latte.functional.disentanglement.modularity

Module Contents

Functions

modularity(z, a[, reg_dim, discrete, thresh])

Calculate Modularity between latent vectors and attributes
modularity(z, a, reg_dim=None, discrete=False, thresh=1e-12)

Calculate Modularity between latent vectors and attributes

Modularity is a letent-centric measure of disentanglement based on mutual information. Modularity measures the degree in which a latent dimension contains information about only one attribute, and is given by

\[\operatorname{Modularity}(\{a_i\}, z_d) = 1-\dfrac{\sum_{i≠j}(\mathcal{I}(a_i, z_d)/\mathcal{I}(a_j, z_d))^2}{|{a_i}| -1},\]

where \(j=\operatorname{arg}\max_i \mathcal{I}(a_i, z_d)\), and \(\mathcal{I}(\cdot,\cdot)\) is mutual information.

reg_dim is currently ignored in Modularity.

Parameters:

  • z (np.ndarray, (n_samples, n_features)) – a batch of latent vectors

  • a (np.ndarray, (n_samples, n_attributes) or (n_samples,)) – a batch of attribute(s)

  • reg_dim (Optional[List], optional) – regularized dimensions, by default None. Attribute a[:, i] is regularized by z[:, reg_dim[i]]. If None, a[:, i] is assumed to be regularized by z[:, i].

  • discrete (bool, optional) – Whether the attributes are discrete, by default False

  • thresh (float, optional) – threshold for mutual information, by default 1e-12. Latent-attribute pair with variance below thresh will have modularity contribution zeroed.

Returns:

Modularity for each latent vector dimension

Return type:

np.ndarray, (n_features,)

References

[1]

  1. Ridgeway and M. C. Mozer, “Learning deep disentangled embeddings with the F-statistic loss,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, pp. 185–194.