selector.measures.diversity

Molecule dataset diversity calculation module.

selector.measures.diversity.compute_diversity(feature_subset: array, div_type: str = 'shannon_entropy', normalize: bool = False, truncation: bool = False, features: array = None, cs: int = None) → float

Compute diversity metrics.

Parameters

feature_subsetnp.ndarray

Feature matrix.

div_typestr, optional

Method of calculation diversity for a given molecule set, which includes “entropy”, “logdet”, “shannon entropy”, “wdud”, “gini coefficient” “hypersphere_overlap”, and “explicit diversity index”. The default is “entropy”.

normalizebool, optional

Normalize the entropy to [0, 1]. Default is “False”.

truncationbool, optional

Use the truncated Shannon entropy. Default is “False”.

featuresnp.ndarray, optional

Feature matrix of entire molecule library, used only if calculating hypersphere_overlap_of_subset. Default is “None”.

csint, optional

Number of common substructures in molecular compound dataset. Used only if calculating explicit_diversity_index. Default is “None”.

Returns

float, computed diversity.

selector.measures.diversity.explicit_diversity_index(x: ndarray, cs: int) → float

Compute the explicit diversity index.

Parameters

x: ndarray of shape (n_samples, n_features)

Feature matrix of n_samples samples in n_features dimensional feature space.

csint

Number of common substructures in the compound set.

Returns

edi_scaledfloat

Explicit diversity index.

Notes

This method hasn’t been tested.

This method is used only for datasets of molecular compounds.

Papp, Á., Gulyás-Forró, A., Gulyás, Z., Dormán, G., Ürge, L., and Darvas, F.. (2006) Explicit Diversity Index (EDI): A Novel Measure for Assessing the Diversity of Compound Databases. Journal of Chemical Information and Modeling 46, 1898-1904.

selector.measures.diversity.gini_coefficient(x: ndarray)

Gini coefficient of bit-wise fingerprints of a database of molecules.

Measures the chemical diversity of a database of molecules defined by the following formula:

\[G = \frac{2 \sum_{i=1}^L i ||y_i||_1 }{N \sum_{i=1}^L ||y_i||_1} - \frac{L+1}{L},\]

where \(y_i \in \{0, 1\}^N\) is a vector of zero and ones of length the number of molecules \(N\) of the i`th feature, and :math:`L is the feature length.

Lower values mean more diversity.

Parameters

xndarray(N, L)

Molecule features in L bits with N molecules.

Returns

float :

Gini coefficient in the range [0,1].

References

Weidlich, Iwona E., and Igor V. Filippov. “Using the gini coefficient to measure the chemical diversity of small‐molecule libraries.” (2016): 2091-2097.

selector.measures.diversity.hypersphere_overlap_of_subset(x: ndarray, x_subset: array) → float

Compute the overlap of subset with hyper-spheres around each point

The edge penalty is also included, which disregards areas outside of the boundary of the full feature space/library. This is calculated as:

\[g(S) = \sum_{i < j}^k O(i, j) + \sum^k_m E(m)\]

where \(i, j\) is over the subset of molecules, \(O(i, j)\) is the approximate overlap between hyperspheres, \(k\) is the number of features and \(E\) is the edge penalty of a molecule.

Lower values mean more diversity.

Parameters

xndarray

Feature matrix of all molecules.

x_subsetndarray

Feature matrix of selected subset of molecules.

Returns

float :

The approximate overlapping volume of hyperspheres drawn around the selected points/molecules.

Notes

The hypersphere overlap volume is calculated using an approximation formula from Agrafiotis (1997).

Agrafiotis, D. K.. (1997) Stochastic Algorithms for Maximizing Molecular Diversity. Journal of Chemical Information and Computer Sciences 37, 841-851.

selector.measures.diversity.logdet(x: ndarray) → float

Compute the log determinant function.

Given a \(n_s \times n_f\) feature matrix \(x\), where \(n_s\) is the number of samples and \(n_f\) is the number of features, the log determinant function is defined as:

where the \(I\) is the \(n_s \times n_s\) identity matrix. Higher values of \(F_\text{logdet}\) mean more diversity.

Parameters

x: ndarray of shape (n_samples, n_features)

Feature matrix of n_samples samples in n_features dimensional feature space,

Returns

f_logdet: float

The volume of parallelotope spanned by the matrix.

Notes

The log-determinant function is based on the formula in Nakamura, T., Sci Rep 2022. Please note that we used the natural logrithim to avoid the numerical stability issues, theochem/Selector#229.

References

Nakamura, T., Sakaue, S., Fujii, K., Harabuchi, Y., Maeda, S., and Iwata, S.., Selecting molecules with diverse structures and properties by maximizing submodular functions of descriptors learned with graph neural networks. Scientific Reports 12, 2022.

selector.measures.diversity.nearest_average_tanimoto(x: ndarray) → float

Computes the average tanimoto for nearest molecules.

Parameters

xndarray

Feature matrix.

Returns

natfloat

Average tanimoto of closest pairs.

Notes

This computes the tanimoto coefficient of pairs with the shortest distances, then returns the average of them. This calculation is explictly for the explicit diversity index.

Papp, Á., Gulyás-Forró, A., Gulyás, Z., Dormán, G., Ürge, L., and Darvas, F.. (2006) Explicit Diversity Index (EDI): A Novel Measure for Assessing the Diversity of Compound Databases. Journal of Chemical Information and Modeling 46, 1898-1904.

selector.measures.diversity.shannon_entropy(x: ndarray, normalize=True, truncation=False) → float

Compute the shannon entropy of a binary matrix.

Higher values mean more diversity.

Parameters

xndarray

Bit-string matrix.

normalizebool, optional

Normalize the entropy to [0, 1]. Default=True.

truncationbool, optional

Use the truncated Shannon entropy by only counting the contributions of one-bits. Default=False.

Returns

float :

The shannon entropy of the matrix.

Notes

Suppose we have \(m\) compounds and each compound has \(n\) bits binary fingerprints. The binary matrix (feature matrix) is \(\mathbf{x} \in m \times n\), where each row is a compound and each column contains the \(n\)-bit binary fingerprint. The equation for Shannon entropy is given by 1 and 3,

\[H = \sum_i^m \left[ - p_i \log_2{p_i } - (1 - p_i)\log_2(1 - p_i) \right]\]

where \(p_i\) represents the relative frequency of 1 bits at the fingerprint position \(i\). When \(p_i = 0\) or \(p_i = 1\), the \(SE_i\) is zero. When completeness is True, the entropy is calculated as in 2 instead

\[H = \sum_i^m \left[ - p_i \log_2{p_i } \right]\]

When normalize is True, the entropy is normalized by a scaling factor so that the entropy is in the range of [0, 1], 2

\[H = \frac{ \sum_i^m \left[ - p_i \log_2{p_i } - (1 - p_i)\log_2(1 - p_i) \right]} {n \log_2{2} / 2}\]

But please note, when completeness is False and normalize is True, the formula has not been used in any literature. It is just a simple normalization of the entropy and the user can use it at their own risk.

References

1

Wang, Y., Geppert, H., & Bajorath, J. (2009). Shannon entropy-based fingerprint similarity search strategy. Journal of Chemical Information and Modeling, 49(7), 1687-1691.

2(1,2)

Leguy, J., Glavatskikh, M., Cauchy, T., & Da Mota, B. (2021). Scalable estimator of the diversity for de novo molecular generation resulting in a more robust QM dataset (OD9) and a more efficient molecular optimization. Journal of Cheminformatics, 13(1), 1-17.

3

Weidlich, I. E., & Filippov, I. V. (2016). Using the Gini coefficient to measure the chemical diversity of small molecule libraries. Journal of Computational Chemistry, 37(22), 2091-2097.

selector.measures.diversity.wdud(x: ndarray) → float

Compute the Wasserstein Distance to Uniform Distribution(WDUD).

The equation for the Wasserstein Distance for a single feature to uniform distribution is

\[WDUD(x) = \int_{0}^{1} |U(x) - V(x)|dx\]

where the feature is normalized to [0, 1], \(U(x)=x\) is the cumulative distribution of the uniform distribution on [0, 1], and \(V(x) = \sum_{y <= x}1 / N\) is the discrete distribution of the values of the feature in \(x\), where \(y\) is the ith feature. This integral is calculated iteratively between \(y_i\) and \(y_{i+1}\), using trapezoidal method.

Lower values of the WDUD mean more diversity because the features of the selected set are more evenly distributed over the range of feature values.

Parameters

x: ndarray of shape (n_samples, n_features)

Feature matrix of n_samples samples in n_features dimensional feature space.

Returns

float :

The mean of the WDUD of each feature over all molecules.

Notes

Nakamura, T., Sakaue, S., Fujii, K., Harabuchi, Y., Maeda, S., and Iwata, S.. (2022) Selecting molecules with diverse structures and properties by maximizing submodular functions of descriptors learned with graph neural networks. Scientific Reports 12.