Skip to content

MutualInfo

Mutual Information between two clusterings.

The Mutual Information 1 is a measure of the similarity between two labels of the same data. Where \(|U_i|\) is the number of samples in cluster \(U_i\) and \(|V_j|\) is the number of the samples in cluster \(V_j\), the Mutual Information between clusterings \(U\) and \(V\) can be calculated as:

\[ MI(U,V) = \sum_{i=1}^{|U|} \sum_{v=1}^{|V|} \frac{|U_i \cup V_j|}{N} \log \frac{N |U_i \cup V_j|}{|U_i| |V_j|} \]

This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won't change the score.

This metric is furthermore symmetric: switching y_true and y_pred will return the same score value. This can be useful to measure the agreement of two independent label assignments strategies on the same dataset when the real ground truth is not known.

The Mutual Information can be equivalently expressed as:

\[ MI(U,V) = H(U) - H(U | V) = H(V) - H(V | U) \]

where \(H(U)\) and \(H(V)\) are the marginal entropies, \(H(U | V)\) and \(H(V | U)\) are the conditional entropies.

Parameters

  • cm (river.metrics.confusion.ConfusionMatrix) – defaults to None

    This parameter allows sharing the same confusion matrix between multiple metrics. Sharing a confusion matrix reduces the amount of storage and computation time.

Attributes

  • bigger_is_better

    Indicate if a high value is better than a low one or not.

  • requires_labels

    Indicates if labels are required, rather than probabilities.

  • works_with_weights

    Indicate whether the model takes into consideration the effect of sample weights

Examples

>>> from river import metrics

>>> y_true = [1, 1, 2, 2, 3, 3]
>>> y_pred = [1, 1, 1, 2, 2, 2]

>>> metric = metrics.MutualInfo()
>>> for yt, yp in zip(y_true, y_pred):
...     print(metric.update(yt, yp).get())
0.0
0.0
0.0
0.215761
0.395752
0.462098

>>> metric
MutualInfo: 0.462098

Methods

get

Return the current value of the metric.

is_better_than
revert

Revert the metric.

Parameters

  • y_true
  • y_pred
  • sample_weight – defaults to 1.0
update

Update the metric.

Parameters

  • y_true
  • y_pred
  • sample_weight – defaults to 1.0
works_with

Indicates whether or not a metric can work with a given model.

Parameters

  • model (river.base.estimator.Estimator)

References

  1. Wikipedia contributors. (2021, March 17). Mutual information. In Wikipedia, The Free Encyclopedia, from https://en.wikipedia.org/w/index.php?title=Mutual_information&oldid=1012714929 ↩