Skip to content

PrevalenceThreshold

Prevalence Threshold (PT).

The relationship between a positive predicted value and its target prevalence is propotional - though not linear in all but a special case. In consequence, there is a point of local extrema and maximum curvature defined only as a function of the sensitivity and specificity beyond which the rate of change of a test's positive predictive value drops at a differential pace relative to the disease prevalence. Using differential equations, this point was first defined by Balayla et al. 1 and is termed the prevalence threshold (\phi_e).

The equation for the prevalence threshold 2 is given by the following formula

\[ \phi_e = \frac{\sqrt{TPR(1 - TNR)} + TNR - 1}{TPR + TNR - 1} \]

with

\[ TPR = \frac{TP}{P} = \frac{TP}{TP + FN}, TNR = = \frac{TN}{N} = \frac{TN}{TN + FP} \]

Parameters

  • cm – defaults to None

    This parameter allows sharing the same confusion matrix between multiple metrics. Sharing a confusion matrix reduces the amount of storage and computation time.

  • pos_val – defaults to True

    Value to treat as "positive".

Attributes

  • bigger_is_better

    Indicate if a high value is better than a low one or not.

  • requires_labels

    Indicates if labels are required, rather than probabilities.

  • sample_correction

  • works_with_weights

    Indicate whether the model takes into consideration the effect of sample weights

Examples

>>> from river import metrics

>>> y_true = [False, False, False, True, True, True]
>>> y_pred = [False, False, True, True, False, True]

>>> metric = metrics.PrevalenceThreshold()
>>> for yt, yp in zip(y_true, y_pred):
...     print(metric.update(yt, yp).get())
0.0
0.0
1.0
0.36602540378443876
0.44948974278317827
0.41421356237309503

>>> metric
PrevalenceThreshold: 0.414214

Methods

clone

Return a fresh estimator with the same parameters.

The clone has the same parameters but has not been updated with any data. This works by looking at the parameters from the class signature. Each parameter is either - recursively cloned if it's a River classes. - deep-copied via copy.deepcopy if not. If the calling object is stochastic (i.e. it accepts a seed parameter) and has not been seeded, then the clone will not be idempotent. Indeed, this method's purpose if simply to return a new instance with the same input parameters.

get

Return the current value of the metric.

revert

Revert the metric.

Parameters

  • y_true (bool)
  • y_pred (Union[bool, float, Dict[bool, float]])
  • sample_weight – defaults to 1.0
  • correction – defaults to None
update

Update the metric.

Parameters

  • y_true (bool)
  • y_pred (Union[bool, float, Dict[bool, float]])
  • sample_weight – defaults to 1.0
works_with

Indicates whether or not a metric can work with a given model.

Parameters

  • model (river.base.estimator.Estimator)

References

  1. Balayla, J. (2020). Prevalence threshold (\(\phi\)_e) and the geometry of screening curves. PLOS ONE, 15(10), e0240215. DOI: 10.1371/journal.pone.0240215 ↩

  2. Wikipedia contributors. (2021, March 19). Sensitivity and specificity. In Wikipedia, The Free Encyclopedia, from https://en.wikipedia.org/w/index.php?title=Sensitivity_and_specificity&oldid=1013004476 ↩