ROCAUC¶
Receiving Operating Characteristic Area Under the Curve.
This metric is an approximation of the true ROC AUC. Computing the true ROC AUC would require storing all the predictions and ground truths, which isn't desirable. The approximation error is not significant as long as the predicted probabilities are well calibrated. In any case, this metric can still be used to reliably compare models between each other.
Parameters¶
-
n_thresholds
Default →
10
The number of thresholds used for discretizing the ROC curve. A higher value will lead to more accurate results, but will also cost more time and memory.
-
pos_val
Default →
True
Value to treat as "positive".
Attributes¶
-
bigger_is_better
Indicate if a high value is better than a low one or not.
-
requires_labels
Indicates if labels are required, rather than probabilities.
-
works_with_weights
Indicate whether the model takes into consideration the effect of sample weights
Examples¶
from river import metrics
y_true = [ 0, 0, 1, 1]
y_pred = [.1, .4, .35, .8]
metric = metrics.ROCAUC()
for yt, yp in zip(y_true, y_pred):
metric.update(yt, yp)
metric
ROCAUC: 87.50%
The true ROC AUC is in fact 0.75. We can improve the accuracy by increasing the amount of thresholds. This comes at the cost more computation time and more memory usage.
metric = metrics.ROCAUC(n_thresholds=20)
for yt, yp in zip(y_true, y_pred):
metric.update(yt, yp)
metric
ROCAUC: 75.00%
Methods¶
get
Return the current value of the metric.
is_better_than
Indicate if the current metric is better than another one.
Parameters
- other
revert
Revert the metric.
Parameters
- y_true — 'bool'
- y_pred — 'bool | float | dict[bool, float]'
- w — defaults to
1.0
update
Update the metric.
Parameters
- y_true — 'bool'
- y_pred — 'bool | float | dict[bool, float]'
- w — defaults to
1.0
works_with
Indicates whether or not a metric can work with a given model.
Parameters
- model — 'base.Estimator'