Skip to content

RollingROCAUC

Rolling version of the Receiving Operating Characteristic Area Under the Curve.

The RollingROCAUC calculates the metric using the instances in its window of size S. It keeps a queue of the instances, when an instance is added and the queue length is equal to S, the last instance is removed. The metric has a tree with ordered instances, in order to calculate the AUC efficiently. It was implemented based on the algorithm presented in Brzezinski and Stefanowski, 2017.

The difference between this metric and the standard ROCAUC is that the latter calculates an approximation of the real metric considering all data from the beginning of the stream, while the RollingROCAUC calculates the exact value considering only the last S instances. This approach may be beneficial if it's necessary to evaluate the model's performance over time, since calculating the metric using the entire stream may hide the current performance of the classifier.

Parameters

  • window_size

    Default → 1000

    The max length of the window.

  • pos_val

    Default → True

    Value to treat as "positive".

Attributes

  • bigger_is_better

    Indicate if a high value is better than a low one or not.

  • requires_labels

    Indicates if labels are required, rather than probabilities.

  • works_with_weights

    Indicate whether the model takes into consideration the effect of sample weights

Examples

from river import metrics

y_true = [ 0,  1,  0,  1,  0,  1,  0,  0,   1,  1]
y_pred = [.3, .5, .5, .7, .1, .3, .1, .4, .35, .8]

metric = metrics.RollingROCAUC(window_size=4)

for yt, yp in zip(y_true, y_pred):
    metric = metric.update(yt, yp)

metric

RollingROCAUC: 75.00%

Methods

get

Return the current value of the metric.

is_better_than

Indicate if the current metric is better than another one.

Parameters

  • other

revert

Revert the metric.

Parameters

  • y_true — 'bool'
  • y_pred — 'bool | float | dict[bool, float]'

update

Update the metric.

Parameters

  • y_true — 'bool'
  • y_pred — 'bool | float | dict[bool, float]'

works_with

Indicates whether or not a metric can work with a given model.

Parameters