ARFClassifier

Adaptive Random Forest classifier.

The 3 most important aspects of Adaptive Random Forest ¹ are:

1. inducing diversity through re-sampling

2. inducing diversity through randomly selecting subsets of features for node splits

3. drift detectors per base tree, which cause selective resets in response to drifts

It also allows training background trees, which start training if a warning is detected and replace the active tree if the warning escalates to a drift.

Parameters

• n_models

  Type → int

  Default → 10

  Number of trees in the ensemble.

• max_features

  Type → bool | str | int

  Default → sqrt

  Max number of attributes for each node split.
  - If int, then consider max_features at each split.
  - If float, then max_features is a percentage and int(max_features * n_features) features are considered per split.
  - If "sqrt", then max_features=sqrt(n_features).
  - If "log2", then max_features=log2(n_features).
  - If None, then max_features=n_features.

• lambda_value

  Type → int

  Default → 6

  The lambda value for bagging (lambda=6 corresponds to Leveraging Bagging).

• metric

  Type → metrics.base.MultiClassMetric | None

  Default → None

  Metric used to track trees performance within the ensemble. Defaults to metrics.Accuracy()`.

• disable_weighted_vote

  Default → False

  If True, disables the weighted vote prediction.

• drift_detector

  Type → base.DriftDetector | None

  Default → None

  Drift Detection method. Set to None to disable Drift detection. Defaults to drift.ADWIN(delta=0.001)`.

• warning_detector

  Type → base.DriftDetector | None

  Default → None

  Warning Detection method. Set to None to disable warning detection. Defaults to drift.ADWIN(delta=0.01)`.

• grace_period

  Type → int

  Default → 50

  [Tree parameter] Number of instances a leaf should observe between split attempts.

• max_depth

  Type → int | None

  Default → None

  [Tree parameter] The maximum depth a tree can reach. If None, the tree will grow indefinitely.

• split_criterion

  Type → str

  Default → info_gain

  [Tree parameter] Split criterion to use.
  - 'gini' - Gini
  - 'info_gain' - Information Gain
  - 'hellinger' - Hellinger Distance

• delta

  Type → float

  Default → 0.01

  [Tree parameter] Allowed error in split decision, a value closer to 0 takes longer to decide.

• tau

  Type → float

  Default → 0.05

  [Tree parameter] Threshold below which a split will be forced to break ties.

• leaf_prediction

  Type → str

  Default → nba

  [Tree parameter] Prediction mechanism used at leafs.
  - 'mc' - Majority Class
  - 'nb' - Naive Bayes
  - 'nba' - Naive Bayes Adaptive

• nb_threshold

  Type → int

  Default → 0

  [Tree parameter] Number of instances a leaf should observe before allowing Naive Bayes.

• nominal_attributes

  Type → list | None

  Default → None

  [Tree parameter] List of Nominal attributes. If empty, then assume that all attributes are numerical.

• splitter

  Type → Splitter | None

  Default → None

  [Tree parameter] The Splitter or Attribute Observer (AO) used to monitor the class statistics of numeric features and perform splits. Splitters are available in the tree.splitter module. Different splitters are available for classification and regression tasks. Classification and regression splitters can be distinguished by their property is_target_class. This is an advanced option. Special care must be taken when choosing different splitters. By default, tree.splitter.GaussianSplitter is used if splitter is None.

• binary_split

  Type → bool

  Default → False

  [Tree parameter] If True, only allow binary splits.

• min_branch_fraction

  Type → float

  Default → 0.01

  [Tree parameter] The minimum percentage of observed data required for branches resulting from split candidates. To validate a split candidate, at least two resulting branches must have a percentage of samples greater than min_branch_fraction. This criterion prevents unnecessary splits when the majority of instances are concentrated in a single branch.

• max_share_to_split

  Type → float

  Default → 0.99

  [Tree parameter] Only perform a split in a leaf if the proportion of elements in the majority class is smaller than this parameter value. This parameter avoids performing splits when most of the data belongs to a single class.

• max_size

  Type → float

  Default → 100.0

  [Tree parameter] Maximum memory (MB) consumed by the tree.

• memory_estimate_period

  Type → int

  Default → 2000000

  [Tree parameter] Number of instances between memory consumption checks.

• stop_mem_management

  Type → bool

  Default → False

  [Tree parameter] If True, stop growing as soon as memory limit is hit.

• remove_poor_attrs

  Type → bool

  Default → False

  [Tree parameter] If True, disable poor attributes to reduce memory usage.

• merit_preprune

  Type → bool

  Default → True

  [Tree parameter] If True, enable merit-based tree pre-pruning.

• seed

  Type → int | None

  Default → None

  Random seed for reproducibility.

Attributes

• models

Examples

from river import evaluate
from river import forest
from river import metrics
from river.datasets import synth

dataset = synth.ConceptDriftStream(
    seed=42,
    position=500,
    width=40
).take(1000)

model = forest.ARFClassifier(seed=8, leaf_prediction="mc")

metric = metrics.Accuracy()

evaluate.progressive_val_score(dataset, model, metric)

Accuracy: 71.17%

The total number of warnings and drifts detected, respectively

model.n_warnings_detected(), model.n_drifts_detected()

(2, 1)

The number of warnings detected by tree number 2

model.n_warnings_detected(2)

1

And the corresponding number of actual concept drift detected

model.n_drifts_detected(2)

1

Methods

learn_one

n_drifts_detected

Get the total number of concept drifts detected, or such number on an individual tree basis (optionally).

Parameters

• tree_id — 'int | None' — defaults to None

Returns

int: The number of concept drifts detected.

n_warnings_detected

Get the total number of concept drift warnings detected, or the number on an individual tree basis (optionally).

Parameters

• tree_id — 'int | None' — defaults to None

Returns

int: The number of concept drift warnings detected.

predict_one

Predict the label of a set of features x.

Parameters

• x — 'dict'
• kwargs

Returns

base.typing.ClfTarget | None: The predicted label.

predict_proba_one

Predict the probability of each label for a dictionary of features x.

Parameters

• x — 'dict'

Returns

dict[base.typing.ClfTarget, float]: A dictionary that associates a probability which each label.

---

1. Heitor Murilo Gomes, Albert Bifet, Jesse Read, Jean Paul Barddal, Fabricio Enembreck, Bernhard Pfharinger, Geoff Holmes, Talel Abdessalem. Adaptive random forests for evolving data stream classification. In Machine Learning, DOI: 10.1007/s10994-017-5642-8, Springer, 2017. ↩