Skip to content

OXTRegressor

Online Extra Trees regressor.

The online Extra Trees1 ensemble takes some steps further into randomization when compared to Adaptive Random Forests (ARF). A subspace of the feature space is considered at each split attempt, as ARF does, and online bagging or subbagging can also be (optionally) used. Nonetheless, Extra Trees randomizes the split candidates evaluated by each leaf node (just a single split is tested by numerical feature, which brings significant speedups to the ensemble), and might also randomize the maximum depth of the forest members, as well as the size of the feature subspace processed by each of its trees' leaves.

On the other hand, OXT suffers from a cold-start problem. As the splits are random, the predictive performance in small samples is usually worse than using a deterministic split approach, such as the one used by ARF.

Parameters

  • n_models

    Typeint

    Default10

    The number of trees in the ensemble.

  • max_features

    Typebool | str | int

    Defaultsqrt

    Max number of attributes for each node split.
    - If int, then consider max_features at each split.
    - If float, then max_features is a percentage and int(max_features * n_features) features are considered per split.
    - If "sqrt", then max_features=sqrt(n_features).
    - If "log2", then max_features=log2(n_features).
    - If "random", then max_features will assume a different random number in the interval [2, n_features] for each tree leaf.
    - If None, then max_features=n_features.

  • resampling_strategy

    Typestr | None

    Defaultsubbagging

    The chosen instance resampling strategy:
    - If None, no resampling will be done and the trees will process all instances. - If 'baggging', online bagging will be performed (sampling with replacement). - If 'subbagging', online subbagging will be performed (sampling without replacement).

  • resampling_rate

    Typeint | float

    Default0.5

    Only valid if resampling_strategy is not None. Controls the parameters of the resampling strategy.
    . - If resampling_strategy='bagging', must be an integer greater than or equal to 1 that parameterizes the poisson distribution used to simulate bagging in online learning settings. It acts as the lambda parameter of Oza Bagging and Leveraging Bagging.
    - If resampling_strategy='subbagging', must be a float in the interval \((0, 1]\) that controls the chance of each instance being used by a tree for learning.

  • detection_mode

    Typestr

    Defaultall

    The concept drift detection mode in which the forest operates. Valid values are:
    - "all": creates both warning and concept drift detectors. If a warning is detected, an alternate tree starts being trained in the background. If the warning trigger escalates to a concept drift, the affected tree is replaced by the alternate tree.
    - "drop": only the concept drift detectors are created. If a drift is detected, the affected tree is dropped and replaced by a new tree.
    - "off": disables the concept drift adaptation capabilities. The forest will act as if the processed stream is stationary.

  • warning_detector

    Typebase.DriftDetector | None

    DefaultNone

    The detector that will be used to trigger concept drift warnings. Defaults to drift.ADWIN(0.01)`.

  • drift_detector

    Typebase.DriftDetector | None

    DefaultNone

    The detector used to detect concept drifts. Defaults to drift.ADWIN(0.001)`.

  • max_depth

    Typeint | None

    DefaultNone

    The maximum depth the ensemble members might reach. If None, the trees will grow indefinitely.

  • randomize_tree_depth

    Typebool

    DefaultFalse

    Whether or not randomize the maximum depth of each tree in the ensemble. If max_depth is provided, it is going to act as an upper bound to generate the maximum depth for each tree.

  • track_metric

    Typemetrics.base.RegressionMetric | None

    DefaultNone

    The performance metric used to weight predictions. Defaults to metrics.MAE()`.

  • disable_weighted_vote

    Typebool

    DefaultTrue

    Defines whether or not to use predictions weighted by each trees' prediction performance.

  • split_buffer_size

    Typeint

    Default5

    Defines the size of the buffer used by the tree splitters when determining the feature range and a random split point in this interval.

  • seed

    Typeint | None

    DefaultNone

    Random seed to support reproducibility.

  • grace_period

    Typeint

    Default50

    [Tree parameter] Number of instances a leaf should observe between split attempts.

  • delta

    Typefloat

    Default0.01

    [Tree parameter] Allowed error in split decision, a value closer to 0 takes longer to decide.

  • tau

    Typefloat

    Default0.05

    [Tree parameter] Threshold below which a split will be forced to break ties.

  • leaf_prediction

    Typestr

    Defaultadaptive

    [Tree parameter] Prediction mechanism used at leaves.
    - 'mean' - Target mean
    - 'model' - Uses the model defined in leaf_model
    - 'adaptive' - Chooses between 'mean' and 'model' dynamically

  • leaf_model

    Typebase.Regressor | None

    DefaultNone

    [Tree parameter] The regression model used to provide responses if leaf_prediction='model'. If not provided, an instance of linear_model.LinearRegression with the default hyperparameters is used.

  • model_selector_decay

    Typefloat

    Default0.95

    [Tree parameter] The exponential decaying factor applied to the learning models' squared errors, that are monitored if leaf_prediction='adaptive'. Must be between 0 and 1. The closer to 1, the more importance is going to be given to past observations. On the other hand, if its value approaches 0, the recent observed errors are going to have more influence on the final decision.

  • nominal_attributes

    Typelist | None

    DefaultNone

    [Tree parameter] List of Nominal attributes. If empty, then assume that all attributes are numerical.

  • min_samples_split

    Typeint

    Default5

    [Tree parameter] The minimum number of samples every branch resulting from a split candidate must have to be considered valid.

  • binary_split

    Typebool

    DefaultFalse

    [Tree parameter] If True, only allow binary splits.

  • max_size

    Typeint

    Default500

    [Tree parameter] Maximum memory (MB) consumed by the tree.

  • memory_estimate_period

    Typeint

    Default2000000

    [Tree parameter] Number of instances between memory consumption checks.

  • stop_mem_management

    Typebool

    DefaultFalse

    [Tree parameter] If True, stop growing as soon as memory limit is hit.

  • remove_poor_attrs

    Typebool

    DefaultFalse

    [Tree parameter] If True, disable poor attributes to reduce memory usage.

  • merit_preprune

    Typebool

    DefaultTrue

    [Tree parameter] If True, enable merit-based tree pre-pruning.

Attributes

  • instances_per_tree

    The number of instances processed by each one of the current forest members. Each time a concept drift is detected, the count corresponding to the affected tree is reset.

  • models

  • n_drifts

    The number of concept drifts detected per ensemble member.

  • n_tree_swaps

    The number of performed alternate tree swaps. Not applicable if the warning detectors are disabled.

  • n_warnings

    The number of warnings detected per ensemble member.

  • total_instances

    The total number of instances processed by the ensemble.

Examples

from river import datasets
from river import evaluate
from river import metrics
from river import forest

dataset = datasets.synth.Friedman(seed=42).take(5000)

model = forest.OXTRegressor(n_models=3, seed=42)

metric = metrics.RMSE()

evaluate.progressive_val_score(dataset, model, metric)
RMSE: 3.127311

Methods

learn_one
predict_one

Predict the output of features x.

Parameters

  • x'dict'

Returns

base.typing.RegTarget: The prediction.

Notes

As the Online Extra Trees change the way in which Hoeffding Trees perform split attempts and monitor numerical input features, some of the parameters of the vanilla Hoeffding Tree algorithms are not available.


  1. Mastelini, S. M., Nakano, F. K., Vens, C., & de Leon Ferreira, A. C. P. (2022). Online Extra Trees Regressor. IEEE Transactions on Neural Networks and Learning Systems.