AdaptiveRandomForestRegressor¶
Adaptive Random Forest regressor.
The 3 most important aspects of Adaptive Random Forest 1 are:
-
inducing diversity through re-sampling
-
inducing diversity through randomly selecting subsets of features for node splits
-
drift detectors per base tree, which cause selective resets in response to drifts
Notice that this implementation is slightly different from the original algorithm proposed in 2. The HoeffdingTreeRegressor
is used as base learner, instead of FIMT-DD
. It also adds a new strategy to monitor the predictions and check for concept drifts. The deviations of the predictions to the target are monitored and normalized in the [0, 1] range to fulfill ADWIN's requirements. We assume that the data subjected to the normalization follows a normal distribution, and thus, lies within the interval of the mean \(\pm3\sigma\).
Parameters¶
-
n_models (int) – defaults to
10
Number of trees in the ensemble.
-
max_features – defaults to
sqrt
Max number of attributes for each node split.
- Ifint
, then considermax_features
at each split.
- Iffloat
, thenmax_features
is a percentage andint(max_features * n_features)
features are considered per split.
- If "sqrt", thenmax_features=sqrt(n_features)
.
- If "log2", thenmax_features=log2(n_features)
.
- If None, thenmax_features=n_features
. -
aggregation_method (str) – defaults to
median
The method to use to aggregate predictions in the ensemble.
- 'mean'
- 'median' - If selected will disable the weighted vote. -
lambda_value (int) – defaults to
6
The lambda value for bagging (lambda=6 corresponds to Leveraging Bagging).
-
metric (river.metrics.base.RegressionMetric) – defaults to
MSE: 0.
Metric used to track trees performance within the ensemble. Depending, on the configuration, this metric is also used to weight predictions from the members of the ensemble.
-
disable_weighted_vote – defaults to
True
If
True
, disables the weighted vote prediction, i.e. does not assign weights to individual tree's predictions and uses the arithmetic mean instead. Otherwise will use themetric
value to weight predictions. -
drift_detector (base.DriftDetector) – defaults to
ADWIN
Drift Detection method. Set to None to disable Drift detection.
-
warning_detector (base.DriftDetector) – defaults to
ADWIN
Warning Detection method. Set to None to disable warning detection.
-
grace_period (int) – defaults to
50
[Tree parameter] Number of instances a leaf should observe between split attempts.
-
max_depth (int) – defaults to
None
[Tree parameter] The maximum depth a tree can reach. If
None
, the tree will grow indefinitely. -
split_confidence (float) – defaults to
0.01
[Tree parameter] Allowed error in split decision, a value closer to 0 takes longer to decide.
-
tie_threshold (float) – defaults to
0.05
[Tree parameter] Threshold below which a split will be forced to break ties.
-
leaf_prediction (str) – defaults to
model
[Tree parameter] Prediction mechanism used at leaves. - 'mean' - Target mean - 'model' - Uses the model defined in
leaf_model
- 'adaptive' - Chooses between 'mean' and 'model' dynamically -
leaf_model (base.Regressor) – defaults to
None
[Tree parameter] The regression model used to provide responses if
leaf_prediction='model'
. If not provided, an instance ofriver.linear_model.LinearRegression
with the default hyperparameters is used. -
model_selector_decay (float) – defaults to
0.95
[Tree parameter] The exponential decaying factor applied to the learning models' squared errors, that are monitored if
leaf_prediction='adaptive'
. Must be between0
and1
. The closer to1
, the more importance is going to be given to past observations. On the other hand, if its value approaches0
, the recent observed errors are going to have more influence on the final decision. -
nominal_attributes (list) – defaults to
None
[Tree parameter] List of Nominal attributes. If empty, then assume that all attributes are numerical.
-
splitter (river.tree.splitter.base.Splitter) – defaults to
None
[Tree parameter] The Splitter or Attribute Observer (AO) used to monitor the class statistics of numeric features and perform splits. Splitters are available in the
tree.splitter
module. Different splitters are available for classification and regression tasks. Classification and regression splitters can be distinguished by their propertyis_target_class
. This is an advanced option. Special care must be taken when choosing different splitters.By default,tree.splitter.EBSTSplitter
is used ifsplitter
isNone
. -
min_samples_split (int) – defaults to
5
[Tree parameter] The minimum number of samples every branch resulting from a split candidate must have to be considered valid.
-
binary_split (bool) – defaults to
False
[Tree parameter] If True, only allow binary splits.
-
max_size (int) – defaults to
500
[Tree parameter] Maximum memory (MB) consumed by the tree.
-
memory_estimate_period (int) – defaults to
2000000
[Tree parameter] Number of instances between memory consumption checks.
-
stop_mem_management (bool) – defaults to
False
[Tree parameter] If True, stop growing as soon as memory limit is hit.
-
remove_poor_attrs (bool) – defaults to
False
[Tree parameter] If True, disable poor attributes to reduce memory usage.
-
merit_preprune (bool) – defaults to
True
[Tree parameter] If True, enable merit-based tree pre-pruning.
-
seed (int) – defaults to
None
If
int
,seed
is used to seed the random number generator; IfRandomState
,seed
is the random number generator; IfNone
, the random number generator is theRandomState
instance used bynp.random
.
Attributes¶
-
models
-
valid_aggregation_method
Valid aggregation_method values.
Examples¶
>>> from river import datasets
>>> from river import evaluate
>>> from river import metrics
>>> from river import ensemble
>>> from river import preprocessing
>>> dataset = datasets.TrumpApproval()
>>> model = (
... preprocessing.StandardScaler() |
... ensemble.AdaptiveRandomForestRegressor(n_models=3, seed=42)
... )
>>> metric = metrics.MAE()
>>> evaluate.progressive_val_score(dataset, model, metric)
MAE: 1.874094
Methods¶
append
S.append(value) -- append value to the end of the sequence
Parameters
- item
clear
S.clear() -> None -- remove all items from S
clone
Return a fresh estimator with the same parameters.
The clone has the same parameters but has not been updated with any data. This works by looking at the parameters from the class signature. Each parameter is either - recursively cloned if it's a River classes. - deep-copied via copy.deepcopy
if not. If the calling object is stochastic (i.e. it accepts a seed parameter) and has not been seeded, then the clone will not be idempotent. Indeed, this method's purpose if simply to return a new instance with the same input parameters.
copy
count
S.count(value) -> integer -- return number of occurrences of value
Parameters
- item
extend
S.extend(iterable) -- extend sequence by appending elements from the iterable
Parameters
- other
index
S.index(value, [start, [stop]]) -> integer -- return first index of value. Raises ValueError if the value is not present.
Supporting start and stop arguments is optional, but recommended.
Parameters
- item
- args
insert
S.insert(index, value) -- insert value before index
Parameters
- i
- item
learn_one
pop
S.pop([index]) -> item -- remove and return item at index (default last). Raise IndexError if list is empty or index is out of range.
Parameters
- i – defaults to
-1
predict_one
Predicts the target value of a set of features x
.
Parameters
- x (dict)
Returns
Number: The prediction.
remove
S.remove(value) -- remove first occurrence of value. Raise ValueError if the value is not present.
Parameters
- item
reset
Reset the forest.
reverse
S.reverse() -- reverse IN PLACE
sort
References¶
-
Gomes, H.M., Bifet, A., Read, J., Barddal, J.P., Enembreck, F., Pfharinger, B., Holmes, G. and Abdessalem, T., 2017. Adaptive random forests for evolving data stream classification. Machine Learning, 106(9-10), pp.1469-1495. ↩
-
Gomes, H.M., Barddal, J.P., Boiko, L.E., Bifet, A., 2018. Adaptive random forests for data stream regression. ESANN 2018. ↩