Regression¶
ChickWeights¶
Summary¶
| Model | MAE | RMSE | R2 | Memory in Mb | Time in s |
|---|---|---|---|---|---|
| Adaptive Model Rules | 24.1943 | 37.2166 | 0.725319 | 0.046977 | 5.25855 |
| Adaptive Random Forest | 26.1016 | 40.8094 | 0.669725 | 1.19043 | 56.006 |
| Aggregated Mondrian Forest | 25.6742 | 41.7123 | 0.65479 | 8.21412 | 127.415 |
| Bagging | 23.1143 | 36.6311 | 0.733893 | 0.628034 | 38.0203 |
| Exponentially Weighted Average | 121.818 | 141.004 | -2.94294 | 3.09241 | 55.8851 |
| Hoeffding Adaptive Tree | 23.3739 | 37.6579 | 0.718766 | 0.0947332 | 7.99029 |
| Hoeffding Tree | 23.1619 | 36.7336 | 0.732402 | 0.0440512 | 6.29305 |
| Linear Regression | 23.7587 | 37.0377 | 0.727954 | 0.00421047 | 3.21471 |
| Linear Regression with l1 regularization | 23.7577 | 37.078 | 0.727361 | 0.00444126 | 9.7485 |
| Linear Regression with l2 regularization | 25.2738 | 38.5885 | 0.704694 | 0.00423336 | 1.22128 |
| Passive-Aggressive Regressor, mode 1 | 24.3423 | 37.596 | 0.71969 | 0.00345898 | 1.10187 |
| Passive-Aggressive Regressor, mode 2 | 100.624 | 143.066 | -3.05911 | 0.00345898 | 1.16798 |
| River MLP | 51.4078 | 80.9203 | -0.298584 | 0.0123129 | 28.2295 |
| Stochastic Gradient Tree | 68.7588 | 80.358 | -0.280601 | 1.12059 | 22.3803 |
| Streaming Random Patches | 23.7097 | 38.4416 | 0.706938 | 0.355182 | 93.4014 |
| [baseline] Mean predictor | 50.2509 | 71.1144 | -0.00292947 | 0.000490189 | 0.302835 |
| k-Nearest Neighbors | 24.8406 | 39.2016 | 0.695236 | 2.88522 | 40.0878 |
Charts¶
TrumpApproval¶
Summary¶
| Model | MAE | RMSE | R2 | Memory in Mb | Time in s |
|---|---|---|---|---|---|
| Adaptive Model Rules | 1.39847 | 2.43336 | -1.02372 | 0.114429 | 9.38293 |
| Adaptive Random Forest | 0.800378 | 2.11495 | -0.528761 | 1.28462 | 87.4457 |
| Aggregated Mondrian Forest | 0.268533 | 0.349421 | 0.958184 | 16.9323 | 186.034 |
| Bagging | 0.908203 | 2.23718 | -0.710572 | 1.31579 | 82.0689 |
| Exponentially Weighted Average | 40.7546 | 40.7905 | -567.663 | 5.27613 | 141.452 |
| Hoeffding Adaptive Tree | 0.921313 | 2.23942 | -0.713986 | 0.138225 | 16.7576 |
| Hoeffding Tree | 0.956103 | 2.24987 | -0.730022 | 0.148639 | 11.7656 |
| Linear Regression | 1.31455 | 3.91198 | -4.23035 | 0.00497341 | 11.5379 |
| Linear Regression with l1 regularization | 1.15377 | 3.82872 | -4.01007 | 0.0052042 | 13.3595 |
| Linear Regression with l2 regularization | 1.87151 | 4.13052 | -4.83107 | 0.0049963 | 4.15677 |
| Passive-Aggressive Regressor, mode 1 | 4.98403 | 6.97667 | -15.6354 | 0.00443554 | 2.99338 |
| Passive-Aggressive Regressor, mode 2 | 31.0933 | 34.6257 | -408.765 | 0.00443554 | 4.72475 |
| River MLP | 1.58058 | 5.03392 | -7.66066 | 0.0133505 | 32.2432 |
| Stochastic Gradient Tree | 9.42975 | 17.9379 | -108.972 | 3.08244 | 52.4507 |
| Streaming Random Patches | 0.656697 | 1.98434 | -0.345761 | 1.06461 | 134.903 |
| [baseline] Mean predictor | 1.56755 | 2.20286 | -0.658483 | 0.000490189 | 1.08177 |
| k-Nearest Neighbors | 0.641679 | 1.59417 | 0.131425 | 5.03263 | 123.301 |
Charts¶
Datasets¶
ChickWeights
Chick weights along time.
The stream contains 578 items and 3 features. The goal is to predict the weight of each chick along time, according to the diet the chick is on. The data is ordered by time and then by chick.
Name ChickWeights
Task Regression
Samples 578
Features 3
Sparse False
Path /Users/mastelini/miniconda3/envs/river-benchmark/lib/python3.10/site-packages/river/datasets/chick-weights.csv
TrumpApproval
Donald Trump approval ratings.
This dataset was obtained by reshaping the data used by FiveThirtyEight for analyzing Donald Trump's approval ratings. It contains 5 features, which are approval ratings collected by 5 polling agencies. The target is the approval rating from FiveThirtyEight's model. The goal of this task is to see if we can reproduce FiveThirtyEight's model.
Name TrumpApproval
Task Regression
Samples 1,001
Features 6
Sparse False
Path /Users/mastelini/miniconda3/envs/river-benchmark/lib/python3.10/site-packages/river/datasets/trump_approval.csv.gz
Models¶
Linear Regression
Pipeline (
StandardScaler (
with_std=True
),
LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
)
Linear Regression with l1 regularization
Pipeline (
StandardScaler (
with_std=True
),
LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=1.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
)
Linear Regression with l2 regularization
Pipeline (
StandardScaler (
with_std=True
),
LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=1.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
)
Passive-Aggressive Regressor, mode 1
Pipeline (
StandardScaler (
with_std=True
),
PARegressor (
C=1.
mode=1
eps=0.1
learn_intercept=True
)
)
Passive-Aggressive Regressor, mode 2
Pipeline (
StandardScaler (
with_std=True
),
PARegressor (
C=1.
mode=2
eps=0.1
learn_intercept=True
)
)
k-Nearest Neighbors
Pipeline (
StandardScaler (
with_std=True
),
KNNRegressor (
n_neighbors=5
engine=SWINN (
graph_k=20
dist_func=FunctionWrapper (
distance_function=functools.partial(, p=2)
)
maxlen=1000
warm_up=500
max_candidates=50
delta=0.0001
prune_prob=0.
n_iters=10
seed=None
)
aggregation_method="mean"
)
)
Hoeffding Tree
Pipeline (
StandardScaler (
with_std=True
),
HoeffdingTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
)
)
Hoeffding Adaptive Tree
Pipeline (
StandardScaler (
with_std=True
),
HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=True
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=42
)
)
Stochastic Gradient Tree
SGTRegressor (
delta=1e-07
grace_period=200
init_pred=0.
max_depth=inf
lambda_value=0.1
gamma=1.
nominal_attributes=[]
feature_quantizer=StaticQuantizer (
n_bins=64
warm_start=100
buckets=None
)
)
Adaptive Random Forest
Pipeline (
StandardScaler (
with_std=True
),
[]
)
Aggregated Mondrian Forest
[]
Adaptive Model Rules
Pipeline (
StandardScaler (
with_std=True
),
AMRules (
n_min=200
delta=1e-07
tau=0.05
pred_type="adaptive"
pred_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
splitter=TEBSTSplitter (
digits=1
)
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
fading_factor=0.99
anomaly_threshold=-0.75
m_min=30
ordered_rule_set=True
min_samples_split=5
)
)
Streaming Random Patches
Pipeline (
StandardScaler (
with_std=True
),
SRPRegressor (
model=HoeffdingTreeRegressor (
grace_period=50
max_depth=inf
delta=0.01
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
)
n_models=10
subspace_size=0.6
training_method="patches"
lam=6
drift_detector=ADWIN (
delta=1e-05
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
warning_detector=ADWIN (
delta=0.0001
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
disable_detector="off"
disable_weighted_vote=True
drift_detection_criteria="error"
aggregation_method="mean"
seed=42
metric=MAE ()
)
)
Bagging
Pipeline (
StandardScaler (
with_std=True
),
[HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=False
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
)]
)
Exponentially Weighted Average
Pipeline (
StandardScaler (
with_std=True
),
[LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
), HoeffdingAdaptiveTreeRegressor (
grace_period=200
max_depth=inf
delta=1e-07
tau=0.05
leaf_prediction="adaptive"
leaf_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
model_selector_decay=0.95
nominal_attributes=None
splitter=TEBSTSplitter (
digits=1
)
min_samples_split=5
bootstrap_sampling=True
drift_window_threshold=300
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
switch_significance=0.05
binary_split=False
max_size=500.
memory_estimate_period=1000000
stop_mem_management=False
remove_poor_attrs=False
merit_preprune=True
seed=None
), KNNRegressor (
n_neighbors=5
engine=SWINN (
graph_k=20
dist_func=FunctionWrapper (
distance_function=functools.partial(, p=2)
)
maxlen=1000
warm_up=500
max_candidates=50
delta=0.0001
prune_prob=0.
n_iters=10
seed=None
)
aggregation_method="mean"
), AMRules (
n_min=200
delta=1e-07
tau=0.05
pred_type="adaptive"
pred_model=LinearRegression (
optimizer=SGD (
lr=Constant (
learning_rate=0.01
)
)
loss=Squared ()
l2=0.
l1=0.
intercept_init=0.
intercept_lr=Constant (
learning_rate=0.01
)
clip_gradient=1e+12
initializer=Zeros ()
)
splitter=TEBSTSplitter (
digits=1
)
drift_detector=ADWIN (
delta=0.002
clock=32
max_buckets=5
min_window_length=5
grace_period=10
)
fading_factor=0.99
anomaly_threshold=-0.75
m_min=30
ordered_rule_set=True
min_samples_split=5
)]
)
River MLP
Pipeline (
StandardScaler (
with_std=True
),
MLPRegressor (
hidden_dims=(5,)
activations=(, , )
loss=Squared ()
optimizer=SGD (
lr=Constant (
learning_rate=0.001
)
)
seed=42
)
)
[baseline] Mean predictor
StatisticRegressor ( statistic=Mean () )
Environment¶
Python implementation: CPython Python version : 3.12.12 IPython version : 8.29.0 river : 0.23.0 numpy : 2.0.2 scikit-learn: 1.5.1 pandas : 2.2.3 scipy : 1.14.1 Compiler : GCC 13.3.0 OS : Linux Release : 6.11.0-1018-azure Machine : x86_64 Processor : x86_64 CPU cores : 4 Architecture: 64bit