UCBRegressor

Model selection based on the UCB bandit strategy.

Due to the nature of this algorithm, it's recommended to scale the target so that it exhibits sub-gaussian properties. This can be done by using a preprocessing.TargetStandardScaler.

Parameters

• models

  The models to choose from.

• metric – defaults to None

  The metric that is used to compare models with each other. Defaults to metrics.MAE.

• delta – defaults to 1

  Exploration parameter.

• burn_in – defaults to 100

  The number of initial steps during which each model is updated.

• seed (int) – defaults to None

  Random number generator seed for reproducibility.

Attributes

• best_model

  The current best model.

• burn_in

• delta

• models

• seed

Examples

>>> from river import datasets
>>> from river import evaluate
>>> from river import linear_model
>>> from river import metrics
>>> from river import model_selection
>>> from river import optim
>>> from river import preprocessing

>>> models = [
...     linear_model.LinearRegression(optimizer=optim.SGD(lr=lr))
...     for lr in [0.0001, 0.001, 1e-05, 0.01]
... ]

>>> dataset = datasets.TrumpApproval()
>>> model = (
...     preprocessing.StandardScaler() |
...     preprocessing.TargetStandardScaler(
...         model_selection.UCBRegressor(
...             models,
...             delta=1,
...             burn_in=0,
...             seed=42
...         )
...     )
... )
>>> metric = metrics.MAE()

>>> evaluate.progressive_val_score(dataset, model, metric)
MAE: 0.410815

>>> model['TargetStandardScaler'].regressor.bandit
Ranking   MAE        Pulls   Share
     #3   1.441458       8    0.80%
     #1   0.291200     242   24.18%
     #2   0.808878      19    1.90%
     #0   0.204892     732   73.13%

>>> model['TargetStandardScaler'].regressor.best_model
LinearRegression (
  optimizer=SGD (
    lr=Constant (
      learning_rate=0.01
    )
  )
  loss=Squared ()
  l2=0.
  l1=0.
  intercept_init=0.
  intercept_lr=Constant (
    learning_rate=0.01
  )
  clip_gradient=1e+12
  initializer=Zeros ()
)

Methods

append

S.append(value) -- append value to the end of the sequence

Parameters

• item

clear

S.clear() -> None -- remove all items from S

copy

count

S.count(value) -> integer -- return number of occurrences of value

Parameters

• item

extend

S.extend(iterable) -- extend sequence by appending elements from the iterable

Parameters

• other

index

S.index(value, [start, [stop]]) -> integer -- return first index of value. Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

Parameters

• item
• args

insert

S.insert(index, value) -- insert value before index

Parameters

• i
• item

learn_one

Fits to a set of features x and a real-valued target y.

Parameters

• x (dict)
• y (numbers.Number)

Returns

Regressor: self

pop

S.pop([index]) -> item -- remove and return item at index (default last). Raise IndexError if list is empty or index is out of range.

Parameters

• i – defaults to -1

predict_one

Predict the output of features x.

Parameters

• x

Returns

The prediction.

remove

S.remove(value) -- remove first occurrence of value. Raise ValueError if the value is not present.

Parameters

• item

reverse

S.reverse() -- reverse IN PLACE

sort

References

---

1. Lai, T. L., & Robbins, H. (1985). Asymptotically efficient adaptive allocation rules. Advances in applied mathematics, 6(1), 4-22. ↩

2. Upper Confidence Bounds - The Multi-Armed Bandit Problem and Its Solutions - Lilian Weng ↩

3. The Upper Confidence Bound Algorithm - Bandit Algorithms ↩