Bayes-UCB bandit policy.
Bayes-UCB is a Bayesian algorithm for the multi-armed bandit problem. It uses the posterior distribution of the reward of each arm to compute an upper confidence bound (UCB) on the expected reward of each arm. The arm with the highest UCB is then pulled. The posterior distribution is updated after each pull. The algorithm is described in [^1].
The reward object that is used to update the posterior distribution.
Number of initial observations per arm before using the posterior distribution.
Type → int | None
Random number generator seed for reproducibility.
Return the list of arms in descending order of performance.
import gym from river import bandit from river import proba from river import stats env = gym.make( 'river_bandits/CandyCaneContest-v0' ) _ = env.reset(seed=42) _ = env.action_space.seed(123) policy = bandit.BayesUCB(seed=123) metric = stats.Sum() while True: action = policy.pull(range(env.action_space.n)) observation, reward, terminated, truncated, info = env.step(action) policy = policy.update(action, reward) metric = metric.update(reward) if terminated or truncated: break metric
the p-th quantile of the beta distribution for the arm
This method is a generator that yields the arm(s) that should be pulled. During the burn-in phase, all the arms that have not been pulled enough times are yielded. Once the burn-in phase is over, the policy is allowed to choose the arm(s) that should be pulled. If you only want to pull one arm at a time during the burn-in phase, simply call
- arm_ids — 'list[ArmID]'
ArmID: A single arm.
Rewrite update function