We consider the stochastic multi-armed bandit (MAB) problem in a setting where a player can, at a cost, pre-observe one or multiple arms before playing one of them in each round. Apart from the classic trade-o between exploration (trying out more arms to nd the best one) and exploitation (sticking with the arm believed to oer the highest reward), we encounter an additional dilemma in each single round, i.e., pre-observing more arms gives a higher chance to play the best one, but incurs a larger cost which decreases the overall reward. We design an Observe-Before-Play (OBP) policy for arms with Bernoulli rewards, which could be generalized to any i.i.d. reward distributions bounded in [0, 1]. Our strategy could enable a better policy for secondary spectrum access in Cognitive Ratio Networks, where users can sense multiple channels' occupancies before choosing one on which to transmit. To evaluate our policy, we dene the regret as the gap between the expected overall reward gained by our OBP policy and that obtained by the expected optimum, which always chooses an optimal sequence of arms to pre-observe based on the perfect knowledge of the arm distributions. Experiments show that our OBP policy has sub-linear regret and can outperform the classical MAB algorithm when the cost of pre-observations is relatively low. Multi-armed bandit, pre-observations.
In a recent work REF , the authors study a bandit problem with Bernoulli rewards where in each round, the decision maker proposes an ordered list of the K arms, and plays the first arm with observed reward equal to 1.
Observe Before Play: Multi-armed Bandit with Pre-Observations
PERV
Computer Science Mathematics