We consider a variant of the multi-armed bandit model, which we call multi-armed bandit problem with known trend, where the gambler knows the shape of the reward function of each arm but not its distribution. This new problem is motivated by different online problems like active learning, music and interface recommendation applications, where when an arm is sampled by the model the received reward change according to a known trend. By adapting the standard multi-armed bandit algorithm UCB1 to take advantage of this setting, we propose the new algorithm named A-UCB that assumes a stochastic model. We provide upper bounds of the regret which compare favourably with the ones of UCB1. We also confirm that experimentally with different simulations
Similar to @cite_5 , authors in @cite_3 propose a Thompson Sampling strategy equipped with a Bayesian change point mechanism to tackle this problem. They develop algorithms for a variety of cases with constant switching rate: when switching occurs all arms change (Global Switching), switching occurs independently for each arm (Per-Arm Switching), when the switching rate is known and when it must be inferred from data.
@cite_5 @cite_3
1944301273 2116082319
Thompson Sampling has recently been shown to achieve the lower bound on regret in the Bernoulli Multi-Armed Bandit setting. This bandit problem assumes stationary distributions for the rewards. It is often unrealistic to model the real world as a stationary distribution. In this paper we derive and evaluate algorithms using Thompson Sampling for a Switching Multi-Armed Bandit Problem. We propose a Thompson Sampling strategy equipped with a Bayesian change point mechanism to tackle this problem. We develop algorithms for a variety of cases with constant switching rate: when switching occurs all arms change (Global Switching), switching occurs independently for each arm (PerArm Switching), when the switching rate is known and when it must be inferred from data. This leads to a family of algorithms we collectively term Change-Point Thompson Sampling (CTS). We show empirical results in 4 articial environments, and 2 derived from real world data: news click-through[Yahoo!, 2011] and foreign exchange data[Dukascopy, 2012], comparing them to some other bandit algorithms. In real world data CTS is the most eective. The apparatus includes an arm pivoted between its end portions, a foot pedal for operating one of the end portions of the arm, and a switch operated by the other end portion of the arm. The pedal may be the accelerator pedal of an automotive vehicle. When the pedal is depressed, the switch operates immediately, even before the engine of the vehicle speeds up. The preferred switch has a special construction which is also part of the invention. In this paper, a physically interactive control scheme is developed for a manipulator robot arm. The human touches the robot and applies force in order to make it behave as he she likes. The communication between the robot and the human is maintained by a physical contact with no sensors. The intent of the human is estimated by observing the change in control effort. The robot receives the estimated human intent and updates its position reference accordingly. The developed method uses the principle of conservation of zero momentum for position-controlled systems. A switching scheme is developed that goes between the modes of pure impedance control with a fixed-position reference and interactive control under human intent. The switching mechanism uses neither a physical switch nor a sensor; it observes the human intent and puts the robot into interactive mode, if there is any. When the human intent disappears, the robot goes into the pure-impedance-control mode, thus stabilizing in the left position. A portable alarm device with a battery or low voltage operated sound signal triggered by a magnetic reed switch which is closed to complete the circuit by a magnet attached to a movably mounted arm, the poles of the magnet being positioned perpendicular to the longitudinal dimension of the contact strips of the reed switch to cause the reed switch to close when the magnet is in either of two positions relative to the switch thereby permitting the device to be used with doors which open inwardly or outwardly, with sliding doors, in drawers or with windows on two sides of a room. The device may also be constructed to activate a light signal simultaneously with the sound signal. In a modified form of the device, for use in the purse or in a pocket, the magnet is positioned with its poles parallel to the longitudinal dimension of the contact strips of the reed switch to cause the reed switch to close when the magnet is in one position over the center portion of the reed switch.