Orthogonal Projection in Linear Bandits The expected reward in a linear stochastic bandit model is an unknown linear function of the chosen decision vector. In this paper, we consider the case where the expected reward is an unknown linear function of a projection of the decision vector onto a subspace. We call this the projection reward. Unlike the classical linear bandit problem, we assume that the projection reward is unobservable. Instead, the observed reward at each time step is the projection reward corrupted by another linear function of the decision vector projected onto a subspace orthogonal to the first. Such a model is useful in recommendation applications where the observed reward is corrupted by each individualxe2x80x99s biases. In the case where there are finitely many decision vectors, we develop a strategy to achieve O(log T ) regret, where T is the number of time steps. In the case where the decision vector is chosen from an infinite compact set, our strategy achieves O(T 23(log T )12) regret. Simulations verify the efficiency of our strategy.

An ArmWise Randomization Approach to Combinatorial Linear SemiBandits Combinatorial linear semibandits (CLS) are widely applicable frameworks of sequential decisionmaking, in which a learner chooses a subset of arms from a given set of arms associated with feature vectors. Existing algorithms work poorly for the clustered case, in which the feature vectors form several large clusters. This shortcoming is critical in practice because it can be found in many applications, including recommender systems. In this paper, we clarify why such a shortcoming occurs, and we introduce a key technique of armwise randomization to overcome it. We propose two algorithms with this technique: the perturbed C2 UCB (PC2 UCB) and the Thompson sampling (TS). Our empirical evaluation with artificial and realworld datasets demonstrates that the proposed algorithms with the armwise randomization technique outperform the existing algorithms without this technique, especially for the clustered case. Our contributions also include theoretical analyses that provide high probability asymptotic regret bounds for our algorithms.