  Structured stochastic multi-armed bandits provide accelerated regret rates
over the standard unstructured bandit problems. Most structured bandits,
however, assume the knowledge of the structural parameter such as Lipschitz
continuity, which is often not available. To cope with the latent structural
parameter, we consider a transfer learning setting in which an agent must learn
to transfer the structural information from the prior tasks to the next task,
which is inspired by practical problems such as rate adaptation in wireless
link. We propose a novel framework to provably and accurately estimate the
Lipschitz constant based on previous tasks and fully exploit it for the new
task at hand. We analyze the efficiency of the proposed framework in two folds:
(i) the sample complexity of our estimator matches with the
information-theoretic fundamental limit; and (ii) our regret bound on the new
task is close to that of the oracle algorithm with the full knowledge of the
Lipschitz constant under mild assumptions. Our analysis reveals a set of useful
insights on transfer learning for latent Lipschitzconstants such as the
fundamental challenge a learner faces. Our numerical evaluations confirm our
theoretical findings and show the superiority of the proposed framework
compared to baselines.
