Title: Provably Efficient Multi-Task Meta Bandit Learning via Shared Representations
Venue: NeurIPS 2025
Accepted: yes
Keywords: Meta learning

Multi-task learning

Linear bandits

Low dimensional learning

Representation learning
Abstract: Learning-to-learn or meta-learning focuses on developing algorithms that leverage prior experience to quickly acquire new skills or adapt to novel environments. A crucial component of meta-learning is representation learning, which aims to construct data representations capable of transferring knowledge across multiple tasks—a critical advantage in data-scarce settings. We study how representation learning can improve the efficiency of bandit problems. We consider $T$ $d$-dimensional linear bandits that share a common low-dimensional linear representation. We provide provably fast, sample-efficient algorithms to address the two key problems in meta-learning: (1) learning a common set of features from multiple related bandit tasks and (2) transferring this knowledge to new, unseen bandit tasks. We validated the theoretical results through numerical experiments using real-world and synthetic datasets, comparing them against benchmark algorithms.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles meta learning by identifying shared representations in the context of bandit problems. The core contributions are an explore-commit design to first estimate the shared representation and then exploit it in solving source tasks and the target task, with SVD for spectral initialisation. This leads to the possibility of analysing a linear bandits setup without restrictions in previous works. The proof shows that bounds can be derived for characterising the error of identified representation and the regrets of the target task, with an improvement over the previous studies. The theoretical results are verified on real-world datasets about movie recommendation task, showing notable improvement on cumulative regrets comparing to baseline UCB methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n* General scope of meta-learning of bandits problems, which have good potential to decision-making problems that may bring real-world impacts.\n* Improved theoretical bounds.\n* Demonstration of effectiveness on a real-world dataset. Highly commendable for a theoretical work for me.\n\nWeakness:\n* Unclear about the implication of the selected exploration strategy.\n* Assumptions about feature distribution might be restrictive.\n\nQUESTIONS:\nMy main question is about the distribution and the exploration policy of generating action $x_{n,t}$. To what extent the conclusions and improvements can still hold if the distribution is not as simple as a unimodal like Gaussian? I see this is listed as a research outlook in the conclusion but would still like to hear the authors' opinions as for now. \n\nPotentially more critical to practitioners, what if the exploration policy is no longer a stationary one as the assumption here. There could be lots of design space in lieu of active exploration/learning. I can see this might break the exploration-then-commit paradigm but in practice it is not always common to see a complete offline stage or an exploration policy agnostic to what has been explored before.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a meta‐learning framework for linear contextual bandits that learns a shared low‐dimensional representation across source tasks, and consequently transfers it to the target task.  For the upstream phase, the authors propose an ETC algorithm using spectral initialization to handle the non-convex nature of the problem. Authors prove that it recovers the subspace with error $\\widetilde{O} (\\sqrt{r / T})$ and leads to $\\widetilde{O}(\\sqrt{r  N  T})$ cumulative regret.  For the downstream transfer, they develop a subspace restricted OFUL variant, which leads to $\\widetilde{O}(\\sqrt{r  d  N})$ regret, compared with the naive $\\widetilde{O} (d\\sqrt{N})$ regret in linear bandits. Empirical results on synthetic data (multiple trials) and real‐world datasets (single trial) confirm significant improvements over baselines.\n\nSTRENGTHS AND WEAKNESSES:\n+ The paper is generally well-written and easy to follow, with clear notation definitions, problem definitions, as well as assumptions needed for achieving the theoretical outcomes.\n+ Based on my knowledge, the theoretical results in this paper are interesting. Authors support their theoretical finds with established error bounds, sample complexity and cumulative regret, which are comprehensive.\n+ Experiments are conducted on both synthetic and real datasets in order to demonstrate the effectiveness of the proposed pipeline.\n\n-\tMy main concern aligns with the stated limitation from the authors, namely the Gaussian assumption of feature vectors (Assumption 3.1). In particular, it is insufficient to only justify that this assumption is for the sake of theoretical analysis, instead of justifying it is viable for real application scenarios. Therefore, more thorough discussions for this assumption are needed.\n\n-\tMeanwhile, the assumption on noise $\\eta$ also tends to be stronger than most existing works: it is assumed to be i.i.d. rather than conditional independent in most existing works.\n\n-\tIt would also be beneficial if authors can include the standard deviation results for the experiments. In particular, please also conduct multi-trial experiments for real datasets and report the standard deviation outcomes.\n\nQUESTIONS:\n1.\tCould the authors please give additional justifications on applying the gaussian assumption (Assumption 1)? I believe it is important to provide sufficient discussions for this assumption as it is fundamental to your theoretical results, and such assumptions are also not standard from existing linear bandit works. \n2.\tCould the authors include additional discussions comparing your work with the kernelized multi-task learning bandits [1]? In [1], there are no such assumptions on the task feature assumptions, while their task features depend on the arm features within each task. By the way, I do not think it is reasonable to discuss this paper in your Related Works section in its current form, since [1] is not a linear bandit work.\n[1] Deshmukh, Aniket Anand, Urun Dogan, and Clay Scott. \"Multi-task learning for contextual bandits.\" Advances in neural information processing systems 30 (2017).\n\nLIMITATIONS:\nThere is not a designated limitations section. Rather, authors only states that relaxing the Gaussian modeling on task features will be considered as future works. In this context, I suggest authors include more detailed and thorough discussions on their Gaussian assumption of the task features. For instance, why would such assumption be reasonable for real bandit learning scenarios? Could it be justified from perspectives beyond merely deriving the theoretical results presented in this paper?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of multi-task linear representation learning in the linear bandit setting. Each linear bandit task $t$ has reward vector $\\theta\\_t^\\*=B^\\* w \\_t^\\*$ for a shared ground-truth subspace parameterized by $B^* \\in \\mathbb{R}^{d \\times r}$ and a task-specific vector $w\\_t^*$. The problem then becomes to learn the ground truth subspace using the provided in order to facilitate generalization to a new downstream task with the same structure. The paper proposes an Explore-then-Commit (ETC) algorithm to estimate the ground-truth subspace and an OFUL-based algorithm to solve the downstream task. These algorithms are theoretically analyzed and simulations are provided to verify the theoretical results.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The formulation is rigorous. The theoretical results are clearly explained. The paper is well-written overall.\n2. The theoretical results demonstrate the utility of representation learning in reducing the new-task regret bound from $O(d\\sqrt{N})$ to $O(\\sqrt{drN})$. A helpful proof sketch is provided, and the analysis seems non-trivial.\n3. The simulations are thorough and consistent with the theory, and also well-explained. \n\nWeaknesses\n1. The paper discusses the series of prior works that have also studied mutli-task linear representation learning in the bandit context, and mentions that these works either use a convex relaxation or assume access to the solution of a non-convex problem in order to obtain their theoretical results on subspace learning. However, in the formal analysis the paper does not discuss what distinguishes their analysis from prior work, which would be helpful to gauge the novelty considering the extensive prior study of this problem.\n2. The level of interest of this problem to the community is not obvious, considering apparent decline in interest in bandits, and also that this method is limited to linear analysis.\n\nQUESTIONS:\nPlease see weekness 1 -- can you further clarify the novelty?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}