Title: Pareto Optimal Risk-Agnostic Distributional Bandits with Heavy-Tail Rewards
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi-Objective Bandits

Bandits with Heavy-Tail Rewards

Robust Distribution Estimation
Abstract: This paper addresses the problem of multi-risk measure agnostic multi-armed bandits in heavy-tailed reward settings. 
We propose a framework that leverages novel deviation inequalities for the $1$-Wasserstein distance to construct confidence intervals for Lipschitz risk measures. 
The distributional LCB (DistLCB) algorithm is introduced, which achieves asymptotic optimality by deriving the first lower bounds for risk measure aware bandits with explicit sub-optimality gap dependencies.
The DistLCB is further extended to multi-risk objectives, which enables Pareto-optimal solutions that consider multiple aspects of reward distributions.
Additionally, we provide a regret analysis that includes both gap-dependent and gap-independent bounds for multi-risk settings. 
Experiments validate the effectiveness of the proposed methods in synthetic and real-world applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn risk-sensitive applications such as quantitative finance or healthcare, where outcomes in the tails of the distribution—like rare but catastrophic events—are crucial. Classical bandit algorithms aim to maximize expected rewards, which limits their usefulness in these aforementioned risk-sensitive areas.\nThis paper proposes a new framework for tackling multi-risk agnostic multi-armed bandit problems with heavy-tailed reward function. While previous work has explored risk-aware bandits using measures like CVaR, mean-variance, or distortion risk, these methods typically optimize a single risk criterion and often assume light-tailed distributions. In contrast, this paper addresses the need to handle multiple risk measures simultaneously under heavy-tailed noise.\n\nTo this end, the paper presents a robust estimation method called the Median of Empirical Quantiles (MoEQ), which extends the median-of-means technique to quantile estimation. This estimator is compatible with a broad class of Lipschitz risk measures without requiring risk-specific confidence bounds. They also derive new deviation inequalities for the 1-Wasserstein distance, which allow the construction of confidence intervals even in heavy-tailed settings. These tools form the basis of a new algorithm, DistLCB, which gets optimal performance guarantees across a wide range of risk criteria. The algorithm is further extended to handle multi-risk objectives, and enables identification of Pareto-optimal actions rather than a single best arm. The paper then provides both gap-dependent and gap-independent regret bounds for this setting and also establishes asymptotic lower bounds with explicit dependence on sub-optimality gaps. Experimental results on synthetic and real-world data demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**\n\nThe paper has a fine quality. It provides some novel statistical theoretical results. For example, the paper gives new deviation inequalities for the 1-Wasserstein distance under heavy-tailed distribution. It also introduces the DistLCB, which achieves near-optimality via the first lower bounds for certain types of risk-aware bandits. Overall, the theoretical results have a good quality with well-established theorems and mathematical analysis.\nRegarding the empirical results, the experiments look carefully designed to me and cover both synthetic and real-world datasets. The results also seem to support the theoretical claims. \n\n**Clarity**\n\nThe paper has good clarity. The mathematical definitions and assumptions are introduced in a clear manner without ambiguity, which is important for a theoretical work. It is great to see that the main theoretical results are accompanied with high-level explanations. \nThe notations are a bit heavy but I personally do not think there is room for huge improvement. \n\n**Significance**\nLinear Bandit is an important area of research in various fields such as statistics and computer science. \nThe paper tackles a highly relevant and underexplored problem in this area: how to perform risk-aware decision-making in multi-objective bandit settings when rewards are heavy-tailed. The heavy-tailedness is a big challenge because it prohibits the use of powerful concentration inequalities and makes the classical statistical estimator like quantile estimator unstably distributed.  \nThe results bridge multiple research directions—distributionally robust learning, heavy-tailed statistics, and multi-objective optimization—filling a clear gap in the literature.\n\n**Originality**\n\nThe paper is original in the following respects. It is the first to provide deviation inequalities for 1-Wasserstein distance under infinite-variance (heavy-tailed) settings. The MoEQ estimator represents a novel adaptation of median-of-means to quantile estimation. The DistLCB and MR-DistLCB algorithms offer a principled way to conduct exploration in multi-risk settings with strong theoretical backing. The lower bound results for SRM and DRM are also new contributions that extend existing bandit theory.\n\nQUESTIONS:\n**1**\nWhile the core results are rigorous, some theoretical analyses (e.g., lower bounds) are limited to only SRM and DRM, leaving out UBSR and CERM. This restricts the generality of the claims, even though the algorithms themselves can handle arbitrary Lipschitz risk measures. Some aspects of the empirical evaluation, such as sensitivity to the tail index pp, are not deeply explored.\n\n**2** Sensitivity to Tail Index $p$\nIn heavy-tailed settings, the tail index $p$ seems to play an important role in determining both the difficulty of estimation and the tightness of regret bounds. For example, the convergence rate of the MoEQ estimator and the confidence width explicitly depend on $p$. Lower values of $p$ mean heavier tails, which should in principle make the problem harder. The regret bounds also degrade as $p \\to 1$. However, it seems the paper does not thoroughly investigate this in experiments. I think synthetic settings are totally fine. But I do think it is important for this work to show that the algorithm maintains performance across a range of tail indices, or at least discuss limitations if it doesn’t. Afterall, the core motivation of the paper is robustness under heavy tails. \n\n\n**[Minor issues]**\n**3** \nThis is very minor but I personally suggest it would be better to also explain why heavy-tailedness brings so much challenge to the risk-aware bandit. The paper should explicitly and briefly explain (in the Appendix perhaps) that, in light-tailed settings, the problem is significantly easier, and existing methods suffice. This will make the paper more self-contained and friendly to readers not familiar with the area of bandit.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors consider a problem of multi-armed bandit with heavy-tailed rewards and multi-risk measures, i.e., the performance of an algorithm is measured by multiple risk-sensitive measures, and the reward distributions are heavy-tailed. They provided asymptotic and minimax lower bounds for single-risk bandits with heavy-tailed rewards for some classes of risk measures (SRM and DRM). They derived a concentration inequality for Lipschitz risk measures, and proposed LCB-type algorithms called DistLCB (single-risk case) and MR-DistLCB (multi-risk case). Using real-world and synthetic datasets, they demonstrated MR-DistLCB overall outperforms the baselines.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths\n    - The heavy-tailed reward and multi-risk setting would be important for practical applications (such as finance).\n    - The deviation inequality (Theorem 4.4 and Cor. 4.5) extends existing results for the heavy-tailed reward setting for a wide class of risk measures. \n    - Although this is a theoretical paper, the experiments are conducted using a real-world dataset. \n- Weaknesses\n    - As far as I understand, the lower bound analysis focuses on the single risk case. Therefore, the optimality of MR-DistLCB is unclear.\n    - Although theoretical results are novel, the technical challenges are unclear. For example, could you elaborate technical challenge of Theorem 4.4 (or other main results)?\n    - The proposed methods seem standard LCB-type algorithms, and there seems to be limited novelty in the algorithms.\n\nQUESTIONS:\n- Is there a discussion of lower bounds for the multi-risk case? Do the authors believe that MR-DistLCB is optimal?\n- Could you elaborate on technical challenges of the theoretical results (e.g. Theorem 4.4).\n- Theorem 6.3 focuses on SRM and DRM. How strong is this assumption? For example, in the domain of finance, are most popular risk measures covered by these classes of risk measures?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of regret minimization in heavy-tailed bandits. The focus is on risk-aware bandits, and the goal is to actually minimize the cumulative risk incurred by the algorithm. The concept of risk generalizes the usual expected value, and is assumed to be Lipschitz w.r.t. to a distance notion (e.g., 1-Wasserstein). The paper introduces a new estimator, Median of Empirical Quantiles, to estimate the CDF of a random variable while being robust to its heavy-tailedness. The estimator is then equipped with deviation inequalities, which are used to construct a UCB-like algorithm to deal with their regret minimization problem. The authors prove upper bounds for the regret of the resulting algorithm and matching lower bounds. Lower bounds only hold for the case of two specific risk measures.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n\n- The paper addresses a very general problem, and new technical tools are developed that may be of independent interest (e.g., MOEQ and its deviation inequalities).\n\n- The paper provides a comprehensive experimental evaluation, comparing their algorithm to baselines on both real-world and synthetic data.\n\n- Both gap-dependent and worst-case types of guarantees are provided, and even though they match only for two types of risk measures, the authors discuss this point and highlight the technical challenge involved.\n\nWeaknesses\n\n- The paper is hard to read in some points, and there are some typos (e.g., \"measure\" in line 95). In some points, there is an abrupt passage to the statement of a result, e.g. lines 141, 248, 305, where the result is immediately after the section title. Improving the writing there.\n\nQUESTIONS:\nCan the authors better clarify what the main technical challenge is when trying to extend the lower bound from the considered risk measures to a general one?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of risk-aware stochastic multi-armed bandits under heavy-tailed rewards. The authors develop a unified framework for minimizing a wide class of Lipschitz risk measures when only a finite $p$-th moment exists for some $p \\in (1,2]$. The central technical contribution is the Median of Empirical Quantiles (MoEQ) estimator, which admits new deviation bounds under the 1-Wasserstein distance. Based on this, the authors propose DistLCB for single-risk minimization, and MR-DistLCB for multi-risk Pareto optimization. The paper also establishes regret upper and lower bounds under heavy tails and provides experimental results on synthetic and financial datasets.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The proposed approach generalizes many prior works by covering both light-tailed and heavy-tailed settings, as well as both single-risk and multi-risk objectives.\n2. The Wasserstein deviation analysis for MoEQ appears to be new and enables confidence interval construction for arbitrary Lipschitz risk measures.\n3. The authors provide both minimax and gap-dependent regret bounds, and importantly, derive new lower bounds for SRM and DRM in the heavy-tailed regime.\n4. The authors also provide formal Pareto regret guarantees across multiple risk measures under heavy-tailed noise.\n\n### Weaknesses\n\n1. The term \"risk-agnostic\" is used in several places to describe the estimator or algorithms, but this term is ambiguous and potentially misleading. In other literature, “risk-agnostic” may imply risk-neutral, that is mean-based. This could lead to confusion about the intended scope of the methods, especially since the paper focuses on minimizing Lipschitz risk measures rather than mean-based risks.\n2. Unreasonable assumptions in Corollary 4.5: The sample complexity bound for the full-batch version of MoEQ (FB-MoEQ) assumes that the offline sample size $N = O(n^4)$, which is excessively large and unlikely to hold in any practical setting. This weakens the usefulness of the corollary and the empirical claims based on it.\n3. While the deviation analysis is novel, the estimator itself, i.e., median over quantiles of batches, is a natural variant of median-of-means and empirical quantiles. The innovation lies more in the analysis than in the estimator’s form.\n4. The experimental results are promising but narrow in scope. Comparisons to strong baselines are minimal, and the choice of synthetic distributions could be better motivated.\n\nQUESTIONS:\n1. Please clarify the intended meaning of \"risk-agnostic\" and consider replacing it with a more precise term such as “risk-function-agnostic” or “uniform over Lipschitz risks.”\n2. Is the assumption $N = O(n^4)$ in Corollary 4.5 necessary for the FB-MoEQ bound to hold? Can this be relaxed, or does it reflect a limitation of the MoEQ analysis?\n3. Could the empirical results include more direct comparisons to robust baselines under heavy-tailed noise, such as TED or trimmed estimators?\n4. Is it possible to extend the regret lower bounds to other Lipschitz risks beyond SRM/DRM, such as UBSR or CERM?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}