Revisiting Follow-the-Perturbed-Leader with Unbounded Perturbations in Bandit Problems
Authors: Jongyeong Lee
Junya Honda
Shinji Ito
Min-hwan Oh
Venue: NeurIPS 2025 poster ()
Keywords: multi-armed bandits
best-of-both-worlds
FTPL
utility model
extreme value theory
Primary area: theory
TL;DR: This paper aims to broaden the theoretical foundation of FTPL and emphasize the need for further investigation to better understand the behavior of FTPL in broader settings.

Abstract: Follow-the-Regularized-Leader (FTRL) policies have achieved Best-of-Both-Worlds (BOBW) results in various settings through hybrid regularizers, whereas analogous results for Follow-the-Perturbed-Leader (FTPL) remain limited due to inherent analytical challenges. 
    To advance the analytical foundations of FTPL, we revisit classical FTRL-FTPL duality for unbounded perturbations and establish BOBW results for FTPL under a broad family of asymmetric unbounded Fréchet-type perturbations, including hybrid perturbations combining Gumbel-type and Fréchet-type tails.
    These results not only extend the BOBW results of FTPL but also offer new insights into designing alternative FTPL policies competitive with hybrid regularization approaches.
    Motivated by earlier observations in two-armed bandits, we further investigate the connection between the $1/2$-Tsallis entropy and a Fréchet-type perturbation.
    Our numerical observations suggest that it corresponds to a symmetric Fréchet-type perturbation, and based on this, we establish the first BOBW guarantee for symmetric unbounded perturbations in the two-armed setting.
    In contrast, in general multi-armed bandits, we find an instance in which symmetric Fréchet-type perturbations violate the standard condition for BOBW analysis, which is a problem not observed with asymmetric or nonnegative Fréchet-type perturbations. 
    Although this example does not rule out alternative analyses achieving BOBW results, it suggests the limitations of directly applying the relationship observed in two-armed cases to the general case and thus emphasizes the need for further investigation to fully understand the behavior of FTPL in broader settings.

Revisiting Followthe-Pertubeed-Leeddd with Unbounded Perturbations in Bandit Problems
Jongveong Lee
Junya Honda Kyoto University, RIKEN AIP honda@i k...oi..yoouuua.aa ac.
Korea Institute of Science and Technology" jongyeongokist..o wtt.nnkiiittrro.
Shinji Ito
Min-hwan Oh
The University of Tokyo, RIKEN AIP shinji@min feiit. i.us u.utokko..as jc.
Scoul National University minoh@snu.co ac. ac.kr
Abstract
Follow-the- thhe-Regularized-Leeeeee (FTRL) policies have achieved Best-of-Both- Worlds (BOBW) results in various settings through hybrid regularizers, whereas analogous results for Bealowtthe-Pertubeed---- inherent analytical challenges. To advance the analytical foundations of FTPL, we revisit classical FTRL-FTPL duality for unbounded perturbations and establish BOBW results for FTPL under broad family of asymmetric unbounded Fréchet- type perturbations, including hybrid perturbations combining Gumbel-type and Fréchet-type tails. These results not only extend the BOBW results of FTPL but also offer new insights into designing alternative FTPL, policies competitive with hybrid regularization approaches. Motivated by earlier observations in two-armed bandits, we further investigate the connection between the 1/2-Tsallis entropy and Fréchet-type perturbation. Our numerical observations suggest that it corresponds to a symmetric Fréchet-type perturbation, and based on this, we establish the first BOBW guarantee for symmetric unbounded perturbations in the two-armed setting. In contrast, in general multi-arman armed bandits, we find an instance in which symmetric Fréchet-type perturbations violate the key condition for standard BOBW analysis, which is a problem not observed with asymmetric or nonnegative Fréchet-type perturbations. Although this example does not rule out alternative analyses achiev- ing BOBW results, it suggests the limitations of directly applying the relationship observed two-armed cases to the general case and thus emphasizes the need for further investigation to fully understand the behavior of FTPL in broader settings.
(FTPL) remain limited due to
Introduction
In multi-armed bandit (MAB) problems, an agent plays an arm I from set of K arms each round t e [T] = (1....,T) T) over a time horizon T. After playing an arm, the agent observes only the loss
lt,1, of the played arm, where the loss vectors f, (f,,1,...,, (e,x) environment. Given the constraints of partial feedback, the agent must handle the tradeoff between gathering information about the arms and playing arms strategically to minimize total loss.
[0. 1] are determined by the
Although numerous policies have been developed for MAB, many of them can be largely classified into two primary frameworks, namely Follow-the tht--erturreedd....
(FTPL) [32 and Follow- the Regularized-Leade (FTRL) [6]. While FTPL was inspired by game theoretic approach [22 25
and FTRL emerged from the context of online optimization (26). both frameworks have analogs
He was affiliated with Scoul National University the time of submission.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

in discrete choice theory, a field in economics that models decision decision-making through probabilistic frameworks to maximize utility over finite alternatives [5T] In particular, FTPL is known as the
additive random utility model
S 50]. while FTRL corresponds to the representative agent model [4]
Beyond these conceptual analogies, a line of work has formalized the relationship between FTPL and FTRL in discrete choice theory [2] In particular, when the joint perturbation distribution has strictly positive density on IR* the existence of corresponding regularizer, along with detailed
results on its properties, has been established [9] 44 A classical example is the multinomial logit model 40), known to be equivalent to both FTPL with Gumbel perturbations (also known as Exp3 7 48) and FTRL with Shannon entropy regularizer [4]. In the context of online learning, Abernethy et al. 3 discussed this relationship for general perturbations, where the formal theorem was later established for the independent and identically distributed (i.i.d.) perturbations absolutely continuous with respect to Lebesgue measure by Suggala and Netrapalli [49 Proposition 3.1].
While the above results mainly discuss the transformation of FTPL into FTRL, Abernethy et al. [1] demonstrated that (nearly) every instance of FTRL can be viewed as a special case of FTPL in one-dimensional online optimization and more general equivalences are discussed by Feng et al. [9 Nevertheless, when K 2 4, no FTPL counterpart exists for FTRL with log-barrier regularizer 27 Proposition 2.2] and Tsallis entropy regularizer [33 Theorem 8]. These findings indicate that FTRL strictly subsumes FTPL as a special case.
Despite this narrower coverage, FTPL has gained significant attention due to its computational efficiency and simplicity, making it suitable for variety of problems in online learning, including combinatorial semi-bandits [43] online learning with non-linear losses [7]. and MDP bandits [15] Still, while FTRL policies have achieved optimal results in several problems such graph bandits [16 and partial monitoring [52], comparable progress for FTPL has been relatively underexplored.
This gap is primarily due to the complexity of expressing arm-selection probabilities of FTPL, which poses significant analytical challenges despite its computational efficiency in practice. In the standard analysis of FTRL and FTPL, a key factor in achieving the optimal adversarial regret is evaluating the stability of the arm-selection probability against the changes in the estimated cumulative loss. Abernethy et al. 2 tackled this challenge by leveraging the hazard function of perturbations, but it only resulted in near-optimal regret of O (VKTlog K) Later, motivated by the observation in two- armed bandits that FTRL with B-Tsallis entropy roughly correspond to Fréchet-type perturbations with tail index (1 (1-3) -3- Kim and Tewari (3) conjectured that FTPL with Fréchet-type distributions could achieve optimal O VK) regret. Recently, this conjecture has been partially validated, as it was shown that FTPL with nonngetative Fréchet-type distributions, under certain conditions, indeed obtain the optimal O(VKT) regret and even Best-of-Both- Worldd (BOBW) result [22] 28 38).
Here, unfamiliar readers can think of Fréchet-type as any distributions whose right tail decays polynomially.
Contribution Since hybrid regularizers are typically constructed by summing distinct regulariz- ers 22 (6), a natural analogue for FTPL is to construct hybrid perturbations, either by summing perturbations drawn from different distributions or by specifying different behaviors for the left and right parts of the density. This motivates the study of asymmetric or unbounded perturbations, where "unbounded" distributions in this paper refers to distributions that are unbounded on both the positive and negative sides and "eemi-infinite" refers to those unbounded on only one side. However, the requirement for existing results is casily violated when the perturbation has strictly positive density on E since existing results require the ratio of density to cumulative distribution function, f/F, be monotonically decreasing over the entire support.
In this paper, we first relax these previous conditions for the BOBW guarantee by providing milder conditions under which FTPL with unbounded perturbations achieves optimal O(VKT) regret and even logarithmic stochastic regret, thereby achieving the BOBW result. As shown below, the key conditions related to the difference in tail indices of right and left tails, where (right) tail index roughly corresponds to the order of polynomial decay, Pix-DIX [X Z x] RJ =a.
Proposition 1.1 (asymmetric, informal). Let Drr denote the unimodal Fréchet-type distribution fully supported on RR under mild conditions, with right and left tails characterized by tail indices 2 and > O respectively. If a E R21 U (co). FTPL with D2 achieves BOBW guarantee for K > 2.
Here, Fréchet-type perturbation with infinite index denotes the Gumbel-type family, which of exponential tail such as exponential distribution and Gamma distribution. Consequently, D. with a

---

oo can be seen as hybrid perturbation, combining the Gumbel-type and Fréchet-type distributions, analogous the hybrid regularizers widely used in FTRL [T] 311. While remains unclear which regularizers are associated with these hybrid perturbations, we expect that our findings can provide insights into developing an FTPL policy that serves as an alternative to a hybrid regularizer for FTRL. For example, we expect it can approximately reproduce the combination of Tsallis entropy with Shannon entropy, -x log æ, or Shannon entropy for the complement, - - I)) log(1 - z), where the latter has been used in FTRL policies [52 [6].
While asymmetry appears natural the context of hybrid perturbations, one may wonder whether it is essential or just a technical artifact. To explore this, we revisit the original motivation for using Fréchet- type perturbation, specifically its equivalence FTRL with Tsallis entropy in the two-armed bandit setting [33] where the latter is well-known BOBW policy [55]. Although the exact distribution is not available in closed form, our numerical analysis suggests that the perturbation associated with 1/2-Tssllii entropy for K = 2 can be realized by a symmetric Fréchet-type distribution. Therefore, it natural to expect that the BOBW guarantee can be extended to symmetrie Fréchet-type at least for
K = 2, and perhaps even K 2 3. However, our analysis reveals limitation in the latter case.
Proposition 1.2 (symmetric, informal). For FTPL with Fréchet-type distributions with both left and right tail indices equal to 2, the BOBW result holds when = 2. However, for K 2 3, the key conditions required by standard BOBW analyses are violated unlike the asymmetric tail indices.
Details this key condition provided in later sections. Here, it is worth noting that no perturbation exactly reproduces Tsallis entropy for K 2 4 [33.. Yet, the regularizer induced by the symmetric Pareto distribution, the simplest example of symmetric Fréchet-typp type with density f f(r) 1/[[x]+ behaves like the Tsallis entropy even in the counterexample for K 2 3. Although alternative, non- standard techniques might be able to establish general BOBW guarantees for symmetric Fréchet perturbations, our findings, together with the impossibility result in Kim and Tewari (3), highlights the difficulty of extending the BOBW guarantees beyond K 2. Further investigation is therefore needed to develop a deeper and more comprehensive understanding of FTPL in broader settings.
2 Preliminaries
In bandit problems, the loss vectors l; are determined typically in either stochastic or adversarial manner. In the stochastic setting, the loss vector l, i.i.d. from an unknown but fixed distribution over (0, 1|k Hence, one can define the expected losses of arms μ4; == [R[,,.] and the optimal arm 8" E arg min μ4. The suboptimality gap of each arm is denoted by A, = #i 14- and the optimal problem-dependent regret bound is known to be Σt (log T T/A,) [36. On the other hand, in the adversarial setting, an (adaptive) adversary determines the loss vector based the history of the decisions, and thus no specific assumptions are made about the loss distribution. In this environment, the optimal regret bound is KT) g. A policy is called a BOBW policy when the policy achieves (near) optimal guarantee for both stochastic and adversarial setting (10).
2.1 Seeoo-tth-Perturbeeee Leader and Follow-the- Reghhaa-egul Leader policies
Since the agent only observes the loss of the played arm, L, one uses the loss estimator l. Let L, - &e denote the estimated cumulative loss up to round t, with L, = >x=1 la as the true cumulative loss. Then FTPL is policy that plays arm
1=
1, arg min (E[K]
T1,s
where Ti DK
(FTPL)
where Th is the learning rate specified later and r, ( denotes the random perturbation vector drawn from the distribution DK over R In the case of i.i.d. perturbations, which are common in the bandit literature, we denote the common distribution by D. The probability of playing an arm
i E [K] by FTPL, given L. is denoted by Wie = dx(Te D)), where for À E R
(,(); Pr P>DK
arg min {)₃ - -r33) JE(K]
When the perturbations are i.i.d., we denote (b; D by (): We denote the distribution function and density function of D by F and f. respectively. Mcanwhile, FTRL plays arm
3

---

according to the probability vector Pt. i... 1 Pe. defined as
{(esD) + V(p) PEPx-1
P: - p( Lu; V) = arg min
(FTRL)
7/t S
where Px denotes K dimensional probability simplex and V PPk-1 + Rgo denotes a regularizer function. Therefore, FTPL and FTRL are equivalent if (A; D) (A;(A; v) holds for any À E R
For example, is known that p(A; Vs)= - <5(;; Gumbel), where Vs(p) = EvEK] pi log denotes negative Shannon entropy [4].
Vs(p))-Enekk
For the unbiased loss estimator, both FTPL and FTRL policies often use an importance-weighted (IW) estimator & = f,t,/ Pr[l. =]] of the loss vector é, where e; is the i-th standard basis vector. In general, Φ; does not have closed form, making computations of o; and the IW estimator & diffieult. To address this, FTPL policies usually construct & with a geometric resampling estimator of 1 1/ instead of explicitly computing we,s [28 43 Unlike FTPL, FTRL can directly use Pt, to construct & since it plays arm according pt obtained by solving optimization problem.
2.2 Standard analysis techniques
The standard FTRL regret analysis proceeds by decomposing the regret (largely) into two pieces, (i) stability term, which is usually related to the Pt itself, and (ii) a penalty term that is related to the value of regularizer function (p:) 30 37 45 By choosing the regularizer and learning rate so that these two terms are of the same order, several FTRL policies obtain the optimal performance [16 [2]]22)) 55).
In this context, it is known that uniform bound on
3/2 is sufficient condition to guarantee ((v KT) regret, where (A) = (A) 19. as this ensures the stability and penalty terms remain of the same order. For FTRL with 3-Tsallis entropy, defined as 113 Σnς refined
analysis showed that uniform bound on P²/pp²
2-8 is sufficient to guarantee O(VKT) regret for (0.1) 1) (2) Zimmert and Seldin (55) further provided a unified analysis that covers [0. 1]. achieving O O(VKT log K) regret for B 1, O (VKTTlog 7') regret for B 0 and even BOBW guarantee for 3 1/2..
While FTRL with Tsallis entropy has achieved the optimal O(VKT) regret in MAB problems, Kim and Tewari (33 showed that no corresponding FTPL policy exists for FTRL with the Tsallis entropy regularizer when K 2 4. This makes unclear how to apply the standard FTRL techniques to FTPL to achieve the optimal results. As a result, only a near optimal O(KKT K) regret
was obtained by considering uniform bound on -0//02 (2), rather than - To address this challenge, Honda et al. 28 and Lee al. derived refined bounds on (n( that depends on 1/ //.,, which is indeed related to w rather than relying uniform constant. Here, denotes the index of Fréchet-type perturbations. The dependency of 1/a arises naturally from observations in two-armed bandits, where FTRL with 3-Tsallis entropy roughly corresponds to FTPL with Fréchet-type perturbations with index (1 -B)-1, which will be elaborated in Section
1/a
This refined analysis enables the derivation of BOBW guarantees for FTPL policies, illustrating optimal performance (up to multiplicative constant) in both stochastic and adversarial settings. In particular, they derived that for adversarial regret, -6' (n,Le) //,,, 00;,-//) when Le, is the -t smallest among the components of Lt and for stochastic regret, (n, / = O(1LL,)). These bounds provide more refined characterization of the sensitivity of arm-sclection probabilities to changes in estimated losses, leading to improved regret analyses for FTPL.
2.3 Previous assumptions on perturbations in FTPL literature
In econometrics, FTPL with i.i.d. Gumbel distribution has been extensively studied since its arm- selection probability model can be written in the closed-form, known the multinomial logit model
40 On the other hand, in the online learning literature, Kalai and Vempala [2) introduced FTPL with the exponential distribution. Following this work, FTPL policies for bandits have mainly adopted exponential distributions [34] 35 43 46). where oear-optimal results are established.
Beyond distribution-speciif analysis, based on the relationship between FTRL and FTPL., Abernethy et al. (2) provided unified analysis for perturbations with bounded hazard function f/(1 F). How- ever, using this result with best tuning of parameters always leads to near-optimal O( /KTT/Klo log K)

---

regret [31, which provokes to explore different approaches to achieve the minimax optimality. While there no corresponding FTPL for FTRL with Tsallis entropy in general. Honda al. [28] showed that FTPL with Fréchet perturbation with shape 2 can achieve optimal P(VKT) regret and even BOBW result, which is generalized to Fréchet-type perturbations satisfying certain conditions [38] To avoid unnecessary technicalities, we state here minimal, sufficient definition and the key relevant conditions, where the full and precise statements appear in Appendix A for completeness.
Definition 2.1 (Informal). A distribution is said to be Fréchet-type type with index if the tail function satisfies 1 - F(x) = (x =) for æ >> 0, where O hides polylogarithmie factors. Assumption 2.2. SuppD C [0, oo) and the hazard function f/(1 F) is bounded. Assumption 2.3. f/F is monotonically decreasing in the whole support.
Let Da denote the set of distributions satisfying all the conditions in Lee et al. 38 given in Appendix A. including Assumptions 2.2 and 2.3 where index >> represents the tail index. It was shown that FTPL with D E D. can achieve O(VKT) adversarial regret for any > 1 and Although the nonnegativity assumption was introduced for simplicity, worth noting that Assumption 2.3 can be easily violated when suppD = R, which implicitly requires the use of semi- cemi-infinite perturbations.
further O log T/A,) stochastic regret for
Relationship between FTPL and FTRL
In this section, we revisit classical results in discrete choice theory under the assumption that the density strictly positive on RK While the existence of corresponding regularizer for FTPL has been extended to general distributions [19,49] 49), unbounded perturbations remain the most thoroughly analyzed and thus offer a solid foundation for understanding FTPL. We also note that, unlike the bandit settings where FTPL typically uses i.i.d. perturbations, discrete choice models allow for correlated perturbations.
3.1 Classical results in discrete choice theory
For perturbations fully supported R-K, the existence of corresponding regularizer has been established by classical results discrete choice theory 27 41
Following the conventions in discrete choice theory, where the objective is to maximize rewards (which is analogous minimizing losses), in this section, we redefine the arm-selection probability for arm in terms of cumulative rewards v, which corresponds to the negative loss, instead of cumulative loss À
arg max (v, r3)
(v(()) = Pr
(2)
jE(K]
where rys are random perturbations. This definition relates the loss-based definition (A) (v) as the gap of cumulative losses X; in (T) can be written in terms of the gap of cumulative rewards by A; - miny. A = max, V/ - 14. is known that for any given FTPL policy, corresponding FTRL always exists, although the associated regularizer may lack a closed-form
Lemma 3.1 (Theorem 2.1 in Hofbauer and Sandholm (27)). For defined in (2). let the joint distribution of random vector T be of finite mean and absolutely continuous, and fully supported on k Then, there exists a regularization function V Int (Pk-1) + R. such that
p(v)
arg max
((P (p)},
pEInt (Px-1)
where Int. denotes the interior. Moreover, can obtained as the Legendre transformation of the potential function & ip, i.e.,
V(p) 4' 4'(p) = sup (p, v) (DV), where VΦ(V) = v(v).
VERK
Here, potential function of (TV), denoted by (vv). is known as surplus function of the discrete choice model [2]. which is defined as (Dv) TE, [max]g[k] (K) ii + rz] so that d E, [arg max, 14 r.] (P)). By definition, the location normalization on reward и does not change the arm-selection probability. Therefore, we can assume Vk without loss of generality.
5

---

3.2 Perturbations on the real line: connection to generalized entropy
requires DK to have strictly positive density on T& which might appear somewhat restrictive. For this reason, Lemma 3.I extended to general distributions with absolutely continuous density in Feng et al. 19 and Suggala and Netrapalli [49]. Still, it is known that several useful properties hold when we assume perturbations are fully supported on R For example, Norets and Takahashi [44] showed that yp is a bijection between the space of reward vectors I RK-1 x {0} and that of arm-selection probabilities Int (Px-1), that is, for any p Int (Pk-1). there exists unique E I k x {0}, and vice versa. Furthermore, Fosgerau et al. [21] showed that the associated regularizer can be expressed as a generalized entropy:
Lemma 3.1
((p) p log S(p), where S RR B is the inverse function of (c
They also showed that the arm-selection probability can be expressed by using expected Bregman divergence under some distribution associated with the negative generalized entropy. Investigating unbounded perturbations therefore natural for two reasons: (i) reconnects FTPL theory with the perturbations already studied thoroughly, and (ii) it enables the design of analogues to hybrid regularizers in FTRL. The following sections formalize this intuition and shows that a broad class of asymmetric Fréchet-type perturbations is sufficient to achieve BOBW guarantees.
4 BOBW guarantee with asymmetric perturbations
We now consider class of hybrid perturbations whose left and right tails follow different types of distributions, family of unimodal asymmetric distributions Ua.3 supported on R, defined by
0.9
LP
-35P
0.6
if 2 0,
F(x:Ul.,9)
2
2
F F(-r:De)
if x < O,
where Dr D, and Dz D are Fréchet-type distributions over [0, oo) with tail indices a > 0 and 3 >> 0, respectively. For exam- ple, one can consider Laplace-Pareto distribution with density and distribution defined by
Figure 1: Density examples
x < O,
x O,
[LP(x)
and FLr(x))
(++1)
2 0,
2)++1) 1)
I 0.
Although, strictly speaking, the Laplace distribution Gumbel-type distribution, it can be seen as Fréchet-type distribution with oo, that is, D Another example is the asymmetric Pareto distribution, which is equivalent to a generalized Pareto distribution with shape 3 on the negative side and Lomax distribution with shape 2 on the positive side, such that
(3/2)*
x < O,
0,
fasPa,3
(3/2-2)*
and FasP (x) = 33/2-233
(5)
[x+1)3
x 0,
2
& 2 0.
For illustration, we provide the plot of density functions of fLP and fasPa.s in Figure
In the previous BOBW analysis of FTPL, Assumption 2.3 which requires the monotonic decrease of f/F, played a crucial role in bounding - /2 However, for Ug,9, the interval where f/F increases can be unbounded, e.g.. asymmetric Pareto distribution (5), and so we cannot directly apply the previous analysis. To address this issue, we derive tighter evaluation of - by explicitly accounting for the negative term induced by (f'(x) for x < O, which leads the following result.
Theorem 4.1. Let a; be the rank of X; in the nondecreasing order of X1...., AK (ties are broken arbitrarily). When for any R that
and B 2
then FTPL with the unimodal distribution Ua, satisfies
(((2) (((A)
S 50 1/) ^ ^o
A4 -x--)))))
1
Theorem 4.1 shows that when the left tail index 3 of the distribution (associated with negative perturbations) is lighter than the right tail index a by two, the standard analysis techniques used in Honda et al. (88 and Lee et al. 38 can be largely applied, leading to the following results.

---

Corollary 4.2. Ifa > 1 andd 3 a+2, then FTPL with Ma,3 and learning rate of order k - 2//b/V /vE achieves O KT) adversarial regret for all 1. Moreover, a 2, FTPL achieves stochastic regret of E (log T)) provided there is a unique optimal arm.
Similarly to Theorem 4.1, establishing Corollary 4.2 requires a more refined evaluation to address the unbounded interval where f'(x) > 0 in the regret decomposition. We can also derive explicit regret bounds for specific distribution in 02 by explicitly using the expression of the distributions, whose example is given as follows.
Theorem 4.3, FTPL with learning rate 1)t = m and the Laplace- Parreto distributions in 4 satisfies
Reg(T) < (60m
5.7
KT +
2K
+ +
log(T + V KT
27
2m
whose dominant term optimized as Reg(T) v 49.25v KT + OO(V K + K KlggT) when m = 0.23.
Using the techniques from Theorem 4.1 which address additional terms due to negative perturbations, we can also obtain logarithmic regret in the stochastic setting with unique optimal arm.
Theorem 4.4. Assume that i = arg min K) Ms is unique and let A min A.. Then, FTPL
with learning rate T/ = m/ Vt for m > 0 and Laplace- PPrreto perturbation in (4) satisfies
RTg(T)= (60m 2 log T
(2K
log(T (T' + +1)+6 + (107m +3/m (3/m)2KK K
+
e²
0.035A;
27
Although the dominant term in Theorem 4.4 is larger compared to that of FTRL policies, it can be further optimized by using the same arguments as in Honda et al. [28 Remark 12]. Theorems 4.3 and hndg4.4 the potential of using hybrid perturbations, as Laplace-Parett perturbations can be roughly viewed as hybrid regularizers combining Shannon and Tsallis entropies. This is because the Laplace distribution belongs to the Gumbel-type, which are associated with the Shannon entropy. Remark 4.5. While provided concrete results for asymmetric perturbations with different tails, it is also natural to consider hybrid perturbations formed by summing perturbations generated from different distributions. In many such cases, the density may not be available in simple form. Nevertheless, our conditions are sufficiently general to be verified even in such settings.
5 FTPL with symmetric perturbations
While asymmetric unbounded perturbations would suffice for constructing hybrid perturbations, is important to examine whether the asymmetry is truly essential merely technical artifact for better understanding of FTPL. To this end, we revisit the original motivation for using Fréchet-type, namely, its equivalence to FTRL with Tsallis entropy in the two-armed bandits (33. Since FTRL with Tsallis entropy is well-established BOBW policy, this connection would provide a natural starting point for investigating this question, even though such equivalence is established only for K
When = 2, Pi becomes a line segment, where we can write p = (x, - x) for some x E (0, 1). In this case, there exists a unique reward vector v(p) = (c(x), 0) for (c(x), = x (p) 44 Accordingly, we can define a regularizer V + R that coincides with V(p) in terms of x as
((x) == ((x,1 1-
1-11-x) x)) = xc(x) 4 ((((x)))). Vz E (0,
which satisfies (x( c(x). On the other hand, have Pr(c(=)) + 2 r2] in two-armed bandits by definition, which implies that (x(x) is equivalent to the quantile function of r2 Tı-
When Va(p) 3-3 r. denotes B-Tsallis entropy regularizer for 3 E (0, let DJa denote the
distribution r2 TI and F(::D) be a CDF of D. Then, we have F(c(x);D)+ = x, that is
3
(x (1- 1))x))= x))3 DTs = x,
(6)
which shows that DJ' is symmetric. By letting c(x), we obtain for B E (0,
2 ff DF:)
8 1-8 (x (1 3-1-11 3
1
lim x
lim z-1
1 1- B' (7)
1- F(z; D DTv
7

---

1/2-Tsalli -sallsls
0.9
Symmetic Pareoo syhte
2
0.2
-20
-10
20
Figure 2: Cumulative sum of IFT.
Figure 3: Output of IFT and densities.
Here. 7 known
Mises condition for Fréchet-type, which is a sufficient condition for distribution to be Frécher-type [2]]. This implies that D i.e., the distribution of difference
between two random variables, is Fréchet-type with index (1 (1-3) -)
While distribution (2) D2 satisfying 66 can be easily constructed by dependent perturbations
in Feng et al. [19] as explained in Appendix A.3 it remains unclear whether D is realizable by two
i.i.d. perturbations rı, ~ Da. Kim and Tewari 33 simply conjectured that it can be realized by ii.d. Fréchet-type D. Though it still difficult show the existence of i.i.d. perturbation, we can formally show that D. if it exists, is indeed Fréchet-type, as stated below.
X- for X ~ Let (r1,72) be i.i.d. from DD and 'D Lemma 5.1. Let DD denote the distribution - be distribution of
r2. Then, Df is Fréchet-type, then either 'D or -D is Fréchet-type.
The proof of this lemma is given in Appendix G.1 where we analyze the tail behavior of distributions using results from extreme value theory and tail convolutions. Recall that these observations are made under the assumptions of Lemma 3.1 which implies that D is also fully supported on R and absolutely continuous it exists.
5.1 Perturbation associated with Tsallis entropy: numerical observation
In two-armed bandits, we can view FTRL with Tsallis entropy as FTPL with some Fréchet-type perturbations. While the existence of i.i.d. perturbation reproducing Tsallis entropy not formally shown, numerical approximations suggest the following conjecture.
Conjecture 5.2. There exists a symmetric distribution such that the corresponding FTPL reproduces FTRL with 1 1/2-Tsallis entropy regularizer.
To prove Conjecture 5.2 it suffices to consider the characteristic function distributions. Let g(t) be the characteristic function of DJa, i.e., (Efea((n -ra)] and g that of DJ' By definition of D. we have g(t) g(t)g(-t). Since DT= is symmetric by (6). g(t) is real and positive function. Therefore, proving Conjecture 5.2 is equivalent to showing that vē is characteristic function of some probability distributions. We expect that this conjecture can be proved by showing that vī is positive definite, which is necessary and sufficient condition, according to Bochner's theorem [18).
Numerical validation By definition, 9 can be expressed in terms of the quantile function c(-) as
{.² cittel) dp ['' exp exp(-it(p
g(t) 100=/0 sta s(x; (TT; =
11/2 1/2 -1- p1)pp)1/2) dy.
which can computed numerically for any E R. If Vg valid characteristic function, then applying the inverse Fourier transform (IFT) vē should yield the appropriate density function. Note that Vø is real-valued regardless of symmetry of DTs, g(t) itself is real and positive.
Figure 2 shows that the cumulative sum of the IFT numerically converges to 1, suggesting that vg would be valid characteristic function. In Figure 3 the blue solid line represents the IFT of vg. while the red dashed and green dotted lines represent the density functions of the symmetric Pareto with shape (Fréchet-type) and standard Laplace (Gumbel-type) distributions, respectively. As shown there, the IFT of Vo closely resembles the density of symmetric Pareto distribution with shape parameter 2, though minor fluctuations appear due to numerical approximations. This behavior supports the conjecture that DT2 is symmetric and Fréchet-type with index 2. Details on
the numerical evaluation and further results including the imaginary part are provided in Appendix

---

5.2 Two-armed bandits
The numerical observations indicate that FTRL with 1/2-Tsallis entropy, BOBW policy, corresponds to FTPL with certain unimodal symmetric Fréchet-type distribution with index 2. While [ and the previous analysis do not extend to the symmetric Fréchet-type perturbations, it is reasonable to expect that FTPL with certain unimodal symmetric Fréchet distributions can achieve BOBW, at least in two-armed bandits. One of the simplest examples of such distributions the symmetric Pareto distribution, illustrated in Figure 3]
where we obtain the following positive result as expected.
Proposition 5.3. Let the perturbations be i..dd. from the symmetric Pareto distribution with shape 2 whose density function and distribution function are respectively defined as
[(x)
1 (10) 133 and F(x)=
2(1 2(1-x)2
I < 0,
r20 0. 2(++1)
Then, (/ is uniformly bounded for any X E R22 and E (1,2)-
Corollary 5.4. FTPL with symmetric Pureto perturbutions with shape 2 in Proposition 3.3 achieves O(VKT) adversarial regret and Ollog T/ //A stochastic regret when K 2
Note that [(x)/F(x)) शা| for x < O, which is increasing on RR. This observation demonstrates
that Assumption 2.3 and asymmetry of the perturbation are not necessary conditions, as far as K 2.
5.3 Generalization to multi-armed bandits
While standard Fréchet and Pareto perturbations whose support R2 do not correspond to FTRL. with Tsallis entropy, they can indeed attain optimal results for general K [28 38 Although there is no perturbation associated with Tsallis entropy for K 2 4 [33]. one may expect the optimality of FTPL with symmetric Fréchet-type perturbations as achieved by asymmetric (Corollary 4.2) and
nonnegative perturbations 38) One would guess the main difficulty for K 2 3 arises when @ or A;
not identical to each other, complexity that does not occur in K = 2. Surprisingly, however, find that the problem happens even when the losses of the currently suboptimal arms are identical, i.e, when d lies on the line segment as in K 2.
Proposition 5.5. Let the perturbations be i.i.d. from unimodal Fréchet-type Uo with aj- 2, satisfying x (x; Da2)/ (11) F(x; Da $ 2 for any x >> O and j E (1,2). If E RKK satisfies (0.c. ......... c), then for + 1 and c 2 2/K.
(())
52 c+2
and -0,(1) (a(A)
2 Ω KVK
(8)
KVK
Although the condition in Proposition 5.5 may seem restrictive, it is known to hold for well-known
Fréchet-type distributions such as Fréchet, Pareto and Student-t (38]. The LHS of ( means that
-0'/0"/ (1) does not hold, which is the key tool to obtain O KT) regret with simple learning rate 9 (55) In addition, while the RHS of (8) does not contradict another key tool -0'/6 ((1)
for O WVKT KT log K) regret [2), it violates the key property, -6' / = ((1/),) used for achieving logarithmic regret in the stochastic setting (28 38). The proof of Proposition 5.5 and additional results for K = 3 with any a > 1 and with symmetric Pareto distributions are given in Appendix G These results suggest that perturbations with equally heavy tails lose some essential properties preserved in FTPL with asymmetric Fréchet-type perturbations and FTRL with Tsallis entropy.
Building on this result, it might be reasonable to expect that the corresponding regularizer for symmetric Pareto distribution behaves differently from 1 1/2-Tsallis entropy when K 2 3. However, we obtain similar results even in cases in Proposition 5.5 where the symmetric Pareto fails.
Proposition 5.6. When K = 3 and p- = (I, 1-x 1-2) for x E (1/3,1) 1), let Var(x) be a correspond- ing regularizer of the symmetric Pareto distribution with shape considered Proposition 5.3 Then, it holds that
2
1
2/2
2V1 - - -1 Vz))))))) V <
V1-- 11-
which implies lim Vzp(x) (x) is of the same order up to multiplicative constant with the limit of derivative of 1/2-Ts /--lisllis entropy regularizer since (z)
v2 1/

---

Proposition (.6 that similar results to the K = 2 case can be obtained even when the probability vector lies on the line segment in P2. Notably, the case considered Proposition [5.6]is the same as that in Proposition 5.5 which serves as counterexample where the standard analysis cannot be applied. This suggests that there are limitations to arguments based solely on analogies from the equivalence in two-armed bandits.
Remark 5.7. These results do not rule out the possibility of FTPL with symmetric Fréchet-type distributions achieving BOBW guarantees or optimal adversarial regret. Still, the ratio between ø' and & have been believed to be the key for the regret bound. For example, it is conjectured by Abernethy et al. [2 Conjecture 4.5] that FTPL with Gaussian perturbation suffers linear regret because of the unbounded - regret bound for FTPL with Gaussian perturbation still suffers the regret with additional log T factor, i... O(v WKVKKllog log K log T)
/é. /0//6.
Although this conjecture was later shown to be false, the known
[39] Our conjecture is in parallel to these observations when we try to achieve the optimal O(VKT) regret, instead of (VVKT log K) regret. We conjecture that the behavior of -0'/ó of symmetric perturbations may introduce additional factors depending on K or T, thereby limiting the achievable regret near-optimal rates.
Conclusion
In this paper, we first extended the BOBW guarantee of FTPL to asymmetric perturbations under mild conditions, including a hybrid of Gumbel-type and Fréchet-type perturbations. Given the development of several FTPL policies with hybrid regularizers, a promising direction for future work designing computationally efficient FTPL counterparts to FTRL policies by incorporating hybrid perturbations in complicated settings. While FTRL with 1/2-Tsallis entropy regularizer (numerically) corresponds to FTPL with symmetric Fréchet-type perturbations, and FTPL with its tail-equivalent perturbations achieves BOBW guarantee in two-armed bandits, found that this result does not straightforwardly extend to general case. This finding highlights the limitations of directly extending the relationship observed in two-armed bandits the general case, emphasizing the need for further investigation to better understand the behavior of FTPL in broader settings.
Acknowledgments and Disclosure of Funding
JL was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 14-2024-00395300 by AI-Bio Research Grant through Scoul National University, and this work was also supported by the grant Nos. 2023-00262155; 2024-00460980; and 2025-0230471 17 (IITP) funded by the Korea government (the Ministry of Science and ICT). JH was supported by JST/CREST Innovative Measurement and Analysis (Grant Number JPMJCR2333), SI was supported by JSPS KAKENHI Grant Number JP25K03 884 MO was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.
RS-2022-NRON 853 and RSS2003-00222263)
References
[1] Jacob Abernethy, Chansoo Lee, Abhinay Sinha, and Ambuj Tewari. Online linear optimization via smoothing. In Conference on learning theory, pages 807-823. PMLR, 2014.
[2] Jacob Abernethy, Chansoo Lee, and Ambuj Tewari. Fighting bandits with a new kind of smoothness. Advances in Neural Information Processing Systems, 28, 2015.
[3] Jacob Abernethy, Chansoo Lee, and Ambuj Tewari. Perturbation techniques in online learning and optimization. Perturbations, Optimization, and Statistics, 233. 2016.
[4] Simon Anderson, André De Palma, and Jacques-Franqee Thisse. representative consumer theory of the logit model. International Economic Review, pages 461-466, 1988.
[5] Simon P Anderson, André De Palma, and Jacques-Françole Thisse Discrete choice theory of product differentiation. MIT press, 1992.
[6] Jean- Audibert and Séhastien Bubeck. Minimax policies for adversarial and stochastic bandits. In Annual Conference Learning Theory, volume 7, pages 1-122, 2009.
10

---

[7] Peter Auer, Nicolo Cesa- -inachiacchi Yoav Freund, and Robert E Schapire. Gambling in rigged casino: The adversarial multi-armed bandit problem. In Annual Foundations of Computer Science, pages 122-331. IEEE, 1995.
[8] Peter Auer, Nicolo Cesa- Bianchi, Yoav Freund, and Robert E Schapire. The nonstochastic multiarmed bandit problem. SIAM journal on computing, 32(1):48-777 2002.
[9] Sébastien Bubeck. Five miracles of mirror descent, 2019. sciencesconf
URL https:/ / h/dps:///dp2299 httpaaaaa/ppageeebbbbbb...//e pdf Lecture note of HDPA-2019.
[10] Sébastien Bubeck and Aleksandrs Slivkins. The best of both worlds: Stochastic and adversarial bandits. In Annual Conference Learning Theory, pages 42. PMLR, 2012.
[11] Sébastien Bubeck, Michael Cohen, and Yuanzhi Li. Sparsity, variance and curvature in multi- armed bandits. In International Conference on Algorithmic Learning Theory, pages 111-127.
PMLR, 2018.
[12] Botao Chen, Jongyeong Lee, and Junya Honda. Geometric resampling in nearly linear time for follow-the mallow-the-perturhed- perturhed-leader with sett-of-both-wotde guarantee in bandit problems. In International Conference on Machine Learning. 2025.
[13] Khai Xiang Chiong, Alfred Galichon, and Matt Shum. Duality in dynamic discrete-choice models. Quantitative Economics, 7(1):83-115, 2016.
[14] Daren BH Cline. Convolution tails, product tails and domains of attraction. Probability Theory and Related Fields, 72(4):529-555 1986.
[15] Yan Dai, Haipeng Luo, and Liyu Chen. ballow-the-eerturbed-leedd for adversarial markov decision processes with bandit feedback. Advances in Neural Information Processing Systems, 35:1 35:11437-11449, 2022.
[16] Chris Dann, Chen-Ye Yu Wei, and Julian Zimmert. A blackbox approach to best of both worlds in bandits and beyond. In Annual Conference on Learning Theory, pages 5503-5570. PMLR, 2023.
[17] Miroslav Dudík, Nika Haghtalab, Haipeng Luo, Robert Schapire, Vasilis Syrgkanis, and Jennifer Wortman Vaughan. Oracle-efficient online learning and auction design. Journal of the ACM, 67(5):1-57, 2020.
[18] William Feller, An introduction to probability theory and its applications, Volume 2, volume 81.
John Wiley & Sons, 1991
[19] Guiyun Feng, Xiaobo Li, and Zizhuo Wang. On the relation between several discrete choice models. Operations research, 65(6):1516-1555 2017.
[20] Ronald Aylmer Fisher and Leonard Henry Caleb Tippett. Limiting forms of the frequency distribution of the largest or smallest member of sample. In Mathematical proceedings of the Cambridge Philosophical Society, pages 180-190. Cambridge University Press, 1928.
[21] Mogens Fosgerau, Emerson Melo, Andre De Palma, and Matthew Shum. Discrete choice and rational inattention: A general equivalence result. International Economic Review, (1)) 1569-1589, 2020.
[22] Drew Fudenberg and David M Kreps. Learning mixed equilibria. behavior, 5(3):320-367, 1993.
Games and economic
[23] Boris Gnedenko. Sur la distribution limite du terme maximum d' unue serie aleatoire. Annals of Mathematics, pages 423-453, 1943.
[24] Laurens Haan and Ana Ferreira. Extreme value theory: An introduction, volume 3. Springer, 2006.
[25] James Hannan. Approximation to bayes risk repeated play. Contributions to the Theory of Games, 3(2), 1957.

---

[26] Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends@ in Optimization, 2(3-4):157-325, 2016.
[27] Josef Hofbauer and William H Sandholm. On the global convergence of stochastic fictitious play. Econometrica, 70(6):2263-2294, 2002.
[28] Junya Honda, Shinji Ito, and Taira Tsuchiya. both-worlds for bandit problems. In International Conference Algorithmic Learning Theory, volume 201, pages 726-754. PMLR, 2023.
wwwtttteeeerredde-eee achieves best-of-
[29] Shinji Ito. Hybrid regret bounds for combinatorial semi-bandits and adversarial linear bandits. In Advances in Neural Information Processing Systems, volume 34, pages 2654-2667 Curran Associates, Inc., 2021.
[30] Shinji Ito, Taira Tsuchiya, and Junya Honda. Adaptive learning rate for folloa-the-regularizee- leader: Competitive analysis and west-of-both-worldd In Annual Conference Learning Theory, pages 2522-2563. PMLR, 2024.
[31] Tiancheng Jin, Junyan Liu, and Haipeng Luo. Improved best-of- wothls guarantees for multi-armes bandits: FTRL with general regularizers and multiple optimal arms, Advances in Neural Information Processing Systems, 36:30918 1-009188-977. 2023.
[32] Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences, 71(3):291-30) 2005.
[33] Backjin Kim and Ambuj Tewari. On the optimality of perturbations in stochastic and adver- sarial multi-armed bandit problems. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[34] Tomáš Kocák, Gergely Neu, Michal Valko, and Rémi Munos. Efficient learning by implicit ex- ploration in bandit problems with side observations. Advances in Neural Information Processing Systems, 27, 2014.
[35] Jussi Kujala and Tapio Elomaa. On following the perturbed leader in the bandit setting. In International Conference on Algorithmic Learning Theory, pages 371-385. Springer, 2005.
[36] Tze Leung Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Ad- vances in Applied Mathematics, 6(1):4 1985.
[37] Tor Lattimore and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020.
[38] Jongyeong Lee, Junya Honda, Shinji Ito, and Min-hwan Oh. seltowtth--erturbe---eee with Fréchet-type tail distributions: Optimality in adversarial bandits and best-of- both-worlds In Conference Learning Theory, volume 247 of PMLR, pages
PMLR, 2024.
[39] Zifan Li and Ambuj Tewari. Beyond the hazard rate: more perturbation algorithms for adversar- ial multi-armed bandits. Journal Machine Learning Research, 18(183):1-24, 2018.
[40] Daniel McFadden. Conditional logit analysis of qualitative choice behavior. In Fontiers in Econometrics, pages 142. Academic press, 1974.
[41] Daniel McFadden. Econometric models for probabilistic choice among products. Journal of Business, pages S13-S29, 1980.
[42] Daniel McFadden. Econometric models of probabilistic choice. Structural Analysis of Discrete Data with Econometric Applications, 1981
[43] Gergely Neu and Gábor Bartók. Importance weighting without importance weights: An efficient algorithm for combinatorial semi-bandits. Journal of Machine Learning Research, 17(154): 1-21, 2016.
[44] Andriy Norets and Satoru Takahashi. the surjectivity of the mapping between utilities and choice probabilities. Quantitative Economics, 4(1):149-155, 2013.
12

---

[45] Francesco Orabona. A modern introduction to online learning, arXiv preprint arXiv:1912.13213. 2019.
[46] Jan Poland. FPL analysis for adaptive bandits. In International Symposium on Stochastic Algorithms, pages 58-69. Springer, 2005.
[47] Sidney Resnick. Extreme values, regular variation, and point processes, volume 4. Springer Science & Business Media, 2008.
[48]
Gilles Stoltz. Incomplete information and internal regret in prediction of individual sequences. PhD thesis, Université Paris Sud-Paris XI, 2005.
[49] Arun Suggala and Praneeth Netrapalli. Follow the perturbed leader: Optimism and fast parallel algorithms for smooth minimax games. Advances in Neural Information Processing Systems, 0112316-223200 2020.
[50] Louis Leon Thurstone. Three psychophysical laws. Psychological Review, 34(6):424, 1927.
[51] Kenneth E Train. Discrete choice methods with simulation. Cambridge university press, 2009.
[52]
Taira Tsuchiya, Shinji Ito, and Junya Honda. Exploration by optimization with hybrid regular- izers: Logarithmic regret with adversarial robustness in partial monitoring, In International Conference on Machine Learning, 2024.
[53] Viktor Witkovsky. Numerical inversion of a characteristic function: An alternative tool to form the probability distribution of output quantity in linear measurement models. Acta IMEKO, 5 (3):32 44, 2016.
[54] Viktor Witkovsky. CharFunTool, httppp//ggtaab..ooa//aaatttataaa
2017.
[55] Julian Zimmert and Yevgeny Seldin. Tsallis-NF:: An optimal algorithm for stochastic and adversarial bandits. The Journal of Machine Learning Research, 22(1):1310-1358 2021.
[56] Julian Zimmert, Haipeng Luo, and Chen-Yu Wei. Beating stochastic and adversarial semi- bandits optimally and simultaneously. In International Conference on Machine Learning, pages 7683-7692. PMLR, 2019.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We provide all the claims and the limitations in abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
13

---

Justification: In Remark 5.7, we explicitly acknowledge that our negative result is limited in scope, as we do not formally prove impossibility. However, based on our analysis, we expect that a counterexample exists in which FTPL. with symmetric distribution fails to achieve the desired guarantees.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For most of our theoretical results, we explicitly state the required assumptions. However, some technical conditions specific Fréchet-type distributions are deferred to Appendix A. to avoid unnecessary distractions in the main text. For completeness, all detailed proofs are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
14

---

Answer: [Yes]
Justification:
We provide the MATLAB code in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We provide the full codes in Appendix
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c ec/
public/guides, http:///uuiiss//eeuuuuiiisssss
Soodesubmisssionool111
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https /niiss cc/ waallic/ggiides//ddeeiuiei
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
15

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [No]
Justification:
We provide the full codes in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: We do not include experimental results that needs any kind of statistical significance testing.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [No]
16

---

Justification: Our code is very light and can be run on any recent computer or laptop without special hardware requirements.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This is theoretical work.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This is theoretical work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
17

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This is theoretical work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We explicitly cite CharFunTool in Appendix F
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
18

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMss if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy hbtrge///aaarrppeccc////ofeeeeep for what should or should not be described.
19

---

Appendix Contents
Additional details omitted in main paper
A.1 Previous assumptions for BOBW guarantee
21
A.2 Derivation of potential function
22
Equivalence results two-armed bandits: the existence correlated perturbations
B
Proof of T'hoorem 4
asymmetric perturbations
24
Proof of T'Toorerr 4.3)
Laplace-Pareet perturbation
26
Proof of Lemma
27
Proof of Lemma
29
Proof of Lemma
Proof of Lemma
................................
32
Proof of (20)
33
Proofs the stochastic regret for Laplace- Pareto perturbations
34
D.I Regret lower bounds
34
D.2 Regret for the optimal arm
36
D.3 Proof of Theorem
39
Proofs of Corollary 4.2
40
E.
Adversarial regret analysis
41
Stochastic regret analysis
Details on numerical validation
42
Proofs results under symmetric perturbations
42
Proof of Lemma 5.1
42
G.2
Proof of Proposition 5.3
43
Proof of Proposition 5.5
45
G.4
44
Proof of Proposition
47
G.5
Counterexample for general index in three- armed bandits
49
6.6
Specific results for symmetric Pareto distribution
51
20

---

A Additional details omitted in main paper
Here, provide the details omitted in the main paper due to space constraints for completeness.
Notation Throughout the appendix, we define the gap each vector element from its minimum using underlines, ie., À À min As Therefore, the arm-selection probability based on the gap of the cumulative loss can be interchanged with that based on the cumulative rewards since 2 max; V V4 holds. When D is absolutely continuous, o(:; D) can be written
Φ (x;
Pr PI
=
arg min {Ax - rr = /II jØz
II
F(z dF) Ao).
jE(K]
Also, we denote the geometric resampling estimator of Wt, by wi,i
Definition of Fréchet-type perturbation Here, we readily follow the notation used in Lee et al. 38] To define the Fréchet-type perturbation, first require the notion of regular variation defined as below.
Definition A.1 (Regular variation (24)). An eventually positive function g. which becomes positive after a certain point, is called regularly varying at infinity with index a, RV if
lim g(tx)
t. AA
0.
g(z)
If g(x) is regularly varying with index O, then y called slowly varying.
Then, a necessary and sufficient condition for distribution to be Fréchet-type expressed in terms of regular variation as below.
Proposition A.2. distribution is Fréchet-type with index a > 0 iff its right endpoint infinite and the tail function, 1 F, is regularly varying at infinity with index i.e. - F RV -a. In
this case,
x 2 0, (exp(-x-99
FT (an
n 20000
0,
x <<0, 0
where an inf (x: 2 1-
Moreover, if 'D is Fréchet-type, we can express the tail distribution with a slowly varying function SF RV,
F())= x Ys > 0.
Note that by definition, slowly varying function Sp(x) grows most polylogarithmicaly, For further details on Fréchet-type distributions, see Haan and Ferreira 24 and Resnick [47].
A.1 Previous assumptions for BOBW guarantee
In the previous analysis, Lee et al. 33 showed the optimality of FTPL with perturbations satisfying following assumptions.
Assumption
2.2
suppDa C [Amin: 00) for some Fini 2 and the hazard function I- [-F(z)
f(x)
bounded.
Assumption
2.3
[(x)
is monotonically decreasing in æ 2 Fmin-
F(w)
Assumption A.3. F(x) has density function f(x) that is decreasing 2 xo for some 3o > Zmin-
Assumption A.4. There exist positive constants M, (D and Mi- = Mc(D.) satisfying
E XEND,
max X,/at iE[k]
$ M,
1
Ex XeNDy max;e Xz/ak
< Mi
for 04 = inf (x: F(x) 2 1/k) and satisfies A S ax S A, k for some positive constants
A,, An.
21

---

With general support, correlated perturbations (discrete theory literature)
With general support, i.i.d. perturbations (online learning literature)
Bounded hazard function for right-tails (near-optimal]
Gaussian unbounded hazard
Gamma (Exponential)
Fréchet-type perturbation (BOBW literature)
near-optimal sto.
semi-infinite
regret [12]
near retarolpimmal adv. regret
One-sided perturbations
additional log in adv. regret [39]
Fréchet, Pareto (12, 28, 38)
Sec. 5:
Symmetric Fréchet-type
Unbounded, asymmetric (Sec. 4)
BOBW K
connection to discrete choice theory
Gumbel
near-optimal adv. regret
>> connection to FTRL with hybrid regularizer
Exp3
(bounded hazard)
MNL model
Laplace-Pareto
= FTRL with
our conjecture cannot be optimal
BOBW (Theorems 4.3 and 4.4)
Shannon entropy
(at most near-optimal)
hybrid-type (Gumbel-type Fréchet-type)
Figure 4: Perturbation landscape summarizing distributions studied in the literature.
Assumption A.5. lim
(z) a and =f(x) bounded almost everywhere on [Tmin 00). f(m)
T(a)
To simplify the analysis and avoid technical complications related the boundedness of the hazard function, this paper focuses tail-equivalent distribution satisfying the assumption by considering the truncated version E of F when we consider the general distributions. Specifically, we define the truncated distribution function
Fx (z) = Pr|X 1 F(x+ +1 F(1) 1 F
>> 0,
which is supported on I., This truncation technique was also used to provide OVVK'' K) regret Then, for any D. Da. its distribution function can be written as
(x; = Ō((x 1)79),
0,
For simplicity, we denote F by F since we only consider truncated one this paper.
Note that by Corollary 4.2 Assumption 2.3 can be relaxed to the existence of xp > #min such that
f/F is decreasing in æ 2 To unless the left tail is lighter than the right tail.
For a quick overview of the position of this paper, see Figure
A.2 Derivation of potential function
For any i.i.d. perturbations under assumptions in Lemma 3.I, we can express the potential function as
( = Σ [E[K]
zf(2
v4 II
F(z v2)d2,
(9)
J#1
By definition of potential function Tb and its convex conjugate V, it holds ((v)+V(p) Eie[K] Pits iff p E @D(v). Chiong al. 13 showed that
V(p) = P Pi arg max(u; + +r;)) jE[K]
Let us assume is of decreasing order, i.e., v1 2 V2 2 ... 2 ux holds without loss of generality, where 2, holds. By definition, when consider the perturbation distributions supported
22

---

on R, we obtain
$(V)
max(r; + vz) ri = = f())dr
20
(x (z)
F(x+ + vj)dæ
5#1
((x)
(+)())(()) II
F(u + v2)dyda
131,2
cf(x v1)
F(x v2)do
j#1
f(x) Σ
ufly v.) II
Fly - F(y-2))dup dydd
i#1
jF1,
zf(x v1) II
-1ydz
$#1
f(z vn)
ufly 14) II Fly - vy)dddr. 1
#
By changing the order of integral, obtain that
{ [xx v1) Σ
uf(y vs)
II
Flu dj)dyd
171,
yf v4) II Fly - u2) 1 f(x vs vi)dady
Σ
DC
1
j
Σ
uf(y v4) II
F(y - vjydy.
11
jfi
Therefore,
((v) = Σ
Σ zf(2 14) II
- vy)de.
(E[K]
Since p and v are bijective, as discussed in Section 3.2 we have
V(p)
Σ
(p)( ((v())), Vp e Int (PK-1).
(10)
KE[K]
Therefore, by 10 and letting = =(()) ((v), it holds that
-V(w) Σ
(n v;) + Σ
2f(2 + 21) F[x 2j)
dz
jE[K]
Σ
Wi
A +
Σ JE[K]
2f(2
F(z + (1=+A))ds II 145
Note that the above potential function and corresponding regularization function can obtained for any perturbations with density f and distribution F' supported on R.
23

---

Here, one can check that
@Φ(v)
=ff vz) II F(z kj)dz
dv:
00
+
Σ =1(2-v;))(C 13) -1v1 -vi)
II
F(z vz)dz
-
iti
[(z v:) II F(z 1j)dz (p(()):
(I1)
j7i
where 1T) holds by partial integral. To be precise, we have
2f 14) II F( 22)d2
-2f(2 1/2--100 - vz) II F(z vj)
jti
f(z vz) II F[ (1; v1) + Σ 2f(2 v.) II
F(2 v1)dz
jti
iti
f(z vi) II
F(z - vz)) Σ
ff(z - v/:)
II 191.
F(z v1)d2.
174
A.3 Equivalence results in two-armed bandits: the existence of correlated perturbations
In discrete choice theory, as the assumptions in Lemma 3.1 implies, the additive random utility model (ARUM) admits correlated perturbations, whereas FTPL in bandit problems typically assumes ii.d. perturbations. The equivalence between ARUM and FTRL in two-dimensional case was established as follows.
Lemma A.6 (Theorem 4 in Feng et al. [19). For any differentiable choice welfare function C(vi, there exists a distribution D of {ri,r2) such that
C(v,, v2) B(r (v1 11, + r2)]
While the notation of choice welfare function in Feng et al. 19 a general notion to define discrete choice model, it suffices to interpret type of potential function & in this section, such that the choice model (arm-selection probability) is given by (p()) = (
This lemma shows the equivalence between differentiable choice functions and ARUM. Furthermore, Theorem 2 in Feng et al. [19] shows the equivalence between differentiable choice function and FTRL with (essentially) strictly convex regularizer in general case, implying that for any FTRL with strictly convex regularizer V, there exists a differentiable choice function such that p(v; V) = VC(v). and vice versa. Since 3-Tsallis entropy regularizer is strictly convex function, this implies that there exists a corresponding ARUM. is important to note that this result does not necessarily imply the independence of r and r2, as suggested by Lemma A.6 Specifically, one can define a distribution D of (r1,72 T2) as
(f1,72) (C(0, 0) max (5, C), max (-5,0))
where & is a random variable with distribution function (x) = c(x). In this case, whenever 71 is observed, the value of r2 determined by definition D, which shows dependency. While this example illustrates how strictly convex regularizers can be associated with correlated perturbations, we expect that the 3-Tsallis entropy regularizer corresponds i.i.d. perturbations, as such constructed examples appear artificial and were introduced to address a broad class of (essentially) strictly convex regularizers.
B Proof of Theorem asymmetric perturbations
Here, we assume \ .... S Ax without loss of generality, where o; holds.

---

By definition, for any E [K].
(1())
f( + II jpz F(2 + 22)d2
(1(A)
1_ + F(z 2j)dz
2 -f + 11 jزز F(2 + 22)
(unimodality of f)
1 f(= A 1 F(z 2
Then, we first show the existence constant Ç satisfying
S2 - - P(z + 2j)dz 5
b#z F(= + 2y)
(12)
for any A. When L, it holds trivially. Let consider * 1. Define G(z) II F(z 2z). which is increasing with respect to 2. Then,
S - 2i F(z + 2j)dz So -o- -f + II 1 F(z 22)dz
S ff(x + (()
S0 +
S -f"(= + ()F())(())
S -f'(z + (())))(())
S 4. ["(x F()ddd
S -f((z F())
Since F(z) 2 1/2 and -f'(z + A) 2 0 hold on = 2 0, we have
2
-5'(2 + 2)dz
0
f(A.)
Hence,
1 - f' A)F()dd
2
+ +
Jo -f' + A))(2)d
[()) -2
By Assumption A.5
it holds that for 2 220
- (=)) =
(2 (2+1)a42
Also, by definition of F of Ua, we have
F(2;Ua.)) = F(-2; Dp)
Vz <.
2
(1-
Here,
max =E[-X,.0] (2 A; 1)o+2 (1-))3
(4, + +10+2 V
(4, 1)3*
Therefore, whenever / 2 22
2 f(2,)
=
f(2)
(z + A + (1=
-2
A
f(2,) 0
(1+ jat2
+ 1)a+1
= (())
(1 + X1 2.)002
25

---

Therefore, by applying 12 we obtain
(1(A)
000 -f + A) II jut F(z + 25)dz
1(++1)
(1(2)
1_ f(z + 2, II joz F(= + 2,)dz
Jo f( + II جهز F(2 + 2j)d2
$ (5 +
Jo f(z 2 11 F(= 2,)d2
Since Ua,a is equivalent Da Da 0 2 0. which satisfies all assumptions in Appendix A.1
(with Tmin = 0), Lemmas and 10 in Lee et al. [8] concludes the proof.
C Proof of Theorem Laplace-Pareed perturbation
The proof given in this section readily follows those given by Honda et al. 28 and Lee et al. (38). with the main distinction being terms related to the negative perturbations. For clarity, we omit notation related L.P we focus solely the Laplace-Parets distribution defined in (4)
We begin by decomposing the regret. Using Lemma from Lee et al. [38], the regret can be decomposed as follows.
(T)
( (1, - -w(+1)
1
E E - -1410
E PLP (maxee Fi.s
76-1
Thc
71
1=1
1=1
Li m(tt)|
1
1
B -1411. B [max ri
76-1
111
1/1
1=1
( (e, We wh++
1
Blreth 1
K
(13)
Tb+1
Th
2m/1
stability
penalty
where the second inequality follows that the block maxima of LP can be upper bounded by that of the Pareto distribution with shape P2.
Then, remains to bound two terms, stability term and penalty term. For the stability term, we can further decompose it into two terms.
Lemma C.1. It holds that
Σ E (4e, - 1=1
B EB -lml 1=1
4K 27
2ca
log
m
177+1
To prove this lemma, we address certain terms that ignored in previous approaches, which considered only positive perturbations. This is necessary because, due to symmetry, there exists semi-infinite interval where I' 0, resulting in loose upper bound.
Then, the first term can be bounded as follows.
Lemma C.2. For any [K), if Lugs is the og smallest among
(Er.s) }. then
(e.()r(( - 4:11
30
10e
E
Vo,
m
L,
In the proof of Lemma we apply the same techniques in the proof of Theorem ensures that the terms are of the desired order.
which
Finally, the penalty term can be bounded as follows.
Lemma C.3. It holds that
E
Tth Tt.'" Le
5.7KK A igit
Σ
26

---

By applying Lemmas C.I
C.3
into
13
with
1/2
m v2²
we obtain
(Tg
T π
1 5.7VK +
[4K
+ -2
log
7/2
+
VKT n
1t+1
1/1
27
11T+1
2711
T
T
1
1
30m
TT + 2(VK - +)
+5.7VK
1/111
1=1
4K
+
2
log
?
+
VKT
27
Tr++
2mh
60mV KT
5.7VK
2K
V
Vt+++ 1 vt)
+e?
log(T' + +1) + 2m
Vt
m
27
5.7(vT+1-1) VK+ + (
2K
V
120m
KTT
+ +
log(T + +1+
27
2m
120m
5.7
KT
2K
+ e² log(T + +1) +
vk=
m
27
2m
C.1 Proof of Lemma
C.1
For generic L E R,R,K define I 1 min, L. Then, by definition of é we have
0 Ōn
(1(())) = [ L + +
II F(z + n4,
+
$ f(z nL,) Σ +
II F(z F(=+ + nLn)) +(
dz. (14)
jti
(+..)
By definition of f. one can see that f'(x) > O for x < 0 and f'(x) < for x > Therefore, we have
( + [n1))
Il F(z + /L2
$
Lif'(z + nL,)
II F(2 + nL,)dz
= =Lf(2 + n1,)) II F(z F(2 nL,))
- { Lif(2 + nL1))
Σ S(z n1zg))
II F(z + nta) dz [
3+1
[ Li((2 nL4) Σ [(2
(n())))) ((nL))))
II (+1,)
F(z + nLn dz
8z1
L
0 1
L.f(z nLL) Σ f(z f(z+nL,)
II F(z + nLn) dz
(nL, + 1)3
(
<
4
L.f(z + nbin) Σ f(z + nnb))
II
F(z mbL)) dz.
(15)
271
27

---

By injecting the result of
15 into
14
we obtain for any i E [K]
(n S 27nj
/
f(z+
II F(z + nti) de 1715
dn
j7i
(f.)
f(2 + mL))
E
L))(( nLn)
II F(z nL) 1818
dz. (16)
j7i
(1,)
Similarly to the proof of Lemma 8 in Lee et al.
38
we obtain for 2 0 that
II F(++ = II (1- F(z +
I+i.
S exp
-E Iifi
F(z + nnni))
$ e² exp exp
Σ (1- F(= + |E[K]
= =x2exp exp
Σ [E[K]
2(2 + nT
where the last inequality follows from F(x) E [0, 1] for all x R, i.e., et-F(x) and the last equality follows from the definition of Fxx) for 2 0. Therefore, we have
(t)) se S 1( n())
Σ [z (((=+11)) +n,))) jti
eep Σ [Е
dz
2( nL 2
f0
I 2
f(z
Σ
(z
exp
Σ |E[A]
de
2(z + n )2
jE(K]
(1+n)
1
Σ jE[K]
+ nL 3
L
exp
Σ KE[K]
2( + T211) )
dz
? x
1
Σ
2
exp
Σ [E(K)
dz,
(17)
(2+ (I+nL
2(2 + nn
1
2(2 + n1
jE[K]
which implies
26
Σ (t.)
Σ
Σ jElK]
exp Σ
dz
11
Ju
22 + L 3
2(x + n1,,}
2(2 + nTL)²
K
kE[K]
[E[K]
2e
K 1i1 dw
Jo
we
(by w = ΣlE(K] 2(
1
EiE(K] F(z + nTL))
17
2e
11
7
On the other hand, we have
Σ (tt) (t)= L Σ f(z LL)
Σ [L L))(z + #n
II
F(z + (La)) (())
dz = 0. (18)
KE[K]
KE[K]
This is because the value of f(= + nL) + nL,) [Tz4 F(z + nLn) remains unchanged when i and are swapped, which makes the integrand zero.
28

---

Let L -,e Then, we obtain
E ol lo) olnta + +b)))]|
=
Σ
E
I wi (di(Te)) (nc++ L))
KE[K]
ille, I on
a
Σ
E
114
8 Σ
E
On
Kc|K]
Σ
E
On
KE[K]
M++1
By combining the results in 17 and
18 with
16
we obtain
E (e,, M(ncLe + ε.) - (l + +b,1)\
S
Σ
E
T
4
+ (tt) + (zi)dn)
27n
KE|K]
7
4K
2c²
dr
27n
M
4K
+ 2
log
27
Tc++1
Therefore,
T
Σ
LB
(mm(i (ln (ane(le+++))) +(11)\| 5
4K
+
2e
log
27
1/T
1=1
C.2 Proof of Lemma
By injecting f and F, it holds that
0₂(A)= [ A (
II
F(z + 2,)dz
II 3
F(z + 2y)dz,
A 1)3
jzi
and
(()) = ax,
(A) = 20
II F( j#i
-3
Il j#1
F(z + #()+)1)dee
(2 4 1)a
Therefore, obtain for any X E 10, oc)
-o(()) 3
100
1
II
F(z + 2,)d2 1 2z+tA) II
F(x 2j)dz
(z +
A + 1)a
jti
<3
II F(z+ + 2j)dZ.
(z A + 1)
j71
Moreover, have for any 2 0,
Hi( + xe4) 53
II F[ + Xz)dz
2 + A3 + x + 1)d
A A-
600
<3
F(z + A3 x)dz
(2 + A, + 1)
J#1
<3
1
Il
F(2 + 2j)d2.
(19)
(a + A + 1)4
J+1
29

---

Based on these results, we can derive the upper bounds on <(A) o(A + nele) for any A E [0, 00)*
((1) - ( + m1))) =
(-di(ncLe + xe;))dr
3
1
II F(z + 2,)dadr
(by (I9
(= + 2 1)a
2
jti
00
S 3
II F(z + 2j)dz
(2 2 + 1)a
F(z 2,)dz.
(++
1)1
jz
Here, let us decompose the integral above into two terms.
II
F(z
D2)dz
00
II F(z+ A,)d2+
II 1
F(z +
dj)de.
4
0
(2+ 2
11,
12.1
By explicitly considering the formulation of distribution in Proposition 4.1
we can obtain for all
i
[K] that
1,, T
< 4, AA E [0, Ox
(20)
whose detailed computation is given in Section
C.5 for completeness.
By 20 we obtain
((A) Φ() + m.li) S 15m44,
1
II
F(2 + 2j)d2.
(=+ + 1)4
For notational simplicity. we define
(A) = / (2+
F(z + Xj)d2. II J#1
1)4
= Since wiI, follows the geometric distributions with expectation w,J,, given i., and I,, it holds that
2
2
1
2
E
W
W
L
W
Wr
W J
2
(21)
30

---

Therefore, when I = i and {Ay} are sorted, we obtain
(ell -inniiiinnnn +em)) En
E
15 Beres Lo0
< E
1h "
II
P(= + mn ddz
(z + mbne + 1)a i71
Peeeeeneee Wh,t woZ
< <30mg.
We (Telee Le
s 30
(by 4,s E 10, 1)
W/et
U/' (1
30
L
1 f(z + (eeL Live Fz + 1/1 Le )da
(T
3/s (i Iue) in oe (n)Le)
s 30m.E
Le
30,,E
[ f(z +
-ii F(z + mele, dd
where 6.(A)= 10 f(z Ao) I1 F(z+ Az)dz. Then, the following lemma concludes the proof.
Lemma C.4. If Xs is the a4-th smallest among {2y} (ties are broken arbitrarily), then
(
VT
22
(A)
00
C.3 Proof of Lemma C3
Since EXOLP [X| 1. it holds that
E Blret, re. L1 5
[ :/ren k..]
E
E 1[4 = il(re.. 2 Opron Lz]
2(z + meLes + 1)2 )), dz (22)
00
1
II 1
(2+ + 1)2 i#3
i/1"
100
2 + MeL, + 1)2 dz
J0
dz Σ
(2 mh
)2
31

---

Let f(2) = Ei 2(stmbe, +13a (0. 2(=+1)²
Then, we can also bound (2) by
Il
dz
(z + meler + 1)²
2( m:Les 132
jti
e 0
Σ
d2
(z + mehes + 1)2
0
$ 2e
((z)e-((o)dz
0
(2(K-1)
00
K
K
$ 2e
- 2e
exp
de
22(K-1) 2( 1)2
2(2 + 1)2
=222K
VK
w0 1/2 "dw
V2
0
5.7VK.
C.4 Proof of Lemma
C.4
We first show that (a(A) (x
is monotonically increasing in A for any j + i and A E [0, o0)
By definition, it holds that
(A) So (I+ iip F(z + X1)
(A)
So [(= + X;) jætt F(2 + -2y)dz
Jo (#+ActI)=
Iiizi
[1 2(=+Ay+Ij
dz
1o (*+A++1))
Iizzi jzi
dz
2(z+A, +1y²
1 (2+A,)²
|
,( 3
dz
11i4s
Since 1 ا is monotonically decreasing for 2 applying Lemma 9 in Lee et al. (38 implies 227
that (A) is monotonically decreasing in Aÿ-
(A)
By the monotonicity of bh (A)) (A), we have
(
I vi((A*)
{ A2 i i x0, j > i.
where 1==
=
( (A)
2x(A+)²
By definition, we have
i-1
1
(x(2*)=
(2 >2 1)4 1
dz
2(x + ) + 1)²
V2w² (1 w-w)i--dd dw
(by
+1y
V2B
2((, + 1)2 2
32

---

where B(x;a,b) = -oo (1-1(1- t)a-l dt denotes the incomplete Beta function. Similarly, we obtain
i-1
(i(Aa)= = /
1
dz
+++2+11 1)3
1122
(1-)
V2B
2(4: + 1)2:
Therefore, we can directly apply the result in Lee et al. 2(,+1) instead of (2,+1)2, which gives
38
Appendix C.2.2.] with = 2 and
(A*) (x²)²)
VT
2
A
0
3
C.5 Proof of (20)
When 1, it holds trivially by T1 0. Assume i 1. Let Gi(z) A 0 and (z) increasing, we have
I1 F(z Aÿ)- Since
0
222
1
G.(z)dz
(2 X
+ 1)4
22
(o)d2
(2 + 2₂ + 1)
1
1
1,,
Gx())d=
(e \ 1)
2(x 1)²
1
1
>
C.(O)dz.
(a+ d + 1)1
2(x 1)2
Therefore,
S D, +11 2 ddz dz
20
11 <
12,
Jo [x+)+11+ 1イ= de
ش²t dz
(at)
2
So
(++x,+1frdz
3(4 1)3
2d.
(2+ 1)a
d Here, ds (++2,+1)4 2= 26 (2+2, holds, whose maximum in the interval of integral occurs at either ~2; or z 0.. Therefore, we obtain that
(1)2
1
3(4+ 1)3
2 ( A 1)4 22 S 3(2, + 1)22
c
-22
(2, + 114
where V denotes the max operator. Since A, > 0, it easy to see
3 1)3
31,
33.
( + 1) A ++1
On the other hand, by simple calculus, we obtain for 0
9 (12 (12+ 7-3) <
3(x + 1) 2r
where equality holds when 1+v3 This completes the proof.
33

---

D Proofs of the stochastic regret for Laplace-Pareto perturbations
In this section, provide the proof of Theorem 4.4
based on the self-bounding techniques, where
we adopt idea in Honda et al. [28] and Lee et al. 38 Firstly, define event D, by
D2 =
Σ Egeis
- $5 (1 + )eLe.)
(23)
where on Dr
L = and m1e 21, #1.
(24)
D.1 Regret lower bounds
Here, we provide the regret lower bounds, which are used for the self-bound technique.
Lemma D.1. Let A == min D,. Then, holds that
(i) On Dr. Eipr ArWa, 86574 Eigi" (mile,))
A.
and W1 2 e1/
(ii) On Di. Eigr Arwrs 2 A - e-14a).
Although the proof is largely the same as in Lemma 22 of Lee et al. for completeness.
38 we include the details here
Proof. Let L' min Lyf. Then, by definition of w, we have
Σ A,wii
Σ A II
F(z + mLe.s) dz
6$1
8#1
100
Aif(z + +
II JE(K)
F(z + L
00
1- F-F + melrow
2
Σ Aif(z + tn
cxp
dz
(25)
F(z + mcLe.))
-0
jElK]
- F(z + 11e Leys))
F(x) dz,
Σ Aif(( + +
exp
exp
F(z + vcLe.
F(2)
(26)
where 25 holds since e 1n2 -1 < 1 - holds for < 1.
34

---

(i) On De, we obtain
1 F(z melen
(-sp-11-1)) F(z)
11((())
1 1- F(z)
Σ
fz + mBr.)
exp
Σ
F(z m velee
exp
dz
1711
iF
2 26
Σ
Ais(z + melira)
exp
-2 Σ (1 - F(z + melusl)) dz
0
Σ
Ai + relee)
exp
Σ
dz
(1
Egli*
5/4 |
2 554
Σ + +obenn) dz
(by definition of D, in
-6/5 Σ
A,(1 (niLes))
ispi""
= 5/4 5/4 Σ
A
2
Σ ipp
D
(nLis 2 for j * on D,)
2(1 + m 1,,e,,)
865/1
(neLe,))²*
i
On the other hand, on Dr, have
1
II jti"
F(z + mL
(|2) + 1)3
II jf1
F(z + ne 1eLendde
(2+ + 1)3
F(z + mnTers dz
0 (2+ 1)3 exp
F(z + m Intees)
8
-2 Σ (1 F(z + m dz
0 21 12+1 1)3 cxp
j
(2+ 1)3 exp
Σ
dz
(1 + T/e 12
jz
100
2 1/4
(=+ 1)3 dz
(by definition of D, in 23
11/1
2 0.3894,
2
which concludes the proof of the case (i).
35

---

(ii) From 26 on Di we have
- F(z + mibea)
F(2)
Σ 171
Aif(z + mcb..))
exp
exp
de
F(z + Tole,e)
F(z)
1 1 F(z + meleed)
(-
1 F())
A
f(z + (lst,olen))) Σ
exp
exp
dz
F(z + mobe,)
F(z)
Ae-1
%
Σ f(z veLe,)
-2 Σ (1 F(z + nebe,)) dz
exp
s71"
A
exp
-2 Σ
F(IeLea))
2
j
A
1- exp
Σ
1
2
(1 + neleo )2
2 A e- (1 e 1/11
Ejfi (Itmle,)) 2 on Di)
2
which concludes the proof.
D.2 Regret for the optimal arm
To apply the self-bounding technique, need express the regret of the optimal arm using the statistics of the other arms. Although the proof is largely the same as Lemma 11 of Honda et al. (8). include the details here for completeness, accounting for some additional terms due to negative perturbations.
Similarly to the proof in Lee et al. 38 we begin by introducing the following lemma.
Lemma D.2 (Partial result of Lemma 11 in Honda al. 28 For any i and ( E (0, 1), it holds that
[I[eus
5 17c
Ea, Le $ -
11) 5 +e
1
77
and when 4i
vi and (= (4e/21) 1/3 holds that
- - - ( +e < 2743m² 77m.
71
Based this result, obtain the following lemma.
Lemma D.3. On for any Ç E (0, 1), it holds that
E Les (px fiee) or (m 2.)) Ln < (1 -1-C
1362/1
Σ
1 1
+
(1 1---"
( 1)
6
+e)-
4t1 Les 1 - --
11
Proof. Recall that L,,- and Le., > for all A 1* on D,. Moreover, from 24 L, 2 1/1
holds for all j + it on D.. Then, consider two cases separately: (i) way m
S and (ii) w S.
36

---

ii When WiTh S (, we have Egs5 - S by definition. For any x - C and i + i", we have
n
di dnllll + =xcx)) = / f(z + m(L,))
II iti
F 2 mcLe, + x1 = i*] i) dz
000
f(z + mLr,))
II
F 2 + mbrs + x1هi = i*) +1)de
jfi
f(z + neL4, + nax)
II r( + m dz.
s
Here, note that whenever x S S. Lus- xx 2 (1 - c)L,, holds for all i + i on De. Then, by
differentiating with respect to x, we obtain
d dæ
Di (melee + xes-)) = 7)
{= + mr f(z + m2))
II F F(= + miLee) )de de j43,8"
20
((= + ne Mn, - M(=) 8F1,3"
II f(z+m )d)dlz dz
f(2 (12 milee - -x) s()dde + ni
[o { 5(2 ( (2 - x) (()dd..
S
(27)
For to term, since ne (Le, - x) 2 0 in case (i), we obtain for any i # i that
$: Sm
(2 mohei - x) + 1)3 (= + 1)s dz
1
Lo (++ 1)3 dz
s 1)
(1 + (1 - (1-
11
6(1 - mmL 6(1 L)
On the other hand, t; term can be decomposed into two terms by
t. Tt (l.,.. =))
f(z f(+ (h f(x))) + (-m(ta.- / 5(z
milees - -a)) (z)ds.
f,2
Since f(x) = 22x on x < 0, it holds that
t.,1 = 1t
exp 42 + n mn(Le - x) 2))dz dz
00
33 (1)) n
4
nie 3y (1-
4
< 171
1
(by e < I
for forz> -1)
4 3(1 - SSmieee +
12(1- - s)Lr,i
For the second term, we have
2, 1t 1
c²2
dz,
-mi(le,--)) (z + ne nelLe, - x) + 1)3
37

---

where the maximum of (0tc-11 Fj E [-c,0] occurs at either -c 0 since its minimum achieved at -c 2' This implies that
1 (a + me,,, - x) + 1)3 $ (nn(,,, x) + 1)3
2²2
V Vexp( --mc 2))
(28)
Here, since (y+1)3
2 for all y 2 1.15, obtain
n m(b,, 2)
1 T[m (Le, x)) 2 1.15) f22 S m (nc - 2)) + 1)3
(n (nely, - -5) 1)2
(29)
2(1 - -C)L,
On the other hand, when n (L x) E (0, 1.15), we need more careful approach to provide a tight bound. Similarly to (0 we first evaluate h.2 on Dj n {melee x) s 1.15], which satisfies
S mm n) lsttel,, +1+1 dz
1/2
So (atmi +1] (zt1prdz
00
(stmil, =2)+13 (2+1)5 de
When (L,, x) S 1.15, the denominator can be evaluated
Joa (z molLes x) + 1)3 (2 + 1)3
1
1
dz /
(a + 1)3 de
(=+2.15))
0.028.
Therefore, on Dr (nllee 2)) S 1.153. we obtain
0.5
3
1, v
< 18 <
0.028
(1 - L),,
Combining this result with 29 on D,, it holds that
3 (1 (1- (1-
11.
Therefore, for any i o i", we obtain
or (n (ni + xex.))
3 1/4
<
dr
(1 L)L.,
38

---

Combining this with Σ, (:(A) = 1, we have
E [1le, - C/nele,, (n. - or (ne [he + (1)) Lng
=I(114+ = i, ,,,,i-- S //niler (o- (0-
114e
or (ne + k.)) Ln
=<1ll1 i", ll, S S/ndlee (1 2r (n + &()) [c]]
1[4 = i, h. s 5/mbles E
E
(Dr(OnLi) - -allmll))-clmmll--- -a))) 1
3+ 1/4 (1
14 = M
L
L)
26
3+ 1/4
Le
(by (1
Whi" i71
L)
< 4e 4/3 + 1/4)
Σ
(by (i) of Lemma D.I
(1 (1- Lن,
For the case where C/m,, we can directly apply Lemma D.2
which concludes the proof.
D.3 Proof of Theorem
We begin by revisiting the regret decomposition in
13
and Lemma
C.I
where we obtain
T
1
RTg(T) v
E
C,, -ln(
Σ
Elret1,443 r241.. + Ci
7
1
E
EE
-
- - ++il))
1
(ret1.eer - ret1.)))
+ Ci
M1+1
Σ
EE
H(nne)) - M(n (())
re41.111 r1+1,5
+
(30)
2mvt
for C1 = K
+
2K 27
+ e²) log(T + 1) and m/. m/vi.
If Lt satisfies D. defined in
23 then the inner expectation is bounded by
(4, p6n L.) ((n + +Q,)) + 1e+1. - 1,t+1. L
2mvi
10ev2
+ 1/4 1
(1- 5) Lesi m²Les) )
+
1
11-0-1(1 (1 -1-
1
1)
( C
+e
L
(by Lemmas and
D33
60 + m m2
1 T- -) +
1
(31)
L,,
17:
where we set ( = 1 (4e/21)2/3 following Honda et al.
288
On the other hand, Dú, the inner expectation can be bounded by
E
(L., Mntte) p(n (L. +(0)) + Pttl. Te41.s
Lg
2mvl
KT 5.7
K
< <60mg
V
(by Lemmas C.2 and
C.3
2m
t
K
s (107m + +3/m)
(32)
39

---

By combining
31
32
and Lemma D.2 with 30 and
=1- - (4e/21) we obtain
7
60 + m -
K
Reg(T) (T)
Σ
E
1 [
(107m + +3/m)
+ + 2743m² + 77m. (33)
L.::
[=1
On the other hand, by the lower bound given in Lemma D.1
we have
A
(1 E 1/4)
Reg(T) (T) N
E
1[D
(34)
8c5/4
m2 Lr
De
i+10"
From
33
34
/2, we obtain
Reg(T)
2
T
60 + m 2 tm ²A;
+ 1[Di] (107m 33mm)
- 1-/4 1/4
1
1 [(rr] Σ
L,
16e L
2e
Afi
+ C1 + 2743m² 2 + 77m
T
(60m + m 122
1/4
[D
1[D;] (107m + (107++3/mm) 3/m)
1A, (465//)
2e
1=1
+ + + 2743m² + 77m
(by az baz S a²/4b for b > 0)
(60m + m 113
T
max (107m + 13/mm////1- 0.015, + + 2743m 77m
0.071.;
(60m + m -1)(11 + log T)
(107m + +3/m)²K
+ C1 + 2743m² + 77m
0.074;
0.04A
log A
θ
K log T
which concludes the proof.
Proofs of Corollary 4.2
Here, we provide the proof of Corollary 4.2 based on the discussion given in Appendices C and D
The main idea recast our arguments the form analyzed in previous works 28 38).
Firstly, as in (3). we can decompose the regret as follows:
Reg(? (T)
T
E
le W1 - 12+4))
1
1
El Elr - r241.+ E, Uhn [maxie(k] T1.a]
7
1h.
T'
lee wi - -we+t>|
1
Elret1,444 - 7441,++ EnnDD [max{e] ria)
<
E
7
171
1=1
T
1 -Erret, -141++++ M,A.VKK
Me++
Th
1=1
where the last inequality directly follows from Lemma Lee et al. since D, is assumed to satisfy all the conditions.
40

---

E.1 Adversarial regret analysis
As shown in Appendix [ it suffices generalize Lemmas C.1 Here, the generalization of Lemma C.2 can be obtained by Theorem 4.1
The generalization of Lemma C.1 is straightforward by discussion after (16 where the main difference comes from the additional (i) term induced by the negative parts. Specifically, the first two terms in (T6) can be directly bounded by using Lemma 8 in Lee et al. [88. and shown in [8 the sum of 2 terms over all i zero for any distribution.
Similarly, the generalization of Lemma C.3 follows directly from Lemma 13 in Lee et al. 33 The introduction of negative parts only decreases the expected value of perturbations, which turn reduces the penalty term unless the expected value is positive, shown in Appendix C.3
E.2 Stochastic regret analysis
To obtain BOBW guarantee, we further need to generalize Lemmas D.I and D.3 For Lemma D.I we can directly apply Lemma 22 in Lee et al. [38], since the lower bound can be obtained by restricting the integral interval to the positive side.
To generalize Lemma
D.3 one can see that it suffices to bound the term
i, - [ 5(2 to - f(z)de, f())ddd
when æ S </nl since the induced part by t; term in 27 can be analyzed by Lemma 25 in Lee et al. Then, we can decompose t; into two terms by
38
t& Th =x)
f(z)dz+ + L S (mm(h,,-o) f(z
f(2 (( xx) (2)))((()d))))))
m milee - xi) f(z)dz.
tia1
142
Since f(x) is the density of D3 on < O, we have
me mall,, -m)
5 mo 2)) f(z)dz s nif(0) / (2(l)
f(2)d=
m -00
=mf(0)F(-T(( x))
F( 1 -() (),
1
mif())
(m
1
f(0je
(1
(1-c))La.t
as desired. For the second term, can do the similar derivations in 28 by considering the maximum of 12+ (
41

---

1FT of
0.6
0.8
1FT of /0
(V
x10-16
0.5
0.5
-VV0,
0.4
0.4
0.3
0.2
-1
0.1
-10
-20
-10
0
10
20
Figure 5: Imaginary part after IFT.
Figure 6: Sanity check with normal distribution.
Details on numerical validation
In this section, provide the Matlab code for the inverse Fourier transform (IFT) and additional plots to confirm whether the obtained IFT is rallev. The code implemented based on the code of cf2 in Toar [53 54).
X Generate IFT given quantile function c(x). x_min -20.0; x_max 20.0; w-azis number of samples
3 N= 2-11; k = (N-(N))))
(0.5-N/2+k) (2*pi / (x_max-x_min)) X frequency awis 5 quantile_func = ((p) -(p (-p(2-1))) (-1p)) % given c(x)
7 fun @(w, p) exp(1i * function g
X compute characteristic
cffun = @(w) (rrayfun@@(v_aaa quantile_funceeee
sqrt (integral (@(p) (un p)
0.0001, 0.9999, 'ArrayValued' true)), x);
9
cf cffun cffun (s(N/(+::
10
(ef -in):::::));
for the negative side
11
X Do inverse Fourier Transform
12
(x_max-x_min)/1
13
= (-1).-((1-1/)) (ttp:/N((xxx_iiaa)__
14
(-1).-(-24xx_min)
www.iiiinnmmmxxmmmme_.aaa *k);
15
ifft C.*fft
16
pdf real (ifft); img-part imag (ifft); cdf cumsum (pdf dx)
Note that in line compute the square root of the integral, i.e., the square root of the characteristic function g. As illustrated in Figure 5 the imaginary part resulting from the IFT is negligible, with the y-axis scale on the order of 10 which can be attributed to numerical errors.
To validate the correctness of our code, we perform sanity check by modifying the quantile function the Matlab code to that the standard normal distribution, given by c(x) = - (2x - 1), where erf-1 denotes the inverse error function. Since the characteristic function of (V, o) is exp(-o2t) (((()/22))) /2). the characteristic function of ( 1) g()) = exp(-t?/2). Therefore, applying the IFT to v should yield the density of (0, 1/v2). As shown in Figure 6 the result confirms the correctness of our implementation.
G Proofs of results under symmetric perturbations
Here, we provide proof's for results in Section
G.1 Proof of Lemma 5.1
Lemma 5.1 (restated) Let -D denote the distribution of -XX. Let (ri, 12) i.i.d. from continuous D and D be distribution of T2. Then, if D is Fréchet-type, then either D or -D is Fréchet-rype.
42

---

Proof. According to the Seaber-Tpppet-Gneee normalized maximum converges, such limit distributions must belong one of three types of extreme value distributions: Fréchet, Gumbel, and Weibull, Distributions with polynomial tails (i.e., heavy tails) are classified as Fréchet-type distributions, while distributions with exponential tails are Gumbel-type, and bounded tails (i.e., bounded maximum) belong to either the Gumbel-type or the Weibull-type [47].
(FTG) theorem [20 23], if the distribution of
Whenever the tails of distribution of X are either exponential or bounded, the distribution of X - also has either an exponential right tail or a bounded right tail. is straightforward to verify this when the distributions have bounded tails. In the case where both distributions have exponential tails, classical results on convolution tails indicate that their convolution also exhibits an exponential tail, where the general proof can be found in Cline [14] and references therein. This implies that if tails of D3 are either Gumbel-type or Weibul-type, D does not have a heavy-tail, meaning that not Fréchet-type.
G.2 Proof of Proposition 5.3
Here, we assume A1 I Xg. so that X 0 and A2 for some constants 0 without loss of generality. For i = 1, by the unimodality of symmetric Pareto distribution, we have
(A) (0) (A))3/22
-_x0 ((
(1 f())F(=+ +
3/2
S0 -f(=) + c)dz
(unimodality of f)
(1 f(z)F(2 ())
3/2
22
-f (001F(++ccc) +c
(A) 1/2)
3
(1-
1
2
de
0
(s+ 1)4
2(2 c 1)2
< 2v2.
(35)
For i = 2, we have
(2(A)
(a2(3/3
((+ c())) 3/2
For the denominator term, we have
f
f( (c)+d)(;)dz>> (r f(2 ((2+c)F(( c))
1
1
dz
2 Jo
(= 1)3
(36)
1)
For the numerator term, we consider two cases separately, where (i) c E (0, and (ii) 2 1.5. When (0,1.5), we have
f(z f(z+c) > (2.5)
and
((()d)
-1'(s) c)dz
-f(2)](2 c)d2
-f'(2)f(z + c)dz < 1.
(by 35
43

---

Therefore, we have
(A)
S 125.
(A2(5/5
When c 2 1.5, we decompose the numerator into three terms.
-3
((())) 2(1 c) 2) d2
h
3
3
d2+
1
d2
2( + +c+ 1)a(1 2)²
(2 c++ 1)a
2(z + 1)2
c
f2
For the last term, we have
3 dz (c+ 1)3
1
ts 5
(37)
(c+ = + 11
For the first term, by partial fraction decomposition, we obtain
1/c²
-2/c²
3/c²
-4/6
4/c
1/c²
(1- - c) - 2)2
(1-2-c))
(1-2-c) c]3
(1-2-10
c)2
1 2
1 - (1 - 2)
Therefore, we have
111 11
2
1/c²
-2/c3
3/C²
-4/65
4/c
1/c²
+
+ (1
1-2-c +
I- (1 2) dz
2
c)1
(1 ccs
c)²
1/c²
-1/c3
3/C²
11-3-0
1/e²
+
+ (1 2 c)
+ c
log
+ 1-
3(1
c)3
(1 2 c)3
1- 2
1
4
log(c + 1) + (c+1))
362
c3
c
which implies
1
3
3
t1 =
+
+
log(c + 1)
2c
2
2c
2c (c+
1
1
(by log(c + 1) c)
20
2
Similarly, for the second term, partial fraction decomposition gives
1/(c+ 2)2
2/(c+ ((c+2)) 2)3
3/ 2)
4/(c + 2)5 4/(c+
1/lc+ 2)
(++ ()=(1-2)- =
(2+ 1)a²
(=+ 1)3 (2+c+1)² 2
1-z (1 (1-2)²
Therefore, we have
-1/(c 2)² 3 1/2 = 3(2+c+1)) 1)3
2.
11/cc + 2)3
-3/(c+2)*
log (+++1))
1/(c+ 2)2 ==
+ 2+c+1 +
+
(=+c+ 1)2
2)3
1-2
2 1==-
(c 2)4 2) + 8log(c+1)
3(c +
(c+2)3 2)3 +
(c+ 2)5
4
33 -3(ct () 2)2
+
+ (c+ 1)(c + 2)4
(c+ 1)² 2)3
which implies for c 0
1
3
6
12log(c+1)
t2
2(c+2) 2)2
2(c 2/3
(c+
(c+2)3
3
4
+
+ 2)3 +
2
3(c + 1) (c (c 2)
(c+ (c+
(c+ () 2
1
3
18
3
4
+
+
+
+
2g
2(c + 2/3
(c 2
2
3(c () 2)²
(ct 1) + 2)3
(c+ 1)(c 2)
44

---

Therefore,
3
3
18
f + 12 $
+
+
+
2c
2(c+2)
2)3
2c
(c+
(c+ 1) + 2)4 2)z) (38)
(ct () + 2)3 +
2
3(c + () 2)
By combining (36 and 37
with
(38
we have for any > 0 that
-22(1)
(ct11
-+ )
1c+1)²
(02(1))3/2
S 12
c
c+2)
ca
144(c+ + 1)3
12(c+1) 1)
48(c + 1)2
2)4 + 8.
(c+
(c+2)3
(c+
Then, for c 2 1.5, we have
- (A)( S 121,
(A))5/2
which concludes the proof.
G.3 Proof of Proposition 5.5
Consider symmetric perturbation distributions 22 whose tails of both sides are Fréchet-type with index 2. Let 22, and D2,r denote the distribution of left and right sides, respectively. Here, we assume that for all x > 0
x. fx; D2,) 1 F(x; D2.)
2 and rf(x; (2.))
This condition is known as sufficient condition to satisfy Assumption 2.3 Also, it is not that restrictive conditions as several well-known Fréchet-type distributions, such as Fréchet, Pareto, and Student-t distributions, satisfy this condition [38] see Appendix A]. Under this assumption, we have
for x > 0
(F. F(-2;(122) = SF, and 1 1 F(; = 2(x 1)
2(x + 1)²
where the corresponding slowly varying function Sp. (x) is an increasing function for 0. Here, slowly varying function g(æ) satisfies lim g(m) = 0 for any >> 0, so that is asymptotically negligible compared to any polynomial function. .
By definition Φ, we have for #
Φi(A) = _ f(z+ x)F(()))
II F(z + dj)dz.
1
Then, we obtain for K 2 3 and A = (0.c...,.) c) with e> 0 that
((A
f' ((() (2 ccdz
F)()) - -c) (z)dz
(K 2) ()) L f(z f(==c))(t)FM-22()8)
(K -2)F(2 - k-)))
K-2 000
f(=)
F(z
(K - 1)f(2)(Fk-2))) (2)d2
K -1
F(2)
K K-2
2VK-1
F(z - c) f(2)
F(z) (K (
K-1
.K-1
45

---

Then, since Sp is increasing, we have for c 2 2
F( F(VK c) Sp.(c + vK)
Sp (c(c+1 VR)
Sr. (vK +1)
2(c K)2
2ca
2c²
uniformly over E IVK- - 1,2V - -I..
By Assumption 2.3 F/F is decreasing for > 0. Therefore, we have for E [V that
1,2VK -1]
2 f(2VK -1) (2V -1) F(z) F(2NK 1) 2
f(x)
4KvK
since F(2VK- - 1) 2 1/2 and f(x) S₁, (z)( 1)3. Therefore, we obtain
K 2
2
f(z)
F(: 6)
(K (K-1)f( 1))(3))
K
VK-1
F(z)
(K F) (VK+1) 11)
2VK-1 f(z)
(K (K))))(2)pkk>(()
2(K - 1]c²
VR-1
F(z)
2 (VK
(2VK-2KK) 5,VK 1)
2VR-1
(K- (K)
8K JKVK(K 1)c²
VR-1
(K 2)5 DASn(VER (VK + 5, 1)
pk pk 1)
SKVK(K - 1)c²
(vk +1S,, (2VK -1) 1) pk 1 16KV
(: K 2
Here, for y = 1, we obtain
FK (k) = (1 (1 F(y))) exp((( - log(1 (1 F(y))))
- -(K 1) F'())
2 exp
(39)
F(y)
exp(
2(K 1)(1 F())))
(: (::F(z) 2 1/2 for 2 0)
Sx(y)
exp
2(K 1) (y + 1)²
exp
(SP 11)22
(::y=
where 33 follows from log(1 (1- x) >
-2 I-r
for E (0.1), 1]. This implies
((22) SE(VK + (2VK exp
(5. exp(-S (2,K - 1)/2)
C(D, K)
(40)
KK2
16K 2
Here, C(D, K) is constant that only depends the distribution and K. Even though C(D, K) depends on K, it is important to note that its dependency is at most logarithmic order since both Sp and Sy slowly varying functions. For example, C(D, K) 2 eex((-1/2) holds when D is symmetric Pareto with shape 2.
Next, we have
(a(A) =
f(z ((2+c) + c)dz
-c0
f(z F()) c)d2+
+(=+)) +c)dz
41
42
f(2 Fl2 cdds.
46

---

Then, we have
$1 $1 S f(z + c) 22 c)dz F(-c) 1
K - 2K-1
For the second term, obtain
12 L/ -)) (2)d2
= 1oz [(=)F(=- (=)==) {" [(=)F(z -c)F c)F
2/2
0012 < F((c/22) /o ((=))k->(>)d= + f [(z)dz $2F(-c/2),
0/2
where the last inequality follows from symmetry so that 1 - F(2) = =F(-2) for 2 > 0. For the last term, we have
ts f S Lo f(z ((++c) f(=+c)F(2)F=* K 7( +cdds
F(c)=
Therefore, we have for K 2 3 that
(A(A) F(-c) + +FF(cc/2)+
25
F( F(-c/2)
8
25 SF (c/2)
25 (p/(cc2)
(41)
8 (c/2 + 1)2
22+
By combining (40 and (11.
we have for c 2v K that
(A)
2(c 2) C(D,K) = á
(A(
(c/2(
16K KK2
KVK
where Ω hides any dependency. Note that Assumption 2.3 implies lim sup2-100 Sr(x) 00. Moreover,
(1 2
c+2
3/2 91 (2)
KvK
which concludes the proof.
G.4 Proof Proposition 5.6
Since (p()) is a bijective function, there exists a unique c(x) satisfying
(((c()),000) = x, 2
for any x E [ 1). When < J. there exists unique c(x) satisfying
1 -
p((0, c(z), x;
2
Note that c(1/3) = 0 holds since the perturbations independently identically distributed. Let us define another function V : 1) + Rço satisfying Vzx) V(z, (1 - x)/2, (1 2)/2). From 3 it holds that
v(x)= = fac(2) - ((c(()) 0, 0)), ((1 x)c(x) - ((00, c(x), c()))
if 2 1/3
if < 1/3
Let us consider x: 2 1/3 case, where arg max, = 1 as Proposition the potential function in ( we have
5.3
By explicitly considering
4 ((c(z),0,0) = zf(2 c(x)) +2 +2ʃ
c(x))dz.
47

---

Therefore, by relationship between regularization function and potential function given in 100 for 1/3, we have
('(x) = c(x) + xxd - <()) =-(==-((())) (z)) + l c()) (z) d (()))(((((()))000))
(by
= c(x)-
Thus, the derivative of V (xx) depends c(x), which is related to the inverse function of yo.
Fix c 2 0. Then, it holds that
PI (1c(c, 0,0)) 1 (p(((c,000)
From 42
it holds that
(2((c,0,0) 0)) o,((0,,cc))
3+ 1/16
4
(c+ 1)²
(c 1)2
On the other hand,
c,((0,c,)) = L- [(++c)
> ((=+c)F(()F(2+t))) / ((=+))(()F((2+0))
2 F(-c) 1 f(= f(=+c)F((= + c)dc + F(()) /-d f(z+ c) +
(9))F((c)) F(-c) + (1- F2()))
2
MO)((-)) F(-c) + 2 (1 F(c))
2 2
(F(c)
F(()F(())) F(-c)
+ F(-c)
2
2
F(-c)
(F(c) E (1/2,1)
4
Therefore, obtain
1
4
S {po((c,,,0)) <
8(c+ 1)2
(c+
which implies that for x E 11)3,1)
8
1x 5
(c(x) + 1)2
4(c(x) + 1)2
Hence, we obtain
d(x) 0
-
Note that the derivative of ] Tsallli entropy at ptp = (xi, 1 1
1 + V2
(1)2(1)
V
which implies that the derivative of Vpp) roughly coincides with that of J-Tsallis entropy when + 1.
48

---

G.5 Counterexample for general index in three-armed bandits
Proposition G.1. Let the unimodal symmetric Fréchet-rype perturbation Da with index defined on R, which is equivalent to a distribution in Da on R4. and
(0 20,
(Unimodality)
-1- = - ( (1=+ V2 2 0 (Symmetric Fréchet-type)
L ((st
C(D,) S Da), Vc 0.
(Condition)
for a constant (D) that depends onty the distribution and is independent of c. When K 3 and À E R2 satisfies X = (0, c, c) for some с >> O, then for + 1, it holds that
((() Ω (((2)
(A) 2 (c+ 1)0/2
(a (a+1)C()))
and
3/2 (1)
(a ()+1)C(2a))1
Proof. By definition of for any X E [0, 00)* K we have
(( 1-x - seY F(z + 21)dz
(A I- f(z 2₂) 1I F(= 45)dz
Let us consider K = and = (0, c, c) for some c > 0. Then, for * 1, the numerator is written as
- -' X₂)
F(: x,)dz
#$1
f'(2 + (c)))) + +c)dz
-f'(= (()) + ccdz + -f' ((() c)d=
t2
+ -(===c))(()(=++++
For the first term, since f' + 0 for 2 -c, we have
t1 = f' -
((()
2 F(-c) ----- (f' c) + c)dz = F(-c) / (()))
For the second term, since 7(=+0) 0 for -c, we have
12 = L (()) (()
2 F(-c) 1" -
c)F c)dz (f)===(-)=- J. (-- ((())()))
For the third term, we have
t3 / -((==))P((=F((++00
F(-o) (- =
F(-c) f.- -f())
49

---

Therefore, we obtain
-f'(z + A II
F(= Aj)d= 2 F(-c) (=))
jfi
= F((cc) (z)dz
= (- Jo (
(symmetry)
2F(-c) J/ (
Sy(2) 2
(z+ (1+1)o0t 1)a+1
dz,
where Sy(z) denotes the corresponding slowly varying function. Note that the slowly varying function is function satisfying
g(xx) g(z)
lim
1, Vt > O,
which implies Sy(2) = o(2") for any O. Moreover, by Assumption A.5 lim inf Sy(z)
holds (refer to Lee et al. [38, Appendix A] for more details). Here, (z > 0 holds for all 2 since we consider the unimodal symmetric distribution. Therefore, there exists constant cf such that Sy(2)= 2 ef for all = 2 0. Hence, we have
2F(-c)
Sy(2) ([ ((2+1)at1
2
de 2 2F(-c) 11++laa1] dz
//(
cf 2
20, =
(a 1)(cc 1)o²) )
1
21+1
since is distribution function of symmetric Fréchet-type with F(-z) 1- F()) 0 ((++))
for 0.
For the denominator, for + 1, we have
f( A) II F(z 2,)dz
f(z + ((=+)) cddz
1" ((E+ +
+(=+c) ((=))) cddz+
4
[ f(= F(e)
For the first term, we obtain
$1 11 I ((2)))(22)((2+
< F(-c) / +(2+c) ccdd
F(-c)
f(z + c)dz = F(-c)
5 2
4
50

---

The second term is directly bounded by condition, i.c., 12 - ((-c) For the third term, we obtain
PC
$3
00
-
1(=+c)d==1-F(c F(-cc)
Therefore,
f(z + 2x) II F(2 S +/44)
Therefore, obtain
-(( 0.(A)
i1 + 12 + ts
2r²
=
$1 + $2
1
22 + (D(C 5/4)
= -(11)
and
((2)
f1 + f2 +
(2a 1)(C(D.) 5/4)) 1)
207
Pi (22
+ +3
G.6 Specific results for symmetric Pareto distribution
Here, we explicitly substitute the density function and distribution function of symmetric Pareto distribution.
Proposition G.2. Let the perturbations be i.i.d. from the symmetric Pareto distribution with shape 2 considered Proposition for + it holds that
5.3 When = 3 and E R. satisfies = (0, c, c) for some c > 0, then
((())
1
c+1
2
and
(A)
31
11
Proof. Let the same notation used in the proof of Proposition G.1 numerator, we obtain
For the first term of the
-3
1
i1
(1 2 - c) 2(1 2)2 2(1 - c)2 dz
-1
000 (1 - c0 2)2 dz
3
|-00 (1 2 0)6 de -3
-1
4(c + 1)2
20(c + 1)2
For the second term, we obtain
0
3
1
(1 (1-
1
t2
dz
(2 C + 1)4 2(1 22122)2 2)²
2(z + c 1)2
2
(s+c+ 1)º(1 - 2)2 dz
(# + 1) 2)
3 0
1
( 1 c+ 11++1)11111 dz
(: ( S 1, Vz (
3
dz= = 4(c 1)2
4(c + 1)2
(1 + 1)
4(c + 1)5*
51

---

For the third term, we obtain
3
1
1
f3
1
1
da
(z
(2+ 1)4
2(2 1)3
2( + c 1)2
100
3/2
1
I
dz
c++ 1)
2(2 + c c 1)3
0
2(c+ 1)3
20(c + 1)5*
Therefore,
1--0 (= + 2:) II
00
F(=+ + d, 2
10(c 1)2 +
2(c 1)3
20(c 1)
j#1
10(c 1)
For the denominator term, we obtain
1
81 =
c)3 2(1 2)² 2(1 c)2 dz
(1
1
1
d=
4(e + 1)2
-00 (1
c)5
16(c + 1)2"
For the second term, we obtain
[.
0
1
1
$
-
de
(a c 1)32(1 2(1 2)²
2(2 + c + 1)2
1
(2 + + 1)3 2(1 2)² dz.
By partial fractional decomposition, one obtain
1
1/(c + 2)² (++c+1)) 1)3 + 2/(c+ 2)3 3/(c+ 22
1/(c+ 2) (=+c+1)² +cc+1 (1-z)²
3/(c + 2)
+
c++ 1)3 21=
1-2
Therefore,
1
1
(a + + 1)3 2(1 - 2)² dz
1
1
3 1)
1 (c+
1-
+ 2(c + 2)
4(c 2)2
(c 2)3
c+1
1
3 3log(c 1)
+
+
2(c + 2)3
c+1
2(c + 2)
1
3
3c
+
4(c + 2)2
2(c + 2)3
(c+2)-
$
4(c + 2)2
2 + 2)3
10
5
4(c + 2)²
2(c + 1)2
For the third term, we obtain
1
2(z + 112) )
1-
dz
(2 (2+c+ 1)3
2(2 + c + 1)2
1
(2+c+ 1)3 1)3d2
2[c+1)²
52

---

Therefore, for K = 3, when X = c0,
Lo f(z + An) II
3+ 3 1/16
F(z + 2
(42)
(c 1)2
jtt
In sum, for K == 3, when X (0, c)., we obtain
-A((1)
1
1
(x(A)
103 + 1/16
31
Moreover, obtain
((()) 2
1/10
96v3
c+1
(c+
(c 1)
(2) 7 73/(43 .333)
1715
11
53


Reviews:
[{"review_id": "rHqxHFzl1W", "reviewer": "Reviewer_zV4p", "summary": "Follow-the-Regularized-Leader (FTRL) and Follow-the-Perturbed-Leader (FTPL) are well-known meta-algorithms for online learning problems that achieve optimal rates in both adversarial and stochastic settings. In some cases, they even attain a best-of-both-worlds (BOBW) guarantee.\n\nFor FTRL, prior work has shown that hybrid regularizers, typically combining negative entropy and Tsallis entropy, can be used to obtain a BOBW guarantee. For FTPL, recent research has demonstrated how to use hybrid perturbations to achieve a similar BOBW result.\n\nThis paper extends that line of work by refining the conditions on the types of perturbations used in FTPL and by analyzing both asymmetric and symmetric unbounded perturbations. Specifically, the authors introduce:\n\nA family of asymmetric perturbations that can achieve BOBW guarantees in the general case.\n\nA family of symmetric perturbations that achieve BOBW guarantees in the special case of two arms.", "questions": "See the above.\n\nLine 156, Possible typo?", "limitations": "Limitations are addressed.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe paper deepens our theoretical understanding of FTPL, particularly how to design hybrid perturbations that can mimic the behavior of FTRL with hybrid regularizers.\n\nThe study advances the long-term goal of understanding how FTPL can be made comparable to FTRL in a principled and efficient way.\n\nThere are several technically non-trivial contributions related to the analysis of FTPL, which I consider to be the main strength of the paper.\n\nWeaknesses: \n\nAlthough the paper provides additional insights into how to design FTPL algorithms with BOBW guarantees, prior work has already established that FTPL can achieve BOBW under certain settings.\n\nAs a result, the contribution of this paper feels incremental.\n\nI am unsure whether the contribution is significant enough. For me, it is borderline.", "was_revised": true, "final_justification": "POST REBUTTAL:\nMy initial score may have been a bit harsh, so I’m raising it from 3 to 4 (borderline accept). \nThe technical contribution is solid, and the problem is both interesting and important.", "rebuttal": "Thank you for taking the time to review our paper.\n\n> Comment: The contribution of this paper feels incremental.\n\nWe believe our contribution goes beyond distributional extensions in Section 4 as explained below.\nBy considering perturbations fully supported on $\\mathbb{R}$, it becomes possible to clearly connect FTPL to discrete choice theory (Section 3, Appendix A.3), which offers a more principled approach for understanding and designing FTPL.\nIn addition, Section 5.2 proves the first BOBW guarantee under symmetric heavy-tailed perturbations for $K=2$, building on the FTPL–FTRL duality discussed in Section 5.1. \nSection 5.3 then establishes a new negative result for $K\\geq 3$, revealing the fundamental limitations of current analysis techniques.\nTaken together, these contributions go beyond simply showing that “another distribution works,” as they advance theoretical understanding and suggest new directions for FTPL design grounded in duality and tail behavior.\n\n---\n\nA more detailed explanation is summarized below:\n\n### A1. FTPL–FTRL correspondence and regret analysis beyond known cases\n\nOne motivation of our work is to explore when the recent successes of FTRL, particularly with Tsallis entropy and hybrid regularizers, can be replicated by FTPL policies, which are simpler and optimization-free.\nIn this regard, we revisit the relationship between regularization and perturbation and investigate under what conditions such design is possible in Section 3, which may inspire the future design of FTPL.\n\nIn the two-arm setting, we show that FTRL with Tsallis entropy corresponds to FTPL with symmetric Fréchet-type tails, which numerically resemble symmetric Pareto distributions (Section 5.2). \nWe then establish the first BOBW guarantee for this symmetric unbounded case not covered in previous FTPL work.\n\n---\n\n### A2. Revealing limitations for symmetric perturbations in the general case\n\nIn contrast to the positive two-arm result, Section 5.3 shows that FTPL with symmetric perturbations fails to achieve BOBW guarantees for $K\\geq 3$ under very widely used framework.\nProposition 5.5 shows that key stability terms can be unbounded in this case, revealing a concrete limitation of current analysis techniques.\nThis suggests that perturbation designs effective in two-arm settings may not generalize to larger action spaces, revealing the limitation on the approach that relies on the FTRL–FTPL correspondence established in the two-arm case [3, 32].\n\nAs briefly discussed in Section 1 and Remark 5.7, Abernethy et al. [2] analyzes FTPL using $-\\phi_i'/\\phi_i$ term, showing near-optimal $O(\\sqrt{KT\\log K})$ regret for perturbations with bounded hazard functions.\nSince their regret decomposition is quite tight, they conjectured that FTPL with Gaussian perturbations, whose hazard functions are unbounded, could suffer linear regret under an adaptive adversary.\nIn our case, where the goal is $O(\\sqrt{KT})$ regret, the analysis relies on the term $-\\phi_i'/\\phi_i^{3/2}$ as done in recent BOBW studies for FTRL and FTPL [27, 30, 37, 53].\nProposition 5.5 thus strongly suggests the existence of such adversary that forces FTPL with symmetric perturbations to suffer $\\Omega(\\sqrt{KT\\log K})$ regret.\nSince the current decomposition is already almost tight (up to minor refinements), our results at least show that the current technique fails to obtain optimal regret even though the current decomposition is already almost tight.\nPlease also see the response to Q2 of Reviewer QAp1.\n\n---\n\n### A3. Bridging to discrete choice theory\n\nWe also see our results as contributing to a broader unification of FTPL design and discrete choice models extensively studied in econometrics and operations research. \nWhile prior FTPL analyses in the ML literature were often focused on perturbations with nonnegative support, our generalization to perturbations on $\\mathbb{R}$ aligns more naturally with well-established (discrete choice) models like the multinomial logit or nested logit. \nThis connection enables the application of insights and techniques from those mature fields, which may inspire future FTPL design in online learning contexts."}, {"review_id": "TkWXuvt6kf", "reviewer": "Reviewer_QAp1", "summary": "The authors study Follow-the-Perturbed-Leader (FTPL) policies for a large class of asymmetric distributions and derive Best-of-Both-Worlds (BOBW) regret guarantees, i.e., that are optimal in both the stochastic and adversarial settings. The authors also study FTPL with symmetric distributions, where they show that BOBW guarantees are achievable when the number of arms is $K=2$, but that their analysis do not generalize to $K \\ge 3$.", "questions": "1. Can you give insights regarding why the stochastic regret guarantee of Corollary 4.2 only holds for alpha=2?\n2. Regarding the implications of the negative result in Section 5.3, as you note in Remark 5.7, your results do not constitute a full impossibility proof. Could you elaborate on what this would imply for the prospect of achieving BOBW guarantees with symmetric perturbations? Does it suggest that an entirely new analytical technique is needed, or that perhaps only a small subset of \"well-behaved\" symmetric distributions might work?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This paper is theoretically very solid. All statements in the main text are proven in great detail in the appendices.\n\nI found the paper to be well-written and generally clear. The extensive literature review in Sections 2,3, and Appendix A.1 help greatly with situating this work and understanding its contributions. Section 4 is well organized and it is commendable that the leading constants are tracked in the statements of Theorems 4.3 and 4.4. However, I found the exposition in Section 5 somewhat harder to follow.\n\nIn my opinion, the main contribution of the paper is the derivation of BOBW regret guarantees for FTPL under asymmetric perturbations  (Section 4). The BOBW guarantee for symmetric perturbations in two-armed bandits (Section 5.2) is also a meaningful contribution, and the negative results of Section 5.3 are particularly valuable for future work,  but as the authors point out in Remark 5.7, they only show the failure of a specific analysis technique, which does not entail that FTPL with symmetric frechet-type distributions cannot achieve BOBW guarantees.\n\nAdditionally, I have noticed a few minor typos and/or potentially confusing wording (which do not impact my overall positive assessment):\n- Line 105: `might be enable'\n- Line 135: I believe there is a typo in the definition of the importance-weighted estimator. Shouldn't its $i$-th coordinate be  $\\frac{\\ell_{t,I_t}}{\\mathbb{P}(I_t=i)}\\mathbb{1}[I_t=i]$ ?\n- Line 250: 'let $\\lambda_i$ is the...' -> `let $\\lambda_i$ be the...'. I also personally found the wording \"let $\\lambda_i$ be the $\\sigma_i$-th smallest among $\\lambda_1,\\dots,\\lambda_K$\" slightly confusing, and would recommend something like \"let $\\sigma_i$ be the position of $\\lambda_i$ in the nondecreasing ordering of $\\{\\lambda_1,...\\lambda_K\\}$\"\n- Line 298: 'if exists' -> 'if it exists'\n- Line 349: 'do not occur' -> `that does not occur'", "was_revised": true, "final_justification": "The authors have addressed my questions well. I remain confident that this paper contributes significantly to the FTPL literature, so I am keeping my score.", "rebuttal": "Thank you for taking the time to review our paper and pointing several typos in the current manuscript. We will revise the manuscript accordingly.\n\n---\n\n### Q1. Insights on optimal result only for $\\alpha= 2$ in stochastic setting\n\nWhile $\\alpha$ also influences the leading constants in the adversarial setting [37], its role is particularly critical in the stochastic setting, where the logarithmic regret is the goal.\nThe tail index $\\alpha$ controls the trade-off between exploration and exploitation in FTPL.\nA small $\\alpha$ induces heavier tails and more frequent large deviations, encouraging exploration (e.g., by selecting arms with large $\\hat{L}_t$) but potentially preventing exploitation.\nConversely, larger $\\alpha$ leads to lighter tails and more exploitative behavior, but with insufficient exploration to correct early misestimates.\n\nThis phenomenon is analogous to the role of $\\beta$ in $\\beta$-Tsallis entropy, which similarly influences the exploration–exploitation balance of FTRL. \nIn particular, Zimmert and Seldin [53] identified $\\beta = 1/2$ (corresponding to $\\alpha = 2$ in our setting) as a special case that achieves optimality in both cases, and provided an intuitive explanation for its effectiveness in their Section 4.3.\nThey also discussed why $\\beta \\neq 1/2$ may lead to suboptimality in Section 4.2, which further supports the delicate balance needed in tuning this parameter.\n\n---\n\n### Q2. Further implication on BOBW guarantee with symmetric perturbations\n\nAs you mentioned, our current analysis does not formally rule out the possibility of achieving a BOBW guarantee with symmetric perturbations. \nWhile we do not provide a concrete counterexample, we conjecture the existence of an adaptive adversary that induces $\\Omega(\\sqrt{KT\\log K})$ regret in this setting.\n\nThis conjecture is analogous to that in Abernethy et al. [2], who showed that FTPL achieves $O(\\sqrt{KT\\log K})$ regret when the hazard function $-\\phi_i'/\\phi_i$ is uniformly bounded.\nTheir analysis is based on the following decomposition, believed to be tight:\n\n$$\n\\begin{aligned}\n\\sum\\_{t=1}^T \\mathbb{E}\\left[\\langle w\\_t - e\\_{i^* }, \\hat{\\ell}\\_t \\rangle\\right]  \n    &= \\sum\\_{t=1}^T \\mathbb{E}\\left[\\langle \\phi(\\eta_t \\hat{L}\\_{t}) - \\phi(\\eta_t \\hat{L}\\_{t+1}), \\hat{\\ell}\\_t \\rangle\\right] + \\sum\\_{t=1}^T \\mathbb{E}\\left[\\langle \\phi(\\eta\\_t \\hat{L}\\_{t+1}) - e_{i^*}, \\hat{\\ell}\\_t \\rangle\\right] \\\\\\\\\n    &\\geq \\sum\\_{t=1}^T \\mathbb{E}\\left[\\langle \\phi(\\eta\\_t \\hat{L}\\_{t}) - \\phi(\\eta\\_t \\hat{L}\\_{t+1}), \\hat{\\ell}\\_t \\rangle\\right] \\\\\\\\\n    &\\approx \\sum\\_{t=1}^T \\sum\\_{i=1}^K \\mathbb{E}\\left[\\langle -1[I\\_t = i] \\eta\\_t \\ell\\_{t,i} \\frac{\\phi\\_i'(\\eta\\_t \\hat{L}\\_t)}{\\phi\\_i(\\eta\\_t \\hat{L}\\_t)}, \\hat{\\ell}\\_{t,i} \\rangle\\right] \\\\\\\\\n    &\\approx \\sum\\_{t=1}^T \\sum\\_{i=1}^K \\mathbb{E}\\left[-\\eta\\_t \\ell\\_{t,i}^2 \\frac{\\phi\\_i'(\\eta\\_t \\hat{L}\\_t)}{\\phi\\_i(\\eta\\_t \\hat{L}\\_t)}\\right].\n\\end{aligned}\n$$\n\nTo achieve $O(\\sqrt{KT\\log K})$ regret, it suffices to uniformly bound $\\phi_i'/\\phi_i$ under appropriate $\\eta_t$. \nSince this is not possible for Gaussian perturbations (which have unbounded hazard), they conjectured that FTPL with Gaussian perturbations suffers linear regret under an adaptive adversary.\nIn our case, the goal is to achieve the sharper $O(\\sqrt{KT})$ regret, which requires $\\phi_i'$ to be smaller than $\\phi_i^{3/2}$ rather than $\\phi_i$.\nProposition 5.5 shows that this is not valid under FTPL with symmetric perturbations, which strongly suggests the suboptimality of symmetric perturbations.\nAlthough our analysis does not yield a formal lower bound, the level of informality is similar to that of Abernethy et al. [2], as both rely on decompositions regarded as tight and use them to argue that certain perturbations likely fail in adversarial settings."}, {"review_id": "QXDpvIZXke", "reviewer": "Reviewer_vYoW", "summary": "This work generalizes the BOBW analysis in previous works [1,2] by showing that FTPL can achieve BOBW results using symmetric unbounded Fréchet-type perturbations. Further, the authors also show that FTPL with the symmetric Pareto distribution with shape 2 is also feasible to achieve the BOBW results, in the 2-arm setting. On the negative side, the authors also demonstrate that in the general MABs with more than 2 arms, FTPL with symmetric Fréchet-type perturbations does not satisfy the usual condition for establishing BOBW results.\n\n[1] Honda et al. Follow-the-Perturbed-Leader achieves best-of-both-worlds for bandit problems. ALT, 2023.\n\n[2] Lee et al. Follow-the-perturbed-leader with Fréchet-type tail distributions: Optimality in adversarial bandits and best-of-both-worlds. COLT, 2024.", "questions": "1. There seems to be a concurrent work [3] which simply uses the i.i.d. Fréchet distribution with shape 2 on each base arm and achieves the BOBW results for $m$-set CMAB. Could the authors briefly discuss whether the results of FTPL with hybrid perturbations can be extended to the CMAB setting? If so, are there any advantages of the method in this work compared to that in [3]?\n\n[3] Zhan et al. Follow-the-Perturbed-Leader Approaches Best-of-Both-Worlds for the m-Set Semi-Bandit Problems. arXiv, 2025.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. **Motivation and Novelty**: Given the progress of FTRL with hybrid regularizers in various bandit problems, it is of great interest to study FTPL with hybrid perturbations. I think both the techniques and the results in this work are valuable and novel to the online learning literature. \n2. **Writing**: This paper is generally well written.\n\n**Weaknesses**\n\nThough most parts of the writing are clear, I personally feel that the writing may be further improved if some parts of the paper could be further explained or discussed. For instance,\n1. For the term “unbounded perturbations” throughout the paper, I am not fully sure how the authors define it. For me, [1] consider the Fréchet distribution with shape 2 on the support $\\mathbb{R}_{\\ge 0}$, which seems to be “unbounded” to me. However, after a complete reading of this paper,  I feel that the authors seem to denote “unbounded distribution” by meaning that a distribution has support on both the positive and negative parts.\n2. From Line 2 to Line 8, the authors seem to express an opinion that FTRL with a hybrid regularizer (roughly) corresponds to FTPL with perturbation, which has support on both the positive and negative parts. I am wondering whether FTRL with a hybrid regularizer can (roughly) correspond to FTPL with a summation of two independent perturbations that only have supports on the positive parts. If so, shouldn’t it be an easier starting point to investigate the FTPL counterparts of FTRL with hybrid regularizers?\n\nBesides, as the main results in the paper body are a bit heavy, I would like to suggest that the authors include a table summarizing the results of this work. I understand that it might be difficult to fit into the main body. Perhaps including such a table in the appendix might also benefit the readers.", "was_revised": true, "final_justification": "The authors have resolved all my previous questions.", "rebuttal": "Thank you for taking the time to review our paper and for your valuable feedback.\nWe will include a summary table or figure (of Venn diagram) in the revised version to provide a clear overview of the position and scope of our results.\nAs pointed out, the term \"unbounded distribution\" was intended to refer to distributions with support on both the positive and negative real line.\nWe acknowledge that this terminology was unclear and will revise the manuscript to clarify our notation accordingly.\n\nWe address your specific questions and comments below. \nThroughout, by \"semi-infinite support\" we mean a distribution satisfying $\\mathrm{supp} F \\subseteq [\\nu,\\infty)$ for some finite $\\nu\\in \\mathbb{R}$.\n\n---\n\n### Q1. Hybrid regularization and hybrid perturbation (weakness 2)\n\nWe agree that summing two independent perturbations with semi-infinite supports and different tail indices $\\alpha_1, \\alpha_2$ can be a promising and perhaps more accessible starting point.\nHowever, it remains unclear what forms of such perturbations can effectively mimic hybrid regularizers that achieve a BOBW guarantee.\nHence, it would be more beneficial to find a more systemic approach rather than adhoc approach.\nIn previous hybrid regularization approaches, the regularizer is constructed from two distinct types. \nTo name a few examples, as discussed in Line 82-89:\n* Zimmert et al. [54] used Tsallis entropy with Shannon entropy applied to the complement in combinatorial semi-bandits, i.e., $-\\sqrt{x} + \\gamma (1-x)\\log(1-x)$\n* Jin et al. [30] used a Tsallis entropy with log-barrier to extend BOBW guarantees of FTRL in the MAB setting $-\\sqrt{x} + \\gamma \\log x$.\n\nIn some earlier works (not necessarily in the BOBW context), hybrid regularizers are designed to penalize small probabilities, i.e., $x\\to 0$, typically using log-barrier terms like $\\log x$.\nIn such cases, summing two positive-support perturbations may capture the desired behavior as you mentioned.\nHowever, in the BOBW literature, especially for combinatorial semi-bandits, hybrid regularizers commonly include an additional penalty near $x\\to 1$, such as $(1-x)\\log(1-x)$ (Shannon entropy to the complement).\nTo the best of our knowledge, all known BOBW hybrid regularizers in combinatorial settings fall into this category [28, 54]. Capturing such effects via FTPL, penalizing both over-exploitation ($x\\to 1$) and insufficient exploration ($x \\to 0$), likely requires perturbations with support on both positive and negative sides.\n\nFrom this perspective, a natural candidate for hybrid perturbation in more complex settings would be a mixture or composition of Fréchet-type and another perturbation corresponding to a non-Tsallis entropy defined on $\\mathbb{R}$.\nAs discussed in Section 3.2, the perturbations on $\\mathbb{R}$ correspond to generalized entropies, via the inverse gradient of the potential function, which defines the associated FTRL policy.\nIn this sense, perturbations on $\\mathbb{R}$ may offer a more unified and systematic way to study the behavior of FTPL with the regularization–perturbation duality, particularly when leveraging insights from discrete choice theory.\nFor this reason, we view the study of perturbations on $\\mathbb{R}$ as a more principled and broadly applicable approach to designing FTPL policies.\n\n---\n\n### Q2. Advantages of hybrid perturbations compared to Zhan et al. (2025) (question 1)\n\nThank you for pointing out this interesting reference.\nIt is indeed interesting that a simple choice of Fréchet perturbations can achieve the BOBW guarantee.\n\nAs far as we understand, one limitation of the result in Zhan et al. (2025) lies in the looseness of the minimax regret bound, where they obtain a near-optimal guarantee of $O(\\sqrt{mT}(\\sqrt{d\\log d}+m^{5/6}))$.\nWhile this improves upon the previous FTPL bound with exponential distribution of $O(m\\sqrt{dT\\log(d/m)})$ by Neu and Bartok [41], especially when $m$ does not scale with $d$, it remains suboptimal compared to the optimal $O(\\sqrt{mdT})$ bounds by FTRL methods in Zimmert et al. [54] and Ito [28].\nAs Zhan et al. (2025) noted, achieving tighter bounds (and BOBW guarantee) are likely to require fundamentally different or more refined techniques. \nIn that context, designing hybrid perturbations could be one of promising directions."}, {"review_id": "fggs6mB1cF", "reviewer": "Reviewer_FXse", "summary": "This paper studies FTPL for MAB. While all existing results either use unbounded symmetric perturbation (e.g., Laplace noise) or non-negative Frechet-type noise. This work is the first to show that BOBW bounds can be achieved even using unbounded but asymmetric perturbation, that is, Frechet distribution on both sides but with different tails.", "questions": "n/a", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. BOBW guarantee of FTPL with asymmetric unbounded perturbation is a new result. It also needs new analysis.\n2. Sharp remarks/discussions are presented.\n\nWeaknesses\n\n1. My main concern is the motivation of the significance of the result. \"These results not only extend the BOBW results of FTPL but also offer new insights into designing alternative FTPL policies competitive with hybrid regularization approaches.\" How is asymmetric perturbation exactly connected to hybrid regularier in FTRL? How would this new result let us mimick the effect of hybrid regularizer from FTRL into FTPL?", "was_revised": true, "final_justification": "my final recommendation would be 4: Borderline accept. I highly appreciate the technical novelty of this paper but I'm not sure about how useful the results could be, e.g., establishing connections to FTRL", "rebuttal": "Thank you for taking the time to review our paper. We have addressed your questions and comments below.\n\n---\n\n### Q1. How is asymmetric perturbation exactly connected to hybrid regularizer in FTRL\n\nIn FTRL, additional regularization terms are often introduced to stabilize the arm-selection probabilities, e.g., to prevent abrupt changes as in Jin et al. [30].\nIn the FTPL framework, such stabilizing effect can be implicitly controlled by the choice of the perturbation distribution.\nAsymmetric perturbations can therefore be interpreted as a way to control the behavior of FTPL, where the lighter left tail introduces a stabilizing effect similar to that of an auxiliary regularizer in FTRL, such as Shannon entropy [54].\nWhile similar effects might be reproduced by summing two independent semi-infinite perturbations with different tail indices, we expect that introducing a small probability of negative perturbations can subtly increase the conservativeness of FTPL, without altering its overall structure.\n\nHowever, it remains open which class of perturbations actually corresponds to hybrid regularizers.\nThis correspondence may be related to either a summation of independent perturbations or a single perturbation with asymmetric tail behavior, as considered in our work.\nNevertheless, we believe that asymmetric perturbations, particularly those supported on $\\mathbb{R}$ provide a reasonable starting point, motivated by the regularization–perturbation duality from discrete choice theory, as discussed in Section 3.2 and in our first response to Reviewer vYoW."}]

Author rebuttal:

Meta-review: The paper studies the Follow-the-Perturbed-Leader family of bandit meta-algorithms known for achieving optimal scaling for both stochastic and adversarial settings (the so called Best of Both Worlds guarantees). Here the authors study a family of perturbation distributions for FTPL policies and derive BoBW guarantees for asymmetric unbounded perturbations. They further derive similar guarantees for symmetric perturbations in problem instances with two arms ($K=2$), and demonstrate that there exist problem instances with $K>2$ where their analysis technique does not suffice (the conditions shown to be necessary for the derivation of BoBW guarantees do not hold).

Overall the paper is well written and the reviewers agree that the results further the understanding of an important family of methods, FTPL, and details further connections with the FTRL (Follow the regularized leader) family of meta-algorithms. Reviewers explicitly mention the importance of studying hybrid perturbations in the FTPL, given the success of hybrid regularizers in the FTRL approaches and the lighter computation required in FTPL family of algorithms.

In light of this contribution, I recommend acceptance for this submission and encourage the authors to address the reviewer comments in their final version.
Decision: Accept (poster)