Follow-the-Perturbed-Leader Nearly Achieves Best-of-Both-Worlds for the m-Set Semi-Bandit Problems
Authors: Jingxin Zhan
Yuchen Xin
Chenjie Sun
Zhihua Zhang
Venue: NeurIPS 2025 poster ()
Keywords: multi-armed bandit
combinatorial semi-bandit
m-set semi-bandit
best-of-both-worlds
follow-the-perturbed-leader
Fréchet distribution
Primary area: optimization

Abstract: We consider a common case of the combinatorial semi-bandit problem, the $m$-set semi-bandit, where the learner exactly selects $m$ arms from the total $d$ arms. In the adversarial setting, the best regret bound, known to be $\mathcal{O}(\sqrt{nmd})$ for time horizon $n$, is achieved by the well-known Follow-the-Regularized-Leader (FTRL) policy. However, this requires to explicitly compute the arm-selection probabilities via optimizing problems at each time step and sample according to them. This problem can be avoided by the Follow-the-Perturbed-Leader (FTPL) policy, which simply pulls the $m$ arms that rank among the $m$ smallest (estimated) loss with random perturbation. In this paper, we show that FTPL with a Fr\'echet perturbation also enjoys the near optimal regret bound $\mathcal{O}(\sqrt{nm}(\sqrt{d\log(d)}+m^{5/6}))$ in the adversarial setting and approaches best-of-both-world regret bounds, i.e., achieves a logarithmic regret for the stochastic setting. Moreover, our lower bounds show that the extra factors are unavoidable with our approach; any improvement would require a fundamentally different and more challenging method.

Follov-the-Peruubbe--LLadd Nearly Achieves Best-of-Both-Wordde for the m-Set Semi-Bandit Problems
Jingxin Zhan
Yuchen Xin School of Mathematical Sciences Peking University Beijing, China 100871 23011 2301110087@pku, edu. cn
School of Mathematical Sciences Peking University Beijing, China 100871 xxdxz edu. cn
Chenjie Sun
Zhihua Zhang
School of Mathematical Sciences Peking University Beijing, China 100871 scj2330stu pku. edu.
School of Mathematical Sciences Peking University
Center Intelligent Computing Great Bay University, China
zhzhang@math pku. edu. cn
Abstract
We consider common case of the combinatorial semi-bandit problem, the m-set semi-bandit, where the learner exactly selects arms from the total d arms. In the adversarial setting, the best regret bound, known to be O( O(Vnmd) for time horizon n, is achieved by the well-known Follow-the-Regie Reghhe-Regrriied--eeee -Lader (FTRL) policy. However, this requires to explicitly compute the selectiin probabilities via optimizing problems each time step and sample according to them. This problem can be avoided by the -eetowtt-e--rruube----lee Leader (FTPL. policy, which simply pulls the m arms that rank among the m smallest (estimated) loss with random perturbation. In this paper, we show that FTPL with Fréchet perturbation also enjoys the near optimal regret bound OVvim /d log(d) + m²/6)) in the adversarial setting and approaches logarithmic regret for the stochastic setting. Moreover, lower bounds show that the extra factors are unavoidable with our approach; any improvement would require fundamentally different and more challenging method.
-ft-ooffoothoorlll regret bounds, i.e., achieves
Introduction
The combinatorial semi-bandis problem (Cesa-Bianche and Lugosi, 2012] is an important online decision-making problem with partial information feedback, and has many practical applications such in shottest-path problems [Gai et al., 2012], ranking [Kveton et al., 2015a), multi-task bandits (Cesa-Bianchi and Lugosi, 2012] and recommender systems [Zou et al., 2019]. The semi-bandit problem is sequential game that involves learner and an environment, both interacting over time. In particular, the problem setup consists of d fixed arms, and at each round = 1, ... the learner selects a combinatorial action--a subset arms from a predefined set A {0, 1.dd Simultaneously, the environment generates loss vector [0. 1]d The learner then incurs loss of (At, Ex), where A, E A is the selected action, and receives semi-bandit feedback o, = A, @ f,, representing the losses associated with the selected arms only (here, denotes element-wise multiplication).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this work, we focus on a common instance of the semi-bandi setting, the m-set semi semi-bandit [Kveton et al., 2014], where each action consists of exactly m arms. That is, the action set is given by A = {a E {0, 1jd: ||a|| = m), with S m S d. The performance of the learner is quantified by the pseudo-regret, defined as Reg,, == R [-== A az; 4 where a. = arg A A E باس= (a, & represents the optimal fixed action in hindsight. The expectation is taken over the randomness of both the learner's decisions and the loss. The combinatorial semi- bandit problem has been studied primarily under two frameworks: the stochastic setting and the adversarial setting.
In the adversarial setting, assumptions made about the generation of the loss vectors 4; they can be chosen arbitrarily, possibly in an adaptive manner [Kveton et al., 2015a, Neu, 2015, Wang and Chen, 2018J. The optimal regret bound is (V(ndd [Audibert et al., 2014] (when m S d/2). In the stochastic setting, the losses f1, ......................... Ln E [o, 1] are independent and identically distributed samples drawn from an unknown but fixed distribution D. For each arm i {l..... d]. the expected loss is denoted by V = Eewp(e [4:] E [o, 1]. The suboptimality gap of arm is expressed min ut)+ and the minimum gap is A 15isd, O<á,
max by A, = (v+- (I,,,...d /.).////m jEZ
min A, There
many algorithms that were shown to achieve logarithmic regrets. For example, Kveton et al. [2015a] and Wang and Chen [2018] derived O (d-m) log(n) regrets in m-set semi-bandits.
In real-world scenarios, it often unclear whether the environment follows a stochastic or adversarial pattern, making it desirable to design policies that offer regret guarantees in both settings. To address this challenge, particularly the classical multi-armed bandit setting, a line of research has focused on Best-of- Best--f-Booth-- Worlds (BOBW) W) algorithms, which aim to achieve near-optimal performance in both regimes. A pioneering contribution in this direction was made by Bubeck and Slivkins [2012], who introduced the first BOBW algorithm. More recently, the well-known Tsallis-INT algorithm was proposed by Zimmert and Seldin [2019]. the context of combinatorial semi-bandits, related advancements have been made by Zimmert et al. [2019], Ito [2021] and Tsuchiya et al. [2023].
However, most existing BOBW algorithms are Follow-the- Follow-the-Reuularizee Regularized-Leatee (FTRL) policies and require to explicitly compute the arm-sclection probabilities by solving optimizing problems at each time step and sample according to it. This problem, particularly in combinatorial semi-bandits [Neu, 2015], has attracted interest and can be avoided by the policy, which simply pulls the m arms that rank among the m smallest (estimated) loss with random perturbation. More precisely, the FTPL, algorithm selects the action arg milaa.A (Ix Hh a), where rt., denotes a random perturbation drawn from a specified distribution, 1); is the learning rate, and L is an estimate of the cumulative loss for arm i, defined as Lr,- = Σ'= Le.s.
(FTPL)
Honda al. [2023] first proved that FTPL with Fréchet perturbations of shape parameter a successfully achieves BOBW guarantees in the original bandit setting (i.e., when m = 1), which was recently generalized by Lee et al. [2024]. They analyzed general Fréchet-type tail distributions and underscored the effectiveness of the FTPL approach. Nevertheless, in m-set semi-bandits the
arm-selection probabilities Wt, = (n L.) are much more complicated compared to the original setting, making it harder to analyze the regret for FTPL.
1.1 Contribution
In this work, we show that FTPL with Fréchet perturbations achieves O O(ynml (dloo(()))) + m²/0)) regret in the adversarial regime and O log(n)) regret in the stochastic regime simultaneously. A
This is the first FTPL algorithm to approach the BOBW guarantee in the semi- semi-bandit setting when m S d/2. Technically, first, we use the standard analysis framework for FTRL algorithms (originally introduced by Lattimore and Szepesvári [2020]), and extend it to cases where the convex hull of the action set lacks interior points-i.e.. i.e. to m--ee [2023]'s proof. Second, we generalize Honda et al. [2023]'s analytical techniques and handle the challenges posed by the complex structure of arm-selection probabilities in m-set semi- bandit.. Moreover, by establishing lower bounds, we demonstrate that our current approach has been pushed 5/5 to its limit limit-namely, the log(d) and 21/z factors cannot be removed. Any further improvement would likely require adopting a different and more challenging line of analysis.
biii-baandtttteeeehh simplifying Honda et al.
2

---

1.2 Related Works
The FTPL algorithm was originally introduced by Gilliland [1969] in game theory and later rediscovered and formalized by Kalai and Vempala [2005]. FTPL has since gained significant attention for its computational efficiency and adaptability across various online learning scenarios, including MAB [Abernethy et al.. 2015]. linear bandits [McMahan and Blum, 2004], MDP bandits [Dai et al., 2022], combinatorial semi-bandits [Ncu, 2015, Neu and Bartók, 2016] and Differential Privacy [Wang and Zhu, 2024]. However, in MAB, due to the complicated expression of the arm- selection probability in FTPL, it remains a open problem [Kim and Tewari, 2019] for a long time that dose there exist a perturbation achieve the optimal regret bound O( D(Vnd) in the adversarial setting, which had been already achieved by FTRL policies [Audibert and Bubeck, 2009]. Kim and Tewari [2019] conjectured that the corresponding perturbations should be of Fréchet-type tail distribution and it was shown to be true by Honda et al. [2023], Lee et al. [2024].
FTPL
BOBW Following the influential work of Bubeck and Slivkins [2012], a broad line of research has explored BOBW algorithms across diverse online learning settings. These include, but are not limited to, MAB [Zimmert and Seldin, 2019]. the problem of prediction with expert advice [de Rooij et al., 2013, Gaillard et al., 2014, Luo and Schapire, 2015], linear bandits [Ito and Takemura, 2023, Kong et al., 2023], dueling bandits [Saha and Gaillard, 2022], contextual bandits [Kuroki et al., 2024], episodic Markov decision processes [Jin et al., 2021] and especially, combinatorial semi-bandite [Wei and Luo, 2018, Zimmert et al., 2019, Ito, 2021, Tsuchiya et al., 2023].
2 Preliminaries
In this section, we formulate the problem and introduce the FTPL policy.
2.1 The Problem Setting and Notation
We consider the m-set combinatorial semi-bandit problem with action set A = fa E {0, 1] ||a|| = m). where each action selects subset of m arms and d 2 At each round t = 1,2, 2, the learner chooses an action A₁ E A, while the environment generates loss vector E [0, 1jd, The learner incurs loss (AA, ( and observes semi-bande feedback = A₁ @ le, i.e., the losses for the chosen arms only. The goal is to minimize the pseudo-regret Reg,, := E E (Ar - a., 4)], where a, arg min E >>tm {a, [a)] is the optimal fixed action in hindsight. In the adversarial setting, the loss vectors l, may be arbitrary and adaptive. In the stochastic setting, they are i.i.d. samples from a fixed but unknown distribution D. Let v; =- EE-D [f,] denote the expected loss of arm Define the
suboptimality gap of arm as A; == (v: (n...c( [Z]mm min f vz) + (A less formal way to put it is: the gap from the m-th smallest value.), and the minimum gap as A = ming;,,00 Ag. Here (z)+ = = 0 = max(=, 0).
To analyze regret, for any À (Ai...., Ad) E R2, we let A be the (A)-th smallest among the >; (ties are broken arbitrarily) and 2, = ()i max min X,)+. We denote by F, the filtration zi<m jēT
a (A,, 01, .,...., Ar, ot), and by 1 the all-one vector. Let ab = min(a, b) and V b = max(a, b).
2.2 Fréchet distribution
We consider the Fréchet distribution with shape parameter 2 (denoted F2), the density and CDF of which are
f(x) = 2x 3
- x 2 0,
respectively. In the following, "Fréchet" refers to this distribution without specifying the parameter. This choice is motivated by a sequence of prior studies on the FTPL algorithm in the MAB setting [Abernethy et al., 2015, Kim and Tewari, 2019, Honda et al., 2023, Lee al., 2024].
Briefly speaking, this choice stems from an intuitive probabilistic property of the distribution: we draw d independent samples from it, the order of the expectation of the maximum vd, which corresponds to the optimal regret bound /nd the MAB problem. Our subsequent analysis (Lemma E.2) shows that this property can be generalized to the case where the order of the expectation of the
3

---

Algorithm 1: FTPL wit geometric resampling for m--set Semi- bandits
Initialization :T =0 for / ,,...,7' do
Sample r, (f1,1 ................................ F(,d) iid. from F2
Play A, = argmin (L: Tt/ Tr/n.
Observe o A, 0 l.
for 1, ,,...,
Set K O.
repeat
Ke Kt,i +1.
// geometric resampling
Sample r' = (ri....,,y) i..dd from F2.
A4 argmin (Lr //n, a). until A/. =
11
Set W Ke bes = o,,Wie and l441, = Les les.
12
13
end
end
sum of the top-mn results is /md,, which likewise suggests the effectiveness of this distribution in the m-set cemi-bandits problem.
2.3 FTPL Policy
We study the heap--The-Perturbed--essss (FTPL) algorithm (Algorithm 1), which selects actions based on perturbed cumulative estimated loss L. E'= lg. At round t. the learner pulls the m arms that rank among the m smallest estimated loss with random perturbation rt/ne, where ra Rd has i.i.d. components drawn from the Fréchet distribution Fz. and 17, Ott-1/3) 1/2) is the learning rate. The probability selecting arm given Le is Wi, = (x) where for E Rd
(A)) Prr X among the top m largest values in r AI...., rd da).
(I)
Then by Lemma C.3, have (a(A) (A),, where
(A) = 1 1((++))))= m-1
Σ ( ==
Σ
II OET qEI.
II (1
((r+d2))
dx.
()))
We denote the true cumulative loss as L, Eb=1 function ó as (0 Qa) and wi as (Wr,1, Wr.)).
| la For convenience, we also denote the vector
Geometric Resampling In FTRL policies, Importance Weighted (IW) estimators are commonly used, where l.; A, for i = 1...., d. However, in FTPL algorithms, the action probabilities Wy.i often hard to compute directly. To address this, the geometric resampling technique (Neu and Bartók, 2016] is frequently employed. This method replaces w with an unbiased estimator wi.i Specifically, after selecting action At and observing outcome o., for each i i= 1....,d,, d, we repeatedly resample r' (rí, ...,.....7 r'd) i.i.d. from Fa and compute A arggmnn.. (L, r'/nes a) until A/,, 1, i.e., arm i is "selected". Let Kr be the number of such resamples; then by the properties of the geometric distribution, [E(K,,) = w... computation, we only need to compute K2,, for arms actually selected by A, [Honda et al., 2023]. Since A 0 implics é, 0, the remaining estimates can be omitted.
wt.s
1_ so we define w = K2,-- To reduce

---

Viewing as Mirror Descent FTPL can be interpreted as Mirror Descent [Abernethy et al., 2015, Lattimore and Szepesvári, 2020]. For all R let
$ (A) =B ma X,
aEA
(2)
Σ Bl(rs + A1)= 1 X, among the largest values in A1. Aa 1/-
Then, by exchanging expectation and the derivation (or see Lemma C.1), it clear that ( = ((-)) and (AA) is convex. Consider the Fenchel dual of Φ, (=( supzerd (z. u) (xx). Then FTPL can regarded as Mirror Descent with potential d", because wy o(n = ( However, is worth noting that V = -n:h, generally does not hold in this case, because for all t R, +(A+11) ((A) by its definition, and then V obviously not invertible.
3 Main Results
In this section, we present our main theoretical results, including the regret bounds and our new analyses of the regret decomposition. Theorem 3.1. In the adversarial setting, Algorithm I with learning rate tJs 1/vl satisfies
Reg. = O (Vnm( / (ogg(d) + +m//))
The proof is given in Section 4.3. Furthermore, Appendix C.3 provides lower bounds showing that current method (Section 4.1) is essentially inatt-ini that the log(d) and m/e factors inherent to our analysis. Thus, removing them would likely require fundamentally different and more sophisticated techniques.
In the stochastic setting, we assume that there are at most mn arms with A, = O. other words, we assume the uniqueness of the optimal action a.. This is common assumption in BOBW problems [Zimmert and Seldin, 2019, Zimmert et al., 2019, Honda al., 2023].
Theorem 3.2. In the stochastic setting, the optimal action is unique, then Algorithm I with learning rate = 1/VL satisfies
log(n) A:
+ +0 +0 (m') log(d) +m# mdd?)
Reg, -
Σ 1,
where = min;,,,20 Ag-
Its proof can be found in Appendix A. Therefore, FTPL with Fréchet perturbations approaches BOBW when S d/2. In addition, similar to Zimmert and Seldin [2019], Zimmert al. [2019], our algorithm adopts a simple time-decaying learning rate schedule т) 1/vt. Our results can be readily extended to general setting with 7) c/ /l for any 0.
3.1 Regret Decomposition
We follow the standard FTRL analysis framework for FTPL, originally by Lattimore and Szepesvári [2020, Theorem 30.4], extending it to m-set semi-bandits where the convex hull of the action set A has no interior points and hence 4 and V are not inverses of each other. For convenience, the following, let ?/u +oo. Then the regret can be decomposed in the following way: Lemma 3.3.
2 Σ ((Li) - o(mL(1z))) +
Reg.
E
Σ
E 1 (a.)
4" x0 (we)
T--
Stability Term
Penalty Term
Its proof is deferred in Appendix B.1. For the penalty term, we need the following result, whose proof can be found in Appendix E.I.
5

---

Lemma 3,4. For all X E R., let a ( Then (a =-E((r, A),, where A
arg m (r A). Furthermore, for all E A, we have 4- (a) S -E((r,))
Combining Lemmas 3.3 and 3.4, have
a
Regn
E
Σ MnLLi - - b(nc))))))
+
B Kr,, A: - a.)l
T--
Stability Term
Penalty Term
It is worth noting that in the decomposition of Honda et al. [2023] (see Lemmas 3 and therein) and of Lee et al. [2024]) (see Lemmas and therein), the stability term
Mente) - 100---00n o( Amerleetl)
1=1
which is further decomposed into
ln omnLe o(n((())))
E
(i.e., the stability term in our decomposition) and
Σ (ee, pln
E
Therefore, these two terms both need to be controlled in their analysis. More specifically, the second term is bounded by O(n) in Honda et al. [2023], while it is bounded by O O log
th in Lee et al. [2024]. In contrast, our decomposition does not require controlling the second term. Moreover, our penalty term is almost the same theirs. Consequently, on the one hand, we simplify their proof; on the other hand, our bound is tighter. This also further demonstrates the superiority of the FTRL-based analysis framework.
Remark 3.1. Furthermore, by Generalized Pythagoras Identity (Lemma G.2), for the stability term, obtain tighter upper bound Σt= nhs Li -mLL)), which is more popular in the analyses of FTRL policies and usually approximated by the sum of e ( L.) However, such an approximate relationship difficult to establish FTPL because V2 ( (n [2+1) and (2 L,) may not be close enough.
4 Proof Outline
This section begins with analyses for the stability term and the penalty term, followed by proof for Theorem 3.1 and sketch for Theorem 3.2, whose details can be found in Appendix A. Although our analysis follows the framework in Honda et al. [2023], directly applying their approach fails in the m-set semi-bandit setting due to the intricate structure of the arm selection probabilities.
4.1 Stability Term
For the stability terms, informally, we will show that
a o.
E (- Mm m
L(((
2 (A) (x)
and hence, the key component of the analysis lies in bounding the quantity the definition, is upper bounded by (A). (i, However, each VI.N (A) is sum over many terms. To
which, by

---

V., (A)
effectively bound this ratio, our strategy is to apply union bound over all individual terms such that [Z] < m and & I. where we define
12 1,5 (A)
1 (a v (a) Sx
\ 1/(+
[1
x2)
II
F(r + 1z)
(3)
GET
94I, 9ëi
To this end, we require the following generalization of Honda et al. [2023]'s result, whose proof can be found in Appendix C.I.
Lemma 4.1. For any I = (1,... .d]. IE Idd such that X and any N 2 let
(N) = I nc
I F(x + X²)) II F(x + Ag) dr.
(x+
9EZ
Then for all k > O, J.. is increasing on Ag 2 forrr I (.,),((A)
In the MAB setting (i.e., the case m 1). there are no 1 - F terms, and we can leverage monotonicity let certain X tend infinity, making the corresponding
terms approach 1, which greatly simplifies the form of the ratio. However, for general m. removing - F terms would require sending the corresponding Xy to negative infinity. This is not feasible, as the monotonicity does not hold generally. To address this difficulty, we rely on the following result:
Lemma 4.2. For all J 2 O, 2 N 2 and M 1, let μ4 E foror all isiS S M, and define
HN f
100
K (x+ (++
II (1 F(x + pz) dz.
For all k E NN we have
HNNK S
M
k/3
CN.K
A 1
HN
where CN. is a positive constant only depending on N and k. Furthermore, if K 2 M, then we have
HA
M
+1) M
K
1/2
A )
Sc
log
H3
where C is a positive constant.
The proof of this lemma is tedious; it constitutes the most difficult part of the stability term analysis. Therefore, we defer to Appendix C.2. We also have lower bounds in Appendix C.3, showing that the logarithmic term and M2/33 are inevitable. Based on Lemma 4.2, we have the following result, whose proof is deferred in Appendix D.I.
Lemma 4.3. There exists > O such that all 2 1 and 1 sisa d
m logíd) (L,) ( ml/3
oi(L.) 2m o,(L)) S 2m.
E [ees (delmlla) delmett)) [r-]] < C. Las At
Note that by the definition in Section 2.1, L too when (L(L) S m. As direct corollary, have:
Lemma 4.4. There exists C > 0 such that for all
E (l, - ((pLL41)) F1-1 < Cn((Vmllog(d) + m m/3).
Proof. By Lemma 4.3, the left-hand side is less than
m log(d)
Cn 2m 4/3 +
Σ
S C' mnd + m 1/3).
V
(T.(..)
2.))().)))

---

Finally, we also need different upper bound making use of L, in the stochastic environment and the proof be found in Appendix D.2, which used new technique compared to Honda al. [2023]. Their proof relies on the uniqueness of the optimal arm, while there are in the mesett semi-bandits.
Lemma 4.5. If
E (meIa mm++1
< 2m
then
I8 ((nL))
sC
Σ 1=m+1
(Lnd mewers) m2 Ineed
where C is absolute positive constant.
4.2
Penalty Term
Then we present our analyses for the penalty term. Lemma 4.6. For all X E IRd, we have
" (a(a) "0 (o()) & 5v
Furthermore, ifa arg min
E (a', A). then
$* (a 45 (x())) 2
E
A-1
The proof is given in Appendix E.2. It is worth noting that the first part of the result stems from key observation: if one draws i.i.d. samples from the Fréchet distribution, then the expected sum the top m largest values among them can be upper bounded by (V/md). Then clearly, we have: Lemma 4.7. For all 2 1, we have
E E[ 20 (we)) S 5V
Furthermore, if max Li S 18i5m m+isigd
min Les and a. (L,
,,....0) 0), then
m mf
d-m of
E (a.) - x0 1.--1l 2m
i=
4.3 Proof for Theorem 3.1
Combining Lemmas 3.3, 4.4 and 4.7 with 1)t
1/v7, we have
Reg.. I C
Σ ne Tel log(d) +m 4/3 +5
Vmd S C" Vnm( (dlog(d)+m 5/6),
t=1 11
Th-1
(=1
where we applied Lemma G.5.
4.4 Proof Sketch for Theorem 3.2
W.I.O.G., we assume that V1 S U/ S ... S va and then a₂ (1,., 0...... ,0)
We apply the
m
d-m ofo
technique by Honda et [2023] and hence define the event A Σ (n -2
11 On
2m
i=m+t
one hand, by Lemma 4.4, 4.5 and 4.7, one can show that
n
Reg, 50
Σ E
(A.)
Σ Lit + 1l 4ART
[Vd dlg((d) m//6)
n
d
dw
+0
+0 (md?)
Vt
(=m++
+1
8

---

On the other hand, using the fact that
72
Reg
D
W
Wesi
Li=++1
in Appendix we will show that
Reg,
Σ 1=1
E
-
Σ
A.L., + 1
i=m+1
111
Hence, with Reg,, = 3 Reg,, -2 Reg,, 3 + 131 - + +0 (md+), one can get the logarithmic result by noting that L,, tA, = -( and V (Vllog(d) + m //0 and dwe. are less than A and Awr, respectively when t is large enough. Details can be found in Appendix A.
/
15
E9P2
2000
1000
20000
60392
60003
03000
110000
10
17'
122
109
301
30000
TTH
32"
JOCB4n
6000
32"
2000
327
20022
42000
00000
00500
130002
1.7
160
135
161
Timm
Figure I: Comparisons of our algorithm FTPL and several existing algorithms. The left side is in linear scale and the right is in log-log scale.
5 Experiments
In this section, we evaluate the empirical performance of FTPL and several benchmark algorithms on the m-set semi-bandit problem. We compare our method against five established baselines: for the stochastic setting, we include COMBUCB [Kveton et al., 2015b] and THOMPSON SAMPLING [Gopalan et al., 2014]; the adversarial setting, we use EXP2 [Audibert et al., 2014] and LOGBAR- RIER [Iuo et al., 201 8), corresponding to FTRL with generalized Shannon entropy and log barrbarrer regularizers, respectively. We also compare against the BOBW algorithm- FTRI with a hybrid regularizer (hereafter referred to as "hybrid") [Zimmert et al., 2019] in both settings.

---

Following [Zimmert et al., 2019], we run experiments on a specific instance of the m-set semi- bandit with parameters d 10, m = 5, and n 10 The loss for arm at time has mean Pti- and the realized loss is 0 with probability 1 - Uu and with probability Vti independently across arms and time. In the stochastic environment, the losses are generated from a stationary distribution where the mean loss for arm at time ł is given by V/, = A if i S 5, and V/s 2 + A otherwise, with = 0.1. In the adversarial environment, we employ the adversarial setting detailed in Zimmert et al. [2019], framework with numerous practical applications. This setting divides the time horizon into phases: ,,...,Ri,,
Ri, n2 + I,..., .?,..... ,,....,7......................,,
-1..1.... n. The duration of phase s is N, = 1.6*, and the
mean losses configured follows:
+ 1
i
otherwise.
2
where A 0.1 and in +. + is used if time falls within an odd-numbered phase, and otherwise. We sample a sequence of loss vectors from the above setting and fix it as our adversarial environment, then run the algorithms to be compared on this fixed sequence. Across all experiments, we estimated the pseudo-regret using 20 repetitions. The resulting average pseudo-regret for each algorithm over time is presented in Figure 1. Our experiments are conducted on a server with 4 NVIDIA RTX 4090 GPUs and Intel(R) Xe Gold 6132 CPU @ 2.60GHz.
6 Concluding Remarks
To summarize, have shown that FTPL with Fréchet perturbations achieves both D(Vum(Vdlog(c)) + m)/))) regret in the adversarial regime and OiE,,, A. regret in the stochastic regime. This makes it the first FTPL algorithm to approach the Best-of-Both-World (BOBW) guarantee in the m-set semi-bandit setting when < d/2. Our analysis has been built upon the standard FTRL. framework, which we extend to accommodate the lack of interior points in the convex hull of the m-sse action space. In doing so, we simplify and partially extend the arguments of Honda et al. [2023], and attempt to address the technical challenges arising from the intricate structure of arm-selection probabilities in the semi-bande setting.
log(n)
Va, can be established to eliminate An important open question is whether sharper upper bound on Va,i
the log(d) and m 5/6 d17z factors in the regret bound, thereby enabling FTPL to achieve the BOBW guarantee. Appendix C.3 suggests that Lemma 4.2 is already tight, meaning that these factors cannot be removed. Therefore, obtaining tighter bound on Vi.5 is not possible through bounding Va. the term-wise ratio; instead, one must analyze the ratio of the full summations directly, which is substantially more challenging [Chen and Honda, 2025]. Moreover, it has been shown that FTRL algorithms [Zimmert et al., 2019] can achieve the BOBW guarantee even in the regime where m d/2. Whether there exists FTPL algorithm capable of matching this performance remains an intriguing open problem. Promising future directions include extending our analysis to more general Fréchet distributions [Lee et al., 2024], investigating broader classes of the semi-bandit settings, or removing the assumption of the uniqueness of optimal actions (our additional experiments suggest that the assumption may be not essential).
Acknowledgments and Disclosure of Funding
We would like thank anonymous reviewers for pointing out an issue with Lemma 4.1 in the previous version. We resolved this by correspondingly modifying Lemma 4.2, and still proved that -ertturbee- Leaderr approaches the Best-of-Bothh Bothooo-ssssss for the m-set semi-bandit problem.
This work has been supported by the National Natural Science Foundation of China (No. 12271011 and No. 12350001) and the MOE Project of Key Research Institute of Humanities and Social Sciences N1o222JJD1100011
References
Jacob Abernethy, Chansoo Lee, and Ambuj Tewari. Fighting bandits with a new kind of smoothness, 2015.
10

---

Jean- Jean-Yves Audibert and Sébastien Bubeck. Minimax policies for adversarial and stochastic bandits. In Proceedings of the 22th anmual conference on learning theory, pages 217-226, Montreal, Canada, June 2009. URI.tttar / /npp. hal.
Jean- Yves Audibert, Sébastien Bubeck, and Gābor Lugosi. Regret in online combinatorial optimiza- tion. Mathematics of Operations Research, 39(1):31 (113))33-45, 45, 2014.
Sébastien Bubeck and Aleksandrs Slivkins. The best of both worlds: Stochastic and adversarial bandits. In Shie Mannor, Nathan Srebro, and Robert C. Williamson, editors, Proceedings of the 25th Annual Conference on Learning Theory, volume 23 of Proceedings of Machine Learning Research, pages 42.1 42.23, Edinburgh, Scotland, 25-27 Jun 2012. PMLR.
Nicolò
and Gábor Lugosi. Combinatorial bandits. Journal of Computer and System Sciences, 78(5): 140--1422, 2012. ISSN 0022-0000. doi: 001. JCSS Special Issue: Cloud Computing 2011.
Botao Chen and Junya Honda. Note on follow-the-perturbell problems, 2025. URL https: //tpps////axvvv.///aa http:/awwvvvo...rree////// wttpoag///aa..//eee 12490.
in combinatorial
Yan Dai, Haipeng Luo, and Liyu Chen. follow-the-perturbed-eee for adversarial markov decision processes with bandit feedback, 2022.
Steven de Rooij, Tim van Erven, Peter D. Grünwald, and Wouter M. Koolen. Follow the leader if you can, hedge if you must, 2013.
Rick Durrett. Probability: Theory and examples. 2019.
Yi Gai, Bhaskar Krishnamachari, and Rahul Jain. Combinatorial network optimization with unknown variables: Multi-armed bandits with linear rewards and individual observations. /EEF/ACM Transactions on Networking, 20(5): 1468- 2012.
Pierre Gaillard, Gilles Stoltz, and Tim van Erven. A second-order bound with excess losses. In Maria Florina Balcan, Vitaly Feldman, and Csaba Szepesvári, editors, Proceedings of The 27th Conference Learning Theory, volume 35 of Proceedings of Machine Learning Research, pages 176-196, Barcelona, Spain, 13-15 Jun 2014. PMLR.
Dennis C. Gilliland. Approximation to bayes risk in sequences of non-finite games. The Annals of Mathematical Statistics, 40(2):467-474, 1969. ISSN 00034851, 21688990. URL http: // //www/ jstor. wwwgootggaaaboooo.o httssstallle//e 22399463.
Aditya Gopalan, Shie Mannor, and Yishay Mansour. Thompson sampling for complex online prob- lems. In Eric Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 100- 108, Bejing, China, 22-24 Jun 2014. PMLR. URL htttp:://proceeeeeee gopalan14 html.
mlr h....peess/v2/.
Junya Honda, Shinji Ito, and Taira Tsuchiya. Worlds for Bandit Problems. In Shipra Agrawal and Francesco Orabona, editors, Proceedings of The 34th International Conference on Algorithmic Learning Theory, volume 201 of Proceedings of Machine Learning Research, pages 726-754. PMLR, 20 Feb-23 Feb 2023.
hewp--thhe-Petuubed--eeeeee Achieves Best-offBotth
Shinji Ito. Hybrid regret bounds for combinatorial semi-bandits and adversarial linear bandits. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 2654-2667. Curran Associates, Inc., 2021.
Shinji Ito and Kei Takemura. Best-of-three-orrdss linear bandit algorithm with variance-adaptive regret bounds, 2023.
Tiancheng Jin, Longbo Huang, and Haipeng Luo. The best of both worlds: stochastic and adversarial episodic mdps with unknown transition, 2021.

---

Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences, 71(3):291-307, 2005. ISSN 0022-0000. doi: https://doi.org 10. 2004. URI. https::////s..... /tts:://wwwe http//ww.ciieeeee sciencedirect. com com///clence/articll//hhh. $0022000004001394, Learning Theory 2003.
Backjin Kim and Ambuj Tewari. On the optimality of perturbations in stochastic and adversarial multi-armed bandit problems, 2019. URL https://arxiv org/abs.///..o..oooo... ha///bb///a
Fang Kong, Canzhe Zhao, and Shuai L.i. Best-of-three-worlds analysis for linear bandits with hollow-tie-regnlaiiiee-eeeed algorithm, 2023. URL https://// wttp://.rrgabbb////eeeeeee 2303.06825.......
Yuko Kuroki, Alberto Rumi, Taira Tsuchiya, Fabio Vitale, and Nicolò Cesa-Bianchi, Best-of- both-worlds algorithms for linear contextual bandits. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li, editors, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 1216-1224. PMLR, 02-04 May 2024.
Branislav Kveton, Zheng Wen, Azin Ashkan, Hoda Eydgahi, and Brian Eriksson. Matroid bandits: fast combinatorial optimization with learning. In Proceedings of the Thirtieth Conference on Uncertainty Artificial Intelligence, 14, page 420-429, Arlington, Virginia, USA, 2014. AUAI Press. ISBN 9780974903910.
Branislav Kveton, Csaba Szepesvari, Zheng Wen, and Azin Ashkan. Cascading bandits: Learning to rank in the cascade model. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 767-776, Lille, France, 07-09 Jul 2015a. PMLR.
Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. Tight regret bounds for stochastic combinatorial semi-bandits, 2015b. URL https: : /ttxiv/xivv//v/////v/////////avv org/abs abs 14100.11..0 0949.
Lattimore and C. Szepesvári. Bandit Algorithms. Cambridge University Press, 2020. ISBN 108448888
Jongyeong Lee, Junya Honda, Shinji Ito, and Min-hwan Oh. Follow-the-perturbed perturbed-leader with fréchet-type tail distributions: Optimality in adversarial bandits and cost-of-both-worldd In Shipra Agrawal and Aaron Roth, editors, Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pages 3375 3430. PMLR, 30 Jun-03 Jul 2024.
Haipeng Luo and Robert E. Schapire. Achieving all with no parameters: Adaptive normalhedge, 2015.
Haipeng Luo, Chen-Yu Wei, and Kai Zheng. Efficient online portfolio with logarithmic regret, 2018. www.arwwwwwioooaaaeoeooeeeeeeeee www.///////...oo...o./.// 07430.
URL
H. Brendan McMahan and Avrim Blum. Online geometric optimization in the bandit setting against an adaptive adversary. In John Shawe-Taylor and Yoram Singer, editors, Learning Theory, pages 109-123, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg.
Gergely Neu. First-order regret bounds for combinatorial semi-bandits. In Peter Grünwald, Elad Hazan, and Satyen Kale, editors, Proceedings of The 28th Conference on Learning Theory, volume 40 of Proceedings of Machine Learning Research, pages 1360-1375, Paris, France, 03-06 Jul 2015. PMLR.
Gergely Neu and Gábor Bartók. Importance weighting without importance weights: An efficient algorithm for combinatorial semi-bandits, 2016.
John Pryce. R. tyrell rockafellar, convex analysis. Proceedings of The Edinburgh Mathematical Society PROC EDINBURGH MATH SOC, 18, 12 1973. doi: 00.1017/5001300115000000 42.
Aadirupa Saha and Pierre Gaillard. Versatile dueling bandits: Best-of-both world analyses for learning from relative preferences. Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 1901 1902.... PMLR, 17-23 Jul 2022.
12

---

Taira Tsuchiya, Shinji Ito, and Junya Honda. Further adaptive best-of-both both-worlds algorithm for combinatorial semi-bandits. In Francisco Ruiz, Jennifer Dy. and aun-Willem van Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 8117-8144. PMLR, 25-27 Apr 2023.
Siwei Wang and Wei Chen. Thompson sampling for combinatorial
In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 Proceedings of Machine Learning Research, pages 5114-5122. PMLR, 10-15 Jul 2018.
Siwei Wang and Jun Zhu. Optimal learning policies for differential privacy in multi-armed bandits. Journal of Machine Learning Research, 25(314):1-52, 2024. URL http::ttttmll parggpaper//////e v25/21-12 10567111677... html.
Chen- Yu Wei and Haipeng Luo. More adaptive algorithms for adversarial bandits, 2018.
Julian Zimmert and Tor Lattimore. Connections between mirror descent, thompson sampling and the information ratio, 2019. URL
oogooogg/aaoo httg/bbs/1900 11000.18817 11817.
Julian Zimmert and Yevgeny Seldin. An optimal algorithm for stochastic and adversarial bandits. In Kamalika Chaudhuri and Masashi Sugiyama, editors, Proceedings of the Swenty-Secone Interna- fional Conference Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 467-475. PMLR, 16-18 Apr 2019.
Julian Zimmert, Haipeng Luo, and Chen-Yu Yu Wei. Beating stochastic and adversarial semi-bandits optimally and simultancously. In International Conference on Machine Learning, pages 7683-7692. PMLR, 2019.
Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, and Dawei Yin. Reinforcement learning to optimize long-term user engagement in recommender systems. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2810-2815 2019.
13

---

Appendix
Proof for Theorem 3.2
15
Decomposition
16
B.1 Proof for Lemma 3.3
17
Important Facts
17
C.I Proof for Lemma 4.1
18
C.2 Proof for Lemma 4.2
18
Lower Bounds
20
D Stability Term
22
D.1 Proof for Lemma 4.3
25
D.2 Proof for Lemma 4.5
26
E Penalty Term
26
E.1 Proof for Lemma 3.4
27
E.2 Proof for Lemma 4.6
27
F Probability of the Best Action
28
Auxiliary Lemma
30
14

---

A Proof for Theorem 3.2
W.L.O.G., we assume that V1 S V2 S S Da and then a, = -L,,.., 1,0, 0.....0))
Define the event
m of d-m of
A. - Σ (1r 1=m+1
2m and w =P{{A ax F(-1).
Our plan is to apply the self- eeff-bounding constrain technique by Honda et al. [2023], Zimmert and Seldin [2019]. We first derive the upper bound. On the one hand, if Le satisfies Ar, which implies that Lt,i then combining Lemma 3.3, 4.5 and 4.7, the regret in round should be
max Li, S min 15i5m
m+lgisd
bounded by
d +C Σ
dwr -m2 -22-Vt/2d
C
Σ i=m+1
L
Vt
i=
where used that M/+1 ?i v++1+++-- Vt <
1 nT/2 for the penalty term. On the other
2
hand, if L, doesn't satisfy Ac, similarly, by Lemma 3.3, 4.4 and 4.7, the regret in round / is less than
VVT (V((Vloo(()) + m00)
Putting Eq. (4) and Eq. (5) together, one can get
m d E + 1 (V + m///
dw
Reg,, S C
E E
4A4
+0
E
+Cmd8
1=1
i=m-1
1=1
-==++1
11
where we applied Lemma G.6 for the last term.
We then show the lower bound. Clearly, we have
Reg,,
E
Σ A
Wr
Σ E
L
Σ
A(1-ww)) Aaw. + [(4 A(1 - w')
(6)
1=1
Li=++1
i=++1
where we applied Lemma F.3 for the second term. Then by Lemma F.I and Lemma F.2, we have
Reg..
>
C"
E
1 (A.)
+
Σ ALL + 1 (4g)
i=++1
111
where C' is an absolute positive constant. Besides, similar to Eq. (6), we also have
Regn. 2
A
We
1=1
Li=m++
IV
Hence,
Reg, = 3 Reg,, -2 Reg, < (3T - ITT) + (3TT - IV) + Cmd?.
For 31 111, equals
d Σ L - C't Σ م ]
3C
+ +>
1
(30
m (v (V dlog(d) + m3/6) -c m
-
i=m+1
i=m+1
For the first term, since
bx? a² /4b for O, then there exists C" such that
O"
3C
L Σ O't Σ A
A
1m
i=m+1
15

---

3Cm (v /llogd))mmm/)) CAA
Note that 3C / (T (Vdlgg)())
m S C' in
after Vi
tten we have
ac vetoeliott
31 - 111
3C
V
m
(Vdlog(d) + m²/0)
t
1=1 1=1++
C log(n)
(d
m/33)
A
i=m++1
where C₁ and C2 are absolute positive constants and we used (x + y) s 2(x² + 82) for positive and y-
Then it suffices to bound 377 IV, which equals
d
3Cdwsss Aw:h)
VL
1i=++1
Similarly,
3C
A after V 3Cd Hence,
Vi
202
3Cdww,
3Cd
Camd
Σ
Awass
S m
<
Vt
Li=++1
where C3 is an absolute positive constant and we used that Eim Wt by Lemma C.2. We complete the proof by putting everything together.
B Decomposition
In this section, we give the detailed proof for the regret decomposition. Lemma B.1.
Let l1. ... ln E TRd and = (bTTT)) where (Ne)r= is decreasing with voo +00
and L, = Σ tg. Then for all a E A
Σ (a, a, (.) S E Cle ((1l(411 +
Σ
(Φ'(a) - 55' +a(()))
M--1
Proof. For convenience, let Mh41 Th. and an+1 -(m Note that ( Le), then by Lemma G.1,-7ph, E (a:), which implies that d, arg min (2)), where (i(x) =
('(x) + (z, Le). We then have "
Σ (ar a, 4)) = Σ
(or 0141, L6) +
Σ
(a141, - Σ (a, Ei)
1=1
t=1
t=l
1=1
n
x (a((2)))
4- (a((+)))
(al Oc+1, L1) + Σ
4
D (a(41)
x (a(41)
Mc+1
mec
[=1
(a)
4^(a)
(a))(a)
((((a=
7/+11
1/t
(et (,+1, + Σ (a)) $7 + Σ
1 11]
(x'(a) x²(a)))
118
Mh-1
+ 0 n+1 (4x+1) - 12+1(a) (a).
Since for all a A, ++ (0z+1) (p)+1(a), have
(0r a, Ex) <
Σ
((a ac+1, (x) ).(a))
6 (a.- 1)
Σ
n
1
1
(0'(a)
x0 (a:))
7
1=1
1=1
16

---

Then by the definition,
(ar)) - 07 (a++1) = ((p(((++))) (a(a)) (as+1 - az, -(-1)) = -D 1 - -1+1
1)t
where we used Lemma G.3 by noting that v = a, and ( 11111 = a1+1. Finally, by Lemma G.2 (taking - 1,+1, y -[t+ ++1 and - T4), have
- 0(+1,) Do (- Lu, 7++11++1)) & (4,, v T.t)
1/t
since Do D0(r. v) + (2,2,x) 2 0. We complete the proof by putting them together.
Remark B.1. The overall proof framework is based on Lattimore and Szepesvári [2020, Exercise 28.12), with the latter part inspired by Zimmert and Lattimore [2019, Lemma 3).
B.1 Proof for Lemma 3.3
Proof. Noting that B[A, F = ((T,Le), we have
Reg. = Σ (o(nLe)) a.. (2) =E
n
(((ee))) - an. a....)
Then suffices to apply Lemma B.I.
C Important Facts
In this section, we present some important facts to be used in our analyses. Lemma C.1. For all IRd, we have ( o( A) and (()) convex over R.d
Proof. By Eq. (2), since for all < i < d, E Elri < +o,, one can exchange expectation and the derivative, then we have
((A) E
[L (ry A; is among the top vn largest values in + A rr da o,(-1),
OA:
because
1 { r. among top largest values in + A1..... Aa O, a.s. a7:
This shows that ( = x(-A). For convexity, it suffices to note that taking maximum and expectation keeps convexity.
Lemma C.2. For all E IRd, we have Ed= Ps(A) = m.
Proof. By the definition, we have
Σ (a(a))
Σ 1 (rs among top largest values 11 AA,... Aa3 m.
Lemma C.3. (i(A) (A)
Proof. Because
(c(A) = E, [P (theee exist at most m 1 ri - ,i-1 Aj-1
Ti+1 * Ta As that are larger than rs A; xb1,
then it suffices to note that the conditional probability inside is just
Σ
II 1ET
F(x Xg))
II qEI.g
F(x F(x Nq)
=
17

---

C.1 Proof for Lemma 4.1
Proof. We follow the proof by Honda et al. [2023]. Let Q(x) h(x)(2 X1) "INEI F(x
Ag)) ILa Fax + >q). If q & I, then
J (.) = 2/1 +oc
//-0 (x (())) == 2J (A).
dig
Hence,
(LN+ Q7g ((N
=2
JNN=
Note that
(A
22.20 (x (0+)) dr
|
Q(x)Q(y) 2,) Ax) (g) Ax) dr dy.
and similarly,
Z = 2 / J Q(x)Q()) x,) + ( (g)) - A1) dx dy,
2.020
then we have J MN J.NN
Q())(()) [10 A,) (x + (y+ (v+
1.N20
Xq) (x+ (x Aq) A.) - * dz dy
(z))((v) (I + X₂) (y ) 3 ( + - (y+ A1) dr dy 2
J.y20
which implies that x(2) increases with Xg 2 0. (x,xz(N)
C.2 Proof for Lemma 4.2
We divide the proof into two parts. Recall that for all 0, K, M 2 1, 2 and μs R for all
1 < < M, we defined
M
HN = / (z+ -)-
K tnt
II
(1
F(x + wz) dz.
Lemma C.4. For all N+, we have
HN+k <
(M
M k/3
A )uth)
CNNN
HN
K
where CN, is positive constant only depending N and k.
Proof. The upper bound of ~ is obvious because (x -N < 1r - 11) -N and hence, in
( (M) 1/3 where Cyy- 8/N -2 Let z+p and
the following assume that μ -1 CN
M
( + w)) /A,u2
g(u) = u N-2 N-2- kn²
II
111
where A is constant such that f g(u) du 1. Consider random variable U with pdf g(u), then clearly,
H Hy
E
H
18

---

Our plan is to find some > 0 such that g(u) decays rapidly after ug. then we expect that the order of E[U*) is roughly ug.
Let E(u) log g()) and y:
+ p4 for all 1si5 S M, then when 0,
de
N 2 2Ku + >
1 10:20)
(u)
du
u1 V/
e -1
Then it suffices to show that when u 2 ug == CN K
1/3
12, dł (u) I uuu/2. Because note that du
MAp 2 4K-1/3 2 4K-1/2 and f=- 2 2no, then by Lemma G.10, we have
EU S u E|U (n- 1 (2-120000) $ (1 + 2k!!)a
Note that sup=20
1, then we have
de (u)
2
2M
2K +
du
w²
When 2 up. clearly, since Cу is large enough, we have
2M
Ku
w
2
Also, since V N-2 K when 2 No. we have N-2 - Ku. Therefore, when 2 no. du (u) <
de
- KKu/2, which completes our proof.
Lemma C.5. If 2 M. then we have
H4
M
K
+ 111)
1/2
A A
log
H3
M
where C a positive constant.
Proof. We still use the definition of g and U in the proof of Lemma C.4 (N = 3 and h: = 1). Let
VAp = (* log (7r + +)) 1))2/² When K < 32M, by Lemma C.4, the result holds clearly when C is large enough. Hence we then assume that K > 32M. Then ug < 1/3. Similarly, in the following. we also assume that μ 11 2 C'ug, where C" > 3 is a large constant to be chosen and is not depending on K and M.
Similarly, our plan is still to find some un > 0 such that g(u) decays rapidly after ug We will first compute the ratio g(sua) for 1, and then find suitable choice of ug such that this ratio decays g(ua) rate comparable the Gaussian tail.
However, the challenge is that we also need to ensure that g(ua) is not too large in order to concretely control the decay rate of g(sug). To illustrate this, will use simple fact: if we choose tig large enough that (u)) starts decreasing after 3 then the values of g(u) over the interval [33] 2np should all be larger than (uu)) Since g(u) is probability density function, we then have
25p
g(u) du 2 g(n) 3
u0
which naturally gives an upper bound on g(u)). For the convenience of presentation, in the following formal proof, scaled un to 3uo-
For all 2 t Z 1, we have
g(suo) g(tua)
F
1. + 14g
K -=
83kg
144
11
Euto
19

---

If fflsts 3, then tuo < 1. Hence, by Lemma G.7, we have
g(su)) 8 0 Kug(o?-( g(tua) t
M
80 -K
8 M
S
(8)
t uuu
u0
2
Then on the one hand, for all & [1, have
g(3uo) g(tua) S <3-5kk
(8)
M
I
30-5-M
5M IoRl 15 +1)
(8K M
K where used the definition of uo in the last inequality. Since
8, have
N7
(3uo)
S 3e 5M 5MM
& +1) (111)+200 +2M log( 1) < 1.
g g(tuo)
Then since C² 2 2,
g(u) du > /
2n0
1 2
g(u) du 2 vog(3vo),
which implies that g(3u)) S "00"
On the other hand, by Eq. (8), for all 8 2 3, we have
g(sua) (3u0)
S se Kup 9)
M
log(())) M log for +1)(0--0)) + +MM (og(8)+ Mogg (
Then since K > 32M 2 32, one can find C" > 3 that is not depending on K and M and large enough (one can then pick an C" larger than ( such that for C", we have
g(sua) g(3u0)
I 4ff loR 3 +1)0
Therefore,
J( sg(sup) ds sg(3mm) [ s log (oo(((++1))) +1)+100/2 ds S up Som f sc-M Mog 15 +1) ds
log
=w0 -1
M
K
+1))
c² M lox +1)/2 - 110 -1
M
Then have
[" S
ug(u) du Cu + u/ ( sg(suo) ds S (C" + 1)u0-
E[Ul S Cm
C.3 Lower Bounds
In this section, we will prove lower bounds for Lemma 4.2, showing that the logarithmic term and 11/3 are inevitable.
Lemma C.6. For all μ RR K 2 1, and N 3, define
UN(P) / - Ne- #h (1- F-= + 1J) dr.
Then there exists C > 0 such that for all K 2 2, have
(2(p) C'
log( K)
sup
HER
(2(
K
20

---

Proof. Let w = _, then r(w) V -log and
UN(H)
= K N
/ log w) 222 (1- -=(x(w) + pb) dw.
2
which implies that
VK Ua(1)) Up(1)
(o log +1 F(x(w) + 14)) dw ([(-log(W))
1o F(x(w) + p)
where W a random variable with p.d.f. proportional to (1 - F(x(w) + (we(1110))
An intuitive understanding of this result is that the pdf of W, given by 1 F(z(w) + pd) is
monotonically decreasing on [0, 1] and clearly exhibits sharp drop at e _²= the pdf is equal to 1 before this point and then gradually decays afterward. Our calculation will show that most of the
density is concentrated in the first half of the interval, and thus the expectation of (- log W 1/2
KK of a larger order than simply substituting W E x² Thus, by choosing μ K) the desired lower bound follows.
2K
Clearly, for all 0, we have
EI( log (og 2 <P S {)
(9)
Let μ which implies that
log(K) and
2K
log(K) K)
Then when S * one can see that x(w) + μ S
[1 F(x(w) + [D] dw = = 1-1
Then it suffices to show that
[1 F(x(w) [4] dw = =
(10)
since then we have
K-1/2
P(W ---= K-1/2 + OK--1/2 2 C,
where C is a positive constant. Combining Eq. (9) and Eq. (10) leads the desired result.
To show Eq. (10), one should note that when w -KK (K- 1/4 1/29-2
((x(w) + p1) 1 (n(w) 1n
S K-1/2
(x(w) 12)
K (V -log (V-1o-
where we used that 1 - -- < all x E R. Then
1/2--- f [I - F(x(w) + +1] dw S K-1/2
Finally, since
-== (--0(K-11/)/))))-=--1 (
(K 11x-1/44-11/---
and noting that, by s log(K) 2
+ - s(K-1/2 2k 1/4 o(1),
- s K 11/4/11/22
(K (K-1/4 1)2
we then have
1/44,
F(((w) [)] dw S <e- -x-11-1144-1133 - o(K o(K-1 1/2,
where we used that (1)) 1 o(1). It suffices to put everything together.
21

---

Lemma C.7. For all E RR M,K 2 1, and N 2 3, define
Rx(U)= = I. 1.00
N E - (1- F(x + ni)) M dr.
I
Then there exists C' > 0 such that for all M 2 2K. we have
(1
M
1/3
sup PR (M)
2
K
Proof. We still use the definition of 9 and U in the proof of Lemma C.4 (N = 3 and k = 1). Our plan
show that that exists j IR and corresponding C > 0 such that when 0 S u S uo == C (x) 1/3
du (u) 2 0, which implies that g(u) increases with a when u S uo. Then we have
U S M0
g(u) du S |
no g(u) du,
P
up
which implies that P (U S
S 2. Then by Markov's 's inequality, we have
EEU
2
uo pP
U
2
40
2
2
4
which is just the desired result.
To this end, let μ 1 > 0 and y = + 1. then
df du (u) = 2Ku +
2M
M/K
+
2Ku
udya (ett -1 - 1
u²yy (e -1
11
Since S y, when u S (M) 1/2 (e- 1)-3 1, we have
M/K
M/K
M/K (u + 1)3 (e 1) 2 1.
y² (er' -1) (u+ + 1)3 (ei² -1)
Take C = (c- 1) 3 > 0 and then one can see that when M 2 2K, un = C (M) 1/3
M $ (e- 1)-+
1/- - 1. Then when 0 s S un C (
(
M
1/3
du
de
( 2 0.
D Stability Term
In this section, we provide our results related to the stability term. We start with showing some important properties for Vi,N. Lemma D.1. For all C (1,... .d} such that [ < m and i & I, where sis d, we have
Vix(()=
f Ay (o + A
1
II
F(x Xn)) II
F(x + Ag) dr.
9EI
QET
Then for all 1 Sis d such that oA(A) > m, we huve
(1,N(A)= = [ max min 2, (a + A, 1/ n-
Σ
Σ
II
F(x+Xg))
II
F(x + 2g) dr.
S=0
IC
WET
Proof. For the first result, it suffices to note that F(x) = 0 when x < 0, then when x S - min X3x
j$Z
the integrand in Eq. (3) is just O. For the second result, recall that
Vi,(())= = m-
Σ
Σ
(A)
==
22

---

and note that for all I C fI, df such that [Z] < m and i $ I.
(-min min \3 - j4z
(-
max min > Z. JET
min I./Z]cm
min A jez
then the result follows from that for every VI (A), the lower limit of the integral can be further reduced max min Ag.
I, <m JEI
Lemma D.2. The followings hold:
N- 1. For all N 2, (1, - N-I-
2. For all 1 Usis and N 23, VA, is increasing in all Az, 4 i and decreasing in X
3. VA,N(A) = -( (A,
Lx (I (I+ e
d-1
M 1/(+2)))
Σ
E
11- F(x+
II
F(x + F(x+
2g) dx 20. 0.
Proof. For the first result, obviously, suffices to consider the case when o; > m. By Lemma D.I, we have
(A, = /
x
max min A (x N, e/(ooaa 2.12/cm
1/(++11)
m-
Σ
E
II
F(x + Xn))
II
F(x + 2g) dx
== IS
44E
qdI.
1 !--
max min + A de
(=+ de
N
-
where in the inequality, we upper bound the conditional probability inside (see Lemma C.3) by For the rest results, consider a random variable rf with density
--Ne 1/22 -1121222 1 (a220)
g(x)
e 1/0² (x>0)-
5tx IN-e--1/a dx
r N-N11
Then similar to Eq. (1),
(A) D( N-1 2
Pthhre exist at most m - 1 of rı X1. ; A-11 A4r"
va Aa that are larger than r/ - Ax),
where T1. ... 71-1.- ... .d Lid F2. Then these properties hold obviously.
Lemma D.3. For all S i d, we have
Ve,(A) VA,(A)
log(d) (2) m
oi(A)) 2m
2A Ad
o2(A)= 2m,
where C a positive constant.
Proof. For any I I {1.-... .1.....,d ,d} such that [[] < m and i I, by Lemma D.I, we have
(A) =
600
1
F(x + X₂)) II II GET
F(x + X²) dr
min Xy (x + A₂) N
287
9E4
(11)
F(z + II F(z + 2) dz,
(= + )
GET
23

---

where we denoted that 1, = Xq- min X for all 1 1 q S d and we denoted that + min Az.
5&z
j41
Then, clearly, (A)
(A) (2 )-1-1 which implies that
(A) (A)
max (A (di s.
max min A4) S A. z.jz|<m,iez jęz
For the second part, by Eq. (11) and Lemma 4.1, we have
(A)( JAAA(AA) JAA,I(X))
So (5+2/)0
(x+2f)
1IEEI F(= x²)) dz
(12)
(A) J.NNZ(A)) J(NNI(A*)
(((A)
So (2427788
[ FF(= + 2;)) dz
where we denoted that
too
q $ I and Ag 2 A
A
q E I and Xg S X
E
and A; is the oA smallest in (Ag To apply Lemma 4.2, one should note that here K = (A) and M []. If (,;()) 2m, then K 2 (A()) 2 2 M, then by the second result in Lemma 4.2, there exists > 0 such that the right hand in Eq. (12)
M K
(K
m log(d)
s c
log
+1) S V
M
(A(A) - m
If (A) < 2m, similarly, by the first result in Lemma 4.2, the right hand in Eq. (12) is less than
M
1/3
K
Lemma D.4. For all 1 < i d, N 3 and K. 2 1, we have
Vl,++e()) Vi,(A)
< CN.
where CN.K is a posilive constant.
Proof. Recall that in Lemma D.2 we have
d-1
A. (x + X1 1
Σ
Σ
II F(x+ xg)) II F(*+1p)
dr 2200 0,
GET.
TEI.
then the proof follows from the case (A) S 2m in Lemma D.3 by noting that [Z] < d.
Lemma D.5. There exists C> 0 such that for all E [0, and sis d,
Ve.(A + +ac;)
S.
Vi,((A)
& log(Vv.(AA) is upper bounded by Cd. By the definition, Proof. It suffices to show that
ax,
a (o,())
2x. V2,((A)
0%
Vio(A)
where
0 ~0 8
(1) = f ON, (x
1/(++A1)2
Σ
Σ
I[ 9ET
F(x + >g))
II
F(x + 2g)
de,
0A,
8=m
IC (1.... d) -
q
24

---

since lim a= (a+2,)0 1/(x+A,)² Note that 0 aA; (x+A,)u 1/(tt>)))) S (a+A;)0 1/(x+A,))
then it's clear that aA, Vio(A) (A). Hence, by Lemma D.4, there exits C such that BA; log(Vv,(A)) - Cd.
a
Then we show our the results about the continuity of Φ. Lemma D.6. There exists >> O such that for all Sisd, and E IR,, if (a( and w/ = cx + dei), then the followings hold:
mh log(d)
(2(A) > 2m
1. w w'= < Cwa X7 A
m 1/3
(A(A) S 2m.
2. w w' S Cd((1 w)a, ifa sd!
Proof. For the first result, note that for all &. E [0,
0o:
0.(2 (1 t)ae;)
(A (1 -))as).
Then,
Ooi (A+(1-1)aex) (V+, (Tfx(A+(--(eee)) (1 (A+, (V),
where we used Lemma D.2 the final inequality. Recall that w (x( (A)) then by Lemma D.3,
0b: (A + (1 - t)aet) S C'w A7
log(d) oz(A) m
03 (A) > 2m
02:
m 1/3
o, (A) < 2m.
Therefore,
- w = //' d
m mlgg(d) 0.()-m
oz(A) > 2m
6: + (1 - (1=t)ae; dt S C" wa A A
(13)
dt
m 1/3
(x(A) S 2m.
For the second result, let x- = Φ. then w w = 0:(A + ac;) - (A)... Similarly, for all t E [0, 1].
d
(A + tae;))
(A + taee).
dt
O);
Since now ē,(A) = (A), then clearly,
døi (A) = -VV + 4V,, 2 (V) 0λ,
Hence, combing Lemma D.5 and Lemma D.4, we have
0Φ + lae;)) S 4V,6(A tae;) s CV.,(A) S C'dVi,(A) Cn -0--m) w).
ax;
Finally, one can obtain the result by the way similar to Eq. (13).
D.I Proof for Lemma 4.3
Proof. By Lemma D.2,
di(ml(11) 2 di(nc + ee),
then
les (o.(rel - (s(ne s les (pi(m)l)) alve nele, ei))
Denote that
V llgild)
(,( > 2m
A Lif ABb
(i(t))
1/3
oi(L)) < 2m.
25

---

Hence, by Lemma D.6
fe w²) S CAwll S 2,
where we denoted that w si((ee)) and w' @s nels es). By Lemma G.4, B|K?, 9t-,, At] I 2w Then
Ele,, w') 11-1 Arl s 2CA w A
Hence, since B[Ar, 8 = w, then
El w') F1-11 s 2CA.
D.2
Proof for Lemma 4.5
Proof. By Lemma 4.3, it's clear that
Σ
B Ele (di(nEt)) (((tet))) | sc
Σ L...
imm+1
then suffices to tackle the sum for Sis S m. By Lemma D.6 and following the same argument in Lemma 4.3, for all 1 S m, we have
E [(nKi eple, (d)()) - de(nnett)) | Fr-] Cn.dwi (1 w(,s),
where we denoted that w, (n)Le) and used the fact that nele, d= when mi Ke S d=
Note that by Lemma F.I, when Σ (mrL..) -2 -. for all 1 S m,
Whes 2 w 2 1/2.
(14)
Hence,
m
m
Σ L Lmm (dr(maL))
| s 20rid Σ [1- w1,) 20n
Σ
W7,ss
i=m-1
d Σ Wtis = m by Lemma C.2. Finally, for all 1 m,
where we used that
1=1
E 1 [nnKe los (de((el - delmaleti) F < E (n Ar F.-1]
E I (mKeidd 1) i F,-1],
which, by Lemma G.4, is less than
(1 - w.,s) S (1 w.,1)
where we used Eq. (14) in the final inequality. It suffices to combine everything together naively.
E
Penalty Term
In this section, we present our results related to the penalty term.
Lemma E.1. I ~ F2, then for all æ 2 1, we have
B r 2 x < 4x.
Proof. By the definition, we have
Eir r x| 1- uf(u) du
F(x) < 2x [ uf(u) du s 2x
<2n²
( 2u -u du 4x,
where the first inequality used that - - 2 x/2 when 0 S x < and the second inequality used
that e 1/ < 1.
26

---

Lemma E.2. Consider Ti. ra Lid, F2 with r(k) as the kth order statistic for all S S
Then for all m S d, the expectation of the largest m numbers, say [k==d-m+ T(k) is less than 5/md.
Proof. Clearly,
E
Σ
r(k)
S
Vmd+E
Σ r(k) 1 (ros S Vmd+
Σ
E
rh 1 (m2vvrmm
J B=ddm-+
Lk m+1
km
Then it suffices to note that, by Lemma E.I, we have
E rk 1 (122 d/m $ 44/d/mP (r* vd/m) 4Vd/m (1-cmm/1)) 44/m/d.
where the final inequality used that 1 - - S
E.1 Proof for Lemma 3.4
Proof. If (A), which implies that E (A), then by Lemma G.I, we have
x" (a) = (A, a) - (A) = E A) (A) = (A, A}] B((r + X, A)] = - [r, A}},
where we used that [[[A] (->)) in the second equality and the third equality follows from the definition of 4. If a E A, for all E Rd we have (z) 2 (r + x, a)}, which implies that
s Φ'(a) (a) sup (z, a) (( .r,a) (r; a))
F,2 Proof for Lemma 4.6
Proof. By Lemma 3.4,
(a(a) - (o(()) S B( A a)},
(15)
where arg max (r + X, a). Then for the first result, since r E RR
ক (a) कh (d(A)) S E[max((, a²)],
w'EA
which is less than mv by Lemma E.2.
For the second result, W.L.O.G., assume that .... >a. then a (1....,,,,,,..... and m
d-m of0
by Eq. (15),
(a) (o(A)) <
Σ
im++1
By the definition of A, for all > m, have
Eir, (A,= =E, Er.E (A,=1) r; x Ad]
2(x +
-1/(++\]² m-1
Σ
Σ
I
F(x+
II
F(x + 2g) dx
(x A₂)
Ig(
(0.
EEI
(A) < 2X,
where we used Lemma D.2 in the final inequality. This completes our proof.
27

---

Probability of the Best Action
In this section, we present our results related to the lower bound for the regret. Lemma F.1. For all IR,
lei
W.
P( min 12
(r+ X;)
max Isidd
(r:
}z)},
where r1, 1 ,dd Lidd F2. Then, have the followings:
d
1. If Σ
<
then w. 2 2.
2m
1=
d
2. 1f Σ
2
then wx S
2m
16m
1=
Proof. If Σ di + oc, then there exists i S m <j d such that X 2 Xz. W.L.O.G..
i=m+1
we assume that Am 2 Am+1. Denote that X; r2 - for all S i d, then clearly,
W/ =P(1,, Xm are the m largest values among X,, Xa}
SP(X1, X Xm+1 are the m largest values among XI, Xa) == =w'--
Note that wo + w/, I 1, then have w, 1/2
Then we assume that Σ AF < to0, which implies that max X max A4. W.L.O.G., we m+
1sism
assume that Å1 -...5 Am Am+1 ... Ad. Hence, max min Aÿ- Am and 2 An I.7ZK<m JIZ
for all i > m. By the definition of W..
(x+
w. = 2
Σ (x+
smm+1
II (1 (1- e1 -(ttde- (04t)220) dr. (16)
Am+1 i=m+1
1sksm
We first prove the lower bound for w.. Since for all k S m, dp S Ams then
(x+ (1 1 e (a+dm) ((ot2220) ) dr.
W. 2 Am+t i=m+1
Σ (a+ Az)
Then by Bernoulli's inequality, we have
d
(a+t)))
W. 22
(x + 2x) -è jam+l
[ (1-me (a+Am)
110t2m203 dx
Am+1 i=m+1
d
(x+11))
+o0
d
E. (a+dg) 2 =-(x+
=2
(x No)
sam+n
dr
2m
Σ
(x+ A:) 1:) s=m+n
dx
Amt1 imm+1
Am imm+l
400
S (2-Aa) -2 2 --(x+>m)
=1
1
2(x + Am)
jimil
dx
400
=1 - m
1
22 -3
s=m+1
dz
21 m
1 -
2z -3 3 --
dz
0
A-2
=1- m
1 1 e
28

---

{
j-mil (o+2,) 2
Σ (o+dy) 2
where in the third line we used that
=2 Σ
(x+
j-m'l
i=m+1
d
and in the fourth line we used that A, X Am for all >> m. Hence, when
Σ
1
1. by 2m
x=++1
that 1 1 2 I, we have
Wx 2 1
2m 2
We then show the upper bounds. By Eq. (16), clearly,
d
(x+2,)) (1 -1-t-dm 2012)220)) dr
Wx 52
(x + 2o)
s=m+1
Am+1
m
d
Σ (K+t2) -2 (ctthm))
=1
2
(x+ 15) - jam=+
dr
Am (=++1
(17)
+00 d
S
(x+2,)-²
=1
cc (2+
-3
s=m+1
dz
i=m+1
to0
E (3-2,)
51
2
(=+
3
s=m-1
dz.
11
im++1
If Σ
1
2
note that when Am+1. 1 S (=+Am-1) p. then
2m
1=m+1
Σ
(=+2,)
<5 Σ (2+ 12
j=m+1
j=m+1
Hence, by Eq. (17),
t00
Σ (=+2))
W. 51
2
Σ
(a+
-3
jmm+1
dz
i=++1
1
5
Σ
(2m
2m +2,1
j=m+1
5
<1
j-m+1
5
5 Km
<1
e
<
16m
where we used that
e
2 when [0, 1] in the final inequality.
d
Lemma F2. If
Σ A 2
7m then all m <i d. dx(A)) 2 1.27"
(=++1
Proof. By the definition of ø; in Eq. (1), clearly, for all m <isd,
(x(A) >P( rj X is the largest in I' Amy ... , .d dd }
(18)
exp Σ b (2 + )x
dz.
(2 + 2z}²
min $
j=m
29

---

d
Since Σ A,
similar to Lemma F.1, W.L.O.G., we assume that A1 .... Am < Am+1 S A X Am for all > m. Therefore, by Eq. (18),
2m
m=
Ad. Hence, max min A3- Am and
I <m J&Z
we have
(A(A) 2
exp
Σ
dz
(2 + 1
(2 + 2,)²
Jm
d
Am+1 (++ A1)
exp
( + )3) 2
dz.
j=m
Note that when z 2 2m+1
d
Σ
(=+
Σ j=m+1
Aj 2 + + $2
Σ
2j <
1,
j= (2 21)
j=
j=m+1
then by Eq. (17),
0:(A)20
/ 2(2 + x 1 de cc- (2, + 12,
Am+1)
-2 1;²
4e
Am+1
where in the final inequality we used that A I A for all 2 m + 1.
Lemma F.3. Use the definition of w. in Lemma F.I, then we have
W. (A)
imm
Proof. Clearly,
- w. =Pf min (ri - Xr) <
max
(ri
Xx)
isism
m+1 m+1sied
d
=P
(ri - A; is among the top m largest values in r1 - ",, Ad b
i=m+1
(c(A).
i=m+1
G Auxiliary Lemma
Lemma G.1 (Theorem 23.5 in Pryce [1973]). For any continuous convex function g and any vector I, the following conditions on vector x" are equivalent to each other:
1. x² +gg(z).
2. x E 0g"(x")) (x*).
3. g(x) + 9" (x="= (a., x*).
Lemma G.2 (Generalized Pythagoras Identity).
Do + (=(2, (= (V VQ(v), z).
Proof. It suffices to expand the left hand by the definition that
Dz v) (z))
+(v) (x y. (v))))
30

---

Lemma G.3. If = (x(x) and (y) then
Doly,x) 4 (v)) (u v, v.
Remark G.1. Informally, this is just the folklore that Da (y, = Dø- (), However, remains umproven whether & is differentiable everywhere, making it incomvenient to discuss its Bregman divergence directly.
Proof. By Lemma G.I, we have
(*(u) = (u.z) - ((x), (v) = (v, v) -DV))
Then the right hand equals
( - (x)) (u, y 2) =b()) (σx) (V y x) (v)
Lemma G.4. If K is sampled from the Geometric distribution with parameter p E (0, 1), then
E|K"| & 3. Furthermore, for all n E N E K n] = (1-p) p)"
Proof. For the first result,
1 1- 2 12
= = + Var(K)
p2
p
For the second result, by direct calculation, we have
+00
+00
E[K - KAn Σ P(K2))
p- (1 =- p)"
1mn+1
Rmn+1
Lemma G.5. For all N
S 2Vn.
Vk
Proof.
1 VTa
11 /"
dr s 22nn.
vx
k=
t00
Lemma G.6. For all x (0, let f(x)= =
Σ
2
Vtr
then there exists C > 0 such that f(x)
<
C 22
E=1
Proof. It suffices to note that
f(x) < (10 Vtv di =
+00
too
toc
42
2
du
12 w2- do.
x2
Lemma G.7. For all μ I. and y 2 x > ify 2
then we have
1 F(x (1)
< Syy
- Fly +p)
31

---

Proof. If x + S y, then
1 F(x + 11)
1 1 Flv Fy +1) - 1-
When + 1 S 0. the right hand equals and is clearly less than 8y². Otherwise, y + μ > 0 and then suffices to note that
--F Fly + 11) = 1 e- (a++) 2 - 3/4 2
where we used y + S y + μ S 2y in the first inequality and - E= 2 x/2 when 0 S x S in the second inequality.
If + μ > y, then y + μ 2 x # y 2 and similarly.
1 Fl + (1) 1 -E (y + 22)
Also, since for all x 2 0, - - S
we then have
1 F- + μم = 1- e- (a+1)) 2 1 (x + 22
Therefore,
1 F(x + p))
y + #
y + y - x
2
2
<2
< S 8y²
- Fly + 1))
+ μ1
y
Lemma G.8 (Theorem 1.2.6 in Durrett [2019]). For x > 0,
(x-'-x- xx exp(-x²/22 < { expl- 12) dy S xx exxeeppp--
Lemma G.9. IfY NN10, 02) and 1 > 0, then for all k E N, we have
By (22 k!! k!! op. k-1 oph-1 - 20 /V2T
where we define that 0!! = 1.
Proof. If k = 0, by Lemma G.8, we have
σ
2² // 2n.
B[4 = (
I -e
P
If
x² 20
B|y 1 [Y2))
ye
dy
22 207
V2TO²
v VTT
Assume that the statement holds for all integers
0, I.... k - 1, then
400
2 yeee 20 dy
o²
E E|y*1 (v2m)
yk-l d -e 202
2
22T0²
d²
2 207
+ (k - 1)0 'E y [Y224)]
V2T02
Then it suffices to note that a & μl and
+ (k - 1). (I: 2)!! < k (I- 2)!! = k!!
Lemma G.10. For any random variable X, let its density be denoted by f(x). If there exist J 2x > and pl a such that
dlog (z) -x/02, Vpl 2 4
dx
then all k N if 2 24, we have
E & 1(22X2M) 2k!! nta
32

---

Proof. Consider Y ~ ~N(0, o²), then we will show that
E[x* ((2222m) - E | 2 Y 2 pl]
(19)
which just equals
22
(2222u)) P( 2
E 1 (Y2n)) $
V (
k!! op R lo
20
(20)
P( 2 Y 2
[ e dr
J
22
dx
where we applied Lemma G.9 in the last inequality. By Lemma G.8, we have
0 σ² /
0 03
dr 2
e
2
2
e
20
c
(21)
H
2p
2n²
2²
where used p¹ 2μ in the last incquality. Since μ o, we have combining Eq. (20) and Eq. (21), we have
2
1
20
Hence,
Y 2 k!!
o4 e
20
2k!! 4
B| p
3r Ap
5
20
3
2
2
The right hand is clearly less than 2μtx when μ 2 2a. To show Eq. (19), it suffices to show that for all 0 t pl,
P(X (22x22) 2) S P(Y 2 tl pl 2 Y 2).). p).
(22)
which holds when t S 1 because the right hand becomes 1 by the definition. In fact, we can prove stronger result: for all p² 2 1>
P(X (22222) 2)) P(X
P(Y 2 2
<
(23)
1 4222221 2 1)
P(Y 2 M 2 Y 1I)
If there exists to > y that violates Eq. (23), then have 7 [1/2X2n] 2te) P(X (2'2X2n)21)
P(Y2H122V22)
which is equivalent to
P(1 (12222)) 2 to) P(u S (1/2221) < to)
P(Y 2 2ul YY
P(Y < to F 2 2 μ)
because x+1
increases with positive x. Then have
Jr' f(x) dx f f(x)do do
2 p - 20 dr
(24)
f4 20 dr
By homogeneity, may assume f(to) c 20
Now since for all /² 2 n. have
flx) < -x/022 then for all 2 to. f(x) S 24 dar contradicts with Eq. (24).
for all μ S x to. f(x)
Iv
which
33

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The theoretical assumptions in our paper are well-established and supported by substantial body of existing literature, with relevant references duly cited. Moreover, we discuss potential directions for improvement in the concluding remarks.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
34

---

Answer: [Yes]
Justification: Our paper presents complete set of assumptions along with a fully self- contained proof.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We disclose all details of experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
35

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We provide the code in supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
Sundides/CodSSumisseeee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /nips. cttps/sccppbllcccccccccc http:/pbllcggguudeeeeeeeep guides/ /CooeS/ ssion agionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We disclose all details of experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We consider and report the statistical significance of our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
36

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide full information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htapps:///neue http:////er.../.eeeee
line??
Answer: [Yes]
Justification:
Our research confirms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
Our paper only focuses on the theory bandits, and there no societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
37

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper only focuses on the theory of bandits, and there is no societal impact.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We properly use and cite these assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
38

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
39

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in our paper does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: : htt///eeriss../e/ httpe//ips.ceoooneeeeeeeeeeeeeee / for what should or should be described.
40


Reviews:
[{"review_id": "XlpZGT6N1C", "reviewer": "Reviewer_7BAU", "summary": "The authors introduced Follow-the-Perturbed-Leader-type algorithm w/ Fréchet-distributed perturbations for the $m$-set combinatorial semi-bandit problem, where a learner selects exactly $m$ out of $d$ arms per round. It is the first to achieve Best-of-Both-Worlds guarantees in this setting: near-optimal $\\mathcal{O}(\\sqrt{n m d \\log d})$ regret under adversarial rewards and logarithmic $\\mathcal{O}(\\sum_{i:\\Delta_i>0} \\frac{\\log n}{\\Delta_i})$ regret for stochastic bandits. FTPL avoids the computational expense of prior FTRL-based BOBW methods by replacing per-step optimization with simple perturbed loss minimization. The analysis reveals the $\\log d$ factor is unavoidable via their proof techniques, establishing a \"fundamental limit of the *proof strategy*\".", "questions": "- This paper is self-contained, especially regarding technical details. It is also desirable to have a self-contained high-level illustration of the intuition behind the \"Fréchet distribution with shape parameter 2\" that enables the BOBW guarantee, right?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n- The high-level idea of using geometric resampling to tackle with the lack of full-information feedback aligns with the intuition in the semi-bandit setting and the regret decomposition in Lemma 3.4 is novel can correct.\n- The implication that FTPL without complex tweaks can achieve the best-of-both-worlds guarantee for semi-bandits is a clear and desirable.\n- The \"lower bound for the proof technique\" part (Lemma C.5) is a interesting contribution, which is seldom seen of its kind in the literature.\n\n### Weaknesses\n\n- The last clause from Line 191 to Line 192 is interesting, but its proof from Line 502 to Line 503 is not clear. To be frank, if the authors consider this part as a trivial argument, please give a more clear explanation. However, as far as I can tell, this part is either highly non-trivial or even **incorrect**.\n  - **I'm willing to reconsider my evaluation if the authors can convince me that this part is not wrong**.\n- Lemma C.5 deserves a more intuitive discussion in the Appendix, as it is a key counterpart of the main result.", "was_revised": true, "final_justification": "Based on the rebuttal, this submission will be a mathematically correct one, which adapts well-established technical toolkits developed by Honda et al. 2023 from bandits to semi-bandits. Therefore, my final decision is borderline accept.", "rebuttal": "Thank you for your valuable review and constructive suggestions. Regarding the weaknesses and questions, we provide the following detailed responses:\n\n**Weakness 1**: \nWe must acknowledge our oversight: the second half of **Lemma 4.1** was indeed incorrect. To address this issue, we have **strengthened Lemma 4.2** by **removing the assumption $\\mu_i \\leq \\mu$**, which eliminates the need for the monotonicity argument used in the original Lemma 4.1.\n\n>Lemma 4.2 (**modified**):  For all $\\mu\\ge0$, $K\\ge 1$, $ N\\ge 3$ and $M\\ge 1$, let $\\mu_i\\in R$ for all $1\\leq i\\leq M$, and define\n    $$\n    H_{N}=\\int_0^{+\\infty} (x+\\mu)^{-N}e^{-\\frac{K}{(x+\\mu)^2}}\\prod_{i=1}^M\\left(1-F(x+\\mu_i)\\right) d x.\n    $$\n    For all $k\\in\\mathcal{N}^+$, we have\n    $$\n    \\frac{H_{N+k}}{H_N}\\leq C_{N,k}\\left(\\left(\\frac{M}{K}\\right)^{k/3}\\wedge \\mu^{-k}\\right),\n    $$\n    where $C_{N,k}$ is a positive constant only depending on $N$ and $k$. Furthermore, if $K\\ge M$, then we have\n    $$\n    \\frac{H_{4}}{H_3}\\leq C\\left(\\left(\\frac{M}{K}\\log\\left(\\frac{K}{M}+1\\right)\\right)^{1/2}\\wedge \\mu^{-1}\\right),\n    $$\n    where $C$ is a positive constant.\n\n Under the more general condition on $\\mu_i$, the **second part of Lemma 4.2 remains valid (when $K\\ge M$)**, while the **first part becomes weaker**. As a result, the **adversarial regret bound becomes slightly worse** than before:\n\n$$\n\\mathcal{O}\\left(\\sqrt{nmd}\\left(\\sqrt{\\log(d)}+\\frac{m^{5/6}}{d^{1/2}}\\right)\\right).\n$$\n\nThe additional factor $\\frac{m^{5/6}}{d^{1/2}}$ is negligible when $d \\gg m$.\n\nIn the **stochastic setting**, the **logarithmic regret guarantee still holds**, though the additive term becomes slightly larger:\n\n$$\\mathcal{O}\\left(\\sum_{i,\\Delta_i>0}\\frac{\\log(n)}{\\Delta_i}\\right)+\\mathcal{O}\\left(\\frac{1}{\\Delta}(m^2d\\log(d)+m^{11/3}+md^2)\\right)$$\n\nTherefore, our results **still demonstrate that the FTPL algorithm approaches the BOBW guarantee**.\n\n## Sorry for the inconvenience. Due to space limitations, we leave the proof of the revised Lemma 4.2 in our response to Reviewer 1er3.\nSince the derivation of the modified regret bound based on this lemma does not differ significantly from the corresponding parts in the original version of the paper, we do not provide the full derivation here. However, we are happy to include it in a follow-up response if necessary.\n\n\n**Weakness 2**:\nWe appreciate your interest in the lower bound part of our Lemma C.5. We acknowledge that the current proof of the lower bound may be difficult to follow. \n\n\nAn intuitive understanding of this result is that the pdf of $W$, given by $1 - F(x(w) + \\mu)$, is monotonically decreasing on $[0,1]$ and clearly exhibits a sharp drop at $e^{-\\frac{K}{\\mu^2}}$: the pdf is equal to $1$ before this point and then gradually decays afterward. Our calculation shows that most of the density is concentrated in the first half of the interval, and thus the expectation of $(-\\log(W))^{1/2}$ is of a larger order than simply substituting $W = e^{-\\frac{K}{\\mu^2}}$. Thus, by choosing $\\mu = -\\sqrt{\\frac{2K}{\\log(K)}}$, the desired lower bound follows.\n\n\nBesides, we would like to emphasize that, due to the appearance of the $\\frac{m^{5/6}}{d^{1/2}}$ factor in our new results, we have established a corresponding updated lower bound. This shows that the new Lemma 4.2 is tight, and therefore, the extra factors beyond the optimal rate in the adversarial setting are unavoidable under our approach. To achieve the tightest possible rate, one would have to resort to alternative methods. \n\n>Lemma C.7: For all $\\mu\\in R$, $M, K\\ge 1$, and $ N\\ge 3$, define\n    $$\n    R_{N}(\\mu)=\\int_0^{+\\infty} x^{-N}e^{-\\frac{K}{x^2}}\\left(1-F(x+\\mu)\\right)^M dx.\n    $$\n    Then there exists $C>0$ such that for all $M\\ge 2K$, we have\n    $$\n    \\sup_{\\mu\\in R}\\frac{R_{4}(\\mu)}{R_3(\\mu)}\\ge C\\left(\\frac{M}{K}\\right)^{1/3}.\n    $$ \n\n\nThis new lower bound is originated from the following simple observation: if a random variable $X$ supported on $[0,1]$ has a probability density function $f$, and there exists $x_0 \\in (0,1]$ such that $f$ is non-decreasing on $[0,x_0]$, then it holds that $E[X] \\geq x_0/4$. This follows from the fact that $P(0 \\leq X \\leq x_0/2) \\leq P(x_0/2 \\leq X \\leq x_0)$, so $P(0 \\leq X \\leq x_0/2) \\leq 1/2$, which implies $E[X] \\geq x_0/4$.\n\n>Proof for Lemma C.7: Let $u = \\frac{1}{x}$. By the change of variable in the integral, we obtain\n$$\nR_N(\\mu) = \\int_0^{+\\infty} u^{N-2} e^{-K u^2}  \\left(1 - F\\left(\\frac{1}{u} + \\mu \\right)\\right)^M  du.\n$$\nWe now consider a random variable $U$ with probability density function $g(u)$ defined as\n$$\ng(u) = \\frac{1}{\\Lambda} u^{N-2} e^{-K u^2}\\left(1 - F\\left(\\frac{1}{u} + \\mu \\right)\\right)^M, \\quad u \\geq 0,\n$$\nwhere $\\Lambda$ is the normalization constant. With this, we have\n$$\n\\frac{R_4(\\mu)}{R_3(\\mu)} = E[U].\n$$\nLet $\\ell(u)=\\log g(u)$, then our plan is to show that that exists $\\mu\\in R$ and corresponding $C>0$ such that when $0\\leq u\\leq u_0:=C\\left(\\frac{M}{K}\\right)^{1/3}$, $\\frac{d \\ell}{d u}(u)\\ge 0$.\nTo this end, let $\\mu=1>0$ and $y=\\frac{1}{u}+1$, then$$\n        \\frac{d\\ell}{du}(u)=\\frac{1}{u}-2Ku+\\frac{2M}{u^2y^3\\left(e^{\\frac{1}{y^2}}-1\\right)}=\\frac{1}{u}+2Ku\\left(\n        \\frac{M/K}{u^3y^3\\left(e^{\\frac{1}{y^2}}-1\\right)}-1\n        \\right).\n        $$\n        Since $1\\leq y$, when $u\\leq \\left(\\frac{M}{K}\\right)^{\\frac13}(e-1)^{-\\frac13}-1$, we have\n        $$\n        \\frac{M/K}{u^3y^3\\left(e^{\\frac{1}{y^2}}-1\\right)}=\\frac{M/K}{(u+1)^3\\left(e^{\\frac{1}{y^2}}-1\\right)}\\ge\\frac{M/K}{(u+1)^3\\left(e-1\\right)}\\ge 1.\n        $$\n        Take $C=(e-1)^{-\\frac13}-2^{-\\frac13}>0$ and then one can see that when $M\\ge 2K$, $u_0:=C\\left(\\frac{M}{K}\\right)^{1/3}\\leq \\left(\\frac{M}{K}\\right)^{\\frac13}(e-1)^{-\\frac13}-1$. Then when $0\\leq u\\leq u_0:=C\\left(\\frac{M}{K}\\right)^{1/3}$, $\\frac{d\\ell}{du}(u)\\ge 0$.\n\n\n**Question**:\nWe appreciate your suggestion and will include this clarification. This choice is motivated by a sequence of prior studies on the FTPL algorithm in the multi-armed bandit (MAB) setting. \n\nBriefly, according to the decomposition in our Lemma 3.3, consider $X_1, \\ldots, X_d$ as i.i.d. random variables drawn from the perturbation distribution $\\mathcal{D}$. With a properly chosen learning rate, the penalty term corresponds to the expected sum of the top $m$ values among $\\sqrt{n} X_1, \\ldots, \\sqrt{n} X_d$. \n\nNoting that the optimal regret bound in the adversarial setting is $\\sqrt{nmd}$, the intuition is that we want to choose a distribution $\\mathcal{D}$ such that the expectation of the sum of the top $m$ among $X_1, \\ldots, X_d$ scales as $\\sqrt{dm}$. \nAccording to our Lemma E.2, the Fréchet distribution with shape parameter $2$ satisfies this condition exactly. As for how we discovered that the Fréchet distribution satisfies such a condition, one can refer to classic results from extreme value theory."}, {"review_id": "gR7jAKOcQA", "reviewer": "Reviewer_S1FU", "summary": "This paper studies the BoBW semi-bandit problem. Utilizing and refining recent technical advancements in FTPL with Frechet-type perturbations, this paper derived an FTPL-based algorithm achiving the $\\tilde{\\mathcal O}(\\sqrt{nmd})$ adversarial regret and $\\mathcal O(\\sum_i \\log n \\Delta_i^{-1})$ stochastic regret. While the regret is slightly sub-optimal in terms of logarithmic factors, FTPL avoids the hard-to-solve global optimization problem common in FTRL.", "questions": "See weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. The elegance and efficiency of FTPL is for sure a plus.\n2. New analytical tools for FTRL is proposed (i.e., it's not a simple adaptation of recent BoBW FTPL results to semibandits).\n3. Numerical illustrations are given.\n\nWeakness:\n\nThe paper is written in a super technical way and lacks some justifications even for experts in this area. For example, around Line 169 the claim \"which is a stronger result and simplifies the proof compared to those of Honda et al. [2023], Lee et al. [2024],\" is not well-elaborated -- while I can get the intuition why this might be simpler as an expert in FTRL, I am not sure whether my understanding is correct.\n\nThe notations are also too many to be easily interpreted.\n\nAlso, the authors are also strongly encouraged to explain the technical difficulty / insights in proving these results: What leads to this stronger result? Is it because of the FTRL-for-FTPL style analysis? If yes, what do the previous works of Honda et al. and Lee et al. use, and what stopped them from using a similar idea? (Note that I do not mean something like \"Line 169 is simply due to an existing observation\", but rather suggest the authors to be clear about their contributions.)\n\nOverall, I'm leaning towards an accept because of the good results provided as well as the new contribution to the somehow-scarce FTPL technical toolbox; nevertheless, as I always worked on FTRL instead of FTPL, I'm not super certain how significant this contribution is.", "was_revised": true, "final_justification": "I am satisfied with Authors' responses and decide to keep my original evaluation.", "rebuttal": "We thank you for your review and suggestions. Regarding the question about Lemma 3.3, we provide the following clarifications.\n\nFirst, our result is stronger in the sense that, compared to previous work by Honda and Lee et al., we obtain an almost identical penalty term, but with a simpler stability term. Specifically, their stability term takes the form\n\n$$\nE\\left[\\sum\\_{t=1}^{n}\\langle\\hat{\\ell}\\_t,\\phi(\\eta_t \\hat{L}\\_t)-\\phi(\\eta\\_{t+1} \\hat{L}\\_{t+1})\\rangle\\right],\n$$\n\nwhich they further decompose into two parts:\n$$\nE\\left[\\sum_{t=1}^n \n    \\langle\\hat{\\ell}\\_t,\\phi(\\eta\\_t \\hat{L}\\_t)-\\phi(\\eta\\_{t} \\hat{L}\\_{t+1})\\rangle\\right] \\quad \n    \\text{and}\n    \\quad\nE\\left[\\sum_{t=1}^n \n    \\langle\\hat{\\ell}\\_t,\\phi(\\eta\\_{t} \\hat{L}\\_{t+1})-\\phi(\\eta\\_{t+1} \\hat{L}\\_{t+1})\\rangle\\right],\n$$\nand control each part separately. However, under our decomposition, the second term is not needed, simplifying the analysis.\n\nSecond, we believe this stronger result indeed stems from adopting an FTRL-for-FTPL-style analytical framework. The Bregman-divergence-based argument is more principled, allowing us to apply the Generalized Pythagoras Identity (Lemma G.2). In contrast, the decomposition in Honda and Lee’s work is more ad hoc, and directly based on the update form \n$$\nA_t = \\operatorname{argmin}_{a \\in \\mathcal{A}} \\langle \\hat{L}_t - r_t / \\eta_t, a \\rangle.\n$$\n\nWe conjecture there are two main reasons why they did not use the FTRL-for-FTPL framework:  \n(1) The regularization function corresponding to FTPL's dual view does not have a closed-form expression, making it hard to analyze;  \n(2) Their decomposition is based on the update rule of $A_t$ (i.e., the maximizer of an inner product), which naturally leads to a decomposition where the stability term also takes the form of an inner product. This inner-product form is more convenient for analysis in the context of FTPL algorithms.\n\nAs mentioned in Remark 3.1, unlike standard FTRL analyses where the stability term is often approximated by a sum of the form $\\eta\\_t E[\\|\\hat{\\ell}\\_t\\|^2\\_{\\nabla^2 \\Phi(-\\eta\\_t \\hat{L}\\_t)}]$, this approach does not extend easily to FTPL. This is because $\\nabla^2 \\Phi(-\\eta\\_t \\hat{L}\\_t)$ and $\\nabla^2 \\Phi(-\\eta\\_t \\hat{L}\\_{t+1})$ may differ significantly (we can only control their diagonal entries) — in FTPL, $\\nabla^2 \\Phi$ is not a diagonal matrix, which is a key difference from typical FTRL algorithms. \n\nIn contrast, the inner-product form of the stability term $\\langle \\hat{\\ell}\\_t, \\phi(\\eta\\_t \\hat{L}\\_t) - \\phi(\\eta\\_t \\hat{L}\\_{t+1}) \\rangle$ can be written as \n$$\n\\sum\\_{i=1}^d \\hat{\\ell}\\_{t,i} \\left( \\phi\\_i(\\eta\\_t \\hat{L}\\_t) - \\phi_i(\\eta\\_t \\hat{L}\\_{t+1}) \\right),\n$$\nwhich, according to Lemma D.2, can be upper bounded by \n$$\n\\sum\\_{i=1}^d \\hat{\\ell}\\_{t,i} \\left( \\phi\\_i(\\eta\\_t \\hat{L}\\_t) - \\phi\\_i\\left(\\eta\\_t \\left(\\hat{L}\\_t + \\hat{\\ell}\\_{t,i} \\cdot e_i\\right)\\right) \\right).\n$$\nIn this way, we only need to consider the diagonal entries of $\\nabla^2 \\Phi$.\n\nHowever, in any case, we find that a similar and even stronger form of the stability term can also be easily derived using the FTRL-based analysis framework, which further demonstrates the superiority of this framework."}, {"review_id": "jKgxQ0QenG", "reviewer": "Reviewer_JPYF", "summary": "This paper introduces and analyzes the Follow-the-Perturbed-Leader (FTPL) policy with Frechet perturbations for the m-set semi-bandit problem, which aims overcome the computational complexities of the traditional Follow-the-Regularized-Leader (FTRL) policies. The main contribution is achieving Best-of-Both-Worlds (BOBW) regret guarantees, making it the first FTPL algorithm to approach this performance in the m-set semi-bandit setting when $m\\leq d/2$. Specifically, it demonstrates a near-optimal regret in the adversarial setting and logarithmic regret in the stochastic setting. The authors also establish a lower bound and indicate that the logarithmic factor $(log(d))$ in the adversarial regret is inherent to their analytical approach, and eliminating it would require a fundamentally different method. Furthermore, the work advances theoretical analysis by extending the standard FTRL framework to m-set semi-bandits, a setting where the convex hull of the action set lacks interior points, thereby simplifying previous proofs. The effectiveness of the proposed FTPL algorithm is also supported by empirical validation against various established baselines.", "questions": "1) Could you provide a more in-depth discussion of the specific technical challenges that prevent the removal of the $log(d)$ factor in the adversarial regret bound? It is only noted that obtaining a tighter bound is not possible through bounding the term-wise ratio; and instead, one must analyze the ratio of the full summations directly, which is substantially more challenging. A clearer understanding of this analytical hurdle would be valuable.\n\n2) What are the specific hurdles encountered when attempting to extend the FTPL's BOBW guarantee to the $m > d/2$ regime. It is indicated that FTRL algorithms have achieved BOBW guarantees in this setting, but whether FTPL can match this remains an open problem.\n\n3) What are the implications if there are more than $m$ arms with zero suboptimality gap? How would the analytical framework (e.g., the regret decomposition for the stochastic setting) break down or degrade if this assumption were relaxed?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n\n1) The FTPL policy achieves optimal BOBW regret in both adversarial and stochastic settings, which is an advantage for uncertain environments.\n\n2) FTPL is computationally efficient as it directly selects $m$ arms with smallest perturbed losses, avoiding complex arm-selection probability computations required by FTRL. It uses geometric resampling for unbiased estimators.\n\n3) The paper extends the standard FTRL analysis framework to m-set semi-bandits where the action set's convex hull lacks interior points, simplifying prior proofs by Honda et al.. It also provides a lower bound showing that the log(d) factor in adversarial regret is inherent to their approach.\n\n4) The paper provides detailed empirical validation by comparing FTPL against five baselines (COMBUCB, Thompson Sampling, EXP2, LogBarrier, and a hybrid FTRL) across both stochastic and adversarial environments, with specified parameters, loss generation, and repetitions.\n\n# Weaknesses\n\n1) The theoretical analysis for the stochastic setting (Theorem 3.2) assumes the uniqueness of the optimal action, which might not hold universally in practice.\n\n2) While its application seems novel, FTPL itself is an existing algorithm. The contribution is an extension and refinement of prior FTRL/FTPL analysis, rather than a new algorithmic approach.\n\n3) The FTPL method's adversarial regret includes an unavoidable $log(d)$ factor, preventing it from strictly matching the minimax optimal bound.\n\n4) The BOBW guarantees are only applicable when $m \\leq d/2$. Extending these results to the $m > d/2$ regime remains an open problem.", "was_revised": true, "final_justification": "I am keeping my score, which remains positive.\n\nMy concern regarding the unavoidable log(d) factor in the adversarial regret bound has been effectively addressed. The strengthening of Lemmas and related lower bounds clearly demonstrate that this factor is inherent to your current analytical approach. The discussions confirmed the complexity and rigor of your theoretical analysis for FTPL in the m-set semi-bandit setting, especially concerning the arm-selection probabilities and extending the FTRL framework where the convex hull of the action set lacks interior points. The clarified regret decomposition also enhanced confidence in the proof's integrity. The comprehensive empirical evaluation remains a strong aspect, supporting the practical efficacy of your proposed method across stochastic and adversarial settings.\n\nThe assumption of a unique optimal action in the stochastic setting persists. While your rebuttal suggests FTPL can achieve logarithmic regret without this assumption, it remains an unproven conjecture within the paper and is deferred as future work. The theoretical guarantee in the paper still relies on this specific assumption. The paper's Best-of-Both-Worlds (BOBW) guarantee currently applies when m ≤ d/2. Extending FTPL to the m > d/2 regime remains an acknowledged open problem, noted as an intriguing and challenging future direction.\n\nMy decision is primarily driven by the theoretical contribution of presenting the first FTPL algorithm with Best-of-Both-Worlds guarantees in the m-set semi-bandit setting", "rebuttal": "Thank you for your valuable review and constructive suggestions. Regarding the weaknesses and questions, we provide the following detailed responses:\n\n**Weakness 1**: Though the uniqueness of the optimal action is a common assumption in BOBW problems  [Zimmert and Seldin, 2019, Zimmert et al., 2019, Honda et al., 2023], we would like to emphasize that this is primarily for technical convenience (see our response to Question 3 for further discussion). Intuitively, the case where the optimal action is not unique is actually easier for the learner. In the MAB setting, some classical BOBW algorithms based on FTRL have already been shown to achieve logarithmic regret even without the uniqueness assumption, though such results require more delicate analysis (e.g., Parameter-Free Multi-Armed Bandit Algorithms with Hybrid Data-Dependent Regret Bounds, Shinji Ito; Improved Best-of-Both-Worlds Guarantees for Multi-Armed Bandits: FTRL with General Regularizers and Multiple Optimal Arms, Tiancheng Jin, Junyan Liu, and Haipeng Luo).\n\nOur additional experiments (We're sorry, but due to the new NeurIPS policy, we are temporarily unable to upload images.) show that even when the optimal action is not unique, FTPL still achieves logarithmic regret in the stochastic setting. This supports our conjecture that the assumption is not essential. We view this as a promising direction for future work and will include a discussion in the revised version. Thank you for bringing this up.\n\n**Weakness 2**: We acknowledge that our work does not introduce algorithmic innovations. Our main contribution lies in the theoretical analysis of the FTPL algorithm in the semi-bandit setting, which is technically challenging. Specifically:\n\n1. Our analysis of the ratio $\\frac{V_{i,4}}{V_{i,3}}$—where both the numerator and denominator involve complex integrals and summations—is nontrivial. By carefully analyzing the ratio of corresponding terms in the numerator and denominator (including both upper and lower bounds), we obtain a nontrivial result for the first time, and further demonstrate the theoretical limitations of this approach.\n\n2. We would also like to emphasize that our result in the stochastic setting should not be overlooked — it is not a straightforward extension (see Lemma 4.5 for details).\n\n\n**Weakness 3**: We demonstrate that this is difficult to avoid within our proof framework by establishing a lower bound. Any further improvement would likely require adopting a different and more challenging line of analysis.\n\n\n\n**Weakness 4**: We believe it is likely that, without modifying the algorithm, FTPL with a Fréchet distribution of shape parameter 2 cannot achieve the optimal performance in the adversarial setting when $m \\ge d/2$, and therefore fails to attain the BOBW guarantee. Please refer to our response to Question 2 for details.\n\n\n**Question 1**:\nWe provide a detailed explanation of this issue. According to our analysis in Section 4.1, the stability term in round $t$ can be bounded by\n$$\n\\eta_t \\sum_{i=1}^d \\frac{V_{i,4}(\\lambda)}{V_{i,3}(\\lambda)}.\n$$\nThe **main challenge** lies in the fact that $V_{i,N}(\\lambda) = \\sum_{I,\\ |I|<m,\\ i \\notin I} V_{i,N}^{I}(\\lambda)$ is a sum over a collection of complicated integrals, which makes it difficult to directly compute $\\frac{V_{i,4}(\\lambda)}{V_{i,3}(\\lambda)}$. A natural idea is to use the upper bound:\n$$\n\\frac{V_{i,4}(\\lambda)}{V_{i,3}(\\lambda)} \\leq \\sup_{I,\\ |I|<m,\\ i \\notin I} \\frac{V_{i,4}^{I}(\\lambda)}{V_{i,3}^{I}(\\lambda)}.\n$$\nHowever, our lower bound shows that when $\\sigma_i(\\hat{L}_t) > 2m$, we have\n$$\n\\sup\\_{\\lambda} \\sup\\_{I,\\ |I|<m,\\ i \\notin I} \\frac{V\\_{i,4}^{I}(\\lambda)}{V\\_{i,3}^{I}(\\lambda)} \\ge \\sqrt{\\frac{m \\log(d)}{\\sigma\\_i(\\hat{L}\\_t) - m}}.\n$$\nThis implies that, under this approach, the upper bound of the stability term is at least\n$$\n\\eta_t\\sum_{i:\\ \\sigma_i(\\hat{L}_t) > 2m} \\sqrt{\\frac{m \\log(d)}{\\sigma_i(\\hat{L}_t) - m}} = \\Omega\\left(\\sqrt{\\frac{m d \\log(d)}{t}}\\right).\n$$\n\n**Question 2**: According to the result of \"Beating Stochastic and Adversarial Semi-bandits Optimally and Simultaneously\" by Julian Zimmert, Haipeng Luo, and Chen-Yu Wei (Theorem 3), when $ m > d/2 $, the optimal regret bound in the adversarial setting is \n$$\nO\\left((d - m)\\sqrt{\\log\\left(\\frac{d}{d - m} \\right)n}\\right).\n$$\nHowever, if we choose the Fréchet distribution with shape parameter 2 as the perturbation distribution, the resulting penalty term remains \n$$\nO(\\sqrt{nmd}),\n$$\n**regardless of the relationship between $ m $ and $ d $.** This stems from a simple fact: consider $ X_1, \\ldots, X_d $ as i.i.d. random variables drawn from the Fréchet distribution with shape parameter 2. Then, the expectation of the sum of the top $ m $ among $ X_1, \\ldots, X_d $ scales as $ \\sqrt{dm} $. \n\nHowever, this is too large when compared to $(d - m)\\sqrt{\\log\\left(\\frac{d}{d - m} \\right)}$ in the optimal bound in the case where $ m > d/2 $ (when $m=d-1$, for example). Therefore, we believe that, without modifying the algorithm, FTPL with a Fréchet distribution (shape parameter 2) is unlikely to achieve the optimal regret in the adversarial setting when $ m \\ge d/2 $.\n\nIt is important to emphasize that the algorithm in \"Beating Stochastic and Adversarial Semi-bandits Optimally and Simultaneously\" that achieves the BOBW guarantee even when $ m > d/2 $ is based on FTRL with a hybrid regularizer. The hybrid regularizer used there is relatively complex and thus can adapt to various values of $ m $ and $ d $. To modify FTPL to handle more general cases, it may similarly require more sophisticated perturbation distributions.\n\n\n**Question 3**:\nThe difficulty of the case where the optimal action is not unique lies in the following aspects:\n\n1. The lower bound used in the self-bounding constraint technique,\n$$\nReg_n \\ge \\Delta \\sum_{t=1}^n E\\left[\n\\sum_{i=m+1}^d w_{t,i}\n\\right],\n$$\nno longer holds. This is because, in this case, the $m$-th and $(m+1)$-th arms may have the same expected loss, and both can potentially appear in some optimal action sets. As a result, selecting the $(m+1)$-th arm does not necessarily incur at least $\\Delta$ regret. Therefore, a more refined characterization of the lower bound is required, which becomes significantly more cumbersome.\n\n2. Once a new lower bound is obtained, we must correspondingly strengthen Lemma 4.5 to match the upper and lower bounds. This requires more sophisticated analytical techniques from FTRL algorithms (see \"Parameter-Free Multi-Armed Bandit Algorithms with Hybrid Data-Dependent Regret Bounds\", Shinji Ito; and \"Improved Best-of-Both-Worlds Guarantees for Multi-Armed Bandits: FTRL with General Regularizers and Multiple Optimal Arms\", Tiancheng Jin, Junyan Liu, and Haipeng Luo)."}, {"review_id": "4HXArKI7XP", "reviewer": "Reviewer_1er3", "summary": "This paper studies the best-of-both-worlds (BOBW) achievability of the Follow-the-Perturbed-Leader (FTPL) algorithm in the combinatorial semi-bandit setting. The authors demonstrate that FTPL achieves near-optimal regret in adversarial environments and logarithmic regret in stochastic settings. While similar results have been established for FTRL-based approaches, this is the first result of its kind for FTPL. This contribution is particularly valuable, as FTPL does not require solving complex optimization problems, which can be computationally expensive in combinatorial settings.", "questions": "Regarding the weaknesses, I would like to point out a recent note by Chen and Honda (2025), which argues that the current proof of Lemma 4.1 is incomplete and contains a technical flaw. Furthermore, their numerical validation suggests that the current argument may not hold as intended. I am curious whether the authors believe the proof can be modified or extended to rigorously establish the desired $\\sqrt{nmd\\log d}$ regret bound.\n\nI would also like to emphasize that I fully acknowledge both works were developed independently, and the positive results presented by Chen and Honda have no influence on my overall evaluation. My assessment is based entirely on the concerns about the proof of Lemma 4.1, which plays a critical role in bounding the stability term and supporting the main theoretical results.\n\n---\nChen, Botao, and Junya Honda. \"Note on Follow-the-Perturbed-Leader in Combinatorial Semi-Bandit Problems.\" arXiv preprint arXiv:2506.12490 (2025).", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Strength\n\nThis paper presents the first BOBW result for FTPL in combinatorial settings, along with improved adversarial regret bounds. Specifically, the authors achieve a regret of $\\sqrt{nmd\\log d}$ improving upon the previous bound of $m\\sqrt{nd\\log(d/m)}$ established by Neu and Bartók (2016). \nAlthough several FTRL-based approaches have achieved optimal adversarial regret (without logarithmic dependencies), they typically require solving complex optimization problems to compute arm-selection probabilities at each round. In combinatorial settings, these optimization steps become significantly more computationally demanding compared to standard multi-armed bandit problems. In this regard, achieving BOBW guarantees with FTPL, an algorithm that avoids solving such optimization problems, is particularly appealing and makes the contribution of this paper especially valuable.\n\n---\n\n### Weakness\n\nIn the current analysis of the stability term, Lemma 4.1 plays a key role, establishing the foundation for Lemmas 4.2, 4.3, and 4.4, which collectively address the main analytical challenges in studying FTPL. While the paper provides a detailed proof showing that $J_{i,N+k,I}/J_{i,N,I}$ is increasing with respect to $\\lambda_q \\geq 0$ for $q\\not\\in I$, the corresponding decreasing result for $q\\in I$ is not provided. The authors simply state that similar arguments would suffice, but I believe this step is not straightforward and may not follow from analogous reasoning.\n\nThis is because, as noted by the authors in Line 193, in the multi-armed bandit setting, the corresponding terms do not involve $1-F$ factors.\nThis structural difference becomes crucial when considering the derivative with respect to $\\lambda_q$ for $q\\in I$, where the presence of $1-F$ terms introduces additional dependencies that were not present in the multi-armed case and $q\\not\\in I$ case. \nTHerefore, the decreasing property for $q\\in I$ may require a more careful, explicit argument rather than a direct extension of the existing proof.\n\nTo be more precise, fix $s \\in I$.\nThen, the partial derivatives of $J_{i,N,I}$ with respect to $\\lambda_s$ is\n$$ \nJ_{i,N,I}^{s}(\\lambda) := -\\frac{1}{2}\\frac{\\partial J_{i,N,I}(\\lambda)}{\\partial \\lambda_s} =\\int_0^\\infty \\frac{e^{-1/(z+\\lambda_s)^2}}{(z+\\lambda_i)^N(z+\\lambda_s)^3} \\prod_{q \\in I \\setminus \\\\{s\\\\}}(1-F(z+\\lambda_q)) \\prod_{q \\not\\int I} F(z+\\lambda_q)  dz.\n$$\nThen, similarly to Line 499,\n$$\n\\frac{\\partial}{\\partial \\lambda_s} \\frac{J_{i,N+k,I}(\\lambda)}{J_{i,N,I}(\\lambda)} = -2 \\cdot \\frac{J_{i,N+k,I}^s(\\lambda)J_{i,N,I}(\\lambda)-J_{i,N+k,I}(\\lambda)J_{i,N,I}^s(\\lambda)}{J_{i,N,I}(\\lambda)^2}.\n$$\nWhile the exponential term can be absorbed into the $Q$ term (as defined in Line 496) when $s\\not\\in I$, since $F$ is expressed in terms of those exponential terms, the same absorption is not possible under the current definition of $Q$ when $s\\in I$.\nIn this case, the exponential term must be explicitly included in the product involving $1-F$.\nTherefore, it is necessary to carefully reconsider the analysis for the $s\\in I$ case by defining\n$$\nQ(z)=\\frac{1}{(z+\\lambda_i)^N}\\prod_{q \\in I \\setminus \\\\{s\\\\}}(1-F(z+\\lambda_q)) \\prod_{q \\not\\in I} F(z+\\lambda_q).\n$$\nThen, if we follow the same steps, we need to consider the sign of\n$$\n\\frac{e^{-1/(z+\\lambda_s)^2}}{(z+\\lambda_s)^3(1-e^{-1/(z+\\lambda_s)^2})}-\\frac{e^{-1/(w+\\lambda_s)^2}}{(w+\\lambda_s)^3(1-e^{-1/(w+\\lambda_s)^2})}.\n$$\nDeriving such results is not straightforward, and therefore the authors should provide detailed proofs to support these claims.", "was_revised": true, "final_justification": "During the rebuttal, the authors provided a revised analysis to address the issue in Lemma 4.1.\nAt least for the new Lemma 4.2, the analysis appears correct, and the updated regret becomes\n$$ O(\\sqrt{nmd\\log d} + m^{4/3}\\sqrt{n}). $$\nTherefore, when $(d \\log d)^{\\frac{3}{5}} \\leq m \\leq d/2$, the adversarial regret becomes worse than expected. However, as the authors noted, it cannot be improved within the current proof framework.\n\nNevertheless, this work presents the first BOBW guarantee for FTPL under the combinatorial semi-bandit setting, where the adversarial regret is at least an improvement over the previous FTPL bound of $O(m\\sqrt{nd\\log (d/m)})$, while also achieving logarithmic stochastic regret. Although I expect that the adversarial regret can be improved further, doing so would likely require a different type of analysis or even a different choice of perturbations, which would be beyond the scope of this paper.\n\nAlthough the current rebuttal policy makes it difficult to fully verify every detail, I think the modified proof is convincing. Therefore, I will raise my score to 4. Although I cannot change my confidence score, while lowering my confidence to 3.", "rebuttal": "Thank you for your valuable review and constructive suggestions. Regarding the weaknesses and questions, we provide the following detailed responses:\n\n**Weakness**:\nWe must acknowledge our oversight: the second half of **Lemma 4.1** was indeed incorrect. To address this issue, we have **strengthened Lemma 4.2** by **removing the assumption $\\mu_i \\leq \\mu$**, which eliminates the need for the monotonicity argument used in the original Lemma 4.1.\n\n>Lemma 4.2 (**modified**):  For all $\\mu\\ge0$, $K\\ge 1$, $ N\\ge 3$ and $M\\ge 1$, let $\\mu_i\\in R$ for all $1\\leq i\\leq M$, and define\n    $$\n    H_{N}=\\int_0^{+\\infty}(x+\\mu)^{-N}e^{-\\frac{K}{(x+\\mu)^2}}\\prod_{i=1}^M\\left(1-F(x+\\mu_i)\\right) d x.\n    $$\n    For all $k\\in\\mathcal{N}^+$, we have\n    $$\n    \\frac{H_{N+k}}{H_N}\\leq C_{N,k}\\left(\\left(\\frac{M}{K}\\right)^{k/3}\\wedge \\mu^{-k}\\right),\n    $$\n    where $C_{N,k}$ is a positive constant only depending on $N$ and $k$. Furthermore, if $K\\ge M$, then we have\n    $$\n    \\frac{H_{4}}{H_3}\\leq C\\left(\\left(\\frac{M}{K}\\log\\left(\\frac{K}{M}+1\\right)\\right)^{1/2}\\wedge \\mu^{-1}\\right),\n    $$\n    where $C$ is a positive constant.\n\n Under the more general condition on $\\mu_i$, the **second part of Lemma 4.2 remains valid (when $K\\ge M$)**, while the **first part becomes weaker**. As a result, the **adversarial regret bound becomes slightly worse** than before:\n\n$$\n\\mathcal{O}\\left(\\sqrt{nmd}\\left(\\sqrt{\\log(d)}+\\frac{m^{5/6}}{d^{1/2}}\\right)\\right).\n$$\n\nThe additional factor $\\frac{m^{5/6}}{d^{1/2}}$ is negligible when $d \\gg m$.\n\nIn the **stochastic setting**, the **logarithmic regret guarantee still holds**, though the additive term becomes slightly larger:\n\n$$\\mathcal{O}\\left(\\sum_{i,\\Delta_i>0}\\frac{\\log(n)}{\\Delta_i}\\right)+\\mathcal{O}\\left(\\frac{1}{\\Delta}(m^2d\\log(d)+m^{11/3}+md^2)\\right)$$\n\nTherefore, our results **still demonstrate that the FTPL algorithm approaches the BOBW guarantee**.\n\nWe will include the complete proof of the new version of Lemma 4.2 at the end of our response. Since the derivation of the modified regret bound based on this lemma does not differ significantly from the corresponding parts in the original version of the paper, we do not provide the full derivation here. However, we are happy to include it in a follow-up response if necessary.\n\nIn addition, we have provided a new lower bound to demonstrate that the newly introduced $\\frac{m^{5/6}}{d^{1/2}}$ term is also unavoidable. Please refer to our response to Reviewer 7BAU (Weakness 2) for further details.\n\n**Question**:\nRegarding the work of Chen, Botao, and Junya Honda that you mentioned, please allow us to reiterate a few points:\n\n1. Our work is still the first to show that FTPL approaches the BOBW guarantee. We would like to emphasize that our result in the stochastic setting should not be overlooked — it is not a straightforward extension (see Lemma 4.5 for details).\n\n2. Although our result in the adversarial setting is slightly weaker, as an early work studying the BOBW problem for FTPL in the semi-bandit setting, we theoretically demonstrate the limitations of current techniques by providing a matching lower bound. This contribution helps clear the path for future progress.\n\n3. Compared to Lemma 4.1, the proof of Lemma 4.2 plays a more critical role in our analysis. Our careful treatment of the ratio $H_{N+k}/H_N$ allows us to avoid relying on the monotonicity condition in the original Lemma 4.1.\n\n>Proof of modified Lemma 4.2: Let $u = \\frac{1}{x + \\mu}$. By the change of variable in the integral, we obtain\n$$\nH_N=\\int_0^{\\mu^{-1}} u^{N-2}e^{-Ku^2}\\prod_{i=1}^M \\left(1-F\\left(\\frac{1}{u}+\\mu_i-\\mu \\right)\\right)du.\n$$\nWe now consider a random variable $U$ with probability density function $g(u)$ defined as\n$$\ng(u) = \\frac{1}{\\Lambda}u^{N-2}e^{-K u^2}\\prod_{i=1}^M\\left(1-F\\left(\\frac{1}{u}+\\mu_i-\\mu \\right)\\right),\\quad u\\geq 0,\n$$\nwhere $\\Lambda$ is the normalization constant. With this, we have\n$$\n\\frac{H_{N+k}}{H_N} = E[U^k].\n$$\n**Our plan** is to find some $u_0 > 0$ such that $g(u)$ **decays rapidly** after $u_0$, then we expect that the order of $E[U^k]$ is roughly $u_0^k$.\n**We first show the first result.** The upper bound of $\\mu^{-k}$ is obvious because $(x+\\mu)^{-N-k}\\leq \\mu^{-k}(x+\\mu)^{-N}$ and hence, in the following we assume that $\\mu^{-1}\\ge C_{N}'\\left(\\frac{M}{K}\\right)^{1/3}$, where $C_{N}'=8\\sqrt{N-2}$.  Let $\\ell(u)=\\log g(u)$ and $y_i=\\frac{1}{u}+\\mu_i-\\mu$ for all $1\\leq i\\leq M$, then when $u\\ge 0$, \n        $$\n        \\frac{d \\ell}{d u}(u)=\\frac{N-2}{u}-2Ku+2\\sum_{i=1}^M\\frac{1_{\\{y_i\\ge 0\\}}}{u^2y_i^3\\left(e^{\\frac{1}{y_i^2}}-1\\right)}.\n        $$\nThen we will show that when $u\\ge u_0:=C_{N}'\\left(\\frac{M}{K}\\right)^{1/3}/2$, $\\frac{d \\ell}{d u}(u)\\leq -Ku/2$. Intuitively, this suggests that for $u \\ge u_0$, $U$ exhibits a Gaussian tail. Then, by applying Lemma G.10 (which we include at the end), we can conclude that\n$$\nE[U^k] \\leq u_0^k + E[U^k 1_{\\{\\mu^{-1} \\ge U \\ge u_0\\}}] \\leq (1 + 2k!!) u_0^k.\n$$\nTo show that $\\frac{d \\ell}{d u}(u)\\leq -Ku/2$, it suffices to note that $\\sup_{x\\ge0}\\frac{1}{x^3\\left(e^{\\frac{1}{x^2}}-1\\right)}<1$, then we have\n        $$\n        \\frac{d\\ell}{du}(u)\\leq\\frac{N-2}{u}-2Ku+\\frac{2M}{u^2}.\n        $$\n        When $u\\ge u_0$, clearly, since $C_N$ is large enough, we have\n        $$\n        \\frac{2M}{u^2}\\leq \\frac{Ku}{2}.\n        $$\n        Also, since $u_0>\\sqrt{\\frac{N-2}{K}}$, when $u\\ge u_0$, we have\n        $\\frac{N-2}{u}\\leq Ku.$\nTherefore, when $u\\ge u_0$, $$\\frac{d\\ell}{du}(u)\\leq -Ku/2.$$ **For the second part ($N=3$ and $k=1$)**, let $$u_0=\\left(\\frac{M}{K}\\log\\left(\\frac{K}{M}+1\\right)\\right)^{1/2}.$$ When $K\\leq 32M$, by the first part, the result holds clearly when $C$ is large enough. Hence we then assume that $K>32M$. Then $u_0<1/3$. Similarly, in the following, we also assume that $\\mu^{-1}\\ge C'u_0$, where $C'>3$ is a large constant to be chosen and is not depending on $K$ and $M$.\nFor all $s\\ge t\\ge 1$, we have\n        $$\n        \\frac{g(su_0)}{g(tu_0)}=\\frac{s}{t}\\mathrm{e}^{-Ku_0^2(s^2-t^2)}\\prod_{i=1}^M\\frac{1-F\\left(\\frac{1}{su_0}+\\mu_i-\\mu\\right)}{1-F\\left(\\frac{1}{tu_0}+\\mu_i-\\mu\\right)}.\n        $$\n        If $1\\leq t\\leq 3$, then $tu_0<1$. Hence, by Lemma G.7 (which we include at the end), we have$$\n            \\frac{g(su_0)}{g(tu_0)}\\leq \\frac{s}{t}\\mathrm{e}^{-Ku_0^2(s^2-t^2)}\\left(\\frac{8}{t^2u_0^2}\\right)^M\\leq s\\mathrm{e}^{-Ku_0^2(s^2-t^2)}\\left(\\frac{8}{u_0^2}\\right)^M\n $$\n        Then on the one hand, for all $t\\in[1,2]$, we have\n        $$\n         \\frac{g(3u_0)}{g(tu_0)}\\leq 3\\mathrm{e}^{-5Ku_0^2}\\left(\\frac{8}{u_0^2}\\right)^M\\leq 3\\mathrm{e}^{-5M\\log\\left(\\frac{K}{M}+1\\right)}\\left(\\frac{8K}{M}\\right)^M<1,\n        $$\n        where we used the definition of $u_0$ in the second inequality and $\\frac{K}{M}>8$ in the last inequality.\n        Then since $C'\\ge 2$,\n        $$\n        1\\ge \\int_{u_0}^{\\mu^{-1}}g(u) du\\ge \\int_{u_0}^{2u_0}g(u) du\\ge u_0 g(3u_0),\n        $$\n        which implies that $g(3u_0)\\leq u_0^{-1}.$\nOn the other hand, for all $s\\ge 3$, we have\n        $$\n        \\frac{g(su_0)}{g(3u_0)}\\leq s\\mathrm{e}^{-Ku_0^2(s^2-9)}\\left(\\frac{8}{u_0^2}\\right)^M\\leq\\mathrm{e}^{\\log (s)-M\\log\\left(\\frac{K}{M}+1\\right)(s^2-9)+M\\log(8)+M\\log\\left(\\frac{K}{M}\\right)}.\n        $$\n        Then since $K>32M\\ge 32$, one can find $C''>3$ that is not depending on $K$ and $M$ and large enough (one can then pick an $C'$ larger than $C''$) such that for all $s\\ge C''$, we have\n        $$\n        \\frac{g(su_0)}{g(3u_0)}\\leq\\mathrm{e}^{-\\frac{M}{2}\\log\\left(\\frac{K}{M}+1\\right)s^2}.\n        $$\n        Therefore,\n        $$\n\\int_{C''}^{\\frac{\\mu^{-1}}{u_0}}s g(su_0)ds\\leq g(3u_0)\\int_{C''}^{+\\infty}s\\mathrm{e}^{-M\\log\\left(\\frac{K}{M}+1\\right)s^2/2} ds\\leq u_0^{-1}\\int_{C''}^{+\\infty}s\\mathrm{e}^{-M\\log\\left(\\frac{K}{M}+1\\right)s^2/2} ds,\n        $$\n        which equals \n        $$\nu_0^{-1}\\left(M\\log\\left(\\frac{K}{M}+1\\right)\\right)^{-1}\\mathrm{e}^{-C''^2M\\log\\left(\\frac{K}{M}+1\\right)/2}\\leq u_0^{-1}.\n        $$\n        Then we have\n        $$\n            E[U]\\leq C''u_0+\\int_{C''u_0}^{\\mu^{-1}} ug(u)du=C''u_0+u_0^2\\int_{C''}^{\\frac{\\mu^{-1}}{u_0}} sg(su_0)ds\\leq (C''+1)u_0.\n        $$\n\nFinally, we include the auxiliary lemmas:\n>Lemma G.7: For all $\\mu\\in R$ and $y\\ge x>0$, if $y\\ge 1$, then we have\n    $$\n    \\frac{1-F(x+\\mu)}{1-F(y+\\mu)}\\leq 8y^2.\n    $$\n> Proof: If $x+\\mu\\leq y$, then\n        $$\n        \\frac{1-F(x+\\mu)}{1-F(y+\\mu)}\\leq\\frac{1}{1-F(y+\\mu)}.\n        $$\n        When $y+\\mu\\leq 0$, the right hand equals $1$ and is clearly less than $8y^2$. Otherwise, $y+\\mu>0$ and then it suffices to note that\n        $$\n        1-F(y+\\mu)=1-e^{-(y+\\mu)^{-2}}\\ge 1-e^{-y^{-2}/4}\\ge y^{-2}/8,\n        $$\n        where we used $y+\\mu\\leq y+x+\\mu\\leq 2y$ in the first inequality and $1-e^{-x}\\ge x/2$ when $0\\leq x\\leq 1$ in the second inequality.\nIf $x+\\mu>y$, then $y+\\mu\\ge x+\\mu>y\\ge 1$ and similarly,\n        $$\n        1-F(y+\\mu)=1-e^{-(y+\\mu)^{-2}}\\ge(y+\\mu)^{-2}/2.\n        $$\n        Also, since for all $x\\ge 0$, $1-e^{-x}\\leq x$, we then have\n        $$\n        1-F(x+\\mu)=1-e^{-(x+\\mu)^{-2}}\\leq(x+\\mu)^{-2}.\n        $$\n        Therefore,\n        $$\n        \\frac{1-F(x+\\mu)}{1-F(y+\\mu)}\\leq 2\\left(\\frac{y+\\mu}{x+\\mu}\\right)^2<2\\left(\\frac{y+y-x}{y}\\right)^2\\leq 8\\leq 8y^2.\n        $$\n\n>Lemma G.10: For any random variable $ X $, let its density be denoted by $ f(x) $. If there exist $\\mu\\ge2\\sigma > 0$ and $\\mu'>\\mu$ such that\n$$\n\\frac{d\\log f}{dx}(x) \\leq -x/\\sigma^2, \\quad \\forall \\mu'\\ge x \\ge \\mu,\n$$\nthen for all $k\\in\\mathcal{N}^+$, if $\\mu'\\ge2\\mu$, we have\n$$\nE[X^k  1_{\\{\\mu'\\ge X \\ge \\mu\\}}]\\leq 2k!!\\cdot\\mu^k.\n$$\n\nLemma G.10 can be viewed as a comparison result: the expectation is maximized when equality holds, in which case $X$ becomes a truncated Gaussian, and the expectation can be computed inductively. Due to space limitations, we omit the full proof but can provide it in a follow-up if needed."}]

Author rebuttal:

Meta-review: This paper studies the m-set combinatorial semi-bandit problem and shows that FTPL with Fréchet perturbations achieves the near-optimal regret in the adversarial setting while also attaining logarithmic regret in the stochastic case, generalizing similar best-of-both-world guarantees of FTPL previously known for standard bandits (Honda et al. 2023). The authors further prove that the extra logarithmic factor in their bound is unavoidable without fundamentally new techniques.

Reviewers pointed out a mistake in the original proof, but were eventually convinced by the fix the authors proposed. Other than that, reviewers are generally positive about this paper. Given the long line of work on this best-of-both-world topic, I personally would be more excited if the analysis is significantly different from that of previous work and/or the results are presented for the general semi-bandit setting (instead of just m-set). That said, I would still recommend a borderline accept, given that it is a good addition to this line of work after all.
Decision: Accept (poster)