No-Regret M${}^{\natural}$-Concave Function Maximization: Stochastic Bandit Algorithms and NP-Hardness of Adversarial Full-Information Setting
Authors: Taihei Oki
Shinsaku Sakaue
Venue: NeurIPS 2024 poster (2024)
Keywords: online learning
discrete convex analysis
combinatorial bandit
Primary area: online_learning
TL;DR: We present stochastic bandit algorithms and prove NP-hardness of the adversarial setting for online M${}^{\natural}$-concave function maximization.

Abstract: M${}^{\natural}$-concave functions, a.k.a. gross substitute valuation functions, play a fundamental role in many fields, including discrete mathematics and economics. In practice, perfect knowledge of M${}^{\natural}$-concave functions is often unavailable a priori, and we can optimize them only interactively based on some feedback. Motivated by such situations, we study online M${}^{\natural}$-concave function maximization problems, which are interactive versions of the problem studied by Murota and Shioura (1999). For the stochastic bandit setting, we present $O(T^{-1/2})$-simple regret and $O(T^{2/3})$-regret algorithms under $T$ times access to unbiased noisy value oracles of M${}^{\natural}$-concave functions.  A key to proving these results is the robustness of the greedy algorithm to local errors in M${}^{\natural}$-concave function maximization, which is one of our main technical results. While we obtain those positive results for the stochastic setting, another main result of our work is an impossibility in the adversarial setting. We prove that, even with full-information feedback, no algorithms that run in polynomial time per round can achieve $O(T^{1-c})$ regret for any constant $c > 0$ unless $\mathsf{P} = \mathsf{NP}$.  Our proof is based on a reduction from the matroid intersection problem for three matroids, which would be a novel idea in the context of online learning.

No-Regred Mi-Concave Function Maximization: Stochastic Bandit Algorithms and NP-Hardness of Setting
Adversarial
Taihei Oki Hokkaido University
Shinsaku Sakaue'
The University of Tokyo and RIKEN AIP Tokyo, Japan
Hokkaido, Japan oki@icrt Aeidi.redd. www..ai..accccoooooocooooooooo.. jc.
sakaue@mist.i.u-tooe ac.jp
Abstract
M9-concave functions, a.k.a. gross substitute valuation functions, play a fundamen- tal role in many fields, including discrete mathematics and economics. In practice, perfect knowledge of Mc-concave functions is often unavailable a priori, and we can optimize them only interactively based on some feedback. Motivated by such situations, we study online Mc-concave function maximization problems, which are interactive versions of the problem studied by Murota and Shioura (1999). For the stochastic bandit setting, we present O(T -/sss/l regret and O )(Te/3)-regret -rrgrret algorithms under 7 times access to unbiased noisy value oracles of M9-concave functions. A key proving these results is the robustness of the greedy algorithm to local errors MR-concave function maximization, which one of our main technical results. While obtain those positive results for the stochastic setting, another main result of our work is an impossibility in the adversarial setting, We prove that, even with full-information feedback, no algorithms that run in poly- nomial time per round can achieve O(T!-c) regret for any constant > 0 unless P NP. Our proof is based on reduction from the matroid intersection problem for three matroids, which would be novel idea in the context of online learning.
1 Introduction
Mc-concave functions form fundamental function class in discrete convex analysis [32], and various combinatorial optimization problems written as MF-concave function maximization. In economics, Mi-concave functions (restricted to the unit-hypercube) are known as gross substitute valuations [19, 13, 25]; in operations research, M*-concave functions often used in modeling resource allocation problems [46, 30J. Furthermore, M'-concave functions form a theoretically interesting special case of (Db-Jsubmodular functions that the greedy algorithm can exactly maximize (see, Murota and Shioura [33]. Murota [32, Note 6.21], and Soma [48, Remark 3.3. while it is impossible for the submodular case [34, 10] and the greedy algorithm can find only approximately optimal solutions [35]. Due to the wide-ranging applications and theoretical importance, efficient methods for maximizing M²-coneave functions have been extensively studied [33, 46, 30, 20. 39].
When comes to maximizing MR-concave functions in practice, we hardly have perfect knowledge of objective functions in advance. For example, is difficult know the exact utility agent gains from some items, which is often modeled by a gross substitute valuation function. Similar issues are also prevalent in submodular function maximization, and researchers have addressed them by developing no-approximate-rgrre algorithms in various settings, including stochastic/adversarii environments
"Equal contribution, alphabetical order.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

and full innliinoomaatto/nnddii feedback [49, 14, 43, 54, 16, 36, 37, 38, 51, 55, 11. 40]. On the other hand, no-regret algorithms for Mc-concave function maximization have not been well studied, despite the aforementioned importance and practical relevance. Since the greedy algorithm can exactly solve M9-concave function maximization, an interesting question is whether we can develop no-regret algorithms-in in the standard sense without approximation- for M*-concave function maximization.
1.1 Our contribution
This paper studies online Mi-concave function maximization for the stochastic bandit and adversarial full-information settings. Below are details of our results.
In Section 4, we study the stochastic bandit setting, where we can only observe values of an underlying Mi-concave function perturbed by sub-Gaussian noise. We first consider the stochastic optimization setting and provide times we can access the noisy value oracle. We then convert it into an ((TT-/))-cuulaaiiv regret algorithm (Theorem 4.3), where T is the number of rounds, using the peplore-then-commee technique. En route to developing these algorithms, we show that the greedy algorithm for Me-concave function maximization is robust to local errors (Theorem 3. I). which one of our main technical contributions and is proved differently from related results in submodular and M function maximization.
regret algorithm (Theorem 4.2), where T is the number of
we establish the NP-hardness of no-regret learning for the adversarial -ull-information setting. Specifically. Theorem 5.2 shows that unless P = NP, no algorithms that run in polynomial time in each round can achieve poly(N) - - regret for any constant > 0, where poly (NV stands for any polynomial of N, the per-round problem size. Our proof is based on the fact that maximizing the sum of three M9-concave functions is at least as hard as the matroid intersection problem for three matroids, which is known to be NP-hard." 2 We carefully construct concrete online Mc-concave function maximization instance that enables reduction from this NP-hard problem. Our high-level idea, namely, connecting sequential decision-making and finding common base of three matroids, might be useful for proving hardness results in other online combinatorial optimization problems.
In Section
1.2 Related work
There is large stream of research on no-regret submodular function maximization. Our stochastic bandit algorithms are inspired by a line of work on explore-then-commit algorithms for stochastic bandit problems [37, 38, 11] and by a robustness analysis for extending the offline greedy algorithm to the online setting [36]. However, unlike existing results for the submodular case, the guarantees of our algorithms in Section 4 involve no approximation factors. Moreover, while robustness properties similar to Theorem 3.1 are widely recognized in the submodular case, our proof for the Mi-concave case substantially differs from them. See Appendix A for a detailed discussion.
Combinatorial bandits with linear reward functions have been widely studied [6, 9, 8, 41], and many studies have also considered non-linear functions [7, 21, 15, 29]. However, the case of MC-concave functions has not been well studied. Zhang et al. [53] studied stochastic minimization of Li-convex functions, which form another important class in discrete convex analysis [32] but fundamentally differ from MR-convex functions. Apart from online learning, a body of work has studied maximizing valuation functions approximately from samples to do with imperfect information [2. 3, 4].
Regarding hardness results in online learning, most arguments are typically information -thooretic. For instance, the minimax regret of hopeless games in partial monitoring is ?(T) [24, Section 37.2]. By contrast, we establish the NP-hardness of the adversarial full-information online MC-concave function maximization, even though the offline Mi-concave function maximization is solvable in polynomial time. Such situation is rare in online learning. One exception is the case studied by Bampis et al. [5]. They showed that no polynomial-time algorithm can achieve sub-linear approximate regret for some online min-max discrete optimization problems unless NP = RP, even though their offline counterparts are solvable in polynomial time. Despite the similarity in the situations, the problem class and proof techniques are completely different. Indeed, while their proof is based on the NP-hardness of determining the minimum size of a feasible solution, it can be done in polynomial time for M -oncavve function maximization [47, Corollary 4.2). They also proved the NP- of
Note that this fact alone does not immediately imply the hardnessss of no-regret learning since the learner take different actions across rounds and each conccae function maximization instance is not NP hardd..
2

---

the multi-inan instance setting, which is similar to the maximization of the sum of Mc-concave functions. However, they did not relate the hardness of
problems to that of no-regret learning.
2 Preliminaries
Let V = (1...., N) be a ground set of size N. Let be the all-zero vector. For E V. let e; E RV denote the ith standard vector, i.e., the ith element 1 and the others are 0: let co 0 for convenience. For x E RV and 5 C V. let (S)) = Eies** zt. Slightly abusing notation, let =<(i) = x x((i) = Ty.
For function f zV + R U } on the integer lattice zV, its effective domain is defined as function f is called proper if dom f * V. We say proper function / ZV + ( is M'-concave if for every 2, y dom f and E V with x(i): v(i).
dom f {z E ZV f(x)= > -o0
there exists j E V U (0) with x(i) < y(i) or 0 such that the following inequality holds:
f(x) + f(v) S f(x - e, + eg) + f(y + e; - ee))
(I)
Similarly, we say f ZV + R U {+oo} is M -f is Ma-concave. If xVV & v(V). M²-coneave functions satisfy more detailed conditions, as follows.
Proposition 2.1 (Corollary of Murota and Shioura [33, Theorem 4.2]). Let ZV -UU(- be an M -c-cccvvee function. Then, the following conditions hold for every 2.Y E dom
(a) (V) y vV), Aj V with x x(j) < y(i). (x) f(y) I f(x + 6j) + f(y - eg) holds.
(b) a S (V). Vi E V with x01) > y(i). 3j E V with x(j) < v()), (1) holds.
Let ba = = a(i) S (i) < b(i) } be an integer interval of a, E (Z (=x)=)))) and f be Mi-concave. If dom fn + 0. restricting dom f to [a, bj preserves the Mi-concavity [32. Proposition 6.]. The sum of M Mi-concave functions is not necessarily Mc-concave [32, Note 6.16]. In this paper, we do not assume monotonicity, i..., x S y (element wise)) does not imply f(x) S f(y).
2.1 Examples of M -onccavee functions
Maximum- on bipartite graphs. Let (V, W; be a bipartite graph, where the set V of N left-hand-side vertices is a ground set. Each edge ij E E is associated with weight wa E R. Given sources Z allocated to the vertices in V. let fxx be the maximum- flow value, i.e.,
f(x) = max <E DEENO
{ EijEE WiyKis Vi E V, ΣjHEE Sug = xy Vj E W, EMNEE Siz = y/
This function f is MR-concave; indeed, more general functions specified by convex-cost flow problems on networks are MD-coneave [32, Theorem 9.27]. If we restrict the domain to {0, 13V and regard V as a set of items, W as a set of agents, and waj 2 0 as the utility of matching an item i with an agent j. the resulting set function f {0. 1 + R2zo coincides with the OXS valuation function known in combinatorial auctions [45, 25), which is special case of the following gross substitute valuation.
Gross substitute valuation. In economics, an agent's valuation (a non-negative monotone set function of items) is said to be gross substitule (GS) if, whenever the prices of some items increase while the prices of the other items remain the same, the agent keeps demanding the same-priced items that were demanded before the price change [19, 25]. M7-concave functions can be viewed as an extension of GS valuations to the integer lattice [32. Section 6.8]. Indeed, the class of M2-concave functions restricted to {0, 13V is equivalent to the class of GS valuations [13].
Resource allocation. M9-concave functions also arise in resource allocation problems [46, 30], which are extensively studied in the operations research community. For example, given n univariate concave functions / : > IR U }oo and a positive integer K, a function ʃ defined by f(x) = f. if 2 2 and xVV)= < K and f(z) = - oo otherwise is Mi-concave. More general examples of Mo-conaave functions used in resource allocation are given in, e.g., Moriguchi et al. [30].
More examples can be found in Murota and Shioura [33, Section 2] and Murota [32, Section 6.3). As shown above, MC-concave functions are ubiquitous in various fields. However, those are often difficult to know perfectly in advance: we may neither know all edge weights in maximum-flow problems, exact valuations of agents, nor fis' values at all points in resource allocation. Such situations motivate us to study how to maximize them interactively by selecting solutions and observing some feedback.
3

---

Algorithm 1 Greedy-style procedure with possibly erroneous local updates
1: 2o =0 -0
2: for k k= 1..... K;
3: Select in E VV U {0)
> Standard greedy selects in E arg max f(#x-1 + e.).
4: 3 Fk-1 + Ci.
2.2 Basic setup
Similar to bandit convex optimization [23], we consider learner who interacts with sequence of Mc-concave functions, ,,....,/f f" over 7 rounds. To avoid incurring f'(x) -oo, we assume that dom f dom f are identical dom ft. We also assume 0 dom f and dom f S Z which are reasonable in all the examples in Section 2.1. We consider constrained setting where the learner's action r E dom f must satisfy x<V) S K. If dom f S 10. this is equivalent cardinality constraint common in set function maximization. Let X' { x E dom f =vV) I K) denote the set of feasible actions, which the learner is told in advance. (More precisely, poly(N)- time membership oracle of X' is given.) Additional problem settings specific to stochastic bandit and adversarial full- iall-information cases provided in Sections 4 and 5, respectively.
the
3 Robustness of greedy Mi-concave function maximization to local errors
This section studies a gtyedy-style procedure with possibly erroncous local updates for M Mc-concave function maximization, which will be useful for developing stochastic bandit algorithms in Section 4. Let f z + R {-oc} be an Mi-concave function such that 0 E dom f C Z%%,. which we want to maximize under x<V) I K. Let x² E arg max f(x) x E dom (vV) s K } be an optimal solution. We consider the procedure in Algorithm 1. If is known priori and é f1:-... ig selected as in the comment in Step 3, it coincides with the standard greedy algorithm for Mi-concave function maximization and returns optimal solution [33]. However, when f is unknown, we may select different -1,-.., ix than those selected by the exact greedy algorithm. Given any x E ZV and update direction E V U {0}. we define the local error of
err(i ()) = max f(x + ev) f(x e.) 20
d'EVufo)
which quantifies how much direction deviates from the choice of the exact greedy algorithm when is given. The following result states that local errors affect the eventual suboptimality only additively, ensuring that Algorithm 1 applied to -c-cccavee function maximization is robust to local errors.
Theorem 3.1. For any , ik E VU (0), holds that f(xx) 2 f(x*)
E err(1x ())-))
Proof. The claim is vacuously true if crr(is xx-1) 1) = +oo occurs for some k < K. Below, we focus on the case with finite local errors. For h: = 0, 1,.,,., K, we define
y% = ( y IR. v(V) < K - k + (v)
where y 2 3% is read element-wise That is, Di. C X consists of feasible points that be reached from I% by the remaining K 1 updates (see Figure 1). Note that x" E Yo and Yk = {ax) hold.
To prove the theorem, we will show that the following inequality holds for any i E {1....,K] K]:
max f(v)= max DEV'-1
f(y) err(ix (xx-1).
yEli
Take yk-1 E arg max.eea--..- (v) and yk E arg max f(v). If J f())) 2 f( we are done since err(iz xx-1) 0. Thus, we assume f()) < (()-1),, which implies yk-1 E Va- 1 Pk. Then, we can prove the following helper claim by using the M'-concavity of f.
Helper claim. If yk-1 E V--1 Ve. there exists j E V {0} such that yk-1 + Cip - ej V/ and
f(xx) + ((x-11 & f(x Ci + ez)) + s(uk + Ci. - -ez).
(4)
Assuming the helper claim, we can easily obtain (3). Specifically, (i) f(yk-1 + Bity - eg) S f())
holds due to 9k-1 Ci. - ej 3% and the choice of yk. and (ii) err(is xk-1) 2 f f(z=-1 + c2)

---

12
V
2006
11
11
Figure I: Images of 3% on 2 The set of integer points in the trapezoid is the feasible region X. Left: the gray area represents V/-1 consisting of points reachable from Tę-1- Middle: if iz = 0 (case 1), Fk-1 Ty holds and J shrinks to Vk. the darker area, since the constraint on y(V) gets tighter. Right: is = i1 (cases 2 and 3), the area, Vk, reachable from F& = zk-1 + C₁2 shifts along ei,
fax-1 +eeg)) (0x-es - holds due the definition of the local error (2). Combining these with (4) and rearranging terms imply (3) as follows:
(4) f(J)) 2 f(n ti, - -e,) fykk + f(x)) f( - Csa + +e,) 2 f(k--1) eer(ik 1x1-))
(1)
Given (3), the theorem follows from simple induction on k 1,..., K. For each k, we will prove
max f(w) 2 f(x'(())))))))))))
Σ err(iw xww-1)
(5)
yE
The case of 1 follows from since x²' E Vo- is true for k: 1, (3) and the induction hypoth-
esis imply maXgeys fly) 2 maxye)i- f(y) - err(is zk=1) 2 f(x*) > EE=1 err(iw 1
err(ix (x thus obtaining (5). Since Vx {xx) } holds, setting k: K in (5) yields Theorem 3.1.
The rest of the proof is dedicated proving the helper claim, which do by examining the following three cases. The middle (right) image in Figure/ illustrates case (cases 2 and 3).
Case 1: is O. Due to z TRe yk-1 Vk-1 \. 13:% implies (V-)(()) K - (k- + 1
Thus, 24 VV) = - (V) = (V- (K-++1) < VV) holds. From Proposition 2.1 (a), there
exists j E with n4 (1) < 1 that satisfies (4). Also, Mk-1 1 2 -4-1 1 Ta. 2* xx < y2 (1).
and (y/- (V) = K - k Ia IV)) K h: + xx xx(VV imply yk-1 ej E Jkk
Case 2: is 0 and (V) S yv))(V). In this case, 1k-1 Va-1 V' implies yk-1 2 xk-1 1 and
2k-1 2 F4. = Tk-1 + Cig, hence (1x) 1 From Proposition 2.1 (b), there exists j E V
with (1) < yk (j) that satisfies (4). Since l/- Fk-1 Eg Ci and zx(() < 9 3)-1()) 10).
have 0k-1 + c42 - -e 2 Zk. Also, we have (JA-1 + Cit -e,) 8k 1 S K (k 1)
Ix-1 (1) K k (V).. Therefore, we have Vk-1 Eis ej E J
Case 3: i, # and 24 (V) > yk- yk-1(V), Since 1 E Va-1, we have yk-1 2 -k-1- We also have
(V) = 2% (VV) + These imply yk-1 = - Therefore, (4) with j = 0 holds by yk 1k-1(V) <
equality, and - + Ci. 2b V%. also holds.
The robustness property in Theorem 3.1 plays a crucial role in developing stochastic bandit algorithms in Section Furthermore, the robustness would be beneficial beyond the application stochastic bandits since Mi-concave functions often involve uncertainty in practice, as discussed in Section 1. Note that Theorem 3.1 has been known even in the field of discrete convex analysis and that the above proof substantially differs from the original proof for the greedy algorithm without local errors for M -c-occave function maximization [33]. Indeed, the original proof does not consider set like Dkk which is crucial in our proof. It is also worth noting that Theorem 3.1 automatically implies the original result on the errorless case by setting err(ix xx-1) ) = for k. We also emphasize that while Theorem 3.1 resembles robustness properties known in the submodular case [49, 14, 36, 37, 38, 11], ours different from them in that involves no approximation factors and requires careful inspection of the solution space, as in Figure 1. See Appendix A for detailed discussion.
5

---

4 Stochastic bandit algorithms
This section presents no-regret algorithms for the following stochastic bandit setting.
Problem setting. For = 1,... T, the learner selects x' E X and observes (xxx ( where f" zV - [0, i.i.d. -sub-Gaussian noises, i.e., E[cxp(Ae")] S exp(X2/2) /2) for À TR.3 Let E arg max f" (a) denote the offline best action. In Theorem 4.2, we will discuss the simple-regret minimization setting, where the learner selects TTT+1 E X' after the Tth round to minimize the expected simple regret:
U { {-oo] is an unknown M²-concave function and (e') is sequence of
sRegr = f=xx) - [[
Here, the expectation is about the learner's randomness, which may originate from noisy observations and possible randomness in their strategy. This is a stochastic bandit optimization setting, where the learner aims to find the best action without caring about the cumulative regret over the T rounds. On the other hand, Theorem 4.3 is about the standard regret minimization setting, where the learner aims to minimize the expected cumulative regret (or the pseudo-regret, strictly speaking):
Regr- = T (x²) - E Σ M())
In this section, assume that T is large enough to satisfy T 2 K(N + 22 to simplify the discussion.
Pure-exploration algorithm. Below, we will use a UCB-type algorithm for pure exploration in the standard stochastic multi-armed bandit problem as building block. The algorithm based on MOSS (Minimax Optimal Strategy in the Stochastic case) and is known achieve an O simple regret as follows. For completeness, we provide the proof and the pseudo-code in Appendix B.
Proposition 4.1 (Lattimore and Szepesvári [24, Corollary 33.3]). Consider a stochastic multi-armer bandit instance with M arms and T" rounds, where T" M. Assume that the reward of the ith arm in the tth round, denoted by Y satisfies the following conditions: 14: == E[ E [0 and Y b4 is AsubbGaausian Then, there is an algorithm that, after pulling arms T' times, randomly returns i {I....,M] MM with μ* Blu4] O( O(MM/T)), where μ³' = maax(M1........ ................................ J.)-
Given this fact and our robustness result Theorem 3.1, it not difficult to develop an OTT-1/2)- simple regret algorithm; we select ig in Algorithm with the algorithm in Proposition 4.1 consuming [T/K] rounds and bound the simple regret by using Theorem 3.1, as detailed below. Also, given the O(T--/ regret algorithm, an ) algorithm follows from the explore-then- commit technique, described subsequently. Therefore, we think of these no-regret algorithms as byproducts and the robustness result in Theorem 3.1 as our main technical contribution on the positive side. Nevertheless, we believe those algorithms are beneficial since no-regret maximization of Mc-concave functions have not been well studied, despite their ubiquity as discussed in Section 2.1.
The following theorem presents our result regarding an O(T-1/2) --smmmp regret algorithm.
Theorem 4.2. There is algorithm that makes up to T queries to the noisy value oracle of f* and outputs x such that sRegr = O(K3/2 /N/T).
Proof. Based on Algorithm 1, we consider randomized algorithm consisting of K phases. Fixing realization of 14-1, we discuss the kth phase. We consider the following multi-armed bandit instance with at most N + arms and [T/ K rounds. The arm set is {i E V U {0} #x-1 + e, E X }. i.e., the collection of all feasible update directions; note that the learner can construct this arm set since X is told in advance. In each round & {1,.... [T/K] the reward of an arm E V U [0} is given by Y," = "f'=="(ze (zx=1 + +e) + where e is the -ssu--Gasssinn noise. Let μ4 = [| = f (ax-1+ +e) E [0, denote the mean reward of the arm and μM max,evu(0) 14 the optimal mean reward. If we
apply the algorithm in Proposition 4.1 to this bandit instance, it randomly returns is E VU (0} such P EC pi, - O(v KN/T), consuming [T/K] queries.
that [E[ err(is 1xx-1) I
"Restricting the range of f to [0, 1] and the sub- Gaussian constant to 1 is for simplicity: our results extend to any range and Lub-Gaussian constant. Note that any zero-ment grro random variable in [-1, +1] is

---

Consider sequentially selecting is as above and setting 24: = 2k-1 + Cig+ thus obtaining 21. 21....: 2K. For any realization of ......, ix, Theorem 3.1 guarantees f (œx) - - err(ix [xt By taking the expectations of both sides and using the law total expectation, we obtain
K
K
("xx))BIf( (xx)
Σ
eir(is (x
B| Blerr(i (ax-1) | -z-1) = =O(K
Thus, x² T+1 = KK achieves the desired bound. The number of total queries is K K ST
We then convert the ((TT-1/)-sinna regret algorithm into an ((T2/3))regge algorithm by using the sxplore-tee-commie technique as follows.
Theorem 4.3. There is an algorithm that achieves Regr O(KN1/372/3)
Proof. Let f T be the number of exploration rounds, which will tune later. If we use the algo- rithm of Theorem 4.2 allowing π queries, we can find x E X' with sReg, ((33/2 N/T). If we commit to x= T+1 in the remaining T --- rounds, the total expected regret is
f
-T sRegr S + ..egs = O(T TK /N/T). Σ M(a")
Regr E
By setting / = ((KN1/372/3 we obtain Regr = O(KN1/37213
5 NP-hardness of adversarial full-information setting
This section discusses the NP-hardness of the following adversarial full-information setting.
Problem setting. An oblivious adversary chooses an arbitrary sequence of M'-concave functions, f!..... fT, where ft ZV + [o, 1 {- for t 1, 1,.... T, in secret from the learner. Then, fort 1, .,.... T, the learner selects x' X' and observes f", i.e., full-information feedback. More precisely, we suppose that the learer gets free access to poly(NV- value oracle of f by observing f since M*-concave functions may not have polynomial-sin representations in general. The learner aims to minimize the expected cumulative regret:
('(x)
Σ MMa
(6)
where the expectation about the learner's 's randomness. To simplify the subsequent discussion, we focus on the case where the constraint is specified by K = N and f², / are defined on (0, 1;;; therefore, the set of feasible actions is X { E dom f x<V) S K =
f
For this setting, there simple no-regret algorithm that takes exponential time per round. Specifi- cally, regarding each x E N' as an expert, we use the standard multiplicative weight update algorithm to select I ....... . 2r [26, 12]. Since the number of experts is |X| = 2N, this attains an expected regret bound of O(VT log |x] S (Nly(N) while taking prohibitively long voly(N)[{)] 2 2 time per round. An interesting question is whether similar regret bound is achievable in polynomial time per round. Thus, we focus on the polynomial-time -ime randomized learner, as with Bampis et al. [5].
Definition 5.1 Polynomial-time randomized learner). We say an algorithm for computing T1...., FT is a polynomial-time -ie randomized learner if, given poly( (N) )-time value oracles of revealed functions, runs in poly(N, T') time in each round, regardless of realizations of the algorithm's randomness."
Note that the per-round time complexity can depend polynomially on T. Thus, the algorithm can use past actions, 3 ... -t as inputs for computing Ir, long the per-round time complexity is polynomial in the input size. The following theorem is our main result on the negative side.
'While this definition does not cover so called efficient Las Vegas algorithms, which run in polynomial time in expectation, requiring polynomial runtime for every realization is standard in randomized computation [1].
7

---

Theorem 5.2. In the adversarial full-information setting, for any constant > 0. no polynomial-ting randomized learner can achieve regret bound of poly(N) T in expectation unless P NP.
5.1 Proof of Theorem 5.2
As preparation for proving the theorem, we first show that it suffices to prove the hardness for any polynomial-tine deterministic learner and some distribution on input sequences of functions, which follows from celebrated Yao's principle [52]. We include the proof in Appendix C for completeness.
Proposition 5.3 (Yao [52]). Let A be a finite set of all possible deterministic learning algorithms that run in polynomial time per round and FLT a finite set of sequences of M -C-cccavee functions,
f'...., fT. Let Rogr(a. f(.)) be the cumulative regret deterministic learner a E A achieves on sequence f'=T= (J any distribution on F..T holds that
5") Fii Then, for any polynomial-time randomized learner A and
max [E [Regr fA,f f(:T)] f1. EFITT } min B (a.
A}.
Note that the left-hand side is nothing but the expected cumulative regret (6) of a polynomial-timg randomized learner A on the worst-case input f²= Thus, to prove the theorem, it suffices to show that the right-hand side, i.e., the expected regret of the best polynomial-time deterministic learner on some input distribution q, cannot be as small as poly(N) () T unless P = NP. To this end, we will construct a finite set FF. of sequences of M concave functions and a distribution on it.
The fundamental idea behind the subsequent construction of M9-concave functions is that the matroid intersection problem for three matroids (the 3-matroid intersection problem, for short) is NP-hard.
3-matroid intersection problem. A matroid M over V is defined by a non-empty set family B C 2 such that for B1, B2 B and e B1 \ Bg. there exists j E B, B1 such that B1 fi) e B. Elements in B are called bases. We suppose that, given a matroid, we can test whether a given S C V a base in poly(N) time. (This is equivalent to the standard poly --)-time independence testing.) The 3-matroid intersection problem asks to determine whether three given matroids M,, M2, M3. over a common ground set V have common base B E B1 B2 B3 or not.
Proposition 5.4 (cf. Schrijver [44, Chapter 41]). The 3-matroid intersection problem is NP-hard.
We construct Mi-concave functions that appropriately encode the 3-matroid intersection problem. Below, for any B C V, let g E {0, 1) denote vector such that 1 1 (i) = 1 if and only if E B.
Lemma 5.5, Let M be a matroid over V and B C 2 its base family. There is a function f (0, 1) - [0, 1] such that (i) fxx 1 if and only if x 1a for some B E B and f(x) 1 1 1/N otherwise, (ii) f is M--concaee, and (iii) fx) can be computed in poly poly(N) time at every E (0, 13V.
Proof. Let ||-I|i denote the -oomm. We construct the function f (00 13 + [0, 1] as follows:
(xx =
min 1ll1 (x E <0.13")
N BEB
Since 0 < ||y/| < N for y E [-1, +1] f(x) takes values in [0, 1]. Moreover, mingeøllr 1 l| is zero if x = 1g for some B E B and at least otherwise, establishing (i). Below, we show that f is (ii) Mo-ccaa and (iii) computable in poly(N) time.
We prove that T(x) == mingeBle | 1Bll is M*-convex, which implies the M Mi-concavity of Let òg zz (0, +oc) be the indicator function of B, i... 68(2) = 0 x 1g for some B B and too otherwise. Observe that is the integer infimal convolution of ||- and ōg. Here, the integer infimal convolution of two functions f2 z +
{+oo} is function of x E zV defined as (JiDz Dz (x) = min (a y) fa(y): y E ZV). and the Mi-concavity is preserved under this operation [32, Theorem 6.15]. Thus, the M Mi-convexity of 7(x) (I1-112 2z 85) (x)) follows from the Mi-convexity of the n-rrm ||-112 [32, Section 6.3) and the indicator function 8g [32, Section 4. I].
Next, show that r(x) is computable in Poly(N) time for every {0, 1)V, which implies the poly (N) -timm computability of fxx) As described above, T is the integer infimal convolution of
$ Our result does not exclude the possibility of polynomial no-regret learning with an exponential factor in the regret bound. However, we believe whether voly(N). - regret is possible or not is of central interest.
8

---

11-11 and 6g, i.e., 7(x) = min + +v()): : E Z Since the function y + ||-- yl1 is M convex [32, Theorem 6.15]. =2x) is the minimum value of the sum of the two Mc-coeexx functions. While the sum of two M Mc-convex functions fı, f::2= + Zzo U {+oo] is no longer Mc-conve in general, we can minimize it via reduction to the M-convex submodular flow problem [32, Note 9.30]. We can solve this by querying f1 and f2 values poly(N, (N, log L, log M) times, where L is the minimum of the fo -d-immettrr of dom fi and dom f2 and M is an upper bound on function values [18. 17]. In our case of [v()) = |æ - yll and (2(y) = 5z (y). we have I, = and M < N, and we can compute (v) and f2(y) values in poly (N) time (where the latter is due to the poly(N) membership testing for B). Therefore, ((x) is computable in Poly(N) time, and so is (z)
Now, we are ready to prove Theorem 5.2.
Proof of Theorem 5.2. Let M1, M2, M3 be three matroids over V and fı, /2. J3 functions defined as in Lemma 5.5, respectively. Let Fil be a finite set such that each f² (1 11=1,.... 1...., T) is either S1, f2. fa- Let q be distribution on FI: that sets each f* to fi, f2, or fa with equal probability.
Suppose for contradiction that some polynomial-time deterministic learner achieves poly(NV) T regret in expectation for the above distribution q. Let T be the smallest integer such that the regret bound satisfies poly(N) TT-- < IN + T > p3Ny (N)) Note that T is polynomial in N since 0 is constant. We consider the following procedure.
Run the polynomial- deterministie learner on the distribution q and obtain H, for 1, 1,..,, T. If some satisfies fi (2+) f2(11) = fa(rz) = 1, output "Yes" and otherwise "No."
If M2. M3 have common base B B3 nB n B3, we have (1B) = f2(1g) fs On the other hand, if Ie # 1 for every / E Bi B3, E[f*(=*)] S 1- 3N holds from Lemma 5.5 and the fact that f is drawn uniformly from {f1, f2. fs}. Thus, to achieve the poly(NV -' regret for T > (3.N Molylol(())) (1)) the learner must return corresponding to some common base at least once among T rounds. Consequently, the above procedure outputs "Yes." If M1, M2, M, have no common base, none of æ1,..., JT can be common base, and hence the procedure outputs "No." Therefore, the above procedure returns correct answer to the 3-matroid intersection problem. Recall that T is polynomial in N. Since the learner runs in T poly((, T)) time and we can check (a,) = f2xx,) (a(x)) = for / = 1, ..... T in T poly (N) time, the procedure runs in poly (N) time. This contradicts the NP-hardness of the 3-matroid intersection problem (Proposition 5.4) unless = NP. Therefore, no polynomial-time deterministic learner can achieve poly(N) T1- regret in expectation. Finally, this regret lower bound applies to any polynomial-tine time randomized learner the worst-case input due to Yao's principle (Proposition 5.3), completing the proof.
Remark 5.6. One might think that the hardness simply follows from the fact that no-regret learning in terms of (6) is too demanding. However, similar criteria are naturally met in other problems: there are efficient no-regret algorithms for online convex optimization and no-approximate-rege algorithms for online submodular function maximization. What makes online Mc-concave function maximization NP-hard is its connection to the 3-matroid intersection problem, as detailed in the proof. Consequently, even though offline M#-concave function maximization is solvable in polynomial time, no polynomial-timg randomized learner can achieve vanishing regret in the adversarial online setting.
6 Conclusion and discussion
This paper has studied no-regret Mi-concave function maximization. For the stochastic bandit setting, we have developed ( /
regret and O(K //KK)333r//3)rrrr -regrett algorithms. A crucial ingredient is the robustness of the greedy algorithm to local errors, which we have first established for the Mi-concave case. For the adversarial full-information setting, we have proved the NP-hardness of no-regret learning through a reduction from the 3-matroid intersection problem.
Our stochastic bandit algorithms are limited to the sub- Gao--aussin noise model, while our hardness result for the adversarial setting comes from somewhat pessimistic analysis. Overcoming these limitations by exploring intermediate regimes between the two settings, such as stochastic bandits with adversarial corruptions [28], will be an exciting future direction from the perspective of beyond the worst-case analysis [42]. We also expect that our stochastic bandit algorithms have room for

---

improvement, considering existing regret lower bounds for stochastic combinatorial (semi- -bandits with linear reward functions. For top-K combinatorial bandits, there is a bound of (((N/e2) for any (e, 8)-PAC algorithm [41]. Since our 0(K3/2 N/T))-smple regret bound implies that we can achieve an E-error in expectation with O/K3N/E") samples, our bound seems tight when K = (1), while the K factors would be improvable. Regarding the cumulative regret bound, there is an 92( V NT) lower bound for stochastic combinatorial semi-bandits [22]. Filling the gap between our O OKKN N1/332//5) upper bound and the lower bound is an open problem. (Since we have assumed 7 = ()(N) in Section 4, our upper bound does not contradict the lower bound.) We believe that our upper bound ís essentially tight considering recent minimax regret bound by Tajdini et al. [50] for bandit submodular maximization, which we discuss in detail below. Regarding the adversarial setting, it will be interesting to explore no-approximate- algorithms. If M conccce functions are restricted to {0, 13/, the resulting problem is a special case of online submodular function maximization and hence vanishing 1 1/2-approximate regret is already possible [43, 16, 36]. We may be able to improve the approximation factor by using the M'-concavity.
lower
Discussion on the tightness of the O(KN1/322/3 bound. As mentioned above, obtaining a tight regret bound for stochastic bandit Mi-concave maximization is left open. Nevertheless, we conjecture that our 03323/372/3 bound in Theorem 4.3 is tight unless we admit exponential factors in K. The rationale behind this conjecture lies in a recent result by Tajdini et al. [50]. They studied stochastic bandit monotone submodular maximization with a ground set of size N and a cardinality constraint of K, and they showed that there is lower bound of
N K (M="))
Ω (K (K-ii -))11///372/33
on robust greedy regret, which compares the learner's actual reward with the output of the greedy algorithm, denoted by Sgr, applied to the underlying true submodular function. Here, i S K the
16 ( (N-K)² S T; see Tajdini et al. [50, Theorem 2.3] for details. 6 This largest positive integer with NiKe lower bound suggests that the O (KN118822/ regret for stochastic bandit submodular maximization, which can also be achieved by the poglore-then-oomii strategy, is inevitable in general. We can interpret the / (""")T term as the regret achieved by regrading all (~=*) subsets as arms and using a UCB-type algorithm. Thus, the lower bound consists of the two regret terms achieved by the explore-then-conee
and the UCB applied exponentially many arms.
Currently, we have observed that the proof of the lower bound by Tajdini et al. [50] does not directly apply to our stochastic bandit -onccav maximization problem. Specifically. the function used in their proof for obtaining the lower bound is submodular but not Mi-concave. Nevertheless, the problem setting of Tajdini et al. [50] and our problem in Section 4, with the domain restricted to {0, 1)", have notable connections:
1. Since the greedy algorithm applied to the unknown true MD-concave function f can find an optimal solution x², we have x= Sgr- Therefore, the notion of robust greedy regret in Tajdini et al. [50] essentially coincides with the standard regret in our case.
2. Both the O(KN1/3T2/3) and (V(*+)))) regret bounds discussed above can also be
achieved by the explore-then-commi and UCB strategies, respectively, in our M'-concave case, where the former is exactly what our Theorem 4.3 states.
Considering these facts, we expect that we can construct a hard instance of stochastic bandit M concave maximization similar to Tajdini et al. [50] to establish the same regret lower bound. Therefore, we conjecture that our O K /37/2/33 regret bound in Theorem 4.3 is tight in K, N, and T. if we want to avoid the exponential factor, which generally scales as N regardless of the value of T.
"More precisely, the lower bound of Tajdini et al. [50, Theorem 2.3] applies to the class of non-adaptive greedy algorithms, which specify error thresholds only depending on T,, N, and K. Our algorithm in Section 4, which runs MOSS in each iteration for [T/K] rounds, falls into this category. Tajdini et al. [50. Theorem 2.1] also shows that a weaker lower bound, with the first term replaced with (K- 1)1/3 N 1/3/2//3 applies to all stochastic bandit submodular maximization algorithms.
10

---

Acknowledgements
The authors thank the anonymous reviewers for their valuable feedback, particularly for bringing our attention to the recent result by Tajdini et al. [50]. This work was supported by JST ERATO Grant Number JPMJER 1903, JST CREST Grant Number JPMJCR24Q2, JST FOREST Grant Number JPMJFR232L, and JSPS KAKENHI Grant Numbers 1722 and 24K21315.
References
[1] S. Arora and B. Barak. Computational Complexity: A Modern Approach. Cambridge University Press, 2009 (cited on page 7).
[2] M. F. Balcan, F. Constantin, S. Iwata, and L. Wang. Learning valuation functions. In Proceed- ings of the 25th Annual Conference on Learning Theory, volume 23, pages 4. 1.4.24.. PMLR, 2012 (cited on page 2).
[3] E. Balkanski, A. Rubinstein, and Y. Singer. The power of optimization from samples. In Advances Neural Information Processing Systems, volume 29, pages 4017-4025. Curran Associates, Inc., 2016 (cited on page 2).
[4] E. Balkanski, A. Rubinstein, and Y. Singer. The limitations of optimization from samples. Journal of the ACM, 69(3):1-33, 2022 (cited on page 2).
[5] E. Bampis, D. Christou, B. Escoffier, and K. T. NGuyen. Online learning for min-max discrete problems. Theoretical Computer Science, 930:209-222 217, 2022 (cited on pages 2, 7).
[6] N. Cesa- BiancBianchiii and G. Lugosi. Combinatorial bandits. Journal of Computer and System Sciences, 78(5):1404-1422 2012 (cited page 2).
[7] W. Chen, Y. Wang, and Yuan. Combinatorial multi-armed bandit: general framework and applications. In Proceedings of the 30th International Conference on Machine Learning. volume 28, pages 151-159 PMLR, 2013 (cited on page 2).
[8] A. Cohen, T. Hazan, and T. Koren. Tight bounds for bandit combinatorial optimization. In Proceedings of the 30th Annual Conference Learning Theory, volume 65, pages 629-642. PMLR, 2017 (cited on page 2).
[9] R. Combes, M. S. Talebi, A. Proutiere, and M. Lelarge. Combinatorial bandits revisited. In Advances Neural Information Processing Systems, volume 28, pages 2112-2 Curran Associates, Inc., 2015 (cited on page 2).
[10] U. Feige. A threshold of In for approximating set cover. Journal of the ACM, 45(4))634-652, 1998 (cited on page 1).
[11] F. Fourati, C. J. Quinn, M.-S. Alouini, and V. Aggarwal. Combinatorial stochastic-greedy bandit. Proceedings of the 38th AAAI Conference on Artificial Intelligence, 38(1 122052 12060, 2024 (cited on pages 2, 5, 14).
[12] Y. Freund and R. E. Schapire. A Decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-13) 1997 (cited page 7).
[13] S. Fujishige and Z. Yang. A note Kelso and Crawford's ' gross substitutes condition. Mathe- matics of Operations Research, 28(3):463-44) 4693,466,400 2003 (cited on pages 1, 3).
[14] D. Golovin, A. Krause, and M. Streeter. Online submodular maximization under matroid constraint with application to learning assignments. arXiv: 1407. 1082, 2014 (cited on pages
5, 14).
[15] Y. Han, Y. Wang, and X. Chen. Adversarial combinatorial bandits with general non-linear reward functions. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 4030-4039. PMLR, 2021 (cited page 2).
[16] N. Harvey, C. Liaw, and T. Soma. Improved algorithms for online submodular maximiza- tion via first-order regret bounds. In Advances in Neural Information Processing Systems, volume 33, pages 123-133. Curran Associates, Inc., 2020 (cited on pages 10).
[17] S. Iwata, S. Moriguchi, and K. Murota. A capacity scaling algorithm for M-convex submodular flow. Mathematical Programming, (03(1):181-23) -02, 2005 (cited on page 9).
[18] S. Iwata and M. Shigeno. Conjugate scaling algorithm for Fenchel-type duality in discrete convex optimization. SIAM Journal on Optimization, 13(1):204-211 2002 (cited on page 9).
11

---

[19] A. S. Kelso and V. P. Crawford. Job matching, coalition formation, and gross substitutes. Econometrica. 50(6):1483- 1504, 1982 (cited on pages 3).
[20] R. Kupfer, S. Qian, E. Balkanski, and Y. Singer. The adaptive complexity of maximizing a gross substitutes valuation. In Advances in Neural Information Processing Systems, volume 33, pages 19817-19827. Curran Associates, Inc., 2020 (cited on page 1).
[21] B. Kveton, Z. Wen, A. Ashkan, and C. Szepesvari. Combinatorial cascading bandits. In Advances Neural Information Processing Systems, volume 28, pages 1450-1458. Curran Associates, Inc., 2015 (cited on page 2).
[22] B. Kveton, Z. Wen, A. Ashkan, and C. Szepesvari. Tight regret bounds for stochastic com- binatorial cani-bandits. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, volume 38, pages 535-543 PMLR, 2015 (cited page 10).
[23] T. Lattimore. Bandit convex optimisation. arXiv:2402.065: 06535, 2024 (cited on page 4).
[24] T. Lattimore and C. Szepesvári. Bandit Algorithms. Cambridge University Press, 2020 (cited on pages 2, 6, 14, 15).
[25] B. Lehmann, D. Lehmann, and N. Nisan. Combinatorial auctions with decreasing marginal utilities. Games and Economic Behavior, 55(2):270-29) 296, 2006 (cited on pages 1, 3).
[26] N. Littlestone and M. K. Warmuth. The weighted majority algorithm. Information and Compu- tation, 222-21, 1994 (cited on page 7).
[27] L. H. Loomis. On a theorem of von Neumann. Proceedings of the National Academy of Sciences of the United States of Americu, 32(8):213-215, 1946 (cited page 15).
[28] T. Lykouris, V. Mirrokni, and R. Paes Leme. Stochastic bandits robust to adversarial corrup- tions. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing. pages 114-122. ACM, 2018 (cited on page 9).
[29] N. Merlis and S. Mannor. Tight lower bounds for combinatorial multi-armed bandits. In Proceedings of the 33rd Annual Conference on Learning T'heory, volume 125, pages 2830-
2857. PMLR, 2020 (cited on page 2).
[30] S. Moriguchi, A. Shioura, and N. Tsuchimura. M-convex function minimization by continu- ous relaxation approach: proximity theorem and algorithm. SIAM Journal Optimization,
21 (3)66333-666 2011 (cited on pages 1, 3).
[31] R. Morwani and P. Raghavan. Randomized Algorithmss. Cambridge University Press, 1995 (cited on page 15).
[32] K. Murota. Discrete Convex Analysis, volume 10. SIAM, 2003 (cited on pages 1-3, 8, 9).
[33] K. Murota and A. Shioura. M-convex function on generalized polymatroid. Mathematics of Operations Research, (4(199955105 1999 (cited on pages 3-5, 14).
[34] G. Nemhauser and L. A. Wolsey. Best algorithms for approximating the maximum of a submodular set function. Mathematics of Operations Research, 377):177-188, 1978 (cited on page 1).
[35] G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for maxi- mizing submodular set functions Mathematical Programming, 14(1):265-294, 1978 (cited on page 1).
[36] R. Niazadeh, N. Golrezaei, J. R. Wang, F. Susan, and A. Badanidiyuru. Online learning via offline greedy algorithms: applications in market design and optimization. In Proceedings of the 22nd ACM Conference on Economics and Computation, pages 737-738. ACM, 2021 (cited
on pages 2, 5, 10, 14).
[37] G. Nie, M. Agarwal, A. K. Umrawal, V. Aggarwal, and C. John Quinn. An explore-then- commit algorithm for submodular maximization under full-bandit feedback. In Proceedings of the 38th Conference on Uncertainty in Artificial Intellignnee volume 180, pages 1541-1551. PMLR, 2022 (cited pages 2, 5, 14).
[38] G. Nic, offline algorithms to solve combinatorial multi-armed bandit problems with bandit feedback. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages
Nadew, Y. Zhu, V. Aggarwal, and C. Quinn. A framework for adapting
PMLR, 2023 (cited on pages 2, 5, 14).
[39] T. Oki and S. Sakaue. Faster discrete convex function minimization with predictions: the M- convex case. In Advances in Neural Information Processing Systems, volume 36, pages 68576- 68588. Curran Associates, Inc., 2023 (cited on page 1).
12

---

[40] S. Pasteris. A. Rumi, F. Vitale, and N. Gesa-Bianche Sum max submodular bandits. In Proceed- ings of the 27th International Conference on Artificial Intelligence and Statistics, volume 238, PMIR, 2024 (cited on page 2).
pages
[41] I. Rejwan and Y. Mansour. Top-k combinatorial bandits with full-bandit feedback. In Pro- ceedings of the st International Conference on Algorithmic Learning Theory, volume 117, pages 752-776. PMLR, 2020 (cited pages 2, 10).
[42] T. Roughgarden. Beyond the Worst-Case Analysis of Algorithms. Cambridge University Press, 2021 (cited on page 9).
[43] T. Roughgarden and J. R. Wang- An optimal learning algorithm for online unconstrained submodular maximization. In Proceedings of the 31st Annual Conference on Learning Theory, volume 75, pages 1307-1325. PMLR, 2018 (cited on pages 2, 10).
[44] A. Schrijver. Combinatorial Optimization, volume 24. Springer, 2003 (cited on page 8).
[45] L. S. Shapley. Complements and substitutes in the opttmal assignment problem. Naval Re-
search Logistics Quarterly, 9(1):45 48, 1962 (cited on page 3).
[46] A. Shioura. Fast scaling algorithms for M-convex function minimization with application to the resource allocation problem. Discrete Applied Mathematics, 134(1):303-3 -166, 2004 (cited pages 3).
[47] A. Shioura. M-convex function minimization under LI-distance constraint and its application to dock reallocation in bike-sharing system. Mathematics of Operations Research, 47(2):1566 1611, 2022 (cited on page 2).
[48] T. Soma. Submodular and Sparse Optimization Methods for Machine Learning and Communi- cation. PhD thesis, The University of Tokyo, 2016 (cited page 1).
[49] M. Streeter and D. Golovin. An online algorithm for maximizing submodular functions. In Advances in Neural Information Processing Systems, volume 21, pages 1577-1584. Curran Associates, Inc., 2008 (cited on pages 2, 5, 14).
[50] Tajdini, L. Jain, and K. Jamieson. Nearly minimax optimal submodular maximization with bandit feedback. arXiv:2310.1844
2023. To appear in NeurlPS 2024. (cited on pages 10, 11). [51] Z. Wan, J. Zhang, W. Chen, X. Sun, and Z. Zhang. Bandit multi-lineat DR-submodular maximization and its applications on adversarial submodular handits. In Proceedings of the 40th International Conference Machine Learning, volume 202, pages 35491-35524. PMLR, 2023 (cited on page 2).
[52] A. C.-C. Yao. Probabilistic computations: toward a unified measure of complexity. In Pro- ceedings of the 18th Annual Symposium on Foundations of Computer Science, pages 222-227. IEFE, 1977 (cited on pages 8, 15).
[53] H. Zhang, Z. Zheng, and J. Lavaei. Stochastic L'-convex function minimization. In Advances in Neural Information Processing Systems, volume 34, pages 13004-1301 Curran Associates, Inc., 2021 (cited page 2).
[54] M. Zhang, L. Chen, H. Hassani, and A. Karbasi. Online continuous submodular maximization: from full-information to bandit feedback. In Advances in Neural Information Processing Systems, volume 32, pages 9210-9221. Curran Associates, Inc., 2019 (cited on page 2).
[55] Q. Zhang, Z. Deng, Z. Chen, K. Zhou, H. Hu, and Yang. Online learning for non-monoton DR-submodular maximization: from full information to bandit feedback. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, volume 206, pages 3515 3537. PMLR, 2023 (cited page 2).
13

---

A Differences of Theorem 3.1 from robustness results in the submodular case
The basic idea of analyzing the robustness is inspired by similar approaches used in online submodular function maximization [49, 14, 36, 37, 38, 11]. However, our Theorem 3.1 for the M4-coneave case is fundamentally different from those for the submodular case.
At a high level, an evident difference lies in the comparator in the guarantees. Specifically, we need to bound the suboptimality compared to the optimal value in the Mi-concave case, while the comparator an approximate value in the submodular case.
At a more technical level, we need to work on the solution space in the MC-concave case, while the proof for the submodular case follows from analyzing objective values directly. Let overview the standard technique for the case of monotone submodular function maximization under the cardinality constraint, which is the most relevant our case due the similarity in the algorithmic procedures. In this case. a key argument is that in each iteration, the marginal increase in the objective value lower bounded by a 1/K fraction of that gained by adding an optimal solution, minus the local error. That is, regarding f: 1) - R as submodular set function, the submodularity implies f(zx) f 2 K ( V -==)) maximum. Consequently, by rearranging terms in the same way as the proof of the (1 - 1/e)- approximation, one can confirm that local errors accumulate only additively over K iterations. In this way, the robustness property directly follows from incorporating the effect of local errors into the inequality for deriving the (1 1/)e-approximaaiio in the submodular case. By contrast, in proof of Theorem 3.1 for the Me-concave case, we need to look at the solution space to ensure that the local update by ig with small crr(is 1x-1) does not deviate much from Vk-1, as highlighted in (3) (and this also differs from the original proof without errors [33]). After establishing this, can obtain the theorem by induction by virtue of the non non-trivial design of V%s (k = 0, O...., K), which satisfies * E Vo and VK- {œk).
err(ik Æx-1), where V is the element-wise
B MOSS for pure exploration in stochastic multi-armed bandit
We overview the MOSS-bort based pure-exploration algorithm used in Section 4. For more details, see Lattimore and Szepesvári [24, Chapters 9 and 33].
Let L[A} take if A is true and 0 otherwise, and let log (x) = log max I1 Given a stochastic multi-armed bandit instance with M arms and 7" rounds, we consider an algorithm that randomly selects arms A' ...... AT
A'.....AT' E {l...., .M. Fort 1...., T", let Y* be random variable representing the learner's s reward in the round, (t) = E. T{A' = i} the number of times the ith arm is selected up to round i, and (1) = (.( Σ. I(A* = Y) the empirical mean reward of the ith arm up to round t. Given these definitions, the MOSS algorithm can be described as in Algorithm 2.
Algorithm 2 MOSS
Input: Bandit instance with M arms and T rounds E (1...., M) during the first M rounds 2: for ( = M I,...,T"'
I: Choose each
3: Choose A = arg max,,e(I..... Po(t - 1) + V
4 log+ T" 11(111) (+i((-1)
Let a- ....,, T" denote the realization of A ,.....,. AT", respectively, after running the MOSS algo- rithm. Then, we set the final output to i E (1....,M, M) with probability + Ei= Ifa' = =t}. This procedure gives an M
regret algorithm, as stated in Proposition 4.1.
Proposition 4.1 (Lattimore and Szepesvári [24, Corollary 33.3]). Consider a stochastic multi-armer bandit instance with M arms and T" rounds, where T" M. Assume that the reward of the ith arm in the Ith round, denoted by Y satisfies the following conditions: μú; == E[Y;] E [0, 1] and Y μ.; is -sub-Gaussian. Then, there is an algorithm that, after pulling arms T" times, randomly returns
i {1...., M) with p Elua] D(/M/T), where jt" = max(H1,..... pm }.
14

---

max e. Proof. Since the suboptimality of the ith arm, defined by μz -4, is at most for all i E {1, M). the MOSS algorithm enjoys cumulative regret bound of Regr = T" j - E - Mat <
39 + M (see Lattimore and Szepesvári [24, Theorem 9.1]). Consider setting the final output to E f1....,MI } with probability 7h. Ei=1 I(a' = i), where a' denote the realization of A Then, holds that μ² Empplt]] = Regr./ /T" (see Lattimore and Szepesvári [24, Proposition 33.2)). The right-hand side is at most (39 + +M//T x 40/"/ completing the proof.
C Proof of Proposition 5.3
Proposition 5.3 (Yao [52]). Let A be finite set of all possible deterministic learning algorithms that run in polynomial time per round and F1. a finite set of sequences of M Mc-concae functions, Rogr (a. f1:T) be the cumulative regret deterministic learner a E A achieves on f f'...., fT. Let
sequence fl.T" (f ..... "....."T' fr) Fl.T. Then, for any polynomial-tinn time randomized learner A and any distribution on Fi. holds that
max (A, ((T)) f1 E EFITT m B (a, f(T)) :aE::aE.. AEEA..
Proof. We use the proof idea that of Yao's principle (see, e.g., Motwani and Raghavan [31, Section 2.2]). First, note that any polynomial- -imiil--immeeeeee randomized learner can be viewed as a polynomial- time deterministic learner with access random tape. Thus, can take A be chosen according to some distribution on the family, A, all possible polynomial deterministic learners.
Consider an |.| x |F1: matrix M, whose entry corresponding row E A and column far E Fll is Regz-(a, (a, f..)) For any polynomial-time randomized learner and any distribution on FIT it holds that
max E (Reg (A, (1)] f E FF min max y' Me ena
en
min E Eu [Regr(a, f(:))] A),
where y' and o' denote probability vectors on and Fil respectively, and e and e, denote the standard unit vectors of f1: and a, respectively. The equality is due to Loomis' theorem [27]. n
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clarified our scope (no-regret Mi-concave function maximization) and contributions (stochastic bandit algorithms and NP-hardness of the adversarial setting).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Section 6 discusses limitations and possible directions for overcoming them.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
Assumptions are clearly stated in Section 2.2 and problem-setiee paragraphs in Sections 4 and 5. All theoretical results are followed by complete proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [NA]
Justification:
This paper does not include experiments.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer:
[NA]
Justification:
This paper does not include experiments requiring code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (ittps//nipps.c/ppblic Buides/CodSSubbissionnooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (htips: wetpdieee/oeeeuueiiiiiiioooooooo
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA]
Justification: This paper does not include experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: This paper does not include experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
18

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification:
This paper does not include experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
This paper conforms, in every respect, with the NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This paper is dedicated to theoretical results. While there are potential societal consequences of our work, we feel none of them must be specifically discussed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: This paper does not existing assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswithcode, www.i//oodeoooodatsseeeeeee com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "hIdoOaAGuO", "reviewer": "Reviewer_yBbo", "summary": "This paper studies the online optimization problem with $M^\\natural$-concave function, which has many real-world implications such as maximum-flow on bipartite graphs, gross substitute valuation, resource allocation and bandit problem. $M^\\natural$-concave function forms the fundamental basis of discrete concave analysis. In the stochastic bandit setting, they present algorithms with $O(T^-1/2)$ simple regret and $O(T^2/3)$ cumulative regret. These results leverage the robustness of the greedy algorithm to local errors, a significant technical contribution of the paper.\n\nAdditionally, the author also presents the result in the adversarial online learning setting, the paper proves that achieving sub-linear regret is NP-hard, even with full-information feedback, which establishes a distinct difference from the offline setting.", "questions": "I am more curious about the conclusion in the bandit example.\n\n1. In theorem 4.2, does $sReg_T$ is equivalent to the error in the Best-arm identification problem?\n\n2. Does the bit-O term in theorem 4.2 and 4.3 hide the logarithmic term?\n\n3. The theorem 4.3 claims that the proposed algorithm for stochastic bandit achieves cumulative regret as $O(K N^{1/2} T^{2/3}$. While according to [1], the tight lower bound of a consistent bandit algorithm is $\\Theta(K^{1/2}T^{1/2})$. What kind of factor do you think causes the gap between your upper and lower bound?\n\n[1] Auer, Peter, et al. \"The nonstochastic multiarmed bandit problem.\" *SIAM journal on computing* 32.1 (2002): 48-77.", "limitations": "Please refer to Questions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper focuses on $M^\\natural$-concave functions, which is a crucial class in discrete convex analysis with wide applications, thus benefiting the analysis on a large variety community.\nThe author brings both stochastic bandit and adversarial full-information settings. \nEspecially for the adversarial setting, at least linear regret justification will guide us to avoid building sub-linear regret algorithms.\nThis theorem provides a profound understanding of the challenges and encourages us to think more potential alternative model assumptions and solutions for optimizing $M^\\natural$-concave functions in the online scenarios.", "weaknesses": "No obvious weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the reviewer's insightful comments and positive evaluation. We address the questions below.\n\n> 1. In theorem 4.2, does $sReg_T$ is equivalent to the error in the Best-arm identification problem?\n\nOur ${\\rm sReg}_T$ is essentially the same as the error measure in the best-arm identification, like the one used by Rejwan and Mansour [41]. To clarify the connection to Rejwan and Mansour [41] mentioned in the conclusion section, there is a bit of difference: they consider the $(\\varepsilon, \\delta)$-PAC guarantee, i.e., the high probability bound on the expected error, whereas our ${\\rm sReg}_T$ focuses solely on the expected error. Nevertheless, their sample complexity lower bound [41, Theorem 10] of $T\\gtrsim\\Omega(N/\\varepsilon^2)$ applies to any $K\\le N/2$ and $\\delta \\in (0, 1/2)$, and hence Markov's inequality implies that our bound of ${\\rm sReg}_T = O(K^{3/2}\\sqrt{N/T})$ in Theorem 4.2 is tight with respect to $N$ and $T$ when $K=O(1)$.\n\n> 2. Does the bit-O term in theorem 4.2 and 4.3 hide the logarithmic term?\n\nThe big-O notation in Theorems 4.2 and 4.3 does not hide any logarithmic terms. Our algorithms in Section 4 employ the minimax optimal strategy (MOSS) (e.g., Lattimore and Szepesvári [24, Chapter 9]), whose regret bound is free of $\\log T$ factors that are typically present in the regret bounds of standard UCB-type algorithms. Consequently, our regret bounds do not involve logarithmic factors. To add further, this strategy could slightly improve previous explore-then-commit approaches for stochastic bandit submodular maximization [37, 38], which contain $\\log T$ factors.\n\n> 3. The theorem 4.3 claims that the proposed algorithm for stochastic bandit achieves cumulative regret as $O(KN^{1/2}T^{2/3})$. While according to [1], the tight lower bound of a consistent bandit algorithm is $\\Theta(K^{1/2}T^{1/2})$. What kind of factor do you think causes the gap between your upper and lower bound?\n>\n> [1] Auer, Peter, et al. \"The nonstochastic multiarmed bandit problem.\" SIAM journal on computing 32.1 (2002): 48-77.\n\nWe greatly value this insightful question. In our opinion, the primary challenge stems from the presence of exponentially many arms: even if we restrict the domain to $\\\\{0, 1\\\\}^V$, there are about $N^K$ arms. This leads to the following dilemma:\n\n1. If we aim to achieve $O(\\sqrt{T})$ regret, we may naively apply a UCB-type algorithm to the multi-armed bandit problem with about $N^K$ arms. However, this approach results in $O(\\sqrt{N^K T})$ regret, and the algorithm requires exponential time per round in general. \n2. Alternatively, we can design more efficient strategies based on offline algorithms. In our case, we have designed a no-regret strategy by executing a UCB-type algorithm in each iteration in the greedy algorithm, as described in Section 4. This method employs $K$ no-regret learners; however, only a single bandit feedback is available in each round. Due to this limited feedback, we need sufficient exploration across $K$ iterations in the greedy algorithm, as the explore-then-commit strategy does. Consequently, we incur $T^{2/3}$ regret as in Theorem 4.3.\n\nFurthermore, as detailed in the above [global response](https://openreview.net/forum?id=NnoAj91HZX&noteId=RjP8EjP159), a recent lower-bound result by Tajdini et al. (2023) suggests that we cannot do significantly better than the above two strategies in the case of submodular maximization. While there might be a slight possibility of achieving better regret by focusing on the $\\text{M}^\\natural$-concave case, we conjecture that we cannot achieve $\\sqrt{T}$ regret that depends only polynomially on $K$ and $N$.\n\nWe hope our responses have adequately addressed the reviewer's questions. Please do not hesitate to reach out during the discussion period if there are any further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AI3QWeVCd3", "reviewer": "Reviewer_ugrH", "summary": "This paper studies the online version of M-concave function maximization. With the help of a new lemma that bounds local errors for greedy algorithms, the authors derive an efficient $T^{-1/2}$ simple regret algorithm together with a $T^{2/3}$ regret algorithm for stochastic settings. Moreover, the authors show an interesting computational hardness result for adversarial settings with full information.", "questions": "1. What is the main technical challenge in getting $\\sqrt{K}$ regret for stochastic settings? Specifically, what is the difficulty when implementing the UCB-type exploration on the fly rather than doing an explore-then-commit algorithm?", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper studies a new online learning problem and proposes an efficient algorithm with sublinear regret for stochastic settings.\n2. A new computational hardness result for adversarial settings with full information is given, which introduces a new understanding of online optimization.", "weaknesses": "1. The regret bound for stochastic setting is not tight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate the reviewer's effort in reviewing our paper and providing thoughtful comments. Below is our answer to the question. \n\n> Questions:\n> \n> 1. What is the main technical challenge in getting $\\sqrt{K}$ regret for stochastic settings? Specifically, what is the difficulty when implementing the UCB-type exploration on the fly rather than doing an explore-then-commit algorithm?\n\nWe appreciate this insightful question. Our response assumes the question refers to $\\sqrt{T}$ regret (not $\\sqrt{K}$). Please let us know if this is not your intended question!\n\nThe fundamental difficulty lies in the fact that there are exponentially many arms. Indeed, if the domain is restricted to $\\\\{0, 1\\\\}^V$, we can achieve a $\\sqrt{T}$ regret bound by regarding all subsets of size up to $K$ as arms and naively applying a UCB-type algorithm to this multi-armed bandit problem. However, this approach leads to a regret bound of $O\\left(\\sqrt{\\binom{N}{K}T}\\right)$, which depends exponentially on $K$, and the resulting algorithm requires exponential time per round in general. By contrast, the explore-then-commit strategy can achieve the $O(KN^{1/3}T^{2/3})$ regret bound as in Theorem 4.3, albeit with the worse dependence on $T$, and the algorithm runs in polynomial time per round.\n\nAs detailed in the [global response](https://openreview.net/forum?id=NnoAj91HZX&noteId=RjP8EjP159), a recent lower bound by Tajdini et al. (2023) implies that, in the submodular maximization case, we cannot do significantly better than the naive UCB applied to $\\binom{N}{K}$ arms and the explore-then-commit strategy. While there is a slight possibility of achieving better regret bounds by focusing on the $\\text{M}^\\natural$-concave case, we conjecture that a similar lower bound likely exists and that our $O(KN^{1/3}T^{2/3})$ regret is essentially tight unless we allow the regret bound to depend exponentially on $K$.\n\nWe hope this explanation helps the reviewer better understand the challenge in stochastic bandit $\\text{M}^\\natural$-concave maximization and casts our results in a more favorable light. Please do not hesitate to ask further questions during the discussion period.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fFTNKgy2DH", "reviewer": "Reviewer_opkE", "summary": "The authors consider online $M^\\sharp$-concave optimization, similar to problems like online convex optimization and online DR-submodular optimization.  $M^\\sharp$-concave function classes include resource allocation, valuation, and flow problems, and unlike DR-submodular functions can be exactly optimized (at least under a cardinality constraint) by a simple greedy algorithm.  The authors show standard online adaptation of the greedy algorithm analogous to similar work for submodular functions, though prove the robustness of the offline algorithm to oracle value errors in a manner distinct from previous robustness analyses (such as for submodular functions).  The authors also show that unlike OCO and online DR-submodular optimization, the adversarial setting is fundamentally harder than the stochastic setting – one cannot (with poly-time per round complexity) achieve sublinear regret.", "questions": "1. What is known about the hardness of offline optimization for maximizing sums of (possibly non-monotone) $M^\\sharp$-concave functions? \nFor the adversarial setting, the authors use an interesting construction based on matroid intersection to prove hardness for no-approximation sublinear regret (with poly-time per round computation).  But I wonder if it could have been reached in a more straightforward manner.  Namely, for the stochastic setting, the regret is based on a single $M^\\sharp$-concave function $f^*$ for which in the offline setting the greedy algorithm can find the optimal solution. In the adversarial setting, however, the regret is based on a sum of $M^\\sharp$-concave functions and in line 100 we are told that in general the class of $M^\\sharp$ functions is not closed under addition.", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors consider an interesting class of functions for online optimization, which are related to, but much easier to optimize than, (DR-)submodular functions.  They show an (potentially quite) interesting dichotomy in hardness between the online stochastic setting and the online adversarial setting, unlike related classes like online convex optimization or online DR-submodular optimization.  For the online stochastic setting they get analogous results to prior work for (DR-)submodular, but show through a matroid-intersection construction that one cannot obtain sublinear regret.  \n\n2. The robustness analysis is distinct from that used in submodular bandit papers.  As described, the standard proof(s) for offline cardinality-constrained $M^\\sharp$-concave maximization does not have a structure that lends itself to accounting for the function value impact in sequential mistakes by the greedy algorithm.  Thus, the authors develop a new proof that clearly shows such additive accumulation, and subsequently permits almost direct adaptation to the online stochastic setting.  (one minor note - line 191 “ours is different from them in that it involves no approximation factors” that aspect I don’t see in and of itself as a meaningful distinction since the offline problem is not NP-hard; from my reading it is how the offline proof is structured as to whether it can be easily modified to account for value oracle errors that matters.) \n\n3.  I found the paper overall well written, with good organization, logical flow, discussions, etc.", "weaknesses": "1. The stochastic setting algorithms and analysis adapting a greedy algorithm from the offline setting (including an ETC method getting $T^{2/3}$) are straightforward (once a robustness result is in hand), though the authors do acknowledge that in the main paper.  \n\n2. This is a somewhat minor point – since the need for a novel robustness analysis arises due to the offline proof of the greedy algorithm’s not already being in a manner that can be easily adapted to account for local errors (which is unlike the proof of $1-1/e$ for the greedy approximation algorithm for cardinality constrained submodular maximization, as noted in Appendix A), I think it would have been helpful to include the proof(s) for constrained $M^\\sharp$-concave maximization (specialized to cardinality constraint) so that this need would be more apparent, that no standard proof has a sequential exchange argument that would already lend itself more readily to analyzing robustness and that the robustness result used in the offline setting with exact value oracles clearly constitutes a distinct proof.  \n\n3. minor point – it is not clear to what extent the positive result in the stochastic setting can be generalized beyond a simple cardinality constraint.  \n\n4. minor point – there is no implementation or experiments for toy flow/valuation/resource allocation problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate the reviewer's dedication to reviewing our paper and providing many insightful comments.\n\nFirst, we would like to respond to some comments regarding weaknesses. \n\n> Weaknesses:\n> \n> 1. The stochastic setting algorithms and analysis adapting a greedy algorithm from the offline setting (including an ETC method getting $T^{-2/3}$) are straightforward (once a robustness result is in hand), though the authors do acknowledge that in the main paper.\n\nWe acknowledge the reviewer's point. Indeed, while our algorithm is straightforward, significantly improving our $O(T^{2/3})$ regret seems challenging unless we admit exponential dependence on $K$. Please refer to the above [global response](https://openreview.net/forum?id=NnoAj91HZX&noteId=RjP8EjP159) for details. \n\n> 2. This is a somewhat minor point – since the need for a novel robustness analysis arises due to the offline proof of the greedy algorithm’s not already being in a manner that can be easily adapted to account for local errors (which is unlike the proof of $1-1/e$ for the greedy approximation algorithm for cardinality constrained submodular maximization, as noted in Appendix A), I think it would have been helpful to include the proof(s) for constrained $M^\\sharp$-concave maximization (specialized to cardinality constraint) so that this need would be more apparent, that no standard proof has a sequential exchange argument that would already lend itself more readily to analyzing robustness and that the robustness result used in the offline setting with exact value oracles clearly constitutes a distinct proof.\n\nWe appreciate this valuable suggestion. The original proof by Murota and Shioura [33 Section 3], which is based on a convexity argument along the trajectory of an augmenting-type algorithm, is a quite different approach that does not readily align with the robustness analysis. To add further, we wish to highlight that our proof of Theorem 3.1, with $\\mathrm{err}(i_k \\mid x_{k-1}) = 0$, serves as an alternative proof of the optimality of the greedy algorithm for the offline $\\text{M}^\\natural$-concave maximization.\n\n> 3. minor point – it is not clear to what extent the positive result in the stochastic setting can be generalized beyond a simple cardinality constraint.\n\nThis is an interesting future direction. Indeed, in the offline setting, the greedy-type algorithm can solve more general $\\text{M}^\\natural$-concave maximization problems. When it comes to online/bandit settings, extending the robustness analysis akin to Theorem 3.1 will be a key point of investigation.\n\nNext, we would like to respond to the following question. \n\n> Questions:\n> \n> 1. What is known about the hardness of offline optimization for maximizing sums of (possibly non-monotone) $M^\\sharp$-concave functions? For the adversarial setting, the authors use an interesting construction based on matroid intersection to prove hardness for no-approximation sublinear regret (with poly-time per round computation). But I wonder if it could have been reached in a more straightforward manner. Namely, for the stochastic setting, the regret is based on a single $M^\\sharp$-concave function $f^*$ for which in the offline setting the greedy algorithm can find the optimal solution. In the adversarial setting, however, the regret is based on a sum of $M^\\sharp$-concave functions and in line 100 we are told that in general the class of $M^\\sharp$ functions is not closed under addition.\n\nIn the offline setting, maximizing the sum of more than two $\\text{M}^\\natural$-concave functions is NP-hard in general. This is recognized as folklore within the field of discrete convex analysis; those familiar with $\\text{M}^\\natural$-concavity would readily infer this NP-hardness due to the connection to the 3-matroid intersection, by encoding a base family of a matroid with an $\\text{M}^\\natural$-concave indicator function $f:\\\\{0, 1\\\\}^V \\to \\\\{0, -\\infty\\\\}$. To our knowledge, however, no explicit proof exists in the literature (although the NP-hardness is mentioned in a seminar material by Kazuo Murota, titled \"Discrete Convex Analysis,\" used in the 2015 Summer School at Hausdorff Institute of Mathematics). Our work would be the first to provide an explicit proof, although the NP-hardness itself is already widely recognized. \n\nMore importantly, our Lemma 5.5 implies a slightly stronger result: even if $\\text{M}^\\natural$-concave functions on $\\\\{0, 1\\\\}^V$ are restricted to take values in $[0, 1]$ (forbidden taking $-\\infty$), maximizing the sum of more than two $\\text{M}^\\natural$-concave functions remains NP-hard. This restriction is crucial to ensure that our NP-hardness result is meaningful because, if $f_t$ can take $-\\infty$, the learner who does not know $f_t$ a priori cannot achieve even a bounded regret, making the hardness result (Theorem 5.2) vacuous. The reduction from the 3-matroid intersection with $\\text{M}^\\natural$-concave functions that do not take $-\\infty$ has not been even recognized in the literature. Thus, our Lemma 5.5, despite being somewhat specific, provides a new crucial technical result. We believe that no significantly more straightforward approach could establish meaningful NP-hardness.\n\nWe hope this clarification has effectively addressed the reviewer's question and highlighted the value of our technical contributions. Please do not hesitate to reach out during the discussion period if further questions remain.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CRvKzSjiTE", "reviewer": "Reviewer_PdPM", "summary": "This paper considers online learning variants of ${\\rm M}^\\natural$-concave function maximization.  These types of function generalize maximum flows on graphs (where the variable is the vector of source values on each node), gross substitute valuations in economics, and have applications in resource allocation.\n\nThis paper has two main results:\n\n - A learning strategy with $O(K N^{1/3} T^{2/3})$-regret in the stochastic setting (here we can query a function at a given input and observe its output perturbed by mean 0, 1-subgaussian noise and we have $T$ rounds to perform these queries).  This is obtained by showing that the greedy algorithm is \"robust to local errors\", and is the key technical step for this result.  The regret bound follows by an application of known results for pure-exploration + the explore-then-commit paradigm.\n\n - NP-hardness for the adversarial setting.  Unless P=NP, it is impossible for any polynomial time learner to achieve sub-linear regret in the adversarial setting.  This is done via a clever reduction from the 3-matroid-intersection problem.", "questions": "Please clarify the relationship between ${\\rm M}^{\\natural}$ concave maximization and submodular function maximization.  I found it hard to understand the relationship between these as it pertains to the results in this paper and the discussion therein.  For example, the conclusion states that if the domain is restricted to $\\{0,1\\}^V$ then we get online submodular function maximization as a special case.  Here, the offline problem is NP-hard so I'm not sure how to interpret the fact that we can get sublinear-regret with respect to $\\mathbb{E}[f^*(x^*)]$ with a polynomial-time algorithm.\n\nSome very minor comments:\n- ${\\rm M}^\\natural$-convexity is used in section 5, but not defined in the paper", "limitations": "The authors have adequately addressed and discussed limitations of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- This paper considers online learning for a very general class of functions, which makes the results potentially widely applicable.\n - The \"robust to local errors\" Theorem for $\\rm{M}^\\natural$-concave functions may be of independent interest.\n - The proofs are well written and easy to follow, which is challenging to do for the very technical topic at hand.  \n - The NP-hardness result is interesting and shows a sharp difference between the stochastic and adversarial case for this problem.", "weaknesses": "- For the upper bound, the techniques are fairly straightforward once we have Theorem 3.1.  I am wondering if more can be done, e.g. regret that is sublinear in $K$, improving the bound for $T$ to $O(\\sqrt{T})$, or sublinear approximate regret in the adversarial setting.  These suggestions have been pointed out by the authors themselves, but they are relevant questions.\n- The stochastic online setting for this problem could be better motivated than the paragraph in lines 122-126.  More concrete examples would be helpful to clarify the applicability of these results beyond the nice mathematics.\n- No experimental evaluation.  While this paper makes a nice theoretical contribution, it could be helpful to demonstrate that the theory is applicable to a problem that prior methods were not suitable for.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful to the reviewer for the thoughtful comments and positive evaluation. First, we would like to address the following comment regarding weakness.  \n\n> Weaknesses:\n> - For the upper bound, the techniques are fairly straightforward once we have Theorem 3.1. I am wondering if more can be done, e.g. regret that is sublinear in $K$, improving the bound for $T$ to $O(\\sqrt{T})$, or sublinear approximate regret in the adversarial setting. These suggestions have been pointed out by the authors themselves, but they are relevant questions.\n\nWe appreciate the feedback and insights provided. As regards improving the $O(KN^{1/3}T^{2/3})$ regret bound in Theorem 4.3 for the stochastic bandit setting, we have discovered a recent preprint by Tajdini et al. (2023) that presents a relevant lower bound. In the submodular case, their lower bound implies that significant improvements to the $O(KN^{1/3}T^{2/3})$ regret bound are impossible without admitting exponential factors in $K$. While their result does not directly apply to our $\\text{M}^\\natural$-concave setting, we conjecture that a similar lower bound exists and that our $O(KN^{1/3}T^{2/3})$ regret bound is tight unless we admit exponential factors in $K$. For more details, please refer to the above [global response](https://openreview.net/forum?id=NnoAj91HZX&noteId=RjP8EjP159).\n\nNext, we would like to answer the following question. \n\n> Questions:\n> \n> Please clarify the relationship between $\\text{M}^\\natural$-concave maximization and submodular function maximization. I found it hard to understand the relationship between these as it pertains to the results in this paper and the discussion therein. For example, the conclusion states that if the domain is restricted to ${0, 1}^V$ then we get online submodular function maximization as a special case. Here, the offline problem is NP-hard so I'm not sure how to interpret the fact that we can get sublinear-regret with respect to $\\mathbb{E}[f^*(x^*)]$ with a polynomial-time algorithm.\n\nWe wish to clarify that the correct relationship we intended to describe in the conclusion is: \n$$\n\\text{class of $\\text{M}^\\natural$-concave functions on $\\\\{0, 1\\\\}^V$}\\subseteq \\text{class of subomdular functions}, \n$$\nwhich is the opposite of the relationship mentioned in the reviewer's comment. We apologize for any confusion and appreciate it if you could notify us of any incorrect expressions in our manuscript. \n\nImportantly, $\\text{M}^\\natural$-concave maximization on $\\\\{0, 1\\\\}^V$ is a special case that the greedy algorithm can *exactly* solve in polynomial time. This has been established by Murota and Shioura [33] and can also be derived from our Theorem 3.1 with $\\mathrm{err}(i_k \\mid x_{k-1}) = 0$. Since submodular maximization forms a larger problem class, sublinear regret with respect to $\\mathbb{E}[f^*(x^*)]$ for stochastic bandit $\\text{M}^\\natural$-concave maximization does *not* imply exact algorithms for submodular maximization. Therefore, our sublinear regret bounds in Section 4 do not contradict any known results in $\\text{M}^\\natural$-concave or submodular maximization.\n\nWe hope this clarification has effectively addressed the reviewer's concerns. Please do not hesitate to let us know if further questions remain. \n\n> Some very minor comments:\n> - $\\text{M}^\\natural$-convexity is used in section 5, but not defined in the paper\n\nWe thank the reviewer for pointing this out. We will clarify that the $\\text{M}^\\natural$-convexity is defined by the negative of $\\text{M}^\\natural$-concavity, analogous to the standard convexity--concavity relationship.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## **Global response: a discussion on the tightness of the $O(KN^{1/3}T^{2/3})$ regret bound in Theorem 4.3**
We sincerely thank all reviewers for their efforts in reviewing our paper and providing invaluable feedback. We are pleased to see that all reviewers have positively evaluated our work.

Upon reviewing the comments, we noted that multiple reviewers are curious about whether our  $O(KN^{1/3}T^{2/3})$ regret upper bound in Theorem 4.3 for stochastic bandit $\text{M}^\natural$-concave maximization is improvable. We revisited this issue and discovered an interesting recent preprint by Tajdini et al. (2023), titled "Minimax Optimal Submodular Optimization with Bandit Feedback." This paper studies stochastic bandit monotone submodular maximization with a ground set of size $N$ and a cardinality constraint of $K$. They showed that there is a lower bound of 
$$ \Omega\left(\min_{i\le K}\ (K-i)N^{1/3}T^{2/3} + \sqrt{\binom{N-K}{i}T}\right) $$
on *robust greedy regret*, which intuitively compares the learner's actual reward with that of the output, denoted by $S_{\rm gr}$, of the greedy algorithm applied to the underlying true function. This lower bound implies that $O(KN^{1/3}T^{2/3})$ robust greedy regret is inevitable when $T$ is small in the submodular case; furthermore, the explore-then-commit strategy can achieve this regret bound. The $\sqrt{\binom{N-K}{i}T}$ term can be interpreted as the regret bound achieved by regrading all $\binom{N-K}{i}$ subsets as arms and using a UCB-type algorithm. Thus, the lower bound represents the best mix of the two regret terms achieved by the explore-then-commit strategy and the UCB applied to exponentially many arms. 

Currently, we have confirmed that the proof for establishing the lower bound by Tajdini et al. (2023) does not directly apply to our stochastic bandit $\text{M}^\natural$-concave maximization problem. Specifically, the function used by Tajdini et al. (2023) for obtaining the lower bound is submodular but not $\text{M}^\natural$-concave. Nevertheless, the situations of Tajdini et al. (2023) and our problem in Section 4, with the domain restricted to $\\{0, 1\\}^V$, are remarkably similar: 

1. Since the greedy algorithm applied to the unknown true $\text{M}^\natural$-concave function $f^*$ can find an optimal solution $x^*$, we have $x^* = S_{\rm gr}$ and hence the notion of robust greedy regret in Tajdini et al. (2023) essentially coincides with the standard regret in our case. 
2. The explore-then-commit strategy and the UCB with exponentially many arms also achieve the $O(KN^{1/3}T^{2/3})$ and $O\left( \sqrt{\binom{N-K}{i}T} \right)$ regret bounds, respectively, in the $\text{M}^\natural$-concave case, where the former is the very result our Theorem 4.3 states. 

Considering these facts, it is highly plausible that we can construct a hard instance of stochastic bandit $\text{M}^\natural$-concave maximization similar to Tajdini et al. (2023) to establish the same regret lower bound. Thus, we conjecture that our $O(KN^{1/3}T^{2/3})$ regret bound in Theorem 4.3 is tight in $K$, $N$, and $T$ if we want to avoid exponential factors, such as $N^K$, regardless of the value of $T$, which is a common desideratum in the context of combinatorial bandits. We will include an extensive discussion of this open problem in our revised manuscript. 

We deeply thank dedicated reviewers, whose insightful comments have brought our attention to this interesting connection to Tajdini et al. (2023). We hope the information provided above helps reviewers understand that our $O(KN^{1/3}T^{2/3})$ regret bound in Theorem 4.3 may be close to being tight.

Meta-review: This paper studied the online bandit problem of maximizing M-concave functions. Through a new error decomposition, it established regret guarantees in the stochastic setting. More noteworthy was the hardness of regret minimization in the adversarial case, which the reviewers also appreciated. Happy to recommend this paper for acceptance.
Decision: Accept (poster)