Title: Optimal, Efficient and Practical Algorithms for Assortment Optimization
Venue: NeurIPS 2024
Accepted: no
Keywords: Active online assortment optimization

Preference feedback

Subsetwise utility maximization

Assortment selection algorithms

Plackett Luce model

Regret minimization

Pairwise Rank-Breaking

Concentration guarantee

Practical algorithms

Empirical evaluations
Abstract: We address the problem of active online assortment optimization problem with preference feedback, which is a framework for modeling user choices and subsetwise utility maximization. The framework is useful in various real-world applications including ad placement, online retail, recommender systems, and fine-tuning language models, amongst many others. The problem, although has been studied in the past, lacks an intuitive and practical solution approach with simultaneously efficient algorithm and optimal regret guarantee. E.g., popularly used assortment selection algorithms often require the presence of a ``strong reference" which is always included in the choice sets, further they are also designed to offer the same assortments repeatedly until the reference item gets selected---all such requirements are quite unrealistic for practical applications. In this paper, we designed efficient algorithms for the problem of regret minimization in assortment selection with \emph{Plackett Luce} (PL) based user choices. We designed a novel concentration guarantee for estimating the score parameters of the PL model using `\emph{Pairwise Rank-Breaking}', which builds the foundation of our proposed algorithms. Moreover, our methods are practical, provably optimal, and devoid of the aforementioned limitations of the existing methods.

1

Abstract

We address the problem of active online assortment optimization problem with preference feedback, which is a framework for modeling user choices and subsetwise utility maximization. The framework is useful in various real-world applications including ad placement, online retail, recommender systems, and fine-tuning language models, amongst many others. The problem, although has been studied in the past, lacks an intuitive and practical solution approach with simultaneously efficient algorithm and optimal regret guarantee. E.g., popularly used assortment selection algorithms often require the presence of a 'strong reference' which is always included in the choice sets, further they are also designed to offer the same assortments repeatedly until the reference item gets selected-all such requirements are quite unrealistic for practical applications. In this paper, we designed efficient algorithms for the problem of regret minimization in assortment selection with Plackett Luce (PL) based user choices. We designed a novel concentration guarantee for estimating the score parameters of the PL model using ' Pairwise Rank-Breaking ', which builds the foundation of our proposed algorithms. Moreover, our methods are practical, provably optimal, and devoid of the aforementioned limitations of the existing methods. Empirical evaluations corroborate our findings and outperform the existing baselines.

1 Introduction

Studies have shown that it is often easier, faster and less expensive to collect feedback on a 21 relative scale rather than asking ratings on an absolute scale. E.g., to understand the liking 22 for a given pair of items, say (A,B), it is easier for the users to answer preference-based 23 queries like: 'Do you prefer Item A over B?", rather than their absolute counterparts: 'How 24 much do you score items A and B in a scale of [0-10]?". Due to the widespread applicability 25 and ease of data collection with relative feedback, learning from preferences has gained much 26 popularity in the machine-learning community, especially the active learning literature which 27 has applications in Medical surveys, AI tutoring systems, Multi-player sports/games, or any 28 real-world systems that have ways to collect feedback in terms of preferences. The problem 29 is famously studied as the Dueling-Bandit (DB) problem in the active learning community 30 [41, 3, 45, 46, 44], which is an online learning framework for identifying a set of 'good' items 31 from a fixed decision-space (set of items) by querying preference feedback of actively chosen 32 item-pairs. Consequently, the generalization of Dueling-Bandits, with subset-wise preferences 33 has also been developed into an active field of research. For instance, applications like 34 Web search (e.g. Google, Bing, or even in some versions of ChatGPT), online shopping 35 (Amazon, App stores, Google Flights), recommender systems (e.g. Youtube, Netflix, Google 36 News/Maps, Spotify) typically involve users expressing preferences by choosing one result (or 37 a handful of results) from a subset of offered items and often the objective of the system is to 38

Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.

identify the 'most-profitable' subset to offer to their users. The problem, popularly termed 39 as 'Assortment Optimization' is studied in many interdisciplinary literature, e.g. Online 40 learning and bandits [10], Operations research [40, 2], Game theory [15], RLHF [20, 30], to 41 name a few. 42

43

44

45

46

47

48

49

50

51

52

53

Problem (Informal): Active Optimal Assortment (AOA) Active Assortment Optimization (a.k.a. Utility Maximization with Subset Choices) [13, 2, 23, 22] is an active learning framework for finding the 'optimal' profit-maximizing subset. Formally, assume we have a decision set of [ K ] := { 1 , 2 , . . . K } of K items, with each item being associated with the score (or utility) parameters θ := ( θ 1 , θ 2 , . . . , θ K ) (without loss of generality assume θ 1 ≥ θ 2 ≥ . . . ≥ θ K ≥ 0). At each round t = 1 , 2 , . . . , the learner or the algorithm gets to query an assortment (typically subsets containing up to m -items) S t ⊆ [ K ], upon which it gets to see some (noisy) relative preferences across the items in S t , typically generated according to an underlying Plackett-Luce (PL) choice model with parameters θ (1). Further, to allow the event where no items are selected, we also model a No-Choice (NC) item, indexed by item-0, with PL parameter θ 0 ∈ R + .

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

(Objective 1.) Topm : identify the topm item-set: { θ 1 , . . . , θ m } , for some m ∈ [1 , K ].

(Objective 2.) Wtd-Topm : A more general objective could also consider a weight (or price) r i ∈ R + associated with the item i ∈ [ K ], and the goal could be to identify the assortment (subset) with maximum weighted utility 1 , as detailed in Sec. 2.

Related Works and Limitations: As stated above, the problem of AOA is fundamental in many practical scenarios, and thus widely studied in multiple research areas, including Online ML/learning theory and operations research.

· In the Online ML literature, the problem is well-studied as Multi-Dueling Bandits [39, 14], or Battling Bandits [35, 34, 11], which is an extension of the famous Dueling Bandit problem [46, 45]. The main limitation of this line of work is the lack of practical objectives, which either aim to identify the 'best-item' 1(= arg max i ∈ [ K ] θ i ) within a PAC (probably approximately correct) framework [36, 16, 17, 31] or quantifying regret against the best items [35, 12]. Note the latter actually leads to the optimal subset choice of repeatedly selecting the optimal item, arg max i θ i , m times, i.e. (1 , 1 , . . . 1), which is unrealistic from the viewpoint of real-world system design. Selecting an assortment of distinct topm items (Topm -AOA) or maximum expected utility (Wtd-Topm -AOA) makes more sense.

· On the other hand, a similar line of the problem has been studied in operations research and dynamic assortment selection literature, where the goal is to offer a subset of items to the customers in order to maximize expected revenue. The problem has been studied under different user choice models, e.g. PL or Multinomial-Logit models [2], Mallows and mixture of Mallows [22], Markov chain-based choice models [23], single transition model [27] etc. While these works indeed consider a more practical objective of finding the best assortment (subset) with the highest expected utility for a regret minimization objective, (1) a major drawback in their approach lies in the algorithm design which requires to keep on querying the same set multiple times , e.g. [2, 29, 18, 1]. Such design techniques could be impractical to be deployed in real systems where users could easily get annoyed if the same items are shown again and again. For example, in ad-placement, music/movies/news/tweets/reels recommendations, offering the same assortment could increase user dissatisfaction and disengagement.

(2) The second major drawback of this line of work lies in the structural assumption of their underlying choice models which requires the existence of a reference/default item, that needs to be part of every assortment S t . This leads to assuming a No-Choice item, typically denoted as item-0, which is a default choice of any assortment S t . Further a stronger and more unrealistic assumption lies in the fact that they require to assume that the above pivot is stronger than the rest of the K items, i.e. θ 0 ≥ max i ∈ [ K ] θ i , i.e. the No-Choice (NC) action is the most likely outcome of any assortment S t . This is often unrealistic, e.g., during user interactions with language models, or online shopping, or Route recommendation in GPS navigation, a NC action is highly improbable. Consequently, such assumption limits the use in real-systems. In the existing literature [2, 28, 1, 24], such assumptions are primarily

adapted solely for theoretical needs, precisely for maintaining concentration bounds of the 92 PL parameters θ , and hence not well justified from a practical viewpoint. Some recent 93 developments also generalized the AOA problem to linear MNL scores to incorporate large 94 actions embedded in d -dimension [43, 42, 28], however, their approaches are either limited 95 to the above restrictions or suffer sub-optimal regret guarantees without those assumptions 96 (e.g. the regret bound of [28] is O ( d 3 / 2 √ T ) which is suboptimal by a d -factor). Considering 97 the above limitations of the AOA literature, we set to answer two questions: 98

99

100

Can we consider a general AOA model where the default item, like the NC item defined above, is not necessarily the strongest one, i.e. θ 0 ≥ max i ∈ [ K ] θ i ?

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

Can we design a practical and regret optimal algorithm for the AOA framework, without needing to play the same repetitive actions and yet converge to the optimal assortment?

Contributions We answer these questions in the affirmative and present best of all scenarios. We design practical algorithms on practical AOA framework with practical objectives-Unlike the existing approaches of the AOA, literature [2, 18], we do not have to keep playing the same assortment multiple times, neither require a strongest default item (like NC satisfying θ 0 ≥ max i ∈ [ K ] θ i ). Moreover, our objectives do not require us to converge to a multiset of replicated arms like (1 , 1 , . . . 1), but converge to the utility-maximizing set of distinct items. We list our contributions below:

1. A General AOA Setup: We work with a general problem of AOA for PL model, which requires no additional structural assumption of the θ parameters such as θ 0 ≥ max i θ i , unlike the existing works. We designed algorithms for two separate objectives Topm and Wtd-Topm as discussed above (Sec. 2).

Practical, Efficient and Optimal Algorithm: In Sec. 3, we give a practical, efficient and optimal algorithm for MNL Assortment (up to log factors and the magnitude of θ max ). The regret bound of our algorithm AOA-RB PL (Alg. 1) yields ˜ O ( √ KT ) regret for both Topm and Wtd-Topm objective. Our algorithms use a novel parameter estimation technique for discrete choice models based on the concept of Rank-Breaking (RB) which is one of our key contributions towards designing the efficient and optimal algorithm. This enables our algorithm to perform optimally without requiring the No-Choice item to be the strongest. Appendix A details the key concept of our parameter estimation technique exploiting the concept of RB. Our resulting algorithm plays optimistically based on the UCB estimates of PL parameters and does not require repeating the same subset multiple times, justifying our title.

3. Improvement with Adaptive Pivots: In Sec. 4, we refine the performance of our algorithm by employing the novel idea of 'adaptive pivots' (a reference item) and proposed AOA-RB PL -Adaptive. Performance-wise this removes the asymptotic dependence on θ max = max i θ i /θ 0 in the regret analysis. This enables the algorithm to work effectively in scenarios where the No-Choice item is less likely to be selected, i.e., θ max /greatermuch 1. This leads to a huge improvement in our experiments, especially in the range of low θ 0 , where AOA-RB PL -Adaptive drastically outperforms over the existing baseline. Comparison of our regret bound with existing work is detailed in Table 1.

Emperical Analysis. Finally, we corroborate our theoretical results with empirical evaluations (Sec. 5), which certify our superior performance in the general AOA setups.

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

It is also worth mentioning that our proposed algorithm and their respective regret analysis could be extended to any general random utility (RUM) based preference models [38, 37], as explained in Rem. 1. However, to keep the focus on the AOA problem and ease the presentation, we stick to the special case of MNL choice model based preferences.

2 Problem Setup

We write [ n ] = { 1 , 2 , ..., n } and 1 {·} denotes the indicator function. The symbol ≲ , employed in the proof sketches, represents a coarse inequality.

We consider the sequential decision-making problem of Active Optimal Assortment (AOA), with preference/choice feedback. Formally, the learner is given [ K ], a finite set of K items ( K > 2). At each decision round t = 1 , 2 , . . . , the learner selects a subset S t ⊆ [ K ] of up to m items, and receives some (stochastic) feedback about the item preferences of S t , drawn according to some unknown underlying Plackett-Luce (PL) choice model (1) with parameters θ = ( θ 1 , θ 2 , . . . , θ K ) ∈ R K + . We assume θ 1 ≥ θ 2 ≥ . . . ≥ θ K without loss of generality. An interested reader may check App. A.1 for a detailed discussion on PL models. Given any assortment S t we also consider the possibility of 'no-selection' of any items given an S t . Following the literature of [2], we model this mathematically as a No-Choice (NC) item, indexed by item-0, and its corresponding PL utility parameter θ 0 . Unlike most existing literature on assortment selection, we are not assuming θ 0 /negationslash≥ max i ∈ [ K ] θ i . Further, since the PL model is scale independent, we set θ 0 = 1 and scale the rest of the PL parameters.

Feedback model The feedback model formulates the information received (from the 'environment') once the learner plays a subset S t ⊆ [ K ] of at most m items. Given S t we consider the algorithm receives a winner feedback (or index of an item) i t ∈ S t ∪ { 0 } , drawn according to the underlying PL choice model as:

We consider the following two objectives for the learner:

1. Topm -Ojective. One simple objective could be to identify the topm item-set: { θ 1 , . . . , θ m } , for some m ∈ [1 , K ]. The performance of the learner can be captured by minimizing the following regret:

2. Wtd-Topm -Objective. Here, each itemi is associated with a weight (for example price) r i ∈ R + , and the goal is to identify the set of size at most m with maximum weighted utility. One could measure the regret of the learner as:

denotes S ∗ := argmax S ⊆ [ K ] || S |≤ m R ( S, θ ) is the optimal utility-maximizing subset. This objective corresponds to the standard objective in the MNL litterature [2].

3 A Practical and Efficient Algorithm for AOA with PL

In this section, we introduce our first algorithm, which works for both objectives.

3.1 Algorithm Design

At each time t , our algorithm (Alg. 1) maintains a pairwise preference matrix ̂ P t ∈ [0 , 1] n × n , whose ( i, j )-th entry ̂ p ij,t records the empirical probability of i having beaten j in a pairwise

duel, and a corresponding upper confidence bound p ucb ij,t . Let [ ˜ K ] := [ K ] ∪{ 0 } . We define for 172 each pair ( i, j ) ∈ [ ˜ K ] × [ ˜ K ], 173

174

175

176

where w ij,t = ∑ t -1 s =1 1 { i s = i, j ∈ S s } denotes the number of pairwise wins of itemi over j and n ij,t = w ij,t + w ji,t being the number of times ( i, j ) has been compared. The above UCB estimates p ucb ij,t are further used to design UCB estimates of the PL parameters θ i as follows

The estimates θ ucb i,t s are then used to select the set S t , that maximizes the underlying objective. 177 This optimization problem transforms into a static assortment optimization problem with 178 upper confidence bounds θ ucb i,t as the parameters, and efficient solution methods for this case 179 are available (see e.g., [7, 21, 32]). 180

Algorithm 1 AOA for PL model with RB (AOA-RB PL )

2: init: ˜ K ← K +1, [ ˜ K ] = [ K ] ∪ { 0 } , W 1 ← [0] ˜ K × ˜ K

1: input: x > 0

3: for t = 1 , 2 , 3 , . . . , T do

5: Define for all i , p ucb ii,t = 1 2 and for all i, j ∈ [ ˜ K ] , i = j

4: Set N t = W t + W /latticetop t , and ̂ P t = W t N t . Denote N t = [ n ij,t ] ˜ K × ˜ K and ̂ P t = [ ̂ p ij,t ] ˜ K × ˜ K .

/negationslash





m

items from argsort(



{

θ

(

for Top-

1

m

S

⊆

[

K

]

||

i

7:

S

t







8: Play S t

9:

R

m

S, θ

ucb

t

)

ucb ucb

K,t

, . . . , θ

objective

,

m

for Wtd-Top-

S

t

Receive the winner

|≤

∈

K

objective

] (drawn as per (1))

end for

11:

3.2 Analysis: Concentration Lemmas

We start the analysis by providing two technical lemmas, whose proofs are deferred to the appendix and that provide confidence bounds for the θ i .

Lemma 1. Let T ≥ 1 and x > 0 . Then, with probability at least 1 -3 KTe -x , for all t ∈ [ T ] and i ∈ [ K ] : θ i ≤ θ ucb i,t atleast one of the following two inequalities is satisfied

The above lemma depends on n i 0 ,t the number of times items i have been compared with item 0 up to round t . The latter is controlled using the following lemma:

Lemma 2. Let T ≥ 1 and x > 0 . Then, with probability at least 1 -KTe -x : simultaneously for all t ∈ [ T ] and i ∈ [ K ]

where τ i,t = ∑ t -1 s =1 1 { i ∈ S s } denotes the number of rounds item i got selected before round t . 190

)

Top- argmax

181

182

183

184

185

186

187

188

189

,t

}

,

191

192

193

194

195

196

197

198

199

200

201

202

3.3 Analysis: Topm Objective:

We are now ready to provide the regret upper bound for Algorithm 1 with Topm objective. Theorem 3 (Topm Objective) . Let θ max ≥ 1 . Consider any instance of PL model on K items with parameters θ ∈ [0 , θ max ] K , θ 0 = 1 . The regret of Alg. 1 with parameter x = 2log T is bounded as

The above rate of ˜ O ( KT ) is optimal (up to log-factors), as a lower bound can be derived from standard multi-armed bandits [5, 6]. We only state here a sketch of the proof of Theorem 3. The detailed proof is deferred to the App. B.

Proof Sketch of Theorem 3. Let us define for any S ⊆ [ K ],

Let E be the high-probability event such that both Lemma 1 and 2 holds true. Then, P ( E ) ≥ 1 -4 TKe -x . Let us first assume that E holds true. Then, by Lemma 1, Θ S ∗ ≤ Θ ucb S ∗ ≤ Θ ucb S t , which yields

where τ 0 = 138 x ( m +1) 2 θ 2 max corresponds to an exploration phase needed for the confidence 203 upper bounds of Lem 1 and 2 to be satisfied. Then, noting that if E holds true, we can show 204 by Lemma 2, that 1 { τ i,t ≥ τ 0 } ≤ 1 { n i 0 ,t ≥ 69 x ( θ 0 + θ i ) } . Therefore, we can apply Lemma 1 205 that entails, 206

207

208

209

where we used ∑ n i =1 1 / √ i ≤ 2 √ n and ∑ i τ i,t = mT together with Jensen's inequality in the last inequality. We thus have under the event E that Reg top T ≤ O ( θ 3 / 2 max √ xKT ) and the proof is concluded by taking the expectation with x = 2log T to control P ( E c ).

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

3.4 Analysis: Wtd-Topm Objective

We turn now to the analysis of the Wtd-Topm objective (2). We start by stating a lemma from [2] that shows that the expected utility R ( S ∗ , θ ) that corresponds to the optimal assortment S ∗ = argmax S ⊂ [ K ] , | S |≤ m R ( S, θ ) is non-decreasing in the parameters θ .

Lemma 4 (Lemma A.3 of [2]) . Assume θ ucb i ≥ θ i for all i ∈ [ K ] , then R ( S ∗ , θ ) ≤ R ( S ∗ , θ ucb ) .

Theorem 5 (Wtd-Topm Objective) . Let θ max ≥ 1 . Then, for any θ ∈ [0 , θ max ] K and weights r ∈ [0 , 1] K , the weighted regret of AOA-RB PL (Alg. 1) with x = 2log T

√

The complete proof is postponed to App. B. The rate Ω( KT ) is optimal as proved by the lower bound in [19] for MNL bandit problems for θ max = 1. Our result recovers (up to a factor √ log T ) the one of [2] when θ max = 1. However, their algorithm relies on more sophisticated estimators that necessitate epochs repeating the same assortment until the No-Choice item is selected. Note for our problem setting, where it is possible to have θ max /greatermuch θ 0 = 1, the length of these epochs could be of O ( Kθ max ), which could be potentially very large when θ max /greatermuch 1. This reduces the number of effective epochs, leading to poor estimation of the PL parameters. We see this tradeoff in our experiments (Sec. 5) where the MNL-UCB algorithm of [2] yields linear O ( T ) regret for such choice of the problem parameters.

226

227

228

229

230

231

232

233

234

Remark 1 (Beyond MNL Models) . Although, in this paper, we primarily focused on MNL based choice models, it is worth mentioning that our proposed algorithms can be generalized to more general random utility based models (RUMs) [9, 33] pursuing the ideas from [36] that extends the RB based parameter estimation technique to any RUM ( θ ) choice models. Our algorithms and analyses thus apply to any general RUM ( θ ) based choice models; we stick to the special case of MNL models in this paper for brevity and keep the main focus on the AOA problem and the related algorithmic novelties.

Proof sketch of Thm. 5. Let E be the high-probability event such that both Lemma 1 and 2 are satisfied. Then,

because R ( S t , θ ucb t ) ≥ R ( S ∗ , θ ucb t ) ≥ R ( S ∗ , θ ) under the event E by Lemma 4. We now 235 upper-bound the first term of the right-hand-side 236

Because Θ ucb S t ,t ≥ Θ S t under the event E by Lemma 1. Then, using r i ≤ 1, we further upper237 bound using an exploration parameter τ 0 = O (log( T )) so that the upper-confidence-bounds 238 in Lemmas 1 and 2 are satisfied 239

where the last inequality is by Cauchy-Schwarz inequality. Now, the term A T ( i ) above may 240 be upper-bounded using Lemmas 1 and 2, 241

where in the last inequality we used that ∑ T n =1 n -1 ≤ 1 + log T . Substituting into (6), 242 Jensen's inequality entails, 243

The proof is finally concluded by applying Cauchy-Schwarz inequality which yields: 244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

Finally, combining the above result with (5) and (7) concludes the proof

Choosing x = 2log T ensures TP ( E c ) ≤ O (1) and τ 0 ≤ O (log T ).

4 Improved dependance on θ max with Adaptive Pivot Selection

A problem with Algorithm 1 stems from estimating all θ i based on pairwise comparisons with item 0. When θ max /greatermuch θ 0 = 1, item 0 may not be sampled enough as the winner, leading to poor estimators. This deficiency contributes to the suboptimal dependence on θ max observed in Theorems 3 and 5 and in prior work, such as [2]. We propose the following fix to optimize the pivot. For all i, j ∈ [ K ] ∪ { 0 } we define γ ij = θ i θ j , and the estimators:

where p ucb ij,t are defined in (3). For all rounds t , the algorithm AOA-RB PL -Adaptive selects

We offer below a regret bound that underscores the value of optimizing the pivot when θ max /greatermuch K . Note that while the algorithm and analysis are presented for the weighted objective with winner feedback only, it can be adapted to other objectives by replacing R ( S, θ ) with the new objective in the analysis, as long as Lemma 4 remains valid.

Theorem 6. Let θ max ≥ 1 . For any θ ∈ [0 , θ max ] K and weights r ∈ [0 , 1] K , the weighted regret of AOA-RB PL -Adaptive is upper-bounded as

as T →∞ for the choice x = 2log T (when definining p ucb ij,t ).

√

Asymptotically, when θ max is constant, the regret is O ( K T log T ), eliminating any dependence on θ max . This allows for handling scenarios where the No-Choice item is highly unlikely, which is not achievable in previous works such as [2, 1]. [2] did attempt in their Thm. 4 to relax the assumption of θ max = θ 0 and shows a bound of order O ( max { θ max /θ 0 , 1 } 1 / 2 √ KT ) , which unfortunately blows to ∞ as θ 0 → 0 or equivalently θ max → ∞ , leading to a vacuous bound. Here, lies the stark improvement and one of the key contributions, as also corroborated in our experimental evaluation Sec. 5 (Fig. 2).

The proof is deferred to the App. B, with a key step relying on selecting the pivot j t = argmax j ∈ S t ∪{ 0 } θ j . The use of | ̂ θ ucb i,t -θ i | ≤ | γ ucb ij t ,t -θ i | provides confidence upperbounds with an improved dependence on θ max , leveraging the fact that θ j t ≥ θ i . Due to the varying pivot over time, a telescoping argument introduces an additive factor √ K .

5 Experiments

We provide here a synthetic experiments. All results are averaged across 100 runs. We evaluate the performance of our main algorithm AOA-RB PL -Adaptive (Sec. 4), referred as 'Our Alg-1 (Adaptive Pivot)", with the following two algorithms: AOA-RB PL (Sec. 3) referred as 'Our Alg-2 (No-Choice Pivot)", and MNL-UCB, the state-of-the-art algorithm for AOA ([2], Alg. 1).

Different PL ( θ ) Environments. We report our experiment results on two datasets with K = 50 items: (1) Arith50 with PL parameters θ i = 1 -( i -1)0 . 2 , ∀ i ∈ [50]. (2) Bad50

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

with PL parameters θ i = 0 . 6 , ∀ i ∈ [50] \ { 25 } and θ 25 = 0 . 8. For simplicity of computing the assortment choices S t , we assume r i = 1 , ∀ i ∈ [ K ].

(1). Averaged Regret with weak NC ( θ max /θ 0 /greatermuch 1) (Fig. 1): In our first experiment, we set set m = 5 and θ 0 /θ max = 0 . 01 and report the average regret of the above three algorithms for our two objectives.

Avg-Topm-Regret

15000

Our Alg-2 (No-Choice pivot)

MNL-UCB

Our Alg-1 (Adaptive pivot)

arith50

150

+Our Alg-2 (No-Choice pivot)

MNL-UCB

Our Alg-1 (Adaptive pivot)

arith50

Avg-Topm-Regret

400

300

Bad50

+Our Alg-2 (No-Choice pivot)

Our Alg-1 (Adaptive pivot)

MNL-UCB

+Our Alg-2 (No-Choice pivot)

+MNL-UCB

Our Alg-1 (Adaptive pivot)

Bad50

10000

100

200

5000

50

100

0.5

口

5000

←⊥

10000

1500

0

0

5000

10000

0

2000

4000

6000

8000

Q

0

2000

←1

4000

6000

(2). Averaged Regret vs No-Choice PL Parameter ( θ max /θ 0 ) (Fig. 2): In this experiment, we evaluate the regret performance of our algorithm AOA-RB PL -Adaptive. We report the experiment on Artith50 PL dataset and set the subsetsize m = 5, θ max /θ 0 = { 1 , 0 . 5 , 0 . 1 , 0 . 05 , 0 . 01 , 0 . 005 , 0 . 001 } . Fig. 2 shows the increase in the performance gap between our algorithm AOA-RB PL -Adaptive (with adaptive pivot) with decreasing θ 0 /θ max .

Avg-Fin-Topm-Reg

400

300

Arith50

MNL-UCB

Our Alg-2 (No-Choice pivot)

Our Alg-1 (Adaptive pivot)

60

40

MNL-UCB

Our Alg-2 (No-Choice pivot)

Our Alg-1 (Adaptive pivot)

Arith50

Avg-Topm-Regret

300

200

400

+k=4

k=2

Arith50 (m=30)

0.15

0.1

+k=1

+k=4

k=2

Arith50 (m=30)

200

0.05

100

100

0.001

0.005

0.01

0.05

0.1

0.5

1

0.0010.0050.01

0.05

0.1

0.5

1

1000

2000

←1

3000

4000

5000

0

1000

2000

←⊥

3000

4000

5000

(3). Averaged Regret vs Length of the rank-ordered feedback ( k ) (Fig. 3): We also run a thought experiment to understand the tradeoff between learning rate with k -length rank-ordered feedback, where given any assortment S t ⊆ [ K ] of size m , the learner gets to see the topk draws ( k ≤ m ) from the PL model without replacement. This is a stronger feedback than the winner (i.e. top-1 for k = 1) feedback and, as expected, we see in Fig. 3 an improved regret (for both notions) when increasing k . The experiment are run on the Artith50 dataset with m = 30 and k ∈ { 1 , 2 , 4 , 8 } .

6 Conclusion

We address the Active Optimal Assortment Selection problem with PL choice models, introducing a versatile framework ( AOA ) that eliminates the need for a strong default item, typically assumed as the No-Choice (NC) item in the existing literature. Our proposed algorithms employ a novel 'Rank-Breaking' technique to establish tight concentration guarantees for estimating the score parameters of the PL model. Our approach stands out for its practicality and avoids the suboptimal practice of repeatedly selecting the same set of items until the default item prevails. This is beneficial when the default item's quality ( θ 0 ) is significantly lower than the quality of the best item ( θ max ). Our algorithms are computationally efficient, optimal (up to log factors), and free from restrictive assumptions on the default item.

Future Works. Among many interesting questions to address in the future, it will be interesting to understand the role of the No-Choice (NC) item in the algorithm design, precisely, can we design efficient algorithms without the existence of NC items with a regret rate still linear in θ max ? Further, it will be interesting to extend our results to more general choice models beyond the PL model [18, 22, 23]. What is the tradeoff between the subsetsize m and the regret for such general choice models? Extending our results to large (potentially infinite) decision spaces and contextual settings would also be a very useful and practical contribution to the literature of assortment optimization.

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

References

Shipra Agrawal, Vashist Avadhanula, Vineet Goyal, and Assaf Zeevi. Thompson sampling for the mnl-bandit. In Conference on learning theory , pages 76-78. PMLR, 2017.

Shipra Agrawal, Vashist Avadhanula, Vineet Goyal, and Assaf Zeevi. Mnl-bandit: A dynamic learning approach to assortment selection. Operations Research , 67(5):1453-1485, 2019.

Nir Ailon, Zohar Karnin, and Thorsten Joachims. Reducing dueling bandits to cardinal bandits. In International Conference on Machine Learning , pages 856-864. PMLR, 2014.

Jean-Yves Audibert, Rémi Munos, and Csaba Szepesvári. Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science , 410(19):18761902, 2009.

Peter Auer. Using upper confidence bounds for online learning. In Foundations of Computer Science, 2000. Proceedings. 41st Annual Symposium on , pages 270-279. IEEE, 2000.

Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning , 47(2-3):235-256, 2002.

Vashist Avadhanula, Jalaj Bhandari, Vineet Goyal, and Assaf Zeevi. On the tightness of an lp relaxation for rational optimization and its applications. Operations Research Letters , 44(5):612-617, 2016.

Hossein Azari, David Parkes, and Lirong Xia. Random utility theory for social choice. In Advances in Neural Information Processing Systems , pages 126-134, 2012.

Hossein Azari, David Parks, and Lirong Xia. Random utility theory for social choice. Advances in Neural Information Processing Systems , 25, 2012.

Viktor Bengs, Róbert Busa-Fekete, Adil El Mesaoudi-Paul, and Eyke Hüllermeier. Preferencebased online learning with dueling bandits: A survey. Journal of Machine Learning Research , 2021.

Viktor Bengs, Róbert Busa-Fekete, Adil El Mesaoudi-Paul, and Eyke Hüllermeier. Preferencebased online learning with dueling bandits: A survey. J. Mach. Learn. Res. , 22:7-1, 2021.

Viktor Bengs, Aadirupa Saha, and Eyke Hüllermeier. Stochastic contextual dueling bandits under linear stochastic transitivity models. In International Conference on Machine Learning , pages 1764-1786. PMLR, 2022.

Gerardo Berbeglia and Gwenaël Joret. Assortment optimisation under a general discrete choice model: A tight analysis of revenue-ordered assortments. arXiv preprint arXiv:1606.01371 , 2016.

Brian Brost, Yevgeny Seldin, Ingemar J. Cox, and Christina Lioma. Multi-dueling bandits and their application to online ranker evaluation. CoRR , abs/1608.06253, 2016.

Niladri S Chatterji, Aldo Pacchiano, Peter L Bartlett, and Michael I Jordan. On the theory of reinforcement learning with once-per-episode feedback. arXiv preprint arXiv:2105.14363 , 2021.

Xi Chen, Sivakanth Gopi, Jieming Mao, and Jon Schneider. Competitive analysis of the top-k ranking problem. In Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 1245-1264. SIAM, 2017.

Xi Chen, Yuanzhi Li, and Jieming Mao. A nearly instance optimal algorithm for top-k ranking under the multinomial logit model. In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 2504-2522. SIAM, 2018.

Xi Chen, Chao Shi, Yining Wang, and Yuan Zhou. Dynamic assortment planning under nested logit models. Production and Operations Management , 30(1):85-102, 2021.

Xi Chen and Yining Wang. A note on a tight lower bound for mnl-bandit assortment selection models. arXiv preprint arXiv:1709.06109 , 2017.

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep 365 reinforcement learning from human preferences. Advances in neural information processing 366 systems , 30, 2017. 367

James Davis, Guillermo Gallego, and Huseyin Topaloglu. Assortment planning under the 368 multinomial logit model with totally unimodular constraint structures. Work in Progress , 2013. 369

370

371

372

Antoine Désir, Vineet Goyal, Srikanth Jagabathula, and Danny Segev. Assortment optimization under the mallows model. In Advances in Neural Information Processing Systems , pages 4700-4708, 2016.

Antoine Désir, Vineet Goyal, Danny Segev, and Chun Ye. Capacity constrained assortment 373 optimization under the markov chain based choice model. Operations Research , 2016. 374

375

376

James A Grant and David S Leslie. Learning to rank under multinomial logit choice. Journal of Machine Learning Research , 24(260):1-49, 2023.

377

378

379

380

381

Minje Jang, Sunghyun Kim, Changho Suh, and Sewoong Oh. Optimal sample complexity of m-wise data for top-k ranking. In Advances in Neural Information Processing Systems , pages 1685-1695, 2017.

Ashish Khetan and Sewoong Oh. Data-driven rank breaking for efficient rank aggregation. Journal of Machine Learning Research , 17(193):1-54, 2016.

Kameng Nip, Zhenbo Wang, and Zizhuo Wang. Assortment optimization under a single 382 transition model. 2017. 383

384

385

Min-hwan Oh and Garud Iyengar. Thompson sampling for multinomial logit contextual bandits. Advances in Neural Information Processing Systems , 32, 2019.

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

Mingdong Ou, Nan Li, Shenghuo Zhu, and Rong Jin. Multinomial logit bandit with linear utility functions. arXiv preprint arXiv:1805.02971 , 2018.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 35:27730-27744, 2022.

Wenbo Ren, Jia Liu, and Ness B Shroff. PAC ranking from pairwise and listwise queries: Lower bounds and upper bounds. arXiv preprint arXiv:1806.02970 , 2018.

Paat Rusmevichientong, Zuo-Jun Max Shen, and David B Shmoys. Dynamic assortment optimization with a multinomial logit choice model and capacity constraint. Operations research , 58(6):1666-1680, 2010.

Aadirupa Saha and Suprovat Ghoshal. Exploiting correlation to achieve faster learning rates in low-rank preference bandits. In International Conference on Artificial Intelligence and Statistics , pages 456-482. PMLR, 2022.

Aadirupa Saha and Aditya Gopalan. Active ranking with subset-wise preferences. International Conference on Artificial Intelligence and Statistics (AISTATS) , 2018.

Aadirupa Saha and Aditya Gopalan. Combinatorial bandits with relative feedback. In Advances in Neural Information Processing Systems , 2019.

Aadirupa Saha and Aditya Gopalan. PAC Battling Bandits in the Plackett-Luce Model. In Algorithmic Learning Theory , pages 700-737, 2019.

Aadirupa Saha and Aditya Gopalan. Best-item learning in random utility models with subset choices. In International Conference on Artificial Intelligence and Statistics , pages 4281-4291. PMLR, 2020.

Hossein Azari Soufiani, David C Parkes, and Lirong Xia. Computing parametric ranking models 409 via rank-breaking. In ICML , pages 360-368, 2014. 410

Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue. Multi-dueling bandits with 411 dependent arms. In Conference on Uncertainty in Artificial Intelligence , UAI'17, 2017. 412

Kalyan Talluri and Garrett Van Ryzin. Revenue management under a general discrete choice 413 model of consumer behavior. Management Science , 50(1):15-33, 2004. 414

Yisong Yue, Josef Broder, Robert Kleinberg, and Thorsten Joachims. The k -armed dueling 415 bandits problem. Journal of Computer and System Sciences , 78(5):1538-1556, 2012. 416

Yu-Jie Zhang and Masashi Sugiyama. Online (multinomial) logistic bandit: Improved regret 417 and constant computation cost. Advances in Neural Information Processing Systems , 36, 2024. 418

Zihan Zhang and Xiangyang Ji. Regret minimization for reinforcement learning by evaluating 419 the optimal bias function. In Advances in Neural Information Processing Systems , pages 420 2827-2836, 2019. 421

Masrour Zoghi, Zohar S Karnin, Shimon Whiteson, and Maarten De Rijke. Copeland dueling 422 bandits. In Advances in Neural Information Processing Systems , pages 307-315, 2015. 423

Masrour Zoghi, Shimon Whiteson, Remi Munos, Maarten de Rijke, et al. Relative upper 424 confidence bound for the k -armed dueling bandit problem. In JMLR Workshop and Conference 425 Proceedings , number 32, pages 10-18. JMLR, 2014. 426

Masrour Zoghi, Shimon A Whiteson, Maarten De Rijke, and Remi Munos. Relative confidence 427 sampling for efficient on-line ranker evaluation. In Proceedings of the 7th ACM international 428 conference on Web search and data mining , pages 73-82. ACM, 2014. 429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: In the abstract, we list the main claims of this paper in a general fashion. Then, in the introduction we state them in more detail. They accurately reflect the paper's contribution and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations and assumptions of our work throughout the paper. Additional limitations are highlighted in the discussion.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors are encouraged to create a separate "Limitations" section in their paper.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The assumptions can be found in the Problem Setup section and in the paragraphs before the theorems and remarks. We provide proof sketches in the main text and complete proofs in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All assumptions should be clearly stated or referenced in the statement of any theorems.

All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Theorems and Lemmas that the proof relies upon should be properly referenced.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: experimental details including algorithms and setups are clearly provided. In addition, the main contribution of the paper is theoretical and synthetic experiments are mostly provided as an illustration.

Guidelines:

The answer NA means that the paper does not include experiments.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: experimental setups are synthetic and can easily be reproduced.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

Please see the NeurIPS code and data submission guidelines ( https://nips. cc/public/guides/CodeSubmissionPolicy ) for more details.

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All details are provided to reproduce the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The full details can be provided either with the code, in appendix, or as supplemental material.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

588

Justification:

[Yes]

589

Guidelines:

590

591

The answer NA means that the paper does not include experiments.

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The assumptions made should be given (e.g., Normally distributed errors).

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We do not see any potential negative social impact of this work and it follows the NeurIPS code of ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work addresses the problem of designing efficient and optimal algorithms for different assortment selection problems with MNL models. Our work is purely theoretical and studies a fundamental mathematical optimization framework that is unrelated to societal considerations

Guidelines:

The answer NA means that there is no societal impact of the work performed.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper poses no such risks.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

694

Justification: [NA]

695

Guidelines:

696

697

The answer NA means that the paper does not use existing assets.

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

The authors should cite the original paper that produced the code package or dataset.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

The authors should state which version of the asset is used and, if possible, include a URL.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not release new assets.

The paper should discuss whether and how consent was obtained from people whose asset is used.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

Supplementary: Optimal, Efficient and Practical Algorithms for Assortment Optimization

A Preliminaries: Some Useful Concepts for PL choice models

A.1 Plackett-Luce (PL): A Discrete Choice Model

A discrete choice model specifies the relative preferences of two or more discrete alternatives in a given set. A widely studied class of discrete choice models is the class of Random Utility Models (RUMs), which assume a ground-truth utility score θ i ∈ R for each alternative i ∈ [ n ], and assign a conditional distribution D i ( ·| θ i ) for scoring item i . To model a winning alternative given any set S ⊆ [ n ], one first draws a random utility score X i ∼ D i ( ·| θ i ) for each alternative in S , and selects an item with the highest random score.

One widely used RUM is the Multinomial-Logit (MNL) or Plackett-Luce model (PL) , where the D i s are taken to be independent Gumbel distributions with parameters θ ′ i [8], i.e., with probability densities

Moreover assuming θ ′ i = ln θ i , θ i > 0 ∀ i ∈ [ n ], it can be shown in this case the probability that an alternative i emerges as the winner in the set S /owner i becomes: P ( i | S ) = θ i ∑ j ∈ S θ j .

Other families of discrete choice models can be obtained by imposing different probability distributions over the utility scores X i , e.g. if ( X 1 , . . . X n ) ∼ N ( θ , Λ ) are jointly normal with mean θ = ( θ 1 , . . . θ n ) and covariance Λ ∈ R n × n , then the corresponding RUM-based choice model reduces to the Multinomial Probit (MNP) .

A.2 Rank Breaking

Rank breaking (RB) is a well-understood idea involving the extraction of pairwise comparisons from (partial) ranking data, and then building pairwise estimators on the obtained pairs by treating each comparison independently [26, 25], e.g., a winner a sampled from among a, b, c is rank-broken into the pairwise preferences a /follows b , a /follows c . We use this idea to devise estimators for the pairwise win probabilities p ij = P ( i |{ i, j } ) = θ i / ( θ i + θ j ) for our problem setting. We used the idea of RB in both our algorithms (AOA-RB PL and AOA-RB PL -Adaptive) to update the pairwise win-count estimates w i,j,t for all the item pairs ( i, j ) ∈ [ K ] × [ K ], which is further used for deriving the empirical pairwise preference estimates ̂ p ij,t , at any time t .

A.3 Parameter Estimation with PL based preference data

Lemma 7 (Pairwise win-probability estimates for the PL model [34]) . Consider a PlackettLuce choice model with parameters θ = ( θ 1 , θ 2 , . . . , θ n ) , and fix two items i, j ∈ [ n ] . Let S 1 , . . . , S T be a sequence of (possibly random) subsets of [ n ] of size at least 2 , where T is a positive integer, and i 1 , . . . , i T a sequence of random items with each i t ∈ S t , 1 ≤ t ≤ T , such that for each 1 ≤ t ≤ T , (a) S t depends only on S 1 , . . . , S t -1 , and (b) i t is distributed as the Plackett-Luce winner of the subset S t , given S 1 , i 1 , . . . , S t -1 , i t -1 and S t , and (c) ∀ t : { i, j } ⊆ S t with probability 1 . Let n i ( T ) = ∑ T t =1 P ( i t = i ) and n ij ( T ) = ∑ T t =1 P ( { i t ∈ { i, j }} ) . Then, for any positive integer v , and η ∈ (0 , 1) ,

B Omitted Proofs from Sec. 3 and Sec. 4 795

B.1 A concentration bounds for the p ij,t 796

We first prove below a concentration inequality based on Bernstein's inequality for the 797 estimators p ij,t . 798

Lemma 8. Let ( i, j ) ∈ [ K ] × [ K ] . Let T ≥ 1 and x > 0 . Then, with probability at least 799 1 -3 Te -x , 800

simultaneously for all t ∈ [ T ] .

801

Proof of Lemma 8. Let T ≥ 1, x > 0 and i, j ∈ [ K ]. Applying Thm. 1 of [4], with probability 802 at least 1 -β ( x, T ), we get simultaneously for all t ∈ [ T ], 803

where β ( x, T ) = 3 inf 1 <α ≤ 3 min { log T log α , T } e -x/α ≤ 3 Te -x . Note that the inequality holds 804 true although n ij,t is a random variable. This, shows the first inequality 805

For the second inequality, (9) implies 806

Furthermore, because x ↦→ x (1 -x ) is 1-Lipschitz on [0 , 1], we have 807

Plugging back into (10), we get 810

Therefore, 808

which yields 809

811

812

813

814

815

816

817

818

B.2 Proof of Lemma 1

Proof. Let i ∈ [ K ] and x > 0. Then, by a union bound on Lemma 8 and 2, with probability at least 1 -4 Te -x , (8) and (4) hold true for all t ∈ [ T ]. We consider this high-probability event in the rest of the proof. Define the function f : x ↦→ x/ (1 -x ) + on [0 , 1] (with the convention f (1) = + ∞ ), so that θ ucb i,t = f ( p ucb i 0 ,t ) and θ i = f ( p i 0 ). Because f is non-decreasing, and p ucb i 0 ,t ≥ p i 0 by (8), we have

Furthermore, denote

In the rest of the proof we assume, n i 0 ,t ≥ 69 x ( θ 0 + θ i ). Then, using that θ 0 θ i ≤ θ 0 + θ i 819 since θ 0 = 1, it implies 820

and 821

Thus, because f is non-decreasing 822

which concludes the proof. 823

824

825

826

827

828

829

830

B.3 Proof of Lemma 2

Proof. Let T ≥ 1 and i ∈ [ K ]. Recall that τ i,t = ∑ t -1 s =1 1 { i ∈ S s } is the number of times i was played at the start of round t and n i 0 ,t = ∑ t -1 s =1 1 { i t ∈ { i, 0 } , i ∈ S t } is the number of times i or 0 won up to round t when played together. When i is played the probability of 0 or i to win is

Therefore, applying Chernoff-Hoeffding inequality together with a union bound (to deal with the fact that τ i,t is random), we have with probability at least 1 -Te -x

simultaneously for all t ∈ [ T ]. Noting that 831

if τ i,t ≥ 2 x ( θ 0 +Θ S ∗ ) 2 ≥ 2 x ( θ 0 +Θ S ∗ ) 2 ( θ 0 + θ i ) 2 concludes the proof. 832

B.4 Proof of Theorem 3 833

Proof. Let us define for any S ⊆ [ K ], 834

Let E be the high-probabality event such that both Lemma 1 and 2 holds true. Then, 835 P ( E ) ≥ 1 -4 TKe -x . Let us first assume that E holds true. Then, by Lemma 1, 836

where ¯ τ i 0 = 2 x ( θ 0 +Θ S ∗ ) max { θ 0 +Θ S ∗ , 69 } ≤ 138 x ( m +1) 2 θ 2 max , where θ max := max i θ i . 837 Then, noting that if E holds true, by Lemma 2, we also have n i 0 ,t ≥ 1 2( θ 0 +Θ S ∗ ) ( θ 0 + θ i ) τ i,t , 838 which yields 839

Therefore, we can apply Lemma 1 that entails, 840

where we used ∑ n i =1 1 / √ i ≤ 2 √ n and ∑ n i =1 i -1 ≤ 1 + log n . We thus have 841

Therefore, 842

E [ Reg top T ] ≤ 12 √ 2 xmKθ 3 max +16 θ 3 / 2 max √ 2 xKT +88 xmKθ 2 max ( 1 + log ( mT K )) +4 mKT 2 e -x θ max . Choosing x = 2log T concludes the proof. 843

B.5 Proof of Theorem 5 844

Proof. Let E be the high-probabality event such that Lemma 1 and 2 are satisfied, so that 845 P ( E ) ≥ 1 -4 KTe -x . Then, denoting x ∧ y := min { x, y } , 846

because R ( S t , θ ucb t ) ≥ R ( S ∗ , θ ucb t ) ≥ R ( S ∗ , θ ) under the event E by Lemma 4. Then, using 847 R ( S ∗ , θ ) ≤ max i r i ≤ 1, we get 848

Let us upper-bound the second term of the right-hand-side 849

where the last inequality is by Cauchy-Schwarz inequality. Now, the term A T ( i ) above may 850 be upper-bounded as follows 851

Now, since under the event E by Lemma 2, τ i,t ≥ 138 x ( m +1) 2 θ 2 max implies 852

Therefore, we can apply Lemma 1, which further upper-bounds 853

where we used n i 0 ,t ≥ 69 x ( θ 0 + θ i ) mθ max in the last inequality. Then, we get 854

Substituting into (16), we then obtain using Cauchy-Schwarz inequality, 855

856

857

858

859

860

861

862

Finally, replacing into Inequality (15) yields

Choosing x = 2log T concludes the proof.

B.6 Proof of Theorem 6

The proof follows the one of Theorem 5, except that the concentration lemmas should be generalized to any pairs ( i, j ) instead of only with respect to item 0, whose proofs are left to the reader and closely follows the one of Lemma 1 and 2. For simplicity, this proof is performed up to universal multiplicative constants, using the rough inequality ≲ .

and 872

This implies, 873

Replacing γ ij = θ i /θ j and γ j 0 = θ j concludes the proof. 874

Lemma 11. Let T ≥ 1 and x > 0 . Then, with probability at least 1 -K ( K +1) Te -x 875

/negationslash where τ ij,t := ∑ t -1 s =1 1 {{ i, j } ⊆ S s } simultaneously for all t ∈ [ T ] and i = j ∈ [ K ] .

Proof of Theorem 6. Let E be the high-probabality event of Lemmas 10 and 11 are satisfied, so that P ( E ) ≥ 1 -4 K 2 Te -x . First, note that since we have under the event E , ̂ θ ucb t ≤ θ ucb t , our procedure also satisfies the regret upper-bound

of Theorem 5. Indeed, all upper-bounds of the proof of Theorem 5 remain valid upper-bounds 880 except the probability of the event E c which is O ( T -1 ) for x = 2log T . 881

√

Let us now prove that we also have R T ≤ O ( K T log T ) with no asymptotic dependence on 882 θ max when T →∞ . 883

876

877

878

879

/negationslash

Lemma 9. Let T ≥ 1 and x > 0 . Then, with probability at least 1 -3 K ( K + 1) Te -x , 863 simultaneously for all t ∈ [ T ] and i = j in [ ˜ K ] : γ ij := θ i θ j ≤ γ ucb ij,t and one of the following 864 two inequalities is satisfied 865

866

867

868

Lemma 10. Let T ≥ 1 and x > 0 . Then, with probability at least 1 -3 K ( K +1) Te -x , simultaneously for all t ∈ [ T ] and i ∈ [ K ] : ̂ θ ucb i,t := min j γ ucb ij,t γ ucb j 0 ,t ≥ θ i and for all j one of the following two inequalities is satisfied

or 869

Proof of Lemma 10. The proof follows from Lemma 9. If n ij,t > Cx (1 + γ ij ) and n j 0 ,t > 870 Cx (1 + θ j ) for some large enough constant C, we have 871

Then, 884

Then, using R ( S ∗ , θ ) ≤ max i r i ≤ 1, we get 885

Follow the proof of Theorem 5, we upper-bound the second term of the right-hand-side 886 of (19): 887

where j t = argmax j ∈ S t ∪{ 0 } θ j , where the last inequality is by definition of ̂ θ ucb i,t . Now, 888 from Lemma 10, paying an additive exploration cost to ensure that n ij,t ≳ x (1 + γ ij ) and 889 n j 0 ,t ≳ x (1 + θ j ) 2 θ j for all j ∈ S t such that θ j ≥ θ 0 . From Lemma 11, this is satisfied if for 890 some constant C > 0 891

Such a condidtion can be wrong for a couple ( i, j ) ∈ S 2 t at most during CK 2 m 2 θ 2 max x = 892 O (log T ) rounds (since τ ij,t increases then). Thus, for C large enough, 893

where the last inequality is because using that { i, j t , 0 } ⊆ S t , we have 894

and 895

Then, by Cauchy-Schwarz inequality we further get 896

where the last inequality is by Jensen's inequality and the equality by setting x = 2log T to 897 control the probability that E c occurs. This concludes the proof. 898

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the Adaptive Optimal Assortment (AOA) problem a.k.a. Utility Maximization with Subset Choices. The goal of this problem is to find the optimal profit-maximizing subset of size up to m (Top-m-Objective) or its weighted variant (Wtd-Top-m-Objective). Given a selected subset, the feedback follows the Plackett-Luce (PL) choice model that returns an item from the subset or a \"no-choice\" option. The probability of choosing each item is proportional to their underlying score/utility values.\n\nThe paper proposes a new algorithm, AOA-RB, that is claimed to be practical, efficient, and optimal. Compared to previous works, this algorithm does not require sampling the same subset repeatedly nor assumes a strongest default item. Later, the authors extend this algorithm with adaptive pivots that further improves performance.\n\nThe theoretical analysis shows that AOA-RB obtains regret guarantees that build on a novel \"Rank-Breaking\" parameter estimation technique for the discrete choice model.\n\nThe performance of AOA-RB is further demonstrated in numerical experiments using synthetic datasets.\n\nSTRENGTHS:\nProblem Statement\n- Clear presentation and motivation. Easy-to-follow section.\n\nAlgorithm\n- Clear strengths are the relaxation of previous assumptions, e.g., repeated sampling of the same subset or the assumption of a strong default item\n- The algorithm is well-presented and easy-to-follow.\n- The adaptive pivot extension of the AOA-RB is a clear improvement that provides significant improvements\n\nTheoretical Analysis\n- The new concentration lemmas in Section 3.2. are claimed to be novel by the authors.\n- Regret guarantees are provided for both objectives. The main strength is Theorem 6 which analyses the regret of the adaptive pivot version of the algorithm and shows a regret bound that does not blow to $\\infty$ in corner cases.\n\nExperiments\n- The numerical experiments section further demonstrates the performance improvement of AOA-RB over the state-of-the-art MNL-UCB algorithm. It highlights especially the benefits of the adaptive pivots.\n\nWEAKNESSES:\nIntroduction, Related Works, and Contribution\n- Certain claims are not supported, e.g., Line 21 \"Studies have shown that it is often easier...\" but it lacks citation which studies the authors refer to.\n- I found some citations to be misplaced or non-supportive of the claims it is used for, e.g., [11] is used in Line 62 as a reference for Battling Bandits while it is a survey of dueling bandits. Similarly, citations [45, 46] are used for dueling bandits while they are only two examples from the literature. It would be great if authors could use consistent citations, e.g., surveys when they refer to broader literature and individual publications when specifics are important.\n- Table 1 is provided for the comparison of regret guarantees but the authors do not describe it. It would be great if they could comment on the differences between the algorithms.\n\nProblem Setting\n- Limitations are not mentioned in the problem statement. For example, how restrictive is the Plackett-Luce model, and whether the approach could be extended to other models? I see that it is mentioned in Remark 1 but could be commented on in Section 2 as well.\n- Both Top-m and Wtd-Top-m consider the (weighted) utility optimization problem. However, for most of the applications used as motivation, e.g., assortment optimization and recommender systems, the utility of the user which dictates the selected feedback, and the utility/profit of the subset selection (platform) are misaligned. Could the authors comment on how to formulate these problems in their setting?\n\nAlgorithm\n- The $argmax_{S\\subseteq [K], |S|\\leq m}$ optimization is non-trivial and could be computationally expensive for large values for $K$.\n- The authors claim that AOA-RB is practical, efficient, and optimal. While the theoretical analysis supports the last two claims, I struggle to find the intuition behind the algorithm. Could the authors elaborate further on this point?\n\nExperiments\n- Numerical experiments demonstrate performance only in synthetic data. Given the clear application and motivation of the paper, I would like to see experiments that reflect these problems.\n- I recommend the authors to use larger figures. Axes and titles are hardly visible in the printed version.\n- Only one baseline is considered. It would be appreciated if the authors could include the other algorithms mentioned in Table 1 for numerical comparison besides the theoretical one.\n\nWhile the paper is easy to read and follow even for readers not familiar with all the works in the area, the inconsistent citations and unsupported claims have to be addressed before the paper would reach publication standards.\n\nQUESTIONS:\nIt would be appreciated if the authors could comment on the questions outlined in the Weaknesses section.\nSome further questions are the following:\n- It is not evident to me how the proposed framework applies to the application areas described in the Introduction section. Could the authors further comment on it and formulate some more rigorously for demonstration?\n- The paper compares its performance closely to MNL-UCB [2]. Could the authors further elaborate on the similarities and differences between their proposed algorithms and the MNL-UCB?\n\nLIMITATIONS:\nLimitations are mentioned in the paper, however, it is often not directly connected, e.g., the assumption of the PL model is only addressed in Remark 1. I would suggest the authors address limitations more clearly when they appear for easier readability.\nThe work is mainly theoretical without any immediate direct societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the online MNL assortment optimization problem, where the goal is to learn MNL parameters while suggesting assortments, with the goal of either learning the top-m highest utility items or learning the maximum revenue set with m items. They use a UCB-based approach on pairwise win rates to get a UCB for utilities, which can then be fed into a traditional assortment optimization algorithm. The authors show this approach achieves asymptotically optimal regret and does not require assumptions used by previous approached. The basic algorithm relies on comparisons between each item and the no-choice option, but they also introduce a more sophisticated adaptive pivot approach that works better when the no-choice option is rarely selected. In experiments on synthetic data, their assortment optimization approach performs significantly better than the previous state-of-the-art.\n\nSTRENGTHS:\n1. The problem studied is natural and important. \n2. The presentation is generally clear.\n3. The technical quality seems good, although I cannot attest to the correctness of all the proofs in the appendix.\n4. The UCB approach on pairwise win rates is clever and appears original.\n\nWEAKNESSES:\n1. The algorithms and proofs could use some additional description/intuition. Some of the steps in the proofs take rather large leaps.\n\nQUESTIONS:\n1. The regret bounds in Table 1 for [2] (Thm 1) and [2] (Thm 4) don't seem to match up with the bounds actually in [2], which have some extra terms.\n2. Eq. (1) is a multinomial logit, but it's referred to in the paper as Plackett-Luce. Plackett-Luce is the repeated-selection ranking version of MNL, so I think the model in this paper should be referred to as MNL. (Ah, I see that lines 294-300 use Placket-Luce, but I think it makes more sense to call the model MNL throughout the paper, since that's the focus.)\n3. After line 253, what is the justification for defining $\\hat \\theta$ as that product of $\\gamma$s? This definition seems to still depend on observations of comparisons between j and the no-choice option, doesn't it?\n4. The title is quite general, but the paper is very specifically about online assortment optimization rather than the static problem. It would be good to make this clear from the title.\n\nMinor comments:\n1. The statement of Lemma 1 needs to be proofread and fixed\n2. Line 196: shouldn't this be $\\tilde O(\\sqrt{KT})$?\n\nLIMITATIONS:\nI think the limitations of the paper were adequately stated", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of active online assortment optimization problem with preference feedback, which has been extensively studied. The paper argues that the previous studies have some unrealistic assumptions such as: there is a ‘strong reference’ which is always included in the choice sets; the same assortments can be repeatedly selected. Without these assumptions, they propose some efficient algorithms for the problem of regret minimization in assortment selection with Plackett Luce (PL) based user choices.\n\nSTRENGTHS:\nThe paper proves the regret bounds of the proposed online learning algorithms. The regret bounds are proved based on some concentration guarantee for estimating the score parameters of the PL model using ‘Pairwise Rank-Breaking’.\n\nWEAKNESSES:\n1.  I cannot fully understand the motivation of the paper. The paper says that two major drawbacks of the previous studies include: the existing algorithms assume that the ``no-choice’’ option is stronger than the other choices, and they may query the same set of items for multiple times. It seems that the focus of the paper is to address these drawbacks. However, I think that these ``drawbacks’’ may not be real. First, it is natural that most of the customers will not choose any product, so it is very reasonable to assume that no-choice option is stronger. Second, in the typical assortment optimization scenario where customers arrive online one by one, showing the same set of items to different customers for multiple times absolutely will not cause any problem.  So I think that addressing these ``drawbacks’’ has very limited value.\n2.  The regret bounds proposed by the paper is actually K\\sqrt{T}\\log T. It seems that this regret bound is weaker than those of the previous studies such as [2] (at least by log factors on T). The authors may argue that their bounds are better when \\theta_{max}\\rightarrow \\infty, but this depends on the assumptions made on specific application scenarios, which is questionable as explained in my last comment.\n3.  The experiments are conducted using some specific values of \\theta and hence are not very convincing. I think that more experiments on more applications are necessary to demonstrate the superiority of the paper.\n\nQUESTIONS:\n1.\tSince the feedbacks from the same customer may be correlated, how do you use concentration bounds to learn the underlying distributions?\n2.\tThe proposed algorithms seem to be UCB-style algorithms. Can you explain the key differences and novelties of your algorithm compared to the classic UCB algorithm?\n\nLIMITATIONS:\nsee the above", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of active assortment optimization in MNL model.\n\nIn the problem of assortment optimization, we have  a large universe of products i=1,2,\\dots N, each of which generates a given revenue r_i for the seller.  In MNL model  each product i has a value \\theta_i to the customers and when customers are offered a subset of products they choose each item (including the no-choice option) with a probability proportional to their value. We also assume there is a no-choice option with revenue 0. The seller’s objective is to identify the assortment of products which generates maximum expected revenue. \n\nIn the active version of the problems, the values of items  \\theta_1, \\theta_2,\\dots , \\theta_N, are not known to the seller. Thus, the seller  shows a  subset of items from the universe to the customers at  rounds 1,2, \\dots ,T  and estimates \\theta_i s  based on the observations. After approximating these values, the seller may solve the problem in static setting and find the optimal assortment.  This strategy is known as exploration and exploitation. \n\nIn active assortment optimization, the objective is to minimize the regret of the algorithm which is defined as the summation over rounds t=1,2,\\dots T the difference of the expected revenue in each round from the optimal revenue.\n\nPrior works for instance [2] provided an algorithm for this problem by estimating at each round a high probability upper bound for the values \\theta_i,  and then solve the static problem using the upper-bounds. In [2] the authors assume that \\theta_0 (the value assigned to no-choice option and thus its probability ) is the highest among all items. \n\nThe submitted manuscript claims that they provide an algorithm with a similar regret bound to [2] which does not have the restriction of assuming the no-choice option has the highest value. Their suggested approach is similar to that of [2] (finding high probability upper bounds for the parameters) but it is hard to follow all details of obtaining the upper bound and how it removes the restriction imposed on the value of the no-choice option. \n\nThe result, if true, is interesting but I found the paper hard to read and got lost in section 3.1. I think that the paper will benefit greatly from rewriting and improving the presentation. \n\nI will detail my confusions as follows: \n\n- In Equation (3) on line 173 there is a variable x which is not defined up to this point. I understand that x appears to bound the probability of error in Lemma 1. But you have to introduce it before you use it the first time. \n- Between line 176 and 177 what is the + sign on the denominator of the equation? you use this notation again in another equation between lines 252 and 253.\n- In equation 3 you show an upper bound on \\hat{p_ijt} which then turns to a bound on \\theta_i s. But in Lemma 1 you have shown a different upper bound for \\theta_i. Can you explain the connection of these two bounds. \n\nA few minor typos:\n\nLemma 1. atleast-> at least\n^ucb is sometimes with roman font and sometimes normal font. \n\n\n\n\n[2] Shipra Agrawal, Vashist Avadhanula, Vineet Goyal, and Assaf Zeevi. Mnl-bandit: A dynamic learning approach to assortment selection. Operations Research, 67(5):1453–1485, 2019.\n\nSTRENGTHS:\n- The problem of active assortment optimization is a fundamental problem in revenue management \n- The result is interesting if correct, as it removes an important restriction from prior algorithms.\n\nWEAKNESSES:\n- The results are poorly presented and it is hard to follow the paper. The paper lacks an explanation of main intuitions . \n- The technique seems to be similar to [2] as both papers obtain high probability upper bounds for the parameters and then solve it in an static setting. An intuitive explanation of how the given different upper bound is obtained, why it is correct, and how it removed the restriction on no-choice option is not provided.\n\nQUESTIONS:\nI am mainly confused by the high probability upper bounds in Eq (3). Can you provide some intuition where is comes from and how it turns to an upper bound on \\theta_i?\nFor instance in the introduction you mention you use Rank-Breaking technique. Can you explain in simple words what rank breaking is? how does this technique make your work (and in particular deriving the upper bound on \\theta_i s ) different from [2] and how it shows up in equation (3)\n-------------------------\n\nMy confusions were addresses after reading the rebuttal.\n\nLIMITATIONS:\nlimitations are not discussed but there are several future directions that have been discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}