Title: Cascading Contextual Assortment Bandits
Venue: NeurIPS 2023
Accepted: yes
Keywords: cascade bandit

assortment bandit

upper confidence bound

exploration and exploitation

combinatorial optimization
Abstract: We present a new combinatorial bandit model, the \textit{cascading contextual assortment bandit}. This model serves as a generalization of both existing cascading bandits and assortment bandits, broadening their applicability in practice. For this model, we propose our first UCB bandit algorithm, UCB-CCA. We prove that this algorithm achieves a $T$-step regret upper-bound of $\tilde{\mathcal{O}}(\frac{1}{\kappa}d\sqrt{T})$, sharper than existing bounds for cascading contextual bandits by eliminating dependence on cascade length $K$. To improve the dependence on problem-dependent constant $\kappa$, we introduce our second algorithm, UCB-CCA+, which leverages a new Bernstein-type concentration result. This algorithm achieves $\tilde{\mathcal{O}}(d\sqrt{T})$ without dependence on $\kappa$ in the leading term. We substantiate our theoretical claims with numerical experiments, demonstrating the practical efficacy of our proposed methods.

Cascading Contextual Assortment Bandits

Abstract

We present a new combinatorial bandit model, the cascading contextual assortment bandit . This model serves as a generalization of both existing cascading bandits and assortment bandits, broadening their applicability in practice. For this model, we propose our first UCB bandit algorithm, UCB-CCA . We prove that this algorithm achieves a T -step regret upper-bound of ˜ O ( 1 κ d √ T ) , sharper than existing bounds for cascading contextual bandits by eliminating dependence on cascade length K . To improve the dependence on problem-dependent constant κ , we introduce our second algorithm, UCB-CCA+ , which leverages a new Bernstein-type concentration result. This algorithm achieves ˜ O ( d √ T ) without dependence on κ in the leading term. We substantiate our theoretical claims with numerical experiments, demonstrating the practical efficacy of our proposed methods.

1 Introduction

Sequential interactions between users and a recommender agent are often modeled as the multi-armed bandit problem or one of its variants [15]. In practice, a user typically encounters multiple items per round of interaction rather than a solitary item. Two popular models that capture this aspect are the cascading bandit [13; 14; 18] and the assortment bandit [4; 5; 7; 22], also often known as multinomial logistic bandits.

In the cascading bandit problem [13; 14; 18; 29; 25; 28], the agent selects a cascade of K items from a total of N items each round. These selected items are sequentially presented one at a time to a user. If the user clicks on a presented item, the cascading round ends. If not, the agent proceeds to reveal the next item from the cascade. This process continues until either the user clicks on an item or all K items in the cascade have been presented without a click. Once a round ends, a next round commences with a newly selected list of K items.

In the assortment bandit problem [4; 5; 7; 8; 9; 7; 22; 23], the agent presents an assortment of M items all at once, then receives user choice feedback on the assortment. A user may opt for one of the M items presented or choose none at all, concluding the round in either case. Both cascading and assortment bandit problems are significant combinatorial variations of the multi-armed bandit problem and have been extensively examined both theoretically and in practice.

However, a more commonly encountered scheme in real-world applications is a generalization of these two settings, where a cascade of assortments is sequentially revealed in each round. This approach is evident in video streaming services, where assortments of recommended contents are revealed as users scroll through webpages or mobile applications. Similar experiences can also be found in various online retail services and search engines. To address this, we propose a new interactive model, which we term cascading assortment bandits . In the cascading assortment bandits, the agent chooses a cascade of assortments that consists of K assortments with each assortment containing M items. The agent reveals one assortment at a time in the cascade. The cascade concludes if the user clicks on one of the items contained in a given assortment. If not, the agent proceeds

<latexi sh

1_b

64="

qYnQGMP

/r+T2

c

H

K

u

o

>A

B83

VDLSg

F

W

JN

pkRX

E

9

U

zOZI

v

d

f

m

j

7

C

0

5

w

y

<latexi sh

1_b

64="Up7

LJ

+TW

ro

FPVRY

u3

>A

B8

c

D

SgM

2

f

Zd

k

w

q

H

j

N

I

z

m

yQ

9

0

n

v

O

C

5

/

X

K

G

E

<latexi sh

1_b

64="3JWLXAy9BE

g

rdq/IjukcC7

>

8

VD

S

M

F

2p

f

Z

GK

R

H

O

Nm

Q

n

5

P

Y

T

z

v

o

+

0

<latexi sh

1_b

64="Xmv5y

8pIVK

Q

GWM

7

E

>A

B

3

c

DLSg

F

j

9Z

Y

r

oRC

N

T

U

/w

0

dz

+

2

H

P

k

O

q

J

f

u

n

Scanned &

not clicked

Not scanned

Current scan

Single item

Single assortment

Scanned &

not clicked

Not scanned

Current scan

Multi-Armed Bandit

Cascading Bandit

Cascading Assortment Bandit

K

= 1

, M

= 1

K

= 5

, M

= 1

K

= 1

, M

= 4

K

= 5

, M

= 4

to unveil the subsequent assortment in the cascade. This cycle continues until either an assortment receives a click from the user or the agent depletes the pre-selected assortments.

The cascading assortment bandit problem is the strict generalization of both the cascading bandits model and the assortment bandits model. It is also a generalization of the simple multi-armed bandit problem. That is, if K = 1 and M = 1 , the problem is the simple multi-armed bandit. If K > 1 and M = 1 , the problem corresponds to the cascading bandit. If K = 1 and M > 1 , we recover the assortment bandit problem. The illustrations on comparisons between the cascading assortment bandit and the other combinatorial bandits are presented in Figure 1.

In order to accommodate the generalization of the interactive model across items and assortments, we also incorporate the feature information of items and parametrization of a click model in the cascading assortment bandit model. Hence, we name the model as cascading contextual assortment bandit (see Section 2.2 for the formal definition of the problem setting). 1 Under this newly proposed combinatorial bandit model, we posit the following question:

Can we design a provably efficient algorithm for cascading contextual assortment bandits?

To address the question at hand, we first have to overcome the technical challenges inherent in each special case of our problem setting: the cascading contextual bandit and the contextual assortment bandit. Firstly, in the cascading contextual bandit [18; 25], a longstanding issue has been the suboptimal dependence on the cascade length, K . Intuitively, one would expect that as K increases in the cascading model, the regret should either diminish or at least remain constant; performance deterioration should not occur. However, all existing regret bounds for cascading contextual bandits scale proportionally to K [18; 29; 17; 28; 25]. This finding is not just counter-intuitive, but also suboptimal (for further discussions, refer to Section2.4). As a result, (i) eradicating the suboptimal dependence on cascade length K has been recognized as an open problem, even within the cascading contextual bandit setting. 2

Further, in the context of assortment bandits, there is a widely recognized suboptimal dependence on the problem-specific constant κ , as demonstrated in the existing assortment bandit literature [9; 22; 23]. This problem-specific constant κ (in Assumption 4.2) represents the curvature of the multinomial logit (MNL) function. Recent studies [24; 3] have demonstrated an improved dependence on κ , albeit only multiplied by logarithmic factors. However, this improvement comes at the expense of an increased dependence on the assortment size M , a conclusion that is both counter-intuitive and suboptimal. Thus, (ii) decreasing the κ dependence without escalating the dependence on M still poses an unresolved issue. While addressing either of the two challenges (i) and (ii) can be daunting individually, tackling both issues simultaneously poses an even greater challenge in both our algorithm design and regret analysis.

To this end, we design novel upper confidence bound (UCB) algorithms for contextual cascading assortment bandits, tackling both technical challenges. We show that our proposed algorithms achieve provable guarantees on regret performances overcoming the longstanding technical challenges. Our regret bounds show sharper results than those of the existing contextual cascading bandits or assortment bandits. We corroborate our theoretical claims through numerical experiments, thus ensuring that both our newly proposed bandit framework and the proposed algorithms establish provable efficiency and practical applicability.

Our main contributions are summarized as follows.

We formulate a general combinatorial bandit model, named cascading contextual assortment bandit that encompasses the existing cascading bandits and assortment bandits. This novel problem setting is observed in many practical applications.

We first propose a UCB bandit algorithm UCB-CCA for the cascading contextual assortment bandit and establish the T -step regret upper-bound of ˜ O ( 1 κ d √ T ) (in Theorem 4.3). This regret bound is tighter than the existing bounds for cascading contextual bandits, where we not only remove the longstanding, unnecessary dependence on K but also establish the result without dependence on M .

While UCB-CCA is an efficient algorithm achieving both the statistical efficiency and practical performances (shown in Section 7), its regret bound includes dependence on the inverse of a problem-dependent constant κ , which can be potentially large in the worst case. To improve the dependence on κ , we propose our second algorithm UCB-CCA+ , which exploits a new Bernstein-type concentration result, taking into account the effects of the local curvature of the MNL model. We prove that UCB-CCA+ achieves ˜ O ( d √ T ) without the dependence on κ in the leading term (only scaling with logarithmic factors), hence significantly improving the regret of UCB-CCA without increasing the other dependencies. Hence, we successfully solve the two technical challenges (i) and (ii) mentioned above.

As an independent contribution, we prove that a greedy algorithm for the cascading assortment optimization problem gives a 0.5 approximation of the optimal solution (discussed in Section 6). To our best knowledge, this is the first rigorous result showing the approximation guarantee even for the contextual cascading bandit problem, instead of simply assuming access to an approximation optimization oracle.

We evaluate our proposed methods in numerical experiments and show that the practical performances support our theoretical claims. Hence, our proposed algorithms along with our newly proposed bandit model establish provable efficiency and practical applicability.

2 Preliminaries

2.1 Notation

Define [ n ] as a set of positive integers from 1 to n . Let |·| be the length of a sequence or the cardinality of a set. For a vector x ∈ R d , we denote the /lscript 2 -norm of x as || x || 2 and the V -weighted norm of x for a positive-definite matrix V as || x || V = √ x /latticetop V x . The determinant and trace of a matrix V are det( V ) and trace ( V ) , respectively. λ min ( V ) denotes the minimum eigenvalue of a matrix V .

2.2 Cascading Contextual Assortment Bandit Problem

Consider [ N ] , a set of N items. Let A be a set of candidate assortments of items with size M , i.e., A := { A ⊆ [ N ] : | A | = M } . A cascade S is an ordered sequence of K assortments chosen from A where all the items in these K assortments are distinct. Then, the set of all feasible cascades S can be defined as follows.

At round t , feature vectors { x ti ∈ R d , i ∈ [ N ] } for every item are revealed to the decision-making agent. Each feature vector x ti may contain the contextual information of the user at round t and the item i . After observing this contextual information, at round t , the agent recommends a cascade S t = ( A tk ) k ∈ [ K ] to the user, where A tk ∈ A represents the k -th assortment of the cascade at round t . The user scans the assortments in S t one by one. If the items in A tk do not attract the user, the user moves on to the next assortment A t,k +1 . The user stops at the O t -th assortment when the user is attracted by an item in the O t -th assortment and clicks on the item.

After the user clicks on the item, the agent observes a sequence of user choices y t = ( y tk ) k ∈ [ O t ] where a binary vector y tk = ( y tk 0 , y tk 1 , ..., y tkM ) represents user choices on assortment A tk . Let y tkm = 1 if the m -th item i m in A tk is clicked by the user, and y tkm = 0 for items that are not clicked on. For each assortment, there is an outside option . That is, there is a probability that the user may not click any of the items in A tk . If the user does not choose any items, y tk 0 = 1 and y tkm = 0 for all m ∈ [ M ] . The user choice for each assortment is given by the multinomial logit (MNL) choice model [21]. For this MNL model, there is an unknown time-invariant parameter θ ∗ ∈ R d . We define the true weight of item i in round t as w ∗ ti := x /latticetop ti θ ∗ . Also, we let the vector representation of the weights be defined as w ∗ t := ( w ∗ ti ) i ∈ [ N ] for convenience.

Under this model, the user's click probability of the m -th item in A tk and the probability of the outside option in A tk is given respectively by

where item i 0 represents the outside option. The user choice y tk is sampled from the multinomial distribution, y tk ∼ MNL { 1 , ( p t ( i m | A tk , w ∗ t ) ) M m =0 } , where the argument 1 indicates that y tk is a single-trial sample. Hence, ∑ M m =1 y tkm is always 1. Also, we denote measurement noise as /epsilon1 tkm := y tkm -p t ( i m | A tk , w ∗ t ) . Since /epsilon1 tkm is bounded in [0 , 1] , /epsilon1 tkm is σ 2 -sub-Gaussian with σ 2 = 1 / 4 . It is important to note that /epsilon1 tkm across items in the same assortment is not independent due to the substitution effect in the MNL model.

The expected reward function of a combinatorial action S t based on w ∗ t is given by

The formulation above is also known as the cascade model with disjunctive objective, where the user stops at the first attractive item [13; 14; 18].

2.3 α -Approximation Oracle and α -Regret

The exact combinatorial optimization to compute an optimal cascade of assortments can be computationally expensive. Therefore, we allow for approximate optimization. We assume that the

agent has access to an α -optimization oracle to compute a α -approximation solution of the cascade optimization problem with α ≤ 1 . For approximate optimization, we prove that a greedy selection for the cascading assortment optimization problem gives a 0.5 approximation of the optimal solution, which may be of independent interest (see Section 6).

Formally, for a given an α -optimization oracle and a weight parameter w , the oracle outputs an approximately optimal cascade ˆ S ∗ = O α ( w ) ∈ S satisfying f ( ˆ S ∗ , w ) ≥ α f ( S ∗ , w ) where S ∗ ∈ arg max S ∈ S f ( S, w ) is an optimal assortment without approximation. The instantaneous α -regret of cascade S t in round t is defined as R α ( t, S t ) := E [ α f ( S ∗ t , w ∗ t ) -f ( S t , w ∗ t )] where S ∗ t ∈ arg max S ∈ S f ( S, w ∗ t ) is an true optimal assortment. Then, the goal of the agent is to minimize the cumulative α -regret defined as

2.4 Suboptimal Dependence on Problem Dependent Parameters

In this subsection, we discuss the main technical challenges faced in the regret analysis of our problem setting. In particular, the suboptimal dependence on cascade length K has been a long-standing open problem even in contextual cascading bandits.

2.4.1 Dependence on Length of Cascade K

The previous literature on the contextual cascading bandits [18; 17; 26; 25] bounds the instantaneous regret in each round, utilizing the monotonicity and Lipschitz continuity of the expected reward function f . A simple adaptation of the previously known techniques to our problem would result in the following upper bound for the instantaneous regret R α ( t, S t ) for S t = ( A t 1 , A t 2 , ..., A tK ) .

where β t is a suitable confidence radius chosen by an algorithm, and V t is a positive definite gram matrix. Then, the dependence on the length of cascade K and the assortment size M would appear in the regret bound after summing the right-hand side of Eq.(1) over the time horizon and applying the Cauchy-Schwarz inequality. Because of this reason, even when M = 1 , there still exists dependence on K which appears in the regret bounds of all previous contextual cascading bandits (see Table 1).

To overcome this challenge, we present a new Lipschitz continuity of the expected reward function to derive the regret bound independent of M and K by replacing the summation with the maximum over assortments and a cascade (see Section 4.3 for more details).

2.4.2 Dependence on Worst-Case Scanning Probability

Analogous to the existing algorithms for the contextual cascading bandits [18; 26], the gram matrix V t contains the rank-1 matrices of observed items accumulated up to round t , i.e., V t = ∑ t τ =1 ∑ O τ k =1 ∑ i ∈ A τ k x τ i x /latticetop τ i + λ I . However, there exists an out-of-control issue, that is, the summation of the rank-1 matrices over O t +1 to | S t | in Eq.(1) is not included in the gram matrix V t . Note that this issue also arises in cascading contextual assortment bandits. Adapting a technique used in the existing literature [18] to mitigates this issue, let p t,S t be the probability of examining all assortments in S t and p ∗ = min t ∈ [ T ] min S ∈ S p t,S t be the worst-case probability of examining a cascade over all rounds and all feasible cascades. A simple adaptation of the existing methods would result in the following bound for the expected instantaneous regret.

This concedes the dependence on p ∗ , which can be exponentially small in the worst case. We overcome this challenge by designing an algorithm that offers the assortment containing the most uncertain item as the first assortment in a cascade. We discuss this salient feature of the proposed algorithm in more detail in Section 3.2.

Algorithm 1 UCB-CCA

Input : confidence radius β t and ridge penalty parameter λ ≥ 1 1: for t = 1 , . . . , T do 2: Observe x ti for all i ∈ [ N ] 3: Compute u ti = x /latticetop ti ˆ θ t -1 + β t -1 || x ti || V -1 t -1 for all i ∈ [ N ] 4: Compute a candidate cascade S ′ t ← ( A ′ tk ) k ∈ [ K ] = O α ( u t ) 5: Find optimistic exposure assortment index k ∗ in ( k ∗ , i ∗ ) = argmax k ∈ [ K ] ,i ∈ A ′ tk || x ti || V -1 t -1 6: Optimistic exposure swap S t ← ( A tk ) k ∈ [ K ] where A tk :=    A ′ tk ∗ if k = 1 A ′ t 1 if k = k ∗ A ′ tk otherwise 7: Offer S t , and observe user feedback O t and y t = ( y tk ) k ∈ [ O t ] 8: Update V t ← V t -1 + ∑ O t k =1 ∑ i ∈ A tk x ti x /latticetop ti 9: Compute the regularized MLE ˆ θ t by solving ∇ θ [ /lscript t ( θ ) + λ 2 || θ || 2 2 ] = 0 10: end for

3 Algorithm: UCB-CCA

3.1 Upper Confidence Bounds and Confidence Set

UCB-CCA utilizes the upper confidence bounds (UCB) technique [6; 1; 16] to compute an optimistic action based on optimistic estimates of each item's weight, u ti = x /latticetop ti ˆ θ t -1 + β t -1 ( δ ) || x ti || V -1 t -1 for all i ∈ [ N ] . The confidence radius β t ( δ ) is specified to maintain a high-probability confidence set C t ( δ ) for the unknown parameter θ ∗ , although the algorithm does not explicitly compute C t ( δ ) .

Setting a proper confidence radius β t ( δ ) can guarantee that θ ∗ lies within the confidence set with probability 1 -δ . On the event that θ ∗ ∈ C t ( δ ) , the UCB weight u ti serves as an upper bound of a true weight w ∗ ti := x /latticetop ti θ ∗ for every item i ∈ [ N ] . We denote the UCB weight vector as u t = ( u ti ) i ∈ [ N ] for convenience.

3.2 Optimistic Exposure Swapping

A distinctive element of UCB-CCA is what we call optimistic exposure swapping , a procedure crucial for eliminating dependence on the worst-case scanning probability, as elaborated in Section 2.4.2. This technique strategically positions the assortment containing the item with the highest uncertainty among the top MK items in the first slot of the cascade of assortments.

In each round t , the α -approximate oracle O α ( u t ) outputs a candidate cascade S ′ t , determined by the UCB weights u t . It is important to note that S ′ t is not immediately presented to the user. Instead, after S ′ t is derived using the optimization oracle O α ( u t ) , the algorithm identifies the index k ∗ of an assortment that includes the item with the largest || x ti || V -1 t -1 in S ′ t .

Subsequently, the algorithm swaps the positions: the assortment A ′ tk ∗ is moved to the top of S t , becoming A t 1 , and the initially top assortment A ′ t 1 in S ′ t is relocated to the k ∗ -th position of S t , now A tk ∗ . The positions of the other assortments remain the same, that is, A tk = A ′ tk for all k ∈ [ K ] \ { 1 , k ∗ } . 3 This procedure is viable as the expected reward is unaffected by the display order of assortments in the cascade, as shown in Lemma 4.5.

3.3 Regularized Maximum Likelihood Estimation

UCB-CCA computes a regularized maximum likelihood estimate of the unknown parameter θ ∗ . The negative log-likelihood is given by /lscript t ( θ ) = -∑ t -1 τ =1 ∑ O τ k =1 ∑ M m =0 y τ km log p τ ( i m | A τ k , w τ ) , where

w t = ( w ti ) i ∈ [ N ] is a weight vector, and its element is w ti = x /latticetop ti θ . For penalty parameter λ ≥ 1 , the /lscript 2 -regularized MLE is given by

4 Regret Analysis of UCB-CCA

4.1 Regularity Condition

Assumption 4.1. x 1 for all round t and items i [ N ] , and also θ ∗ 1 .

Assumption 4.2. There exists κ > 0 such that for all t ∈ [ T ] , any assortment A ∈ A , and any item i ∈ A , inf θ ∈ R d p t ( i | A,w ) p t ( i 0 | A,w ) ≥ κ , where w = ( w i ) i ∈ [ N ] and w i = x /latticetop i θ .

|| || ti ≤ ∈ || || ≤

Discussion of Assumptions. Assumption 4.1 makes the regret bound independent on the scale of the feature vector and parameter. This is the standard assumption used in the contextual bandit literature [1; 18; 22]. Assumption 4.2 is the standard regularity assumption in the contextual assortment bandit literature [8; 27; 22; 7; 23], adapted from the standard assumption for the link function in the generalized linear contextual bandit literature [16] to ensure that the Fisher information matrix is non-singular.

4.2 Regret Bound of UCB-CCA

Theorem 4.3 ( α -regret upper bound of UCB-CCA ) . Suppose Assumptions 4.1 and 4.2 hold, and we run UCB-CCA for total T rounds with β t = 1 2 κ √ d log ( 1 + tKM d λ ) +4log t + √ λ κ and with λ ≥ 1 , Then, the α -regret of UCB-CCA is upper-bounded by

Discussion of Theorem 4.3. Theorem 4.3 establishes that UCB-CCA achieves a regret bound of ˜ O ( d κ √ T ) . Notably, this regret bound removes dependence on p ∗ completely and removes polynomial dependence on K , achieving the best-known bound in contextual cascading bandits [18; 25; 20], a special case of our problem setting. Apart from the generalization we consider in this work, a key distinction between our work and the previous contextual cascading bandit models lies in our adoption of the MNL model, as opposed to the linear model assumed by the existing literature. This model choice introduces a dependence on the parameter κ within the regret bound of UCB-CCA , which is an aspect we address and refine in subsequent sections. It is essential to highlight that our work tackles a more general problem yet achieves improved bounds concerning the key problem parameters previously considered suboptimal. The factor comprised of the length of the cascade, ( K/ ( K +1)) K +1 , in Theorem 4.3 is also notable, which is bounded above by 1 regardless of the value of K . Consequently, the regret bound does not increase polynomially with K , ensuring scalability.

4.3 Proof Outline

In this subsection, we present the proof sketch of Theorem 4.3. One of the key components of the regret analysis that enables carving off the dependence on K is the following lemma.

Lemma 4.4 (Maximal Lipschitz continuity) . Suppose u ti ≥ w ∗ ti for all i ∈ [ N ] . Then

Lemma 4.4 demonstrates that the difference between the reward functions under the UCB parameter and the true parameter is upper bounded by the maximal difference between the UCB and true parameters. This implies that the regret bound remains unaffected by increases in the cascade length K or the assortment size M .

Eliminating the dependence on p ∗ is another key element of our analysis. To this end, we first show that the order of assortments in the cascade model with the disjunctive objective does not affect the expected reward. We formalize this property in the following lemma.

Lemma 4.5. Let p k be the probability that the user clicks on any item in A k . Given a collection of assortments { A 1 , · · · , A K } with probabilities { p 1 , · · · , p K } , their order of display does not matter. Further, for every permutation ρ : [ K ] → [ K ] , we have

Consolidating these key results, we proceed to bound the cumulative regret. We begin by leveraging the monotonicity of the expected reward function and the definition of the α -approximate optimization oracle to bound the cumulative regret.

The second inequality is from Lemma 4.4, letting C K := ( K/ ( K +1)) K +1 . The third inequality is given by the concentration of the UCB weights (see Lemma B.5). Note that the assortment including the item with the largest value of || x ti || V -1 t is always examined by the user since it is included in the first assortment of S t by the optimistic exposure swapping technique as described in Section 3.2. Note that a change in the order of assortments incurred by the optimistic exposure swapping does not affect the expected reward which is shown in Lemma 4.5. Hence, for every round t ∈ [ T ] , we obtain

Therefore, the last equality in Eq.(4) is given by Eq.(5). Then, we can apply the maximal version of elliptical potential lemma (see Lemma B.8) to bound the cumulative regret.

5 Improved Dependence on κ

While UCB-CCA achieves the regret bound of ˜ O ( 1 κ d √ T ) improving dependence on K , the bound includes the problem-dependent constant κ . This implies a potential risk of the regret bound becoming large when κ becomes very small. In order to circumvent this challenge, we propose a new optimism in the face of uncertainty (OFU) algorithm, UCB-CCA+ , which exhibits a regret bound that is independent of κ in the leading term. The pseudocode of UCB-CCA+ is detailed in Algorithm 2.

5.1 Algorithm: UCB-CCA+

5.1.1 Confidence Set

UCB-CCA+ computes a regularized MLE ˆ θ t , following the same procedure described in Section 3.3. Then, the algorithm constructs a new confidence set centered around ˆ θ t utilizing a Bernstein-type tail inequality for self-normalized martingales [11; 3]. Nevertheless, a simple adaptation of the previous approaches may incur increased dependence on M . Hence, a more intricate analysis and refined algorithmic strategy are imperative to effectively address this challenge.

First, we define g t ( θ ) := ∑ t τ =1 ∑ k ∈ [ O τ ] ∑ i ∈ A τ k p τ ( i | A τ k , w τ ) x τ i + λ t θ where w t = ( w ti ) i ∈ [ N ] and w ti = x /latticetop ti θ . We also denote the partial derivative of p t ( i | A tk , w t ) with respect to w ti as ˙ p t ( i | A τ k , w τ ) := p t ( i | A tk , w t ) p t ( i 0 | A tk , w t ) . Additionally, we define the new design matrix containing local information, denoted as H t ( θ ) := ∑ t τ =1 ∑ k ∈ [ O τ ] ∑ i ∈ A τ k ˙ p τ ( i | A τ k , w τ ) x τ i x /latticetop τ i + λ t I d , and, for convenience, H t := ∑ t τ =1 ∑ k ∈ [ O τ ] ∑ i ∈ A τ k ˙ p τ ( i | A τ k , w ∗ τ ) x τ i x /latticetop τ i + λ t I d . Then, the algorithm constructs a confidence set as below:

Algorithm 2 UCB-CCA+ Input : confidence radius γ t and ridge penalty parameter λ ≥ 1 1: for t = 1 , . . . , T do 2: Observe x ti for all i ∈ [ N ] 3: Construct a confience set B t -1 ( δ ) as defined in Eq.(6) 4: Compute a candidate cascade ( S ′ t = ( A ′ tk ) k ∈ [ K ] , θ t ) = arg max S ∈ S , θ ∈ B t ( δ ) f ( S, w t ) 5: Find optimistic exposure assortments L t,H and L t,V (and their positions h and v ) in S ′ t 6: S t ← ( A tk ) k ∈ [ K ] where A tk =              L t,H if k = 1 L t,V if k = 2 A ′ t 1 if k = h A ′ t 2 if k = v A ′ tk otherwise 7: Offer S t and observe O t , y t = ( y tk ) k ∈ [ O t ] 8: Update H t ← H t -1 + ∑ O t k =1 ∑ i ∈ A tk ˙ p t ( i | A tk , w ∗ t ) x ti x /latticetop ti 9: Update V t ← V t -1 + ∑ O t k =1 ∑ i ∈ A tk x ti x /latticetop ti 10: Compute the regularized MLE ˆ θ t by solving ∇ θ [ /lscript t ( θ ) + λ 2 || θ || 2 2 ] = 0 11: end for

where the confidence radius γ t ( δ ) is suitably specified to ensure that the true parameter θ ∗ lies in the confidence set B t ( δ ) with high probability. On the event of θ ∗ ∈ B t ( δ ) , The following lemma bounds the weighted /lscript 2 -norm of the difference between θ and θ ∗ .

Lemma 5.1. Suppose θ ∗ ∈ B t ( δ ) . Then, for any θ ∈ B t ( δ ) , we have || θ -θ ∗ || H t ≤ 6 γ t ( δ ) .

5.1.2 Doubly Optimistic Exposure Swapping

UCB-CCA+ still faces the challenge outlined in Section 2.4.2. The complication is exacerbated for UCB-CCA+ as the algorithm concurrently updates two gram matrices, H t and V t , which only contain the information of the observed items. Building upon the technique of optimistic exposure swapping detailed in Section 3.2, in each round t , UCB-CCA+ assigns the assortment L t,H -containing the item with the largest uncertainty with respect to H t -1 among the top KM items - to the first slot of cascade S t . Similarly, the algorithm places L t,V -with the item that has the largest uncertainty with respect to V t -1 -in the second slot of S t .

5.2 Regret Analysis of UCB-CCA+

Theorem 5.2 (Regret upper bound of UCB-CCA+ ) . Suppose Assumptions 4.1 and 4.2 hold, and we run UCB-CCA+ for total T rounds with λ ≥ 1 and γ t ( δ ) := 3 √ λ t 2 + 2 √ λ t log ( ( λ t + KMt/d ) d/ 2 λ -d/ 2 t δ ) +

where C 1 = 36 and C 2 = 216(1 + Me ) .

Discussion of Theorem 5.2. Theorem 5.2 establishes the regret bound of ˜ O ( d √ T ) . The leading term of the regret bound is independent of κ . Although the second term exhibits dependence on κ , the term only scales logarithmically in T , whose comparative effect diminishes as t increases compared to the leading term. Hence, the worst-case regret guarantee of UCB-CCA+ improves from that of UCB-CCA . The comprehensive proof of Theorem 5.2 is provided in the appendix.

6 Approximation Algorithm for Optimal Combinatorial Action

In this section, we show that computing the optimal cascade is, in general, a weakly NP-hard problem and give a (fast) polynomial time algorithm that the agent can use to compute a 0 . 5 -approximation to

the optimal cascade for any weight w . We assume that the MNL weights are given and consider the problem of finding the cascade that maximizes the expected reward. This is an optimization problem on selecting a sequence of K assortments with size M each from a ground set of N items. The number of feasible cascades is O ( ( N M ) K ) . In fact, we establish the following hardness result.

Lemma 6.1. For general M , the optimization problem is weakly NP-hard even for K = 2 .

The hardness of our problem follows from the hardness result of [19] for a related setting of unconstrained cascade optimization where the size of each assortment can be arbitrary. In light of this hardness, we turn our attention to finding fast approximation algorithms for the problem. While there has been a lot of recent work on the unconstrained cascade optimization problem (see [19; 12; 10] and the references therein), none of the previous algorithms apply to our constrained setting (where the size of each assortment is M ).

We consider the following greedy approach for the problem. Consider arbitrary weights at round t , i.e. w t = ( w ti ) i ∈ [ N ] , is given. We order the items in [ N ] in decreasing order of given weights and consider the following cascade of assortments.

We call these assortments 'decreasing order assortments'. Let OPT denote the value of the optimal solution to the problem. We establish that showing these assortments in any possible sequences gives a good approximation to the problem.

Lemma 6.2. Let OPT denote the overall click probability in the optimal solution. The decreasing order assortments D 1 , ..., D K , shown in any order, have overall click probability at least 0 . 5 OPT .

We present the proof in Appendix E. We also show that our algorithm is optimal for M = 1 , which captures the classic cascade optimization problem.

7 Numerical Experiments

In this section, we evaluate the performances of our proposed algorithms UCB-CCA and UCB-CCA+ in numerical experiments and compare their performances with the existing combinatorial bandit algorithms CombCascade and C 3 -UCB . For simulations, we generate a random sample of the unknown time-invariant parameter θ ∗ from N (0 , 1) at the beginning of the simulation. We sample N feature vectors from N (0 , 1) in each round t . At each round t , the oracle computes a sequence of assortments in decreasing order, forming a cascade S t based on given w t . We assess the cumulative regret of UCB-CCA , UCB-CCA+ , CombCascade , and C 3 -UCB . Note that a user's choice for UCB-CCA and UCB-CCA+ is determined by the MNL logit choice model, whereas C 3 -UCB utilizes a linear model and CombCascade is a non-contextual model. Figure 2 and Figure 3 indicate that both UCB-CCA and UCB-CCA+ significantly outperform C 3 -UCB and CombCascade . We also observe that UCB-CCA+ shows a slight performance advantage over UCB-CCA , although the difference is not statistically significant. While UCB-CCA+ has a sharper worst-case regret guarantee, UCB-CCA can provide favorable practical performances, with simpler implementation and computational efficiency.

Acknowledgments and Disclosure of Funding

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (RS-2023-00222663, No. 2022R1C1C1006859, No. 2022R1A4A103057912, No. 2021M3E5D2A01024795) and by Creative-Pioneering Researchers Program through Seoul National University and by Naver.

References

Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems , 24, 2011.

Marc Abeille, Louis Faury, and Clément Calauzènes. Instance-wise minimax-optimal algorithms for logistic bandits. In International Conference on Artificial Intelligence and Statistics , pages 3691-3699. PMLR, 2021.

Priyank Agrawal, Theja Tulabandhula, and Vashist Avadhanula. A tractable online learning algorithm for the multinomial logit contextual bandit. European Journal of Operational Research , 2023.

Shipra Agrawal, Vashist Avadhanula, Vineet Goyal, and Assaf Zeevi. Thompson sampling for the mnlbandit. In Conference on Learning Theory , pages 76-78. PMLR, 2017.

Shipra Agrawal, Vashist Avadhanula, Vineet Goyal, and Assaf Zeevi. Mnl-bandit: A dynamic learning approach to assortment selection. Operations Research , 67(5):1453-1485, 2019.

Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research , 3(Nov):397-422, 2002.

Xi Chen, Yining Wang, and Yuan Zhou. Dynamic assortment optimization with changing contextual information. Journal of machine learning research , 2020.

Wang Chi Cheung and David Simchi-Levi. Assortment optimization under unknown multinomial logit choice models. arXiv preprint arXiv:1704.00108 , 2017.

Wang Chi Cheung and David Simchi-Levi. Thompson sampling for online personalized assortment optimization problems with multinomial logit choice models. Available at SSRN 3075658 , 2017.

Elaheh Fata, Will Ma, and David Simchi-Levi. Multi-stage and multi-customer assortment optimization with inventory constraints. arXiv preprint arXiv:1908.09808 , 2019.

Louis Faury, Marc Abeille, Clément Calauzènes, and Olivier Fercoq. Improved optimistic algorithms for logistic bandits. In International Conference on Machine Learning , pages 3052-3060. PMLR, 2020.

Pin Gao, Yuhang Ma, Ningyuan Chen, Guillermo Gallego, Anran Li, Paat Rusmevichientong, and Huseyin Topaloglu. Assortment optimization and pricing under the multinomial logit model with impatient customers: Sequential recommendation and selection. Operations research , 69(5):1509-1532, 2021.

Branislav Kveton, Csaba Szepesvari, Zheng Wen, and Azin Ashkan. Cascading bandits: Learning to rank in the cascade model. In International conference on machine learning , pages 767-776. PMLR, 2015.

Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. Combinatorial cascading bandits. Advances in Neural Information Processing Systems , 28, 2015.

Tor Lattimore and Csaba Szepesvári. Bandit algorithms . Cambridge University Press, 2020.

Lihong Li, Yu Lu, and Dengyong Zhou. Provably optimal algorithms for generalized linear contextual bandits. In International Conference on Machine Learning , pages 2071-2080. PMLR, 2017.

Shuai Li and Shengyu Zhang. Online clustering of contextual cascading bandits. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.

Shuai Li, Baoxiang Wang, Shengyu Zhang, and Wei Chen. Contextual combinatorial cascading bandits. In International conference on machine learning , pages 1245-1253. PMLR, 2016.

Nan Liu, Yuhang Ma, and Huseyin Topaloglu. Assortment optimization under the multinomial logit model with sequential offerings. INFORMS Journal on Computing , 32(3):835-853, 2020.

Xutong Liu, Jinhang Zuo, Siwei Wang, John CS Lui, Mohammad Hajiesmaili, Adam Wierman, and Wei Chen. Contextual combinatorial bandits with probabilistically triggered arms. In International Conference on Machine Learning , pages 22559-22593. PMLR, 2023.

Daniel McFadden. Modeling the choice of residential location. Transportation Research Record , (673), 1978.

Min-hwan Oh and Garud Iyengar. Thompson sampling for multinomial logit contextual bandits. Advances in Neural Information Processing Systems , 32, 2019.

Min-hwan Oh and Garud Iyengar. Multinomial logit contextual bandits: Provable optimality and practicality. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9205-9213, 2021.

Noemie Perivier and Vineet Goyal. Dynamic pricing and assortment under a contextual mnl demand. Advances in Neural Information Processing Systems , 35:3461-3474, 2022.

Daniel Vial, Sujay Sanghavi, Sanjay Shakkottai, and R Srikant. Minimax regret for cascading bandits. arXiv preprint arXiv:2203.12577 , 2022.

Kun Wang. Conservative contextual combinatorial cascading bandit. IEEE Access , 9:151434-151443, 2021.

Xue Wang, Mike Mingcheng Wei, and Tao Yao. Online assortment optimization with high-dimensional data. Available at SSRN 3521843 , 2019.

Zixin Zhong, Wang Chi Chueng, and Vincent YF Tan. Thompson sampling algorithms for cascading bandits. The Journal of Machine Learning Research , 22(1):9915-9980, 2021.

Shi Zong, Hao Ni, Kenny Sung, Nan Rosemary Ke, Zheng Wen, and Branislav Kveton. Cascading bandits for large-scale recommendation problems. In Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence , 2016.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the contextual cascading assortment bandit problem, and proposes low regret algorithms for this problem.\n\nSTRENGTHS:\n1. This paper studies a novel problem by combining ideas from assortment bandits and cascading bandits. \n\t2. The paper introduces some new algorithmic ideas.\n\nWEAKNESSES:\nIt seems to me the optimal cascading assortment should obey some structural properties, i.e., should similar items appear in the same assortment? Should different items appear in the same assortment? Should we pair desirable items together? Or should we pair high-weight and low-weight items together for an ‘anchoring effect’? It will be interesting to hear insights on this. \n\nThe writing contains noticeable grammatical errors and typos\n\nThe reward function in this work seems to greatly simplify the problem and removes the difficulty in prior work. See questions section below. \n\n---------------\nUpdate. Some issues in writing include:  \n1. several sentences in paragraph from line 215-227\n2. line 272, the $Ht^{-1}$ I assume should be subscript. \n3. Not writing out the constant in line 268 and 278 is strange.\n\nQUESTIONS:\n1. The first claim the paper makes is that they are able to remove the dependence on K in the cascading bandit problem by using a swapping technique. However, I believe this depends on the specific loss function used, therefore this claim is not entirely accurate. Specifically, the swapping technique only holds when the order does not matter, but this does not generally hold for cascading bandits. \n2. The second claim is the regret removes $\\kappa$ term. However, the regret definition is different from assortment bandits, therefore is this claim really accurate? \n3. In section 2.4, is the description talking about the cascading bandit problem or cascading assortment bandit problem?\n\nLIMITATIONS:\nThe author did not address limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a new contextual combinatorial multi-armed bandit model, which generalizes the contextual cascading bandits and assortment bandits.  For the offline problem when item parameters are known, the authors propose a 0.5-approximate solution. For the online problem where parameters are not known a priori, the authors first propose a UCB algorithm, called UCB-CCA, which yields a regret bound of $\\tilde{O}(\\kappa^{-1}d\\sqrt{T})$ regret bound. To remove the unsatisfying $\\kappa$ which may be relevant to the cascade length $K$, the authors further leverage Bernstein-type concentration and propose a new algorithm UCB-CCA+, which removes the $\\kappa^{-1}$ dependence and achieves regret bounds that are independent of $K$. Finally, the authors conduct experiments to show the practical efficacy of the proposed methods.\n\nSTRENGTHS:\nOverall, I feel this is a decent work that is suitable to put in the combinatorial MAB literature.\n\n1. From the model perspective, the model is new and general, which covers contextual cascading bandits and MNL bandits as degenerate cases.\n2. For the results, Table 1 gives a clear comparison with existing works and this paper gives the first regret bound for this new model. Interestingly, when the \n3. For the analysis, this paper not only is grounded by existing works from MNL bandit, but also gives some new techniques \n4. For the writing, it is clear and intuitive supported by intuitive figures and tables.\n\nWEAKNESSES:\nOverall, I do not have major concerns, yet I have some minor comments, which I hope to get some clarification to validate my understanding.\n\n1. In line 8 of Algorithm UCB-CCA+, the algorithm uses the true parameter $w_t^*$ which is unknown, is it a typo?\n2. In line 4 of Algorithm UCB-CCA+, it is a combinatorial optimization problem over a confidence radius $B_t(\\delta)$, it is NP-hard in general? Is any computational-efficient method for this?\n3. I cannot find the lower bound result for the current problem. Is the current result matches the lower bound?\n\nQUESTIONS:\nPlease comment point 1,2,3 in the above weakness part.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations and there are no potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the Cascading Contextual MNL bandits problem. Two effective algorithms UCB-CCA and UCB-CCA+ are proposed. Compared to existing cascading bandits and MNL bandits, the regrets of the two algorithms have some better dependence on the length of cascades and \\kappa. Numerical simulations demonstrate the effectiveness of the proposed algorithms.\n\nSTRENGTHS:\nS1. Combining cascading bandits and MNL bandits is interesting and can find real applications.\n\nS2. The proposed UCB-CCA algorithm has a regret independent of K, the length of cascades.\n\nS3. The results of numerical simulations are good.\n\nWEAKNESSES:\nW1. This paper focuses on removing the dependence on K and \\kappa. It seems to me that improving the dependence on d may be more helpful than removing the dependence on K, since in practice K could be a small constant and the dimension of contextual vectors could be large. Note that [18] has an algorithm for MNL contextual bandits that has a \\sqrt{dT} regret.\n\nW2. In theorem 5.2, the dependence on \\kappa is removed by increasing T. Such a treatment explicitly assumes that T is much larger than 1/\\kappa, which makes some sense in practice. However, I am not sure if this practical assumption is appropriate in the theoretical analysis of regret. \n\nW3. In the numerical simulations, the authors report the cumulative regrets of algorithms. What about the curves of revenues of algorithms? Since all algorithms adopted in experiments have sublinear regrets, the cumulative regret is actually an insignificant term compared to the revenue. I wonder if the difference between the proposed algorithms and the baseline is still as large as reported in Fig 2 when reporting the curves of cumulative revenues.\n\nW4. The submission file and the full version (Supplementary Material) are inconsistent with each other in some parts.\n\nQUESTIONS:\nQ1. For UCB-CCA+, as \\kappa is removed in the regret, does it mean that we only need the assumption that the Fisher information matrix is invertible?\n\nLIMITATIONS:\nThe authors discuss the limitations of their work in Appendix.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a new combinatorial bandit problem that generalizes the existing cascading and assortment bandits. The authors first propose a UCB-based algorithm, UCB-CCA, that achieves a tighter regret bound than existing bounds for cascading contextual bandits by eliminating the dependence on cascade length $K$. They also introduce an improved algorithm, UCB-CCA+, and use a Bernstein-type concentration to prove a regret bound without $\\kappa^{-1}$ dependence, where $\\kappa$ is a problem-dependent constant in the regret bound of UCB-CCA. Numerical experiments validate the effectiveness of the proposed algorithms.\n\nSTRENGTHS:\n1) This paper is the first to study the combination of contextual cascading and assortment bandits. This new problem is well-motivated by real-world applications in recommender systems.\n2) One of the main technical contributions is the new Lipschitz continuity of the expected reward function for contextual cascading assortment bandits in Lemma 4.2, which helps prove the regret bound of UCB-CCA is independent of $K$ and $M$. (However, I have a question about the proof of this Lipschitz continuity; see below.)\n3) The proposed UCB-CCA+ algorithm achieves an improved regret bound than that of UCB-CCA, solving the two technical challenges (dependence on cascade length and $\\kappa$) faced by contextual cascading and assortment bandits simultaneously.\n\nWEAKNESSES:\n1) For contextual combinatorial bandits, there is a recent result [A] that provides a regret bound independent of the cascade length $K$ using a variance adaptive algorithm. Moreover, its regret bound can get rid of $p^*$, which raises a concern that whether the optimistic exposure swapping in Section 3.3 is necessary or whether the $p^*$ issue can be resolved by a more involved analysis.\n2) The Lipschitz continuity in Lemma 4.2 is a key component of the analysis. However, the proof in line 402-405 is unclear to me. I would appreciate it if the authors could add more details about the proof; a simple example of the non-contextual cascading bandit can also be helpful.\n3) Although UCB-CCA+ achieves a good regret bound, there is no discussion on the lower bound of contextual assortment combinatorial bandits: would it be similar to that of the contextual combinatorial bandits or contextual assortment bandits?\n\n[A] Xutong Liu, Jinhang Zuo, Siwei Wang, John CS Lui, Mohammad Hajiesmaili, Adam Wierman, and Wei Chen. Contextual Combinatorial Bandits with Probabilistically Triggered Arms. In International Conference on Machine Learning, 2023.\n\nQUESTIONS:\nTypo: line 207: In round (at every round) $t$\n\nLIMITATIONS:\nSee Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the cascading contextual assortment bandit problem and provides a UCB type algorithm. This problem is motivated by online content recommendation systems. They develop a UCB algorithm that is applicable to this problem setting, and prove that their algorithm improves upon existing regret bound rates in the cascading and assortment bandit problems respectively.\n\nSTRENGTHS:\n- Based on the authors' discussion, the scaling rate of their regret bounds is both sharper and more interpretable/intuitive than that in existing literature. It seems their regret bound improves upon those both in the assortment and cascading bandit literatures respectively.\n- The authors have a nice discussion about why we expect the regret to decrease with $K$ and then show how their result shows this type of dependence. Additionally, their result and discussion of how their regret bound scales with $\\kappa$ in a way that is not worsening its dependency on $M$ seems nice.\n- The overall writing and presentation in the paper pretty good. I think the table 1 is quite useful and Figure 1 was very helpful for understanding the problem.\n\nWEAKNESSES:\n- In the evaluation, it was not clear to me why you only compared to C^3-UCB and not the other methods listed in table 1. While the algorithm has these regret guarantees, its not entirely clear if the algorithm really performs well in practice from the simulations in the evaluation section.\n- Only applicable to environments in which the probability that a user clicks is determined by a generalized linear model. Additionally, the feature vector $x_{ti}$ captures both contextual information on the user and the item. It's not clear how this kind of vector could be chosen in practice.\n- There is not a real related works section in the main paper. I see there is one in the Appendix.\n\nQUESTIONS:\n- It is not clear to me that Assumption 2.3 as written will ever hold. Do you really need to take an infinum over all $\\theta \\in \\mathbb{R}^d$? Based on my understanding, this means that if you take $\\theta = \\lambda \\cdot x_i$ you can make $w_i = x_i^\\top \\theta$ arbitrarily small or large with the choice of $\\lambda$. Then, based on the model from line 127, it seems you could make $p_t(i_m | A_{tk}, w)$ arbitrarily small. Can you explain there any reasonable settings where Assumption 2.3 will hold? If there are, please have a discussion of this and also more information about how to interpret Assumption 2.3.\n- You say below assumption 2.2 that the regret bound is $c$ times larger if you allow the norm of $x_i$ and $\\theta^\\star$ respectively to be bounded below $c$. Is \\emph{knowledge} of $c$ needed by the algorithm? In other words, does your algorithm currently implicitly take advantage of knowledge that you assume that $x_i$ and $\\theta^\\star$ are bounded by $1$? If so, this would severely limit the practical applicability of the approach. If this is the case, this limitation should be discussed. If it is not, it should also be mentioned when discussing scaling by $c$.\n- Could you a sentence or two (or more) about how you suggest to choose ridge penalty $\\lambda$?\n\nLIMITATIONS:\n- I would like the authors to address the questions / limitations I list in the weaknesses section.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-26"}