SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Hyun-jun Choi
nschj1@snu.ac.kr
Rajan Udwani
Berkeley, CA, USA
rudwani@berkeley.edu
Min-hwan Oh
minoh@snu.ac.kr
We present a new combinatorial bandit model, the cascading contextual assortment bandit. This model serves as a generalization of both existing cascading bandits and assortment bandits, broadening their applicability in practice. For this model, we propose our first UCB bandit algorithm, UCB-CCA. We prove that this algorithm achieves a  $T$ -step regret upper-bound of  $\tilde{\mathcal{O}}\left(\frac{1}{\kappa} d\sqrt{T}\right)$ , sharper than existing bounds for cascading contextual bandits by eliminating dependence on cascade length  $K$ . To improve the dependence on problem-dependent constant  $\kappa$ , we introduce our second algorithm, UCB-CCA+, which leverages a new Bernstein-type concentration result. This algorithm achieves  $\tilde{\mathcal{O}}(d\sqrt{T})$  without dependence on  $\kappa$  in the leading term. We substantiate our theoretical claims with numerical experiments, demonstrating the practical efficacy of our proposed methods.
Sequential interactions between users and a recommender agent are often modeled as the multi-armed bandit problem or one of its variants [15]. In practice, a user typically encounters multiple items per round of interaction rather than a solitary item. Two popular models that capture this aspect are the cascading bandit [13; 14; 18] and the assortment bandit [4; 5; 7; 22], also often known as multinomial logistic bandits.
In the cascading bandit problem [13, 14, 18, 29, 25, 28], the agent selects a cascade of  $K$  items from a total of  $N$  items each round. These selected items are sequentially presented one at a time to a user. If the user clicks on a presented item, the cascading round ends. If not, the agent proceeds to reveal the next item from the cascade. This process continues until either the user clicks on an item or all  $K$  items in the cascade have been presented without a click. Once a round ends, a next round commences with a newly selected list of  $K$  items.
In the assortment bandit problem [4; 5; 7; 8; 9; 7; 22; 23], the agent presents an assortment of  $M$  items all at once, then receives user choice feedback on the assortment. A user may opt for one of the  $M$  items presented or choose none at all, concluding the round in either case. Both cascading and assortment bandit problems are significant combinatorial variations of the multi-armed bandit problem and have been extensively examined both theoretically and in practice.
However, a more commonly encountered scheme in real-world applications is a generalization of these two settings, where a cascade of assortments is sequentially revealed in each round. This approach is evident in video streaming services, where assortments of recommended contents are revealed as users scroll through webpages or mobile applications. Similar experiences can also be found in various online retail services and search engines. To address this, we propose a new interactive model, which we term cascading assortment bandits. In the cascading assortment bandits, the agent chooses a cascade of assortments that consists of  $K$  assortments with each assortment containing  $M$  items. The agent reveals one assortment at a time in the cascade. The cascade concludes if the user clicks on one of the items contained in a given assortment. If not, the agent proceeds
Figure 1: Comparisons between the cascading assortment bandit and the other combinatorial bandits. The cascading assortment bandit subsumes the multi-armed bandit ( $K = 1$ ,  $M = 1$ ), the cascading bandit ( $K > 1$ ,  $M = 1$ ), and the assortment bandit ( $K = 1$ ,  $M > 1$ ).
to unveil the subsequent assortment in the cascade. This cycle continues until either an assortment receives a click from the user or the agent depletes the pre-selected assortments.
The cascading assortment bandit problem is the strict generalization of both the cascading bandits model and the assortment bandits model. It is also a generalization of the simple multi-armed bandit problem. That is, if  $K = 1$  and  $M = 1$ , the problem is the simple multi-armed bandit. If  $K > 1$  and  $M = 1$ , the problem corresponds to the cascading bandit. If  $K = 1$  and  $M > 1$ , we recover the assortment bandit problem. The illustrations on comparisons between the cascading assortment bandit and the other combinatorial bandits are presented in Figure [1].
In order to accommodate the generalization of the interactive model across items and assortments, we also incorporate the feature information of items and parametrization of a click model in the cascading assortment bandit model. Hence, we name the model as cascading contextual assortment bandit (see Section 2.2 for the formal definition of the problem setting). Under this newly proposed combinatorial bandit model, we posit the following question:
Can we design a provably efficient algorithm for cascading contextual assortment bandits?
To address the question at hand, we first have to overcome the technical challenges inherent in each special case of our problem setting: the cascading contextual bandit and the contextual assortment bandit. Firstly, in the cascading contextual bandit [18; 25], a longstanding issue has been the suboptimal dependence on the cascade length,  $K$ . Intuitively, one would expect that as  $K$  increases in the cascading model, the regret should either diminish or at least remain constant; performance deterioration should not occur. However, all existing regret bounds for cascading contextual bandits scale proportionally to  $K$  [18; 29; 17; 28; 25]. This finding is not just counter-intuitive, but also suboptimal (for further discussions, refer to Section 2.4). As a result, (i) eradicating the suboptimal dependence on cascade length  $K$  has been recognized as an open problem, even within the cascading contextual bandit setting.
Further, in the context of assortment bandits, there is a widely recognized suboptimal dependence on the problem-specific constant  $\kappa$ , as demonstrated in the existing assortment bandit literature [9; 22; 23]. This problem-specific constant  $\kappa$  (in Assumption 4.2) represents the curvature of the multinomial logit (MNL) function. Recent studies [24; 3] have demonstrated an improved dependence on  $\kappa$ , albeit only multiplied by logarithmic factors. However, this improvement comes at the expense of an increased dependence on the assortment size  $M$ , a conclusion that is both counter-intuitive and suboptimal. Thus, (ii) decreasing the  $\kappa$  dependence without escalating the dependence on  $M$  still poses an unresolved issue. While addressing either of the two challenges (i) and (ii) can be daunting individually, tackling both issues simultaneously poses an even greater challenge in both our algorithm design and regret analysis.
<sup>1</sup>Note that a non-contextual version of the cascading assortment bandit is a special case of the cascading contextual assortment bandit with a one-hot encoded feature vector for each item. Hence, when we aim to provide efficient algorithms for cascading contextual assortment bandit, we also address the non-contextual cascading assortment bandit which has not been studied previously.
2A concurrent work [20] addresses this suboptimal dependence on  $K$  under the linear model assumption. Our work tackles this challenge under the MNL model in a more general setting.
Table 1: Comparisons of algorithms for contextual cascade and assortment bandits as well as for cascading contextual assortment bandits.  $N$  is the number of ground items,  $K$  is a length of cascade,  $d$  is a dimension of feature vectors and  $T$  is total rounds.  $\kappa$  is a problem-dependent parameter for the MNL model. See Appendix A for more discussions.
Algorithm	Context	Cascade	Assortment	Click Model	Regret Bound
CombCascade [14]	×	○	×	×	O(√KNT)
C3-UCB [18]	○	○	×	Linear	O(d√KT)
EE-MNL [5]	×	×	○	MNL	O(√NT)
TS-MNL [22]	○	×	○	MNL	O(1/κd3/2√T)
UCB-MNL [23]	○	×	○	MNL	O(1/κd√T)
LinTS-Cascade [28]	○	○	×	Linear	O(d3/2K√T)
CascadeWOFUL [25]	○	○	×	Linear	O(√d2T+dTK)
VAC2-UCB [20]	○	○	×	Linear	O(d√T)
UCB-CCA (Algorithm 1)	○	○	○	MNL	O(1/κd√T)
UCB-CCA+ (Algorithm 2)	○	○	○	MNL	O(d√T)
To this end, we design novel upper confidence bound (UCB) algorithms for contextual cascading assortment bandits, tackling both technical challenges. We show that our proposed algorithms achieve provable guarantees on regret performances overcoming the longstanding technical challenges. Our regret bounds show sharper results than those of the existing contextual cascading bandits or assortment bandits. We corroborate our theoretical claims through numerical experiments, thus ensuring that both our newly proposed bandit framework and the proposed algorithms establish provable efficiency and practical applicability.
Our main contributions are summarized as follows.
- We formulate a general combinatorial bandit model, named cascading contextual assortment bandit that encompasses the existing cascading bandits and assortment bandits. This novel problem setting is observed in many practical applications.
- We first propose a UCB bandit algorithm UCB-CCA for the cascading contextual assortment bandit and establish the  $T$ -step regret upper-bound of  $\tilde{\mathcal{O}}\left(\frac{1}{\kappa} d\sqrt{T}\right)$  (in Theorem 4.3). This regret bound is tighter than the existing bounds for cascading contextual bandits, where we not only remove the longstanding, unnecessary dependence on  $K$  but also establish the result without dependence on  $M$ .
- While UCB-CCA is an efficient algorithm achieving both the statistical efficiency and practical performances (shown in Section 7), its regret bound includes dependence on the inverse of a problem-dependent constant  $\kappa$ , which can be potentially large in the worst case. To improve the dependence on  $\kappa$ , we propose our second algorithm UCB-CCA+, which exploits a new Bernstein-type concentration result, taking into account the effects of the local curvature of the MNL model. We prove that UCB-CCA+ achieves  $\tilde{\mathcal{O}}(d\sqrt{T})$  without the dependence on  $\kappa$  in the leading term (only scaling with logarithmic factors), hence significantly improving the regret of UCB-CCA without increasing the other dependencies. Hence, we successfully solve the two technical challenges (i) and (ii) mentioned above.
- As an independent contribution, we prove that a greedy algorithm for the cascading assortment optimization problem gives a 0.5 approximation of the optimal solution (discussed in Section 6). To our best knowledge, this is the first rigorous result showing the approximation guarantee even for the contextual cascading bandit problem, instead of simply assuming access to an approximation optimization oracle.
- We evaluate our proposed methods in numerical experiments and show that the practical performances support our theoretical claims. Hence, our proposed algorithms along with our newly proposed bandit model establish provable efficiency and practical applicability.
Define  $[n]$  as a set of positive integers from 1 to  $n$ . Let  $|\cdot|$  be the length of a sequence or the cardinality of a set. For a vector  $x \in \mathbb{R}^d$ , we denote the  $\ell_2$ -norm of  $x$  as  $||x||_2$  and the  $V$ -weighted norm of  $x$  for a positive-definite matrix  $V$  as  $||x||_V = \sqrt{x^\top V x}$ . The determinant and trace of a matrix  $V$  are  $\operatorname{det}(V)$  and  $\operatorname{trace}(V)$ , respectively.  $\lambda_{\min}(V)$  denotes the minimum eigenvalue of a matrix  $V$ .
2.2 Cascading Contextual Assortment Bandit Problem
Consider  $[N]$ , a set of  $N$  items. Let  $\mathcal{A}$  be a set of candidate assortments of items with size  $M$ , i.e.,  $\mathcal{A} := \{A \subseteq [N] : |A| = M\}$ . A cascade  $S$  is an ordered sequence of  $K$  assortments chosen from  $\mathcal{A}$  where all the items in these  $K$  assortments are distinct. Then, the set of all feasible cascades  $S$  can be defined as follows.
$$
\mathcal {S} := \left\{S = \left(A _ {1}, \dots , A _ {K}\right) \mid A _ {k} \in \mathcal {A} \text {f o r a l l} k \in [ K ], \cap_ {k = 1} ^ {K} A _ {k} = \emptyset \right\}
$$
At round  $t$ , feature vectors  $\{x_{ti} \in \mathbb{R}^d, i \in [N]\}$  for every item are revealed to the decision-making agent. Each feature vector  $x_{ti}$  may contain the contextual information of the user at round  $t$  and the item  $i$ . After observing this contextual information, at round  $t$ , the agent recommends a cascade  $S_t = (A_{tk})_{k \in [K]}$  to the user, where  $A_{tk} \in \mathcal{A}$  represents the  $k$ -th assortment of the cascade at round  $t$ . The user scans the assortments in  $S_t$  one by one. If the items in  $A_{tk}$  do not attract the user, the user moves on to the next assortment  $A_{t,k+1}$ . The user stops at the  $O_t$ -th assortment when the user is attracted by an item in the  $O_t$ -th assortment and clicks on the item.
After the user clicks on the item, the agent observes a sequence of user choices  $y_{t} = (y_{tk})_{k\in [O_{t}]}$  where a binary vector  $y_{tk} = (y_{tk0},y_{tk1},\dots,y_{tkM})$  represents user choices on assortment  $A_{tk}$ . Let  $y_{tkm} = 1$  if the  $m$ -th item  $i_m$  in  $A_{tk}$  is clicked by the user, and  $y_{tkm} = 0$  for items that are not clicked on. For each assortment, there is an outside option. That is, there is a probability that the user may not click any of the items in  $A_{tk}$ . If the user does not choose any items,  $y_{tk0} = 1$  and  $y_{tkm} = 0$  for all  $m\in [M]$ . The user choice for each assortment is given by the multinomial logit (MNL) choice model [21]. For this MNL model, there is an unknown time-invariant parameter  $\theta^{*}\in \mathbb{R}^{d}$ . We define the true weight of item  $i$  in round  $t$  as  $w_{ti}^{*}\coloneqq x_{ti}^{\top}\theta^{*}$ . Also, we let the vector representation of the weights be defined as  $w_{t}^{*}\coloneqq (w_{ti}^{*})_{i\in [N]}$  for convenience.
Under this model, the user's click probability of the  $m$ -th item in  $A_{tk}$  and the probability of the outside option in  $A_{tk}$  is given respectively by
$$
p _ {t} \left(i _ {m} \mid A _ {t k}, w _ {t} ^ {*}\right) = \frac {\exp \left(w _ {t i _ {m}} ^ {*}\right)}{1 + \sum_ {j \in A _ {t k}} \exp \left(w _ {t j} ^ {*}\right)} \quad \text {a n d} \quad p _ {t} \left(i _ {0} \mid A _ {t k}, w _ {t} ^ {*}\right) = \frac {1}{1 + \sum_ {j \in A _ {t k}} \exp \left(w _ {t j} ^ {*}\right)}
$$
where item  $i_0$  represents the outside option. The user choice  $y_{tk}$  is sampled from the multinomial distribution,  $y_{tk} \sim \mathrm{MNL}\big\{1, (p_t(i_m|A_{tk}, w_t^*))_{m=0}^M\}$ , where the argument 1 indicates that  $y_{tk}$  is a single-trial sample. Hence,  $\sum_{m=1}^{M} y_{tkm}$  is always 1. Also, we denote measurement noise as  $\epsilon_{tkm} := y_{tkm} - p_t(i_m|A_{tk}, w_t^*)$ . Since  $\epsilon_{tkm}$  is bounded in  $[0,1]$ ,  $\epsilon_{tkm}$  is  $\sigma^2$ -sub-Gaussian with  $\sigma^2 = 1/4$ . It is important to note that  $\epsilon_{tkm}$  across items in the same assortment is not independent due to the substitution effect in the MNL model.
The expected reward function of a combinatorial action  $S_{t}$  based on  $w_{t}^{*}$  is given by
$$
f(S_{t},w_{t}^{*}) = \sum_{k = 1}^{K}\left\{\prod_{\dot{k} = 1}^{k - 1}p_{t}(i_{0}|A_{\dot{t}\dot{k}},w_{t}^{*})\right\} \sum_{i\in A_{tk}}p_{t}(i|A_{tk},w_{t}^{*}) = 1 - \prod_{k = 1}^{|S_{t}|}p_{t}(i_{0}|A_{tk},w_{t}^{*}).
$$
The formulation above is also known as the cascade model with disjunctive objective, where the user stops at the first attractive item [13; 14; 18].
2.3  $\alpha$ -Approximation Oracle and  $\alpha$ -Regret
The exact combinatorial optimization to compute an optimal cascade of assortments can be computationally expensive. Therefore, we allow for approximate optimization. We assume that the
agent has access to an  $\alpha$ -optimization oracle to compute a  $\alpha$ -approximation solution of the cascade optimization problem with  $\alpha \leq 1$ . For approximate optimization, we prove that a greedy selection for the cascading assortment optimization problem gives a 0.5 approximation of the optimal solution, which may be of independent interest (see Section 6).
Formally, for a given an  $\alpha$ -optimization oracle and a weight parameter  $w$ , the oracle outputs an approximately optimal cascade  $\hat{S}^{*} = \mathbb{O}^{\alpha}(w) \in S$  satisfying  $f(\hat{S}^{*}, w) \geq \alpha f(S^{*}, w)$  where  $S^{*} \in \arg \max_{S \in S} f(S, w)$  is an optimal assortment without approximation. The instantaneous  $\alpha$ -regret of cascade  $S_{t}$  in round  $t$  is defined as  $\mathcal{R}^{\alpha}(t, S_{t}) := \mathbb{E}[\alpha f(S_{t}^{*}, w_{t}^{*}) - f(S_{t}, w_{t}^{*})]$  where  $S_{t}^{*} \in \arg \max_{S \in S} f(S, w_{t}^{*})$  is an true optimal assortment. Then, the goal of the agent is to minimize the cumulative  $\alpha$ -regret defined as
$$
\mathcal {R} ^ {\alpha} (T) := \sum_ {t = 1} ^ {T} \mathcal {R} ^ {\alpha} \left(t, S _ {t}\right) = \sum_ {t = 1} ^ {T} \mathbb {E} \left[ \alpha f \left(S _ {t} ^ {*}, w _ {t} ^ {*}\right) - f \left(S _ {t}, w _ {t} ^ {*}\right) \right].
$$
2.4 Suboptimal Dependence on Problem Dependent Parameters
In this subsection, we discuss the main technical challenges faced in the regret analysis of our problem setting. In particular, the suboptimal dependence on cascade length  $K$  has been a long-standing open problem even in contextual cascading bandits.
2.4.1 Dependence on Length of Cascade  $K$
The previous literature on the contextual cascading bandits [18; 17; 26; 25] bounds the instantaneous regret in each round, utilizing the monotonicity and Lipschitz continuity of the expected reward function  $f$ . A simple adaptation of the previously known techniques to our problem would result in the following upper bound for the instantaneous regret  $\mathcal{R}^{\alpha}(t,S_{t})$  for  $S_{t} = (A_{t1},A_{t2},\dots,A_{tK})$ .
$$
\mathcal {R} ^ {\alpha} (t, S _ {t}) \leq \sum_ {k = 1} ^ {K} \sum_ {i \in A _ {t k}} \beta_ {t} \| x _ {t i} \| _ {V _ {t - 1} ^ {- 1}} \tag {1}
$$
where  $\beta_{t}$  is a suitable confidence radius chosen by an algorithm, and  $V_{t}$  is a positive definite gram matrix. Then, the dependence on the length of cascade  $K$  and the assortment size  $M$  would appear in the regret bound after summing the right-hand side of Eq.(1) over the time horizon and applying the Cauchy-Schwarz inequality. Because of this reason, even when  $M = 1$ , there still exists dependence on  $K$  which appears in the regret bounds of all previous contextual cascading bandits (see Table 1).
To overcome this challenge, we present a new Lipschitz continuity of the expected reward function to derive the regret bound independent of  $M$  and  $K$  by replacing the summation with the maximum over assortments and a cascade (see Section 4.3 for more details).
2.4.2 Dependence on Worst-Case Scanning Probability
Analogous to the existing algorithms for the contextual cascading bandits [18:26], the gram matrix  $V_{t}$  contains the rank-1 matrices of observed items accumulated up to round  $t$ , i.e.,  $V_{t} = \sum_{\tau=1}^{t}\sum_{k=1}^{O_{\tau}}\sum_{i\in A_{\tau k}}x_{\tau i}x_{\tau i}^{\top} + \lambda I$ . However, there exists an out-of-control issue, that is, the summation of the rank-1 matrices over  $O_{t} + 1$  to  $|S_{t}|$  in Eq.(I) is not included in the gram matrix  $V_{t}$ . Note that this issue also arises in cascading contextual assortment bandits. Adapting a technique used in the existing literature [18] to mitigate this issue, let  $p_{t,S_t}$  be the probability of examining all assortments in  $S_{t}$  and  $p^{*} = \min_{t\in [T]}\min_{S\in S}p_{t,S_{t}}$  be the worst-case probability of examining a cascade over all rounds and all feasible cascades. A simple adaptation of the existing methods would result in the following bound for the expected instantaneous regret.
$$
\mathbb {E} \left[ \mathcal {R} ^ {\alpha} (t, S _ {t}) \right] = \mathbb {E} \bigg [ \mathcal {R} ^ {\alpha} (t, S _ {t}) \mathbb {E} \bigg [ \frac {1}{p _ {t , S _ {t}}} \mathbb {1} \left\{O _ {t} = | S _ {t} | \right\} \mid S _ {t} \bigg ] \bigg ] \leq \frac {1}{p ^ {*}} \mathbb {E} \big [ \mathcal {R} ^ {\alpha} (t, S _ {t}) \mathbb {1} \left\{O _ {t} = | S _ {t} | \right\} \big ].
$$
This concedes the dependence on  $p^*$ , which can be exponentially small in the worst case. We overcome this challenge by designing an algorithm that offers the assortment containing the most uncertain item as the first assortment in a cascade. We discuss this salient feature of the proposed algorithm in more detail in Section 3.2.
Algorithm 1 UCB-CCA
Input: confidence radius  $\beta_{t}$  and ridge penalty parameter  $\lambda \geq 1$
1: for  $t = 1, \dots, T$  do
2: Observe  $x_{ti}$  for all  $i \in [N]$
3: Compute  $u_{ti} = x_{ti}^{\top}\hat{\theta}_{t - 1} + \beta_{t - 1}||x_{ti}||_{V_{t - 1}^{-1}}$  for all  $i\in [N]$
4: Compute a candidate cascade  $S_t' \gets (A_{tk}')_{k \in [K]} = \mathbb{O}^\alpha(u_t)$
5: Find optimistic exposure assortment index  $k^{*}$  in  $(k^{*},i^{*}) = \operatorname *{argmax}_{k\in [K],i\in A_{tk}^{\prime}}||x_{ti}||_{V_{t}^{-1}}$
6: Optimistic exposure swap  $S_{t} \gets (A_{tk})_{k \in [K]}$  where  $A_{tk} := \begin{cases} A_{tk^*}' & \text{if } k = 1 \\ A_{t1}' & \text{if } k = k^* \\ A_{tk}' & \text{otherwise} \end{cases}$
7: Offer  $S_{t}$ , and observe user feedback  $O_{t}$  and  $y_{t} = (y_{tk})_{k\in [O_{t}]}$
8: Update  $V_{t} \gets V_{t-1} + \sum_{k=1}^{O_{t}} \sum_{i \in A_{tk}} x_{ti} x_{ti}^{\top}$
9: Compute the regularized MLE  $\hat{\theta}_t$  by solving  $\nabla_{\theta}\left[\ell_t(\theta) + \frac{\lambda}{2} ||\theta||_2^2\right] = \mathbf{0}$
3 Algorithm: UCB-CCA
3.1 Upper Confidence Bounds and Confidence Set
UCB-CCA utilizes the upper confidence bounds (UCB) technique [6, 11, 16] to compute an optimistic action based on optimistic estimates of each item's weight,  $u_{ti} = x_{ti}^{\top}\hat{\theta}_{t - 1} + \beta_{t - 1}(\delta)||x_{ti}||_{V_{t - 1}^{-1}}$  for all  $i\in [N]$ . The confidence radius  $\beta_{t}(\delta)$  is specified to maintain a high-probability confidence set  $C_t(\delta)$  for the unknown parameter  $\theta^{*}$ , although the algorithm does not explicitly compute  $C_t(\delta)$ .
$$
C _ {t} (\delta) := \left\{\theta \in \mathbb {R} ^ {d}: | | \hat {\theta} _ {t} - \theta | | _ {V _ {t}} \leq \beta_ {t} (\delta) \right\}.
$$
Setting a proper confidence radius  $\beta_{t}(\delta)$  can guarantee that  $\theta^{*}$  lies within the confidence set with probability  $1 - \delta$ . On the event that  $\theta^{*} \in C_{t}(\delta)$ , the UCB weight  $u_{ti}$  serves as an upper bound of a true weight  $w_{ti}^{*} \coloneqq x_{ti}^{\top} \theta^{*}$  for every item  $i \in [N]$ . We denote the UCB weight vector as  $u_{t} = (u_{ti})_{i \in [N]}$  for convenience.
3.2 Optimistic Exposure Swapping
A distinctive element of UCB-CCA is what we call optimistic exposure swapping, a procedure crucial for eliminating dependence on the worst-case scanning probability, as elaborated in Section 2.4.2. This technique strategically positions the assortment containing the item with the highest uncertainty among the top  $MK$  items in the first slot of the cascade of assortments.
In each round  $t$ , the  $\alpha$ -approximate oracle  $\mathbb{O}^{\alpha}(u_t)$  outputs a candidate cascade  $S_{t}^{\prime}$ , determined by the UCB weights  $u_{t}$ . It is important to note that  $S_{t}^{\prime}$  is not immediately presented to the user. Instead, after  $S_{t}^{\prime}$  is derived using the optimization oracle  $\mathbb{O}^{\alpha}(u_t)$ , the algorithm identifies the index  $k^{*}$  of an assortment that includes the item with the largest  $\| x_{ti}\|_{V_{t - 1}^{-1}}$  in  $S_{t}^{\prime}$ .