Product Ranking for Revenue Maximization with Multiple Purchases
Authors: Renzhe Xu
Xingxuan Zhang
Bo Li
Yafeng Zhang
xiaolong chen
Peng Cui
Venue: NeurIPS 2022 Accept (2022)
Keywords: Revenue maximization
Multiple purchases
Product ranking

Abstract: Product ranking is the core problem for revenue-maximizing online retailers. To design proper product ranking algorithms, various consumer choice models are proposed to characterize the consumers' behaviors when they are provided with a list of products. However, existing works assume that each consumer purchases at most one product or will keep viewing the product list after purchasing a product, which does not agree with the common practice in real scenarios. In this paper, we assume that each consumer can purchase multiple products at will. To model consumers' willingness to view and purchase, we set a random attention span and purchase budget, which determines the maximal amount of products that he/she views and purchases, respectively. Under this setting, we first design an optimal ranking policy when the online retailer can precisely model consumers' behaviors. Based on the policy, we further develop the Multiple-Purchase-with-Budget UCB (MPB-UCB) algorithms with $\tilde{O}(\sqrt{T})$ regret that estimate consumers' behaviors and maximize revenue simultaneously in online settings. Experiments on both synthetic and semi-synthetic datasets prove the effectiveness of the proposed algorithms.

Product Ranking for Revenue Maximization with Multiple Purchases
Renzhe Xu', Xingxuan Zhang's Bo Li?, Yafeng Zhang', Xiaolong Chen², Peng Cui DDepartment of Computer Science and Technology. Tsinghua University, Beijing, China "Scoool of Economics and Management, Tsinghua University, Beijing, China Meituan Inc., Beijing, China
xrz19972ggmail. com, xiagsuanzhang@hotmmal com, libo@sem. tsinghua. edu cn yafengz4outlook. com, chenxiaolong 3eme tuan. com, cuip@tsinghua edu. cn
Abstract
Product ranking is the core problem for revenue-maximizing online retailers. To design proper product ranking algorithmss, various consumer choice models are proposed to characterize the consumers' behaviors when they are provided with a list of products. However, existing works assume that cach consumer purchases most one product or will keep viewing the product list after purchasing a product, which does not agree with the common practice in real scenarios. In this paper, we assume that each consumer can purchase multiple products at will. To model consumers' willingness view and purchase, set random attention span and purchase budget, which determines the maximal amount of products that he/she views and purchases, respectively. Under this setting, we first design optimal ranking policy when the online retailer can precisely model consumers' behaviors. Based on the policy, we further develop the (MPB-UCB) algorithms with Õ( regret that estimate consumers' behaviors and maximize revenue simultaneously in online settings. Experiments on both synthetic and semi-synthetic datasets prove the effectiveness of the proposed algorithms. The source code is available at https: / //g thub. com/ hoom/winxxz///// Maaxxr//PPBBUS UCB.
UCB
1 Introduction
Online retailing has become increasingly popular over the last decades [17, 28, 52]. The way of product ranking is the crux for online retailers because it determines the consumers' shopping behaviors [17] and thus influences the retailers' revenue [20, 49]. For instance, the probability of consumers' purchasing from firm clicking advertisement is strongly related to the display order [8, 3, 33]. Therefore, it is crucial for online retailers to design proper product ranking algorithms for revenue maximization.
The consumer choice model, when faced with a list of products, the basis of the problem [17]. Generally speaking, existing works assume that the consumers view the list of products sequentially and select products according to a choice model [43, 42, 32, 38, 18, 26, 25, 37]. However, most of them assume that each consumer purchases most one product and stop browsing immediately afterward, which does not agree with the common practice since a consumer may expect multiple purchases website
app [34, 35, 16, 28). Recently, several works consider multiple- purchase setting while their targets are not revenue maximization [28, 15), or they assume that consumers always continue viewing the product list after purchasing any product, which is not practical in real scenarios [40].
"Corresponding Author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

In this paper, we propose more realistic consumer choice model to characterize consumer behaviors under multiple-purchann settings. Following [13, 17, 40], we assume that the consumers view the product list sequentially and purchase product when its utilityy exceeds certain threshold. Similar to [17], we assume the existence of the attention span for each consumer, which determines the maximal amount of products that the consumer is willing to view. More importantly. instead of forcing the consumer to quit viewing once she purchases one product, we allow each consumer to conduct multiple purchases at will and explore product ranking models for revenue maximization. Specifically, we constrain each consumer with a purchase budget to determine the maximal amount of purchases. Both the attention span and the purchase budget are assumed to obey the geometric distribution [13,40] 40].
Under this setting, we study the optimal product ranking policy to maximize online retailers' revenue. We first characterize the optimal ranking policy that achieves the maximal revenue when the online retailer knows the consumer parameters, including the purchase probabilities for each product and the distribution of the attention span and purchase budget. Afterward, we investigate the problem in online learning frameworks where consumer parameters are unknown to online retailers. In both non-comtextual (i.e,, all consumers share the same parameters) and contextual settings (i.e., consumers have personalized parameters), we provide the algorithms that could fit consumers' parameters and maximize revenue simultaneously. We further theoretically prove that the algorithms achieve Ö(vT) regrets in both settings. Finally, we conduct experiments to verify the effectiveness of the algorithms.
with-Budget UCB (MPB-UCB)
To conclude, our contributions are highlighted as follows:
We propose a novel consumer choice model to deal with multiple-purchase activities of consumers in online retailing scenarios.
We characterize the optimal ranking policy and further design the MPB-UCB algorithms with ((VT) regret based on them in both non-contextual and contextual online settings. We conduct experiments on both synthetic and semi-synthetic datasets to prove the effec- tiveness of our method in the purthple-purchase setting.
2
Related Works
Consumer choice model Traditional choice models adopt the multinomial logic (MNL) [12, 22, 43] models to characterize consumers' behaviors. However, these models can not describe the behaviors when the consumers are provided with a list of products. As result, several choice models are proposed recently. In detail, Abeliuk et al. [2] add position bias term in the traditional MNL. model. Gallego et al. [30], Aouad and Segev [4] suppose that the products are framed into a set of virtual web pages. Consumers select product, if any, from these pages following a general choice model (such as MNL). Flores et al. [29] propose the Sequential Multinomial Logit (SML), which assumed a two-stage MNL model and supposed that products are divided into two stages with prior knowledge. Liu et al. [41], Feldman and Segev [27]. Gao et al. [31] further extend the setting to multiple stages. These works suppose that each stage contains multiple products while recently several works assume that a product is treated as a single stage [17, 13, 14]. These models are also closely related to the cascade model [12, 36, 38, 54, 19]. Gao et al. [32] further propose the general cascade click models. Following these works, we suppose that each stage contains only one product and the consumers view the products sequentially.
In addition, several works consider the setting where consumers may have multiple interactions with the platforms. However, most of them focus on the click model [15, 16, 34-36] or MNL-based consumer choice models [48, 53, 50]. In addition, Ferreira et al. [28] maximize the number of consumers who engage with the site. Liang et al. [40] aim to maximize the revenue in the multiple- purchase setting while they assume that consumers always continue viewing the product list after purchasing any product, which is not practical in real scenarios.
Online learning and multi-armed bandit model consumers' behaviors and maximize revenue at the same time, which is closely related to the multi-armed bandit framework [57, Many works [5, 9, 11, 10, 24] have been proposed to deal with demand learning price experimentation problems with the help of the framework in the field
Online retailers need to design proper algorithms to

---

of revenue management. Recently, more methods [40, 17, 13, 14, 6, 44] adopt similar techniques in the product ranking setting, which inspires the algorithm proposed in this paper.
3 Preliminaries
Let [N] (1. ] be the set of all products owned by an online retailer (he). The product
A generates revenue тı when consumer purchases it. Let T Tmax - maxke[N] T'k be the maximal revenue for all products and or on the products for the t-th consumer (she). More precisely, Ak E [AV] 0z.k E [M] represents the product displayed in the k-th position for her. Reversely, F2.k denotes the position of the k-th product in the ranking list. Furthermore, for any subset Q S [N] that represents the possible positions in the ranking list, we use Tt to denote the corresponding set the products w.E. the position set Q, i.e.,
{rr) We use the notation o -I [N] denote the ranking policy
for, & Q}. In addition, we use max(.) to denote the maximal element in set.
Consumer choice model for multiple purchases When the t-th consumer arrives, she views the list of products sequentially. We assume that the behaviors of different consumers are independent. The probability of product k being purchased by her is denoted as Aτ, and let A, = {Ax, ) We assume for tractability that consumer interests in any products independent, which is common in product ranking papers [4, 18, 23, 49]. This indicates that the purchase behavior of any product set does not affect the purchase probabilities of other products.
To characterize the consumers' behaviors under multiple-purchase settings, we assume that the consumer endowed with the attention span V2 and purchase budget Br, which determine the maximal number of the products that she will view and purchase, respectively. In reality, the consumer always has random attention span and purchase budget [17). Therefore, following [13, 40], we assume V ~ Geo(1 [1 - gt) and B₁ Geo(1 sx) are independent geometric distributions with parameters - and 1 - 8t. respectively. As a result, P(V. = v) q: -i:=2(1 - Q:) for any v 2 1 and P(B, 6) it (1 sx) for any b 2 1. With attention span V22 and purchase budget Br, the consumer keeps viewing the products until she views V, or purchases B, products.
Because of the attention span V,, the consumer will not purchase products whose positions are greater than V, in the ranking list. Let Q C [Vr] denote the possible positions of the purchased products by the consumer. The corresponding indices purchased products are dr Therefore, the probability that the consumer purchases the product set drQQ is given by
TIke )coo.)) (Thεraano (1 deoes))
if < B
pbay (Q: Ve, B., =
TTEE de,
(Ikevv)))a (1 )oo
if | = B.. (I)
|f > B,.
Specifically, the probability pbuy (Q; Vr, Be, oz) is calculated based on the cardinality of Q.
When [Q] B,, the consumer does not meet her purchase budget and stops browsing the list after viewing V, products. When [Q] = Br, the consumer stops browsing as long as she purchases the last product in the list Q. When [Q > B,, the probability becomes 0 since the number of purchases Ql exceeds her purchase budget.
Remark. The setting of the purchase budget provides rational way to characterize the consumers' behaviors with multiple purchases and extends previous works that suppose consumers would purchase most one product [13, 17]. Furthermore, when s; - O, the consumers would tend to purchase at most one product and our setting degenerates to the setting proposed in [13]. Thus the previous setting can be considered a special case of our setting. In addition, Liang et al. [40] assume that consumers always continue viewing the product list after purchasing any product, which is not practical in real scenarios. By contrast, the purchase budget proposed in our paper leads a different and more practical consumer choice model. In addition, our model introduces an extra parameter (i.e., the parameter), which makes our problem more challenging.
Revenue optimization The online retailer chooses a ranking policy o, + [N] on the products for the t-th consumer. For the consumer with fixed purchase probabilities for each product A attention
3

---

span Ve, and purchase budget Bi, the expected revenue could be written as
Br) = Σ poay Bi,
Σ Tor.
(2)
QEIVI
In reality, the attention span Vi and purchase budget Br are random and can not be estimated accurately by the online retailer. As result, the retailer calculates the expected revenue for the 1-th consumer by sampling V, from Geo (t) and sampling B, from Geo 8t), L.e,
Revenue (ar:; Xe, 4t,8t) st) Gvo(1Go(11- ( B [R(V,, Be; A o.)]
(3)
To maximize the revenue, the optimal ranking policy is given by
ai ( ft. 8t) arg max Revenue (r,
(4)
dx-[M]
For simplicity, we use of to denote oj (>x, Qt, 8t).
We also need the following assumption to guarantee the tractability of the proposed problem. In reality, consumers always have finite attention span, which indicates that qe is always smaller than 1. Assumption 3.1. There exists a known constant EQ > such that for all q: EQ.
Basic assumption
4 Optimal Ranking Policy Given Consumers' Characteristics
In this section, we introduce the optimal ranking policy when the online retailer exactly knows the consumers' parameters, including the purchase probability for each product X, random attention span (parametrized with g), and random purchase budget (parametrized with s).
We drop the subscript in this section for simplicity. The optimal ranking strategy stems from the following theorem.
Theorem 4.1. Under Assumption 3.1. the following holds for the optimal ranking strategy o' (A, 9.s)
>o; visiaN, g- 2 1 2 -11))
No To
(5)
Remark. The expected revenue of single product of is Ao;To; which grows the purchase probability and the revenue generated by the product increase. Yet purchasing a product reduces the purchase probability of the following products, and thus the expected revenue of other products decreases as Arą increases. Furthermore, when s - O, the term in Equation (5) becomes the same as that in [13, Theorem 1] if ignoring the marketing fatigue (i.e., overexposure to unwanted marketing messages) term in their result. As a result, Theorem 4.1 consistently extends the results on the single-purchase setting [13] the multiple-purchase setting.
In addition, Liang et al. [40] assume that the online retailer will stop recommending products with a certain probability
wi to avoid the cost brought by marketing fatigue. As result, the parameter wi is part of the ranking policy instead consumers' characteristics and should be optimized to achieve maximal revenue. In our paper, we focus on the multiple purchases with budget setting and leave the extension to the case with the marketing fatigue as future work.
According to Theorem 4.1, the online retailer can provide the optimal ranking policy o" by sorting q(1 =s)>x) for the k-th product. The
the products in descending order with value (Aurr) time complexity is O(N log N).
Online Learning of the Ranking Policy
In this section, we consider more realistic scenario where the online retailer has no prior knowledge about consumers' characteristies. At each timestamp, consumer arrives and the online retailer can only provide proper product ranking policies based on the feedback of previous consumers. As result, the retailer needs to design online learning algorithms to model consumers' behaviors and maximize revenue in the meantime. We consider two settings (i.e., the non-contextual setting where all consumers share the same parameters (Section 5.1) and the contextual setting where consumers have personalized behaviors (Section 5.2)) and develop the Multiple-Purchase--e Purchase-with. Budget UCB (MPB-UCB) algorithms on both settings. Before introducing the details of the algorithms, we first formally define the notations characterize the consumers' behaviors.

---

Notations for consumers' behaviors Let 42 E 24 be the random variable that denotes the number of products viewed by the f-th consumer. Let random variable "t. E {0. 1} denote whether the consumer purchases the product displayed in the k-th position (i.e., product ot.kk if she views the product. Specifically, Y = 1 if and only if the product is purchased. Let Y {Tt, )
Let random variable 1/.& E {0, } denote whether the consumer continues viewing products she views the product displayed in the k-th position but does not buy it. 1t, is observed only when & 4 and Yt,k = Let random variable pr, E {0, 1 denote whether the consumer keeps viewing products if she buys the product in the k-th position. Mr, is observed only when & I o; and Nt
Let 7,, = J -
and μ,, (=
Let Ber(.) denote the Bernoulli distribution and we have the following result.
Lemma 5.1. frt, M..Kfeo,.sKK. are independent of each other, Iss are independent of each other, and {He, 11,,1, C are independent of each other. In addition, Vt > 0 and 1 < k < N, It.k Ber (Nor,), ) Tt. Ber(g)), and pt,k Ber(g.st)
Lemma 5.1 is the basis of the algorithms that estimate parameters X, q, and from historical data. In addition, compared with 8t, it easier to estimate gr from data and we denote it as wy _ gr
5.1 Non-contextual Online Setting
In this subsection, assume that all consumers share the same parameters, including the purchase probabilities on different products and the parameters on the distribution of the attention span and purchase budget. In detail, assume that there exist X, q, such that X >, qt q. and
A) Estimation of Parameters
To estimate the parameters at timestamp i, we leverage the observed behavior of consumers before timestamp We first calculate the following statistics that can help provide unbiased estimators of the consumer parameters.
In detail, let Or be the number of times that the product k is observed and Cr.k be the num- ber of times the product is purchased, i.e., Ct, A Et. E [or, = k] and cr - Ets EE [ = k] T [Tc, 1]. In addition, Let D2 be the number of times that consumers view product and do not purchase it and de be the number of times that consumers continue viewing the list after viewing a product without purchasing it, i.e., D. A EL=EE Ihu = 0] and de A Ei= Ets 1 [hrs 0] 77,1. Furthermore, let D. be the number of times that consumers purchase product and d." be the number of times that consumers continue viewing the list after purchasing a product, i.e., D.W & Σ. E ,
I = 1] and dw & Ei= [7,, Pr.
With these statistics, the parameters X, q. and w qs are estimated as follows.
de, 2 ct,t/CC, á. _ ot DD. ih a d!" /D..
(6)
We provide the following proposition that gives the enror bound our parameter estimation approach. Proposition 5.2. For all E [T], with probubility at least 6Nr-3, the following holds:
Wk E (N7, See xel
2 2log
1 1
5
2log D
and 10:: -gsl S v
22 log
Ca
DW
B) Algorithm
Following the classic optimism in the face of uncertainty principle [1], we design UCB-like algorithm that learns consumer behaviors and maximizes revenue simultaneously. In detail, according to Proposition 5.2, we learn the optimistic estimators of the parameters given follows
d e min 1, 1, + V
2logt
C.u . a min 1 - eq. qi +
2logt D.
v. 2 min
Te, ti + V
2 log t
V
D.
(8)
5

---

Algorithm 1: MPB-UCB (Non-contextual)
Input: Products revenue r and nyper-paramette EQ- 2 Initialization: λu, - 1 for k E [N]. do = 1 - eQ. wo = EQ.
for t=1:7 do
Let oe be the optimal offline policy from Theorem 4.1 with À 7x-1. = Q: and
8 s= -tt11/qt111
Offer ranking policy or and observe Pt, Yx. nhe Pc- Update statistics Ct Ct.k, Di d.. D. and di".. Calculate Ae, qt, and Wh by Equations (6) and (8).
end
As shown in Algorithm 1, when the 1-th consumer arrives, the online retailer displays the products based on Theorem 4.1 with 7 It-1, and 2 shown in Line 4. Afterward, the online retailer updates the statistics according to consumers' feedback as shown in Lines and The estimators are then updated in Line 7.
Regret Analysis
To theoretically analyze the performance of Algorithm we first introduce the regret, which measures the total difference between the maximal revenue achieved by o²* and the cumulative reward by an online algorithm after rounds.
Reg
Revenue ; A,g,s) Revenue (ar.; x).
(9)
Regr random variable and the randomness comes from the uncertainty consumers' behaviors. As a result, we focus on E [Regyl. which is a common routine in bandit literature [46]. The performance of the algorithm is given by the following theorem.
Theorem 5.3. Under Assumption 3.1 (with parameter eQ) for any > 1, the regret achieved by Algorithm is bounded by
B Regrl S C Tmax N WogTT
(10)
where C is a absolute constant (independent of problem parameters) and Tmax = max Tk- Remark. We analyze different parameters' impacts the regret bound in Equation (10). Firstly, the regret grows at Ö(VT), which is standard result in online learning algorithms [46]. Secondly, the regret bound linearly related to N, which is similar to the results cascade bandit models [54] and revenue management literature [17]. Finally, the bound depends on Tmax/Eq, which measures the maximal expected revenue the online retailer could achieve from each consumer.
5.2 Contextual Online Setting
We consider a more realistic scenario where the consumers' behaviors are various and depend on their own features. In detail, let x; mh be the feature of the 1-th consumer with dimension m, and Ut. E my be the joint feature of the (-th consumer and k-th product with dimension my. Dt.k can be the concatenation of consumer features and product features fusion through transformation such as matrix multiplication. In this paper, we adopt concatenation for simplicity. Let || denote the & norm. Following [17], we consider the classic linear bandit setting [7, 21, 39] and the ground-truth data-generating process is given by the following assumptions.
Assumption 5.1. There exist constant vectors AA E RRy,,, ,8 mR and g MR, such that 81 x, g8. qe = x g. and
A vi.B* E [N],
(11)
Assumption 5.2. For any t t>> and & [V], ||x $ and || < 1. In addition, there exists constant U > 0 such that || ||3 || <
U.
Remark. These two assumptions are common revenue maximization and línear bandits [1, 46). Assumption 5.1 can be satisfied with kernel functions for complex data-generating processes and Assumption 5.2 can be achieved by normalization. Thus the assumptions are rational and easily attainable in real applications.

---

A)
Estimation of Parameters
we For the estimation of B and 88, we consider the ridge regression method. Given the observation of consumer's behaviors till the t-th timestamp, the estimation BAA 3e is given by
BA argmin Bea
(uuoo, 9 - -mub) + aal1s11ē ( Pe:
(12)
B argmin BERm
Mni = 0] (I nux)) ax 1|31 (29 -1
p..
(13)
Here 2/A
Eu= E k= Nu, Vu,ou.tee,ou. + antmyy o -E k= uu,
2
Eim tx 1/Tu, of + aqIm., and pf E Ex= 1/rkk O.T,,KNu.
Img and
In, are the identity matrices with dimension my and my, respectively.
For the estimation of gs, similar to Section 5.1, we focus on regressing Wy = q18; instead. Note that Wy = 8: x, / ((8)T Tt vec( ) ((35) where vec(.) denotes the vectorization operation that maps matrix to vector. Therefore, define 2 vec(x.z, ) and 3 - vec( and we can get wi = 2, 3W. Similarly, we estimate 3W by conducting ridge regression as follows
D. argmin SER
I = =1] (1xz -u.kk + aa aaw1 (x:²) Pi:
(14)
Here D E Eks Ihr, = 1| + aw Imn and P.v = -E Eke Ihak I is the identity matrix with dimension m..
11
Let || = /3 AB for any positive definite matrix A. Proposition 5.4 gives the estimation error bound of the coefficients.
Proposition 5.4. Under Assumptions 5.1 and 5.2 (with parameter U), for any t 2 1, with probability at least
3t 2 the following holds:
BA 8* 1m23 r(t mt, am,
88 S T(, mr, st, and and |30" 3W x:
st,t, aw). (15)
Here Tt,, m. a)) = 1/2, m log (1 (mmm))))) + 4 4ogg + a 2// + 1)?.
B) Algorithm
Given the estimation of BA, 80 and 3W we estimate de, qt and wy follows.
Se, = Projoo.)
w. + T(t - 1, an) [Bulll(s)) 1
q: = Projjo,1-oo)
2. 32, + T(l - 1, mz, a2) || (x2
(16)
Projj.aa
3 T(L - 1, m ow || (w )1
Here Projja.a()) (-) is function that projects the input into the interval [a, b].
As shown in Algorithm 2, when the 1-th consumer arrives, the online retailer observes the consumer features I, and Vt, for l E [N] and calculates the intermediate feature 2, Afterward, he calculates Ne, qt. and ii according to Equation (16) in Line 5. Then the ranking policy a, is given as shown in Line 6 and the statistics are updated in Lines 7 and 8. Finally, the parameters 3.A. 30 and 3: are estimated in Line 9 for the next iteration.
C) Regret Analysis
We analyze the performance of Algorithm 2 by investigating its regret bound. For a sequence of T consumers with features X = (a:) and Y {(V.. N 1T the regret given by
Reg, X T, Yr) Revenue (aī; de, gt, st) Revenue (#:: A st).
(17)
7

---

Algorithm 2: MPB-UCB (Contextual)
Input:
Products revenue r and hnyper-paaameet <Q. OA, aq, and ow.
2 Initialization: 30 = O, g = O. and By 0.
for 1=1:7 do
Observe consumer features 24, and St for [N]. Let =t voc())e0)))
Calculate At, qt. and ñH according to Equation (16).
Let o be the optimal ranking policy from Theorem 4.1 with À = de, Gt. and = w/r/q. Offer ranking policy o,, and observe Pt. Yz. Th pay-
Q Calculate statistics ΣA pt. E Pt IW and p!"
Calculate estimated parameters B 3P, and Pt according Equations (12), (13) and (14).
end
Similar to the non-contextual setting, Regr(Xx YT) is random variable and we focus on E[Regr( XT, Y.)]. The performance of the algorithm given by the following theorem.
Theorem 5.5. Under Assumptions 3.1 (with parameter eq). 5.1, and 5.2 (with parameter for any T > 1, the regrel uchieved by Algorithm 2 is bounded by
B[Regr (XT, Y
C Tmax
(x(T,my mys 0A) (m,ma, aq) + x(T, m aw))
(18)
EQ
Here C is absolute constant (independent of problem parameters), Tmax max Tk. and
( log 1
TN + +loog + "2 (PFUU+1)))) -
+1))
TN TN2m (1+
X(T, m. a)
ma a + +a=1)
(19)
ma
Remark. We analyze different parameters' impacts on the regret bound in Equation (19). Firstly, the regret grows at Õ(vT), standard result in online learning algorithms [46]. Secondly, the regret depends on Ö(N), which shares similar result with the revenue management literature [17]. Thirdly, similar Theorem 5.3, the bound depends linearly on f that measures the maximal expected revenue the online retailer could achieve from each consumer. Fourthly, the bound depends on the dimensions of features (i.e., my and myy) by ((ymyy and (m.z) because the feature dimension for the estimation of Wt 9i is m² and then the result is consistent with [17, 54]. Finally, the bound depends on OU + U) since 3| || s 2 and |39 || S U, which agrees with [17].
6 Experiments
In this section, we conduct experiments on synthetic data to verify the performances of the online ranking policies. Reports and discussions of more results including the expected revenue and the ratio over the expected revenue of the optimal ranking policy are Section B.
Baselines We adopt the following baselines. Firstly, Cao and Sun [13] considered the single- purchase setting, which is the simplified version of our multiple-purchanl setting. We denote it as Single Purchase. Secondly, that consumers always keep viewing the ranking list after purchasing products. Furthermore, we implement two explore-then-exxlooi-bbssss
adopt the method in [40] (denoted as Keep Viewing). which assumes
algorithms (denoted Explore Then Exploit and Explore Then Exploit B, respectively) to verify the advantages of our method that could balance exploration and exploitation.
6.1 Synthetic Data
Data Data-generating processes For the non-contextual setting, we consider N = 50 and 300 products
and 7 100, 000 consumers. The revenue for each product r' uniformly distributed [0, and the purchase probabilities for each product are uniformly sampled from [0, 0.3). We set q = 0.9 and = 0.5 and 0.8 to test the performance of different models on characterizing consumers' behaviors with multiple purchases.

---

N
300,
300,
Non- Non-contextual
Regret
2008
1000
10000
25.0
5000
104.0
3000
1000
50, Bens
0.5
50, 8mas
300, Aerss
0.5
.V
300, Besast
Contextual
6000
2000
100m0
2000
4000
10700
4000
10.0
# of customers
Single Purchase
Keep Viewing
Explore Then Exploit
Explore Then Exploit B
MPB-UCB (Ours)
Figure 1: Experimental results synthetic data. We plot the regret curves for different settings by varying = 50/300 and s/5max = 0.5/0.8 8 in both non-contextual and contextual scenarios.
For the contextual setting, we consider The revenue тą is also uniformly distributed in [0. 1]. Both the dimension of consumer features and product features are set to 5 (i.e,, my = 5). The joint feature yt, (the feature of the 1-th consumer and concatenate of the corresponding consumer and product feature that my 10. The consumer features and product features are uniformly sampled from /0.8//m.. 1/ 1//m,,l and [0, ///)) A Afterward, the coefficients 13A, f and 338 in Equation (11) are uniformly sampled from [0, 1]. To ensure A lt 8, have similar ranges in the contextual setting, normalize the coefficients such that the maximal elements in Ae, and (denoted as Amax and 4max) are 0.3 and 0.9, respectively. Similarly, by normalizing 35, we set the maximal element in (denoted as 8max) 0.5 and 0.8.
50 and 300 products and T 100, 000 consumers.
k-th product)
Results and analysis We implement the experiments for 5 distinct simulations by resampling consumers' behaviors while their basic characteristics remain unchanged. Specifically, in both the non-contextual and contextual settings, the purchase probabilities (i.e., A₂) and parameters on the span attention and purchase budget (i.e., and s,) are the same. However, the same consumer in distinct simulations may purchase different products due randomness.
We then evaluate our method (MPB-UCB) and baselines via calculating the regret according to Equations (9) and (17). The results shown in Figure and method outperforms all baselines in both settings with various parameters. On the one hand, the Single Purchase and Keep Viewing baselines consider different consumer choice models and not directly applicable here to the more realistic nultiple-purchase setting. On the other hand, compared with the explore-then-exxlooi--bssss algorithms, our method integrates the advantages of traditional UCB-like algorithms, leading better exploration-exploiaae Explore Then Exploit B incorporates learning processes during the exploration phase, leading to better performance compared with Explore Then Exploit A.
trade-off for smaller regret. In addition, as introduced in Section B,
6.2 Semi-synthetic Data
We utilize the Ad Display/Click Data from Taobao which contains ad display/purchase logs (26 million records) of 1,140,000 anonymized users from the website of Taobao for 8 days. Because the behavior logs only contain the purchase information of the users a category of a brand, we view (brand, category) pair in the dataset as a product and the average prices of the ads this pair as the revenue for the product.
Because we can not obtain consumers' behaviors when we offer new product ranking policy, following [14], we first estimate the parameters (i.e,, X, in the non-contextual setting and 8² 8 3 in the contextual setting) with all of the data. Afterward, we use the estimated parameters to simulate consumers' behaviors when we provide them with different ranking lists of products.
'https://tianch" www.aaacc.o/ttaaaaaaaaaaaaaaaaaa htt.:aataaa.ttttttttaaaaaaaaaaaa License: CC BY-NC 4.0.

---

Non contextual, 50
Non contextual, N 300
Contextual, N
Contextual, 300
1300
Regret
1000
20.00
# of customers
Single Purchase
Keep Viewing
Explore Than Exploit A
Explore Then Exploit
B
MPB-UCB (Ours)
Figure 2: Experimental results the semi-synthetic data. We plot the regret curves for different
settings by varying N = 50/300 both non-contextual and contextual scenarios.
Data processing Since each consumer may launch and leave the platform many times during the days, we need to distinguish different launching activities of the same consumer. Specifically, we suppose that a consumer stops viewing the product list if it has been more than 10 minutes since her last interaction with the platform. Afterward, we can estimate the parameters in both non-contextual and contextual settings.
In the non-contextual setting, we calculate the statisties CLAss Ct.k. Dj. df D. and di from the data and estimate A, directly according to Lemma 5.1. The estimation is = 0.870 and 0.907. Afterward, we sample . 50 and 300 products with prices no more than 200 and purchase probabilities at least 0.1. Similar to the synthetic data, we set 7 100, 000.
In the contextual setting, as consumers' features and the number of ads, the logarithm of the average, standard deviation, maximum, and median of the ads' prices in the (brand, category) pair products' features. With the statistics Cr,b, Cr,k. Di df DW and d! we use the ridge regression with regularization strength 1 estimate 3d, 89, and B Afterward, we sample N 50 and 300 products with prices no more than 200 and purchase probabilities at least 0.1. In addition, we sample ' = 100, 000 consumers.
the gender, age level, consumption grade, and shopping level
Results and analysis Similar to the synthetic experiments, we implement the experiments for distinct simulations by resampling consumers' behaviors while their basic characteristics remain unchanged. The results of the regret are shown Figure As shown in the figure, our method outperforms all baselines in different scenarios. Firstly, the Single Purchase and Keep Viewing baselines are not directly applicable here since they consider different consumer choice models. worth mentioning that the estimated is large from the dataset. As result, the consumers prefer to keep viewing products if they purchase any product, making the Keep Viewing baseline outperforms Single Purchase in this setting. Secondly, our method combines the benefits of existing JCBB-liee algorithms, resulting in a superior exploration-exploiaatioo trade-off with less regret compared with wheeorreeee-exlooiit
methods. In addition, similar to the analysis in the synthetic experiments, Explore Then Exploit outperforms Explore Then Exploit in most cases.
Conclusion
To conclude, propose novel consumer choice model to deal with multiple-purchant activities of consumers in online scenarios. We characterize the optimal ranking policy and further design the MPB-UCB algorithms with (VTT) regret both non-contextual and contextual online settings. We conduct extensive experiments to prove the effectiveness of our method.
Acknowledgments
Peng Cui's research was supported in part by National Key R&D Program of China (No. 2018AAA0102004, No. 2020AAA01063000, National Natural Science Foundation of China (No. U1936219, 62141607), Beijing Academy of Artificial Intelligence (BAAI). Bo Li's research was supported by the National Natural Science Foundation of China (No.72171131), the Tsinghua Univer- sity Initiative Scientific Research Grant (No. 2019THZWICIN 1), Technology and Innovation Major Project of the Ministry of Science and Technology of China under Grants A220AAA0100000 and 2020AAA0108403.
10

---

References
[1] Yasin Abbasi- Yadkor,, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011. 5, 26, 28
[2] Andrés Abeliuk, Gerardo Berbeglia, Manuel Cebrian, and Pascal Van Hentenryck. Assortment optimization under multinomial logit model with position bias and social influence. 4OR, 14 (1):57-75,2016, 2016. 2
[3] Ashish Agarwal, Kartik Hosanagar, and Michael D Smith. Location, location, location: An analysis of profitability of position in online advertising markets. Journal of marketing research, 18(6):1057-1073, 2011.
[4] Ali Aouad and Danny Segev. Display optimization for vertically differentiated locations under multinomial logit preferences. Management Science, 67(6):3519-3550, 2021. 2,
[5] Victor F Araman and René Caldentey. Dynamic pricing for nonperishable products with demand learning. Operations research, (7(5):1169-1188 2009. 2
[6] Arash Asadpour, Rad Niazadch, Amin Saberi, and Ali Shameli. Ranking an assortment of products via sequential submodular optimization. arXiv preprint arXiv:2002.09458 2020.
[7] Peter Auer. Using confidence bounds for exploitation-exxplrraiio trade-offs. Journal of Machine Learning Research, 3(Nov):397-422, 2002. 6
[8] Michael R Baye, J Rupert Gatti, Paul Kattuman, and John Morgan. Clicks, discontinuities, and firm demand online. Journal of Economics & Management Strategy, 18(4))935-975, 2009.
[9] Omar Besbes and Assaf Zeevi. Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Operations Research, 57(6;:107--1200, 2009. 2
[10] Omar Besbes and Assaf Zeevi. Blind network revenue management. Operations research, 60
(6):1537-1550, 2012. 2
[11] Josef Broder and Paat Rusmevichientong. Dynamic pricing under general parametric choice model. Operations Research, 60(4):965-980, 2012. 2
[12] Juan José Miranda Bront, Isabel Méndez-Diat and Gustavo Vulcano. A column generation algorithm for choice-based network revenue management. Operations research, 57(3):769-788. 2009.
[13] Junyu Cao and Wei Sun. Dynamic learning of sequential choice bandit problem under marketing fatigue. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3264-3271 2019. 3. 15
[14] Junyu Cao, Wei Sun, and Zuo-Jun Max Shen. Sequential choice bandits: Learning with marketing fatigue. Available at SSRN 33552/1, 2019. 2, 3, 9, 15
[15] Junyu Cao, Wei Sun, Zuo-Jun Max Shen, and Markus Ettl. Fatigue-awan bandits for dependent click models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3341-3348, 2020. 1,2
[16] Olivier Chapelle and Ya Zhang. A dynamic bayesian network click model for web search ranking. In Proceedings of the 18th international conference World wide web, pages 1-10, 2009. 1,2
[17] Ningyuan Chen, Anran Li, and Shuoguang Yang. Revenue maximization and learning in products ranking. In Proceedings of the 22nd ACM Conference on Economics and Computation, pages 316-317, 2021. 1, 2, 3, 6, 8, 15
[18] Yuxin Chen and Song Yao. Sequential search with refinement: Model and application with click-stream data. Management Science, 63(12):4345-4333 2017. 1,3

---

[19] Wang Chi Cheung, Vincent Tan, and Zixin Zhong. A thompson sampling algorithm for cascading bandits. In The 22nd International Conference on Artificial Intelligence and Statistics, pages PMLR, 2019. 2
[20] Roberta Comunian, Caroline Chapain, and Nick Clifton. Location, location, location: exploring the complex relationship between creative industries and place. Creative Industries Journal, (1):5-10, 2010.
[21] Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedback. In 2 Annual Conference on Learning Theory, pages 355-366, 2008.
[22] James Davis, Guillermo Gallego, and Huseyin Topaloglu. Assortment optimization under variants of the nested logit model. Operations Research, 62(2):250- 273, 2014. 2
[23] Babur De los Santos and Sergei Koulayev. Optimizing chick-through in online rankings with endogenous search refinement. Marketing Science, 36(4):542-564, 3017.
[24] Arnoud Den Boer. Dynamic pricing and learning: historical origins, current research, and new directions. Surveys in operations research and management science, 20(1):1-18, 2015. 2
[25] Mahsa Derakhshan, Negin Golrezaei, Vahideh Manshadi, and Vahab Mirrokni. Product ranking on online platforms. Management Science, 2022.
[26] Daria Dzyabura and John Hauser. Recommending products when consumers learn their preference weights. Marketing Science, 38(3):417-441, 2019.
[27] Jacob Feldman and Danny Segev. Improved approximation schemes for mnl-driven sequential assortment optimization. Available SSRN 3440645, 2019. 2
[28] Kris Ferreira, Sunanda Parthasarathy, and Shreyas Sekar. Learning to rank an assortment of products. Management Science, 2021. 1,2
[29] Alvaro Flores, Gerardo Berbeglia, and Pascal Van Hentenryck. Assortment optimization under the sequential multinomial logit model. European Journal of Operational Research, 273(3): 1052-1064, 2019. 2
[30] Guillermo Gallego, Anran Li, Van-Anh Truong, and Xinshang Wang. Approximation algorithms for product framing and pricing. Operations Research, 68(1):134-16)) 2020.
[31] Pin Gao, Yuhang Ma, Ningyuan Chen, Guillermo Gallego, Anran Li, Paat Rusmevichientong and Huseyin Topaloglu. Assortment optimization and pricing under the multinomial logit model with impatient eustomers: Sequential recommendation and selection. Operations research, 69
(5):1509-1532, 2021. 2
[32] Xiangyu Gao, Stefanus Jasin, Sajjad Najafi, and Huanan Zhang. Joint learning and optimization for multi-product pricing (and ranking) under general cascade click model. Management Science, 2022. 1,2
[33] Anindya Ghose and Sha Yang. An empirical analysis of search engine advertising: Sponsored search in electronic markets. Management science,
2009.
[34] Fan Guo, Chao Liu, Anitha Kannan, Tom Minka, Michael Taylor, Yi-Min Wang, and Christos Faloutsos. Click chain model in web search. In Proceedings of the 18th international conference on World wide web, pages 1 11-20, 2009. 1,2
[35] Fan Guo, Chao Liu, and Yi Min Wang. Efficient multiple-click models in web search. In Proceedings of the second acm international conference web search and data mining, pages 124-131, 2009.
[36] Sumeet Katariya, Branislav Kveton, Csaba Szepesvari, and Zheng Wen. Dem bandits: Learning to rank with multiple clicks. In International Conference Machine Learning, pages 1215- 1224. PMLR, 2016.
[37] Sergei Koulayev. Search for differentiated products: identification and estimation, The RAND Journal of Economics, 45(3):553-575, 2014.
12

---

[38] Branislav Kveton, Csaba Szepesvari, Zheng Wen, and Azin Ashkan. Cascading bandits: Learning rank in the cascade model. International Conference on Machine Learning. pages 767-776. PMLR, 2015. 1.2
[39] Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual- approach to personalized news article recommendation. Proceedings of the 19th international conference on World wide web, pages 661 2010. 6
[40] Yuan Liang, Chunlin Huang, Xiuguo Bao, and Ke Xu. Sequential dynamic event recommen- dation in event-based social networks: An upper confidence bound approach. Information Sciences, 542:1-23, 2021. 1, 2, 3, 4, 8, 15
[41] Nan Liu, Yuhang Ma, and Huseyin Topaloglu. Assortment optimization under the multinomial logit model with sequential offerings. INFORMS Journal on Computing, 32(3):835-853, 853,, 2020. 2
[42] Qing Liu and Neeraj Arora. Efficient choice designs for conscder-then-choose model. Market- ing Science, 30(2):321-338, 2011.
[43] Siddharth Mahajan and Garrett Van Ryzin. Stocking retail assortments under dynamic consumer substitution. Operations Research, 49(3):334-351, 2001. 1,
[44] Rad Niazadeh, Negin Golrezaei. Joshua R Wang, Fransisca Susan, and Ashwinkumar Badani- diyuru. Online learning via offline greedy algorithms: Applications in market design and optimization. Proceedings of the 22nd ACM Conference on Economics and Computation,
pages 737-738, 2021. 3
[45] Herbert Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58(5):527-535, 1952.
[46] Aleksandrs Slivkins. Introduction to multi-armed bandits. arXiv preprint
2019. 6. 8
[47] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
[48] Theja Tulabandhula, Deeksha Sinha, and Prasoon Patidar, Multi-purchass behavior: Modeling and optimization. arXiv preprint arXiv:2006.00 08055, 2020. 2
[49] Raluca M Ursu. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science, 37(4)5530-55, 2018. 1,3
[50] Mengmeng Wang, Xun Zhang, and Xiaolong Li. Multiple-purchee choice model: Estimation and optimization. Available SSRN 4147705, 2022.
[51] Zheng Wen, Branislav Kveton, Michal Valko, and Sharan Vaswani. Online influence maxi- mization under independent cascade model with semi-bandit feedback. Advances in neural information processing systems, 30, 2017. 27
[52] Renzhe Xu, Xingxuan Zhang, Peng Cui, Bo Li, Zheyan Shen, and Jiazheng Xu. Regulatory instruments for fair personalized pricing. In Proceedings of the ACM Web Conference 2022, pages 4-15, 2022.
[53] Heng Zhang, Hossein Piri, Woonghee Tim Huh, and Hongmin Li. Assortment optimization under multiple-discrete customer choices. Available at SSRN 3988982, 2021. 2
[54] Shi Zong, Hao Ni, Kenny Sung, Nan Rosemary Ke, Zheng Wen, and Branislav Kveton. Cascad- ing bandits for large-scale recommendation problems. arXiv preprint arXiv: 1xXvv:1603005359, 2016
2, 6, 8
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section A.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section A.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Assump- tions 3.1, 5.1, and 5.2.
(b) Did you include complete proofs of all theoretical results? [Yes] See Section C.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as a URL)? [Yes] See Section B. (b) Did you specify all the training details (e.g., data splits, hyper-parameters, how they were chosen)? [Yes] See Section B.
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes] See Figure and Figure 2 in the main body and other figures the appendix.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Section B.
4. If you using existing assets (e.g., code, data, models) or curating/releasing
assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] The dataset url is included the paper.
(b) Did you mention the license of the assets? [Yes] The license is included in the paper.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [Yes] It public dataset with the CC BY-NC 4.0 license.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] The data has been anonymized.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "mLoJGE0-4mY", "reviewer": "Reviewer_oMCA", "summary": "This paper studies a multi-product purchase problem. Authors design an optimal ranking policy when the online retailer can precisely model customers’ behaviors. Most of the existing work assume that each customer purchases at most one product and stops browsing immediately. This paper proposes a more realistic setting that customers view the product list sequentially and purchase a product when its utility exceeds a certain threshold with a certain purchase budget. In both non-contextual and contextual settings, they provide UCB-like algorithms and theoretically prove that the algorithms achieve \\tilde{O}(\\sqrt{T}) regrets in both settings.", "questions": "1.\tAccording to the purchase probability specified in Eq. (1), the purchase probabilities towards different products are independent of each other. How do you verify this assumption? Please also discuss the corresponding applications.\n\n2.\tCould you emphasize the major technical contribution of the online algorithm? For example, how does the analysis differ from the existing work? Which part of the single purchase problem cannot be used in this problem? If it can be emphasized before the main results, it will help to improve the paper.\n\n3.\tIt is assumed that $V_t$ follows the geometric distribution with parameter 1-q_t and B_t follows the geometric distribution with parameter 1-s_t. I think Theorem 4.1 heavily relies on this assumption. Can the result be generalized to other distributions?\n\n4.\tIn Section 5, two Theorems specify the regret upper bounds for both non-contextual and contextual problems. Could you also prove that they are both tight?\n", "limitations": "The attention span and the purchase budget are both limited to the geometric distribution. It would improve the paper if they can be generalized to other distributions.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \nThis paper considers a more practical setting where a single customer can make multiple purchases with certain purchase budget and attention span.\nTheorem 4.1 provides a nice structure of the optimal ranking policy.\nThe study of this ranking problem is comprehensive. It studies both the non-contextual and contextual version.\nThe paper is nicely written.\n\nWeaknesses:\n\nAlthough the paper considers a more practical problem, it still has some limitations: the paper assumes that V_t follows the geometric distribution with parameter 1-q_t and B_t follows the geometric distribution with parameter 1-s_t. Each item costs one unit of budget.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "cg4Lv7tHP1A", "reviewer": "Reviewer_UNt8", "summary": "The paper studies the problem of product ranking in the following setting (1) customers may purchase more than one product, and (2) customers quit after viewing/purchasing a certain number of products. They first characterize the optimal ranking policy when the customer parameters are known. The paper then investigates the case when customer parameters are unknown. Two UCB-like algorithms with O(\\sqrt{T}) regrets are proposed, one for the non-contextual setting (i.e., all customers share the same parameters), and the other for the contextual setting (i.e., customers have personalized parameters). Finally, the effectiveness of the algorithms is tested by simulations. \n", "questions": "1. Could you briefly explain the technical challenges and contributions compared to Liang et al. (2021)?", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\nThe paper identifies some limitations of the previous literature and considers a setting that is an “intersection” of several previous setups (mainly Liang et al. (2021), Chen et al. (2021), and Cao and Sun (2019)), which makes their setting more general or realistic compared to each of the previous papers. The paper is complete in the sense that it investigates all the scenarios that have been previously considered (known parameter/non-contextual online setting/contextual online setting) and provides both theoretical analysis and experimental results. I did not check the whole proof, but the theoretical results seem solid and it seems non-trivial to derive the results in this new setting.\n\nWeakness:\nFor the conceptual contribution, this is an incremental work based on several previous papers. The main concepts and ideas have appeared in previous literature. For the experimental result in the main text, the baselines are some previous methods that are designed under different assumptions. I am not sure how meaningful it is to compare these methods when the synthetic customer data are generated according to a new model. \n\nDisclaimer:\nMy evaluation of the theoretical contribution may not be accurate because I was not able to access the full pdf of Liang et al. (2021), which seems to contain quite similar results according to the outline. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "RIv0Mjxajvf", "reviewer": "Reviewer_tuu8", "summary": "The paper develops vanilla and contextual bandit approaches to maximize revenue given an unknown parametric customer that purchases multiple products.\n", "questions": "See above.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "One of the major issues with this paper is the extremely light and poor comparison with prior work that diminishes the key contributions. Firstly, multiple purchases have already been modeled in the OR literature that are similar to the MNL (e.g., see \"Multi-Purchase Behavior: Modeling and Optimization\" by Tulabandhula et al, \"Assortment Optimization Under Multiple-Discrete Customer Choices\" by Zhang et al, etc). Of course, some of these works focus on the revenue maximization problem in the offline setting. Given this context, the multiple purchase contribution looks like yet another model among this growing collection.\n\nFor the online setting, in the non-contextual case,there have been many works that follow the same template (define how the user parametrically interacts with a sequence of lists) such as \"Thompson sampling for combinatorial semi-bandits\" by Wang et al. and \"Dynamic learning of sequential choice bandit problem under marketing fatigue\" by Cao et al. Especially the latter work shows almost the same model (sans the budget constraint and multi purchase) with near identical results (characterization of the optimal solution in the offline known behavior case) and deriving regret bounds (albeit sparingly for the contextual case in this particular prior work). Given that this type of proof strategy is well known at this point, it is important to highlight what makes the construction of the algorithm and its analysis difficult. \n\nThese two issues make one believe that the contributions are incremental and the authors should make an effort to highlight differences. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ESSOJNSMvR", "reviewer": "Reviewer_PDX6", "summary": "The authors study the problem of product ranking based on a more realistic situation where customers can have multiple purchases and continues viewing products even after buying a product, both being subject to an overall attention span and overall budget. They characterize the optimal ranking policy and design UCB-like algorithms with a provable \\tildeO(\\sqrt{T}) regret based on both contextual and non-contextual situations. They also show extensive experiments to prove their efficacy on the simulated and real-world datasets.\n", "questions": "See above.", "limitations": "NA", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is very well written (though with minor typos (see below)) and is quite easy to follow. The related work section throws good light into the existing literature on which this work is based. The thorough theoretical section is impressive and the results look good. \n\nThe major comment is on the time complexity of the algorithms in practice. Section 4 shows the time complexity as O(N Log N). Please add a discussion on how Algorithms 1 and 2 can put into practice with similar time complexity and appropriate parallelization for each t. \n\nMinor typos:\n1. Line 53, 314: rankling -> ranking\n2. Line 210: management\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept