Product Ranking for Revenue Maximization with Multiple Purchases
Anonymous Author(s)
AffiliationAddress email
Abstract
Product ranking is the core problem for revenue-maximizing online retailers.
Todesign proper product ranking algorithms, various customer behavior models areproposed to characterize the customers’ behaviors when they are provided a list ofproducts.
However, existing works assume that each customer purchases at mostone product or will keep viewing the product list after purchasing a product, whichdoes not agree with the common practice in real scenarios.
We assume that eachcustomer can purchase multiple products at will in this work.
To model customers’willingness to view and purchase, we set a random attention span and purchasebudget, which determines the maximal amount of products that he/she views andpurchase, respectively.
Under this setting, we first design an optimal ranking policywhen the online retailer can precisely model customers’ behaviors.
Based on thepolicy, we further develop UCB-like algorithms with ˜ O ( √ T ) regret that estimatescustomers’ behaviors and maximize revenue simultaneously in online settings.
Experiments on both synthetic and real-world datasets prove the effectiveness ofthe proposed algorithms.
Introduction
Online retailing has become increasingly popular over the last decades [17, 27].
The way of productranking is the crux for online retailers because it determines the customers’ shopping behaviors[17] and thus influences the retailers’ revenue [20, 47].
For instance, the probability of customers’purchasing from a firm or clicking an advertisement is strongly related to the display order [8, 3, 32].
Therefore, it is crucial for online retailers to design proper product ranking algorithms for revenuemaximization.
The customers’ behavior, when faced with a list of products, is the basis of the problem [17].
Generallyspeaking, existing works assume that the customers view the list of products sequentially and selectproducts according to a choice model [42, 41, 31, 37, 18, 25, 24, 36].
However, most of them assumethat each customer purchases at most one product and stop browsing immediately afterward, whichdoes not agree with the common practice since a customer may expect multiple purchases on awebsite or an app [33, 34, 16, 27].
Recently, several works consider a multiple-purchase setting whiletheir targets are not revenue maximization [27, 15], or they assume that customers always continueviewing the product list after purchasing any product, which is not practical in real scenarios [39].
In this paper, we propose a more realistic setting to characterize the customer behaviors undermultiple-purchase settings.
Following [13, 17, 39], we assume that the customers view the productlist sequentially and purchase a product when its utility exceeds a certain threshold.
Similar to [17], we assume the existence of the attention span for each customer, which determines the maximalamount of products that the customer is willing to view.
More importantly, instead of forcing thecustomer to quit viewing once she purchases one product, we allow each customer to conduct multiplepurchases at will and explore product ranking models for revenue maximization.
Specifically, weconstrain each customer with a purchase budget to determine the maximal amount of purchases.
Boththe attention span and the purchase budget are assumed to obey the geometric distribution [13, 39].
Under this setting, we study the optimal product ranking policy to maximize the online retailers’revenue.
We first characterize the optimal ranking policy that achieves the maximal revenue whenthe online retailer knows the customer parameters, including the purchase probabilities for eachproduct and the distribution of the attention span and purchase budget.
Afterward, we investigatethe problem in online learning frameworks where customer parameters are unknown to onlineretailers.
In both non-contextual ( i.e. , all customers share the same parameters) and contextualsettings ( i.e. , customers have personalized parameters), we provide UCB-like algorithms [1] thatcould fit customers’ parameters and maximize revenue simultaneously.
We further theoretically provethat the algorithms achieve ˜ O ( √ T ) regrets in both settings.
Finally, we conduct experiments to verifythe effectiveness of the algorithms.
To conclude, our contributions are highlighted as follows:
• We propose a novel setting to deal with multiple-purchase activities of customers on online
scenarios.• We characterize the optimal ranking policy and further design UCB-like algorithms with ˜ O ( √ T ) regret based on them in both non-contextual and contextual online settings.
• We conduct extensive experiments to prove the effectiveness of our method on the multiple-
purchase setting.
Related works
Customers’ behavior model Traditional choice models adopt the multinomial logic (MNL) [12,22, 42] models to characterize customers’ behaviors.
However, these models can not describe thebehaviors when the customers are provided with a list of products.
As a result, several choice modelsare proposed recently.
In detail, Abeliuk et al.
[2] add a position bias term in the traditional MNLmodel.
Gallego et al.
[29], Aouad and Segev [4] suppose that the products are framed into a set ofvirtual web pages.
Customers select a product, if any, from these pages following a general choicemodel (such as MNL).
Flores et al.
[28] propose the Sequential Multinomial Logit (SML), whichassumed a two-stage MNL model and supposed that products are divided into two stages with priorknowledge.
Liu et al.
[40], Feldman and Segev [26], Gao et al.
[30] further extend the setting tomultiple stages.
These works suppose that each stage contains multiple products while recentlyseveral works assume that a product is treated as a single stage [17, 13, 14].
These models are alsoclosely related to the cascade model [12, 35, 37, 48, 19].
Gao et al.
[31] further propose the generalcascade click model.
Following these works, we suppose that each stage contains only one productand the customers views the products sequentially.
In addition, several works consider the setting where customers may have multiple interactions withthe platforms.
However, most of them focus on the click model [15, 16, 33–35].
In addition, [27]maximizes the number of customers who engage with the site.
Liang et al.
[39] aimed to maximize therevenue in the multiple-purchase setting while they assume that customers always continue viewingthe product list after purchasing any product, which is not practical in real scenarios.
Online learning and multi-armed bandit
Online retailers need to design proper algorithms tomodel customers’ behaviors and maximize the revenue at the same time, which is closely relatedto the multi-armed bandit framework [46, 44].
Many works [5, 9, 11, 10, 23] have been proposedto deal with demand learning or price experimentation problems with the help of the frameworkin the field of revenue management.
Recently, more methods [39, 17, 13, 14, 6, 43] adopt similartechniques in the product ranking setting, which inspires the algorithm proposed in this paper.
Preliminaries
Let [ N ] ≜ { 1 , 2 , . .
, N } be the set of all products owned by an online-retailer (he).
The productk generates revenue r k when a customer purchases it.
Let r max ≜ max k ∈ [ N ] r k be the maximalrevenue for all products and r = { r k } Nk =1 .
We use the notation σ t ⊣ [ N ] to denote the ranking policyσ t on the products for the t -th customer (she).
More precisely, ∀ k ∈
[ N ] , σ t,k ∈
[ N ] represents theproduct displayed in the k -th position for her.
Reversely, σ − 1 t,k denotes the position of the k -th productin the ranking list.
Furthermore, for any subset Q ⊆ [ N ] , we use σ − 1 t,Q to denote the positions ofthe products in Q and σ − 1 t,Q = { σ − 1 t,k : k ∈ Q } .
In addition, we use max( · ) to denote the maximalelement in a set.
The customers’ behavior model When the t -th customer (she) arrives, she views the list ofproducts sequentially.
The probability of product k being purchased by her is denoted as λ t,k andlet λ t = { λ t,k } Nk =1 .
To characterize the customers’ behaviors under multiple-purchase settings, weassume that the customer is endowed with the attention span V t and purchase budget B t , whichdetermine the maximal amount of the products that she will view and purchase, respectively.
In reality,the customer always have random attention span and purchase budget [17].
Therefore, following[13, 39], we assume V t ∼ Geo(1 − q t ) and B t ∼ Geo(1 − s t ) are both geometric distributions withparameter 1 − q t and 1 − s t , respectively.
As a result, P ( V t = v ) =
q v − 1 t (1 − q t ) for any v ≥ 1 and
P ( B t = b ) =
s b − 1 t (1 − s t ) for any b ≥ 1 .
With attention span V t and purchase budget B t , the customer keeps viewing the products untilshe views V t or purchases B t products.
The top V t products shown by the ranking policy σ t is
A t = { σ t,i : i ∈
[ V t ] } .
As a result, the probability that the customer purchases a subset of products
Q ⊆
A t is given by
[Equation]
Here the condition σ − 1 t,k ≤ max( σ − 1 t,Q ) requires that the ranking policy σ t displays the product kearlier than some products in the set Q .
Specifically, the probability P buy ( Q ; V t , B t , λ t , σ t ) iscalculated based on the cardinality of Q .
When | Q | < B t , the customer does not meet her purchasebudget and stops browsing the list after viewing V t products.
When | Q | =
B t , the customer stopsbrowsing as long as she purchases the last product in the list Q .
When | Q | > B t , the probabilitybecomes 0 since the amount of purchases | Q | exceeds her purchase budget.
Remark.
The setting of the purchase budget provides a rational way to characterize the customers’behaviors with multiple purchases and extends previous works that suppose customers would purchaseat most one product [13, 17].
Furthermore, when s t → 0 , the customers would tend to purchaseat most one product and our setting degenerates to the setting proposed in [13].
Thus the previoussetting can be considered as a special case of our setting.
Revenue optimization The online-retailer chooses a ranking policy σ t ⊣ [ N ] on the products forthe t -th customer.
For the customer with fixed purchase probabilities for each product λ t , attentionspan V t , and purchase budget B t , the expected revenue could be written as
[Equation]
In reality, the attention span V t and purchase budget B t are random and can not be estimatedaccurately by the online retailer.
As a result, the retailer calculates the expected revenue for the t -thcustomer by sampling V t from Geo(1 − q t ) and sampling B t from Geo(1 − s t ) .
Revenue
[Equation]
To maximize the revenue, the optimal ranking policy is given by
[Equation]
For simplicity, we use σ ∗ t to denote σ ∗ t ( λ t , q t , s t ) .
Basic assumption We also need the following assumption to guarantee the tractability of theproposed problem.
In reality, customers always have finite attention span, which indicates that q t isalways smaller than 1 .
Assumption 3.1.
There exists a known constant ϵ q > 0 such that for all t > 0 , q t ≤ 1 − ϵ q .
Optimal ranking policy given customers’ characteristics
In this section, we introduce the optimal ranking policy when the online retailer exactly knows thecustomers’ parameters, including the purchase probability for each product λ , random attention span(parametrized with q ), and random purchase budget (parametrized with s ).
We drop the subscript t in this section for simplicity.
The optimal ranking strategy stems from thefollowing theorem.
Theorem 4.1.
Under Assumption 3.1, for any 1 ≤ i < N , the following holds for the optimal rankingstrategy σ ∗ ( λ , q, s ) .
[Equation]
The expected revenue of a single product σ ∗ i is λ σ ∗ i r σ ∗ i , which grows as the purchase probabilityand the revenue generated by the product increases.
Yet purchasing a product reduces the purchaseprobability of the following products, and thus the expected revenue of other products decreases asσ ∗ i increases.
Furthermore, when s → 0 , the term in Equation (5) become the same as that in [13,
Theorem 1].
As a result, Theorem 4.1 consistently extends the results on the single purchase setting[13] to the multiple purchase setting.
As a result, according to Theorem 4.1, the online retailer can provide the optimal ranking policy σ ∗by sorting the products in a descending order with value ( λ k r k ) / (1 − q + q (1 − s ) λ k ) for the k -thproduct.
The time complexity is O
( N log N ) .
Online learning of the ranking policy
In this section, we consider a more realistic scenario where the online retailer has no prior knowledgeabout customers’ characteristics.
At each timestamp, a customer arrives and the online retailercan only provide proper product ranking policies based on the feedbacks of previous customers.
As a result, it is essential for the retailer to design online learning algorithms to model customers’behaviors and maximize the revenue in the meantime.
We consider two settings, i.e. , the noncontextual setting where all customers share the same parameters (Section 5.1) and the contextualsetting where customers have personalized behaviors (Section 5.2).
Before introducing the details ofthe algorithms, we first formally define the notations to characterize the customers’ behaviors.
Notations for customers’ behaviors Let Φ t be the random variable that denotes the amount ofproducts viewed by the t -th customer and Γ t = { γ t,k } Φ t k
=1 be the purchase activities.
Specifically,γ t,k = 1 if
and only if the product at the k -th position ( i.e. , σ t,k ) is purchased.
Let random variable η t,k denote whether the customer continues viewing products after she views thek -th product but does not buy it and η t = { η t,k } Φ t k =1 .
Let random variable µ t,k denote whether thecustomer continues viewing products if she buys the k -th product and µ t = { µ t,k } Φ t k
=
So η t,k isobserved only when γ t,k = 0 and µ t,k is observed only when γ t,k = 1 .
Let Ber( · ) denote the Bernoulli distribution and we have the following result on the distribution ofthese random variables.
Lemma 5.1.
γ t,k are i.i.d.
random variables and γ t,k ∼ Ber (cid:0) λ σ t,k (cid:1) .
η t,k are i.i.d.
random variablesand
η t,k ∼ Ber( q t ) .
µ t,k are i.i.d.
random variables and µ t,k ∼ Ber( q t s t ) .
Lemma 5.1 is the basis of the algorithms introduced later that estimate parameters λ , q , and s fromhistorical data.
In addition, compared with s t , it is easier to estimate q t s t from data and we denote itas w t ≜ q t s t .
5.1 Non-contextual online setting
In this subsection, we assume that all customers share the same parameters, including the purchaseprobabilities on different products and the parameters on the distribution of the attention span andpurchase budget.
To be specific, we assume that there exist λ , q, s such that λ t = λ , q t = q , ands t = s .
A) Estimation of parameters
To estimate the parameters at timestep t , we leverage the observed behavior of customers beforetimestep t .
We first calculate the following statistics that can help provide unbiased estimators of thecustomer parameters.
In detail, let C t,k be the total number of times that the product k is observed and c t,k be thenumber of times the product k is purchased, i.e. , C t,k ≜ (cid:80) tu =1 (cid:80) Φ t
i
=
I [ σ t,i = k ] and c t,k ≜ 178 (cid:80) tu =1 (cid:80) Φ t i =
I [ σ t,i =
k ] I [ γ t,i = 1] .
In addition, Let D qt be times that customers view a product andnot purchases it and d qt be the times that customers continue viewing the list after viewing a productwithout purchasing it, i.e. , D
Furthermore, let D wt be times that customers purchase a product and d wt be the times that customerscontinue viewing the list after purchasing a product, i.e. , D wt ≜ (cid:80) tu =1 (cid:80) Φ t i =
I [ γ t,i = 1] and d wt ≜ 183 (cid:80) tu =1 (cid:80) Φ t i =
I [ γ t,i = 1] µ t,i .
With these statistics, the parameters λ , q , and w = qs are estimated as follows.
[Equation]
According to the Hoeffding’s inequality, we provide the following proposition that gives the errorbound of our parameter estimation approach.
Proposition 5.2.
For any t ≥ 1 , with probability at least 1 − 2( N + 2) t − 2 , the following holds:
[Equation]
B) Algorithm
Following the classic optimism in the face of uncertainty principle [1], we design a UCB-like 190 algorithm that learns customer behaviors and maximizes revenue simultaneously.
In detail, accordingto Proposition 5.2, we learn the optimistic estimators of the parameters given as follows
[Equation]
(cid:115) log t
C t
[Equation]
k (cid:41) , ˜ q t ≜ min (cid:40) 1 − ϵ q , ˆ q t + (cid:115) log t
D qt (cid:41) , ˜ w t ≜ min (cid:40) ˜
q t , ˆ w t + (cid:115) log t
D wt (cid:41) .
As shown in Algorithm 1, when the t -th customer arrives, the online retailer displays the productsbased on Theorem 4.1 with ˜ λ
t − 1 , ˜ q t − 1 , and ˜ w t − 1 / ˜
q t − 1 shown in Step 4.
Afterward, the onlineretailer updates the statistics according to customers’ feedbacks as shown in Steps 5 and 6.
Theoptimistic estimators are then calculated in Step 7.
1 Input: Products revenue r and hyperparameter ϵ q .
2 Initialization: ˜ λ 0 ,k = 1 for k ∈ [ N ] , ˜ q 0 = 1 − ϵ q , ˜ w 0 = 1 − ϵ q .
3 for t=1:T do
Let σ t be the optimal offline policy from Theorem 4.1 with λ = ˜ λ
t − 1 , q = ˜ q t − 1 , and s = ˜
w t − 1 / ˜
q t − 1 .
Offer ranking policy σ t and observe Φ t , Γ t , η t , µ t .
Update statistics C t,k , c t,k , D qt , d qt , D wt , and d wt .
Calculate ˜ λ t,k , ˜ q t , and ˜ w t by Equations (6) and (8).
8 end
C) Regret analysis
To theoretically analyze the performance of Algorithm 1, we first introduce the regret, which measuresthe total difference between the maximal revenue achieved by σ ∗ and the cumulative reward by anonline algorithm after T rounds.
[Equation]
Reg T is a random variable and the randomness comes from the uncertainty in customers’ behaviors.
As a result, we focus on E [Reg T ] , which is a common routine in bandit literature [45].
Theperformance of the algorithm is given by the following theorem.
Theorem 5.3.
Under Assumption 3.1 (with parameter ϵ q ), for any T > 1 , the regret achieved by
Algorithm 1 is bounded by
[Equation] [Equation]
where C is a absolute constant (independent of problem parameters) and r max = max k ∈
[ N ] r k .
Remark.
We analyze different parameters’ impacts on the regret bound in Equation (10).
Firstly, theregret grows at ˜ O ( √ T ) , which is a standard result in online learning algorithms [45].
Secondly, theregret bound linearly related to N , which is similar to the results in cascade bandit models [48] andrevenue management literature [17].
Finally, the bound depends on r max /ϵ q , which measures themaximal expected revenue the online retailer could achieve from each customer.
5.2 Contextual online setting
We consider a more realistic scenario where the customers’ behaviors are various and depend on theirown features.
In detail, let x t ∈ R m x be the feature of the t -th customer with dimension m x andy t,k ∈ R m y be the joint feature of the t -th customer and k -th product with dimension m y .
y t,k can bethe concatenation of customer features and product features or fusion through a transformation suchas matrix multiplication.
In this paper, we adopt concatenation for simplicity.
Let ∥ · ∥ denote the ℓ 2norm.
Following [17], we consider the classic linear bandit setting [7, 21, 38] and the ground-truthdata-generating process is given by the following assumptions.
Assumption 5.1.
There exists constant β λ ∈ R m y , β q ∈ R m x , and β s ∈ R m x , such that
[Equation]
Assumption 5.2.
For any t > 0 and k ∈ [ N ] , ∥ x t ∥ ≤ 1 and ∥ y t,k ∥ ≤ 1 .
In addition, there exists aconstant U > 0 such that ∥ β λ ∥ , ∥ β q ∥ , ∥ β s ∥ ≤ U .
Remark.
These two assumptions are common in revenue maximization and linear bandits [1, 45].
Assumption 5.1 can be satisfied with kernel functions for complex data-generating processes and
Assumption 5.2 can be achieved by normalization.
Thus the assumptions are rational and easilyattainable in real applications.
A) Estimation of parameters
For the estimation of λ t and q t , we use the ridge regression method to estimate β λ and β q .
Given theobservation of the t -th customer’s behaviors, the estimation ˆ β t,λ , ˆ β t,q is given by
[Equation]
Here Σ t,λ = (cid:80) tu =1 (cid:80) Φ u k =1 y u,σ u
[Equation]
k y ⊤ u,σ u,k
+ α λ
I m y , ρ
[Equation]
,λ = (cid:80) tu =1 (cid:80) Φ u k =1 γ u,k y u,σ u,k , Σ t
[Equation]
q =(cid:80) tu =1 (cid:80)
Φ u k =
I
[ γ u,k = 0] x u x ⊤ u + α q
I m x , and ρ t,q = (cid:80) tu =1 (cid:80) Φ u k
=
I [ γ u,k = 0] η u,k x u .
I m x and
I m y are the identity matrices with dimension
m x and m y , respectively.
For the estimation of s t , similar to Section 5.1, we estimate w t =
q t s t instead.
Note that w t =
q t s t = x
⊤ t β q β ⊤ s x t = vec( x t x ⊤ t ) ⊤ vec( β q β ⊤ s ) , where vec( · ) denotes the vectorization operation thatmaps a matrix to a vector.
As a result, define z t ≜ vec( x t x ⊤ t ) and β w ≜ vec( β q β ⊤ s ) and we can getw t =
z ⊤ t β w .
Similarly, we estimate β w by conducting ridge regression as follows
[Equation]
Here
[Equation]
I m 2 x is the identity matrix with dimension m 2 x .
Let ∥ β
∥
A = (cid:112) β ⊤ Aβ for any positive definite matrix A .
Proposition 5.4 gives the estimation errorbound.
Proposition 5.4.
Under Assumptions 5.1 and 5.2 (with parameter U ), for any t ≥ 1 , with probabilityat least 1 − 3 t − 2 , the following holds:
[Equation]
Here τ ( t, m, α ) = 1 / 2 (cid:112) m log (1 + tN/ ( mα ))
+ 4 log t + α 1 /
( U + 1) 2 .
B) Algorithm
Given the estimation of β λ , β q and β w we estimate λ t,k , q t and w t as follows.
[Equation]
As shown in Algorithm 2, when the t -th customer arrives, the online retailer observes the customerfeatures x t and y t,k for k ∈ [ N ] before learns the intermediate feature z t and calculate ˜ λ t,k , ˜ q t , and˜ w t according to Equation (16).
Then the ranking policy σ t is given as shown in Step 6.
Then thestatistics are calculated in Step 6 and 7 and finally the parameters ˆ β t,λ , ˆ β t,q , and ˆ β t,w are estimated in
Step 9 for the next iteration.
C) Regret analysis
We analyze the performance of Algorithm 2 by investigating its regret bound.
For a sequence of Tcustomers with features X T = { x t } Tt =1 and Y T = {{ y t,k } Nk =1 } Tt =1 , the regret is given by
Reg T
[Equation]
, Y
[Equation] [Equation]
T
[Equation]
t =
[Equation]
Revenue (
[Equation]
, q
[Equation]
s
[Equation] [Equation]
[Equation]
λ
˜ λ t q = ˜
q t , s = ˜ w t
˜ q t .
t t µ
Similar to the non-contextual setting, Reg T ( X T , Y T ) is a random variable and we focus on 254 E [Reg T ( X T , Y T )] .
As a result, the performance of the algorithm is given by the following theorem.
Theorem 5.5.
Under Assumptions 3.1 (with parameter ϵ q ), 5.1, and 5.2 (with parameter U ), for any
T > 1 , the regret achieved by Algorithm 2 is bounded by
[Equation]
Here C is a absolute constant (independent of problem parameters), r max = max k ∈
[ N ] r k , andχ ( T, m, α
[Equation]
=
[Equation]
Remark.
We analyze different parameters’ impacts on the regret bound in Equation (19).
Firstly,the regret grows at ˜ O ( √ T ) , a standard result in online learning algorithms [45].
Secondly, the regretdepends ˜ O
( N ) , which shares a similar result with the revenue management literature [17].
Thirdly,similar to Theorem 5.3, the bound depends linearly on r max /ϵ q that measures the maximal expectedrevenue the online retailer could achieve from each customer.
Fourthly, the bound depends on thedimensions of features ( i.e. , m x and m y ) by ˜ O ( √ m y ) and ˜ O
( m x ) because the feature dimension forthe estimation of w t =
q t s t is m 2 x
and then the result is consistent with [17, 48].
Finally, the bounddepends on O ( U 2 + U ) since ∥ β w ∥ ≤ U 2 and ∥ β q ∥ , ∥
β λ ∥ ≤ U , which agrees with [17].
Experiments
In this section, we conduct experiments on synthetic data to verify the performances of the onlineranking policies.
More experimental results on both synthetic and real-world datasets are in appendix.
Data-generating processes For the non-contextual setting
, we consider N = 50 and 300 productsand T = 100 , 000 customers.
The revenue for each product r k is uniformly distributed in [0 , 1] andthe purchase probabilities for each product are uniformly sampled from [0 , 0 . 2] .
We set q = 0 .
8 ands = 0 .
2 and 0 .
5 to test the performance of different models on characterizing customers’ behaviorswith multiple purchases.
For the contextual setting, we consider N = 50 and 300 products and T = 30 , 000 customers.
The revenue r k is also uniformly distributed in [0 , 1] .
Both the dimension of customer featuresand products features are set to 10 (i.e., m x = 10 ).
The joint feature y t,k (the feature of the t -thcustomer and k -th product) is a concatenate of the corresponding customer and product feature so thatm y = 20 .
The customer features and product features are both uniformly sampled from [0 , 1 / √ m x ]to satisfy the normalization constraint in Assumption 5.2.
Afterward, the coefficients β λ , β q , andβ s in Equation (11) are uniformly sampled from [0 , 1] .
To ensure λ t,k , q t , s t have similar range inthe contextual setting, we normalize the coefficients such that the maximal elements in λ t,k and q t(denoted as λ max and q max ) are 0 .
2 and 0 . 8 , respectively.
Similarly, by normalizing β s , we set themaximal element in s t (denoted as s max ) to 0 . 2 and 0 . 5 .
[Figure]
Baselines We adopt the following baselines.
Firstly, Cao and Sun [13] considered the singlepurchase setting, which is the simplified version of our multiple purchase setting.
We denote it as
Single purchase .
Secondly, we adopt the method in [39] (denoted as Keep viewing ), which assumesthat customers always keep viewing the ranking list after purchasing products.
Furthermore, weimplement two explore-then-exploit algorithms (denoted as Explore then exploit A and Explorethen exploit B , respectively) to verify the advantages of our method that could balance explorationand exploitation.
Results and analysis
We implement the experiments for 10 distinct simulations by resamplingcustomers’ behaviors while their basic characteristics remain unchanged.
Specifically, in both thenon-contextual and contextual settings, the purchase probabilities ( i.e. , λ t ) and parameters on thespan attention and purchase budget ( i.e. , q t and s t ) are the same.
However, the same customer indistinct simulations may purchase different products due to randomness.
We then evaluate our method and baselines and present the regret via calculating the regret (accordingto Equations (9) and (17)).
The results are shown in Figure 1 and our method outperforms all baselinesin both settings with various parameters.
On the one hand, the Single purchase and Keep viewingbaseline considers different customers’ behaviors settings and is not directly appliable here for themore realistic multiple purchase setting.
On the other hand, compared with the explore-then-exploitbased algorithms, our method integrates the advantages from traditional UCB-like algorithms, leadingto the better exploration-exploitation trade-off for smaller regret.
Furthermore, as a accumulationof regret of each customer, the reported overall regret grows as the number of customers increases.
Basically, the gap between our method and Single purchase decreases as N and s/s max decreasesince the expected number of purchase is approaching to 1.
Thus as an extension of the singlepurchase setting, the multiple purchase setting is more realistic and challenging.
Moreover, thegrowth gradient of our method is smoother than its counterparts, suggesting better scalability oncustomer numbers and applicability to real application scenarios.
Reports and discussions of moreresults including the expected revenue and the ratio over the expected revenue of the optimal rankingpolicy are in Appendix.
Conclusion
To conclude, we propose a novel setting to deal with multiple-purchase activities of customers ononline scenarios.
We characterize the optimal ranking policy and further design UCB-like algorithmswith ˜ O ( √ T ) regret based on them in both non-contextual and contextual online settings.
We conductextensive experiments to prove the effectiveness of our method.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See details in the appendix.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See detailsin the appendix.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Assump-
tion 3.1, 5.1, and 5.2.
(b) Did you include complete proofs of all theoretical results?
[Yes] See details in thesupplementary material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Details can befound in the appendix
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] Details can be found in the appendix
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[Yes] See Figure
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See details in the appendix4.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]