Product Ranking for Revenue Maximization with Multiple Purchases
Abstract
Product ranking is the core problem for revenue-maximizing online retailers.
To design proper product ranking algorithms, various consumer choice models are proposed to characterize the consumers’ behaviors when they are provided with a list of products.
However, existing works assume that each consumer purchases at most one product or will keep viewing the product list after purchasing a product, which does not agree with the common practice in real scenarios.
In this paper, we assume that each consumer can purchase multiple products at will.
To model consumers’ willingness to view and purchase, we set a random attention span and purchase budget, which determines the maximal amount of products that he/she views and purchases, respectively.
Under this setting, we first design an optimal ranking policy when the online retailer can precisely model consumers’ behaviors.
Based on the policy, we further develop the Multiple-Purchase-with-Budget UCB (MPB-UCB) algorithms with ˜ O ( √ T ) regret that estimate consumers’ behaviors and maximize revenue simultaneously in online settings.
Experiments on both synthetic and semi-synthetic datasets prove the effectiveness of the proposed algorithms.
The source code is available at https://github.com/windxrz/MPB-UCB .
Introduction
Online retailing has become increasingly popular over the last decades [17, 28, 52].
The way of product ranking is the crux for online retailers because it determines the consumers’ shopping behaviors [17] and thus influences the retailers’ revenue [20, 49].
For instance, the probability of consumers’ purchasing from a firm or clicking an advertisement is strongly related to the display order [8, 3, 33].
Therefore, it is crucial for online retailers to design proper product ranking algorithms for revenue maximization.
The consumer choice model, when faced with a list of products, is the basis of the problem [17].
Generally speaking, existing works assume that the consumers view the list of products sequentially and select products according to a choice model [43, 42, 32, 38, 18, 26, 25, 37].
However, most of them assume that each consumer purchases at most one product and stop browsing immediately afterward, which does not agree with the common practice since a consumer may expect multiple purchases on a website or an app [34, 35, 16, 28].
Recently, several works consider a multiplepurchase setting while their targets are not revenue maximization [28, 15], or they assume that consumers always continue viewing the product list after purchasing any product, which is not practical in real scenarios [40].
In this paper, we propose a more realistic consumer choice model to characterize consumer behaviors under multiple-purchase settings.
Following [13, 17, 40], we assume that the consumers view the product list sequentially and purchase a product when its utility exceeds a certain threshold.
Similar to [17], we assume the existence of the attention span for each consumer, which determines the maximal amount of products that the consumer is willing to view.
More importantly, instead of forcing the consumer to quit viewing once she purchases one product, we allow each consumer to conduct multiple purchases at will and explore product ranking models for revenue maximization.
Specifically, we constrain each consumer with a purchase budget to determine the maximal amount of purchases.
Both the attention span and the purchase budget are assumed to obey the geometric distribution [13, 40].
Under this setting, we study the optimal product ranking policy to maximize online retailers’ revenue.
We first characterize the optimal ranking policy that achieves the maximal revenue when the online retailer knows the consumer parameters, including the purchase probabilities for each product and the distribution of the attention span and purchase budget.
Afterward, we investigate the problem in online learning frameworks where consumer parameters are unknown to online retailers.
In both non-contextual ( i.e. , all consumers share the same parameters) and contextual settings ( i.e. , consumers have personalized parameters), we provide the Multiple-Purchase-with-Budget UCB (MPB-UCB) algorithms that could fit consumers’ parameters and maximize revenue simultaneously.
We further theoretically prove that the algorithms achieve ˜ O ( √ T ) regrets in both settings.
Finally, we conduct experiments to verify the effectiveness of the algorithms.
To conclude, our contributions are highlighted as follows:
• We propose a novel consumer choice model to deal with multiple-purchase activities of consumers in online retailing scenarios.
• We characterize the optimal ranking policy and further design the MPB-UCB algorithms with ˜ O ( √ T ) regret based on them in both non-contextual and contextual online settings.
• We conduct experiments on both synthetic and semi-synthetic datasets to prove the effectiveness of our method in the multiple-purchase setting.
Related Works
Consumer choice model Traditional choice models adopt the multinomial logic (MNL) [12, 22, 43] models to characterize consumers’ behaviors.
However, these models can not describe the behaviors when the consumers are provided with a list of products.
As a result, several choice models are proposed recently.
In detail, Abeliuk et al.
[2] add a position bias term in the traditional MNL model.
Gallego et al.
[30], Aouad and Segev [4] suppose that the products are framed into a set of virtual web pages.
Consumers select a product, if any, from these pages following a general choice model (such as MNL).
Flores et al.
[29] propose the Sequential Multinomial Logit (SML), which assumed a two-stage MNL model and supposed that products are divided into two stages with prior knowledge.
Liu et al.
[41], Feldman and Segev [27], Gao et al.
[31] further extend the setting to multiple stages.
These works suppose that each stage contains multiple products while recently several works assume that a product is treated as a single stage [17, 13, 14].
These models are also closely related to the cascade model [12, 36, 38, 54, 19].
Gao et al.
[32] further propose the general cascade click models.
Following these works, we suppose that each stage contains only one product and the consumers view the products sequentially.
In addition, several works consider the setting where consumers may have multiple interactions with the platforms.
However, most of them focus on the click model [15, 16, 34–36] or MNL-based consumer choice models [48, 53, 50].
In addition, Ferreira et al.
[28] maximize the number of consumers who engage with the site.
Liang et al.
[40] aim to maximize the revenue in the multiplepurchase setting while they assume that consumers always continue viewing the product list after purchasing any product, which is not practical in real scenarios.
Online learning and multi-armed bandit
Online retailers need to design proper algorithms to model consumers’ behaviors and maximize revenue at the same time, which is closely related to the multi-armed bandit framework [47, 45].
Many works [5, 9, 11, 10, 24] have been proposed to deal with demand learning or price experimentation problems with the help of the framework in the field of revenue management.
Recently, more methods [40, 17, 13, 14, 6, 44] adopt similar techniques in the product ranking setting, which inspires the algorithm proposed in this paper.
Preliminaries
Let [ N ] ≜ { 1 , 2 , . .
, N } be the set of all products owned by an online retailer (he).
The product k generates revenue r k when a consumer purchases it.
Let r max ≜ max k ∈ [ N ] r k be the maximal revenue for all products and r = { r k } Nk =1 .
We use the notation σ t ⊣ [ N ] to denote the ranking policy σ t on the products for the t -th consumer (she).
More precisely, ∀ k ∈
[ N ] , σ t,k ∈
[ N ] represents the product displayed in the k -th position for her.
Reversely, σ − 1 t,k denotes the position of the k -th product in the ranking list.
Furthermore, for any subset Q ⊆ [ N ] that represents the possible positions in the ranking list, we use σ t,Q to denote the corresponding set of the products w.r.t.
the position set Q , i.e. , σ t,Q = { σ t,k : k ∈ Q } .
In addition, we use max( · ) to denote the maximal element in a set.
Consumer choice model for multiple purchases When the t -th consumer arrives, she views the list of products sequentially.
We assume that the behaviors of different consumers are independent.
The probability of product k being purchased by her is denoted as λ t,k and let λ t = { λ t,k } Nk =1 .
We assume for tractability that consumer interests in any products are independent, which is common in product ranking papers [4, 18, 23, 49].
This indicates that the purchase behavior of any product set does not affect the purchase probabilities of other products.
To characterize the consumers’ behaviors under multiple-purchase settings, we assume that the consumer is endowed with the attention span V t and purchase budget B t , which determine the maximal number of the products that she will view and purchase, respectively.
In reality, the consumer always has random attention span and purchase budget [17].
Therefore, following [13, 40], we assume V t ∼ Geo(1 − q t ) and B t ∼ Geo(1 − s t ) are independent geometric distributions with parameters 1 − q t and 1 − s t , respectively.
As a result, P ( V t = v ) =
q v − 1 t (1 − q t ) for any v ≥and P ( B t = b ) =
s b − 1 t (1 − s t ) for any b ≥ 1 .
With attention span V t and purchase budget B t , the consumer keeps viewing the products until she views V t or purchases B t products.
Because of the attention span V t , the consumer will not purchase products whose positions are greater than V t in the ranking list.
Let Q ⊆
[ V t ] denote the possible positions of the purchased products by the consumer.
The corresponding indices of purchased products are σ t,Q .
Therefore, the probability that the consumer purchases the product set σ t,Q is given by
[Equation]
Specifically, the probability P buy ( Q ; V t , B t , λ t , σ t ) is calculated based on the cardinality of Q .
When | Q | < B t , the consumer does not meet her purchase budget and stops browsing the list after viewing V t products.
When | Q | =
B t , the consumer stops browsing as long as she purchases the last product in the list Q .
When | Q | > B t , the probability becomes 0 since the number of purchases | Q | exceeds her purchase budget.
Remark.
The setting of the purchase budget provides a rational way to characterize the consumers’ behaviors with multiple purchases and extends previous works that suppose consumers would purchase at most one product [13, 17].
Furthermore, when s t → 0 , the consumers would tend to purchase at most one product and our setting degenerates to the setting proposed in [13].
Thus the previous setting can be considered a special case of our setting.
In addition, Liang et al.
[40] assume that consumers always continue viewing the product list after purchasing any product, which is not practical in real scenarios.
By contrast, the purchase budget proposed in our paper leads to a different and more practical consumer choice model.
In addition, our model introduces an extra parameter ( i.e. , the parameter), which makes our problem more challenging.
Revenue optimization The online retailer chooses a ranking policy σ t ⊣
[ N ] on the products for the t -th consumer.
For the consumer with fixed purchase probabilities for each product λ t , attention span V t , and purchase budget B t , the expected revenue could be written as
[Equation]
In reality, the attention span V t and purchase budget B t are random and can not be estimated accurately by the online retailer.
As a result, the retailer calculates the expected revenue for the t -th consumer by sampling V t from Geo(1 − q t ) and sampling B t from Geo(1 − s t ) , i.e. ,
[Equation]
To maximize the revenue, the optimal ranking policy is given by
[Equation]
For simplicity, we use σ ∗ t to denote σ ∗ t ( λ t , q t , s t ) .
Basic assumption We also need the following assumption to guarantee the tractability of the proposed problem.
In reality, consumers always have finite attention span, which indicates that q t is always smaller than 1 .
Assumption 3.1.
There exists a known constant ϵ Q > 0 such that for all t > 0 , q t ≤ 1 − ϵ Q .
Optimal Ranking Policy Given Consumers’ Characteristics
In this section, we introduce the optimal ranking policy when the online retailer exactly knows the consumers’ parameters, including the purchase probability for each product λ , random attention span (parametrized with q ), and random purchase budget (parametrized with s ).
We drop the subscript t in this section for simplicity.
The optimal ranking strategy stems from the following theorem.
Theorem 4.1.
Under Assumption 3.1, the following holds for the optimal ranking strategy σ ∗ ( λ , q, s ) .
[Equation]
Remark.
The expected revenue of a single product σ ∗ i is λ σ ∗ i r σ ∗ i , which grows as the purchase probability and the revenue generated by the product increase.
Yet purchasing a product reduces the purchase probability of the following products, and thus the expected revenue of other products decreases as λ σ ∗ i increases.
Furthermore, when s → 0 , the term in Equation (5) becomes the same as that in [13, Theorem 1] if ignoring the marketing fatigue ( i.e. , overexposure to unwanted marketing messages) term in their result.
As a result, Theorem 4.1 consistently extends the results on the single-purchase setting [13] to the multiple-purchase setting.
In addition, Liang et al.
[40] assume that the online retailer will stop recommending products with a certain probability 1 − w t to avoid the cost brought by marketing fatigue.
As a result, the parameter w t is a part of the ranking policy instead of consumers’ characteristics and should be optimized to achieve maximal revenue.
In our paper, we focus on the multiple purchases with budget setting and leave the extension to the case with the marketing fatigue as future work.
According to Theorem 4.1, the online retailer can provide the optimal ranking policy σ ∗ by sorting the products in descending order with value ( λ k r k ) / (1 − q + q (1 − s ) λ k ) for the k -th product.
The time complexity is O
( N log N ) .
Online Learning of the Ranking Policy
In this section, we consider a more realistic scenario where the online retailer has no prior knowledge about consumers’ characteristics.
At each timestamp, a consumer arrives and the online retailer can only provide proper product ranking policies based on the feedback of previous consumers.
As a result, the retailer needs to design online learning algorithms to model consumers’ behaviors and maximize revenue in the meantime.
We consider two settings ( i.e. , the non-contextual setting where all consumers share the same parameters (Section 5.1) and the contextual setting where consumers have personalized behaviors (Section 5.2)) and develop the Multiple-Purchase-with-Budget UCB (MPB-UCB) algorithms on both settings.
Before introducing the details of the algorithms, we first formally define the notations to characterize the consumers’ behaviors.
Notations for consumers’ behaviors Let
Φ t ∈ Z
+ be the random variable that denotes the number of products viewed by the t -th consumer.
Let random variable γ t,k ∈ { 0 , 1 } denote whether the consumer purchases the product displayed in the k -th position ( i.e. , product σ t,k ) if she views the product.
Specifically, γ t,k = 1 if
and only if the product is purchased.
Let γ t = { γ t,k } Φ t k =1 .
Let random variable η t,k ∈ { 0 , 1 } denote whether the consumer continues viewing products if she views the product displayed in the k -th position but does not buy it.
η t,k is observed only when k ≤ Φ t and γ t,k = 0 .
Let random variable µ t,k ∈ { 0 , 1 } denote whether the consumer keeps viewing products if she buys the product in the k -th position.
µ t,k is observed only when k ≤ Φ t and γ t,k = 1 .
Let η t = { η t,k } Φ t k =1 and µ t = { µ t,k } Φ t k =1 .
Let Ber( · ) denote the Bernoulli distribution and we have the following result.
Lemma 5.1.
{ γ t,k } t> 0 , 1 ≤ k<N are independent of each other, { η t,k } t> 0 , 1 ≤ k<N are independent of each other, and { µ t,k } t> 0 , 1 ≤ k<N are independent of each other.
In addition, ∀ t > 0 and 1 ≤ k < N , γ t,k ∼ Ber (cid:0) λ σ t,k (cid:1) , η t,k ∼ Ber( q t ) , and µ t,k ∼ Ber( q t s t ) .
Lemma 5.1 is the basis of the algorithms that estimate parameters λ , q , and s from historical data.
In addition, compared with s t , it is easier to estimate q t s t from data and we denote it as w t ≜ q t s t .
5.1 Non-contextual Online Setting
In this subsection, we assume that all consumers share the same parameters, including the purchase probabilities on different products and the parameters on the distribution of the attention span and purchase budget.
In detail, we assume that there exist λ , q, s such that λ t = λ , q t
=
q , and s t
= s .
A) Estimation of Parameters
To estimate the parameters at timestamp t , we leverage the observed behavior of consumers before timestamp t .
We first calculate the following statistics that can help provide unbiased estimators of the consumer parameters.
In detail, let C t,k be the number of times that the product k is observed and c t,k be the number of times the product k is purchased, i.e. , C t,k ≜ (cid:80) tu =1 (cid:80) Φ t i
=
I [ σ t,i = k ] and c t,k ≜ (cid:80) tu =1 (cid:80) Φ t i =
I [ σ t,i =
k ] I [ γ t,i = 1] .
In addition, Let D Qt be the number of times that consumers view a product and do not purchase it and d Qt be the number of times that consumers continue viewing the list after viewing a product without purchasing it, i.e. , D Qt ≜ (cid:80) tu =1 (cid:80)
Φ t i =
I [ γ t,i = 0] and d Qt ≜ (cid:80) tu =1 (cid:80) Φ t i =
I [ γ t,i = 0] η t,i .
Furthermore, let D Wt be the number of times that consumers purchase a product and d Wt be the number of times that consumers continue viewing the list after purchasing a product
, i.e. , D Wt ≜ (cid:80) tu =1 (cid:80) Φ t i
=
I [ γ t,i = 1] and d Wt ≜ (cid:80) tu =1 (cid:80) Φ t i =
I [ γ t,i = 1] µ t,i .
With these statistics, the parameters λ , q , and w = qs are estimated as follows.
[Equation]
We provide the following proposition that gives the error bound of our parameter estimation approach.
Proposition 5.2.
For all t ∈ [ T ] , with probability at least 1 − 6 Nt − 3 , the following holds:
[Equation]
B) Algorithm
Following the classic optimism in the face of uncertainty principle [1] [Equation] we design a UCB-like algorithm that learns consumer behaviors and maximizes revenue simultaneously.
In detail, according to Proposition 5.2, we learn the optimistic estimators of the parameters given as follows
[Equation]
Let σ t be the optimal offline policy from Theorem 4.1 with λ = ˜ λ
t − 1 , q = ˜ q t − 1 , and s = ˜
w t − 1 / ˜
q t − 1 .
t t t , η t , t .
As shown in Algorithm 1, when the t -th consumer arrives, the online retailer displays the products based on Theorem 4.1 with ˜ λ
t − 1 , ˜ q t − 1 , and ˜ w t − 1 / ˜
q t − 1 shown in Line 4.
Afterward, the online retailer updates the statistics according to consumers’ feedback as shown in Lines 5 and 6.
The estimators are then updated in Line 7.
C) Regret Analysis
To theoretically analyze the performance of Algorithm 1, we first introduce the regret, which measures the total difference between the maximal revenue achieved by σ ∗ and the cumulative reward by an online algorithm after T rounds.
[Equation]
Reg T is a random variable and the randomness comes from the uncertainty in consumers’ behaviors.
As a result, we focus on E [Reg T ] , which is a common routine in bandit literature [46].
The performance of the algorithm is given by the following theorem.
Theorem 5.3.
Under Assumption 3.1 (with parameter ϵ Q ), for any T > 1 , the regret achieved by Algorithm 1 is bounded by
[Equation]
(10) where C is a absolute constant (independent of problem parameters) and r max = max k ∈
[ N ] r k .
Remark.
We analyze different parameters’ impacts on the regret bound in Equation (10).
Firstly, the regret grows at ˜ O ( √ T ) , which is a standard result in online learning algorithms [46].
Secondly, the regret bound is linearly related to N , which is similar to the results in cascade bandit models [54] and revenue management literature [17].
Finally, the bound depends on r max /ϵ Q , which measures the maximal expected revenue the online retailer could achieve from each consumer.
5.2 Contextual Online Setting
We consider a more realistic scenario where the consumers’ behaviors are various and depend on their own features.
In detail, let x t ∈ R m x be the feature of the t -th consumer with dimension m x and y t,k ∈ R
m
y be the joint feature of the t -th consumer and k -th product with dimension m y .
y t,k can be the concatenation of consumer features and product features or fusion through a transformation such as a matrix multiplication.
In this paper, we adopt concatenation for simplicity.
Let ∥ · ∥ denote the ℓnorm.
Following [17], we consider the classic linear bandit setting [7, 21, 39] and the ground-truth data-generating process is given by the following assumptions.
Assumption 5.1.
There exist constant vectors β Λ ∈ R
m
y , β Q ∈ R m x , and β S ∈ R m x , such that
[Equation]
Assumption 5.2.
For any t > 0 and k ∈ [ N ] , ∥ x t ∥ ≤ 1 and ∥ y t,k ∥ ≤ 1 .
In addition, there exists a constant U > 0 such that ∥ β Λ ∥ , ∥ β Q ∥ , ∥ β S ∥ ≤ U .
Remark.
These two assumptions are common in revenue maximization and linear bandits [1, 46].
Assumption 5.1 can be satisfied with kernel functions for complex data-generating processes and
Assumption 5.2 can be achieved by normalization.
Thus the assumptions are rational and easily attainable in real applications.
A) Estimation of Parameters
For the estimation of β Λ and β Q , we consider the ridge regression method.
Given the observation of consumer’s behaviors till the t -th timestamp, the estimation ˆ β Λ t , ˆ β Qt is given by
[Equation]
Here Σ Λ t = (cid:80) tu =1 (cid:80)
Φ u
[Equation]
=1 y u,
[Equation]
y ⊤
[Equation]
+ α Λ
[Equation] [Equation]
Λ
[Equation]
=
[Equation]
tu
[Equation]
Φ
[Equation]
y
[Equation]
u
[Equation]
k ,
[Equation]
(cid:80) tu =1 (cid:80)
Φ u k =
I
[ γ u,k = 0] x u x ⊤ u + α Q
I m x , and ρ Qt = (cid:80) tu
=1 (cid:80)
Φ u k =
I
[ γ u,k = 0] η u,k x u .
I m x
and I m y are the identity matrices with dimension m x and m y , respectively.
For the estimation of β S , similar to Section 5.1, we focus on regressing w t =
q t s t instead.
Note that w t = q
[Equation]
s
[Equation]
=
[Equation]
Q (
[Equation] [Equation]
x
[Equation]
= vec( x
[Equation]
x ⊤ t ) ⊤ vec( β Q ( β S ) ⊤ ) , where vec( · ) denotes the vectorization operation that maps a matrix to a vector.
Therefore, define z t ≜ vec( x t x ⊤ t ) and β W ≜ vec( β Q ( β S ) ⊤ ) and we can get w t =
z ⊤ t β W .
Similarly, we estimate β W by conducting ridge regression as follows
[Equation]
Here
[Equation]
I m 2 x is the identity matrix with dimension m 2 x .
Let ∥ β
∥
A = (cid:112) β ⊤ Aβ for any positive definite matrix A .
Proposition 5.4 gives the estimation error bound of the coefficients.
Proposition 5.4.
Under Assumptions 5.1 and 5.2 (with parameter U ), for any t ≥ 1 , with probability at least 1 − 3 t − 2 , the following holds:
[Equation]
Here τ ( t, m, α ) = 1 / 2 (cid:112) m log (1 + tN/ ( mα ))
+ 4 log t + α 1 /
( U + 1) 2 .
B) Algorithm
Given the estimation of β Λ , β Q and β W we estimate λ t,k , q t and w t as follows.
[Equation]
Here Proj [ a,b ] ( · ) is a function that projects the input into the interval [ a, b ] .
As shown in Algorithm 2, when the t -th consumer arrives, the online retailer observes the consumer features x t and y t,k for k ∈ [ N ] and calculates the intermediate feature z t .
Afterward, he calculates ˜ λ
t,k , ˜ q t , and ˜ w t according to Equation (16) in Line 5.
Then the ranking policy σ t is given as shown in Line 6 and the statistics are updated in Lines 7 and 8.
Finally, the parameters ˆ β Λ t , ˆ β Qt , and ˆ β Wt are estimated in Line 9 for the next iteration.
C) Regret Analysis
We analyze the performance of Algorithm 2 by investigating its regret bound.
For a sequence of T consumers with features X T = { x t } Tt =1 and Y T = {{ y t,k } Nk =1 } Tt =1 , the regret is given by
Reg T ( X T , Y T ) =
T (cid:88) t =
Revenue
[Equation]
Similar to the non-contextual setting, Reg T ( X T , Y T ) is a random variable and we focus on E [Reg T ( X T , Y T )] .
The performance of the algorithm is given by the following theorem.
Theorem 5.5.
Under Assumptions 3.1 (with parameter ϵ Q ), 5.1, and 5.2 (with parameter U ), for any
T > 1 , the regret achieved by Algorithm 2 is bounded by
[Equation] [Equation]
Here C is a absolute constant (independent of problem parameters), r max = max k ∈
[ N ] r k , and χ ( T, m, α ) =
[Equation]
Remark.
We analyze different parameters’ impacts on the regret bound in Equation (19).
Firstly, the regret grows at ˜ O ( √ T ) , a standard result in online learning algorithms [46].
Secondly, the regret depends on ˜ O
( N ) , which shares a similar result with the revenue management literature [17].
Thirdly, similar to Theorem 5.3, the bound depends linearly on r max /ϵ Q that measures the maximal expected revenue the online retailer could achieve from each consumer.
Fourthly, the bound depends on the dimensions of features ( i.e. , m x and m y ) by ˜ O ( √ m y ) and ˜ O
( m x ) because the feature dimension for the estimation of w t =
q t s t is m 2 x
and then the result is consistent with [17, 54].
Finally, the bound depends on O ( U 2 + U ) since ∥ β W ∥ ≤ U 2 and ∥ β Q ∥ , ∥ β Λ ∥ ≤ U , which agrees with [17].
Experiments
In this section, we conduct experiments on synthetic data to verify the performances of the online ranking policies.
Reports and discussions of more results including the expected revenue and the ratio over the expected revenue of the optimal ranking policy are in Section B.
Baselines We adopt the following baselines.
Firstly, Cao and Sun [13] considered the singlepurchase setting, which is the simplified version of our multiple-purchase setting.
We denote it as
Single Purchase .
Secondly, we adopt the method in [40] (denoted as Keep Viewing ), which assumes that consumers always keep viewing the ranking list after purchasing products.
Furthermore, we implement two explore-then-exploit-based algorithms (denoted as Explore Then Exploit A and
Explore Then Exploit B , respectively) to verify the advantages of our method that could balance exploration and exploitation.
6.1 Synthetic Data
Data-generating processes For the non-contextual setting
, we consider N = 50 and 300 products and T = 100 , 000 consumers.
The revenue for each product r k is uniformly distributed in [0 , 1] and the purchase probabilities for each product are uniformly sampled from [0 , 0 . 3] .
We set q = 0 .
9 and s = 0 .
5 and 0 .
8 to test the performance of different models on characterizing consumers’ behaviors with multiple purchases.
[Figure]
For the contextual setting
[Figure]
we consider N = 50 and 300 products and T = 100 , 000 consumers.
The revenue r k is also uniformly distributed in [0 , 1] .
Both the dimension of consumer features and product features are set to 5 ( i.e. , m x = 5 ).
The joint feature y t,k (the feature of the t -th consumer and k -th product) is a concatenate of the corresponding consumer and product feature so that m y
=
The consumer features and product features are uniformly sampled from [0 . 8 / √ m x , 1 / √ m x ] and [0 , 1 / √ m x ] .
Afterward, the coefficients β Λ , β Q , and β S in Equation (11) are uniformly sampled from [0 , 1] .
To ensure λ t,k , q t , s t have similar ranges in the contextual setting, we normalize the coefficients such that the maximal elements in λ t,k and q t (denoted as λ max and q max ) are 0 .
3 and 0 . 9 , respectively.
Similarly, by normalizing β S , we set the maximal element in s t (denoted as s max ) to 0 .
5 and 0 . 8 .
Results and analysis
We implement the experiments for 5 distinct simulations by resampling consumers’ behaviors while their basic characteristics remain unchanged.
Specifically, in both the non-contextual and contextual settings, the purchase probabilities ( i.e. , λ t ) and parameters on the span attention and purchase budget ( i.e. , q t and s t ) are the same.
However, the same consumer in distinct simulations may purchase different products due to randomness.
We then evaluate our method (MPB-UCB) and baselines via calculating the regret according to
Equations (9) and (17).
The results are shown in Figure 1 and our method outperforms all baselines in both settings with various parameters.
On the one hand, the Single Purchase and Keep Viewing baselines consider different consumer choice models and are not directly applicable here to the more realistic multiple-purchase setting.
On the other hand, compared with the explore-then-exploit-based algorithms, our method integrates the advantages of traditional UCB-like algorithms, leading to a better exploration-exploitation trade-off for smaller regret.
In addition, as introduced in Section B,
Explore Then Exploit B incorporates learning processes during the exploration phase, leading to better performance compared with Explore Then Exploit A.
6.2 Semi-synthetic Data
We utilize the Ad Display/Click Data from Taobao 1 , which contains ad display/purchase logs ( million records) of 1,140,000 anonymized users from the website of Taobao for 8 days.
Because the behavior logs only contain the purchase information of the users on a category of a brand, we view a (brand, category) pair in the dataset as a product and the average prices of the ads in this pair as the revenue for the product.
Because we can not obtain consumers’ behaviors when we offer a new product ranking policy, following [14], we first estimate the parameters ( i.e. , λ , q , s in the non-contextual setting and β Λ , β Q , β S in the contextual setting) with all of the data.
Afterward, we use the estimated parameters to simulate consumers’ behaviors when we provide them with different ranking lists of products.
[Figure]
Data processing Since each consumer may launch and leave the platform many times during thedays, we need to distinguish different launching activities of the same consumer.
Specifically, we suppose that a consumer stops viewing the product list if it has been more than 10 minutes since her last interaction with the platform.
Afterward, we can estimate the parameters in both non-contextual and contextual settings.
In the non-contextual setting, we calculate the statistics C t,k , c t,k , D Q t , d Q t , D Wt , and d Wt from the data and estimate λ , q , s directly according to Lemma 5.1.
The estimation is q = 0 .
870 and s = 0 .
Afterward, we sample N = 50 and 300 products with prices no more than 200 and purchase probabilities at least 0 .
Similar to the synthetic data, we set T = 100 , 000 .
In the contextual setting, we set the gender, age level, consumption grade, and shopping level as consumers’ features and the number of ads, the logarithm of the average, standard deviation, maximum, and median of the ads’ prices in the (brand, category) pair as products’ features.
With the statistics C t,k , c t,k , D Qt , d Qt , D Wt , and d Wt , we use the ridge regression with regularization strengthto estimate β Λ , β Q , and β S .
Afterward, we sample N = 50 and 300 products with prices no more than 200 and purchase probabilities at least 0 .
In addition, we sample T = 100 , 000 consumers.
Results and analysis Similar to the synthetic experiments, we implement the experiments fordistinct simulations by resampling consumers’ behaviors while their basic characteristics remain unchanged.
The results of the regret are shown in Figure 2.
As shown in the figure, our method outperforms all baselines in different scenarios.
Firstly, the Single Purchase and Keep Viewing baselines are not directly applicable here since they consider different consumer choice models.
It is worth mentioning that the estimated s is large from the dataset.
As a result, the consumers prefer to keep viewing products if they purchase any product, making the Keep Viewing baseline outperforms
Single Purchase in this setting.
Secondly, our method combines the benefits of existing UCB-like algorithms, resulting in a superior exploration-exploitation trade-off with less regret compared with explore-then-exploit-based methods.
In addition, similar to the analysis in the synthetic experiments,
Explore Then Exploit B outperforms Explore Then Exploit A in most cases.
Conclusion
To conclude, we propose a novel consumer choice model to deal with multiple-purchase activities of consumers in online scenarios.
We characterize the optimal ranking policy and further design the
MPB-UCB algorithms with ˜ O ( √ T ) regret in both non-contextual and contextual online settings.
We conduct extensive experiments to prove the effectiveness of our method.
Acknowledgments
Peng Cui’s research was supported in part by National Key R&D Program of China (No.
2018AAA0102004, No. 2020AAA0106300), National Natural Science Foundation of China (No.
U1936219, 62141607), Beijing Academy of Artificial Intelligence (BAAI).
Bo Li’s research was supported by the National Natural Science Foundation of China (No.72171131), the Tsinghua University Initiative Scientific Research Grant (No. 2019THZWJC11), Technology and Innovation Major
Project of the Ministry of Science and Technology of China under Grants 2020AAA0108400 and