Discussion Paper No. 35

Topic Modeling of Market Responses forLarge-Scale Transaction Data
Tsukasa Ishigaki
Nobuhiko Terui
Tadahiko Sato
Greg M. Allenby

February 23, 2015

Data Science and Service Research
Discussion Paper

Center for Data Science and Service Research
Graduate School of Economic and Management
Tohoku University27-1 Kawauchi, Aobaku
Sendai 980-8576, JAPANTopic Modeling of Market Responses forLarge-Scale Transaction Databy

Tsukasa Ishigaki
Tohoku University
Nobuhiko Terui
Tohoku University
Tadahiko Sato
University of Tsukuba
Greg M. Allenby
Ohio State University

February 23, 2015

Topic Modeling of Market Responses forLarge-Scale Transaction Data

Abstract
Large-scale databases in marketing track multiple consumers across multiple productcategories. A challenge in modeling these data is the resulting size of the data cube,
which often has thousands of consumers and thousands of choice alternatives with prices
and merchandising variables changing over time. We develop a heterogeneous topic modelfor these data, and employ variational Bayes techniques for estimation that are shown tobe accurate in a simulation study. We find the model to be highly scalable and useful for
identifying eﬀective marketing variables for consumers, including infrequent purchasers.

Keywords: Cross-category Analysis, Data Cube, Hierarchical Bayes Moldel, Market
Response, Panel Data, Personalization, Topic Model, Variational Bayes Inference† This paper is a revised version of a discussion paper by T. Ishigaki, N. Terui, T.
Sato and G. Allenby, A Large-Scale Marketing Model using Variational Bayes Inference
for Sparse Transaction Data, Data Science and Service Research Discussion Paper, No.
18, 20141

1Introduction

Modern analytic techniques in marketing are continuously confronted with the necessityof extracting relevant information from large volumes of data by identifying importantdrivers of consumer behavior. It is common for datasets to record household purchasesof products that are orders of magnitude larger than what current models of behavior
are currently capable. Existing models of choice and demand, for example, are typicallylimited to less than twenty or so product alternatives that are tracked across possibly
hundreds of consumers (see Rossi et al. 2005, Chintagunta and Nair 2011).
Increasing the number of products analyzed is problematic because of potential complexities in the structure of demand and the accompanying increase in the required number of model parameters. Increasing the number of respondents is also problematic because of computational constraints arising from respondent heterogeneity that is foundto be important in describing demand and deriving policy implications. While a variety of dimension-reducing techniques have been studied in the fields of statistics anddata-mining, the presence of heterogeneous consumers and heterogeneous purchase environments with prices and other variables changing over occasions requires the use ofmodel-based inference as opposed to methods applied directly to the marginal data (Anderson 2003).
Naik et al. (2008) discusses three solutions to the challenges in massive data analysis:
increasing computer power, employing alternative approaches for data analysis, and using
scalable estimation methods. In this paper, we combine the second and third options to
obtain improved inferences about consumer behavior in large datasets. Thus, instead ofattempting to build an economic model of choice across dozens of product categories,
explicitly modeling the presence of substitutes, complements and an inter-related set ofbudget constraints, we extend the voting bloc model of Spirling and Quinn (2010) and
Grimmer (2011) that are a variation of topic models used to conduct large-scale analysis

2

of text data (Blei et al. 2003). These models make the simplifying assumption that votes,
words or purchases are outcomes of latent probabilities that describe the occurrence of
events. We extend these models so that the latent probabilities are a function of a brand’s
own marketing variables.
The topic model is a generalization of a finite mixture model in which each data point
is associated with a draw from a mixing distribution (The and Jordon 2010). Models
of voting blocs (Spirling and Quinn 2010) track the votes of legislators (aye or nay)
across multiple bills, with each bill associated with a potentially diﬀerent concern or issue.
Similarly, the latent Dirichlet allocation (LDA) model of Blei et al. (2003) allocates words
within documents to a small number of latent topics whose patterns are meaningful andinterpretable. Each vote and each word is associated with a potentially diﬀerent issue or
topic, and hence the mixing distribution is applied to the individual vector of observations
and not to the entire set of observations (e.g., series of votes a legislator or set of words
by an author) of the panelist. In our analysis of household purchases, we allow the vectorof observed purchases across all product categories on an occasion to be related to adiﬀerent latent context (topic, or issue). This allows us to view a consumer’s purchases as
responding to diﬀerent needs or occasions (e.g., family dinner, snacks, etc.), and allows us
to identify the ensemble of goods that collectively define latent purchase segments acrossa large number of products.
We obtain a scalable estimation method by employing variational Bayes (VB) inference
as in Jordan et al. (1999) and Bishop (2006), instead of the standard Markov chain
Monte Carlo (MCMC) inference. MCMC methods can incur large computational costin large-scale problems. VB inference approximates a posterior distribution of target byvariational optimization in a computationally eﬃcient manner. Our approach combines
variational Bayes (VB) methods, as in Jordan et al. (1999) and Bishop (2006), with a
topic-like probit model to obtain a computationally feasible model of consumer purchasesthat is scalable to large databases. Individual-level inference is possible in our model,3where we can identify the marketing variables that are eﬀective for specific individuals
and the products for which they are eﬀective. Our model is therefore similar to adaptive
personalization systems proposed by Ansari and Mela (2003), Rust and Chung (2006),
Chung et al. (2009) and Braun and McAuliﬀe (2010). However, it is diﬀerent in that
our model structure facilitates analysis of a much larger array (i.e., at least an order ofmagnitude) of oﬀerings across multiple product categories.
Our model identifies the latent state, or topic assignment, for each consumer at eachpoint in time, providing information about the array of products a consumer will likely
purchase. We do not make a-priori assumptions about substitute and complementary
goods in the spirit of market basket analysis in data mining. Our model takes an exploratory approach to analysis and does not test assumptions of the form of the utility
function across hundreds of oﬀerings. However, our model does include marketing variables so that their eﬀects on choice can be measured and used in prediction.
In the next section, we propose a model for consumer purchases in multiple productcategories. Section 3 describes a variational Bayes inference scheme for the model andsimulation studies that verify the precision of VB estimate and scalability of the model.
In section 4, we first discuss the joint segmentation of consumers and items for crosscategory analysis, and propose a method of decompressing the information obtained in the
reduced-dimensional space to make marketing decisions in the original large-scale original
space. Section 5 applies the model to customer purchases in a general merchandise store.
Discussion and concluding remarks are oﬀered in Section 6.

2

Model Development

The analysis of large-scale transactional data is challenging because of the sparsity ofobserved purchases. Most consumers do not purchase in most product categories onmost shopping trips, and when a purchase is recorded in one category it is frequently

4for just one oﬀering. The actual sample size of transactional data is much smaller thanthe data space reflected by a data cube with dimensions corresponding to the number ofconsumers, number of products and time. In this situation, standard random-eﬀect model
specifications break down because of the high frequency of non-purchase for almost every
brand.
In choice models, maximum likelihood estimates of brand intercepts are driven to negative infinity if the brand is never purchased by the household, and since most households
do not ever purchase most brands, standard random-eﬀect models result in excessive negative shrinkage of the intercepts. Similarly, the prevalence of non-purchase makes it appear
that consumers are not price sensitive because they do not react to competitive price discounts, when in fact they may be making a quick trip to the store and may not evenbe exposed to prices in many categories. The analysis of large-scale transactional data
must therefore employ additional assumptions about heterogeneity and price responsiveness not typically made in the analysis of revealed preference data. We relate consumerpurchases to latent segments as is done in models for text analysis that greatly reduces
the dimensionality of the model. Response parameters are then introduced in the reduced
dimensional space by connecting each choice to their own marketing variables with a hierarchical probit model. We do not attempt to model cross-price and cross-merchandising
eﬀects because of the large number of brands under study.

2.1

Dimensional Reduction by Topic Models

Dimensional reduction is an important technique in massive data analysis. Here we briefly
introduce the idea of introducing a latent variable that is common in topic models in thecontext of consumer purchases. We seek the probability p(i|c) that consumer c purchases item i. We assume the dataset includes C consumers and I product items throughT periods. However, the probabilities cannot be accurately calculated because of data
sparseness. The topic model calculates p(i|c) by introducing a latent class z ∈ {1 . . . Z}

5

whose dimension is significantly smaller than the number of consumers and items.
The latent variable is used to represent the sparse data matrix as a finite mixture ofvectors commonly found in topic models:








p (1|z) [p (i = 1|c = 1) · · · p (i = 1|c = C)
]
Z 
∑




.
.
.
.
=
 ..  p (z|1) · · · p (z|C)

..
..
..





 z=1 p (I|z)p (i = I|c = 1) · · · p (i = I|c = C)

(1)

More specifically, we decompose a large probability matrix of size I × C to two small
probability matrices of sizes I × Z and Z × C based on the property of conditional
independence. The main diﬀerence between voting blocs model and LDA is assumed
distributions for probabilities p(i|c) in the I × Z matrix. The voting blocs model supposes a Bernoulli distribution for the probability p(i|c). LDA assumes a categorical (i.e.,
multinomial) distribution for the probability matrix.
The latent classes z serve to define types of purchase baskets across the I products.
The first term on the right side of (1) defines a vector of choice probabilities for each item
under study, assuming that the purchase occasion is of type z. Items with high probability
are likely to be jointly present in the basket, so our model identifies likely bundles of goods
purchased for diﬀerent types of shopping trips. The second term is the probability thata consumer’s purchases are of type z. Our model does not model heterogeneity in a
traditional manner, where there is a common set of response parameters for all purchasesof an individual. We instead assume that each purchase belongs to one of Z types, andthat respondents can also be characterized in terms of the probability their purchases areof these types.
In the analysis of purchase behavior using topic models for large consumer transaction
data, Iwata et al. (2009) extracted dynamic patterns between purchased product items
and consumer interests. Ishigaki et al. (2010) fused heterogeneous transaction data andconsumer lifestyle questionnaire data, while Iwata et al. (2012) identified consumer pur-6

chase patterns by using a topic model with price information on the purchased products.
These approaches identify patterns among consumers and product items. The labeled
LDA proposed by Ramage et al. (2009), and the supervised LDA of Blei and McAuliﬀe(2007) extend the topic models by incorporating additional data in the analysis. However,none of these approaches are suitable for relating marketing variables to individual consumer choices as explanation variables. In the following sections, we construct a modelthat links marketing variables with consumers and products.

2.2

A Reduced Dimensional Market Response Model

Let ycit denote consumer c’s purchase record of product i at time t, assigning ycit = 1
if consumer c purchased the item, and ycit = 0 otherwise. Denote ucit as the utility of
consumer c’s purchase record of product i at time t. We assume a binary probit modelwith ucit > 0 if ycit = 1, and ucit ≤ 0 if ycit = 0. We couple the topic model in (1) withthe binary choice probability as in a voting bloc model to obtain the choice probability:p (ucit > 0) =Z∑p (uit > 0|z) p (z|c)

(2)

z=1
(z)

We denote the utility associated with the latent class z as uit , and then the choice(z)probability can be represented as p(uit > 0|z) = p(uit > 0). Assuming a linear Gaussian(z)

structure on the utility uit for marketing variables, the right hand side of (1) can be
represented as:)( T
F xit β z1 
Z 
∑


..
 [p (z|1) · · · p (z|C)]

.



z=1
( T
)
F xIt β zI(3)where β Tzi = (βzi1 , . . . , βziM ) is a response coeﬃcient vector of latent class z with respect toitem i, xTit = (xit1 , . . . , xitM ) is a vector of M marketing variable for item i at time t, and7F () is the cumulative distribution function (CDF) of the standard normal distribution.
In our empirical study, xit includes price and promotional variables.
We next set a categorical distribution θcz for the probability p(z|c) that consumer cbelongs to the latent class z. The categorical distribution is multinomial with parametersθ c . The θ c is specified so that the selection probability of consumer c with respect to item
i is conditionally independent if the latent class z is given. That is, all of the information
about respondent heterogeneity of purchases is conveyed through the latent classes. Then,the right hand side of (1) is represented by:



( T
)
F xit β z1 
Z 
∑


..

 [θ1z · · · θCz ].

z=1  ()
F xTIt β zI

(4)

Finally, segment-level heterogeneity is introduced through a hierarchical model witha random eﬀect for response coeﬃcient β zi :β zi ∼ NM (µi , Vi )(5)

where the prior distributions for µi and Vi follow an M-dimensional multivariable normal
distribution NM (µ̃, σ̃ 2 Vi ) and an inverse-Wishart distribution IW (W̃ ,w̃), where µ̃, σ̃ 2 ,W̃ and w̃ are parameters specified by the analyst. We assume that the M -dimensionalcoeﬃcient vector β zi for each segment, z, is a draw from a distribution with mean andcovariance that is item-specific.
We specify a prior distribution for θ c , assuming the Dirichlet distribution as the natural
conjugate prior distribution of categorical distribution:θ c ∼ Dirichlet (γ̃)
The likelihood is given as:

8(6)ℓ ({ycit } | {θ c } , {β zi } , {xit }) =C ∏ ∏∑
Z∏[θcz p (ycit |xit , β zi , z)](7)c=1 i∈Ic t∈Tc z=1

where p(ycit |xit , βzi , z) denotes the kernel of the binary probit model conditional on z, Tc
denotes a subset of t in which consumer c purchased any item in a store, and Ic is a subset
of items i purchased by consumer c at least once during the period t = 1, . . . , T , that is,
{ I
}
{ T
}
∑
∑
Tc ∈ t| ycit > 0 and Ic ∈ i| ycit > 0 .
i=1

t=1

Equation (7) is diﬃcult to use directly because the likelihood includes summations overlatent class z. Instead, we employ a data augmentation approach by Tanner (1987) withrespect to latent variable z. We introduce variables zcit ∈ {1, . . . , z . . . , Z} denoting thelabel of the latent class for each consumer c, each purchased item i, and each purchasing
event t. Conditioning on the zcit for each purchasing transaction, as in the LDA of Blei
et al. (2003), the likelihood in (7) simplifies to:ℓ ({ycit } | {θ c } , {zcit } , {β zi } , {xit }) =

C ∏ ∏
∏

p (zcit = z|θ c ) p (ycit |xit , β zi , zcit = z)c=1 i∈Ic t∈Tc

(8)where p(zcit = z|θ c ) denotes a categorical distribution when θ c is given. Hereinafter,(zcit = z) is denoted as zcit to simplify notation.
Our model for large-scale transaction data is diﬀerent from related standard models
in two respects. First, the likelihood is defined over brands and time periods in whichpurchases are observed to take place at least once as indicated by the variables Tc and
Ic . It is composed of not only purchase but also non-purchase occations for identifying
market response parameter. In this sense, our model diﬀers from topic models used intext analysis where the likelihood is formed using the words present in a corpus, not thewords that are not present. Second, heterogeneity is introduced at the observation-level,
allowing the diﬀerent transactions of a household to reflect diﬀerent latent states, z at9

every (c, i, t), which is denoted by zcit . It provides us with useful information for characterizing respondents and brands, and predicting their purchases. This diﬀers from thetraditional latent class model where the likelihood of all household purchases contributes
to inferences about a respondent’s latent class membership (z) and parameters (β).
The posterior distribution of parameters including latent variables of states {zcit } and(z)augmented utilities {ucit } of probit model is then given by:
(

{
}
)
(z)
p {θ c }, {zcit }, ucit , {β zi }, {µi }, {Vi } | {xit }, {ycit }= p ({θ c } | {zcit })× p ({zcit } | {θ c , β zi , xit , ycit })
)
({
}
(z)
× p ucit | {β zi , zcit , xit , ycit }× p ({µi , Vi } | {β zi })
(
{
})
(z)
× p {β zi } | ucit , µi , Vi , xit
(
{
}
)
(z)
∝ p {θ c }, {zcit }, ucit , {β zi }, {µi }, {Vi }, {xit }, {ycit }
[C
][ I
]
Z
∏
∏
∏
=
p (θ c )p (µi , Vi )p (β zi | µi , Vi )
c=1

[C
∏∏ ∏

i=1

z=1

(

)

(z)

]

p (zcit | θ c ) p ucit | β zi , zcit , xit , ycit p (ycit | β zi , zcit , xit )(9)c=1 i∈Ic t∈Tc

3

Variational Bayes Inference

We introduce VB inference in order to achieve computational feasibility for large-scaletransaction data. VB inference approximates the posterior, or target distribution in a
Bayesian model. The advantage of this method over MCMC is low computational cost.
VB also takes advantage of parameters that can be decomposed into several mutuallyindependent groups. This is necessary for our analysis using a large database.
The target and approximate distributions are denoted as p and q, respectively. The
10

latter is called the variational distribution. Distributions p and q share a parameter set
Θ. In general, when the data D is given, the log marginal likelihood log p(D) of thetarget distribution is decomposed into two components as:log p (D) = L (q) + KL (q ∥p)∫{}
L (q) = q (Θ) log p (D, Θ) q(Θ)−1 dZ∫{}
KL (q ∥p) = − q (Θ) log p (Θ|D) q(Θ)−1 dZ

(10)
(11)
(12)

L(q) is called variational lower bound in VB inference, and KL (q ∥p) is the KullbackLeibler divergence of the target and variational distributions. As is well known, KL (q ∥p)
is zero if p and q are the same distribution. Therefore, a reasonable solution to estimating
the posterior distribution p is the variational distribution q for which KL (q ∥p) is minimized. However, it is diﬃcult to evaluate the value of KL (q ∥p ) because the expression
involves a posterior distribution of p(Θ|D).
In contrast, L(q) involves a joint distribution p(D, Θ) that is easily evaluated in manycases because it is obtained as the product of the prior and the likelihood in Bayesianmodels. We note that maximizing L(q) is equivalent to minimizing KL (q ∥p ) because
the log marginal likelihood of the target distribution is constant for a given dataset. In
this situation, assuming that the distribution q and parameter set Θ are decomposable
(
)
J
∏
(j)∗
for some groups, the parameters are called variational parameters q (Θ) =
qj Θ
j=1

and can be maximized by the following updating algorithm (Jordan et al., 1999):
Θ(j)∗{new} ← arg max L
Θ(j)∗

(∏ J
j

(
))
qj Θ(j)∗∝ exp (E k̸=j [log p (D, Θ)]) .

(13)

The Ek̸=j [ ] are the expectation value associated with qj distributions over all pa∗

rameters Θ(j) , where k ̸= j. The variational parameters are updated for each variational

11

∗parameter set Θ(j) until convergence of the algorithm. The initial variational parameters
are proper random values. The VB is guaranteed to converge after several iterations be∗cause L(q) is convex with respect to each qj (Θ(j) ) (Bishop, 2006). The variational lower
bound monotonically increases as the iteration proceeds; therefore, convergence can be
confirmed by checking the value of L(q) at each iteration.

3.1

VB for the Proposed Model

We introduce the variational distributions and parameters for the proposed model. The
parameters and variational parameters are denoted as
{
{
}
}
(z)
Θ = {θ c }, {zcit }, ucit , {β zi }, {µi }, {Vi } and
{
{
}
{
}
}
(z)∗
Θ∗ = {θ ∗c } , {z ∗cit } , ucit , {β ∗zi } , Vizβ∗ , {µ∗i } , {σiµ∗ } , {wi∗ } , {Wi∗ } respectively, whilethe variational distributions are configured as
q (Θ | Θ∗ , {xit }, {ycit })
[C
][ C
][ C
]
)
∏
∏∏ ∏
∏ ∏ ∏ ( (z) (z)∗=qc (θ c | θ ∗c )qz (zcit | z ∗cit )qu ucit | ucit , z ∗cit , β ∗zi .xit
c=1

[ I Z
∏∏
i=1 z=1c=1 i∈Ic t∈Tc

(

][ I
) ∏
β∗

qβ β zi | β ∗zi , Vzi

c=1 i∈Ic t∈Tc

qµ,V (µi , Vi | µ∗i , σiµ∗ , wi∗ , Wi∗ )

]
(14)

i=1

where qc is a Dirichlet distribution with variational parameter θ ∗c , qz is a categorical
distribution with variational parameter z ∗cit , qu is a truncated normal distribution with(z)∗

parameter zcit and variational parameter ucit , qβ is an M -dimensional multivariable normal distribution with two variational parameters (mean vector β ∗zi and covariance matrix
Vziβ∗ ), and qµ,V is a multivariable normal–inverse Wishart distribution with variationalparameters µ∗i , σiµ∗ , wi∗ , Wi∗ . Here, to realize eﬀective variational inference, we assume
that all variational parameters are independent. The update equation and the derivationsof the variational parameters are detailed in Appendix A.

12

3.2

Simulation StudyIn this subsection we examine the performance of the proposed VB estimator. In addition
to computational time, VB has another advantage over MCMC in that it is not prone tothe label switching problem encountered in MCMC estimation (see Puolamaki and Kaski,2009).
We examine the precision of the estimates and computational time separately. The
first simulation evaluates the recovery of true parameter values by VB, and the second
simulation examines scalability. We compare the computational times of VB to MCMC,
ignoring label switching problem encountered with MCMC estimation. We show that
MCMC becomes too computationally demanding as the size of the dataset increases, andthat VB provides a computationally eﬃcient and accurate approximation to the posterior.
3.2.1

Simulation Dataset

In this simulation study, purchase records are generated by simulation using marketing
variables. The marketing variables are extracted from a real customer database of ageneral merchandise store. The marketing variables vector is composed of price (P̄it ),display (Dit ), and feature (Fit ); that is, xTit = [1 P̄it Dit Fit ] . P̄it is the discount rate to
the maximum price of item i in the observational period. Display and feature are binary
entries, equal to one if the item i is displayed or featured at time t, and zero otherwise.
Here, the value of P̄it is normalized into interval [0, 1] in order to conform the scale ofdiscount rate to the scale of dummy variables.
We assume that any customers belong to one of three segments characterized byresponse coeﬃcients for marketing variables. First segment (Segment 1) has a response
coeﬃcient β̄ 1 = [−0.5, 1, 0, 0]T , that is, customers in the segment sensitively respond to
discount of product items and are not aﬀected from display or feature. Similarly, we
employ β̄ 2 = [−0.5, 0, 1, 0]T and β̄ 3 = [−0.5, 0, 0, 1]T as response coeﬃcient vectors forsecond (Segment 2) and third segments (Segment 3) that are influenced from display and13

feature promotion only, respectively. The three vectors are set as true values of response
parameter. This setting means that any product items have the same properties on the
response to marketing promotions for a simplification of analysis. The verification orcheck of parameter estimation will be too complicated if we employ diﬀerent coeﬃcient
vector for each product items.
Next, we make coeﬃcient vectors of individual customers. Here, we suppose that eachsegment consists of 100 customers and 50 product items are in a store. The individual
coeﬃcients vectors ᾱci are generated by followings; ᾱci ∼ N (β̄ 1 , σI) (c = 1, · · · , 100),ᾱci ∼ N (β̄ 2 , σI) (c = 101, · · · , 200) and ᾱci ∼ N (β̄ 3 , σI) (c = 201, · · · , 300), and σ is set
as 0.1. Then, the utilities for 30 days are simulated by ūcit = xTit ᾱci + ϵ̄cit (ϵ̄cit ∼ N (0, 1))
and the purchased records {ȳcit } are generated as ȳcit = 1 if ūcit > 0 and ȳcit = 0 otherwise.
3.2.2

Precision of Estimates

In this subsection, we examine how well the parameters of β̄ 1 , β̄ 2 and β̄ 3 are recovered in
the proposed model with VB. Here, we generate ten simulation datasets by the procedures
above, and we set hyper-parameters as γ̃ = [0.1, · · · , 0.1]T , µ̃ = [0, · · · , 0]T , σ̃ 2 = 1,
W̃ = I M and w̃ = 10 and appropriate initial values. I M is the identity matrix of size M .
In VB estimation, the iterations are terminated when the variational lower bound improves
by less than 10−5 % of the current value in two consecutive iterations (the variational
lower bound is described in Appendix B). These settings for the hyper-parameters andthe stopping rule of the VB iterations are adopted in all empirical studies hereafter.
Table 1 displays the means and standard deviations of estimates using the ten simu∑
lation dataset. The numbers in Table 1 are calculated as 50−1 Ii=1 β̂ zi (β̂ zi represents aestimated posterior mean of β zi .). The results indicate that the VB estimates are close
to true values for all parameters in every segment.

Table 1: Estimates of Simulation Data

14

3.2.3

Scalability

Scalability is investigated for: C = {1000, 5000, 10000}, I = {100, 500, 1000}, T = 30
and Z = {5, 10, 20}. Thus, 27 diﬀerent scenarios were explored in the scalability study.
The MCMC estimator is described in Appendix C, and we forecast the simulation times
for 6,000 MCMC samples from ten samples for computational feasibility. We note thatthe selection of 6,000 MCMC samples is consistent with the simulation study of Braunand Mcauliﬀe (2010). The simulated data is the same as used in above, and the results
reported below were calculated in identical computational environment (64-bit version of
Python 2.7.5 with Numpy, implemented on a 3.5 GHz processor (Quad-Core Xeon; Intel
Corp.) with 64 GB memory).
Table 2 reports computation time in hours for the VB and MCMC estimators. For bothalgorithms, the computational cost increases linearly with the size of the dataset specified
in terms of the number of consumers, items, and latent classes. In all scenarios, the timesof MCMC computations exceed those of VB. The VB algorithm is approximately 20 to 50
times more eﬃcient than MCMC, depending on the scenario. The time of computation
using large-scale data (C = 10000, I = 1000) by MCMC is estimated to be over 450
hours, and thus we recognize that MCMC is not applicable for our problem. The resultsof the simulation show that VB estimates are reliable in precision and computationallyfeasible for analysis. In contrast, MCMC becomes increasingly prohibitive as the numberof customers and choice alternatives increases.

Table 2: Simulation Time by VB and MCMC

4

Joint Segmentation and Personalization
∗

(z)∗

(z)∗

The variational estimates β̂ zi , ûcit , θ̂c , and ẑcit can be transformed into statistics thatare relevant for segmentation and targeting using the M dimensional vector of probability
15

(z)∗

ẑcit , z =, 1, ..., Z at each point of data cube:
qz (zcit | ẑ ∗cit )
∗

(15)
(z)∗

Given the variational Bayes estimates Λ̂ = {β̂ zi , ûcit , θ̂c }, we obtain the probability ofcustomer segment membership by aggregating over products (i) and time (t):
(

)

∑

∑

(z)∗
t∈Tc ẑcit × I(ycit = 1)
∑ ∑
(z)∗
zk =1
i∈I
t∈Tc ẑcit × I(ycit = 1)

p c ∈ z|Λ̂ = ∑Z

i∈I

(16)

and aggregating over customers (c) and time (t) yields the probability of product segment
membership:(p i ∈ z|Λ̂

)

∑C ∑

(z)∗
t∈Tc ẑcit × I(ycit = 1)
= ∑Z ∑C ∑
(z)∗
zk =1
c
t∈Tc ẑcit × I(ycit = 1)
c=1

(17)

where I(·) is the indicator function equal to one if the augment holds and zero otherwise.
We take the sums over the instances of purchase because we believe that non-purchasecan occur for many reasons other than non-membership (e.g., having large household
inventory of the product). Our estimates of customer and product latent membership are
driven by customer actions and not their inactions.
We can also construct market response estimates for each respondent and each prod∗(z)∗

uct from Λ̂ = {β̂ zi , ûcit , θ̂c } by projecting the estimates of latent utility on marketing
variables. That is, the estimates are obtained from an auxiliary regression of latent util(k)∗ity Ûci

(k)∗

(z)∗

stacked by ûcit with the state k = argmax ẑcit changing over time on thecorresponding marketing variables Xci constituted by xit (t ∈ Tc ).
)−1
(
(k)∗
Xci T Ûci .
α̂ci = Xci T Xci(18)

The estimates above provide a bridge between the granularity of the model, where heterogeneity is introduced at each point in the data cube, and managerial inferences and16

decisions that are made across products (e.g., which customers to reward), across customers (e.g., which products to promote) and over time. In addition, the standard t testin the standard linear regression models can be used for testing significance of estimates.

5

Empirical Analysis

A customer database from a general merchandise store, recorded from April 1 to June 30
in 2002, is used in our analysis. A customer identifier, price, display, and feature variables were recorded for each purchase occasion. The dataset contains 94,297 transactions
involving 1,650 consumers and 500 items. The items were chosen by being displayed andfeatured at least once in the data period. The marketing variables are price (Pit ), display(Dit ), and feature (Fit ); that is, xTit = [1 Pit Dit Fit ] . Pit is the price relative to the
maximum price of item i in the observational period. The display and feature are binary
entries, equal to one if the item i is displayed or featured at time t, and zero otherwise.

5.1

Cross-category analysis

Our model of purchase behavior allows for observation-level heterogeneity that acknowledges that each purchase occasion can be viewed as the building-block for analysis. Someoccasions are associated with large trips to the store while other occasions may have be
more focused on a specific set of oﬀerings. Moreover, consumers may exhibit behavior
consistent with multiple occasions, or topics, over time. While it may be desirable forfirms to classify goods and respondents to segments for the purpose of understanding
diﬀerent types of customers and goods, our model is capable of conducting analysis at amore disaggregate level. Alternatively, our model can be used to associated both oﬀerings and customers to latent topics, or segments, for understanding and managing market
basket purchases.
We illustrate such cross-category analysis using a z = 10 topic solution. Conditioning17

on the number of segments is common practice in the machine learning literature. We
tried, but were not successful in estimating z as part of our model (see appendix B) andleave this as an area for future research..

Table 3: Joint Segmentation for Cross-category Analysis

Table 3 displays the result of the joint segmentation of products and consumers using
equations (16) and (17). The five products with highest probability and their averagelevels of marketing activity are shown for each segment. The first column reports the brand
name, the second column reports the product category associated with the oﬀering, andthe remaining columns display the average level of marketing activity, i.e., the average
price rate, average display rate, and average feature rate. The title of each segment
contains the numbers of items and customers jointly classified into the same segment.
The segments are interpreted as follows.
The first segment has 31 consumers and 9 items are assigned to it. This segment
contains beverages across diﬀerent categories with small discount rates and low rates
of feature advertising. The second segment is characterized as being composed of theidentical brands in the desert category. The items are infrequently discounted and havea higher rate of display than the first segment. Segments 3 through 7 have relatively
fewer consumers and items, and they exhibit greater variation in the level of marketing
activity. In particular, Segment 5 contains two oﬀerings in both the ice cream and dressing
categories with the same brand names, both with relatively high rates of display andfeature activity. Segment 6 contains contains mainly items from the drink category andis similar in marketing activity with segment 5. Segment 7 is also comprised of drink
items with higher marketing level as well as other items with lower level of activities. The
items in segment 8 are comprised of variety of product categories with relatively higher
level of display. Segment 9 is the largest cluster with 946 consumers and 332 items. It is
18characterized as having the highest level of display activity. Segments 8 and 10 containthe less discounting and more displayed items, and the former is double and triple sizedin consumers and items.
The potential use of this information is in managing cross-category behavior. Knowing
the products typically purchased for diﬀerent types of shopping trips can be used to determine the range of impact of price promotions and merchandising activity. If consumers
have a budget for a particular shopping occasion, rather than for a particular productcategory, then the influence of a price reduction will have a broader eﬀect in traditionalmodels of demand. Our model allows for the identification of the boundary of eﬀects aspart of the topic, or latent segment, characterization.

5.2

Individual-level parameter estimates

The management of pricing, displays and feature activity within a store involves decisions
that cut across time and consumers, and requires knowledge of which product categories
are most sensitive to these actions. More recently, targeted coupon delivery systems haveallowed for the individual-level customization of prices. Managing these decisions requiresa view of the sensitivity of consumers and product categories to these actions.
Individual-level estimates of market response is obtained by using the equation (18)and two sided significance test on each estimate with the level of 5% is conducted byt test for deciding eﬀectiveness of marketing variables in empirical analysis. We notethat customers will display variation in their sensitivity to variables such a price acrossproduct categories because of varying aspects of the product categories (e.g., necessary
versus luxury goods, amount of product diﬀerentiation, price expectations) and diﬀerent
purposes of the shopping visit over time (e.g., shopping for one’s self or others, large
versus small shopping trip, etc.).
We can marginalize α̂ci by either of its arguments, c and i, to obtain characterizations of customers and items useful for analysis. The empirical marginal distribution of19

consumer parameter estimates is obtained by averaging across the 500 products in our{C}∑analysis, i.e.,α̂ci /C . A histogram of 500 items for each marketing variable arec=1

displayed on the left side of Figure 1, and provides information about the general distribution of heterogeneity faced by the firm for actions such as price customization. We
find that the individual-level estimates to be plausible in that the price coeﬃcients arenegative and the display and feature coeﬃcients are estimated to be positive.
We can also summarize heterogeneity across consumers and examine the distributionof marketing variables for the 500 products in our analysis. The empirical marginal distri{I}∑butions of individual products, averaging over the 1650 consumers, i.e., ofα̂ci /I ,i=1

are depicted on the right of Figure 1. The products that never displayed and featured
in the data period have been omitted from the histograms. These estimates are usefulfor deciding which product categories should receive merchandising support in the formof in-store displays and feature advertising. We find that the estimates are plausible inmost product categories with negative price coeﬃcients, and positive display and featurecoeﬃcients, but there exists fairly wide variation in the eﬀectiveness of these variables
across products. Many product categories appear to be unresponsive to merchandisingeﬀorts.

Figure 1: Marginal Distribution of Individual Parameter Estimates:
Figure 2 provides a two dimensional summary of the data and coeﬃcient estimates.
Figure 2(a) is a scatter plots of two dimensional data cube with respect to customers(i) and products (c), aggregated along the time (t) dimension. If a customer has neverpurchased a specific product in the dataset, the coordinate (i, c) is colored “white,” andit is “black” if they have purchased the product at least once. We observe that customeritem space is still very sparse.
Figures 2(b)-2(d) show the results of testing with a 5% level of significance level for nonzero individual response coeﬃcients. In figure 2(b), the coordinates with a significant price20

coeﬃcient is indicated as “black” and “white” shows that the estimate is insignificant.
The eﬀectiveness of display and feature promotions are similarly defined. We find that
our model produces many significant price, display and feature coeﬃcients.

Figure 2: Personalized Eﬀective Marketing Variables for All Customers and Items
Figure 3 provides a close up of Figure 2 for 100 products and customers. An interesting
aspect of our analysis is that significant coeﬃcients can arise even when a customer has
never purchase a product because of the imputation present in the topic model for nonpurchases. The topic model greatly reduces the dimensionality of the data cube, as shownin Equation (1), and results in individual-level estimates in a sparse data environment.
Our analysis yields coeﬃcient estimates at the individual- and product-level by way ofthe latent topics that transcend the product categories. Our model enables marketers todevelop eﬀective pricing and promotional strategies by recognizing the presence of latent
topics, or shopping baskets, present at each point in time in the data cube.

Figure 3: Personalized Eﬀective Marketing Variables for 100 Customers and Items

6

Discussion

The unit of analysis in marketing is not a person or a product, but a person embedded
within a context of action for which a product might be useful. Consumers find value
in the goods and services that help them deal with issues in their lives, which is time
specific. It is not surprising that shopping behavior is therefore time specific, with some
shopping trips encompassing a large number of purchases and expenditures, while othershopping trips having a much smaller number of items being purchased. We propose amodel for dealing with a large number of oﬀerings by recognizing the presence of shopping
heterogeneity at each point in time, and employ a topic model for dealing with the many
choice alternatives available at retailers.
21This paper addresses three challenges in estimating models of demand in large databases:i) the large number of available products, ii) the large number of consumers who purchasethese products, and iii) the sparseness of transaction data. Existing models in marketingand methods of estimation tend to focus on a narrow set of products and a subset of
consumers to understand the richness of the competitive environment within a productcategory among a random sample of consumers. This goal, however, is often at odds withthe goals of practitioners who want to score existing datasets to identify a wide set ofcustomers and products to allocate promotional budgets and increase sales.
We propose a descriptive model of demand based on the idea of topic models where
products purchased by consumers take the place of words used by authors in creating
documents. We allow for a product’s purchase probability to be aﬀected by price, display
and feature advertising variables, but do not treat purchases to arise from a process ofconstrained utility maximization. The advantage of this approach is that it allows us toside-step complications associated with competitive eﬀects and model a much larger set of
products than that possible with existing economic models. By retaining prices and othermarketing variables in our model we can still predict the eﬀect of these variables on ownsales. This tradeoﬀ is inevitable in the analysis of large-scale databases where purchases
are tracked across thousands of products. The proposed model links the characteristics
of consumer segments to marketing variables, and it is applicable to both segment-level
and individual-level marketing across a large set of products.
The scalability and predictive performance of the proposed models were confirmed
through a simulation study involving variational Bayes inference. In our analysis, we
imposed a fairly conservative convergence criteria for VB of 10−5 %, but also found thatcoarser thresholds produced similar results. We therefore believe that estimation times
can be further reduced in practice from those reported in this paper.
Our model allows us to engage in the joint segmentation of consumers and items byusing the posterior probability of latent state which is allocated to every point of data22

cube. The information on response to marketing eﬀorts in a reduced dimensional spacecompressed by topic model is decompressed into original space by using variational Bayes
inference to obtain the individual response parameters in data cube. We show how themodel can be used to produce information useful for personalized marketing for both
specific customers and specific products, and eﬀectively deals with data spareness due toinfrequent consumer purchases.
Our model assumes the stability of the topic structure over time. However, it is possiblethat consumer’s market response and purchase patterns change over time because offactors such as new trends, state dependence and the arrival of new purchase and delivery
technologies. We believe the development of a dynamic topic model for purchase is aninteresting extension of our work, and leave this for future research.

23

Appendix A: Derivation of VB Algorithm for Proposed Model
This appendix details the variational inference of proposed model. The update procedure
derives from the analytical calculation of Equation (13). The update equation for eachvariational parameter is obtained from the following expectation values
E ̸=qj [log p (D, Θ)] ≡ E k̸=j [log p (D, Θ)]∫)∏ (= log p (D, Θ)qi Θ(i)∗ dΘ(i)∗ ,

(A1)

k̸=j

where D = {{xit } , {ycit }}.
The update procedures of variational parameters θ ∗c , z ∗cit , ucit , β ∗iz , Vizβ∗ , µ∗i , σiµ∗ , wi∗ ,(z)∗

and Wi∗ are presented below.

A.1

Optimization of θ ∗c

The Dirichlet and categorical distributions are of the following forms:∏ZZ
Γ (γ̃z ) ∏ γ̃z −1
z=1
)
θcz
Dirichlet (θ c | γ̃) = (∑
Z
Γ
γ̃
z=1
z
z=1
Categorical (zcit | θ c ) =

Z
∏

(A2)

δ(zcit =z)
θcz

z=1

where Γ (·) is the gamma function and δ(zcit = z) is the Dirac delta function defined asδ(zcit = z) = 1 if zcit = z and δ(zcit = z) = 0. The expectation value E ̸=qθ [log p (D, Θ)]24

is then calculated for each c as
E ̸=qθ [log p (D, Θ)] = log p (θ c ) + E qz [log p ({zcit } | θ c )] + const.
[(
)]
Z
Z
(∑ Z
) ∑
∑
∑∑
∗
= log Γγ̃z −logΓ (γ̃z ) +
γ̃z +
zcitz
−1
log θcz + const,
z−1

z=1

z=1

i∈Ic t∈Tc(A3)∗
where, zcitz
is a element of z ∗cit . Here and hereafter, const. denotes any terms not includedin the relevant parameters. The second line of the above equations describes a log-Dirichlet∑ ∑ ∗function with parameter γ̃z +zcitz . Therefore,i∈Ic t∈Tcθ ∗c ← γ̃ +∑∑z ∗cit(A4)i∈Ic t∈Tc

A.2

Optimization of z ∗cit

Here we denote a digamma function as Ψ (·), which will be useful for later discussion,(z)

and summarize the property of truncated normal distribution in the probit model. ucit(z)follows a normal distribution with mean xTit β zi and variance 1. Moreover, ucit must satisfy(z)ycit = 1 if ucit > 0 and ycit = 0 if ucit ≤ 0. Therefore, ucit is generated from a truncatednormal distribution as()
 T N(0,∞) xT β , 1
if ycit = 1
zi
it
(z)
ucit ∼.
()
 T N(−∞,0) xTit β zi , 1 if ycit = 0(A5)

where T N(n1 ,n2 ) (·, ·) denotes a normal distribution truncated from n1 to n2 . The distri(z)bution of ucit is therefore expressed as(p(z)
ucit | β zi , zcit , xit , ycit

)

{
)2 }
1
1 ( (z)T= (z) √ exp − ucit − xit β zi
2Ωcit 2π
1

25

(A6)

{ (
)}y {
(
)}(1−ycit )(z)with Ωcit ≡ F xTit β zi cit 1 − F xTit β zi
. In addition, the expectation value
and variance are expressed as
[
]
(z)
(z)
E ucit = xTit β ∗zi + φcit
[
]
(
)2
(z)
(z)
T ∗ (z)V ucit = 1 − xit β zi φcit − φcit

(A7)(
) (z)∗{ ()}y {()}(1−ycit )(z)(z)∗
where φcit ≡ (−1)(1−ycit ) f xTit β ∗zi /Ωcit and Ωcit ≡ F xTit β ∗zi cit 1 − F xTit β ∗zi.
Thus, the expected value E ̸=qz [log p (D, Θ)] is given as
E ̸=qz [log p (D, Θ)] = E qc [log p (zcit | θ c )]
)]
[
(
(z)
+ E qu ,qβ log p ucit | β zi , zcit , xit , ycit + const.

(A8)∗
The first term in the right hand side of Equation (A8) is obtained as Ψ (θcz
)−
(∑
)
Z
∗
Ψ
z−1 θcz (Blei et al. 2003), while the second term is evaluated as

[
)2 ]
[
(
)]
√
1 ( (z)(z)(z)
Tu − xit β zi
E qu ,qβ log p ucit | β zi , zcit , xit , ycit = E qu ,qβ − log 2πΩcit −
2 cit
[(
[(
)2 ]
] 1
[
] 1
[
)2 ]
(z)
(z)
(z)
+ const.
= −E qβ log Ωcit − E qu ucit
+ E qu ,qβ ucit xTit β zi − E qβ xTit β zi
2
2
(A9)
∗
To solve Equation (A8) for zcitz, we must evaluate the four terms of Equation (A9).

The first term includes a CDF from which the expectation value is diﬃcult to obtain
analytically. Thus, we expand the term as a zeroth-order Taylor expansion in terms of
the CDF of normal distribution and the logarithm function. Such bold approximationis standard strategies for adapting topic models with VB to practical computation (forexamples, zeroth-order Taylor approximation by Asuncion et al. (2009) and Sato andNakagawa (2012), and zeroth and first order delta approximation by Braun and McAuliﬀe26

(2010)). The four expectation values in Equation (A9) are then written as[](z)
E qβ log Ωcit ≈ const,
[(
)2 ]
[
] (
)2
(z)
(z)
(z)
= V ucit + xTit β ∗zi + φcit ,E qu ucit
[
] (
)
)
(z)
(z) (
E qu ,qβ ucit xTit β zi = xTit β ∗zi + φcit xTit β ∗zi + xTit Vziβ∗ xit ,
[(
)2 ]
(
)2
Eqβ xTit β zi
= xTit Vziβ∗ xit + xTit β ∗zi .

(A10)

∗
Finally, zcitz
is updated as

∗
←
zcitz

exp (ρcitz )
Z
∑

,

(A11)

exp (ρcitj )

j=1

where
∗
ρcitz = Ψ (θcz
)−Ψ

A.3

(∑ Z
z−1

∗
θcz

)

1
1
(z)
+ xTit β ∗zi φcit + xTit Vziβ∗ xit .
2
2

(A12)

(z)∗

Optimization of ucit(z)∗

Similar to Equations (A3) and (A9), the expected value that optimizes ucit is
[
(
)]
(z)
E ̸=qu [log p (D, Θ)] = E qz ,qβ log p ucit | β zi , zcit , xit , ycit + const.

(A13)

(z)

Here we seek the mean vector of the truncated normal distribution of ucit . Therefore, theupdate equation becomes
ucit ← xTit β ∗zi + φcit .
(z)

(z)∗

A.4

(A14)

Optimization of β ∗zi and Vziβ∗First, we derive an inverse Wishart distribution function and adopt some well-known
properties of multivariable normal and inverse Wishart distributions (Anderson 2003,

27

Bishop 2006).
(

w̃/2

)

W̃

+1
− w̃+M
2

{

)}
1 (
−1
exp − tr W̃ Vi
,
2

|Vi |
2w̃M Γ(w̃/2)
( ∗
)
M
∑
wi + 1 − m
E qV [log |Vi |] =Ψ
+ M log 2 + log Wi∗−1 ,2
m=1[ −1 ]
E qV Vi= wi∗ Wi∗−1 ,[]Eqµ ,qV (β zi − µi )T Vi−1 (β zi − µi ) = (β zi − µ∗i )T wi∗ Wi∗−1 (β zi − µ∗i ) + σiµ∗ .
IW W̃ , w̃ =(A15)

We obtain the optimization procedures of β ∗zi and Vizβ∗ by the following expected value:E ̸=qβ [log p (D, Θ)] = E qµ ,qV [log p (β zi | µi , Vi )]
[
({
}
)]
(z)
+ E qu ,qz log p ucit | β zi , {zcit , xit , ycit } + const.
[]
1= − E qµ ,qV (β zi − µi )T Vi−1 (β zi − µi )
2
[(
C
)2 ]
1 ∑∑
(z)
T
−
E qu ,qz ucit − xit β zi
+ const.
(A16)
2 c=1 t∈Tc

The first and second terms of the second line are given by the last and third lines ofEquation (A10), while the third and fourth terms are given by Equations (A2) and (A3),respectively, derived in a manner similar to quation (A9). β ∗zi and Vziβ∗ are then arithmetically updated as{}−1 { ∗ ∗−1 ∗}β ∗zi ← wi∗ Wi∗−1 + Xzi XiT
wi Wi µi + Xzi ūziVziβ∗ ←{ ∗ ∗−1}−1
wi Wi + Xzi XiT

(A17)

where
[{
}
(z)∗
ūzi ≡ ucit

c=1,···C,t∈Tc

]T
,

[
]
Xi ≡ {xit }c=1,···C,t∈Tc ,

28

]
[
∗
xit }c=1,···C,t∈Tc .
Xzi ≡ {zcitz

The ūzi is vector and Xi and Xzi are matrices. The number of elements in ūi , Xi and
Xzi are decided by the size of the consumer base and by Tc .

A.5

Optimization of µ∗i , σiµ∗ , wi∗ , and Wi∗

Here we consider a joint distribution of a multivariable normal distribution of µi andan inverse Wishart distribution of Vi , and derive the update equations for four types ofvariational parameters from this joint distribution. To this end, we require the following
expectation value from the joint distribution function:
E ̸=qµ ,qV [log p (D, Θ)] = log p (µi , Vi ) + Eqβ [log p ({β zi } | µi , Vi )] + const.

}
1
1w̃ + M + 1
1 {= − log |Vi | − σ̃µ−1 (µi − µ̃µ )T Vi−1 (µi − µ̃µ ) −log |Vi | − tr W̃ Vi−1
2
2
2
2
Z
[
]
1
1∑
− Z · Eqβ [log |Vi |] −Eq (µi − β zi )T Vi−1 (µi − β zi ) + const.
(A18)
2
2 z=1 βFirst, we extract from this expectation value all terms linked to multivariable variationalµ∗
parameters µµ∗i and σi ; that is1
E ̸=qµ [log p (D, Θ)] = − σ̃µ−1 (µi − µ̃µ )T Vi−1 (µi − µ̃µ )
2
Z
[
]
1∑
−
Eqβ (µi − β zi )T Vi−1 (µi − β zi ) + const.
2 z=1

(A19)

The second term in the above equation is obtained in the same manner as Equation (A15).
The multivariable normal distribution function is then constructed in a straightforwardmanner as follows:
µ∗i ←

)−1
( −1
σ̃µ + Z(σ̃µ−1 µ̃µ +()−1σiµ∗ ← σ̃µ−1 + Z
.

Z
∑

)
β ∗zi

,

z=1

(A20)

Next, we optimize wi∗ and Wi∗ using Equation (A15) and the relationship log q (Vi ) =29

log q (µi , Vi ) − log q (µi | Vi ).
E ̸=qV [log p (D, Θ)] = E ̸=qµ ,qV [log p (D, Θ)] − E ̸=qµ [log p (D, Θ)](A21)

The expectation value E ̸=qV [log p (D, Θ)] is calculated in a straightforward manner byusing Equations (A16) and (A17). Finally, we obtain the update equations for wi∗ andWi∗ asWi∗ ← W̃ +∑Z
z=1

Vziβ∗ + σ̃µ−1 µ̃µ̃ +∑Zz=1) ∗ ∗T( −1
β ∗zi β ∗Tzi − σ̃µ + Z µi µi ,wi∗ ← w̃ + Z.

(A22)

Notice that σiµ∗ and wi∗ are constant if the hyperparameters and the number latent classare given.

Appendix B: Variational Lower Bound of Proposed
Model
The variational lower bound L (Θ∗ ) is given by∗∫ [L (Θ ) =][]p (Θ, {xit } , {ycit })p (Θ, {xit } , {ycit })q (Θ|Θ ) logdΘ = E qΘβ log
q (Θ|Θ∗ )q (Θ|Θ∗ )
∗

(p)

(p)

(p)

(q)

(q)

(q)

(p)
(q)
(q)
= Lθ + L(p)z + Lu + Lβ + Lµ,V − Lθ − Lz − Lu − Lβ − Lµ,V ,where, each component of L (Θ∗ ) is expectation of variables of proposed model. The
(p)

(q)

expectations excepting Lu and Lu are followings;(p)Lθ = E qc [log p ({θ c })]
]
[
C
Z
Z
)}
(∑ Z
(∑ Z
) ∑
{
∑
∑
∗
∗
,
)−Ψ
θcz
=
log Γ
γ̃z −
logΓ (γ̃z ) +(γ̃z − 1) Ψ (θcz
c=1

z=1

z=1

z=1

30

z=1

L(p)
z = E qz ,qc [log p ({zcit } | {θ c })]
=

Z
C ∑∑∑
∑

{
(∑ Z
)}
∗
∗
∗
zcitz
Ψ (θcz
)−Ψ
θcz
,
z=1

c=1 i∈Ic t∈Tc z=1(p)Lβ = E qβ ,qµ ,qV β [log p ({β zi } | {µi , Vi })]


)
( ∗
M
∑
wi + 1 − mIZ
M log 2π ++ M log 2 + log Wi∗−1
Ψ

1 ∑∑
2


m=1
=−

,
2 i=1 z=1 

{
}
T
µ∗
µ∗
∗
∗ −1
∗
∗
∗ −1 β∗+ (µ∗zi − µµ∗
)
w
(W
)
(µ
−
µ
)
+
tr
w
(W
)
V
+
σ
i
i
zi
i
i
i
i
zi
i

(p)

Lµ,V = E qµ ,qV β [log p ({µi , Vi })]


[
]
µ∗
µ∗
µ∗
µ T
µ
∗
∗ −1
−1
 M log 2π + σ̃µ (µi − µ̃ ) wi (Wi ) (µi − µ̃ ) + σi



)
(
{
}


w̃
I
−1

+ tr W̃ (Wi∗ )1 ∑ −w̃ log W̃ + log 2 + 2 log Γ

2
=−

,

2 i=1 
{
}
(
)
M


∗
∑
wi + 1 − m
∗ −1 +(w̃ + M + 2)
Ψ
+ M log 2 + log (Wi )2
m=1

Lθ = E qc [log qc ({θ c } | {θ ∗c })]
[
]
C
Z
Z
(∑ Z
) ∑
{
(∑Z
)}
∑
∑
∗
∗
∗
∗
∗
=
log Γ
θcz
−
logΓ (θcz)+(θcz− 1) Ψ (θcz
)−Ψ
θcz
,
(q)

c=1

z−1

z=1

z=1

Lz(q) = E qz [log qz ({zcit } | {z ∗cit })]
=

C ∑∑∑
Z
∑
c=1 i∈Ic t∈Tc z=1

31

∗
∗
,
log zcitz
zcitz

z−1

(q)
Lβ = E qβ

[
(
{
})]
β∗
∗
log qβ {β zi } | µzi , Vzi

1 ∑∑=−{M log (2πe) + log |Vzi∗ |}2 i=1 z=1
I

Z

and
[
]
(q)
µ∗
∗
∗
Lµ,V = E qµ ,qV β log qµ,V β ({µi , Vi } | {µµ∗i , σi , wi , Wi })
( ∗)


1
wi
µ∗
∗
∗
∗

 M log 2π + log |σi | − wi log |Wi | + wi M log 2 + 2 log Γ 2


}
{
I


( ∗
)
M
1 ∑

∑
w
+
1
−
m
=−
i
∗ −1
.
 + (w∗ + M + 2)
)
Ψ
+
M
log
2
+
log
(W

2 i=1 
i
i
2


m=1


+wi∗ + 1

B.1(q)Derivation of L(p)u − Lu(z)

The entropy of ucit is given by}( T ∗ )2
1 { [ 2](z)∗
T ∗Entropy = −E ξ − 2xit µzi E [ξ] + xit µzi + log(2π) − log Ωcit ,2where, ξ is a random variable of the distribution (Grimmer 2010 b). Therefore,
[
({
}
)]
(z)
(q)
L(p)
−
L
=E
log
p
u
{β
,
z
,
x
,
y
}
qu ,qβ ,qz
it citzi cit
u
u
cit
[
({
} {
})]
(z)
(z)∗
− E qu log qu ucit
ucit , xit , ycit
I()}]
1 ∑[ {β∗∗ ∗T
T r Xi Xi µzi µzi + Vzi
=−
2 i=1
}
C ∑∑{
∑
1 ∗ ( T ∗ )2(z)∗∗
+ θcitz log Ωcit .
θx µ+
2 citz it zi
c=1 i∈I t∈Tcc

The value of L (Θ∗ ) is calculated by summation of the above ten expectations.

32

Appendix C: Gibbs Sampler
The joint posterior distribution, assuming conditional independence between variables,
provides the full conditional posterior distributions:θ c | − ∼ p (θ c | zcit )zcit | − ∼ p (zcit | θ c , {β zi }, {xit }, {ycit })
(
)
(z)
(z)
ucit | − ∼ p ucit | zcit , β zi , xit , ycit()(z)
β zi | − ∼ p β zi | {ucit }, µi , Vi , {xit }(C1)µi | − ∼ p (µi | {β zi }, Vi )
Vi | − ∼ p (Vi | {β zi }, µi )C.1

Sampling of θ c

The θ c is generated by a Dirichlet categorical relation. The Dirichlet distribution is aconjugate prior of a categorical distribution. For each consumer c, nc = [nc1 , · · · , ncZ ]Tdenotes the number of generated latent classes zcit by categorical distribution of parameterθ c in each MCMC step. A Dirichlet categorical relation gives the posterior distribution
with respect to θ c asp (θ c | −) = p (θ c ) p (zcit | θ c ) = Diriclet (nc + γ̃)

C.2

(C2)

Sampling of zcit | −The posterior probability of (zcit = z) is given as(z)θcz Ωcit

Pr {zcit = z | θ c , {xit } , {β zi } , {ycit }} = ∑Z(j)
j=1 θcz Ωcit

33

,

(C3)

C.3

(z)

Sampling of ucit | −
(z)

(z)

The distribution of ucit is described in Appendix A.2. ucit is sampled from a truncatednormal distribution in Equation (A5). This well-known sampling approach is called data
augmentation (Tanner, 1987).

C.4

Sampling of β zi , µi , and Vi

The full conditional posterior distribution of β iz , µi , and Vi is derived from a hierarchicallinear regression model. In our case, β zi for each i and each z is sampled from
)
}
)
(
{(
T (z)
β iz ∼ NM R−1 X̄ziuzi + Vi−1 µi , R−1 ,(z)Twhere R ≡ X̄zi
X̄zi + Vi−1 , uzi ≡

[{
}
(z)
ucit

]T

c∈zc =z, t∈Tc

(C4)

[
]T
and X̄zi ≡ {xit }c∈zc =z,t∈Tc .

µi is sampled from(µi ∼ NM−1(Z + σ̃µ )

Z
∑

)
−1

β zi , Vi + (Z + σ̃µ )

IM

,

(C5)

z=1

[
for each i. Here, the hyperparameters are set to µ̃ =]T
0 0 0 0.

Finally, Vi for each i is sampled from()
Vi ∼ IW w̃ + Z, W̃ + B T B ,where B ≡

Z
∑
z=1

(
)
Z
∑
−1
β zi − Zβ zi .
z=1

34

(C6)

References[1] Anderson, T. W. (2003). An Introduction to Multivarite Statistical Analysis, Wiley, U.S.A[2] Ansari, A., and Mela, C. F. (2003). “E-Customization”. Journal of Marketing Research., 40,
131-145.
[3] Asuncion, A., Welling, M., Smyth, P. and Teh, Y.W. (2009). “On Smoothing and Inference
for Topic Models” , Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial
Intelligence, 27-34.
[4] Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer, U.S.A.
[5] Blattberg, R.C., Kim, B.D., and Neslin, S.A. (2009). Database Marketing: Analyzing andManaging Customers,, Springer: PA.
[6] Blei, D.M., Ng, A.Y., and Jordan, M.I. (2003). “Latent Dirichlet Allocation,” Journal of
Machine Learning Research, 3, 993-1022.
[7] Blei, D., and McAuliﬀe. J. (2007). “Supervised Topic Models,” Proceedings of Neural Information Processing System, 3, 993-1022.
[8] Braun, M., and McAuliﬀe. J. (2010). “Variational Inference for Large-Scale Models of Discrete Choice,” Journal of the American Statistical Association, 105, 324-335.
[9] Chintagunta, P.K., and Nair. H.S. (2011). “Discrete-Choice Models of Consumer Demand inMarketing,” Marketing Science, 30, 977-996.
[10] Chung, T.S., Rust, R., and Wedel. M. (2009). “My Mobile Music: An Adaptive Personalization System for Digital Audio Players,” Marketing Science, 28, 52-68.
[11] Corduneanu, A., and Bishop, C.M. (2001). “Variational Bayesian Model Selection for Mixture Distributions. In: Jaakkola, T., Richardson, T. (Eds.)”, Articial Intelligence and Statistics, Morgan Kaufmann: Los Altos, CA, 2734.
[12] Grimmer, J. (2011). “An Introduction to Bayesian Inference via Variational Approximations,” Political Analysis, 19, 32-47.
[13] Ishigaki, T., Takenaka T., and Motomura. Y. (2010). “Category Mining by Heterogeneous
Data Fusion Using PdLSI Model in a Retail Service,” Proceeding of IEEE International
Conference on Data Mining, 857-862.
[14] Iwata, T., Watanabe, S., Yamada, and T., Ueda, N,. (2009). “Topic Tracking Model for
Analyzing Consumer Purchase Behavior,” , Proceeding of International Joint Conference on
Artificial Intelligence, 1427-1432.

35

[15] Iwata, T., and Sawada, H., (2012). “Topic Model for Analyzing Purchase Data with Price
Information,” Data Mining and Knowledge Discovery, 26, 559-573.
[16] Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., and Saul, L.K. (1999). An Introduction to
Variational Methods for Graphical Models,” Machine Learning, 37, 183-233.
[17] Puolamaki, K. and S. Kaski (2009). “Bayesian solutions to the label switching problem,”
In Advances in Intelligent Data Analysis VIII, 381-392, Springer[18] Naik, P., Wedel, M., Bacon, L., Bodapati, A., Bradlow, E., Kamakura, W., Kreulen, J.,
Lenk, P., Madigan and D.M., Montgomery, A. (2008). “Challenges and opportunities in highdimensional choice data analyses,” Marketing Letter, 19, 201-213.
[19] Ramage, D., Hall, D., Nallapati, R., and Manning, C.D. (2009). “Labeled LDA: a Supervised Topic Model for Credit Attribution in Multi-labeled Corpora,” Proceedings of the 2009
Conference on Empirical Methods in Natural Language Processing, 248-256.
[20] Rossi, P.E., Allenby, G.M., and McCulloch, R. (2005). Bayesian Statistics and Marketing,John Wiley & Sons: Chichester, UK.
[21] Rust, R.T. and Chung, T.S. (2005). “Marketing Models of Service and Relationships,”
Marketing Science, 25, 560-580.
[22] Spirling, A. and Quinn, K. (2010). “Identifying Intraparty Voting Blocs in the U.K. Houseof Commons”, Journal of the American Statistical Association, 105, 447-457.
[23] Sato, I. and Nakagawa, H. (2012). “Rethinking Collapsed Variational Bayes Inference for
LDA,” Proceedings of International Conference on Machine Learning, 999-1006.
[24] Tanner, M.A. and Wong, W.H. (1987). “The Calculation of Posterior Distributions by Data
Augmentation,” Journal of the American Statistics Association, 82, 528-540.
[25] Teh, Y.W. and M. I. Jordan. (2010). “Hierarchical Bayesian nonparametric models withapplications,” eds N. Hjort, C. Holmes, P. Mueller, and S. Walker, Bayesian Nonparametrics:
Principles and Practice, Cambridge University Press, Cambridge, UK:, 2010.
[26] Tsiptsis, K. and Chorianopoulos, A. (2010). Data Mining Techniques in CRM: Inside
Customer Segmentation, Wiley: UK.
[27] Wedel, M. and Kamakura, W.A. (1999). Market Segmentation: Conceptual and Methodological Foundations, Kluwer Academic Publishers: U.S.A.

36

Table 1: Estimates of Simulation Data
Estimates (Posterior mean)
Intercept
Discount
Display
FeatureSegment 1 -0.45 (0.03) 0.89 (0.05) -0.03 (0.05) 0.04 (0.01)
Segment 2 -0.50 (0.01) 0.07 (0.04) 0.91 (0.02) 0.02 (0.02)
Segment 3 -0.51 (0.01) 0.04 (0.03) 0.00 (0.03) 0.93 (0.03)
Simulated data (C = 300, I = 50, T = 30).

Table 2: Simulation Time by VB and MCMC

Z

5

VB
10

20
5
C = 1000
100 0.6 0.8
1.1
5.3
500 1.4 1.7
2.3
21.7
1000 2.0 2.2
2.7
49.0
C = 5000
100 2.1 2.3
3.0
23.4
I 500 2.3 3.2
5.2
65.5
1000 4.4 5.2
8.2 128.7
C= 10000
100 3.5 4.2
5.7
49.4
500 5.3 7.0 10.4 213.3
1000 8.9 12.6 17.2 430.1
The number means hour.

37

MCMC
10

20

7.1
29.6
54.6

14.2
41.7
62.4

30.3
81.2
144.0

46.8
104.1
166.2

67.9
261.0
482.7

102.5
343.0
580.8

Table 3: Joint Segmentation for Cross Category Analysis

Brand
No.1
No.2
No.3
No.4
No.5
Brand
No.8
No.9
No.10
No.11
No.12
Brand
No.17
No.18
No.19
No.18
No.19
Brand
No.6
No.14
No.12
No.22
No.20
Brand
No.24
No.21
No.25
No.26
No.9

Segment 1 (C=31,I=9)Category
Price
Display
Drink
.99
.06
Coﬀee
.89
.10
Iced noodle.77.60
Bean paste
.75
.21
Coke
.89
.24
Segment 3 (C=22, I=4)
Category
Price
Display
Fish sausage .93
.08
Water
.60
.47
Detergent.69.20
Ice cream
.91
.02
Water
.87
.11
Segment 5 (C=24, I=5)
Category
Price
Display
Soup
.84
.16
Dressing.76.72
Ice cream.76.57
Dressing.83.42
Ice cream.82.14
Segment 7 (C=67, I=14)
Category
Price
Display
Desert
.96
.13
Food
.90
.03
Sugar
.99
.26
Drink
.77
.63
Drink
.75
.52
Segment 9 (C=946, I=332)
Category
Price
Display
Cleaner
.85
.48
Sauce
.74
.35
Snack
.86
.16
Noodle.68.98
Energy drink .68.88

Feature

.06
.02
.03
.05
.02
Feature

.08
.04
.26
.02
.04
Feature

.09
.09
.22
.15
.10
Feature

.06
.01
.05
.17
.16
Feature

.11
.07
.09
.09
.06

38

Brand
No.6
No.7
No.6
No.6
No.6
Brand
No.13
No.14
No.13
No.15
1No.6
Brand
No.20
No.9
No.11
No.20
No.21
Brand
No.12
No.22
No.20
No.23
No.15
Brand
No.27
No.12
No.11
No.19
No.25

Segment 2 (C=114, I=28)
Category
Price
Display
Desert
.94
.13
Drink
.72
.92
Desert.94.17
Desert.93.22
Desert.93.19
Segment 4 (C=28, I=6)
Category
Price
Display
Noodle
89.
.23
Food
.90
.03
Noodle.78.09
Fish sausage .91
.01
Drink
.87
.11
Segment 6 (C=26, I=6)
Category
Price
Display
Drink
.81
.29
Drink
.76
.33
Ice cream
.99
.03
Drink
.75
.31
Drink
.64
.73
Segment 8 (C=267, I=68)
Category
Price
Display
Cookie
.98
.29
Coﬀee
.81
.28
Ice cream.89.36
Dressing.74.80
Fish sausage .91.01
Segment 10 (C=124, I=28)
Category
Price
Display
Drink
.99
.25
Water
.87
.26
Ice cream.99.03
Yoghurt
.88
.10
Curry
.67
.98

Feature

.06
.24
.04
.05
.06
Feature

.05
.01
.11
.01
.04
Feature

.17
.02
.03
.17
.11
Feature

.06
.08
.02
.08
.01
Feature

.11
.01
.03
.16
.08

0

2-

0
F
05 qer
u
ne
1 yc
05

2-

2

2

1

0

0

1-

1-

tpecretnI

1

yalpsiD

stcudorp laudividnI )b(1-

1-

erutaeF
1

0

ecirP
1

F
er
uq
ne
yc

2-

0
04
1
00

F
er
uq
ne
yc

0
F
05 er
uqe
1 nc
05 y

2-

2

2

0

1-

1-

2-

0
5 Fr
00 qe
u
ne
51 yc
00

2-

2

2

1

0

0

1-

1-

tpecretnI

1

yalpsiD

sremotsuc laudividnI )a(

ecirP

0

erutaeF
1

1

0
F
r
4 qe
00 nue
yc

0
4
00
01
00

2-

0
5
00
51
00

2-

Figure 1: Marginal Distribution of Parameter Estimates of Individual Consumers and Items
2

2

0
04
08

F
er
uq
ne
yc

F
er
uq
ne
yc

39

yalpsiD )c(ecirP )b(

drocer desahcruP )a(Figure 2: Personalized Eﬀective Marketing Variables for Individual Consumers and Items: All Customers and Items

erutaeF )d(stcudorP

C
u
st
mo
er
s

stcudorP

ecirP )a(stcudorP

C
us
t
mo
er
s

stcudorP

C
u
st
mo
er
s

stcudorP

C
u
st
mo
er
s

40

sr
e
m
ots
uC

sr
e
m
ots
u
C

Products

Products(a) Purchased record(b) Price

sr
e
m
ot
su
C

sr
e
m
ot
su
C

Products

Products(c) Display(d) Feature

Figure 3: Personalized Eﬀective Marketing Variables for Individual Consumers and Items:
100 Customers;100 Items

41