Data Min Knowl Disc
DOI 10.1007/s10618-012-0281-y

Topic model for analyzing purchase data with priceinformation
Tomoharu Iwata · Hiroshi Sawada

Received: 17 November 2011 / Accepted: 13 June 2012
© The Author(s) 2012

Abstract We propose a new topic model for analyzing purchase data with priceinformation. Price is an important factor in consumer purchase behavior. The proposed model assumes that a topic has its own price distributions for each item aswell as an item distribution. The topic proportions, which represent a user’s purchasetendency, are influenced by the user’s purchased items and their prices. By estimating
the mean and the variance of the price for each topic, the proposed model can cluster related items taking their price ranges into consideration. We present its efficient
inference procedure based on collapsed Gibbs sampling. Experiments on real purchasedata demonstrate the effectiveness of the proposed model.
Keywords Purchase log · Probabilistic topic modeling · Gibbs sampling · Clustering
1 Introduction
It is important to model user purchase behavior because it helps us to find a marketingstrategy and perform trend analysis, and provides recommendations and personalized
advertisements. Topic models have been used for analyzing log data of human behavior such as web usage (Jin et al. 2004) and news article reading (Das et al. 2007) as
well as purchase behavior (Iwata et al. 2009). A topic model is a probabilistic generative model for discrete data, where each user has his/her own probability distributionof selecting each topic, and each topic has a purchase probability distribution overResponsible editor: Bing Liu.
T. Iwata (B) · H. Sawada
NTT Communication Science Laboratories, 2-4, Hikaridai, Seika-cho, Soraku-gun,Kyoto 619-0237, Japane-mail: iwata.tomoharu@lab.ntt.co.jp

123

T. Iwata, H. Sawada

items. Topic models are mainly used for analyzing text data, where users and items inpurchase data correspond to documents and words in text data, respectively. “Topic”
in purchase data represents a latent category of items that are likely to be purchasedby a particular set of users. By using topic models, we can obtain a low dimensional
representation for each user, and find latent purchase patterns of co-occurrence items.
In this paper, we propose a new topic model for analyzing purchase data with priceinformation. Price is an important factor in consumer purchase behavior. People sometimes give up the idea of buying favorite items because they are too expensive, and buy
reasonably priced items even if they do not greatly care for them. The proposed model
has a distribution over items for each topic, in the same way as standard topic models,such as latent Dirichlet allocation (LDA) (Blei et al. 2003). In addition, the proposedmodel assumes that each topic has its own price distribution for each item, where the
distributions have different means and variances depending on topics and items. Thus,
topic proportions, which represent a user’s purchase tendency, are influenced by theitems purchased by the user and their prices.
Concurrently purchased items can differ depending on their prices. For example,
high ranking wines might often be purchased with expensive cheeses and hams; on theother hand, cheap wines might be purchased with vegetables for use in cooking. The
price range can also affect purchase behaviors in relation to other items. For example,
fashionable people who buy clothes in a wide price range might often buy fashion
magazines; thrifty people who buy only cheap clothes are not likely to buy fashion
magazines. By inferring price distributions with their means and variances as wellas item distributions, the proposed model can find clusters of concurrently purchased
items taking their price ranges into consideration.
The proposed model is a Bayesian probabilistic model. We use conjugate priors forall of the parameters in the proposed model, i.e. Dirichlet priors for multinomial parameters, and Gaussian-Gamma priors for Gaussian parameters. Therefore, the inference
can be efficiently performed based on collapsed Gibbs sampling by integrating out theparameters.
Standard topic models, such as LDA, are used for modeling samples (e.g. users anddocuments), each of which is represented by a set of discrete values (e.g. purchased
items and words). More recently, topic models have been proposed that incorporate
other information such as authors (Rosen-Zvi et al. 2004), time (Wang and McCallum2006; Blei and Lafferty 2006; Iwata et al. 2010) and annotations (Blei and Jordan
2003). The proposed model is different from these models in that it can model sets ofitems where a continuous value (price) is associated with each item. Standard topic
models can cluster items that are likely to be purchased by a specific user group.
However, they do not distinguish differences in price. One approach can model the
price distribution of an item by using a Gaussian mixture, and it can find clusters
of price ranges. However, it does not cluster concurrently purchased items whosepurchase is price dependent. The proposed model resolves these problems by incorporating price distributions into topic models and inferring item and price distributions
simultaneously.
The remainder of this paper is organized as follows. In Sect. 2, we propose a topic
model for purchase data with price information. In Sect. 3, we present its efficient
inference procedure based on collapsed Gibbs sampling. In Sect. 4, we outline related123

Analyzing purchase data with price information

work. In Sect. 5, we demonstrate the effectiveness of the proposed method by using
real purchase data. Finally, we present concluding remarks and a discussion of future
work in Sect. 6.
2 Proposed model
Suppose that we have a set of purchase logs of U users. The purchase log of user uu
consists of purchased items and their prices (xu , vu ). Here, xu = {xun }N
n=1 representsuitems that are purchased by user u, and vu = {vun }N
n=1 represents their prices. Ournotation is summarized in Table 1.
The proposed model finds latent topics that are influenced by both concurrently
purchased items and their prices. The proposed model first generates an item to bepurchased, and then generates its price. The generative process for purchased items
is the same as that of standard topic models. Each user has topic proportions θu thatare sampled from a Dirichlet distribution. For each of the Nu purchases, a topic zun
is chosen from the topic proportions, and then item xun is purchased according to a
topic-specific multinomial distribution φzun .
Each topic has its own price distribution for each item, which is assumed to be a
Gaussian distribution, Normal(μki , λ−1ki ). By setting different means and precisions
depending on the topics, we can analyze the price and its range that are specific to
the topic. The mean μki and the precision λki are sampled respectively from Gaussian and Gamma distributions, which are conjugate priors of a Gaussian distribution.
After topic zun and item xun are sampled, its price vun is determined according to the
topic-specific Gaussian distribution, Normal(μzun xun , λ−1
zun xun ).
In summary, the proposed model assumes the following generative process for aU
set of purchase logs with price information X = {xu }U
u=1 and V = {vu }u=1 :1. For each topic k = 1, . . . , K:(a) Draw item probability φk ∼ Dirichlet(β)(b) For each item i = 1, . . . , I :Table 1 Notation

Symbol Description
U

Number of users

I

Number of items

K

Number of latent topics

Nu

Number of purchased items of user u

xun

nth purchased item of user u

vun

Price of nth purchased item of user uzun

Latent topic of nth purchased item of user u

θu
φk

K
Topic
 proportions for user u, θu = {θuk }k=1 , θuk ≥ 0,
θ
=
1
k uk
Multinomial distribution over items for topic k,φk = {φki }Ii=1 , φki ≥ 0, i φki = 1

μki

Mean price of item i for topic k

λkiPrecision (inverse of variance) of price of item i for topic k123

T. Iwata, H. Sawada
Fig. 1 Graphical model
representation of the proposed
topic model for purchase datawith price information

α
θ

β

z

x

v
N

φ

η

μ

γ
a

U

λ
I

Fig. 2 Graphical model
representation of the standard
topic model

K

α
θ

b

I

β

z

φ

x
N

U

K

i. Draw price precision λki ∼ Gamma(ai , bi )ii. Draw price mean μki ∼ Normal(ηi , (γi λki )−1 )2. For each user u = 1, . . . , U :(a) Draw topic proportions θu ∼ Dirichlet(α)(b) For each transaction n = 1, . . . , Nu :i. Draw topic zun ∼ Multinomial(θu )ii. Draw item xun ∼ Multinomial(φzun )iii. Draw price vun ∼ Normal(μzun xun , λ−1
zun xun ),where α and β are Dirichlet parameters, a = {ai }Ii=1 and b = {bi }Ii=1 are shape andinverse scale parameters of Gamma distributions, respectively, and η = {ηi }Ii=1 andγ = {γi }Ii=1 are mean and precision parameters of Gaussian distributions, respectively.
Figure 1 shows a graphical model representation of the proposed topic model forpurchase data, which represents dependencies among variables. Here the shaded andunshaded nodes indicate observed and latent variables, respectively, the plate indicates
replicates, and the value in the plate indicates the number of replicates. For comparison,
we present a graphical model of the standard topic model, or latent Dirichlet allocationin Fig. 2. The proposed model is an extension of the standard topic model, where pricev is generated depending on topic z and item x. In addition, priors for parameters ofGaussian distribution are also incorporated for robust Bayesian inference.
The joint distribution of purchased items X, their prices V , and latent topics Z =Nu{zu }U
u=1 , where z u = {zun }n=1 , is described as follows:
P (X, V , Z|α, β, η, γ , a, b) = P (Z|α)P (X|Z, β)P (V |X, Z, η, γ , a, b).

(1)

We can integrate out multinomial parameters in the first and second factors, {θu }U
u=1
and {φk }Kk=1 , because we use Dirichlet distributions for their priors, which are conjugate to multinomial distributions. The first factor on the right hand side of (1) is
calculated by:123

Analyzing purchase data with price information

P (Z|α) =

Nu
U  

u=1

P (zun |θu )P (θu |α)dθu ,

(2)

n=1

and we have the following equation by integrating out {θu }U
u=1 :P (Z|α) =Γ (αK)
Γ (α)KU  k Γ (Nku + α),Γ (Nu + αK)u(3)

where Γ (·) is the gamma function, and Nku is the number of purchased items assigned
to topic k in the history of user u. Similarly, the second factor is given as follows by
integrating out {φk }K
k=1 :
P (X|Z, β) =K  i Γ (Nki + β)

Γ (βI )
Γ (β)IkΓ (Nk + βI )

(4)

,


where Nki is the number of times item i has been assigned to topic k, and Nk = i Nki .
We can also integrate out Gaussian parameters in the third factor of (1),IK{{μki }Ii=1 }K
k=1 and {{λki }i=1 }k=1 , because we use Gaussian and Gamma distributions
for their priors, which are conjugate to Gaussian distributions. The third factor is given
as follows:
P (V |X, Z, η, γ , a, b) =

K 
I


P (Vki |η, γ , a, b),(5)

k=1 i=1

where Vki is a set of prices of item i that is assigned to topic k, andP (Vki |η, γ , a, b)  −1P (v|μki , λ−1=ki )P (μki |ηi , (γi λki ) )P (λki |ai , bi )dμki dλkiv∈Vki

= (2π )

N

− 2ki

a

Γ (aki ) bi iaki
Γ (ai ) bki



γi
γki

1
2

(6)

.

Here, ηki and γki are hyperparameters of posterior distributions for mean μki , P (μki |X,
V , Z, ηi , γi ) = Normal(ηki , γki−1 ), and aki and bki are hyperparameters of posterior
distributions for precision λki , P (λki |X, V , Z, ai , bi ) = iGamma(aki , bki ). They are
given as follows:ηki =γi η i +v∈Vki v

γi + Nkiγki = γi + Nki ,Nkiaki = ai +
,
2

,

(7)
(8)
(9)

123

T. Iwata, H. Sawadabki = bi +v∈Vki v22+2γi ηi2γki ηki
−
.
2
2

(10)

In this way, the parameters in the proposed model can be integrated out, and thus, anefficient inference procedure can be derived as described in the next section.
3 Inference by Gibbs sampling
Given a set of purchased items with their prices, X and V , we would like to infer thelatent topics Z. Several approaches have been proposed for inferring topic models,such as variational Bayes (Blei et al. 2003), collapsed Gibbs sampling (Griffiths andSteyvers 2004), expectation propagation (Minka and Lafferty 2002), and collapsed
variational Bayes (Teh et al. 2006b). In this section, we present a simple and efficient
inference procedure based on collapsed Gibbs sampling.
Given the current state of all but one variable zj , where j = (u, n), the assignmentof a latent topic to the nth purchased item of user u is sampled from the following
probability:
P (zj = k|X, V , Z\j , α, β, η, γ , a, b)

akx \j 

jNku\j + α Nkxj \j + β Γ (akxj ) bkxj \j∝
Nu\j + αK Nk\j + βI Γ (akxj \j ) bakxj

kxj

γkxj \j
γkxj

1
2

,

(11)

where \j represents the count or hyperparameter when excluding sample j . See
Appendix A for the derivation. The first and second factors in (11) are the sameas those of LDA. The remaining factors are derived from price distributions. The hyperparameters of Gaussian-Gamma distributions excluding sample j are calculated asfollows:
γki ηki − vj,
γki − 1
γki\j = γki − 1,
1
aki\j = aki − ,222
γki\j ηki\jvj2
γki ηkibki\j = bki −
+
−
.
2
2
2

ηki\j =

(12)
(13)
(14)
(15)

The hyperparameters of Dirichlet distributions, α and β, can be estimated by maximizing the joint likelihood (1) by using the fixed-point iteration method describedin (Minka 2000) as follows: (Nku + α) − U K (α)
u
α←α
,
(16)
 k
K
(N
u + αK) − U (αK)
 uki (Nki + β) − KI (β)
β←β 
,
(17)
I
k (Nk + βI ) − K (βI )123

Analyzing purchase data with price information

where (·) is the digamma function defined by (x) = ∂ log∂xΓ (x) . We can set thehyperparameters of Gaussian-Gamma distributions, {ηi , γ , ai , bi }Ii=1 , by using theempirical means and variances of the given data. In the experiments, we used ηi =μ̄i , γi = 1, ai = 1, and bi = 1 + σ̄i2 , where μ̄i and σ̄i2 are the empirical mean andvariance of the price of item i.
At the beginning of the inference, we initialize latent topics so that none of thepurchased items are assigned to any topics. In the first iteration, a latent topic is sampled according to (11) for each item, where it is unnecessary to exclude the current
sample because it has no topic. After that, by iterating Gibbs sampling with (11) andmaximum likelihood estimation with (16) and (17), we can infer latent topics while
optimizing the parameters.
We can estimate parameters θuk , φki , μki and λki using sampled latent topics Z asfollows:
Nku + α,Nu + αK
Nki + βφ̂ki =,
Nk + βIμ̂ki = ηki ,aki.
λ̂ki =bkiθ̂uk =

(18)
(19)
(20)
(21)

The estimated topic proportions {θ̂uk }K
k=1 can be used for a low dimensional representation of user u, and the estimated parameters {φ̂ki , μ̂ki , λ̂ki }Ii=1 can be used to analyze
the characteristics of topic k.
4 Related Work
The proposed model is latent Dirichlet allocation (LDA) (Blei et al. 2003) extended
to include price information. Topic models are successfully used for a wide variety
of applications including information retrieval (Blei et al. 2003; Hofmann 1999), collaborative filtering (Hofmann 2003), language modeling (Wallach 2006; Williamson
et al. 2010), network analysis (Fu et al. 2009) and multilingual text analysis (Mimno
2009) as well as for modeling human behavior (Jin et al. 2004; Das et al. 2007; Iwata
et al. 2009). A number of topic models have been proposed for incorporating different types of information, such as author topic model (ATM) (Rosen-Zvi et al. 2004),
correspondence latent Dirichlet allocation (corr-LDA) (Blei and Jordan 2003), andtopic over time (TOT) (Wang and McCallum 2006). ATM and corr-LDA integrate
authors and annotations into topic models, respectively. The authors and annotations
are discrete variables, and they are associated with each document, but not each word.
In this paper, we analyze purchase data, in which a continuous variable (price) is
associated with each item (word), and prices differ among purchased items of a user
(document). TOT is a topic model for incorporating a continuous variable. TOT captures the dynamics of topics by assuming that each topic has its own beta distributionthat generates the document’s time stamp. In TOT, the time stamp is assumed to be123

T. Iwata, H. Sawada

associated with each word, although it is usually constant across the document. And
the time stamp is generated depending only on the topic; time is conditionally independent of words given the topic. On the other hand, the proposed model assumes thatthe price is generated depending on both the item (word) and the topic, which is naturalfor a generative process of prices. There have been some work on analyzing point ofsales (POS) data, which focus mainly on finding frequent purchase patterns and discovering general sales trends (Hamuro et al. 1998; Thomas and Chakravarthy 2000).
Modeling the effect of price promotion, which is studied in marketing science (Raju
1992; Nijs et al. 2001), is also related to our work. However, the aim is different fromours, which is to model purchase behavior for each user. Purchase data are often used
to enable recommendation methods to learn preference, or purchase behavior, for eachuser (Shani et al. 2005; Iwata et al. 2006; Li et al. 2009). However, these methods do
not take price information into account.

5 Experimental Results
We evaluated the proposed topic model using real purchase data with price information. The data consist of records in an online service for managing household accounts
from February 2009 to November 2010 in Japan. The user can keep his/her household
accounts by recording the purchased items with their prices in the online service. The
data were noisy; since the users can freely choose the name of an item, an item can bereferred to by different names among users, and different items can be referred to by
the same name. For simplicity, we assumed that if the names were different, the items
were different. To protect privacy, we omitted items that were purchased by fewer
than 300 users. We also omitted users who purchased fewer than 30 items. The dataconsisted of 5, 052 items, and 48, 427 users.
Table 2 is the topic extraction result, which shows ten items that are the most likely
to be purchased in each topic. They are estimated with the proposed model using 100
topics. We translated the Japanese names into English. The right and left values ineach column represent the mean price in yen and its standard deviation for the itemin the topic. We can see that related items are clustered, for example, Topic1 is about
drink, Topic2 is about baby supplies, and Topic3 is about food. For these topics, itemsin the same product category are clustered. Some other topics represent lifestyle, anditems from different categories are clustered. For example, Topic5 contains stockings,
cosmetics and magazines, which fashion conscious women are likely to buy. Topic6
includes beer, pachinko and cigarettes, which are items men who like gambling mightbuy. Topic11 contains drinking party, taxi and cleaning, which are items company
workers are likely to buy. From the topic extraction result, we can also see that there
are some topic specific price ranges. Certain items have different price ranges depending on topics. For example, the price of lunch in Topic5 is 1,189 yen, and that in
Topic7 is 757 yen; this indicates that fashion conscious women spend more moneyon lunch in fancy restaurants. In this purchase data, items with the same name might
indicate different things, because users can freely choose the name of an item. We candistinguish them from the result. For example, curry and ramen (noodles) in Topic4
indicate that users have eaten them in restaurants; those in Topic9 indicate that users

123Analyzing purchase data with price information
Table 2 Some examples of extracted topics obtained with the proposed method
Topic 1

Topic 2

Topic 3Canned coffee

103

17

Nappy

1181

253

Sprout35

7Vegetable juice

90

18

Diaper

1158

235

Natto

93

19

Cocktail

173

25

Parts

293

124

Tofu

98

31

canned beer

207

97

Baby wipe

416

216

Cucumber

54

13

Dietary supplement

123

53

Weaning food

152

79

Enokidake

71

21

Canned juice

127

54

Latte

306

148

Onion

104

51
45

Tea

96

8

Daiso

123

80

Carrot

127

Ice

165

77

Baby food

213

122

Chivee96

7

Yogurt

46

19

Milk

1462

533

Kimchi251

78

Sweet bun

206

62

Sweet bun

179

48

Shiitake

157

45

734

334

Topic 4

Topic 5Topic 6

Box lunch

416

128

Stocking

646

278

Beer

Ramen

726

185

Box lunch

469

142

Pachinko

Curry

627

295

Magazine

624

130

Cigarette

790

289

Wheat noodle

473

154

Lunch

1189

363

Juice

365

156

Consumables

320

385

Beauty wash

1615

1037

Sushi

1965 1310

Green tee

154

101

Pasta

961

416

Box lunch

747

254

Beef-on-rice

394

122

Chocolate

182

115

Low-malt beer

611

214

Noodle539

266

Eyelash liner

1307

363

Lottery

2013 1712

Coffee333

154

Napkin244

97

Grilled beef

6941 3350

Pasta

453

208

Tights823

443

Prepared food

1874 1138

Topic 7

Topic 817864 15353

Topic 9

Lunch

757

352

Coffee

104

27

Natto

76

17

Coffee303

132

Green tea

119

31

Wheat noodle

69

56

Magazine

360

165

Tea

116

51

Tofu

66

16

Tea

456

255

Ice coffee

103

17

Ice

172

60

Dinner1884 1532

Vegetable juice

178

58

Ramen

174

96

Food shopping

1208

941

Sports drink

134

40

Toilet tissue

262

65

Subway

310

267

Bread

177

35

Curry

128

73

Snack

599

209

Calpis

150

60

Pasta

178

92

Haircut

5722 2996

Snack

117

27

Toothbrush

110

47

Taxi

2508 1090

Ice-cream

316

141

Chocolate

169

88

Topic 10

Topic 11

Topic 12

Socks

689

367

Drink party

3499

1552

Pudding

100

29

Bread

218

92

Taxi

918

397

Ice

215

70

Pants

1134 1028

Shirt cleaning

141

146

Cream puff

99

27

T-shirts

908

530

Rental DVD

218

186

Rice cracker

145

86

Juice

227

108

Drink money

6016

3490

Cookie

135

67

123

T. Iwata, H. Sawada
Table 2 Continued
Topic 10

Topic 11

Topic 12

One-piece

2293

1783

Golf

9266

5305

Sushi

495

Sandal

1557

1160

Cleaning

902

625

Sweet bun

104

450
31

T-shirts

1058

645

Bullet train

7492

4014

Snack

97

15

Movie

1173

360

Mobile charge

5332

2885

Cheese203

78

Legging697

342

Water charge

2465

1106

Chips

97

23

Each column represents a topic, which shows ten items that are most likely to be purchased. The right andleft values in each column represent the mean price in yen and the standard deviation of the item in the
topic. The same item names with different price ranges are in bold

Fig. 3 Accuracies of predicting
the age when using topic
proportions as input

0.49

accuracy

0.48

0.47

Proposed
LDA
Baseline
0.46
10

20

30

40

50

60

70

80

90

100

#topics

have bought uncooked curry and ramen in grocery stores. We can determine this fromthe price range and other items in the topic. In these ways, the proposed model cancluster items taking their price ranges into consideration.
For a quantitative evaluation measure, we calculated the accuracy of predicting
the user’s age when using topic proportions as input. The ages of some users were
provided. Age was divided into sections, where the width of each section was ten
years. We predicted the age section of users using k-nearest neighbor classifiers. For
finding neighbors, we used Kullback–Leibler (KL) divergence between topic proportions that were estimated with the proposed model. In particular,
neighbors u of userK
u were selected by minimizing the following equation, k=1 θuk log θθuk . The highuk
prediction accuracy implies that the estimated topic proportions appropriately represent intrinsic characteristics of the user. We compared the proposed model and LDA
by using their topic proportions as input. We generated 100 sets of training data, eachof which contained the purchase logs of 3,000 users that were randomly sampled.
Figure 3 shows the results obtained with different numbers of topics, where we set the
number of neighbors at k = 100. The baseline plot represents the accuracies when
the set of purchased items was used as the input of the k-nearest neighbor classifiers. The proposed model did not achieve the significant improvement over the LDA;
the standard deviation bars of the proposed model overlapped with the accuracies of123

Analyzing purchase data with price information
Fig. 4 KL divergences of pricedistributions between different
topics. The x-axis is the numberof topics

100
Proposed
LDA

KL divergence

10

1

0.1

0.01
10

20

30

40

50

60

70

80

90

100

#topics

LDA. However, the proposed model outperformed LDA for all numbers of topics, andachieved significant improvement over the baseline method. This result indicates thatthe proposed model can learn information about users adequately by incorporating
price information into topic models.
We might obtain the topic specific price ranges by calculating that means and standard deviations after extracting topics using LDA. We compared this approach withthe proposed model in terms of specificity of price ranges for each topic. The specificity is evaluated by using the following average KL divergence of price distributions
between different topics,1
I K(K − 1)IK

i=1 k=1 k  =k

P (v|μki , λ−1ki ) log

P (v|μki , λ−1ki )P (v|μk  i , λ−1k i )

dv,(22)

which is averaged over all items and all pairs of topics. The specificity becomes highwhen the price distributions differ depending on topics. Figure 4 shows the result. The
KL divergences of the proposed model were higher than those of LDA for all numbersof topics. This result implies that we can extract characteristic topics that have specificprice ranges by inferring topics using both item and price information.
Figure 5 shows the log joint likelihoods (1) over iterations in the inference with different numbers of topics (K = 10, 30, 50, 100). As the number of iterations increases,
the log likelihood increases, and eventually converges to a certain point, although there
are small fluctuations because the inference is based on Gibbs sampling. This result
indicates that the inference procedures described in Section 3 can appropriately findlatent topics and parameters that fit with the given purchase log data. The log likelihoods drastically increases in the small number of iterations.
6 Conclusion
We have proposed a topic model for analyzing purchase data with price information and its efficient inference procedure. We have confirmed experimentally that the123

T. Iwata, H. Sawada(a) -4.9e+06-5e+06log joint likelihood

Fig. 5 Log joint likelihoods
over iterations with different
numbers of topics (a), and its
zoom of 0 to 100 iterations (b)-5.1e+06-5.2e+06-5.3e+06-5.4e+06
K=100
K=50
K=30
K=10

-5.5e+06-5.6e+06-5.7e+06-5.8e+06

0

500

1000

1500

2000

2500

3000

#iterations(b)-5e+06log joint likelihood-5.1e+06-5.2e+06-5.3e+06-5.4e+06-5.5e+06
K=100
K=50
K=30
K=10

-5.6e+06-5.7e+06-5.8e+06

10

20

30

40

50

60

70

80

90 100#iterations

proposed model can learn intrinsic user characteristics, and can cluster related itemswith their prices.
Although our results have been encouraging to date, our model can be further
improved in a number of ways. First, we can automatically infer the number of topics by extending the model to a nonparametric Bayesian model such as hierarchical
Dirichlet processes (Teh et al. 2006a). Second, we can incorporate information otherthan price into topic models to allow us to analyze purchase data. Purchase data might
contain other information, such as purchase times, stores, types of payments and user
attributes. Third, the proposed model is applicable to recommender systems that cansuggest items that match with the user’s preference and whose prices coincide withthe user’s money sense. Most of the existing work on recommender systems focuseson the recommendation of items within the same category, such as movies, books andweb pages, where the prices are the same or vary little. However, when recommending diverse items, such as food, electrical products and furniture, price information
becomes important since their prices vary widely. Fourth, the inference procedurecan be extended further to reduce the computational time by using an online learning
approach (Hoffman et al. 2010; Sato et al. 2010) and parallelization (Newman et al.
2007). Finally, we would like to evaluate our model further by applying it to other realpurchase data sets.

123

Analyzing purchase data with price information

Appendix A
Derivation of (11)In this appendix, we give the derivation of (11).
P (zj = k|X, V , Z\j )∝ P (zj = k, xj , vj |X \j , V\j , Z\j )= P (zj = k|Z\j )P (xj |X \j , zj = k, Z\j )P (vj |xj , X \j , V\j , zj = k, Z\j ), (23)The first factor of (23) becomes,
P (zj = k|Z\j )
= P (zj = k|θu )P (θu |Z\j )dθu
Γ (Nu\j + αK)  Nuk \j +α−1
= θuk θuk dθuk  Γ (Nuk  \j + α) k Γ (Nu\j + αK)
N  +α  Nuk  \j +α−1
= 
θuk 
dθu
θukuk \jk  Γ (Nuk  \j + α)k  =kΓ (Nu\j + αK) Γ (Nuk\j + α + 1) k  =k Γ (Nuk  \j + α)= 
Γ (Nu\j + αK + 1)
k  Γ (Nuk  \j + α)
Nku\j + α,=
Nu\j + αK(24) αk −1Γ (αk )k
where we usedθdθ=
in the fourth equation, which is the nork kΓ ( k αk )malizing constant of the Dirichlet distribution, and Γ (x + 1) = xΓ (x) in the fifth
equation. In a similar way, the second factor of (23) becomes,

P (xj |X \j , zj = k, Z\j )
= P (xj |φk )P (φk |X \j , Z\j )dφk
Γ (Nk\j + βI )  Nki  \j +β−1
= φkxj φki 
dφki  Γ (Nki  \j + β) i 
Nkx \j +β  Nki  \j +β−1
Γ (Nk\j + βI )
= 
θki 
dφk
θkxj ji  Γ (Nki  \j + β)i  =xj
Γ (Nk\j + βI ) Γ (Nkxj \j + β + 1) i  =xj Γ (Nki  \j + β)= Γ (Nk\j + βI + 1)
i  Γ (Nki  \j + β)
Nkxj \j + β=.
Nk\j + βI

(25)

123

T. Iwata, H. Sawada

The third factor of (23) becomes,
P (vj |xj , X \j , V\j , zj = k, Z\j ) =P (vj |μkxj , λ−1kxj )P (μkxj |X \j , V\j , λkxj )P (λkxj |X \j , V\j )dμkxj dλkxj −1=
N (vj |μkxj , λ−1kxj )N (μkxj |ηkxj \j , (λkxj γkxj ) )G(λkxj |akxj \j , bkxj \j )dμkxj dλkxj
1=

γkx2 j \j bkxjj\j  akx \j2πΓ (akxj \j )akxj \j ∝

akx \jλkxjj

Γ (akxj ) bkxj \jγkxj \jΓ (akxj \j ) bakxj

γkxj

kxj



λkxj γkxj(μkxj − ηkxj )2 exp(−λkxj bkxj )dμkxj dλkxj
exp −
2
1
2

,

(26)


 1

2 dμ = 2π 2 and λa−1 exp(−λb)dλ = Γ (a)(μ−η)
where we used exp − λγ2λγbain the third equation, which are the normalizing constant of the Gaussian and Gamma
distributions, respectively.

References
Blei DM, Jordan MI (2003) Modeling annotated data. In: SIGIR ’03, pp 127–134
Blei DM, Lafferty JD (2006) Dynamic topic models. In: ICML ’06, pp 113–120
Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022
Das AS, Datar M, Garg A, Rajaram S (2007) Google news personalization: scalable online collaborative
filtering. In: WWW ’07, pp 271–280
Fu W, Song L, Xing EP (2009) Dynamic mixed membership block model for evolving networks. In: ICML
’09, pp 329–336
Griffiths TL, Steyvers M (2004) Finding scientific topics. Proc Nat Acad Sci 101(1):5228–5235
Hamuro Y, Katoh N, Matsuda Y, Yada K (1998) Mining pharmacy data helps to make profits. Data Min
Knowl Disc 2(4):391–398
Hofmann T (1999) Probabilistic latent semantic analysis. In: UAI ’99, pp 289–296
Hofmann T (2003) Collaborative filtering via Gaussian probabilistic latent semantic analysis. In: SIGIR’03. ACM Press, New York, pp 259–266
Hoffman M, Blei D, Bach F (2010) Online learning for latent Dirichlet allocation. In: NIPS ’10
Iwata T, Saito K, Yamada T (2006) Recommendation methods for extending subscription periods. In: KDD’06, pp 574–579
Iwata T, Watanabe S, Yamada T, Ueda N (2009) Topic tracking model for analyzing consumer purchasebehavior. In: IJCAI ’09, pp 1427–1432
Iwata T, Yamada T, Sakurai Y, Ueda N (2010) Online multiscale dynamic topic models. In: KDD ’10, pp
663–672
Jin X, Zhou Y, Mobasher B (2004) Web usage mining based on probabilistic latent semantic analysis. In:
KDD ’04, pp 197–205
Li M, Dias BM, Jarman I, El-Deredy W, Lisboa PJ (2009) Grocery shopping recommendations based onbasket-sensitive random walk. In: KDD ’09, pp 1215–1224
Mimno D, Wallach HM, Naradowsky J, Smith DA, McCallum A (2009) Polylingual topic models. In:
EMNLP ’09, pp 880–889
Minka T (2000) Estimating a Dirichlet distribution. Technical report. MIT, Cambridge
Minka T, Lafferty J (2002) Expectation-propagation for the generative aspect model. In: UAI ’02, pp
352–359123

Analyzing purchase data with price information
Newman D, Asuncion A, Smyth P, Welling M (2007) Distributed inference for latent Dirichlet allocation.
In: NIPS ’07, pp 1081–1088
Nijs VR, Dekimpe MG, Steenkamps JBE, Hanssens DM (2001) The category-demand effects of pricepromotions. Market Sci 20(1):1–22
Raju JS (1992) The effect of price propotion on variability in product category sales. Market Sci 11(3):207–
220
Rosen-Zvi M, Griffiths T, Steyvers M, Smyth P (2004) The author-topic model for authors and documents.
In: UAI ’04, pp 487–494
Sato I, Kurihara K, Nakagawa H (2010) Deterministic single-pass algorithm for lda. In: NIPS ’10
Shani G, Heckerman D, Brafman RI (2005) An MDP-based recommender system. J Mach Learn Res
6:1265–1295
Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Hierarchical Dirichlet processes. J Am Stat Assoc
101(476):1566–1581
Teh YW, Newman D, Welling M (2006) A collapsed variational Bayesian inference algorithm for latent
Dirichlet allocation. In: NIPS ’06, 1378–1385
Thomas S, Chakravarthy S. (2000) Incremental mining of constrained associations. In: HIPC ’00, 547–558
Wallach HM (2006) Topic modeling: Beyond bag-of-words. In: ICML ’06, 977–984
Wang X, McCallum A (2006) Topics over time: a non-Markov continuous-time model of topical trends.
In: KDD ’06, 424–433
Williamson S, Wang C, Heller K, Blei D (2010) The IBP-compound dirichlet process and its application
to focused topic modeling. In: ICML ’10, 1151–1158

123