Topic Tracking Model for Analyzing Consumer Purchase Behavior
Tomoharu Iwata
Shinji Wanatabe Takeshi Yamada Naonori Ueda
NTT Communication Science Laboratories,
2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto, Japan{iwata,watanabe,yamada,ueda}@cslab.kecl.ntt.co.jp
Abstract
We propose a new topic model for tracking timevarying consumer purchase behavior, in which consumer interests and item trends change over time.
The proposed model can adaptively track changes
in interests and trends based on current purchaselogs and previously estimated interests and trends.
The online nature of the proposed method means
we do not need to store past data for current inferences and so we can considerably reduce the computational cost and the memory requirement. We
use real purchase logs to demonstrate the effectiveness of the proposed method in terms of the prediction accuracy of purchase behavior and the computational cost of the inference.

1

Introduction

Modeling consumer purchase behavior in e-commerce is an
important task, because it enables online stores to provide
recommendations, personalized advertisements, and perform
trend analyses. Recently there has been great interest in analyzing discrete data using topic models, where the discrete
data are mainly text corpora but also include human behavior
log data [Jin et al., 2004; Das et al., 2007]. A topic model
is a hierarchical Bayesian model, in which a user (document)
is modeled as a mixture of topics, where a trend in a topic
is modeled as a purchase probability distribution over catalog items (words). However, traditional topic models do not
fulfill two important requirements for modeling consumer behavior in online stores. The first is the adaptability to changes.
Consumer interests and item trends change over time for avariety of reasons. The second is the ease with which incremental model updates can be achieved. Huge amounts of data
are accumulated every day in real online stores.
In this paper, we propose a topic model, which we callthe Topic Tracking Model (TTM), that satisfies the above two
requirements. With TTM, interests for each consumer andtrends for each topic are sequentially inferred using newly obtained data and previously estimated interests and trends. The
computational cost and the memory requirement of TTM are
low because past data are not required for the inference andneed not be stored.

The persistency of interests differs among consumers.
Some consumers significantly change their interests and some
do not. Furthermore, it also differs according to periods.
There may be periods during which a consumer’s interestschange greatly and there may also be long periods when theseinterests remain unchanged. Therefore, we need to infer theinterest persistency for each consumer and for each time period. We also infer the trend persistency for each topic andfor each time period because it differs among topics and overtime according to new releases of items, seasons, and socialand economic environments.
Probabilistic Latent Semantic Analysis (PLSA) [Hofmann,
1999] and Latent Dirichlet Allocation (LDA) [Blei et al.,2003] are two representative topic models. However, thesemodels assume that samples are exchangeable, and therefore they cannot capture the dynamics. More recently, topic
models with dynamics, such as the Dynamic Topic Model(DTM) [Blei and Lafferty, 2006], Dynamic Mixture Model(DMM) [Wei et al., 2007], and Topic over Time (ToT) [Wangand McCallum, 2006], have been proposed. TTM has an advantage over the other models in its adaptability to changes
because TTM models dynamics of both interests and trends,
and it infers their persistency for each time interval from thegiven data.
In the rest of this paper, we assume that the given data arepurchase logs. However, our method is applicable to a wide
range of human behavior data with timestamps, such as web
access log, blog, e-mail and discourse data.

2

Preliminaries

The goal of this paper is to estimate the probability that useru purchases item i at time t, P (i|u, t), which represents thepurchase behavior of user u, where u ∈ {1, · · · , U } is a user
index, and i ∈ {1, · · · , I} is an item index. We assume thatt is a discrete variable, and we can set the unit time interval
arbitrarily at, for example, one day or one week.
In topic modeling, P (i|u, t) is obtained by the following
equation with the assumption of the conditional independenceof u and i given latent topic z,
P (i|u, t) =ZX

φt,u,z θt,z,i ,

(1)

z=1

where Z is the number of topics, φt,u,z = P (z|u, t) rep-

resents the interests of a user, which is the probability thatuser
P u is interested in topic z at time t, where φt,u,z ≥ 0,z φt,u,z = 1, and θt,z,i = P (i|z, t) represents trends in a
topic, which is the probability that item i is selected
P and purchased from topic z at time t, where θt,z,i ≥ 0, i θt,z,i = 1.
In LDA, interests φt,u = {φt,u,z }Z
z=1 are assumed to havea Dirichlet prior as follows:
Y γ −1z
P (φt,u |γ) ∝φt,u,z,(2)(b) For each purchase m = 1, · · · , Mt,u :i. Draw topic
zt,u,m ∼ Multinomial(φt,u )ii. Draw item
xt,u,m ∼ Multinomial(θ t,zt,u,m ).
As in LDA, each item xt,u,m is sampled from a topic-specificmultinomial distribution.
Figure 1 (a) shows a graphical model representation of
TTM, where shaded and unshaded nodes indicate observed
and latent variables, respectively.

where γ = {γz }Z
z=1 , γz > 0, is a set of Dirichlet parameters.
In this assumption, current interests are independent of thepast interests, and their dynamics are not incorporated.

4

z

3

Topic Tracking Model

To model the dynamics of user interests, we assume that themean of the user interests at the current time are the sameas those at a previous time unless otherwise confirmed by
the newly observed data. In particular, we use the following Dirichlet prior, in which the mean of the current interestsis the same as the mean of the previous interests φ̂t−1,u , andthe precision is αt,u ,Y α φ̂
t,u t−1,u,z −1
P (φt,u |φ̂t−1,u , αt,u ) ∝φt,u,z,(3)zwhere Dirichlet parameters in (2) are factorized into the meanand precision γt,u,z = αt,u φ̂t−1,u,z . The precision αt,u represents the interest persistency, which is a measure of how
consistently user u maintains her interests at time t comparedwith those at t − 1. We estimate αt,u for each time period
and for each user because the variability of interests depends
on both time and user. Since this is a conjugate prior, the inference is simple as in LDA, which is a topic model without
dynamics [Griffiths and Steyvers, 2004].
To model the dynamics of trends, we use the following
Dirichlet distribution for the prior of trends θ t,z = {θt,z,i }Ii=1
in the same way as interests,
Y β θ̂
t,z t−1,z,i −1
P (θ t,z |θ̂ t−1,z , βt,z ) ∝θt,z,i,(4)iwhere the mean is the previous trends θ̂ t−1,z , and the precision is βt,z . This prior is also conjugate.
Mt,u
Let xt,u = {xt,u,m }m=1
be a set of items purchased byuser u at time t, where xt,u,m is the mth item purchased byuser u at time t, and Mt,u is the number of items purchasedby user u at time t. TTM assumes the following generative
process for X t = {xt,u }U
u=1 , which is a set of all items purchased by U users at time t,
1. For each topic z = 1, · · · , Z:(a) Draw trends
θ t,z ∼ Dirichlet(βt,z θ̂ t−1,z )2. For each user u = 1, · · · , U :(a) Draw interests
φt,u ∼ Dirichlet(αt,u φ̂t−1,u )Inference

We estimate the parameters in TTM based on stochastic EM
algorithm, in which Gibbs sampling of latent topics and maximum joint likelihood estimation of parameters are alternately
iterated [Wallach, 2006].
Let t be the current time. Suppose that we have purchaselogs X t at the current time t, the mean of the previous interests Φ̂t−1 = {φ̂t−1,u }U
u=1 , and the mean of the previous trends Θ̂t−1 = {θ̂ t−1,z }Z
z=1 . By using these data andparameters, we estimate current interests Φt = {φt,u }U
u=1
and trends Θt = {θ t,z }Z
z=1 as well as interest persistenciesZ
αt = {αt,u }U
u=1 and trend persistencies β t = {βt,z }z=1 .
We infer latent topics based on the collapsed Gibbs sampling [Griffiths and Steyvers, 2004], in which the joint distribution of data and latent topics is required. Let Z t ={z t,u }U
u=1 be a set of latent topics of all users at time t, andMt,uz t,u = {zt,u,m }m=1
. Since we use conjugate priors for parameters Φt and Θt , we can integrate out the parameters in
the joint distribution as follows:
P (X t , Z t |Φ̂t−1 , Θ̂t−1 , αt , β t )= P (Z t |Φ̂t−1 , αt )P (X t |Z t , Θ̂t−1 , β t )Z=
P (Z t , Φt |Φ̂t−1 , αt )dΦt
Z×P (X t , Θt |Z t , Θ̂t−1 , β t )dΘtQY
Γ(αt,u )z Γ(nt,u,z + αt,u φ̂t−1,u,z )=QΓ(nt,u + αt,u )z Γ(αt,u φ̂t−1,u,z )uQYΓ(βt,z )i Γ(nt,z,i + βt,z θ̂t−1,z,i )×,(5)Q
Γ(nt,z + βt,z )i Γ(βt,z θ̂t−1,z,i )zwhere nt,u,z is the number of items purchased by user u attime t that have been assigned to topic z, nt,z,i is the numberof times item i has been assigned to topic z at time t, andΓ(x) is the gamma function.
The assignment of a latent topic is estimated using the following equation, which is calculated from (5),P (zj = k|X t , Z t\j , Φ̂t−1 , Θ̂t−1 , αt , β t )∝nt,u,k\j + αt,u φ̂t−1,u,k nt,k,xj \j + βt,k θ̂t−1,k,xj,(6)nt,u\j + αt,unt,k\j + βt,kwhere j = (t, u, m), and \j represents a count that does notinclude a purchase j.

t-1

t

t-2

t-1

t

t-1

t

t-1

t

α

α

α

α

α

γ

γ

φ

φ

φ

φ

φ

φ

φ

φ

φ

z

z

z

z

z

z

z

z

z

x

M

x
U

θ

β

x
M

β

M

θ

θ

Z

U

Z

(a) TTM

β

x
U

M

θ

Z

β

x
U

M

U

x

θ

Z

β

θ
Z

(b) TTM with long dependences

M

x
U

Z

θ

x

M

x

M
θ

U

M

Z

γ

Z

(c) DTM(d) DMM

Figure 1: Graphical model representations of Topic Tracking Model (TTM), Dynamic Topic Model (DTM), and Dynamic
Mixture Model (DMM).
The persistency parameters αt and β t are estimated by
maximizing the joint distribution (5). The following updating rules for maximizing the joint distribution are derived byusing the bounds in [Minka, 2000],Pz φ̂t−1,u,z At,u,z
αt,u ← αt,u(7)
Ψ(nt,u + αt,u ) − Ψ(αt,u )of yesterday but also on the interests of a week ago. Therefore, to capture the long term dependences, we modify theprior of interests φt,u so that it can depend on past L interests{φ̂t−l,u }L
l=1 as follows:

where At,u,z = Ψ(nt,u,z + αt,u φ̂t−1,u,z ) − Ψ(αt,u φ̂t−1,u,z ),and Ψ(·) is the digamma function defined by Ψ(x) =∂ log Γ(x), and,∂xPi θ̂t−1,z,i Bt,z,i
βt,z ← βt,z,(8)
Ψ(nt,z + βt,z ) − Ψ(βt,z )

(9)

(11)
where the mean is proportional to the weighted sum of thepast L interests of the user, and αt,u,l represents how the interests at t are related to the l-previous interests. The information loss is reduced by considering long term dependences.
Furthermore, the bias of the inference is reduced by using
multiple estimates. When we use only the previous interests,
the inference might be biased because it depends on just one
estimate.
In a similar fashion, we can also modify the prior of thetrends θ t,u so that it can depend on past L trends {θ̂ t−l,u }L
l=1
in order to make the inference more robust as follows:
Y P βt,z,l θ̂t−l,z,i −1l)∝
P (θ t,z |{θ̂ t−l,z , βt,z,l }L
θt,z,i, (12)
l=1

nt,z,i + βt,z θ̂t−1,z,iθ̂t,z,i =.
(10)nt,z + βt,zThese estimates are used to estimate interests and trends at
the next time period t + 1.
TTM uses only current data X t (not past data) when inferring current interests Φt and trends Θt , Therefore, TTM
can greatly reduce the computational cost compared with theconventional models that use past data. Moreover, TTM also
reduces the memory requirement because past data need not
be stored.

where the mean is proportional to the weighted sum of thepast L trends of the topic, and βt,z,l represents how the trends
at t are related to the l-previous trends. A graphical model of
TTM with long term dependences is shown in Figure 1 (b).
Since the priors in (11) and (12) are still Dirichlet distributions and conjugate priors, the parameters Φt and Θt in
the joint distribution can be integrated out as in (5). Therefore, the inference can be achieved by using a stochastic EM
algorithm in the same way with the model described in theprevious section. Each latent topic is sampled according to
the following probability,

where Bt,z,i = Ψ(nt,z,i + βt,z θ̂t−1,z,i ) − Ψ(βt,z θ̂t−1,z,i ). By
iterating Gibbs sampling with (6), and maximum likelihood
estimation with (7) and (8), we can estimate latent topics Z t ,and parameters αt and β t .
After the iterations, the means of φt,u,z and θt,z,i are obtained as follows:
φ̂t,u,z =5nt,u,z + αt,u φ̂t−1,u,z,
nt,u + αt,uCapturing long term dependences

In the above discussion, we infer interests φt,u using an estimate of the previous interests φ̂t−1,u . However in practice,the interests of today may depend not only on the interests

P (φt,u |{φ̂t−l,u , αt,u,l }Ll=1 ) ∝

Y

P

α

φ̂

l t,u,l t−l,u,z
φt,u,z

−1

,

z

i

P (zj = k|X t , Z t\j , {Φ̂t−l , Θ̂t−l , αt,l , β t,l }L
l=1 )
P
P
nt,u,k\j + l αt,u,l φ̂t−l,u,k nt,k,xj \j + l βt,k,l θ̂t−l,k,xj
PP,∝
nt,u\j + l αt,u,lnt,k\j + l βt,k,l(13)

Z
where αt,l = {αt,u,l }U
u=1 , and β t,l = {βt,z,l }z=1 . The update rule of weight αt,u,l is as follows:P
φ̂t−l,u,z A0t,u,zPzP
αt,u,l ← αt,u,l,Ψ(nt,u + l0 αt,u,l0 ) − Ψ( l0 αt,u,l0 )(14)Pwhere A0t,u,z=Ψ(nt,u,z +
α
φ̂
)
−
t,u,l
t−l,u,zlPΨ( l αt,u,l φ̂t−l,u,z ). The update rule of weight βt,z,lis as follows:P0θ̂t−l,z,i Bt,z,iPiP
βt,z,l ← βt,z,l, (15)
Ψ(nt,z + l0 βt,z,l0 ) − Ψ( l0 βt,z,l0 )P0where Bt,z,i=Ψ(nt,z,i +l βt,z,l θ̂t−l,z,i ) −PΨ( l βt,z,l θ̂t−l,z,i ). The means of φt,u,z and θt,z,i are
obtained as follows:P
nt,u,z + l αt,u,l φ̂t−l,u,zP,(16)
φ̂t,u,z =nt,u + l αt,u,lP
nt,z,i + l βt,z,l θ̂t−l,z,iPθ̂t,z,i =.
(17)nt,z + l βt,z,lThese estimates are efficiently used for the inference fromt + 1 to t + L.

6

Related work

Topic models with dynamics have been proposed recently.
These include the Dynamic Topic Model (DTM) [Blei and
Lafferty, 2006], Dynamic Mixture Model (DMM) [Wei etal., 2007], and Topic over Time (ToT) [Wang and McCallum, 2006]. A graphical model of DTM is shown in Figure 1 (c). DTM analyzes the time evolution of topics in document collections, in which a document is assumed to have
only one timestamp. Therefore, DTM does not consider the
dynamics in each document (user), and it cannot be used formodeling user behavior dynamics, which is the goal of thispaper. Since DTM uses a Gaussian distribution for the dynamics, the inference is intractable owing to the nonconjugacy of the Gaussian and multinomial distributions, and it
requires complicated variational approximations. In contrast
to DTM, the inference of TTM is relatively simple because
it uses Dirichlet distributions that are conjugate for multinomial distributions. A graphical model of DMM is shown in
Figure 1 (d). DMM only considers a single dynamic sequenceof documents, which corresponds to a single user over time.
On the other hand, TTM considers multiple users with different dynamics as well as the dynamics of trends for multiple
items in each topic, which is essential for analyzing consumerpurchase behavior. Furthermore, DMM assumes that the interest persistency is fixed and known. Therefore, it is not
appropriate for purchase logs, in which distributions sometimes change greatly, and are sometimes stable. ToT needs all
the samples over time for the inference. Therefore, it cannot
be updated sequentially, and is not appropriate for data thatare continuously accumulated. Singular Value Decomposition (SVD) is also used for analyzing multiple timeseries [Papadimitriou et al., 2005] as well as topic models. However,since SVD implicitly assumes Gaussian noise, it is not appropriate for discrete data [Hofmann, 1999], such as text corpora
and purchase logs.
TTM is related to the Kalman filter [Welch and Bishop,
1995] even though TTM deals with discrete variables andthe Kalman filter deals with continuous variables. In the
Kalman filter, since both the transition and emission distributions are Gaussian, and the conjugate distribution of theGaussian is also Gaussian, the inference is tractable. On theother hand, with topic models, since the emission distribution is the multinomial, and the conjugate distribution of themultinomial is the Dirichlet, the inference in the same way
is infeasible. DTM is inferred as with the Kalman filter by
mapping the Gaussian to the multinomial using the softmax
function. In contrast, TTM is inferred by assuming that thetransition distribution is the Dirichlet, in which the mean isthe previous mean of the multinomial, and the variance isadaptively estimated from the given data.

7

Experiments

We evaluated TTM using two real purchase log data sets consisting of movie and cartoon data. The movie data are the logs
of a movie downloading service from May 14th to August
31st, 2007, in which the numbers of users, items, and transactions are 70,122, 7,469, and 11,243,935, respectively. The
cartoon data are the logs of a cartoon downloading service forcell phones from January 1st to May 31st, 2006, in which thenumbers of users, items, and transactions are 143,212, 206,
and 12,642,505, respectively. We considered cartoons withdifferent volumes to be the same item. We omitted items andusers that appeared fewer than ten times. We set the unit timeinterval at one day.
For the evaluation measurement, we employed the N -best
prediction accuracy of purchase items. We used data until theprevious day t − 1 as training data to infer the model, andpredicted purchase items at day t as test data. The probability
that user u purchases item i at time t was calculated using theestimated parameters at the previous day, Φ̂t−1 and Θ̂t−1 , asfollows:
X
P (x = i|u, t; Φ̂t−1 , Θ̂t−1 ) =φ̂t−1,u,z θ̂t−1,z,i . (18)z

The N -best accuracy represents the percentage that thepurchased items are contained in the set of N -highest
P (x=i|u, t; Φ̂t−1 , Θ̂t−1 ) items.
In order to demonstrate the effectiveness of modeling the
dynamics of interests and trends, we compared TTM withthree topic models, LDAall, LDAonline, and LDAone, which
are based on LDA. LDAall is an LDA that uses all the pastdata for inference. LDAonline is an online learning extensionof LDA, in which the parameters are estimated using those of
the previous day and the newly obtained data [Banerjee andBasu, 2007]. LDAone is an LDA that uses just the previous
day data for inference. For a fair comparison, the hyper parameters in these LDAs were optimized using stochastic EM
as described in [Wallach, 2006].
The N -best accuracies for movie and cartoon data sets averaged over time are shown in Table 1. TTM1 and TTM10

12

N
1
2
3
4
5

(b) cartoon
LDAall LDAonline LDAone TTM1
TTM1027.0 (3.3) 26.0 (3.5) 24.8 (4.5) 26.8 (4.2) 30.5 (3.4)
37.3 (3.6) 35.1 (4.2) 32.4 (4.9) 34.2 (4.5) 39.9 (3.5)
43.7 (3.9) 41.1 (4.8) 37.2 (5.3) 39.8 (4.6) 45.9 (3.3)
48.5 (4.0) 45.8 (5.1) 40.9 (5.3) 44.5 (4.6) 50.6 (3.2)
52.4 (4.2) 49.6 (5.4) 44.1 (5.4) 48.5 (4.6) 54.4 (3.0)

TTM10
TTM1
LDAallLDAoneLDAonline

10

accuracy

8
6
4
2
0

20

40

60

80

100

time

(a) movie
60

TTM10
TTM1
LDAallLDAonline
LDAone55
50
45

accuracy

Table 1: Average N -best accuracies (%) over time. The digitin the bracket is the standard deviation.
(a) movie
TTM1
TTM10
N LDAall LDAonline LDAone
1 1.21 (0.61) 1.08 (0.54) 1.91 (0.78) 2.22 (0.91) 2.46 (0.92)
2 2.18 (0.79) 2.00 (0.78) 3.52 (1.22) 3.99 (1.33) 4.47 (1.36)
3 3.06 (1.04) 2.81 (1.02) 5.04 (1.64) 5.60 (1.75) 6.35 (1.85)
4 3.90 (1.27) 3.56 (1.24) 6.24 (1.90) 6.82 (2.01) 7.82 (2.15)
5 4.70 (1.51) 4.26 (1.44) 7.37 (2.20) 7.92 (2.26) 9.20 (2.42)

40
35
30
25
20
15

20

40

60

80

100

120

140

47

(b) cartoon

46
45
accuracy

accuracy

time

6.5
6.4
6.3
6.2
6.1
6
5.9
5.8
5.7
5.6
5.5

Figure 3: Three-best accuracies (%) for each day.

44
43
42
41
40

1

2

3

4

5

6
L

(a) movie

7

8

9

10

39

1

2

3

4

5

6

7

8

9

10

L

(b) cartoon

Figure 2: Average three-best accuracies (%) of TTM withdifferent dependent lengths L.
represent the proposed topic tracking models, which use parameters of one- and ten-previous days, or L = 1 and L = 10,
respectively. For both of the data sets, the highest accuracies
are achieved by TTM10, and this result indicates that TTM10
can appropriately predict consumer behavior by considering
dynamics and efficiently using information in the past data.
The accuracies achieved by LDAall and LDAonline are low
because they do not consider the dynamics. The reason forthe low accuracy of LDAone is that it uses only current dataand ignores the past information. The accuracies achieved by
LDAall and LDAonline for the cartoon data are higher thanthat of LDAone because the cartoon data has smaller dynamics than the movie data. The high accuracies of TTM10 inboth of the data sets that have different dynamics represent
its high adaptability.
Figure 2 shows the average three-best accuracies of TTM
with different dependent lengths L. The accuracy increased
as L increased. This result implies that TTM became robust
by using long term dependences. The accuracy saturated ataround L = 7, and then slightly decreased possibly due tooverfitting. Therefore, we can achieve high accuracy withthese data sets by storing estimated parameters for a week.
The three-best accuracies for movie and cartoon data sets foreach day are shown in Figure 3. The accuracies varied witha period of seven days because new items are released every
week, and the accuracies when new items were released wereTable 2: Average computational time (sec).
LDAall LDAonline LDAone TTM1 TTM10
movie 12,380.3
523.7
503.2 455.3 707.0
cartoon 12,528.4
663.2
674.6 633.9 936.9

low.
Table 2 shows the daily average computational time whenusing a computer with a Xeon5355 2.66GHz CPU. The
computational time for LDAonline, LDAone and TTM areroughly the same since they use the same number of samples for inference. The computational time for LDAall islong although its accuracy was the second highest for the
cartoon data set since LDAall uses all samples for inference.
Moreover, LDAall requires more memory than TTM. TTM10
processed data that consisted of over 100,000 transactions
per day in about 12 minutes, and it can deal with the hugeamounts of data that are observed in real online stores. Figure 4 shows the average computational time for TTM withdifferent dependent lengths L. The computational time increases linearly with L. Note that the time for TTM is muchsmaller than that for LDAall even TTM with L = 10.
Figures 5 and 6, respectively, show the average α and βwith different l values of TTM10. The sum of the values foreach day and for each user or topic are normalized to one.
The parameters decrease as l increases in the movie data set.
This result implies that recent interests and trends are more
informative as regards estimating current interests and trends,
which is an intuitive result. In the cartoon data set, the parameters at l = 7 are high because the same cartoons withdifferent volumes are released weekly.
Figure 7 shows some examples of estimated parameter βof TTM1 for each day with the movie data. The trends ina topic (red) have a high dependence on the previous trends,

900

600
550
500
450

850
800

beta

650

0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

750
700
650

1

2

3

4

5

6

7

8

9

600

10

1

2

3

4

L

5

6

7

8

9

10

0.15
0.14
0.13
0.12
beta

950

700

computational timecomputational time

750

0.1
0.09
0.08
0.07

1

2

3

4

5

L

(a) movie

0.11

6

7

8

9

0.06

10

1

2

3

4

l

(b) cartoon

6

7

8

9

10

l

(a) movie

Figure 4: Average computational time (sec) of TTM with different dependent lengths L.

5(b) cartoon

Figure 6: Average normalized β with different lengths l of
TTM10.
4000

0.2
0.16

alpha

alpha

0.18
0.14
0.12
0.1
0.08
0.06

1

2

3

4

5

6
l

(a) movie

7

8

9

10

0.18
0.17
0.16
0.15
0.14
0.13
0.12
0.11
0.1
0.09
0.08
0.07

3500
3000
2500

beta

0.22

1500
1000
500
0

1

2

3

4

5

6

7

8

9

10

0

20

40

60

80

100

time

l

(b) cartoon

Figure 5: Average normalized α with different lengths l of
TTM10.
and the trends in a topic (green) have a low dependence on
the previous trends. Even in a topic, the dependence changes
day by day. For example, in a topic (red), the dependence on
the day of a new release is low and those of the other days arehigh. In this way, TTM can adapt flexibly to changes.

8

2000

Conclusion

In this paper, we proposed a probabilistic consumer purchasebehavior model, called the Topic Tracking Model (TTM), fortracking the interests of individual consumers and trends ineach topic. We confirmed experimentally that TTM could
estimate purchase behavior appropriately and process large
amounts of data.
In future work, we could determine the unit time interval
and the length of dependence automatically from the given
data. The number of topics would be automatically inferredby extending TTM to a nonparametric Bayesian model suchas the Dirichlet process mixture model [Teh et al., 2006]. We
will evaluate the proposed method further by comparing itwith other methods for modeling dynamics, such as hidden
Markov models, dynamic topic models, and dynamic mixture
models.

References[Banerjee and Basu, 2007] Arindam Banerjee and Sugato Basu.
Topic models over text streams: A study of batch and online unsupervised learning. In SDM ’07, 2007.
[Blei and Lafferty, 2006] David M. Blei and John D. Lafferty. Dynamic topic models. In ICML ’06, pages 113–120, 2006.

Figure 7: Examples of β of TTM1 for each day with themovie data.
[Blei et al., 2003] David M. Blei, Andrew Y. Ng, and Michael I.
Jordan. Latent Dirichlet allocation. Journal of Machine Learning
Research, 3:993–1022, 2003.
[Das et al., 2007] Abhinandan S. Das, Mayur Datar, Ashutosh
Garg, and Shyam Rajaram. Google news personalization: scalable online collaborative filtering. In WWW ’07, pages 271–280,
2007.
[Griffiths and Steyvers, 2004] Thomas L. Griffiths and Mark
Steyvers. Finding scientific topics. Proceedings of the National
Academy of Sciences, 101 Suppl 1:5228–5235, 2004.
[Hofmann, 1999] Thomas Hofmann. Probabilistic latent semantic
analysis. In UAI ’99, pages 289–296, 1999.
[Jin et al., 2004] Xin Jin, Yanzan Zhou, and Bamshad Mobasher.
Web usage mining based on probabilistic latent semantic analysis. In KDD ’04, pages 197–205, 2004.
[Minka, 2000] Thomas Minka. Estimating a Dirichlet distribution.
Technical report, MIT, 2000.
[Papadimitriou et al., 2005] Spiros Papadimitriou, Jimeng Sun, andChristos Faloutsos. Streaming pattern discovery in multiple timeseries. In VLDB ’05, pages 697–708, 2005.
[Teh et al., 2006] Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M.
Blei. Hierarchical Dirichlet processes. Journal of the American
Statistical Association, 101(476):1566–1581, 2006.
[Wallach, 2006] Hanna M. Wallach. Topic modeling: Beyond bagof-words. In ICML ’06, pages 977–984, 2006.
[Wang and McCallum, 2006] Xuerui Wang and Andrew McCallum. Topics over time: a non-Markov continuous-time modelof topical trends. In KDD ’06, pages 424–433, 2006.
[Wei et al., 2007] Xing Wei, Jimeng Sun, and Xuerui Wang. Dynamic mixture models for multiple time-series. In IJCAI ’07,
pages 2909–2914, 2007.
[Welch and Bishop, 1995] Greg Welch and Gary Bishop. An introduction to the Kalman filter. Technical report, University ofNorth Carolina at Chapel Hill, 1995.