Dynamic Segmentationu s i n g M a r k o v -Switching Model
Takuya SATOMURA
Discussion Paper 01-09

April 2001
この研究は「大学院経済学研究科・経済学部記念事業」
基金より援助を受けた、記して感謝する。
Graduate School of Economics andOsaka School of International Public Policy (OSIPP)
Osaka University, Toyonaka, Osaka 560-0043, JapanDynamic Segmentation using Markov-Switching Model
Takuya SATOMURA
Graduate School of Economics, Osaka University
1-7 Machikaneyama, Toyonaka, Osaka, 560-0043, Japan
telephone +81-6-6850-5233

fax +81-6-6850-5277

e-mail:satomura@econ.osaka-u.ac.jp
April 9, 2001

abstract
In this paper, we propose a new methodology of modeling dynamic segmentation.
A probability that one belongs to a segment is deﬁned as a markov process, and aprobability that one chooses a brand is deﬁned as a multinomial logit model. The
estimation of such model has been diﬃcult because of the complicated calculation oflog likelihood. Using Markov-switching model, we can estimate the model parameters
and latent states (=segments) at each purchase occasion.
In the empirical study of using scanner panel data, We have estimated an instability of segment in the instant coﬀee category. We assumed that there are two
preference structures, one is under the inﬂuence of loyalty and the other is not. The
empirical result showed that there are some people that change their own segment
within data period.

keywords: dynamic segmentation, Markov-switching model, state space model,
choice models

Introduction
In the marketing literature, the stability of a segment is one of the most importantcriteria. Most probabilistic choice models in the marketing area assume the stabilityof preference and/or choice processes. However, if the preference and/or choiceprocesses change over time, we will fail to identify the segment using a model basedon the assumption of stability. Modeling the instability of segment is necessary forthe description of the change of structure.
There are two major approaches to take into account the instability of segment.
One approach is manifest change, and the other is latent change(Wedel and Kamakura 1998 p.159). In the manifest change, the segment membership is stable, butchanges may occur in the preference or choice structure of customers in a segment
over time. In the latent change, the preference structure of segments is stable, butchanges may occur in segment size and/or the segment membership of consumers
over time.
In this paper, we will take into account not only that individuals diﬀer in theirpreferences but also that they change a segment which they belong to. So, latentchange is proper for our application.
In a latent class model, mixture components are used as a prior of the segment
size, and assumed that mixture components may change over a time. There are twomain approaches in this class of models, based on diﬀerent reparameterizations of
the priors for the mixture components(Wedel and Kamakura 1998 p.168). One isconcomitant variable model, and the other is latent Markov model.
The concomitant variable models assume that latent class probabilities dependon observable variables(Wedel and Kamakura 1998 p.168; Yang and Allenby 2000).

1

These models need observable variables to deﬁne segment change.
The latent Markov models assume that latent class probabilities depend on
markov-switching, so we do not need concomitant variables to deﬁne segment change(Poulsen 1990; Böckenholt and Langeheine 1996; Ramaswamy 1997). However, if
the number of ﬁnite time period increased, multiple integral is necessary to obtainthe likelihood of the latent Markov model. Those latent Markov models are not
suﬃcient to express the change if the number of ﬁxed time period increases.
To solve this problems, We adopt a non-Gaussian state-space modeling of nonstationary time series (Kitagawa 1987). When the state-space is represented as Markov
process in the state-space model, it is called Markov-switching model(Hamilton
1989). We present our proposed model at the next section.

1

Model

1.1

Modelling Markov-Switching process

Suppose that the hth household in the panel (h = 1, 2, . . . , N ) is faced with a choiceof M brands at the tth purchase occasion (t = 1, 2, . . . , Th ). Following the ﬁnite
mixture formulation of Kamakura and Russell(1989), we also assume that there
exist S market segments. Segment s(= 1, 2, . . . , S) contains households who haverelatively similar preference and responses to marketing mix variables.
Let Sht denote the random variable that takes a value s, and Sht = s when hth
household belongs to segment s at purchase occasion t. Let yht = m denote the eventthat household h chooses brand m at purchase occasion t , and ψht = (yh1 , . . . , yht ).
We assume that evolution of Sht depends upon Sh,t−1 , in which case the processof St is named as first order Markov-switching process.

2

Denote the element of transition matrix from state i to statej by pij , and let
P r(Sht = j|Sh,t−1 = i, ψh,t−1 ) = P r(Sht = j|Sh,t−1 = i) = pijS

0 ≤ pij ≤ 1,(1)

pij = 1.

j=1

1.2

Modeling Brand choice

We assume that choice of a brand in the segment s is governed by a multinomial logit
model. Let xhtm denote the vector of attributes. Then the conditional probability
that brand m is chosen, given Sht = s, is speciﬁed asexp(xhtm β s ).
P r(yht = m|Sht = s) = Mi=1 exp(xhtm β s )(2)where β s is a parameter vector in segment s.
The unconditional probability is given by the weighted sum of segment probabilities.
P r(yht = m|ψh,t−1 ) =S
P r(Sht = s|ψh,t−1 )P r(yht = m|Sht = s).

(3)

s=1

1.3

Estimation

For parameter and state estimation, we need to evaluate P r(Sht |ψht ), the conditional
probability of Sht = s given observations ψht (= yh1 , . . . , yht ). Recursive formulas for
obtaining the One Step Ahead Prediction, Filtering, Smoothing are as follows:
One Step Ahead Prediction:
Calculate the P r(Sht = j|ψh,t−1 ) using ψh,t−1 and equation(1),P r(Sht = j|ψh,t−1 ) =
=
=

S

i=1
S

i=1
S


P r(Sht = j, Sh,t−1 = i|ψh,t−1 ).
P r(Sht = j|Sh,t−1 = i, ψh,t−1 )P r(Sh,t−1 = i|ψh,t−1 )pij P r(Sh,t−1 = i|ψh,t−1 ).

i=1

3

(4)

Filtering:
Update the P r(Sht = j|ψht ) using Bayes’ law,
P r(Sht = j|ψht ) = P r(Sht = j|yht , ψh,t−1 )=p(yht |Sht = j, ψh,t−1 )P r(Sht = j|ψh,t−1 ).
P r(yht |ψh,t−1 )

(5)

where
p(yht |ψh,t−1 ) =Sp(yht |Sht = j, ψh,t−1 )P r(Sht = j|ψh,t−1 ).

(6)

j=1

Smoothing:After parameters of the model are estimated, we can make inference on Sht using all
information (=ψhTh ) in the sample. P r(Sht = j|ψhTh ) is the smoothed probability.
Oppositely, P r(Sht = j|ψht ) is the ﬁltered probability.
Calculate the joint probability of Sht = j and Sh,t+1 = k based on ψhTh :P r(Sht = j, Sh,t+1 = k|ψhTh )= P r(Sh,t+1 = k|ψhTh )P r(Sh,t = j|Sh,t+1 = k, ψhTh )= P r(Sh,t+1 = k|ψhTh )P r(Sh,t = j|Sh,t+1 = k, ψht )==P r(Sh,t+1 = k|ψhTh )P r(Sh,t = j, Sh,t+1 = k|ψht )P r(Sh,t+1 = k|ψht )P r(Sh,t+1 = k|ψhTh )P r(Sh,t = j|ψht )P r(Sh,t+1 = k|Sht = j).
P r(Sh,t+1 = k|ψht )(7)

and P r(Sht = j|ψhTh ) is
P r(Sht = j|ψhTh ) =SP r(Sht = j, Sh,t+1 = k|ψhTh ).

(8)

k=1

Begin with p(Sh,Th −1 = j|ψh,Th ), we can calculate p(Sh,Th −2 = j|ψh,Th ), . . . , p(Sh,1 =4j|ψh,Th ). The validity of going from the second line to the third line of equation (7)is in Appendix:A

1.4

Model Identification

The log likelihood function for h is given byln Lh =Th

ln p(yht |ψh,t−1 ).

(9)

t=1

And the log likelihood function for all households is given byln L =Hln Lh .

(10)h=1

We can Estimate the parameter by maximizing the ln L numerically. In thisapproach, Akaike’s information criterion(AIC) is
AIC = −2max ln L + 2(number of f ree parameters).

(11)

and other infromation criterion are obtained in the same way as AIC.

1.5

Advantage of Using Markov-Switching ModelIf we could not use the Markov-switching Model, the log likelihood of household his

ln Lh = ln{

Sh1 Sh2...

p(Sh1 )p(yh1 |Sh1 )Thp(Sht |Sh,t−1 )p(yht |Sht )}.

(12)

t=2

ShTh

Compared with (9), (12) is so diﬃcult to ﬁnd the parameters which maximize log
likelihood. Markov-switching model has advantage of simplicity at the calculationof the log likelihood.

5

2

APPLICATIONIn this section we estimate the model using scanner panel data set1 .

2.1

Data

The data analyzed here consist of 52 weeks scanner panel records of households in theTokyo metropolitan area, during the year 1993. The number of 97 households whopurchased instant coﬀee between 12 and 24 times during this period was selected.
To keep the model estimation manageable, we chose four top-selling brands(BrandA–
BrandD), and the rest was gathered to one brand(Brand E 2 ).

Brand

Marketshare

Average
Discount Rate

A
B
C
D
E

0.286
0.171
0.133
0.078
0.331

0.794
0.777
0.747
0.835
0.734

The purpose of this analysis is to know whether people change their preference
structure. The eﬀect of loyalty( last purchased brand) to the utility may diﬀer overthe person. Will s/he be also stable all over the time? If not, when did s/he change
preference structure? This information will be very important when we plan therelationship marketing( ex. electronic couponing) .

2.2

Model Variables

We begin by specifying the functional form of the choice model. The deterministic
component of utility for brand m at time t, given that household h is a member of6

segment s, isxhtm β s = βs,price PRICE mt + βs,display DISPLAY mt + βs,lbp LBP mt + βs,int,m . (13)where:
PRICE mt = (actual shelf price of brandm at time t)÷(highest shelf price of brandm during the period).
DISPLAY mt = 1 if brandm was displayed at time t for h ,and 0 otherwise.
LBP hmt = 1 if brand m was purchased by household hat time t − 1, and 0 otherwise.
βs,price , βs,display , βs,lbp = parameters to be estimated.
βs,int,m = constant for brandm, segment s, to be estimated.
In this application, our interest is in the dynamics of the inﬂuence of loyalty to
the utility. So we consider two segments, Segment1 is under the inﬂuence of loyalty(β1,lbp = 0, to be estimated) and Segment2 is not (set β2,lbp = 0 a priori).

2.3

Steady-State Probability

To start the Filtering, we have to deﬁne the start probability Sh0 . We employ thesteady-state probability as Sh0 (Kim and Nelson 1999 p.66). Derivation of SteadyState Probability is stated in Appendix B. When Segment Size was determined to
be two,P r(Sh0 = 1) =P r(Sh0 = 2) =7

1 − p22.
2 − p11 − p22
1 − p11.
2 − p11 − p22

(14)
(15)

2.4

Estimation Results

Model Comparison
Before the estimation, we set three comparison models.
Model 1: Logit model with Markov-switchingThis model assumes that there are two segments and household will switch thesegment.
Lh =

T 
S


[

P r(Sht = j|ψh,t−1 )p(yht |Sht = j)].

(16)

t=1 j=1

Model 2: Logit model with latent segment
This model assumes that there are two segments and any household does not switch
segment.
Lh =

S


T


[πj

j=1

p(yht |Sht = j)].

(17)

t=1

where πj is size of segment j.

Model 3: Normal logit modelThis model assumes that there is only one segment.
Lh =

T


[p(yht )].

(18)

t=1

The information criteria of the models are shown in Table 1. AIC, BIC andCAIC are smallest at the Model1. We can say that Model1 is ﬁtest for the data.

8

Feature of the segment
The estimated parameters of the Model1 are shown in Table 2.

We have set

β2,lbp = 0 a priori, and β1,lbp is estimated positive. The inﬂuence of loyalty is
positive in Segment1. Comparison of price parameters and display parameters,
β2,price > β1,price and β2,display > β1,display , so we can also say that Segmen2 is apromotion sensitive segment. As shown in Table 2, switching probability p11 and
p22 , which is recurrent probability, are close to 1. The switching probability between
Segment1 and Segment2 is very small.

Dynamics of Segmentation
The example of dynamics of segmentation are shown in Fig 1 and Fig 2). The
household#1 suddenly changes segment at purchase occasion 9. On the other hand,
the household#2 is stable over the time. As we intended, the dynamics of segmentation was estimated.

3

Discussion

In this study, we demonstrated a methodology for modeling the latent change between the segments in a short period. A probability that one belongs to a segment isdeﬁned as a markov process, and a probability that one chooses a brand is deﬁned asa multinomial logit model. The estimation of such model has been diﬃcult becauseof the complicated calculation of log likelihood. Using Markov-switching model, we
can estimate the model parameters and latent states at each purchase occasion. In
the empirical study, We have estimated an instability of segment in the instant coffee category. We assumed that there are two preference structures, one is under the9

inﬂuence of loyalty and the other is not. The empirical result showed that there aresome people that change their own segment within data period.
The main limitation of our model is that the parameters of transition probability are dependent on an unobservable state variable (Sht ), that is an outcome ofan unobservable Markov process. This means that inference of ShT is based on aconditional distribution, not on a joint distribution. Another limitation is that thelikelihood function of the parameters is still not presented in simple form. MCMC (Markov chain Monte Carlo ) approach will enable us to solve these problems. Albert
and Chib (1993) have made MCMC analysis of Markov-switching model.
Application of our model to another marketing area is also an interesting. We
have not investigated why s/he switches her/his segment. If we use the concomitantvariable( Kamakura, Wedel and Agrawal 1994) to the Markov-switching probability,
we will be able to account for this problem. This analysis is also important frommanagerial perspective. The future research is expected.

Appendix A
The validity of going from the second line to the third line of equation (7) is given
by Kim and Nelson(1988 p.68).
Deﬁne f˜h,t+1,Th = (yh,t+1 , yh,t+2 , . . . , yhTh ) , for Th > t. That is, f˜h,t+1,Th is thevector of observations from date t + 1 to Th . Then we have
P r(Sht = j|Sh,t+1 = k, ψhTh )= P r(Sht = j|Sh,t+1 = k, f˜h,t+1,Th , ψht )=g(Sht = j, f˜h,t+1,Th |St+1 = k, ψht )g(f˜h,t+1,Th |St+1 = k, ψht )10

=P r(Sht = j|Sh,t+1 = k, ψht )g(f˜h,t+1,Th |Sh,t+1 = k, Sht = j, ψht )g(f˜h,t+1,T |Sh,t+1 = k, ψht )h= P r(Sht = j|Sh,t+1 = k, ψht )(19)

The above holds as g(f˜h,t+1,Th |Sh,t+1 = k, Sht = j, ψht ) = g(f˜h,t+1,Th |Sh,t+1 =k, ψht ), which suggests that if Sh,t+1 were somehow known, then yh,t+1 would containno information about Sht beyond that contained in Sh,t+1 and ψh,t .

Appendix B
Derivation of Steady-State Probability Used to Start the Filter is below(Kim and
Nelson 1999 p.70).




 p11



 p
 12
∗
P =

 ..
 .




p21

· · · pS1 


p22

···

..
.

..

.



pS2 


.. 

. 




(20)

p1S p2S · · · pSS

where iS P ∗ = iS with iS = [1 1 . . . 1] . If we let πt be a vector of S × 1
steady-state probabilities, we have



πt =







 P r(St = 1)   π1t 

 


 


 

 P r(S = 2)   π t  2t 

=


 


 

 ···
  ··· 

 


 


 


P r(St = S)iS πt = 1.

(21)

πSt

(22)

Then according to the deﬁnition of steady state probabilities, we have πt+1 = P∗ πt

11and πt+1 = πt , and thus
πt = P ∗ πt =⇒ (IS − P ∗ )πt = 0S .

(23)

where 0S is an S × 1 matrix of zeros. Combining equations(22)and(23) ,we have












 IS − P ∗ 
 0S 
 0S 







 πt =  , or Aπt = 
.







iS

1

(24)

1

Multiply both sides of the above equation by (A A)−1 A . Then, 0S πt = (A A)−1 A 

(25)

1

That is, the matrix of steady-state probabilities, πt , is the last column of the matrix(A A)−1 A .

12

Notes
1. Scanner panel data was oﬀered from one of the Japanese supermarket. The nameof supermarket is secret because of the duty to protect privileged information. Thisdata set is same as Moriguchi and Mori(1995).

2. The Value of Price and Display of Brand E was computed as following.
• If one choose one of BrandE, the price and the promotion of Brand E is thischosen Brand.
• Otherwise, the price is set to the largest discount rate and the promotion is
set to 1 if at least one of Brand E is promoted.

13

References
Albert, J.H. and S. Chib (1993),“Bayesian Analysis of Binary and Polychotomous
Response Data,” Journal of the American Statistical Association, 88,669-679.
Böckenholt,U. and R. Langeheine (1996),“Latent Change in Recurrent Choice Data,”
Psychometrika, 61,285-301.
Hamilton, J.D. (1989),“A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle.” Econometrica,57,357-384
Kamakura, W.A. and G.J. Russell (1989),“A Probabilistic Choice Model for Market Segmentaion and Elasticity Structure,” Journal of Marketing Research,
26,379-90.
Kamakura, W.A., M. Wedel and J. Agrawal (1994),“Concomitant Variable LatentClass Models for Conjoint Analysis,” International Journal of Researchin Marketing, 11,451-64.
Kim, Chang-Jin, and C. R. Nelson (1999), State-Space Models with Regime Switching,
The MIT Press.
Kitagawa, G. (1987),“Non-Gaussian state space modeling of nonstationary time series,(with discussion)”, Journal of American statistical Association, 82,10321063.
Moriguchi, T. and M. Mori (1995),“A Brand Choice Model Considering Evoked
Set(in Japanese)”, Journal of Marketing Science (published by JIMS), 26,8898.

14

Poulsen, C.S. (1990),“Mixed Markov and Latent Markov Modeling Applied toBrand Choice Behavior,” Intern. J. of Research in Marketing, 7,5-19.
Ramaswamy, V (1997),“Evolutionary preference segmentation with panel survey
data: An application to new products,” Intern. J. of Research in Marketing,
14,57-80.
Yang, S. and G.M. Allenby (2000),“A Model for Observation, Structural, andHousehold Heterogeneity in Panel Data,” Marketing Letters, 11,137-149.
Wedel, M. and W.A.Kamakura (1998), Market Segmentation, Kluwer Academic
Publishers.

15

Table 1: Model Comparison

LL
AIC
BIC
CAIC

Model 1−1268.54
2567.08
2643.68
2643.70

Model 2−1352.09
2732.18
2803.68
2803.69

Model 3−1387.24
2788.482824.232824.23

Table 2: Estimation Results
Segment1
β1price

Segment2β2price−2.540(−4.45)a−12.206(−8.25)0.467(3.17)
1.977(18.49)−0.346(−2.12)β2display

β2bint1

1.425(6.33)

β1int2−0.037(−0.28)β2int2−1.261(−3.81)β1int3−1.378(−5.58)−0.694(−3.98)0.961(10.35)β2int30.291(1.64)−1.112(−3.59)0.954(10.00)β1display
β1lbp
β1int1β1int4p11

β2lbp

β2int4p22

a Asymptotic t-statistics in parenthese.
b Set to 0 a priori.

16

0.984
(6.38)
0b

Figure 1: The dynamics of state probability in household#1

Figure 2: The dynamics of state probability in household#2

17