Modeling User Behavior in Recommender Systems
based on Maximum Entropy
Tomoharu Iwata,

Kazumi Saito,

Takeshi Yamada

NTT Communication Science Laboratories
2-4, Hikaridai, Seika-cho, Keihanna Science City Kyoto 619-0237 Japan{iwata,saito,yamada}@cslab.kecl.ntt.co.jp

ABSTRACT
We propose a model for user purchase behavior in online
stores that provide recommendation services. We model thepurchase probability given recommendations for each user
based on the maximum entropy principle using features thatdeal with recommendations and user interests. The proposed model enable us to measure the effect of recommendations on user purchase behavior, and the effect can be used
to evaluate recommender systems. We show the validity of
our model using the log data of an online cartoon distribution service, and measure the recommendation effects forevaluating the recommender system.

Categories and Subject Descriptors
H.2.8 [Database Management]: Database Applications—
Data Mining; H.3.3 [Information Storage and Retrieval]:
Information Retrieval and Search—Information Filtering

General Terms
Algorithms,Experimentation

Keywords
recommendation, user model, maximum entropy principle

1. INTRODUCTIONRecommender systems are widely used by online stores
because they can improve user convenience and store profits. Recommendations would change user behavior, and it
is important for online stores to understand user behavior
when items are recommended. If online stores know thedegree of change in purchase behavior brought about byrecommendations, they can predict the degree to which therecommendations can increase cross-sell. Although a number of user behavior models have been proposed, they do
not consider the effect of recommendations.
In this paper, we propose a model for user behavior in
online stores that provide recommendation services basedon the maximum entropy principle. The maximum entropy
principle is a method for estimating a probabilistic distribution that satisfies all the constraints in the given data while
maximizing the entropy [2]. In our model, we estimate purchase probabilities for each user given recommendations under the constraints present in the purchase log data. One ofCopyright is held by the author/owner(s).
WWW2007, May 8–12, 2007, Banff, Canada.
.

the advantages of the maximum entropy principle is that wecan integrate arbitrary features in the model. We integrateuser interest information and recommendation information
for a user behavior model by employing the maximum entropy principle.
From the log data, our model can estimate the degree ofchange in purchase probabilities induced by recommendations. We call this degree of change the recommendation
effect. It is difficult to evaluate recommender systems [1].
Most recommender systems have been evaluated in terms of
their accuracy in predicting ratings or next purchase items.
However, the high predictive accuracy of the recommendations does not necessarily imply good recommendations.
Those recommendations are likely to recommend ordinary
items that users are already aware of. Although other factors, such as novelty or serendipity [3] have been proposed
for use in evaluating recommender systems, it is difficult tomeasure them quantitatively. The recommendation effectestimated with our model can be used to evaluate the recommender system itself. The recommendation effect can bedifferent for different users. Some users are more likely topurchase items than others when recommended. We estimate the recommendation effect for each user. We evaluate
our model using real log data, and we estimate the recommendation effect and the difference among users. To thebest of our knowledge, this is the first report describing therecommendation effect in relation to individual users in areal online store.

2.

USER BEHAVIOR MODEL WITH RECOMMENDATIONSLet S = {sj }Vj=1 be a set of items, and r(u) ⊂ S be a
set of recommended items. Based on the maximum entropy
principle, we estimates P (s|u, r(u)), which is the probability that user u purchases item s when an item set r(u) isrecommended.
The item that a user purchases depends on the recommended items and the user interests. We use two typesof features to model user purchase behavior. The first iswhether item s is recommended to user u or not:(1 if s ∈ r(u),z(r(u), s) =(1)0 otherwise.
The second is related to user interest. We used first-orderMarkov transitions si → sj as interest features, since thelast purchased item most reveals the interest [2]:8><1 if item si is the last purchasedyij (u, s) =item of user u and s = sj ,>:0 otherwise.

(2)

Table 1: Predictive accuracies of N -best items (%).
N
1
2
3
45No effect model
17.4 24.9 33.9 39.8 42.1
Uniform effect model 17.4 24.9 34.2 39.8 44.6
Individual effect model 17.6 26.3 35.8 41.5 45.1

With the maximum entropy principle, the probability thatuser u purchases item s when item set r(u) is recommended
can be modeled as follows:

7000

P (s|u, r(u))
„X
«
1exp
αij yij (u, s) + φz(r(u), s) , (3)Z(u, r(u))
i,jPPwhere Z(u, r(u)) = k exp( i,j αij yij (u, sk )+φz(r(u), sk ))is the normalization term, and α = {αij } and φ are unknown parameters to be estimated. Note that only one αij
in the summation is effective for a given pair of (u, s). Since
the purchase probability is increased exp(φ) times by therecommendation r(u), φ represents the recommendation effect. Some users are likely to be affected by recommendations, and others are not. Therefore, we estimate therecommendation effect φn for each user un . The unknown
parameter α and φn can be estimated by maximizing thelog likelihood.

3. EXPERIMENTAL RESULTS
We evaluated our model in terms of predictive accuracies of next purchase items under a certain recommendation
provided using the log data of an online cartoon distribution service for cell-phones in Japan. In this online store,
five cartoons are recommended that are chosen manually byhuman experts on each cartoon distribution page.
We compared following three models: no effect model,
uniform effect model, and individual effect model. No effect model assumes the recommendation effect φ = 0, whichuses only user interests for features as same as conventionaluser models. Uniform effect model has uniform recommendation effects φ for all users. Individual effect model has
the recommendation effects φn for each user un . We firstestimated parameters about user interests α using the log
data without recommendations, and used the parameters α
for all three models. The log data consisted of 23,354 users,
218,231 transactions, and 128 items. Next, we estimated theaverage recommendation effect φ for all users using the log
data with recommendations and used the parameters φ for
the uniform effect model. The log data consisted of 44,338
users, 604,673 transactions. Finally, we estimated recommendation effects for each user φn with a Gaussian priorwith mean φ and variance 1/η. We used η = 1. The test
data was the last item of each user in the log data withrecommendations, that was excluded in the estimation.
For the evaluation measurements, we used the predictive
accuracies of N -best items. Table 1 shows the results. The
accuracy of the individual effect model is higher than that ofthe others. This result shows that estimating the individual
recommendation effect is important in terms of predicting
purchase behavior. The average recommendation effect was
φ = 0.24. This means that the purchase probability is increased 1.27(= exp(0.24)) times by recommendations, andthe recommendations lead to increased purchases. Figure 1shows a histogram of users with recommendation effect φnestimated in the individual effect model. This figure can6000

the number of users=8000

5000
4000
3000
2000
1000
0−3−2−101
recommendation effect φ23Figure 1: Histogram of users with recommendation
effect φn in the individual effect model.

be viewed to characterize the recommendations used in theonline store. It has two peaks at around φn = 0.1 and 0.8.
The recommendations did not change purchase behaviors ofusers around φn = 0.1, and this might be because they wererecommended ordinary items that the users would purchaseeven without recommendations. On the other hand, the recommendations affected purchase behaviors of users aroundand above φn = 0.8, and the one reason might be their highserendipity.

4.

CONCLUSIONIn this paper, we proposed a model for user behavior inonline stores that provide recommendation services. We estimates the probability of purchasing an item given recommendations for each user based on the maximum entropy
principle. Using our model, we can measure the recommendation effects. In the experiments, we showed the validityof our model using the log data of a real online store. We
used simple features to model user interests. We need toimprove our user behavior model using other features, suchas user demographic information, and content information.
We would like to evaluate our model further by applying it
to other online stores.

5.

REFERENCES[1] J. L. Herlocker, J. A. Konstan, L. G. Terveen, andJ. T. Riedl. Evaluating collaborative filtering
recommender systems. ACM Transactions onInformation Systems, 22(1):5–53, 2004.
[2] T. Iwata, K. Saito, and T. Yamada. Recommendation
methods for extending subscription periods. In
Proceedings of the Conference on Knowledge Discovery
and Data Mining, pages 574–579, 2006.
[3] C.-N. Ziegler, S. M. Mcnee, J. A. Konstan, andG. Lausen. Improving recommendation lists throughtopic diversification. In Proceedings of the international
conference on World Wide Web, pages 22–32, 2005.