DualDiv: Diversifying Items and Explanation Styles
in Explainable Hybrid Recommendation
Kosetsu Tsukuda and Masataka Goto
National Institute of Advanced Industrial Science and Technology (AIST), Japan{k.tsukuda, m.goto}@aist.go.jp

ABSTRACT͗ƌĞĐŽŵŵĞŶĚĞĚĂƌƚŝƐƚ

In recommender systems, item diversification and explainable recommendations improve users’ satisfaction. Unlike traditional explainable recommendations that display a single explanation foreach item, explainable hybrid recommendations display multiple
explanations for each item and are, therefore, more beneficial forusers. When multiple explanations are displayed, one problem isthat similar sets of explanation styles (ESs) such as user-based,
item-based, and popularity-based may be displayed for similar
items. Although item diversification has been studied well, thequestion of how to diversify the ESs remains underexplored. In
this paper, we propose a method for diversifying ESs and a framework, called DualDiv, that recommends items by diversifying boththe items and the ESs. Our experimental results show that DualDiv
can increase the diversity of the items and the ESs without largely
reducing the recommendation accuracy.

;ĂͿŽƌŝŐŝŶĂůCCS CONCEPTS• Information systems Recommender systems.

KEYWORDS

recommender systems, explainable recommendations, diversity
ACM Reference Format:
Kosetsu Tsukuda and Masataka Goto. 2019. DualDiv: Diversifying Items
and Explanation Styles in Explainable Hybrid Recommendation. In Thirteenth ACM Conference on Recommender Systems (RecSys ’19), September
16–20, 2019, Copenhagen, Denmark. ACM, New York, NY, USA, 5 pages.
https://doi.org/10.1145/3298689.3347063

1 INTRODUCTION
Most studies dealing with recommender systems (RSs) have traditionally put priority on recommendation accuracy [1] where users’
preferences toward items are often measured based on their histories of item consumption [4]. However, it has been recognized thatthe diversity of items in the recommended list is also an importantfactor for increasing users’ satisfaction with RSs because diversified recommended items allow users to browse a wider range ofitem types [9, 18]. Recommendation diversification aims to generate a ranked list of items in which items are dissimilar to each otherbut nonetheless relevant to the user’s preference [3, 14, 18].
To improve the persuasiveness and user satisfaction of RSs, it
is also useful to provide explanations so that users can understandPermission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
RecSys ’19, September 16–20, 2019, Copenhagen, Denmark© 2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-6243-6/19/09…$15.00
https://doi.org/10.1145/3298689.3347063

ܽଵϭƐƚ ƌŽĐŬ
ďĂŶĚ

ϮŶĚ

ܽଶ

ƌŽĐŬ ďĂŶĚܽଷϯƌĚ ƌŽĐŬ ϵϬƐ

݁ଵ
݁ଷ
݁ଵ

݁ଵ
݁ଷ

͗ƚĂŐ

͗^;ĞǆƉůĂŶĂƚŝŽŶƐƚǇůĞͿ;ĐͿŝƚĞŵĂŶĚ^;ďͿŝƚĞŵĚŝǀĞƌƐŝĨŝĐĂƚŝŽŶ
ĚŝǀĞƌƐŝĨŝĐĂƚŝŽŶ
݁ଵ
݁ଵ
ܽଵ
ܽଵ
ƌŽĐŬ ďĂŶĚ ݁ଷƌŽĐŬ ďĂŶĚ ݁ଷ

ܽଷ

ƌŽĐŬ

ܽସ

ƉŽƉ

ϵϬƐ

݁ଵ
݁ଷ

݁ଵ
݁ସ

ܽଷ

ƌŽĐŬ

ܽସ

ƉŽƉ

ϵϬƐ

݁ଶ
݁ସ

݁ଵ
݁ସ

Figure 1: DualDiv framework concept.
why each item is recommended [13]; such approach is called explainable recommendation (ER) [16]. In ERs, a concrete explanation is generated from the explanation style (ES). An ES can beuser-based (e.g., “this item is recommended because users similar
to you like it.”), popularity-based (e.g., “this item is recommended
because it is popular among users.”), etc. Although most studies related to ERs had focused on a single ES, where a single explanation
is displayed for each item [6, 11, 12, 15], it has been reported thatusers prefer multiple ESs, where many explanations are displayedfor each recommended item [8]. A recommendation that can deal
with multiple ESs is called explainable hybrid recommendation [8].
Because of such usefulness of item diversification and ERs, multiple explanations in item-diversified recommendations could be
beneficial to users. However, there are some problems associatedwith explanations. A study by Kouki et al. [8] has reported that
subjective persuasiveness of each ES is different from one user toanother. To cope with such difference, displaying all possible explanations generated from all ESs, say seven ESs, is not appropriate: it has been reported that giving too many explanations leads
to the user’s information overload, so displaying three or four explanations is usually sufficient. However, it is difficult for a RS toknow in advance which ESs are preferred by each user. Moreover,even when preferred ESs are known for a specific user, it may besubject to change depending on the domain (e.g., songs and books)
and context (e.g., time and user’s intent). Therefore, it may not be
ideal to fix three or four ESs for all items.
In light of the above, we propose a new framework DualDiv
for diversifying both the items and the ESs. We explain intuitively
DualDiv by using toy examples in Fig. 1 which shows three resultsof artist recommendations to a user. List (a) represents the original recommendation results where, for each recommended artist,
tags attached to an artist as well as ESs are displayed. In list (a), because the user likes rock music, top three artists are rock-related.
To make the recommendation results more attractive to the user,
the list is diversified in terms of tags, and list (b) is generated. Now,
a pop artist a 4 is recommended instead of a 2 . However, for the tworock artists a 1 and a 3 , the same ESs e 1 (e.g., user-based ES) and e 3(e.g., social-based ES) are displayed. If the user does not think theseESs are persuasive, she may select neither of the recommended

RecSys ’19, September 16–20, 2019, Copenhagen, Denmark

artists. To solve this problem, DualDiv also diversifies the ESs as
shown in list (c): the ESs e 2 (e.g., popularity-based ES) and e 4 (e.g.,
item-based ES) are displayed for a 3 because e 1 and e 3 are displayed
for a 1 . Note that in list (c), only the ESs are diversified, while theorder of the artists is the same as in list (b). If the user thinks e 2
is persuasive, she would select a 3 and listens to a 3 ’s songs. Thus,
DualDiv aims to increase the probability of a user accepting at least
one recommended item by diversifying both the items and the ESs.
Our main contributions are as follows. (1) We propose a framework DualDiv that recommends items by diversifying both theitems and the ESs. To the best of our knowledge, this is the firststudy to propose the concept of ES diversification. DualDiv is a
general framework and can be applied to any domain of item recommendations. (2) We propose a method for diversifying ESs thatavoids displaying the same ESs for similar items as much as possible. Our method can also diversify ESs among all recommended
items regardless of similarities between the items, by changing theparameter. (3) Our experiments in the context of artist recommendations show that DualDiv can diversify artists and ESs without
largely reducing recommendation accuracy in terms of Recall.

2

DUALDIV

Given a target user, DualDiv generates a list of items by diversifying both the items and the ESs as follows. We first generate recommendations with personalized explanations for each item basedon the method of Kouki et al. [8] (Section 2.2). We then generate
an item-diversified list by using the method of Dou et al. [5] (Section 2.3). Because ESs cannot be diversified just by combining these
two methods, in Section 2.4, we present DualDiv core method fordiversifying the ESs. In this paper, we develop DualDiv in the domain of artist recommendations by using the data of Last.fm.

2.1

Notation

Let U and A denote the sets of users and artists, respectively. For
each artist a ∈ A, a set of tags attached to a is defined as Ta , andeach tag t ∈ Ta has a relevance score with respect to a that isrepresented by trel(a, t). For u ∈ U , Au ⊂ A represents a set ofartists preferred by u, and Fu ⊂ U represents a set of u’s friends
on Last.fm. Moreover, let Apop ⊂ A denote a set of top 1,000 popularartists on Last.fm obtained by Last.fm API. Given the target useru, the set of candidate artists for a recommendation is defined as∪
C = u ′ ∈Fu Au ′ ∪ Apop \ Au [8]. Therefore, our goal is to generate
a personalized ranked list of artists from C.

2.2

Explainable Hybrid Recommendation

Our first step is to generate recommendations with personalized
explanations. To this end, we adopt a hybrid recommender system, called HyPER [7]. HyPER makes use of probabilistic soft logic(PSL) [2] that enables HyPER to develop recommendation models
through a set of template rules in first-order logic syntax. HyPER
automatically learns to balance the different input signals and computes the probability that the user accepts each item.
To compute the probability that user u listens to the recommended artist a ∈ C, we use the following eight rules [8].
Rule 1: SIMUSERSCF (u, u ′ ) ∧ PREFERS(u ′, a) ⇒ LISTENS(u, a)Rule 2: SIMARTISTSCF (a, a ′ ) ∧ PREFERS(u, a ′ ) ⇒ LISTENS(u, a)Rule 3: SIMARTISTScontent (a, a ′ ) ∧ PREFERS(u, a ′ ) ⇒ LISTENS(u, a)Rule 4: SIMARTISTSlast.fm (a, a ′ ) ∧ PREFERS(u, a ′ ) ⇒ LISTENS(u, a)Kosetsu Tsukuda and Masataka Goto

Table 1: Example explanations generated from ESs: artist“The Beatles” is recommended to a user.
ES
e1
e2
e3
e4
e5
e6
e7

We recommend The Beatles because:
Last.fm users Anna, Tom, and John with whom you
share similar music tastes, listens to The Beatles.
People who listen to your preferred artists Bob Dylan
and Johnny Cash also listen to The Beatles.
The Beatles has similar tags with your preferred artist
The Kinks
According to Last.fm, artist The Beatles is similar toyour preferred artist Queen.
Artist The Beatles is tagged with 60s and britpop that are
in your profile.
Your friend Lisa likes artist The Beatles.
Artist The Beatles is very popular in the Last.fm database with 3.67 M listeners and 516.17 M playcounts.

Rule 5: TAG(a, t) ∧ TAG(a ′, t) ∧ PREFERS(u, a ′ ) ⇒ LISTENS(u, a)Rule 6: SIMFRIENDS(u, u ′ ) ∧ PREFERS(u ′, a) ⇒ LISTENS(u, a)Rule 7: POPULARARTIST(a) ⇒ LISTENS(u, a)Rule 8: ¬LISTENS(u, a)
In all rules, the atom LISTENS(u, a) represents the inferred probability that user u will listen to a, while the remaining atoms arebinaries. Rule 1 captures the intuition that similar users like similar artists. Similarity between two users is computed based on theset of common artists they prefer. SIMUSERSCF (u, u ′ ) is 1 iff u ′ is included in the top 20 similar users of u. Rule 2, 3, and 4 capture theintuition that a user listens to similar artists. In Rule 2, similarity
between two artists is computed by the set of common users who
prefer both artists, while in Rule 3, similarity is computed according to the set of common tags attached to the artists. Similarity in
Rule 4 is obtained through Last.fm API. In all cases, artists in the
top 20 similar artists of a have values of 1 for SIMARTISTS. Rule 5
states that a user listens to artists with the same tag. Rule 6 captures the intuition that friends share similar tastes in music andlisten to the same artists. Rule 7 implies that a user tends to listen
to popular artists. Finally, Rule 8 models a general belief that a user
will not listen to an artist. Due to space limitation, we do not givethe details of the similarity computation methods. We refer readers
to Kouki et al. [8] for more details.
After HyPER computes LISTENS(u, a) for all artists in C, we generate a ranked list of the artists by sorting them in descending orderof LISTENS(u, a). For each artist, we also create personalized explanations. In HyPER, each rule has an ES and explanations are generated from the ESs. Example explanations for seven ESs are shownin Table 1, where ES ei corresponds to Rule i. Kouki et al. [8] havereported that the order of average subjective persuasiveness of theseven ESs is e 4 → e 2 → e 3 → e 7 → e 5 → e 6 → e 1 . They havealso reported that displaying three or four explanations for eachartist is sufficient. Following this insight, in this paper, we display
three explanations for each recommended artist. Given target useru, let R exp and Eac denote the ranked list of all artists in C and thesets of all ESs created for a ∈ R exp , respectively.

2.3

Item Diversification

Having obtained R exp , our next step is to diversify the artists in
R exp . In this paper, we generate the artist-diversified recommendation list based on the diversification method proposed by Dou etal. [5], which is a general form of the xQuAD framework [10].

DualDiv

RecSys ’19, September 16–20, 2019, Copenhagen, Denmarka ∈R exp \S n∪where Φ(a, Sn ,T ) computes the diversity of a and T = a ∈C Ta ;specifically, Φ(a, Sn ,T ) represents a tag richness score of a given
the set Sn . α is a parameter that controls the tradeoff between relevance (i.e., LISTENS(u, a)) and diversity. The term Φ(a, Sn ,T ) is de∑composed as Φ(a, Sn ,T ) = t ∈T w t · ntrel(a, t) · φ(t, Sn ), wherew t is the importance of tag t for u. Following Santos et al. [10], we
consider all tags as being equally important: w t = |T1 | . In turn,ntrel(a, t) represents the normalized relevance of artist a with retrel(a,t )spect to tag t and is given by ntrel(a, t) = ∑ ′ trel(a,t ′ ) . Finally,t ∈Taφ(t, Sn ) is the discounted importance of tag t given Sn , where∏φ(t, Sn ) = 1 if n = 0 and φ(t, Sn ) = a ′ ∈Sn [1 − ntrel(a ′, t)] otherwise. Based on the above iterative process, we re-rank all artists
in R exp ; let R div denote the ranked list of diversified artists.

2.4

Having obtained R div , our final step is to diversify ESs for eachartist. Note that in this step, the order of artists in R div does notchange. One simple approach to select three ESs for each artist
is by selecting them according to the order of average subjective
persuasiveness of ESs mentioned in Section 2.2. However, in thisapproach, similar sets of ESs may be often displayed for recommended artists. To solve this problem, we propose a method that
also employs a greedy algorithm and generates a diversified ranked
list of ESs for each artist. Our method is based on the following criterion: similar sets of ESs should not be displayed for similar artists.
Our method is applied for each artist in top-to-bottom order of R div .
Given n + 1th ranked artist an +1 ∈ R div , our goal is to select threeESs, which are denoted by Eas n +1 , from Eac n +1 created in Section 2.2.
Note that when we create Eas n +1 , Eas i for ai (1 ≤ i ≤ n) has alreadybeen created. We create Eas n +1 based on (1) if e ∈ Eac n +1 is includedin Eas i and (2) if an +1 is similar to ai .
Fig. 2 shows an example process of selecting mth (1 ≤ m ≤ 3)
ES for a 3 (i.e., n = 2), where a 3 is similar to a 1 and dissimilar to a 2 .
When Eac 3 consists of five ESs, e 2 is selected first because it is the
only ES not included in either of Eas 1 and Eas 2 . Then e 6 is selected
because it is included in only Eas 2 of dissimilar artist a 2 . Finally,
all of the remaining ESs (i.e., e 1 , e 4 , and e 7 ) are included in Eas 1 ofsimilar artist a 1 ; but e 4 and e 7 are also included in Eas 2 . Therefore,
to increase the overall diversity of ESs, e 1 is selected.
Formally, the mth ES e ∈ Eac n +1 is given by:arg maxe ′ ∈E ac n +1 \E as n +1

[Ψ(e ′, an +1 )].

(1)Ψ(e ′, an +1 ) represents the diversity of e ′ and is decomposed by:)∑ (Ψ(e ′, an +1 ) =1 − δ (e ′ ∈ Easi ) · sim(an +1 , ai ),1 ≤i ≤n݉ ൌ ͳǣ ݁ଶ ŝƐƐĞůĞĐƚĞĚ͘‫ܧ‬௔௦య ൌ ݁ଶ݉ ൌ ʹǣ ݁଺ ŝƐƐĞůĞĐƚĞĚ͘‫ܧ‬௔௦య ൌ ݁ଶ ǡ ݁଺݉ ൌ ͵ǣ ݁ଵ ŝƐƐĞůĞĐƚĞĚ͘‫ܧ‬௔௦య ൌ ݁ଵ ǡ ݁ଶ ǡ ݁଺

Figure 2: An example process to select three ESs Eas 3 for a 3 .
in R div are dissimilar, almost the same sets of ESs may be displayed
for all artists. To solve this problem, we consider the ES duplications between two arbitrary artists and expand Eq. 1 as follow:e=arg max[β · Ψ(e ′, an +1 ) + (1 − β) · Ω(e ′, an +1 )], (2)e ′ ∈E ac n +1 \E as n +1

where Ω(e ′, an +1 ) =∑1 ≤i ≤n()
1 − δ (e ′ ∈ Eas i ) and β is a param-eter. If e ′ is already selected for many high-ranking artists in R div ,the importance of e ′ decreases regardless of similarity. When morethan one ESs have the same score in Eq. 2, the ES having the highest order of average subjective persuasiveness is selected 1 .

3

EXPERIMENTS

This section reports on the evaluation of our proposed framework
DualDiv using the Last.fm dataset.

Explanation Style Diversificatione=‫ܧ‬௔௖య ൌ ݁ଵ ǡ ݁ଶ ǡ ݁ସ ǡ ݁଺ ǡ ݁଻‫ܧ‬௔௦భ ൌ ݁ଵ ǡ ݁ସ ǡ ݁଻‫ܧ‬௔௦మ ൌ ݁ସ ǡ ݁଺ ǡ ݁଻Ͳ ܽଷ ŝƐƐŝŵŝůĂƌƚŽܽଵͲ ܽଷ ŝƐĚŝƐƐŝŵŝůĂƌƚŽܽଶ

Their method [5] employs a greedy algorithm which iteratively
selects items and generates a diversified ranking list. Let Sn denotethe top n artists selected so far. The n + 1th artist is given by:
an +1 = arg max [α · LISTENS(u, a) + (1 − α) · Φ(a, Sn ,T )],where δ (x) is 1 when x is true and 0 otherwise. sim(an +1 , ai )represents similarity between an +1 and ai , and is given bysim(an +1 , ai ) = ∥Tan +1 ∩Tai ∥/∥Tan +1 ∪Tai ∥. Although Eq. 1 considers duplications of ESs between similar artists, those between
dissimilar artists are not taken into account. Therefore, if all artists

3.1

Evaluation Setup[Dataset] As target users, we randomly selected 300 users ofAmerican nationality from Last.fm dataset [17]. By using theLast.fm API, we collected the following data: top 20 friends and top
25 preferred artists for each target user, top 20 preferred artists foreach friend, and top 20 tags for each artist. In summary, our dataset
consisted of 5,789 users, 29,435 artists, and 31,969 tags. For each target user, five preferred artists were randomly sampled as test data
and added to C (described in Section 2.1), and the remaining 20
artists were used as training data.
[Comparisons] In addition to our proposed framework DualDiv that diversifies both the items and the ESs, we use the following three comparisons.
• NonDiv: this method diversifies neither artists nor ESs and corresponds to Fig. 1 (a). The recommendation list R exp generated
in Section 2.2 is used for this method. Regarding ESs, we selectthree ESs from Eac in the order of average subjective persuasiveness.
• ItemDiv: this method, which corresponds to Fig. 1 (b), diversifies only artists and uses the recommendation list R div generated in Section 2.3. The ESs for each artist are selected in thesame way as in NonDiv.
• RandDiv: this method also uses R div for artist ranking but randomly selects three ESs for each artist. This is also a kind of ES
diversification because, unlike in NonDiv and ItemDiv, there
are no specific priorities among ESs.
For ItemDiv, RnadDiv, and DualDiv, the value of α in Section 2.3
is set to 0.75, while β in Section 2.4 is set to 0.5 for DualDiv.
[Evaluation Metrics] We evaluate the recommendation accuracy in terms of Recall, where Recall@k is defined by the ratio ofartists in the user’s test data that are correctly included in Rkdiv ,which is the top k artists in R div . Regarding artist diversity, we
use AILD (artist ILD, where ILD stands for intra-list diversity [18]),
1 When |E ccan +1 | < 3, the number of selected ESs is equal to |E an +1 | .

RecSys ’19, September 16–20, 2019, Copenhagen, Denmark

Kosetsu Tsukuda and Masataka Goto

Table 4: Example results for a user of NonDiv, ItemDiv, and DualDiv. Note: see Table 1 for example explanations generated
from each ES. (Abbreviations: The Human League (THL), Orchestral Manoeuvres in the Dark (OMD), Backstreet Boys (BB))

Rank
1
2
3
4
5

NonDiv
Recall@5=0.4, AILD@5=0.558
EILD@5=0.350, WEILD@5=0.137
ArtistTags ESs (Eas )THL electronic, 80s e 2 , e 3 , e 4
Anything Box electronic, 80s e 3 , e 4 , e 5
OMD electronic, 80s e 2 , e 3 , e 4
Dead or Alive electronic, 80s e 2 , e 3 , e 4Book of Love electronic, 80s e 2 , e 4 , e 5

ItemDiv
Recall@5=0.2, AILD@5=0.701
EILD@5=0.400, WEILD@5=0.0768
ArtistTags ESs (Eas )THL electronic, 80s e 2 , e 3 , e 4
Anything Box electronic, 80s e 3 , e 4 , e 5
OMD electronic, 80s e 2 , e 3 , e 4BBpop, 90s e 3 , e 4 , e 7*NSYNCpop, 90s e 3 , e 4 , e 7

Table 2: Comparison results.
Recall@5
Recall@10
Recall@20
AILD@5
AILD@10
AILD@20
EILD@5
EILD@10
EILD@20WEILD@5
WEILD@10
WEILD@20

NonDiv
0.115
0.190
0.311
0.680
0.715
0.745
0.391
0.454
0.517
0.117
0.122
0.128

ItemDiv

0.433
0.480
0.537
0.09850.100
0.0993

RandDiv
0.111
0.173
0.275
0.753
0.776
0.804
0.644
0.632
0.628
0.159
0.141
0.122

Table 3: Effects of parameter β in DualDiv.
DualDiv
EILD@5WEILD@5

0.731
0.699
0.686
0.184
0.157
0.135

which is computed as follows:∑∑a i ∈R kdiv

a j ∈R kdiv \{a i } dist(ai , a j ),(3)k(k − 1)
where dist(ai , a j ) represents the distance between two artists andis computed by dist(ai , a j ) = 1 − sim(ai , a j ) (see Section 2.4 about
the definition of sim(ai , a j )). The high value of AILD indicates that
artists in the recommendation list are diversified well. Similarly,
we use EILD (explanation style ILD) to evaluate the ES diversity.
EILD@k is also computed as in Eq. 3, but dist(ai , a j ) is given bydist(ai , a j ) = 1−∥Eas i ∩Eas j ∥/∥Eas i ∪Eas j ∥. EILD puts equal weights
between arbitrary pairs of artists. In contrast, we also evaluate the
ES diversity by using WEILD (weighted EILD), which is given by:∑∑a i ∈R kdiv a j ∈R kdiv \{a i } sim(ai , a j ) · dist(ai , a j )WEILD@k =.
k(k − 1)dist(ai , a j ) is the same as in EILD. By using WEILD, we can evaluate if dissimilar sets of ESs are selected between similar artists.
exp
Regarding NonDiv, Rk is used instead of Rkdiv for all metrics.
AILD@k =3.2

DualDiv
Recall@5=0.2, AILD@5=0.701
EILD@5=0.700, WEILD@5=0.207
ArtistTags ESs (Eas )THL electronic, 80s e 2 , e 3 , e 4
Anything Box electronic, 80s e 3 , e 4 , e 5
OMD electronic, 80s e 2 , e 5 , e 6BBpop, 90s e 4 , e 5 , e 7*NSYNCpop, 90s e 3 , e 5 , e 7

Results

Table 2 shows comparison results. Note that ItemDiv, RandDiv,
and DualDiv have the same values of Recall and AILD because
RandDiv and DualDiv use the same artist ranking as ItemDiv.
We can see that these three methods can improve AILD without
largely reducing Recall compared to NonDiv. In addition, in termsof EILD, DualDiv significantly outperforms the other three methods. Although RandDiv also outperforms NonDiv and ItemDiv,
when there are many, say seven, candidate ESs for an artist (i.e.,|Eac | = 7), RandDiv may be able to randomly select ESs that are
not overlapping with those of similar artists. However, when there
are, for example, only four candidate ESs, it rarely happens that theappropriate three ESs are selected by chance. In contrast, DualDiv

β =0
0.734
0.179

β = 0.25
0.731
0.184

β = 0.5
0.731
0.184

β = 0.75
0.730
0.184

β =1
0.725
0.184

can greedily select the three most appropriate ESs one by one regardless of the number of candidate ESs. Hence, DualDiv shows
better performance than RandDiv. In ItemDiv, because the artists
are diversified, the scores of sim(ai , a j ) tend to be lower than in
NonDiv; this leads to the lower WEILD than NonDiv. Nonetheless
DualDiv, which uses the same artist ranking with ItemDiv, outperforms NonDiv. These results indicate that DualDiv can successfully
select dissimilar sets of ESs, especially between similar artists.
Next, we evaluate the effects of parameter β in DualDiv. Whenβ is small, DualDiv aims to reduce duplications of ESs among all
artists. When β is large, it aims to reduce duplications especiallybetween similar artists. In Table 3, as expected, EILD is high forsmall β while WEILD is high for large β. Therefore, we can tune
DualDiv by changing the value of β depending on the type of ES
diversification we want to obtain.
Finally, Table 4 shows top five recommendation results of NonDiv, ItemDiv, and DualDiv for a user, where only characteristic
tags for each artist are listed. Due to space limitation, explanations
generated from each ES are not shown. In terms of tags, all artists
in NonDiv are related to “electronic” and “80s,” while in DualDiv,
two artists related to “pop” and “90s” are included. This enables
the user to browse a wide range of artists. In terms of ESs, DualDiv shows six kinds of ESs, while NonDiv and ItemDiv shows fourand five kinds of ESs, respectively. Therefore, even if this user puts
the highest priority on e 6 , she can find at least one artist explained
based on that ES. In addition, unlike ItemDiv, DualDiv can selectmore diverse sets of ESs for similar artists (i.e., five ESs for artists
ranked at 1-3 and four ESs for artists ranked at 4-5). This leads toa higher value of WEILD and increases the probability with which
the user accepts at least one of the recommended similar artists.

4

CONCLUSION

This paper proposed DualDiv, a framework for recommending
items by diversifying both the items and the ESs. Our experimental results using Last.fm dataset showed that DualDiv was able todiversify the artists and ESs without largely reducing Recall. As
future work, we plan to develop DualDiv in other domains such as
movies and sightseeing spots, because DualDiv as a general framework can be applied to any domain of item recommendations.
[Acknowledgments] This work was supported by JST ACCEL Grant
Number JPMJAC1602, Japan.

DualDivREFERENCES[1] G. Adomavicius and A. Tuzhilin. Toward the next generation of recommendersystems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):734–749, 2005.
[2] S. H. Bach, M. Broecheler, B. Huang, and L. Getoor. Hinge-loss markov random fields and probabilistic soft logic. The Journal of Machine Learning Research,
18(1):3846–3912, 2017.
[3] F. Belém, R. Santos, J. Almeida, and M. Gonçalves. Topic diversity in tag recommendation. In Proceedings of the 7th ACM Conference on Recommender Systems,
RecSys ’13, pages 141–148, 2013.
[4] R. Burke. Hybrid recommender systems: Survey and experiments. User Modelingand User-Adapted Interaction, 12(4):331–370, 2002.
[5] Z. Dou, S. Hu, K. Chen, R. Song, and J. R. Wen. Multi-dimensional search result
diversification. In Proceedings of the 4th ACM International Conference on Web
Search and Data Mining, WSDM ’11, pages 475–484, 2011.
[6] J. L. Herlocker, J. A. Konstan, and J. Riedl. Explaining collaborative filtering
recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, CSCW ’00, pages 241–250, 2000.
[7] P. Kouki, S. Fakhraei, J. Foulds, M. Eirinaki, and L. Getoor. HyPER: A flexible
and extensible probabilistic framework for hybrid recommender systems. In Proceedings of the 9th ACM Conference on Recommender Systems, RecSys ’15, pages
99–106, 2015.
[8] P. Kouki, J. Schaffer, J. Pujara, J. O’Donovan, and L. Getoor. Personalized explanations for hybrid recommender systems. In Proceedings of the 24th International
Conference on Intelligent User Interfaces, IUI ’19, pages 379–390, 2019.
[9] P. Pu, L. Chen, and R. Hu. A user-centric evaluation framework for recommendersystems. In Proceedings of the 5th ACM Conference on Recommender Systems,
RecSys ’11, pages 157–164, 2011.

RecSys ’19, September 16–20, 2019, Copenhagen, Denmark[10] R. L. Santos, C. Macdonald, and I. Ounis. Exploiting query reformulations forweb search result diversification. In Proceedings of the 19th International Conference on World Wide Web, WWW ’10, pages 881–890, 2010.
[11] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Item-based collaborative filtering
recommendation algorithms. In Proceedings of the 10th International Conferenceon World Wide Web, WWW ’01, pages 285–295, 2001.
[12] A. Sharma and D. Cosley. Do social explanations work?: Studying and modelingthe effects of social explanations in recommender systems. In Proceedings of the
22nd International Conference on World Wide Web, WWW ’13, pages 1133–1144,
2013.
[13] N. Tintarev. Explanations of recommendations. In Proceedings of the 1st ACM
Conference on Recommender Systems, RecSys ’07, pages 203–206, 2007.
[14] S. Vargas, L. Baltrunas, A. Karatzoglou, and P. Castells. Coverage, redundancy
and size-awareness in genre diversity for recommender systems. In Proceedingsof the 8th ACM Conference on Recommender Systems, RecSys ’14, pages 209–216,
2014.
[15] J. Vig, S. Sen, and J. Riedl. Tagsplanations: Explaining recommendations usingtags. In Proceedings of the 14th International Conference on Intelligent User Interfaces, IUI ’09, pages 47–56, 2009.
[16] Y. Zhang, G. Lai, M. Zhang, Y. Zhang, Y. Liu, and S. Ma. Explicit factor models
for explainable recommendation based on phrase-level sentiment analysis. In
Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’14, pages 83–92, 2014.
[17] Y. Zhang, J. Tang, Z. Yang, J. Pei, and P. S. Yu. Cosnet: Connecting heterogeneous
social networks with local and global consistency. In Proceedings of the 21st ACM
SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’15, pages 1485–1494, 2015.
[18] C.-N. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen. Improving recommendation lists through topic diversification. In Proceedings of the 14th International
Conference on World Wide Web, WWW ’05, pages 22–32, 2005.