Justifying Recommendations using Distantly-Labeled Reviews andFine-Grained Aspects
Jianmo Ni, Jiacheng Li, Julian McAuley
University of California, San Diego{jin018,j9li,jmcauley}@ucsd.edu

Abstract
Several recent works have considered the
problem of generating reviews (or ‘tips’) as aform of explanation as to why a recommendation might match a user’s interests. While
promising, we demonstrate that existing approaches struggle (in terms of both quality
and content) to generate justifications that are
relevant to users’ decision-making process.
We seek to introduce new datasets and methods to address this recommendation justification task. In terms of data, we first propose an ‘extractive’ approach to identify review segments which justify users’ intentions;this approach is then used to distantly label
massive review corpora and construct largescale personalized recommendation justification datasets. In terms of generation, we
design two personalized generation models
with this data: (1) a reference-based Seq2Seq
model with aspect-planning which can generate justifications covering different aspects,and (2) an aspect-conditional masked language
model which can generate diverse justifications based on templates extracted from justification histories. We conduct experiments on
two real-world datasets which show that ourmodel is capable of generating convincing anddiverse justifications.

1

Introduction

Explaining, or justifying, recommendations tousers has the potential to increase their transparency and reliability. However providing meaningful interpretations remains a difficult task,
partly due to the black-box nature of many recommendation models, but also because we simply
lack ground-truth datasets specifying what ‘good’justifications ought to look like.
Previous work has sought to learn user preferences and writing styles from crowd-sourced reviews (Dong et al., 2017; Ni and McAuley, 2018)

Review examples:
I love this little stand! The coconut mocha chiller andcaramel macchiato are delicious.
Wow what a special find. One of the most unique andspecial date nights my husband and I have had.
Tip examples:Great food. Nice ambiance. Gnocchi were very good.
I can’t get enough of this place.
Justification examples:
The food portions were huge.
Plain cheese quesadilla is very good and very cheap.

Table 1: In contrast to reviews and tips, we seek toautomatically generate recommendation justifications
that are more concise, concrete, and helpful for decision making. Examples of justifications from reviews,
tips, and our annotated dataset are marked in bold.

to generate explanations in the form of natural language, e.g. generating synthesized reviews similar to those that users would write about a product. However, a large portion of review text (ortext from ‘tips’) is often of little relevance to mostusers’ decision making (e.g. they describe verbose
experiences or general endorsements) and may not
be appropriate to use as explanations in terms ofcontent and language style. As a result, existingmodels that learn directly from reviews (or tips)
may not capture crucial information that explains
users’ purchases. Table 1 shows examples of reviews, tips and ideal justifications. More recently,
there has been work studying the task of tip generation where tips are concise summaries of reviews(Li et al., 2017). Though tips are concise and somesubset of them might be suitable as candidates
for recommendation justifications, only a few ecommerce systems provide tips accompanied withreviews. Even in systems where tips are available,
the number of tips is usually far smaller than thenumber of reviews. These approaches hence sufferfrom generalizability issues, especially in settingswhere user interactions are highly sparse.
On the other hand, generating diverse responses
is essential in personalized content generation scenarios such as justification generation. Insteadof always predicting the most popular reasons,
it’s preferable to present diverse justifications fordifferent users based on their personal interests.
Recent work has shown that incorporating priorknowledge into generation frameworks can greatly
improve diversity. Prior knowledge could include
story-lines in story generation (Yao et al., 2019),
or historical responses in dialogue systems (Weston et al., 2018).
In this work, our goal is to generate convincing
and diverse justifications. To address the challengeof lacking ground-truth data about ‘good’ justifications, we propose a pipeline that can identify
justifications from massive corpora of reviews or
tips. We extract fine-grained aspects from justifications and build user personas and item profiles
consisting of sets of representative aspects. To
improve generation quality and diversity, we propose two generation models (1) a reference-based
Seq2Seq model with aspect-planning, which takes
previous justifications as a reference and can produce justifications based on different aspects, and(2) an aspect-conditional masked language modelthat can generate diverse justifications from templates extracted from previous justifications.
Our contributions are threefold:• To facilitate recommendation justification generation, we propose a pipeline to identify justification candidates and build aspect-based user
personas and item profiles from massive corpora of reviews. With this approach, we areable to build large-scale personalized justification datasets. We use these extractive justification segments in the task of explainable recommendation and show that these are better training sources instead of whole reviews.
• We propose two models based on reference
attention, aspect-planning techniques and apersona-conditional masked language model.
We show that adding such personalized information enables the models to generate justifications
with high quality and diversity.
• We conduct extensive experiments on two realworld datasets from Yelp and Amazon Clothing.
We provide an annotated dataset about ‘good’justifications on the Yelp dataset and show thatthe binary classifier trained on this dataset gener-

alizes well to the Amazon Clothing dataset. We
study different decoding strategies and compare
their effect on generation performance.

2

Dataset Generation

In this section, we introduce the pipeline to extract high quality justifications from raw user reviews. Specifically, our goal here is to identify
review segments that can be used as justifications
and build a personalized justification dataset uponthem. Our pipeline consists of three steps:
1. Annotating a set of review segments with binary
labels, i.e., to determine whether they are ‘good’or ‘bad’ justifications.
2. Training a classifier on the annotated subset andapplying it to distantly label all the review segments to extract ‘good’ justifications for eachuser and item pair.
3. Applying fine-grained aspect extraction for the
extracted justifications, and building user personas and item profiles.
2.1

Identifying Justifications From Reviews

The first step is to extract text segments from reviews that are appropriate to use as justifications.
Instead of a complete sentence or phrase, we define each segment as an Elementary Discourse
Unit (EDU; (Mann and Thompson, 1988)) whichcorresponds to a sequence of clauses. We use themodel of Wang et al. (2018) to obtain EDUs from
reviews. Recent works have shown that EDUs canimprove the performance of document-level summarization (Bhatia et al., 2015) and opinion summarizaiton (Angelidis and Lapata, 2018).
After preprocessing the reviews into EDUs, weanalyzed the linguistic differences between recommendation justifications and reviews, and built
two rules to filter the segments that are unlikely
to be suitable justifications: (1) segments with
first-person or third-person pronouns, and (2) too
long or short. Next, two expert annotators were
exposed to 1,000 segments among those not filtered out and asked to determine whether they are‘good’ justifications. Labeling was performed iteratively, followed by feedback and discussion, until the quality was aligned between the two annotators. At the end of the process, the interannotator agreement for the binary labeling task
(good vs. bad), measured by Cohen’s kappa (Cohen, 1960), was 0.927 after alignment. Then, theannotators further labeled 600 segments. Overall,Method
BOW-Xgboost
CNN
LSTM-MaxPool
BERT
BERT-SA (one epoch)
BERT-SA (three epoch)

F1

RecallPrecision

0.559
0.644
0.675
0.747
0.481
0.491

0.679
0.596
0.703
0.700
0.975
1.000

0.475
0.700
0.650
0.800
0.320
0.325

Table 2: Performance for classifying review segments
as good or bad for recommendation justification.

24.8% of the segments were labeled good.
2.2

Yelp
The Tuna is pretty amazing
Appetizers and pasta are excellent here
An excellent selection of both sweet and savory crepes
It was filled with delicious food, fantastic music anddancingAmazon-Clothing
The quality of the material is great
Great shirt, especially for the price.
The seams and stitching are really niceFit the bill for a Halloween costume.

Table 3: Examples of justifications with fine-grained
aspects in our annotated dataset. The fine-grained aspects are italic and underlined.

Automatic Classification

Our next step is to propagate labels to the complete
review corpus. Here we adopt BERT (Devlin et al.,2019) to fine-tune on our classification task, wherea [CLS] token is added to the beginning of eachsegment and the final hidden state (i.e., output of
BERT) corresponding to this token is fed into alinear layer to obtain the binary prediction. Cross
entropy is used as the training loss.
We split the annotated dataset into Train, Dev,
and Test sets with a 0.8/0.1/0.1 ratio, fine-tune the
BERT classifier on the Train set and choose thebest model on the Dev set. After three epochsof fine-tuning, BERT can achieve an F1-scoreof 0.80 on the Test set. We compare the performance of BERT with multiple baseline models: (1) a XGBoost model which uses Bags-ofWords as sentence features (2) a convolutional
neural network (CNN) with three convolution layers and one linear layer (3) a long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997)
network with a max-pooling layer, and a linear
layer (4) a BERT sentiment classifier (BERT-SA)
trained on the complete Yelp dataset for one epoch
and three epochs. To obtain the pre-trained wordembeddings for the CNN and LSTM models, weapplied fastText (Bojanowski et al., 2016) on the
Yelp Review dataset. We set the embedding dimension to 200 and used default values for otherhyper-parameters.
Table 2 presents results for our binary classification task. The BERT classifier has higher F1-scoreand precision than other classifiers. The BERTSA model after three epochs only achieves an F1score of 0.491, which confirms the difference between sentiment analysis and our good/bad task,i.e., even if the segment has positive sentiment, it
might be not suitable as a justification.

2.3

Fine-grained Aspect Extraction

Finally, we extract the fine-grained aspects thateach justification covers. Fine-grained aspects
are properties of products that appear among auser’s opinions. We adopt the method proposedby Zhang et al. (2014) to build a sentiment lexicon which includes a set of fine-grained aspects
from the whole dataset. We then use simple rules
to determine which aspects appear in each justification.1 Table 3 presents a set of examples fromour dataset. Each example consists of a justification that a user has written about an item, andmultiple fine-grained aspects mentioned in the justification. Note that we only annotated the Yelp
dataset, trained a classifer on that and applied themodel on both Yelp and Amazon Clothing dataset.
As shown in Table 3, the trained classifier works
well on both datasets.

3

Approach

3.1

Problem Definition

For each user u (or item i), we build a justificationreference D = {d1 , . . . , dlr } consisting of justifications that the user has written (or justifications
about the item) on the training set, where lr is the
maximum number of justifications. We also obtaina user persona (or item profile) A = {a1 , . . . , aK }based on the fine-grained aspects that the user’s (oritem’s) previous justifications have covered, where
K is the maximum number of aspects.
Given a user u and an item i, as well as the theirjustification reference Du and Di , and u’s persona
Au and i’s profile Ai , our target is to predict the1

For each aspect, if its singular or plural exists in the tokenized justification, then we consider that this aspect exists
in that justification.

P(food)

Projection Layer
User
FA
2
Item
⨁
⨁ ℎ
⨁
Representation
Representation
Representationℎ2

Attention Layer
RNN Layer

Last
HiddenEmbedding Layer
User
Reference

Item
Reference

Fine-grained
Aspect

The

food

ℎ1

ℎ2

ℎ3

<SOS>

The

food

portions were
ℎ4

huge
ℎ5

portions were

Figure 1: Structure of the reference-based Seq2Seq model with Aspect Planningjustifications Ju,i = {w1 , w2 , . . . , wT } that would
explain why item i fits user u’s interests, where T
is the length of the justification.
3.2

Reference-based Seq2Seq Model

Our base model follows the structure of a standard Seq2Seq (Sutskever et al., 2014) model. Our
framework, called ‘Ref2Seq’, views the historicaljustifications of users and items as references andlearns latent personalized features from them. Figure 1 shows the structure of our Reference-based
Seq2Seq Model. It includes two components: (1)two sequence encoders that learn user and item
latent representations by taking previous justifications as references; (2) a sequence decoder incorporating representations from users and items togenerate personalized justifications.
Sequence Encoders. Our user encoder and sequence encoder share the same structure, which
includes an embedding layer, a two-layer bidirectional GRU (Cho et al., 2014), and a projection layer. The inputs are a user (or item) reference
D consisting of a set of historical justifications.
These justifications pass a word embedding layer,then go through the GRU and yield a sequence ofhidden states e ∈ Rls ×lr ×n :→←E = Embedding(D), e = GRU(E) = e + e ,(1)
where ls denotes the length of the sequence, n is
the hidden size of the encoder GRU, E ∈ Rls ×lr ×n→
is the embedded sequence representation, and e←
and e are the hidden vectors produced by a forward and a backward GRU (respectively).
To combine information from different ‘references’ (i.e. justifications), the hidden states arethen projected via a linear layer:ê = We · e + be ,(2)

where ê ∈ Rls ×n is the final output of the encoder,and We ∈ Rlr , be ∈ R are learned parameters.
Sequence decoder. The decoder is a two-layer
GRU that predicts the target words given a start token. The hidden state of the decoder is initializedusing the sum of the last hidden state of the user
and item encoders. The hidden state at time-step t
is updated via the GRU unit based on the previous
hidden state and the input word. Specifically:h0 = euls + eils , ht = GRU(wt , ht−1 ),(3)where euls and eils are the last hidden states of theuser and item encoder output êu and êi .
To explore the relation between the reference
and generation, we apply an attention fusion layer
to summarize the output of each encoder. For theuser and item reference encoder, the attention vector is defined as:
a1t =

ls
X

1
αtj
ej ,

j=1
>
1
αtj
= exp(tanh(vα1 (Wα1 [ej ; ht ] + b1α )))/Z,(4)
where a1t ∈ Rn is an attention vector on the se1 is an attention
quence encoder at time-step t, αtjscore over the encoder hidden state ej and decoder
hidden state ht , and Z is a normalization term.
Aspect-Planning Generation. One of the challenges for generating justifications is how to improve controllability, i.e., directly manipulate thecontent being generated. Inspired by ‘plan-andwrite’ (Yao et al., 2019), we extend the base modelto an Aspect-Planning Ref2Seq (AP-Ref2Seq)
model where we plan a fine-grained aspect before
generation. This aspect planning can be considered as an extra form of supervision instead ofa hard constraint to make justification generation
more controllable.

When generating the justification for user u anditem i, we first provide a fine-grained aspect a as aplan. The aspect a is fed into the word embedding
layer to obtain the aspect embedding Ea . Then, we
compute the scores between the embedding of the
aspect and the decoder hidden state as:
a2t = αt2 Ea ,>αt2 = exp(tanh(vα2 (Wα2 [Ea ; ht ] + b2α )))/Z,(5)
where a2t ∈ Rn is an attention vector and αt2 is anattention score.
The attention vectors a1ut of user u, a1it of item i,and a2t of fine-grained aspect a, are concatenatedwith the decoder hidden state at time-step t andprojected to obtain the output word distribution P .
The output probability for word w at time-step t isgiven by:p(wt ) = tanh(W1 [ht ; a1ut ; a1it ; a2t ] + b1 ),(6)

where wt is the target word at time-step t. Given
the probability p(wt ) at each time step t, the modelis trained using a cross-entropy loss comparedagainst the ground-truth sequence.
3.3

Aspect Conditional Masked LanguageModelu wrote about item i, we adapt the pre-trained
BERT model (Devlin et al., 2019) into an encoderdecoder network with (1) an aspect encoder which
encodes the user persona and item profile into latent representations and (2) a masked language
model sequence decoder that takes in a masked
justification and predicts the masked tokens.
Aspect Encoder. Our aspect encoder shares
the same WordPiece embeddings (Wu et al., 2016)as BERT. The encoder feeds the intersection offine-grained aspects from the user persona anditem profile Aui = {a1 , . . . , aK 0 } into the embedding layer and obtains the aspect embedding0
Aui ∈ RK ×n , where K 0 is the number of common fine-grained aspects and n is the dimensionof the WordPiece embeddings.
Masked Language Model Sequence Decoder.
We use the masked language model in the pretrained BERT model as our sequence decoder andadd attention over the aspect encoder’s output. As
shown in Figure 2, the input to the decoder is a
M = {w , . . . , w } withmasked justification Ju,i
1
T
multiple tokens be replaced as [MASK]. The decoder’s output T ∈ RT ×n is then fed to the attention layer to calculate an attention score with theoutput of the encoder:0

Though Seq2Seq-based models can achieve highquality output, they often fail to generate diverse
content. Recent works in natural language generation (NLG) tried to combine generation methods
with information retrieval techniques to increase
the generation diversity (Li et al., 2018; Baheti
et al., 2018). The basic idea follows the paradigmof retrieve-and-edit—which is to first retrieve historical responses as templates, and then edit the
template into new content. Since our data is annotated with fine-grained aspects, it naturally fits into
this type of retrieve-and-edit paradigm. Meanwhile, masked language models have shown great
performance in language modeling. Recent work
(Wang and Cho, 2019; Mansimov et al., 2019) has
shown that by sampling from the masked language
model (e.g. BERT), it is able to generate coherent
sentences.
Inspired by this work, we want to extend suchan approach into a conditional version—we explore the use of an Aspect Conditional Masked
Language Model (ACMLM) to generate diverse
personalized justifications. Figure 2 shows the
structure of our Aspect Conditional Masked Language Model. For a justification Ju,i that user

a3t =

K
X

3
αtj
Aj ,

j=1
>

3
αtj
= exp(tanh(vα3 (Wα3 [Aj ; Tt ] + b3α )))/Z.
(7)
The attention vector a3t is then concatenatedwith the decoder hidden state at time-step t andsent to a linear projection layer to obtain the output word distribution P . The output probability
for word w at time-step t is given by:p(wt ) = tanh(W2 [Tt ; a3t ] + b2 )(8)

where wt is the target word at time-step t.
Masking Procedure. The original BERT paper applies a flat rate (15%) to decide whether tomask a token. Unlike their approach, we adopt ahigher rate to mask fine-grained aspects since they
are more important in justifications. Specifically,
if we encounter a fine-grained aspect, we will replace it with a [MASK] token 30% of the time;
while for other words, we will replace them with a[MASK] token 15% of the time.
During training, the model will only predict
those masked tokens and calculate a cross-entropy
loss on them.

Projection Layer

P(atmosphere)

P(service)atmosphere andservice

are

top

notch

𝐶𝐶

𝑇𝑇 1

𝑇𝑇 2

𝑇𝑇 3

BERT

𝑇𝑇 4

𝑇𝑇 5

𝑇𝑇 6

𝐸𝐸 𝐶𝐶𝐶𝐶𝐶𝐶

𝐸𝐸 1

𝐸𝐸 2

𝐸𝐸 3

𝐸𝐸 4

𝐸𝐸 5

𝐸𝐸 6

Attention Layer
Embedding Layer
Fine-grained Aspects′

𝐴𝐴𝑢𝑢𝑢𝑢 = [𝐴𝐴1𝑢𝑢𝑢𝑢 , … , 𝐴𝐴𝑘𝑘𝑢𝑢𝑖𝑖 ][CLS][MASK]

and[MASK]

are

top

notch

Figure 2: Structure of the Aspect Conditional Masked Language Model

Iter 0Iter 5Iter 10Iter 15Iter 20

universe [MASK] is extremely friendly and persona ##ble
the [MASK] is extremely friendly and persona##ble
the [MASK] is extremely friendly and persona##ble
the staff are extremely cool and persona ##ble
the staff are extra kind , persona ##ble

Table 4: Examples of the generation output of
ACMLM at different iterations.

Generation by Sampling from Masked Templates. We next discuss how to generate justifications from the trained ACMLM. We follow thesampling strategy of Wang and Cho (2019) to generate justifications. Instead of generating froma sequence of all [MASK] tokens, we start withmasked templates generated from historical justifications about the target item. These masked templates include prior knowledge about the item andcan increase the speed of sampling convergence.
Table 4 shows an example of the generation process. We initialize the template sequence X 0 as
(universe, [MASK], . . . , ##ble) with length T . At
each iteration i, a position ti is sampled uniformly
at random from {1, . . . , T } and the token at ti(i.e. xiti ) of the current sequence X i is replacedby [MASK]. After that, we obtain the conditional
probability of xti as1i
exp(1h(xti )> fθ (X\t
))),
i )
i
Z(X\t
i
(9)
where 1h(xti ) is a one-hot vector with index xti
i is the sequence we obtain after reset to 1, X\tiplacing the token at position ti of X i by [MASK],i ) is the output after feeding X ifθ (X\t\ti intoithe ACMLM as in Equation (8), and Z is thenormalization term. We then sample x̃ti from
Equation (9), and construct the next sequence byip(xti |X\t=i

X i+1 = (xi1 , . . . , x̃ti , . . . , xiT ). After repeating
this procedure N times, the final output is considered as the generation output.2

4

Experiments

4.1

Datasets

With our proposed pipeline (Section 2), we construct two personalized justification datasets fromexisting review data—Yelp and Amazon Clothing.34 We further filter those users with fewer than
five justifications. For each user, we randomly
hold out two samples from all of their justifications to construct the Dev and Test sets. Table 5shows the statistics of our two datasets.
4.2

Baselines

For automatic evaluation, we consider three baselines: Item-Rand is a baseline which randomly
chooses a justification from the item’s historicaljustifications. LexRank is a strong unsupervised
baseline that is widely used in text summarization(Erkan and Radev, 2004). Given all historical justifications about an item, LexRank can select one
justification as the summary. We then use thatas the justification for all users. Attr2Seq (Donget al., 2017) is a Seq2Seq baseline that uses attributes (i.e. user and item identity) as input.
By default, all models use beam search during generation. Recently, there have been works
showing that the generation output of sampling
methods is more diverse and suitable on highentropy tasks (Holtzman et al., 2019). To this end,
we explore another decoding strategy— ‘Top-k
sampling’ (Radford et al., 2019) in experiments
2
We set N proportional to the length T of the initialmasked template to prevent the generation diverging too
much from the original template.
3
https://www.yelp.com/dataset/challenge4
http://jmcauley.ucsd.edu/data/amazon

Dataset
Yelp
Amazon Clothing

Train

Dev

Test

# Users# Items# Aspects

1,219,962
202,528

115,907
57,947

115,907
57,947

115,907
57,94751,94850,2402,041
581

Table 5: Statistics of our datasets.
DatasetModel
Item-Rand
LexRank
Attr2seq
Ref2Seq
AP-Ref2Seq
Ref2Seq (Top-k)
ACMLM

Yelp
BLEU-3
0.440
2.290
7.890
4.380
3.390
1.630
0.700

BLEU-4
0.150
0.920
0.000
2.450
1.830
0.700
0.280

Distinct-1
2.766
1.738
0.049
0.188
0.326
0.818
1.322

Amazon Clothing
Distinct-2
20.151
8.509
0.095
1.163
2.094
11.92714.319BLEU-3
1.620
3.480
1.720
8.780
13.9103.960
2.420

BLEU-4
0.680
2.250
0.560
5.670
12.5002.130
1.590

Distinct-1
2.400
2.407
0.076
0.141
0.557
0.697
0.942

Distinct-2
11.85314.956
0.352
1.240
3.661
10.858
9.312

Table 6: Performance on Automatic Evaluation.

and include a variant of our model: Ref2Ref (Topk).5
For human evaluation, we include two baselines: Ref2Seq (Review) and Ref2Seq (Tip), bothof which are the same model as Ref2Seq modelbut trained on the original review and tip data,
respectively. Comparisons with these two baselines demonstrates that training on our annotateddataset tends to generate text more suitable as justifications.
4.3

Implementation DetailWe use PyTorch6 to implement our models.
For Req2Seq and AP-Ref2Seq, we set the hidden size and word embedding size as 256. We apply a dropout rate of 0.5 for the encoder and 0.2
for the decoder. The size of the justification reference lr is set to 5 and the number of fine-grained
aspects K in the user persona and item profile isset to 30. We train the model using Adam withlearning rate 2e−4 and stop training either when
it reaches 20 epochs or the perplexity does not improve (on the Dev set). For ACMLM, we build ourmodel based on the BERT implementation from
HuggingFace.7 We initialize our decoder using
the pre-trained ‘Bert-base’ model and set the max
sequence length to 30. We train the model for 5
epochs using Adam with learning rate 2e−5 . For
models using beam search, we set the beam sizeas 10. For models using ‘top-k’ sampling, we set
5
At each time step, the next word is sampled from the top
k possible next tokens, according to their probabilities.
6
http://pytorch.org/docs/master/index.html
7
https://github.com/huggingface/pytorch-pretrainedBERT

Model
Ref2Seq (Review)
Ref2Seq (Tip)
Ref2Seq
Ref2Seq (Top-k)
ACMLM

R

I

D

3.02
3.25
3.87
3.95
3.23

2.39
2.35
3.13
3.34
3.29

2.10
2.34
2.96
3.39
3.42

Table 7: Performance on Human Evaluation, where
R,I,D represents Relevance, Informativeness andDiversity, respectively.

k to 5. For ACMLM, we use a burn-in step equal
to the length of the initial sequence. Our data andcode are available online.8
4.4

Automatic Evaluation

For automatic evaluation, we use BLEU, Distinct1, and Distinct-2 (Li et al., 2015) to measure theperformance of our model. As shown in Table 6,
our reference-based models achieve the highest
BLEU scores on both datasets except for BLEU3 on Yelp. This confirms that Ref2Seq is able
to capture user and item content to generate themost relevant content, compared with unpersonalized models such as LexRank and personalized
models that do not leverage historical justifications
such as Attr2Seq.
On the other hand, recent works have reported
that models achieving higher diversity scores
will have lower scores on overlap-based metrics(e.g. BLEU) for open-domain generation tasks(Baheti et al., 2018; Gao et al., 2018). We makea similar observation for our personalized justifi8

https://github.com/nijianmo/recsys justification.git

ModelShake Shack

Teharu Sushi

MGM Grand Hotel

Ground Truth

The burger was good

The rolls are pretty great , typical rolls not that many specials

Room was very clean comfortable

LexRankA great burger and fries.

Sushi ?

Great rooms.

Ref2Seq (Review)

i love trader joe ’s , i love trader
joe ’sthe food was good and the service was greati love this place ! the food isalways good and the service isalways great

Ref2Seq (Tip)this place is awesome

love this placecome here

Ref2Seq

this place has some of the best
burgers

the sushi is delicious

the room was nice

Ref2Seq (Top-k)

the fries are amazing

fresh and delicious sushi

open hotel for hours

ACMLM

breakfast sandwiches are overall very filling

overall fun experience with half
price sushi

family style dinner , long time
shopping trip to vegas, family
dining , cheap lunch

Table 8: Comparisons of the generated justifications from different models for three businesses on the Yelp dataset.

cation generation task. As shown in Table 6, both
sampling-based methods Ref2Seq (Top-k) andACMLM achieve higher Distinct-1 and Distinct-2,
while their BLEU scores are lower than Seq2Seq
based models using beam search. Therefore, we
also perform human evaluation to validate the generation quality of our proposed methods.
4.5

Human Evaluation

We conduct human evaluation on three aspects:(1) Relevance measures whether the generated
output contains information relevant to an item;(2) Informativeness measures whether the generated justification includes specific information thatis helpful to users; and (3) Diversity measures how
distinct the generated output is compared withother justifications.
We focus on the Yelp dataset and sample 100
generated examples from each of the five models as shown in Table 7. Human annotators are
asked to give a score in the range [1,5] (lowest to
highest) for each metric. Each example is rated
by at least three annotators. The results showthat both Ref2Seq (Top-k) and ACMLM achieve
higher scores on Diversity and Informativenesscompared to other models.
4.6

Qualitative Analysis

Here we study the following two qualitative questions:
RQ1: How do training data and methods affectgeneration? As Table 8 shows, models trained onreviews and tips tend to generate generic phrases(such as ‘i love this place’) which often do notinclude information that helps users to make de-DatasetAspects

Generated Output

Yelp

diningpastrychickensandwichthe dining room is nicethe pastries were pretty goodthe chicken fried rice is the best
the pulled pork sandwich is thebest thing on the menu

product
priceleathergreat product , fast shippong
design is nice , good pricecomfortable leather sneakers .
classic
sturdy , great city walking shoes

AmazonClothing

walking

Table 9: Generated justifications from AP-Ref2Seq.
The planned aspects are randomly selected from users’
personas.

cisions. Other models trained on the justification datasets tend to mention concrete information(e.g. different aspects). LexRank tends to generate
relevant but short content. Meanwhile, samplingbased models are able to generate more diverse
content.
RQ2: How does aspect planning affect generation? To mitigate the trade-off between diversity and relevance, one approach is to add moreconstraints during generation such as constrained
Beam Search (Anderson et al., 2017). In our work,
we extend our base model Ref2Seq by incorporating aspect-planning to guide generation. As
shown in Table 9, most planned aspects are present
in the generated outputs of AP-Req2Seq.

5

Related Work

Explainable Recommendation There has beena line of work that studies how to improve theexplainability of recommender systems. Catherine and Cohen (2017) learn latent representationsof review text to predict ratings. These representations are then used to find the most helpful reviews for given a particular user and item
pair. Another popular direction is to generate text
to justify recommendations. Dong et al. (2017)
proposed an attribute-to-sequence model to generate product reviews which utilizes categorical attributes. Ni et al. (2017) developed a multi-task
learning method that considers collaborative filter and review generation. Li et al. (2019b) generated tips by considering ‘persona’ information
which can capture the language style of users andcharacteristics of items. However, these works use
whole reviews or tips as training examples, which
may not be appropriate due to the quality of review text. More recently, Liu et al. (2019) proposed a framework to generate fine-grained explanations for text classification. To achieve labels forhuman-readable explanations, they constructed adataset from a website which provides ratings andfine-grained summaries written by users. Unfortunately, most websites do not provide such finegrained information. On the other hand, our work
identifies justifications from reviews, uses them as
training examples and shows these are better data
source for explainable recommendation via extensive experiments.

get sentiment. In this work, we also introduces aconditional masked language model but considers
more fine-grained aspects.

Diversity-aware NLG Diversity is an important
aspect of NLG systems. Recent works have focused on digesting prior knowledge to improvegeneration diversity. Yao et al. (2019) proposed amethod to incorporate planned story-lines in story
generation. Li et al. (2019a) developed an aspectaware coarse-to-fine review generation method.
They predict an aspect for each sentence in thereview to capture the content flow. Given the aspects, a sequence of sentence sketches is generated and a decoder will fill in the slots of eachsketch. In dialogue systems, several works havestudied frameworks to extract templates from historical responses, which are then edited to form
new responses (Weston et al., 2018; Wu et al.,2018). Similarly, the extract-and-edit paradigm
has been studied in style transfer tasks in NLG(Li et al., 2018). Wu et al. (2019) proposed an
attribute aware masked language model for nonparallel sentiment transfer. They first mask out thesentimental tokens and then train a masked language model to infill the masked positions for tar-

References

6

Conclusion

In this work, we studied the problem of personalized justification generation. To build high quality justification datasets, we provided an annotateddataset and proposed a pipeline to extract justifications from massive review corpora. To generate convincing and diverse justifications, we developed two models: (1) Ref2Seq which leverages historical justifications as references during generation; and (2) ACMLM, which is an
aspect conditional model built on a pre-trained
masked language model. Our experiments showedthat Ref2Seq achieves higher scores (in terms of
BLEU) and ACMLM achieves higher diversity
scores compared with baselines. Human evaluation showed that reference-based models obtainhigh relevance scores and sampling based methods
led to more diverse and informative outputs. Finally, we showed that aspect-planning is a promising way to guide generation to produce personlized and relevant justifications.
Acknowledgements. This work is partly supported by NSF #1750063. We thank all the reviewers for their constructive suggestions.

Peter Anderson, Basura Fernando, Mark Johnson, andStephen Gould. 2017. Guided open vocabulary image captioning with constrained beam search. In
EMNLP.
Stefanos Angelidis and Mirella Lapata. 2018. Summarizing opinions: Aspect extraction meets sentimentprediction and they are both weakly supervised. In
EMNLP.
Ashutosh Baheti, Alan Ritter, Jiwei Li, and William B.
Dolan. 2018. Generating more interesting responses
in neural conversation models with distributional
constraints. In EMNLP.
Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein.
2015. Better document-level sentiment analysis
from rst discourse parsing. In EMNLP.
Piotr Bojanowski, Edouard Grave, Armand Joulin, andTomas Mikolov. 2016. Enriching word vectors withsubword information. Transactions of the Association for Computational Linguistics, 5:135–146.
Rose Catherine and William W. Cohen. 2017.
Transnets: Learning to transform for recommendation. In RecSys.

Kyunghyun Cho, Bart van Merrienboer, aglar Gülehre,
Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using
rnn encoder-decoder for statistical machine translation. In EMNLP.

Jianmo Ni, Zachary C. Lipton, Sharad Vikram, and Julian J. McAuley. 2017. Estimating reactions and recommending products with generative models of reviews. In IJCNLP.

Jacob Willem Cohen. 1960. A coefficient of agreement
for nominal scales.

Jianmo Ni and Julian McAuley. 2018. Personalized review generation by expanding phrases and attending
on aspect-aware representations. In ACL.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. Bert: Pre-training of deep
bidirectional transformers for language understanding. In NAACL.

Alec Radford, Jeff Wu, Rewon Child, David Luan,
Dario Amodei, and Ilya Sutskever. 2019. Language
models are unsupervised multitask learners.

Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata,
Ming Zhou, and Ke Xu. 2017. Learning to generate
product reviews from attributes. In EACL.
Günes Erkan and Dragomir R. Radev. 2004. Lexrank:Graph-based lexical centrality as salience in text
summarization. J. Artif. Intell. Res., 22:457–479.
Jun Gao, Wei Bi, Xiaojiang Liu, Junhui Li, and Shuming Shi. 2018. Generating multiple diverse responses for short-text conversation. In AAAI.

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.
Sequence to sequence learning with neural networks. In NIPS.
Alex Wang and Kyunghyun Cho. 2019. Bert has amouth, and it must speak: Bert as a markov random
field language model. CoRR, abs/1902.04094.
Yizhong Wang, Sujian Li, and Jingfeng Yang. 2018.
Toward fast and accurate neural discourse segmentation. In EMNLP.

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long
short-term memory. Neural Computation, 9:1735–
1780.

Jason Weston, Emily Dinan, and Alexander H. Miller.
2018. Retrieve and refine: Improved sequence generation models for dialogue. In SCAI@EMNLP.

Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin
Choi. 2019. The curious case of neural text degeneration. CoRR, abs/1904.09751.

Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han,
and Songlin Hu. 2019. Mask and infill: Applying
masked language model to sentiment transfer. In IJCAI.

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao,and William B. Dolan. 2015. A diversity-promoting
objective function for neural conversation models.
In HLT-NAACL.
Juncen Li, Robin Jia, He He, and Percy S. Liang. 2018.
Delete, retrieve, generate: A simple approach to sentiment and style transfer. In NAACL-HLT.
Junyi Li, Wayne Xin Zhao, Ji-Rong Wen, and YangSong. 2019a. Generating long and informative reviews with aspect-aware coarse-to-fine decoding. In
ACL.
Piji Li, Zihao Wang, Lidong Bing, and Wai Lam.
2019b. Persona-aware tips generation. In WWW.
Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and
Wai Lam. 2017. Neural rating regression with abstractive tips generation for recommendation. In SIGIR.
Hui Liu, Qingyu Yin, and William Yang Wang. 2019.
Towards explainable nlp: A generative explanation
framework for text classification. In ACL.
William C. Mann and Sandra A. Thompson. 1988.
Rhetorical structure theory: toward a functional theory of text.
Elman Mansimov, Alex Wang, and Kyunghyun Cho.
2019. A generalized framework of sequence generation with application to undirected sequence models.
ArXiv, abs/1905.12790.

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V.
Le, Mohammad Norouzi, Wolfgang Macherey,
Maxim Krikun, Yuan Cao, Qin Gao, Jeff Klingner,
Apurva Shah, Melvin Johnson, Xiaobing Liu,
Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato,
Taku Kudo, Hideto Kazawa, Keith Stevens, George
Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason
Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals,
Gregory S. Corrado, Macduff Hughes, and Jeffrey
Dean. 2016. Google’s neural machine translation
system: Bridging the gap between human and machine translation. CoRR, abs/1609.08144.
Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. 2018. Response generationby context-aware prototype editing. In AAAI.
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin
Knight, Dongyan Zhao, and Rui Yan. 2019. Planand-write: Towards better automatic storytelling. In
AAAI.
Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang,
Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation basedon phrase-level sentiment analysis. In SIGIR.