You May Like This Hotel Because ...: Identifying Evidence for Explainable Recommendations (2020)

Abstract: Explainable recommendation is a good way to improve user satisfaction. However, explainable recommendation in dialogue is challenging since it has to handle natural language as both input and output. To tackle the challenge, this paper proposes a novel and practical task to explain evidences in recommending hotels given vague requests expressed freely in natural language. We decompose the process into two subtasks on hotel reviews: Evidence Identification and Evidence Explanation. The former predicts whether or not a sentence contains evidence that expresses why a given request is satisfied. The latter generates a recommendation sentence given a request and an evidence sentence. In order to address these subtasks, we build an Evidence-based Explanation dataset, which is the largest dataset for explaining evidences in recommending hotels for vague requests. The experimental results demonstrate that the BERT model can find evidence sentences with respect to various vague requests and that the LSTM-based model can generate recommendation sentences.

## introduction recently, dialog systems using natural language processing technology have been adopted in interactive services such as call centers @xcite . one challenging issue in a real-world scenario is vague requests foot_0 from users. for example, in a hotel booking service, users often ask operators for "a child-friendly hotel" or "a convenient inn." to respond to such vague requests, human operators need to explain the reason why the given request is satisfied. an example response would be, "this hotel has a large kids' space, so i recommend it for families with children like you." responding to vague requests with evidences is effective because it not only strengthens the recommendation, but also urges users to make more concrete requests such as "i don't need a kids' space but want a baby stroller rental service." several studies have addressed explainable recommendations that produce natural language sentences @xcite @xcite . one major approach is feature-based explanations. @xcite generated explanation sentences using templates with slots, for example, "you might be interested in @xcite , on which this product performs well." however, by handling only predefined and limited features, this study cannot explain detailed evidences for each hotel such as "a view of mount fuji and lake kawaguchi." furthermore, this study does not accept natural language requests as inputs, which is a major bottleneck for building dialog-based interactive systems. in this study, we propose a novel and practical task to identify and explain evidences that satisfy a given vague request expressed freely in natural language. specifically, assuming a practical situation of recommendation, we address a hotel booking service. when choosing a hotel on an interactive service, users make a wide range of vague requests, which differ from predefined aspects @xcite , emotional expressions @xcite and questions @xcite . in order to satisfy vague requests by recommending hotels with evidences, the system must understand a given request, associate the request to a hotel with specific evidence, and generate an explanation (recommendation sentence) for the evidence. to address these challenges, we decompose the process into two subtasks: evidence identification and evidence explanation. the former predicts whether a sentence contains evidence that expresses why a given request is satisfied. the latter generates a recommendation sentence given the evidence sentence. in order to focus on evidence explanations for requests, we assume that recommending hotels are given in advance in this study. for these subtasks, we present an evidencebased explanation dataset, which is the largest dataset for explaining evidences in recommending hotels for vague requests. assuming that titles of hotel reviews often correspond to vague requests, the dataset includes 37,280 hotel reviews with annotations for vague requests, evidence sentences for the requests, recommendation sentences based on the evidence sentences. the key feature of the dataset is the variety of requests: it includes 15,767 unique types of requests written in natural language. this dataset is publicly available 2 . we report experiments for the two subtasks in section 3. we build a bert @xcite model for the first subtask, which predicts whether a sentence contains evidence for a request. experimental results show that the model can detect evidence sentence for various requests with a high (79.94) f1-score, and that the score does not drop so much even for requests unseen in the training data. we present encoderdecoder models for the second subtask, which rewrite an evidence sentence into a recommendation sentence. the experiments demonstrate that an lstm @xcite based model achieves the bleu score @xcite of 56.09 with a gold evidence sentence given and that of 45.38 without a gold sentence (only a re-2 https://github.com/megagonlabs/ebe-dataset view and a request is given). we also report experiments when the two subtasks are combined to generate a recommendation sentence for a given review. the contributions of this paper are as follows: ## dataset creation in this section, we describe the procedure to create the evidence-based explanation dataset. the dataset is expected to include (i) vague requests from users, (ii) items (in this study, hotel candidates), (iii) evidence where an item satisfies an request, (iv) and a recommendation sentence based on each evidence. as a corpus that meets these requirements, we use review data on jalan foot_1 , which is a major hotel booking service in japan. on jalan, users can enter reviews after their stay at the hotel. in addition to review texts, jalan accepts ratings for some specific aspects (e.g., 'service' and 'cleanliness'), similarly to other booking services @xcite . although some aspects are similar to vague requests(e.g., "good service" or "cheap hotel"), the number of such predefined aspects is very limited and cannot cover diverse requests, such as "dog-friendly hotel." consequently, we created a new dataset using review titles and review texts. in the review texts, users describe their impressions on the service of the hotel based on their real experiences. additionally, the review titles often summarize the most salient point of the experiences and often include similar expressions to vague requests such as "dog-friendly hotel." hence, assuming that some review titles express vague requests and that the corresponding review texts contain evidence, we extracted vague requests from review titles and annotated evidence sentences for requests in review texts. finally, we rewrote the evidence sentences into recommendation sentences. figure 1 illustrates the overall pipeline to construct the dataset. it consists of three steps. ## experiments using the annotated dataset, we conducted two experiments. (1) evidence identification and (2) evidence explanation. the former predicts whether a sentence contains evidence for a request, whereas the latter generates a recommendation sentence. ## related work several studies addressed explainable recommendations @xcite @xcite @xcite @xcite . in feature-based explanations, @xcite generated textual sentences as explanations using templates such as "you might be interested in [feature], on which this product performs well." in aspect-based explanations, @xcite discovered latent ratings on each aspect, and selected sentences related to each aspect to help users better understand the opinions given a set of review texts with the overall ratings. @xcite formulated a problem called personalized reason generation and generated a recommendation sentence given a song name, author, and user tag as input. the inputs of those studies were user vectors created from the user's action history or limited aspects. however, our study deals with a wide range of natural language requests for a dialog system in the hotel booking domain. in the field of sentiment analysis, research that extracts evidence based on sentiment expressions has attracted attention @xcite @xcite . @xcite extracted the cause of a target emotional expression based on a rule. @xcite annotated an emotional expression and its cause. these studies aimed to gather useful information to extract emotional expressions and provide evidence simultaneously by examining the reputations for specific products. although our study also aims to collect useful information, the requests are not limited to emotional expressions. in addition, we generate recommendation sentences. our study can be viewed as a special application of argument mining in the domain of hotel review. @xcite used manually annotated arguments of evidence-conclusion discourse relations in 110 hotel reviews. the study showed the effectiveness of several combinations of argumentbased features. in japanese, @xcite proposed a method to collect consents and dissents for queries that can be answered with yes or no. as part of that, they extracted evidence using rules. our dataset is useful as training data to extract evidence in argument mining. ## conclusion we proposed a novel task of predicting an evidence to satisfy a request and generating a recommendation sentence. we built an evidence-based explanation dataset for the task. the experimental results demonstrated that the bert model could find evidence sentences with respect to various vague requests and that the lstm-based model could generate recommendation sentences. future directions of this study include choosing the best evidence sentence from multiple candidate sentences for a vague request from a user and developing a concierge service that can recommend a hotel with evidence.