Iterative Multi-document Neural Attention for Multiple Answer Prediction

Abstract: People have information needs of varying complexity, which can be solved by an intelligent agent able to answer questions formulated in a proper way, eventually considering user context and preferences. In a scenario in which the user profile can be considered as a question, intelligent agents able to answer questions can be used to find the most relevant answers for a given user. In this work we propose a novel model based on Artificial Neural Networks to answer questions with multiple answers by exploiting multiple facts retrieved from a knowledge base. The model is evaluated on the factoid Question Answering and top-n recommendation tasks of the bAbI Movie Dialog dataset. After assessing the performance of the model on both tasks, we try to define the long-term goal of a conversational recommender system able to interact using natural language and to support users in their information seeking processes in a personalized way.

We are surrounded by a huge variety of technological artifacts which “live” with us today. These artifacts can help us in several ways because they have the power to accomplish complex and time-consuming tasks. Unfortunately, common software systems can do for us only specific types of tasks, in a strictly algorithmic way which is pre-defined by the software designer. Machine Learning (ML), a branch of Artificial Intelligence (AI), gives machines the ability to learn to complete tasks without being explicitly programmed.
People have information needs of varying complexity, ranging from simple questions about common facts which can be found in encyclopedias, to more sophisticated cases in which they need to know what movie to watch during a romantic evening. These tasks can be solved by an intelligent agent able to answer questions formulated in a proper way, eventually considering user context and preferences.
Question Answering (QA) emerged in the last decade as one of the most promising fields in AI, since it allows to design intelligent systems which are able to give correct answers to user questions expressed in natural language. Whereas, recommender systems produce individualized recommendations as output and have the effect of guiding the user in a personalized way to interesting or useful objects in a large space of possible options. In a scenario in which the user profile (the set of user preferences) can be represented by a question, intelligent agents able to answer questions can be used to find the most appealing items for a given user, which is the classical task that recommender systems can solve. Despite the efficacy of classical recommender systems, generally they are not able to handle a conversation with the user so they miss the possibility of understanding his contextual information, emotions and feedback to refine the user profile and provide enhanced suggestions. Conversational recommender systems assist online users in their information-seeking and decision making tasks by supporting an interactive process BIBREF0 which could be goal oriented with the task of starting general and, through a series of interaction cycles, narrowing down the user interests until the desired item is obtained BIBREF1 .
In this work we propose a novel model based on Artificial Neural Networks to answer questions exploiting multiple facts retrieved from a knowledge base and evaluate it on a QA task. Moreover, the effectiveness of the model is evaluated on the top-n recommendation task, where the aim of the system is to produce a list of suggestions ranked according to the user preferences. After having assessed the performance of the model on both tasks, we try to define the long-term goal of a conversational recommender system able to interact with the user using natural language and to support him in the information seeking process in a personalized way.
In order to fulfill our long-term goal of building a conversational recommender system we need to assess the performance of our model on specific tasks involved in this scenario. A recent work which goes in this direction is reported in BIBREF2 , which presents the bAbI Movie Dialog dataset, composed by different tasks such as factoid QA, top-n recommendation and two more complex tasks, one which mixes QA and recommendation and one which contains turns of dialogs taken from Reddit. Having more specific tasks like QA and recommendation, and a more complex one which mixes both tasks gives us the possibility to evaluate our model on different levels of granularity. Moreover, the subdivision in turns of the more complex task provides a proper benchmark of the model capability to handle an effective dialog with the user.
For the task related to QA, a lot of datasets have been released in order to assess the machine reading and comprehension capabilities and a lot of neural network-based models have been proposed. Our model takes inspiration from BIBREF3 , which is able to answer Cloze-style BIBREF4 questions repeating an attention mechanism over the query and the documents multiple times. Despite the effectiveness on the Cloze-style task, the original model does not consider multiple documents as a source of information to answer questions, which is fundamental in order to extract the answer from different relevant facts. The restricted assumption that the answer is contained in the given document does not allow the model to provide an answer which does not belong to the document. Moreover, this kind of task does not expect multiple answers for a given question, which is important for the complex information needs required for a conversational recommender system.
According to our vision, the main outcomes of our work can be considered as building blocks for a conversational recommender system and can be summarized as follows:
The paper is organized as follows: Section SECREF2 describes our model, while Section SECREF3 summarizes the evaluation of the model on the two above-mentioned tasks and the comparison with respect to state-of-the-art approaches. Section SECREF4 gives an overview of the literature of both QA and recommender systems, while final remarks and our long-term vision are reported in Section SECREF5 .
Given a query INLINEFORM0 , an operator INLINEFORM1 that produces the set of documents relevant for INLINEFORM2 , where INLINEFORM3 is the set of all queries and INLINEFORM4 is the set of all documents. Our model defines a workflow in which a sequence of inference steps are performed in order to extract relevant information from INLINEFORM5 to generate the answers for INLINEFORM6 .
Following BIBREF3 , our workflow consists of three steps: (1) the encoding phase, which generates meaningful representations for query and documents; (2) the inference phase, which extracts relevant semantic relationships between the query and the documents by using an iterative attention mechanism and finally (3) the prediction phase, which generates a score for each candidate answer.
The input of the encoding phase is given by a query INLINEFORM0 and a set of documents INLINEFORM1 . Both queries and documents are represented by a sequence of words INLINEFORM2 , drawn from a vocabulary INLINEFORM3 . Each word is represented by a continuous INLINEFORM4 -dimensional word embedding INLINEFORM5 stored in a word embedding matrix INLINEFORM6 .
The sequences of dense representations for INLINEFORM0 and INLINEFORM1 are encoded using a bidirectional recurrent neural network encoder with Gated Recurrent Units (GRU) as in BIBREF3 which represents each word INLINEFORM2 as the concatenation of a forward encoding INLINEFORM3 and a backward encoding INLINEFORM4 . From now on, we denote the contextual representation for the word INLINEFORM5 by INLINEFORM6 and the contextual representation for the word INLINEFORM7 in the document INLINEFORM8 by INLINEFORM9 . Differently from BIBREF3 , we build a unique representation for the whole set of documents INLINEFORM10 related to the query INLINEFORM11 by stacking each contextual representation INLINEFORM12 obtaining a matrix INLINEFORM13 , where INLINEFORM14 .
This phase uncovers a possible inference chain which models meaningful relationships between the query and the set of related documents. The inference chain is obtained by performing, for each inference step INLINEFORM0 , the attention mechanisms given by the query attentive read and the document attentive read keeping a state of the inference process given by an additional recurrent neural network with GRU units. In this way, the network is able to progressively refine the attention weights focusing on the most relevant tokens of the query and the documents which are exploited by the prediction neural network to select the correct answers among the candidate ones.
Given the contextual representations for the query words INLINEFORM0 and the inference GRU state INLINEFORM1 , we obtain a refined query representation INLINEFORM2 (query glimpse) by performing an attention mechanism over the query at inference step INLINEFORM3 : INLINEFORM4 
where INLINEFORM0 are the attention weights associated to the query words, INLINEFORM1 and INLINEFORM2 are respectively a weight matrix and a bias vector which are used to perform the bilinear product with the query token representations INLINEFORM3 . The attention weights can be interpreted as the relevance scores for each word of the query dependent on the inference state INLINEFORM4 at the current inference step INLINEFORM5 .
Given the query glimpse INLINEFORM0 and the inference GRU state INLINEFORM1 , we perform an attention mechanism over the contextual representations for the words of the stacked documents INLINEFORM2 : INLINEFORM3 
where INLINEFORM0 is the INLINEFORM1 -th row of INLINEFORM2 , INLINEFORM3 are the attention weights associated to the document words, INLINEFORM4 and INLINEFORM5 are respectively a weight matrix and a bias vector which are used to perform the bilinear product with the document token representations INLINEFORM6 . The attention weights can be interpreted as the relevance scores for each word of the documents conditioned on both the query glimpse and the inference state INLINEFORM7 at the current inference step INLINEFORM8 . By combining the set of relevant documents in INLINEFORM9 , we obtain the probability distribution ( INLINEFORM10 ) over all the relevant document tokens using the above-mentioned attention mechanism.
The inference GRU state at the inference step INLINEFORM0 is updated according to INLINEFORM1 , where INLINEFORM2 and INLINEFORM3 are the results of a gating mechanism obtained by evaluating INLINEFORM4 for the query and the documents, respectively. The gating function INLINEFORM5 is defined as a 2-layer feed-forward neural network with a Rectified Linear Unit (ReLU) BIBREF5 activation function in the hidden layer and a sigmoid activation function in the output layer. The purpose of the gating mechanism is to retain useful information for the inference process about query and documents and forget useless one.
The prediction phase, which is completely different from the pointer-sum loss reported in BIBREF3 , is able to generate, given the query INLINEFORM0 , a relevance score for each candidate answer INLINEFORM1 by using the document attention weights INLINEFORM2 computed in the last inference step INLINEFORM3 . The relevance score of each word INLINEFORM4 is obtained by summing the attention weights of INLINEFORM5 in each document related to INLINEFORM6 . Formally the relevance score for a given word INLINEFORM7 is defined as: INLINEFORM8 
where INLINEFORM0 returns 0 if INLINEFORM1 , INLINEFORM2 otherwise; INLINEFORM3 returns the word in position INLINEFORM4 of the stacked documents matrix INLINEFORM5 and INLINEFORM6 returns the frequency of the word INLINEFORM7 in the documents INLINEFORM8 related to the query INLINEFORM9 . The relevance score takes into account the importance of token occurrences in the considered documents given by the computed attention weights. Moreover, the normalization term INLINEFORM10 is applied to the relevance score in order to mitigate the weight associated to highly frequent tokens.
The evaluated relevance scores are concatenated in a single vector representation INLINEFORM0 which is given in input to the answer prediction neural network defined as: INLINEFORM1 
where INLINEFORM0 is the hidden layer size, INLINEFORM1 and INLINEFORM2 are weight matrices, INLINEFORM3 , INLINEFORM4 are bias vectors, INLINEFORM5 is the sigmoid function and INLINEFORM6 is the ReLU activation function, which are applied pointwise to the given input vector.
The neural network weights are supposed to learn latent features which encode relationships between the most relevant words for the given query to predict the correct answers. The outer sigmoid activation function is used to treat the problem as a multi-label classification problem, so that each candidate answer is independent and not mutually exclusive. In this way the neural network generates a score which represents the probability that the candidate answer is correct. Moreover, differently from BIBREF3 , the candidate answer INLINEFORM0 can be any word, even those which not belong to the documents related to the query.
The model is trained by minimizing the binary cross-entropy loss function comparing the neural network output INLINEFORM0 with the target answers for the given query INLINEFORM1 represented as a binary vector, in which there is a 1 in the corresponding position of the correct answer, 0 otherwise.
The model performance is evaluated on the QA and Recs tasks of the bAbI Movie Dialog dataset using HITS@k evaluation metric, which is equal to the number of correct answers in the top- INLINEFORM0 results. In particular, the performance for the QA task is evaluated according to HITS@1, while the performance for the Recs task is evaluated according to HITS@100.
Differently from BIBREF2 , the relevant knowledge base facts, taken from the knowledge base in triple form distributed with the dataset, are retrieved by INLINEFORM0 implemented by exploiting the Elasticsearch engine and not according to an hash lookup operator which applies a strict filtering procedure based on word frequency. In our work, INLINEFORM1 returns at most the top 30 relevant facts for INLINEFORM2 . Each entity in questions and documents is recognized using the list of entities provided with the dataset and considered as a single word of the dictionary INLINEFORM3 .
Questions, answers and documents given in input to the model are preprocessed using the NLTK toolkit BIBREF6 performing only word tokenization. The question given in input to the INLINEFORM0 operator is preprocessed performing word tokenization and stopword removal.
The optimization method and tricks are adopted from BIBREF3 . The model is trained using ADAM BIBREF7 optimizer (learning rate= INLINEFORM0 ) with a batch size of 128 for at most 100 epochs considering the best model until the HITS@k on the validation set decreases for 5 consecutive times. Dropout BIBREF8 is applied on INLINEFORM1 and on INLINEFORM2 with a rate of INLINEFORM3 and on the prediction neural network hidden layer with a rate of INLINEFORM4 . L2 regularization is applied to the embedding matrix INLINEFORM5 with a coefficient equal to INLINEFORM6 . We clipped the gradients if their norm is greater than 5 to stabilize learning BIBREF9 . Embedding size INLINEFORM7 is fixed to 50. All GRU output sizes are fixed to 128. The number of inference steps INLINEFORM8 is set to 3. The size of the prediction neural network hidden layer INLINEFORM9 is fixed to 4096. Biases INLINEFORM10 and INLINEFORM11 are initialized to zero vectors. All weight matrices are initialized sampling from the normal distribution INLINEFORM12 . The ReLU activation function in the prediction neural network has been experimentally chosen comparing different activation functions such as sigmoid and tanh and taking the one which leads to the best performance. The model is implemented in TensorFlow BIBREF10 and executed on an NVIDIA TITAN X GPU.
Following the experimental design, the results in Table TABREF10 are promising because our model outperforms all other systems on both tasks except for the QA SYSTEM on the QA task. Despite the advantage of the QA SYSTEM, it is a carefully designed system to handle knowledge base data in the form of triples, but our model can leverage data in the form of documents, without making any assumption about the form of the input data and can be applied to different kind of tasks. Additionally, the model MEMN2N is a neural network whose weights are pre-trained on the same dataset without using the long-term memory and the models JOINT SUPERVISED EMBEDDINGS and JOINT MEMN2N are models trained across all the tasks of the dataset in order to boost performance. Despite that, our model outperforms the three above-mentioned ones without using any supplementary trick. Even though our model performance is higher than all the others on the Recs task, we believe that the obtained result may be improved and so we plan a further investigation. Moreover, the need for further investigation can be justified by the work reported in BIBREF11 which describes some issues regarding the Recs task.
Figure FIGREF11 shows the attention weights computed in the last inference step of the iterative attention mechanism used by the model to answer to a given question. Attention weights, represented as red boxes with variable color shades around the tokens, can be used to interpret the reasoning mechanism applied by the model because higher shades of red are associated to more relevant tokens on which the model focus its attention. It is worth to notice that the attention weights associated to each token are the result of the inference mechanism uncovered by the model which progressively tries to focus on the relevant aspects of the query and the documents which are exploited to generate the answers.
Given the question “what does Larenz Tate act in?” shown in the above-mentioned figure, the model is able to understand that “Larenz Tate” is the subject of the question and “act in” represents the intent of the question. Reading the related documents, the model associates higher attention weights to the most relevant tokens needed to answer the question, such as “The Postman”, “A Man Apart” and so on.
We think that it is necessary to consider models and techniques coming from research both in QA and recommender systems in order to pursue our desire to build an intelligent agent able to assist the user in decision-making tasks. We cannot fill the gap between the above-mentioned research areas if we do not consider the proposed models in a synergic way by virtue of the proposed analogy between the user profile (the set of user preferences) and the items to be recommended, as the question and the correct answers. The first work which goes in this direction is reported in BIBREF12 , which exploits movie descriptions to suggest appealing movies for a given user using an architecture tipically used for QA tasks. In fact, most of the research in the recommender systems field presents ad-hoc systems which exploit neighbourhood information like in Collaborative Filtering techniques BIBREF13 , item descriptions and metadata like in Content-based systems BIBREF14 . Recently presented neural network models BIBREF15 , BIBREF16 systems are able to learn latent representations in the network weights leveraging information coming from user preferences and item information.
In recent days, a lot of effort is devoted to create benchmarks for artificial agents to assess their ability to comprehend natural language and to reason over facts. One of the first attempt is the bAbI BIBREF17 dataset which is a synthetic dataset containing elementary tasks such as selecting an answer between one or more candidate facts, answering yes/no questions, counting operations over lists and sets and basic induction and deduction tasks. Another relevant benchmark is the one described in BIBREF18 , which provides CNN/Daily Mail datasets consisting of document-query-answer triples where an entity in the query is replaced by a placeholder and the system should identify the correct entity by reading and comprehending the given document. MCTest BIBREF19 requires machines to answer multiple-choice reading comprehension questions about fictional stories, directly tackling the high-level goal of open-domain machine comprehension. Finally, SQuAD BIBREF20 consists in a set of Wikipedia articles, where the answer to each question is a segment of text from the corresponding reading passage.
According to the experimental evaluations conducted on the above-mentioned datasets, high-level performance can be obtained exploiting complex attention mechanisms which are able to focus on relevant evidences in the processed content. One of the earlier approaches used to solve these tasks is given by the general Memory Network BIBREF21 , BIBREF22 framework which is one of the first neural network models able to access external memories to extract relevant information through an attention mechanism and to use them to provide the correct answer. A deep Recurrent Neural Network with Long Short-Term Memory units is presented in BIBREF18 , which solves CNN/Daily Mail datasets by designing two different attention mechanisms called Impatient Reader and Attentive Reader. Another way to incorporate attention in neural network models is proposed in BIBREF23 which defines a pointer-sum loss whose aim is to maximize the attention weights which lead to the correct answer.
In this work we propose a novel model based on Artificial Neural Networks to answer questions with multiple answers by exploiting multiple facts retrieved from a knowledge base. The proposed model can be considered a relevant building block of a conversational recommender system. Differently from BIBREF3 , our model can consider multiple documents as a source of information in order to generate multiple answers which may not belong to the documents. As presented in this work, common tasks such as QA and top-n recommendation can be solved effectively by our model.
In a common recommendation system scenario, when a user enters a search query, it is assumed that his preferences are known. This is a stringent requirement because users cannot have a clear idea of their preferences at that point. Conversational recommender systems support users to fulfill their information needs through an interactive process. In this way, the system can provide a personalized experience dynamically adapting the user model with the possibility to enhance the generated predictions. Moreover, the system capability can be further enhanced giving explanations to the user about the given suggestions.
To reach our goal, we should improve our model by designing a INLINEFORM0 operator able to return relevant facts recognizing the most relevant information in the query, by exploiting user preferences and contextual information to learn the user model and by providing a mechanism which leverages attention weights to give explanations. In order to effectively train our model, we plan to collect real dialog data containing contextual information associated to each user and feedback for each dialog which represents if the user is satisfied with the conversation. Given these enhancements, we should design a system able to hold effectively a dialog with the user recognizing his intent and providing him the most suitable contents.
With this work we try to show the effectiveness of our architecture for tasks which go from pure question answering to top-n recommendation through an experimental evaluation without any assumption on the task to be solved. To do that, we do not use any hand-crafted linguistic features but we let the system learn and leverage them in the inference process which leads to the answers through multiple reasoning steps. During these steps, the system understands relevant relationships between question and documents without relying on canonical matching, but repeating an attention mechanism able to unconver related aspects in distributed representations, conditioned on an encoding of the inference process given by another neural network. Equipping agents with a reasoning mechanism like the one described in this work and exploiting the ability of neural network models to learn from data, we may be able to create truly intelligent agents.
This work is supported by the IBM Faculty Award "Deep Learning to boost Cognitive Question Answering". The Titan X GPU used for this research was donated by the NVIDIA Corporation.

Figure and table captions:
Table 1: Comparison between our model and baselines from [6] on the QA and Recs tasks evaluated according to HITS@1 and HITS@100, respectively.
Figure 1: Attention weights q̃i and D̃qi computed by the neural network attention mechanisms at the last inference step T for each token. Higher shades correspond to higher relevance scores for the related tokens.

Questions about this paper:
How well does their model perform on the recommendation task?

Answers:
Proposed model achieves HITS@100 of 30.0 compared to best baseline model result of 29.2 on recommendation task.