Generative Question Answering: Learning to Answer the Whole Question

Discriminative question answering models can overfit to superficial biases in datasets , because their loss function saturates when any clue makes the answer likely . We introduce generative models of the joint distribution of questions and answers , which are trained to explain the whole question , not just to answer it . Our question answering ( QA ) model is implemented by learning a prior over answers , and a conditional language model to generate the question given the answer— allowing scalable and interpretable many-hop reasoning as the question is generated word-by-word . Our model achieves competitive performance with comparable discriminative models on the SQUAD and CLEVR benchmarks , indicating that it is a more general architecture for language understanding and reasoning than previous work . The model greatly improves generalisation both from biased training data and to adversarial testing data , achieving state-of-the-art results on ADVERSARIALSQUAD . 1 INTRODUCTION . Question answering tasks are widely used for training and testing machine comprehension and reasoning ( Rajpurkar et al. , 2016 ; Joshi et al. , 2017 ) . However , high performance has been achieved with only superficial understanding , as models exploit simple correlations in the data ( Weissenborn et al. , 2017 ; Zhou et al. , 2015 ) . For example , in Visual QA ( Agrawal et al. , 2017 ) , the answer to What colour is the grass ? can be memorised as green without considering the image ( Figure 1 ) . We argue that this over-fitting to biases is partly caused by discriminative loss functions , which saturate when simple correlations allow the question to be answered confidently , leaving no incentive for further learning on the example . We propose generative QA models , using Bayes ’ rule to reparameterise the distribution of answers given questions in terms of the distribution of questions given answers . We learn a prior over answers and a conditional language model for generating the question—reducing question answering to sequence-to-sequence learning ( Sutskever et al. , 2014 ) , and allowing many-hop reasoning as the model explains the whole question word-by-word . Generative loss functions train the model to explain all question words , even if the answer is obvious . For example , a model can not assign high probability to generating the question What colour is the grass ? without learning a dependency between the image and the word grass . We show that this method allows much improved generalisation from biased training data and to adversarial test data , compared to state-of-the-art discriminative models . Word-by-word generative modelling of questions also supports chains of reasoning , as each subpart of the question is explained in turn . Existing methods use a pre-specified number of reasoning steps ( Sukhbaatar et al. , 2015 ; Hudson & Manning , 2018 ) , which may be too many steps on easy cases , and too few on long and complex questions . We instead perform an interpretable reasoning step for each question word , and achieve 97.7 % accuracy on the CLEVR benchmark ( Johnson et al. , 2017 ) . Our approach opens a promising new direction for question answering , with strong results in language understanding , reasoning and generalisation . Is the purple thing the same shape as the large gray rubber thing ? Does the green rubber object have the same shape as the gray thing that is on the right side of the big purple object ? Whilst filming in Mexico City , speculation in the media claimed that the script had been altered to accommodate the demands of Mexican authorities reportedly influencing details of the scene and characters , casting choices , and modifying the script in order to portray the country in a “ positive light ” in order to secure tax concessions and financial support worth up to $ 20 million for the film . This was denied by producer Michael G. Wilson . Which Bond producer would not confirm that the film had been changed to accommodate Mexican authorities ? 2 MODEL . 2.1 OVERVIEW . We assume a dataset of examples with a question q = q0 .. T , answer a , and context c ( in our experiments , c is a document or image , but alternatives such as knowledge graphs could be used ) . We train models to minimize the negative log likelihood of the joint distribution of questions and answers given the context , − log p ( q , a|c ) , which we decompose using the chain rule as : L = − log p ( a|c ) − ∑ t log p ( qt|a , c , q0 .. t−1 ) ( 1 ) First c is encoded , using a recurrent model for text ( §2.2 ) and a using a convolutional model for images ( §2.3 ) . Then , a prior over answers p ( a|c ) is evaluated by scoring all possible answers ( 2.4 ) . The likelihood of the question p ( q|a , c ) is modelled using a conditional language model ( §2.5 ) . At test time , the answer maximizing p ( q , a|c ) is returned1 ( §2.7 ) . Hyperparameters and training details are fully described in Appendix A . 2.2 DOCUMENT ENCODER . Answer-independent Context Representation We use contextualised word representations , similar to ELMo ( Peters et al. , 2018 ) . We use character-based word embeddings ( Kim et al. , 2016 ) and train a 2-layer LSTM language model in the forward and reverse directions , using the WikiText-103 corpus ( Merity et al. , 2016 ) for domain-specificity2 . These parameters are frozen , and not finetuned . Contextualised word representations are computed as a fully connected layer applied to the concatenation of the word embedding and the hidden states of each layer of the language model . We sum this representation with a trainable vector of size d if the word occurs in the article title , giving the encoder information about the article topic . We follow this with bidirectional LSTMs of size d/2 , concatenating the output in each direction and adding residual connections after each layer . 1argmaxa p ( q , a|c ) = argmaxa p ( a|q , c ) , so this is the most likely answer given the question . 2The original ELMo implementation was trained on GBW ( Chelba et al. , 2014 ) and fine-tuned on SQUAD . Answer Encoder We assume answers a are a span i .. j of the document . We represent the answer as a weighted sum of words within the span . For each word representation ∑ k=i .. j σ ( w · c̃k ) c̃k , where w ∈ Rd is a trainable vector , and c̃k is the kth word in the answer-independent document representation . In contrast to previously proposed span representations ( Lee et al. , 2016 ; He et al. , 2018 ) , this approach allows the model to select arbitrarily many head words from the span . Answer-dependent Context Representation Generative training makes it feasible to model more complex interactions between the answer and context than discriminative training , because only the correct answer is used . On SQUAD , we compute an answer-dependent document representation . Here , we take the output of the answer-independent representation of each context word , and concatenate it with 32-dimensional embeddings of : a binary feature for whether the word is contained in the answer , its position relative to the answer start , and its position relative to the answer end . We also concatenate the element-wise product of the word representation and the answer encoding . We feed the result into 3 further layers of residual bidirectional LSTMs of size d/2 . 2.3 IMAGE ENCODER . We use a simple image encoder , leaving reasoning to the question decoder . Following Johnson et al . ( 2017 ) , we take pre-trained features from the conv4 layer ResNet-101 model ( He et al. , 2016 ) , giving 1024-dimensional features for a 14x14 image grid . We apply dropout , and project these representations to size d using a 1x1 convolution , followed by batch normalisation ( Ioffe & Szegedy , 2015 ) and a ReLU activation ( Nair & Hinton , 2010 ) . We then use 2 blocks of 3x3 convolutions , batch normalisation and ReLUs , and concatenate the final representation with a 32-dimensional positional encoding . 2.4 ANSWER PRIOR . We found modelling p ( a|c ) , the distribution over answers given the context , to be straightforward . On SQUAD , we concatenate the start and end representations of the answer-independent context representation , combine them with a single hidden layer of size 2d and ReLU activation , and project this representation down to a score sendpoints ( a , c ) . We also add an additional score based only on the length of the answer slength ( a ) . Finally , we calculate : p ( a|c ) = exp ( s endpoints ( a , c ) +slength ( a ) ) ∑ a′ exp ( s endpoints ( a′ , c ) +slength ( a′ ) ) On CLEVR , we simply apply a fully connected layer of size 2d and ReLU activation to the image representation , followed by a projection to the space of all possible answers and a softmax . 2.5 QUESTION DECODER . We generate question words left-to-right with teacher forcing . We first embed words independently of the context ( §2.5.1 ) , then use a multi-layer RNN with attention to model interactions between the question and context ( §2.5.2 ) , and finally compute the likelihood of the next word ( §2.5.3 ) . 2.5.1 INPUT WORD EMBEDDINGS . To represent SQUAD question words independently of the answer and document , we use a pretrained left-to-right language model ( which can be viewed as a uni-directional version of ELMo ) , followed by a trainable LSTM layer of size d. On CLEVR , we simply train embeddings of size d . 2.5.2 DECODER BLOCKS . Decoder blocks are composed of a question self-attention layer ( Vaswani et al. , 2017 ) and a questionto-context attention mechanism , which are combined and fed into an LSTM ( Figure 3 ) . The context attention query is computed using both the previous layer state and the self-attention value . Blocks after the first are connected with residual connections . Our attention mechanisms are implemented as in Vaswani et al. , except that we use a single-headed attention , and a bias term bj is added to the query-key score for context position j . This bias term ( calculated as a dot product of a shared trainable vector and context encoding cj ) allows the model to easily filter out parts of the context which are irrelevant to the question . A final block again uses self-attention and question-to-document attention , which are then combined with a Gated Linear Unit layer ( Dauphin et al. , 2017 ) to output a vector of size d. The GLU layer uses a gating mechanism to select the relevant information for predicting the subsequent word . 2.5.3 OUTPUT WORD PROBABILITIES . Rare words are more challenging for generative than discriminative models , because it is easier to guess the meaning of a rare word from morphological clues than to generate it . SQUAD contains many rare words , because of the specialised vocabulary in Wikipedia . We improve modelling of rare words using a combination of an approximate character-based softmax and copy mechanism . Word Softmax The simplest approach is to learn an embedding per word , replacing infrequent words with an unknown token . This approach can not discriminate between different rare words . We use this method on CLEVR , which has a small vocabulary . Character Softmax While there is an infinite space of possible output words , we approximate it by only normalising over the union of a list of frequent words , and any additional word that appears in any question or any document in the batch . We build a character-based representation for each candidate word using the pre-trained character CNN from §2.2 , and add a trainable linear projection to size d. We combine character-based and word-based representations by summation . Pointer Mechanism We use a pointer mechanism ( Vinyals et al. , 2015 ) to improve the likelihood of the specialised vocabulary present in SQUAD , by copying article words . From the final hidden state ht , the model first chooses whether to copy using a simple classifier pcopy ( h ) = σ ( wcopy · h ) , where wcopy is a trainable vector of size d. Then , the model interpolates between generating a word with softmax pgen ( h ) and copying context word ci using the question-to-context attention probability from the final layer αit : p ( qt|q0 : t−1 , c , a ) = pcopy ( ht ) ∑ i α i t1ci=qt + ( 1− pcopy ( ht ) ) pgen ( qt|ht )

Summary: In this paper, authors proposed a generative QA model, which optimizes jointly the distribution of questions and answering given a document/context. More specifically, it is decomposed into two components: the distributions of answers given a document, which is modeled by a single layer neural network; and the distribution of questions given an answer and document, which is modeled by a seq2seq model with a copy mechanism. During inference, it firstly extracts the most likely answer candidates, then evaluates the questions conditioned on the answer candidates and document and finally returns the answer with the max joint score from two aforementioned components.