Uncertainty Guided Global Memory Improves Multi-Hop Question Answering (2023)

Abstract: Transformers have become the gold standard for many natural language processing tasks and, in particular, for multi-hop question answering (MHQA). This task includes processing a long document and reasoning over the multiple parts of it. The landscape of MHQA approaches can be classified into two primary categories. The first group focuses on extracting supporting evidence, thereby constraining the QA model’s context to predicted facts. Conversely, the second group relies on the attention mechanism of the long input encoding model to facilitate multi-hop reasoning. However, attention-based token representations lack explicit global contextual information to connect reasoning steps. To address these issues, we propose GEMFormer, a two-stage method that first collects relevant information over the entire document to the memory and then combines it with local context to solve the task. Our experimental results show that fine-tuning a pre-trained model with memory-augmented input, including the most certain global elements, improves the model’s performance on three MHQA datasets compared to the baseline. We also found that the global explicit memory contains information from supporting facts required for the correct answer.

## introduction transformer @xcite and its variants @xcite have become one of the most popular solutions for various nlp tasks. particularly, transformers are applied to solve the multi-hop question answering (mhqa) tasks @xcite @xcite that require reasoning over multiple parts of the long document to answer the question. the problem of reasoning in mhqa has attracted a lot of research recently @xcite . one group of methods focuses on the usage of subnetworks or dedicated modules to extract evidence from a long document and then solve the questionanswering (qa) task based on the detected evidence facts @xcite @xcite @xcite . the resulting performance of such models highly depends on the evidence extraction method quality, limiting the qa model context to a small number of pre-selected facts. another group of methods addresses the general long document encoding problem by sparsifying the attention patterns to enlarge the maximal input sequence length @xcite @xcite . despite the merits of these models, their attention-based token representations combine local and global information in the same vector. the high-level contextual features are spread over a long sequence which makes it harder to access them. to address the described problems, we propose gemformer (global explicit memory transformer). gemformer is a method for augmenting the pre-trained language model with memory to store global information relevant to the task. it processes long input concatenated with a memory sequence consisting of tokens from input that are important to solve the task. token importance is defined by the language model uncertainty. ## related work augmentation of the neural network model with memory provides additional space to store relevant information that can be used to improve model performance and reduce computational costs. early examples of memory-augmented neural network architectures such as rnn and lstm @xcite used hidden states as internal memory. @xcite and @xcite introduced the external type of memory, where a separate network manipulated memory. with the growing popularity of the attention mechanism, attention was adopted for model-memory interaction @xcite ., 2 nd stage: target task training global memory 1 st stage: memory population ... lm head roberta question segment 1 lm head roberta segment 1 roberta question gm ... task heads y 1 y 2 y 3 question segment n segment n roberta question gm task heads figure 1: global explicit memory transformer. the input segments are processed by roberta+lm head to generate prediction distributions for uncertainty estimation. global memory is populated based on the given entropy condition. then, the memory-augmented segments question+memory+context are used to obtain the mhqa predictions y 1 , y 2 , y 3 . ## global explicit memory we implemented gemformer with a robertabase @xcite model as a backbone. global explicit memory is a sequence of document tokens that are important for correct reasoning and answer prediction. the model's uncertainty is used to measure the importance of inputs. given an input sequence @xmath0, t 2 , . . . , t m ] of m tokens and a vocabulary of size n, we first obtain a vector @xmath1. then we calculate the entropy @xmath2. in our experiments, we tested two memory population conditions for entropy: the general motivation behind the entropy-based memory population is the following. the model is fed with a question and a context, and the entropy is computed for each contextual token. this entropy is conditional with respect to the question and token-surrounding context. such entropy value measures how much entropy a token has remaining if we have already learned the question and the context. in other words, how easily the token can be predicted given the question and the context. taking into account that a document is a collection of question-relevant and distractor paragraphs, the entropy of a task-relevant token should be lower than the entropy of the irrelevant one. the gemformer architecture is depicted in figure 1 . to fit roberta maximum sequence length limit, the contextual document is split into segments and each segment is concatenated to the question. input processing consists of two stages: 1) document comprehension and memory population, and 2) task predictions generation using memory-enhanced inputs. in the first stage, question-context segments are passed through the roberta model with lm head to obtain crossdictionary distributions for entropy estimation. context tokens with little independent semantic content, such as special separators, punctuation marks, and stop words, are not considered for the memory population (see details in appendix b). global memory is then collected from the remaining tokens from the document based on the chosen entropy conditions (eq. 1). in the second stage, question and global memory tokens are concatenated with each segment for the mhqa task training. the model's weights for the first stage are updated every epoch with the weights of the model trained to solve the target task in the second stage. we evaluated gemformer on three english mhqa datasets: hotpotqa distractor setting @xcite , 2wikimultihopqa @xcite and musique-ans @xcite . further in the paper, we will refer to them as hp, 2w, and msq respectively. data preprocessing, training, and inference details are described in appendix b. ## results and discussion as baselines in our experiments we used a roberta fine-tuned on the task without memory and a roberta fine-tuned with memory bank of 200 yake! @xcite keyword tokens from the contextual document (see we hypothesize that a model combining a tuned roberta encoder with a frozen lm head for uncertainty estimation tends to detect parts of a text that are essential for the answer by reducing their uncertainty (see also appendix b). to test this hypothesis, we used a variable-size memory consisting of tokens with entropy lower than a given threshold (low h in equation 1 ). the fine-tuned model with constant entropy threshold θ outperformed the highest performing baseline memoryfree model by 1.6 joint f1 points for hp, 1.59 and 0.58 points for 2w answer and supporting evidence prediction, and 1.25 f1 points for msq supporting paragraphs prediction. we also tested a dynamical entropy threshold as a value of the fifth percentile of the document token entropy assuming that this might help to better cover the full supporting evidence (low (@xmath3. such a memory selection rule showed the best answer f1 for msq and led to a slight improvement compared to the task-tuned roberta baseline but was weaker than fixed θ for hp and 2w. we also evaluated the chatgpt 2 (gpt-3.5turbo-0613 checkpoint) with memory generated by gemformer low (@xmath4. the results are presented in ablation study to verify the proposed memory augmentation, we conducted an ablation study on hp. results are shown in the random memory ablation was to fill memory with tokens selected randomly from the document. it showed that global memory with the arbitraryselected content can not only fail to improve predictions but can actually degrade the model performance. we also provide the comparison of our bestperforming gemformer low (@xmath7. ## conclusions in this study, we demonstrated how utilizing uncertainty-based global explicit memory can enhance the model performance on mhqa tasks. our findings indicate that utilizing low entropy context tokens can aid the model in mhqa reasoning, but only when the entropy estimation model is specifically fine-tuned to the target task. experiments show that higher-performing models use larger memory sizes with better coverage of supporting facts.