Clues Before Answers: Generation-Enhanced Multiple-Choice QA (2022)

Abstract: A trending paradigm for multiple-choice question answering (MCQA) is using a text-to-text framework. By unifying data in different tasks into a single text-to-text format, it trains a generative encoder-decoder model which is both powerful and universal. However, a side effect of twisting a generation target to fit the classification nature of MCQA is the under-utilization of the decoder and the knowledge that can be decoded. To exploit the generation capability and underlying knowledge of a pre-trained encoder-decoder model, in this paper, we propose a generation-enhanced MCQA model named GenMC. It generates a clue from the question and then leverages the clue to enhance a reader for MCQA. It outperforms text-to-text models on multiple MCQA datasets.

## introduction multiple-choice question answering (mcqa) aims at selecting the correct answer from a set of options given a question. this long-standing challenge in natural language processing (nlp) requires machines to have a wealth of knowledge, such as commonsense knowledge @xcite and scientific knowledge @xcite @xcite , and have reasoning skills such as multi-hop reasoning @xcite and logical reasoning @xcite @xcite . mcqa has made great progress with the development of pre-trained language models (plms). basically there are two types of plms that are suitable for different tasks. bert @xcite and its variants such as roberta @xcite and albert @xcite are encoder-only models, being more suitable for natural language understanding (nlu) tasks including mcqa and other classification and regression tasks. t5 @xcite and bart @xcite are encoder-decoder models, being more suitable for natural language generation (nlg) tasks. however, encoder-decoder models can also be applied to mcqa @xcite . this is enabled by the text-to-text framework, which transforms data in different tasks into a unified text-to-text format so that knowledge spanning many and various tasks can be learned, aggregated, and used by a single model. research question to fit mcqa, existing implementations of the text-to-text framework take all the options as input and are trained to generate one of the options, i.e., to copy some tokens from the input. however, this is inconsistent with how encoder-decoder models are pre-trained so that their underlying knowledge may not be sufficiently exploited. indeed, @xcite have found that in classification and regression tasks, the decoder layer is often under-utilized. one research question is how to apply pre-trained encoder-decoder models in a more natural way to mcqa, in particular, to exploit their nlg capabilities. our contribution our idea is inspired by human behavior. when reading a question, humans are sometimes triggered to associate the question with their background knowledge to form some clues even before reading the options. for simple questions, a clue may be exactly the correct answer, while for complex questions, clues may play an auxiliary role to help humans connect the question with the correct answer. for example, for the question shown in figure 1 , the clue "paper" forms an intermediate concept between "notebook" in the question and "tree" in the correct answer. with this idea, we propose to employ a pretrained encoder-decoder model to generate a clue from the question by exploiting its underlying knowledge, without seeing and being strictly confined to the options as in the text-to-text framework. the clue representation is then leveraged by an encoder-based model to read the options and make prediction. we refer to this generation-enhanced mcqa model as genmc. it significantly outperforms comparable models, in particular, text-to-text models, on five mcqa datasets. outline we discuss related work in section 2, introduce genmc in section 3, describe the experimental setup in section 4, report the results in section 5, and conclude in section 6. code our code is available on github 1 under the apache licence 2.0. ## genmc model in mcqa, a question q is given together with a set of n options @xmath0, . . . , o n } with exactly one option being the correct answer. the key to finding the correct answer is to capture and deeply understand the connection between q and each o @xmath1, which oftentimes is beyond the lexical level and requires a non-trivial entailment process. we follow the trend of building on a pretrained encoder-decoder model and use the encoder to jointly encode q and each o i . however, previous works directly use the decoder to generate an option in o, i.e., using the decoder as a classifier, which may have under-exploited the model's nlg capability @xcite . moreover, a simple joint encoding of q and each o i can only enable lexical-level reasoning @xcite which is insufficient for mcqa tasks. our proposed model genmc overcomes these limitations. building on a pre-trained encoderdecoder model, genmc firstly generates a clue which is indicative of the correct answer, thereby exploiting the nlg capability and underlying knowledge of the pre-trained encoder-decoder model. then genmc employs the generated clue representation as intermediate knowledge connecting the question and the correct answer to interact with and enhance a reader for solving mcqa. our model design mimics how humans solve an mcqa task, i.e., after reading a question, humans may firstly associate it with some of their background knowledge (i.e., looking for clues) that helps them to later identify the correct answer. ## conclusion we present genmc, a simple yet effective model which tailors pre-trained encoder-decoders for mcqa tasks. compared with existing usages of pre-trained encoder-decoders for mcqa, our model fully exploits the pre-trained encoderdecoders' nlg capabilities to generate a clue from the input question, which facilitates deep understanding of question-option pairs. experimental results further verify the superiority of genmc over existing usages. notably, our model achieves promising results without using any provided documents or external corpora, showing an interesting application of plms by directly inducing either commonsense or scientific knowledge from them through clue generation. in the future, we will focus on how to further improve the clue generation quality, which remains a bottleneck of genmc. we hope this work will spur more research in how to better use pre-trained encoder-decoders for not only mcqa, but also beyond; for tasks with divergent structures from the pre-training, a smarter use of plms can boost the performance significantly.