A r T : All-round Thinker for Unsupervised Commonsense Question Answering (2022)

Abstract: Without labeled question-answer pairs for necessary training, unsupervised commonsense question-answering (QA) appears to be extremely challenging due to its indispensable unique prerequisite on commonsense source like knowledge bases (KBs), which are usually highly resource consuming in construction. Recently pre-trained language models (PLMs) show effectiveness as an alternative for commonsense clues when they play a role of knowledge generator. However, existing work either relies on large-scale in-domain or out-of-domain labeled data, or fails to generate knowledge of high quality in a general way. Motivated by human thinking experience, we propose an approach of All-round Thinker (ArT) by fully taking association during knowledge generating. In detail, our model first focuses on key parts in the given context, and then generates highly related knowledge on such a basis in an association way like human thinking. Besides, for casual reasoning, a reverse thinking mechanism is especially added to further enhance bidirectional inferring between cause and effect. ArT is totally unsupervised and KBs-free. We evaluate it on three commonsense QA benchmarks: COPA, SocialIQA and SCT. On all scales of PLM backbones, ArT shows its brilliant performance and outperforms previous advanced unsupervised models.

## introduction commonsense question-answering (qa) has been a more challenging natural language understanding (nlu) task than conventional qa tasks, for it requires extra commonsense knowledge, which cannot be directly acquired from the given context, to make an appropriate answer @xcite . figure 1 gives a typical example. to select the correct cause of the fact "my body cast a shadow over the grass", models should know that "light is obscured to produce shadows" and "the sun emits light". as human beings, we can immediately make this association and judgment because above knowledge is deeply branded in our mind due to daily seeing and hearing, and maybe longterm education, which is so-called commonsense. however, it could be much difficult for models to be equipped with such ability. besides, commonsense qa usually takes an unsupervised setting which makes this task even more difficult. this setting means there is no labeled training data available as commonsense is too broad to build a sufficient labeled training dataset @xcite . to deal with it, prior work focused on building large-scale knowledge bases (kbs), also known as knowledge graphs (kgs), which usually contain millions of nodes and edges to record the relation between entities as relation triple: <e 1 , r, e 2 > @xcite . qa models then can be injected with commonsense through retrieving over @xcite . though impressive improvements are gained, such method is resource consuming in building or finding a good and suitable kb (e.g. building concept-net needs 30 gb of ram foot_0 ). recently, pre-trained language models (plms) have been widely used and proved to be effective in commonsense qa @xcite . thanks to the self-supervised pre-training strategy on largescale unlabeled text (e.g. webtext @xcite and wikipedia foot_1 ), plms are competent for many tasks even under a zero-shot setting. and fine-tuning plms on task-specific data in a supervised way can further produce even stronger results @xcite @xcite . since highquality labeled datasets are rare, researching on unsupervised commonsense qa is still of great significance. with the help of plms, existing studies have explored some good solutions @xcite @xcite , however they either rely on large-scale in-domain or out-of-domain labeled data, or need to be specifically designed for different tasks. in this work, we focus on designing a simple and general method to solve commonsense qa tasks in a strictly unsupervised way. based on two empirical observations from human thinking, (1) given a question with context, people firstly tend to focus on several key parts (as marked in red in figure 1 ) and then make corresponding associations to choose the right answer; our proposed model is strictly unsupervised and kbs-free for all it needs is plms. we test art and validate its effectiveness on three commonsense qa benchmark datasets: copa @xcite , socialiqa @xcite and sct @xcite . our contribution is summarized as follows: ## all-round thinker inspired by the behaviour and thought of human beings during solving questions requiring commonsense, we propose all-round thinker (art). figure 2 gives its overview. we will firstly introduce the task definition and the basic solution, and then describe the two principal methods of art in detail: notes taking (nt) and reverse thinking (rt). ## rt for other questions although rt is designed to enhance causal reasoning, we also explored if rt has the potential to help other questions. to investigate this, we apply it on other two datasets: socialiqa and sct. note that in these tasks the opposite question is hard to define, therefore we simply exchange the position of option and context, that is st i @xmath0, q, o @xmath1, q, c >. we employ three scoring functions on basis of gpt-2 medium baseline, as shown in ## conclusion commonsense qa has been a challenging task for it requires extra knowledge beyond the given con-text. in consideration of the high resource consumption of building knowledge bases (kbs) and the rarity of high-quality labeled data, this work aims at addressing commonsense qa in a fully kbs-free and unsupervised way. inspired by the association process of human thinking, we propose all-round thinker (art), which first focuses on key parts in the given context, and then generates highly related knowledge on such a basis in an association way. besides, a reverse thinking mechanism is introduced to further enhance bidirectional inferring for causal reasoning as human will do. we test art on three benchmarks: copa, socialiqa and sct. art outperforms previous advanced unsupervised models and shows stable performance on all scales of plm backbones.