Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge (2022)

Abstract: To perform well on a machine reading comprehension (MRC) task, machine readers usually require commonsense knowledge that is not explicitly mentioned in the given documents. This paper aims to extract a new kind of structured knowledge from scripts and use it to improve MRC. We focus on scripts as they contain rich verbal and nonverbal messages, and two relevant messages originally conveyed by different modalities during a short time period may serve as arguments of a piece of commonsense knowledge as they function together in daily communications. To save human efforts to name relations, we propose to represent relations implicitly by situating such an argument pair in a context and call it contextualized knowledge. To use the extracted knowledge to improve MRC, we compare several fine-tuning strategies to use the weakly-labeled MRC data constructed based on contextualized knowledge and further design a teacher-student paradigm with multiple teachers to facilitate the transfer of knowledge in weakly-labeled MRC data. Experimental results show that our paradigm outperforms other methods that use weakly-labeled data and improves a state-of-the-art baseline by 4.3% in accuracy on a Chinese multiple-choice MRC dataset C ^3 , wherein most of the questions require unstated prior knowledge. We also seek to transfer the knowledge to other tasks by simply adapting the resulting student reader, yielding a 2.9% improvement in F1 on a relation extraction dataset DialogRE, demonstrating the potential usefulness of the knowledge for non-MRC tasks that require document comprehension.

## introduction given a question and a document, machine reading comprehension (mrc) tasks usually aim to select the correct answer option(s) from all options @xcite or extract a span as the answer from the document @xcite . for many mrc tasks, which are mostly in multiplechoice formats, machine readers require extensive general world knowledge unstated in the given documents to perform well on tasks. recent studies have showed the usefulness of other humanannotated mrc datasets @xcite or data from relevant tasks such as natural language inference @xcite . there is also a trend towards taking advantage of existing crowdsourced general world knowledge graphs such as con-ceptnet @xcite or automatically constructed graphs @xcite to improve mrc tasks @xcite @xcite . however, most progress has been limited to english, and it is expensive, time-consuming, laborious, and error-prone to construct clean, large-scale datasets of mrc-related tasks or general world knowledge graphs, which are not always available even for high-resource languages such as chinese. this paper aims to extract a new kind of structured general world knowledge from external unstructured chinese corpora, which is seldom studied, and investigate the use of the knowledge-based augmented data to improve mrc. typically, each piece of general world knowledge is represented as a triple that contains two phrases (e.g., ("finding a lost item", "happiness") and the relation (e.g., causes) between phrases, which can be one of a small pre-defined set of relations @xcite . however, naming relations often requires substantial human efforts (e.g., guidelines, annotations, or relation-specific patterns), and it is also unclear whether we need to explicitly represent relations if the final goal is to improve downstream tasks that do not directly depend on the reliability of relations in triples from other sources. once we have decided not to name relations, one natural question is whether we could implicitly represent relations between two phrases. we suggest that adding context in which the phrases occur may be useful as such a context constrains the possible relations between phrases without intervening in the relations explicitly @xcite . hereafter, we call a triple that contains a phrase pair and its associated context as a piece of contextualized knowledge for convenience. besides verbal information that is written or spoken, it is well accepted that nonverbal information is also essential for face-to-face communication @xcite . we regard related verbal and nonverbal information as the phrase pair; we treat the context in which the verbal-nonverbal pair occurs as the context. such triples contain rich commonsense knowledge as verbal and nonverbal information function together in communications, and this kind of knowledge is assumed to be known by most people without being formally taught, just as commonsense knowledge. for example, as shown in to use contextualized knowledge to improve mrc, we randomly select nonverbal messages from the same script as distractors to convert each piece of knowledge into a weakly-labeled mrc instance (section 3). we explore different two-stage fine-tuning strategies to use the weakly-labeled mrc data: for example, we first train a model on the combination of the weakly-labeled data and the target mrc data that is human-annotated but smallscale, and then, we fine-tune the resulting model on the target data (section 4). however, we observe that increasing the amounts of weakly-labeled data does not lead to noticeable gains than using one subset of the data. inspired by teacher-student studies that facilitate the transfer of clean knowledge @xcite @xcite , we further design a teacher-student paradigm with multiple teachers, each trained with a subset of the hard-labeled, weakly-labeled data, and we feed soft-labeled, weakly-labeled/clean data to a student in a two-stage fashion, similar to the two-stage fine-tuning strategy mentioned earlier (section 5). we evaluate our method on c 3 , a representative multiple-choice general-domain mrc dataset for chinese wherein most questions require commonsense knowledge beyond the given contents @xcite . to date, c 3 is the only public chinese dataset of this task. experimental results show that our multi-teacher paradigm leads to +4.3% in accuracy over a state-of-the-art baseline @xcite and also outperforms other methods such as pre-training upon all scripts and hard-label fine-tuning that uses the same structured knowledge. also, under the same framework, constructing weakly-labeled mrc data based on verbalnonverbal knowledge leads to bigger gains compared to other strong data augmentation strategies (and some are even based on human-annotated commonsense knowledge graphs @xcite ). finally, we seek to transfer the knowledge to help other tasks by adapting the final student mrc model, yielding +2.9% in f1 on the chinese set of a document-level relation extraction dataset dialo-gre @xcite ) over competitive baselines. the main contributions are as follows: (i) we suggest that scripts can be a good resource for extracting rich contextualized verbal-nonverbal knowledge with minimal supervision; (ii) we explore and compare several fine-tuning strategies to use weakly-labeled data based on the extracted knowledge and further propose a teacher-student paradigm with multiple teachers to better learn from large-scale weakly-labeled data; and (iii) our empirical results demonstrate the effectiveness of the multi-teacher paradigm and the usefulness of contextualized verbal-nonverbal knowledge for mrc tasks that require commonsense knowledge. ## contextualized knowledge extraction understanding the interactions between verbal and nonverbal information in communications requires prior knowledge, and this knowledge is assumed scene 1 2 interior. runaway office. day. andy: i tried to ask her, but... emily: you never ask miranda. anything. (sighs) all right, i'll take care of the other stuff. you go to calvin klein. andy: me? emily: i'm sorry. do you have a prior commitment? is there some hideous pants convention? andy: so i just, what, go down to the calvin klein store and ask them... 3 emily rolls her eyes so hard they almost eject from her head. emily: you're not going to the store. andy: of course not. i'm going...(thinking)...to his house. emily (oh god): you are catching on quickly. we always send assistants to a designer's home on their very first day. you're going to his showroom. i'll give you the address. andy: sorry. got it. what's the nearest subway stop? emily: good god. you do not. under any circumstances. take public transportation. andy: i don't? pattern type nonverbal verbal bc oh god emily: you are catching on [. . .] i'll give you the address. i sighs emily: you never ask miranda. anything. all right [. . .] klein. i thinking andy: of course not. i'm going......to his house. o emily rolls her eyes so hard andy: so i just, what, go down to the they almost eject from her head. calvin klein store and ask them... to be known by most people without being taught, just as commonsense knowledge. we propose to use interrelated verbal and nonverbal information as phrases in the classical triple-style knowledge representation and situate them in a context. formally, we call a triple (v, c, n) as a piece of contextualized knowledge, containing a pair of related verbal information v and nonverbal information n, and the associated context c. we choose to extract contextualized knowledge from film and television show scripts as plentiful verbal and nonverbal messages frequently co-occur in scripts, and they can be easily separated. scenes in a script are delimited by blank lines. based on the relative position of verbal and nonverbal information (intra-turn or cross-turn), we extract four subsets of contextualized knowledge (b c , b n , i, and o): we roughly regard a phrase as a speaker name if it appears in the first span of other turns in the same scene. we do not extract phrases in parentheses or action lines as nonverbal information if they are terminologies for script writing such as "o.s.", "v.o." "cont'd", "beat", "jump cut", and "fade in". 1 all contextualized knowledge triples extracted from a scene share the same context c, i.e., the scene itself. we do not exploit scene headings (e.g., when and where a scene takes place) (marked by 2 in ## mrc instance generation we mainly discuss how to convert the extracted triples into multiple-choice instances and leave its extension to other types (e.g., extractive or abstractive) of mrc tasks for future research. we generate one instance for each piece of contextualized knowledge. for each triple (v, c, n), we remove n from context c, and we regard the remaining content as the reference document, verbal information v as the question, and the nonverbal information n as the correct answer option. to generate distractors (i.e., wrong answer options), we randomly select n items from all the unique nonverbal messages in other triples, which are extracted using the same type of patterns from the same script as @xcite . note that although we only generate one instance based on each triple, it is easy to generate more instances by changing distractors. ## two-stage fine-tuning we aim to use the constructed weakly-labeled data to improve a downstream mrc task. given weaklylabeled data generated based on contextualized knowledge extracted from scripts, we first use the weakly-labeled data in conjunction with the training set of the target mrc data as the training data to train the model and then fine-tune the resulting model on the target mrc data as illustrated in figure 1. we do not adjust the ratio of clean data to weakly-labeled data observed during training as previous joint training work on other tasks such as machine translation @xcite . another way is to perform separate training (or sequential transfer learning @xcite ): we can first train the model on the weakly-labeled data and then fine-tune it on the target data. we assume that using the small-scale, clean data to guide weakly-supervised training is more helpful, and we compare joint and separate training in section 6.5. ## multi-teacher paradigm in our preliminary experiment, we observe that increasing the amounts of weakly-labeled data does not lead to noticeable gains than using one subset of the weakly-labeled data. inspired by previous teacher-student frameworks @xcite ) that train a multi-domain student model with multiple teacher models to help knowledge transfer, and each teacher model is trained on a clean domain-specific subset, we extend the multi-teacher idea to let models learn better from large-scale weakly-labeled mrc data for improving a target mrc task. as introduced in section 3, we can have multiple subsets of weakly-labeled data generated based on verbal-nonverbal knowledge extracted by different types of patterns ((e.g., b c or i) in (section 2)). let v denote a set of labeled instances, w 1 , . . . , w ℓ denote ℓ sets of weakly-labeled instances, and @xmath0. for each instance t, we let m t denote its total number of answer options, and h (t) be a hard label vector (one-hot) such that h (t) @xmath1. we train ℓ teacher models, denoted by t 1 , . . . , t ℓ , and optimize t i by minimizing @xmath2, θ t i ). l 1 is defined as we then train a student model, denoted by s, still in a two-stage fashion. in stage one (i.e., weaklysupervised fine-tuning), we optimize s by minimizing @xmath3, θ s ), where l 2 is defined as in stage two (i.e., supervised fine-tuning), we further fine-tune the resulting s after stage one by we collect 100,46 scripts in chinese, and most of them are intended for films and television shows. 2 after segmentation and filtering, we obtain 199,280 scenes, each of which contains at least one piece of contextualized knowledge. we generate four subsets of weakly-labeled mrc data based on contextualized knowledge extracted by four types of patterns. for comparisons, we use existing humanannotated triples about commonsense knowledge in the chinese version of conceptnet @xcite as well as span-style augmented multiplechoice mrc data @xcite extracted from all scripts. we set the number of distractors n to five for weakly-labeled mrc instances. for evaluation, we use c 3 , so far as we know, the only public multiple-choice mrc data for chinese with a focus on commonsense knowledge. about 86.8% of questions in c 3 involve prior knowledge (i.e., linguistic, domain-specific, and commonsense knowledge) unstated in the given texts, and all instances are carefully designed by experts such as 2 https://www.1bianju.com . ## conclusions this paper introduces how to extract contextualized verbal-nonverbal knowledge from scripts and use the knowledge to improve multiple-choice mrc. we compare different strategies to use the knowledge-based weakly-labeled data and further propose a multi-teacher paradigm to better learn from large-scale weakly-labeled data. experimental results show the effectiveness of multi-teacher and usefulness of the knowledge for different tasks.