Lite Unified Modeling for Discriminative Reading Comprehension (2022)

Abstract: As a broad and major category in machine reading comprehension (MRC), the generalized goal of discriminative MRC is answer prediction from the given materials. However, the focuses of various discriminative MRC tasks may be diverse enough: multi-choice MRC requires model to highlight and integrate all potential critical evidence globally; while extractive MRC focuses on higher local boundary preciseness for answer extraction. Among previous works, there lacks a unified design with pertinence for the overall discriminative MRC tasks. To fill in above gap, we propose a lightweight POS-Enhanced Iterative Co-Attention Network (POI-Net) as the first attempt of unified modeling with pertinence, to handle diverse discriminative MRC tasks synchronously. Nearly without introducing more parameters, our lite unified design brings model significant improvement with both encoder and decoder components. The evaluation results on four discriminative MRC benchmarks consistently indicate the general effectiveness and applicability of our model, and the code is available at https://github.com/Yilin1111/poi-net .

## introduction machine reading comprehension (mrc) as a challenging branch in nlu, has two major categories: generative mrc which emphasizes on answer generation @xcite , and discriminative mrc which focuses on answer prediction from given contexts @xcite . among them, discriminative mrc is in great attention of researchers due to its plentiful application scenarios, such as extractive and multi-choice mrc two major subcategories. given a question with corresponding passage, extractive mrc asks for precise answer span extraction in passage @xcite multi-choice mrc example ... in addition, lynn's pioneering efforts also provide public educational forums via green scenesa series of three hour events, each focusing on specific topics teaching hoosiers how to lead greener lifestyles. ... q: what can we learn about green scenes? a. it is a scene set in a three-hour film. b. it is a series of events focusing on green life. (golden) c. it is a film set in central indiana. d. it is a forum focusing on green lifestyle. extractive mrc example ... early versions were in use by 1851, but the most successful indicator was developed for the high speed engine inventor and manufacturer charles porter by charles richard and exhibited at london exhibition in 1862. ... q: where was the charles porter steam engine indicator shown? golden answer: london exhibition imprecise answer 1: london exhibition in 1862 imprecise answer 2: exhibited at london exhibition in mrc field, several previous works perform general-purpose language modeling with considerable computing cost at encoding aspect @xcite @xcite , or splice texts among diverse mrc tasks simply to expand training dataset @xcite , without delicate and specialized design for sub- the decoding design simulates human reconsideration and integration abilities at global level, with especial effect for multi-choice mrc. to handle compound questions with limited attention, human will highlight critical information in turns, and update recognition and attention distribution iteratively. inspired by above reconsideration strategy, we design iterative co-attention mechanism with no additional parameter, which iteratively exe-cutes the interaction between passage and questionoption (q -o) pair globally in turns. in the multichoice example from ## our model we aim to design a lightweight, universal and effective model architecture for various subcategories of discriminative mrc, and the overview of our model is shown in figure 1 , which consists of four main processes: encoding ( §2.1), interaction ( §2.2), integration ( §2.3) and output ( §2.4). ## ablation studies in this section, we implement poi-net on albert base for further discussions, and such settings have the similar quantitative tendency to poi-net on albert xxlarge . to evaluate the contribution of each component in poi-net, we perform ablation studies on race and squad 1.1 development sets and report the average results of three random seeds in ## conclusion in this work, we propose pos-enhanced iterative co-attention network (poi-net), as a lightweight unified modeling for multiple subcategories of discriminative mrc. poi-net utilizes pos embedding to encode pos attributes for the preciseness of answer boundary, and iterative co-attention mechanism with integration strategy is employed to highlight and integrate critical information at decoding aspect, with almost no additional parameter. as the first effective and unified modeling with pertinence for different types of discriminative mrc, evaluation results on four extractive and multi-choice mrc benchmarks consistently indicate the general effectiveness and applicability of our model.