PARROT : Zero-Shot Narrative Reading Comprehension via Parallel Reading (2023)

Abstract: Narrative comprehension is a challenging task that requires a deep understanding of the foundational elements of narratives. Acquiring this skill requires extensive annotated data. To mitigate the burden of data annotation, we present Parrot, a zero-shot approach for narrative reading comprehension through parallel reading, which involves two parallel narratives that tell the same story. By leveraging one narrative as a source of supervision signal to guide the understanding of the other, Parrot abstracts the textual content and develops genuine narrative understanding. Evaluation conducted on two narrative comprehension benchmarks demonstrates that Parrot surpasses previous zero-shot approaches and achieves comparable performance to fully supervised models. The code will be available at https://github.com/zhaochaocs/Parrot.

## introduction narratives have long been recognized as a valuable resource for linguistic, scientific, cultural, and social learning @xcite @xcite @xcite . narrative comprehension, therefore, is considered a fundamental aspect of human intelligence @xcite and an important tool for cognitive development and meaning-making @xcite . with this motivation, previous research has tackled the task of narrative reading comprehension, which involves automatically comprehending a given narrative and answering questions related to it @xcite . however, in comparison to general text comprehension, which typically focuses on the understanding of named entities and factual information @xcite , narrative comprehension presents unique challenges. specifically, it requires understanding the foundational elements of narratives. these elements include events along with their temporal and causal connections; settings narrative 𝒩: us navy colonel pete "maverick" mitchell is a test pilot of the hypersonic "darkstar" scramjet program. rear admiral chester "hammer" cain plans to cancel "darkstar" in favor of funding a more promising drone program. to meet the final contract specification and save the program, maverick unilaterally changes the target speed for that day's test from mach 9 to mach 10. narrative 𝒩 ! : colonel pete "maverick" mitchell is a test pilot in the mojave desert. one day, rear admiral chester "hammer" cain attempted to suspend the hypersonic flight test machine "darkstar" project and intended to reallocate the funds to an unmanned drone program under his command. before cain arrived, maverick decided to push the prototype to its contractually specified speed of mach 10. q1: who is the test pilot? [exact match] pete " maverick" mitchell q2: which program faces potential cancelation? @xcite darkstar q3: what's the relation between @xcite maverick is the test pilot of darkstar q4: what motivates hammer to fund drone program? @xcite he thinks that program is more promising q5: why does maverick pilot darkstar at mach 10? [event causality] to meet the final contract specification and save the program such as the time, place, and environment; as well as characters, including their motivations, desires, emotions, and relationships with other characters. together they exhibit intricate plot structures and involve complex character interactions, making it challenging for machines to comprehend. despite the availability of extensively annotated data for general text reading comprehension, there is currently a lack of sufficient annotated data in the narrative domain, and it is not optimal to directly use models trained on general text data for narrative reading comprehension. hence, there is a need to develop data-efficient learning approaches for narrative reading comprehension. to address the aforementioned challenges, our idea is to leverage parallel reading: reading two parallel narratives that convey the same story but differ in various aspects of story-telling style. this narrative 𝒩 ! colonel pete "maverick" mitchell is a test pilot in the mojave desert. one day, rear admiral chester "hammer" cain attempted to suspend the hypersonic flight test machine "darkstar" project and intended to reallocate the funds to an unmanned drone program under his command. before cain arrived, maverick decided to push the prototype to its contractually specified speed of mach 10. ## related works data-efficient reading comprehension has been addressed by automatically generating (passage, question, answer) triples for pre-training a model @xcite . however, it requires annotated data to train a question-generation module. one possible solution is to replace it with a heuristic-based question-generation module @xcite @xcite . another option is to replace natural questions with cloze questions by masking the answer from the passage @xcite . researchers have proposed masking and predicting recurring spans within the passage as answers @xcite . in contrast to these approaches that focus on general extractive reading comprehension, par-rot specifically addresses the unique challenge of open-ended narrative comprehension and leverages parallel reading for abstractive comprehension and free-form question answering. ## parrot in narrative reading comprehension, the input is a narrative n and a question q, while the output is a concise answer a. we develop parrot , a zeroshot solution for this problem. parrot utilizes a masked language modeling (mlm) based pretraining approach, which incorporates a selective span masking strategy to mask essential narrative elements (sec. 3.1) and a parallel reading strategy to learn to predict the masked spans (sec. 3.2). next, to utilize the pre-trained model in a zero-shot fashion, we transform the downstream narrative reading comprehension task to match the format of the pre-training task (sec. 3.3). figure 2 shows an overall illustration. ## experiments in this section, we evaluate the performance of parrot . ## analysis we conduct analysis to better understand the behavior of parrot . ## conclusion we introduce parrot , a novel zero-shot approach for narrative reading comprehension based on pretraining. by selectively masking significant elements within the narrative and pre-training the model to predict these spans through parallel reading, parrot learns to abstract essential textual content and gains a genuine understanding of the narrative. experimental results on two diverse narrative datasets demonstrate the superiority of parrot , showcasing its effectiveness in enhancing narrative reading comprehension. our analysis further emphasizes the significance of employing a diverse range of masked spans and leveraging the parallel reading strategy during model pre-training.