Natural Response Generation for C hinese Reading Comprehension (2023)

Abstract: Machine reading comprehension (MRC) is an important area of conversation agents and draws a lot of attention. However, there is a notable limitation to current MRC benchmarks: The labeled answers are mostly either spans extracted from the target corpus or the choices of the given candidates, ignoring the natural aspect of high-quality responses. As a result, MRC models trained on these datasets can not generate human-like responses in real QA scenarios. To this end, we construct a new dataset called Penguin to promote the research of MRC, providing a training and test bed for natural response generation to real scenarios. Concretely, Penguin consists of 200k training data with high-quality fluent, and well-informed responses. Penguin is the first benchmark towards natural response generation in Chinese MRC on a relatively large scale. To address the challenges in Penguin, we develop two strong baselines: end-to-end and two-stage frameworks. Following that, we further design Prompt-BART : fine-tuning the pre-trained generative language models with a mixture of prefix prompts in Penguin. Extensive experiments validated the effectiveness of this design.

## introduction machine reading comprehension (mrc) aims to empower the machine to correctly answer queries based on the given passages or paragraphs, which is an active field in nlp. much of this popularity can be attributed to the release of many annotated and publicly available datasets @xcite @xcite @xcite . formally, these mrc efforts can be classified into two most popular streams foot_0 from the answer type perspective: span-extraction @xcite @xcite @xcite and multiple choices @xcite @xcite . the former requires the model to locate the text span in the given passage as the answer, e.g., squad @xcite and newqa @xcite . squad, for example, introduces extracted span answers. the latter needs to choose the answer from a list of candidates like race @xcite . considering most of the existing works are built in english, several efforts have been proposed to further advance mrc in chinese such as dureader @xcite , cmrc 2018 @xcite , reco @xcite . nonetheless, despite significant achievements in the various types and on a relatively large scale, a significant barrier that has rarely been discussed in previous mrc efforts remains: annotated answers among these datasets are almost limited to spans in the source document or the given candidates. while spans directly copied from the source document and simple words like "yes/no" can be seen as ground-truth answers, they don't sound like natural responses in real-life scenarios. in reality, the responses of humans are always generated with flexibility and information rather than being forced to select certain type spans ( e.g., entities and person) or one of the candidates. this work devotes itself to natural response generation for chinese mrc, catering to real-world qa scenarios. intuitively, we can try two existing technical routes to generate natural responses towards mrc at the moment: 1) optimizing generative language models (glms) such as bart and t5 with predetermined and inflexible answers on the above benchmarks. 2) use extremely large-passage ...bream, which is traditionally steamed or braised, is a popular fish. soup should of course be cooked, but soles is not suitable for this dish because of its fleshy nature... ...鳊鱼一直备受人们喜爱，鳊鱼的传 统做法是清蒸或者红烧。当然不排 除可以烧汤，但是鉴于鳊鱼的肉质 特色，不是很适合烧汤的... ## conclusion in this paper, we summarize the limitation of current mrc efforts, and propose a new dataset named penguin for natural response generation in chinese mrc. penguin requires the predicted responses not only be accurate but also natural-sounding, catering to real scenarios. thereafter, we propose end-to-end and two-stage grc pipelines to address the challenge in penguin, achieving promising results. moreover, we design prompt-tuning, which involves fine-tuning the pre-trained genera-tive language models with a mix of different prefix prompts in penguin, making the system perform better and more efficiently. as an extension of future work, we will explore other techs to build more human-like grc models.