NCUEE - NLP at MEDIQA 2021: Health Question Summarization Using PEGASUS Transformers (2021)

Abstract: This study describes the model design of the NCUEE-NLP system for the MEDIQA challenge at the BioNLP 2021 workshop. We use the PEGASUS transformers and fine-tune the downstream summarization task using our collected and processed datasets. A total of 22 teams participated in the consumer health question summarization task of MEDIQA 2021. Each participating team was allowed to submit a maximum of ten runs. Our best submission, achieving a ROUGE2-F1 score of 0.1597, ranked third among all 128 submissions.

## introduction consumers increasingly use online resources to meet their health information needs. however, health information needs are usually complex and to be expressed in natural language @xcite . in general, health questions tend to consist of considerable contextual information that may hinder automatic question answering (qa) systems. paraphrasing and summarizing the questions has been shown to substantially improve qa performance @xcite . therefore, effective summarization methods for consumer health questions could play an important role in enhancing medical qa performance. automatic text summarization is the process of computationally shortening texts to find or generate the most informative sentences that represent the most important or relevant information within the original content. there are two general approaches to summarization: extraction and abstraction. in extractive summarization methods, a summary is extracted from the original texts, but the extracted sentences are not modified in any way. abstractive summarization methods learn a semantic representation of the original content, and then use this representation to generate a summary that is closer to what a human might express in terms of original content. mediqa 2021 is the second edition of the mediqa challenge collocated with the bionlp 2021 workshop, focusing on summarization in the medical domain with three tasks: consumer health question summarization, multi-answer summarization, and radiology report summarization. we only participated the first question summarization (qs) task, in the domain of abstractive summarization. the goal of this task is to promote the development of new summarization methods that specifically address the challenges of long and complex consumer health questions. the recently developed transformer in nlp is a novel neural architecture that aims to solve sequence-to-sequence tasks in handling long dependencies and usually achieves promising results. this achievement motivates us to explore the use of a transformer-based model to tackle the question summarization problem in the medical domain. this paper describes the ncuee-nlp (national central university, dept. of electrical engineering, natural language processing lab) system for the qs task of the mediqa challenge at the bionlp 2021 workshop. our solution explores the use of pre-trained pegasus transformers @xcite and fine-tuning on the downstream question summarization task using our collected and processed datasets. a total of 22 teams participated in this task. each participating team was allowed to submit a maximum of ten runs. our best submission had a rouge2-f1 score of 0.1597, ranking third among all 128 submissions. ## the ncuee-nlp system figure 1 shows our ncuee-nlp system architecture for the qs task. specifically, our system is comprised of two main parts: 1) pegasus transformers, and 2) fine-tuning. details are introduced as follows. ## conclusions this study describes the ncuee-nlp system in the consumer health question summarization task of the mediqa 2021 challenge, including system design, implementation and evaluation. we used the pegasus transformers and fine-tuned the downstream summarization task using our collected and processed datasets. a total of 22 teams participated in the task, each submitting at least one entry. our best submission had a rouge2-f1 score of 0.1597, ranking third place among all 128 submissions. f-008-003-through pervasive artificial intelligence research (pair) labs, taiwan.