NLM at MEDIQA 2021: Transfer Learning-based Approaches for Consumer Question and Multi-Answer Summarization (2021)

Abstract: The quest for seeking health information has swamped the web with consumers’ healthrelated questions, which makes the need for efficient and reliable question answering systems more pressing. The consumers’ questions, however, are very descriptive and contain several peripheral information (like patient’s medical history, demographic information, etc.), that are often not required for answering the question. Furthermore, it contributes to the challenges of understanding natural language questions for automatic answer retrieval. Also, it is crucial to provide the consumers with the exact and relevant answers, rather than the entire pool of answer documents to their question. One of the cardinal tasks in achieving robust consumer health question answering systems is the question summarization and multi-document answer summarization. This paper describes the participation of the U.S. National Library of Medicine (NLM) in Consumer Question and Multi-Answer Summarization tasks of the MEDIQA 2021 challenge at NAACL-BioNLP workshop. In this work, we exploited the capabilities of pre-trained transformer models and introduced a transfer learning approach for the abstractive Question Summarization and extractive Multi-Answer Summarization tasks by first pre-training our model on a task-specific summarization dataset followed by fine-tuning it for both the tasks via incorporating medical entities. We achieved the second, sixth and the fourth position for the Question Summarization task in terms ROUGE-1, ROUGE-2 and ROUGE-L scores respectively.

## introduction healthcare consumers often query over the web to find a quick and reliable answer to their healthcare information needs. on average, 6 million people only in the united states seek health-related information on the internet every day @xcite . one way to facilitate such informationseeking activities is to build a natural language question answering (qa) system that can extract precise answers from the myriad of health-related information sources @xcite . though existing search engines respond to the general health-related queries to some extent, users often reach out to specialized medical websites or online health communities for seeking personalized high-quality, and trustworthy answers for their complex health questions. moreover, consumers while expressing their medical concern on these sources except the involvement of healthcare professionals (hps) for a quality suggestion and virtual observation @xcite . however, the participation of hps in large-scale discussion forums or medical websites is time-consuming and expensive. furthermore, the consumers' questions are very descriptive and contain several peripheral information (like patient's medical history), which contributes to the challenges of understanding natural language questions for automatic answer retrieval @xcite . these elaborated details are often not required for providing the relevant answers. hence, novel strategies should be devised for automatic question simplifications and answer retrieval. towards this, we study the tasks of question summarization (qs) and multi-answer summarization (mas) as a part of mediqa 2021 (asma ben abacha, 2021) shared task challenge. for the task of question summarization (qs), we proposed the transfer learning approach by utilizing multiple pre-trained transformer @xcite models. in our best run, we fine-tuned the pre-trained models on a variety of question summarization datasets and proposed a medical entities coverage technique to select the best question summary from the pool of question summaries obtained from the various transformer models. we also explored the transfer learning approach for the multi-answer summarization task. specifically, the proposed method uses the text-to-text transfer transformer (t5) relevance-based re-ranking model @xcite . in our best system, we first fine-tuned t5 on msmarco passage and then mediqa-qa 2019 datasets. it first ranks the sentences of the answers and then rejoins the top-k sentences as a summary. ## related work existing works on the summarization can be broadly categorized into (i) extractive and (ii) abstractive approach which are discussed as follows: extractive summarization: the recent development in the neural network and transformer based models has led to the significant progress in extractive document summarization. majority of the models focus on the encoder-decoder model @xcite @xcite , recurrent neural network @xcite , and stateof-the-art transformers encoders @xcite . for instance, @xcite and @xcite proposed an encoder-decoder model as a binary classifier to decide whether the input sentence will be part of the summary or not. @xcite utilize a pointer generator network @xcite to sequentially select sentences from the document for generating the extractive summary. other decoding techniques, such as ranking @xcite has also been utilized for content selection. recently several studies have explored pre-trained language models in summarization for contextual word representations @xcite . ## conclusion and future work in this paper, we describe our submissions for the tasks of question summarization and multi answer summarization at mediqa 2021 shared task. for the question summarization task, our best run achieved the second-best rouge-1 score among all the submitted runs in the shared task. we also obtained the competitive scores in terms of various evaluation metrics over the other participant's runs. for the multi-answer summarization task, our t5based approach achieved good performances compared to participants' systems. in the future, we will explore the techniques to integrate the medical entities and semantics in the pre-trained transformer models for the task of question summarization. further, we will also explore the abstractive approaches for multi-answer summarization.