UCSD -Adobe at MEDIQA 2021: Transfer Learning and Answer Sentence Selection for Medical Summarization (2021)

Abstract: In this paper, we describe our approach to question summarization and multi-answer summarization in the context of the 2021 MEDIQA shared task (Ben Abacha et al., 2021). We propose two kinds of transfer learning for the abstractive summarization of medical questions. First, we train on HealthCareMagic, a large question summarization dataset collected from an online healthcare service platform. Second, we leverage the ability of the BART encoder-decoder architecture to model both generation and classification tasks to train on the task of Recognizing Question Entailment (RQE) in the medical domain. We show that both transfer learning methods combined achieve the highest ROUGE scores. Finally, we cast the question-driven extractive summarization of multiple relevant answer documents as an Answer Sentence Selection (AS2) problem. We show how we can preprocess the MEDIQA-AnS dataset such that it can be trained in an AS2 setting. Our AS2 model is able to generate extractive summaries achieving high ROUGE scores.

## introduction the 2021 medical nlp and question answering (mediqa) shared task @xcite is comprised of three tasks, centered around summarization in the medical domain: question summarization, multi-answer summarization, and radiology report summarization. in this paper, we focus on the first two tasks. in question summarization, the goal is to generate a one-sentence formal question summary from a consumer health question -a relatively long question asked by a user. in multi-answer summarization, we are given a one-sentence question and multiple relevant answer documents, and the aim is to compose a questiondriven summary from the answer text. in this paper, we first show that transfer learning from pre-trained language models can achieve very high results for question summarization. sequenceto-sequence language model bart @xcite has achieved state-of-the-art results in various nlp benchmarks, including in the cnn-dailymail news article summarization dataset @xcite . we leverage this success and train bart on summarization datasets from the medical domain @xcite @xcite . moreover, we find that training on a different task in the medical domain -recognizing question entailment (rqe) @xcite ) -can yield better improvements, especially in terms of rouge precision scores. second, we tackle the extractive track of the multi-answer summarization task, and we cast multi-answer extractive summarization as an answer sentence selection (as2) problem. a limitation of bart is that the input to its abstractive summarization cannot be as long as the multiple documents in this task. we therefore propose to mitigate this weakness by proposing to cut up the input into pairs of sentences, where the first sentence is the input question, and the second one is a candidate answer. we then train our bart model to score the relevance of each candidate answer with regards to its corresponding question. we also describe in this paper the algorithm used to extract an as2 dataset from an multi-document extractive summarization dataset. ## question summarization our approach to question summarization involves two kinds of transfer learning. first, we train our model to learn from medical summarization datasets. second, we show that transfer learning from other tasks in the medical domain increases rouge scores. ## conclusions this paper describes the approach taken by our team, ucsd-adobe, at the 2021 mediqa shared task. we tackle the tasks of question summarization and multi-answer summarization. for question summarization, we propose two kinds of transfer learning. first, we propose to pretrain on a large-scale dataset of abstractive summarization of medical questions, healthcaremagic. our results show that training on this dataset enhances performance in both validation and test sets. then, we propose to transfer from another medical question-based task: recognizing question entailment. this binary classification task increases performance, and precision scores in particular. in the test results, the highest rouge scores are achieved by a model trained on both transfer learning methods. we tackle the extractive track of the multianswer summarization task. we propose to cast the question-driven extractive summarization of multiple answer documents as an answer sentence selection problem. we show how we can transform the mediqa-ans dataset into an as2 dataset. we show that we achieve good rouge scores with and without transfer learning from question summarization on the validation set. in the test results, the model without question summarization training achieves the highest rouge scores.