UET fishes at MEDIQA 2021: Standing-on-the-Shoulders-of-Giants Model for Abstractive Multi-answer Summarization (2021)

Abstract: This paper describes a system developed to summarize multiple answers challenge in the MEDIQA 2021 shared task collocated with the BioNLP 2021 Workshop. We present an abstractive summarization model based on BART, a denoising auto-encoder for pre-training sequence-to-sequence models. As focusing on the summarization of answers to consumer health questions, we propose a query-driven filtering phase to choose useful information from the input document automatically. Our approach achieves potential results, rank no.2 (evaluated on extractive references) and no.3 (evaluated on abstractive references) in the final evaluation.

## introduction in the past several decades, biomedicine and human health care have become one of the major service industries. they have been receiving increasing attention from the research community and the whole society. the rapid growth of volume and variety of biomedical scientific data make it an exemplary case of big data @xcite . it is an unprecedented opportunity to explore biomedical science and an enormous challenge when facing a massive amount of unstructured and semistructured data. the development of search engines and question answering systems has assisted us in retrieving information. however, most biomedical retrieved knowledge comes from unstructured text form. without considerable medical knowledge, the consumer is not always able to judge the correctness and relevance of the content @xcite . it also takes too much time and labour to process the whole content of these documents rather than extracting the useful compressed content. automatic summarization is a challenging application of biomedical natural language processing. it generates a concise description that captures the salient details (called summary) from a more complex source of information @xcite . summarization can be particularly beneficial for helping people easily access electronic health information from search engine and question answering systems. mediqa 2021 foot_0 @xcite tackles three summarization tasks in the medical domain. task 2-summarization of multiple answers challenge aims to promote the development of multi-answer summarization approaches that could simultaneously solve the aggregation and summarization problems posed by multiple relevant answers to a medical question. there are two approaches to summarization: extractive and abstractive. extractive summarization, i.e., choose important sentences from the original text, is extensively researched but have several limitations: (i) it is unable to keep the coherence of the answer, (ii) the information compressed may be incomplete because information may take many sentences to expose, and (iii) it must include nonrelevant part of a relevant sentence. recently, the research has shifted towards more promising approaches, i.e. abstractive summarization, which can overcome these problems give higher precision than extractive summaries @xcite . abstractive text summarization is the task of generating a short and concise summary that captures the salient ideas of the source text. the generated summaries potentially contain new phrases and sentences that may not appear in the source text. abstractive summarization helps resolve the dangling anaphora problem and thus helps generate readable, concise and cohesive summaries. in abstractive summary, we can merge several relate sentences or make them shorter, i.e., removing the redundancy part. our proposed model for the multi-answer summarization task follows abstractive summarization approaches. we try to process original answers as a shorter representation while preserving information content and overall meaning. we take advantage of bart, a pre-trained model combining bidirectional and auto-regressive transformers @xcite . we construct an architecture with two filtering phases to choose the more concise input for bart. since the summary should be question-oriented, the coarse-grained filtering phase removes question-irrelevant sentences. the fine-grained filtering phase is then used to cut-off noise phases. the remaining of this paper is organized as follows: section 2 gives brief introduction of some state-of-the-art related work. section 3 describes task data and our proposed model. section 4 is the experimental results and our discussion. and finally, the conclusion. ## related work because of the complexity of natural language, abstractive summarization is a challenging task and has only been of interest in recent years. @xcite proposed an abstractive summarization system for product reviews by taking advantage of their discourse tree structure. a important subgraph in the discourse tree were then selected by using pagerank algorithm. a natural language summary was then generated by applying a template-based nlg framework. according to current research trends, witnessing the success of deep learning in other nlp tasks, researchers have started considering this framework as an promising solution for abstractive summarization. @xcite used an attentional encoder-decoder recurrent neural networks and several models such as key-words modeling, sentence-to-word hierarchy structure, and emitting rare words, etc. @xcite proposed an lstm-cnn based ats model to construct new sentences by exploring fine-grained phrases from source sentences (of cnn and dailymail) and combining them. @xcite used a bottom-up attention technique to improve the deep learning model by over-determining phrases in a source document that should be part of the summary. inspired by the successful application of deep learning methods for machine translation, abstractive text summarization is specifically framed as a sequence-to-sequence learning task. bart is a transformer-based pretrained denoising encoder-decoder model that is applicable to a very wide range of end tasks, includes summarization. it combines a bidirectional encoder and an auto-regressive decoder @xcite . there are several bart-based model, example includes distilbart foot_1 and question-driven bart @xcite . question-driven bart re-trained bart on objectives designed to improve its general ability to understand the content of text (including document rotation, sentence permutation, text-infilling, token masking and token deletion) and fine-tuned the model for biomedical data. another recently published abstractive summarization framework is pegasus @xcite , it masks important sentences and generates those gap-sentences from the rest of the document as an additional pretraining objective. ## conclusion this paper presents a systematic study of our abstractive approach to question-driven summarization problem, specifically depending on mediqa 2021 -task 2: multi-answer summarization. we present a model improved and optimized based on bart -a state-of-the-art method for abstractive summarization called ssg (standing on the shoulders of giants). the proposed model has a potential performance, being the runner-up of the shared task. our best performance achieved a rouge-2 f 1 is 0.470 evaluated on extractive summarization references and 0.147 evaluated on abstractive summarization references . experiments were also carried out to verify the rationality and impact of model components and the compressed ratio. the results demonstrated the contribution and robustness of all techniques and hyper-parameters. besides, the error analysis was made to analyze the sources of the errors. the evidence pointed out some imperfection of the sentence selecting strategy, the ranking score combination, and the question analyzer. in further works, there could be several ways: applying machine learning model, deeply question-analyzing, sentence clustering, etc. applied to extend the ability of the model. our source code will be released publicly to support the reproducibility of our work and facilitate other related studies.