UET rice at MEDIQA 2021: A Prosper-thy-neighbour Extractive Multi-document Summarization Model (2021)

Abstract: This paper describes a system developed to summarize multiple answers challenge in the MEDIQA 2021 shared task collocated with the BioNLP 2021 Workshop. We propose an extractive summarization architecture based on several scores and state-of-the-art techniques. We also present our novel prosper-thy-neighbour strategies to improve performance. Our model has been proven to be effective with the best ROUGE-1/ROUGE-L scores, being the shared task runner up by ROUGE-2 F1 score (over 13 participated teams).

## introduction biomedical documents are available with the tremendous amount on the internet, together with several search engines (e.g., pubmed® 1 ) and question-answering systems (e.g., chiqa 2 ) developed. however, the returned results of these systems still contain a lot of noise and duplication, making them difficult for users without medical knowledge to quickly grasp the main content and get the necessary information. hence, generating a shorter condensed form with important information would benefit many users as it saves time and can retrieve massive useful information. this motivation leads to the growing interest among the research community in developing automatic text summarization methods. the bionlp-mediqa 2021 shared task 3 @xcite aims to attract further research efforts in text summarization and their applications in medical question-answering (qa). this shared task is motivated by a need to develop relevant methods, techniques, and gold standards for text summarization in the * contributed equally & names are in alphabetical order † * corresponding author 1 https://pubmed.ncbi.nlm.nih.gov/ 2 https://chiqa.nlm.nih.gov/ 3 https://sites.google.com/view/ mediqa2021 medical domain and their application to improve the domain-specific qa system. task 2 -summarization of multiple answers focuses on developing multi-document summarization approaches that could synthesize and compress information from answers to a medical question. according to @xcite a summary is defined as 'a text that is produced from one or more texts, that conveys important information in the original text(s), and that is no longer than half of the original text(s) and usually, significantly less than that'. automatic text summarization is the task of condensing the document(s) and generating a compressed summary, which is shorter but preserves key information content and overall meaning. a summary can be generated through extractive or abstractive approaches (or hybrid). typically, to produce an abstractive summarization, we need to use advanced linguistic techniques to 'understand' the text as well as re-generate the summary in natural language from useful information. up to now, the research community is focusing more on extractive summarization. this approach tries to achieve coherent and meaningful summaries in a more simple and faster way than the abstractive approach. extractive summarization chooses important sentences (or phrases) from the original documents (without any modification) and merges them to generate a summary. our proposed model for the multi-answer summarization task follows extractive summarization approaches. we try to select sentences containing the most important information in the original answers. our novel contributions are: (i) proposing the question-driven scores to ensure that the summary is the answer to the question, (ii) proposing prosper-thy-neighbour (ptn) strategies, which increase the constraint of neighbouring sentences, to take advantage of paragraph information in the answer. (iii) combining several scores that successfully applied for summarization problem, includ-ing tf-idf, lexrank, and textrank with optimized weights, (iv) improving the maximal marginal relevance technique (mmr) for multi-document summarization with bert-based embedding to improve the performance. the remaining of this paper is organized as follows: section 2 gives a brief introduction to some state-of-the-art related works. section 3 describes task data and our proposed model. section 4 is the experimental results and our discussion. and finally, the conclusion. ## related works from the early 1950s, various methods have been proposed for extractive summarization @xcite . some of them are based on the idea of using scores to choose the most important phrases in the documents. term frequency-inverse document frequency (tf-idf) @xcite is a frequency-based score to detect important sentences by calculating the scores of its words. lexrank @xcite and textrank @xcite are two graph-based methods that rank sentences/words using their degree centrality. maximal marginal relevance (mmr) @xcite is one of the most well-known approaches for multi-document summarization. it is a diversity-based re-ranking method based on the document similarities and can be used to remove redundancy in the summaries. although encouraging results have been reported, most of these scores are applied individually. since each score type has its unique contribution, combining them may help to improve the performance. hence, we propose an architecture to take advantage of several scores with weights and calculate a final combined score. with the advent of machine learning techniques in nlp, many research projects tried to apply machine learning methods to extractive summarization tasks, from the naive bayes, decision tree, support vector machine @xcite to deep learning models. most recently, @xcite improved the bidirectional auto regressive transformer (bart) with a question-driven approach, but it is more well-known for abstractive summarization, which is not discussed in-depth in this paper. ## conclusions this paper presents a systematic study of our extractive approach to the mediqa 2021 -task 2: multi-answer summarization. we combined and optimized several scoring criteria such as tf-idf, lexrank, textrank, query-based, keywords-based and mmr scores. we also developed a strategy called prosper-thy-neighbour to take advantage of adjacent sentences in the answers. the proposed model has a potential performance, being the runner-up of the shared task. our best performance achieved a rouge-2 f 1 is 0.504, comparable to that of the highest-ranked system with 0.507. experiments were also carried out to verify the rationality and impact of model components and the compressed ratio. the results demonstrated the contribution and robustness of all techniques and hyper-parameters. the error analysis was made to analyze the sources of the errors. the evidence pointed to some imperfection of the sentence selecting strategy, the ranking score combination and the question analyzer. our proposed system is extensible in several ways: applying machine learning model, deeply question-analyzing, sentences clustering, etc. we will release our source code on the public repository to support the re-producibility of our work and facilitate other related studies.