Abstractive Approaches To Multidocument Summarization Of Medical Literature Reviews (2022)

Abstract: Text summarization has been a trending domain of research in NLP in the past few decades. The medical domain is no exception to the same. Medical documents often contain a lot of jargon pertaining to certain domains, and performing an abstractive summarization on the same remains a challenge. This paper presents a summary of the findings that we obtained based on the shared task of Multidocument Summarization for Literature Review (MSLR). We stood fourth in the leaderboards for evaluation on the MSˆ2 and Cochrane datasets. We finetuned pre-trained models such as BART-large, DistilBART and T5-base on both these datasets. These models’ accuracy was later tested with a part of the same dataset using ROUGE scores as the evaluation metrics.

## introduction the last few decades have witnessed a wide range of research applications in the field of natural language processing, especially text summarization. text summarization has been applied in a number of domains including healthcare and medicine. with the tremendous amounts of big data getting generated in the medical industry each day, there is a need realized for effective techniques to summarize the data for further purposes. with the exponential rise in data getting accumulated in hospital databases and medical research labs, the need is increasing correspondingly. text summarization in the healthcare domain has enabled far-reaching benefits for medical professionals. effective summarization techniques help researchers and other individuals to parse long documents effectively, and gain valuable insights in shorter time periods. the history of text summarization in nlp dates back to 1958, when the first paper on text sum- * equal contribution † equal contribution ‡ equal contribution marization was published. since then, its incorporation in healthcare has been widely done. text mining and nlp methods have played an essential role in developing automatic text processing tools @xcite . automatic text summarization, thus proves to be an effective means of gaining valuable information from large documents and reports. in the medical domain, many approaches have been proposed for effective document summarization @xcite . subfields in the biomedical domain where summarization is used include medical literature @xcite , evidencebased medical care @xcite , clinical notes @xcite , and drug information extraction @xcite . summarization approaches are broadly classified as abstractive and extractive. in extractive summarization @xcite , important sentences from the text are directly extracted and put into the summary, whereas for abstractive summarization @xcite , new sentences depicting the summary of the topic are formed. summarization approaches based on the number of documents can be classified as single document and multi-document(more than one documents are searched). in this paper, we present our findings obtained from performing multidocument summarization on the ms^2 @xcite and cochrane @xcite datasets. we finetune a few models on the ms^2 and cochrane datasets, and research upon the best possible hyperparameters that could give us good results. we experimented with the bart-large model @xcite provided by facebook ai on huggingface, the cnn version of the distilbart model @xcite , and t5-base model @xcite ) for text summarization. we preprocessed the inherently messy data provided, and generated summarizations on the same. we have experimented and compared the results of the aformentioned models. the datasets were provided by allenai. we have used the rouge evaluation metric @xcite for comparing summarization accuracies. ## data preparation the ms^2 and cochrane datasets were provided to us in the csv format. the input dataset consisted of the following columns: "reviewid", "pmid", "title" and "abstract", whereas the target dataset consisted of the following columns: "reviewid" and "target". for the ms^2 dataset, additional 'reviews-info' files were included, which consisted of background information associated with the review. however, we didn't utilize them for training purposes. in data preprocessing, the reviews present in the ms^2 and cochrane datasets contain unnecessary delimiters and redundant line breakers, which made it necessary to clean them, before they could be passed to the model. we used simple pandas preprocessing @xcite on the csv files, and cleaned these reviews into simple plain text which could be passed to the model. we mapped each of the documents corresponding to a particular review id, to the corresponding target summary in the target dataset, thus establishing a many-to-one relationship between the abstracts and the targets. we then removed all the other columns which were unnnecessary for summarization ("background", "title", etc). newly formed dataframes, consisting of the source texts (multiple documents merged together for each review id) and the target text (target summaries) were formed and passed for preprocessing. we used the pretrained bart-base tokenizer provided by facebook ai for the bart-large and dis-tilbart models, whereas for the t5-base model training, the t5-base tokenizer was used. both of these tokenizers are available open-source on the huggingface foot_0 model hub. we trained the bart-large and the distilbart-cnn models on the datasets, by instantiating seq2seq models @xcite and arguments provided by simple transformers. we later modified some of the arguments by making the maximum length for each sequence equal to 140. due to limited ram available on the cuda used, we faced memory errors. hence, after each epoch, the weights directory was overwritten for memory availability. maximum sequence length for the tokenized sequences of each input document was set to 512. for t5 (text-to-text transfer transformer), we used the t5-base models @xcite , after providing t5-base tokenization, and trained them with the same aforementioned hyperparameters. all the above mentioned hyperparameters were giving the best possible results, and hence we proceeded with the use of the same. we finetuned the basic configurations specified in the fairseq documentation. 3 4.2 evaluation metrics rouge score @xcite , which stands for recall-oriented understudy for gisting evaluation, was used as the evaluation metric. to cal-3 https://fairseq.readthedocs.io/en/latest/index.html culate the rouge score we used the rouge metric provided by huggingface library foot_2 . we recorded rouge1, rouge2, rougel and rougelsum scores for our summaries. rouge1 measured the overlap of unigram between the candidate and the reference summaries whereas rouge2 compared the bigram similarities between the summaries. rougel and rougelsum measured the longest common subsequence (lcs) @xcite words between predicted and target summaries. all the rouge scores recorded are scored out of 1; where, closer to 1 means more accurate summaries. ## results for the results please refer to ## competition results we obtained high rouge1 and deltaei-macrof1 scores for the multi-document summarization task on the cochrane dataset. we stood 5th when ranked according to rougel metric. for the ms^2 data summarization subtask, we stood 4th when ranked according to the rougel metric. we attained high delta ei-avg scores for the summarization subtask. the scores obtained in the mslr ms^2 and cochrane subtask are given in ## conclusion thus, we implemented multi-document summarization of different clinical studies and their literature surveys in the medical field. we implemented various architectures and analysed their performance. finally, we evaluated the models using rouge metric. we plan to explore other models and tokenization methods to provide more accurate summarizations. also, we plan to train the models on different medical survey datasets for better results in our summarizations.