Towards Summarizing Healthcare Questions in Low-Resource Setting (2022)

Abstract: The current advancement in abstractive document summarization depends to a large extent on a considerable amount of human-annotated datasets. However, the creation of large-scale datasets is often not feasible in closed domains, such as medical and healthcare domains, where human annotation requires domain expertise. This paper presents a novel data selection strategy to generate diverse and semantic questions in a low-resource setting with the aim to summarize healthcare questions. Our method exploits the concept of guided semantic-overlap and diversity-based objective functions to optimally select the informative and diverse set of synthetic samples for data augmentation. Our extensive experiments on benchmark healthcare question summarization datasets demonstrate the effectiveness of our proposed data selection strategy by achieving new state-of-the-art results. Our human evaluation shows that our method generates diverse, fluent, and informative summarized questions.

## introduction online health information search is becoming conventional for more and more consumers every day. a recent survey showed that on average eight million people in the united states seek health-related information on the internet every day 1 . one challenge towards assisting consumers in their healthcare information search is automatic question understanding. generally consumers' questions are overly descriptive and include several peripheral information (as shown in figure -1 ), which are not necessary to answer questions. therefore, in this study we tackle the task of consumer health question understanding by summarizing the question. automatic text summarization is a non-trivial task in natural language processing (nlp) that aims to generate human-readable, concise text con-1 https://pewrsr.ch/3l6m3mv taining salient information of the original document. the recent development in large-scale neural language models @xcite have led to significant performance on several abstractive summarization task. however, their accuracy is partially due to the availability of largescale human-annotated training data. moreover, some domains such as biomedical and medical require domain experts to create high-quality training datasets, which is tedious to create at a large-scale level. a potential solution that has shown effectiveness in other generation and translation tasks is to augment the large-scale synthetically generated samples with a human-annotated training set. however, a limited study focused on data selection strategy in summarization, particularly for abstractive summarization. the majority of the traditional data selection methods are based on word replacement that mainly generates a synthetic sentence by changing one or multiple words with their synonyms @xcite or with a language model predicted words @xcite . however, these methods make minor changes to the original sentence and therefore fall short of generating a diversified sentence. to address this research gap, we present a novel data selection strategy for abstractive consumer health question (chq) summarization task. inspired by the success of the round-trip translation (rtt) @xcite -a process of translating the sentences to a pivot language and then back translating to the original language, we aim to explore the effect of rtt as a data augmentation method in chq summarization. however, not all the data samples obtained from rtt are diverse and can contain redundant information. towards this, we enhance the capability of rtt by devising multiple optimal data selection strategies to select diverse and informative questions, which leads to the significant performance improvement of the chq summarization system. our first data selection strategy frechét question distance (fqd) is based on frechét distance @xcite , which measures the distribution distance between the gold and round-trip translated question. the fqd ensures that questions having near similar or very different distributions should not be selected as additional data to train the summarization system. we propose precision recall question distance (prqd) as our second data selection strategy, which disentangles the question distributions divergence into two components: precision and recall. these two components ensure that the selected additional data brings diversity to the whole training dataset. it is achieved by finding the trade-off between precision and recall of the distributions of the gold and round-trip translated questions. our final data selection strategy question semantic volume (qsv) is based on maximizing the semantic area formed by the points obtained from the semantic representation of the questions. the qsv aims to select the questions which maximize the semantic area leading to the selection of the additional questions which are nonredundant and diverse in nature. we evaluated the effect of the additional data generated using the rtt and our proposed data selection objective measures on benchmark chq summarization dataset and two low-resource open domain datasets. we assess the role of each objective measure in rtt based data selection technique using five different pivot languages. our results show that the rtt-based data selection method helps to improve the performance of the summarization system. we summarize the contribution of the work as follows: neural abstractive summarization: the recent advancement of neural networks models, particularly sequence-to-sequence (seq2seq) @xcite ) models, attention mechanism @xcite , copy mechanism @xcite , coverage mechanism @xcite has propelled the development of efficient abstractive summarization approaches on numerous opendomain datasets. several other methods have utilized the reinforcement learning (rl) @xcite @xcite to guide the models to generate faithful summaries. recently, several studies have investigated the pre-trained language models in the abstractive summarization task @xcite and have achieved the state-ofthe-art performance. besides the supervised models, various other unsupervised approaches have utilized variational autoencoders for automatic summarization @xcite @xcite . consumer health question (chq) summarization: while major progress has been made in open-domain abstractive summarization, chq summarization is a relatively new task. ben abacha and demner-fushman (2019) defined the task of summarizing chq and introduced a benchmark dataset containing 1000 consumer questions summaries. recently, a first shared task was organized by ben abacha et al. ( 2021 ) with the task of summarizing consumer health questions, radiology reports, and multi-document answers. the majority of the works @xcite @xcite used pre-trained language models, ensemble approaches, and knowledge-based methods for the chq summarization task. a few other new methods @xcite have enhanced the capability of transformer model by inducing the latent knowledge. in the literature, several works have explored the concept of rtt in machine translation @xcite , sentence construction @xcite , and style-transfer @xcite . our works advances the existing studies in the consumer health question summarization by proposing an unsupervised framework to optimally select the diverse and information rtt questions, which leads to significant improvement without the need of additional labelled data. ## conclusion this work propose novel data selection strategy based on the concept of round-trip translation for consumer health question summarization. we devised three major data selection objective measures: fqd, prqd and qsv based on the distributional distance to optimally select the diverse and informative samples from the pool of round-trip translated data. extensive experiments show that proposed methods can effectively improve the performance without any additional labelled data. we also achieves new state-of-the-art results on benchmark consumer healthcare question summarization datasets. in future, we plan to explore these objective measures on other resource-scarce tasks. we obtained the first two principal components using the scikit-learn foot_6 library @xcite . the convex hull is computed using the sci-py qhull foot_7 library @xcite . to compute rouge, we use the py-rouge implementation 10 . to update the model parameters, we used adam (kingma and ba, 2015) optimization algorithm with the learning rate of 7e -5 in all the experiments. we also used the cosine annealing @xcite based learning rate decay scheduler, where the learning rate decreases linearly from the initial learning rate in the optimizer to 0. we have checked for the software usage agreements. the licence details of the used software are as follows: prophetnet and mpnet huggingface (apache-2.0 license), google translate (apache-2.0 license), scikit-learn (bsd-3-clause license), scipy (bsd-3-clause license).