Focus-Driven Contrastive Learning for Medical Question Summarization (2022)

Abstract: Automatic medical question summarization can significantly help the system to understand consumer health questions and retrieve correct answers. The Seq2Seq model based on maximum likelihood estimation (MLE) has been applied in this task, which faces two general problems: the model can not capture well question focus and and the traditional MLE strategy lacks the ability to understand sentence-level semantics. To alleviate these problems, we propose a novel question focus-driven contrastive learning framework (QFCL). Specially, we propose an easy and effective approach to generate hard negative samples based on the question focus, and exploit contrastive learning at both encoder and decoder to obtain better sentence level representations. On three medical benchmark datasets, our proposed model achieves new state-of-the-art results, and obtains a performance gain of 5.33, 12.85 and 3.81 points over the baseline BART model on three datasets respectively. Further human judgement and detailed analysis prove that our QFCL model learns better sentence representations with the ability to distinguish different sentence meanings, and generates high-quality summaries by capturing question focus.

## introduction a growing number of health questions are raised by consumers on websites nowadays, which are usually written in natural language and including detailed and peripheral information not related to the answers. summaries of such questions can greatly improve the performance in retrieving relevant answers @xcite . accordingly, the medical question summarization task is defined as summarizing the consumer health questions (chq) into frequently asked questions (faq), which are shorter but remain essential information of the original question to get correct the seq2seq neural models have been widely used in abstractive summarization @xcite @xcite and show promising potentials, and they have also been applied in medical question summarization and achieve current state-of-the-art results. @xcite apply the pointergenerator model for this task. @xcite present a reinforcement learning framework with question-type identification reward and questionfocus recognition reward. @xcite propose a multitask learning method by treating recognizing question entailment as an auxiliary task. in the medical question summarization task, the input question chq is always lengthy and contains redundant information, where some salient medical entities and the semantic focus of question are vital to understand users' intention. but it still remains a challenging task for the existing methods to capture the question focus. as described in the example . 1, the focus "gender dysphoria" is mis-replaced by "weight loss" in the summary generated by the finetuned bart, resulting in a completely different meaning from the original sentence. for the medical question summarization task, the generated question summary is required to semantically close to the reference question. however, in most of current pre-trained models such as bart @xcite , the model adopts maximum likelihood estimation (mle) and mainly focuses on the accuracy of the prediction of masked tokens, but does not guarantee to the semantic similarity or dissimilarity of the whole sentences. to address this issue, some previous works adopt reinforcement learning (rl) in text summarization task @xcite , but rl suffers from the noise gradient estimation problem @xcite , which makes the training process unstable and sensitive to hyper-parameters. to alleviate these problems, we propose a novel question focus-driven contrastive learning (qfcl) framework for medical question summarization, as illustrated in figure 1 . in our model, we introduce a "double anchors" strategy for contrastive learning, by utilizing the sentence representation of chq as an anchor and the generated summary as another anchor, and regarding the golden refer-ence faq as the positive sample. in addition, we present a "focus-driven hard negatives generator" to construct hard negative samples, by replacing the focus phrases with other phrases sharing the same attribute. through contrastive learning, we minimize the distance between chq/generated summary and golden reference, and maximize the distance between chq/generated summary and other negative samples. by using the double anchors, our model is able to extract sentence-level semantic features to alleviate the problem of mle. with the help of hard negatives generator, the model learns to pay more attention to question focus and thus produces high quality summary. we conduct extensive experiments on three medical question summarization datasets: meqsum (ben abacha and demner-fushman, 2019), health-caremagic and icliniq @xcite . our proposed model outperforms previous best results by a wide margin, achieving new state-of-the-art results on all three datasets. compared with the baseline bart, our model brings a relative performance gain of 12.2%, 28.7% and 9.6% on meqsum, cliniq and healthcaremagic respectively. through analysis, we prove that our model significantly gains the power of distinguishing the semantics between generated summaries and negative samples, and our model generates high-quality summaries capturing more question focuses. ## model given an input question chq, which is written by consumers and contains lengthy and complex information, the medical question summarization task aims to automatically generate a question summary that is a frequently asked question (faq), capturing the essential information to help efficiently retrieve correct answers. a more detailed structure of our proposed qfcl model is presented in figure 2 . ## conclusion in this paper, we introduce a novel question focusbased contrastive learning framework qfcl for medical question summarization. in the proposed model, we adopt a "double anchor" strategy, by considering both the input question chq and the generated summary as comparing anchors. and we exploit a "hard negatives generator" to generate hard negative samples based on the question focus. our model significantly improves the performance on three medical question summarization datasets, and achieves new state-of-the-art results. in the future, we would like to find a more effective way to do question focus recognition.