Joint Summarization-Entailment Optimization for Consumer Health Question Understanding (2021)

Abstract: Understanding the intent of medical questions asked by patients, or Consumer Health Questions, is an essential skill for medical Conversational AI systems. We propose a novel data-augmented and simple joint learning approach combining question summarization and Recognizing Question Entailment (RQE) in the medical domain. Our data augmentation approach enables to use just one dataset for joint learning. We show improvements on both tasks across four biomedical datasets in accuracy (+8%), ROUGE-1 (+2.5%) and human evaluation scores. Human evaluation shows joint learning generates faithful and informative summaries. Finally, we release our code, the two question summarization datasets extracted from a large-scale medical dialogue dataset, as well as our augmented datasets.

## introduction in order to answer questions, conversational ai systems have to first understand the intent of questions @xcite . this is particularly important for medical conversational agents @xcite , as consumer health questions (chq) are often long and contain peripheral information not needed to answer the question. approaches to medical question understanding include query relaxation @xcite , question entailment recognition @xcite @xcite and summarization @xcite . we approach the problem of medical question understanding using joint learning of medical question pairs in the two tasks of question summarization and recognizing question entailment (rqe). previous work on combining summarization and entailment uses at least two datasets -one for each task. we start from the observation that, given a pair of questions a and b, where a is the longer question, a entails b if and only if b is a summary of a. using this observation, we propose a data augmentation scheme to use a single dataset for joint learning, instead of two. then, we propose a simple, simultaneous joint learning approach with fully shared model parameters. our findings show that joint learning performs significantly better than single-task training. our joint learning approach brings about an 8% increase in accuracy in the rqe task compared to singletask training, and shows an average of 2.5% increase in rouge-1 f1 scores across three medical question summarization datasets. additionally, we perform human evaluation and find our approach generates more informative question summaries. our results suggest the rqe objective makes our summaries more similar in style to the chq. finally, we release the two consumer health question summarization datasets we extracted from an existing large-scale medical dialogue dataset, our augmented datasets and our code. ## joint learning for consumer health question understanding we consider the joint learning of medical question summarization and recognizing question entail-ment (rqe). in both tasks, a question pair includes a first medical question, written in an informal style by a patient -thus called a consumer health question (chq). the second medical question is shorter, and often written in a formal style by medical experts: it is a frequently asked question (faq). the inspiration for our joint learning scheme stems from the observation that a chq entails an faq, if and only if the faq is a summary of the chq. our data-augmented joint learning approach to consumer health question understanding has two main components. first, we use our equivalence observation to propose a scheme for data augmentation. second, we show our joint learning model architecture and learning objective. ## conclusions we propose a novel data-augmented joint learning approach for the tasks of rqe and question summarization. our data augmentation method extends a dataset such that it can be used for both tasks. our results show improvements in both tasks, across three question summarization datasets (+2.5% in rouge-1 f1) and one rqe dataset (+8% accuracy). we perform a human evaluation for our generated summaries: we find that our approach generates more informative summaries for formally written faqs, and summaries that are faithful to patient writing styles in the more extractive icliniq dataset. finally, we make our datasets, code and training details publicly available.