Learning Health-Bots from Training Data that was Automatically Created using Paraphrase Detection and Expert Knowledge (2020)

Abstract: A key bottleneck for developing dialog models is the lack of adequate training data. Due to privacy issues, dialog data is even scarcer in the health domain. We propose a novel method for creating dialog corpora which we apply to create doctor-patient interaction data. We use this data to learn both a generation and a hybrid classification/retrieval model and find that the generation model consistently outperforms the hybrid model. We show that our data creation method has several advantages. Not only does it allow for the semi-automatic creation of large quantities of training data. It also provides a natural way of guiding learning and a novel method for assessing the quality of human-machine interactions.

## introduction current data-driven dialog models require large quantities of training data. because of privacy issues, the situation is even worse in the health domain where data is particularly scarce. in this work, we propose a novel method for automatically creating the training data necessary to learn a chatbot which can mimick a doctor in doctor-patient interactions. specifically, we combine expert knowledge provided by physicians with automatic paraphrase extraction techniques. we first ask experts (physicians) to specify typical doctor-patient interactions occurring in the context of clinical studies when talking about the four main topics generally discussed in these studies namely, sleep, mood, anxiety, leisure. formally, the specification takes the form of a dialog tree whose nodes are labelled with either an example doctor question or an example patient input. each node in the tree is associated with a unique identifier which can be viewed as a simple form of dialog state. we then enrich this initial dialog data by extracting paraphrases for patient turns from an online forum. this data generation method has several advantages. first, it allows for a straightforward integration of expert knowledge in data generation, model learning and model evaluation as we can use the dialog turn identifiers both to guide learning and to assess the model (by comparing the sequences it follows with the expert defined sequences). more generally, the association of each dialog turn with a dialog turn identifier which reflects its position in the dialog tree and the consistent use of this identifier during data creation, model learning and model evaluation allows for increased interpretability. second, this method helps achieve good coverage as we can ensure that the data does contain all possible dialog paths. this is not the case with wizard-of-oz (woz) and crowdsourcing data collections approaches where the coverage of the possible dialog paths depends on the crowdworker decisions and input. third, by instantiating each dialog with different paraphrases, we can increase linguistic diversity i.e., we can create dialogs that have the same structure but different wording. in sum, our work makes the following contributions. we propose a novel method for creating training data for dialog models. we apply this method to create training data for a bot mimicking doctor-patient interaction in the context of clinical studies. we use the created data to learn a generation and a hybrid classification/retrieval dialog model, we show that the generation model generally outperforms the classification model and we provide a detailed analysis of the models results using automatic metrics, human evaluation and qualitative analysis. ## related work various methods have been proposed to facilitate the creation of training data for dialog. previous work has explored woz experiments in which two humans interact based on some pre-defined scenario and the dialogs resulting from these interactions are collected @xcite or crowdsourcing settings where workers provide continuations to incomplete dialogs @xcite . both approaches are time intensive. crowdsourcing is also expensive while the humanhuman dialogs that are collected by both approaches may be very different from the human-machine interactions that should be learned to support efficient human-machine communication where typically, chat messages are restricted in length. other work has relied on already available dialog data or on question/answer pairs extracted from online forums @xcite @xcite . in the health domain however, such data is extremely scarce and difficult to obtain. when obtainable, it also requires extensive pre-processing due to anonymization constraints. another line of research has been to acquire data through machine-machine simulations @xcite @xcite . in particular, @xcite combines pre-defined dialog outlines with templatebased verbalizations of dialog turns to automatically create a synthetic dialog corpus. our work is similar to @xcite but differs from it in two main ways. first, instead of using templates, we use automatically extracted paraphrases to enrich the initial dialogs. second we experiment with two dialog models to investigate how domain knowledge (in the form of dialog tree positional information) can best be exploited to guide learning and to support error analysis. ## creating dialog corpora to create training data for the dialog bot, we start by collecting typical dialog outlines from an expert. we then extract paraphrases for the patient turns from a health forum and filter out dialog interactions with low coherence. 1 collecting an initial corpus from an expert. studies have shown that the closed questionnaires traditionally used in the context of clinical studies are ineffective in gathering correct and precise information about the patient status because the patients get used to the questions and routinely input the same answers from one interaction to the next. our long-term goal is to develop a human-machine dialog system that would complement standard clinical questionnaires by regularly engaging the patient in a dialog about the questionnaire topics. since our target users are chronic pain patients, it is more important to keep them engaged for a long period rather than getting all information at the first interaction. to create our dialog corpus, we asked a physician to formalize typical patient-doctor interactions occurring in the context of a clinical study in the form of a dialog tree describing which questions need to be asked and for each question, which answers are possible. the interactions cover four domains namely, sleep, mood, anxiety and leisure activities and the dialog tree has 58 nodes. a fragment of the dialog tree created for the sleep domain is shown in figure 1 on the left and an example dialog for the sleep domain in the same figure on the right. we call the data collected from the expert d init . extracting paraphrases. we extract paraphrases for the patient turns provided by the expert from the healthboard 2 forum in several steps as follows. as patient turns are mostly assertive responses to the doctor questions, we start by filtering out questions from the forum data to keep only those utterances which are assertions 3 . to this end, we use a binary stacked bi-lstm classifier trained on the switchboard dataset. we then compare each patient turn in d init together with its context (p , the preceding doctor turn) with the assertive utterances extracted from the forum. for each sequence d + p of contextualised patient turns in d init and each (assertive) utterance u in the forum, we create an s-bert embedding (cf. figure 2 left ). we then retrieve from the forum all utterances u whose cosine similarity with a contextualized patient turn d + p is higher than 0.70. finally, we use maximal marginal relevance (mmr) to select from this pool of candidates a subset of paraphrases which maximises both similarity (the paraphrases should be semantically similar to the input turn) and diversity (the resulting set of paraphrases should be maximally diverse 4 ). we stop selecting sentences as soon as mmr score becomes negative as a negative mmr score indicates that adding more paraphrases will not increase diversity. as illustrated in figure 2 , we apply this paraphrase extraction process not only to create paraphrases for a single turn but also to create paraphrases which summarise 3 consecutive turns. in this way, we can derive compressed versions of the initial dialogs. for instance, we can derive the short dialog in (2) from the longer dialog interaction shown in (1). p1: no d2: what keeps you awake ? p2: i have pain in the legs we refer to the set of paraphrases that summarise three consecutive turns as short and those that summarise a single turn as long. filtering paraphrases. we compute cosine and bertscore on the s-bert embeddings of each pair c, d of context-doctor interactions (where the context is the string concatenation of the three preceding turns) created in the previous step and keep only those pairs for which both scores are higher than the corresponding scores for the corresponding turn in the initial corpus (init). ## health bot models we aim to learn a model which mimicks a physician in the kind of doctor-patient interaction that is typical of clinical studies conversations. as we derive the training data from the dialog tree, each patient turn and each doctor query is associated with a dialog state (a node in that dialog tree). we use this dual information (dialog turn and dialog state) to train and compare two models for response generation: a classification model which, given the last three turns of a doctor-patient interaction, predicts a dialog state and outputs the corresponding doctor query; and a generative sequence-to-sequence model which auto-regressively generates an answer while conditioning on the last three dialog turns. for both models, we use a pre-training and fine-tuning approach similarly to that presented in @xcite . classification model. given a dialog context (3 dialog turns), the classification model predicts a dialog state and outputs the corresponding doctor query. thus, the classification model is a multi-class classifier with 58 target classes, the 58 dialog states defined by the expert dialog tree. we use the pytorch implementation of @xcite )'s pre-training and fine-tuning approach provided by huggingface foot_0 and the default hyper-parameter settings. the input to the model consists of three turns p 1 d 1 p 2 . we concatenate these three turns, prefixing each turn with its dialog state identifier and separating them with a delimiter token. each token is represented by the sum of three embeddings: a word and a position embeddings which are learnt in the pre-training phase; and a turn embedding (learned during fine tuning) indicating whether the token belongs to a patient or to a doctor turn. the input to the model is the sum of all three types -word, position and turn embeddings for each token in the input sequence. the pre-trained model is the generative pre-trained transformer-based (gpt-2) language model trained on the bookscorpus dataset (7,000 books from different genres including adventure, fantasy, and romance). the parameters are initialized to the smallest version of the gpt-2 model weights opensourced by @xcite . we fine-tune the pretrained language model on our data by passing the input turns through the pretrained model and feeding the final transformer block's activation to an added linear output layer followed by a softmax to predict a probability distribution over the target classes. generative model. to generate (rather than retrieve) doctor queries, we use the transfertransfo foot_1 model @xcite which combines a pretrained language model with a transformer-based generation model fine tuned on dialog data using multi-task learning. multi-tasking combines a language modeling loss with a next turn classification loss. for the latter, the model is trained to distinguish a correct continuation from one randomly chosen distractor. as for the classification model, we use the gpt-2 language model pretrained on the bookscorpus. for fine-tuning, we use the same augmented representations as for classification, i.e. each input consists of the three previous turns with a separator and a dialog state identifier between each turn. from this sequence of input tokens, a sequence of input embeddings for the transformer is constructed by summing the word and positional embeddings learned during the pre-training phase and the turn embeddings learned during fine-tuning. multi-task learning is done, as in the transfertransfo model, by jointly optimizing the language modeling and the next-turn classification loss. ## results we compare the classification and the generation models using both automatic metrics and human evaluation. we present various ablation settings to analyse the impact of dialog state information on performance. and we display an example dialog between a human and the generative model in in the oracle setting (oracle), dialog state information is provided for all dialog turns in the input, at training and at test time. this gives an upper bound of how the system would perform given perfect dialog state information. we compare this oracle setting with a standard setting (classif and gen) in which only the dialog state associated with the doctor queries are given. at training time, this is the reference dialog state associated with the doctor query. at test time, it is the dialog state of the doctor query predicted by the model. to analyse the impact of dialog state information on performance, we also execute an ablation study considering models where (i) no dialog state information is given in the input but the model is trained to predict the output dialog state (predict only) and (ii) a model where dialog state information is not used at all (no dialog states). ## conclusion using paraphrase identification techniques and a dialog tree to model expert knowledge about doctorpatient interactions, we proposed a novel method to create training data for dialog models and we used data created using this method to learn health chatbots that cover the main topics standardly used in the questionnaires of clinical studies. we compared two models, a generative and a hybrid classification/retrieval model and we showed that the expert knowledge captured by the dialog tree both helps guide learning and facilitate error analysis. results analysis highlights three main directions for future research. first, additional paraphrase techniques could be explored to create a more balanced dataset. as shown in