Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning (2020)

Abstract: We present a multi-task learning framework to enable the training of one universal incremental dialogue processing model with four tasks of disfluency detection, language modelling, part-of-speech tagging and utterance segmentation in a simple deep recurrent setting. We show that these tasks provide positive inductive biases to each other with optimal contribution of each one relying on the severity of the noise from the task. Our live multi-task model outperforms similar individual tasks, delivers competitive performance and is beneficial for future use in conversational agents in psychiatric treatment.

## introduction conversational technologies offer a remarkable addition to the current approaches for providing mental healthcare. communications with these conversational agents have been found to include discoverable psychological distress signs, such as the rate of filled pauses, speech rate, and various temporal and turn-related characteristics @xcite . in human-human automatic analysis of patient-doctor conversations it has also been found that different types of disfluency can indicate levels of adherence to medication @xcite . markers of disfluency also hold predictive power for identification of cognitive disorders @xcite . such devices are mainly used for processing content which is then analyzed offline. there is much work on detecting disfluencies for offline analysis of transcripts with gold standard utterance segmentation within much of the current effort on disfluency detection on telephone conversations begun by @xcite . however, given that these models do not operate for live systems and rely on rich transcription data including the pre-segmentation of dialogue acts, to facilitate more costeffective study of other data, it would be easier to be able to perform directly and incrementally off the speech signal, or at least from automatic speech recognition (asr) results as they arrive into the system. the incremental model must work with minimum latency as it receives word-by-word data and does so without modifying its initial assumptions and providing its best decisions as soon as possible in line with the principles set out in @xcite . we combine incremental identification of disfluencies with three other tasks that are essential for active conversational models to provide a favorable inductive bias to disfluency detection and to study the way these tasks interact. we explore a multi-task learning (mtl) framework to enable the training of one universal model with four tasks of disfluency detection, language modelling, part-of-speech (pos) tagging and utterance segmentation, which in the data we use is also equivalent to dialogue act segmentation. multi-task learning seeks to improve learning efficiency and predictive power by learning from a shared representation with multiple objectives. we investigate the entire power set of these tasks to investigate the interaction between them. we experiment with two different methods for losses: a naive weighted sum of losses where the weights of loss are uniform, and a loss function based on maximizing the gaussian likelihood with task-dependent uncertainty. we train and test a simple neural model for the different tasks, experimenting with all the combinations of the tasks, different loss functions for each of the tasks, and also experiment with different input representations (words vs. words with word duration). ## related work although significant research has been done on disfluency detection as an individual task, most of this work uses transcripts as texts rather than using data from live speech, with the intention of 'cleaning' such texts of disfluent content for eventual post-processing. they are performed on pre-segmented utterances in the switchboard corpus of telephone conversations @xcite . sequence tagging models with start-inside-outside (bio) style tags have been used in many studies to detect disfluencies. the most common techniques have utilized discriminative models as a classifier like conditional random fields (crfs) @xcite . such methods are insufficient if we intend to measure context from repairs and edit words for disfluency detection, that is not only useful in psychiatric domain but also logical for a dialogue framework that aims to measure a clear understanding of user statements. methods focusing on incremental performance have been uncommon. @xcite used a pipeline of classifiers and language model features in a highly incrementally operating system without looking-ahead. incremental dependency parsing paired with removal of disfluency was also investigated @xcite . two studies have applied recurrent neural networks for live disfluency detection. one approach, using a simple elman recurrent neural network (rnn), investigates incremental detection, with an objective coupling detection efficiency with low latency @xcite . a development of this work proposed a joint task of incremental disfluency detection and utterance segmentation, with the performance of both tasks jointly improving over equivalent systems doing the individual tasks @xcite . there has been research on utterance segmentation as an individual task and also in joint models. @xcite makes use of a range of lexical and acoustic features. @xcite used prosodic and lexical features to implement a dnn combined with a crf classifier for broadcast news speech. @xcite used syntactic ground-truth information, in a rare incremental approach, to predict whether the current word on switchboard is the end of the utterance (dialog act). @xcite used utterance segmentation in a joint framework with dependency parsing. language models have been used as an auxiliary task for disfluency detection, based on the intuition that disfluency occurrences will be indicated and edited from the context to improve the prediction of the next words @xcite , most recently using variants of rnn langauge models @xcite . pos tags have also been used as an input feature in detecting disfluencies, with slight boosts in disfluency detection possible @xcite . in this paper we define a live setting of joint tasks which include disfluency detection, utterance segmentation, language modelling and pos tagging. we present a simple deep learning system after defining the tasks in the next section, which simultaneously detects disfluencies, assigns pos tags, predicts upcoming utterance boundaries and following words from incremental word hypotheses and derived information. john [ likes reparandum + { uh } interregnum loves ] repair mary (1) | a uh flight [ to boston + { uh i mean } to denver ] on friday | thank you | disfluency f e f f f e e e rps-5 rpnsub f f f f utterance segmentation .w--w--w--w--w--w--w--w--w--w--w--w. .w--w. pos tags dt u h n n in n n p u h p rp v b in n n p in n n p v b p rp in the absence of reparandum and repair, the disfluency reduces to an isolated edit term. a marked, lexicalized edit term such as a filled pause ("uh" or "um") or more phrasal terms like "i mean" and "you know" can occur. recognizing these elements and their structure is then the task of disfluency detection. the task of detecting incremental disfluencies adds to the challenge of doing this in real time, wordby-word from left to right. disfluency identification is then cast as the same challenge a human processor faces with a disfluent utterance: only when the interregnum is detected, or perhaps even when the repair onset is encountered, does it become apparent the earlier content is now to be regarded as "to be repaired", i.e. identified as the reparandum. therefore, the task cannot be established as a simple sequence labeling task in which the tags for the reparandum, interregnum and repair phases are allocated left-to-right over words as seen in the above example; in this case, it will require the assumption that "likes" would be repaired, at a point where there is no evidence to make it available. we follow @xcite , using a tag set that encodes the start of the reparandum only at a time when it can be inferred, mainly when the repair begins -the individual disfluency detection task is to tag words as in the top line of tags in fig. 1 as either fluent (f ) an edit term (e), a repair onset word (rps-n for the reparandum starting n words back) and a repair end word of the type repeat (rpnrep), substitution (rpnsub) or delete (rpndel). incremental utterance segmentation utterance segmentation has a strong interdependence with disfluency detection, as standard disfluency detection operates on an utterance or dialogue act level. the two tasks may be more difficult done separately as without utterance segmentation, disfluent restarts and repairs may be predicted at fluent utterance boundaries @xcite , while without disfluency detection the utterance segmentation may predict an utterance boundary at a repair onset point. so we presume that the tasks can be carried out together in a supplementary manner. we characterize incremental utterance segmentation as the real-time word-by-word decision as to whether the current utterance is ending. we are shifting from strictly reactive, silent-signaled approach to prediction. following @xcite , we use four tags to define ranges of acoustic data which are the time spans of forced aligned gold standard words, equivalent to a bies utterance scheme (beginning, inside, end and single) to allow for prediction. the tag set allows information to be captured from the previous context of the word to determine whether this word continues an existing utterance (the prefix) or begins anew (the . prefix), and also allows online prediction of whether the following word will continue the existing utterance (thesuffix) or whether the current word completes the utterance (the . suffix)-see fig. 1 for the incremental utterance segmentation tags around a w symbol for the current word. unlike @xcite we do not employ a joint tag set, instead looking to use multi-task learning as described below to combine these tasks into one learning regime. incremental pos tagging part-of-speech (pos) tags can improve disfluency detection on different settings. we combine pos tagging as an additional task with a similar structure to utterance segmentation and disfluency detection to motivate the model to learn better features for syntactic and semantic structures which can improve the other tasks without requiring additional training data. pos tagging itself could be helped by information about utterance segmentation and disfluency structure, as the parallelism between the reparadum and repair in substitutions as shown in the repeated in n n p sequences in fig. 1 show, allowing better disambiguation of 'to' with this extra information. language modelling the central idea of our approach to language modelling is that the probability of the current word can be more accurately modelled with knowledge from the other dialogue phenomena detected as described above, compared to the standard approach of just using the previous word values, modelling it as part of a joint task @xcite . a secondary assumption is that disfluencies, each of which has a context-conditioned probability, can be represented as word-like occurrences themselves @xcite . utterance segmentation and disfluency detection are optimized only on the basis of accurate labels. while each token in the input has a target tag, some contribute next to nothing to the training process. the disfluency detection and utterance segmentation models can acquire a bias in label distribution by gaining additional information from the majority labels. so we are using another objective that would allow the models to make full use of the dataset. in addition to learning how to predict labels for each word, we also suggest optimizing different parts of the architecture as a language model. the role of predicting the next word would require the model to study more general patterns of semantic and syntactic structure, which can then be reproduced to predict individual words more accurately. this purpose is also generalizable to every task and dataset that does not include additional annotated training details. with a simple adjustment in the model, we introduce a second parallel output layer for each token, improving it to predict the next word. we predict the next word in the sequence only based on the hidden representation from the left-to-right moving lstm. ## model architecture for this task, we use the incremental neural network model the long short-term memory net (lstm) as the baseline architecture. for our individual disfluency detection, utterance segmentation and postagging tasks, the model takes the input and uses an lstm to add a label to each token. for our language modelling task, the model predicts the likelihood of the current word given the previous words. the overall architecture of our model is shown in figure 2 . first, the input tokens are mapped to distributed word embeddings. an lstm is used for constructing context-dependent representations for every word @xcite . the lstm, together with the word embedding from the current step, takes the hidden state from the preceding time step as input and produces a new hidden state: then, the representation is passed through a feedforward layer: the objective function is an objective of regular language modeling e l , which is the negative loglikelihood of the current word in the sequence given the previous words. the network is designed to predict the current label at each token position in terms of disfluency detection, utterance segmentation and pos tagging and the likelihood of the next word in sequence by predicting the likelihood of each word in the vocabulary. the added language modeling objective enables the model to learn the better representations of words, that are then used for other tasks. to boost mtl's performance on the target dataset and to learn several tasks simultaneously, we propose two loss function methods: first, combining multi-objective losses with a naive approach and simply carrying out a weighted linear loss sum for each task: with σ the model's trainable observation noise parameter -capturing how much noise we have in the outputs. e i is one of four indexed to the individual tasks: e lm , e disf , e seg and e p os . ## experimental set-up with various tasks, we assess the proposed architecture in joint models with different number of tasks. we experimented with two types of inputs: word embeddings and the duration of the current word in addition to the word embeddings. the word embedding was initialized with pre-trained vectors available to the public, generated using 50-dimensional embedding trained on google news @xcite . the neural network model has been implemented using tensorflow @xcite . the lstm hidden layers are 200. to minimize computational complexity in these experiments, the language modeling task predicted only the 7000 most commonly used words, with an additional token representing all the other words. results on the development set was also used to find the best model to be evaluated on the test set. we train all models for a maximum of 50 epochs, otherwise stop training if there is no advancement on the best score on the transcript validation set after 7 epochs. the code used in the experiments are publicly available in an online repository. 1 data we use standard switchboard training data (all conversation numbers startning sw2*,sw3 * in the penn treebank iii release: 100k utterances, 650 k words) and use standard heldout data (ptb iii files sw4[5-9] *: 6.4 k utterances, 49 k words) as our validation set. we test standard test data (ptb iii files 4[0-1] *) with partial words and punctuation stripped away from all files. evaluation criteria we calculate f1 accuracy for repair onset detection f rps and for edit term words f e , which includes interregna. for utterance segmentation we also use word-level f1 scores for utterance boundaries (end-of-utterance words) f uttseg . we calculate accuracy acc p os for pos tagging and p erplexity for the language modelling task. we measure latency and also the stability of output over time, which is essential to the live nature of the model. we use the first time to detection (ftd) metric of @xcite for latency: the average distance (in number of words) consumed before the first detection of gold standard repairs from the repairs onset word. for stability, from the evaluation of incremental processors by @xcite , we evaluate the edit overhead (eo) of the output labels -the proportion of the unnecessary editing (insertion and deletion) necessary to attain the final labels produced by the model. ## results the models using the timing outperform those with lexical information only on the utterance segmentation metrics and f e , whilst having lower performance on f rps , language modelling and pos tagging. we get our highest results in all tasks from the model with four joint tasks. edit term detection works very well at 0.922, nearing the state-of-the-art on the switchboard reported at 0.938. our lowest perplexity is 64.3. additional training objective leads to more accurate language modelling in all joint models except when it is trained with disfluency detection alone. the best result for pos tagging is 0.965. it is important to mention that it's difficult to do lm and pos tagging comparisons on this particular dataset. however the high quality of our neural pos tagging and language modelling on un-segmented data and in a live setting are comparable to state-of-the-art approaches, and are useful enough to improve utterance segmentation and disfluency detection, in line with our main goal. ## error analysis different repair types categorizing repetitions as verbatim repeats, substitutes as the other repairs marked with a repair phase, and deletes as those without one, we see in adding other tasks to the task of disfluency detection improves the repetitions detection but we don't observe an improvement in models with more than two tasks. while the performance improvements of joint models in detecting substitutions are small, they are consistent across all models with different number of tasks. we get the best result for substitutions in our single disfluency model at 0.65 and for repeats in our joint model with language modelling at 0.93 when the loss function is naive. for deletes, single disfluency detection model and joint segmentation and disfluency detection models get the best results at 0.34. different repair lengths while our best system still suffers from a vanishing gradient problem to predict repairs with longer reparanda, we can observe in our joint models do not show improvements in detecting longer repairs with reparanda of 5 words or longer. the combined number of instances in the test set with reparanda over 5 words are 59 and our best model predict 26 of them correctly. ## conclusion we have presented a multi-task learning framework to enable the training of one universal incremental model with four tasks of disfluency detection, language modelling, part-of-speech tagging and utterance segmentation. we have observed that these tasks produce favorable inductive biases to each other with utterance segmentation and disfluency detection get the most benefits. we note that the optimal weighting of each task relies heavily on the severity of the noise from the task. we showed that word timing information helps the utterance segmentation and disfluency detection in online setting and adding new tasks with the exception of language modelling does not have a remarkable negative effect on the incremental metrics. the results show that our framework can be suitable for online conversational systems, such as conversational agents in the mental health domain. in future work we intend to analyze the interactions between different task as they occur in real time. monitoring the interaction after each word could help highlight informative moments that contribute more to optimisation of our models. furthermore, we intend to use raw acoustic features as the input for a strongly time-linear model.