Multi-Task Learning using Dynamic Task Weighting for Conversational Question Answering (2020)

Abstract: Conversational Question Answering (ConvQA) is a Conversational Search task in a simplified setting, where an answer must be extracted from a given passage. Neural language models, such as BERT, fine-tuned on large-scale ConvQA datasets such as CoQA and QuAC have been used to address this task. Recently, Multi-Task Learning (MTL) has emerged as a particularly interesting approach for developing ConvQA models, where the objective is to enhance the performance of a primary task by sharing the learned structure across several related auxiliary tasks. However, existing ConvQA models that leverage MTL have not investigated the dynamic adjustment of the relative importance of the different tasks during learning, nor the resulting impact on the performance of the learned models. In this paper, we first study the effectiveness and efficiency of dynamic MTL methods including Evolving Weighting, Uncertainty Weighting, and Loss-Balanced Task Weighting, compared to static MTL methods such as the uniform weighting of tasks. Furthermore, we propose a novel hybrid dynamic method combining Abridged Linear for the main task with a Loss-Balanced Task Weighting (LBTW) for the auxiliary tasks, so as to automatically fine-tune task weighting during learning, ensuring that each of the task’s weights is adjusted by the relative importance of the different tasks. We conduct experiments using QuAC, a large-scale ConvQA dataset. Our results demonstrate the effectiveness of our proposed method, which significantly outperforms both the single-task learning and static task weighting methods with improvements ranging from +2.72% to +3.20% in F1 scores. Finally, our findings show that the performance of using MTL in developing ConvQA model is sensitive to the correct selection of the auxiliary tasks as well as to an adequate balancing of the loss rates of these tasks during training by using LBTW.

## introduction the task of conversational question answering (convqa), which consists in answering a question from a given passage in the form of a dialogue has become a vital task for machine reading comprehension (mrc). in the convqa task, in order to predict an answer, the system needs to extract text spans from a given passage and understand the question based on the given conversational history. recently, the advancement in neural language modeling such as bert @xcite , and the introduction of two large-scale datasets, namely coqa @xcite and quac @xcite have further boosted research in the convqa task. in particular, quac introduces a main task, namely answer span prediction, which consists in answering a question by extracting text spans from a given passage as well as some auxiliary tasks, namely yes/no prediction, follow up prediction and unanswerable prediction. recently, multi-task learning (mtl), which is a way to learn multiple different but related tasks simultaneously, has emerged as a popular solution to tackle all these tasks in a uniform model @xcite . mtl can also be used to leverage the auxiliary tasks to improve the performance of a system on the main task. for example, for the quac dataset, @xcite ) adopted an mtl approach that learns the auxiliary tasks and the main task by sharing the encoder, and showed an improvement in the used convqa model. mtl methods can be categorised into static or dynamic methods. in the static mtl methods, each of the task's weights used to combine the loss functions of the various used tasks during training are unchanged throughout the learning phase, which may divert training resources to unnecessary tasks. in contrast, in the dynamic mtl methods, each of the task's weights are adjusted automatically to balance the loss rate @xcite or to balance the weights across tasks @xcite . however, the implementation of a mtl dynamic method is more complicated and has a lower training efficiency than static methods. most existing @xcite conversational question answering (con-vqa) models that leverage multi-task learning (mtl) use the static method with unchanged tasks' weights during the training epochs. for instance, the recently-proposed history attention mechanism (ham) model @xcite attempted to apply multi-task learning in order to improve the effectiveness of conversational qa. however, the tasks' weights in the model were unchanged during the training state and emphasise the main task. flowdelta @xcite ) is a convqa model that also employed a static mtl method, which sets all tasks' weights equal to one. in the static mtl methods used in ham and flowdelta, all of the tasks' weights have not been adjusted throughout the learning phase. as a result, training resources could be diverted to unnecessary tasks with a negative impact on the performance of the learned models. to improve the effectiveness of multi-task learning for conversational question answering, we propose a novel method, called hybrid task weighting, which focuses on adjusting the tasks' weights by modelling the difference between the tasks' weights, while still prioritising on the main task. our contributions are summarised as follows: (1) we leverage dynamic multi-task learning with bert 1 to effectively address the task of learning answer span prediction with its auxiliary tasks including yes/no prediction, follow up prediction, and unanswerable prediction; (2) to further enhance the performance of multi-task learning, we introduce a hybrid strategy, which automatically fine-tunes the multiple tasks' weights along the learning steps. our method uses abridged linear for the primary task and loss-balanced task weighting for the auxiliary tasks; (3) the proposed hybrid method yields the best performance improvements over the baselines on the quac dataset. ## related work in the following, we discuss related work about multi-task learning, conversational question an1 our preliminary experiments found bert to be more effective than albert or roberta. multi-task learning: mtl is a learning paradigm, which has achieved success in many machine learning applications, including natural language processing @xcite , speech processing @xcite , and computer vision @xcite . for further background on mtl, we refer the readers to the recent review by @xcite . the mtl methods can be classified into static methods or dynamic methods based on their weighting strategy @xcite . in a static method, before training the network, each of the task's weights is set manually, then these weights are fixed throughout the training of the network @xcite . in contrast, the dynamic methods initialise each of the task's weights at the beginning of the training and automatically update the weights during the training process @xcite @xcite . typically, the mtl networks can be classified into either hard or soft parameter sharing networks @xcite . in hard parameter sharing, also known as multi-head, the network is applied by employing separate task-specific output layers on top of a shared encoder. in soft parameter sharing, all parameters are task-specific but all networks have mechanisms to handle the cross-task learning. following @xcite @xcite , we employ a hard parameter sharing mtl approach because this network type reduces the risk of overfitting @xcite . conversational question answering: con-vqa is a machine reading comprehension (mrc) task where questions are formed in conversations. hence, a convqa approach needs to deal with the conversation history to accurately understand and answer the current question. to handle the conversation history, existing works @xcite prepended previous questions and answers to the current question while @xcite ) employed a history selection mechanism. some prior studies have integrated the conversation history into neural language models such as bert: for example, @xcite proposed the positional history answer embedding (poshae) approach, which uses a feature vector to encode the position of the answer in the conversation history in the current question; similarly, @xcite ; @xcite used a context feature to mark historical answers in the passage. the two recent large-scale convqa datasets, quac @xcite and coqa @xcite , have facilitated further research on this task. the differences between these datasets are that the questions in coqa are predominantly factoid in nature, while most questions in quac are nonfactoid. moreover, quac also contains three auxiliary tasks; in contrast, coqa only provides an unanswerable prediction task as an auxiliary task. hence, due to the presence of multiple auxiliary tasks, our mtl study focuses on the quac dataset. multi-task learning for conversational question answering models: recently, existing works @xcite on mtl for convqa have successfully adopted static mtl methods. however, there is still room for improvement since during the learning phase the weights for the auxiliary tasks are unchanged and therefore they not adjusted relative to the importance of the different tasks. we include these mtl methods as baselines in our present work. instead, in this paper, we take advantage of a dynamic method in mtl for convqa. the goal of our proposed model is to improve the effectiveness of the convqa task. as far we know, no prior work has addressed the use of dynamic mtl methods for the convqa task. in our proposed mtl approach, we employ the abridged linear @xcite for the primary task and the loss-balanced task weighting @xcite for the auxiliary tasks, which prioritises the primary task after step t during training by setting the task's weight to one while also automatically fine-tuning the tasks' weights by balancing the loss ratio of the auxiliary tasks. in our model, we employ bert @xcite , which is still a widely used and popular pre-trained model, with customised features following @xcite ). in the following section, we describe in detail our convqa model. ## the bert convqa model we first define the task in section 3.1. an overview of the proposed convqa model is provided in section 3.2. section 3.3 describes how additional features are integrated with the bert encoder. then we explain how predictions are made for the main answer span prediction task as well as the auxiliary tasks in sections 3.4 & 3.5, respectively. ## research questions in this paper, we address two key research questions. firstly, one of our central contributions is the comparison of existing multi-task learning (mtl) strategies, when used in the same conversational question answering (convqa) model both in terms of effectiveness and efficiency. by doing this, we investigate whether there is an actual difference between the static and dynamic loss weighting methods, in guiding the learning process. moreover, to the best of our knowledge, there has been no previous study that investigated dynamic loss weighting for the convqa task on the quac dataset. hence, our first research question is: rq1: what is the most effective and efficient multi-task learning method for convqa? secondly, we investigate the effectiveness of the combination of the auxiliary tasks to improve the performance of the main quac task, namely we posit the following research question: rq2: does applying the proposed mtl conv-qa model using each of the auxiliary tasks result in effectiveness improvements over learning using only the main task? ## experimental setup in this section, we describe the used dataset, quac, and its auxiliary tasks in section 6.1. we present the list of our baselines in section 6.2. we discuss the used evaluation metrics in section 6.3, and the applied hyper-parameter settings in section 6.4. ## experimental results we first report our evaluation results for various mtl methods using our convqa model in section 7.1. our findings for the usefulness of the auxiliary tasks in mtl are detailed in section 7.2. ## conclusions we have proposed a method for conversational question answering, which learns to predict the correct answer span, by applying multi-task learning (mtl). our proposed hybrid mtl method makes use of evolving weighting by abridged linear for learning the main task, while the auxiliary tasks are addressed using loss-balanced task weighting. our experiments on the quac dataset demonstrated that our convqa model learned through multi-task learning by using a hybrid approach has the best effectiveness, yielding statistically significant improvements over the baselines. furthermore, we showed that the use of a combination of the auxiliary tasks resulted in an enhancement to the main task performance compared to single-task learning. for future work, we plan to consider the integration of a question re-writer as well as the use of an attention mechanism for capturing the dialog context.