B andit MTL : Bandit-based Multi-task Learning for Text Classification (2021)

Abstract: Task variance regularization, which can be used to improve the generalization of Multi-task Learning (MTL) models, remains unexplored in multi-task text classification. Accordingly, to fill this gap, this paper investigates how the task might be effectively regularized, and consequently proposes a multi-task learning method based on adversarial multi-armed bandit. The proposed method, named BanditMTL, regularizes the task variance by means of a mirror gradient ascent-descent algorithm. Adopting BanditMTL in the multi-task text classification context is found to achieve state-of-the-art performance. The results of extensive experiments back up our theoretical analysis and validate the superiority of our proposals.

## introduction multi-task learning (mtl), which involves the simultaneous learning of multiple tasks, can achieve better performance than learning each task independently @xcite . it has achieved great success in various applications, ranging from summary quality estimation @xcite ) to text classification @xcite . in the multi-task text classification context, mtl simultaneously learns the tasks by minimizing their empirical losses together; for example, by minimizing the mean of the empirical losses for the included tasks. however, it is common for these tasks to be competing. minimizing the losses of some tasks increases the losses of others, which accordingly increases the task variance (variance between the task-specific loss). large task variance can lead to over-fitting in some tasks and under-fitting in others, which degenerates the generalization performance of an mtl model. to illustrate this issue, *corresponding author. to address the problem caused by uncontrolled task variance, it is necessary to implement task variance regularization, which regularizes the variance between the task-specific losses during training. however, existing deep mtl methods, including both adaptive weighting sum methods @xcite @xcite and multi-objective optimization-based methods @xcite , ignore the task variance. overlooking task variance degenerates an mtl model's generalization ability. to fill this gap and further improve the generalization ability of mtl models, this paper proposes a novel mtl method, dubbed banditmtl, which jointly minimizes the empirical losses and regularizes the task variance. banditmtl is proposed based on linear adversarial multi-armed bandit and implemented with a mirror gradient ascent-descent algorithm. our proposed approach can improve the performance of multi-task text classification. moreover, to verify our theoretical analysis and validate the superiority of banditmtl in the text classification context, we conduct experiments on two classical text classification problems: sentiment analysis (on reviews) and topic classification (on news). the results demonstrate that applying variance regularization can improve the performance of a mtl model; moreover, banditmtl is found to outperform several state-of-the-art multitask text classification methods. ## related works multi-task learning methods jointly minimize taskspecific empirical loss based on multi-objective optimization @xcite @xcite or optimizing the weighted sum of the task-specific loss @xcite @xcite . the multi-objective optimization based mtl can converge to an arbitrary pareto stationary point, the task variance of which is also arbitrary. while the weighted sum methods focus on minimizing the weighted average of the task-specific empirical loss, they do not consider the task variance. to fill the gap in existing methods, this paper proposes to regularize the task variance, which will significantly impact the generalization performance of mtl models. variance-based regularization has been used previously in single-task learning to balance the tradeoff between approximation and estimation error @xcite @xcite . in the single-task learning setting, the goal of variance-based regularization is to regularize the variance between the loss of training samples @xcite . while these variance-based regularization methods can improve the generalization ability of single-task learning models, they do not fit the multi-task learning setting. this paper thus first proposes a novel variancebased regularization method for multi-task learning to improve mtl models' generalization ability by regularizing the between-task loss variance. ## preliminaries consider a multi-task learning problem with t tasks over an input space x and a collection of task spaces {y t } t @xmath0. for each task, we have a set of i.i.d. training samples d @xmath1, y t ) and (x t , y t ) = {x t i , y t i } nt @xmath2, where n t is the number of training samples of task t. in this paper, we focus on the neural network-based multi-task learning setting, in which the tasks are jointly learned by sharing some parameters (hidden layers). let h(•, θ) : {x } t @xmath3, where θ ∈ θ is the vector of the model parameters. θ = (θ sh , θ 1 , ..., θ t ) consists of θ sh (the parameters shared between tasks) and θ t (the task-specific parameters). we denote h t (•, θ sh , θ t ) : @xmath4. the task-specific loss function is denoted as l t (•, •) : y t × y @xmath5, 1] t . the empirical loss of the task t is defined as lt (θ sh , θ t )= 1 nt nt @xmath6, θ sh , θ t ), y t i ). the transpose of the vector/matrix is represented by the superscript , and the logarithms to base e are denoted by log. ## bandit-based multi-task learning to avoid uncontrolled task variance, we need to develop a learning method that regularizes the task variance during training. regularized loss minimization (rlm) is a learning method that jointly minimizes the empirical risk and a regularization function, and is thus a natural choice. while rlm is widely used in single-task learning, it cannot be directly used in multi-task learning to regularize the task variance. in this section, we propose a surrogate for rlm in mtl and accordingly develop a novel mtl method, namely banditmtl. ## conclusion this paper proposes a novel multi-task learning algorithm, dubbed banditmtl. it fills the task variance regularization gap in the field of mtl and achieves state-of-the-art performance in real-world text classification applications. moreover, our proposed banditmtl is model-agnostic; thus, it could potentially be used in other natural language processing applications, such as multi-task named entity recognition. 4 hardware specification and environment our experiments are conducted on a ubuntu 64-bit linux workstation, having 10-core intel xeon silver cpu (2.20 ghz) and nvidia geforce rtx 2080 ti gpus with 11gb graphics memory.