M eta W eighting: Learning to Weight Tasks in Multi-Task Learning (2022)

Abstract: Task weighting, which assigns weights on the including tasks during training, significantly matters the performance of Multi-task Learning (MTL); thus, recently, there has been an explosive interest in it. However, existing task weighting methods assign weights only based on the training loss, while ignoring the gap between the training loss and generalization loss. It degenerates MTL’s performance. To address this issue, the present paper proposes a novel task weighting algorithm, which automatically weights the tasks via a learning-to-learn paradigm, referred to as MetaWeighting. Extensive experiments are conducted to validate the superiority of our proposed method in multi-task text classification.

## introduction multi-task learning (mtl) simultaneously learns multiple related tasks and aims to achieve better performance than learning each task independently @xcite . it has achieved great success in various applications; especially, in the text classification context, mtl can significantly outperform single task learning @xcite . in mtl, it is common for the including tasks to be competing. if we cannot properly balance these tasks, some tasks might dominate the training process and hurt the performance of other tasks, a phenomenon known as task imbalance. to address the task imbalance, the most widely used method is task weighting, which adaptively assigns weights on the tasks during training to balance their impacts. various task weighting methods have been proposed and can be used in multi-task text classification, such as @xcite @xcite . however, existing task weighting methods compute the task weights only based on training losses or corresponding gradients. they ignore the gap between the training loss and generalization loss. to illustrate this gap, we report observations of our four-task topic classification experiment in figure 1 . the detailed experimental settings are introduced in the experiment section. figure 1 demonstrates that the training losses and generalization losses (estimated by the test losses) have different magnitudes; moreover, they have different patterns, such as a task might have the largest training loss but the lowest generalization loss among the tasks. this gap causes a mismatch between the task weights and tasks' generalization performance, which reduces effectiveness of the task weighting. to tackle this issue, this paper proposes a novel task weighting method based on a bi-level optimization problem, which aims to find task weights that explicitly optimize the generalization performance. our proposed method computes task weights by solving this bi-level optimization problem and performs in a learning-to-learn manner; thus, dubbed metaweighting. metaweighting can improve the performance of multi-task text classification. to verify our theoretical analysis and validate the superiority of metaweighting, we conduct experiments on two classical text classification problems: sentiment analysis (on reviews) and topic classification (on news). the results demonstrate that metaweighting outperforms several state-of-the-art multi-task text classification methods. ## related works existing task weighting strategies can be divided into two categories: weight adaptation methods and pareto optimization (po)-based methods. the weight adaptation methods adaptively adjust the tasks' weights during training based on pre-defined heuristic, such as uncertainty @xcite , task difficulty prioritization @xcite , gradient normalization @xcite , weight average @xcite and task variance regularization @xcite . these methods only use training losses or their gradients to compute task weights while ignores the gap between the training loss and generalization loss. besides, the po-based methods formulate mtl as a multi-objective optimization problem and aim to find an arbitrary pareto stationary solution @xcite @xcite @xcite . however, in these methods, the learning objectives only involve training losses; thus, they can only achieve pareto stationary points w.r.t training losses. they also ignore the gap between the training loss and generalization loss. moreover, @xcite proposes that the po-based methods can be also regarded as weight adaptation methods for they optimize the weighted sum of training losses as well. overlooking the gap between the training loss and generalization loss would degenerate the performance of mtl. this paper proposes a novel meta learning-based task weighting method to solve this issue. there are some works adopt meta learning-based weighting methods in multilingual learning, e.g., @xcite and @xcite . however, these works cannot solve multi-objective optimization problems. by contrast, this paper proposes a novel method which can solve multi-objective optimization problems. ## preliminaries consider a multi-task learning problem with t tasks over an input space x and a collection of task spaces {y t } t @xmath0. for each task, we have a set of i.i.d. training samples d @xmath1, y i t } n @xmath2. the training samples are sampled from an identical distribution p t . based on the training sets {d t } t @xmath3, we learn an mtl model from a parameterized hypothesis class h, which shares some parameters across tasks. let θ s represent the parameters shared between tasks (task-sharing param-eters), while θ t represent the task-specific parameters. h(•, θ s , θ 1 , ..., θ t ) : @xmath4, while h(•, θ s , θ t ) : @xmath5. the loss function is represented by l(•, •) : y t × y @xmath6, 1] t . for each task, the generalization loss is l t (θ) = e (xt,yt)∼pt l(h(x t , θ s , θ t ), y t ), and the training loss is defined as l tr t (θ, d t ) = 1 |dt| (xt,yt)∈dt l(h(x t , θ s , θ t ), y t ). in this paper, each training set d t is randomly divided into two subsets: support set d s t and query set d q t . correspondingly; moreover, the support loss is defined as l s t (θ, d s t ) = 1 |d s t | (xt,yt)∈d s t l(h(x t , θ s , θ t ), y t ), and the query loss is defined as l q t (θ, d q t ) = 1 |d q t | (xt,yt)∈d q t l(h(x t , θ s , θ t ), y t ). ## theoretical analysis in this section, we study the generalization error bound for mtl; furthermore, we compare the bound w.r.t training loss and the bound w.r.t the query loss. the comparison presents that the query loss is a more accurate estimation of the generalization loss than the training loss. firstly, we derive the generalization error bound w.r.t training loss for mtl. theorem 2. assume we have n training samples for each task. let σ = {{σ t i } n @xmath7. then, ∀δ ∈ [0, 1], for all h(•, θ s , θ foot_0 , ..., θ t ) ∈ h, with probability of at least 1δ: proof. the proof is provided in appendix a. next, we derive the generalization error bound w.r.t query loss for mtl. theorem 3. assume we have m training samples for each task. ∀δ ∈ [0, 1], with probability of at least 1δ, for all h(•, θ s , θ 1 , ..., θ t ) ∈ h, we have proof. the proof is provided in appendix a. comparing the bound ( 14 ) and ( 16 ), we can find that the upper bound for the query loss is tighter than that for the training loss. taking m to be order of n, the query loss is a more accurate estimate of the generalization loss than the training loss by a factor that depends on the rademacher complexity. ## experiments in this section, we perform experimental studies on sentiment analysis to evaluate the performance of our proposed metaweighting and verify our theoretical analysis. ## conclusion this paper presents that the gap between the training loss and the generalization loss, which is overlooked by existing task weighting methods, is nonnegligible; furthermore, to narrow this gap, a novel task weighting method (dubbed metaweighting) is proposed. in metaweighting, multi-task text classification is formulated as a multi-objective bilevel programming problem, and then solved in a learning-to-learn manner. metaweighting automatically learns the task weights without any predefined heuristic and achieves state-of-the-art performance. it has the potential to forge new trends in task weighting research.