ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning
Authors: Junguang Jiang
Baixu Chen
Junwei Pan
Ximei Wang
Dapeng Liu
jie jiang
Mingsheng Long
Venue: NeurIPS 2023 poster (2023)
Keywords: Auxiliary-Task Learning
Negative Transfer
TL;DR: We systematically analyze the cause of negative transfer in auxiliary-task learning and propose a novel method, ForkMerge, to effectively mitigate negative transfer.

Abstract: Auxiliary-Task Learning (ATL) aims to improve the performance of the target task by leveraging the knowledge obtained from related tasks. Occasionally, learning multiple tasks simultaneously results in lower accuracy than learning only the target task, which is known as negative transfer. This problem is often attributed to the gradient conflicts among tasks, and is frequently tackled by coordinating the task gradients in previous works. However, these optimization-based methods largely overlook the auxiliary-target generalization capability. To better understand the root cause of negative transfer, we experimentally investigate it from both optimization and generalization perspectives. Based on our findings, we introduce ForkMerge, a novel approach that periodically forks the model into multiple branches, automatically searches the varying task weights by minimizing target validation errors, and dynamically merges all branches to filter out detrimental task-parameter updates. On a series of auxiliary-task learning benchmarks, ForkMerge outperforms existing methods and effectively mitigates negative transfer.

Fore Mitigating Negative Transfer in Auxiliary -aask Learning
Junguang Jiang, Baixu Chen," Junwei Pan Ximei Wang*, Dapeng Liu Jie Jiang
Mingsheng Long
School of Software, BNRist, Tsinghua University, China
Tencent Inc, China
{jig20, cbx22) cbx22))maaall. @maila. wta.e...maaaananaale.ee.. tainghua...... edu cn, fjonaspan, messixmwang, bea.iiiu.zeuudteeenttcccm
mingshengOtaingguu.. edu.cn
Abstract
Auxiliary- -aaikr Learning (ATL) aims to improve the performance of the target task by leveraging the knowledge obtained from related tasks. Occasionally, learning multiple tasks simultaneously results in lower accuracy than learning only the target task, which is known as negative transfer. This problem often attributed the gradient conflicts among tasks, and is frequently tackled by coordinating the task gradients in previous works. However, these optimization-basee methods largely overlook the auxiliary-target generalization capability. To better understand the root cause of negative transfer, we experimentally investigate from both optimization and generalization perspectives. Based our findings, introduce ForkMerge, novel approach that periodically forks the model into multiple branches, automati- cally searches the varying task weights by minimizing target validation errors, and dynamically merges all branches to filter out detrimental task sask-paraetter updates. On series of auxiliary- -tak learning benchmarks, ForkMerge outperforms existing methods and effectively mitigates negative transfer.
1 Introduction
Deep neural networks have achieved remarkable success in various machine learning applications. such as computer vision [23] (2], natural language processing [62] [1 [7], and recommendation
systems [46]. However, one major challenge in training deep neural networks is the scarcity of labeled data. In recent years, Muxiliary-Task Learning (ATL) has emerged as promising technique
to address this challenge 667 39 43. ATL improves the generalization of target tasks by leveraging
the useful signals provided by some related auxiliary tasks. For instance, larger-scale tasks, such as user click prediction, can be utilized as auxiliary tasks to improve the performance of smaller-seale target tasks, such as user conversion prediction in recommendation [47] (3). Self-supervised tasks unlabeled data can serve as auxiliary tasks to improve the performance the target task in computer
vision and natural language processing, without requiring additional labeled data 33 69
However, in practice, learning multiple tasks simultaneously sometimes leads to performance degra- dation compared to learning only the target task, a phenomenon known negative transfer [84] 75
Even in large language models, negative transfer problems may still exist. For example, RLHF key component of ChatGPT [57]. achieves negative effects on nearly half of the multiple-choice question tasks when post-training GPT-4 (58). There has been a significant amount of methods
proposed to mitigate negative transfer in ATL 7 13 39). Notable previous studies attribute negative transfer to the optimization difficulty, especially the gradient conflicts between different tasks, and propose to mitigate negative transfer by reducing interference between task gradients
"Equal contribution.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Other works focus on selecting the most relevant auxiliary tasks and reducing negative transfer by avoiding task groups with severe task conflicts [7 However, despite the significant efforts to address negative transfer, its underlying causes are still not fully understood.
79 15.
In this regard, experimentally analyze potential causes of negative transfer in ATL from the perspectives of optimization and generalization. From an optimization view, our experiments suggest that gradient conflicts do not necessarily lead to negative transfer. For example, weight decay, special auxiliary task, can conflict with the target task in gradients but still be beneficial to the target performance. From a generalization view, we observe that negative transfer is more likely to occur when the distribution shift between the multi-task training data and target test data is enlarged.
Based above findings, present a new approach named ForkMerge. Since cannot know which task distribution combination leads to better generalization in advance, and training models for each possible distribution is prohibitively expensive, we transform the problem of combining task distributions into that of combining model hypotheses. Specifically, we fork the model into multiple branches and optimize the parameters of different branches on diverse data distributions by varying the task weights. Then at regular intervals, we merge and synchronize the parameters of each branch to approach the optimal model hypothesis. In this way, we will filter out harmful parameter updates to mitigate negative transfer and keep desirable parameter updates to promote positive transfer.
The contributions of this work are summarized follows: (1) We systematically identify the problem and analyze the causes of negative transfer in ATL. (2) We propose ForkMerge, a novel approach to mitigate negative transfer and boost the performance of ATL. (3) We conduct extensive experiments and validate that ForkMerge outperforms previous methods on a series of ATL benchmarks.
2 Related Work
2.1 Auxiliary-Tash Learning
Auxiliary-Tast -ask Learning (ATL) enhances a model's performance on a target task by utilizing knowl- edge from related auxiliary tasks. The two main challenges in ATL are selecting appropriate auxiliary tasks and optimizing them jointly with the target task. To find the proper auxiliary tasks for ATL, recent studies have explored task relationships by grouping positively related tasks together and assigning unrelated tasks to different groups to avoid task interference 71 70). Once auxiliary tasks are determined, most ATL methods create unified loss by linearly combining the target and auxiliary losses. However, choosing task weights is challenging due to the exponential increase in search space with the number of tasks, and fixing the weight of each task loss can lead to negative transfer [32]. Recent studies propose various methods to automatically choose task weights, such as using one-step multi-step gradient similarity [15] 39 91, minimizing representation-bue based task distance [2] or gradient gap [67], employing a parametric cascade auxiliary network [54], or from the perspective of bargaining game (66). However, these methods mainly address the optimization difficulty after introducing auxiliary tasks and may overlook the generalization problem.
Recently, AANG [10] formulates a novel searching space of auxiliary tasks and adopts the meta- learning technique, which prioritizes target task generalization, to learn single-step task weightings. This parallel finding highlights the importance of the target task generalization and we further introduce the multi-step task weightings to reduce the estimation uncertainty. Another parallel method, ColD Fusion [2], explores collaborative multitask learning and proposes to fuse each contributor's 's parameter to construct a shared model. In this paper. we further take into account the diversity of tasks and the intricacies of task relationships and derive method for combining model parameters from the weights of task combinations.
2.2 Multi-Task Learning
Different from ATL, Multi-Task Learning (MTL) aims to improve the performance of all tasks by learning multiple objectives from a shared representation. To facilitate information sharing and minimize task conflict, many multi-task architectures have been designed, including hard- pard-parameter
sharing (30, 22 24 and soft-parameter sharing [5]] 64
16
46
44
48 72 Another line of work
aims to optimize strategies to reduce task conflict. Methods such as loss balancing and gradient balancing propose to find suitable task weighting through various criteria, such as task uncertainty
(28), task loss magnitudes [44), gradient norm
[5, and gradient directions [79] 6 40 41 25| 55

---

Although MTL methods can be directly used to jointly train auxiliary and target tasks, the asymmetric task relationships in ATL are usually not taken into account in MTL.
2.3 Negative Transfer
Negative Transfer (NT) is a widely existing phenomenon in machine learning, where transferring knowledge from the source data model can have negative impact on the target learner [63 60
27.. To mitigate negative transfer, domain adaptation methods design importance sampling or instance weighting strategies to prioritize related source data 77 83]. Fine-tuning methods filter out detrimental pre-trained knowledge by suppressing untransferable spectral components in the representation 4.. MTL methods use gradient surgery task weighting to reduce the gradient
conflicts across tasks 779 76 25 42]. Different from previous work, we propose to dynamically
filter out harmful parameter updates in the training process mitigate negative transfer. Besides, provide an in-depth experimental analysis of the causes of negative transfer in ATI,, which is rare in this field yet will be helpful for future research.
3 Negative Transfer Analysis
Problem and Notation. In this section, we assume that both the target task Tig: and the auxiliary task Taux are given. Then the objective is to find model parameters & that achieve higher performance on the target task by joint training with the auxiliary task,
min Lig( + Lox
(1)
where is the training loss, and is the relative weighting task and the target task. Our final objective is maxe [P(0), where P is the relative performance measure for the target task Ti.. such as the accuracy in classification. Next we define the Transfer
between the auxiliary
Gain to measure the impact of Tamx on Tigl-
Definition 3.1 (Transfer Gain, TG). Denote the model obtained by some ATL algorithm A as A (Tgi, Taux, A) and the model obtained by single-task learning on target task as A (Tig).. Let P be the performance measure the target task Tig Then the algorithm A can be evaluated by
TG(A, A) = P(B Taux; A)) ((0(TT))).
(2)
Going beyond previous work on Negative Transfer (NT) 75 in ATL into two types.
84
we further divide negative transfer
Definition 3.2 (Weak Negative Transfer, WNT). For some ATL algorithm A with weighting hyper-parameter X, weak negative transfer occurs if TG, A) < 0. Definition 3.3 (Strong Negative Transfer, SNT). For some ATL algorithm A, strong negative transfer occurs if maxx>0 TGG( < O.
Figure I Iillustrates the difference between weak negative transfer and strong negative transfer. The most essential difference is that we might be able to avoid weak negative transfer by selecting a proper weighting hyper-parametts parameter X, yet we cannot avoid strong negative transfer this way.
TG
WNT
Next, we will analyze negative transfer in ATL from two different perspectives: optimization and generalization. We conduct our analysis on a multi-domain image recognition dataset DomainNet [6]] with 18 (23] pre-trained ImageNet. Specifically, we use task Painting and Quickdraw in DomainNet as target tasks respectively to showcase weak negative transfer and strong negative transfer, and mix all other tasks in DomainNet as auxiliary tasks. We will elaborate on the DomainNet dataset in Appendix C.3 and provide the detailed experiment design in Appendix B
SNT
Figure 1: Weak Negative Transfer (WNT) vs. Strong Negative Transfer (SNT).
3.1 Effect of Gradient Conflicts
It is widely believed that gradient conflicts between different tasks will lead to optimization difficulties (79 40], which in turn lead to negative transfer. The degree of gradient conflict is usually measured
by the Gradient Cosine Similarity [79 76 15].
3

---

Definition 3.4 (Gradient Cosine Similarity, GCS), Denote dij as the angle between two task gradients g; and gj. then we define the gradient cosine similarity as cos diy and the gradients as conflicting when cos cos 0ij < O.
In Figure 2 we plot the correlation curve between gradient cosine similarity and transfer gain. Somewhat counterintuitively, we observe that negative transfer and gradient conflicts strongly correlated, and negative transfer might be severer when the task gradients are highly consistent.
Finding 1. Negative transfer is not necessarily caused by gradient conflicts and gradient conflicts do not necessarily lead negative transfer.
It seems contradictory to the previous work 79 15 considers the optimization convergence during training, while our experiments we further consider the generalization during evaluation (transfer gain is estimated on the validation set). Although the
and the reason is that previous work mainly
conflicting gradient of the auxiliary task will increase the training loss of the target task and slow down its convergence speed [37), it may also play role similar to regularization (32, reducing the
over-fitting of the target task, thereby reducing its generalization error. To confirm our hypothesis,
repeat the above experiments with the auxiliary task replaced by L2 regularization and observe similar phenomenon as shown in Figure (d)), which indicates that the gradient conflict ATL is not necessarily harmful, it may serve proper regularization.
Figure 2 also indicates that the weighting hyper-parameter parameter in ATL has large impact on negative transfer. A proper À not only reduces negative transfer but also promotes positive transfer.
64
B
-1.
-15
P
(6) Thrgeee Q Andiaty:Domiiike
(6 Thrgee Auailary: Eegidaibation
(9) Toreet Andilirr Regubrtatioo
Aucilary Dorroshdee
Figure 2: The effect of gradient conflicts. The correlation curve between Transfer Gain (TG) and Gradient Cosine Similarity (GCS) under different X. For fair comparison, each data point starts from the same model parameters in the middle of the training process and updates with one-step multi-task gradient descent. P and Q are short for Painting and Quickdraw tasks, respectively.
3.2 Effect of Distribution Shift
Next, we will analyze negative transfer from the perspective of generalization. We notice that adjusting will change the data distribution that the model is fitting. For instance, when the model only fits the data distribution of the target task. and when À = 1. the model will fit the interpolated distribution of the target and auxiliary tasks. Formally, given the target distribution Tig. and the auxiliary distribution Taux, the interpolated distribution of the target and auxiliary task is Timerr
Tinter T-TT + ZT Z Bernoulli
where À is the weightiig hwper--aramene parameter. Figure 3 a) quantitatively visualizes the distribution shift under different À using t-SNE [74].
To quantitatively measure the distribution shift in ATL, we introduce the following definitions.
Following the notations of [53] we consider multiclass classification with hypothesis space F of scoring functions :
where (x, indicates the confidence of predicting x as y.
Definition 3.5 (Confidence Score Discrepancy, CSD). Given scoring function hypothesis F. denote the optimal hypothesis on distribution D as fū. then confidence score discrepancy between

---

distribution D and D' induced by F is defined by
(D _ RDDD max (b(,,)) yey
Confidence score discrepancy between training and test data indicates how unconfident the model is on the test data, which is expected to increase when the data shift enlarges 59 50
Trunning Dara for Brgettuucloyy
A=
X=1/16
A=14
X=1
(WNT)
(SNT)
Figure 3: The effect of distribution shift. (a) Visualization of training distribution and test distribution under different/ x. (b) For weak negative transfer tasks, as À increases, Confidence Score Discrepancy (CSD) first drops and then rises and Transfer Gain (TG) is first positive and then negative. For strong negative transfer tasks, CSD increases monotonically and TG remains negative.
Figure 3 b) indicates the correlation between confidence score discrepancy and transfer gain. For weak negative transfer tasks, when À increases first, the introduced auxiliary tasks will shift the training distribution towards the test distribution, thus decreasing the confidence score discrepancy between training and test data and improving the generalization of the target task. However, when continues to increase, the distribution shift gradually increases, finally resulting in negative transfer. For strong negative transfer tasks, there is a large gap between the distribution of the introduced auxiliary tasks and that of the target task. Thus, increasing À always enlarges confidence score discrepancy and always leads to negative transfer. In summary.
Finding 2. Negative transfer is likely occur if the introduced auxiliary task enlarges the distribution shift between training and test data for the target task.
Methods
In Section 4.1 based above analysis, will introduce how mitigate negative transfer when the auxiliary task is determined. Then in Section[4.2. we will further discuss how to use the proposed method to select appropriate auxiliary tasks and optimize them_jointl with the target task simultaneously.
4.1 ForkMerge
In this section, we assume that the auxiliary task Taux is given. When updating the parameters 0 with Equation ( training step f,, we have
=
(0) + Agaux (0:)),
(5)
where 1 is the learning rate, Sig and gaux are the gradients calculated from Lags and Laux respectively. Section 3.1 reveals that the gradient conflict between gigi and gaux does not necessarily lead to negative transfer as long as À is carefully tuned and Section 3.2) shows that negative transfer is related to generalization. Thus we propose dynamically adjust À according to the target validation performance P to mitigate negative transfer:
max P(f+41) P(. (0)) + Agaux >g)xxx(())).
5

---

Inical Parameter
Fork
Algorithm ForkMerge Training Pipeline.
initial model parameter Po. total iterations T. interval At Ensure: linal model parameterr 07. lask relevance A
Require:
%
Opaimizzzion
fork model into 2 copies (0 jo-
(0)e()
mijn Coge(0) (o) de.
for to do 8, +-8
de
b initialization
while
T do to
independent update
Merge
for 1' t At
97, 11
f9 (( () + Run (0")
Orose
end for
end for
Final Parameter
X + arg maxa P((1 - 1)0 A01+Ar)
D search validation
Figure 4: ForkMerge training pipeline. The model parameters will be forked into branches, the target task loss and the other jointly trained, and be merged at regular inter- vals of At steps.
o+ + (1 x-)o ter A
merge parameters
optimized with
for t 100100 do
0º iiti 07-ac end for
synchronize parameters
At
end while
Equation (6) is bi-level optimization problem. One common approach is first approximate P with the loss of batch of data on the validation set, and then use first-order approximation to solve [8 43 However, these approximations within a single step of gradient descent introduce large noise to the estimation of X and also increase the risk of over-fitting the validation set. To tackle these issues, we first rewrite Equation (6) equally as
maxp((1 A)0++1(00 (1)+
(7)
where (0) = 8, Big (6,) and (1) 0, (ggg (0,) gaix gm,(())) (0.)). The proof is in Appendix A. Note that we assume the optimal X satisfies 0 S S 1, which can be guaranteed by increasing
the scale Laux when necessary. Yet an accurate estimation of performance p in Equation (7) is still computationally expensive and prone over-fitting, thus we extend the one gradient step to At steps,
X = arg maxp A)0 + 10+ar(1)).
8
A
Algorithm. As shown in Figure 4 and Algorithm I the initial model parameters 0, at training step will first be forked into two branches. The first one will be optimized only with the target task loss Ligi for At iterations to obtain (0)):(0), while the other will be jointly trained for At iterations to obtain (1)):(1). Then we will search the optimal X" that linearly combines the above two sets of parameters to maximize the validation performance P. When weak negative transfer occurs in the joint training branch, we can select proper X between 0 and And when strong negative transfer occurs, we can simply set X* to O. Finally, the newly merged parameter 0ita/ = (1 - x=)0:+s: + + +*0++Ae(1)) will join a new round, being forked into two branches again and repeating the optimization process for Af times.
Discussion. Compared to grid searching A, which is widely used in practice, ForkMerge can dynamically transfer knowledge from auxiliary tasks to the target task during training with varying X In terms of computation cost, ForkMerge has a lower complexity as it only requires training 2 branches while grid searching has cost proportional to the number of parametrr to be searched.
4.2 ForkMerge for Task Selection Simultaneously
When multiple auxiliary tasks are available, we can simply mix all the auxiliary tasks together to form single auxiliary task. This simple strategy actually works well in most scenarios (see Section 5.2) and is computationally cheap. However, when further increasing the performance is desired, we can also dynamically select the optimal weighting for each auxiliary task. Formally, the objective

---

when optimizing the model for the target task To with multiple auxiliary tasks {Th ] K k== is
K
min To (o( + Σ (x,
(9)
where Ex= du S 1 and Vk, Au 2 O. Using gradient descent to update O, at training step I, we have
(A) = 0/
(Oe),
(10)
k=
where λo = 1. Given K task-weighting vectors { =k= that satisfies w = 1Bi k or i = i.c. the k-th and 0-th dimensions of w are I and the rest are O, and vector A that satisfies
As
k 0,
A
(11)
Ake
k + 0,
then optimizing A in Equation 10 is equivalent to
K
A = arg maxP
(x0++1(w"))
(12)
R==
In Equation (12). the initial model parameters are forked into K + branches, where one branch is optimized with the target task, and the other branches are jointly optimized with one auxiliary task and the target task. Then we find the optimal A' that linearly combines the K + 1 sets of parameters to maximize the validation performance (see proof of Equation (12) and the detailed algorithm in Appendix A.2). The training computational complexity of Equation 12 is ((K). which is much lower than the exponential complexity of grid searching, but still quite large. Inspired by the early-stop approximation used task grouping methods 71, we can prune the forking branches with A₁ = 0 (strong negative transfer) and only keep the branches with the largest K' < K values in after the early merge step. In this way, those useless branches with irrelevant auxiliary tasks can be stopped early. Additionally, we introduce greedy search strategy in Algorithm to further reduce the computation complexity when grid searching all possible values of A.
Lastly, we introduce a general form of ForkMerge. Assuming B candidate branches with task- weighting vectors vd (b=1, 1,..., B), the goal is to optimize A"
A arg maxp
Σ (slr-ar(e()).
(13)
From a generalization view, the mixture distributions constructed by different v lead to diverse data shifts from the target distribution, yet cannot predict which leads to better generalization. Thus, we transform the problem of mixture distribution into that of mixture hypothesis (49 and the models
trained on different distributions are combined dynamically via A to approach the optimal parameters. Here, Equation 12 is a particular case by substituting B K and v 1l = b 1 or Oj. By comparison, Equation 13 allows us to introduce human prior into ForkMerge by constructing more efficient branches, and also provides possibilities for combining ForkMerge with previous task grouping methods [81 [ 17 The detailed algorithm of Equation] 13 can be found in Algorithm
5 Experiments
We evaluate the effectiveness of ForkMerge under various settings, including multi-task learning, multi-domain learning, and semi-supervised learning. First, in Section we illustrate the prevalence of negative transfer and explain how ForkMerge can mitigate this problem. Section We examine whether ForkMerge can mitigate negative transfer when joint training the auxiliary and target tasks, and compare with other methods. In Section 5.3 we further use ForkMerge for task selection simultaneously. Experiment details can be found in Appendix C We will provide additional analysis and comparison experiments in Appendix [D] The codebase for both our method and the compared methods will be available at https: /tt::/gitiii github....... com/ thhml/ForkMezee

---

5.1 Motivation Experiment
Negative Transfer widespread across different tasks. In Figure (a), we visualize the transfer gains between 30 task pairs on DomainNet, where the auxiliary and target tasks are equally weighted, and we observe that negative transfer is common in such case (23 of 30 combinations lead to negative transfer). Besides, as mentioned in Definition and 333 whether negative transfer occurs is related to specific ATI. algorithm, in Figure (b), we observe that negative transfer in all 30 combinations can be successfully avoided when we use ForkMerge algorithm. This observation further indicates the limitation task grouping methods [71 [7], since they use Equal Weight between tasks and may
discard some useful auxiliary tasks.
Mixture of hypotheses is an approximation of mixture of distribution. Figure uues the ternary heatmaps to visualize the linear combination of set of three models optimized with different task weightings for 25 iterations, including a single-task model and two multi-task models. Similar to mixing distributions for weak negative transfer task Painting (see Figure 3). the transfer gain when mixing models Painting and Painting+Red first increases and then decreases. Also similar to mixing distributions for strong negative transfer task Quickdraw, the transfer gain when mixing models Quickdraw and Quickdraw+Real decreases monotonically. Besides, Figure 6 also indicates a good property of deep models: the loss surfaces of over-parametert quite well-behaved and smooth after convergence, which has also been mentioned by previous works and provides an intuitive explanation of the merge step in ForkMerge.
deep neural networks are
20 35
1.5%%
et ge
04
(WNTT)
12.%% Juh
100% 18.5%
(30 Liuual Weight
(ii PookMtzggg
Figure 6: Ternary heatmap for mixture of model hypotheses. Each triangle vertex rep- resents an optimized model, e.g., P+R is the model jointly optimized with Painting and Real tasks. Each point inside the triangle corre- sponds mixture of model hypotheses and its heat value measures the Transfer Gain (TG).
Figure 5: Negative Transfer on DomainNet. The rows of each matrix represent auxiliary tasks, and the columns represent target tasks. The blue and red cells correspond to negative and positive trans- fer gain. Deeper colors indicate stronger impacts.
5.2 Use ForkMerge for Joint Optimization
First, we use ForkMerge only for joint training of the target and auxiliary tasks. When datasets contain multiple tasks, we will mix all tasks together to form a single auxiliary task for ForkMerge. Yet for the compared methods, a distinction is still made between different tasks for better performance.
Specifically, we compare ForkMerge with: (1) Single Task Learning (STL): (2) EW, which assigns equal weight all tasks; (3) GCS (15). an ATL approach using gradient similarity between target and auxiliary tasks; (4) OL A__ 339 an ATL approach adjusting the loss weight based on gradient inner product; (5) ARMI 667 ATL approach adjusting the loss weight based on gradient difference; (6) Auto-A [43] an ATL method that estimates loss weight through finite-difference approximation [18]; (7) Post-train, an ATL method that pre-trains the model on all tasks and then fine-tunes it for each task separately. (8) UW (28). which adjusts weights based on task uncertainty; (9) DWA [44), which adjusts weights based on loss change; (10) MGDA (65.. which computes a convex combination of gradients with a minimum norm to balance tasks; (11) GradNorm [5). which rescales the gradient norms of different tasks to the same range; (12) PCGrad (79], which eliminates conflicting gradient components; (13) IMTL [1], which uses an update direction with equal projections on task gradients; (14) CAGrad 40 which optimizes for the average loss and minimum decrease rate across tasks; (15) NashMTL [55]. which combines the gradients using the Nash bargaining solution. Since different tasks have varying evaluation metrics, we will report the average per-task performance improvement for each method using Am. as defined in Appendix C
8

---

Auxiliary-Task Scene Understanding. We evaluate on the widely-used multi task scene understand- ing dataset, NYUv2 [68. which contains 3 tasks: 13-class semantic segmentation, depth estimation, and surface normal prediction. Following (55. we use 636, 159 and 654 images for training, val- idation, and test. Our implementation is based on LibMTI. 38 and MTAN [4 The results are presented in Table [ depth benefit from ATL and only normal task gets worse. In such cases, our method still achieves significant improvement on all tasks. We also find that Post-train serves as strong baseline in most of our ATL experiments. Its drawback is that it fails to consider the task relationship in the pre-training phase, and suffers from catastrophic forgetting during the fine- finettuiing process.
Negative transfer is not severe on this dataset, where both segmentation and
image recognition dataset, DomainNet [6]], which contains 6 diverse visual domains and approximately 0.6 million images distributed among 345 categories, where the task difference is reflected in the marginal distribution. Our implementation is based on TLlib [26]- As the original DomainNet does not provide a separate validation set, we randomly split 50% data from the test set as the validation set. The results are presented in T DomainNet contains both positive transfer tasks (Clipart), weak negative transfer tasks (Infograph, Painting, Real, Sketch), and strong negative transfer tasks (Quickdraw). When negative transfer occurs, previous ATL methods lead to severe performance degradation, while our method can automatically avoid strong negative transfer and improve the performance over STL in other cases.
Auxiliary-Domaa Image Revognition. Further, we evaluate on the widely-used
Table 1: Performance on NYUv2 dataset.
Table 2: Performance on DomainNet dataset.
Methods
Segmentation
Depth
Normal
A
Methods
C
Q
R
Avg
A 1
mloUt
Aeet
Abs Err,
Ere
Mean,
STL
77.6
41.4
71.8
73,0
84.6
70.2
69.8
51.42
74.14
17.37
22.82
78.0
50.
63.0
-9.62%
EW
52.13
24.14
0.30%
UW
79.1
35.8
68.2
50.5
77.9
67.0
63.1
-9.58%
UW
52.51
74.72
39.15
16.56
23.99
0.63%
DWA
78.3
38.2
67.8
51.4
77.3
67.2
63.4
-9.15%
DWA
52.10
74.45
39.26
16.57
24.12
0.07%
MGDA
78.1
69.2
51.0
67.3
63.8
-8.07%
MGDA
50.79
73.81
39.19
16.25
23.14
1.44%
GradNona
78.4
09.4
52.9
79.0
67.7
64.4
-7.68%
GradNoom
52.25
16.37
23.86
PCGmd
78.3
38.0
08.2
50.4
77.4
61.3
63.3
-9.52%
PCGmd
51.77
74.72
38.91
16.36
24.31
0.22%
IMTI.
79.4
38.6
68.6
53.7
79.3
67.6
64.5
-7.55%
IMTL
52.24
74,73
39.
15.92
23.25
2.10%
CAGiad
79.1
69.4
53.6
-7.35%
CAGod
52.04
74.25
16.30
1,41%
NashMILL
71.8
39.5
73.5
56.9
-18.5%
NadMIL
51.73
74.10
16.50
23.21
1.11%
67.6
56.6
62.3
-11.0%
52.67
16.64
24.10
OL_AUXX
68.2
33.5
65.3
54.1
76.3
60.9
59.7
-14.8%
OL AUX
52.07
74.28
39.32
16.30
23.98
0.17%
ARML
75.6
36.8
67.8
52.4
77.6
64.2
62.4
-10.7%
ARML
52.73
74.35
39,61
16.65
23.89
0.37%
Auto X
76.3
37.8
70.2
56.3
79.7
67.1
61.9
-7.18%
Auto-A A
$2.40
74.62
39.25
16:25
23.38
1.17%
Post-train
78.7
72.7
73,0
94.7
71.2
70.4
+1.0%%
Pod--tnin
52.08
1.49%
ForkMerge
79.9
42.7
73.5
73,0
85.2
72.0
71.1
+2.00%
ParkMerge
53.67
75.64
38.91
16.47
22.18
4.03%
5.3 Use ForkMerge for Task Selection Simultaneously
As mentioned in Section 4.2) when there are multiple auxiliary task candidates, we use ForkMerge to simultancously select auxiliary tasks and jointly train them with the target task, which is denoted as ForkMerge
Auxiliary-Tass Scene Understanding. In NYUv2, we have 2 auxiliary tasks for any target task, thus we can construct 3 branches with different task weights in Equation 12] In this way, we are able to select auxiliary tasks adaptively by learning different for different branches in the merge step. As shown in Table this strategy yields better overall performance.
Auxiliary-Donaa Domain Image Recognition. For any target task in DomainNet, we can construct up to 6 branches with different task weights in Equation [2] which is computationally expensive. As mentioned in Section 4.2, we will prune the branches after the first merge step to reduce the computation cost. Table 4 reveals the impact of the pruning strategy. As the number of branches increases, the gain brought by auxiliary tasks will enlarge, while the gain brought by each branch will reduce. Therefore, pruning is effective strategy to achieve better balance between performance and efficiency. In practical terms, when confronted with multiple auxiliary tasks, users have the flexibility to tailor the number of branches to align with their available computational resources.
CTR and CTCVR Prediction. We evaluate on AliExpress dataset [36], recommendation dataset from the industry, which consists of 2 tasks: CTR and CTCVR, and 4 scenarios and more than 100M records. Our implementation is based on MTReclib [85). For any target task in AliExpress, we can construct up to 8 branches with different task weights, and we prune to 3 branches after the first merge

---

Table 3: Effect of of branch number NYUv2. Table 4: Effect of of branch number on DomainNet.
Methods
F3
Segmentation
Depth
Normal
A
Methods
B
I
R
Am
A m
mloU/
Pix Aect
Abs Erri
Rel Err.
Meunj
STL
77.6
41.
718
73.0
84.6
70.2
STL
51.42
74.14
41.74
17.37
22.82
PorkMerge
79.9
42.7
735
750
85.2
12.0
2,0%
2.0%
52.13
74.51
39.00
16,43
2414
0.30%
73.
730
ForkMerge
53.67
75.64
58.01
16.47
2118
4.03%
ForkMerge
44.
1.1%
RorkMerge
75.78
81.3
44.4
74.7
752
85.3
73.4
3.6%
0.7%
step. The results are presented in Table Note that improving on such a large-scale dataset with auxiliary tasks is quite difficult. Still, ForkMerge achieves the best performance with Am 1.30 %.
Semi-Supervised Learning (SSL), We also evaluate on two SSL datasets, CIFAR-I0 31 and SVHN [56]. Following (67),
Self-supervised Semi-supervised Learning (S4L) 82 our baseline
algorithm and use 2 self-supervised tasks, Rotation [19] and Exempler-MT 14, as our auxiliary tasks. Table/ppresentt the test error of S4L using different ATL approaches, along with other SSL methods, and shows that ForkMerge consistently outperforms the compared ATL methods. Note that do not aim to propose novel state-of-the-art SSL method in this paper. Instead, we find that some SSL methods use ATL and the auxiliary task weights have a great impact (see Grid Search in Table Thus, we use ForkMerge to improve the auxiliary task training within the context of SSL.
Table 5: Performance on AliExpress dataset.
Table 6: Peformance (test error) on CIFAR-10 and SVHN datasets.
Methods
CTR
CTCVR
Avg
ES
NL
US
ES
FR
NL
US
CIEAR-10
SVHN labels)
STL
0,7299
0.7316
0.7237
0,2077
0,8778
0.8682
0.8652
(.8659
0.7963
Methods
labels)
Am
EW
0,7299
0.7300
0.7248
0,7008
0,8855
0.8515
0,8605
018618
0,7931
-0,39%
STL
20.3
12.60
UW
0.7048
0.8814
0,8709
0,8999
(U8793
0.7966
33 IT-Modei
31.5%
MGDA
DWA
0.6953
0.7317
0.8926
0.7297
0,7000
0.7061
0.6676
0.8663
0,8215
0.8695
0,8145
0,7978
0.8696
0.8484
0.7917
0,7480
0.7937
-0.25%
-3.8%%
VAT 3
Tencher
66
13.9
5.63
43.8%
GradNorm
PCGrad
0.7239
0.7178
0.7101
0.7035
0.8851
0.8671
0,8479
088465
0.8413
0,8685
0.7988
-0.79%
Poondo-Tahel
17.8
7.62
26.4%
IMTL
0.7268
0.0852
0.9472
0.8502
08881
0,5282
0,7822
-2.20%
S41. + EW
15.7
7,83
30.7%
CAGrad
0.7250
0.7271
0.7225
0.69%
0.8712
0,8650
0,8417
0.8648
0.7900
-0.77%
S4L GraNNona
35.3%
NashMILL
0.7229
0.7245
0,7272
0,6972
0,8562
0.8606
038607
018497
0,7881
-1.000%
7.02
35.6%
OL_AUX
Forg
GCS
Post train
ARML
Auto-
0.7291
0.7229
0,7311
0.7282
0.7402
0.7211
0.7282
0.7245
0.7227
0.7427
0.7254
0.7239
0.7236
0.7272
0.7263
6.7416
0,7086
0.7114
0,7050
0.6972
0.7069
0.8779
0,8562
0.8928
0.780
0,8852
0.8889
0.8651
0.8671
0.8606
0,8646
0.8808
0.8786
0.8667
0,8610
0.8654
0.8753
0,8640
0.8675
0.8727
0.8497
0.8613
0,8750
0.5670
0,8753
0.7947
0.7981
0.7977
0.7949
0.2979
0.3067
+1.30%
+0.54%
+0.18%
+0.55%
+0.19%
-0.09%
S4L ARML
S4I. DI, DILAIIX
S41. Grdd Search
S41. Post-tain
ForkMerge
16.1
13.8
1577
7.85
6.07
5.89
7.82
41.0%
46.3%
43.2%
20.88
42.5%
30.4%
6 Conclusion
Methods have been proposed to mitigate negative transfer in auxiliary-task learning, yet there still lacks an in-depth experimental analysis on the causes of negative transfer. In this paper, we systematically delved into the negative transfer issues and presented ForkMerge, an approach to enable auxiliary-task learning with positive transfer gains. Experimentally, ForkMerge achieves state- of-the-art accuracy on four different auxiliary-task learning benchmarks, while being computationally efficient. We view the integration of previous task grouping methods with our auxiliary task learning approach as a promising avenue for further research.
Acknowledgements
We would like thank many colleagues, in particular Yuchen Zhang, Jialong Wu, Haoyu Ma, Yuhong Yang, and Jincheng Zhong, for their valuable discussions. This work was supported by the National Key Research and Development Plan (2020AA40109201), the National Natural Science Foundation of China (62022050 and 620210000 1002), and the Beijing Nova Program (200110006880041)
References
[1] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV,
10

---

2018.
[2] Shuxiao Chen, Koby Crammer, Hangfeng He, Dan Roth, and Weijie J Su. Weighted training for cross-task learning. ICLR, 2022.
[3] Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton, Big self-supervised models are strong semi-supervised learners. In NeurlPS, 2020.
[4] Xinyang Chen, Sinan Wang. Bo Fu, Mingsheng Long, and Jianmin Wang. Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning. In NeurlPS, 2019.
[5] Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In ICML, 2018.
[6] Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick sign: Optimizing deep multitask models with gradient sign dropout. In NeurlPS, 2020.
[7] Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In NeurlPS, 2017.
[8] Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[9] Lucio M Dery, Yann Dauphin, and David Grangier. Auxiliary task update decomposition: The good, the bad and the neutral. In ICLR, 2021.
[10] Lucio M Dery, Paul Michel, Mikhail Khodak, Graham Neubig, and Ameet Talwalkar. Aang: Automating auxiliary learning. In ICLR, 2023.
[11] Jacob Devlin, Ming- Wei Chang. Kenton I Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACI, 2019.
[12] Shachar Don- Yehiya, Elad Venezian, Colin Raffel, Noam Slonim, Yoav Katz, and Leshem Choshen. Cold fusion: Collaborative descent for distributed multitask finetuning. arXiv preprint 1rX1 2022.
[13] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.
[14] Alexey Dosovitskiy, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox. Dis- criminative unsupervised feature learning with convolutional neural networks. In NeurIPS, 2014.
[15] Yunshu Du, Wojciech M Czarnecki, Siddhant M Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, and Balaji Lakshminarayanan. Adapting auxiliary losses using gradient similarity. arXiv preprint arXiv: 1812.0222,, 2018.
[16] Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR, abs/1701
2017.
[17] Christopher Fifty. Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for
learning. In NeurIPS. 2021.
[18] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In ICML, 2017.
[19] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. ICLR, 2018
[20] Ian Goodfellow, Oriol Vinyals, and Andrew Saxe. Qualitatively characterizing neural network optimization problems. In ICLR, 2015.

---

[21] Priya Goyal, Piotr Dollár, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch SGD: training imagenet in I hour. arXiv preprint (oKiv:7706002677 2017.
[22] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017.
[23] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[24] Falk Heuer, Sven Mantowsky, Saqib Bukhari, and Georg Schneider. Multtaakk-cettemet (men); Efficient and diverse multitask learning using an anchor free approach. In ICCV, 2021.
[25] Adrián Javaloy and Isabel Valera. Rotograd: Gradient homogenization in multitask learning, In ICLR, 2022.
[26] Junguang Jiang, Baixu Chen, Bo Fu, and Mingsheng Long, Transfer-lcuming-libee https certhuu.com// wtt.uuu.commmmmm///raatttttt
Librrry 2020.
[27] Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05880 2022.
[28] Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty weigh losses for scene geometry and semantics. In CVPR, 2018.
[29] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[30] lasonas Kokkinos. Ubernet: Training 'universal' convolutional neural network for low-, mid- and high-level vision using diverse datasets and limited memory. In CVPR, 2017.
[31] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[32] Vitaly Kurin, Alessandro De Palma, Ilya Kostrikov, Shimon Whiteson, and M Pawan Kumar. In defense of the unitary scalarization for deep multi-task learning. In NeurlPS, 2022.
[33] Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In ICLR, 2017.
[34] Dong- Hygg Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML, 2013.
[35] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In NeurlPS, 2018.
[36] Pengcheng Li, Runze Li, Qing Da, An-Xiang Zeng, and Lijun Zhang. Improving multi-scenario learning to rank in e-commerce by exploiting task relationships in the label space. In CIKM, 2020.
[37] Baijiong Lin, YE Feiyang, Yu Zhang, and Ivor Tsang. Reasonable effectiveness of random weighting: A litmus test for multi-task learning. In TMLR, 2022.
[38] Baijiong Lin and Yu Zhang. LibMTL: A python library for multi-task learning. arXiv preprint arXiv:2203. 14338, 2022.
[39] Xingyu Lin, Harjatin Baweja. George Kantor, and David Held. Adaptive auxiliary task weighting for reinforcement learning. In NeurlPS, 2019.
[40] Bo Liu, Xingehao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conffict-averse gradient descent for multi-task learning. In NeurlPS, 2021.
[41] L Liu, Y Li, Z Kuang, J Xue, Y Chen, W Yang, Liao, and Wayne Zhang. Towards impartial multi-task learning. In ICLR, 2021.
[42] Shengchao Liu, Yingyu I Liang, and Anthony Gitter. Loss-balanced task weighting to reduce negative transfer in multi-task learning. In AAAI, 2019.
12

---

[43] Shikun Liu, Stephen James, Andrew J Davison, and Edward Johns. Auto-lambda: Disentangling dynamic task relationships. In TMLR, 2022.
[44] Shikun Liu, Edward Johns, and Andrew J Davison. End-to-end multi-task learning with attention. In CVPR, 2019.
[45] Hya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts, In ICLR, 2017.
[46] Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-expert.. In SIGKDD, 2018.
[47] Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang. Zelin Hu, Xiaoqiang Zhu, and Kun Gai. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In SIGIR, 2018.
[48] Kevis-Kokitsi Kokitsi Maninis, Ilija Radosavovic, and lasonas Kokkinos. Attentive single-tasking of multiple tasks. In CVPR, 2019.
[49] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation with multiple sources. In NIPS, 2008.
[50] Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In NeurlPS, 2021.
[51] Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In CVPR, 2016.
[52] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training:
a regularization method for supervised and semi-supervised learning. In TPAMI, 2018.
[53] Mehryar Mohri and Andres Muñoz Medina. New analysis and algorithm for learning with drifting distributions. In International Conference on Algorithmic Learning Theory, 2012.
[54] Aviv Navon, Idan Achituve, Haggai Maron, Gal Chechik, and Ethan Fetaya. Auxiliary learning by implicit differentiation. In JCLR, 2021.
[55] Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning bargaining game. In ICML, 2022.
[56] Yuval Netzer, Tao Wang. Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NeurIPS, 2011.
[57] OpenAl. Introducing chatgpt, 2022.
[58] OpenAL. Gpt-4 technical report, 2023.
[59] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sehastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In NeurlPS, 2019.
[60] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. In TKDE, 2010.
[61] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. ICCV, 2019.
[62] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Technical report, OpenAI, 2018.
[63] Michael T. Rosenstein. To transfer or not to transfer. In NewrIPS, 2005.
[64] Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. CoRR, abs/1606.04671, 2016.
13

---

[65] Ozan Sener and Vladlen Koltun. Multi-task learning as multi oulti--bjeccive optimization. In NeurIPS, 2018.
[66] Aviv Shamsian, Aviv Navon, Neta Glazer, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Auxiliary learning as asymmetric bargaining game, arXiv preprint arXiv:2301.13501, 2023.
[67] Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, and Huijuan Xu. Auxiliary task reweighting for minimum-data learning. In NeurIPS, 2020.
[68] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012.
[69] Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020.
[70] Xiaozhuang Song, Shun Zheng, Wei Cao, James Yu, and Jiang Bian. Efficient and effective multi-task grouping via meta learning task combinations. In NeurIPS, 2022.
[71] Trevor Standley, Amir Zamir, Dawn Chen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In ICML, 2020.
[72] Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. Progressive layered extraction (ple): novel multi-task learning (mt) model for personalized recommendations, In RecSys, 2020.
[73] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. NeurIPS, 2017.
[74] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 2008
[75] Zirui Wang, Zihang Dai, Barnabás Póczos, and Jaime Carbonell. Characterizing and avoiding negative transfer. In CVPR, 2019.
[76] Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In ICLR, 2021.
[77] Derrick Xin, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, and Orhan Firat. Do current multi-task optimization methods in deep learning even help? In NewrIPS, 2022.
[78] Yang You, Jing Li, Jonathan Hseu, Xiaodan Song, James Demmel, and Cho-Jui Hsieh. Reducing BERT pre-training time from 3 days to 76 minutes. In ICLR, 2020.
[79] Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. In NeurlPS, 2020.
[80] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
[81] Amir Roshan Zamir, Alexander Sax, William B. Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In CVPR. 2018.
[82] Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, and Lucas Beyer. S41: Self-supervised semi-supervised learning. In ICCV, 2019.
[83] Jing Zhang, Zewei Ding, Wanqing Li, and Philip Ogunbona. Importance weighted adversarial nets for partial domain adaptation. In CVPR, 2018.
[84] Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu. A survey on negative transfer. IEEE/CAA Journal of Automatica Sinica, 2022.
[85] Yongchun Zhu, Yudan Liu, Ruobing Xie, Fuzhen Zhuang, Xiaobo Hao, Kaikai Ge, Xu Zhang, Leyu Lin, and Juan Cao. Learning to expand audience via meta hybrid experts and critics for recommendation and advertising. In KDD, 2021.
14

---

A
Algorithm Details
A.1 ForkMerge
Proof of Equation
x arg max maxX((0+11)
arg maxP(0, (Bgr( + Aaux (0:)))
arg maxp ((0: (g (8:)) ))-)) + 17mux (8+)))
arg max? P(1 A)(0, 1g (0+) + 1(0, 71(tarr +aaux (0t)))
= arg MnaxP((11 ()) (1))
Remarks the search step.
We provide two search strategies as follows, and we use the first strategy in our experiments.
Exhaustively searching the task-weighting hyper-paramettr À through manually specified subset of the hyper-parameter space, such as {0, 0.2, 0.4, 0.6, 0.8, 1.0).
Grid Search:
Binary Search: Repeatedly dividing the search interval of X in half and keep the better haper-parameter.
Random search, bayesian optimization, gradient-based optimization, and other paper-parrmeter optimization methods can also be used here, and they are left to be explored in follow-up work.
In practice, the costs of estimating P in the search step usually negligible. Yet when the amount of data in the validation set is relatively large, we can sample the validation set to reduce the cost of estimating P.
Remarks extension from the one gradient step to At steps.
1. It can effectively reduce the average cost of estimating P at each step and avoid over-fitting the validation set.
2. It allows longer-term rewards from auxiliary tasks and leads safer task transfer. For instance, when the accumulated gradients of some auxiliary tasks are harmful to the final target performance, the merging step can cancel the effect of these auxiliary tasks by setting their associated weights À to 0, to mitigate strong negative transfer.
It increases the risk to produce bad model parameters. However, such risk is still low since deep models usually have smooth loss surfaces after convergence as shown in Section 5.1
(O Segnesraion
Daytk
Nacoal
DA
Ar
Figure 7: Effect of the merging step At on NYUv2.
Figure 7 illustrates that an appropriate At can effectively promote the performance of the ForkMerge algorithm, indicating the necessity of the extension from the one gradient step previous work At steps. When At is small, the estimation of À is short-insight and might fail to remove the harmful parameter updates when negative transfer occurs, which also indicates the limitations of methods that use single-step gradient descent to estimate X 15 43 When A/ is large, the risk to get bad model parameters from the linear combination will also increase. Therefore, our experiment, we use the validation set to pick proper At for each dataset and use for all tasks in this dataset.
15

---

A.2 Use ForkMerge to Select Tasks Simultaneously
Detailed Algorithm. Algorithm 2 provides the general optimization process for any task-weighting
vector
{1") B
1b=1- For Equation
have B K + 1 and v 1[ b 1 or i 0). For
Equation 13 we have constraints on B or nd.
Algorithm 2 ForkMerge Training Pipeline with Multiple Branches
Require: initial model parameter Og. task-weighting vector {v") Jb=1: total iterations T, interval Á/ Ensure: final model parameter f fork model into B copies )b=1
2: for = 1 to B do
3:
0g + 0o
initialization
4: end for
5: while
6: for b
1 to B do
independent update
7:
for = to + At 1 do
8:
0, 0r Ex 1/
9:
end for end for
10:
A" + arg maxA P Es A6644A44
11:
12:
search A on the validation set
14:
>> merge parameters
for 1 to do
16:
0.-:: x- ai
D synchronize parameters
end for
18:
t + At
end while
12 Proof of Equation
The goal of selecting X in Equation (00) to maximize the validation performance of model f.+1,
X arg max ((0++1)
arg maxP(0: 10.-11 17 Σ (0e)((e))
// //gaadient descent
arg maxP (B0.ooA 17
Σ (xg)((O)))
arg maxp (0.) - Σ Aegx((.))
1/5o =1
k=0
arg maxp ((1
(0, - (0.)) +
Ax) (B)) + Σ (n(-11B((00)
11 Σ A6 51
kz0
k=
A=
arg maxP((1
(0: (0)) +
Av(Or (0,) - v7g(((0)))
k
k
By definitions of A and (w KK
k=
Air = 0,
A
140
dt: k # O,
1.
i O = k.
w/
0,
otherwise,
16

---

we can prove that optimizing À in Equation I0 is equivalent to optimizing A as follows:
arg max A
Remarks on the search step.
Grid searching all possible values of A is computationally expensive especially when ||]] is large. Thus, here introduce a greedy search strategy in Algorithm [ which reduces the computation complexity from exponential complexity to ([]
Algorithm 3 Greedy Search of A
Require: A list of model parameters fI.... 8g sorted in decreasing order of P(Ob). Ensure: optimal linear combination coefficient A
1: unnormalized combination coefficient A e1
initialization
2: for 2 to B do
3: set upper bound U + E' Am
4: grid search the optimal Am in range [0, U] to maximize P( TAT Σ Em=1 0m0m
5:
end for
6:
A
1 A
D normalization
B Analysis Details
In this section, we provide the implementation details of our analysis experiment in Section
conduct our analysis on the multi-domain image recognition dataset DomainNet [6]]. our analysis, task Painting and Quickdraw in DomainNet examples of weak negative transfer and strong negative transfer, and other tasks (Real, Sketch, Infograph, Clipart) in DomainNet as auxiliary tasks. Details of these tasks summarized in Table g We use ResNet-18 [23 pre-trained on ImageNet
S for all experiments.
B.1 Effect of Gradients Confficts
First, we optimize the model on the target task for T = 25K iterations to obtain 0r. We adopt mini-batch SGD with momentum of 0.9 and batch size of 48, and the initial learning rate is set as 0.01 with cosine annealing strategy [45
Ausiarr DomairNee
Ausllory Domaiehlee
Andiory:: bricaston
Ausillary
Figure 8: The distribution of Gradient Cosine Similarity (GCS). P and Q are short for Painting and Quickdraw tasks, respectively.
We repeatedly sample mini-batch of data and estimate the gradients for the target and auxiliary task Sig: and gax- Figure (8 plots the distribution of gradient cosine similarity (GCS) between gig and gaux We find that the gradients of different tasks are nearly orthogonal (cos diy As 0) in most cases, and highly consistent gradients severely conflicting gradients are both relatively rare.
17

---

Then, we optimize the same OT with one-step multi task gradient descent estimated from different data to obtain different 0T+1.
(A+1(A) = (7(gig(())) + Agax((r))
(14)
where 7/ 0.01 and takes values from {0, T R2 2) 1]. We evaluate 0T+1 (A) and Or+1 (0) on the validation set of the target task to calculate the transfer gain (TG) from single-step multi-task gradient descent
TG(A) P(0r+1 ((0r+1 (A
(15)
Note that we omit the notation algorithm in Equation (14 and (T3) for simplicity. Then, in Figure we mark the GCS and TG of each data point and fit them with 3-order polynomial to obtain the corresponding correlation curve.
B.2 Effect of Distribution Shift
Qualitative Visualization. We visualize by t-SNE 774 in Figure 3(a) the representations of the training and test data by the model 0r trained in Section B.1 For better visualization, only keep the top 10 categories with the highest frequency in DomainNet. To visualize the impact of on the interpolated training distribution, we let the frequency of auxiliary task points be proportional to X. In other words, when the weighing hyper-paramen parameter the auxiliary task increases, the effect of the auxiliary task the interpolated distribution will also increase.
Figure 3 provides the t-SNE visualization of training and test distributions when takes values from {0, 16: 3: 1). We observe that for weak negative transfer tasks, when À initially increases, the area of training distribution can better cover that of the test distribution. But as À continues to increase, the distribution shift between the test set and the training set will gradually increase. For strong negative transfer tasks, however, the shift between the interpolated training distribution and the test distribution monotonically enlarges as À increases.
Quantitative Measure. First, we jointly optimize the model on the target task and auxiliary tasks with different weighting hyper-parameter for 25K iterations to obtain (A). We adopt the same typer-parameter as in Section B. Then evaluate (A(A) on the test set of the target tasks and calculate the average confidence on the test set. We can calculate the confidence score discrepancy (CSD) by Definition 3.5 and the transfer gain (TG) by
(G(A) (()) - (0))
(16)
Again, omit the notation of algorithm A for simplicity. Finally, plot the curve between CSD and TG under different À in Figure 3 b).
C Experiment Details
C.1 Definition of Am
55 37], we report Am as the performance measure, which is the average per-task performance improvement of method m relative the STL baseline b. Formally, Am where M3,s and Mm,,k the performance of the k-th task obtained by the baseline method and the compared method m. zμ is set to 0 if higher value indicates better performance for the k-th task and otherwise
Following
t EK (-1)* (Mm,, Mt.x.)
C.2 Auxiliary-Taa Task Scene Understanding on NYU
Experiment Details. We use DeepLabV3+ architecture M. where a ResNet-50 network pretrained on the ImageNet dataset [8] with dilated convolutions is used as a shared encoder among tasks and the Atrous Spatial Pyramid Pooling module is used as task-specific head for each task. Following [44 79), each method is trained for 200 epochs with the Adam optimizer 29 and batch size of 8. The initial learning rate is 10-1 and halved x 10-5 after 100 epochs. In ForkMerge, the parameters are merged every 10 epochs. Table 7 presents the full evaluation results of Table
18

---

Table 7: Performance on NYUv2 dataset.
Segmentation
Depth
Normal
Methods
A m
Angle Distance
Within
mloUt
Pix Acct
Abs Err
Rel Err!
Mean.
Median!
11.25"
22.51
301
STL
51.42
74.14
41.74
17.37
22.82
16.23
36.58
62.75
73.52
FW
52.13
74.51
39.03
16.43
24.14
17.62
33.98
59.63
70.93
0.30%
UW
52.31
74.72
39.15
16.56
23.99
17.36
34.46
60.13
71.32
0.63%
RLW
$2.88
74.99
39.75
16.67
23.83
17.23
34,76
60.42
71.50
0.66%
MGDA
50.79
73.81
39.19
16.25
23.14
16.46
36.15
62.17
72.97
1.44%
GradNorm
52.25
74.54
39.31
16.37
23.86
17.46
34,13
60.09
71.45
0.56%
PCGrad
51.77
74.72
38.91
16.36
24.31
17.66
33.93
59.43
70.62
0.22%
TMTL
52.24
74.73
39.46
15.92
23.25
16.64
35.86
61.81
72.73
2.10%
GradVac
52.84
74.77
39.48
16.28
24.00
17.49
34,21
59.94
71.26
0.75%
52.04
1.41%
NoshMTL
51.73
74.10
39.55
16.50
23.21
16.74
35.39
61.80
72.92
1.11%
GCS
52.67
74.59
39.72
16.64
24.10
17.56
34,04
59.80
71.04
0.09%
AL_AUX
$2.07
74.28
39.32
16.30
23.98
17.87
33.89
59.53
71.08
0.17%
ARMI.
Auto-A
52.40
74.62
39.25
16.25
23.38
17.20
34,05
61.18
72.05
1.17%
Post-train
52.08
74.86
39.58
16.77
22.98
16.48
36.04
62.27
73.20
1.49%
ForkMerge
53.67
75.64
38.91
16.47
22.18
15.60
37.93
64.29
74.81
4.03%
C.3 Auxiliary-Domain Image Recognition DomainNet
Dataset Details. As the original DomainNet 66] does not provide a separate validation set, we randomly split 50% data from the test set as the validation set, and use the rest 50% data as the test set. For each task, the proportions of training set, validation set, and test set are approximately 115%. Table 8 summarizes the statistics of this dataset. DomainNet under Custom (research-only, com-commercial) license.
70%/15% 15%.
Table 8: Overview of DomainNet dataset.
Tasks
#Train
#Val
#Test
Description
Clipart
33.5K
7.3K
7.3K
collection clipart images photos and real world images sketches specific objects infographic images painting depictions of objects drawings game "Quick Draw"
Real
120.9K
26.0K
26.0K
Sketch
48.2K
10.5K
10.5K
Infograph
36.0K
7.8K
7.8K
Painting
50.4K
10.9K
10.9K
Quickdraw
120.7K
Experiment Details. We adopt mini-batch SGD with momentum of 0.9 and batch size of 48. We search the initial learning rate (0.003, 0.01, 0.03) and adopt cosine annealing strategy 445 to adjust learning rate during training. We adopt ResNet-101 pretrained on ImageNet as the backbone. Each method trained 50K iterations. In ForkMerge, the parameters are merged every 12.5K iterations.
C.4 CTR and CTCVR Prediction AliExpress
Dataset Details. AliExpress 366 is gathered from the real-world traffic logs of AliExpress search than 100M records in total. We split the first 90% data in the time sequence to be training set and the rest 5% and 5% to be validation set and test set. AliExpress consists of 2 tasks: click-through rate (CTR) and click-through conversion rate (CTCVR), and 4 scenarios: Spain (ES), French (FR), Netherlands (NL), and America (US). Table /uummarizes the statistics of this dataset. AliExpress under Attrbution-NonCommeriia Share 4.0 International (CC BY-NC-SA 4.0) license.
system in Taobao and contains
Experiment Details. single embedding layer shared by all tasks and multiple independent DNN towers for each task. The embedding dimension for each feature field is 128. Each method trained for 50 epochs using the Adam optimizer, with the batch size of 2048, learning rate of 10 10-3 and weight decay of 10-6
The architecture most methods based on ESMM 47 which consists of
19

---

Table 9: Overview of AliExpress dataset, where CTR= = #Click #llpok/ssmmosooion and CTCVR = #Purchase
#lmpression.
Statisties
FR
#Product
8.7M
7.4M
6M
XM
#Impression
31.6M
27.4M
17.7M
27.4M
841K
535K
382K
#Purchase
19.1K
14.4K
13.8K
10.9K
2.01%:
2.16%
CTCVR
0.00%
0.54%
0.78%
0.40%
C.5 Semi-supervised Learning on CIFARIO and SVHN
Following (67, we first split the original training set of CIFARIO 31 and SVHN (56 into training set and validation set. Then, we randomly sample labeled images from the training set. Table summarizzes the statistics of CIFAR-IO and SVHN.
Dataset Details.
Table 10: Overview of CIFAR- 10 and SVHN datasets.
Datasets
#Labeled
#Unlabeled
#Val
#Test
CIFAR-I0
10000
SVHN
1000
64931
7326
26032
Experiment Details. (1) Auxiliary Tasks. Following (82 67 consider two self-supervised
auxiliary tasks Rotation [19] and Exempler-MT [44. Rotation, we rotate each image by [ 90%, 180", 270"] and ask the network to predict the angle. In Exemplar-MT, the model is trained to extract feature invariant to a wide range of image transformations. (2) Hyper-parameters. We adopt Adam [29] optimizer with initial learning rate of 0.005. We train each method for 200 iterations and decay the learning rate by a factor of 0.2 at 160K iterations. We use Wide ResNet-28-2
80 as the backhone. In ForkMerge, the parameters merged every 10K iterations.
C.6 Data Division Strategy for ForkMerge
As discussed in Section 4.2 in ForkMerge, we can construct branches with different sets of auxiliary tasks. Below we outline the specific data division strategy used in our experiments, which is consistent with previous ATL literature:
For the NYUv2 dataset, multiple tasks share the same input, but their outputs different. In this setup, each branch has the same input data, which includes the entire dataset. The distinction between different branches solely lies in the task weighting vector {v*} Jb=1"
For DomainNet, AliExpress, CIFAR-10, and SVHN datasets, different tasks have both different inputs and outputs. In these cases, for each branch, the task weighting a specific task is set to 0, the data from that particular task will not be used for training the corresponding branch.
D Additional Experiments
D.1 Analysis on the importance of different forking branches
The importance of different forking branches is dynamic. As shown in Figure 9 the relative ratio of each forking branch is dynamic and varies from task to task, which indicates the importance the dynamic merge mechanism.
D.2 Analysis on the computation cost
The computation cost of Algorithm is O(K) and the computation cost the pruned version O(B). Usually, only one model is optimized in most previous multi-task learning methods, yet their
20

---

Brach
Brooh
Booot
Seg
Dep
Nar
Seg: Dep
Nor + Dep
64
19
Segmentation
Depth
Nommal
Figure 9: Importance of different forking branches during training NYUv2.
computational costs
necessarily O(1). Gradient balancing methods, including MGDA 65 GradNorm [5] PCGrad 77). IMTL [4]] GradVac [76). CAGrad [40], NashMTL (55), GCS 15]
AL_ 399 and ARML 67 require computing gradients of each task, thus leading O(k)
complexity. In addition, calculating the inner product or norm of the gradients will bring a calculation cost proportional the number of network parameters. A common practical improvement is to compute gradients of the shared representation 65].
Yet the speedup is aurhitecture-dependent, and
this technique may degrade performance
55
In Figure 10. we also compare the actual training time across these methods on NYUv2. We can observe that ForkMerge does not require the significant performance gains it brings, these additional computational costs are also worth it. Furthermore, our fork and merge mechanism enables extremely easy asynchronous optimization which is not straightforward in previous methods, thus the training time of our method can be reduced (1) when there are multiple GPUs available.
time than most other methods. And considering
300
Training (iee Pr-qpoc (003)
250
200
150
13W
10W
DWAA
11.30
MCDDA Fosthhemmm
IMTI Gadite CAnneNaaM GCS otLAIIX ABMI
Assed
Ohex (B-3)
Ore
Om
(B-21
Juyc
Figure
Training speed of different MTL. methods on NYUv2 (10 repetitions).
D.3 Analysis the convergence and variance
Figure TT plots the validation performance of STL, EW, and ForkMerge throughout the training process NYUv2. Each curve is obtained by optimizing the same method with 5 different seeds. Compared with single-task learning or minimizing the average loss across all tasks, ForkMerge not only improves the final generalization but also speeds up the convergence and reduces the fluctuations during training.
D.4 Comparison with grid searching
we observe that adjusting the task- weightiig hyper- parameter can effectively reduce the negative transfer and promote the positive transfer. [77] also suggests that sweeping the task weights should be sufficient for full exploration of the Pareto frontier at least for convex setups and observe no improvements in terms of final performance from previous MTI. algorithms compared with grid search.
In Section 3
21

---

(a) Segmentation
(b) Desch
(e) Normal
24
56
BW
-74
EW ForkMerge
Forkhderge
Mean Angle Distanes
12
Abs Eat
122
STL BW FarkMepee
270
Epuchs
Epuchs
Epochs
Figure 11: Learning curves comparing different methods on NYUv2. Each curve plots the mean and standard deviation of the validation performance of a method with 5 different random seeds.
we compare the performance of all methods with grid search on NYUv2. In grid search, the weighting hyper-parameter for each task takes values from {0.3, 1.0, 3.0}. and there are 3 tasks NYUv2, thus there are 27 combinations in total. We find that previous methods simply yield performance trade-off points on the scalarization Pareto front, which has also been observed in previous work [77]. In contrast, our proposed ForkMerge yields point far away from the Pareto front and achieves significant improvement over simply optimizing a weighted average of the losses. One possible reason for the gain is that the task weighting in grid search is fixed during training and takes finite values due to the limitation of computing resources while the task weighting in ForkMerge is dynamic in time and nearly continuous in values, thus can better avoid negative transfer and promote positive transfer.
In Figure 12
AAM
Gld Soac
LN
HREA
X000
350m
MMM
Figure 12: Comparison with grid search on NYUv2. We use mloU for semantic segmentation, 1/ absolute error for depth estimation, and 1/ mean angle distance for surface normal prediction. We plot 2D projections of the performance profile for each pair of tasks. Top-right is better.
D.5 Comparison with larger batch size training
In some sense, the multiple branches in ForkMerge increase the equivalent batch size. And it has been revealed that batch size may have great effect on the performance of deep models 21 78). To ablate the influence of batch size, we increase the batch size of the Equal Weighting method. As shown in Table II] the improvement brought by ForkMerge itself significantly larger than simply increasing the batch size.
D.6 ForkMerge with more network architectures
ForkMerge with Vision Transformers. We replace the backbone network ResNet-101 with advanced ViT-Base [3] pretrained on ImageNet-21K K and repeat the experiments on the DomainNet dataset (Section 5.2) As demonstrated in Table when employing the Vision Transformer model, which boasts increased capacity, the risk of overfitting with limited data becomes more pronounced.
22

---

Table 11: Comparison of different methods with larger batch size training.
Segmentation
Depth
Normal
Methods
Batch Size
Angle Distance
Within e
m
mioU?
Acef
Ern)
Err]
Mean]
Median.
11.251
301
EW
52.13
74.51
39.03
16.43
24.14
17.62
33.98
59.63
70.93
0.30%
FW
32
51.40
73.99
38.86
16.20
23.99
17.34
34.58
60.08
70.85
0.55%
ForkMerge
8
54,30
75.78
38.42
16.11
22.41
15.72
37.81
63.89
74.35
4.59%
This makes Single Task Learning (STL) less effective and consequently leads the Equal Weighting (EW) method outperforning STL, causing the Post-train method to fall short of EW and Auto-À. In this case, ForkMerge still exhibited superior performance, validating its efficacy across different network architectures.
Table 12: Performance on DomainNet by replacing the ResNet-101 architecture with the ViT-Base architecture.
Methods
P
Avg
Am
37.8
68.0
EW
81.9
43.7
74.0
71.3
84.1
73.0
71.3
+5.90 +5.90%
Auto-A
Post-train
76.2
38.8
69.5
71.7
83.2
69.7
68.2
+0.51%
ForkMerge
83.0
45.6
76.3
73.2
74.7
73.3
+8.97%
ForkMerge with Multi-task Architectures. ForkMerge is complementary to different multi-task architectures. In Tables 13 and 14 we provide comparison of different optimization strategies with MTAN [44] and MMoE [46] as architectures, which are widely used in multi-task computer vision tasks and multi-task recommendation tasks respectively. On these specifically designed multi-task architectures, ForkMerge is still significantly better than other methods.
Table 13: Performance NYUv2 dataset by replacing the DeepLabV3+ architecture with the MTAN architecture.
Segmentation
Depth
Normal
Methods
A Amt
Angle Distance
Within 1
mloUt
Pix Acef
Abs Err.
Rel Err.
Mean.
Median
11.251
22.51
301
EW
53.27
75.36
39.37
16:38
23.61
17.00
35.00
61.01
72.07
1.62%
53.05
74.79
39.50
16.49
24.05
17.49
34.14
71.13
0.57%
OL_
52.47
74.70
39.27
16.39
23.66
17.43
34.49
59.96
71.76
0.82%
ARMI
52.33
74.59
39.46
16.61
23.57
17.41
34.56
60.12
72.04
0.55%
Auto-A
52.90
75.03
39.67
16.45
22.71
15.60
38.35
64.09
73.92
3.18%
ForkMerge
55.25
76.16
38.45
16.08
21.94
15.22
38.96
65.04
75.33
5.76%
Table 14: Performance AliExpress dataset by replacing the ESMM architecture with the MMoE architecture.
Methods
CTR
CYCTR
A yr.
ES
FR
NL
US
ES
FR
NL
US
0.7225
0.8742
GCS
0.7300
0.7190
0.7270
0.7102
0.8857
0,8773
0.8680
0,8740
0.7989
0,29%
OL_AUX
0.7265
0.7283
0.7264
0.7146
0.5849
0.8750
0.8710
0.8770
0.8005
0.50%
ARML
0.7289
0.7278
0.7248
0.7081
0.8869
0.8801
0.8714
0.8610
0.7986
0.26%
Auto-A
0.7269
0.7273
0.7256
0.7111
0.8827
0.8811
0.8721
0.8726
0.7999
0.42%
ForkMerge
0.7368
0.7349
0.7359
0.7116
0.8942
0.8791
0.8717
0.8840
0.8060
1.20%
23


Reviews:
[{"review_id": "vZPAMTKTZu", "reviewer": "Reviewer_yZ6C", "summary": "This paper strives to mitigate negative transfer in auxiliary-task learning by optimizing the coefficients assigned to auxiliary tasks. By conducting an empirical investigation into the factors contributing to negative transfer, this paper reveals two interesting findings. Based on the findings, a new approach named ForkMerge is proposed to mitigate negative transfer and boost the performance of auxiliary-task learning. Extensive experiments demonstrate the effectiveness of the proposed approach.", "questions": "See Weaknesses", "limitations": "Not sure", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+The problem is well-defined and well-motivated.\n+The findings seem interesting\n+The proposed approach is reasonable and well presented.", "weaknesses": "-It is not clear how the new findings motivate the proposed ForkMerge. \n\n-I'm curious about the performance when directly trying various lambdas (grid search) during multiple training sessions. Although the proposed method is more efficient, there is a concern about potential accuracy trade-offs. I seek to understand if the proposed approach sacrifices accuracy compared to the direct lambda variation method. \n\n-The simple post-train method leads to superior performance over other complex approaches, which makes me doubt the significance of research efforts in this area over the past years. This paper claims that the main drawback of post-train method is that it fails to consider the task relationship in the pre-training phase, and suffers from forgetting during fine-tuning. I’m wondering the performance of the post-train method if considering the task relationship in the pre-training phase and preventing forgetting during fine-tuning such as by distillation. In this scenario, if the post-train method manages to surpass the proposed ForMerge approach, considering ForMerge’s slight improvement in most tasks compared to Post-train.\n\n-The experimental evaluation is limited to small datasets, such as NYUv2 and CIFAR-10, and employs a relatively small network like ResNet-50 (DeepLabV3+). As a result, there is uncertainty regarding the efficacy of the proposed method on larger datasets and powerful networks like transformers, which already excel in single-target tasks. Consequently, assessing the true significance of the proposed method in the current era of deep learning becomes challenging.\n\n-Too may abbreviations make paper hard to follow in some paragraphs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer yZ6C for providing insightful reviews and valuable comments. We have clarified the questions in the following response.\n\n**Q1:** It is not clear how the new findings motivate the proposed ForkMerge.\n\nPlease refer to $\\text{question 1 (Q1)}$ of our global rebuttal.\n\n**Q2:** Comparison with directly grid searching task weighting $\\lambda$.\n\nIn $\\text{Figure 6 of Appendix D.4}$, we present the comparison of all methods with grid search. For grid search, there are a total of $27$ weighting configurations. Our observations are as follows:\n\n- Existing methods typically yield performance trade-off points that lie along the scalarization Pareto front.\n- ForkMerge produces results that are distant from the Pareto front, resulting in substantial performance gains over the grid search technique.\n- This improvement can be attributed to the fact that grid search tends to converge towards suboptimal solutions due to its reliance on fixed auxiliary task weights. Conversely, ForkMerge possesses the capability to dynamically adjust task weights continuously, as depicted in $\\text{Figure 3 of Appendix D.1}$.\n\n**Q3:** Comparison with improved Post-train method.\n\nTo address the reviewer's concern, we will first discuss the Post-train method and then present additional experimental results with an improved version of Post-train.\n\n**Discussion on Post-train Method.**\n\n- Post-train can be seen as a specific instance within ForkMerge's framework. In the first half of the training period, only branches with equal task weighting are employed, while in the second half of the period, only branches with single task weighting are used.\n- Our experimental results demonstrate the superiority of ForkMerge over Post-train across all benchmark tasks. Specifically, ForkMerge achieves substantial performance gains in CTR and CTCVR prediction tasks ($\\text{Table 5}$: $+1.30\\\\%$ *v.s.* $+0.14\\\\%$) and the semi-supervised learning task ($\\text{Table 6}$: $+46.3\\\\%$ *v.s.* $+30.4\\\\%$). \n- Lastly, when the performance of single task learning (STL) is worse than equal weighting (EW), the fixed post-training strategy may not yield performance gains. For example, when employing ViT-Base as the backbone network on the DomainNet dataset, where ViT is particularly data-hungry, the performance of the Post-train approach falls behind that of EW and ForkMerge. For comprehensive results, please consult $\\text{Table 2 of our global rebuttal pdf}$.\n\n**Additional Experiments.**\n\nWe enhance the Post-train method by adopting the Knowledge Distillation (KD) technique [1] to preserve the knowledge. $\\text{Table 1 of our global rebuttal pdf}$ presents the results on the DomainNet dataset. As evident from the table, ForkMerge outperforms the improved Post-train method.\n\n[1] *Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NeurIPS Workshop, 2014.*\n\n**Q4:** Limitation on the scale of evaluation datasets.\n\n- We adopt the NYUv2 dataset in scene understanding task, CIFAR10, and SVHN datasets in semi-supervised learning task as they are widely used in the auxiliary task learning literature. By doing so, we can provide a more fair and meaningful comparison with prior methods.\n- Additionally, as shown in $\\text{Section 5}$, we further conduct experiments on medium scale DomainNet dataset (about $0.6$M images) and large scale AliExpress dataset (over $100$M records). ForkMerge clearly outperforms existing methods on both datasets, affirming its efficacy across datasets of different magnitudes.\n\n**Q5:** Experiments with advanced architectures such as transformers.\n\nTo address the reviewer's concern, we replace the backbone network with ViT-Base [2] pretrained on ImageNet 21K and repeat the experiments on DomainNet of $\\text{Section 5.2}$. $\\text{Table 2 of our global rebuttal pdf}$ presents the results and we have the following observations.\n\n- With ViT-Base, the equal weighting (EW) method achieves higher average accuracy over single task learning. We conjecture that this improvement can be attributed to the data-hungry nature of vision transformers [2]. In this case, the benefits of reducing overfitting through the use of auxiliary tasks may outweigh the potential task interference problem.\n- ForkMerge outperforms compared methods on all tasks (average accuracy: ForkMerge $73.3\\\\%$ *v.s.* Auto-$\\lambda$ $71.5\\\\%$), validating its efficacy across different network architectures.\n\n[2] *Dosovitskiy, Alexey et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR, 2021.*\n\n**Q6:** Too many abbreviations make the paper hard to follow in some paragraphs.\n\nMany thanks to the reviewer's feedback. We will update our camera-ready version as follows:\n\n- We will reduce the use of abbreviations.\n  - For instance, in the analysis part ($\\text{Section 3}$), we will directly use the full concepts such as \"Transfer Gain,\" \"Weak Negative Transfer,\" \"Strong Negative Transfer,\" and \"Confidence Score Discrepancy\" instead of their abbreviations. \n  - Besides, we will include a table that explains the meaning of all abbreviations used in the paper.\n- We will provide more details in the main text rather than solely discussing these issues in the Appendix.\n  - For instance, in the analysis part ($\\text{Section 3}$), we will include a description of the DomainNet dataset, outline our dataset processing steps, and provide details about the network.\n  - In the method part ($\\text{Section 4}$), we will elaborate on the data division strategy for each branch, provide details about how we prune branches and efficiently merge different branches to reduce computation costs. Additionally, we will provide a discussion comparing $\\text{Equation 12}$ and $\\text{Equation 13}$ to clarify their difference.\n  - In the experiment part ($\\text{Section 5}$), we will present more training details, including the specific network architecture and the important hyperparameters."}, {"review_id": "soEHQuWqxl", "reviewer": "Reviewer_G3Ms", "summary": "To fully leverage the knowledge from auxiliary tasks and mitigate negative transfer issues, this paper introduces ForkMerge, which automatically searches for varying task weights for auxiliary tasks by minimizing target validation errors. ForkMerge is evaluated under various settings, including multi-task learning, multi-domain learning, and semi-supervised learning. The results demonstrate that it outperforms existing methods and proves to be effective.", "questions": "W1. The paper does not provide a clear explanation of how the method filters out harmful parameter updates to mitigate negative transfer after merging and synchronizing the parameters of each branch.\n\nW2. Figure 3 would benefit from improved clarity, using a distinct color or different marks would be helpful. Additionally, why does the number of auxiliary task data points increase with the increasing $\\lambda$?\n\nW3. Lacks clarity regarding the data division strategy for each branch. It is not explicitly explained whether each branch is trained with a part of the data or full data. Moreover, the impact of increasing the number of branches on computational cost and its trade-off with efficiency and accuracy is not thoroughly addressed in the paper.\n\nW4. I am curious about the comparison between the learned weights by ForkMerge and the optimal weights. For instance, in Figure 3 (b), for WNT tasks, do the learned weights resemble the optimal weights shown in the figure?\n\nW5. It would be valuable to investigate and illustrate the trajectory of the learned weights over the course of training to understand their dynamics and convergence patterns. To gain a better understanding of the effectiveness of the ForkMerge approach, it would be insightful to compare the performance of fixed learned weights by ForkMerge with dynamically learned weights during training.\n\n", "limitations": "Limitations could be more clarified in the paper.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper demonstrates a well-organized structure. The authors have conducted experiments using diverse datasets and tasks, resulting in promising outcomes. Furthermore, the paper provides theoretical analysis of each component, offering valuable insights into their impact on performance. Overall, the clear structure and in-depth analyses make it an engaging and compelling read.", "weaknesses": "The paper lacks clear explanations for certain sentences, training details, and figure information. For example, there is a lack of clarity regarding the data division strategy for each branch and the effectiveness of the learned weights. Additional elaboration and detailed analysis in these areas would enhance the overall understanding and impact of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer G3Ms for providing insightful reviews and valuable comments. We have clarified the questions in the following response.\n\n**Q1:** Clarification on the data division strategy for each branch.\n\nDepending on the characteristics of auxiliary task learning scenario, there are two circumstances.\n\n- In the case of the NYUv2 dataset, multiple tasks share the same input, but their outputs are different. In this setup, each branch in the ForkMerge algorithm has the same input data, which includes the entire dataset. The distinction between different branches solely lies in the task weighting.\n- In contrast to the NYUv2 scenario, for datasets like DomainNet, different tasks have both different inputs and outputs. In these cases, for each branch, if the task weighting of a specific task is set to $0$, the data from that particular task will not be used for training the corresponding branch. \n\nWe appreciate the reviewer's feedback and will add clarification on the data division strategy for each branch in our camera-ready version.\n\n**Q2:** The trajectory of the learned weights over the course of training.\n\nWe have visualized the trajectory of the learned weights in $\\text{Figure 3 of Appendix D.1}$, which indicates the relative ratio of each forking branch is dynamic and varies from task to task.\n\n**Q3:** Clarification on the effectiveness of the learned weights. Comparison between the learned weights by ForkMerge and the optimal weights.\n\n**Effectiveness of the Learned Weights**.\n\nWe would like to clarify that the learned weights of ForkMerge are a sequence of task weightings rather than a fixed one.\n\n- At each merging step, ForkMerge will search for an optimal task weighting based on the current learning status of all branches.\n- $\\text{Figure 3 in Appendix D.1}$ visually demonstrates the dynamic behavior of task weighting during the training process in ForkMerge. As shown in the figure, there is no evidence to suggest that the task weighting will converge to a fixed value during training. Instead, the algorithm continuously adjusts the task weighting.\n\nRegarding the effectiveness of the learned sequence of task weighting, our experiments in $\\text{Section 5}$ demonstrate that ForkMerge consistently outperforms existing methods on various benchmarks.\n\n**Comparison with Optimal Weights.**\n\nWe are not pretty sure about the \"optimal weights\" mentioned in your question and we understand the possible reference to static weights obtained from grid search.\n\n- We have provided a comparison between ForkMerge and grid searching $\\lambda$ on the NYUv2 dataset, as presented in $\\text{Appendix D.4}$. In $\\text{Figure 6 of Appendix D.4}$, we visualize the performance comparison of all methods with grid search. For grid search, there are a total of $27$ weighting configurations. It can be observed that ForkMerge achieves substantial improvement over grid search on all tasks. \n\nIf there is any confusion or misunderstanding regarding the \"optimal weights\" mentioned in your question, we would be glad to answer any further questions and provide additional clarification.\n\n**Q4:** Comparison with fixed learned weights by ForkMerge.\n\nAs clarified above, defining fixed learned weights for ForkMerge might not be straightforward and may not hold a meaningful interpretation. It is important to note that ForkMerge naturally generates a dynamic sequence of task weightings, continuously adapting during the training process. And our experiments in $\\text{Appendix D.4}$ provide evidence that this dynamic task weighting mechanism outperforms grid searching static weights.\n\n**Q5:** Clarification on how the method filters out harmful parameter updates to mitigate negative transfer.\n\nAs outlined in $\\text{Section 4}$ of our paper, ForkMerge operates through an iterative process involving fork and merge steps:\n\n- During the fork step, each branch in ForkMerge is independently trained. It is in this step that harmful parameter updates might occur, potentially compromising the performance on the target task.\n- In the merge step, ForkMerge searches for the optimal task weighting combination of different branches. This mechanism empowers ForkMerge to dynamically adjust the task weighting for each branch based on its contribution to the overall performance. As a result, ForkMerge can decrease the weighting of branches where negative transfer occurs. In extreme cases, it can set the weighting to $0$, ignoring harmful parameters. After merging, the newly obtained parameters are synchronized across all branches.\n\nBy synchronizing the new parameters to all branches, ForkMerge ensures that the harmful parameter updates experienced during the fork step are effectively filtered out. The filtering process occurs post-merging, allowing each branch to benefit from the collective knowledge while mitigating the influence of negative transfer.\n\n**Q6:** Why does the number of auxiliary task data points increase with the increasing $\\lambda$ in Figure 3?\n\nAs discussed in $\\text{Section 3.2}$, adjusting $\\lambda$ will change the data distribution that the model is fitting. When the weighing hyper parameter of the auxiliary task increases, the effect of the auxiliary task on the interpolated distribution will also increase. As a result, to visualize the impact of $\\lambda$ on the interpolated training distribution, we let the frequency of auxiliary task points be proportional to $\\lambda$ (introduced in $\\text{Appendix B.2}$).\n\n**Q7:** The impact of increasing the number of branches on computational cost and its trade-off with efficiency and accuracy is not thoroughly addressed in the paper.\n\nPlease refer to $\\text{question 2 (Q2)}$ of our global rebuttal."}, {"review_id": "D3305Vsm8Q", "reviewer": "Reviewer_QCm1", "summary": "Auxiliary-Task-Learning (ATL) has been studied from the perspective of optimization, which aims to improve the performance of the target task by leveraging similar tasks. However, ATL can sometimes suffer from negative transfer, where the performance of the target task actually decreases when auxiliary tasks are added. In this paper, the authors take a more holistic perspective by considering both optimization and generalization. They propose a new method called ForkMerge that is able to resolve negative transfer and improve the performance of the target task.", "questions": "* As discussed in the weaknesses section, the connection between the two main observations to the ForkMerge is not strong. Can you provide a high level theoretical analysis of how ForkMerge can make better generalization not just based on the empirical analysis?\n* Forking a large model is very expensive. Did authors consider a simple ensemble based approach which should be also similar in their compute (not the actual model weight merging rather than prediction level)?\n", "limitations": "I don’t see the negative societal impact of this work.\n", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "* ForkMerge is a simple and practical algorithm that demonstrates good empirical observation.\n* Many analytic figures help readers to understand the negative transfer better although they are limited to DomainNet dataset.\n", "weaknesses": "* There seems to be some logical jump between the two observations listed in Section 3 and the proposed algorithm in Section 4. It is quite unclear how they are connected directly, and it’ll be great to have some theoretical formulation for the connection (at least in high-level). For example: why dynamically adjusting \\lambda is superior to the previous method? How does this address the generalization problem? I think this limits the contribution of this work.\n* ForkMerge essentially optimizes a fork of the given model and trains them separately multiple times; hence, it does require significantly larger compute during the model training.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer QCm1 for providing insightful reviews and valuable comments. We have clarified the questions in the following response.\n\n**Q1:** Logical Connection Enhancement.\n\nPlease refer to $\\text{question 1 (Q1)}$ of our global rebuttal.\n\n**Q2**: Why ForkMerge is superior to the previous method?\n\n**Enhanced Generalization.**\n\nWhile methods like GCS adjusted $\\lambda$ based on the training data at each step, this primarily tackled optimization-level challenges without always guaranteeing improved generalization. The dynamic $\\lambda$ adaptation in ForkMerge inherently captures the importance of generalization by directly considering the target validation performance during each merge step.\n\n**Mitigation of Estimation Uncertainty.**\n\nApproaches such as Auto-$\\lambda$, which attempted to estimate $\\lambda$ from a single gradient update, face challenges due to estimation fluctuations. The negative impact of inaccurately estimated $\\lambda$ values on model parameters might be particularly noticeable. ForkMerge addresses this by using a longer time for $\\lambda$ estimation, which reduces the negative impact of estimation errors on model parameters. The introduction of branch merging within our framework further strengthens the training process against inaccuracies in $\\lambda$ estimation, making the algorithm more robust.\n\n**Efficient Computation.** \nThe adoption of techniques such as Grid Search, which entailed exhaustive training and $\\lambda$ tuning, led to escalating computational demands as the number of auxiliary tasks increased, resulting in exponential complexity. In contrast, ForkMerge dynamically adjusts $\\lambda$ during each merging step, significantly reducing the complexity associated with exhaustive search. Furthermore, our approach mitigates the problem of suboptimal solutions that can arise due to fixed auxiliary task weights, as indicated in $\\text{Figure 3 in Appendix D.1}$, and achieves a better trade-off between performance and computation cost.\n\nWe hope these insights provide a clearer perspective on the advantages of ForkMerge's dynamic $\\lambda$ adjustment mechanism and its superiority over previous methods.\n\n**Q3:** Concern on larger compute during the model training.\n\nPlease refer to $\\text{question 2 (Q2)}$ of our global rebuttal.\n\n**Q4**: Consider an ensemble-based approach focusing on merging the prediction?\n\nIn this response, we'd like to highlight the distinct advantages of our ForkMerge algorithm in comparison to the suggested ensemble approach, while addressing the aspect of computational requirements.\n\nWhile ensemble methods indeed provide a means to combine multiple models, especially at the prediction level, it's essential to note that the computational costs associated with ensemble techniques during the testing phase can be substantial. Ensemble methods require making predictions using each individual model and then aggregating these predictions, leading to increased inference time and resource utilization.\n\nThus, we have explored an alternative: conducting ensemble learning with multiple models during the training phase and distilling them into the target task. We use the term Pred-ensemble to represent this method. The experimental results on the DomainNet dataset are presented in $\\text{Table 3 of our global rebuttal pdf}$. The Pred-ensemble method effectively improves the performance on $4$ of $6$ datasets, yet still lags behind our proposed ForkMerge on all tasks."}, {"review_id": "BpmBfNjyse", "reviewer": "Reviewer_uS5H", "summary": "The authors conduct an analysis of negative transfer in auxiliary task learning, finding that gradient conflicts are not necessarily tied to negative transfer, but that auxiliary tasks that induce large distribution shifts from the new training distribution to the test distribution tend to cause negative transfer. The authors then propose ForkMerge, which repeatedly forks a model into branches trained on just the target task and trained on both the target task and auxiliary task(s), and then uses target task validation set performance to determine how to merge the forked models.", "questions": "1. Can the authors describe in more detail the computational overhead of ForkMerge, especially wrt time and memory, and in relation to the existing approaches from Section 5? I am most curious about the comparison to meta-learning.\n\n2. Do the authors predict that these results and the practicality of ForkMerge are limited to settings where one starts with a pretrained initialization?\n\n3. re: definition 3.5, are there alternative measures of distribution shift that the authors have considered? I am personally somewhat skeptical of confidence scores as a basis for measuring distribution shift given that many models are poorly calibrated in the first place.", "limitations": "The authors test their method using only one model, ResNet-18 pre-trained on ImageNet, on image tasks. I won't push for additional experiments, but I would like to see the authors discuss what assumptions from this paper's experiments are expected to be critical for generalization of findings to other settings.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The authors share an interesting and thoughtful analysis, investigating both gradient conflicts and induced distribution shifts as potential causes of negative transfer and concluding that gradient conflicts are not necessarily tied to negative transfer.\n2. The ForkMerge approach that the authors introduce is intuitive and simple, and the results are consistent across a variety of tasks.\n3. Well-motivated and significant problem", "weaknesses": "1. In general, some critical details are omitted altogether in the main text, sometimes with no proper reference to the Appendix when details are missing. In particular:\n- In Section 3, please spell out Painting and Quickdraw (and later, Real in Section 5) and describe DomainNet when first introducing the tasks. If it needs to be brief, the authors can at the very least include a pointer to Appendix C for dataset and task details.\n- I also believe it is worth mentioning \"We use ResNet-18 [8] pre-trained on ImageNet [3] for all experiments.\" in the main text rather than only noting it in the appendix.\n\n2. I would have liked to see more overlap between works cited in the Experiments section and the works discussed in Related Work, especially discussion about and comparison of adaptive auxiliary- and multi-task learning strategies and settings. In particular, [36] and the original meta-learning paper [13]. A couple additional related works the authors may consider adding: https://arxiv.org/abs/2205.14082 \n and https://arxiv.org/abs/2212.01378", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer uS5H for providing insightful reviews and valuable comments. We have clarified the questions in the following response.\n\n**Q1:** Some critical details are omitted in the main text.\n\nThank you for the feedback. Below, we outline our intended revisions:\n\n- We will make sure to spell out acronyms such as \"Painting\", \"Quickdraw\", and \"Real\" in the main text when first introducing them. Moreover, we will provide comprehensive descriptions of these tasks and the DomainNet dataset in the main text.\n\n- In our revised version, we will include a statement in the main text to mention that we utilize ResNet-18 pre-trained on ImageNet. \n\n**Q2**: More related work.\n\nWe have carefully checked the recommended papers and agree that they are closely related to our research. In the camera-ready version, we will add the following discussion in the related work part.\n\nAANG [1] formulates a novel searching space of auxiliary tasks and adopts the meta-learning technique, which prioritizes target task generalization, to learn singlle-step task weightings.  This parallel finding highlights the importance of the target task generalization and we further introduce the multi-step task weightings to reduce the estimation uncertainty. \nAnother parallel method, ColD Fusion [2], explores collaborative multitask learning and proposes to fuse each contributor's parameter to construct a shared model. In this paper, we further take into account the diversity of tasks and the intricacies of task relationships and derive a method for combining model parameters from the weights of task combinations.\n\n[1] *Dery, Lucio M., et al. AANG: Automating Auxiliary Learning. In ICLR, 2023.*\n\n[2] *Don-Yehiya, Shachar, et al. Cold fusion: Collaborative descent for distributed multitask finetuning. arXiv preprint.*\n\n**Q3**: Discussion on the computation cost.\n\nPlease refer to $\\text{question 2 (Q2)}$ of our global rebuttal.\n\n**Q4**: Is ForkMerge limited to settings where one starts with a pretrained initialization?\n\nThe practicality of ForkMerge is not confined to settings with a pretrained initialization. In our experimental evaluation, we also employed the AliExpress, CIFAR-10, and SVHN datasets without leveraging any pretraining, as reported in $\\text{Table 5}$ and $\\text{Table 6}$, respectively. \n\nHowever, we did choose to use pretrained models in the case of DomainNet and NYUv2. This decision was rooted in two key factors. First, we aimed to ensure a fair comparison with prior works. Second, while not a universal requirement, the utilization of pretrained models is a prevalent setting. In numerous real-world applications, the absence of pretraining can significantly degrade performance.\n\n**Q5**: Are there alternative measures of distribution shift that the authors have considered?\n\nIn the field of machine learning, various measures of distribution shift have been introduced, such as the Maximum Mean Discrepancy [3] and $\\mathcal{H}\\Delta\\mathcal{H}$-Divergence [4], which have found applications in classic problems like domain adaptation. However, extending these measures to the context of auxiliary task learning presents certain challenges.\n\nThe primary hurdle arises from the fact that Maximum Mean Discrepancy and $\\mathcal{H}\\Delta\\mathcal{H}$-Divergence are originally defined over marginal distributions. This poses difficulties when dealing with auxiliary task learning scenarios where distinct tasks share a common feature space, yet exhibit variations in their output spaces. To address this limitation, we define the distribution shift in the output space, leading to the formulation of Confidence Score Discrepancy.\n\nWhile it is acknowledged that a deep learning model's predictive confidence on a specific data point might not always align with correctness, we posit that the expected confidence over a distribution captures certain characteristics of that distribution. While model calibration remains a concern, our definition places more emphasis on the relative magnitude of confidence. For instance, if the expected confidence for a test distribution $D_1$ is lower than that of another test distribution $D_2$, we interpret this as indicating a larger distance between the test distribution $D_1$ and the training distribution.\n\n[3] *Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch olkopf, and Alexander Smola. A kernel two-sample test. In JMLR, 2012.*\n\n[4] *Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F. Analysis of representations for domain adaptation. In NeurIPS, 2007.*\n\n**Q6:** Assumption of findings.\n\n- $\\text{Finding 1 (Section 3.1)}$: We believe that the phenomenon we observed, where negative transfer is not solely attributed to gradient conflicts and vice versa, holds general relevance. This observation is supported by numerous instances, such as L2 regularization mentioned in the paper. \n\n- $\\text{Finding 2 (Section 3.2)}$: The assertion that negative transfer is more likely when auxiliary tasks increase distribution shift between training and test data for the target task aligns with the principles of typical supervised learning. The validation of this assertion indeed presents certain challenges. To address this concern, we introduced the concept of Confidence Score Discrepancy, which quantifies the joint distribution distance in auxiliary task learning. It is important to note that the application of this concept assumes the presence of a confidence measure within the model. In cases such as regression tasks like the NYUv2 dataset, where this confidence measure is not as well-defined, its applicability may be limited. This is one of the reasons why we chose to conduct our analysis experiments on the DomainNet dataset, as it offers a suitable environment for investigating these intricacies.\n\nIn summary, we hypothesize that while certain aspects may have limitations in diverse scenarios, the principles underlying them remain widely applicable."}, {"review_id": "HW91iGMRLq", "reviewer": "Reviewer_pM6c", "summary": "The paper tackles the problem of learning multiple tasks together which is known to lead to \"task inteference\" or \"negative transfer\" issues. This is usually tackled by automatically scaling the task weights or gradients based on training statistics (e.g. GradNorm or uncertainty weighing of losses). In particular, the paper studies an asymetric version of the problem, Auxiliary Task Learning (**ATL**) where one task may not be important at inference but can greatly improve performance when jointly trained with the main target task.\n\nThe paper first explore potential causes of task interference under the lens of train/target distribution shifts. Then, the paper proposes **ForkMerge**, a novel optimization method to avoid task interference: For each update of ForkMerge, first the current parameters are duplicated. Then, the weights  are updated separately; one on the target task, while the other is jointly trained for the task and auxiliary tasks. Then, the optimal task weights are found by finding the linear combination of these two sets of weights that maximize validation accuracy (**Equation 12**). The algorithm can be extended to multiple auxiliary tasks which requires an additional separate optimization branch for each new auxiliary task.\n\nThe method is then evaluated on the NYU dataset (3 tasks) and DomainNet (6 domains) and compared to previous MTL and ATL approaches", "questions": "- **Suggestion of writing**: I feel like the paper sometimes overuse abbreviations. For instance only mentioning the tasks **P** and **Q** for DomainNet (Painting and Quickdraw domains ?) assumes the reader is very familiar with the dataset. In additional, there are many abbreviations introduced in the paper which often make the text hard to read (for instance lines 154-159)\n\n- **Distribution shift and hypothesis shift**: I did not understand the point that *lines 213-221* try to convey and the difference between Equations 12 and 13 (outside of renaming $K$ to $B$ and allowing $\\Delta_t$ time steps instead of only 1). Similarly, I'm not sure what insight  **Figure 6**  tries to convey. More specifically: the task weighting $\\lambda$ is initially defined at the loss level (Equation 1); However, it seems that in later sections $\\lambda$ is rephrased as a parameter mixing the distribution of the target and auxiliary tasks; while there are links between these two views, I found that switching between the two interpretations in the text a bit confusing. It is also not clear what it means for multi-task applications: There is only one input distribution (the input images are the same for all tasks) so which distribution shift are we referring to ?\n\n- From the appendix **C.3**, it seems that the test set of DomainNet was split into two parts: one for validation, and the other one for test performance. I think this should be clearly stated in the main paper directly as it means the reported results are not comparable to other DomainNet works", "limitations": "There is no dedicated limitations section but the paper discusses the limitation of dedicating independent optimization branches for every auxiliary task (e.g. branch pruning to avoid handling too many branches in the DomainNet)", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- I liked the analysis section of the paper and it contains interesting insights on the problem of task interference: For instance, gradient conflicts is often named as a cause of negative transfer but its effect/strength is rarely actually measured in practice. Although it would have been interesting to extend the analysis to more diverse scenarios.\n\n- I also like the insights that task weighing methods should take generalization into account, which is captured in the proposed algorithm by finding the optimal task weights via evaluation on the validation set\n\n- The paper presents extensive experiments on two classical multitask/multidomain benchmarks, which are further completed in the supplementary material (e.g. additional backbone)", "weaknesses": "- **Training cost**: the paper should discuss the training cost in terms of memory efficiency more concretely/quantitatively: ForkMerge requires a separate set of parameters for each task. In addition these parameters are also updated independently hence the number of forward/backward passes will also increases with t he number of tasks. Finally, the cost of search for optimal $\\Lambda$ in Equation 13 will also increase with the number of tasks. These costs may remain reasonable in the application scenario with only one auxiliary task, but it's not clear how practical ForkMerge becomes when dealing with more than two tasks like in Section 5.2\n\n- **Finding 2 does not seem very significant**. Finding 2 states that negative transfer is likely to occur when the added auxiliary task increases the train/test distribution shift for the target task: This seems like a pretty classic statement from statistical machine learning theory: For instance generalization bounds for domain adaptation typically include a term measuring the discrepancy between the source / target domains.\n\n- **Chosen baselines might not all be a fair choice**:  the multi-task optimization baselines (MGDA, GradNorm, PCGrad... etc) typically aim to optimize performance on **all tasks**, as opposed to auxiliary task learning where there is a clear bias towards the target tasks: For instance in ForkMerge, every branch includes the target task objective. For that reason, I do not think applying MTL methods \"as-is\" to the problem of ATL is the best baseline. A stronger baseline could be to finetune the task weights in the \"Equal Weights\" (**EW**) baseline for performance on the target task; this might be to costly for DomainNet but would be reasonable for NYU as there are only two auxilair ytasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer pM6c for providing insightful reviews and valuable comments. We have clarified the questions in the following response.\n\n**Q1:** Concern on the training cost.\n\nPlease refer to $\\text{question 2 (Q2)}$ of our global rebuttal.\n\n**Q2:** Contribution of Finding 2.\n\nIt is worth emphasizing that our contribution lies in providing a new perspective, that is, to consider the problem of auxiliary task learning from the standpoint of model generalization, and to offer a quantifiable metric in this context.\n\nIndeed, traditional measures of distribution distance, such as the Maximum Mean Discrepancy [1] and  $\\mathcal{H}\\Delta\\mathcal{H}$-Divergence [2], have been extensively employed in well-established problems like domain adaptation. However, their direct application to the realm of auxiliary task learning encounters specific challenges.\n\nThe primary challenge stems from the original definition of Maximum Mean Discrepancy and $\\mathcal{H}\\Delta\\mathcal{H}$-Divergence, which are based on marginal distributions. To overcome this limitation, we introduce an innovative approach by establishing distribution shift considerations in the output space, leading to $\\text{Definition 3.5}$.\n\n[1] *Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch olkopf, and Alexander Smola. A kernel two-sample test. In JMLR, 2012.*\n\n[2] *Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F. Analysis of representations for domain adaptation. In NeurIPS, 2007.*\n\n**Q3:** Comparison with stronger baseline.\n\nWe follow the reviewer's suggestion and enhance the performance of baselines MGDA, GradNorm, and PCGrad by fine-tuning the model obtained from multitask learning to each task. $\\text{Table 4 of our global rebuttal pdf}$ presents the results on the NYUv2 dataset, where ForkMerge consistently outperforms improved baselines.\n\n**Q4:** Suggestion of writing.\n\nMany thanks to the reviewer's feedback. As revision is not supported in the rebuttal stage, we will update our camera-ready version as follows:\n\n- We will reduce the use of abbreviations. For instance, in the analysis part ($\\text{Section 3}$), we will directly use the full concepts such as \"Transfer Gain,\" \"Weak Negative Transfer,\" \"Strong Negative Transfer,\" and \"Confidence Score Discrepancy\" instead of their abbreviations. \n - Besides, we will include a table that explains the meaning of all abbreviations used in the paper.\n\n**Q5:**  Confusion on the distribution shift and hypothesis shift.\n\n**Explanation about $\\lambda$.**\nIn our paper, $\\lambda$ serves a dual role—it acts as both the task weighting parameter at the loss level and as a coefficient governing the mixture of distributions associated with different tasks.\n\n- As discussed in $\\text{Section 3.2}$, adjusting the task weighting parameter λ leads to a change in the data distribution that the model is adapting to. This occurs due to the fact that varying λ influences the emphasis placed on different tasks during the learning process. We formally define the interpolated distribution in $\\text{Equation 3}$.\n- The equivalence arises from the fact that the model trained on the interpolated distribution $\\mathcal{T}\\_\\text{inter}$ is essentially the same as the model obtained through auxiliary-task learning across the distributions $\\mathcal{T}\\_\\text{tgt}$ and $\\mathcal{T}\\_\\text{aux}$. \n\n**Explanation about distribution.**\nWe emphasize that when referring to distribution, we are specifically addressing joint distribution. This is crucial, as while auxiliary tasks might share the same input space, their output spaces invariably differ due to the inherent nature of diverse tasks.\n\n**Explanation about Figure 6.**\n\n- The mixture of hypotheses is different from the mixture of distributions.  To maintain this distinction, we consistently employ the symbol $\\lambda$ to represent the mixture weights of distributions, while employing $\\Lambda$ to denote the mixture weights of model hypotheses. \n- Regarding $\\text{Figure 6}$, it intends to convey the similarity in outcomes between the mixture of model hypotheses and the mixture of distributions. This alignment in behavior enables us to assert that the mixture of hypotheses is an approximation of the mixture of distributions. \n\n**The difference between Equations 12 and 13.**\n\n$\\text{Equation 13}$ represents a generalized form of $\\text{Equation 12}$, and its generality can be attributed to three key aspects:\n\n- **Customizable Branching**: Unlike $\\text{Equation 12}$ which involves a fixed number of tasks, $\\text{Equation 13}$ allows for a more flexible setting with a user-defined branching number $B$, which allows for scenarios where pruning or modifying the original candidate branches is desired.\n\n- **Enhanced Flexibility in Candidate Branches**: Additionally, $\\text{Equation 13}$ offers a broader perspective on candidate branches. It's not strictly constrained to target-auxiliary task pairs for optimization. Instead, the formulation is open to incorporating any desired custom design of candidate branches, accommodating the integration of prior knowledge or domain-specific insights.\n\n- **Extended Time for Lambda Estimation**: Importantly, $\\text{Equation 13}$ considers the changes in estimating the optimal lambda ($\\lambda$) by extending the time for estimation. We have noticed that methods like Auto-$\\lambda$, which try to estimate $\\lambda$ within a single gradient update, often face challenges due to estimation fluctuations. ForkMerge addresses this by using a longer time for $\\lambda$ estimation, which reduces the negative impact of estimation errors on model parameters and makes the algorithm more robust. \n\n**Q5**: Split of DomainNet.\n\nThanks for the feedback. In our camera-ready version, we will explicitly highlight this split difference in the experimental section. "}, {"review_id": "7fbBIEj0bV", "reviewer": "Reviewer_ECtn", "summary": "This paper considers how to best use auxiliary tasks to improve performance on target tasks. Specifically, a \"ForkMerge\" procedure is proposed which consists of two parallel optimization procedures, one on the target task, and one which includes auxiliary data, and the resulting weights are synchronized at regular intervals. Compared to grid searching for an appropriate interpolation factor, the proposed approach can dynamically alter the interpolation between the two sets of model parameters. Experiments on diverse set of benchmarks and a wide array of baselines show that the proposed approach is effective at improving performance using auxiliary tasks, relative to the state of the art.", "questions": "None.", "limitations": "See \"Weaknesses\"", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper tackles a difficult problem of improving target task performance using auxiliary data.\n\n* The approach is well-motivated and the experiments are quite convincing.\n\n* The justification in terms of reducing distribution shift relative to test data is quite interesting.", "weaknesses": "* The task selection with multiple auxiliary tasks is computationally expensive, and the impact of the pruning procedure is not completely clear.\n\n* There is no adequate discussion of limitations. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer ECtn for providing insightful reviews and valuable comments. We have clarified the questions in the following response.\n\n**Q1:** Impact of pruning strategy.\n\n$\\text{Table 4}$ illustrates the impact of the pruning strategy of ForkMerge. As the number of branches increases, the cumulative benefits derived from auxiliary tasks become more pronounced. However, while the overall gain increases with the number of branches, the individual gain per branch tends to diminish. This phenomenon reveals the trade-off between computational efficiency and task performance. And our pruning strategy is an approach that allows users to customize the number of branches based on their specific needs and available computational resources.\n\n**Q2:** Computational efficiency and limitation.\n\nWe acknowledge the computational demands of task selection involving multiple auxiliary tasks. Indeed, handling such complexity is a challenge that warrants attention. We view the integration of previous task grouping methods [1-3] with our auxiliary task learning approach as a promising avenue for further research. By combining task selection techniques with our method, we envision the potential to alleviate the computational burden while still harnessing the power of auxiliary tasks for improved auxiliary-task learning performance.\n\n[1] *Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. In NeurIPS, 2021.*\n\n[2] *Trevor Standley, Amir Zamir, Dawn Chen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In ICML, 2020.*\n\n[3] *Amir Roshan Zamir, Alexander Sax, William B. Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In CVPR, 2018.*"}]

Author rebuttal:
We would like to sincerely thank all the reviewers for providing insightful reviews and valuable comments. Your reviews are of great importance to us in improving the quality of this work.

**In this global rebuttal, we aim to clarify the common questions from reviewers, and we have responded to each reviewer with a separate response for other questions. The full results of additional experiments are attached in the one-page pdf.**

**Q1:** How do the new findings motivate the proposed ForkMerge?

Below we outline the motivations behind the design of ForkMerge in light of our new findings.

- **[Finding]** $\text{Section 3.1}$ reveals that the presence of gradient conflict does not necessarily lead to negative transfer, as long as the hyperparameter $\lambda$ is appropriately tuned. Additionally, $\text{Section 3.2}$ emphasizes the importance of considering generalization to mitigate negative transfer effectively. **[Algorithm Design]** Based on these findings, we opt to dynamically adjust the hyperparameter $\lambda$ according to the target validation performance in ForkMerge ($\text{Section 4.1}$). 
- **[Finding]** $\text{Section 3}$ highlights that in scenarios where weak negative transfer (WNT) occurs, selecting an appropriate value for $\lambda$ can alleviate the problem. However, in cases of strong negative transfer (SNT), setting $\lambda$ to $0$ becomes necessary. **[Algorithm Design]** In each merging step of ForkMerge, we perform a search step to identify the optimal value of $\lambda$, which effectively mitigates weak negative transfer. Furthermore, for instances of strong negative transfer, ForkMerge is able to set $\lambda$ to 0 to prevent negative transfer ($\text{Section 4.1}$). Additionally, we have introduced a pruning mechanism to remove SNT forking branches, thus reducing the computation cost ($\text{Section 4.2}$).
- **[Finding]** $\text{Section 3.2}$ indicates that negative transfer is likely to occur when the introduced auxiliary task enlarges the distribution shift between the training and test data for the target task. To address this issue, it is crucial to select auxiliary tasks that decrease the distribution shift between training and test data for the target task. **[Algorithm Design]** To address the distribution shift problem, the general form of ForkMerge constructs mixture distributions that comprise diverse data shifts relative to the target distribution. Subsequently, models trained on these different distributions are combined dynamically to approach the optimal parameters ($\text{Section 4.2}$). 

**Q2:** Concern about the computation cost and trade-off between efficiency and accuracy.

**Clarification on the Computation Cost.**

Firstly, we have developed several techniques to reduce computation cost. Below, we provide a detailed explanation:

- **Pruning Strategy:** As introduced in $\text{Section 4.2}$, we can prune the forking branches with $\Lambda_k=0$ and only keep the branches with the largest $K'<K$ values in $\Lambda$ after the early merge step, where $K$ represents the total number of tasks.
  - To illustrate the effectiveness of the pruning strategy, we present results in $\text{Section 5.2}$ on Auxiliary-Domain Image Recognition and CTR and CTCVR Prediction tasks. For instance, on the CTR and CTCVR Prediction task, we initially construct up to $8$ branches with different task weights, but after the first merge step, we prune them to $3$ branches, achieving considerable computational savings.
- **Greedy Strategy in the Merge Step:** In $\text{Algorithm 2 of Appendix A.2}$, we introduce a greedy strategy during the merge step. This modification reduces the computation complexity from exponential to linear complexity when searching for optimal task weighting.
- **Validation Set Sampling:** As mentioned in $\text{Appendix A.1}$, the costs associated with estimating validation performance $\hat{\mathcal{P}}$ in the search step are usually negligible. However, when the validation set size is relatively large, we can resort to sampling to reduce the computational cost further.

Further, we have conducted analysis of the computation cost in $\text{Appendix D.2}$:

- Although only one model is optimized in most previous auxiliary-task learning methods, their computational costs are not necessarily $\mathcal{O}(1)$. For example, gradient balancing methods require computing gradients of each task, thus leading to $\mathcal{O}(K)$ complexity. In addition, calculating the inner product or norm of the gradients will bring a calculation cost proportional to the number of network parameters.
- To support our claims, we access the actual training time across methods on NYUv2. As depicted in $\text{Figure 4 of Appendix D.2}$, ForkMerge does not require more training time than other auxiliary task learning methods, including GCS, OL_AUX, ARML, and Auto-$\lambda$.

**Memory Utilization.**
In terms of memory usage, it's important to note that the optimization of each branch within ForkMerge is entirely independent. This enables us to load only the model parameters corresponding to the particular branch being trained at any given time. Consequently, the storage requirements are comparable to those of single task learning (STL), resulting in minimal memory overhead.

**Trade off between Efficiency and Accuracy.**

In $\text{Section 5.2}$, we provide an analysis on DomainNet to explore the trade-off between efficiency and accuracy. Results in $\text{Table 4}$ reveal that as the number of branches increases, the gain by auxiliary tasks will enlarge while the gain brought by each branch will reduce.

In this paper, we propose to address this trade-off through the pruning strategy, which allows users to customize the number of branches based on their specific needs and available computational resources.

Meta-review: The paper presents a way to improve target task learning via auxiliary tasks. In particular, the “ForkMerge” procedure trains two parallel branches, one on target task and another with target + auxiliary task. The weights of the two branches are linearly interpolated based on validation performance. Experiments show the efficacy of the proposed method over several settings.

The paper is predominantly empirical and the reviewers were initially split in their opinions. There were concerns about the validity of the results with respect to larger datasets/architectures, sensitivity to hyper-parameters, and motivation of the method. Most concerns are sufficiently addressed during the discussion. Most reviewers recommended acceptance of the paper.
Decision: Accept (poster)