Title: ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Auxiliary-Task Learning

Negative Transfer
Abstract: Auxiliary-Task Learning (ATL) aims to improve the performance of the target task by leveraging the knowledge obtained from related tasks. Occasionally, learning multiple tasks simultaneously results in lower accuracy than learning only the target task, which is known as negative transfer. This problem is often attributed to the gradient conflicts among tasks, and is frequently tackled by coordinating the task gradients in previous works. However, these optimization-based methods largely overlook the auxiliary-target generalization capability. To better understand the root cause of negative transfer, we experimentally investigate it from both optimization and generalization perspectives. Based on our findings, we introduce ForkMerge, a novel approach that periodically forks the model into multiple branches, automatically searches the varying task weights by minimizing target validation errors, and dynamically merges all branches to filter out detrimental task-parameter updates. On a series of auxiliary-task learning benchmarks, ForkMerge outperforms existing methods and effectively mitigates negative transfer.

ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning

Abstract

Auxiliary-Task Learning (ATL) aims to improve the performance of the target task by leveraging the knowledge obtained from related tasks. Occasionally, learning multiple tasks simultaneously results in lower accuracy than learning only the target task, which is known as negative transfer. This problem is often attributed to the gradient conflicts among tasks, and is frequently tackled by coordinating the task gradients in previous works. However, these optimization-based methods largely overlook the auxiliary-target generalization capability. To better understand the root cause of negative transfer, we experimentally investigate it from both optimization and generalization perspectives. Based on our findings, we introduce ForkMerge , a novel approach that periodically forks the model into multiple branches, automatically searches the varying task weights by minimizing target validation errors, and dynamically merges all branches to filter out detrimental task-parameter updates. On a series of auxiliary-task learning benchmarks, ForkMerge outperforms existing methods and effectively mitigates negative transfer.

1 Introduction

Deep neural networks have achieved remarkable success in various machine learning applications, such as computer vision [23, 22], natural language processing [62, 11, 57], and recommendation systems [46]. However, one major challenge in training deep neural networks is the scarcity of labeled data. In recent years, Auxiliary-Task Learning (ATL) has emerged as a promising technique to address this challenge [67, 39, 43]. ATL improves the generalization of target tasks by leveraging the useful signals provided by some related auxiliary tasks. For instance, larger-scale tasks, such as user click prediction, can be utilized as auxiliary tasks to improve the performance of smaller-scale target tasks, such as user conversion prediction in recommendation [47, 36]. Self-supervised tasks on unlabeled data can serve as auxiliary tasks to improve the performance of the target task in computer vision and natural language processing, without requiring additional labeled data [34, 69, 11, 3].

However, in practice, learning multiple tasks simultaneously sometimes leads to performance degradation compared to learning only the target task, a phenomenon known as negative transfer [84, 75]. Even in large language models, negative transfer problems may still exist. For example, RLHF [7], a key component of ChatGPT [57], achieves negative effects on nearly half of the multiple-choice question tasks when post-training GPT-4 [58]. There has been a significant amount of methods proposed to mitigate negative transfer in ATL [71, 79, 15, 39]. Notable previous studies attribute negative transfer to the optimization difficulty, especially the gradient conflicts between different tasks, and propose to mitigate negative transfer by reducing interference between task gradients

[79, 15]. Other works focus on selecting the most relevant auxiliary tasks and reducing negative transfer by avoiding task groups with severe task conflicts [71, 17]. However, despite the significant efforts to address negative transfer, its underlying causes are still not fully understood.

In this regard, we experimentally analyze potential causes of negative transfer in ATL from the perspectives of optimization and generalization. From an optimization view, our experiments suggest that gradient conflicts do not necessarily lead to negative transfer . For example, weight decay, a special auxiliary task, can conflict with the target task in gradients but still be beneficial to the target performance. From a generalization view, we observe that negative transfer is more likely to occur when the distribution shift between the multi-task training data and target test data is enlarged.

Based on our above findings, we present a new approach named ForkMerge . Since we cannot know which task distribution combination leads to better generalization in advance, and training models for each possible distribution is prohibitively expensive, we transform the problem of combining task distributions into that of combining model hypotheses. Specifically, we fork the model into multiple branches and optimize the parameters of different branches on diverse data distributions by varying the task weights. Then at regular intervals, we merge and synchronize the parameters of each branch to approach the optimal model hypothesis. In this way, we will filter out harmful parameter updates to mitigate negative transfer and keep desirable parameter updates to promote positive transfer.

The contributions of this work are summarized as follows: (1) We systematically identify the problem and analyze the causes of negative transfer in ATL. (2) We propose ForkMerge , a novel approach to mitigate negative transfer and boost the performance of ATL. (3) We conduct extensive experiments and validate that ForkMerge outperforms previous methods on a series of ATL benchmarks.

2 Related Work

2.1 Auxiliary-Task Learning

Auxiliary-Task Learning (ATL) enhances a model's performance on a target task by utilizing knowledge from related auxiliary tasks. The two main challenges in ATL are selecting appropriate auxiliary tasks and optimizing them jointly with the target task. To find the proper auxiliary tasks for ATL, recent studies have explored task relationships by grouping positively related tasks together and assigning unrelated tasks to different groups to avoid task interference [81, 71, 17, 70]. Once auxiliary tasks are determined, most ATL methods create a unified loss by linearly combining the target and auxiliary losses. However, choosing task weights is challenging due to the exponential increase in search space with the number of tasks, and fixing the weight of each task loss can lead to negative transfer [32]. Recent studies propose various methods to automatically choose task weights, such as using one-step or multi-step gradient similarity [15, 39, 9], minimizing representation-based task distance [2] or gradient gap [67], employing a parametric cascade auxiliary network [54], or from the perspective of bargaining game [66]. However, these methods mainly address the optimization difficulty after introducing auxiliary tasks and may overlook the generalization problem.

Recently, AANG [10] formulates a novel searching space of auxiliary tasks and adopts the metalearning technique, which prioritizes target task generalization, to learn single-step task weightings. This parallel finding highlights the importance of the target task generalization and we further introduce the multi-step task weightings to reduce the estimation uncertainty. Another parallel method, ColD Fusion [12], explores collaborative multitask learning and proposes to fuse each contributor's parameter to construct a shared model. In this paper, we further take into account the diversity of tasks and the intricacies of task relationships and derive a method for combining model parameters from the weights of task combinations.

2.2 Multi-Task Learning

Different from ATL, Multi-Task Learning (MTL) aims to improve the performance of all tasks by learning multiple objectives from a shared representation. To facilitate information sharing and minimize task conflict, many multi-task architectures have been designed, including hard-parameter sharing [30, 22, 24] and soft-parameter sharing [51, 64, 16, 46, 44, 48, 72]. Another line of work aims to optimize strategies to reduce task conflict. Methods such as loss balancing and gradient balancing propose to find suitable task weighting through various criteria, such as task uncertainty [28], task loss magnitudes [44], gradient norm [5], and gradient directions [79, 6, 40, 41, 25, 55].

Although MTL methods can be directly used to jointly train auxiliary and target tasks, the asymmetric task relationships in ATL are usually not taken into account in MTL.

2.3 Negative Transfer

Negative Transfer (NT) is a widely existing phenomenon in machine learning, where transferring knowledge from the source data or model can have negative impact on the target learner [63, 60, 27]. To mitigate negative transfer, domain adaptation methods design importance sampling or instance weighting strategies to prioritize related source data [75, 83]. Fine-tuning methods filter out detrimental pre-trained knowledge by suppressing untransferable spectral components in the representation [4]. MTL methods use gradient surgery or task weighting to reduce the gradient conflicts across tasks [79, 76, 25, 42]. Different from previous work, we propose to dynamically filter out harmful parameter updates in the training process to mitigate negative transfer. Besides, we provide an in-depth experimental analysis of the causes of negative transfer in ATL, which is rare in this field yet will be helpful for future research.

3 Negative Transfer Analysis

Problem and Notation. In this section, we assume that both the target task T tgt and the auxiliary task T aux are given. Then the objective is to find model parameters θ that achieve higher performance on the target task by joint training with the auxiliary task,

where L is the training loss, and λ is the relative weighting hyper-parameter between the auxiliary task and the target task. Our final objective is max θ [ P ( θ )] , where P is the relative performance measure for the target task T tgt, such as the accuracy in classification. Next we define the Transfer Gain to measure the impact of T aux on T tgt .

Definition 3.1 ( Transfer Gain, TG ) . Denote the model obtained by some ATL algorithm A as θ A ( T tgt , T aux , λ ) and the model obtained by single-task learning on target task as θ ( T tgt ) . Let P be the performance measure on the target task T tgt. Then the algorithm A can be evaluated by

Going beyond previous work on Negative Transfer (NT) [75, 84], we further divide negative transfer in ATL into two types.

Definition 3.2 ( Weak Negative Transfer, WNT ) . For some ATL algorithm A with weighting hyper-parameter λ , weak negative transfer occurs if TG ( λ, A ) < 0 .

Definition 3.3 ( Strong Negative Transfer, SNT ) . For some ATL algorithm A , strong negative transfer occurs if max λ> 0 TG ( λ, A ) < 0 .

Figure 1 illustrates the difference between weak negative transfer and strong negative transfer. The most essential difference is that we might be able to avoid weak negative transfer by selecting a proper weighting hyper-parameter λ , yet we cannot avoid strong negative transfer in this way.

Next, we will analyze negative transfer in ATL from two different perspectives: optimization and generalization . We conduct our analysis on a multi-domain image recognition dataset DomainNet [61] with ResNet-18 [23] pre-trained on ImageNet. Specifically, we use task Painting and Quickdraw in DomainNet as target tasks respectively to showcase weak negative transfer and strong negative transfer, and mix all other tasks in DomainNet as auxiliary tasks. We will elaborate on the DomainNet dataset in Appendix C.3 and provide the detailed experiment design in Appendix B.

3.1 Effect of Gradient Conflicts

It is widely believed that gradient conflicts between different tasks will lead to optimization difficulties [79, 40], which in turn lead to negative transfer. The degree of gradient conflict is usually measured by the Gradient Cosine Similarity [79, 76, 15].

λ

TG

0

WNT

SNT

λ

TG

0

WNT

SNT

Definition 3.4 ( Gradient Cosine Similarity, GCS ) . Denote ϕ ij as the angle between two task gradients g i and g j , then we define the gradient cosine similarity as cos ϕ ij and the gradients as conflicting when cos ϕ ij < 0 .

In Figure 2, we plot the correlation curve between gradient cosine similarity and transfer gain. Somewhat counterintuitively, we observe that negative transfer and gradient conflicts are not strongly correlated, and negative transfer might be severer when the task gradients are highly consistent.

It seems contradictory to the previous work [79, 15] and the reason is that previous work mainly considers the optimization convergence during training, while in our experiments we further consider the generalization during evaluation (transfer gain is estimated on the validation set). Although the conflicting gradient of the auxiliary task will increase the training loss of the target task and slow down its convergence speed [37], it may also play a role similar to regularization [32], reducing the over-fitting of the target task, thereby reducing its generalization error. To confirm our hypothesis, we repeat the above experiments with the auxiliary task replaced by L 2 regularization and observe a similar phenomenon as shown in Figure 2(c)-(d), which indicates that the gradient conflict in ATL is not necessarily harmful, as it may serve as a proper regularization.

Figure 2 also indicates that the weighting hyper-parameter λ in ATL has a large impact on negative transfer. A proper λ not only reduces negative transfer but also promotes positive transfer.

(a)  T arget: P

Auxiliary: DomainNet

(b)  T arget: Q

Auxiliary: DomainNet

(c)  T arget: P

Auxiliary: L2 Regularization

(d)  T arget: Q

Auxiliary: L2 Regularization

λ = 1/16

λ = 1/8

λ = 1/4

λ = 1/2

λ = 1

3.2 Effect of Distribution Shift

Next, we will analyze negative transfer from the perspective of generalization. We notice that adjusting λ will change the data distribution that the model is fitting. For instance, when λ = 0 , the model only fits the data distribution of the target task, and when λ = 1 , the model will fit the interpolated distribution of the target and auxiliary tasks. Formally, given the target distribution T tgt and the auxiliary distribution T aux, the interpolated distribution of the target and auxiliary task is T inter ,

where λ is the task-weighting hyper-parameter. Figure 3(a) quantitatively visualizes the distribution shift under different λ using t-SNE [74].

To quantitatively measure the distribution shift in ATL, we introduce the following definitions. Following the notations of [53], we consider multiclass classification with hypothesis space F of scoring functions f : X × Y → R where f ( x, y ) indicates the confidence of predicting x as y .

Definition 3.5 ( Confidence Score Discrepancy, CSD ) . Given scoring function hypothesis F , denote the optimal hypothesis on distribution D as f ∗ D , then confidence score discrepancy between

distribution D and D ′ induced by F is defined by

Confidence score discrepancy between training and test data indicates how unconfident the model is on the test data, which is expected to increase when the data shift enlarges [59, 50].

Q

(SNT)

λ

=1

λ

=0

λ

=1/16

λ

=1/4

P

(WNT)

Test Data

Training Data for Target/Auxiliary

(b)

(a)

Figure 3(b) indicates the correlation between confidence score discrepancy and transfer gain. For weak negative transfer tasks, when λ increases at first, the introduced auxiliary tasks will shift the training distribution towards the test distribution, thus decreasing the confidence score discrepancy between training and test data and improving the generalization of the target task. However, when λ continues to increase, the distribution shift gradually increases, finally resulting in negative transfer. For strong negative transfer tasks, there is a large gap between the distribution of the introduced auxiliary tasks and that of the target task. Thus, increasing λ always enlarges confidence score discrepancy and always leads to negative transfer. In summary,

4 Methods

In Section 4.1, based on our above analysis, we will introduce how to mitigate negative transfer when the auxiliary task is determined. Then in Section 4.2, we will further discuss how to use the proposed method to select appropriate auxiliary tasks and optimize them jointly with the target task simultaneously.

4.1 ForkMerge

In this section, we assume that the auxiliary task T aux is given. When updating the parameters θ t with Equation (1) at training step t , we have

where η is the learning rate, g tgt and g aux are the gradients calculated from L tgt and L aux respectively. Section 3.1 reveals that the gradient conflict between g tgt and g aux does not necessarily lead to negative transfer as long as λ is carefully tuned and Section 3.2 shows that negative transfer is related to generalization. Thus we propose to dynamically adjust λ according to the target validation performance ̂ P to mitigate negative transfer:

Merge

fork

Initial Parameter

Final Parameter

Fork

Independent

Optimization

min

!

ℒ

"#"

(&)

min

!

ℒ

"#"

& +ℒ

$%&

&

Δ( ×

∆( ×

*

∆(

×

&

0

&

0

&

01∆0

(0)

&

01∆0

(1)

&

01∆0

∗

Equation (6) is a bi-level optimization problem. One common approach is to first approximate ̂ P with the loss of a batch of data on the validation set, and then use first-order approximation to solve λ [18, 43]. However, these approximations within a single step of gradient descent introduce large noise to the estimation of λ and also increase the risk of over-fitting the validation set. To tackle these issues, we first rewrite Equation (6) equally as

where θ t +1 (0) = θ t -η g tgt ( θ t ) and θ t +1 (1) = θ t -η ( g tgt ( θ t ) + g aux ( θ t )) . The proof is in Appendix A.1. Note that we assume the optimal λ ∗ satisfies 0 ≤ λ ∗ ≤ 1 , which can be guaranteed by increasing the scale of L aux when necessary. Yet an accurate estimation of performance ̂ P in Equation (7) is still computationally expensive and prone to over-fitting, thus we extend the one gradient step to ∆ t steps,

Algorithm. As shown in Figure 4 and Algorithm 1, the initial model parameters θ t at training step t will first be forked into two branches. The first one will be optimized only with the target task loss L tgt for ∆ t iterations to obtain θ t +∆ t (0) , while the other one will be jointly trained for ∆ t iterations to obtain θ t +∆ t (1) . Then we will search the optimal λ ∗ that linearly combines the above two sets of parameters to maximize the validation performance ̂ P . When weak negative transfer occurs in the joint training branch, we can select a proper λ ∗ between 0 and 1 . And when strong negative transfer occurs, we can simply set λ ∗ to 0 . Finally, the newly merged parameter θ ∗ t +∆ t = (1 -λ ∗ ) θ t +∆ t (0) + λ ∗ θ t +∆ t (1) will join in a new round, being forked into two branches again and repeating the optimization process for ⌈ T ∆ t ⌉ times.

Discussion. Compared to grid searching λ , which is widely used in practice, ForkMerge can dynamically transfer knowledge from auxiliary tasks to the target task during training with varying λ ∗ . In terms of computation cost, ForkMerge has a lower complexity as it only requires training 2 branches while grid searching has a cost proportional to the number of hyper-parameters to be searched.

4.2 ForkMerge for Task Selection Simultaneously

When multiple auxiliary tasks are available, we can simply mix all the auxiliary tasks together to form a single auxiliary task. This simple strategy actually works well in most scenarios (see Section 5.2) and is computationally cheap. However, when further increasing the performance is desired, we can also dynamically select the optimal weighting for each auxiliary task. Formally, the objective

Algorithm 1 ForkMerge Training Pipeline.

when optimizing the model for the target task T 0 with multiple auxiliary tasks {T k } K k =1 is

where ∑ K k =1 λ k ≤ 1 and ∀ k, λ k ≥ 0 . Using gradient descent to update θ t at training step t , we have

where λ 0 = 1 . Given K task-weighting vectors { ω k } K k =0 that satisfies ω k i = ✶ [ i = k or i = 0] , i.e., the k -th and 0 -th dimensions of ω k are 1 and the rest are 0 , and a vector Λ that satisfies

̸

then optimizing λ ∗ in Equation (10) is equivalent to

In Equation (12), the initial model parameters are forked into K +1 branches, where one branch is optimized with the target task, and the other branches are jointly optimized with one auxiliary task and the target task. Then we find the optimal Λ ∗ that linearly combines the K + 1 sets of parameters to maximize the validation performance (see proof of Equation (12) and the detailed algorithm in Appendix A.2). The training computational complexity of Equation 12 is O ( K ) , which is much lower than the exponential complexity of grid searching, but still quite large. Inspired by the early-stop approximation used in task grouping methods [71], we can prune the forking branches with Λ k = 0 (strong negative transfer) and only keep the branches with the largest K ′ < K values in Λ after the early merge step. In this way, those useless branches with irrelevant auxiliary tasks can be stopped early. Additionally, we introduce a greedy search strategy in Algorithm 3 to further reduce the computation complexity when grid searching all possible values of Λ .

Lastly, we introduce a general form of ForkMerge. Assuming B candidate branches with taskweighting vectors ν b ( b = 1 , . . . , B ), the goal is to optimize Λ ∗ :

From a generalization view, the mixture distributions constructed by different ν lead to diverse data shifts from the target distribution, yet we cannot predict which ν leads to better generalization. Thus, we transform the problem of mixture distribution into that of mixture hypothesis [49] and the models trained on different distributions are combined dynamically via Λ ∗ to approach the optimal parameters. Here, Equation 12 is a particular case by substituting B = K +1 and ν b i = ✶ [ i = b -1 or i = 0] . By comparison, Equation 13 allows us to introduce human prior into ForkMerge by constructing more efficient branches, and also provides possibilities for combining ForkMerge with previous task grouping methods [81, 71, 17]. The detailed algorithm of Equation 13 can be found in Algorithm 2.

5 Experiments

We evaluate the effectiveness of ForkMerge under various settings, including multi-task learning, multi-domain learning, and semi-supervised learning. First, in Section 5.1, we illustrate the prevalence of negative transfer and explain how ForkMerge can mitigate this problem. In Section 5.2, We examine whether ForkMerge can mitigate negative transfer when joint training the auxiliary and target tasks, and compare it with other methods. In Section 5.3, we further use ForkMerge for task selection simultaneously. Experiment details can be found in Appendix C. We will provide additional analysis and comparison experiments in Appendix D. The codebase for both our method and the compared methods will be available at https://github.com/thuml/ForkMerge .

̸

5.1 Motivation Experiment

Negative Transfer is widespread across different tasks. In Figure 5 (a), we visualize the transfer gains between 30 task pairs on DomainNet, where the auxiliary and target tasks are equally weighted, and we observe that negative transfer is common in such case ( 23 of 30 combinations lead to negative transfer). Besides, as mentioned in Definition 3.2 and 3.3, whether negative transfer occurs is related to a specific ATL algorithm, in Figure 5 (b), we observe that negative transfer in all 30 combinations can be successfully avoided when we use ForkMerge algorithm. This observation further indicates the limitation of task grouping methods [71, 17], since they use Equal Weight between tasks and may discard some useful auxiliary tasks.

Mixture of hypotheses is an approximation of mixture of distribution. Figure 6 uses the ternary heatmaps to visualize the linear combination of a set of three models optimized with different task weightings for 25 K iterations, including a single-task model and two multi-task models. Similar to mixing distributions for weak negative transfer task Painting (see Figure 3), the transfer gain when mixing models P ainting and P ainting+ R eal first increases and then decreases. Also similar to mixing distributions for strong negative transfer task Q uickdraw, the transfer gain when mixing models Q uickdraw and Q uickdraw+ R eal decreases monotonically. Besides, Figure 6 also indicates a good property of deep models: the loss surfaces of over-parameterized deep neural networks are quite well-behaved and smooth after convergence, which has also been mentioned by previous works [20, 35] and provides an intuitive explanation of the merge step in ForkMerge.

-

-0.9%

-0.8%

-14%

-1.5%

+0.7%

-1.0%

-2.0%

-20%

-2.8%

-2.1%

-

+1.4% -2.1%

-13%

-1.5%

-0.7%

-

-3.1%

-8.2%

-6.9%

-6.9%

-5.6%

-

+1,8% +1.4% +1.3%

-15%

+0.7%

-

+2.4% -1.6%

-0.4%

-8.6%

-2.1%

-

Target Tasks

C

I

P

Q

R

S

Auxiliary Tasks

C

I

P

Q

R

S

-

+4.6% +2.1% +0.0% +1.2% +2.1%

+1.8%

+1.2% +0.0% +0.4% +1.0%

-

+2.4% +2.6%

+0.0% +0.5% +1.7%

-

+0.7% +2.2% +1.1%

+0.1% +0.3%

-

+2.0% +4.8% +2.2%

+0.1%

+1.8%

-

+3.5% +3.6% +2.5%

+0.0% +0.5%

-

Target Tasks

C

I

P

Q

R

S

Auxiliary Tasks

C

I

P

Q

R

S

(a) Equal Weight

(b) ForkMerge

5.2 Use ForkMerge for Joint Optimization

First, we use ForkMerge only for joint training of the target and auxiliary tasks. When datasets contain multiple tasks, we will mix all tasks together to form a single auxiliary task for ForkMerge. Yet for the compared methods, a distinction is still made between different tasks for better performance.

Specifically, we compare ForkMerge with: (1) Single Task Learning (STL); (2) EW, which assigns equal weight to all tasks; (3) GCS [15], an ATL approach using gradient similarity between target and auxiliary tasks; (4) OL_AUX [39], an ATL approach adjusting the loss weight based on gradient inner product; (5) ARML [67], an ATL approach adjusting the loss weight based on gradient difference; (6) Autoλ [43], an ATL method that estimates loss weight through finite-difference approximation [18]; (7) Post-train, an ATL method that pre-trains the model on all tasks and then fine-tunes it for each task separately. (8) UW [28], which adjusts weights based on task uncertainty; (9) DWA [44], which adjusts weights based on loss change; (10) MGDA [65], which computes a convex combination of gradients with a minimum norm to balance tasks; (11) GradNorm [5], which rescales the gradient norms of different tasks to the same range; (12) PCGrad [79], which eliminates conflicting gradient components; (13) IMTL [41], which uses an update direction with equal projections on task gradients; (14) CAGrad [40], which optimizes for the average loss and minimum decrease rate across tasks; (15) NashMTL [55], which combines the gradients using the Nash bargaining solution. Since different tasks have varying evaluation metrics, we will report the average per-task performance improvement for each method using ∆ m , as defined in Appendix C.1.

(a) P (WNT)

(b) Q (SNT)

Optimum

P+C

P+R

Q

Q+R

P

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

w

R

w

R

w

C

w

C

3.315

2.588

1.861

1.133

0.406

-0.321

-1.049

0.000

-

0.212

-

0.424

-

0.637

-

0.849

-

1.061

-

1.273

Auxiliary-Task Scene Understanding. We evaluate on the widely-used multi-task scene understanding dataset, NYUv2 [68], which contains 3 tasks: 13 -class semantic segmentation, depth estimation, and surface normal prediction. Following [55], we use 636 , 159 and 654 images for training, validation, and test. Our implementation is based on LibMTL [38] and MTAN [44]. The results are presented in Table 1. Negative transfer is not severe on this dataset, where both segmentation and depth benefit from ATL and only normal task gets worse. In such cases, our method still achieves significant improvement on all tasks. We also find that Post-train serves as a strong baseline in most of our ATL experiments. Its drawback is that it fails to consider the task relationship in the pre-training phase, and suffers from catastrophic forgetting during the fine-tuning process.

Auxiliary-Domain Image Recognition. Further, we evaluate on the widely-used multi-domain image recognition dataset, DomainNet [61], which contains 6 diverse visual domains and approximately 0 . 6 million images distributed among 345 categories, where the task difference is reflected in the marginal distribution. Our implementation is based on TLlib [26]. As the original DomainNet does not provide a separate validation set, we randomly split 50% data from the test set as the validation set. The results are presented in Table 2. DomainNet contains both positive transfer tasks ( C lipart), weak negative transfer tasks ( I nfograph, P ainting, R eal, S ketch), and strong negative transfer tasks ( Q uickdraw). When negative transfer occurs, previous ATL methods lead to severe performance degradation, while our method can automatically avoid strong negative transfer and improve the performance over STL in other cases.

5.3 Use ForkMerge for Task Selection Simultaneously

As mentioned in Section 4.2, when there are multiple auxiliary task candidates, we can use ForkMerge to simultaneously select auxiliary tasks and jointly train them with the target task, which is denoted as ForkMerge ‡ .

Auxiliary-Task Scene Understanding. In NYUv2, we have 2 auxiliary tasks for any target task, thus we can construct 3 branches with different task weights in Equation 12. In this way, we are able to select auxiliary tasks adaptively by learning different Λ for different branches in the merge step. As shown in Table 3, this strategy yields better overall performance.

Auxiliary-Domain Image Recognition. For any target task in DomainNet, we can construct up to 6 branches with different task weights in Equation 12, which is computationally expensive. As mentioned in Section 4.2, we will prune the branches after the first merge step to reduce the computation cost. Table 4 reveals the impact of the pruning strategy. As the number of branches increases, the gain brought by auxiliary tasks will enlarge, while the gain brought by each branch will reduce. Therefore, pruning is an effective strategy to achieve a better balance between performance and efficiency. In practical terms, when confronted with multiple auxiliary tasks, users have the flexibility to tailor the number of branches to align with their available computational resources.

CTR and CTCVR Prediction. We evaluate on AliExpress dataset [36], a recommendation dataset from the industry, which consists of 2 tasks: CTR and CTCVR, and 4 scenarios and more than 100 M records. Our implementation is based on MTReclib [85]. For any target task in AliExpress, we can construct up to 8 branches with different task weights, and we prune to 3 branches after the first merge

step. The results are presented in Table 5. Note that improving on such a large-scale dataset with auxiliary tasks is quite difficult. Still, ForkMerge achieves the best performance with ∆ m = 1.30% .

Semi-Supervised Learning (SSL). We also evaluate on two SSL datasets, CIFAR-10 [31] and SVHN [56]. Following [67], we use Self-supervised Semi-supervised Learning (S4L) [82] as our baseline algorithm and use 2 self-supervised tasks, Rotation [19] and Exempler-MT [14], as our auxiliary tasks. Table 6 presents the test error of S4L using different ATL approaches, along with other SSL methods, and shows that ForkMerge consistently outperforms the compared ATL methods. Note that we do not aim to propose a novel or state-of-the-art SSL method in this paper. Instead, we find that some SSL methods use ATL and the auxiliary task weights have a great impact (see Grid Search in Table 6). Thus, we use ForkMerge to improve the auxiliary task training within the context of SSL.

6 Conclusion

Methods have been proposed to mitigate negative transfer in auxiliary-task learning, yet there still lacks an in-depth experimental analysis on the causes of negative transfer. In this paper, we systematically delved into the negative transfer issues and presented ForkMerge, an approach to enable auxiliary-task learning with positive transfer gains. Experimentally, ForkMerge achieves stateof-the-art accuracy on four different auxiliary-task learning benchmarks, while being computationally efficient. We view the integration of previous task grouping methods with our auxiliary task learning approach as a promising avenue for further research.

References

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV ,

Methods

STL

Π

-Model [33]

Mean Teacher [73]

VAT [52]

Pseudo-Label [34]

S4L + EW

S4L + GradNorm

S4L + GCS

S4L + OL_AUX

S4L + ARML

S4L + Auto-

λ

S4L + Post-train

S4L + Grid Search

S4L + ForkMerge

‡

CIFAR-10

(4000 labels)

20.3

16.4

15.9

13.9

17.8

15.7

14.1

15.0

16.1

13.7

14.2

15.8

13.8

13.1

SVHN

(1000 labels)

12.80

7.19

5.65

5.63

7.62

7.83

7.68

7.02

7.82

5.89

6.14

7.85

6.07

5.49

∆

m

↑

-

31.5%

38.8%

43.8%

26.4%

30.7%

35.3%

35.6%

29.8%

43.2%

41.0%

30.4%

42.3%

46.3%

2018.

Shuxiao Chen, Koby Crammer, Hangfeng He, Dan Roth, and Weijie J Su. Weighted training for cross-task learning. In ICLR , 2022.

Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self-supervised models are strong semi-supervised learners. In NeurIPS , 2020.

Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long, and Jianmin Wang. Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning. In NeurIPS , 2019.

Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In ICML , 2018.

Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. In NeurIPS , 2020.

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In NeurIPS , 2017.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR , 2009.

Lucio M Dery, Yann Dauphin, and David Grangier. Auxiliary task update decomposition: The good, the bad and the neutral. In ICLR , 2021.

Lucio M Dery, Paul Michel, Mikhail Khodak, Graham Neubig, and Ameet Talwalkar. Aang: Automating auxiliary learning. In ICLR , 2023.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL , 2019.

Shachar Don-Yehiya, Elad Venezian, Colin Raffel, Noam Slonim, Yoav Katz, and Leshem Choshen. Cold fusion: Collaborative descent for distributed multitask finetuning. arXiv preprint arXiv:2212.01378 , 2022.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR , 2020.

Alexey Dosovitskiy, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox. Discriminative unsupervised feature learning with convolutional neural networks. In NeurIPS , 2014.

Yunshu Du, Wojciech M Czarnecki, Siddhant M Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, and Balaji Lakshminarayanan. Adapting auxiliary losses using gradient similarity. arXiv preprint arXiv:1812.02224 , 2018.

Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR , abs/1701.08734, 2017.

Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. In NeurIPS , 2021.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML , 2017.

Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In ICLR , 2018.

Ian Goodfellow, Oriol Vinyals, and Andrew Saxe. Qualitatively characterizing neural network optimization problems. In ICLR , 2015.

Priya Goyal, Piotr Dollár, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch SGD: training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 , 2017.

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV , 2017.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , 2016.

Falk Heuer, Sven Mantowsky, Saqib Bukhari, and Georg Schneider. Multitask-centernet (mcn): Efficient and diverse multitask learning using an anchor free approach. In ICCV , 2021.

Adrián Javaloy and Isabel Valera. Rotograd: Gradient homogenization in multitask learning. In ICLR , 2022.

Junguang Jiang, Baixu Chen, Bo Fu, and Mingsheng Long. Transfer-learning-library. https: //github.com/thuml/Transfer-Learning-Library , 2020.

Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05867 , 2022.

Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In CVPR , 2018.

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2015.

Iasonas Kokkinos. Ubernet: Training a 'universal' convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. In CVPR , 2017.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto , 2009.

Vitaly Kurin, Alessandro De Palma, Ilya Kostrikov, Shimon Whiteson, and M Pawan Kumar. In defense of the unitary scalarization for deep multi-task learning. In NeurIPS , 2022.

Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In ICLR , 2017.

Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML , 2013.

Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In NeurIPS , 2018.

Pengcheng Li, Runze Li, Qing Da, An-Xiang Zeng, and Lijun Zhang. Improving multi-scenario learning to rank in e-commerce by exploiting task relationships in the label space. In CIKM , 2020.

Baijiong Lin, YE Feiyang, Yu Zhang, and Ivor Tsang. Reasonable effectiveness of random weighting: A litmus test for multi-task learning. In TMLR , 2022.

Baijiong Lin and Yu Zhang. LibMTL: A python library for multi-task learning. arXiv preprint arXiv:2203.14338 , 2022.

Xingyu Lin, Harjatin Baweja, George Kantor, and David Held. Adaptive auxiliary task weighting for reinforcement learning. In NeurIPS , 2019.

Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. In NeurIPS , 2021.

L Liu, Y Li, Z Kuang, J Xue, Y Chen, W Yang, Q Liao, and Wayne Zhang. Towards impartial multi-task learning. In ICLR , 2021.

Shengchao Liu, Yingyu Liang, and Anthony Gitter. Loss-balanced task weighting to reduce negative transfer in multi-task learning. In AAAI , 2019.

Shikun Liu, Stephen James, Andrew J Davison, and Edward Johns. Auto-lambda: Disentangling dynamic task relationships. In TMLR , 2022.

Shikun Liu, Edward Johns, and Andrew J Davison. End-to-end multi-task learning with attention. In CVPR , 2019.

Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In ICLR , 2017.

Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In SIGKDD , 2018.

Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In SIGIR , 2018.

Kevis-Kokitsi Maninis, Ilija Radosavovic, and Iasonas Kokkinos. Attentive single-tasking of multiple tasks. In CVPR , 2019.

Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation with multiple sources. In NIPS , 2008.

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In NeurIPS , 2021.

Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In CVPR , 2016.

Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. In TPAMI , 2018.

Mehryar Mohri and Andres Muñoz Medina. New analysis and algorithm for learning with drifting distributions. In International Conference on Algorithmic Learning Theory , 2012.

Aviv Navon, Idan Achituve, Haggai Maron, Gal Chechik, and Ethan Fetaya. Auxiliary learning by implicit differentiation. In ICLR , 2021.

Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning as a bargaining game. In ICML , 2022.

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NeurIPS , 2011.

OpenAI. Introducing chatgpt, 2022.

OpenAI. Gpt-4 technical report, 2023.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In NeurIPS , 2019.

Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. In TKDE , 2010.

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. ICCV , 2019.

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Technical report, OpenAI , 2018.

Michael T. Rosenstein. To transfer or not to transfer. In NeurIPS , 2005.

Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. CoRR , abs/1606.04671, 2016.

Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In NeurIPS , 2018.

Aviv Shamsian, Aviv Navon, Neta Glazer, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Auxiliary learning as an asymmetric bargaining game. arXiv preprint arXiv:2301.13501 , 2023.

Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, and Huijuan Xu. Auxiliary task reweighting for minimum-data learning. In NeurIPS , 2020.

Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV , 2012.

Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS , 2020.

Xiaozhuang Song, Shun Zheng, Wei Cao, James Yu, and Jiang Bian. Efficient and effective multi-task grouping via meta learning on task combinations. In NeurIPS , 2022.

Trevor Standley, Amir Zamir, Dawn Chen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In ICML , 2020.

Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. In RecSys , 2020.

Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NeurIPS , 2017.

Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. In JMLR , 2008.

Zirui Wang, Zihang Dai, Barnabás Póczos, and Jaime Carbonell. Characterizing and avoiding negative transfer. In CVPR , 2019.

Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In ICLR , 2021.

Derrick Xin, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, and Orhan Firat. Do current multi-task optimization methods in deep learning even help? In NeurIPS , 2022.

Yang You, Jing Li, Jonathan Hseu, Xiaodan Song, James Demmel, and Cho-Jui Hsieh. Reducing BERT pre-training time from 3 days to 76 minutes. In ICLR , 2020.

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. In NeurIPS , 2020.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC , 2016.

Amir Roshan Zamir, Alexander Sax, William B. Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In CVPR , 2018.

Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, and Lucas Beyer. S4l: Self-supervised semi-supervised learning. In ICCV , 2019.

Jing Zhang, Zewei Ding, Wanqing Li, and Philip Ogunbona. Importance weighted adversarial nets for partial domain adaptation. In CVPR , 2018.

Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu. A survey on negative transfer. IEEE/CAA Journal of Automatica Sinica , 2022.

Yongchun Zhu, Yudan Liu, Ruobing Xie, Fuzhen Zhuang, Xiaobo Hao, Kaikai Ge, Xu Zhang, Leyu Lin, and Juan Cao. Learning to expand audience via meta hybrid experts and critics for recommendation and advertising. In KDD , 2021.

A Algorithm Details

A.1 ForkMerge

Proof of Equation (7) .

Remarks on the search step.

We provide two search strategies as follows, and we use the first strategy in our experiments.

Grid Search : Exhaustively searching the task-weighting hyper-parameter λ through a manually specified subset of the hyper-parameter space, such as { 0 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 } .

Binary Search : Repeatedly dividing the search interval of λ in half and keep the better hyper-parameter.

Random search, bayesian optimization, gradient-based optimization, and other hyper-parameter optimization methods can also be used here, and they are left to be explored in follow-up work.

In practice, the costs of estimating ̂ P in the search step are usually negligible. Yet when the amount of data in the validation set is relatively large, we can sample the validation set to reduce the cost of estimating ̂ P .

Remarks on extension from the one gradient step to ∆ t steps.

It can effectively reduce the average cost of estimating ̂ P at each step and avoid over-fitting the validation set.

It allows longer-term rewards from auxiliary tasks and leads to safer task transfer. For instance, when the accumulated gradients of some auxiliary tasks are harmful to the final target performance, the merging step can cancel the effect of these auxiliary tasks by setting their associated weights λ to 0 , to mitigate strong negative transfer.

It increases the risk to produce bad model parameters. However, such risk is still low since deep models usually have smooth loss surfaces after convergence as shown in Section 5.1.

(a) Segmentation

(b) Depth

(c) Normal

Choice of

∆

t

(Epochs)

Figure 7 illustrates that an appropriate ∆ t can effectively promote the performance of the ForkMerge algorithm, indicating the necessity of the extension from the one gradient step in previous work to ∆ t steps. When ∆ t is small, the estimation of λ is short-insight and might fail to remove the harmful parameter updates when negative transfer occurs, which also indicates the limitations of methods that use single-step gradient descent to estimate λ [15, 43]. When ∆ t is large, the risk to get bad model parameters from the linear combination will also increase. Therefore, in our experiment, we use the validation set to pick a proper ∆ t for each dataset and use it for all tasks in this dataset.

A.2 Use ForkMerge to Select Tasks Simultaneously

Detailed Algorithm. Algorithm 2 provides the general optimization process for any task-weighting vector { ν b } B b =1 . For Equation (12), we have B = K + 1 and ν b i = ✶ [ i = b -1 or i = 0] . For Equation (13), we have no constraints on B or ν b .

Algorithm 2 ForkMerge Training Pipeline with Multiple Branches

Require: initial model parameter θ 0 , task-weighting vector { ν b } B b =1 , total iterations T , interval ∆ t Ensure: final model parameter θ ∗ T 1: fork model into B copies { θ b } B b =1 2: for b = 1 to B do 3: θ b 0 ← θ 0 ▷ initialization 4: end for 5: while t < T do 6: for b = 1 to B do ▷ independent update 7: for t ′ = t to t +∆ t -1 do 8: θ b t ′ +1 = θ b t ′ -η ∑ k ν b k g k ( θ b t ′ ) 9: end for 10: end for 11: Λ ∗ ← arg max Λ ̂ P ( ∑ b Λ b θ b t +∆ t ) 12: ▷ search Λ on the validation set 13: θ ∗ t +∆ t ← ∑ b Λ ∗ b θ b t +∆ t 14: ▷ merge parameters 15: for b = 1 to B do 16: θ b t +∆ t ← θ ∗ t +∆ t ▷ synchronize parameters 17: end for 18: t ← t +∆ t 19: end while

Proof of Equation (12) .

The goal of selecting λ ∗ in Equation (10) is to maximize the validation performance of model θ t +1 ,

̸

̸

By definitions of Λ and { ω k } K k =0

̸

̸

̸

̸

̸

̸

̸

̸

we can prove that optimizing λ in Equation (10) is equivalent to optimizing Λ as follows:

Remarks on the search step.

Grid searching all possible values of Λ is computationally expensive especially when ∥ Λ ∥ is large. Thus, here we introduce a greedy search strategy in Algorithm 3, which reduces the computation complexity from exponential complexity to O ( ∥ Λ ∥ ) .

Algorithm 3 Greedy Search of Λ ∗

Require: A list of model parameters θ 1 , ..., θ B sorted in decreasing order of ̂ P ( θ b ) .

Ensure: optimal linear combination coefficient Λ ∗

1: unnormalized combination coefficient ˜ Λ ← e

1 ▷ initialization

2: for b = 2 to B do

3: set upper bound U ← 1 b - 1 ∑ b - 1 m =1 ˜ Λ m

4: grid search the optimal ˜ Λ m in range [0 , U ] to maximize ̂ P ( 1 ∥ ˜ Λ ∥ ∑ b m =1 ˜ Λ m θ m )

5: end for

6: Λ ∗ ← 1 ∥ ˜ Λ ∥ ˜ Λ

▷ normalization

B Analysis Details

In this section, we provide the implementation details of our analysis experiment in Section 3.

We conduct our analysis on the multi-domain image recognition dataset DomainNet [61]. In our analysis, we use task Painting and Quickdraw in DomainNet as examples of weak negative transfer and strong negative transfer, and other tasks (Real, Sketch, Infograph, Clipart) in DomainNet as auxiliary tasks. Details of these tasks are summarized in Table 8. We use ResNet-18 [23] pre-trained on ImageNet [8] for all experiments.

B.1 Effect of Gradients Conflicts

First, we optimize the model on the target task for T = 25 K iterations to obtain θ T . We adopt mini-batch SGD with momentum of 0 . 9 and batch size of 48 , and the initial learning rate is set as 0 . 01 with cosine annealing strategy [45].

(a)  T arget: P

Auxiliary: DomainNet

(b)  T arget: Q

Auxiliary: DomainNet

(c)  T arget: P

Auxiliary: L2 Regularization

(d)  T arget: Q

Auxiliary: L2 Regularization

We repeatedly sample a mini-batch of data and estimate the gradients for the target and auxiliary task g tgt and g aux. Figure 8 plots the distribution of gradient cosine similarity (GCS) between g tgt and g aux . We find that the gradients of different tasks are nearly orthogonal ( cos ϕ ij ≈ 0 ) in most cases, and highly consistent gradients or severely conflicting gradients are both relatively rare.

Then, we optimize the same θ T with one-step multi-task gradient descent estimated from different data to obtain different θ T +1 ,

where η = 0 . 01 and λ takes values from { 0 , 1 16 , 1 8 , 1 4 , 1 2 , 1 } . We evaluate θ T +1 ( λ ) and θ T +1 (0) on the validation set of the target task to calculate the transfer gain (TG) from single-step multi-task gradient descent

Note that we omit the notation of algorithm A in Equation (14) and (15) for simplicity. Then, in Figure 2, we mark the GCS and TG of each data point and fit them with a 3 -order polynomial to obtain the corresponding correlation curve.

B.2 Effect of Distribution Shift

Qualitative Visualization. We visualize by t-SNE [74] in Figure 3(a) the representations of the training and test data by the model θ T trained in Section B.1. For better visualization, we only keep the top 10 categories with the highest frequency in DomainNet. To visualize the impact of λ on the interpolated training distribution, we let the frequency of auxiliary task points be proportional to λ . In other words, when the weighing hyper-parameter of the auxiliary task increases, the effect of the auxiliary task on the interpolated distribution will also increase.

Figure 3 provides the t-SNE visualization of training and test distributions when λ takes values from { 0 , 1 16 , 1 4 , 1 } . We observe that for weak negative transfer tasks, when λ initially increases, the area of training distribution can better cover that of the test distribution. But as λ continues to increase, the distribution shift between the test set and the training set will gradually increase. For strong negative transfer tasks, however, the shift between the interpolated training distribution and the test distribution monotonically enlarges as λ increases.

Quantitative Measure. First, we jointly optimize the model on the target task and auxiliary tasks with different weighting hyper-parameter λ for T = 25 K iterations to obtain θ T ( λ ) . We adopt the same hyper-parameters as in Section B.1. Then we evaluate θ T ( λ ) on the test set of the target tasks and calculate the average confidence on the test set. We can calculate the confidence score discrepancy (CSD) by Definition 3.5 and the transfer gain (TG) by

Again, we omit the notation of algorithm A for simplicity. Finally, we plot the curve between CSD and TG under different λ in Figure 3(b).

C Experiment Details

C.1 Definition of ∆ m

Following [55, 37], we report ∆ m as the performance measure, which is the average per-task performance improvement of method m relative to the STL baseline b . Formally, ∆ m = 1 K ∑ K k =1 ( -1) z k ( M m,k -M b,k ) /M b,k where M b,k and M m,k is the performance of the k -th task obtained by the baseline method b and the compared method m . z k is set to 0 if a higher value indicates better performance for the k -th task and otherwise 1 .

C.2 Auxiliary-Task Scene Understanding on NYU

Experiment Details. We use DeepLabV3+ architecture [1], where a ResNet-50 network [23] pretrained on the ImageNet dataset [8] with dilated convolutions is used as a shared encoder among tasks and the Atrous Spatial Pyramid Pooling module is used as task-specific head for each task. Following [44, 79], each method is trained for 200 epochs with the Adam optimizer [29] and batch size of 8 . The initial learning rate is 10 -4 and halved to 5 × 10 -5 after 100 epochs. In ForkMerge, the parameters are merged every 10 epochs. Table 7 presents the full evaluation results of Table 1.

C.3 Auxiliary-Domain Image Recognition on DomainNet

Dataset Details. As the original DomainNet [61] does not provide a separate validation set, we randomly split 50% data from the test set as the validation set, and use the rest 50% data as the test set. For each task, the proportions of training set, validation set, and test set are approximately 70% / 15% / 15% . Table 8 summarizes the statistics of this dataset. DomainNet is under Custom (research-only, non-commercial) license.

Experiment Details. We adopt mini-batch SGD with momentum of 0 . 9 and batch size of 48 . We search the initial learning rate in { 0 . 003 , 0 . 01 , 0 . 03 } and adopt cosine annealing strategy [45] to adjust learning rate during training. We adopt ResNet-101 pretrained on ImageNet as the backbone. Each method is trained for 50 K iterations. In ForkMerge, the parameters are merged every 12 . 5 K iterations.

C.4 CTR and CTCVR Prediction on AliExpress

Dataset Details. AliExpress [36] is gathered from the real-world traffic logs of AliExpress search system in Taobao and contains more than 100 Mrecords in total. We split the first 90% data in the time sequence to be training set and the rest 5% and 5% to be validation set and test set. AliExpress consists of 2 tasks: click-through rate (CTR) and click-through conversion rate (CTCVR), and 4 scenarios: Spain (ES), French (FR), Netherlands (NL), and America (US). Table 9 summarizes the statistics of this dataset. AliExpress is under Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.

Experiment Details. The architecture of most methods is based on ESMM [47], which consists of a single embedding layer shared by all tasks and multiple independent DNN towers for each task. The embedding dimension for each feature field is 128 . Each method is trained for 50 epochs using the Adam optimizer, with the batch size of 2048 , learning rate of 10 -3 and weight decay of 10 -6 .

C.5 Semi-supervised Learning on CIFAR10 and SVHN

Dataset Details. Following [67], we first split the original training set of CIFAR10 [31] and SVHN [56] into training set and validation set. Then, we randomly sample labeled images from the training set. Table 10 summarizes the statistics of CIFAR-10 and SVHN.

Experiment Details. (1) Auxiliary Tasks. Following [82, 67], we consider two self-supervised auxiliary tasks Rotation [19] and Exempler-MT [14]. In Rotation, we rotate each image by [0 ◦ , 90 ◦ , 180 ◦ , 270 ◦ ] and ask the network to predict the angle. In Exemplar-MT, the model is trained to extract feature invariant to a wide range of image transformations. (2) Hyper-parameters. We adopt Adam [29] optimizer with an initial learning rate of 0 . 005 . We train each method for 200 K iterations and decay the learning rate by a factor of 0 . 2 at 160 K iterations. We use Wide ResNet-28-2 [80] as the backbone. In ForkMerge, the parameters are merged every 10 K iterations.

C.6 Data Division Strategy for ForkMerge

As discussed in Section 4.2, in ForkMerge, we can construct branches with different sets of auxiliary tasks. Below we outline the specific data division strategy used in our experiments, which is consistent with previous ATL literature:

For the NYUv2 dataset, multiple tasks share the same input, but their outputs are different. In this setup, each branch has the same input data, which includes the entire dataset. The distinction between different branches solely lies in the task weighting vector { ν b } B b =1 .

For DomainNet, AliExpress, CIFAR-10, and SVHN datasets, different tasks have both different inputs and outputs. In these cases, for each branch, if the task weighting of a specific task is set to 0 , the data from that particular task will not be used for training the corresponding branch.

D Additional Experiments

D.1 Analysis on the importance of different forking branches

The importance of different forking branches is dynamic. As shown in Figure 9, the relative ratio of each forking branch is dynamic and varies from task to task, which indicates the importance of the dynamic merge mechanism.

D.2 Analysis on the computation cost

The computation cost of Algorithm 2 is O ( K ) and the computation cost of the pruned version is O ( B ) . Usually, only one model is optimized in most previous multi-task learning methods, yet their

(a) Segmentation

(b) Depth

(c) Normal

Branch 1

Branch 2

Branch 3

Seg

Seg + Dep

All

Dep

Dep + Nor

All

Nor

Nor + Dep

All

computational costs are not necessarily O (1) . Gradient balancing methods, including MGDA [65], GradNorm [5], PCGrad [79], IMTL [41], GradVac [76], CAGrad [40], NashMTL [55], GCS [15], OL_AUX [39], and ARML [67], require computing gradients of each task, thus leading to O ( K ) complexity. In addition, calculating the inner product or norm of the gradients will bring a calculation cost proportional to the number of network parameters. A common practical improvement is to compute gradients of the shared representation [65]. Yet the speedup is architecture-dependent, and this technique may degrade performance [55].

In Figure 10, we also compare the actual training time across these methods on NYUv2. We can observe that ForkMerge does not require more time than most other methods. And considering the significant performance gains it brings, these additional computational costs are also worth it. Furthermore, our fork and merge mechanism enables extremely easy asynchronous optimization which is not straightforward in previous methods, thus the training time of our method can be reduced to O (1) when there are multiple GPUs available.

EW

UW

DWA

RLW

MGDAGradNormPCGrad

IMTL

GCS

OL_AUX ARML

Auto-

Ours

(B=3)

Ours

(B=2)

Ours

Async

0

50

100

150

200

250

300

Training Time Per-epoch (sec.)

D.3 Analysis on the convergence and variance

Figure 11 plots the validation performance of STL, EW, and ForkMerge throughout the training process on NYUv2. Each curve is obtained by optimizing the same method with 5 different seeds. Compared with single-task learning or minimizing the average loss across all tasks, ForkMerge not only improves the final generalization but also speeds up the convergence and reduces the fluctuations during training.

D.4 Comparison with grid searching λ

In Section 3, we observe that adjusting the task-weighting hyper-parameter λ can effectively reduce the negative transfer and promote the positive transfer. [77] also suggests that sweeping the task weights should be sufficient for full exploration of the Pareto frontier at least for convex setups and observe no improvements in terms of final performance from previous MTL algorithms compared with grid search.

(a) Segmentation

(b) Depth

(c) Normal

In Figure 12, we compare the performance of all methods with grid search on NYUv2. In grid search, the weighting hyper-parameter for each task takes values from { 0 . 3 , 1 . 0 , 3 . 0 } , and there are 3 tasks in NYUv2, thus there are 27 combinations in total. We find that previous methods simply yield performance trade-off points on the scalarization Pareto front, which has also been observed in previous work [77]. In contrast, our proposed ForkMerge yields point far away from the Pareto front and achieves significant improvement over simply optimizing a weighted average of the losses. One possible reason for the gain is that the task weighting in grid search is fixed during training and takes finite values due to the limitation of computing resources while the task weighting in ForkMerge is dynamic in time and nearly continuous in values, thus can better avoid negative transfer and promote positive transfer.

D.5 Comparison with larger batch size training

In some sense, the multiple branches in ForkMerge increase the equivalent batch size. And it has been revealed that batch size may have a great effect on the performance of deep models [21, 78]. To ablate the influence of batch size, we increase the batch size of the Equal Weighting method. As shown in Table 11, the improvement brought by ForkMerge itself is significantly larger than simply increasing the batch size.

D.6 ForkMerge with more network architectures

ForkMerge with Vision Transformers. We replace the backbone network ResNet-101 with advanced ViT-Base [13] pretrained on ImageNet-21K and repeat the experiments on the DomainNet dataset (Section 5.2). As demonstrated in Table 12, when employing the Vision Transformer model, which boasts increased capacity, the risk of overfitting with limited data becomes more pronounced.

This makes Single Task Learning (STL) less effective and consequently leads to the Equal Weighting (EW) method outperforming STL, causing the Post-train method to fall short of EW and Autoλ . In this case, ForkMerge still exhibited superior performance, validating its efficacy across different network architectures.

ForkMerge with Multi-task Architectures. ForkMerge is complementary to different multi-task architectures. In Tables 13 and 14, we provide a comparison of different optimization strategies with MTAN [44] and MMoE [46] as architectures, which are widely used in multi-task computer vision tasks and multi-task recommendation tasks respectively. On these specifically designed multi-task architectures, ForkMerge is still significantly better than other methods.

Peer reviews:
comments: SUMMARY:
This paper considers how to best use auxiliary tasks to improve performance on target tasks. Specifically, a "ForkMerge" procedure is proposed which consists of two parallel optimization procedures, one on the target task, and one which includes auxiliary data, and the resulting weights are synchronized at regular intervals. Compared to grid searching for an appropriate interpolation factor, the proposed approach can dynamically alter the interpolation between the two sets of model parameters. Experiments on diverse set of benchmarks and a wide array of baselines show that the proposed approach is effective at improving performance using auxiliary tasks, relative to the state of the art.

STRENGTHS:
* This paper tackles a difficult problem of improving target task performance using auxiliary data.

* The approach is well-motivated and the experiments are quite convincing.

* The justification in terms of reducing distribution shift relative to test data is quite interesting.

WEAKNESSES:
* The task selection with multiple auxiliary tasks is computationally expensive, and the impact of the pruning procedure is not completely clear.

* There is no adequate discussion of limitations.

QUESTIONS:
None.

LIMITATIONS:
See "Weaknesses"
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the problem of learning multiple tasks together which is known to lead to \"task inteference\" or \"negative transfer\" issues. This is usually tackled by automatically scaling the task weights or gradients based on training statistics (e.g. GradNorm or uncertainty weighing of losses). In particular, the paper studies an asymetric version of the problem, Auxiliary Task Learning (**ATL**) where one task may not be important at inference but can greatly improve performance when jointly trained with the main target task.\n\nThe paper first explore potential causes of task interference under the lens of train/target distribution shifts. Then, the paper proposes **ForkMerge**, a novel optimization method to avoid task interference: For each update of ForkMerge, first the current parameters are duplicated. Then, the weights  are updated separately; one on the target task, while the other is jointly trained for the task and auxiliary tasks. Then, the optimal task weights are found by finding the linear combination of these two sets of weights that maximize validation accuracy (**Equation 12**). The algorithm can be extended to multiple auxiliary tasks which requires an additional separate optimization branch for each new auxiliary task.\n\nThe method is then evaluated on the NYU dataset (3 tasks) and DomainNet (6 domains) and compared to previous MTL and ATL approaches\n\nSTRENGTHS:\n- I liked the analysis section of the paper and it contains interesting insights on the problem of task interference: For instance, gradient conflicts is often named as a cause of negative transfer but its effect/strength is rarely actually measured in practice. Although it would have been interesting to extend the analysis to more diverse scenarios.\n\n- I also like the insights that task weighing methods should take generalization into account, which is captured in the proposed algorithm by finding the optimal task weights via evaluation on the validation set\n\n- The paper presents extensive experiments on two classical multitask/multidomain benchmarks, which are further completed in the supplementary material (e.g. additional backbone)\n\nWEAKNESSES:\n- **Training cost**: the paper should discuss the training cost in terms of memory efficiency more concretely/quantitatively: ForkMerge requires a separate set of parameters for each task. In addition these parameters are also updated independently hence the number of forward/backward passes will also increases with t he number of tasks. Finally, the cost of search for optimal $\\Lambda$ in Equation 13 will also increase with the number of tasks. These costs may remain reasonable in the application scenario with only one auxiliary task, but it's not clear how practical ForkMerge becomes when dealing with more than two tasks like in Section 5.2\n\n- **Finding 2 does not seem very significant**. Finding 2 states that negative transfer is likely to occur when the added auxiliary task increases the train/test distribution shift for the target task: This seems like a pretty classic statement from statistical machine learning theory: For instance generalization bounds for domain adaptation typically include a term measuring the discrepancy between the source / target domains.\n\n- **Chosen baselines might not all be a fair choice**:  the multi-task optimization baselines (MGDA, GradNorm, PCGrad... etc) typically aim to optimize performance on **all tasks**, as opposed to auxiliary task learning where there is a clear bias towards the target tasks: For instance in ForkMerge, every branch includes the target task objective. For that reason, I do not think applying MTL methods \"as-is\" to the problem of ATL is the best baseline. A stronger baseline could be to finetune the task weights in the \"Equal Weights\" (**EW**) baseline for performance on the target task; this might be to costly for DomainNet but would be reasonable for NYU as there are only two auxilair ytasks.\n\nQUESTIONS:\n- **Suggestion of writing**: I feel like the paper sometimes overuse abbreviations. For instance only mentioning the tasks **P** and **Q** for DomainNet (Painting and Quickdraw domains ?) assumes the reader is very familiar with the dataset. In additional, there are many abbreviations introduced in the paper which often make the text hard to read (for instance lines 154-159)\n\n- **Distribution shift and hypothesis shift**: I did not understand the point that *lines 213-221* try to convey and the difference between Equations 12 and 13 (outside of renaming $K$ to $B$ and allowing $\\Delta_t$ time steps instead of only 1). Similarly, I'm not sure what insight  **Figure 6**  tries to convey. More specifically: the task weighting $\\lambda$ is initially defined at the loss level (Equation 1); However, it seems that in later sections $\\lambda$ is rephrased as a parameter mixing the distribution of the target and auxiliary tasks; while there are links between these two views, I found that switching between the two interpretations in the text a bit confusing. It is also not clear what it means for multi-task applications: There is only one input distribution (the input images are the same for all tasks) so which distribution shift are we referring to ?\n\n- From the appendix **C.3**, it seems that the test set of DomainNet was split into two parts: one for validation, and the other one for test performance. I think this should be clearly stated in the main paper directly as it means the reported results are not comparable to other DomainNet works\n\nLIMITATIONS:\nThere is no dedicated limitations section but the paper discusses the limitation of dedicating independent optimization branches for every auxiliary task (e.g. branch pruning to avoid handling too many branches in the DomainNet)", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors conduct an analysis of negative transfer in auxiliary task learning, finding that gradient conflicts are not necessarily tied to negative transfer, but that auxiliary tasks that induce large distribution shifts from the new training distribution to the test distribution tend to cause negative transfer. The authors then propose ForkMerge, which repeatedly forks a model into branches trained on just the target task and trained on both the target task and auxiliary task(s), and then uses target task validation set performance to determine how to merge the forked models.\n\nSTRENGTHS:\n1. The authors share an interesting and thoughtful analysis, investigating both gradient conflicts and induced distribution shifts as potential causes of negative transfer and concluding that gradient conflicts are not necessarily tied to negative transfer.\n2. The ForkMerge approach that the authors introduce is intuitive and simple, and the results are consistent across a variety of tasks.\n3. Well-motivated and significant problem\n\nWEAKNESSES:\n1. In general, some critical details are omitted altogether in the main text, sometimes with no proper reference to the Appendix when details are missing. In particular:\n- In Section 3, please spell out Painting and Quickdraw (and later, Real in Section 5) and describe DomainNet when first introducing the tasks. If it needs to be brief, the authors can at the very least include a pointer to Appendix C for dataset and task details.\n- I also believe it is worth mentioning \"We use ResNet-18 [8] pre-trained on ImageNet [3] for all experiments.\" in the main text rather than only noting it in the appendix.\n\n2. I would have liked to see more overlap between works cited in the Experiments section and the works discussed in Related Work, especially discussion about and comparison of adaptive auxiliary- and multi-task learning strategies and settings. In particular, [36] and the original meta-learning paper [13]. A couple additional related works the authors may consider adding: https://arxiv.org/abs/2205.14082 \n and https://arxiv.org/abs/2212.01378\n\nQUESTIONS:\n1. Can the authors describe in more detail the computational overhead of ForkMerge, especially wrt time and memory, and in relation to the existing approaches from Section 5? I am most curious about the comparison to meta-learning.\n\n2. Do the authors predict that these results and the practicality of ForkMerge are limited to settings where one starts with a pretrained initialization?\n\n3. re: definition 3.5, are there alternative measures of distribution shift that the authors have considered? I am personally somewhat skeptical of confidence scores as a basis for measuring distribution shift given that many models are poorly calibrated in the first place.\n\nLIMITATIONS:\nThe authors test their method using only one model, ResNet-18 pre-trained on ImageNet, on image tasks. I won't push for additional experiments, but I would like to see the authors discuss what assumptions from this paper's experiments are expected to be critical for generalization of findings to other settings.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuxiliary-Task-Learning (ATL) has been studied from the perspective of optimization, which aims to improve the performance of the target task by leveraging similar tasks. However, ATL can sometimes suffer from negative transfer, where the performance of the target task actually decreases when auxiliary tasks are added. In this paper, the authors take a more holistic perspective by considering both optimization and generalization. They propose a new method called ForkMerge that is able to resolve negative transfer and improve the performance of the target task.\n\nSTRENGTHS:\n* ForkMerge is a simple and practical algorithm that demonstrates good empirical observation.\n* Many analytic figures help readers to understand the negative transfer better although they are limited to DomainNet dataset.\n\nWEAKNESSES:\n* There seems to be some logical jump between the two observations listed in Section 3 and the proposed algorithm in Section 4. It is quite unclear how they are connected directly, and it’ll be great to have some theoretical formulation for the connection (at least in high-level). For example: why dynamically adjusting \\lambda is superior to the previous method? How does this address the generalization problem? I think this limits the contribution of this work.\n* ForkMerge essentially optimizes a fork of the given model and trains them separately multiple times; hence, it does require significantly larger compute during the model training.\n\nQUESTIONS:\n* As discussed in the weaknesses section, the connection between the two main observations to the ForkMerge is not strong. Can you provide a high level theoretical analysis of how ForkMerge can make better generalization not just based on the empirical analysis?\n* Forking a large model is very expensive. Did authors consider a simple ensemble based approach which should be also similar in their compute (not the actual model weight merging rather than prediction level)?\n\nLIMITATIONS:\nI don’t see the negative societal impact of this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo fully leverage the knowledge from auxiliary tasks and mitigate negative transfer issues, this paper introduces ForkMerge, which automatically searches for varying task weights for auxiliary tasks by minimizing target validation errors. ForkMerge is evaluated under various settings, including multi-task learning, multi-domain learning, and semi-supervised learning. The results demonstrate that it outperforms existing methods and proves to be effective.\n\nSTRENGTHS:\nThis paper demonstrates a well-organized structure. The authors have conducted experiments using diverse datasets and tasks, resulting in promising outcomes. Furthermore, the paper provides theoretical analysis of each component, offering valuable insights into their impact on performance. Overall, the clear structure and in-depth analyses make it an engaging and compelling read.\n\nWEAKNESSES:\nThe paper lacks clear explanations for certain sentences, training details, and figure information. For example, there is a lack of clarity regarding the data division strategy for each branch and the effectiveness of the learned weights. Additional elaboration and detailed analysis in these areas would enhance the overall understanding and impact of the proposed approach.\n\nQUESTIONS:\nW1. The paper does not provide a clear explanation of how the method filters out harmful parameter updates to mitigate negative transfer after merging and synchronizing the parameters of each branch.\n\nW2. Figure 3 would benefit from improved clarity, using a distinct color or different marks would be helpful. Additionally, why does the number of auxiliary task data points increase with the increasing $\\lambda$?\n\nW3. Lacks clarity regarding the data division strategy for each branch. It is not explicitly explained whether each branch is trained with a part of the data or full data. Moreover, the impact of increasing the number of branches on computational cost and its trade-off with efficiency and accuracy is not thoroughly addressed in the paper.\n\nW4. I am curious about the comparison between the learned weights by ForkMerge and the optimal weights. For instance, in Figure 3 (b), for WNT tasks, do the learned weights resemble the optimal weights shown in the figure?\n\nW5. It would be valuable to investigate and illustrate the trajectory of the learned weights over the course of training to understand their dynamics and convergence patterns. To gain a better understanding of the effectiveness of the ForkMerge approach, it would be insightful to compare the performance of fixed learned weights by ForkMerge with dynamically learned weights during training.\n\nLIMITATIONS:\nLimitations could be more clarified in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper strives to mitigate negative transfer in auxiliary-task learning by optimizing the coefficients assigned to auxiliary tasks. By conducting an empirical investigation into the factors contributing to negative transfer, this paper reveals two interesting findings. Based on the findings, a new approach named ForkMerge is proposed to mitigate negative transfer and boost the performance of auxiliary-task learning. Extensive experiments demonstrate the effectiveness of the proposed approach.\n\nSTRENGTHS:\n+The problem is well-defined and well-motivated.\n+The findings seem interesting\n+The proposed approach is reasonable and well presented.\n\nWEAKNESSES:\n-It is not clear how the new findings motivate the proposed ForkMerge. \n\n-I'm curious about the performance when directly trying various lambdas (grid search) during multiple training sessions. Although the proposed method is more efficient, there is a concern about potential accuracy trade-offs. I seek to understand if the proposed approach sacrifices accuracy compared to the direct lambda variation method. \n\n-The simple post-train method leads to superior performance over other complex approaches, which makes me doubt the significance of research efforts in this area over the past years. This paper claims that the main drawback of post-train method is that it fails to consider the task relationship in the pre-training phase, and suffers from forgetting during fine-tuning. I’m wondering the performance of the post-train method if considering the task relationship in the pre-training phase and preventing forgetting during fine-tuning such as by distillation. In this scenario, if the post-train method manages to surpass the proposed ForMerge approach, considering ForMerge’s slight improvement in most tasks compared to Post-train.\n\n-The experimental evaluation is limited to small datasets, such as NYUv2 and CIFAR-10, and employs a relatively small network like ResNet-50 (DeepLabV3+). As a result, there is uncertainty regarding the efficacy of the proposed method on larger datasets and powerful networks like transformers, which already excel in single-target tasks. Consequently, assessing the true significance of the proposed method in the current era of deep learning becomes challenging.\n\n-Too may abbreviations make paper hard to follow in some paragraphs.\n\nQUESTIONS:\nSee Weaknesses\n\nLIMITATIONS:\nNot sure", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-27"}