Module-Aware Optimization for Auxiliary Learning
Authors: Hong Chen
Xin Wang
Yue Liu
Yuwei Zhou
Chaoyu Guan
Wenwu Zhu
Venue: NeurIPS 2022 Accept (2022)

Abstract: Auxiliary learning is a widely adopted practice in deep learning, which aims to improve the model performance on the primary task by exploiting the beneficial information in the auxiliary loss. Existing auxiliary learning methods only focus on balancing the auxiliary loss and the primary loss, ignoring the module-level auxiliary influence, i.e., an auxiliary loss will be beneficial for optimizing specific modules within the model but harmful to others, failing to make full use of auxiliary information. To tackle the problem, we propose a Module-Aware Optimization approach for Auxiliary Learning (MAOAL). The proposed approach considers the module-level influence through the learnable module-level auxiliary importance, i.e., the importance of each auxiliary loss to each module. Specifically, the proposed approach jointly optimizes the module-level auxiliary importance and the model parameters in a bi-level manner. In the lower optimization, the model parameters are optimized with the importance parameterized gradient, while in the upper optimization, the module-level auxiliary importance is updated with the implicit gradient from a small developing dataset. Extensive experiments show that our proposed MAOAL method consistently outperforms state-of-the-art baselines for different auxiliary losses on various datasets, demonstrating that our method can serve as a powerful generic tool for auxiliary learning.

Module-Aware Optimization for Auxiliary Learning
Hong Chen, Xin Wang" Yue Liu, Yuwei Zhou, Chaoyu Guan, Wenwu Zhu"
Tsinghua University
{h-chen20, liuyue17, zhou- guancy @mails. tsinghua. edu. cn {xin_wang, wang, www.:/wwwwwwwnnaaaaoe edu.cn
Abstract
Auxiliary learning is a widely adopted practice in deep learning, which aims to improve the model performance the primary task by exploiting the beneficial information in the auxiliary loss. Existing auxiliary learning methods only focus balancing the auxiliary loss and the primary loss, ignoring the module-level auxiliary influence, i.e., an auxiliary loss will be beneficial for optimizing specific modules within the model but harmful to others, failing to make full use of auxiliary information. To tackle the problem. we propose a Module-Awand Optimization approach for Auxiliary Learning (MAOAL). The proposed approach considers the module-level influence through the learnable modulc-level auxiliary importance, i.e. the importance of each auxiliary loss to each module. Specifically, the proposed approach jointly optimizes the module-level auxiliary importance and the model parameters in a bi-level manner. In the lower optimization, the model parameters are optimized with the importance parameterized gradient, while in the upper optimization, the module-level auxiliary importance is updated with the implicit gradient from small developing dataset. Extensive experiments show that our proposed MAOAL method consistently outperforms state-of-the-ar baselines for different auxiliary losses various datasets, demonstrating that our method can serve as a powerful generic tool for auxiliary learning
Introduction
Auxiliary learning is common practice deep learning, which utilizes auxiliary losses (generally from related tasks) to benefit the primary task in terms of model performance generalization ability. The auxiliary learning paradigm has shown its effectiveness in various areas including image classification [1 recommendation (355). reinforcement learning 76. etc. Under various scenarios, task-specific auxiliary losses are designed for different purposes, c.g., in recommendation. previous works [5] 3 add the click-through rate prediction loss to help the click conversion rate prediction; in image classification []. the self-supervised auxiliary loss is designed to improve classification accuracy when labels are inadequate; and the losses of head pose estimation and facial attribute inference are utilized to aid facial landmark detection (8)
Existing auxiliary learning methods focus on balancing the importance of each auxiliary loss and the primary loss. Most of them assign a predefined weight to each auxiliary loss and then optimize the sum of the weighted auxiliary losses and the primary loss. The predefined weights generally need carefully tuning with Hyper-parameter Optimization (HPO) tools to prevent negative auxiliary transfer. Recently, several works 19 10 are proposed to automatically assign weights to each auxiliary loss based on the gradient similarity between each auxiliary loss and the primary loss. The more recent works [ further proposes to utilize non-linear auxiliary loss combination to better
* "ooresponning Authors.
OOr code will be released at http://s//ggtttb..co////rraaaee/
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

exploit the beneficial auxiliary information, and [22 proposes to jointly select the most beneficial task and data for auxiliary learning.
Nevertheless, existing approaches ignore the important fact that an auxiliary loss could have different influences on different modules within the model. Here we define a module to be fine-grained part of the model, c.g., a module can be a block in ResNet [13, a transformer block in Bert [4], or a (MLP). The necessity of considering the module-Ievel influence of
layer in Multi-Layye-Perreephhe auxiliary losses has been indicated by research related to multi-task learning. Previous multi-task learning works [S] 16] show that shallow layers (layers close to the input) contain more common information among related tasks, while deep layers (layers close to the output) contain more task- specific information that tends to cause negative transfer. This phenomenon indicates that when the auxiliary loss comes from a related task, if we do not consider the module-level influence, the negative information the deep layers will counterbalance the benefits brought by the shallow layers, resulting in inefficient usage of auxiliary losses. However, these multi-task learning works [15 [6] aim at designing model architectures, not suitable for the auxiliary learning scenario where the model architecture is designed (and fixed) for the primary task, and our target is to utilize the auxiliary losses to better optimize the model. In sum, considering the module-level influence is important to auxiliary learning, and there has been no work on auxiliary learning considering this module-Ievel influence.
To deal with the module-level auxiliary influence, we propose a module-aware optimization approach for auxiliary learning (MAOAL). The comparison of the existing methods and our module- optimization approach is shown in Figure [] The proposed MAOAL approach considers the module- level influence of auxiliary losses by introducing the learnable module-level auxiliary importance, which reflects the importance of each auxiliary loss to each module within the model. Specifically, the module-level auxiliary importance and the model parameters are jointly optimized in a bi-level manner. In the lower optimization, the model parameters are updated with the module-level auxiliary importance parameterized gradient, so that each auxiliary loss can be used to optimize the module it beneficial to, making each module of the model play a better role for the primary task. In the upper optimization, the module-level auxiliary importance is updated with the implicit gradient from a small developing dataset, making the module-Ieve auxiliary importance more accurate. Additionally, the best-response approximation is utilized for efficient implicit gradient calculation. Since the module- level auxiliary importance is learned by a data-dring diiiaddrienn optimization process without requiring specific knowledge of the auxiliary task, the proposed module-aware optimization method can accommodate variety of auxiliary losses. We conduct experiments on different tasks and datasets, and experimental results show that our method consistently outperforms state-of -fate-of-the-art methods. Particularly, when we consider a wider range of auxiliary losses, such as adopting L2 regularization [17 [8] as an auxiliary loss, our method can also deal with the module-level influence in these situations and bring better model performance.
We summarize contributions follows. (i) We are the first to investigate the module-level influence in the general auxiliary learning setting. (ii) We propose a module-aware optimization (MAOAL) method for auxiliary learning, which utilizes the module-level auxiliary importance parameterized gradient to optimize the model parameters, and optimizes the module-level importance with the implicit gradient. (iii) Extensive experiments demonstrate that our method consistently outperforms sate-oofthee-a baselines in various scenarios. Analysis of the modules and the module- level auxiliary importance provides insights for auxiliary learning.
optimize the whole model(-nVoL4 (Vo4))
auxillary loss1 las auxiliary loss2
module-level importance guided
module-aware optimization
Modulo
Meduea
Miducca
Moduled
Moduse
primary ry loss
Medulct
Meded
Redul
Pledule
Manules
Lgont
primary loss
combined losses 4r
Model for the primary task/91
Laa
Model for the primary task10)
auxiliary losses
optimize whole model
Existing methods
Our method
Figure 1: Existing methods utilize the combination of the primary loss and auxiliary losses to update the whole model parameters. In module-aware optimization, auxiliary loss 1 should update the parameters in module 1 1,2,3 and auxiliary loss 2 should update module 2,4 and 5.
2

---

2 Related Work
Auxiliary Learning Auxiliary learning is a common practice in deep learning. This learning paradigm aims to improve the model performance on the primary task with designed auxiliary losses (generally from auxiliary tasks). Most existing works [ utilize the auxiliary losses by linearly combining them with the primary loss and then utilizing the combined loss to optimize the whole model. The weights for the auxiliary losses will be tuned to avoid negative transfer to the primary task. Some recent works [9 19 propose to automatically weight the auxiliary losses during the training process in a dynamic way. Specifically, work 9 calculates the gradient of each auxiliary loss, and utilizes the cosine similarity between the auxiliary gradient and that of the primary loss as the auxiliary loss weight. Work [Q shares a similar idea with [6] but with the goal that the weighted gradient sum should be close to the gradient of the primary task. Moreover, [1i] proposes to learn a non-linear combination for auxiliary losses, and [12 proposes to select not only the task but also each data sample within each task to make full use of the auxiliary information. However, these methods only conduct model-level optimization for auxiliary losses, ignoring the fact that one auxiliary loss could contribute differently to different modules within the model.
Multi-task learning Another line of highly related work to optimize several losses is multi-task learning. Different from the goal of auxiliary learning, multi-task learning aims to obtain good performance on all the learned tasks, while learning with auxiliary losses only focuses on the primary task. Existing multi-task learning methods can be roughly categorized to three parts [20]: multi-task architecture design [6). multi-task optimization (2 and multi-task relationship learning (25). Multi-task architecture design methods aim at designing proper architectures for simultancously learning multiple tasks. Some architecture design works [66 21 point that different tasks share more common information in the shallow layers, which inspires us that module-aware optimization for auxiliary losses is needed. The multi-task optimization methods aim at optimizing the whole model for all tasks with methods like loss weighting (22 23), which can be used to optimize multiple losses. Particularly, work [26] proposes to assign layer-wise to each loss, but it only focuses on multi-task setting and the weight is assigned based on heuristics, which cannot be directly applied to our general auxiliary setting where we only care about the performance of the primary task and the auxiliary losses might by some other regularization terms like 1.2 regularization or disentangled regularization [77-30]
Position of this work This work is mainly inspired by [31. where the machine is expected to automatically select the data, tasks and model to be learned. Previous works have focused on different focus on automatic task selection, and (8) focus on automatic model designing. More recently, some works focus on joint selection of different elements, where the selection space is much more complicated. [12] proposes joint select task and data, [39 40 proposes to joint select data and model, and this work fills the joint task-model selection gap.
aspects. (27 32 35] focus automatic data selection, [I] 6 9
3 The Proposed Method
3.1 Preliminaries and Problem Formulation
Assume that we have a primary task T goou and its corresponding loss Lgoat (primary loss). There are totally K auxiliary losses {La1, La2. Los. ... LLa .., utilized to help the primary task. Let Dtrain, D2 respectively be the training and validation dataset. The model for the pri- mary task is parameterized by 6, which is a deep model composed of different modules, i.e., 0 = =01, 82 ..... , . Jm). The partition for the modules can be flexibly defined by users, e.g., module can be a block in ResNet [13), transformer block in Bert [14], or specified by its designed function [4]]. Previous approaches for auxiliary learning generally adopt the following objective:
K
(o) Legoal (0; Dtrain +
Σ
wi Las(0; Dirain),
where w; is the weight used to balance the auxiliary losses and the primary loss. With proper values set for wy, the model parameters 0 are updated with the widely adopted gradient descent method where is the unrolled step:
0, 0, 7/ B.Vee-144 Lc (0--1).
(2)
3

---

steps OWE optimization
upper optimization
Gradentt descent for (yst Grodient descent for
Drnan
Drown
Dress
Dooin
Modekiyl
Medellg
Moduuz
Mesuat
Mequas
Modulks
Magaek
of
Prikpp
Ml.
Makaet
Modu
Metwed
Hoeued
Madana
Op
Lypart Meruni
Importance paramete loed gradiene
Impprr lance pammetertoe
teparberee parametirieed
Implist gradiet
Importance parometerized rnddent
Imporsance arrntte ized prad antt
pammetertne
Importance
grddeen
gradien
Pan)
11
Maz
Loop1
modue leve auvilary
Loop2...
module
importance
aunilarr
Figure 2: The module-aware optimization method for auxiliary learning, In each loop, the lower optimization is conducted for times followed by one-step upper optimization. In the example, in Loopl, the two auxiliary losses Loi, Le2 update all modules equally. After Loopl optimization, Lot is only important to module2 and is used for updating module2 while Laz only for modulel.
Although this kind of learning paradigm has been widely used in various scenarios, it cannot deal with the module-level influence as indicated in [2] [6]. To avoid confusion, we remind that the superseript of @ indexes the module within the model, and the subscript of 0, indexes the unrolled step in the rest of the paper.
Our goal is to deal with the module-level influence of auxiliary losses and make each auxiliary loss optimize the module which is beneficial to. Specifically, we introduce the module-level auxiliary importance matrix 02, and then optimize the model parameters 0 for the primary task according to the module-level auxiliary importance. Here two key problems are how to obtain the optimal module-level auxiliary importance and how to utilize the importance to optimize the model parameters A. We optimize the module-level auxiliary importance and the model parameters in a bi-level manner, and the whole framework of the module-aware optimization process is shown in Figure 2 the lower optimization, we optimize the model parameters 0 with the importance parameterized gradient. In the upper optimization, we update the module-level auxiliary importance with the implicit gradient from a developing dataset Dae In the next two subsections, describe the lower and upper optimization in detail and summarize the complete algorithm in Section 3.4
{asz KK fi=1j=1. where azy is the importance of auxiliary loss Lai to the module
3.2 Lower Optimization: Importance Parameterized Gradient for Model Parameters
In the lower optimization, we optimize the model parameters 0 on the training dataset Derain. Assuming that we have already known the module- auxiliary importance Aij. we will utilize the importance to guide the optimization process of the model parameters 0. We first obtain the gradient of each module 0 with respect to each auxiliary loss Liet:
V Laii Voalai,,... Vom Lui] Volni(f; Dtrain), 1,2, . K
Then we reweight the gradient of each auxiliary loss with the importance azs and obtain the impor- tance parameterized gradient Vo , or; Devain) as follows:
V Fo or; Dtrain) = (an Lai, 0: Ve Laii ,iim
K, (4)
K
V VoI o; Dtrain) = LeLouatt Dtrain) +
Σ
VoLass oz Dtrain),
(5)
where in Eq, the gradient for each auxiliary loss is masked with its module-level auxiliary importance, where if the 8th auxiliary loss is important to optimizing the jth module, the auxiliary module-level gradient V Lui will be weighted with a larger 0:j and vice versa. In (5) we combine all the weighted auxiliary gradient and the gradient of the primary loss to obtain the importance parameterized gradient, which will be finally used to optimize the model parameters After that, we utilize the widely adopted unrolled gradient descent to optimize the model parameters

---

0 as follows, where is the learning rate.
D+t1 = Oc 1 77 Vo, I(Or, o:; Dirain)
(6)
3.3 Upper Optimization: Implicit Gradient for Module-level Auxiliary Importance
In the upper optimization, our target is to optimize the module-level auxiliary importance oaj- To obtain the gradient of a, we introduce a small developing dataset Daee as in [42] Daeu is a small-size dataset for the primary task split from the validation dataset D... Since we expect that model can finally perform well on the validation dataset D.. our model should also perform well enough on its subset Dew Therefore, it is natural to utilize the loss on Daev to optimize the importance matrix Assume that we have conducted M steps of lower optimization, and the current model parameter is far- The loss of the model on the developing dataset Lyout (0. (a); Ddev) can be calculated. Note that we only calculate the primary loss on Deev because we actually only care about the model performance on the primary task. Directly calculating the gradient of with respect to ( (()m Daco) is not easy. The loss goaa (0)( Daew) directly relies on the model parameters Oas instead of the importance matrix a. implicitly influences the value of (Og (a); Daev) by influencing the gradient of 0 as shown in the lower optimization in [ According to this observation, we use the chain rule obtain the the gradient of with respect to goo (0 (a); Ddev):
V aaLgoot (0 M (a); Daeu)= V Vz ( (a); V.0M
The term Ve goot M ( (a); Daew) is easy to obtain using the autograd tools. However, the term V Va requires more complex exploration the lower unrolled steps. By differentiating and unrolling the process in (o. we obtain the following results:
V.OM
Σ OSTCM 0SJKT
(
II
[T
mn VgL a; Derain (loman))lom---p)) Von T(00, oz Dirain)lom-1-1
where indexes the steps we look back, summing up all the M looking-bace results gives the desired gradient. This means that we know the Jacobi and the Hessian within the past M steps, we can use them to calculate 0M the gradient of a with respect to current model parameters fM. Detailed derivation can be found in the supplementary file. However, this kind of calculation is memory- consuming because have to restore the Jacobi and Hessian during the past M steps. To improve efficiency, we follow the best-response approximation in 42 and assume that fo ",, -0000 all equal aa.. With this assumption, we can approximate (8) in the following manner:
VaOm R
Σ VVLmVa o; Derain)) V V I(OOM; 0; Derain), 0ST<M
where we only use the Hessian and Jacobi the current step. Replacing the term Va in Eq. 7 with (q.(9) gives the desired gradient of a. Moreover, the calculation for (7) can be performed
efficiently using vector-Jacoo Jacobi- products [42 as shown in Algorithm D which is both memory and
computation efficient. This kind of approximation assumes that 6, (0 < T < .M) equals to 0M, which will be more accurate when 0- is around the best response 0* under given a(the best response 0* is the point that the unrolled process in (6) converges to when is fixed). This is why it is called best-response approximation in 42 The effectiveness of this approximation is practically validated
in both previous works 42
and our experiments.
3.4 Overall Algorithm
With both the lower and upper optimization above, we summarize the complete module-aware optimization algorithm for auxiliary learning in Algorithm [ The input for the algorithm contains three datasets: the training dataset Dtrain- the developing dataset dev and the validation dataset D., and the hyperparameters. T and are used for the lower optimization, where T is the steps we conduct lower optimization in one loop and 7/ is the learning rate for the lower optimization. M and 77 are used for the upper optimization, where M is the total looking- back steps in 49) and n22 is the learning rate for the upper optimization. The model parameters are randomly initialized and the importance matrix is initialized with all its elements 1.0. Specifically, in line 4 to line 6, we conduct lower optimization for T times. In line 15 to line 22, we conduct the upper optimization, where line 16 to line 21 is the process that utilizes the vector-Jacobial products to calculate the gradient in
5

---

Eq. The lower optimization and the upper optimization are iteratively conducted until converged. Particularly, in line 7 to line 14, we will evaluate the model performance on the validation dataset D, after each interval loops of lower-and-upper optimization. We always record the 0 with the best performance in Oopt, and finally return the parameters Aopt. The use of the validation dataset is to prevent the model from overfitting Dtrain or Devv Note that we do not directly look back the total steps in the upper optimization, but only look back M steps(M < ()) for faster computation of 7 which is also adopted by [1].
In our algorithm, each auxiliary loss enjoys an independent vector a, {a enabling not only to balance losses, but also consider how important each auxiliary loss is to each module. Without requiring specific knowledge of auxiliary losses, the data-driven optimization of the module-level auxiliary importance enables our method to be applied to the general setting of auxiliary learning.
Algorithm 1 Module-Await Optimization for Auxiliary Learning (MAOAL)
I: input: dataset: Devv, Do: hyperparameters: T, M, t, T2, interval
2: initialize: 8. a, counter 0, per formance.ps 0, opp A
3: while not converged do
4: // lower optimization: update 9 with fixed
5: for = 0 to 1 do
6: Update model parameters 0 using (6)
7: end for
8: I/ evaluate the model on Dr
9:
counter = counter +
if counter%iintrrull
0 then
11:
per formance Metric(0: D.);
formance > per formancccd then
13:
performanceapt per formance, Opp 0;
end if end if
15:
// upper optimization: update a with current A:
17:
p = U I. Ddev);
18:
for T = 1 to do
19:
grad(n VoL(o, oz Derain), O, graad_output v)
20:
p x 4+
21:
end for
22: V aoLgaal (0: Duev) = grad(m VoL(O, a; Dtrain), a, leeugrrdduuuu outputs = p)
23:
772 V a goog Ddev)
24:
end while
25:
Return Oopt
Experiments
In this section, we empirically assess the efficacy of our method on various tasks and datasets. Also, we provide ablations on how our method works and the module influence under different scenarios.
4.1 Experimental Setup
Task and datasets (i) We first focus on the image classification problem with auxiliary tasks. We conduct experiments on two fine-grained image classification datasets, CUB [4] and Oxford-IIIT Pet (45), and two widely adopted general image classification datasets, CIFARIO 10 and CIFARIOM [66 Specifically, on the CUB dataset, there are total 200 species of birds and each image has the label of its attributes, like the wing color and bill shape. In general, requires expert knowledge classify the birds, but it is much easier to discriminate the attributes. Thus, it is a natural practice to regard bird species classification as the primary task, and use the wing color classification and bill shape classification losses as the auxiliary losses. On the Oxford-IIIT Pet, CIFAR 10 and CIFAR1ON 100 dataset, [ and rotate each image with degrees of (0, 90, 180, 270). The primary
we follow the practice task is image classification and we add the rotation degree prediction loss the auxiliary loss. Note

---

Convolution
Res--bookk1
Rox biodbl
Res Res--bookk33
Ress-book4
head1
Convolution
Convolution2
Corvolution
Corvolutions
Uneer Layer
mranooommrr
ITa
Trarsformr
head?
head1
head2
Pa
ResNet18
4-layer CovNiet
AutoiNT
Figure Adopted models and module partitions. Red number refers to module index.
that in the CUB datasets, we adopt pretrained backbone but in other datasets we train from scratch, to show the effectiveness of our method with different initialization. (ii) The second problem we focus on is the user rating prediction in recommendation. We evaluate our methods on two datasets with different sparsity, Amazon Beauty [47] and MovieLens 488] Our primary task is to predict the rating for each user-item pair based on the user features and item features. In this problem, we consider a kind of more generalized auxiliary loss, L.2 regularizer. In recommendation, L.2 regularizer plays an important role to the model performance and is often used an auxiliary loss 11 18] In this setting, the primary loss is the rating regression loss and the auxiliary loss is the L2 regularizer. Detailed dataset information can be found in the supplementary.
Baselines We compare our methods with different baselines that can be used for the general setting of auxiliary learning. Single loss learning (SLL) is a natural baseline that only utilizes the primary loss to train the model. Comparing with this baseline can help judge whether the auxiliary loss is beneficial to the primary task. Additionally, assigning each loss equal weights "1.0"" can be used to show the benefits of different auxiliary methods. Another natural baseline (HPO-tune) is to combine the auxiliary losses with predefined weights, and then use the HPO methods to tune the weights as conducted in M. GCS [9] is a baseline that utilizes the similarity between the gradient of each auxiliary loss and that of the primary loss to weight each auxiliary loss. Auxl. [1 is a recently proposed method that learns a non-linear combination of all the losses. Uncert [22] is a method that
is designed for combining the losses from multiple tasks, where the loss of each task is weighted according to their uncertainty. Note that for the methods that do not rely on Ddev. we add Ddev to their training set for fair comparison.
Implementation We utilize the most widely adopted structure g for the tasks, with one common backbone followed by task-specific heads for all the tasks. In the image classification problem, the auxiliary losses come from some related auxiliary tasks and each task will have its corresponding task-specific head, but the recommendation problem only has one task head for the rating regression because its auxiliary loss is from 1.2 regularizer. For different tasks, we utilize different backbones. For the CUB and Pet dataset, we utilize ResNet18 [3] as the backbone. For the CIFARIO and CIFAR100, we use a 4-layer convolutional Beauty and MovieLens- 1M, we choose AutoINT [49 as the backbone. The model structure and the module partition for the three settings are shown in Figure B We implement the task-specific heads with Multi-layer Perceptron(MLP) whose layer number is searched in (1, 2). In ResNet18, the modules are divided by residual blocks, in 4-ayyer CowNet, the modules are divided by convolutional layer, and in AutoINT, the modules are divided by Transformer block. More detailed implementation can be found in the supplementary file.
For the rating prediction in Amazon-
4.2 Method Effectiveness
We evaluate our method on all the aforementioned tasks and datasets. For the image classification task. we utilize the topl-acoo vopl-accuracy as the metric and for the rating prediction task, we utilize RMSE(rooted mean squared error) as the metric. Higher accuracy for image classification and lower RMSE for rating prediction indicate better performance. The overall results are shown in Table As indicated in previous works [TI 9), losses from auxiliary tasks play a more important role when the labels for the primary task are inadequate. Therefore, we also conduct experiments on the CUB and Pet dataset, where for the primary task, we only utilize 20% labels. The results for this semi-supervised We run all the experiments with three random seeds and report the mean and the std. Note that 1.2 regularizer with coefficient too large for recommendation, so the Equal baseline performs worst on the Beauty and MovicLens dataset.
setting are shown in Table 2
7

---

Alleviating negative information from auxiliary loss The results of CUB and MovieLens in Table | show that the auxiliary losses are harmful the primary task. The baselines that utilize the auxiliary losses generally achieve worse performance than the SLL baseline that only utilizes the single primary loss to train. Our method shows its strong ability to alleviate the negative information from the auxiliary loss, it achieves comparative performance with the SLL baseline even though it does not use Dew for training. It is worth noting that the AuxL which also adopts a bi-level optimization framework shows excellent ability in alleviating the negative impact of the harmful auxiliary losses.
Exploiting useful information in auxiliary losses The results of Pet and Beauty in Table | and the results of Pet-semi and CUB-semi in Table ( show that the auxiliary losses are beneficial to the primary task. All the baselines that utilize auxiliary losses show substantial improvement over the SLL baseline which only uses a single loss. Our method beats all the baselines in exploiting the beneficial auxiliary information, showing the advantage of the module-Ievel consideration. The AuxI. baseline fails to perform well enough probably because the form of the non-linear combination requires a specific design for the non-linear model in specific tasks.
Other Observations (i) Comparing the results of CUB and Pet in Table D to that in Table 2 we can see that when the labels are inadequate for the primary task, the auxiliary losses from related tasks will bring more benefits, which is consistent to previous works [T 9). (ii) The rotation prediction task seems to have little influence on the task of classification on the CIFARIO and CIFARIOO dataset, the different methods that utilize the auxiliary rotation prediction loss do not bring substantial improvement or severely do harm to the task of CIFAR classification. This phenomenon may be caused by the low resolution of the CIFAR images, where the changes brought by rotation may not be so significant. (iii) Although the Beauty and the Movielens experiments both utilize the 1.2 regularizer to help the rating prediction, the auxiliary regularizer shows different effects. In Beauty, the regularizer is helpful but in MovieLens, the regularizer is harmful. This phenomenon is easy to understand because the MovieLens dataset is dense while Beauty is sparse. Regularizer is important to the sparse Beauty by preventing the overfitting problem. In the dense MovieLens, data are enough for each user so the regularizer is not necessary.
Overall, when we utilize different kinds of auxiliary losses, we always expect to exploit the bene- ficial information while resisting the impact of negative information. Our proposed module-aware optimization method, shows its superiority when facing both beneficial and harmful auxiliary losses.
Additionally, based on the reviews, we add additional experiments on the NYUv2 (50) and CIFAR- 100/20 datasets so that NYUv2, we totally have 3 tasks, i.e., segmentation, depth prediction and normal prediction, where the segmentation is the primary task and we adopt the EfficientNet (51) as the backbone, which is split into 8 blocks according to its model structure. In the CIFAR-100/20 dataset, we treated each of the 20 'coarse' classes as one task, where each coarse class contains classes. We regard the "people" classification("abby
validate the ability of our method when facing more auxiliary tasks. In
"boy", "girl", "man", "wommn") '') as the primary task, and the 0-9 coarse class classification as the auxiliary tasks, totally 11 tasks. The ResNet18 adopted as the backbone for the CIFAR- 100/20. The results of the added experiments are shown in Table 3] These experiments show that our method can comply well with more auxiliary losses and also more advanced models.
Table 1: Performance of different methods under different scenarios. The methods with the best performance are bolded.
Dataset
Method
CUB
Pet
CIFARIO
CIFARIO0
Beauty
MovieLens
ACC(%)
ACC(%)
ACC(%)
ACC(%)
RMSE
RMSE
SLL
77.290.63
61.141.03
71.60o.25
46.52..
1. 1.100 0000 0006
0.9068p 0028
Equal
71.68o.s7
67.52p.58
70.60o.30
45.570.25
2.45120.0013
2. 2.25670..0053
HPO-tune
76.72o.71
66.91o.46
71.51o.05
46.330.21
1.0918o.om 0004
0.93799.00 0.008
GCS
71.59o.10
66.69p.77
70.950.02
45.690.01
1.0956..000 0018
0.9137 o. 0001
Uncert
71.641.15
67.42u.s1
71.00..42
46.370.01
1.0942..
0000
0.9883 0000
AuxL
76.990.81
66.41o.97
71.580.05
46.98n.06
1.0893o. 0028
0.9063p.oom
MAOAL
77.30p.17
69.61p.93
72.240.27
47.57
1.08730.0017
0. 0.9045o.0013
8

---

Table 2: Performance of different methods with only 20% label on the primary task.
Dataset
SLL
Equal
HPO-tune
GCS
Uncert
AuxL.
MAOAL
CUB-semi 48. 15..s
50.54..55
50.34h.71
51.06p.59
50.63o.16
48.506.74
52.36n.64
Pet-semi
31.811.17
49.070.91
49.281.25
47.160.57
47.961.01
46. 100.58
51.420.31
Table 3: Performance of different methods in the setting with more auxiliary tasks.
NYUv2
SLL
Equal
HPO-tune
Uncert
GCS
AuxL
MAOAL
m-1OU((%)
32.61o.21
33.150.15
33.09o.25
32.93a.32
32.188.34
33 33.066.14
33,88
pixel-ace({))
68. 42..22
68. 1l..24
68, 68.522.52
68. 68..42
07.588.80
68. 68.366.53
69.17p.44
CIFAR-100/2)
SLL
Equal
HPOO-tune
Uncert
GCS
Auxl.
MAOAL
acc(%)
49.73v.61
51.270.61
51.750.72
48.40
51 51.80a.40
51 51.536.23
53,870.46
4.3 Module-Ievel Auxiliary Importance
modile importonce
modiiu importanne
moduae importanne
shape
Rotation Prodiction
0.04
Rotacion Prodiction
0.02
color
3.2
2.0
0.52
0.02
0,00
0.00
-0.02
-0.00
-0.02
0.4
Roration Predcction
moduk index
modale indox
module index
module indox
(a) CUB
(b) Pet
(c) CIFARIO
(d) CIFARI00
8 20003155
12 Reguratzers
0.04
Reg Regulwizers
1.2
shope
Rotwatio Predittor
0.00305
0.01110
0.00
0.02
c.n-
wing roler
-0.02
g 0.0295
0.01290
module Indes
modile moee
Index
madum Indes
(c) Beauty
(f) MovieLens
(g) CUB-semi
(h) Pet-semi
Figure 4: Module-level auxiliary importance
We expect that the learned module-level auxiliary importance can offer some insights for auxiliary learning. The learned module importance under each scenario shown in Figure 4 and the modules are numbered the same in Figure 3 (i) When the auxiliary losses are harmful
or have no substantial influence(CIFF I0, 100) to primary task, the final learned module-level auxiliary importance will be O. It is little surprising note that in these harmful or unrelated auxiliary settings, our method can achieve comparative or even better performance than the SLL (only use single primary loss) baseline even though we utilize fewer data to train the model. This improvement may benefit from the dynamic weighting and training process. Although the weights for the auxiliary losses are finally set to zero, in the training process, the bi-level framework can utilize the gradient from the auxiliary losses to help the model generalize better on the unseen small Dácc. That is to say, the gradient from the auxiliary losses may guide 0 to a better descent direction for the primary task in the early stages, but the gradient will show little importance when the better direction is found. (ii) When the auxiliary losses come from some related beneficial tasks (CUB-semi, Pet-semi), the importance of modules fits our intuition where the modules close to the input show higher importance while the modules close to the output show lower importance, and this is why our method can better exploit beneficial auxiliary information. Our module-aware optimization stops the auxiliary losses from optimizing deep modules (close to output), thus avoiding negative transfer in deep modules. (iii) However, in (b)Pet, module 1 shows low importance although the module 2,3,4.5 still follows the previous pattern, probably because compared Pet-semi, the labels in Pet for the primary task are enough, so the information from the primary loss is enough to capture the low-level feature the first module, and the gradient from the rotation loss is not so necessary. (iv) L.2 regularizer is beneficial to the Beauty dataset, but its pattern is also different from that of CUB-semi and Pet-semi. Module6 (task head) and modulel (embedding table) need larger I.2 regularizer than middle transformer blocks.

---

The learned module-level auxiliary importance shows different patterns under different scenarios, indicating our method can be applied a variety of auxiliary learning scenarios.
4.4 Influence of Module Choice
In this part, we explore the influence of how we choose the module. We choose different partitions of modules and conduct experiments on the CUB-semi and Pet dataset. The results are shown in Table The Parameter-L means we regard each parameter in the network as a module in our algorithm, which is finer-grained partition. The Block-L means regard each block in the ResNet as a mohule(the same as all the previous experiments), The Model-L means we regard the whole model module, which degenerates the loss weighting methods. We can see that the Model-L variant performs worse than the Block-L, in- dicating the importance of considering the module-level effect of auxiliary losses. Ad- ditionally, we find that regarding each pa- rameter as module does not bring substan- tial improvement compared to the Block-L. This may be due to the reason that such fine-grained module partition will make the importance matrix a contain more parameters. Opti- mizing more parameters in the upper level requires more data samples in Daee 122, which prevents the finer-grained partition from bringing more improvement. We also find that the performance of Paramter-L has obvious performance drop compared to Block-L in the Pet experiment, while in the CUB-semi experiment, the performance drop is less. This is likely because in the CUB-semi setting, we adopt the pretrained backbone for finetuning which can prevent overfitting problem to some extent, but in the Pet experiment we train from scratch.
Table 4: Performance of different choice of modules.
Dataset
Parameter-L
Block-L
Model-L
CUB-semi 52.48o.79
52.366.64
51.160.42
Pet 66. 14.640
69.61o.93
67.180.75
4.5 Module Complexity Influence on Importance with L2 Regularizer
We also explore how the module complexity will influence the learned importance when the auxiliary loss is the I.2 reg-
0.016
30.014
ularizer. We change the hidden dimensions of the task head
5..010
30.01
(2-layer MLP) of AutoINT the Beauty dataset, and plot the
learned importance for each module in Figure S Specifically, we change the hidden dimension of module6 from to 128, and plot their auxiliary module-level importance. It can be that small hidden dimensions like hidden8 and hidden16 will finally bring small regularizer for this module while larger hid- den dimensions like hidden 128 will bring comparatively larger importance. This fits our experience that the higher module complexity requires a larger regularizer to prevent overfitting.
0.006
20,008
Noden
Noden16
Nodend
0.004
Noden64
0.002
Noden 128
0.000
3 module index
Figure 5: Importance changes with module hidden dimension
5 Conclusion
In this paper, we propose module-aware optimization approach for the commonly encountered auxiliary learning scenario. Our method fills the gap that there exists module-level influence of auxiliary losses but there is no general method that can deal with this effect. We conduct extensive experiments to show that our proposed method can achieve superior performance with auxiliary losses from different scenarios, which could serve as powerful tool when we want to obtain benefits from the auxiliary losses. Our method has no negative social impact. Despite its effectiveness, the only limitation of our method could be the computational cost is almost the same as that of the GCS baseline, i.e., we should calculate the gradient for each auxiliary loss and the training time is linear to the number of auxiliary losses. Reducing this kind of additional cost is an interesting future direction.
Acknowledgement
This work was supported in part by the National Key Research and Development Program of China No. 2020AAA0106300 and National Natural Science Foundation of China (No. 62250008, 62222209, 62 62102222).
10

---

References
[1] Lucas Beyer, Xiaohua Zhai, Avital Oliver, and Alexander Kolesnikov. S4L: self-supervised sumi-super learning. In ICCV 2019, Seoul, Korea (South), October 27- November 2019, pages 1476-1485, 2019.
[2] Shikun Liu, Andrew Davison, and Edward Johns. Self-supervised generalisation with meta auxiliary learning. Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 1677-1687,
2019.. 2019.
[3] Hong Wen, Jing Zhang, Yuan Wang, Fuyu Lv, Wentian Bao, Quan Lin, and Keping Yang. Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In SIGIR 2020, Virtual Event, China, July 25-30, 2020, pages 1377-2386- 2020.
[4] Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang. Zhongyuan Wang. and Ji-Rong Wen. S3-rec: supervide learning for sequential recommendation with mutual information maximization. In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Viriual Event, Ireland, October 19-23, 2020, pages 1893-1902. ACM, 2020.
[5] Jiaqi Ma. Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1930-1939, 2018.
[6] Xingyu Lin, Harjatin Singh Baweja, George Kantor, and David Held. Adaptive auxiliary task weighting for reinforcement learning. In NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 777444784,22 2019.
[7] Max Jaderberg. Volodymyr Mnih, Wojeiech Marian Czamecki, Tom Schaul, Joel 7. Leibo, David Silver, and Koray Kavukcuoglu. Reinforcement learning with unsupervised auxiliary tasks, In 5th International Conference on Learning Reprexentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings. OpenReview.net, 2017.
[8] Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi-task learning. In European conference on computer vision, pages 94- 108. Springer, 2014.
[9] Yunshu Du, Wojeiech M. Czamecki, Siddhant M. Jayakumar, Razvan Pascanu, and Balaji Lakshmi- narayanan. Adapting auxiliary losses using gradient similarity. CoRR. 2018
[10] Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, and Huijuan Xu. Auxiliary task reweighting for minimum-date learning. In NeurlPS 2020, December 6-1 2020, virtual, 2020.
[11] Aviv Navon, Idan Achituve, Haggai Maron, Gal Chechik, and Ethan Fetaya. Auxiliary learning by implicit differentiation. In 9th International Conference on Learning Representations, ICLR 2021. Virtual Event, Austria, May 3-7, 2021, 2021.
[12] Hong Chen, Xin Wang, Chaoyu Guan, Yue Liu, and Wenwu Zhu, Auxiliary learning with joint task and data scheduling. In International Conference Machine Learning, pages 3634 3647. PMLR, 2022.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770-778. IEEE Computer Society, 2016.
[14] Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. BERT: pre- pre-training of deep bidirectional transformers for language understanding. In Proceddings of the 2019 Conference of the North American Chapter of the Association for Computational Länguisties: Human Language Technologies, NAACL-HIIT 2019, Minneopolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers), pages 4171-4186. Association for Computational Linguistics, 2019.
[15] Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris. Fully- feature sharing in multi-task networks with applications person attribute classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5334-5343, 2017.
[16] Simon Vandenhende, Stamatios Georgoulis, Bert De Brabandere, and Lue Van Gool. Branched multi-task networks: deciding what layers to share. arXiv preprini arXivv1904.02900 0229000, 2019.
[17] Linmei Hu, Siyong Xu, Chen Li, Cheng Yang, Chuan Shi, Nan Duan, Xing Xie, and Ming Zhou. Graph neural news recommendation with unsupervised preference disentanglement. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4255-4264, 2020.
11

---

[18] Liwei Wu, Shuging Li, Cho-Jui Hsieh, and James Sharpnack. Sse-pt: Sequential recommendation via personalized transformer. In Fourteenth ACM Conference Recommender Systems, pages 328-337, 2020.
[19] Lucio M. Dery, Yann N. Dauphin, and David Grangier. Auxiliary task update decomposition: the good, the bad and the neutral. In 9th International Conference on Learning Representations, ICI.R 2021, Viriual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[20] Michael Crawshaw. Multi-task learning with deep neural networks: A survey. arXiv preprint
mxXiv:200909999 2020.
[21] Pengsheng Guo, Chen- Yu Lee. and Daniel Ulbricht. Learning to branch for multi-task learning. In International Conference on Machine Learning, pages 3854-3863, PMLR, 2020.
[22] Alex Kendall, Yarin Gal, and Roherto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In CVPR 2018, Salt Lake City, UT, USA, Jume 18-22, 2018, pages 7482-7491, 2018.
[23] Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normal- ization for adaptive loss balancing in deep multitask networks. In ICML 2018, pages 793 802. PMLR, 2018.
[24] Victor Sanh, Thomas Wolf, and Sebastian Ruder. A hierarchical multi-task approach for learning embed- dings from semantic tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6949-6956, 2019.
[25] Jie Song. Yixin Chen, Xinehao Wang. Chengehao Shen, and Mingli Song. Deep model transferability from attribution maps, Advances Neural Information Prucessing Systems, 32, 2019.
[26] Zirui Wang. Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In 9th International Conference on Learning Representations, ICLR 2, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[27] Hong Chen, Yudong Chen, Xin Wang, Ruobing Xie, Rui Wang. Feng Xia, and Wenwu Zhu. Curriculum disentangled recommendation with noisy multi- neltbi.feedbak. Advances Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 26924-26936. 2021.
[28] Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. Learning disentangled representations for recommendation. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 5712-5723, 2019.
[29] Xin Wang, Hong Chen, and Wenwu Zhu. Multimodal disentangled representation for recommendation. In 2021 IEEE International Conference Multimedia and Expo, ICME 2021, Shenzhen, China, July 5-9, 2021, pages 6. IEEE, 2021.
[30] Xin Wang, Hong Chen, Yuwei Zhou, Jianxin Ma, and Wenwu 7hu, Disentangled representation learning for recommendation. IEEF Transactions on Pattern Analysis and Machine Intelligence, 2022.
[31] Wenwu Zhu, Xin Wang, and Pengtao Xie. Self-directed machine learning. arXiv preprint arXiv:2201.01220 2022.
[32] Yudong Chen, Xin Wang, Miao Fan, Jizhou Huang, Shengwen Yang, and Wenwu Zhu. Curriculum meta-learning for next POI recommendation. In KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pages 2692-2702. ACM, 2021.
[33] Zeyang Zhang, Ziwei Zhang, Xin Wang, and Wenwu Zhu. Learning solve travelling salesman problem with hardness- adaptive curriculum. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022. pages 9136 9144, AAAI Press, 2022.
[34] Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander G. Hauptmann. Self-paced curriculum learning. In Blai Bonet and Sven Koenig, editors, Proceedings of the Twenty-Ninth AAAI Conference on Artificial Inteligence, January 25-30, 2015, Austin, Texas, USA, pages 2694-2700. AAAI Press, 2015.
[35] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta- weitht-net: Learning an explicit mapping for sample weighting. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14,
2019, Vancouver, BC, Canada, pages 1917-1928, 2019.
12

---

[36] Yijian Qin. Xin Wang, Peng Cui, and Weawu Zhu. Gqnas: Graph network for neural architecture search. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1288-1293. IEEE, 2021.
[37] Vijian Qin, Xin Wang, 7eyang 7hang, and Wenwu Zhu. Graph differenniablee architecture search with structure learning. Advances in Neural Information Processing Systems, 34:1 14:16660-1687,, 2021.
[38] Chaoyu Guan, Xin Wang. Hong Chen, Ziwei Zhang, and Wenwu Zhu. Large-scale graph neural architecture search. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 7968 7981. PMLR, 2022.
[39] Yijian Qin, Xin Wang. Ziwei Zhang, Pengtao Xie, and Wenwu Zhu. Graph neural architecture search under distribution shifts. In International Conference on Machine Learning, pages -1899. PMLR, 2022.
[40] Yuwei Zhou, Xin Wang, Hong Chen, Xuguang Duan, Chaoyu Guan, and Wenwu Zhu. Curriculum-nas: Curriculum weight-sharing neural architecture search. MM '22, Lisboa, Portugal, 2022. Association for Computing Machinery.
[41] Guohao Li, Xin Wang. and Wenwu Zhu. Perceptual visual reasoning with knowledge propagation. In Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, Nice, France, October 21-25, 2019, pages 530-538. ACM, 2019.
[42] Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit diffcrentiation. In The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26 28 August 2020, Online [Palermo, Sicily, Italy], Proceedings of Machine Learning Research, pages 1550-1552, PMLR, 2020.
[43] Ross M. Clarke, Elre Oldewage, José Miguel Hernándezz Scalable one-pass optimisation of high-dimensional weight- weight-update hyperparameters by implicit differentiation. CoRR, abs/21 10. 10461, 2021.
[44] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSI Birds-200-2011 Dataset. Technical Report CNS-TR- TR-2011-001 California Institute of Technology, 2011.
[45] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar. Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3498-3505, 2012.
[46] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Handbook of Syxtemic Autoimmune Diseases, 1(4), 2009.
[47] Ruining He and Julian J. McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Jacqueline Bourdeau, Jim Hendler, Roger Nkambou, Ian Horrocks, and Ben Zhao, editors, Proceedings of the 25th International Conference on World Wide Web, WWW 2016. Monireal, Canada, April 11 15, 2016, pages 507-517, ACM, 2016.
[48] F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 19:111991 2016.
[49] Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via aelf-attentive neural networks. In Wenwu Zhu, Dacheng Tao, Xueqi Cheng, Peng Cui, Elke A. Rundensteiner, David Carmel, Qi He, and Jeffrey Xu Yu, editors. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019, pages 1161-1170. ACM, 2019.
[50] Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012.
[51] Mingxing Tan and Quoe V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97, pages 6105-6114, PMLR, 2019.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See the Conclusion section.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See the Conclusion section.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Refer to the supplementary file.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes] Refer the supplementary file.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "v0hD55mbEPd", "reviewer": "Reviewer_s6Tb", "summary": "This paper proposes a module-aware optimization method for a commonly adopted learning paradigm, learning with auxiliary losses. Inspired by multi-task learning, the authors point out that different auxiliary losses will contribute differently to different modules of the model for improving the primary task. To obtain the optimal importance of each auxiliary loss to each module, the authors design an efficient bi-level optimization based method. Extensive experiments on several datasets and tasks show that the proposed method for auxiliary learning is effective and generic. Additionally, the authors conduct comprehensive ablations from different aspects, which offers insights for auxiliary learning in various scenarios.", "questions": "1. The authors propose to utilize the validation set to prevent overfitting the developing set and the training set. I’d like to know whether the validation set for the proposed method and the baselines are the same, or the validation set for the baselines contain the developing set?\n2. I’d also like to see the time complexity of different baselines and this method.\n", "limitations": "The authors point that the time complexity is the limitation of this method, but it needs more discussing as above. ", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.\tThis paper is well-motivated. The module-aware optimization is important to addressing the module-level conflicts between the primary task and the auxiliary losses. \n2.\tThe related work and the contributions of this paper are sufficiently discussed. It is clearly discussed the differences between existing auxiliary learning, multi-task learning methods and the proposed method. The necessity of this work is obvious.\n3.\tThe proposed method is novel and efficient. The formulated bi-level optimization problem requires to make the gradient of the model parameterized, which is usually memory and time consuming to conduct the upper optimization. With the hyper-gradient derivation, the authors adopt the efficient best-response approximation, which gives a nice solution.\n4.\tThe experiments with different datasets, backbones and the ablations are convincing. The analysis of the whole experiments is quite insightful, and some results are quite interesting, like the learned module-level weights changing with the complexity of the modules.\n5.\tThis paper is well-written and easy to follow. The proposed method is a general method and can be easily applied to various auxiliary learning scenario, which could be a useful plug-in for many tasks.  \n\nWeaknesses:\n1.\tThe weakness of this paper lies in that although the proposed method is efficient compared to the directly unrolling method. The time complexity of both the baselines and this method should also be discussed so that readers can better balance these methods.\n2.\tAlthough the module choice effect is shown in the experiments, there needs more analysis and instructions for readers to decide when to conduct finer-grained module split or must we conduct parameter-level module split with collecting more data in the developing set?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "s6NxO-o7Rg_", "reviewer": "Reviewer_xStN", "summary": "Traditional approaches to auxiliary learning learn a single scalar per-task to adapt auxiliary tasks to the primary task. \nThis paper suggests to learn a scalar per-module for each auxiliary task. The motivation behind this is to allow more fine-grained adaptability in tuning the auxiliary task to improve the primary task. The authors show through experiments that this is an improvement over past methods they considered.", "questions": "1. Could you state the criteria you used to bold the numbers in your tables ? If it is highest average, please state this clearly. Some of the bolded numbers are *not statistically significant* but are still bolded.\n\n2. Were all the table 1, 2 experiments conducted with Block-L ?\n\n3. Is there any principled way / recommendation for deciding when to use Parameter/Block/Model-L ?\n\n4. How do the results change with model size ? With the size of the dev set ?\n\n5. It seems that for each experiment/data all methods only used a single learning rate ? It would be worth strengthening all methods by increasing the space of hyper-parameters (especially learning rate) searched over\n\n\nLess of a question and more of a note - It would be interesting to see the evolution of the module level weights over time instead of just at the end of training as provided in Figure 4 .", "limitations": "This work introduces multiple hyper-parameters which may have to be cross-validation\n1. The granularity of task adaptivity\n2. M - the number of look-back steps\n\nComputing the meta-gradient adds extra overhead but the runtime compared to simple fixed unitary weighting is not discussed", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper is well written and easy to parse\n2. The motivations for the work we clearly stated and situated in the context of previous work\n3. Experimental evaluation over several datasets and multiple baselines over multiple seeds with calculated standard deviations.\n\n**Weaknesses**\n\nMy primary concern with this paper is that the method seems incremental. Particularly,  all the core elements in this work can be stitched together from previous work (which itself would not be criticism per-se if there was novelty in the way the stitching together was done). \nSpecifically, learning auxiliary task adaptive weights via bi-level optimization has been done in [1, 2]. \nFor adapting auxiliary tasks weights on a per-module bases, there are multiple related works that are not referenced.  Layer-adaptive weights are not uncommon in the multitask literature see Section 3.2 of [3] which introduces an index k, for each layer. \nAlso note that in [4], they use the primary task to learn a subspace within which they deconstruct the auxiliary task weights - this abstracts away the choice of Parameter/Block/Model-L introduced in this paper - and represents an automated \"modularization\" approach\n\n\n**Relevant Related Work**\n\n[1] Should we be pre-training? an argument for end-task aware training as an alternative : https://arxiv.org/abs/2109.07437\n\n[2] Auxiliary learning by implicit differentiation:  https://arxiv.org/abs/2007.02693\n\n[3] Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models. https://arxiv.org/abs/2010.05874\n\n[4] Auxiliary task update decomposition: the good, the bad and the neutral. https://arxiv.org/abs/2108.11346", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ZPFe8_SafK", "reviewer": "Reviewer_NPMU", "summary": "This paper focuses on improving the effectiveness of leveraging auxiliary objectives for training deep neural network models. The authors propose a learning algorithm that jointly optimizes 1) the weight of each auxiliary loss on different parts (modules) of the model and 2) model parameters according to the weighted sum of the loss terms. Experiments are conducted on image classification and user rating prediction tasks to verify the efficacy of the proposed approach.", "questions": "Please see Weakness.", "limitations": "The authors address the limitation in Conclusion. No obvious potential negative social impact is observed.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Strengths*\n1. The paper is well-written. The idea is sound, and the technical details are clear. \n2. The proposed concept may have important contributions to the community. Indeed the impact of auxiliary training losses on different parts of the model is not well-understood yet.\n2. The paper provides some interesting observations in the experiments, especially the loss weight visualizations in Section 4.3.\n*Weakness*\n1. The experiments are conducted on a rather small scale (). The proposed approach could be more promising if the authors apply the algorithm to 1) more SOTA networks, e.g., VIT or EffiecientNet, and 2) more large-scale tasks, e.g., semi-supervised object detection.\n2. In Line 294, the authors use the descriptions, \"That is to say, the gradient from the auxiliary losses may guide θ to a better descent direction for the primary task in the early stages, but the gradient will show little importance when the better direction is found\" to explain those cases where the loss weight is learned to be 0. It would be great to visualize the change of the loss weight over time (training loops) to support the authors' arguments here.\n3. The authors should report the computational loss in the paper since it is the limitation of the proposed approach.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2aUwXDNUNJy", "reviewer": "Reviewer_94wP", "summary": "This paper proposes an auxiliary learning framework that updates network parameters in a module level via learnable module-level auxiliary importance. The proposed framework is updated with bi-level optimisation, a standard way to perform auxiliary learning with meta learning.\nAs such it contains two stages: the inner loop: it updates the network parameter by reweighting gradients of each auxiliary loss on each network module via the fixed auxiliary importance mask on the training data; the outer loop: it updates the auxiliary importance mask by the performance of primary task with M inner rollout steps, which uses best-response approximation (assuming network weightings are converged in inner loop) to reduce training time and memory on developing data (a small dataset to split from validation dataset). The framework eventually stores the best network which achieves the optimal validation performance.\n\nThe paper evaluates on multiple small-scale image classification datasets and user rating prediction datasets for regression with multiple network design and with supervised and semi-supervised learning settings. The auxiliary tasks are specifically designed for each dataset, e.g. using available attribute information as auxiliary tasks for fine-grained bird species classification, or using hand-crafted auxiliary tasks like rotation prediction for image classification datasets without attribute information. The proposed framework was compared to multiple well-known auxiliary learning methods, and was shown to improve the performance with a non-trivial margin.\n", "questions": "1. Is the auxiliary importance mask bounded or unbounded?\n\n2. Could you provide the experiments with 3 or more auxiliary tasks, in the standard multi-task/auxiliary task literature, e.g. in NYUv2, CityScapes, CIFAR-100/20 classification, as shown in [1, 2]?\n\n[1] Liu et al. Auto-Lambda: Disentangling Dynamic Task Relationships, 2022\n[2] Yu et al. Gradient Surgery for Multi-Task Learning, 2020\n\n3. Could you provide the experiments with Equal weighting, so it shows the benefit of auxiliary learning method with the same training data?\n\n4. Minor comment: The dataset chosen in Section 4.4 looks a bit suspicious, which includes two settings having the large performance gap. \n", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I found the paper is very easy to follow with very clean writing. The visualisation and explanation of auxiliary importance is also interesting to read.\n\nHowever, I think the design of experiments is a bit insufficient. \n1) the number of auxiliary tasks is only limited to two, most of the time only has one auxiliary task. \n\n2) the benefits on auxiliary tasks are not well reflected in the experiments. E.g. L267-269 claims the related tasks will bring more benefits, but Table 1 shows training with auxiliary tasks simply not brought any performance improvement, on CUB dataset. \n\n3) I wouldn’t call L2 regularisation is an auxiliary task, though it still might fits in a general definition… \n\n4) missing simple baseline: Equal weighting as multi-task learning, just to see the benefit of the proposed/baseline auxiliary methods with exact same training data.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept