Original: Module-Aware Optimization for Auxiliary Learning
Revised: Module-Aware Optimization for Auxiliary Learning

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Auxiliary learning is a widely adopted practice in deep learning, which aims toimprove the model performance on the primary task by exploiting the beneficialinformation in the auxiliary loss.
Revised: Auxiliary learning is a widely adopted practice in deep learning, which aims to improve the model performance on the primary task by exploiting the beneficial information in the auxiliary loss.

Original: Existing auxiliary learning methods only focuson balancing the auxiliary loss and the primary loss, ignoring the module-levelauxiliary influence, i.e., an auxiliary loss will be beneficial for optimizing specificmodules within the model but harmful to others, failing to make full use of auxiliaryinformation.
Revised: Existing auxiliary learning methods only focus on balancing the auxiliary loss and the primary loss, ignoring the module-level auxiliary influence, i.e., an auxiliary loss will be beneficial for optimizing specific modules within the model but harmful to others, failing to make full use of auxiliary information.

Original: To tackle the problem, we propose a Module-Aware Optimizationapproach for Auxiliary Learning (MAOAL).
Revised: To tackle the problem, we propose a Module-Aware Optimization approach for Auxiliary Learning (MAOAL).

Original: The proposed approach considers themodule-level influence through the learnable module-level auxiliary importance,i.e., the importance of each auxiliary loss to each module.
Revised: The proposed approach considers the module-level influence through the learnable module-level auxiliary importance, i.e., the importance of each auxiliary loss to each module.

Original: Specifically, the proposedapproach jointly optimizes the module-level auxiliary importance and the modelparameters in a bi-level manner.
Revised: Specifically, the proposed approach jointly optimizes the module-level auxiliary importance and the model parameters in a bi-level manner.

Original: In the lower optimization, the model parametersare optimized with the importance parameterized gradient, while in the upperoptimization, the module-level auxiliary importance is updated with the implicitgradient from a small developing dataset.
Revised: In the lower optimization, the model parameters are optimized with the importance parameterized gradient, while in the upper optimization, the module-level auxiliary importance is updated with the implicit gradient from a small developing dataset.

Original: Extensive experiments show that ourproposed MAOAL method consistently outperforms state-of-the-art baselines fordifferent auxiliary losses on various datasets, demonstrating that our method canserve as a powerful generic tool for auxiliary learning.
Revised: Extensive experiments show that our proposed MAOAL method consistently outperforms state-of-the-art baselines for different auxiliary losses on various datasets, demonstrating that our method can serve as a powerful generic tool for auxiliary learning 2 .

Original:  Introduction
Revised:  Introduction

Original: Auxiliary learning is a common practice in deep learning, which utilizes auxiliary losses (generallyfrom related tasks) to benefit the primary task in terms of model performance or generalization ability.
Revised: Auxiliary learning is a common practice in deep learning, which utilizes auxiliary losses (generally from related tasks) to benefit the primary task in terms of model performance or generalization ability.

Original: The auxiliary learning paradigm has shown its effectiveness in various areas includingimage classification [1, 2], recommendation [3–5], reinforcement learning [6, 7], etc.
Revised: The auxiliary learning paradigm has shown its effectiveness in various areas including image classification [1, 2], recommendation [3–5], reinforcement learning [6, 7], etc.

Original: Under variousscenarios, task-specific auxiliary losses are designed for different purposes, e g., in recommendation,previous works [5, 3] add the click-through rate prediction loss to help the click conversion rateprediction; in image classification [1], the self-supervised auxiliary loss is designed to improveclassification accuracy when labels are inadequate; and the losses of head pose estimation and facialattribute inference are utilized to aid facial landmark detection [8].
Revised: Under various scenarios, task-specific auxiliary losses are designed for different purposes, e g., in recommendation, previous works [5, 3] add the click-through rate prediction loss to help the click conversion rate prediction; in image classification [1], the self-supervised auxiliary loss is designed to improve classification accuracy when labels are inadequate; and the losses of head pose estimation and facial attribute inference are utilized to aid facial landmark detection [8].

Original: Existing auxiliary learning methods focus on balancing the importance of each auxiliary loss andthe primary loss.
Revised: Existing auxiliary learning methods focus on balancing the importance of each auxiliary loss and the primary loss.

Original: Most of them assign a predefined weight to each auxiliary loss and then optimizethe sum of the weighted auxiliary losses and the primary loss.
Revised: Most of them assign a predefined weight to each auxiliary loss and then optimize the sum of the weighted auxiliary losses and the primary loss.

Original: The predefined weights generallyneed carefully tuning with Hyper-parameter Optimization (HPO) tools to prevent negative auxiliarytransfer.
Revised: The predefined weights generally need carefully tuning with Hyper-parameter Optimization (HPO) tools to prevent negative auxiliary transfer.

Original: Recently, several works [9, 10, 6] are proposed to automatically assign weights to eachauxiliary loss based on the gradient similarity between each auxiliary loss and the primary loss.
Revised: Recently, several works [9, 10, 6] are proposed to automatically assign weights to each auxiliary loss based on the gradient similarity between each auxiliary loss and the primary loss.

Original: Themore recent work [11] further proposes to utilize a non-linear auxiliary loss combination to betterexploit the  beneficial  auxiliary information.
Revised: The more recent works [11] further proposes to utilize a non-linear auxiliary loss combination to better exploit the beneficial auxiliary information, and [12] proposes to jointly select the most beneficial task and data for auxiliary learning.

Original: Nevertheless, existing approaches ignore the important fact that an auxiliary loss could have differentinfluences on different modules within the model.
Revised: Nevertheless, existing approaches ignore the important fact that an auxiliary loss could have different influences on different modules within the model.

Original: Here we define a module to be a fine-grained part of the model, e.g., a module can be a block in ResNet [12], a transformer block in Bert [13], or alayer in a Multi-Layer-Perceptron (MLP).
Revised: Here we define a module to be a fine-grained part of the model, e.g., a module can be a block in ResNet [13], a transformer block in Bert [14], or a layer in a Multi-Layer-Perceptron (MLP).

Original: The necessity of considering the module-level influence ofauxiliary losses has been indicated by research related to multi-task learning.
Revised: The necessity of considering the module-level influence of auxiliary losses has been indicated by research related to multi-task learning.

Original: Previous multi-tasklearning works [14, 15] show that shallow layers (layers close to the input) contain more commoninformation among related tasks, while deep layers (layers close to the output) contain more taskspecific information that tends to cause negative transfer.
Revised: Previous multi-task learning works [15, 16] show that shallow layers (layers close to the input) contain more common information among related tasks, while deep layers (layers close to the output) contain more taskspecific information that tends to cause negative transfer.

Original: This phenomenon indicates that whenthe auxiliary loss comes from a related task, if we do not consider the module-level influence, thenegative information in the deep layers will counterbalance the benefits brought by the shallow layers,resulting in inefficient usage of auxiliary losses.
Revised: This phenomenon indicates that when the auxiliary loss comes from a related task, if we do not consider the module-level influence, the negative information in the deep layers will counterbalance the benefits brought by the shallow layers, resulting in inefficient usage of auxiliary losses.

Original: However, these multi-task learning works [14, 15]aim at designing model architectures, not suitable for the auxiliary learning scenario where the modelarchitecture is designed (and fixed) for the primary task, and our target is to utilize the auxiliary lossesto better optimize the model.
Revised: However, these multi-task learning works [15, 16] aim at designing model architectures, not suitable for the auxiliary learning scenario where the model architecture is designed (and fixed) for the primary task, and our target is to utilize the auxiliary losses to better optimize the model.

Original: In sum, considering the module-level influence is important to auxiliarylearning, and there has been no work on auxiliary learning considering this module-level influence.
Revised: In sum, considering the module-level influence is important to auxiliary learning, and there has been no work on auxiliary learning considering this module-level influence.

Original: To deal with the module-level auxiliary influence, we propose a module-aware optimization approachfor auxiliary learning (MAOAL).
Revised: To deal with the module-level auxiliary influence, we propose a module-aware optimization approach for auxiliary learning (MAOAL).

Original: The comparison of the existing methods and our module-awareoptimization approach is shown in Figure 1.
Revised: The comparison of the existing methods and our module-aware optimization approach is shown in Figure 1.

Original: The proposed MAOAL approach considers the modulelevel influence of auxiliary losses by introducing the learnable module-level auxiliary importance,which reflects the importance of each auxiliary loss to each module within the model.
Revised: The proposed MAOAL approach considers the modulelevel influence of auxiliary losses by introducing the learnable module-level auxiliary importance, which reflects the importance of each auxiliary loss to each module within the model.

Original: Specifically,the module-level auxiliary importance and the model parameters are jointly optimized in a bi-levelmanner.
Revised: Specifically, the module-level auxiliary importance and the model parameters are jointly optimized in a bi-level manner.

Original: In the lower optimization, the model parameters are updated with the module-level auxiliaryimportance parameterized gradient, so that each auxiliary loss can be used to optimize the module itis beneficial to, making each module of the model play a better role for the primary task.
Revised: In the lower optimization, the model parameters are updated with the module-level auxiliary importance parameterized gradient, so that each auxiliary loss can be used to optimize the module it is beneficial to, making each module of the model play a better role for the primary task.

Original: In the upperoptimization, the module-level auxiliary importance is updated with the implicit gradient from a smalldeveloping dataset, making the module-level auxiliary importance more accurate.
Revised: In the upper optimization, the module-level auxiliary importance is updated with the implicit gradient from a small developing dataset, making the module-level auxiliary importance more accurate.

Original: Additionally, thebest-response approximation is utilized for efficient implicit gradient calculation.
Revised: Additionally, the best-response approximation is utilized for efficient implicit gradient calculation.

Original: Since the modulelevel auxiliary importance is learned by a data-driven optimization process without requiring specificknowledge of the auxiliary task, the proposed module-aware optimization method can accommodate avariety of auxiliary losses.
Revised: Since the modulelevel auxiliary importance is learned by a data-driven optimization process without requiring specific knowledge of the auxiliary task, the proposed module-aware optimization method can accommodate a variety of auxiliary losses.

Original: We conduct experiments on different tasks and datasets, and experimentalresults show that our method consistently outperforms state-of-the-art methods.
Revised: We conduct experiments on different tasks and datasets, and experimental results show that our method consistently outperforms state-of-the-art methods.

Original: Particularly, when weconsider a wider range of auxiliary losses, such as adopting L2 regularization [16, 17] as an auxiliaryloss, our method can also deal with the module-level influence in these situations and bring bettermodel performance.
Revised: Particularly, when we consider a wider range of auxiliary losses, such as adopting L2 regularization [17, 18] as an auxiliary loss, our method can also deal with the module-level influence in these situations and bring better model performance.

Original: We summarize our contributions as follows.
Revised: We summarize our contributions as follows.

Original: (i)
Revised: (i)

Original: We are the first to investigate the module-levelinfluence in the general auxiliary learning setting.
Revised: We are the first to investigate the module-level influence in the general auxiliary learning setting.

Original: (ii) We propose a module-aware optimization(MAOAL) method for auxiliary learning, which utilizes the module-level auxiliary importanceparameterized gradient to optimize the model parameters, and optimizes the module-level importancewith the implicit gradient.
Revised: (ii) We propose a module-aware optimization (MAOAL) method for auxiliary learning, which utilizes the module-level auxiliary importance parameterized gradient to optimize the model parameters, and optimizes the module-level importance with the implicit gradient.

Original: (iii) Extensive experiments demonstrate that our method consistentlyoutperforms state-of-the-art baselines in various scenarios.
Revised: (iii) Extensive experiments demonstrate that our method consistently outperforms state-of-the-art baselines in various scenarios.

Original: Analysis of the modules and the modulelevel auxiliary importance provides insights for auxiliary learning.
Revised: Analysis of the modules and the modulelevel auxiliary importance provides insights for auxiliary learning.

Original: [Figure]
Revised: [Figure]

Original:  in module 1,2,3 and auxiliary loss 2 should update module 2,4 and 
Revised: and auxiliary losses to update the whole model parameters. In module-aware optimization, auxiliary loss 1 should update the parameters in module 1,2,3 and auxiliary loss 2 should update module 2,4 and 5.

Original:  Related Work
Revised:  Related Work

Original: Auxiliary Learning Auxiliary learning is a common practice in deep learning.
Revised: Auxiliary Learning Auxiliary learning is a common practice in deep learning.

Original: This learning paradigm aims to improve the model performance on the primary task with designed auxiliary losses (generally from auxiliary tasks).
Revised: This learning paradigm aims to improve the model performance on the primary task with designed auxiliary losses (generally from auxiliary tasks).

Original: Most existing works [1, 3] utilize the auxiliary losses by linearlycombining them with the primary loss and then utilizing the combined loss to optimize the wholemodel.
Revised: Most existing works [1, 3] utilize the auxiliary losses by linearly combining them with the primary loss and then utilizing the combined loss to optimize the whole model.

Original: The weights for the auxiliary losses will be tuned to avoid negative transfer to the primarytask.
Revised: The weights for the auxiliary losses will be tuned to avoid negative transfer to the primary task.

Original: Some recent works [9, 10, 6] propose to automatically weight the auxiliary losses during thetraining process in a dynamic way.
Revised: Some recent works [9, 10, 6, 19] propose to automatically weight the auxiliary losses during the training process in a dynamic way.

Original: Specifically, work [9] calculates the gradient of each auxiliaryloss, and utilizes the cosine similarity between the auxiliary gradient and that of the primary loss asthe auxiliary loss weight.
Revised: Specifically, work [9] calculates the gradient of each auxiliary loss, and utilizes the cosine similarity between the auxiliary gradient and that of the primary loss as the auxiliary loss weight.

Original: Work [10] shares a similar idea with [6], but with the goal that the weightedgradient sum should be close to the gradient of the primary task.
Revised: Work [10] shares a similar idea with [6], but with the goal that the weighted gradient sum should be close to the gradient of the primary task.

Original: Moreover, [11] proposes to learn anon-linear combination for auxiliary losses, showing stronger auxiliary information exploiting ability.
Revised: Moreover, [11] proposes to learn a non-linear combination for auxiliary losses, and [12] proposes to select not only the task but also each data sample within each task to make full use of the auxiliary information.

Original: However, these methods only conduct model-level optimization for auxiliary losses, ignoring the factthat one auxiliary loss could contribute differently to different modules within the model.
Revised: However, these methods only conduct model-level optimization for auxiliary losses, ignoring the fact that one auxiliary loss could contribute differently to different modules within the model.

Original: Multi-task learning Another line of highly related work to optimize several losses is multi-tasklearning.
Revised: Another line of highly related work to optimize several losses is multi-task learning.

Original: Different from the goal of auxiliary learning, multi-task learning aims to obtain goodperformance on all the learned tasks, while learning with auxiliary losses only focuses on the primarytask.
Revised: Different from the goal of auxiliary learning, multi-task learning aims to obtain good performance on all the learned tasks, while learning with auxiliary losses only focuses on the primary task.

Original: Existing multi-task learning methods can be roughly categorized to three parts [18]: multi-taskarchitecture design [19, 15], multi-task optimization [20–22] and multi-task relationship learning [23].
Revised: Existing multi-task learning methods can be roughly categorized to three parts [20]: multi-task architecture design [21, 16], multi-task optimization [22–24] and multi-task relationship learning [25].

Original: Multi-task architecture design methods aim at designing proper architectures for simultaneouslylearning multiple tasks.
Revised: Multi-task architecture design methods aim at designing proper architectures for simultaneously learning multiple tasks.

Original: Some architecture design works [15, 19] point that different tasks share morecommon information in the shallow layers, which inspires us that module-aware optimization forauxiliary losses is needed.
Revised: Some architecture design works [16, 21] point that different tasks share more common information in the shallow layers, which inspires us that module-aware optimization for auxiliary losses is needed.

Original: The multi-task optimization methods aim at optimizing the whole modelfor all tasks with methods like loss weighting [20, 21], which can be used to optimize multiple losses.
Revised: The multi-task optimization methods aim at optimizing the whole model for all tasks with methods like loss weighting [22, 23], which can be used to optimize multiple losses.

Original: The Proposed Method
Revised: Particularly, work [26] proposes to assign layer-wise to each loss, but it only focuses on multi-task setting and the weight is assigned based on heuristics, which cannot be directly applied to our general auxiliary setting where we only care about the performance of the primary task and the auxiliary losses might by some other regularization terms like L2 regularization or disentangled regularization [27–30]. Position of this work This work is mainly inspired by [31], where the machine is expected to automatically select the data, tasks and model to be learned. Previous works have focused on different aspects. [27, 32–35] focus on automatic data selection, [11, 6, 9] focus on automatic task selection, and [36–38] focus on automatic model designing.

Revised: More recently, some works focus on joint selection of different elements, where the selection space is much more complicated.

Revised: [12] proposes to joint select task and data, [39, 40] proposes to joint select data and model, and this work fills the joint task-model selection gap.

Original:   3.1 Preliminaries and Problem Formulation
Revised:  The Proposed Method 3.1 Preliminaries and Problem Formulation

Original: Assume that we have a primary task T goal and its corresponding loss L goal (primary loss).
Revised: Assume that we have a primary task T goal and its corresponding loss L goal (primary loss).

Original: Thereare totally K auxiliary losses { L a 1 , L a 2 , · · · , L ai , · · · , L aK } , utilized to help the primary task.
Revised: There are totally K auxiliary losses { L a 1 , L a 2 , · · · , L ai , · · · , L aK } , utilized to help the primary task.

Original: Let D train , D v respectively be the training and validation dataset.
Revised: Let D train , D v respectively be the training and validation dataset.

Original: The model for the primary task is parameterized by θ , which is a deep model composed of different modules, i.e.,θ =
Revised: The model for the primary task is parameterized by θ , which is a deep model composed of different modules, i.e., θ =

Original: [ θ 1 , θ 2 , · · · , θ j , · · · θ m ] .
Revised: [ θ 1 , θ 2 , · · · , θ j , · · · θ m ] .

Original: The partition for the modules can be flexibly defined by users, e.g., amodule can be a block in ResNet [12], a transformer block in Bert [13], or specified by its designedfunction [24].
Revised: The partition for the modules can be flexibly defined by users, e.g., a module can be a block in ResNet [13], a transformer block in Bert [14], or specified by its designed function [41].

Original: Previous approaches for auxiliary learning generally adopt the following objective:
Revised: Previous approaches for auxiliary learning generally adopt the following objective:

Original: [Equation]
Revised: [Equation]

Original: where w i is the weight used to balance the auxiliary losses and the primary loss.
Revised: where w i is the weight used to balance the auxiliary losses and the primary loss.

Original: With proper valuesset for w i , the model parameters θ are updated with the widely adopted gradient descent methodwhere t is the unrolled step:
Revised: With proper values set for w i , the model parameters θ are updated with the widely adopted gradient descent method where t is the unrolled step:

Original: [Equation]
Revised: [Equation]

Original:  Although this kind of learning paradigm has been widely used in various scenarios, it cannot dealwith the module-level influence as indicated in [19, 15].
Revised: [Figure] Although this kind of learning paradigm has been widely used in various scenarios [Figure] it cannot deal with the module-level influence as indicated in [21, 16].

Original: To avoid confusion, we remind that thesuperscript of θ j indexes the module within the model, and the subscript of θ t indexes the unrolledstep in the rest of the paper.
Revised: To avoid confusion, we remind that the superscript of θ j indexes the module within the model, and the subscript of θ t indexes the unrolled step in the rest of the paper.

Original: Our goal is to deal with the module-level influence of auxiliary losses and make each auxiliary lossoptimize the module which it is beneficial to.
Revised: Our goal is to deal with the module-level influence of auxiliary losses and make each auxiliary loss optimize the module which it is beneficial to.

Original: Specifically, we introduce the module-level auxiliaryimportance matrix
Revised: Specifically, we introduce the module-level auxiliary importance matrix

Original: α = { α ij } Ki =1 mj =1 , where α ij is the importance of auxiliary loss L ai to the moduleθ j , and then optimize the model parameters θ for the primary task according to the module-levelauxiliary importance.
Revised: α = { α ij } Ki =1 mj =1 , where α ij is the importance of auxiliary loss L ai to the module θ j , and then optimize the model parameters θ for the primary task according to the module-level auxiliary importance.

Original: Here two key problems are how to obtain the optimal module-level auxiliaryimportance and how to utilize the importance to optimize the model parameters θ .
Revised: Here two key problems are how to obtain the optimal module-level auxiliary importance and how to utilize the importance to optimize the model parameters θ .

Original: We optimize themodule-level auxiliary importance and the model parameters in a bi-level manner, and the wholeframework of the module-aware optimization process is shown in Figure 2.
Revised: We optimize the module-level auxiliary importance and the model parameters in a bi-level manner, and the whole framework of the module-aware optimization process is shown in Figure 2.

Original: In the lower optimization,we optimize the model parameters θ with the importance parameterized gradient.
Revised: In the lower optimization, we optimize the model parameters θ with the importance parameterized gradient.

Original: In the upperoptimization, we update the module-level auxiliary importance with the implicit gradient from adeveloping dataset D dev .
Revised: In the upper optimization, we update the module-level auxiliary importance with the implicit gradient from a developing dataset D dev .

Original: In the next two subsections, we describe the lower and upper optimizationin detail and summarize the complete algorithm in Section 3.4.
Revised: In the next two subsections, we describe the lower and upper optimization in detail and summarize the complete algorithm in Section 3.4.

Original: [Figure]

Original: 3.2 Lower Optimization: Importance Parameterized Gradient for Model Parameters
Revised: 3.2 Lower Optimization: Importance Parameterized Gradient for Model Parameters

Original: In the lower optimization, we optimize the model parameters θ on the training dataset D train .
Revised: In the lower optimization, we optimize the model parameters θ on the training dataset D train .

Original: Assuming that we have already known the module-level auxiliary importance α ij , we will utilize theimportance to guide the optimization process of the model parameters θ .
Revised: Assuming that we have already known the module-level auxiliary importance α ij , we will utilize the importance to guide the optimization process of the model parameters θ .

Original: We first obtain the gradientof each module θ j with respect to each auxiliary loss L ai :
Revised: We first obtain the gradient of each module θ j with respect to each auxiliary loss L ai :

Original: [Equation]
Revised: [Equation]

Original: Then we reweight the gradient of each auxiliary loss with the importance α ij and obtain the importance parameterized gradient ∇ θ ¯ L ( θ, α ; D train ) as follows:
Revised: Then we reweight the gradient of each auxiliary loss with the importance α ij and obtain the importance parameterized gradient ∇ θ ¯ L ( θ, α ; D train ) as follows:

Original: [Equation]
Revised: [Equation]

Original: where in Eq.(4), the gradient for each auxiliary loss is masked with its module-level auxiliaryimportance, where if the i th auxiliary loss is important to optimizing the j th module, the auxiliarymodule-level gradient ∇ θ j L ai will be weighted with a larger α ij and vice versa.
Revised: where in Eq.(4), the gradient for each auxiliary loss is masked with its module-level auxiliary importance, where if the i th auxiliary loss is important to optimizing the j th module, the auxiliary module-level gradient ∇ θ j L ai will be weighted with a larger α ij and vice versa.

Original: In Eq.(5), wecombine all the weighted auxiliary gradient and the gradient of the primary loss to obtain theimportance parameterized gradient, which will be finally used to optimize the model parameters θ .
Revised: In Eq.(5), we combine all the weighted auxiliary gradient and the gradient of the primary loss to obtain the importance parameterized gradient, which will be finally used to optimize the model parameters θ .

Original: After that, we utilize the widely adopted unrolled gradient descent to optimize the model parametersθ as follows, where η 1 is the learning rate.
Revised: After that, we utilize the widely adopted unrolled gradient descent to optimize the model parameters θ as follows, where η 1 is the learning rate.

Original: [Equation]
Revised: [Equation]

Original: 3.3 Upper Optimization: Implicit Gradient for Module-level Auxiliary Importance
Revised: 3.3 Upper Optimization: Implicit Gradient for Module-level Auxiliary Importance

Original: In the upper optimization, our target is to optimize the module-level auxiliary importance α ij .
Revised: In the upper optimization, our target is to optimize the module-level auxiliary importance α ij .

Original: Toobtain the gradient of α , we introduce a small developing dataset D dev as in [25].
Revised: To obtain the gradient of α , we introduce a small developing dataset D dev as in [42].

Original: D dev is a small-sizedataset for the primary task split from the validation dataset D v .
Revised: D dev is a small-size dataset for the primary task split from the validation dataset D v .

Original: Since we expect that our model canfinally perform well on the validation dataset D v , our model should also perform well enough on itssubset D dev .
Revised: Since we expect that our model can finally perform well on the validation dataset D v , our model should also perform well enough on its subset D dev .

Original: Therefore, it is natural to utilize the loss on D dev to optimize the importance matrix α .
Revised: Therefore, it is natural to utilize the loss on D dev to optimize the importance matrix α .

Original: Assume that we have conducted M steps of lower optimization, and the current model parameter is θ M .
Revised: Assume that we have conducted M steps of lower optimization, and the current model parameter is θ M .

Original: The loss of the model on the developing dataset L goal ( θ M ( α ); D dev ) can be calculated.
Revised: The loss of the model on the developing dataset L goal ( θ M ( α ); D dev ) can be calculated.

Original: Note that weonly calculate the primary loss on D dev because we actually only care about the model performanceon the primary task.
Revised: Note that we only calculate the primary loss on D dev because we actually only care about the model performance on the primary task.

Original: Directly calculating the gradient of α with respect to L goal ( θ M ( α ); D dev ) isnot easy.
Revised: Directly calculating the gradient of α with respect to L goal ( θ M ( α ); D dev ) is not easy.

Original: The loss L goal ( θ M ( α ); D dev ) directly relies on the model parameters θ M instead of the importance matrix α .
Revised: The loss L goal ( θ M ( α ); D dev ) directly relies on the model parameters θ M instead of the importance matrix α .

Original: α implicitly influences the value of L goal ( θ M ( α ); D dev ) by influencing thegradient of θ as shown in the lower optimization in Eq.(6).
Revised: α implicitly influences the value of L goal ( θ M ( α ); D dev ) by influencing the gradient of θ as shown in the lower optimization in Eq.(6).

Original: According to this observation, we use thechain rule to obtain the the gradient of α with respect to L goal ( θ M ( α ); D dev ) :
Revised: According to this observation, we use the chain rule to obtain the the gradient of α with respect to L goal ( θ M ( α ); D dev ) :

Original: [Equation]
Revised: [Equation]

Original: The term ∇ θ M L goal ( θ M ( α ); D dev ) is easy to obtain using the autograd tools.
Revised: The term ∇ θ M L goal ( θ M ( α ); D dev ) is easy to obtain using the autograd tools.

Original: However, the term∇ α θ M requires more complex exploration to the lower unrolled steps.
Revised: However, the term ∇ α θ M requires more complex exploration to the lower unrolled steps.

Original: By differentiating and unrollingthe process in Eq.(6), we obtain the following results:
Revised: By differentiating and unrolling the process in Eq.(6), we obtain the following results:

Original: [Equation]
Revised: [Equation]

Original: where τ indexes the steps we look back, summing up all the M looking-back results gives the desiredgradient.
Revised: where τ indexes the steps we look back, summing up all the M looking-back results gives the desired gradient.

Original: This means that if we know the Jacobi and the Hessian within the past M steps, we can usethem to calculate ∇ α θ M , the gradient of α with respect to current model parameters θ M .
Revised: This means that if we know the Jacobi and the Hessian within the past M steps, we can use them to calculate ∇ α θ M , the gradient of α with respect to current model parameters θ M .

Original: Detailedderivation can be found in the supplementary file.
Revised: Detailed derivation can be found in the supplementary file.

Original: However, this kind of calculation is memoryconsuming because we have to restore the Jacobi and Hessian during the past M steps.
Revised: However, this kind of calculation is memoryconsuming because we have to restore the Jacobi and Hessian during the past M steps.

Original: To improveefficiency, we follow the best-response approximation in [25] and assume that θ 0 , θ 1 , · · · , θ M − 1 allequal to θ M .
Revised: To improve efficiency, we follow the best-response approximation in [42] and assume that θ 0 , θ 1 , · · · , θ M − 1 all equal to θ M .

Original: With this assumption, we can approximate Eq.(8) in the following manner:
Revised: With this assumption, we can approximate Eq.(8) in the following manner:

Original: [Equation]
Revised: [Equation]

Original: α ; D train ) , (9) where we only use the Hessian and Jacobi at the current step.
Revised: where we only use the Hessian and Jacobi at the current step.

Original: Replacing the term ∇ α θ M in Eq.(7)with Eq.(9) gives the desired gradient of α .
Revised: Replacing the term ∇ α θ M in Eq.(7) with Eq.(9) gives the desired gradient of α .

Original: Moreover, the calculation for Eq.(7) can be performedefficiently using vector-Jacobian products [25] as shown in Algorithm 1, which is both memory andcomputation efficient.
Revised: Moreover, the calculation for Eq.(7) can be performed efficiently using vector-Jacobian products [42] as shown in Algorithm 1, which is both memory and computation efficient.

Original: This kind of approximation assumes that θ τ (0 ≤ τ < M ) equals to θ M , whichwill be more accurate when θ τ is around the best response θ ∗ under given α (the best response θ ∗ isthe point that the unrolled process in Eq.(6) converges to when α is fixed).
Revised: This kind of approximation assumes that θ τ (0 ≤ τ < M ) equals to θ M , which will be more accurate when θ τ is around the best response θ ∗ under given α (the best response θ ∗ is the point that the unrolled process in Eq.(6) converges to when α is fixed).

Original: This is why it is calledbest-response approximation in [25].
Revised: This is why it is called best-response approximation in [42].

Original: The effectiveness of this approximation is practically validatedin both previous works [25, 26, 11] and our experiments.
Revised: The effectiveness of this approximation is practically validated in both previous works [42, 43, 11] and our experiments.

Original: 3.4 Overall Algorithm
Revised: 3.4 Overall Algorithm

Original: With both the lower and upper optimization above, we summarize the complete module-awareoptimization algorithm for auxiliary learning in Algorithm 1.
Revised: With both the lower and upper optimization above, we summarize the complete module-aware optimization algorithm for auxiliary learning in Algorithm 1.

Original: The input for the algorithm containsthree datasets: the training dataset D train , the developing dataset D dev and the validation dataset
Revised: The input for the algorithm contains three datasets: the training dataset D train , the developing dataset D dev and the validation dataset

Original: D v , and the hyperparameters.
Revised: D v , and the hyperparameters.

Original: T and η 1 are used for the lower optimization, where T is the steps weconduct lower optimization in one loop and η 1 is the learning rate for the lower optimization.
Revised: T and η 1 are used for the lower optimization, where T is the steps we conduct lower optimization in one loop and η 1 is the learning rate for the lower optimization.

Original: M andη 2 are used for the upper optimization, where M is the total looking-back steps in Eq.(9) and η 2 isthe learning rate for the upper optimization.
Revised: M and η 2 are used for the upper optimization, where M is the total looking-back steps in Eq.(9) and η 2 is the learning rate for the upper optimization.

Original: The model parameters are randomly initialized and theimportance matrix is initialized with all its elements 1.0.
Revised: The model parameters are randomly initialized and the importance matrix is initialized with all its elements 1.0.

Original: Specifically, in line 4 to line 6, we conductlower optimization for T times.
Revised: Specifically, in line 4 to line 6, we conduct lower optimization for T times.

Original: In line 15 to line 22, we conduct the upper optimization, whereline 16 to line 21 is the process that utilizes the vector-Jacobian products to calculate the gradient in
Revised: In line 15 to line 22, we conduct the upper optimization, where line 16 to line 21 is the process that utilizes the vector-Jacobian products to calculate the gradient in

Original: Eq.(7).
Revised: Eq.(7).

Original: The lower optimization and the upper optimization are iteratively conducted until converged.
Revised: The lower optimization and the upper optimization are iteratively conducted until converged.

Original: Particularly, in line 7 to line 14, we will evaluate the model performance on the validation dataset D vafter each interval loops of lower-and-upper optimization.
Revised: Particularly, in line 7 to line 14, we will evaluate the model performance on the validation dataset D v after each interval loops of lower-and-upper optimization.

Original: We always record the θ with the bestperformance in θ opt , and finally return the parameters θ opt .
Revised: We always record the θ with the best performance in θ opt , and finally return the parameters θ opt .

Original: The use of the validation dataset is toprevent the model from overfitting D train or D dev .
Revised: The use of the validation dataset is to prevent the model from overfitting D train or D dev .

Original: Note that we do not directly look back the total
Revised: Note that we do not directly look back the total

Original: T steps in the upper optimization, but only look back M steps( M < T ) for faster computation of
Revised: T steps in the upper optimization, but only look back M steps( M < T ) for faster computation of

Original: Eq.(7), which is also adopted by [25, 11].
Revised: Eq.(7), which is also adopted by [42, 11].

Original: In our algorithm, each auxiliary loss enjoys an independent vector α i = { α ij } mj
Revised: In our algorithm, each auxiliary loss enjoys an independent vector α i = { α ij } mj

Original: =1 , enabling it notonly to balance losses, but also consider how important each auxiliary loss is to each module.
Revised: =1 , enabling it not only to balance losses, but also consider how important each auxiliary loss is to each module.

Original: Withoutrequiring specific knowledge of auxiliary losses, the data-driven optimization of the module-levelauxiliary importance enables our method to be applied to the general setting of auxiliary learning.
Revised: Without requiring specific knowledge of auxiliary losses, the data-driven optimization of the module-level auxiliary importance enables our method to be applied to the general setting of auxiliary learning.

Revised: L ( θ, α ; D train ) , θ, grad _ outputs = v ) 20: p ← p + v 21: end for 22: ∇ α L goal ( θ ; D dev )

Revised: = − grad ( η 1 ∇ θ ¯ L ( θ, α ; D train ) , α, grad _ outputs = p ) 23:

Revised: α ← α − η 2 ∇ α L goal ( θ ; D dev ) 24: end while 25: Return θ opt

Original:  Experiments
Revised:  Experiments

Original: In this section, we empirically assess the efficacy of our method on various tasks and datasets.
Revised: In this section, we empirically assess the efficacy of our method on various tasks and datasets.

Original: Also,we provide ablations on how our method works and the module influence under different scenarios.
Revised: Also, we provide ablations on how our method works and the module influence under different scenarios.

Original: 1: input: dataset: D train , D dev , D v ; hyperparameters: T, M, η 1 , η 2 , interval 2: initialize: θ , α , counter = 0 , performance opt = 0 , θ opt = θ 3: while not converged do 4: // lower optimization: update θ with fixed α 5: for t = 0 to T − 1 do 6:

Original: Update model parameters θ using Eq.(6); 7: end for 8: // evaluate the model on D v 9: counter = counter +10: if counter % interval =

Original: = 0 then 11: performance =

Original: Metric ( θ ; D v ) ; 12: if performance > performance opt then 13: performance opt = performance, θ opt = θ ; 14: end if 15: end if 16: //

Original: upper optimization: update α with current θ ; 17: p = v

Original: ∇

Original: L θ ; D

Original: ) ; 18: for τ = 1 to M do 19: v ← v − grad

Original: ∇ θ ¯ θ α ; D train ) θ grad outputs v ) 20: p ← p + v 21: end for 22: ∇ α L goal ( θ ; D dev )

Original: = − grad ( η 1 ∇ θ ¯ θ, α ; D

Original: ) , α, grad p ) 23: α ← α − dev ) 24: end while 25: Return θ opt

Original: 4.1 Experimental Setup
Revised: 4.1 Experimental Setup

Original: Task and datasets (i)
Revised: Task and datasets (i)

Original: We first focus on the image classification problem with auxiliary tasks.
Revised: We first focus on the image classification problem with auxiliary tasks.

Original: Weconduct experiments on two fine-grained image classification datasets, CUB [27] and Oxford-IIIT
Revised: We conduct experiments on two fine-grained image classification datasets, CUB [44] and Oxford-IIIT

Original: Pet [28], and two widely adopted general image classification datasets, CIFAR10 and CIFAR100 [29].
Revised: Pet [45], and two widely adopted general image classification datasets, CIFAR10 and CIFAR100 [46].

Original: Specifically, on the CUB dataset, there are total 200 species of birds and each image has the label ofits attributes, like the wing color and bill shape.
Revised: Specifically, on the CUB dataset, there are total 200 species of birds and each image has the label of its attributes, like the wing color and bill shape.

Original: In general, it requires expert knowledge to classifythe birds, but it is much easier to discriminate the attributes.
Revised: In general, it requires expert knowledge to classify the birds, but it is much easier to discriminate the attributes.

Original: Thus, it is a natural practice to regardbird species classification as the primary task, and use the wing color classification and bill shapeclassification losses as the auxiliary losses.
Revised: Thus, it is a natural practice to regard bird species classification as the primary task, and use the wing color classification and bill shape classification losses as the auxiliary losses.

Original: On the Oxford-IIIT Pet, CIFAR10 and CIFAR100 dataset,we follow the practice in [1] and rotate each image with degrees of { 0 , 90 , 180 , 270 } .
Revised: On the Oxford-IIIT Pet, CIFAR10 and CIFAR100 dataset, we follow the practice in [1] and rotate each image with degrees of { 0 , 90 , 180 , 270 } .

Original: The primarytask is image classification and we add the rotation degree prediction loss as the auxiliary loss. 
Revised: The primary task is image classification and we add the rotation degree prediction loss as the auxiliary loss. Note

Revised: C o n v o l u t i o n

Original: (ii)
Revised: R e s - b l o c k R e s - b l o c k R e s - b l o c k R e s - b l o c k     head head

Revised: ResNet

Revised: C o n v o l u t i o n

Revised: L i n e a r

Revised: L a y e r     head head 4-layer CovNet

Revised: C o n v o l u t i o n

Revised: C o n v o l u t i o n

Revised: C o n v o l u t i o n

Revised: AutoINT …

Revised: E m b e dd i n g

Revised: T r a n s f o r m e r

Revised: T r a n s f o r m e r

Revised: T r a n s f o r m e r

Revised: T r a n s f o r m e r

Revised: T a s k h e a d that in the CUB datasets, we adopt pretrained backbone but in other datasets we train from scratch, to show the effectiveness of our method with different initialization.

Revised: (ii)

Original: The second problem we focus on is the user rating prediction in recommendation.
Revised: The second problem we focus on is the user rating prediction in recommendation.

Original: We evaluateour methods on two datasets with different sparsity, Amazon Beauty [30] and MovieLens1M [31].
Revised: We evaluate our methods on two datasets with different sparsity, Amazon Beauty [47] and MovieLens1M [48].

Original: Our primary task is to predict the rating for each user-item pair based on the user features and itemfeatures.
Revised: Our primary task is to predict the rating for each user-item pair based on the user features and item features.

Original: In this problem, we consider a kind of more generalized auxiliary loss, L2 regularizer.
Revised: In this problem, we consider a kind of more generalized auxiliary loss, L2 regularizer.

Original: Inrecommendation, L2 regularizer plays an important role to the model performance and is often usedas an auxiliary loss [16, 17].
Revised: In recommendation, L2 regularizer plays an important role to the model performance and is often used as an auxiliary loss [17, 18].

Original: In this setting, the primary loss is the rating regression loss and theauxiliary loss is the L2 regularizer.
Revised: In this setting, the primary loss is the rating regression loss and the auxiliary loss is the L2 regularizer.

Original: Detailed dataset information can be found in the supplementary.
Revised: Detailed dataset information can be found in the supplementary.

Original: Baselines We compare our methods with different baselines that can be used for the general settingof auxiliary learning.
Revised: Baselines We compare our methods with different baselines that can be used for the general setting of auxiliary learning.

Original: Single loss learning (SLL) is a natural baseline that only utilizes the primaryloss to train the model.
Revised: Single loss learning (SLL) is a natural baseline that only utilizes the primary loss to train the model.

Original: Comparing with this baseline can help judge whether the auxiliary lossis beneficial to the primary task.
Revised: Comparing with this baseline can help judge whether the auxiliary loss is beneficial to the primary task.

Revised: Additionally, assigning each loss equal weights ”1.0”(Equal) can be used to show the benefits of different auxiliary methods.

Original: Another natural baseline (HPO-tune) is to combine the auxiliarylosses with predefined weights, and then use the HPO methods to tune the weights as conductedin [1].
Revised: Another natural baseline (HPO-tune) is to combine the auxiliary losses with predefined weights, and then use the HPO methods to tune the weights as conducted in [1].

Original: GCS [9] is a baseline that utilizes the similarity between the gradient of each auxiliary lossand that of the primary loss to weight each auxiliary loss.
Revised: GCS [9] is a baseline that utilizes the similarity between the gradient of each auxiliary loss and that of the primary loss to weight each auxiliary loss.

Original: AuxL [11] is a recently proposed methodthat learns a non-linear combination of all the losses.
Revised: AuxL [11] is a recently proposed method that learns a non-linear combination of all the losses.

Original: Uncert [20] is a method that is designed forcombining the losses from multiple tasks, where the loss of each task is weighted according to theiruncertainty.
Revised: Uncert [22] is a method that is designed for combining the losses from multiple tasks, where the loss of each task is weighted according to their uncertainty.

Original: Note that for the methods that do not rely on D dev , we add D dev to their training set forfair comparison.C o n v o l u t i o n R e s - b l o c k R e s - b l o c k
Revised: Note that for the methods that do not rely on D dev , we add D dev to their training set for fair comparison.

Original: R e s - b l o c k

Original: R e s - b l o c k     head head

Original: ResNet

Original: C o n v o l u t i o n

Original: L i n e a r

Original: L a y e r     head head 4-layer CovNet

Original: C o n v o l u t i o n

Original: C o n v o l u t i o n

Original: C o n v o l u t i o n

Original: AutoINT …

Original: E m b e dd i n g

Original: T r a n s f o r m e r

Original: T r a n s f o r m e r

Original: T r a n s f o r m e r T r a n s f o r m e r T a s k h e a d Implementation We utilize the most widely adopted structure [8] for the tasks, with one commonbackbone followed by task-specific heads for all the tasks.
Revised: Implementation We utilize the most widely adopted structure [8] for the tasks, with one common backbone followed by task-specific heads for all the tasks.

Original: In the image classification problem, theauxiliary losses come from some related auxiliary tasks and each task will have its correspondingtask-specific head, but the recommendation problem only has one task head for the rating regressionbecause its auxiliary loss is from L2 regularizer.
Revised: In the image classification problem, the auxiliary losses come from some related auxiliary tasks and each task will have its corresponding task-specific head, but the recommendation problem only has one task head for the rating regression because its auxiliary loss is from L2 regularizer.

Original: For different tasks, we utilize different backbones.
Revised: For different tasks, we utilize different backbones.

Original: For the CUB and Pet dataset, we utilize ResNet18 [12] as the backbone.
Revised: For the CUB and Pet dataset, we utilize ResNet18 [13] as the backbone.

Original: For the CIFAR10 and
Revised: For the CIFAR10 and

Original: CIFAR100, we use a 4-layer convolutional network(ConvNet).
Revised: CIFAR100, we use a 4-layer convolutional network(ConvNet).

Original: For the rating prediction in AmazonBeauty and MovieLens-1M, we choose AutoINT [32] as the backbone.
Revised: For the rating prediction in AmazonBeauty and MovieLens-1M, we choose AutoINT [49] as the backbone.

Original: The model structure andthe module partition for the three settings are shown in Figure 3.
Revised: The model structure and the module partition for the three settings are shown in Figure 3.

Original: We implement the task-specificheads with Multi-layer Perceptron(MLP) whose layer number is searched in { 1 , 2 } .
Revised: We implement the task-specific heads with Multi-layer Perceptron(MLP) whose layer number is searched in { 1 , 2 } .

Original: In ResNet18, themodules are divided by residual blocks, in 4-layer CovNet, the modules are divided by convolutionallayer, and in AutoINT, the modules are divided by Transformer block.
Revised: In ResNet18, the modules are divided by residual blocks, in 4-layer CovNet, the modules are divided by convolutional layer, and in AutoINT, the modules are divided by Transformer block.

Original: More detailed implementationcan be found in the supplementary file.
Revised: More detailed implementation can be found in the supplementary file.

Original: 4.2 Method Effectiveness
Revised: 4.2 Method Effectiveness

Original: We evaluate our method on all the aforementioned tasks and datasets.
Revised: We evaluate our method on all the aforementioned tasks and datasets.

Original: For the image classification task,we utilize the top1-accuracy as the metric and for the rating prediction task, we utilize RMSE(rootedmean squared error) as the metric.
Revised: For the image classification task, we utilize the top1-accuracy as the metric and for the rating prediction task, we utilize RMSE(rooted mean squared error) as the metric.

Original: Higher accuracy for image classification and lower RMSE forrating prediction indicate better performance.
Revised: Higher accuracy for image classification and lower RMSE for rating prediction indicate better performance.

Original: The overall results are shown in Table 1.
Revised: The overall results are shown in Table 1.

Original: As indicatedin previous works [11, 9], losses from auxiliary tasks play a more important role when the labels forthe primary task are inadequate.
Revised: As indicated in previous works [11, 9], losses from auxiliary tasks play a more important role when the labels for the primary task are inadequate.

Original: Therefore, we also conduct experiments on the CUB and Pet dataset,where for the primary task, we only utilize 20% labels.
Revised: Therefore, we also conduct experiments on the CUB and Pet dataset, where for the primary task, we only utilize 20% labels.

Original: The results for this semi-supervised settingare shown in Table 2.
Revised: The results for this semi-supervised setting are shown in Table 2.

Original: We run all the experiments with three random seeds and report the mean andthe std.
Revised: We run all the experiments with three random seeds and report the mean and the std.

Revised: Note that L2 regularizer with coefficient is too large for recommendation, so the Equal baseline performs worst on the Beauty and MovieLens dataset.

Original: Alleviating negative information from auxiliary loss
Revised: Alleviating negative information from auxiliary loss

Original: The results of CUB and MovieLens in Table 1show that the auxiliary losses are harmful to the primary task.
Revised: The results of CUB and MovieLens in Tableshow that the auxiliary losses are harmful to the primary task.

Original: The baselines that utilize the auxiliarylosses generally achieve worse performance than the SLL baseline that only utilizes the single primaryloss to train.
Revised: The baselines that utilize the auxiliary losses generally achieve worse performance than the SLL baseline that only utilizes the single primary loss to train.

Original: Our method shows its strong ability to alleviate the negative information from theauxiliary loss, it achieves comparative performance with the SLL baseline even though it does notuse D dev for training.
Revised: Our method shows its strong ability to alleviate the negative information from the auxiliary loss, it achieves comparative performance with the SLL baseline even though it does not use D dev for training.

Original: It is worth noting that the AuxL which also adopts a bi-level optimizationframework shows excellent ability in alleviating the negative impact of the harmful auxiliary losses.
Revised: It is worth noting that the AuxL which also adopts a bi-level optimization framework shows excellent ability in alleviating the negative impact of the harmful auxiliary losses.

Original: Exploiting useful information in auxiliary losses The results of Pet and Beauty in Table 1 andthe results of Pet-semi and CUB-semi in Table 2 show that the auxiliary losses are beneficial tothe primary task.
Revised: Exploiting useful information in auxiliary losses The results of Pet and Beauty in Table 1 and the results of Pet-semi and CUB-semi in Table 2 show that the auxiliary losses are beneficial to the primary task.

Original: All the baselines that utilize auxiliary losses show substantial improvement overthe SLL baseline which only uses a single loss.
Revised: All the baselines that utilize auxiliary losses show substantial improvement over the SLL baseline which only uses a single loss.

Original: Our method beats all the baselines in exploitingthe beneficial auxiliary information, showing the advantage of the module-level consideration.
Revised: Our method beats all the baselines in exploiting the beneficial auxiliary information, showing the advantage of the module-level consideration.

Original: The
Revised: The

Original: AuxL baseline fails to perform well enough probably because the form of the non-linear combinationrequires a specific design for the non-linear model in specific tasks.
Revised: AuxL baseline fails to perform well enough probably because the form of the non-linear combination requires a specific design for the non-linear model in specific tasks.

Original: Other Observations (i)
Revised: Other Observations (i)

Original: Comparing the results of CUB and Pet in Table 1 to that in Table 2, we cansee that when the labels are inadequate for the primary task, the auxiliary losses from related taskswill bring more benefits, which is consistent to previous works [11, 9].
Revised: Comparing the results of CUB and Pet in Table 1 to that in Table 2, we can see that when the labels are inadequate for the primary task, the auxiliary losses from related tasks will bring more benefits, which is consistent to previous works [11, 9].

Original: (ii)
Revised: (ii)

Original: The rotation predictiontask seems to have little influence on the task of classification on the CIFAR10 and CIFAR100dataset, the different methods that utilize the auxiliary rotation prediction loss do not bring substantialimprovement or severely do harm to the task of CIFAR classification.
Revised: The rotation prediction task seems to have little influence on the task of classification on the CIFAR10 and CIFAR dataset, the different methods that utilize the auxiliary rotation prediction loss do not bring substantial improvement or severely do harm to the task of CIFAR classification.

Original: This phenomenon may becaused by the low resolution of the CIFAR images, where the changes brought by rotation maynot be so significant.
Revised: This phenomenon may be caused by the low resolution of the CIFAR images, where the changes brought by rotation may not be so significant.

Original: (iii)
Revised: (iii)

Original: Although the Beauty and the MovieLens experiments both utilize the L2regularizer to help the rating prediction, the auxiliary regularizer shows different effects.
Revised: Although the Beauty and the MovieLens experiments both utilize the L regularizer to help the rating prediction, the auxiliary regularizer shows different effects.

Original: In Beauty, the regularizer is helpful but in MovieLens, the regularizer is harmful.
Revised: In Beauty, the regularizer is helpful but in MovieLens, the regularizer is harmful.

Original: This phenomenon is easy tounderstand because the MovieLens dataset is dense while Beauty is sparse.
Revised: This phenomenon is easy to understand because the MovieLens dataset is dense while Beauty is sparse.

Original: Regularizer is importantto the sparse Beauty by preventing the overfitting problem.
Revised: Regularizer is important to the sparse Beauty by preventing the overfitting problem.

Original: In the dense MovieLens, data are enoughfor each user so the regularizer is not necessary.
Revised: In the dense MovieLens, data are enough for each user so the regularizer is not necessary.

Original: Overall, when we utilize different kinds of auxiliary losses [Table] we always expect to exploit the beneficial information while resisting the impact of negative information.
Revised: Overall, when we utilize different kinds of auxiliary losses, we always expect to exploit the beneficial information while resisting the impact of negative information.

Original: Our proposed module-awareoptimization method, shows its superiority when facing both beneficial and harmful auxiliary losses.
Revised: Our proposed module-aware optimization method, shows its superiority when facing both beneficial and harmful auxiliary losses.

Original: [Table]
Revised: Additionally, based on the reviews, we add additional experiments on the NYUv2 [50] and CIFAR100/20 datasets so that we can validate the ability of our method when facing more auxiliary tasks. In NYUv2, we totally have 3 tasks, i.e., segmentation, depth prediction and normal prediction, where the segmentation is the primary task and we adopt the EfficientNet [51] as the backbone, which is split into 8 blocks according to its model structure. In the CIFAR-100/20 dataset, we treated each of the 20 ‘coarse’ classes as one task, where each coarse class contains 5 classes.

Original:  the primary task
Revised: We regard the “people” classification(“baby”, “boy”, “girl”, “man”, “woman”) as the primary task, and the 0-9 coarse class classification as the auxiliary tasks, totally 11 tasks. The ResNet18 is adopted as the backbone for the CIFAR-100/20. The results of the added experiments are shown in Table 3.

Revised: These experiments show that our method can comply well with more auxiliary losses and also more advanced models.

Original: [Figure]
Revised: [Table]

Original: Dataset
Revised: [Table] [Table] [Table] [Table]

Revised: Uncert

Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 4.3 Module-level Auxiliary Importance
Revised: 4.3 Module-level Auxiliary Importance

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original: We expect that the learned module-level auxiliary importance can offer us some insights for auxiliarylearning.
Revised: We expect that the learned module-level auxiliary importance can offer us some insights for auxiliary learning.

Original: The learned module importance under each scenario is shown in Figure 4 and the modulesare numbered the same as in Figure 3.
Revised: The learned module importance under each scenario is shown in Figure 4 and the modules are numbered the same as in Figure 3.

Original: (i)
Revised: (i)

Original: When the auxiliary losses are harmful (CUB,MovieLens)or have no substantial influence(CIFAR10,CIFAR100) to primary task, the final learned module-levelauxiliary importance will be 0.
Revised: When the auxiliary losses are harmful (CUB,MovieLens) or have no substantial influence(CIFAR10,CIFAR100) to primary task, the final learned module-level auxiliary importance will be 0.

Original: It is a little surprising to note that in these harmful or unrelatedauxiliary settings, our method can achieve comparative or even better performance than the SLL(only use single primary loss) baseline even though we utilize fewer data to train the model.
Revised: It is a little surprising to note that in these harmful or unrelated auxiliary settings, our method can achieve comparative or even better performance than the SLL (only use single primary loss) baseline even though we utilize fewer data to train the model.

Original: Thisimprovement may benefit from the dynamic weighting and training process.
Revised: This improvement may benefit from the dynamic weighting and training process.

Original: Although the weightsfor the auxiliary losses are finally set to zero, in the training process, the bi-level framework canutilize the gradient from the auxiliary losses to help the model generalize better on the unseen small
Revised: Although the weights for the auxiliary losses are finally set to zero, in the training process, the bi-level framework can utilize the gradient from the auxiliary losses to help the model generalize better on the unseen small

Original: D dev .
Revised: D dev .

Original: That is to say, the gradient from the auxiliary losses may guide θ to a better descent directionfor the primary task in the early stages, but the gradient will show little importance when the betterdirection is found.
Revised: That is to say, the gradient from the auxiliary losses may guide θ to a better descent direction for the primary task in the early stages, but the gradient will show little importance when the better direction is found.

Original: (ii) When the auxiliary losses come from some related beneficial tasks (CUB-semi,
Revised: (ii) When the auxiliary losses come from some related beneficial tasks (CUB-semi,

Original: Pet-semi), the importance of modules fits our intuition where the modules close to the input showhigher importance while the modules close to the output show lower importance, and this is why ourmethod can better exploit beneficial auxiliary information.
Revised: Pet-semi), the importance of modules fits our intuition where the modules close to the input show higher importance while the modules close to the output show lower importance, and this is why our method can better exploit beneficial auxiliary information.

Original: Our module-aware optimization stops the auxiliary losses from optimizing deep modules (close to output), thus avoiding negative transferin deep modules.
Revised: Our module-aware optimization stops the auxiliary losses from optimizing deep modules (close to output), thus avoiding negative transfer in deep modules.

Original: (iii)
Revised: (iii)

Original: However, in (b)Pet, module 1 shows low importance although the module2,3,4,5 still follows the previous pattern, probably because compared to Pet-semi, the labels in Petfor the primary task are enough, so the information from the primary loss is enough to capture thelow-level feature in the first module, and the gradient from the rotation loss is not so necessary.
Revised: However, in (b)Pet, module 1 shows low importance although the module 2,3,4,5 still follows the previous pattern, probably because compared to Pet-semi, the labels in Pet for the primary task are enough, so the information from the primary loss is enough to capture the low-level feature in the first module, and the gradient from the rotation loss is not so necessary.

Original: (iv) L2regularizer is beneficial to the Beauty dataset, but its pattern is also different from that of CUB-semiand Pet-semi.
Revised: (iv) L regularizer is beneficial to the Beauty dataset, but its pattern is also different from that of CUB-semi and Pet-semi.

Original: Module

Revised: Module

Original: (task head) and module1 (embedding table) need larger L2 regularizer thanmiddle transformer blocks.
Revised: (task head) and module1 (embedding table) need larger L2 regularizer than middle transformer blocks.

Original: The learned module-level auxiliary importance shows different patterns under different scenarios,indicating our method can be applied to a variety of auxiliary learning scenarios.
Revised: The learned module-level auxiliary importance shows different patterns under different scenarios, indicating our method can be applied to a variety of auxiliary learning scenarios.

Original: 4.4 Influence of Module Choice
Revised: 4.4 Influence of Module Choice

Original: In this part, we explore the influence of how we choose the module.
Revised: In this part, we explore the influence of how we choose the module.

Original: We choose different partitions ofmodules and conduct experiments on the CUB-semi and Pet dataset.
Revised: We choose different partitions of modules and conduct experiments on the CUB-semi and Pet dataset.

Original: The results are shown in Table 3.
Revised: The results are shown in Table 4.

Original: [Table] The Parameter-L means we regard each pa- [Table] rameter in the network as a module in ouralgorithm, which is a finer-grained partition.
Revised: The Parameter-L means we regard each parameter in the network as a module in our algorithm, which is a finer-grained partition.

Original: The Block-L means we regard each block inthe ResNet as a module(the same as  previous experiments).
Revised: The Block-L means we regard each block in the ResNet as a module(the same as all the previous experiments).

Original: The Model-L means weregard the whole model as a module, whichdegenerates to the loss weighting methods.
Revised: The Model-L means we regard the whole model as a module, which degenerates to the loss weighting methods.

Original: We can see that the Model-L  variant performs worsethan the Block-L, indicating the importance of considering the module-level effect of auxiliarylosses.
Revised: We can see that the Model-L [Table] variant performs worse than the Block-L, indicating the importance of considering the module-level effect of auxiliary losses.

Original: Additionally, we find that regarding each parameter as a module does not bring substantialimprovement compared to the Block-L.
Revised: Additionally, we find that regarding each parameter as a module does not bring substantial improvement compared to the Block-L.

Original: This may be due to the reason that such a fine-grained modulepartition will make the importance matrix α contain more parameters.
Revised: This may be due to the reason that such a fine-grained module partition will make the importance matrix α contain more parameters.

Original: Optimizing more parametersin the upper level requires more data samples in D dev [25], which prevents the finer-grained partitionfrom bringing more improvement.
Revised: Optimizing more parameters in the upper level requires more data samples in D dev [42], which prevents the finer-grained partition from bringing more improvement.

Revised: We also find that the performance of

Revised: Paramter-L has obvious performance drop compared to Block-L in the Pet experiment, while in the CUB-semi experiment, the performance drop is less.

Revised: This is likely because in the CUB-semi setting, we adopt the pretrained backbone for finetuning which can prevent overfitting problem to some extent, but in the Pet experiment we train from scratch.

Original: 4.5 Module Complexity Influence on Importance with L2 Regularizer
Revised: 4.5 Module Complexity Influence on Importance with L2 Regularizer

Original: [Figure]
Revised: [Figure]

Original: We also explore how the module complexity will influencethe learned importance when the auxiliary loss is the L2 regularizer.
Revised: We also explore how the module complexity will influence the learned importance when the auxiliary loss is the L2 regularizer.

Original: We change the hidden dimensions of the task head(2-layer MLP) of AutoINT in the Beauty dataset, and plot thelearned importance for each module in Figure 5.
Revised: We change the hidden dimensions of the task head (2-layer MLP) of AutoINT in the Beauty dataset, and plot the learned importance for each module in Figure 5.

Original: Specifically,we change the hidden dimension for module 6 from 8 to 128,and plot their auxiliary module-level importance.
Revised: Specifically, we change the hidden dimension of module6 from 8 to 128, and plot their auxiliary module-level importance.

Original: It can be seenthat small hidden dimensions like hidden8 and hidden16 willfinally bring a small regularizer for this module while largerhidden dimensions like hidden128 will bring comparativelylarger importance.
Revised: It can be seen that small hidden dimensions like hidden8 and hidden16 will finally bring a small regularizer for this module while larger hidden dimensions like hidden128 will bring comparatively larger importance.

Original: This fits our experience thatwhen there exists overfitting problem, the higher module complexity requiresa larger regularizer.
Revised: This fits our experience that the higher module complexity requires a larger regularizer to prevent overfitting.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose a module-aware optimization approach forauxiliary learning, the commonlyencountered scenario in deep learning.
Revised: In this paper, we propose a module-aware optimization approach for the commonly encountered auxiliary learning scenario.

Original: Our method fills the gap that there exists module-levelinfluence of auxiliary losses but there is no general method that can deal with this effect.
Revised: Our method fills the gap that there exists module-level influence of auxiliary losses but there is no general method that can deal with this effect.

Original: We conductextensive experiments to show that our proposed method can achieve superior performance withauxiliary losses from different scenarios, which could serve as a powerful tool when we want toobtain benefits from the auxiliary losses.
Revised: We conduct extensive experiments to show that our proposed method can achieve superior performance with auxiliary losses from different scenarios, which could serve as a powerful tool when we want to obtain benefits from the auxiliary losses.

Original: Our method has no negative social impact.
Revised: Our method has no negative social impact.

Original: Despite itseffectiveness, the only limitation of our method could be the computational cost is  the same as thatof the GCS baseline, i.e., we should calculate the gradient for each auxiliary loss and the trainingtime is linear to the number of auxiliary losses.
Revised: Despite its effectiveness, the only limitation of our method could be the computational cost is almost the same as that of the GCS baseline, i.e., we should calculate the gradient for each auxiliary loss and the training time is linear to the number of auxiliary losses.

Original: How to reduce this kind of additional cost will be aninteresting future direction.
Revised: Reducing this kind of additional cost is an interesting future direction.

Original: Checklist
Revised: Acknowledgement

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See the Conclusion section.

Original: (c) Did you discuss any potential negative societal impacts of your work? [Yes] See the Conclusion section. (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: This work was supported in part by the National Key Research and Development Program of China

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results? [Yes] (b) Did you include complete proofs of all theoretical results? [Yes]
Revised: No. 2020AAA0106300 and National Natural Science Foundation of China (No. 62250008, 62222209, 62102222).

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [N/A]

Original: We provide theimplementation details in the supplementary file.

Original: The code for the experiments will bereleased after the review.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [Yes] Refer to the supplementary file.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?

Original: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?

Original: [Yes] Refer to the supplementary file.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]

Original: (b) Did you mention the license of the assets?

Original: [Yes]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of

Original: instructions given to participants and screenshots, ifapplicable?

Original: [N

Original: A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board

Original: (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]