Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a module-aware optimization method for a commonly adopted learning paradigm, learning with auxiliary losses. Inspired by multi-task learning, the authors point out that different auxiliary losses will contribute differently to different modules of the model for improving the primary task. To obtain the optimal importance of each auxiliary loss to each module, the authors design an efficient bi-level optimization based method. Extensive experiments on several datasets and tasks show that the proposed method for auxiliary learning is effective and generic. Additionally, the authors conduct comprehensive ablations from different aspects, which offers insights for auxiliary learning in various scenarios.
strengths_and_weaknesses: Strengths:
1.	This paper is well-motivated. The module-aware optimization is important to addressing the module-level conflicts between the primary task and the auxiliary losses. 
2.	The related work and the contributions of this paper are sufficiently discussed. It is clearly discussed the differences between existing auxiliary learning, multi-task learning methods and the proposed method. The necessity of this work is obvious.
3.	The proposed method is novel and efficient. The formulated bi-level optimization problem requires to make the gradient of the model parameterized, which is usually memory and time consuming to conduct the upper optimization. With the hyper-gradient derivation, the authors adopt the efficient best-response approximation, which gives a nice solution.
4.	The experiments with different datasets, backbones and the ablations are convincing. The analysis of the whole experiments is quite insightful, and some results are quite interesting, like the learned module-level weights changing with the complexity of the modules.
5.	This paper is well-written and easy to follow. The proposed method is a general method and can be easily applied to various auxiliary learning scenario, which could be a useful plug-in for many tasks.  

Weaknesses:
1.	The weakness of this paper lies in that although the proposed method is efficient compared to the directly unrolling method. The time complexity of both the baselines and this method should also be discussed so that readers can better balance these methods.
2.	Although the module choice effect is shown in the experiments, there needs more analysis and instructions for readers to decide when to conduct finer-grained module split or must we conduct parameter-level module split with collecting more data in the developing set?

questions: 1. The authors propose to utilize the validation set to prevent overfitting the developing set and the training set. I’d like to know whether the validation set for the proposed method and the baselines are the same, or the validation set for the baselines contain the developing set?
2. I’d also like to see the time complexity of different baselines and this method.

limitations: The authors point that the time complexity is the limitation of this method, but it needs more discussing as above.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper focuses on improving the effectiveness of leveraging auxiliary objectives for training deep neural network models. The authors propose a learning algorithm that jointly optimizes 1) the weight of each auxiliary loss on different parts (modules) of the model and 2) model parameters according to the weighted sum of the loss terms. Experiments are conducted on image classification and user rating prediction tasks to verify the efficacy of the proposed approach.
strengths_and_weaknesses: *Strengths*
1. The paper is well-written. The idea is sound, and the technical details are clear. 
2. The proposed concept may have important contributions to the community. Indeed the impact of auxiliary training losses on different parts of the model is not well-understood yet.
2. The paper provides some interesting observations in the experiments, especially the loss weight visualizations in Section 4.3.
*Weakness*
1. The experiments are conducted on a rather small scale (). The proposed approach could be more promising if the authors apply the algorithm to 1) more SOTA networks, e.g., VIT or EffiecientNet, and 2) more large-scale tasks, e.g., semi-supervised object detection.
2. In Line 294, the authors use the descriptions, "That is to say, the gradient from the auxiliary losses may guide θ to a better descent direction for the primary task in the early stages, but the gradient will show little importance when the better direction is found" to explain those cases where the loss weight is learned to be 0. It would be great to visualize the change of the loss weight over time (training loops) to support the authors' arguments here.
3. The authors should report the computational loss in the paper since it is the limitation of the proposed approach.
questions: Please see Weakness.
limitations: The authors address the limitation in Conclusion. No obvious potential negative social impact is observed.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes an auxiliary learning framework that updates network parameters in a module level via learnable module-level auxiliary importance. The proposed framework is updated with bi-level optimisation, a standard way to perform auxiliary learning with meta learning.
As such it contains two stages: the inner loop: it updates the network parameter by reweighting gradients of each auxiliary loss on each network module via the fixed auxiliary importance mask on the training data; the outer loop: it updates the auxiliary importance mask by the performance of primary task with M inner rollout steps, which uses best-response approximation (assuming network weightings are converged in inner loop) to reduce training time and memory on developing data (a small dataset to split from validation dataset). The framework eventually stores the best network which achieves the optimal validation performance.

The paper evaluates on multiple small-scale image classification datasets and user rating prediction datasets for regression with multiple network design and with supervised and semi-supervised learning settings. The auxiliary tasks are specifically designed for each dataset, e.g. using available attribute information as auxiliary tasks for fine-grained bird species classification, or using hand-crafted auxiliary tasks like rotation prediction for image classification datasets without attribute information. The proposed framework was compared to multiple well-known auxiliary learning methods, and was shown to improve the performance with a non-trivial margin.

strengths_and_weaknesses: I found the paper is very easy to follow with very clean writing. The visualisation and explanation of auxiliary importance is also interesting to read.

However, I think the design of experiments is a bit insufficient. 
1) the number of auxiliary tasks is only limited to two, most of the time only has one auxiliary task. 

2) the benefits on auxiliary tasks are not well reflected in the experiments. E.g. L267-269 claims the related tasks will bring more benefits, but Table 1 shows training with auxiliary tasks simply not brought any performance improvement, on CUB dataset. 

3) I wouldn’t call L2 regularisation is an auxiliary task, though it still might fits in a general definition… 

4) missing simple baseline: Equal weighting as multi-task learning, just to see the benefit of the proposed/baseline auxiliary methods with exact same training data.

questions: 1. Is the auxiliary importance mask bounded or unbounded?

2. Could you provide the experiments with 3 or more auxiliary tasks, in the standard multi-task/auxiliary task literature, e.g. in NYUv2, CityScapes, CIFAR-100/20 classification, as shown in [1, 2]?

[1] Liu et al. Auto-Lambda: Disentangling Dynamic Task Relationships, 2022
[2] Yu et al. Gradient Surgery for Multi-Task Learning, 2020

3. Could you provide the experiments with Equal weighting, so it shows the benefit of auxiliary learning method with the same training data?

4. Minor comment: The dataset chosen in Section 4.4 looks a bit suspicious, which includes two settings having the large performance gap. 

limitations: N/A
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: Traditional approaches to auxiliary learning learn a single scalar per-task to adapt auxiliary tasks to the primary task. 
This paper suggests to learn a scalar per-module for each auxiliary task. The motivation behind this is to allow more fine-grained adaptability in tuning the auxiliary task to improve the primary task. The authors show through experiments that this is an improvement over past methods they considered.
strengths_and_weaknesses: **Strengths**

1. The paper is well written and easy to parse
2. The motivations for the work we clearly stated and situated in the context of previous work
3. Experimental evaluation over several datasets and multiple baselines over multiple seeds with calculated standard deviations.

**Weaknesses**

My primary concern with this paper is that the method seems incremental. Particularly,  all the core elements in this work can be stitched together from previous work (which itself would not be criticism per-se if there was novelty in the way the stitching together was done). 
Specifically, learning auxiliary task adaptive weights via bi-level optimization has been done in [1, 2]. 
For adapting auxiliary tasks weights on a per-module bases, there are multiple related works that are not referenced.  Layer-adaptive weights are not uncommon in the multitask literature see Section 3.2 of [3] which introduces an index k, for each layer. 
Also note that in [4], they use the primary task to learn a subspace within which they deconstruct the auxiliary task weights - this abstracts away the choice of Parameter/Block/Model-L introduced in this paper - and represents an automated "modularization" approach


**Relevant Related Work**

[1] Should we be pre-training? an argument for end-task aware training as an alternative : https://arxiv.org/abs/2109.07437

[2] Auxiliary learning by implicit differentiation:  https://arxiv.org/abs/2007.02693

[3] Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models. https://arxiv.org/abs/2010.05874

[4] Auxiliary task update decomposition: the good, the bad and the neutral. https://arxiv.org/abs/2108.11346
questions: 1. Could you state the criteria you used to bold the numbers in your tables ? If it is highest average, please state this clearly. Some of the bolded numbers are *not statistically significant* but are still bolded.

2. Were all the table 1, 2 experiments conducted with Block-L ?

3. Is there any principled way / recommendation for deciding when to use Parameter/Block/Model-L ?

4. How do the results change with model size ? With the size of the dev set ?

5. It seems that for each experiment/data all methods only used a single learning rate ? It would be worth strengthening all methods by increasing the space of hyper-parameters (especially learning rate) searched over


Less of a question and more of a note - It would be interesting to see the evolution of the module level weights over time instead of just at the end of training as provided in Figure 4 .
limitations: This work introduces multiple hyper-parameters which may have to be cross-validation
1. The granularity of task adaptivity
2. M - the number of look-back steps

Computing the meta-gradient adds extra overhead but the runtime compared to simple fixed unitary weighting is not discussed
review_title: null

Meta-review: metareview: The paper investigates module-aware optimization for auxiliary learning, which adjusts the impact of the auxiliary tasks (through their losses) at the module-level instead of treating the model as a whole. The motivation is based on the observation that a certain auxiliary loss may be beneficial for optimizing specific modules in a model but harmful to others at the same time.  A bi-level formulation is employed as a natural solution, where the inner loop optimizes the model parameters and the outer loop optimizes the module-level auxiliary importance. Experiments conducted using different auxiliary losses on diverse datasets show the advantage of the proposed approach over existing baselines.  

Overall, the paper is well written and the problem is clearly motivated. The solution is technically sound and the experimental results show the effectiveness of the proposed module-aware optimization approach. The authors have conducted additional experiments based on the reviewers’ suggestions, which further enhanced the evaluation side of the paper. On the other hand, the overall novelty does not appear to be very strong. Treating the importance of each auxiliary loss to each module as  hyper-parameters and then using a bi-level optimization to optimize both the model parameters and the importance parameters is a very straightforward idea. Reviewers also pointed out some important related works on learning adaptive weights over auxiliary tasks via bi-level optimization, adapting auxiliary tasks weights on a per-module basis,  and layer-wise weight adaptation. The authors are encouraged to reference these important related works and highlight the core technical differences. 

confidence: Less certain
Decision: Accept