Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper pinpoints the key issues of Auxiliary Learning: (1). how to design useful auxiliary tasks, (2) how to combine auxiliary tasks into a single coherent loss. Motived by the issues, this paper proposes a novel Auxiliary Learning frame work, named AuxiLearn. The paper is globally well organized and clearly written. 

Pros:
1.	The motivation is straightforward.
2.	The paper proposes sound the technique contributions. Adopting bi-level optimization in Auxiliary Learning makes sense.
3.	The theoretical analysis in this paper supports the efficiency of the method proposed in this paper.
4.	The experimental results and experimental analysis in this paper is plausible.

Cons:
1.	The efficiency of the hypergradient, which is the main shortage, should be discussed. 
2.	How to determine the number of iterations J in Alg. 2 is not given.

review_title: Good Paper with Sound Technique and Sufficient  Experimental Support

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper studies a variant of multi-task learning, auxiliary learning, where one main task dominates, and other tasks are used to learn a good representation. To achieve this goal, the authors propose a learning-to-learn algorithm. In particular, the auxiliary losses are represented by a vector and then transformed to a new loss term via linear or nonlinear function $h$. They also made two more contributions. First, an approach of new auxiliary task generation is proposed. Second, an implicit differentiation based optimization method is proposed to find the solution. Both theoretical analysis and empirical studies demonstrate the superiority of their proposed model.

Pros:
1. The intuition makes sense. When the main task dominates, its generalization can help guide the weighting of the weights of auxiliary tasks.
2. The theoretical analysis further supports the claim of this work.
3. Comprehensive experiments are made to verify the effectiveness of the proposal.

Cons:
1. The idea of learning new auxiliary tasks is wired and less intuitive. The learning of the whole system is still the main task loss, without any useful supervision (or self-supervision). Therefore, it is doubtful whether the learned task is meaningful, or may involve some chaos in the system. Overall, I think this part does harm to this work and the authors may consider removing it.
2. Notations are sort of unclear. In section 3.2, $\mathbf{l}$ is a $K+1$-dim vector, but it seems $g(\mathbf{l};\phi)$ only maps the $K$ auxiliary tasks' losses to a scalar. 
3. There are some typos and the authors need to polish this paper again.
review_title: The intuition makes sense. Comprehensive experiments are done.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a way to combine the auxiliary tasks' losses. Moreover, when the auxiliary task is unknown, a DNN is utilized to learn the auxiliary task automatically.

1. For the auxiliary tasks combing cases, how to ensure the g is smooth w.r.t. the main task parameters W, i.e., does pdiv{L_T}/pdiv{W} exists? When g is DNN, some non-smooth activation function will break down the framework and make the main task parameters hard to learn.
2. As for the IFT, one assumption or say the requirement for Eq. (3) to hold is the function div{L_T}/div{W} should be continuously differentiable w.r.t. {W, \phi}. This will limit the design of the DNN f and g, such as we cannot use BN and non-smooth activation function (e.g., ReLU, etc.). Although the code and the automatic differentiation tools may still work in practice, it at least wrong from the theoretical perspective.
3. Since several approximations steps are utilized (approximation W^* and inexact vector and Hessian inverse product), it is necessary and better to provide the error analysis since the convergence of the bi-level optimization is not obvious.
4. I do not see much insight from Sec. 4.2. If d{L_A}/d{\phi} > 0 means the auxiliary task is helpless, does it mean that I can make it helpful by assigning it a negative weight say \phi_i? But this is in contradiction with the experimental results in which the monotonic networks tend to have a better performance.

minor comments:
1. Some statements are inconsistent with (a) in figure 1. The subfigure (a) in Fig.1 seems to show that L_T is g(\cdot;\phi), while the end at page 3 states 'L_T = l_mian + g'. It is confused here, since subfigure (b) shows the way of 'L_T = l_mian + g'.
review_title: Overall, It is an okay submission with rich experimental results; however, some main part problems degrade the score.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes to make use of implicit differerntiation to improve auxiliary learning, including learning to combine the manually designed auxiliary tasks and learning new auxiliary tasks. 

Reasons for scores: Overall the paper looks interesting and technically sound. However, strong experiments could have been done for being more convincing. 

Overall the paper is clearly written and technically sound.  The problem studied is significant and should be of interest to a broad range of people in the machine learning community. Some concerns are listed below. 

First, I have concerns on the learning new auxiliary tasks, which is formulated as student-teacher maner. This step sounds somewhat counter-intuitive. The initial teacher network might simply produce meaningless labels which are used to guide the student network; it begs the question that what is the intuitition behind that pushs both the teacher and student network to learn non-trivial solutions.

In general the experiments could have been stronger. Although the authors compare with other multi-task works such as using uncertainty as  a way to combine loss, comparisons with the current best results in each benchmark task is desired as well. This will help reveal where the proposed method stands in terms of accuracy. 

In sec 5.4, it demonstrates the utility of learnable auxiliary in the low-data regime and only 5% of the training data are used. This seems to indicate that the method is only useful in the case of scarce training data. The authors could have reported results under different amount of training data; this is important for readers to understand the range within which the method is expected to help. 

review_title: This paper proposes to make use of implicit differerntiation to improve auxiliary learning. 

final_score: rating: 7: Good paper, accept
review_content: The paper proposes AuxiLearn, a framework that can be used to combine losses from multiple auxiliary tasks (if present) into a single combined, loss function that does not require expensive grid search over possible linear combination. It uses an implicit differentiation-based approach to train a (deep) non-linear network that weighs the various auxiliary losses to optimize the generalization capabilities of the network. In the absence of such pre-defined tasks, a variation of the approach, using teacher-student networks, helps create relevant tasks to improve the performance of the network. Experiments across tasks such as classification (both few-shot and with limited labels) and segmentation show that the approach helps improve the performance of the model on the main task.

Comments:
- There are some issues with notations and the overall presentation that make it a little hard to follow. For example, in section 5.1, why are there different weights/parameters (w* and \tilde{w}) for the helpful and harmful tasks? Should the variation not just be w.r.t epsilon? Similarly, in Section 4.1, while the analysis is nice, how is this used in the proposed framework? Is the Newton update monitored in the proposed framework to identify the more important auxiliary task? If so, then I think that should be the core contribution since there can exist a non-trivial number of auxiliary tasks (e.g. 312 in the CUB dataset), and identifying them would greatly reduce the complexity of training. If the Newton update is not used to "prune" tasks, I am not convinced that it adds value to the overall theme of the paper and the space could be used to describe the evaluation of the approach in more detail (as done in the supplementary).
- Ablation studies are missing that could add more value to the evaluation and help highlight the key contributions. For example, what is the effect of depth on the deep auxiliary network? It is mentioned that a 5 layer network with 10 neurons each was used. Ablations on this configuration would help visualize the effect of network complexity on the aux loss weights. 
- One question that arises is whether the aux losses are scaled based on the amount of data available for each of these tasks. For example, in CUB-200 2011, although there are 312 tasks, only around 130 "tasks" or attributes that have a significant amount of annotations for at least 5 of the classes. Does this have any effect on the combined loss? 
review_title: Interesting Approach to Multi-task learning using auxiliary tasks with strong empirical results

Meta-review: The paper proposes a novel framework to develop useful auxiliary tasks and combine auxiliary tasks into a single coherent loss. The idea is good and the experiments are sufficient to verify the arguments. All the reviewers agree to accept the paper.
Decision: Accept (Poster)