Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:
The authors present a general formulation of different settings in multitask learning (including pretraining regimes), in a setting where the goal is to get best performance for a pre-specified primary task and additional auxiliary tasks. The main idea is to divide the gradients on the auxiliary task into 2 subspaces: a subspace where the gradients influence performance of the primary task and a subspace where they only influence the auxiliary task without changing the loss on the primary task. Within the subspace that does have influence on the primary task, it is easy to compute directions that have a positive or negative effect on the primary task, which allows to create different learning schemes given the gradients that point toward: i) auxiliary influence only, ii) positive influence on auxiliary tass, iii) negative influence on primary task. Experimental results show improvements over previously identified meta learning methods on 2 natural language datasets and 3 image datasets.

Strengths:
The authors present a general framework for an important problem. It has many applications in a wide variety of fields and contributes to thinking about meta learning and pretraining in a more general way.
Explanations, illustrations and mathematical derivations are clear and easy to understand.
The authors show that with a careful choice of hyperparameters, their approach can improve performance, especially in settings with limited data on the primary task. The results on natural language datasets are also interesting, showing that they achieve a higher performance when the auxiliary task doesn’t exactly match the primary task data.

Weaknesses:
Some of the explanations and especially the proof fall apart, when considering the k-largest principal vector of J*. Since k<<D, the sum of all gradients in g_aux will clearly still have a big influence on the performance of the primary task.
The proposed algorithm introduces a lot of additional hyperparameters and not all of those hyper parameters are properly discussed. Eta_aux and eta_prim are properly discussed and the authors convincingly show that these parameters are implicitly set by other methods as well. Figure 2 shows an ablation study but does not help practitioners to set the discussed values, given the large variance over 5 runs.
The choice of the subspace for g_aux seems very critical and the provided experimental results and discussion are somewhat lacking. How can a random choice of subspace basis improve the results? Calculating the randomized_lowrank_approx is only done every n steps, but there is no mention of n later. Some experimental results should be provided to convince the reader that the basis does not change too much given 2 different batches from the primary task.

Other remarks:
The result in Table 2 is much better than the best result in Table 4 on the Cat-vs-Dogs experiment. What is the difference? Can the experiments in Table 4 be repeated to be more comparable to Table 2?
PCGrad most closely resembles this work. What type of subspace basis is used in that work? It would be interesting to see a direct comparison between PCGrad and the proposed method with eta_aux = (alpha_aux, alpha_aux,- alpha_aux) and using the same basis.
Early stopping after 10 epochs seems quite short and might explain some of the large variance in the results.

Minor remarks:
k is introduced without much explanation, which was a bit confusing on first reading. It should be clearly stated that it is a hyper parameter on first mention.

review_title: A good overall idea and well written, but the resulting algorithm might be too tedious to use in practice

final_score: rating: 5: Marginally below acceptance threshold
review_content: ##########################################################################

Summary:
Leveraging the power of the data-rich related tasks have been studied (e.g., pre-training and multitask learning). This paper points out that careful utilization of auxiliary task is required to gain enhanced performance in primary tasks. In order to prevent harming the performance of primary tasks, they suggest the method to decompose auxiliary updates into three directions which have positive, negative and neutral impact on the primary task.

##########################################################################

Reasons for score: 

In this paper, it is highly interesting to see how to use a decomposition from the span of the primary task Jacobian to adapt auxiliary gradients and validate the proposed methodology on image and textual data. Even though this is an interesting setting and the technical solutions presented in the paper look reasonable, the idea seems to be pretty incremental as it stacks multiple existing techniques without many innovations. 

##########################################################################

Pros: 
1. The proposed methodology utilizes  automatic differentiation procedures and randomized singular value decomposition for efficient scalability. 
2. The proposed framework allows the model to treat each auxiliary update independently by its impact on the task of interest, which seems to be interesting.

##########################################################################

Cons: 
Authors need to perform more qualitative and quantitative analysis on the datasets to vilify the effectiveness of the proposed methodology.

##########################################################################


review_title: Review on "Auxiliary task update decomposition: the good, the bad and the neutral"

final_score: rating: 6: Marginally above acceptance threshold
review_content: The work studies the auxiliary task selection in deep learning to resolve the burden of selecting relevant tasks for pre-training or the multitask learning. By decomposing the auxiliary updates, one can reweight separately the beneficial and harmful directions so that the net contribution to the update of the primary task is always positive. The efficient implementation is experimented in text classification, image classification, and medical imaging transfer tasks.

The first contribution is the decomposition algorithm and reweighting of the auxiliary updates. It is a simple idea with a nice insight of treating the primary task and the auxiliary tasks in different manners. The decomposition allows a reweighting on the updates to optimize the primary task as much as possible while keeping the auxiliary tasks providing improvable directions. The second contribution is an efficient mechanism to approximate and calculate the SVD of the Jacobian of the primary task. The mechanism is implemented from an existing randomized approximation method. The third contribution is a set of experiments verifying the proposed method. The experiments include text classification, image classification, and medical imaging transfer tasks. The most salient result is the 99% data efficiency to achieve improvable performance in the medical imaging transfer task.

Concerns

Besides the above positive contributions, following are some concerns from the observations:

1. The relative improvements comparing to the baselines in Table 1 and Table 2 do not seem as much as that in (Gururangan et al. 2020) and (Yu et al., 2020), respectively. 

2. The weights reported in the experiments are 1 or -1 in the experiments. For instance, \eta_aux = (1, 1, -1) is reported in the image classification task.

The reader would expect much better improvements when given the freedom to reassign the weights on the decomposed directions, especially when the harmful part has a negated weight. Moreover, why are the values chosen in \eta 1 or -1? Would there be a nicer balance between, say, the beneficial and the harmful parts? For instance, would \eta = (1, 0.8, -0.9) be a better choice? It would be crucial that the authors can explain furthermore or support further experiments to confirm whether the potential of this decomposition algorithm is fully demonstrated or not.


=====================

Post Rebuttal

I have read the authors' response. All my concerns are addressed properly. However, I still doubt that even the corner cases of \eta have a better performance, would there be a systematic way to find the optimal parameters reflecting the true potential of this method. Thus, I will keep my score unchanged.

review_title: A simple yet insightful idea is implemented while the experiments might not demonstrate the algorithm's full potential.

final_score: rating: 6: Marginally above acceptance threshold
review_content: [Summary] This paper studies auxiliary learning, and propose a decomposition method on the auxiliary gradient into several components, and to select relevant/useful decomposed gradients to maximize the assistance of the primary task.

[Strength] How to adjust auxiliary tasks in a beneficial way is always a challenging problem in multi-task learning. The proposed solution on gradient decomposition is novel and interesting. The improved performance based on the proposed method seems to be non-trivial.

[Weakness] This paper however has some significant weaknesses which I will outline below.
-- Missing details. The subspace of the primary task gradient is composed of all training samples in the primary task, based on the definition of $\mathcal{S}$. However, for any reasonable-size training dataset which contains at least 10k training samples, this process seems to be extremely expensive to compute? The authors have introduced randomised approximation on decomposition step, but accumulate gradients seem to be already taking a lot of computation. I hope the authors could justify this.

-- Unfair experimental setting. My biggest concern is on hyper-parameter tuning on each component of the decomposed auxiliary task gradient. In Eq. 1, the authors decompose each auxiliary task gradient into three components: i) one is lying in the subspace of primary task gradient, ii) one is orthogonal to the primary task gradient ii) the final one is in conflict to the primary task gradient. By selecting different weightings on each component, the authors claim that this formulation can be degraded into prior auxiliary learning methods. I agree that this formulation is general. However, all of these component weightings are selected by hand, and seem to be very different across different datasets and tasks. 

This gives a very unfair comparison to previous methods, simply because these methods are included in one of the hyper-parameter sets of corresponding weighting values. I have noticed that the authors have constrained the search space into 4 sets of weighting, but this does not justify this problem. Ideally, these weighting should be automatically computed during training, and varied based on the dataset.

In addition, the weighting for the primary task also varies across different datasets. So I am wondering whether these task weightings are consistent in baselines, and did authors perform a similar hyper-parameter search on baseline methods as well?

Other minor issues:
This formulation *cannot* be used as pre-training with only auxiliary gradients which are helpful to the primary task, since we do not know the primary task gradient beforehand. Even we know the primary task gradient, (0, 1, 0) is simply putting more weighting on the existing primary task gradient, so it's the same as to tune up the learning rate.

=======================

Rated up after authors' clarification.
review_title: Interesting idea, but questionable experimental setting

Meta-review: After engaging in some good interactive discussions all but one reviewer settled on a rating of marginal accept. The most negative reviewer didn't really provide a clear enough explanation of what was lacking in the work. The other reviewers felt that the observed gains for this multi-task learning framework were clear enough that the work is worthy of some attention by the community. The AC recommends acceptance, but one may consider this recommendation as a just past the line for acceptance recommendation.
Decision: Accept (Poster)