Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a novel multi-task learning method which adjusts task weights dynamically during training, by exploiting task-specific updates of the model parameters between training epochs. Specifically, the proposed model takes the differences between the model’s parameters before and after the singletask update, after that the mixing factors of the model updates are found based on the differences to minimize the loss on the target task’s development data. Empirical studies are performed on tasks of computer vision and natural language understanding.

The paper is well written and easy to follow, the authors summarize the related work in a clear manner. The proposed methodology is intuitive and well-motivated, in the meantime, it is flexible and can be generalized to other variations in terms of models and tasks. 

My major concern about the paper include the following:
1)	Although the proposed method is intuitive and straightforward, it would be necessary to provide some theoretical justification or a formal analysis of the proposed methodology.

2)	Considering the lack of a theoretical justification, the experimental results are not convincing enough to justify the proposed method. The baselines chosen include standard multi-task learning and one single task-oriented approach, which is somewhat limited. Even so, on both of the computer vision and natural language understanding tasks, the proposed method doesn’t consistently outperform the baselines in most cases. The authors did provide sufficient analysis, nevertheless, it doesn’t justify the effectiveness of the algorithm. 

Based on the concerns above, the paper can be improved from both the theoretical and empirical perspectives. 

review_title: The proposed methodology is intuitive and flexible, the experimental results are not convincing enough.

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary: This paper presents an algorithm for multitask learning that learns task weights via an EM-like approach that alternates between updating the model parameters (using task weights) and updating the task weights (using current model parameters, based on the target task development set). 

Experiments: They compare against single task training, standard multitask training (though this isn’t described very clearly but roughly is training jointly on tasks), and another method for learning task weights, Discriminative Importance Weighting (DIW).
They present experiments on MultiMNIST, where the two tasks are to predict the task in the top left and bottom right of two superimposed digits. The proposed algorithm has the beast mean performance, but results are within a standard deviation of the baselines. They also present experiments on 5 NLU tasks (CommitmentBank, COPA, MRPC, RTE, WNLI) with the same baselines. The results on these tasks are mixed, with all multitask methods outperforming single task training (except on WNLI, which is a bit degenerate).

Overall, this paper needs a bit more work. The proposed is quite ad hoc, and with little justification, it’s not clear why we should be doing any of the things the algorithm proposes. For example, in line 11 of Algorithm 1, I don’t understand the intuition arbitrarily subtracting 1 from all weights. From a novelty perspective, I’m not convinced the proposed method is different enough from existing methods. Dynamic task weighting is not particularly new (e.g. the baseline method, DIW, they compare against), and their method starts to look a lot like meta-learning of task weights (like MAML [1], [2], or [3]). The results from the experiments are not convincing to me. On MNIST, the results between all methods are fairly close together, and on the NLU tasks, there’s no clear best algorithm.

Additional notes and questions
1. For the “standard multitask baseline”, are the tasks balanced in size? Do you deterministically train on a batch from both or is it stochastic? This is mostly relevant for the NLU tasks, which have fairly different sizes.
2. On MNIST, given that the algorithm sets the weight of one task to 1.0 and the other to 0.0, why is this algorithm outperforming single-task training?
3. On a similar note, it'd be nice to see a sanity check experiment that the learned weights are sensible (e.g. one task has random labels) or an examples of where the learned weights are binary.
4. I appreciate that the authors report min/max/mean/std of 20 runs on MNIST. It would be nice to see the standard deviations for the NLU tasks for consistency and given the fact that the standard deviations on the MNIST task were important in differentiating significant differences. Similarly, it would be nice to see how the task weights evolve.

Style notes
* Huggingface Transformers now has a citation
* Multitask Learning; Computer Vision; Natural Language Processing/Understanding: lowercase
* “Singletask” should probably be hyphenated or two words.
* “10.000” → “10,000”
* Table 2 could really use headers over the two columns within each task.

[1] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks." ICML. 2017.
[2] Shu, Jun, et al. "Meta-weight-net: Learning an explicit mapping for sample weighting." Advances in Neural Information Processing Systems. 2019.
[3] Wang, Xinyi, Yulia Tsvetkov, and Graham Neubig. "Balancing training for multilingual neural machine translation." arXiv preprint 2004.06748 (2020).
review_title: Ad hoc multitask learning via task weighting with unconvincing experiments

final_score: rating: 3: Clear rejection
review_content: Summary:
This paper proposes a new approach for multi-task learning that estimates the individual task weights through gradient-based meta-optimization on a weighted accumulation of task-specific model updates. Evaluations are performed in a multi-task learning setup on tasks related to computer vision (Multi-MNIST) and natural language understanding (tasks from GLUE and SuperGLUE).

Pros:
1) The paper is easy to follow. 

2) Empirical evaluation is performed on vision and NLU domains.

Cons:
1) I am not completely convinced with the proposed alpha-Variable Importance Learning algorithm. It is not very clear in the discussion how the alpha is different from task-specific weights. For example, in algorithm-1, if you replace deltas in line-10 with line-6, then there is no need to have separate alphas and task-specific weights, where line-9 can calculate the task-specific weights directly. 

2) In general, for a multi-task setup, I would expect to show the multi-task learning with multiple auxiliary tasks (that’s the main motivation of this paper as well). However, the choice of the experimental setup is convincing, especially for the vision domain there is only one auxiliary task. 

3) Both results in Table-1 and Table-2 suggest that the proposed algorithm is not superior over the baselines and previous approaches. The improvements are minor and sometimes lower, and I believe most of the results fall within the statistically insignificant range. 


Overall: 
I think the paper can be made stronger with more thorough discussion on the algorithm and its properties. Further, the experimental results suggest that the proposed algorithm performs more or less similar to previous methods. Hence, there is a lot of scope for further improvement and I would suggest rejecting this paper. I would also suggest the authors to perform more experiments and ablations. 


Questions:
1) How is the alpha different from task-specific weights. Please discuss more on this. In algorithm-1, if you replace deltas in line-10 with line-6, then there is no need to have separate alphas and task-specific weights?


2) Please provide statistical significant scores for all the results. 

3)  What's the reason behind choosing a multi-MNIST dataset with only one auxiliary task? Aren’t there other datasets in a MTL setup with more auxiliary tasks? 

4) Table-2 results for the development set are based on the average of multiple runs, but for test you reported the ensemble, so why don’t you report ensemble for the development set as well?

5) Can you also present some ablations/discussion on the learned importance of an auxiliary task (based on task-specific weight over the training trajectory) vs. any intuitive reason that makes sense of this importance of the axillary task for a given primary task? If there is not such correlation, then also it's good to discuss. 

Other comments: 

1) Please try to expand the introduction section. 

2) Please provide some more ablations. 


review_title: Multi-task learning with gradient-based meta-optimization for learning task-specific weights

Meta-review: This paper proposes \alphaVIL, a method for weighting the task-specific losses in a multi-task setting in order to optimize the performance on a particular target task. The idea is to first collect gradient updates for the model based on all the separate tasks, and then re-weight those updates in order to optimize the loss on a held-out development set for the target task. In practice, this meta-optimization is performed with gradient descent. Experiments on multi-MNIST and several tasks that are part of GLUE and SuperGLUE show that \alphaVIL is close in performance to a baseline multitask method and discriminative importance weighting.

Strengths:
- The idea is intuitively appealing. Directly reweighting tasks as a meta-optimization step is straightforward and appears to not be proposed previously in the literature.
- The paper is clear in its presentation.

Weaknesses:
- The reviewers agree that the main weakness is that the experimental results do not show that \alphaVIL offers any substantial benefits over existing methods. On the multi-MNIST task, while \alphaVIL tends to have the highest mean performance, the difference is small (less than a standard deviation). On the GLUE/SuperGLUE tasks, it outperforms other methods on only 1 out of 10 experiments. There are also no confidence intervals/standard deviations provided to assess the significance of the results.
Decision: Reject