Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper present a meta-learning method for learning parametric loss functions that can generalise across different tasks and model architectures, which is done by encode learning strategies into an adaptive high-dimensional loss.

I think one interesting result is the utilisation of extra information that helps shape the loss landscapes at meta-train time, where as the authors said the extra information can take on various forms, such as exploratory signals or expert demonstrations for RL tasks. This enables a more efficient ways to optimise the original task loss.

Potential improvements:

(1) paper layout, to be honest, I'm not sure if Figure 1 is really needed

(2) related work section seems very long, would be good if it can be shorten and use the extra space for results display

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposes a mete-learning approach to learn a loss function from old tasks which can generalize well to new tasks. The benefits of the proposed approaches are 1) data-driven a loss function and 2) allowing the usage of extra side-information to design the loss function.

Overall, the presentation in this paper is hard for me to understand technical details and see the difference with existing methods. Please see the questions below. I am glad to discuss problems with the authors' reply during the rebuttal.

Q1. "Mφ(y; fθ(x)) that predicts the loss gave the ground truth target y and the predicted target fθ(x)", "the purpose...loss function, or a meta-loss, which generalizes across multiple training contexts or tasks".
- It is better for the authors to visualize all y v.s. f_{\theta}(x) for all experiments. They have done this only for "Section 4.2.1 SHAPING LOSS FOR REGRESSION". It is better to do this for all the experiments. In this way, we can see why meta-loss can better.
- Besides, it is also better to show some example samples where the learning loss is significantly different from the human-designed one. This helps the reader better understand why the meta-loss can better.
- Finally, the authors claim the extra-information used in the meta-training is helpful. How can we see this point? There is not a step-by-step ablation study on this point.

Q2. Except for problem setup, i.e., learning a loss function, what are novelties in using meta-learning techniques?

Q3. The authors present three usages of the proposed framework in Section 3. Could the authors describe one in detail and then briefly mention the other two usages instead of writing them with the same importance? In this way, readers can understand materials and novelties better. 
- For example, I do not understand how exactly gradients are updated on meta-level. The description in Section 3.1. is too brief. 

Q4. Why the convergence speed of the meta-learner is important? e.g., Figure 4(b-c).

Q5. We have some basic restrictions for "loss function", i.e., loss(x, y) >= 0 for any x, y; loss(x, x) = 0. How such basic requirements are ensured by the learned meta-loss?

Q6. Could the authors add more explanation in the experiments and motivation in the main text? Currently, the authors just describe what they have done in the proposed method and what they have observed in experiments, just a list of facts (see Q1).
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper proposes a new meta-learning method (ML3) that meta-learns a loss function that is able to generalize across tasks. Building upon bi-level optimization framework as in MAML, instead of using a task-specific loss function in the inner loop, the authors compute adapted parameters of the model using a parametrized loss network and learn the loss network via backpropagation. Experiments are conducted on supervised sinusoid regression and binary digit classification as well as on model-based and model-free RL benchmarks.

Overall, this paper is an extension to the gradient-based meta-learning algorithms such as MAML. While the idea is natural, there is a prior work [1] that has investigated the effectiveness of learned loss in gradient-based meta-learning, which seems pretty similar to this paper. I wonder how this method could be compared to [1] in various domains. 

Besides, I wonder how important the extra information added during the meta-training time is and the authors should present comparison to ML3 without the extra information.

Moreover, I believe comparing ML3 to more recent meta-learning algorithms such as various MAML variants (e.g. MAML++), PEARL, LEO, etc. would be important to show the effectiveness of ML3. Right now, the method is only compared to ML3 with task loss, which seems not very conclusive.

[1] Yu, T., Finn, C., Xie, A., Dasari, S., Zhang, T., Abbeel, P., & Levine, S. (2018). One-shot imitation from observing humans via domain-adaptive meta-learning. arXiv preprint arXiv:1802.01557.
review_title: Official Blind Review #2

Meta-review: Despite the new ideas in this paper, reviewers feel that it needs to be revised for clarification, and that experimental results are not convincing.  I have down-weighted the criticisms of Reviewer 2 because I agree with the authors' rebuttal.  However, there is still not enough support among the remaining reviews to justify acceptance. 
Decision: Reject