Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: This paper makes a convincing case for improving the usefulness of production level, large scale models by making them quickly editable without extensive retraining. A standard meta-learning method called MAML is used to augment the initial training phase of such models, or to "patch" them later on. The paper demonstrates effectiveness for both image classification and machine translation tasks, covering a wide range of relevant scenarios.

I recommend acceptance because:
- The paper considers a real issue for production models which are becoming widespread, and retraining for targeted modifications is impractical.
- Experiments are consistently well designed and executed. The proposed benchmarks are relevant for future work as well.
- The paper is clear and easy to read.

The method proposed in [1] is very similar, even if it is used in the continual learning settings. I would have liked to see some continual learning solutions used as baselines and/or combined with the proposed method, as they do manage to improve performance in [1]. Some candidates would be L2-regularization and EWC [2].

References:
[1] Khurram Javed, Martha White. Meta-Learning Representations for Continual Learning. CORR 2019. https://arxiv.org/pdf/1905.12588.pdf
[2] James Kirkpatrick et al. Overcoming catastrophic forgetting in neural networks. PNAS 2017. https://arxiv.org/pdf/1612.00796.pdf
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The authors propose Editable Training that edits/updates a trained model using a model-agnostic training technique. Editable training is able to correct mistakes of trained models without retraining the whole model nor harming the original performance. This is attained via meta-learning techniques to avoid catastrophic forgetting, and an editor function to promise mistake correction. The major contribution is a model-agnostic editable training process that is applicable to various neural nets. This paper has brought attention to mistake correction problem in neural networks and proposes a simple and concise solution. In addition, extensive experiments on both small and large-scale image classification and machine translation tasks demonstrate the effectiveness of different editor functions. Overall, this paper is well-written with extensive experimental results. Below are a few concerns I have to the current status of the paper.

1.	It would be interesting to discuss if how a good editor function changes over different models, problems, or even l_e’s. In addition.
2.	In general, a DNN needs to be “edited”/”fixed”, when the training data used are not sufficient, and /or the incoming testing data have a different distribution from the training data. In the latter case, say, if the distribution of new data is significantly different from the training data used so far, it may be worth of re-train the model rather than attempting to “fix” the network. There should be a trade-off between “fixable” vs “not-fixable”. It is unclear how this trade-off is modeled/discussed in the paper.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: 
-----------
Update after the authors' response: the authors addressed some of my concerns and presented some new results that improve the paper. I am therefore upgrading my score to "Weak Accept".
-----------

This paper proposes a way to effectively "patch" and edit a pre-trained neural network's predictions on problematic data points (e.g. where mistakes on these data points can lead to serious repercussions), without necessarily re-training the network on the entire training set plus the problematic samples. More concretely, the edit operation on the problematic samples are done using a few steps of stochastic gradient descent. 

Furthermore, the paper modifies the training procedure to make it easier for the model to perform effective patching, based on three criteria: (i) reliability (i.e. obeying the constraints specified in the edit), (ii) locality (i.e. only minimally changing the model parameters to account for the problematic samples, while ensuring the model's performance on unrelated samples remains consistent and does not degrade), and (iii) efficiency in terms of runtime and memory. To this end, the paper uses a loss term that incorporates these criteria, as weighted by interpolation coefficient hyper-parameters. The edit operation is differentiable with respect to the model parameters, which takes a similar form as the MAML approach and similarly necessitates computing second-order derivatives with respect to the model parameters. Experiments are done on CIFAR-10 toy experiments, large-scale image classification with adversarial examples, and machine translation.

Overall, while the idea of patching neural network predictions is really interesting, I have several concerns regarding the paper in its current form. I am therefore recommending a "Weak Reject". I have listed the pros and cons of this paper below, and look forward to the authors' response to the concerns I have raised.

Pros:
1. The paper is well-written, and clearly motivates the problem and why it is important.

2. The proposed approach is explained very clearly and is easy to understand.

3. The paper correctly identifies the use of distillation loss (Table 4) as a potential confound, and runs a distillation baseline without editable training as an additional baseline. This improves the thoroughness of the experiments, and ensures that the gains can really be attributed to the proposed editable training objective.

4. The paper makes interesting connections to other problems where the editable training approach can potentially be useful, such as catastrophic forgetting and adversarial training.

Cons:
1. I still have my doubts about the locality constraint. If the model makes a mistake on some problematic samples, what we want is not just fixing the model's predictions only on these problematic samples, but also on other samples that share a substantial degree of similarity to the problematic samples, so that the model can avoid making the same broader type of mistakes in the future. On the surface, the locality constraint seems to do the opposite, since it confines the impact of the edit operation only to the problematic samples. In contrast, the alternative of re-training the neural network on the full training set augmented with the problematic samples can potentially overcome this problem (since the entire model parameters are updated, correcting the model's mistakes on the problematic samples would help the model avoid making the same mistakes on other similar samples), although of course computationally much more expensive to do.

2. Related to Point 1 above, the equation defining the locality constraint (Eq. 4) seems to require an expectation of x that is drawn from p(x). How is this quantity computed? Does the model assume that x comes from the empirical distribution? Also, the "control set" (bottom of page 2) is not well defined. This is very important, since the control set defines what examples should (or should not) be changed by the editing operations. These two points should be clarified further.

3. Related to Points 1 and 2 above, I am not sure whether lower "Drawdown" is an indication of better editable training. As mentioned before, ideally the model should not only "patch" its mistakes on problematic samples, but also on samples that are substantially similar to these problematic samples. Naturally this might result in more differences from the initial model, which also means less locality (i.e. higher locality/lower drawdown may not necessarily be a good evaluation metric to evaluate edit effectiveness).   

4. The paper contains a substantial set of experiments and analysis based on toy experiments on CIFAR-10, where the labels of some examples are randomly swapped (i.e. random noise). I think this setup is dangerous, since the paper is drawing conclusions based on how well editable training can fit random noise! Instead, all the experiments should be done on real use cases (e.g. adversarial training or catastrophic forgetting). It is unfortunate that the paper has more content for the toy experiments (2.5 pages) than for the adversarial example experiment in image recognition (1.5 pages) or machine translation experiment (0.5 page).

5. The hyper-parameter experiments for selecting the learning rates can be put in the Appendix instead, rather than taking up nearly half a page in the main text (e.g. Table 1). This would leave more space for experiments, more explanation, intuition, analysis, etc.

6. The improvements (in terms of both image classification error rates or machine translation BLEU score) afforded by editable training are quite small. For instance, in the machine translation experiment, Table 6 indicates that editable training only leads to a 0.04 BLEU improvement. This is really small by standard machine translation literature, and can very well be explained by randomness in hyper-parameter initialisation, etc., rather than a better training procedure. It is hard to draw any conclusions based on such small gains.

More minor points:
1. The use of bold in the Tables are not very consistent. For instance, in Table 3, for column "Test Error Rate", 6.31% is in bold, even though baseline training has an accuracy of 6.3%, which is lower (and thus better). Also, in the "test error drawdown" column, 0.86% is in bold, even though there is a lower entry of 0.65%.

2. Missing citation to the work of Furlanello et al. (2018) for self-distillation.

References
Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born Again Neural Networks. In Proc. of ICML 2018.
review_title: Official Blind Review #2

Meta-review: This paper proposes a method which patches/edits a pre-trained neural network's predictions on problematic data points. They do this without the need for retraining the network on the entire data, by only using a few steps of stochastic gradient descent, and thereby avoiding influencing model behaviour on other samples. The post patching training can encourage reliability, locality and efficiency by using a loss function which incorporates these three criteria weighted by hyperparameters. Experiments are done on CIFAR-10 toy experiments, large-scale image classification with adversarial examples, and machine translation. The reviews are generally positive, with significant author response, a new improved version of the paper, and further discussion. This is a well written paper with convincing results, and it addresses a serious problem for production models, I therefore recommend that it is accepted. 
Decision: Accept (Poster)