Editable Neural Networks
Authors: Anton Sinitsin
Vsevolod Plokhotnyuk
Dmitry Pyrkin
Sergei Popov
Artem Babenko
Venue:  (2020)
Keywords: editing
editable
meta-learning
maml

Abstract: These days deep neural networks are ubiquitously used in a wide range of tasks, from image classification and machine translation to face identification and self-driving cars. In many applications, a single model error can lead to devastating financial, reputational and even life-threatening consequences. Therefore, it is crucially important to correct model mistakes quickly as they appear. In this work, we investigate the problem of neural network editing - how one can efficiently patch a mistake of the model on a particular sample, without influencing the model behavior on other samples. Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model. We empirically demonstrate the effectiveness of this method on large-scale image classification and machine translation tasks.

Published as a conference paper at ICLR 2020
EDITABLE NEURAL NETWORKS
Anton Sinitsin"
Vsevolod Plokhotnyuk
Yandex
National Research University
National Research University
Higher School of Economics
Higher School of Economics ant. sinits i gmail..............m....... com
vsevolod-pl haapppllyanee...mm
Dmitry Pyrkin"
Sergei Popov
National Research University Higher School of Economics ellgasterryandee. ru
Yandex
sapopovéyande--tem..
Artem Babenko
Yandex
National Research University
Higher School of Economics
artem. w www..bbbnnokkplyttch...l.
ABSTRACT
These days deep neural networks are ubiquitously used in a wide range of tasks, from image classification and machine translation to face identification and self- driving cars. In many applications, single model error can lead to devastating financial, reputational and even life-threatening consequences. Therefore, it is cru- cially important correct model mistakes quickly as they appear. In this work, we investigate the problem of neural network editing- - how one can efficiently patch mistake of the model on particular sample, without influencing the model be- havior on other samples. Namely, we propose Editable Training, model-agnostic training technique that encourages fast editing of the trained model. We empiri- cally demonstrate the effectiveness of this method on large-scale image classifica- tion and machine translation tasks.
INTRODUCTION
Deep neural networks match and often surpass human performance on wide range of tasks includ- ing visual recognition (Krizhevsky et al. (2012); ; D. C. Ciresan 1)) machine translation (Hassan al. (2018)) and others (Silver et al. (2016)). However, just like humans, artificial neural networks sometimes make mistakes. As we trust them with more and more important decisions, the cost of such mistakes grows ever higher. A single misclassified image can negligible in academic re- search but can be fatal for pedestrian in front of a self-driving vehicle. A poor automatic translation for single sentence can get person arrested (Hern) (208)) or ruin company's reputation.
Since mistakes are inevitable, deep learning practitioners should be able to adjust model behavior by correcting errors as they appear. However, this is often difficult due to the nature of deep neural networks. In most network architectures, prediction for single input depends on all model param- eters. Therefore, updating neural network to change its predictions on single input can decrease performance across other inputs.
Currently, there are two workarounds commonly used by practitioners. First, one can re-train the model on the original dataset augmented with samples that account for the mistake. However, this computationally expensive as it requires to perform the training from scratch. Another solution is use manual cache (e.g. lookup table) that overrules model predictions on problematic samples.
"Equal contribution

---

Published as a conference paper at ICLR 2020
While being simple, this approach is not robust to minor changes in the input. For instance, it will not generalize to a different viewpoint of the same object or paraphrasing in natural language processing tasks.
In this work, we present an alternative approach that we call Editable Training. This approach involves training neural networks in such way that the trained parameters be easily edited af- terwards. Editable Training employs modern meta-learning techniques (Finn et al. (2017)) to ensure that model's mistakes can be corrected without harming its overall performance. With thorough experimental evaluation, we demonstrate that our method works on both small academical datasets and scauee machine learning tasks. We summarize the contributions of this study as follows:
We address new problem of fast editing of neural network models. We argue that this problem is extremely important in practice but, to the best of our knowledge, receives little attention from the academic community.
We propose Editable Training - model-agnosien method of neural network training that learns models, whose errors can then be efficiently corrected
We extensively evaluate Editable Training on large-scale image classification and machine translation tasks, confirming its advantage over existing baselines.
RELATED WORK
In this section, we aim to position our approach with respect to existing literature. Namely, we explain the connections of Editable Neural Networks with ideas from prior works.
Meta-learning is a family of methods that aim to produce learning algorithms, appropriate for particular machine learning setup. These methods were shown to extremely successful in large number of problems, such as few-shot learning (Finn et al. (2017); Nichol et al. (2018)) learnable optimization (Andrychowicz et al. (2016) and reinforcement learning (Houthooft et al. (2018). ). In- deed, Editable Neural Networks also belong the meta-leaming paradigm, as they basically "learn to allow effective patching". While neural network correction has significant practical importance, are not aware of published meta-learning works, addressing this problem.
Catastrophic forgetting a well-known phenomenon arising in the problem of lifelong/continual learning (Ratcliff (1990). For sequence of learning tasks, turns that after deep neural net- works learn on newer tasks, their performance on older tasks deteriorates. Several lines of research address overcoming catastrophic forgetting. The methods based on Elastic Weight Consolidation (Kirkpatrick et al. (2016) update model parameters based on their importance to the previous learn- ing tasks. The rehearsal baased methods (Robins) (1995)) occasionally repeat learning on samples from earlier tasks to "remind" the model about old data. Finally, line of work (Garnelo et al. (2018); Lin et al. (2019) develops specific neural network architectures that reduce the effect of catastrophic forgetting. The problem of efficient neural network patching differs from continual learning, as our setup is not sequential in nature. However, correction of model for mislabeled samples must not affect its behavior other samples, which is close to overcoming catastrophic forgetting task.
Adversarial training. The proposed Editable Training also bears versarial training (Goodfellow et al. (2015)). which is the dominant approach of adversarial attack defense. The important difference here is that Editable Training aims to learn models, whose behav- ior on some samples can be efficiently corrected. Meanwhile, adversarial training produces models, which are robust certain input perturbations. However, practice
resemblance the ad-
can Editable Train- ing to efficiently cover model vulnerabilities against both synthetic (Szegedy et al. (2013); Yuan al. (2017); Ebrahimi et al. (2017); Wallace et al. (2019) and natural (Hendrycks al. (019 adversarial examples.
'The source code is available online httpp:///gttbb..ooo///////iaaaaa
2

---

Published as a conference paper at ICLR 2020
3 EDITING NEURAL NETWORKS
order to measure and optimize the model's ability for editing, we first formally define the oper- ation of editing neural network. Let [(x, 0) be neural network, with denoting its input and being a set of network parameters. The parameters are learned by minimizing task-specific objective function Crase (0). c.g. cross-entropy for multi-class classification problems.
Then, if we discover mistakes in the model's behavior, we can patch the model by changing its parameters A. Here aim to change model's predictions to misclassified objects, without affecting other inputs. We formalize this goal using the editor function: 0=F le)- Informally, this is a function that adjusts 0 to satisfy a given constraint (0) < 0, whose role is to enforce desired changes in the model's behavior.
subset of inputs, corresponding
For instance, in the case of multi-class classification, L can guarantee that the model assigns input x to the desired label Yref: (0) = maxy, log p(y x 0) - log p(yres (z, 0). Under such definition of le, the constraint (0) I 0 is satisfied iff arg maxy, log p (z, 0) = Yref
To be practically feasible, the editor function must meet three natural requirements:
Reliability: the editor must guarantee Le (0) S 0 for the chosen family of Le(-);
Locality: the editor should minimize influence on f(; outside of satisfying (e( S 0:
Efficiency: the editor should be efficient terms of runtime and memory;
Intuitively, the editor locality aims to minimize changes in model's predictions for inputs unrelated to le. For classification problem, this requirement can be formalized as minimizing the difference
between model's predictions over the "control" set X E. (1( (xx + min.
zEX.
3.1 GRADIENT DESCENT EDITOR
A natural way to implement Edit(0, 1.) for deep neural networks is using gradient descent. Param- eters 0 are shifted against the gradient direction aola (f) for several iterations until the constraint (0) < 0 is satisfied. We formulate the SGD) editor with up l steps and learning rate
Edit (0, k) - {Editt
(0, Eddita 1(0 a Vole Le),
if (0) S o k 0
otherwise
The standard gradient descent editor can be further augmented with momentum, adaptive learn- ing rates (Duchi et al. (2010) Zeiler 2012)) and other popular deep learning tricks (Kingma & Ba (2014) Smith & Topin (2017). One technique that we found practically useful is Resilient Backpropagation: RProp, SignSGD by Bernstein et al. (2018) or RMSProp by Ticleman & Hinton 2012). We observed that these methods produce more robust weight updates that improve locality.
3.2 EDITABLE TRAINING
The core idea behind Editable Training is to enforce the model parameters 0 to be "prepared" for the editor function. More formally, we want learn such parameters 0, that the editor Edit(O, L.) is reliable, local and efficient, as defined in above.
Our training procedure employs the fact that Gradient Descent Editor (T) is differentiable w.r.t. 6. This well-known observation (Finn et al. (2017)) allows us to optimize through the editor function directly via backpropagation (see Figure
Editable Training is performed on minibatches of constraints le ~ p(le) (e.g. images and target labels). First, we compute the edited parameters â Edit(0, by applying up & steps of gradient descent (T). Second, we compute the objective that measures locality and efficiency of the editor function:
3

---

Published as a conference paper at ICLR 2020
Fdit:
(.(+)
Edit
i
edtr
Edit
θ
edit
Model:
0
Tioc
I
I %oc
Batch:
X,
L buse
Obj (o)
X y
buse
Obj
Figure 1: A high-level scheme of editable training: (left) forward pass, (right) backward pass.
Obj(0,le) = Lbase (0) + Cedit Ledu (f) + Cloc (1)
(2)
Ledu( max max (Edit (0,1))
(3)
(ior( E DKI(p(y) 0) [12(y) Edit" (0,
(4)
z~p(x)
Intuitively, (0) encourages reliability and efficiency of the editing procedure by making sure the constraint is satisfied in under k gradient steps. The final term (i) is responsible for locality by minimizing the KL divergence between the predictions original and edited models.
We use hyperparameters Cedit: to balance between the original task-specific objective, editor ef- ficiency and locality. Setting both of them to large positive values would cause the model sacrifice some of its performance for better edit. On the other hand, sufficiently small Cedis, Qioe will not cause any deterioration of the main training objective while still improving all our experiments (see Section 4. We attribute this to the fact that neural networks are typically overparameterized. Most neural networks can accommodate the edit-related properties and still have enough capacity to optimize Obj(O, Le). The learning step and other optimizer parameters (e.g. for RMSProp) are trainable parameters of Editable Training and we optimize them explicitly via gradient descent.
editor function in
EXPERIMENTS
In this section, we extensively evaluate Editable Training on several deep learning problems and compare it to existing alternatives for efficient model patching.
4.1 Toy EXPERIMENT: CIFAR-10
First, we experiment on image classification with the small CIFAR-I0 dataset with standard train/test splits (Krizhevsky et al.). The training dataset is further augmented with random crops and random horizontal flips. All models trained on this dataset follow the ResNet-18 (He et al. (2015) architec- ture and use the Adam optimizer (Kingma & Ba (2014)) with default hyperparameters.
Our baseline ResNet-18 (He et al. (2015)) neural network trained minimize the standard cross- entropy loss without Editable Training. This model provides 6.3% test error rate at convergence.
Comparing editor functions. As a preliminary experiment, we compare several variations of edi- tor functions for the baseline model without Editable Training. We evaluate each editor by applying 1-1000 edits l.. Each edit consists of an image from the test set assigned with a random (likely in- correct) label uniformly chosen from 0 to 9, After N independent edits, we compute three following metrics over the entire test set:
Drawdown - mean absolute difference of classification error before and after performing edit. Smaller drawdown indicates better editor locality.
Success Rate - rate of edits, for which editor succeeds in under k=10 gradient steps.
Num Steps an average number of gradient steps needed to perform a single edit.
Gradient Descent (GD) standard gradient descent.
4

---

Published as a conference paper at ICLR 2020
Editor Function
GD
Scaled GD
RProp
R.MPProp
Momentum
Adam
Drawdown
3.8%
2.81 %
1.99%
1.77%
2.42%
19.4%
Success Rate
98.8%
99.1% %
100%
100%
96.0%
100%
Num Steps
3.54
3.91
2.99
3.11
5.60
3.86
Table 1:
Comparison of different editor functions on the CIFARIO dataset with the baseline ResNet18 model trained without Editable Training.
Scaled GD - like GD, but the learning rate is divided by the global gradient norm from the first gradient step.
RProp- like GD, but the algorithm only uses the sign of gradients: 0- ax sign (V (O))
RMSProp like GD, but the learning rate for each individual parameter is divided by
v /rms; + where rmssy [Vo) and rms++1 = 3 rms; + (1 () (V)le
Momentum GD - like GD, but the update follows the accumulated gradient direction is Vg = 0; p++1 = a (06) + μ Vt-
Adam adaptive momentum algorithm described in Kingma & Ba 2014 with tunable a, 32, To prevent Adam from replicating RMSProp, we restrict 31 to [0.1, 1.0] range.
For each optimizer, we tune all hyperparameters (e.g. learning rate) to optimize locality while ensuring that editor succeeds in under h 10 steps for at least 95% of edits. We also tune the editor function by limiting the subset of parameters it is allowed to edit. The ResNet-18 model consists of six parts: initial convolutional layer, followed by four "chains" of residual blocks and final linear layer that predicts class logits. We experimented with editing the whole model as well as editing each individual "chain", leaving parameters from other layers fixed. For each editor Table []] reports the numbers, obtained for the subset editable parameters, corresponding to the smallest drawdown. For completeness, also report the drawdown of Gradient Descent and RMSProp for different subsets of editable parameters in Table
Editable Layers Gradient Descent
Whole Model
Chain I
Chain 2
Chain
Chain 4
3.8%
18.3%
7.7%
5.3%
4.76%
RMSProp
2.29%
22.8%
1.85%
1.77%
1.99%
Table 2: Mean Test Error Drawdown when editing different ResNet18 layers on CIFAR1O.
Table π and Table demonstrate that the editor function locality is heavily affected by the choice of editing function even for models trained without Editable Training. Both RProp and RMSProp significantly outperform the standard Gradient Descent while Momentum and Adam show smaller gains. In fact, without the constraint 31 > 0.1 the tuning procedure returns 81 = O, which makes Adam equivalent RMSProp. We attribute the poor performance of Adam and Momentum the fact that most methods only make a few gradient steps till convergence and the momentum term cannot accumulate the necessary statisties.
Editable Training. Finally, we report results obtained with Editable Training. On each training
batch, we use a single constraint (0) max,, log xx 0) - llo P(Urer lzz 0), where æ is sampled from the train set and Yref random class label (from 9). The model is then trained by directly minimizing objective with k=10 editor steps and all other parameters optimized by backpropagation.
We compare our Editable Training against three baselines, which also allow efficient model cor- rection. The first natural baseline is Elastic Weight Consolidation (Kirkpatrick et al. (2016)): technique that penalizes the edited model with the squared difference in parameter space, weighted by the importance of each parameter. Our second baseline is a semi- comi-parametric Deep k-Nearest Neighbors (DkNN) model (Papernot & McDaniel (2018)) that makes predictions by using k nearest neighbors in the space of embeddings, produced by different CNN layers. For this approach, we edit the model by flipping labels of nearest neighbors until the model predicts the correct class.
Finally we compare to alternative editor function inspired by Conditional Neural Processes (CNP) (Garnelo et al. (2018)) that refer as Editable+CNP, For this baseline, train specialized
5

---

Published as a conference paper at ICLR 2020
CNP model architecture that performs edits by adding special condition vector to intermediate activations. This vector is generated by an additional "encoder" layer. We train the CNP model to solve the original classification problem when the condition vector is zero (hence, the model behaves as standard ResNet18) 8) and minimize Ledst and Lioe when the condition vector applied.
After tuning the CNP architecture, obtained the best performance when the condition vector is computed with a single ResNet block that receives the image representation via activations from the third residual chain of the main ResNet-18 model. This "encoder" also conditions on the target class ffref with an embedding layer (lookup table) that is added to the third chain activations. The resulting procedure becomes the following: first, apply encoder the edited sample and compute the condition vector, then add this vector to the third layer chain activations for all subsequent inputs.
Training
Editor
Editable
Test Error
Test Error
Success
Num
Procedure
Function
Layers
Rate
Drawdown
Rate
Steps
GD
All
6.3%
3.8%
98.8%
3.54
Baseline Training
RMSProp
Chain
6.3%
1.77%
100%
3.11
GD
All
6.34%
1.42%
100%
3.39
Editable Cloe 0.01
GD
Chain
6.28%
1 1.44%
100%
2.82
RMSProp
Chain
6.31 %
0.86 %
100%
4.13
Editable Qloe 0.1
RMSProp
Chain 3
7.19%
0.65 %
100%
4.76
Editablc+CNP (best)
Cond. vector
Chain
6.33%
1.06%
98.9%
n/a
Baseline Training
GD+EWC
Chain
6.3%
1.92%
100%
3.88
Baseline Training
RMSProp+EWC
Chain
6.3%
1.24%
98.1%
4.03
DkNN & 10
Flip Labels
n/a 6.36%
1.76%
100%
n/a
DKNN & 100
Flip Labels
n/a 7.04%
1.05%
100%
n/a
Table Editable Training ResNet18 on CIFARIO dataset with different editor functions.
Table 3 demonstrates two advantages of Editable Training. First, with =0.0 it is able to reduce drawdown (compared to models trained without Editable Training) while having no significant effect on test errоr rate. Second, editing Chain alone is almost as effective as editing the whole model. This is important because it allows us to reduce training time, making Editable Training x 2.5 times slower than baseline training. Note, Editable+CNN turned out to be almost as effective as models trained with gradient-based editors while being simpler to implement.
4.2 ANALYZING EDITED MODELS
In this section, we aim to interpret the differences between the models learned with and without Editable Training. First, investigate which inputs most affected when the model is edited on a sample that belongs to each particular class. Based on Figure 2 (left), we conclude that edits of baseline model cause most drawdown samples that belong the same class the edited input (prior to edit). However, this visualization loses information by reducing edits to their class labels.
In Figure 2 (middle) apply t-SNE (van der Maaten Hinton (2008)) to analyze the structure of the "edit space". Intuitively, two edited versions of the same model are considered close if they make similar predictions. We quantify this by computing KL-divergence between the model's pre- dictions before and after edit for each of 10.000 test samples. These KL divergences effectively form 1..0000mmmmsionia model descriptor. We compute these descriptors for 4.500 edits applied to models trained with and without Editable Training. These vectors are then embedded in two- dimensional space with the t-SNE algorithm. We plot the obtained charts on Figure P (middle), with point colors denoting original class labels of edited images. As expected, the baseline edits for images of the same class are mapped to close points.
In turn, Editable Training does not always follow this pattern: the edit clusters are formed based on both original and target labels with a highly interlinked region in the middle. Combined with the fact that Editable Training has significantly lower drawdown, this lets hypothesize that with Editable Training neural networks learn representations where edits affect objects the same original class to smaller extent. Conversely, the t-SNE visualization lacks information about the true dimensionality

---

Published as a conference paper at ICLR 2020
Confusion matrix
Baseline training
Editable training
Explained variance ratios
1.0
Original labels
544
1329
0.8
0.7
baseline
1
2 3 Drawdown class
4 5 6
2000
Figure 2: Edited model visualizations (Left) Confusion matrix of baseline model: rows correspond to editing images belonging to each of 10 classes; columns represent drawdowns per individual class. (Middle) t-SNE visualizations. Point color represents original class labels; brightness encodes edit targets (Right) The proportion of explained variance versus the number of components.
of the data manifold. To capture this property, we also perform truncated SVD decomposition of the same matrix of descriptors. Our main interest is the number of SVD components required to explain given percentage of data variance. In Figure 2 (right) we report the explained variance ratio for models obtained with and without Editable Training. These results present evidence that Editable Training learns representations that exploit the neural network capacity to greater extent.
4.3 EDITABLE FINE- TUNING FOR LARGE SCALE IMAGE CLASSIFICATION
Section 4.1 demonstrates the success of Editable Training the small CIFAR- 10 dataset. However, many practical applications require training for many weeks on huge datasets. Re-training such model for the sake of better edits may be impractical. In contrast, it would be more efficient to start from a pre-trained model and fine-tune it with Editable Training.
Here experiment with the ILSVRC image classification task (Deng et al. 2009) and consider two pre-trained architectures: smaller ResNet-18 and deeper DenseNet- 169 (Huang et al. (2016) networks. For each architecture, we start with pre-trained model weights and fine-tune them on the same dataset with Editable Training. More specifically, we choose the training objective Laase as KL-divergence between the predictions of the original network and its fine-tuned counterpart. Intuitively, this objective encourages the network to preserve its original classification behavior, while being trained to allow local edits. Similar Section the editor functions are only allowed to modify subset of neural network layers. We experiment with two choices of such subsets. First, we try edit a pre-existing layer in the network. Namely, we select the third out of four "chains" in both architectures. In the second experiment, layer after the last convolutional layer. We set an extra layer to a residual block with a 4096-unit dense layer, followed by ELU activation (Clevert al. (203)) and another 1024-unit dense layer.
augment each architecture with extra trainable
The evaluation is performed on 1 edits with random target class. We measure the drawdown on the full ILSVRC validation set of 50.000 images. We use the SGD optimizer with momentum 1=0.9. We set the learning rate to 10-5 for the pre-existing layers and 10-3 for the extra block. The ImageNet training data is augmented with random resized crops and random horizontal flips.
Our baselines for this task are the pre-trained architectures without Editable Fine-Tuning, However, noticed that minimizing the KL-divergence L(0) has side-effect of im- proving validation error. We attribute this improvement to the self-distillation phenomenon (Hinton al. (2015); )) Furlanello al. 22018). To disentangle these two effects, we consider an additional baseline where the model is trained to minimize the KL-divergence without Editable Training terms. For fair comparison, we also include basclines that edit an extra layer. This layer is initialized at random for the pre-trained models and fine-tuned for the models trained with distillation.
during experiments,
We use publicly available pre-trained models from

---

Published as a conference paper at ICLR 2020
Model
Training
Editable
Test Error
Mean
Success
Num
Architecture
Procedure
Layers
Rate
Drawdown
Rate
Steps
Pre-trained
Chain 3
30.95%
3.89%
99.8%
3,582
Pre-trained
Extra layer
30.95%
9.18%
100%
4.272
ResNet18
Distillation
Extra layer
30.75%
2.80%
100%
2.63
Editable
Chain
30.53%
3.78%
99.8%
3.616
Editable
Extra layer
30.61% %
0.57%
100%
3.388
Pre-trained
Chain 3
25.49%
5.20%
100%
2.551
Pre-trained
Extra layer
25.47%
9.05%
100%
3.874
DenseNet169
Distillation
Extra layer
24.33%
1.67%
100%
2.822
Editable
Chain 3
24.32%
4.47%
100%
2.556
Editable
Extra layer
24.38%
0.96% %
100%
2.970
Table 4: Editable Training on the ImageNet dataset with RMSProp editor function.
The results in Table show that Editable Training can be effectively applied in the fine-tuning scenario, achieving the best results with an extra trainable layer. In cases Editable Fine- Tunnnnuning took under 48 hours single GeForce 1080 Ti GPU while single edit requires less than 150 ms.
4.3. REALISTIC EDIT TASKS WITH NATURAL ADVERSARIAL EXAMPLES
In all previous experiments, we considered edits with randomly chosen target class. However, in many practical scenarios, most these edits will never occur. For instance, far more likely that an image previously classified "plane" would require editing into "bird" than into "truck" or "ship". To address this consideration, we employ the Natural Adversarial Examples (NAE) data set by Hendryykk et al. (2019). This data set contains 7.500 natural images that particularly hard to classify with neural networks. Without edits, pre-trained model can correctly predict less than 1% of NAEs, but the correct answer is likely be within top-100 10-100 classes ordered by predicted probabilities (see Figure leftt).
The next set of experiments quantifies Editable Training in this more realistic setting. All models are evaluated on sample of 1.000 edits, each corresponding to one Natural Adversarial Example and its reference class. We measure the drawdown from each edit 50.000 ILSVRC test images. We evaluate best techniques from Section 4.3 and their modifications that account for NAEs:
Editable Training: Random model trained to edit on random targets from the uniform distribution, same as Table Compared the same pre-trained and distilled baselines.
Editable Training: Match Ranks model trained to edit ImageNet training images with targets sampled based on their rank under NAE rank distribution (see left).
Editable Training: Train on NAE - model trained to edit 6.500 natural adversarial examples. These NAEs do not overlap with 1.000 NAE examples used for evaluation.
The results in Table 5 (top-left) show that Editable Training significantly reduces drawdown for NAEs even when trained with random targets. However, accounting for the distribution of target classes improves locality even further. Surprisingly enough, training on 6.500 actual NAEs fares no better than simply matching the distribution of target ranks.
For the final set evaluations, we consider two realistic scenarios that are not covered by our previous experiments. First, we evaluate whether edits performed by our method generalize to substantially similar inputs. This behavior is highly desirable since we want to avoid the edited model repeating old mistakes in slightly changed context. For each of 1, 000 NAEs, we find the most similar image from test set based on Inception V3 embeddings (Szegedy et al. 2015). For each such pair, edit 5 augmentations of the first image and measure how often the model predicts the edited class on augmentations of the second image. A model trained with random edits has an accuracy of 86.17 6677% while "Editable Match Ranks" scores 85.9% accuracy. Finally, evaluate if
8

---

Published as a conference paper at ICLR 2020
Answer rank distribution on NAE
Training Procedure
Test Error
Success Rate
Num Steps
Drawdown
Baseline Training
5%
Pre-trained
30.99%
4.54%
100%
3.822
Distillation
30.75%
1.62%
100%
2.192
0%
Editable Training
200
400
600
800
1000
Rank of correct answer
Random edits
30.79%
0.314%
100%
2.594
Test error with standard deviation
Match ranks
30.76%
0.146 %
100%
2.149
1.0
pre-trained match rank
Train on NAE
30.86%
0.167%
100%
2.236
0.8
Table 5: Editing Natural Adversarial Examples for ResNet18:
0.6
(Top-Left) Editor effectiveness when editing N = 1000 NAEs: (Top-Right) Reference class rank distribution for baseline model, (Bottom-Rigg Right) Error rate for edit sequences, ResNet18 baseline and Match Ranks. Pale areas indicate std. deviation over 10 runs.
0.4
10
15
25
sequential edits
our technique can perform multiple edits in a sequence. Figure 3 (bottom-left) demonstrates that our approach can cope with sequential edits without ever being trained that way.
4.4 EDITABLE TRAINING FOR MACHINE TRANSLATION
The previous experiments focused on multi-class classification problems. However, Editable Train- ing can be applied to any task where the model is trained by minimizing a differentiable objective. Our final set of experiments demonstrates the applicability of Editable Training for machine transla- tion. We consider the IWSLT 2014 German German-Engllss translation task with the standard training/test splits (Cettolo et al. (2015). The data is preprocessed with Moses Tokenizer (Koehn al. (2007)) and converted to lowercase. We further apply the Byte-Pair Encoding with 10.000 BPE rules learned jointly from German and English training data. Finally, we train the Transformer (Vaswani et al. 2017 model similar to transformer- base configuration, optimized for IWSLT De-En task
Typical machine translation models use beam search find the most likely translation. Hence we consider an edit to be successful if and only if the log-probability of target translation is greater than log-probability of any alternative translation. So, (0) max,, log p(ys|s, 0) log P(vols, 0), where s is source sentence, denotes target translation and {3: H are alternative translations. During training, we approximate this by finding k=32 most likely translations with beam search using the Transformer model trained normally on the same data. The edit targets are sampled from same model by sampling with temperature 1.2.2. The resulting edit consists of three parts: source sentence, a target translation and set of alternative translations.
We define Lioe as Kl--divergence between the predictions the original and edited model averaged E E. Dz (p(ye) yo.r, 0) p(ye [z, Vo.:Edity (0, (e))), where D over target tokens, Lioe v.
a data batch, x and y are the source and translation phrases respectively, yo. denotes translation prefix. The Edit function optimizes the final decoder layer using RMSProp with hyperparameters tuned as Section 4.1 The results in Table 6 show that Editable Training produces a model that matches the baseline translation quality but has less than half of its drawdown.
Training Procedure
Test BLEU
BLEU Drawdown Success rate
Num Steps
Baseline training, 10-
34.77
0.76
100%.
2.35
Editable, Ctoe= 100. 100 10-3
34.80
0.35
100%
3.07
Editable, =00e= a=3 10
34.81
0.17
100%
5.5
Table 6: Evaluation of editable Transformer models IWSLT14 German-Enghin translation task.
We Transformer configuration "transformer iwslt de en" from Fairseq v0.8.0 Ott et al. 2019

---

Published as a conference paper at ICLR 2020
5 CONCLUSION
In this paper we have addressed the efficient correction of neural network mistakes, highly im- portant task for deep learning practitioners. We have proposed several evaluation measures for comparison of different means of model correction. Then we have introduced Editable Training, editing to address corrections
a training procedure that produces models that allow gradient- of the model behaviour. We demonstrate the advantage of Editable Training against reasonable baselines on large-scale image classification and machine translation tasks.
REFERENCES
Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. Learning to leam by gradient descent by gradient descent. ArXiv, abs/1606.044744, 2016.
Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Animashree Anandkumar. Signsgd: Compressed optimisation for non-convex problems. In Jennifer G. Dy and Andreas Krause (eds.), JCML, volume 80 Proceedings of Machine Learning Research, pp. 559- 568. PMLR, 2018. URL httth:ttt//bbp..... uniii.uiiitiiir.. de /t/d//////eeeeeeeeeeeeeeeeeeeeee conf / cmm httlm/mmm2...... 2018. html #Rernstei mntnnstteiiAAAse
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. Report on the 11 th iwslt evaluation campaign iwslt 2014. 2015.
Djórk-Améé Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). CoRR, 1.002899 2015.
Schmidhuber D. Ciresan, Meier. First superhuman visual pattern recognition. IJCNN, 201
Deng. Dong, R. Socher, L.-J. Li, and L. Fei-Fei. ImageNet: Large-Scale Hierarchical Image Database. In CVPRO9, 2009.
John C. Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Mach. Learn. Res., 12:2121-2159 2010.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In ACL, 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017.
Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anand- kumar. Born-again neural networks. In ICMI., 2018.
Marta Garnelo, Dan Rosenbaum, Chris J. Maddison, Tiago Ramalho, David Saxton, Murray Shana- han, Yee Whye Teh, Danilo Jimenez Rezende, and S. M. Ali Eslami. Conditional neural pro- cesses. ArXiv, abs//807.0166 006111111,.1 2018.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference Learning Representations, 2015. URL http:// arxiv. wttpor///bbbssaaoa 6572
Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan R. Clark, Christian Fed- ermann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, T. M. Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang, and Ming Zhou. Achieving human parity on automatic chinese to english news translation. ArXiv, fass/800305555 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. 2016 IEÉE Conference Computer Vision and Pattern Recognition (CVPR), pp. 770- 778, 2015.
Dan Hendrycks, Kevin Keliang Zhao, Steven Basart, Jacob Steinhardt, and Dawn Xiaodong Song. Natural adversarial examples. ArXiv, 0ttss/907.071177 2019.
10

---

Published as a conference paper at ICLR 2020
Alex Hern. Facebook translates "good morning" into "attack them",, leading to arrest. The Guardian, 2018.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. ArXiv, abs/ 00..01231, 2015.
Rein Houthooft, Yuhua Chen, Phillip Isola, Bradly C. Stadie, Filip Wolski, Jonathan Ho, and Pieter Abbeel. Evolved policy gradients. ArXiv, 800.0102.,4 2018.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261-2269, 2016.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR,
abs/ (bss/442..990, 2014.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agmieszka Saobska-Barrinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic for- getting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 114 13:3521 -3526, 2016.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bo- jar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical ma- chine translation. In Proceedings of the 45th Annual Meeting of the Association for Com- putational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pp. 177-180, Prague, Czech Republic, June 2007. Association for Computational Linguisties. URL https: wwwp ac www.wwawwwwb.rrgo..... org. htt.://.wwr.....ooog/////.11111.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced re- search). http::ttp://ee os toronto. wdu/-krizaciiar..ae.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 25, pp. 1097- 1105. Curran Associates, Inc., 2012. URL http::/papers wipeeiss.piis.. nips wtpp.ccppppp..///////cecpp/pppp. 4824 calaeenne-classiiiee iottiiiontiittooo cetpchlloopp.ooooolltuiioonn -eerralllmlmuuurllllala-all- pdf
Min Lin, Jie Fu, and Yoshua Bengio. Conditional computation for continual learning. ArXiv, abs/1906.06615 06635, 2019.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. ArXiv, abs/ /t//1803.0999 2018.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In NAACL-HLT, 2019.
Nicolas Papernot and Patrick D. McDaniel. Deep k-nearest neighbors: Towards confident, inter- pretable and robust deep learning. ArXiv, hin//803.00765 2018.
Roger Ratcliff.
Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
Anthony V. Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connect. Sci., 7:123- 146. 1995.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Mare Lanctot, Sander Dieleman, Dominik Grewe, John Nham. Nal Kalchbrenner, llya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of go with deep neural networks and tree search. Nature, 529:484 4899 2016.

---

Published as a conference paper at ICLR 2020
Leslie N. Smith and Nicholay Topin. Super-convergence: Very fast training of residual networks using large learning rates. ArXiv, abss/170..01200, 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818-22882 2826, 2015.
Tieleman and G. Hinton. Lecture 6.5 RmsPPop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012.
Laurens der Maaten and Geoffrey E. Hinton. Visualizing data using t-sne. 2008
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances Neu- Curran Associates, Inc., 2017. URL ral Information Processing Systems 30, pp. -attettinn-tttn aalualll---eeddeee pdf]
n ps..pp.... wwtp:cppppppp..eeeeecccteeeeeeet 1ttper//111-aattetiiooo
Eric Wallace, Feng Shi, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing nlp. 2019.
Xiaoyong Yuan, Pan He, Qile Zhu, and Xiaolin Li. Adversarial examples: Attacks and defenses for deep learning. IEEE Transactions on Neural Networks and Learning Systems, 30:2805-2824, 2017.
Matthew D. Zeiler. Adadelta: An adaptive learning rate method. ArXiv, abs/1212.5701, 2012.
12


Reviews:
[{"review_id": "r1x1pGP0tH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n-----------\nUpdate after the authors' response: the authors addressed some of my concerns and presented some new results that improve the paper. I am therefore upgrading my score to \"Weak Accept\".\n-----------\n\nThis paper proposes a way to effectively \"patch\" and edit a pre-trained neural network's predictions on problematic data points (e.g. where mistakes on these data points can lead to serious repercussions), without necessarily re-training the network on the entire training set plus the problematic samples. More concretely, the edit operation on the problematic samples are done using a few steps of stochastic gradient descent. \n\nFurthermore, the paper modifies the training procedure to make it easier for the model to perform effective patching, based on three criteria: (i) reliability (i.e. obeying the constraints specified in the edit), (ii) locality (i.e. only minimally changing the model parameters to account for the problematic samples, while ensuring the model's performance on unrelated samples remains consistent and does not degrade), and (iii) efficiency in terms of runtime and memory. To this end, the paper uses a loss term that incorporates these criteria, as weighted by interpolation coefficient hyper-parameters. The edit operation is differentiable with respect to the model parameters, which takes a similar form as the MAML approach and similarly necessitates computing second-order derivatives with respect to the model parameters. Experiments are done on CIFAR-10 toy experiments, large-scale image classification with adversarial examples, and machine translation.\n\nOverall, while the idea of patching neural network predictions is really interesting, I have several concerns regarding the paper in its current form. I am therefore recommending a \"Weak Reject\". I have listed the pros and cons of this paper below, and look forward to the authors' response to the concerns I have raised.\n\nPros:\n1. The paper is well-written, and clearly motivates the problem and why it is important.\n\n2. The proposed approach is explained very clearly and is easy to understand.\n\n3. The paper correctly identifies the use of distillation loss (Table 4) as a potential confound, and runs a distillation baseline without editable training as an additional baseline. This improves the thoroughness of the experiments, and ensures that the gains can really be attributed to the proposed editable training objective.\n\n4. The paper makes interesting connections to other problems where the editable training approach can potentially be useful, such as catastrophic forgetting and adversarial training.\n\nCons:\n1. I still have my doubts about the locality constraint. If the model makes a mistake on some problematic samples, what we want is not just fixing the model's predictions only on these problematic samples, but also on other samples that share a substantial degree of similarity to the problematic samples, so that the model can avoid making the same broader type of mistakes in the future. On the surface, the locality constraint seems to do the opposite, since it confines the impact of the edit operation only to the problematic samples. In contrast, the alternative of re-training the neural network on the full training set augmented with the problematic samples can potentially overcome this problem (since the entire model parameters are updated, correcting the model's mistakes on the problematic samples would help the model avoid making the same mistakes on other similar samples), although of course computationally much more expensive to do.\n\n2. Related to Point 1 above, the equation defining the locality constraint (Eq. 4) seems to require an expectation of x that is drawn from p(x). How is this quantity computed? Does the model assume that x comes from the empirical distribution? Also, the \"control set\" (bottom of page 2) is not well defined. This is very important, since the control set defines what examples should (or should not) be changed by the editing operations. These two points should be clarified further.\n\n3. Related to Points 1 and 2 above, I am not sure whether lower \"Drawdown\" is an indication of better editable training. As mentioned before, ideally the model should not only \"patch\" its mistakes on problematic samples, but also on samples that are substantially similar to these problematic samples. Naturally this might result in more differences from the initial model, which also means less locality (i.e. higher locality/lower drawdown may not necessarily be a good evaluation metric to evaluate edit effectiveness).   \n\n4. The paper contains a substantial set of experiments and analysis based on toy experiments on CIFAR-10, where the labels of some examples are randomly swapped (i.e. random noise). I think this setup is dangerous, since the paper is drawing conclusions based on how well editable training can fit random noise! Instead, all the experiments should be done on real use cases (e.g. adversarial training or catastrophic forgetting). It is unfortunate that the paper has more content for the toy experiments (2.5 pages) than for the adversarial example experiment in image recognition (1.5 pages) or machine translation experiment (0.5 page).\n\n5. The hyper-parameter experiments for selecting the learning rates can be put in the Appendix instead, rather than taking up nearly half a page in the main text (e.g. Table 1). This would leave more space for experiments, more explanation, intuition, analysis, etc.\n\n6. The improvements (in terms of both image classification error rates or machine translation BLEU score) afforded by editable training are quite small. For instance, in the machine translation experiment, Table 6 indicates that editable training only leads to a 0.04 BLEU improvement. This is really small by standard machine translation literature, and can very well be explained by randomness in hyper-parameter initialisation, etc., rather than a better training procedure. It is hard to draw any conclusions based on such small gains.\n\nMore minor points:\n1. The use of bold in the Tables are not very consistent. For instance, in Table 3, for column \"Test Error Rate\", 6.31% is in bold, even though baseline training has an accuracy of 6.3%, which is lower (and thus better). Also, in the \"test error drawdown\" column, 0.86% is in bold, even though there is a lower entry of 0.65%.\n\n2. Missing citation to the work of Furlanello et al. (2018) for self-distillation.\n\nReferences\nTommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born Again Neural Networks. In Proc. of ICML 2018.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SklpoELX9r", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper makes a convincing case for improving the usefulness of production level, large scale models by making them quickly editable without extensive retraining. A standard meta-learning method called MAML is used to augment the initial training phase of such models, or to \"patch\" them later on. The paper demonstrates effectiveness for both image classification and machine translation tasks, covering a wide range of relevant scenarios.\n\nI recommend acceptance because:\n- The paper considers a real issue for production models which are becoming widespread, and retraining for targeted modifications is impractical.\n- Experiments are consistently well designed and executed. The proposed benchmarks are relevant for future work as well.\n- The paper is clear and easy to read.\n\nThe method proposed in [1] is very similar, even if it is used in the continual learning settings. I would have liked to see some continual learning solutions used as baselines and/or combined with the proposed method, as they do manage to improve performance in [1]. Some candidates would be L2-regularization and EWC [2].\n\nReferences:\n[1] Khurram Javed, Martha White. Meta-Learning Representations for Continual Learning. CORR 2019. https://arxiv.org/pdf/1905.12588.pdf\n[2] James Kirkpatrick et al. Overcoming catastrophic forgetting in neural networks. PNAS 2017. https://arxiv.org/pdf/1612.00796.pdf", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "BylifkEWqH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose Editable Training that edits/updates a trained model using a model-agnostic training technique. Editable training is able to correct mistakes of trained models without retraining the whole model nor harming the original performance. This is attained via meta-learning techniques to avoid catastrophic forgetting, and an editor function to promise mistake correction. The major contribution is a model-agnostic editable training process that is applicable to various neural nets. This paper has brought attention to mistake correction problem in neural networks and proposes a simple and concise solution. In addition, extensive experiments on both small and large-scale image classification and machine translation tasks demonstrate the effectiveness of different editor functions. Overall, this paper is well-written with extensive experimental results. Below are a few concerns I have to the current status of the paper.\n\n1.\tIt would be interesting to discuss if how a good editor function changes over different models, problems, or even l_e’s. In addition.\n2.\tIn general, a DNN needs to be “edited”/”fixed”, when the training data used are not sufficient, and /or the incoming testing data have a different distribution from the training data. In the latter case, say, if the distribution of new data is significantly different from the training data used so far, it may be worth of re-train the model rather than attempting to “fix” the network. There should be a trade-off between “fixable” vs “not-fixable”. It is unclear how this trade-off is modeled/discussed in the paper.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method which patches/edits a pre-trained neural network's predictions on problematic data points. They do this without the need for retraining the network on the entire data, by only using a few steps of stochastic gradient descent, and thereby avoiding influencing model behaviour on other samples. The post patching training can encourage reliability, locality and efficiency by using a loss function which incorporates these three criteria weighted by hyperparameters. Experiments are done on CIFAR-10 toy experiments, large-scale image classification with adversarial examples, and machine translation. The reviews are generally positive, with significant author response, a new improved version of the paper, and further discussion. This is a well written paper with convincing results, and it addresses a serious problem for production models, I therefore recommend that it is accepted. 
Decision: Accept (Poster)