Original: E DITABLE N EURAL N ETWORKS
Revised: E DITABLE N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: These days deep neural networks are ubiquitously used in a wide range of tasks, from image classiﬁcation and machine translation to face identiﬁcation and selfdriving cars.
Revised: These days deep neural networks are ubiquitously used in a wide range of tasks, from image classiﬁcation and machine translation to face identiﬁcation and selfdriving cars.

Original: In many applications, a single model error can lead to devastating ﬁnancial, reputational and even life-threatening consequences.
Revised: In many applications, a single model error can lead to devastating ﬁnancial, reputational and even life-threatening consequences.

Original: Therefore, it is crucially important to correct model mistakes quickly as they appear.
Revised: Therefore, it is crucially important to correct model mistakes quickly as they appear.

Original: In this work, we investigate the problem of neural network editing — how one can efﬁciently patch a mistake of the model on a particular sample, without inﬂuencing the model behavior on other samples.
Revised: In this work, we investigate the problem of neural network editing — how one can efﬁciently patch a mistake of the model on a particular sample, without inﬂuencing the model behavior on other samples.

Original: Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model.
Revised: Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model.

Original: We empirically demonstrate the effectiveness of this method on large-scale image classiﬁcation and machine translation tasks.
Revised: We empirically demonstrate the effectiveness of this method on large-scale image classiﬁcation and machine translation tasks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks match and often surpass human performance on a wide range of tasks including visual recognition (Krizhevsky et al.
Revised: Deep neural networks match and often surpass human performance on a wide range of tasks including visual recognition (Krizhevsky et al.

Original: D. C. Ciresan (2011)), machine translation (Hassan et al.
Revised: D. C. Ciresan (2011)), machine translation (Hassan et al.

Original: and others (Silver et al.
Revised: and others (Silver et al.

Original: However, just like humans, artiﬁcial neural networks sometimes make mistakes.
Revised: However, just like humans, artiﬁcial neural networks sometimes make mistakes.

Original: As we trust them with more and more important decisions, the cost of such mistakes grows ever higher.
Revised: As we trust them with more and more important decisions, the cost of such mistakes grows ever higher.

Original: A single misclassiﬁed image can be negligible in academic research but can be fatal for a pedestrian in front of a self-driving vehicle.
Revised: A single misclassiﬁed image can be negligible in academic research but can be fatal for a pedestrian in front of a self-driving vehicle.

Original: A poor automatic translation for a single sentence can get a person arrested (Hern (2018)) or ruin company’s reputation.
Revised: A poor automatic translation for a single sentence can get a person arrested (Hern (2018)) or ruin company’s reputation.

Original: Since mistakes are inevitable, deep learning practitioners should be able to adjust model behavior by correcting errors as they appear.
Revised: Since mistakes are inevitable, deep learning practitioners should be able to adjust model behavior by correcting errors as they appear.

Original: However, this is often difﬁcult due to the nature of deep neural networks.
Revised: However, this is often difﬁcult due to the nature of deep neural networks.

Original: In most network architectures, a prediction for a single input depends on all model parameters.
Revised: In most network architectures, a prediction for a single input depends on all model parameters.

Original: Therefore, updating a neural network to change its predictions on a single input can decrease performance across other inputs.
Revised: Therefore, updating a neural network to change its predictions on a single input can decrease performance across other inputs.

Original: Currently, there are two workarounds commonly used by practitioners.
Revised: Currently, there are two workarounds commonly used by practitioners.

Original: First, one can re-train the model on the original dataset augmented with samples that account for the mistake.
Revised: First, one can re-train the model on the original dataset augmented with samples that account for the mistake.

Original: However, this is computationally expensive as it requires to perform the training from scratch.
Revised: However, this is computationally expensive as it requires to perform the training from scratch.

Original: Another solution is to use a manual cache (e.g. lookup table) that overrules model predictions on problematic samples.
Revised: Another solution is to use a manual cache (e.g. lookup table) that overrules model predictions on problematic samples.

Original: While being simple, this approach is not robust to minor changes in the input.
Revised: While being simple, this approach is not robust to minor changes in the input.

Original: For instance, it will not generalize to a different viewpoint of the same object or paraphrasing in natural language processing tasks.
Revised: For instance, it will not generalize to a different viewpoint of the same object or paraphrasing in natural language processing tasks.

Original: In this work, we present an alternative approach that we call Editable Training.
Revised: In this work, we present an alternative approach that we call Editable Training.

Original: This approach involves training neural networks in such a way that the trained parameters can be easily edited afterwards.
Revised: This approach involves training neural networks in such a way that the trained parameters can be easily edited afterwards.

Original: Editable Training employs modern meta-learning techniques (Finn et al.
Revised: Editable Training employs modern meta-learning techniques (Finn et al.

Original: to ensure that model’s mistakes can be corrected without harming its overall performance.
Revised: to ensure that model’s mistakes can be corrected without harming its overall performance.

Original: With thorough experimental evaluation, we demonstrate that our method works on both small academical datasets and industry-scale machine learning tasks.
Revised: With thorough experimental evaluation, we demonstrate that our method works on both small academical datasets and industry-scale machine learning tasks.

Original: We summarize the contributions of this study as follows:
Revised: We summarize the contributions of this study as follows:

Original: • We address a new problem of fast editing of neural network models.
Revised: • We address a new problem of fast editing of neural network models.

Original: We argue that this problem is extremely important in practice but, to the best of our knowledge, receives little attention from the academic community.
Revised: We argue that this problem is extremely important in practice but, to the best of our knowledge, receives little attention from the academic community.

Original: • We propose Editable Training — a model-agnostic method of neural network training that learns models, whose errors can then be efﬁciently corrected.•
Revised: • We propose Editable Training — a model-agnostic method of neural network training that learns models, whose errors can then be efﬁciently corrected.•

Original: We extensively evaluate Editable Training on large-scale image classiﬁcation and machine translation tasks, conﬁrming its advantage over existing baselines.
Revised: We extensively evaluate Editable Training on large-scale image classiﬁcation and machine translation tasks, conﬁrming its advantage over existing baselines.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: In this section, we aim to position our approach with respect to existing literature.
Revised: In this section, we aim to position our approach with respect to existing literature.

Original: Namely, we explain the connections of Editable Neural Networks with ideas from prior works.
Revised: Namely, we explain the connections of Editable Neural Networks with ideas from prior works.

Original: Meta-learning is a family of methods that aim to produce learning algorithms, appropriate for a particular machine learning setup.
Revised: Meta-learning is a family of methods that aim to produce learning algorithms, appropriate for a particular machine learning setup.

Original: These methods were shown to be extremely successful in a large number of problems, such as few-shot learning (Finn et al.
Revised: These methods were shown to be extremely successful in a large number of problems, such as few-shot learning (Finn et al.

Original: ; Nichol et al.
Revised: ; Nichol et al.

Original: learnable optimization (Andrychowicz et al.
Revised: learnable optimization (Andrychowicz et al.

Original: and reinforcement learning (Houthooft et al.
Revised: and reinforcement learning (Houthooft et al.

Original: Indeed, Editable Neural Networks also belong to the meta-learning paradigm, as they basically ”learn to allow effective patching”.
Revised: Indeed, Editable Neural Networks also belong to the meta-learning paradigm, as they basically ”learn to allow effective patching”.

Original: While neural network correction has signiﬁcant practical importance, we are not aware of published meta-learning works, addressing this problem.
Revised: While neural network correction has signiﬁcant practical importance, we are not aware of published meta-learning works, addressing this problem.

Original: Catastrophic forgetting is a well-known phenomenon arising in the problem of lifelong/continual learning (Ratcliff (1990)).
Revised: Catastrophic forgetting is a well-known phenomenon arising in the problem of lifelong/continual learning (Ratcliff (1990)).

Original: For a sequence of learning tasks, it turns out that after deep neural networks learn on newer tasks, their performance on older tasks deteriorates.
Revised: For a sequence of learning tasks, it turns out that after deep neural networks learn on newer tasks, their performance on older tasks deteriorates.

Original: Several lines of research address overcoming catastrophic forgetting.
Revised: Several lines of research address overcoming catastrophic forgetting.

Original: The methods based on Elastic Weight Consolidation (Kirkpatrick et al.
Revised: The methods based on Elastic Weight Consolidation (Kirkpatrick et al.

Original: (2016)) update model parameters based on their importance to the previous learning tasks.
Revised: (2016)) update model parameters based on their importance to the previous learning tasks.

Original: The rehearsal-based methods (Robins (1995)) occasionally repeat learning on samples from earlier tasks to ”remind” the model about old data.
Revised: The rehearsal-based methods (Robins (1995)) occasionally repeat learning on samples from earlier tasks to ”remind” the model about old data.

Original: Finally, a line of work (Garnelo et al.
Revised: Finally, a line of work (Garnelo et al.

Original: Lin et al.
Revised: Lin et al.

Original: (2019)) develops speciﬁc neural network architectures that reduce the effect of catastrophic forgetting.
Revised: (2019)) develops speciﬁc neural network architectures that reduce the effect of catastrophic forgetting.

Original: The problem of efﬁcient neural network patching differs from continual learning, as our setup is not sequential in nature.
Revised: The problem of efﬁcient neural network patching differs from continual learning, as our setup is not sequential in nature.

Original: However, correction of model for mislabeled samples must not affect its behavior on other samples, which is close to overcoming catastrophic forgetting task.
Revised: However, correction of model for mislabeled samples must not affect its behavior on other samples, which is close to overcoming catastrophic forgetting task.

Original: Adversarial training.
Revised: Adversarial training.

Original: The proposed Editable Training also bears some resemblance to the adversarial training (Goodfellow et al.
Revised: The proposed Editable Training also bears some resemblance to the adversarial training (Goodfellow et al.

Original: (2015)), which is the dominant approach of adversarial attack defense.
Revised: (2015)), which is the dominant approach of adversarial attack defense.

Original: The important difference here is that Editable Training aims to learn models, whose behavior on some samples can be efﬁciently corrected.
Revised: The important difference here is that Editable Training aims to learn models, whose behavior on some samples can be efﬁciently corrected.

Original: Meanwhile, adversarial training produces models, which are robust to certain input perturbations.
Revised: Meanwhile, adversarial training produces models, which are robust to certain input perturbations.

Original: However, in practice one can use Editable Training to efﬁciently cover model vulnerabilities against both synthetic (Szegedy et al.
Revised: However, in practice one can use Editable Training to efﬁciently cover model vulnerabilities against both synthetic (Szegedy et al.

Original: ; Yuan et al.
Revised: ; Yuan et al.

Original: (2017); Ebrahimi et al.
Revised: (2017); Ebrahimi et al.

Original: ; Wallace et al.
Revised: ; Wallace et al.

Original: and natural (Hendrycks et al.
Revised: and natural (Hendrycks et al.

Original: adversarial examples.
Revised: adversarial examples.

Original:  E DITING N EURAL N ETWORKS
Revised:  E DITING N EURAL N ETWORKS

Original: In order to measure and optimize the model’s ability for editing, we ﬁrst formally deﬁne the operation of editing a neural network.
Revised: In order to measure and optimize the model’s ability for editing, we ﬁrst formally deﬁne the operation of editing a neural network.

Original: Let f ( x, θ ) be a neural network, with x denoting its input and θ being a set of network parameters.
Revised: Let f ( x, θ ) be a neural network, with x denoting its input and θ being a set of network parameters.

Original: The parameters θ are learned by minimizing a task-speciﬁc objective function L base ( θ ) , e.g. cross-entropy for multi-class classiﬁcation problems.
Revised: The parameters θ are learned by minimizing a task-speciﬁc objective function L base ( θ ) , e.g. cross-entropy for multi-class classiﬁcation problems.

Original: Then, if we discover mistakes in the model’s behavior, we can patch the model by changing its parameters θ .
Revised: Then, if we discover mistakes in the model’s behavior, we can patch the model by changing its parameters θ .

Original: Here we aim to change model’s predictions on a subset of inputs, corresponding to misclassiﬁed objects, without affecting other inputs.
Revised: Here we aim to change model’s predictions on a subset of inputs, corresponding to misclassiﬁed objects, without affecting other inputs.

Original: We formalize this goal using the editor function : ˆ θ =
Revised: We formalize this goal using the editor function : ˆ θ =

Original: Edit ( θ, l e ) .
Revised: Edit ( θ, l e ) .

Original: Informally, this is a function that adjusts θ to satisfy a given constraint l e (ˆ θ ) ≤ 0 , whose role is to enforce desired changes in the model’s behavior.
Revised: Informally, this is a function that adjusts θ to satisfy a given constraint l e (ˆ θ ) ≤ 0 , whose role is to enforce desired changes in the model’s behavior.

Original: For instance, in the case of multi-class classiﬁcation, l e can guarantee that the model assigns input x to the desired label y ref : l e (ˆ θ ) =
Revised: For instance, in the case of multi-class classiﬁcation, l e can guarantee that the model assigns input x to the desired label y ref : l e (ˆ θ ) =

Original: max y
Revised: max y

Original: i log p ( y i | x, ˆ θ ) − log p ( y ref | x, ˆ θ ) .
Revised: i log p ( y i | x, ˆ θ ) − log p ( y ref | x, ˆ θ ) .

Original: Under such deﬁnition of l e , the constraint l e (ˆ θ ) ≤ 0 is satisﬁed iff arg max
Revised: Under such deﬁnition of l e , the constraint l e (ˆ θ ) ≤ 0 is satisﬁed iff arg max

Original: y
Revised: y

Original: i log p ( y i | x, ˆ θ ) =
Revised: i log p ( y i | x, ˆ θ ) =

Original: y ref .
Revised: y ref .

Original: To be practically feasible, the editor function must meet three natural requirements:
Revised: To be practically feasible, the editor function must meet three natural requirements:

Original: • Reliability: the editor must guarantee l e (ˆ θ ) ≤ 0 for the chosen family of l e ( · ) ; • Locality: the editor should minimize inﬂuence on f ( · , ˆ θ ) outside of satisfying l e ( ˆ θ ) ≤ 0 ; • Efﬁciency: the editor should be efﬁcient in terms of runtime and memory;
Revised: • Reliability: the editor must guarantee l e (ˆ θ ) ≤ 0 for the chosen family of l e ( · ) ; • Locality: the editor should minimize inﬂuence on f ( · , ˆ θ ) outside of satisfying l e ( ˆ θ ) ≤ 0 ; • Efﬁciency: the editor should be efﬁcient in terms of runtime and memory;

Original: Intuitively, the editor locality aims to minimize changes in model’s predictions for inputs unrelated to l e .
Revised: Intuitively, the editor locality aims to minimize changes in model’s predictions for inputs unrelated to l e .

Original: For classiﬁcation problem, this requirement can be formalized as minimizing the difference between model’s predictions over the ”control” set X c :
Revised: For classiﬁcation problem, this requirement can be formalized as minimizing the difference between model’s predictions over the ”control” set X c :

Original: E x ∈ X c
Revised: E x ∈ X c

Original: #[ f ( x, ˆ θ ) (cid:54) =
Revised: #[ f ( x, ˆ θ ) (cid:54) =

Original: f ( x, θ )] → min .
Revised: f ( x, θ )] → min .

Original: 3. G RADIENT D ESCENT E DITOR
Revised: 3. G RADIENT D ESCENT E DITOR

Original: A natural way to implement Edit ( θ, l e ) for deep neural networks is using gradient descent.
Revised: A natural way to implement Edit ( θ, l e ) for deep neural networks is using gradient descent.

Original: Parameters θ are shifted against the gradient direction − α ∇ θ l e ( θ ) for several iterations until the constraint l e ( θ ) ≤ 0 is satisﬁed.
Revised: Parameters θ are shifted against the gradient direction − α ∇ θ l e ( θ ) for several iterations until the constraint l e ( θ ) ≤ 0 is satisﬁed.

Original: We formulate the SGD editor with up to k steps and learning rate α as:
Revised: We formulate the SGD editor with up to k steps and learning rate α as:

Original: [Equation]
Revised: [Equation]

Original: The standard gradient descent editor can be further augmented with momentum, adaptive learning rates (Duchi et al.
Revised: The standard gradient descent editor can be further augmented with momentum, adaptive learning rates (Duchi et al.

Original: ; Zeiler (2012)) and other popular deep learning tricks (Kingma &
Revised: ; Zeiler (2012)) and other popular deep learning tricks (Kingma &

Original: Ba (2014); Smith & Topin (2017)).
Revised: Ba (2014); Smith & Topin (2017)).

Original: One technique that we found practically useful is Resilient
Revised: One technique that we found practically useful is Resilient

Original: Backpropagation: RProp, SignSGD by Bernstein et al.
Revised: Backpropagation: RProp, SignSGD by Bernstein et al.

Original: (2018) or RMSProp by Tieleman & Hinton (2012).
Revised: (2018) or RMSProp by Tieleman & Hinton (2012).

Original: We observed that these methods produce more robust weight updates that improve locality.
Revised: We observed that these methods produce more robust weight updates that improve locality.

Original: 3. E DITABLE T RAINING
Revised: 3. E DITABLE T RAINING

Original: The core idea behind Editable Training is to enforce the model parameters θ to be ”prepared” for the editor function.
Revised: The core idea behind Editable Training is to enforce the model parameters θ to be ”prepared” for the editor function.

Original: More formally, we want to learn such parameters θ , that the editor Edit ( θ, l e ) is reliable, local and efﬁcient, as deﬁned in above.
Revised: More formally, we want to learn such parameters θ , that the editor Edit ( θ, l e ) is reliable, local and efﬁcient, as deﬁned in above.

Original: Our training procedure employs the fact that Gradient Descent Editor (1) is differentiable w.r.t.
Revised: Our training procedure employs the fact that Gradient Descent Editor (1) is differentiable w.r.t.

Original: θ .
Revised: θ .

Original: This well-known observation (Finn et al.
Revised: This well-known observation (Finn et al.

Original: (2017)) allows us to optimize through the editor function directly via backpropagation (see Figure 1).
Revised: (2017)) allows us to optimize through the editor function directly via backpropagation (see Figure 1).

Original: [Equation]
Revised: [Equation]

Original: Model:
Revised: Model:

Original: Edit:
Revised: Edit:

Original: Batch:
Revised: Batch:

Original: [Equation]
Revised: [Equation]

Original: Editable Training is performed on minibatches of constraints l e ∼ p ( l e ) (e.g. images and target labels).
Revised: Editable Training is performed on minibatches of constraints l e ∼ p ( l e ) (e.g. images and target labels).

Original: First, we compute the edited parameters ˆ θ =
Revised: First, we compute the edited parameters ˆ θ =

Original: Edit ( θ, l e ) by applying up to k steps of gradient descent (1).
Revised: Edit ( θ, l e ) by applying up to k steps of gradient descent (1).

Original: Second, we compute the objective that measures locality and efﬁciency of the editor function:
Revised: Second, we compute the objective that measures locality and efﬁciency of the editor function:

Original: [Equation]
Revised: [Equation]

Original: Intuitively, L edit ( θ ) encourages reliability and efﬁciency of the editing procedure by making sure the constraint is satisﬁed in under k gradient steps.
Revised: Intuitively, L edit ( θ ) encourages reliability and efﬁciency of the editing procedure by making sure the constraint is satisﬁed in under k gradient steps.

Original: The ﬁnal term L loc ( θ ) is responsible for locality by minimizing the KL divergence between the predictions of original and edited models.
Revised: The ﬁnal term L loc ( θ ) is responsible for locality by minimizing the KL divergence between the predictions of original and edited models.

Original: We use hyperparameters c edit , c loc to balance between the original task-speciﬁc objective, editor efﬁciency and locality.
Revised: We use hyperparameters c edit , c loc to balance between the original task-speciﬁc objective, editor efﬁciency and locality.

Original: Setting both of them to large positive values would cause the model to sacriﬁce some of its performance for a better edit.
Revised: Setting both of them to large positive values would cause the model to sacriﬁce some of its performance for a better edit.

Original: On the other hand, sufﬁciently small c edit , c loc will not cause any deterioration of the main training objective while still improving the editor function in all our experiments (see Section 4).
Revised: On the other hand, sufﬁciently small c edit , c loc will not cause any deterioration of the main training objective while still improving the editor function in all our experiments (see Section 4).

Original: We attribute this to the fact that neural networks are typically overparameterized.
Revised: We attribute this to the fact that neural networks are typically overparameterized.

Original: Most neural networks can accommodate the edit-related properties and still have enough capacity to optimize Obj ( θ, l e ) .
Revised: Most neural networks can accommodate the edit-related properties and still have enough capacity to optimize Obj ( θ, l e ) .

Original: The learning step α and other optimizer parameters (e.g. β for RMSProp) are trainable parameters of Editable Training and we optimize them explicitly via gradient descent.
Revised: The learning step α and other optimizer parameters (e.g. β for RMSProp) are trainable parameters of Editable Training and we optimize them explicitly via gradient descent.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we extensively evaluate Editable Training on several deep learning problems and compare it to existing alternatives for efﬁcient model patching.
Revised: In this section, we extensively evaluate Editable Training on several deep learning problems and compare it to existing alternatives for efﬁcient model patching.

Original: 4. T OY E XPERIMENT : CIFAR- 
Revised: 4. T OY E XPERIMENT : CIFAR-

Original: First, we experiment on image classiﬁcation with the small CIFAR-10 dataset with standard train/test splits (Krizhevsky et al.).
Revised: First, we experiment on image classiﬁcation with the small CIFAR-10 dataset with standard train/test splits (Krizhevsky et al.).

Original: The training dataset is further augmented with random crops and random horizontal ﬂips.
Revised: The training dataset is further augmented with random crops and random horizontal ﬂips.

Original: All models trained on this dataset follow the ResNet-18 (He et al. (2015))
Revised: All models trained on this dataset follow the ResNet-18 (He et al. (2015))

Original: architecture and use the Adam optimizer (Kingma & Ba (2014)) with default hyperparameters.
Revised: architecture and use the Adam optimizer (Kingma & Ba (2014)) with default hyperparameters.

Original: Our baseline is ResNet-18 (He et al.
Revised: Our baseline is ResNet-18 (He et al.

Original: neural network trained to minimize the standard crossentropy loss without Editable Training.
Revised: neural network trained to minimize the standard crossentropy loss without Editable Training.

Original: This model provides 6.3% test error rate at convergence.
Revised: This model provides 6.3% test error rate at convergence.

Original: Comparing editor functions.
Revised: Comparing editor functions.

Original: As a preliminary experiment, we compare several variations of editor functions for the baseline model without Editable Training.
Revised: As a preliminary experiment, we compare several variations of editor functions for the baseline model without Editable Training.

Original: We evaluate each editor by applying
Revised: We evaluate each editor by applying

Original: N =1000 edits l e .
Revised: N =1000 edits l e .

Original: Each edit consists of an image from the test set assigned with a random (likely incorrect) label uniformly chosen from 0 to 9.
Revised: Each edit consists of an image from the test set assigned with a random (likely incorrect) label uniformly chosen from 0 to 9.

Original: After N independent edits, we compute three following metrics over the entire test set:
Revised: After N independent edits, we compute three following metrics over the entire test set:

Original: • Drawdown — mean absolute difference of classiﬁcation error before and after performing an edit.
Revised: • Drawdown — mean absolute difference of classiﬁcation error before and after performing an edit.

Original: Smaller drawdown indicates better editor locality.
Revised: Smaller drawdown indicates better editor locality.

Original: • Success Rate — a rate of edits, for which editor succeeds in under k =10 gradient steps.
Revised: • Success Rate — a rate of edits, for which editor succeeds in under k =10 gradient steps.

Original: • Num Steps — an average number of gradient steps needed to perform a single edit.
Revised: • Num Steps — an average number of gradient steps needed to perform a single edit.

Original: [Table]
Revised: [Table]

Original: • Gradient Descent (GD) — standard gradient descent.
Revised: • Gradient Descent (GD) — standard gradient descent.

Original: • Scaled GD — like GD, but the learning rate is divided by the global gradient norm from the ﬁrst gradient step.
Revised: • Scaled GD — like GD, but the learning rate is divided by the global gradient norm from the ﬁrst gradient step.

Original: • RProp — like GD, but the algorithm only uses the sign of gradients: θ − α · sign ( ∇ θ l e ( θ )) .
Revised: • RProp — like GD, but the algorithm only uses the sign of gradients: θ − α · sign ( ∇ θ l e ( θ )) .

Original: • RMSProp — like GD, but the learning rate for each individual parameter is divided by √ rms t + (cid:15) where rms 

Revised: • RMSProp — like GD, but the learning rate for each individual parameter is divided by √ rms t + (cid:15) where rms

Original: =
Revised: =

Original: [ ∇ θ l e ( θ 0 )] 2 and rms t +1 = β · rms t + (1 − β ) · [ ∇ θ l e ( θ )] 2 . • Momentum GD — like GD, but the update follows the accumulated gradient direction ν : ν 0 = 0; ν t +

Revised: [ ∇ θ l e ( θ 0 )] 2 and rms t +1 = β · rms t + (1 − β ) · [ ∇ θ l e ( θ )] 2 . • Momentum GD — like GD, but the update follows the accumulated gradient direction ν : ν 0 = 0; ν t +

Original: = α · ∇ θ l e ( θ 0 ) + µ · ν t .
Revised: = α · ∇ θ l e ( θ 0 ) + µ · ν t .

Original: • Adam — adaptive momentum algorithm as described in Kingma & Ba (2014) with tunable α, β 1 , β 2 .
Revised: • Adam — adaptive momentum algorithm as described in Kingma & Ba (2014) with tunable α, β 1 , β 2 .

Original: To prevent Adam from replicating RMSProp, we restrict β 1 to [0 . 1 , 1 . 0] range.
Revised: To prevent Adam from replicating RMSProp, we restrict β 1 to [0 . 1 , 1 . 0] range.

Original: For each optimizer, we tune all hyperparameters (e.g. learning rate) to optimize locality while ensuring that editor succeeds in under k = 10 steps for at least 95% of edits.
Revised: For each optimizer, we tune all hyperparameters (e.g. learning rate) to optimize locality while ensuring that editor succeeds in under k = 10 steps for at least 95% of edits.

Original: We also tune the editor function by limiting the subset of parameters it is allowed to edit.
Revised: We also tune the editor function by limiting the subset of parameters it is allowed to edit.

Original: The ResNet-18 model consists of six parts: initial convolutional layer, followed by four ”chains” of residual blocks and a ﬁnal linear layer that predicts class logits.
Revised: The ResNet-18 model consists of six parts: initial convolutional layer, followed by four ”chains” of residual blocks and a ﬁnal linear layer that predicts class logits.

Original: We experimented with editing the whole model as well as editing each individual ”chain”, leaving parameters from other layers ﬁxed.
Revised: We experimented with editing the whole model as well as editing each individual ”chain”, leaving parameters from other layers ﬁxed.

Original: For each editor Tablereports the numbers, obtained for the subset of editable parameters, corresponding to the smallest drawdown.
Revised: For each editor Tablereports the numbers, obtained for the subset of editable parameters, corresponding to the smallest drawdown.

Original: For completeness, we also report the drawdown of Gradient Descent and RMSProp for different subsets of editable parameters in Table 2.
Revised: For completeness, we also report the drawdown of Gradient Descent and RMSProp for different subsets of editable parameters in Table 2.

Original: [Table]
Revised: [Table]

Original: different ResNet18 layers on CIFAR10.
Revised: different ResNet18 layers on CIFAR10.

Original: Table 1 and Table 2 demonstrate that the editor function locality is heavily affected by the choice of editing function even for models trained without Editable Training.
Revised: Table 1 and Table 2 demonstrate that the editor function locality is heavily affected by the choice of editing function even for models trained without Editable Training.

Original: Both RProp and RMSProp signiﬁcantly outperform the standard Gradient Descent while Momentum and Adam show smaller gains.
Revised: Both RProp and RMSProp signiﬁcantly outperform the standard Gradient Descent while Momentum and Adam show smaller gains.

Original: In fact, without the constraint β 1 > 0 . 1 the tuning procedure returns β 1 = 0 , which makes
Revised: In fact, without the constraint β 1 > 0 . 1 the tuning procedure returns β 1 = 0 , which makes

Original: Adam equivalent to RMSProp.
Revised: Adam equivalent to RMSProp.

Original: We attribute the poor performance of Adam and Momentum to the fact that most methods only make a few gradient steps till convergence and the momentum term cannot accumulate the necessary statistics.
Revised: We attribute the poor performance of Adam and Momentum to the fact that most methods only make a few gradient steps till convergence and the momentum term cannot accumulate the necessary statistics.

Original: Editable Training.
Revised: Editable Training.

Original: Finally, we report results obtained with Editable Training.
Revised: Finally, we report results obtained with Editable Training.

Original: On each training batch, we use a single constraint l e (ˆ θ ) =
Revised: On each training batch, we use a single constraint l e (ˆ θ ) =

Original: max y
Revised: max y

Original: i log p ( y i | x, ˆ θ ) − log p ( y ref | x, ˆ θ ) , where x is sampled from the train set and y ref is a random class label (from 0 to 9).
Revised: i log p ( y i | x, ˆ θ ) − log p ( y ref | x, ˆ θ ) , where x is sampled from the train set and y ref is a random class label (from 0 to 9).

Original: The model is then trained by directly minimizing objective (2) with k =10 editor steps and all other parameters optimized by backpropagation.
Revised: The model is then trained by directly minimizing objective (2) with k =10 editor steps and all other parameters optimized by backpropagation.

Original: We compare our Editable Training against three baselines, which also allow efﬁcient model correction.
Revised: We compare our Editable Training against three baselines, which also allow efﬁcient model correction.

Original: The ﬁrst natural baseline is Elastic Weight Consolidation (Kirkpatrick et al.
Revised: The ﬁrst natural baseline is Elastic Weight Consolidation (Kirkpatrick et al.

Original: (2016)): a technique that penalizes the edited model with the squared difference in parameter space, weighted by the importance of each parameter.
Revised: (2016)): a technique that penalizes the edited model with the squared difference in parameter space, weighted by the importance of each parameter.

Original: Our second baseline is a semi-parametric Deep k-Nearest
Revised: Our second baseline is a semi-parametric Deep k-Nearest

Original: Neighbors ( DkNN ) model (Papernot & McDaniel (2018)) that makes predictions by using k nearest neighbors in the space of embeddings, produced by different CNN layers.
Revised: Neighbors ( DkNN ) model (Papernot & McDaniel (2018)) that makes predictions by using k nearest neighbors in the space of embeddings, produced by different CNN layers.

Original: For this approach, we edit the model by ﬂipping labels of nearest neighbors until the model predicts the correct class.
Revised: For this approach, we edit the model by ﬂipping labels of nearest neighbors until the model predicts the correct class.

Original: Finally we compare to alternative editor function inspired by Conditional Neural Processes (CNP) (Garnelo et al.
Revised: Finally we compare to alternative editor function inspired by Conditional Neural Processes (CNP) (Garnelo et al.

Original: that we refer to as Editable+CNP .
Revised: that we refer to as Editable+CNP .

Original: For this baseline, we train a specialized
Revised: For this baseline, we train a specialized

Original: CNP model architecture that performs edits by adding a special condition vector to intermediate activations.
Revised: CNP model architecture that performs edits by adding a special condition vector to intermediate activations.

Original: This vector is generated by an additional ”encoder” layer.
Revised: This vector is generated by an additional ”encoder” layer.

Original: We train the CNP model to solve the original classiﬁcation problem when the condition vector is zero (hence, the model behaves as standard ResNet18) and minimize L edit and L loc when the condition vector is applied.
Revised: We train the CNP model to solve the original classiﬁcation problem when the condition vector is zero (hence, the model behaves as standard ResNet18) and minimize L edit and L loc when the condition vector is applied.

Original: After tuning the CNP architecture, we obtained the best performance when the condition vector is computed with a single ResNet block that receives the image representation via activations from the third residual chain of the main ResNet-18 model.
Revised: After tuning the CNP architecture, we obtained the best performance when the condition vector is computed with a single ResNet block that receives the image representation via activations from the third residual chain of the main ResNet-18 model.

Original: This ”encoder” also conditions on the target class y ref with an embedding layer (lookup table) that is added to the third chain activations.
Revised: This ”encoder” also conditions on the target class y ref with an embedding layer (lookup table) that is added to the third chain activations.

Original: The resulting procedure becomes the following: ﬁrst, apply encoder to the edited sample and compute the condition vector, then add this vector to the third layer chain activations for all subsequent inputs.
Revised: The resulting procedure becomes the following: ﬁrst, apply encoder to the edited sample and compute the condition vector, then add this vector to the third layer chain activations for all subsequent inputs.

Original: [Table]
Revised: [Table]

Original: Table 3 demonstrates two advantages of Editable Training.
Revised: Table 3 demonstrates two advantages of Editable Training.

Original: First, with c loc =0 .
Revised: First, with c loc =0 .

Original: 01 it is able to reduce drawdown (compared to models trained without Editable Training) while having no signiﬁcant effect on test error rate.
Revised: 01 it is able to reduce drawdown (compared to models trained without Editable Training) while having no signiﬁcant effect on test error rate.

Original: Second, editing Chain 3 alone is almost as effective as editing the whole model.
Revised: Second, editing Chain 3 alone is almost as effective as editing the whole model.

Original: This is important because it allows us to reduce training time, making Editable Training ≈ 2 .
Revised: This is important because it allows us to reduce training time, making Editable Training ≈ 2 .

Original: 5 times slower than baseline training.
Revised: 5 times slower than baseline training.

Original: Note, Editable+CNP turned out to be almost as effective as models trained with gradient-based editors while being simpler to implement.
Revised: Note, Editable+CNP turned out to be almost as effective as models trained with gradient-based editors while being simpler to implement.

Original: 4. A NALYZING E DITED M ODELS
Revised: 4. A NALYZING E DITED M ODELS

Original: In this section, we aim to interpret the differences between the models learned with and without
Revised: In this section, we aim to interpret the differences between the models learned with and without

Original: Editable Training.
Revised: Editable Training.

Original: First, we investigate which inputs are most affected when the model is edited on a sample that belongs to each particular class.
Revised: First, we investigate which inputs are most affected when the model is edited on a sample that belongs to each particular class.

Original: Based on Figure 2 (left), we conclude that edits of baseline model cause most drawdown on samples that belong to the same class as the edited input (prior to edit).
Revised: Based on Figure 2 (left), we conclude that edits of baseline model cause most drawdown on samples that belong to the same class as the edited input (prior to edit).

Original: However, this visualization loses information by reducing edits to their class labels.
Revised: However, this visualization loses information by reducing edits to their class labels.

Original: [Figure]
Revised: [Figure]

Original: In Figure 2 (middle) we apply t-SNE (van der Maaten & Hinton (2008)) to analyze the structure of the ”edit space”.
Revised: In Figure 2 (middle) we apply t-SNE (van der Maaten & Hinton (2008)) to analyze the structure of the ”edit space”.

Original: Intuitively, two edited versions of the same model are considered close if they make similar predictions.
Revised: Intuitively, two edited versions of the same model are considered close if they make similar predictions.

Original: We quantify this by computing KL-divergence between the model’s predictions before and after edit for each of 10 . 000 test samples.
Revised: We quantify this by computing KL-divergence between the model’s predictions before and after edit for each of 10 . 000 test samples.

Original: These KL divergences effectively form a 10 . 000 -dimensional model descriptor.
Revised: These KL divergences effectively form a 10 . 000 -dimensional model descriptor.

Original: We compute these descriptors for 4 .
Revised: We compute these descriptors for 4 .

Original: 500 edits applied to models trained with and without Editable Training.
Revised: 500 edits applied to models trained with and without Editable Training.

Original: These vectors are then embedded in twodimensional space with the t-SNE algorithm.
Revised: These vectors are then embedded in twodimensional space with the t-SNE algorithm.

Original: We plot the obtained charts on Figure 2 (middle), with point colors denoting original class labels of edited images.
Revised: We plot the obtained charts on Figure 2 (middle), with point colors denoting original class labels of edited images.

Original: As expected, the baseline edits for images of the same class are mapped to close points.
Revised: As expected, the baseline edits for images of the same class are mapped to close points.

Original: In turn, Editable Training does not always follow this pattern: the edit clusters are formed based on both original and target labels with a highly interlinked region in the middle.
Revised: In turn, Editable Training does not always follow this pattern: the edit clusters are formed based on both original and target labels with a highly interlinked region in the middle.

Original: Combined with the fact that Editable Training has a signiﬁcantly lower drawdown, this lets us hypothesize that with Editable
Revised: Combined with the fact that Editable Training has a signiﬁcantly lower drawdown, this lets us hypothesize that with Editable

Original: Training neural networks learn representations where edits affect objects of the same original class to a smaller extent.
Revised: Training neural networks learn representations where edits affect objects of the same original class to a smaller extent.

Original: Conversely, the t-SNE visualization lacks information about the true dimensionality of the data manifold.
Revised: Conversely, the t-SNE visualization lacks information about the true dimensionality of the data manifold.

Original: To capture this property, we also perform truncated SVD decomposition of the same matrix of descriptors.
Revised: To capture this property, we also perform truncated SVD decomposition of the same matrix of descriptors.

Original: Our main interest is the number of SVD components required to explain a given percentage of data variance.
Revised: Our main interest is the number of SVD components required to explain a given percentage of data variance.

Original: In Figure 2 (right) we report the explained variance ratio for models obtained with and without Editable Training.
Revised: In Figure 2 (right) we report the explained variance ratio for models obtained with and without Editable Training.

Original: These results present evidence that Editable
Revised: These results present evidence that Editable

Original: Training learns representations that exploit the neural network capacity to a greater extent.
Revised: Training learns representations that exploit the neural network capacity to a greater extent.

Original: 4. E DITABLE F INE -T UNING FOR LARGE SCALE IMAGE CLASSIFICATION
Revised: 4. E DITABLE F INE -T UNING FOR LARGE SCALE IMAGE CLASSIFICATION

Original: Section 4.1 demonstrates the success of Editable Training on the small CIFAR-10 dataset.
Revised: Section 4.1 demonstrates the success of Editable Training on the small CIFAR-10 dataset.

Original: However, many practical applications require training for many weeks on huge datasets.
Revised: However, many practical applications require training for many weeks on huge datasets.

Original: Re-training such model for the sake of better edits may be impractical.
Revised: Re-training such model for the sake of better edits may be impractical.

Original: In contrast, it would be more efﬁcient to start from a pre-trained model and ﬁne-tune it with Editable Training.
Revised: In contrast, it would be more efﬁcient to start from a pre-trained model and ﬁne-tune it with Editable Training.

Original: Here we experiment with the ILSVRC image classiﬁcation task (Deng et al.
Revised: Here we experiment with the ILSVRC image classiﬁcation task (Deng et al.

Original: and consider two pre-trained architectures: smaller ResNet-18 and deeper DenseNet-169 (Huang et al.
Revised: and consider two pre-trained architectures: smaller ResNet-18 and deeper DenseNet-169 (Huang et al.

Original: (2016)) networks.
Revised: (2016)) networks.

Original: For each architecture, we start with pre-trained model weights 2 and ﬁne-tune them on the same dataset with Editable Training.
Revised: For each architecture, we start with pre-trained model weights 2 and ﬁne-tune them on the same dataset with Editable Training.

Original: More speciﬁcally, we choose the training objective L base ( θ ) as KL-divergence between the predictions of the original network and its ﬁne-tuned counterpart.
Revised: More speciﬁcally, we choose the training objective L base ( θ ) as KL-divergence between the predictions of the original network and its ﬁne-tuned counterpart.

Original: Intuitively, this objective encourages the network to preserve its original classiﬁcation behavior, while being trained to allow local edits.
Revised: Intuitively, this objective encourages the network to preserve its original classiﬁcation behavior, while being trained to allow local edits.

Original: Similar to Section 4.1, the editor functions are only allowed to modify a subset of neural network layers.
Revised: Similar to Section 4.1, the editor functions are only allowed to modify a subset of neural network layers.

Original: We experiment with two choices of such subsets.
Revised: We experiment with two choices of such subsets.

Original: First, we try to edit a pre-existing layer in the network.
Revised: First, we try to edit a pre-existing layer in the network.

Original: Namely, we select the third out of four ”chains” in both architectures.
Revised: Namely, we select the third out of four ”chains” in both architectures.

Original: In the second experiment, we augment each architecture with an extra trainable layer after the last convolutional layer.
Revised: In the second experiment, we augment each architecture with an extra trainable layer after the last convolutional layer.

Original: We set an extra layer to be a residual block with a 4096-unit dense layer, followed by ELU activation (Clevert et al.
Revised: We set an extra layer to be a residual block with a 4096-unit dense layer, followed by ELU activation (Clevert et al.

Original: and another 1024-unit dense layer.
Revised: and another 1024-unit dense layer.

Original: The evaluation is performed on N =1000 edits with random target class.
Revised: The evaluation is performed on N =1000 edits with random target class.

Original: We measure the drawdown on the full ILSVRC validation set of 50 . 000 images.
Revised: We measure the drawdown on the full ILSVRC validation set of 50 . 000 images.

Original: We use the SGD optimizer with momentum µ =0 .
Revised: We use the SGD optimizer with momentum µ =0 .

Original: We set the learning rate to 10 − 5 for the pre-existing layers and 10 − 3 for the extra block.
Revised: We set the learning rate to 10 − 5 for the pre-existing layers and 10 − 3 for the extra block.

Original: The ImageNet training data is augmented with random resized crops and random horizontal ﬂips.
Revised: The ImageNet training data is augmented with random resized crops and random horizontal ﬂips.

Original: Our baselines for this task are the pre-trained architectures without Editable Fine-Tuning.
Revised: Our baselines for this task are the pre-trained architectures without Editable Fine-Tuning.

Original: However, during experiments, we noticed that minimizing the KL-divergence L ( θ ) has a side-effect of improving validation error.
Revised: However, during experiments, we noticed that minimizing the KL-divergence L ( θ ) has a side-effect of improving validation error.

Original: We attribute this improvement to the self-distillation phenomenon (Hinton et al.
Revised: We attribute this improvement to the self-distillation phenomenon (Hinton et al.

Original: ; Furlanello et al.
Revised: ; Furlanello et al.

Original: To disentangle these two effects, we consider an additional baseline where the model is trained to minimize the KL-divergence without Editable Training terms.
Revised: To disentangle these two effects, we consider an additional baseline where the model is trained to minimize the KL-divergence without Editable Training terms.

Original: For fair comparison, we also include baselines that edit an extra layer.
Revised: For fair comparison, we also include baselines that edit an extra layer.

Original: This layer is initialized at random for the pre-trained models and ﬁne-tuned for the models trained with distillation.
Revised: This layer is initialized at random for the pre-trained models and ﬁne-tuned for the models trained with distillation.

Original: [Table]
Revised: [Table]

Original: The results in Table 4 show that Editable Training can be effectively applied in the ﬁne-tuning scenario, achieving the best results with an extra trainable layer.
Revised: The results in Table 4 show that Editable Training can be effectively applied in the ﬁne-tuning scenario, achieving the best results with an extra trainable layer.

Original: In all cases Editable Fine-Tuning took under 48 hours on a single GeForce 1080 Ti GPU while a single edit requires less than 150 ms.
Revised: In all cases Editable Fine-Tuning took under 48 hours on a single GeForce 1080 Ti GPU while a single edit requires less than 150 ms.

Original: 4.3. R EALISTIC E DIT T ASKS WITH N ATURAL A DVERSARIAL E XAMPLES
Revised: 4.3. R EALISTIC E DIT T ASKS WITH N ATURAL A DVERSARIAL E XAMPLES

Original: In all previous experiments, we considered edits with randomly chosen target class.
Revised: In all previous experiments, we considered edits with randomly chosen target class.

Original: However, in many practical scenarios, most of these edits will never occur.
Revised: However, in many practical scenarios, most of these edits will never occur.

Original: For instance, it is far more likely that an image previously classiﬁed as ”plane” would require editing into ”bird” than into ”truck” or ”ship”.
Revised: For instance, it is far more likely that an image previously classiﬁed as ”plane” would require editing into ”bird” than into ”truck” or ”ship”.

Original: To address this consideration, we employ the Natural Adversarial Examples (NAE) data set by Hendrycks et al.
Revised: To address this consideration, we employ the Natural Adversarial Examples (NAE) data set by Hendrycks et al.

Original: This data set contains 7 . 500 natural images that are particularly hard to classify with neural networks.
Revised: This data set contains 7 . 500 natural images that are particularly hard to classify with neural networks.

Original: Without edits, a pre-trained model can correctly predict less than 1% of NAEs, but the correct answer is likely to be within top-100 classes ordered by predicted probabilities (see Figure 5 left).
Revised: Without edits, a pre-trained model can correctly predict less than 1% of NAEs, but the correct answer is likely to be within top-100 classes ordered by predicted probabilities (see Figure 5 left).

Original: The next set of experiments quantiﬁes Editable Training in this more realistic setting.
Revised: The next set of experiments quantiﬁes Editable Training in this more realistic setting.

Original: All models are evaluated on a sample of 1 . 000 edits, each corresponding to one Natural Adversarial Example and its reference class.
Revised: All models are evaluated on a sample of 1 . 000 edits, each corresponding to one Natural Adversarial Example and its reference class.

Original: We measure the drawdown from each edit on 50 . 000 ILSVRC test images.
Revised: We measure the drawdown from each edit on 50 . 000 ILSVRC test images.

Original: We evaluate best techniques from Section 4.3 and their modiﬁcations that account for NAEs:
Revised: We evaluate best techniques from Section 4.3 and their modiﬁcations that account for NAEs:

Original: • Editable Training: Random — model trained to edit on random targets from the uniform distribution, same as in Table
Revised: • Editable Training: Random — model trained to edit on random targets from the uniform distribution, same as in Table

Original: Compared to
Revised: Compared to

Original: the
Revised: the

Original: same
Revised: same

Original: pre trained and distilled baselines
Revised: pre trained and distilled baselines

Original: • Editable Training: Match Ranks — model trained to edit ImageNet training images with targets sampled based on their rank under NAE rank distribution (see 5, left).
Revised: • Editable Training: Match Ranks — model trained to edit ImageNet training images with targets sampled based on their rank under NAE rank distribution (see 5, left).

Original: • Editable Training: Train on NAE — model trained to edit 6 .
Revised: • Editable Training: Train on NAE — model trained to edit 6 .

Original: 500 natural adversarial examples.
Revised: 500 natural adversarial examples.

Original: These NAEs do not overlap with 1 . 

Revised: These NAEs do not overlap with 1 .

Original: NAE examples used for evaluation.
Revised: NAE examples used for evaluation.

Original: The results in Table 5 (top-left) show that Editable Training signiﬁcantly reduces drawdown for
Revised: The results in Table 5 (top-left) show that Editable Training signiﬁcantly reduces drawdown for

Original: NAEs even when trained with random targets.
Revised: NAEs even when trained with random targets.

Original: However, accounting for the distribution of target classes improves locality even further.
Revised: However, accounting for the distribution of target classes improves locality even further.

Original: Surprisingly enough, training on 6 .
Revised: Surprisingly enough, training on 6 .

Original: 500 actual NAEs fares no better than simply matching the distribution of target ranks.
Revised: 500 actual NAEs fares no better than simply matching the distribution of target ranks.

Original: For the ﬁnal set of evaluations, we consider two realistic scenarios where Editable Training can be applied.
Revised: For the ﬁnal set of evaluations, we consider two realistic scenarios that are not covered by our previous experiments.

Original: The ﬁrst experiment evaluates the ability of Editable Training to generalize to substantially similar inputs.
Revised: First, we evaluate whether edits performed by our method generalize to substantially similar inputs.

Original: This behavior is highly desirable since we want to avoid  edited model repeating old mistakes on slightly perturbed inputs.
Revised: This behavior is highly desirable since we want to avoid the edited model repeating old mistakes in a slightly changed context.

Original: For each of 1 , 000test NAEs, we use ﬁve random image augmentations for editing and then evaluate accuracy on 100 augmentations of the same image.
Revised: For each of 1 , 000 NAEs, we ﬁnd the most similar image from test set based on InceptionV3 embeddings (Szegedy et al., 2015). For each [Table]

Original: [Table]
Revised: [Table]

Revised: such pair

Original: [Table]
Revised: [Table]

Original: similar to previous ILSVRC experiments. A model trained with random edits was able to correctly classify 95.2% and the ”Editable + Match Ranks” model scored 96.0%. This allow us to conclude that Editable Training does generalize to substantially similar images.
Revised: we edit 5 augmentations of the ﬁrst image and measure how often the model predicts the edited class on 5 augmentations of the second image. A model trained with random edits has an accuracy of 86.7% while ”Editable + Match Ranks” scores 85.9% accuracy.

Original: Finally
Revised: Finally

Original: [Table]
Revised: [Table]

Original: we evaluate if our technique can perform multiple edits in a sequence.
Revised: we evaluate if our technique can perform multiple edits in a sequence.

Original: Figure 5 (bottom-left) demonstrates that our approach can cope with sequential edits without ever being trained that way.
Revised: Figure 5 (bottom-left) demonstrates that our approach can cope with sequential edits without ever being trained that way.

Original: 4. E DITABLE T RAINING FOR M ACHINE T RANSLATION
Revised: 4. E DITABLE T RAINING FOR M ACHINE T RANSLATION

Original: The previous experiments focused on multi-class classiﬁcation problems.
Revised: The previous experiments focused on multi-class classiﬁcation problems.

Original: However, Editable Training can be applied to any task where the model is trained by minimizing a differentiable objective.
Revised: However, Editable Training can be applied to any task where the model is trained by minimizing a differentiable objective.

Original: Our ﬁnal set of experiments demonstrates the applicability of Editable Training for machine translation.
Revised: Our ﬁnal set of experiments demonstrates the applicability of Editable Training for machine translation.

Original: We consider the IWSLT 2014 German-English translation task with the standard training/test splits (Cettolo et al.
Revised: We consider the IWSLT 2014 German-English translation task with the standard training/test splits (Cettolo et al.

Original: The data is preprocessed with Moses Tokenizer (Koehn et al.
Revised: The data is preprocessed with Moses Tokenizer (Koehn et al.

Original: and converted to lowercase.
Revised: and converted to lowercase.

Original: We further apply the Byte-Pair Encoding with 10 . 000 BPE rules learned jointly from German and English training data.
Revised: We further apply the Byte-Pair Encoding with 10 . 000 BPE rules learned jointly from German and English training data.

Original: Finally, we train the Transformer (Vaswani et al.
Revised: Finally, we train the Transformer (Vaswani et al.

Original: (2017)) model similar to transformer-base conﬁguration, optimized for IWSLT De-En task 3 .
Revised: (2017)) model similar to transformer-base conﬁguration, optimized for IWSLT De-En task 3 .

Original: Typical machine translation models use beam search to ﬁnd the most likely translation.
Revised: Typical machine translation models use beam search to ﬁnd the most likely translation.

Original: Hence we consider an edit to be successful if and only if the log-probability of target translation is greater than log-probability of any alternative translation.
Revised: Hence we consider an edit to be successful if and only if the log-probability of target translation is greater than log-probability of any alternative translation.

Original: So, l e (ˆ θ ) =
Revised: So, l e (ˆ θ ) =

Original: max y
Revised: max y

Original: i log p ( y i | s, ˆ θ ) − log p ( y 0 | s, ˆ θ ) , where s is a source sentence, y 0 denotes target translation and { y i } ki =1 are alternative translations.
Revised: i log p ( y i | s, ˆ θ ) − log p ( y 0 | s, ˆ θ ) , where s is a source sentence, y 0 denotes target translation and { y i } ki =1 are alternative translations.

Original: During training, we approximate this by ﬁnding k =32 most likely translations with beam search using the Transformer model trained normally on the same data.
Revised: During training, we approximate this by ﬁnding k =32 most likely translations with beam search using the Transformer model trained normally on the same data.

Original: The edit targets are sampled from the same model by sampling with temperature τ =1 . 2 .
Revised: The edit targets are sampled from the same model by sampling with temperature τ =1 . 2 .

Original: The resulting edit consists of three parts: a source sentence, a target translation and a set of alternative translations.
Revised: The resulting edit consists of three parts: a source sentence, a target translation and a set of alternative translations.

Original: We deﬁne L loc as KL-divergence between the predictions of the original and edited model averaged over target tokens, L loc =
Revised: We deﬁne L loc as KL-divergence between the predictions of the original and edited model averaged over target tokens, L loc =

Original: E x,y ∈ D 1 | y | (cid:80) t D KL ( p ( y t | x, y 0:
Revised: E x,y ∈ D 1 | y | (cid:80) t D KL ( p ( y t | x, y 0:

Original: t , θ ) || p
Revised: t , θ ) || p

Original: ( y t | x, y 0:
Revised: ( y t | x, y 0:

Original: t Edit kα ( θ, l e ))) , where D is a data batch, x and y are the source and translation phrases respectively, y 0: t denotes a translation preﬁx.
Revised: t Edit kα ( θ, l e ))) , where D is a data batch, x and y are the source and translation phrases respectively, y 0: t denotes a translation preﬁx.

Original: The Edit function optimizes the ﬁnal decoder layer using RMSProp with hyperparameters tuned as in Section 4.1.
Revised: The Edit function optimizes the ﬁnal decoder layer using RMSProp with hyperparameters tuned as in Section 4.1.

Original: The results in Table 6 show that Editable Training produces a model that matches the baseline translation quality but has less than half of its drawdown.
Revised: The results in Table 6 show that Editable Training produces a model that matches the baseline translation quality but has less than half of its drawdown.

Original: Training Procedure
Revised: Training Procedure

Original: [Table]
Revised: [Table]

Original: Baseline

Original: [Table]
Revised: [Table]

Original: [Table]

Original: =
Revised: loc

Original: [Table]
Revised: [Table]

Original: −

Original: [Table] [Table] [Table] [Table]
Revised: Editable,

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper we have addressed the efﬁcient correction of neural network mistakes, a highly important task for deep learning practitioners.
Revised: In this paper we have addressed the efﬁcient correction of neural network mistakes, a highly important task for deep learning practitioners.

Original: We have proposed several evaluation measures for comparison of different means of model correction.
Revised: We have proposed several evaluation measures for comparison of different means of model correction.

Original: Then we have introduced Editable Training, a training procedure that produces models that allow gradient-based editing to address corrections of the model behaviour.
Revised: Then we have introduced Editable Training, a training procedure that produces models that allow gradient-based editing to address corrections of the model behaviour.

Original: We demonstrate the advantage of Editable Training against reasonable baselines on large-scale image classiﬁcation and machine translation tasks
Revised: We demonstrate the advantage of Editable Training against reasonable baselines on large-scale image classiﬁcation and machine translation tasks