Original: Fixing Neural Networks by Leaving the Right Past Behind
Revised: Fixing Neural Networks by Leaving the Right Past Behind

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Prediction failures of machine learning models often arise from deficiencies in trainingdata, such as incorrect labels, outliers, and selection biases.
Revised: Prediction failures of machine learning models often arise from deficiencies in trainingdata, such as incorrect labels, outliers, and selection biases.

Original: However, such data pointsthat are responsible for a given failure mode are generally not known a priori, let alone amechanism for repairing the failure.
Revised: However, such data pointsthat are responsible for a given failure mode are generally not known a priori, let alone amechanism for repairing the failure.

Original: This work draws on the Bayesian view of continuallearning, and develops a generic framework for both, identifying training exampleswhich have given rise to the target failure, and fixing the model through erasinginformation about them.
Revised: This work draws on the Bayesian view of continuallearning, and develops a generic framework for both, identifying training exampleswhich have given rise to the target failure, and fixing the model through erasinginformation about them.

Original: This framework naturally allows leveraging recent advancesin continual learning to this new problem of model repairment, while subsumingthe existing works on influence functions and data deletion as specific instances.
Revised: This framework naturally allows leveraging recent advancesin continual learning to this new problem of model repairment, while subsumingthe existing works on influence functions and data deletion as specific instances.

Original: Experimentally, the proposed approach outperforms the baselines for both identificationof detrimental training data and fixing model failures in a generalisable manner.
Revised: Experimentally, the proposed approach outperforms the baselines for both identificationof detrimental training data and fixing model failures in a generalisable manner.

Original:  Introduction
Revised:  Introduction

Original: Machine learning (ML) models often exhibit unexpected failures once deployed in the “wild”.
Revised: Machine learning (ML) models often exhibit unexpected failures once deployed in the “wild”.

Original: Recentlines of research aim to alleviate different well-known shortcomings in supervised models, such asvulnerability to annotation errors [1] and adversarial attacks [2], sensitivity to data shifts [3], and biases tounderrepresented subgroups [4, 5, 6].
Revised: Recentlines of research aim to alleviate different well-known shortcomings in supervised models, such asvulnerability to annotation errors [1] and adversarial attacks [2], sensitivity to data shifts [3], and biases tounderrepresented subgroups [4, 5, 6].

Original: However, it is often challenging to anticipate beforehand all plausiblefailure scenarios, and protect against them pre-emptively.
Revised: However, it is often challenging to anticipate beforehand all plausiblefailure scenarios, and protect against them pre-emptively.

Original: This motivates developing a technique that isable to repair a model on demand , as new failure cases arise in practice.
Revised: This motivates developing a technique that isable to repair a model on demand , as new failure cases arise in practice.

Original: Undesirable behaviours of ML models commonly stem from defects in the training data.
Revised: Undesirable behaviours of ML models commonly stem from defects in the training data.

Original: However, it is unclear how to detect the causes of such failures automatically, rendering a manual troubleshooting necessary.
Revised: However, it is unclear how to detect the causes of such failures automatically, rendering a manual troubleshooting necessary.

Original: Furthermore, once the problems are uncovered, one would still need to design fixes, which typically involvefurther data curation/collection, and model retraining/redesigning from scratch.
Revised: Furthermore, once the problems are uncovered, one would still need to design fixes, which typically involvefurther data curation/collection, and model retraining/redesigning from scratch.

Original: Executing the above stepsdemand not only time, but also mature expertise in the relevant ML areas, a scarcity in the present job market.
Revised: Executing the above stepsdemand not only time, but also mature expertise in the relevant ML areas, a scarcity in the present job market.

Original: This work introduces an approach to identifying a set of most detrimental training examples that have causedfailure cases observed at test time, and to subsequently repairing the model on these failures by deletingthose culprits.
Revised: This work introduces an approach to identifying a set of most detrimental training examples that have causedfailure cases observed at test time, and to subsequently repairing the model on these failures by deletingthose culprits.

Original: At the basis of both cause identification and repairment steps is the approximation of “counterfactual” posterior distribution where some training examples are assumed absent.
Revised: At the basis of both cause identification and repairment steps is the approximation of “counterfactual” posterior distribution where some training examples are assumed absent.

Original: We formalise this as a
Revised: We formalise this as a

Original: Bayesian continual (un)learning problem [7], where the above counterfactual posterior is estimated by deleting the evidence of selected training data from the current posterior.
Revised: Bayesian continual (un)learning problem [7], where the above counterfactual posterior is estimated by deleting the evidence of selected training data from the current posterior.

Original: We note that, while other factors (e.g.model class and optimisation) may play a role, we focus on “data debugging” and investigate, to what extent,prediction failures could be remedied by only intervening on data and updating the model accordingly.[Figure]
Revised: We note that, while other factors (e.g.model class and optimisation) may play a role, we focus on “data debugging” and investigate, to what extent,prediction failures could be remedied by only intervening on data and updating the model accordingly.

Original: Fig.
Revised: Fig.

Original: 1 gives an overview of the proposed approach for model repairment, which operates in two stepsby 1) identifying causes of failure among training data, and 2) updating the model by erasing the“memories” of those harmful examples.
Revised: 1 gives an overview of the proposed approach for model repairment, which operates in two stepsby 1) identifying causes of failure among training data, and 2) updating the model by erasing the“memories” of those harmful examples.

Original: Importantly, the proposed framework is agnostic to why a particulardatapoint hurts model performance, handling both, issues in the input data and/or labels.
Revised: Importantly, the proposed framework is agnostic to why a particulardatapoint hurts model performance, handling both, issues in the input data and/or labels.

Original: We only requirespecification of the set of failed cases for which we wish to improve performance.
Revised: We only requirespecification of the set of failed cases for which we wish to improve performance.

Original: Our contributions: We develop a framework for repairing machine learning models by erasing memoriesof detrimental datapoints.
Revised: Our contributions: We develop a framework for repairing machine learning models by erasing memoriesof detrimental datapoints.

Original: The framework connects both identification and removal of detrimental data undera (Bayesian) continual learning perspective, which brings forth practical benefits.
Revised: The framework connects both identification and removal of detrimental data undera (Bayesian) continual learning perspective, which brings forth practical benefits.

Original: Firstly, the frameworksubsumes works on influence function [8] and data deletion [9] as specific examples, which are developedindependently, and our work reveals their close connections and limitations.
Revised: Firstly, the frameworksubsumes works on influence function [8] and data deletion [9] as specific examples, which are developedindependently, and our work reveals their close connections and limitations.

Original: Secondly, the generality of ourformulation allows translating any continual learning method into this model repairment setting, and opensdoors to further research.
Revised: Secondly, the generality of our [Figure] formulation allows translating any continual learning method into this model repairment setting, and opens doors to further research.

Original: In particular, we extend Elastic Weight Consolidation [10] – a specific continuallearning algorithm – to cause identification and data removal, and demonstrate improvements over the priorworks in a variety of settings where training data are contaminated with annotation and/or input noise.
Revised: In particular, we extend Elastic Weight Consolidation [10] – a specific continuallearning algorithm – to cause identification and data removal, and demonstrate improvements over the priorworks in a variety of settings where training data are contaminated with annotation and/or input noise.

Original:  Model Repairment by Data Deletion
Revised:  Model Repairment by Data Deletion

Original: Let us consider a prediction model p ( y | θ , x ) that returns a probability distribution of the output y givenan input x .
Revised: Let us consider a prediction model p ( y | θ , x ) that returns a probability distribution of the output y givenan input x .

Original: We make the i.i.d.
Revised: We make the i.i.d.

Original: modelling assumption and denote p ( θ |D ) as the posterior distribution overthe model parameters θ given training data D
Revised: modelling assumption and denote p ( θ |D ) as the posterior distribution overthe model parameters θ given training data D

Original: = { ( x ( n ) ,y ( n ) ) }
Revised: = { ( x ( n ) ,y ( n ) ) }

Original: Nn =1 .
Revised: Nn =1 .

Original: At test time, the posterior predictivedistribution is used to infer label y ⋆ given a new sample x
Revised: At test time, the posterior predictivedistribution is used to infer label y ⋆ given a new sample x

Original: ⋆ :
Revised: ⋆ :

Original: [Equation]
Revised: [Equation]

Original:  where p ( y ⋆ | x ⋆ , θ ) is the likelihood term for sample ( x ⋆ ,y ⋆ ) .
Revised: (1) where p ( y ⋆ | x ⋆ , θ ) is the likelihood term for sample ( x ⋆ ,y ⋆ ) .

Original: While we use approximate posteriors inpractice, we focus on the exact case for now to formalise the problem.
Revised: While we use approximate posteriors inpractice, we focus on the exact case for now to formalise the problem.

Original: Imagine that, in deployment, this model makes incorrect predictions in certain situations.
Revised: Imagine that, in deployment, this model makes incorrect predictions in certain situations.

Original: After collecting aset of examples from such failure mode in the test set 1 , F = { ( x ( n ) f ,y ( n ) f ) } N f n =1 , we would like to repair themodel, such that it improves performance on the failure set F and similar future cases.
Revised: After collecting aset of examples from such failure mode in the test set 1 , F = { ( x ( n ) f ,y ( n ) f ) } N f n =1 , we would like to repair themodel, such that it improves performance on the failure set F and similar future cases.

Original: We also argue thata successful repairment should also maintain a similar level of performance on the rest of test examples.
Revised: We also argue thata successful repairment should also maintain a similar level of performance on the rest of test examples.

Original: These two objectives of model repairment are analogous to those of a medical treatment , in that both aim tofix a specific problem while leaving the “healthy” part as intact as possible.
Revised: These two objectives of model repairment are analogous to those of a medical treatment , in that both aim tofix a specific problem while leaving the “healthy” part as intact as possible.

Original: A further discussion of differentaspects, including generalisation, efficiency and specificity properties is provided in Appendix A.1.
Revised: A further discussion of differentaspects, including generalisation, efficiency and specificity properties is provided in Appendix A.1.

Original: In this work, we assume that the main reason for such failures F is due to the existence of detrimentalexamples in the training data D , for example, noisy labels, low-quality inputs and/or group imbalances.
Revised: In this work, we assume that the main reason for such failures F is due to the existence of detrimentalexamples in the training data D , for example, noisy labels, low-quality inputs and/or group imbalances.

Original: Our hypothesis is that, by removing these harmful datapoints and adapting the model accordingly, themodel can be repaired to return correct predictions for datapoints in F as well as similar future cases.
Revised: Our hypothesis is that, by removing these harmful datapoints and adapting the model accordingly, themodel can be repaired to return correct predictions for datapoints in F as well as similar future cases.

Original: We acknowledge that there might be other reasons for a model to make wrong predictions on F , suchas optimisation and model biases, which are outside the scope of this work.
Revised: We acknowledge that there might be other reasons for a model to make wrong predictions on F , suchas optimisation and model biases, which are outside the scope of this work.

Original: Addressing data-based failuresis complementary to accounting for other problems and is of relevance regardless, as datasets typicallycome with unexpected issues no matter how much we curate them beforehand.
Revised: Addressing data-based failuresis complementary to accounting for other problems and is of relevance regardless, as datasets typicallycome with unexpected issues no matter how much we curate them beforehand.

Original: We formulate the process of model repairment as the following two steps (see Fig.
Revised: We formulate the process of model repairment as the following two steps (see Fig.

Original: 1 for an illustration):
Revised: 1 for an illustration):

Original: Cause identification: Identify a set of detrimental datapoints C in the training data D thatcontributed the most to the failure set F .
Revised: Cause identification: Identify a set of detrimental datapoints C in the training data D that contributed the most to the failure set F .

Original: Treatment:
Revised: Treatment:

Original: Given the set of failure causes C , adapt the model to predict correctly on the failureset F , while maintaining performance on remaining test examples.
Revised: Given the set of failure causes C , adapt the model to predict correctly on the failureset F , while maintaining performance on remaining test examples.

Original: Below we describe a formal framework for performing these two steps.
Revised: Below we describe a formal framework for performing these two steps.

Original: In the first phase of cause identification, we need to define a measure of how much a subset of training examples
Revised: In the first phase of cause identification, we need to define a measure of how much a subset of training examples

Original: C
Revised: C

Original: = { ( x ( n ) c ,y ( n ) c ) } N c n =1 ⊂Dis responsible for the failure cases F .
Revised: = { ( x ( n ) c ,y ( n ) c ) } N c n =1 ⊂Dis responsible for the failure cases F .

Original: To this end, we propose to check how much the posterior predictivedistribution on F changes as a result of deleting C from the training data:
Revised: To this end, we propose to check how much the posterior predictivedistribution on F changes as a result of deleting C from the training data:

Original: [Equation]
Revised: [Equation]

Original: where p ( F|D )2 and p ( F|D\C ) are the posterior predictive distributions before and after removing a subsetof training examples C .
Revised: where p ( F|D ) and p ( F|D\C ) are the posterior predictive distributions before and after removing a subsetof training examples C .

Original: If a given C leads to a large value of r ( C ) in Eq.
Revised: If a given C leads to a large value of r ( C ) in Eq.

Original: 2, it would mean that removing
Revised: 2, it would mean that removing

Original: C from D would have improved the performance of the Bayesian predictive inference on the failure set
Revised: C from D would have improved the performance of the Bayesian predictive inference on the failure set

Original: F .
Revised: F .

Original: The first causal identification step thus entails finding a subset C with the maximal log-density ratior ( C ) .
Revised: The first causal identification step thus entails finding a subset C with the maximal log-density ratior ( C ) .

Original: In the second treatment step, we can directly adopt p ( F|D\C ) as the updated predictive distributionthat needs to be inferred, as it confers the largest improvement over the failure cases.
Revised: In the second treatment step, we can directly adopt p ( F|D\C ) as the updated predictive distributionthat needs to be inferred, as it confers the largest improvement over the failure cases.

Original: In other words, forboth the search of detrimental datapoints and repairment of the model, we need to compute p ( F|D\C ) .
Revised: In other words, forboth the search of detrimental datapoints and repairment of the model, we need to compute p ( F|D\C ) .

Original: The central computational question is, therefore, concerned with the efficient calculation of p ( F|D\C )without retraining the model from scratch .
Revised: The central computational question is, therefore, concerned with the efficient calculation of p ( F|D\C )without retraining the model from scratch .

Original: The reasons for avoiding retraining are: (1) in the cause identification step, it is computationally prohibitive as retraining needs to be done for every subset C⊂D ; and (2)in the treatment step, the resulting retrained model may be drastically different from the original model onewould like to fix (due to, e.g. noises in the SGD-based optimisation and parameter re-initialisation), andmay lose other desirable properties that one may wish to retain.
Revised: The reasons for avoiding retraining are: (1) in the cause identification step, it is computationally prohibitive as retraining needs to be done for every subset C⊂D ; and (2)in the treatment step, the resulting retrained model may be drastically different from the original model onewould like to fix (due to, e.g. noises in the SGD-based optimisation and parameter re-initialisation), andmay lose other desirable properties that one may wish to retain.

Original: In this work, we thus argue for continuallearning approaches [11] to approximate p ( F|D\C ) , given their computational efficiency and their naturalability to maintain model behaviour.
Revised: In this work, we thus argue for continuallearning approaches [11] to approximate p ( F|D\C ) , given their computational efficiency and their naturalability to maintain model behaviour.

Original: We now elaborate on the mathematical details of the two steps.[Table] [Equation] Eq
Revised: We now elaborate on the mathematical details of the two steps.

Original: [Equation]

Original: Find failure causes C : Return the examples with positive influence, C←{ z ∈D :

Original: ˜ r ( z ) > 0 } # Step II: Treatment Delete information of C : Apply a continual (un)learning method to the original posterior q ( θ ) , and obtain the posterior on the corrected data q ⋆⋆ D\C ( θ )

Original: ≈ p ( θ |D\C )

Original: 2.1 Step I: Cause Identification
Revised: 2.1 Step I: Cause Identification

Original: Identifying the set of detrimental examples C requires solving the following optimisation problem
Revised: Identifying the set of detrimental examples C requires solving the following optimisation problem

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: ) where P ( D ) denotes the power set of D .
Revised: where P ( D ) denotes the power set of D .

Original: Solving this comes with multiple computational challenges.
Revised: Solving this comes with multiple computational challenges.

Original: Firstly, a naive approach would require computing the predictive distribution p ( F|D\C ) — and thus, theposterior p ( θ |D\C ) — for every subset C of D , which is prohibitively expensive.
Revised: Firstly, a naive approach would require computing the predictive distribution p ( F|D\C ) — and thus, theposterior p ( θ |D\C ) — for every subset C of D , which is prohibitively expensive.

Original: In the following, wepresent a “predictive” approach that removes this computational burden.
Revised: In the following, wepresent a “predictive” approach that removes this computational burden.

Original: The key idea is to notice thatp ( D\C| θ )
Revised: The key idea is to notice thatp ( D\C| θ )

Original: =
Revised: =

Original: p ( D| θ ) /p
Revised: p ( D| θ ) /p

Original: ( C| θ ) for any C⊂D due to the i.i.d.
Revised: ( C| θ ) for any C⊂D due to the i.i.d.

Original: modelling assumption.
Revised: modelling assumption.

Original: Inserting this into the
Revised: Inserting this into the

Original: Bayes’ rule for computing p ( θ |D\C ) yields:
Revised: Bayes’ rule for computing p ( θ |D\C ) yields:

Original: [Equation]
Revised: [Equation]

Original: Notice that p ( D\C ) /p
Revised: Notice that p ( D\C ) /p

Original: ( D )=
Revised: ( D )=

Original: (cid:82) [Equation] C
Revised: (cid:82) 1 p ( C| θ

Original: [Equation]
Revised: [Equation]

Original:  D| [Equation] D ) d θ again due to the i.i.d.
Revised: p ( D| θ ) p ( θ ) p ( D ) d θ again due to the i.i.d.

Original: assumption.
Revised: assumption.

Original: Therefore, we cancompute the log density ratio r ( C )=
Revised: Therefore, we cancompute the log density ratio r ( C )=

Original: log p ( F|D\C ) − log p ( F|D ) as follows without computing p ( θ |D\C ) :
Revised: log p ( F|D\C ) − log p ( F|D ) as follows without computing p ( θ |D\C ) :

Original: [Equation]
Revised: [Equation]

Original: In this form, we would only need to compute the posterior p ( θ |D , F ) once, and can side-step therequirement of computing p ( θ |D\C ) for inspection of every new C ⊂ D , thereby removing one of thekey computational bottlenecks.
Revised: In this form, we would only need to compute the posterior p ( θ |D , F ) once, and can side-step therequirement of computing p ( θ |D\C ) for inspection of every new C ⊂ D , thereby removing one of thekey computational bottlenecks.

Original: The detailed derivation of Eq.
Revised: The detailed derivation of Eq.

Original: (4) is provided in Sec.
Revised: (4) is provided in Sec.

Original: A.2 in the appendix.
Revised: A.2 in the appendix.

Original: Secondly, we are still left with the combinatorial search for the best subset C , which is also prohibitivewhen the size of training data D is large.
Revised: Secondly, we are still left with the combinatorial search for the best subset C , which is also prohibitivewhen the size of training data D is large.

Original: We address this issue by a first-order Taylor series approximationof r ( C )  .
Revised: We address this issue by a first-order Taylor series approximationof r ( C ) that factorises over the samples in C .

Original: Let us re-write the log density ratio as
Revised: Let us re-write the log density ratio as

Original: [Equation]
Revised: [Equation]

Original: Note that F (0 ,g ( θ ))
Revised: Note that F (0 ,g ( θ ))

Original: [-=
-]{+= 
+}
Revised: [-=
-]{+= 
+}

Original: for any well-defined distribution g ( θ ) .
Revised: for any well-defined distribution g ( θ ) .

Original: We now perform a Taylor expansion of F ( ϵ,g ( θ )) around ϵ = 0 : F ( ϵ,g ( θ )) =
Revised: Assuming g ( θ ) is a distribution, weperform a Taylor expansion of F ( ϵ,g ( θ )) around ϵ =0 : F ( ϵ,g ( θ ))=

Original: − ϵ E g ( θ ) [log p ( C| θ )]
Revised: − ϵ E g ( θ ) [log p ( C| θ )]

Original: + O ( ϵ 2 ) .
Revised: + O ( ϵ 2 ) .

Original: Finally we obtain an approximatelog-density ratio ˆ r ( C ) 
Revised: Thisresults in an approximate log density ratio ˆ r ( C ) ≈

Original: ≈ r ( C ) by plugging thefirst term of the Taylor expansion intothe RHS of Eq.
Revised: r ( C ) by plugging the Taylor expansion into Eq.

Original: [Equation]
Revised: [Equation]

Original: Assuming that data are i.i.d., and defining z =( x ,y ) , the above approximation can be expressed as thesum of individual log density ratios, ˆ r ( C )=
Revised: Assuming that data are i.i.d., and defining z =( x ,y ) , the above approximation can be expressed as thesum of individual log density ratios, ˆ r ( C )=

Original: (cid:80) z ∈C ˆ r ( z ) , where each term is given by
Revised: (cid:80) z ∈C ˆ r ( z ) , where each term is given by

Original: [Equation]
Revised: [Equation]

Original: With this approximation, in order to find a subset C that leads to the maximal ˆ r ( C ) , it suffices to computeˆ r ( z ) for every training example z ∈ D and find the top K examples with largest ˆ r ( z ) values, therebyreducing the search space from O ( |D| !)
Revised: With this approximation, in order to find a subset C that leads to the maximal ˆ r ( C ) , it suffices to computeˆ r ( z ) for every training example z ∈ D and find the top K examples with largest ˆ r ( z ) values, therebyreducing the search space from O ( |D| !)

Original: choices to only O ( |D| ) choices.
Revised: choices to only O ( |D| ) choices.

Revised: Algorithm

Original:  In practice, thefailure causes  correspond to examples z with ˆ r ( z ) > 0 .
Revised: [Table] In practice, the causes of failure will correspond to examples z with ˆ r ( z ) > 0 .

Original: Intuitively, ˆ r ( z ) measures howmuch the  predictive moments of z changes when the model is further trained on F , i.e., computation ofp ( θ |D , F ) .
Revised: Intuitively, ˆ r ( z ) measureshow much the posterior predictive moments of z changes when the model is further trained on F , i.e.,computation of p ( θ |D , F ) .

Original: If the  difference is positive, i.e., ˆ r ( z ) > 0 , it means  the example z ∈D is a conflicting evidenceagainst the test examples in F ; conversely, if ˆ r ( z ) < 0 , then z and F are aligned.
Revised: If the moment difference is positive, i.e., ˆ r ( z ) > 0 , it means that the examplez ∈D is a conflicting evidence against the test examples in F ; conversely, if ˆ r ( z ) < 0 , then z and F arealigned.

Revised: See Algorithm 1 for specification of the whole procedure.

Original: Lastly, for non-linear models (e.g. neural networks), approximate posteriors q ( θ ) ≈ p
Revised: Lastly, for non-linear models (e.g. neural networks), approximate posteriors q ( θ ) ≈ p

Original: ( θ | D ) andq ⋆ D , F ( θ ) ≈ p ( θ |D , F ) are needed due to intractability of the exact posteriors.
Revised: ( θ | D ) andq ⋆ D , F ( θ ) ≈ p ( θ |D , F ) are needed due to intractability of the exact posteriors.

Original: We assume that q ( θ ) isavailable after training, and suffers from the prediction failures F .
Revised: We assume that q ( θ ) isavailable after training, and suffers from the prediction failures F .

Original: As recomputing the posterior q ⋆ D , F ( θ )from scratch can be expensive, we propose to use a continual learning technique [11] and obtain thisquantity by updating the original posterior q ( θ ) .
Revised: As recomputing the posterior q ⋆ D , F ( θ )from scratch can be expensive, we propose to use a continual learning technique [11] and obtain thisquantity by updating the original posterior q ( θ ) .

Original: Finally, we use the following metric ˜ r ( z ) in practiceto calculate the detrimental impact of each training datapoint on the failure set, F :
Revised: Finally, we use the following metric ˜ r ( z ) in practiceto calculate the detrimental impact of each training datapoint on the failure set, F :

Original: [Equation]
Revised: [Equation]

Original: and approximate the failure causes C .
Revised: and approximate the failure causes C .

Original: This metric is generic, and its implementation depends on thespecifics in which both q ( θ ) and q ⋆ D , F ( θ ) are computed, e.g. MLE/MAP point estimates, Laplaceapproximation, variational inference, etc.
Revised: This metric is generic, and the detail implementation depends onthe specifics in which both q ( θ ) and q ⋆ D , F ( θ ) are computed, e.g. MLE/MAP point estimates, Laplaceapproximation, variational inference, etc.

Original: We end with two concrete examples: the first shows that thewell-known linear influence function [8] is a specific instance of Eq.
Revised: We end by providing two concrete examples: the first showsthat the well-known linear influence function [8] is a specific instance of Eq.

Original: (7); and the second is derived byextending a continual learning method, known as Elastic Weight Consolidation (EWC) [10] to causeidentification, and is a key methodological development in our work.
Revised: (7); and the second is derivedby extending a continual learning method, known as Elastic Weight Consolidation (EWC) [10] to causeidentification, and is a key methodological development in our work.

Original: Example 1 (Linear Influence Function): Our proposed metric in Eq.
Revised: Example 1 (Linear Influence Function): Our proposed metric in Eq.

Original: (7) recovers the linear influencefunction from Koh & Liang [8] when point estimates are used for θ .
Revised: (7) recovers the linear influencefunction from Koh & Liang [8] when point estimates are used for θ .

Original: Assume that the model is trainedon data D with parameters ˆ θ , which corresponds to an approximation of MLE/MAP estimates, i.e.,q ( θ )=
Revised: Assume that the model is trainedon data D with parameters ˆ θ , which corresponds to an approximation of MLE/MAP estimates, i.e.,

Original: δ ( θ − ˆ θ )
Revised: [Equation]

Original: ≈ p ( θ |D ) .
Revised: p [Equation] D ) .

Original: After observing the set of failures F , a point estimate of p ( θ |D , F ) is obtainedby performing a single update of natural gradient ascent [12] on the log likelihood of F with step size γ> 0 :
Revised: After observing the set of failures F , a point estimate of p ( θ |D , F ) is obtainedby performing a single update of natural gradient ascent [12] on the log likelihood of F with step size γ> 0 :

Original: [Equation]
Revised: [Equation]

Original:  where ˆ F ˆ θ is the empirical Fisher information matrix.
Revised: (8) where ˆ F ˆ θ is the empirical Fisher information matrix.

Original: This means
Revised: This means

Original: [Equation]
Revised: [Equation]

Original:  if defining the updated posterior as q ⋆ D ,F ( θ )
Revised: (9) if defining the updated posterior as q ⋆ D ,

Original: =
Revised: [Equation] ( θ )=

Original: δ ( θ − ˆ θ ⋆ D , F ) .
Revised: δ ( θ − ˆ θ ⋆ D , F ) .

Original: The negation of the above equation coincideswith the linear influence function (Eq.
Revised: The negation of the above equation coincideswith the linear influence function (Eq.

Original: (2) in [8]) when the failure set is assumed to bea singleton F = { z test } and γ =1 .
Revised: (2) in [8]) when the failure set is assumed to be F = { z ⋆ } and γ =1 .

Original: Note that the sign difference arises since our work aims to quantify the “negative”influence rather than the "positive" one in contrast with the work of Koh and Liang [8].

Original: Example 2 (Elastic Weight Consolidation): The generality of Eq.
Revised: Example 2 (Elastic Weight Consolidation): The generality of Eq.

Original: (7) permits any continual learningmethod of one’s choice for estimating the updated posterior q ⋆ D , F ( θ )
Revised: (7) permits any continual learningmethod of one’s choice for estimating the updated posterior q ⋆ D , F ( θ )

Original: ≈ p ( θ |D , F ) after observing failuresamples F .
Revised: ≈ p ( θ |D , F ) after observing failuresamples F .

Original: Here we illustrate how EWC [10] as a continual learning method can be adopted in the contextof cause identification.
Revised: Here we illustrate how EWC [10] as a continual learning method can be adopted in the contextof cause identification.

Original: EWC approximates p ( θ |D , F ) by first performing the Laplace approximation of theoriginal posterior p ( θ |D ) around the point estimate ˆ θ , and subsequently finding the MAP solution of θ .
Revised: EWC approximates p ( θ |D , F ) by first performing the Laplace approximation of theoriginal posterior p ( θ |D ) around the point estimate ˆ θ , and subsequently finding the MAP solution of θ .

Original: Formally, ˆ θ ⋆ D , F is obtained by maximising the objective below w.r.t.
Revised: Formally, ˆ θ ⋆ D , F is obtained by maximising the objective below w.r.t.

Original: θ via SGD (see Appendix A.3 for details):
Revised: θ via SGD (see Appendix A.3 for details):

Original: [Equation]
Revised: [Equation]

Original: where the off-diagonal elements of ˆ F ˆ θ are dropped for memory reason in practice.
Revised: where the off-diagonal elements of ˆ F ˆ θ are dropped for memory reason in practice.

Original: Intuitively, the first termencourages high accuracy on the failure set F whilst the second/third terms ensures the model parameters donot deviate too much in distribution.

Original: Defining q ( θ )=
Revised: Defining q ( θ )=

Original: δ ( θ − ˆ θ ) and q ⋆ D , F ( θ )=
Revised: δ ( θ − ˆ θ )and q ⋆ D , F ( θ )=

Original: δ ( θ − ˆ θ ⋆ D , F ) , we have that
Revised: δ ( θ − ˆ θ ⋆ D , F ) , we have that

Original: [Equation]
Revised: [Equation]

Original: To compute the above for each datapoint z ∈D , we only need to solve the optimisation problem of Eq.
Revised: To compute the above for each datapoint z ∈D , we only need to solve the optimisation problem of Eq.

Original: (10)by SGD once.
Revised: (10)by SGD once.

Original: We refer to this version of ˜ r ( z ) as EWC-influence function .
Revised: We refer to this version of ˜ r ( z ) as EWC-influence function .

Original: Comparison: The EWC-influence function generalises the linear influence approach.
Revised: Comparison: The EWC-influence function generalises the linear influence approach.

Original: To see this, wederive the fixed point of the EWC objective Eq.
Revised: To see this, wederive the fixed point of the EWC objective Eq.

Original: (10) w.r.t.
Revised: (10) w.r.t.

Original: θ (where we set λ =0 ):
Revised: θ (where we set λ =0 ):

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Then the update in Eq.
Revised: Then the update in Eq.

Original: (8) that is implicitly used by linear influence function can be viewed as (damped)one-step fixed-point iteration update initialised at ˆ θ for solving the fixed-point equation.
Revised: (8) that is implicitly used by linear influence function can be viewed as (damped)one-step fixed-point iteration update initialised at ˆ θ for solving the fixed-point equation.

Original: As EWC-influenceupdate (Eq.
Revised: As EWC-influenceupdate (Eq.

Original: (11)) is obtained by using the optimum of Eq.
Revised: (11)) is obtained by using the optimum of Eq.

Original: (10), it is arguably more accurate than linearinfluence function (Eq.
Revised: (10), it is arguably more accurate than linearinfluence function (Eq.

Original: (9)) for measuring the (detrimental) effect of a datum z to the model failures F .
Revised: (9)) for measuring the (detrimental) effect of a datum z to the model failures F .

Original: 2.2 Step II: Treatment
Revised: 2.2 Step II: Treatment

Original: Once the causes C of the failures F are identified among the training data D , we seek to repair themodel q ( θ ) by erasing the memories of C .
Revised: Once the causes C of the failures F are identified among the training data D , we seek to repair themodel q ( θ ) by erasing the memories of C .

Original: We formalise this problem as the computation of the posteriorp ( θ |D\C ) , i.e., C is absent from the training data D .
Revised: We formalise this problem as the computation of the posteriorp ( θ |D\C ) , i.e., C is absent from the training data D .

Original: A naive approach would re-run approximate inferenceon the whole “corrected” dataset D\C to obtain an approximate posterior q ⋆⋆ D\C ( θ )
Revised: A naive approach would re-run approximate inferenceon the whole “corrected” dataset D\C to obtain an approximate posterior q ⋆⋆ D\C ( θ )

Original: ≈ p ( θ |D\C ) which canbe time consuming.
Revised: ≈ p ( θ |D\C ) which canbe time consuming.

Original: But more fundamentally, by doing so, the obtained q ⋆⋆ D\C ( θ ) may be unrelated to theoriginal q ( θ ) based on which C were identified, due to, e.g. non-convex SGD optimisation issues.
Revised: But more fundamentally, by doing so, the obtained q ⋆⋆ D\C ( θ ) may be unrelated to theoriginal q ( θ ) based on which C were identified, due to, e.g. non-convex SGD optimisation issues.

Original: Moreover,this “model replacement” approach may not maintain other good properties of the original model q ( θ ) .
Revised: Moreover,this “model replacement” approach may not maintain other good properties of the original model q ( θ ) .

Original: Analogous to cause identification (Sec. 2.1), we propose to employ the continual learning approach to estimate efficiently the modified posterior p ( θ |D\C ) .
Revised: Analogous to cause identification (Sec. 2.1), we propose to employ the continual learning approach to estimate efficiently the modified posterior p ( θ |D\C ) .

Original: Applying Bayes’ rule and some algebraic manipulationsyield p ( θ |D\C ) ∝ p ( θ |D ) /p
Revised: Applying Bayes’ rule and some algebraic manipulationsyield p ( θ |D\C ) ∝ p ( θ |D ) /p

Original: ( C| θ ) (see Appendix A.4).
Revised: ( C| θ ) (see Appendix A.4).

Original: Therefore the information about C can be removedby scaling the current posterior p ( θ |D ) by the inverse of p ( C| θ ) and re-normalising.
Revised: Therefore the information about C can be removedby scaling the current posterior p ( θ |D ) by the inverse of p ( C| θ ) and re-normalising.

Original: In other words, wecan treat the approximation of p ( θ |D\C ) as a continual learning task, where the task is to “unlearn” thedatapoints in C while using the posterior distribution p ( θ |D ) as the prior.
Revised: In other words, wecan treat the approximation of p ( θ |D\C ) as a continual learning task, where the task is to “unlearn” thedatapoints in C while using the posterior distribution p ( θ |D ) as the prior.

Original: In practice, the target model to befixed corresponds to the approximate posterior q ( θ ) ≈ p ( θ |D ) .
Revised: In practice, the target model to befixed corresponds to the approximate posterior q ( θ ) ≈ p ( θ |D ) .

Original: Therefore continual (un)learning is done by
Revised: Therefore continual (un)learning is done by

Original: [Equation]
Revised: [Equation]

Original: The above approximation can be carried out with different approximate inference techniques such as
Revised: The above approximation can be carried out with different approximate inference techniques such as

Original: MLE/MAP point estimate, Laplace approximation [10] and variational inference [7].
Revised: MLE/MAP point estimate, Laplace approximation [10] and variational inference [7].

Original: At the end of thissection, we also provide a few examples to concretise this process.
Revised: At the end of thissection, we also provide a few examples to concretise this process.

Original: We highlight the deep connection between the “predictive approach” for cause identification (Sec.
Revised: We highlight the deep connection between the “predictive approach” for cause identification (Sec.

Original: 2.1),and the continual (un)learning for data deletion in the treatment step.
Revised: 2.1),and the continual (un)learning for data deletion in the treatment step.

Original: They share the key idea of editingthe posterior distribution q ( θ ) via continual learning.
Revised: They share the key idea of editingthe posterior distribution q ( θ ) via continual learning.

Original: This can be seen from the factor graph interpretationof posterior distributions, where posterior editing just corresponds to editing the selected factors in thegraph, e.g. insertion of p ( F| θ ) in cause identification, and deletion of p ( C| θ ) in treatment.
Revised: This can be seen from the factor graph interpretationof posterior distributions, where posterior editing just corresponds to editing the selected factors in thegraph, e.g. insertion of p ( F| θ ) in cause identification, and deletion of p ( C| θ ) in treatment.

Original: Example 1 (Fine-tuning on Corrected Data):
Revised: Example 1 (Fine-tuning on Corrected Data):

Original: Given a point estimate of model parameters ˆ θ , i.e.,q ( θ )=
Revised: Given a point estimate of model parameters ˆ θ , i.e.,q ( θ )=

Original: δ ( θ − ˆ θ ) , a simple way to approximate p ( θ |D\C ) is to fine-tune on the corrected dataset D\Cand update the point estimate.
Revised: δ ( θ − ˆ θ ) , a simple way to approximate p ( θ |D\C ) is to fine-tune on the corrected dataset D\Cand update the point estimate.

Original: The new ˆ θ ⋆⋆ D\C of the repaired model are obtained by maximising thelog-likelihood log p ( D\C| θ ) via SGD, starting from ˆ θ .
Revised: The new ˆ θ ⋆⋆ D\C of the repaired model are obtained by maximising thelog-likelihood log p ( D\C| θ ) via SGD, starting from ˆ θ .

Original: Example 2 (Newton Update Removal): Guo et al.
Revised: Example 2 (Newton Update Removal): Guo et al.

Original: [9] proposed a Newton update based method for datadeletion.
Revised: [9] proposed a Newton update based method for datadeletion.

Original: This method reduces to a specific form of Eq.
Revised: This method reduces to a specific form of Eq.

Original: (13) when using negative log-likelihood as its loss:
Revised: (13) when using negative log-likelihood as its loss:

Original: [Equation]
Revised: [Equation]

Original: Here information about C gets deleted by performing a single-step natural gradient descent on theirlog likelihood [12].
Revised: Here information about C gets deleted by performing a single-step natural gradient descent on theirlog likelihood [12].

Original: Also, notice the similarity with the way linear influence [8] is computed in Eq.
Revised: Also, notice the similarity with the way linear influence [8] is computed in Eq.

Original: (8),illuminating the relation between cause identification and treatment steps.
Revised: (8),illuminating the relation between cause identification and treatment steps.

Original: Example 3 (EWC for data deletion): The update rule in Eq.
Revised: Example 3 (EWC for data deletion): The update rule in Eq.

Original: (13) for data deletion is amenable to anycontinual learning approaches.
Revised: (13) for data deletion is amenable to anycontinual learning approaches.

Original: For example, given model parameters ˆ θ , EWC-based deletion obtainsnew parameters ˆ θ ⋆⋆ D\C by maximising the following objective (see Appendix A.4):[Equation]
Revised: For example, given model parameters ˆ θ , EWC-based deletion obtainsnew parameters ˆ θ ⋆⋆ D\C by maximising the following objective (see Appendix A.4):

Original: [Equation]
Revised: [Equation]

Original: where the first term seeks to remove information about C while the remaining terms discourage parametersfrom deviating from the original values.
Revised: where the first term seeks to remove information about C while the remaining terms discourage parametersfrom deviating from the original values.

Original: Contrasting this with Eq.
Revised: Contrasting this with Eq.

Original: (10) again reveals the connectionbetween EWC methods for cause identification and treatment steps.
Revised: (10) again reveals the connectionbetween EWC methods for cause identification and treatment steps.

Original: Comparison: Similar to the comparison made in the cause identification part, EWC for data deletion alsogeneralises the Newton update removal (Eq.
Revised: Comparison: Similar to the comparison made in the cause identification part, EWC for data deletion alsogeneralises the Newton update removal (Eq.

Original: This can again be shown by deriving (damped) one-stepfixed point iterative update starting from ˆ θ to approximate the fixed point of Eq.
Revised: This can again be shown by deriving (damped) one-stepfixed point iterative update starting from ˆ θ to approximate the fixed point of Eq.

Original: (15) when λ =0 .
Revised: (15) when λ =0 .

Original: As EWCfor deletion uses SGD to approximate optimum of Eq.
Revised: As EWCfor deletion uses SGD to approximate optimum of Eq.

Original: (15), it is arguably better than Newton update removalfor erasing the effects of detrimental examples C , while better maintaining performance on other cases.
Revised: (15), it is arguably better than Newton update removalfor erasing the effects of detrimental examples C , while better maintaining performance on other cases.

Original:  Related work
Revised:  Related work

Original: Model Editing .
Revised: Model Editing .

Original: There is a recent surge of interest in developing targeted updates to correct model’s undesirable behaviours, while leaving other desired properties intact.
Revised: There is a recent surge of interest in developing targeted updates to correct model’s undesirable behaviours, while leaving other desired properties intact.

Original: As naive fine-tuning methods often lead tooverfitting to the failure examples and accuracy degradation on others, various strategies have been proposed.
Revised: As naive fine-tuning methods often lead tooverfitting to the failure examples and accuracy degradation on others, various strategies have been proposed.

Original: For example, Zhu et al.
Revised: For example, Zhu et al.

Original: [13] employ a simple regularization technique to minimize parameter changes duringthe fine-tuning phase.
Revised: [13] employ a simple regularization technique to minimize parameter changes duringthe fine-tuning phase.

Original: Subsequent works [14, 15] advocate for a functional regularisation instead, e.g. KLdivergence in the output space, to achieve better regularisation.
Revised: Subsequent works [14, 15] advocate for a functional regularisation instead, e.g. KLdivergence in the output space, to achieve better regularisation.

Original: These lines of work, additionally, propose touse meta-learning [16] to learn to edit the target model, where the latest meta-learning approach is proposedby Mitchell et al.
Revised: These lines of work, additionally, propose touse meta-learning [16] to learn to edit the target model, where the latest meta-learning approach is proposedby Mitchell et al.

Original: Another promising approach [18] performs weight editing so that features of a specific concept (e.g. snow) map to the features of another (e.g. road).
Revised: Another promising approach [18] performs weight editing so that features of a specific concept (e.g. snow) map to the features of another (e.g. road).

Original: A commonality among these approachesis the focus on direct model edits for correction.
Revised: A commonality among these approachesis the focus on direct model edits for correction.

Original: Our work takes an orthogonal and under-explored anglewhere the aim is to “edit” the data instead, by identifying and removing harmful examples which causefailures — in turn, this difference makes our framework complementary to these model-editing approaches.
Revised: Our work takes an orthogonal and under-explored anglewhere the aim is to “edit” the data instead, by identifying and removing harmful examples which causefailures — in turn, this difference makes our framework complementary to these model-editing approaches.

Original: Continual learning.
Revised: Continual learning.

Original: Continual learning is an active research area with a related but broader scope thanmodel repairment, which aims to develop methods that adapt the model for future tasks while maintainingmodel performance on previously learned tasks [11].
Revised: Continual learning is an active research area with a related but broader scope thanmodel repairment, which aims to develop methods that adapt the model for future tasks while maintainingmodel performance on previously learned tasks [11].

Original: We focus on a more targeted problem in this work,yet introduce a framework that allows the use of any continual learning approach for model repairment.
Revised: We focus on a more targeted problem in this work,yet introduce a framework that allows the use of any continual learning approach for model repairment.

Original: Our experiments presents EWC [10] as a practical instantiation of the framework.
Revised: Our experiments presents EWC [10] as a practical instantiation of the framework.

Original: One can also leverageimprovements over EWC such as online EWC [19], or other regularisation-based methods that aremotivated by Bayesian learning principles, such as variational continual learning [7, 20, 21], synapticintelligence [22], and orthogonal gradient descent [23].
Revised: One can also leverageimprovements over EWC such as online EWC [19], or other regularisation-based methods that aremotivated by Bayesian learning principles, such as variational continual learning [7, 20, 21], synapticintelligence [22], and orthogonal gradient descent [23].

Original: As approximations to r ( C ) rely on accurateposterior approximations, advances in Bayesian continual learning methods are expected to improve thepractical effectiveness of model repairment under our framework.
Revised: As approximations to r ( C ) rely on accurateposterior approximations, advances in Bayesian continual learning methods are expected to improve thepractical effectiveness of model repairment under our framework.

Original: Data Selection and Valuation .
Revised: Data Selection and Valuation .

Original: Multiple techniques have been introduced for selecting “influential” trainingexamples on a chosen metric (e.g. test accuracy), such as influence functions [8, 24, 25, 26, 27, 28, 29], Shapley value-based approaches [30, 31, 32] and probability of sufficiency [33].
Revised: Multiple techniques have been introduced for selecting “influential” trainingexamples on a chosen metric (e.g. test accuracy), such as influence functions [8, 24, 25, 26, 27, 28], Shapleyvalue-based approaches [29, 30, 31] and probability of sufficiency [32].

Original: Within the category of influencefunctions, two representative approaches include linear influence function [8] and SGD-influence [27].
Revised: Within the category of influencefunctions, two representative approaches include linear influence function [8] and SGD-influence [27].

Original: Theformer approach performs one-step update only, thus, while efficient, it may be less accurate in reflecting theinfluence of a datum z .
Revised: Theformer approach performs one-step update only, thus, while efficient, it may be less accurate in reflecting theinfluence of a datum z .

Original: The latter approach computes a projected difference between ˆ θ and ˆ θ ⋆ D , F but withˆ θ ⋆ D , F obtained by running SGD fine-tuning on training data without z .
Revised: The latter approach computes a projected difference between ˆ θ and ˆ θ ⋆ D , F but withˆ θ ⋆ D , F obtained by running SGD fine-tuning on training data without z .

Original: Thus SGD-influence is computationally inefficient.
Revised: Thus SGD-influence is computationally inefficient.

Original: Compared to both baselines, our EWC-influence approach achieve the best in both worlds:it produces more accurate influence estimates than linear influence due to better optimisation, while it ismore efficient than SGD-influence as it requires only one optimisation procedure on the given failure set F .
Revised: Compared to both baselines, our EWC-influence approach achieve the best in both worlds:it produces more accurate influence estimates than linear influence due to better optimisation, while it ismore efficient than SGD-influence as it requires only one optimisation procedure on the given failure set F .

Original: Data Deletion .
Revised: Data Deletion .

Original: The detrimental data removal in the treatment step is related to data deletion , a rapidly developing field of machine learning research [34, 9,  35, 36, 37, 38].
Revised: The detrimental data removal in the treatment step is related to data deletion , a rapidly developing field of machine learning research [33, 9, 34, 35, 36, 37].

Original: Closest to our work is variational Bayesianunlearning [39] which extends variational Bayes to data deletion settings.
Revised: Closest to our work is variational Bayesianunlearning [38] which extends variational Bayes to data deletion settings.

Original: But the connection to continuallearning is not explicitly made, and it is limited to applications in logistic regression and sparse Gaussianprocesses.
Revised: But the connection to continuallearning is not explicitly made, and it is limited to applications in logistic regression and sparse Gaussianprocesses.

Original: In general, the main focus of existing data deletion research is to preserve data privacy, anddatapoints to be removed are assumed provided.
Revised: In general, the main focus of existing data deletion research is to preserve data privacy, anddatapoints to be removed are assumed provided.

Original: On the contrary, in this work, we focus on the repairmentof models and propose a unified procedure not only to remove data but also to identify which ones to do so.
Revised: On the contrary, in this work, we focus on the repairmentof models and propose a unified procedure not only to remove data but also to identify which ones to do so.

Revised:  Experiments

Original: Top
Revised: We evaluate the efficacy of the proposed framework in a) identifying the causes of target prediction failuresin Sec. 4.1, and b) repairing the original model by erasing the memories of such causes in Sec. 4.2. Weuse augmented versions of MNIST and CIFAR-10 datasets with simulated annotation and input noise. Such controlled experiments are performed for creating "ground truths” of failure causes – necessary forvalidating the quality of identification methods – and for testing the method in a variety of settings. Baselines.

Revised: For the cause identification task, we compare our approach ( EWC-influence ) against the linearinfluence function [8] and SGD-influence [27].

Revised: To avoid expensive computation of ˆ F − 1 ˆ θ , Koh & Liang

Revised: [Figure]

Revised: [Figure]

Revised: (d) Top

Revised: [Figure]

Original: Causes
Revised: Causes

Original: M N I
Revised: M N I

Original: S T
Revised: S T

Original: C
Revised: C

Original: I F
Revised: I F

Original: A R
Revised: A R

Original: Label = “bird” Label = “bird” Label = “bird”
Revised: Label = “bird” Label = “bird” Label = “bird”

Original: Label = “bird”
Revised: Label = “bird”

Original: Label = “bird” Label = “bird”
Revised: Label = “bird” Label = “bird”

Original: Label = “bird” Label = “bird”
Revised: Label = “bird” Label = “bird”

Original: Label = “bird” Label = “cat”
Revised: Label = “bird” Label = “cat”

Original: Label = “bird”
Revised: Label = “bird”

Original: Label = “bird” Label = “bird”
Revised: Label = “bird” Label = “bird”

Original: Label = “bird” Label = “bird”
Revised: Label = “bird” Label = “bird”

Original: Label = “cat”
Revised: Label = “cat”

Original: Label = 1 Label = Label = 1 Label =
Revised: Label = 1 Label = Label = 1 Label =

Original: Label =
Revised: Label =

Original: Label =
Revised: Label =

Original: Label = 1 Label =
Revised: Label = 1 Label =

Original: Label =
Revised: Label =

Original: Label = 1 Label = 6 Label =
Revised: Label = 1 Label = 6 Label =

Original: Label =
Revised: Label =

Original: Label = Label =
Revised: Label = Label =

Original: Label =
Revised: Label =

Original: [Figure]
Revised: [Figure]

Original: We evaluate the efficacy of the proposed framework in a) identifying the causes of target prediction failuresin Sec. 4.1, and b) repairing the original model by erasing the memories of such causes in Sec. 4.2. Weuse augmented versions of MNIST and CIFAR-10 datasets with simulated annotation and input noise. Such controlled experiments are performed for creating "ground truths” of failure causes – necessary forvalidating the quality of identification methods – and for testing the method in a variety of settings.
Revised: [8] introduced two efficient approximations to the Hessian-vector product

Original: Baselines. For the cause identification task, we compare our approach ( EWC-influence ) against the linearinfluence function [8] and SGD-influence [27]. To avoid expensive computation of ˆ F − 1 ˆ θ , Koh & Liang[8] introduced two efficient approximations to the Hessian-vector product ˆ F − 1 ˆ θ ∇ θ log p ( F| θ ) ; the firstsolves arg min v { v T ˆ F − 1 ˆ θ ∇ θ v − log p ( F| θ ) T v } with gradient descent (GD), while the second uses aniterative algorithm for stochastic approximation (SA) from [40].
Revised: ˆ F − 1 ˆ θ ∇ θ log p ( F| θ ) ; the firstsolves arg min v { v T ˆ F − 1 ˆ θ ∇ θ v − log p ( F| θ ) T v } with gradient descent (GD), while the second uses aniterative algorithm for stochastic approximation (SA) from [39].

Original: We implement these two variants (GD& SA) of linear influence in Pytorch, and use the original implementation for SGD-influence.
Revised: We implement these two variants (GD& SA) of linear influence in Pytorch, and use the original implementation for SGD-influence.

Original: For themodel treatment task, we compare our method ( EWC-deletion ) against Newton update removal [9].
Revised: For themodel treatment task, we compare our method ( EWC-deletion ) against Newton update removal [9].

Original: Thismethod again requires computing a Hessian-vector product for which we employ the same stochasticapproximation technique [40].
Revised: Thismethod again requires computing a Hessian-vector product for which we employ the same stochasticapproximation technique [39].

Original: To isolate the evaluation of cause identification and treatment, we furtherconsider in Section 4.1 fine-tuning on D\C as another repairment strategy, which would return the bestrepairment result if the set C correctly captures the detrimental datapoints.
Revised: To isolate the evaluation of cause identification and treatment, we furtherconsider in Section 4.1 fine-tuning on D\C as another repairment strategy, which would return the bestrepairment result if the set C correctly captures the detrimental datapoints.

Original: Lastly, we set the prior term λto zero in eq.(10) and eq.
Revised: Lastly, we set the prior term λto zero in eq.(10) and eq.

Original: (15) to ensure fair comparison with linear influence and Newton update removal.
Revised: (15) to ensure fair comparison with linear influence and Newton update removal.

Original: Common Set-up.
Revised: Common Set-up.

Original: We train the base classification models on the training split of the “augmented” MNISTand CIFAR-10 datasets.
Revised: We train the base classification models on the training split of the “augmented” MNISTand CIFAR-10 datasets.

Original: For MNIST, we use 6% (3000 samples) of the original training set to make the taskmore challenging.
Revised: For MNIST, we use 6% (3000 samples) of the original training set to make the taskmore challenging.

Original: We use instances of CNNs throughout and train them using the Adam optimiser [41].
Revised: We use instances of CNNs throughout and train them using the Adam optimiser [40].

Original: Thearchitecture and training details can be found in Appendix C. For evaluation, we separate the test set T intothe set of misclassified examples, F (“ failure set ”) and the others, T \F which are correctly classified (“ remaining set ”).
Revised: Thearchitecture and training details can be found in Appendix C. For evaluation, we separate the test set T intothe set of misclassified examples, F (“ failure set ”) and the others, T \F which are correctly classified (“ remaining set ”).

Original: We further split the failure set into query , F q and holdout , F h sets, where we only use the former to identify failure causes C , and use the latter to quantify how generalisably the removal of C can amendthe failure cases.
Revised: We further split the failure set into query , F q and holdout , F h sets, where we only use the former to identify failure causes C , and use the latter to quantify how generalisably the removal of C can amendthe failure cases.

Original: We stress that F q is used for cause identification only, but not for further model adaptation.
Revised: We stress that F q is used for cause identification only, but not for further model adaptation.

Original: 4.1 Identifying Failure Causes
Revised: 4.1 Identifying Failure Causes

Original: Annotation Noise.
Revised: Annotation Noise.

Original: To induce test prediction failures,  we randomly flip labelsin the training set betweensemantically similar classes (e.g. 1 and 7 for MNIST, and cats and dogs for CIFAR-10) according to theconfusion matrices in Fig. 2(a).
Revised: To induce test prediction failures, for the training set we randomly flip labels betweensemantically similar classes (e.g. 1 and 7 for MNIST, and cats and dogs for CIFAR-10) according to theconfusion matrices in Fig. 2(a).

Original: As a result, the classes of miss-classified test examples are concentratedon those classes with label noise as depicted in Fig. 2(b).
Revised: As a result, the classes of miss-classified test examples are concentratedon those classes with label noise as depicted in Fig. 2(b).

Original: To measure the accuracy of identifying incorrectly labelled examples, we inspect the training examplesz ∈D in the descending order of ˜ r ( z ) computed with F q which contains 50% of the miss-classified testcases, and calculate the fraction of incorrectly labelled datapoints in inspected examples.
Revised: To measure the accuracy of identifying incorrectly labelled examples, we inspect the training examplesz ∈D in the descending order of ˜ r ( z ) computed with F q which contains 50% of the miss-classified testcases, and calculate the fraction of incorrectly labelled datapoints in inspected examples.

Original: Fig. 2(c) showsthat EWC-influence identifies more failure causes earlier on compare to other methods, and is the closestmatch to the “Oracle” baseline which has full knowledge of samples with wrong labels.
Revised: Fig. 2(c) showsthat EWC-influence identifies more failure causes earlier on compare to other methods, and is the closestmatch to the “Oracle” baseline which has full knowledge of samples with wrong labels.

Original: Fig. 2(d) showsthat the top few causes according to EWC-influence are the samples with incorrect labels, while the leastharmful ones are the images of the same classes but with the correct labels as shown in Fig.
Revised: Fig. 2(d) showsthat the top few causes according to EWC-influence are the samples with incorrect labels, while the leastharmful ones are the images of the same classes but with the correct labels as shown in Fig.

Original: 7 in Appendix B.
Revised: 7 in Appendix B.

Original: As stipulated in Sec. 2, a set of identified causes C is of higher quality if removing them leads to a largergain in accuracy on the failure set while maintaining performance on others.
Revised: As stipulated in Sec. 2, a set of identified causes C is of higher quality if removing them leads to a largergain in accuracy on the failure set while maintaining performance on others.

Original: To measure such qualityof causes , we fine-tune the base model on D\C and report the accuracy on the failure query set F q , theholdout failure set F h as well as the remaining test set, T \F .
Revised: To measure such qualityof causes , we fine-tune the base model on D\C and report the accuracy on the failure query set F q , theholdout failure set F h as well as the remaining test set, T \F .

Original: Results in Fig.
Revised: Results in Fig.

Original: 3 suggest that removingfailure causes according to EWC-influence yields the highest increase in accuracy on the failure set Fwithout hurting performance in the remaining test set T \F .
Revised: 3 suggest that removingfailure causes according to EWC-influence yields the highest increase in accuracy on the failure set Fwithout hurting performance in the remaining test set T \F .

Original: We also note that all of the methods are ableto fix the failures better than randomly removing datapoints, and more interestingly, for MNIST, whenenough causes are erased ( ≈ 10 3 ), all methods even surpass the case in which all label noise instancesare removed.
Revised: We also note that all of the methods are ableto fix the failures better than randomly removing datapoints, and more interestingly, for MNIST, whenenough causes are erased ( ≈ 10 3 ), all methods even surpass the case in which all label noise instances [Figure] are removed.

Original: This result implies that, while annotation noise is a major detrimental factor, the predictionfailures also arise from other types of harmful examples.
Revised: This result implies that, while annotation noise is a major detrimental factor, the predictionfailures also arise from other types of harmful examples.

Original: Lastly, we evaluate the sample efficiency of cause identification by reducing the size of the query set F q .
Revised: Lastly, we evaluate the sample efficiency of cause identification by reducing the size of the query set F q .

Original: Fig.
Revised: Fig.

Original: 6 in Appendix B shows that all approaches degrade gracefully in repairment performance as the querysize gets smaller, but overall EWC-influence still remains the best in terms of label noise detection andrepairment accuracy on the failure set and the remaining set.
Revised: 6 in Appendix B shows that all approaches degrade gracefully in repairment performance as the querysize gets smaller, but overall EWC-influence still remains the best in terms of label noise detection andrepairment accuracy on the failure set and the remaining set.

Original: Random Input Noise.
Revised: Random Input Noise.

Original: In this experiment, we inject synthetic outliers into MNIST and CIFAR-10 andtest the quality of cause identification.
Revised: In this experiment, we inject synthetic outliers into MNIST and CIFAR-10 andtest the quality of cause identification.

Original: We select a set of target classes — 1, 7, 6, 9 for MNIST andplane, bird, cat, dog for CIFAR-10 — and randomly corrupt 30% of the images in those classes by addingsalt-and-pepper noise (i.e. replacing pixels with extreme values 0 and 255) in MNIST and Gaussian noisein CIFAR-10.
Revised: We select a set of target classes — 1, 7, 6, 9 for MNIST and plane,bird, cat, dog for CIFAR-10 — and randomly corrupt 30% of the images in those classes by adding saltand-pepper in MNIST and Gaussian noise in CIFAR-10.

Original: The top row in Fig.
Revised: The top row in Fig.

Original: 4 shows examples, and those corrupted images constitute roughly 12%of the whole training set.
Revised: 4 shows examples, and thosecorrupted images constitute roughly 12% of the whole training set.

Original: Sec.
Revised: Sec.

Original: C in the appendix provides details.
Revised: C in the appendix provides details.

Original: We use a subgroup of failures in the target classes as the query F q to compute influence values.
Revised: We use a subgroup of failures in the target classes as the query F q to compute influence values.

Original: Surprisingly,the second rows in Fig. 4(a) and (b) show that EWC-influence largely avoids selecting the corrupted imagesas the top 1000 causes for MNIST and the top 20000 causes for CIFAR-10.
Revised: Surprisingly,the second rows in Fig. 4(a) and (b) show that EWC-influence largely avoids selecting the corrupted imagesas the top 1000 causes for MNIST and the top 20000 causes for CIFAR-10.

Original: However, the third and thefourth rows show that removing those causes results in the best treatment performance on failures whilemaintaining the performance at a level similar to other baselines.
Revised: However, the third and thefourth rows show that removing those causes results in the best treatment performance on failures whilemaintaining the performance at a level similar to other baselines.

Original: In fact, removing all the input noise andretraining is not able to fix the failures by much, indicating that EWC-influence is able to correctly avoidthese relatively harmless outliers and detect other more harmful causes.
Revised: In fact, removing all the input noise andretraining is not able to fix the failures by much, indicating that EWC-influence is able to correctly avoidthese relatively harmless outliers and detect other more harmful causes.

Original: Fig.
Revised: Fig.

Original: 8 in Appendix B visualises themost harmful examples identified by EWC-influence.
Revised: 8 in Appendix B visualises themost harmful examples identified by EWC-influence.

Original: Many of them appear to be ambiguous instances innon-target classes, e.g. wonky digits, close-up views of vehicles, a real instance with incorrect label [42], etc.
Revised: Many of them appear to be ambiguous instances innon-target classes, e.g. wonky digits, close-up views of vehicles, a real instance with incorrect label [41], etc.

Original: Adversarial Poisoning.
Revised: Adversarial Poisoning.

Original: To simulate input noise that can induce test-time failures, we introducecontaminated data by randomly corrupting 30% of the training images in those previously mentionedtarget classes.
Revised: To simulate input noise that can induce test-time failures, we introducecontaminated data by randomly corrupting 30% of the training images in those previously mentionedtarget classes.

Original: These poisoned datapoints are adversarial images crafted by the fast gradient sign method(FGSM)
Revised: These poisoned datapoints are adversarial images crafted by the fast gradient sign method(FGSM)

Original: [43] on a separate set of victim models trained on the original clean datasets, and they are labelledby the classes predicted by the victim models.
Revised: [42] on a separate set of victim models trained on the original clean datasets, and they are labelledby the classes predicted by the victim models.

Original: The poisoned datasets are then used to train the base modelsthat are used for evaluation of cause identification.
Revised: The poisoned datasets are then used to train the base modelsthat are used for evaluation of cause identification.

Original: Fig. 4(c) and (d) show that most of the influencefunctions detect the corrupted samples better than the “random” baseline.
Revised: Fig. 4(c) and (d) show that most of the influence functions detect the corrupted samples better than the“random” baseline.

Original: The dashed lines in the thirdrow show that removing all of the corrupted inputs lead to a significant gain in accuracy on the holdoutfailure set in comparison with the random noise setting, illustrating the larger extent of harms caused bydata poisoning.
Revised: The dashed lines in the third row show that removing all of the corrupted inputs lead to asignificant gain in accuracy on the holdout failure set in comparison with the random noise setting, illustrating the larger extent of harms caused by data poisoning.

Original: However, most of the identification methods still outperform this reference by a largemargin.
Revised: However, most of the identification methods stilloutperform this reference by a large margin.

Original: This suggests again for the presence of other influential samples, and EWC-influence is ableto pick up the most important ones, judging by the accuracy on the failure set.
Revised: This suggests again for the presence of other influential samples,and EWC-influence is able to pick up the most important ones, judging by the accuracy on the failure set.

Original: Speed Comparison.
Revised: Speed Comparison.

Original: Table 1 in Appendix B shows the total run-time of cause identification methodson a single GPU for their best sets of hyper-parameters selected based on the treatment accuracy on thefailure set.
Revised: Table 1 in Appendix B shows the total run-time of cause identification methodson a single GPU for their best sets of hyper-parameters selected based on the treatment accuracy on thefailure set.

Original: For both datasets, EWC-influence achieves comparable or shorter run time than the baselines.
Revised: For both datasets, EWC-influence achieves comparable or shorter run time than the baselines.

Original: (a) MNIST + Random Noise (b) CIFAR-10 + Random Noise

Original: Label:

Original: Label:

Original: Label:plane cat bird 1 =>3 => 9 0 =>

Original: (c) MNIST + Adversarial Attacks (d) CIFAR-10 + Adversarial Attacks cat => deer dog => bird plane => deer

Original: [Figure]

Original: 4.2 Comparison of Treatment Methods
Revised: 4.2 Comparison of Treatment Methods

Original: We evaluate the performance of different deletion-based methods for treatment introduced in Sec.
Revised: We evaluate the performance of different deletion-based methods for treatment introduced in Sec.

Original: 2.2 on
Revised: 2.2 on

Original: MNIST and CIFAR10 datasets with simulated annotation noise, used in the previous section.
Revised: MNIST and CIFAR10 datasets with simulated annotation noise, used in the previous section.

Original: We run both
Revised: We run both

Original: EWC-deletion (ours) and Newton update removal [9] methods with early stopping based on the query setaccuracy, and experiment with different hyper-parameter settings (see Sec.
Revised: EWC-deletion (ours) and Newton update removal [9] methods with early stopping based on the query setaccuracy, and experiment with different hyper-parameter settings (see Sec.

Original: C in Appendix) to achieve different trade-offs between failure set accuracy and remaining set performance.
Revised: C in Appendix) to achieve dif- [Figure] [Figure] ferent trade-offs between failure set accuracy and remaining set performance.

Original: Here EWC-influence is usedto identify the causes, and the top 15% examples were removed by the respective deletion methods.
Revised: Here EWC-influence is used to identify the causes, and the top 15% examples were removed by the respective deletion methods.

Original: Suchtrade-off is shown in Fig.
Revised: Suchtrade-off is shown in Fig.

Original: 5, where fine-tuning on D\C is included as an “upper-bound” reference for datadeletion performance.
Revised: 5, where fine-tuning on D\C is included as an “upper-bound” reference for datadeletion performance.

Original: On MNIST, EWC-deletion attains a considerably better trade-off between treatmentand maintenance compared to Newton-update-deletion, and is much closer to the fine-tuning reference.
Revised: On MNIST, EWC-deletion attains a considerably better trade-off between treatmentand maintenance compared to Newton-update-deletion, and is much closer to the fine-tuning reference.

Original: For CIFAR-10, EWC-deletion beats the Newton-update deletion by 5% in the best failure accuracy whilethe order reverses for the best accuracy on the remaining test set but with less than 1% difference.
Revised: For CIFAR-10, EWC-deletion beats the Newton-update deletion by 5% in the best failure accuracy whilethe order reverses for the best accuracy on the remaining test set but with less than 1% difference.

Original:  Conclusions
Revised:  Conclusions

Original: In this work, we develop a generic framework for repairing machine learning models by erasing memoriesof detrimental datapoints.
Revised: In this work, we develop a generic framework for repairing machine learning models by erasing memoriesof detrimental datapoints.

Original: The framework consists of two key components, that are, the mechanism foridentifying the “causes” in training data which are responsible for the given failures, and the adaptationmethod for fixing the model by removing information about them.
Revised: The framework consists of two key components, that are, the mechanism foridentifying the “causes” in training data which are responsible for the given failures, and the adaptationmethod for fixing the model by removing information about them.

Original: Two components are connected under the
Revised: Two components are connected under the

Original: Bayesian view of continual (un)learning, which brings forth several practical benefits.
Revised: Bayesian view of continual (un)learning, which brings forth several practical benefits.

Original: Firstly, the frameworksubsumes some recent works on influence function and data deletion as specific examples, and elucidatetheir limitations.
Revised: Firstly, the frameworksubsumes some recent works on influence function and data deletion as specific examples, and elucidatetheir limitations.

Original: Secondly, the generality of our approach allows leveraging recent advances in continuallearning in this new problem of model repairment.
Revised: Secondly, the generality of our approach allows leveraging recent advances in continuallearning in this new problem of model repairment.

Original: In particular, we extend Elastic Weight Consolidation tocause identification and data deletion, and demonstrate empirically its competitive performance in both tasks.
Revised: In particular, we extend Elastic Weight Consolidation tocause identification and information removal, and demonstrate empirically its competitive performance inboth tasks.

Revised: Future work will investigate the values of adapting more recent continual learning approaches,and also study other types of data correction mechanisms (e.g. label correction, sample/label acquisition).

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on howto answer these questions.
Revised: Please read the checklist guidelines carefully for information on howto answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Original: Youare strongly encouraged to include a justification to your answer , either by referencing the appropriatesection of your paper or providing a brief inline description.
Revised: Youare strongly encouraged to include a justification to your answer , either by referencing the appropriatesection of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code is proprietary at present.
Revised: [No] The code is proprietary at present.

Original: It
Revised: It

Original: will, however, be possible to open-source the code by September this year.
Revised: will, however, be possible to open-source the code by September this year.

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions blockand only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions blockand only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No](d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b)
Revised: [N/A]

Original:  Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]3.
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board

Original: (IRB) approvals, if applicable?
Revised: (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent

Original: on participant compensation?
Revised: on participant compensation?

Original: [N/A]
Revised: [N/A]