Title: Retaining Beneficial Information from Detrimental Data for Neural Network Repair
Venue: NeurIPS 2023
Accepted: yes
Keywords: Model Repair; Fine-tuning
Abstract: The performance of deep learning models heavily relies on the quality of the training data. Inadequacies in the training data, such as corrupt input or noisy labels, can lead to the failure of model generalization. Recent studies propose repairing the model by identifying the training samples that contribute to the failure and removing their influence from the model. However, it is important to note that the identified data may contain both beneficial and detrimental information. Simply erasing the information of the identified data from the model can have a negative impact on its performance, especially when accurate data is mistakenly identified as detrimental and removed. To overcome this challenge, we propose a novel approach that leverages the knowledge obtained from a retained clean set. Our method first identifies harmful data by utilizing the clean set, then separates the beneficial and detrimental information within the identified data. Finally, we utilize the extracted beneficial information to enhance the model's performance. Through empirical evaluations, we demonstrate that our method outperforms baseline approaches in both identifying harmful data and rectifying model failures. Particularly in scenarios where identification is challenging and a significant amount of benign data is involved, our method improves performance while the baselines deteriorate due to the erroneous removal of beneficial information.

Retaining Beneficial Information from Detrimental Data for Deep Neural Network Repair

Abstract

The performance of deep learning models heavily relies on the quality of the training data. Inadequacies in the training data, such as corrupt input or noisy labels, can lead to the failure of model generalization. Recent studies propose repairing the model by identifying the training samples that contribute to the failure and removing their influence from the model. However, it is important to note that the identified data may contain both beneficial and detrimental information. Simply erasing the information of the identified data from the model can have a negative impact on its performance, especially when accurate data is mistakenly identified as detrimental and removed. To overcome this challenge, we propose a novel approach that leverages the knowledge obtained from a retained clean set. Concretely, Our method first identifies harmful data by utilizing the clean set, then separates the beneficial and detrimental information within the identified data. Finally, we utilize the extracted beneficial information to enhance the model's performance. Through empirical evaluations, we demonstrate that our method outperforms baseline approaches in both identifying harmful data and rectifying model failures. Particularly in scenarios where identification is challenging and a significant amount of benign data is involved, our method improves performance while the baselines deteriorate due to the erroneous removal of beneficial information.

1 Introduction

The effectiveness of deep neural networks (DNNs) is greatly influenced by the quantity and quality of the training data from which they are trained. However, obtaining a large amount of high-quality data can be costly. An alternative approach is to acquire training data from sources such as the Web [2], external data suppliers, or crowdsourcing [14]. Although this allows for the collection of a substantial volume of data, the quality of the acquired data cannot be guaranteed. Various issues can arise, including annotation errors [3], data corruption [13], or even deliberate adversarial manipulation [39] by malicious providers. Consequently, a portion of the collected data may have a different distribution than the majority of the data [21]. In this work, we refer to these two categories of data as "detrimental" and "target" data, respectively. The presence of detrimental data in the training set can significantly hinder the performance of a DNN [1]. While recent research has proposed methods to mitigate specific issues like label noise [31, 5, 11], adversarial attacks [28, 39], or data shifts during model training [21, 32], it remains challenging to train a model that generalizes well to target data when the specific cause of the detrimental data is unknown.

It is important to note that in practical scenarios, a DNN model must undergo a quality assurance test before it can be deployed. During this evaluation process, the performance of the trained DNN model is usually assessed using a small reserved set of data known as clean data. This clean data is expected

to have the same distribution as the target data. However, since the training process involves the use of detrimental data, the trained DNN model may produce inaccurate predictions on the clean data, leading to a failure in the quality assurance test. In such cases, it becomes necessary to repair the DNN model to successfully pass the quality assurance test.

Recent approaches to model repair focus on two main steps. Firstly, they aim to identify the detrimental data instances within the original training dataset with a set of clean data, a.k.a, test data. Once identified, these instances can be categorized for further action. Secondly, the model is updated by eliminating the identified detrimental data instances. This repair process helps in improving the model's performance and ensuring that it meets the desired quality standards for deployment. Notable works in this area include studies by Tanno et al.[33] and Kong et al.[23].

In contrast to previous approaches that focus on eliminating the entire detrimental dataset to learn a repaired DNN model, our approach is motivated by the belief that the detrimental data may contain some valuable underlying information. Once this information can be extracted, it can be leveraged to train a more generalized model alongside the target data. Drawing inspiration from domain adaptation techniques [35, 30], we propose a framework that aligns the identified detrimental data with the clean data. By aligning these datasets, we can utilize the aligned detrimental data to repair the DNN model and enhance its generalization ability, all without the need for complete retraining.

To elaborate, our proposed framework begins with an approach rooted in generalization theory to identify a potential set of detrimental data with the assistance of a clean data set, while simultaneously updating the model by mitigating the influence of the detrimental data during training. Subsequently, we introduce an energy-based algorithm [18] to align the identified detrimental data with the clean data. Through this alignment process, we selectively filter out harmful information that could adversely impact the model's generalization performance, while retaining the beneficial information that can bolster the model's overall generalization capabilities. By adopting this approach, we aim to strike a balance between effectively utilizing the beneficial aspects of the detrimental data and minimizing its negative impact on the model's performance. Our proposed framework offers a lightweight alternative to retraining the model from scratch, enabling the model to learn from both the target and aligned detrimental data, thereby improving its ability to generalize effectively.

It is crucial to acknowledge that identifying a perfect detrimental dataset from the training data is an arduous task, if not an impossible one. Previous approaches may discard useful information if certain target data instances are mistakenly identified as detrimental. In our proposed framework, we mitigate this issue by employing an energy-based alignment algorithm to reduce the divergence between the aligned detrimental data and target distributions. In essence, the alignment process serves as a corrective mechanism for potential errors introduced during the detrimental data identification step. By aligning the identified detrimental dataset with the clean data, our framework can rectify misidentifications and ensure that the model benefits from the valuable information contained within the misidentified instances. By leveraging the alignment process, our framework achieves a higher level of resilience, ensuring that the model's generalization performance remains intact, even in the presence of potential misidentifications. This robustness sets our proposed approach apart from previous methods and reinforces its effectiveness in dealing with the challenges associated with identifying and utilizing detrimental data.

Summary of contribution : We introduce a novel framework that acknowledges the potential value of detrimental data in enhancing the training of a more generalized model alongside target data. Our framework aligns the identified detrimental data with clean data, enabling us to leverage this aligned data to repair the DNN model and improve its generalization ability without the need for complete retraining. This approach strikes a balance between utilizing the beneficial aspects of detrimental data and minimizing its negative impact. By rectifying potential misidentifications through alignment, our framework maximizes the utilization of both target and aligned detrimental data, resulting in a more robust and resilient approach compared to previous methods for addressing the challenges associated with utilizing detrimental data. We conducted extensive experiments on real-world datasets to validate the effectiveness of our proposed framework, demonstrating its superiority over existing methods in identifying harmful data and repairing the model to enhance generalization.

2 Methodology

Problem Settings : Assume our goal is to maximize model performance on some target data distribution Z and we have trained a prediction model ˆ θ by minimizing the empirical risk L ( D ; θ ) over a training set D = { z i } N i =1 consisting of N data pairs z i = ( x i , y i ) . While most of the training data are i.i.d. samples from the same distribution as the target data, a small set of training data is corrupted and assumed to be sampled from a detrimental data distribution Z c . We denote the detrimental data in the training set by D c and the remaining by D l . Notably, the existence of detrimental data is unknown during data collection and model training.

Due to the existence of detrimental data, the model absorbs unexpected adverse information which adversely affects its generalization ability, resulting in inaccurate predictions. In this paper, we focus on developing a model repair algorithm that aims to improve performance by removing the detrimental information from the data that causes prediction failures. To facilitate model repair, we assume the availability of a small set D r of clean data that is reserved for validating the model's success or failure and remains concealed from the model during training. And we identify and remove the detrimental information in the data by leveraging the crucial information from these clean data.

Similar to [33], we formulate the process of model repairment as two steps:

Step 1: Cause Identification. Given a reserved set D r from the target distribution, identify a set of detrimental data points in the training set D that contribute to the prediction failures for data in D r .

Step 2: Model Treatment. Given the identified failure cause set, denoted by ˆ D c , refine the model to effectively correct the prediction failures in the target distribution while preserving the performance on remaining

2.1 Step 1: Cause Identification

In order to detect the detrimental data points in the training set, it is necessary to first define a measure that quantifies the extent to which a subset of training data contributes to the failed predictions in the reserved set D r . Then, based on this measure, we can screen out the data points with high failure contributions as the identified data.

Before introducing the measure, it is important to note that the data from detrimental dataset D c lead to generalization failure because they introduce a mismatch between train and test distribution. This is aligned with the generalization theory for mixture distributions [17, 38] which asserts that the degree of divergence between the two distributions impacts the generalization error bound. Additionally, the theory demonstrates that an increase in the amount of data from the target distribution during training reduces the generalization bound and decreases the loss of data from the same distribution. Besides, increasing the amount of target distribution data also enables the learned model to fit more closely to the target distribution and less to the detrimental distribution, thereby increasing the loss of detrimental data in D c . Since the reserved set D r is drawn from the target distribution, we can train the target-distribution-enhanced model, denoted ˆ ϕ on the combined set of D and D r , and identify the data from D c based on the change of loss between the original trained model ˆ θ and the enhanced model ˆ ϕ . The measure is defined as

By estimating s ( z ; ˆ θ, ˆ ϕ ) for each data point z ∈ D , we can identify the data points responsible for the failure, which have a positive value of s ( z ; ˆ θ, ˆ ϕ ) . If we know the size of the cause set, we can identify the cause data by selecting the top |D c | data points with the largest s ( z ; ˆ θ, ˆ ϕ ) values. We denote the identified data set by ˆ D c , and the remaining data set by ˆ D l .

2.1.1 Obtaining the Enhanced Model ˆ ϕ

Considering the large scale of data, training a new model ˆ ϕ from scratch is computationally prohibitive. Instead, a practical solution is to approximate the model trained on the combined dataset of D and D r by performing fine-tuning or continual learning on the well-trained ˆ θ from D using the data in D r . However, due to over-parameterization of the network and the small size of the reserved set D r , a fine-tuned model may simply memorize the data in D r [41], resulting in a negligible impact on the

Training Set D

Reserved Set D

RemainingSet D

i① Cause Set Identification

②Energy-basedAlignment1

Updated

①

Model

Updated

2

Model

Identified

caused set Dc

Repaired

Model θ

loss of the data in the original training set D . This makes the increase of loss s ( z ; ˆ θ, ˆ ϕ ) ineffective in identifying the detrimental data. On the other hand, a fine-tuned model may completely overfit D r while forgetting the knowledge acquired from D [27], which can cause it to fail in approximating ˆ ϕ .

Given that different layers in a neural network capture distinct patterns [40], selectively updating a subset of layers while freezing the remaining layers can update the detrimental knowledge from D c to effective knowledge in D r , while preserving the acquired knowledge from D . After updating the conflict knowledge from D c , the new model may result in a more significant increase in loss on D c . Recent studies [27, 15] support the layer-selective update by observing that selectively fine-tuning a subset of layers improves the fine-tuning generalization for different distribution shifts towards the target distribution. And we empirically validate the effectiveness of layer-selective update in acquiring the knowledge from D r and improving the identification accuracy, as presented in Appendix A.4.

To enable layer-selective update, a criterion for layer selection is required. Note that neural networks have a tendency to minimize loss by learning data patterns in the early epochs, but may end up memorizing the data in the later epochs [4, 9]. Therefore, we assume that the reduction in loss after the first epoch can serve as an indicator of the ability of the selected layers to learn data patterns. Based on this assumption, we estimate the loss after performing a one-epoch update for each subset of layers and select the subset with the smallest loss for further updates. Subsequently, the selected subset is updated while the remaining layers are kept frozen. As the reserved set is small, the criterion for layer selection via loss estimation after a one-epoch update is computationally efficient. It should be noted that in practice, we consider only a pre-defined set of layer subsets, rather than all possible combinations of subsets.

Connection to existing works: EWC-influence, as proposed in [33], utilized the Bayesian view of continual learning to identify the cause data. Specifically, it defined the identification metric as the drop in log-likelihood between a model trained on the combined dataset D and D f , which is a subset of D r containing examples that are mispredicted by ˆ θ , and the current model ˆ θ trained on D . By using negative log-likelihood as the loss function, replacing D r with D f , and applying the continual learning algorithm Elastic Weight Consolidation (EWC) [19] to update all layers, our proposed identification measure recovers the EWC-influence introduced in [33].

Both EWC-influence and the proposed method measure the contribution of a training point to the prediction failure by examining the change in the loss. However, the two measures are derived from different purposes. EWC-influence aims to correct the prediction in D f of using the gain loglikelihood improvement estimated by deleting the data from D , whereas the proposed method aims to improve the generalization to the data from the same distribution of D r by eliminating detrimental information from D . Consequently, the proposed method proposes to measure the contribution to failure by assessing the conflict between D and D r and update the model using all data in D r .

Linear influence [20, 36, 23] is another effective metric to identify the training points that are most responsible for certain predictions. As analyzed in [33], the linear influence is the same as EWCinfluence with the exception that it updates the model via a single update of natural gradient descent. Consequently, our proposed method can recover the linear influence in a similar manner.

2.2 Step 2: Model Treatment

In step 1, we identify a partition of the training data ˆ D c that may contain harmful information leading to a suboptimal performance to the test distribution Z . In this section, we present the second step of our method, which involves refining the model using the identified cause set ˆ D c and the trained model ˆ θ in conjunction with the enhanced model ˆ ϕ . A straightforward solution for repairing the model is to eliminate the harmful information in ˆ D c by removing the identified data from the training set by retraining the model using the remaining data [12] or performing machine unlearning [29] on the identified data [33]. However, identifying a perfect detrimental dataset from the training data is difficult. Simply erasing the information of identified data can negatively impact the model's performance, especially when accurate data is mistakenly identified as detrimental and removed. Furthermore, even the detrimental data contain useful information. For example, an incorrectly annotated image contains a clean input. Recall that the generalization failure is caused by the mismatching between the detrimental distribution and target distribution. Inspired by the domain adaptation framework [35, 30], we propose to align the detrimental data with the target distribution, which can reduce the divergence between the training distribution and the target distribution. The aligning process can rectify the detrimental information in the data, making it possible to reuse the aligned data toward better performance. A theoretical analysis of the following Gaussian mean estimation task supports the potential of this approach to improve performance.

Gaussian mean estimation task: Consider the task where the objective is to estimate the mean of a Gaussian distribution U = N ( µ, σ 2 ) based on a training set D consisting of n training data { x i } n i =1 sampled from the distribution U as well as m training data { x i } n + m i = n +1 sampled from another Gaussian distribution U ′ = N ( µ ′ , σ 2 ) , where µ and µ ′ are the means of the two Gaussian distributions and σ 2 denotes the variance. We define the loss function as L ( x ; θ ) = ( θ -x ) 2 . In this example, we assume n is sufficiently large than m and the distribution divergence between U and U ′ , meausured by | µ -µ ′ | , is sufficiently large, i.e. | µ -µ ′ | > m + n m α ( √ 1 /n -√ 1 / ( n + m )) , such that the training data from U ′ hurt the generalization performance of the model.

The optimal solution is θ ∗ = µ . The empirical risk minimization solution is obtained by minimizing L ( D ; θ ) = 1 n + m ∑ n + m i =1 ( θ -x i ) 2 as ˆ θ = 1 m + n ∑ n + m i =1 x i . We measure the generalization error as the difference between the empirical solution ˆ θ and the optimal solution θ ∗ . Using Gaussian tail bounds [34], with probability at least 1 -p for some probability, the generalization error is upper bounded as

where α = σ √ 2 log(2 /p ) is some constant.

Assume we have identified the failure cause set as { x i } n + m i = n +1 . Directly removing the cause set and retraining the model achieves a generalization error bound as B remove = α √ 1 /n < B .

Proposition 2.1. In the Gaussian mean estimation task, there exists a set of alignment functions: h ( x ) = x -µ ′ + ˜ µ where | µ -˜ µ | << | µ -µ ′ | such that the optimal empirical solution on the combined set of { x i } n i =1 and { h ( x i ) } n + m i = n +1 , i.e., ˜ θ = 1 m + n ( ∑ n i =1 x i + ∑ n + m i = n +1 h ( x i ) ) , achieves a generalization bound B align < B remove .

The detailed proofs for the bound can be found in Appendix B. The alignment function h ( x ) = x -µ ′ + ˜ µ aligns the data in U ′ with U , removing the detrimental information µ ′ and enhancing the beneficial information ˜ µ . Upon alignment, the resulting data set, { h ( x i ) } n + m i = n +1 , approximates the target distribution closely, and utilizing it in training will enhance generalization performance.

We extend the idea of alignment-based model treatment from the Gaussian mean estimation task to general tasks. Concretely, we propose to perform alignment on the identified cause data to obtain the aligned data using an Energy-Based Model (EBM) [25], building upon the energy-based alignment proposed in [18]. EBMs represent a probability density p ( z ) for z ∈ ❘ d as p θ ( z ) = exp( -E θ ( z )) Z ( θ ) , where E θ ( z ) is the energy function which takes z as input and returns a scalar as energy, and Z ( θ ) = ∫ z exp( -E θ ( z )) is the normalizing constant. To align a given data point z ′ with p θ ( z ) , we iteratively adapt z ′ to minimize the energy function [18] as

Algorithm 1 Beneficial Information Retaining (BIR) Model Repair

1: Input: training data D , reserved data D r ; trained model ˆ θ , a pre-defined candidate set of layer subsets. # Step 1: Cause Identification

2: Estimate the loss after a one-epoch update of ˆ θ on D r for each layer subset in the pre-defined set while fixing the remaining layers

3: Select the corresponding layer subset to be updated with the smallest loss

4: Obtain ˆ ϕ by updating the selected layers of model ˆ θ on D r

5: Estimate s ( z ; ˆ θ, ˆ ϕ ) for each data z ∈ D

6: Identify the failure cause set as ˆ D c ←{ z ∈ D : s ( z ; ˆ θ, ˆ ϕ ) > 0 } # Step 2: Model Treatment

7: Perform data alignment for data in ˆ D c and obtain the aligned cause set ˜ D c

8: Repair the model by updating the selected layers of model ˆ θ on ˜ D c

where η is a hyperparameter that controls the adaptation rate. In contrast to existing works [18] which trains an extra EBM to perform alignment, we adopt the approach proposed in [7], which utilizes the classification model θ as an energy-based model (EBM) to represent the data distribution density as p θ ( x, y ) = exp( f θ ( x )[ y ]) Z ( θ ) , where f θ is a classification model which maps each data point x to the logits of K classes, f θ ( x )[ y ] denotes the logit corresponding to the y -th class and Z ( θ ) is the unknown normalization constant.

We use target-distribution-enhanced model ˆ ϕ to approximately characterize the distribution Z . Considering potential label noise, we update x using the energy function of p ˆ ϕ ( x ) instead of p ˆ ϕ ( x, y ) , where we obtain p ˆ ϕ ( x ) = ∑ y ′ p ˆ ϕ ( x, y ′ ) = ∑ y ′ exp( f ˆ ϕ ( x )[ y ′ ]) Z ( ˆ ϕ ) by marginalizing out y . The corresponding energy function is defined as

With the energy function, we align the data ( x, y ) ∈ ˆ D c by first aligning the input x by one-step gradient descent as

then aligning the label y by which is the soft label of ˜ x .

After obtaining the aligned data, we update the model from ˆ θ using the aligned data. Similar to the update of enhanced model ˆ ϕ we update only the layers selected in the cause identification step while keeping the remaining layers frozen.

Finally, the proposed model repair method, named Beneficial Information Retaining (BIR) Model Repair, including the cause identification and model refinement, is summarized in Algo. 1 and illustrated in Fig. 1.

3 Related Works

Our work is closely related to [33], following which we formulate the model repairment problem into a cause identification step and a model treatment step. In [33], Tanno et al. discuss the model repairment problem from a Bayesian perspective and introduce EWC-influence for cause identification and EWC-deletion for model treatment which performs continual unlearning to remove the data. The cause identification relies on data influence estimation, which can be solved by various influence estimation methods [10] such as linear influence [20, 22], SGD influence [12] and Data Shapley Value [6, 16]. For model treatment, existing works [12, 23] proposes to remove the information of the failure cause data by retraining the model with the remaining data or by performing machine unlearning [8, 29]. RDIA [23] proposes an influence-based relabeling framework for reusing harmful training samples. However, RDIA's aggressive relabeling of benign data that is mistakenly identified as detrimental can introduce incorrect information and negatively impact the model's performance,

making it a high-risk approach in the case of imperfect detrimental data identification. Our proposed approach BIR also reuses the identified data to improve model performance. By aligning the identified detrimental dataset with the clean data, our framework rectifies the detrimental information while preserving the beneficial information in the identified set, and updates the model using the aligned data. Unlike RDIA, our alignment strategy does not introduce incorrect information when applied to clean data and achieves satisfactory performance even in cases where there are mistakenly identified clean data in the cause set.

4 Experiments

To evaluate the effectiveness of the proposed BIR model repair algorithm, we conduct extensive experiments to answer the following research questions: Q1 : How does BIR compare to the stateof-the-art baselines in repairing neural networks that produce error predictions? Q2 : How does the proposed failure cause data identification metric compare to the baselines in identifying the failure cause data? Q3 : How does the proposed align-then-update strategy compare to the baselines in model treatment when considering both perfect and imperfect failure cause sets?

Datasets and Network Architectures: We evaluate the effectiveness of model repair algorithms on corrupted versions of MNIST [26] and CIFAR-10 [24] datasets following the setups in [33]. We consider annotation error, input noise, and adversarial attacks in the training data. To simulate annotation error, we randomly flip about 40% of labels between specific similar digits or classes. For input noise, we randomly corrupt 40% of MNIST images and 30% of CIFAR-10 images in a selected set of target classes by adding specific types of noise, such as salt-and-pepper in MNIST or Gaussian noise in CIFAR-10. Additionally, we construct datasets by adversarially attacking 40% of randomly selected training images in the same target classes using FGSM. We denote the corrupted datasets with label noise, input noise, and adversarial attacks by MNIST-Label, MNIST-Input, and MNIST-Adv

0

700

1400

2100

2800

0

90

180

270

360

450

Number of Data Inspected

Number of Corrupted Data Detected

(a) MNIST-Label

0

700

1400

2100

2800

0

90

180

270

360

450

Number of Data Inspected

Number of Corrupted Data Detected

(b) MNIST-Input

0

700

1400

2100

2800

0

90

180

270

360

450

Number of Data Inspected

Number of Corrupted Data Detected

(c) MNIST-Adv

0

9k

18k

27k

36k

45k

0

1500

3000

4500

6000

7500

Number of Data Inspected

Number of Corrupted Data Detected

(d) CIFAR10-Label

0

9k

18k

27k

36k

45k

0

1200

2400

3600

4800

6000

Number of Data Inspected

Number of Corrupted Data Detected

(e) CIAFR10-Input

0

9k

18k

27k

36k

45k

0

1500

3000

4500

6000

7500

Number of Data Inspected

Number of Corrupted Data Detected

(f) CIFAR10-Adv

(CIFAR10-Label, CIFAR10-Input, and CIFAR10-Adv). For MNIST, 3000 samples from the original training set are utilized while the complete original training dataset of CIFAR-10 is employed to train the base models, with a validation set consisting of 10% of the data. Furthermore, we partitioned the original test set, which is uncorrupted, into a reserved set consisting of 300 samples for MNIST and 1000 samples for CIFAR-10, while the remaining set was designated as the testing set. The selected target classes of MNIST are 1,7, 6, and 9; and the selected target classes of CIFAR-10 are 'plane', 'bird', 'cat', and 'dog'. LeNet and Conv4, which is a 4-layer CNN with a fully connected layer, are employed as classifiers for MNIST, while ResNet-20 and ResNet-50 are utilized for CIFAR-10. We also conduct experiments on CIFAR-10N dataset [37], which equips the training set of CIFAR-10 with real-world human-annotated noisy labels. There are 3 noise levels: Worst ( ∼ 41%), Random ( ∼ 18%), and Aggregate ( ∼ 9%).

Baselines: We compare the proposed method with EWC Repair [33], RDIA [23], SGD-influence Repair [12] and Linear-influence-based Repair. EWC Repair utilizes EWC-influence to identify harmful data and EWC-deletion to remove them from the model. RDIA uses linear influence [20] to identify causes, relabel the identified data using an influence-based relabeling function, and retrains the model using the relabeled and remaining data. SGD-influence Repair identifies harmful data using SGD-influence and proposes to retrain the model using the remaining data. Linear-Influence Repair uses the linear influence function [20] to identify the cause data and performs Newton-update removal [8] to eliminate the identified data's information from the model. In practice, retraining the model from scratch can be computationally prohibitive. Therefore, for RDIA, we performed fine-tuning by using both the relabeled data and an equivalent amount of randomly selected data from the remaining dataset. For SGD-influence Repair, fine-tuning was conducted on a small partition of the remaining data that matched the size of the identified data.

Performance Measure : We measure the performance of model repair by the accuracy of the entire test set (denoted by 'Test'). Besides, we measure the success rate of the repair by the test accuracy of the selected corrupted classes (denoted by 'Noisy') and the performance maintaining rate by the test accuracy of the remaining classes (denoted by 'Clean').

Additional information regarding the setups

4.1 Comparison to Model Repairment Baselines

The comparison results for corrupted MNIST and CIFAR-10 are provided in Tabs. 1 and 2. We can observe from the results that the proposed BIR repair algorithm successfully repairs the model and improves the performance of noisy classes in all experiments except for MNIST-Input with Conv4. In contrast, most baselines fail to repair the model for MNIST-Input, MNIST-Adv and CIFAR10-Adv.

Furthermore, BIR achieves the best performance in noisy classes on 10/12 experiments, and the best performance in all classes on 8/12 experiments. These results demonstrate the effectiveness and superiority of BIR in removing detrimental information from the data to repair the model. We also observe that performance in clean classes of all baselines is generally slightly lower than the base model, which is a trade-off for repairing the corrupted data. The comparison results for CIFAR-10N in Tab. 3 show that our method achieves superior performance across all noise levels, demonstrating its effectiveness in mitigating real-world label noise.

4.2 Comparison of Failure Cause Identification

We compare the proposed causes identification method using Eqn. (1) with EWC-influence [33], Linear-influence [20] and SGD-influence [12]. To measure the performance of the failure cause identification, we report the number of identified data points corresponding to samples corrupted with noise. The results in Figure 2 show that all methods perform well when a small number of data points are identified. Our method performs the best overall, except on MNIST-Input. It is worth noting that our method identifies almost all the corrupted data points in MNIST-Label and CIFAR10-Label, where the model performance is significantly impacted by the detrimental information in the training data. However, it is also important to note that the identification is not perfect, and a sufficiently large set of clean data points may be identified as detrimental data. Therefore, the model repair baselines are at risk of removing useful information from the identified set and may fail to repair the model. In contrast, BIR can effectively remove the detrimental information while retaining beneficial information from the identification, achieving satisfactory performance.

4.3 Comparison of Model Treatment

We evaluate model treatment algorithms given both precise and imprecise identified cause sets, with the latter containing 75% corrupted data and 25% clean data. The baselines include EWCdeletion [33], RDIA [23], and Newton-update removal [8] and fine-tuning of randomly selected data from remaining set as baselines. In addition, we employ retraining the model from scratch using the remaining data as a further baseline.

The results are reported in Tabs. 4 and 5. We can observe performance improvement compared to the results reported in Tabs. 1 and 2. And we also observe a decline in performance for most methods

when replacing the precise identified set with the imprecise one, particularly on MNIST-Label and CIFAR10-Label. These results indicate that the performance of the model treatment baselines is contingent on the correctness of identified set. Notably, the performance drop of BIR is significantly less than that of the baselines, illustrating the effectiveness and superiority of BIR in retaining beneficial information and eliminating harmful information from the identified data to repair the model, even when the identified set comprises clean data.

4.4 Extra Experiment Results

We investigate the effect of the layer-selective update strategy on cause identification and model treatment. The results and analysis are presented in Appendix A.4. Moreover, we explore the solution for determining the appropriate size of the cause set when the ground-truth size is unknown and evaluate the effect of the size on performance in Appendix A.5. Additionally, in Appendix A.6, we studied the effect of varying the amount of clean reserved data on our method's performance.

5 Conclusion and Limitations

In this paper, we propose a novel framework for repairing neural networks trained with detrimental data. To identify the data that cause generalization failure, we propose to update the model with a reserved set sampled from the target distribution, which reduces the loss of clean data and increases the loss of corrupted data. After identifying the cause data, we acknowledge the potential value of detrimental data in enhancing the training of a more generalized model alongside target data and propose an energy-based alignment to remove harmful information while retaining beneficial information in the identified data. After obtaining the aligned data, we update the model on these data to repair the model. We conducted extensive experiments on real-world datasets to validate the effectiveness of our proposed framework, demonstrating its superiority over existing methods in identifying harmful data and repairing the model to enhance generalization.

Limitation: the proposed method entails access to the training data for model repairing purposes, a requirement that may be hampered by the possibility of discarding the data after training. Nonetheless, such accessibility is essential to the method's development, which is predicated on the assumption that erroneous training data engenders generalization failure, and thus seeks to remove the adverse information to effect model repair. We leave the investigation of training data-free model repair to the future.

References

Devansh Arpit, Stanisław Jastrz˛ ebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. A closer look at memorization in deep networks. In International conference on machine learning , pages 233-242. PMLR, 2017.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Mohamad Dolatshah, Mathew Teoh, Jiannan Wang, and Jian Pei. Cleaning crowdsourced labels using oracles for statistical classification. Proceedings of the VLDB Endowment , 12(4):376-389, 2018.

Jonathan Frankle, David J Schwab, and Ari S Morcos. The early phase of neural network training. arXiv preprint arXiv:2002.10365 , 2020.

Benoît Frénay and Michel Verleysen. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems , 25(5):845-869, 2013.

Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning , pages 2242-2251. PMLR, 2019.

Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. International Conference on Learning Representations , 2020.

Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030 , 2019.

Guy Hacohen, Leshem Choshen, and Daphna Weinshall. Let's agree to agree: Neural networks share classification order on real datasets. In International Conference on Machine Learning , pages 3950-3960. PMLR, 2020.

Zayd Hammoudeh and Daniel Lowd. Training data influence analysis and estimation: A survey. arXiv preprint arXiv:2212.04612 , 2022.

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems , 31, 2018.

Satoshi Hara, Atsushi Nitanda, and Takanori Maehara. Data cleansing for models trained with sgd. Advances in Neural Information Processing Systems , 32, 2019.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations , 2019.

Pei-Yun Hsueh, Prem Melville, and Vikas Sindhwani. Data quality from crowdsourcing: a study of annotation selection criteria. In Proceedings of the NAACL HLT 2009 workshop on active learning for natural language processing , pages 27-35, 2009.

Long-Kai Huang, Junzhou Huang, Yu Rong, Qiang Yang, and Ying Wei. Frustratingly easy transferability estimation. In International Conference on Machine Learning , pages 9201-9225. PMLR, 2022.

Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 1167-1176. PMLR, 2019.

Sharu Theresa Jose and Osvaldo Simeone. Information-theoretic bounds on transfer generalization gap based on jensen-shannon divergence. In 2021 29th European Signal Processing Conference (EUSIPCO) , pages 1461-1465. IEEE, 2021.

KJ Joseph, Salman Khan, Fahad Shahbaz Khan, Rao Muhammad Anwer, and Vineeth N Balasubramanian. Energy-based latent aligner for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7452-7461, 2022.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017.

Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning , pages 1885-1894. PMLR, 2017.

Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning , pages 5637-5664. PMLR, 2021.

Pang Wei W Koh, Kai-Siang Ang, Hubert Teo, and Percy S Liang. On the accuracy of influence functions for measuring group effects. Advances in neural information processing systems , 32, 2019.

Shuming Kong, Yanyan Shen, and Linpeng Huang. Resolving training biases via influencebased data relabeling. In International Conference on Learning Representations , 2021.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energybased learning. Predicting structured data , 1(0), 2006.

Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist , 2, 2010.

Yoonho Lee, Annie S Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. International Conference on Learning Representations , 2023.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299 , 2022.

Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowledge and data engineering , 22(10):1345-1359, 2010.

Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1944-1952, 2017.

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 , 2019.

Ryutaro Tanno, Melanie F Pradier, Aditya Nori, and Yingzhen Li. Repairing neural networks by leaving the right past behind. Advances in Neural Information Processing Systems , 35:1313213145, 2022.

Martin Wainwright. Chapter 2: Basic tail and concentration bounds. 210B Lecture Notes University , 2015.

Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomputing , 312:135-153, 2018.

Zifeng Wang, Hong Zhu, Zhenhua Dong, Xiuqiang He, and Shao-Lun Huang. Less is better: Unweighted data subsampling via influence function. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 6340-6347, 2020.

Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. arXiv preprint arXiv:2110.12088 , 2021.

Xuetong Wu, Jonathan H Manton, Uwe Aickelin, and Jingge Zhu. Information-theoretic analysis for transfer learning. In 2020 IEEE International Symposium on Information Theory (ISIT) , pages 2819-2824. IEEE, 2020.

Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K Jain. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing , 17:151-178, 2020.

Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems , 27, 2014.

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM , 64(3):107115, 2021.

A Omitted Experiment Details in Sec. 4 and Extra Experiments

A.1 Dataset

MNIST [26] dataset contains 60,000 training images and 10,000 test images of 10-digit classes from 0 to 9, with each image in the size 28 × 28 . We follow [33] to randomly select 3,000 images from the original training set to construct the training set. And we randomly choose 300 images from the original test set to construct the reserved set and use the remaining data as the test set for performance evaluation.

We construct three corrupted versions of MNIST dataset by introducing label noise, input noise, and adversarial attack on the clean training set. Specifically, for MNIST-Label , we randomly select 40% of the data from classes 1 and 7, and flipped their labels to each other; and the same for classes 6 and 9. For MNIST-Input , we randomly corrupted 40% of the data from the target classes 1, 7, 6, and 9 using salt and pepper noise. To create MNIST-Adv , we first train a victim model on the clean training set and then generate adversarial images using the fast gradient sign method (FGSM) for 40% of the data randomly selected from the target classes. The poisoned images in MNIST-Adv are labeled by the classes predicted by the victim model.

CIFAR-10 [24] comprises 60,000 color images divided into 10 categories, each with dimensions of 32 × 32 . The training set has 50,000 images while the original test set is split into a reserved set with 1,000 images and a test set with 9,000 images. We introduce three corrupted versions of the CIFAR-10 dataset, where label noise, input noise, and adversarial attacks are introduced on the target classes: plane, bird, cat, and dog. Specifically, we generate the CIFAR10-Label dataset by randomly selecting 40% of the plane and bird class images and swapping their labels, and similarly for the cat and dog classes. For CIFAR-Input , we apply Gaussian noise to 30% of the images from the target classes. Finally, for CIFAR10-Adv , we use adversarial attacks to corrupt 40% of the randomly selected images from the target classes, similar to MNIST-Adv.

CIFAR-10N [37] is CIFAR-10 dataset with human-annotated noisy labels. In CIFAR-10N dataset, each training image has 3 human-annotated labels. It has 5 noisy-label sets: Random i ( i ∈ { 1 , 2 , 3 } ) uses the i -th submitted label for each image; Aggregate generates the label by majority voting of three noisy labels; Worst randomly selected one label from the wrong labels if there exist any wrongly annotated labels in 3 noisy labels. In the experiments, we adopt 3 out of the 5 noisy-label sets, namely, Aggregate , Random 2 and Worst .

A.2 Network Architectures

For the corrupted MNIST datasets, we use LeNet and Conv4 as the classifier. LeNet is widely used for the tasks on MNIST. It consists of 2 convolutional layers and 3 fullyconnected layers. In the layer selective update, we define Layer 1 to Layer 5 as the first (the first convolutional layer) to the last layer (the last fully-connected layer). The set of candidate layer subsets considered in the layer selective update stage comprises {{ Layer 1 } , { Layer 2 } , { Layer 3 } , { Layer 4 } , { Layer 5 } , { Layer 1-2 } , { Layer 1-3 } , { Layer 1-5 }} . Conv4 consists of 4 convolution layers followed by a fully-connected layer. Similar to LeNet, we define Layer 1 to Layer 5 as the first to the last layer, respectively, and consider the same candidate layer subsets.

The corrupted CIFAR-10 datasets are classified using ResNet-20 and ResNet-50. The CIFAR10N dataset also uses ResNet-20 as classifier. ResNet-20 comprises one convolutional layer, 3 compositional layers, each of which consists of 3 basic blocks, and a fully-connected layer. Layer 1 is defined as the first convolutional layer, Layers 2 to 4 are the 3 compositional layers, and Layer 5 is the fully-connected layer. The same candidate layer subsets are considered. ResNet-50 consists of one convolutional layer, 4 compositional layers, each of which consists of 3, 4, 6, and 3 bottleneck blocks, respectively, and a fully-connected layer. Layer 1 is the first convolutional layer, Layers 2 to 5 are the 4 compositional layers, and Layer 6 is the last fully-connected layer. Its candidate layer subsets are defined as {{ Layer 1 } , { Layer 2 } , { Layer 3 } , { Layer 4 } , { Layer 5 } , { Layer 6 } , { Layer 1-2 } , { Layer 1-3 } , { Layer 1-6 }} . For ResNet-50, The input images are resized to 224 × 224 .

LeNet and Conv4 are both trained from scratch. The ResNet-20 and ResNet-50 model is initialized with the pre-trained model on ImageNet.

A.3 Training Details

In this section, we introduce the settings in training. Specifically, the training set is partitioned into 90% training data and 10% validation data. The base model is trained using the Adam optimizer with a learning rate of 1e-3 and a batch size of 64 for all datasets. And the model is trained for a maximum of 100 epochs, and early stopping after 5 epochs of no progress on the validation data.

To obtain the enhanced model, we update the model on all data in the reserved set using the Adam optimizer with a learning rate of 1e-3 and a batch size of 16. Training is stopped after no progress on the training loss. The update layers are selected based on the training loss estimated after a one-epoch update, i.e., an update going through all reserved data. Once the enhanced model is obtained, we estimate the score in Eqn. (1) for all training data and identify the cause data by selecting the top ˜ K data points with the highest s ( z ; ˆ θ, ˆ ϕ ) values. Here, ˜ K is equal to the size of the ground-truth cause set, i.e., |D c | . For the baseline identification methods, the same number of data points, i.e. ˜ K are selected as the cause data.

During model treatment, we first perform energy-based alignment on the identified data using the enhanced model. Specifically, the data input is updated using Eqn. (4) with η = 1 . 0 and the label is updated using Eqn. (5). We then update the model using the aligned data and ˜ K additional data points that are randomly sampled from the remaining training set. The purpose of the additional data from the remaining set is to alleviate bias toward the identified data. It is worth noting that all baseline methods perform model treatment using both identified data and randomly sampled remaining data. In the proposed method, we initialize the model with ˆ θ and optimize it using the Adam optimizer with a learning rate of 1e-3. For the corrupted MNIST datasets and CIFAR-10 datasets, we set the batch size to 16 and 64, respectively. We terminate the training process when there is no further decrease in the training loss.

A.4 Extra Experiment: The Effect of Layer-Selective Update

In this section, we investigate the effect of the layer-selective update strategy on cause identification and model treatment. Performances are presented in Tabs. 6 and 7, where the AUC between the cause score of the training data calculated by Eqn. (1) and the ground-truth label indicating whether the data is from D c is used to measure identification performance, and the accuracy of the test set of the repaired model with different layers updated is used to evaluate model treatment performance. The

results indicate that updating all layers is suboptimal for both identification and repair performance in most cases. Thus, a layer-selective update is necessary to achieve better performance. We suggest selecting the layers for update based on the loss after one-epoch update on the reserved set. The proposed method achieves the best identification performance in 4/6 cases and the best treatment performance in all datasets except MNIST-Adv, where it is only 0.1% less accurate than the best one. These results highlight the effectiveness of the layer-selective update in enhancing the proposed method's performance.

A.5 Extra Experiment: The Effect of Identified Cause Set Size

In our experiments, we assume knowledge of the ground-truth size of the cause set, denoted as N C , and identify the cause data by selecting the top N C data points with the highest score estimated by Eqn. (1). However, in cases where the number of corrupted data examples is uncertain, selecting an appropriate threshold becomes challenging. To address this issue, a practical solution is to apply a fixed threshold on the score ( s ) computed using Eqn. (1) as a heuristic measure. As a straightforward solution, setting the threshold to 0 identifies all training data with an increased loss, but this often results in identifying more data than the actual number of corrupted points, as demonstrated in our empirical studies presented in Tab. 8. Our investigations indicate that a threshold between log(1 . 01) and log(1 . 1) effectively filters out identified data close in size to the ground truth. Thus, we recommend employing a threshold of log(1 . 05) for practical applications.

Importantly, we emphasize that the repair performance of our proposed BIR method is robust to the size of the identified cause set, as evidenced by the experimental results in Tabs. 9 and 10. Therefore, while identification using thresholds may not be perfect, the repair performance of BIR is hardly affected.

A.6 Extra Experiment: The Effect of Clean Reserved Set Size

In this section, we investigate the impact of varying the amount of reserved clean data on the performance of the proposed model repair method when applied to three corrupted MNIST datasets. The identification and repair performance is reported in Tab. 11. Our empirical analysis reveals three key findings: firstly, with a mere 30 clean samples, our approach resulted in enhanced performance on noisy classes, albeit with a lower identification AUC; secondly, as the number of clean samples increased from 30 to 300, both identification AUC and overall performance demonstrated a marked

improvement; and thirdly, our experiments demonstrated that 100 clean samples were sufficient for achieving effective model repair across these datasets.

B Omitted Technical Details in Sec. 2

C The Gaussian Mean Estimation Task

Definition C.1. (Gaussian Mean Estimation) : Given a training set D consisting of n training data { x i } n i =1 sampled from the distribution U = N ( µ, σ 2 ) as well as m training data { x i } n + m i = n +1 sampled from another Gaussian distribution U ′ = N ( µ ′ , σ 2 ) , where µ and µ ′ are the means of the two Gaussian distributions and σ 2 denotes the variance, the objective is to estimate the mean of the target distribution U . The loss function is defined as L ( x ; θ ) = ( θ -x ) 2 .

Lemma C.2. The empirical solution of Gaussian Mean Estimation is ˆ θ = 1 m + n ∑ n + m i =1 x i . With probability at least 1 -p for some probability, the generalization error is upper bounded as

where θ ∗ = µ is the optimal solution and α = σ √ 2 log(2 /p ) is some constant.

Proof. Since { x i } n + m i = n +1 are sampled from U ′ , { x i -µ ′ + µ } n + m i = n +1 are sampled from U . Using Gaussian tail bounds [34], we have that it holds with probability 1 -p that

Therefore, we have

Remark: Note that in cases where the value of m decreases to zero, indicating the absence of any data from the detrimental distribution U ′ in the training set, the upper bound is then given by

If the distribution divergence between U and U ′ , measured by | µ -µ ′ | , satisfies | µ -µ ′ | > m + n m α ( √ 1 /n -√ 1 / ( n + m )) , then B remove < B . This implies that including data from U ′ in the training set negatively affects performance, and removing these data can improve the performance, given that the divergence is sufficiently large.

Proposition C.3. (Recap of Proposition 2.1) In the Gaussian mean estimation task, there exists a set of alignment functions: h ( x ) = x -µ ′ +˜ µ where | µ -˜ µ | << | µ -µ ′ | such that the optimal empirical solution on the combined set of { x i } n i =1 and { h ( x i ) } n + m i = n +1 , i.e., ˜ θ = 1 m + n ( ∑ n i =1 x i + ∑ n + m i = n +1 h ( x i ) ) , achieves a generalization bound B align < B remove .

Proof. For any alignment function h ( x ) , similar to Eqn. (7), we have

If | µ -˜ µ | << | µ -µ ′ | such that | µ -˜ µ | satisfies

we have

Remark: The alignment function aligns the data in U ′ with that in U . If the alignment is effective, that is | µ -˜ µ | < m + n m α ( √ 1 /n -√ 1 / ( n + m )) , it improves performance over the results of data removal. The degree of improvement is positively correlated with the quality of alignment, as quantified by the magnitude of | µ -˜ µ | .

C.1 Cause Identification in the Gaussian Mean Estimation Task

In the Gaussian mean estimation task, we assume that all the data from the detrimental distribution U ′ are identified before model treatment. In this section, we verify the effectiveness of the metric proposed in Eqn. (1) in identifying the cause data in the training set for this task. Specifically, the metric estimates the contribution of failure by measuring the change in loss between the original trained model and an enhanced model, which is trained with a larger sample size n from U .

In the Gaussian mean estimation task, the expected loss of the data from U is

The forth equality uses the fact that ❊ | ξ -❊ ξ + ε | 2 = ❊ | ξ -❊ ξ | 2 + ε 2 -2 ❊ ( ξ -❊ ξ ) ε = ❊ | ξ -❊ ξ | 2 + ε 2 and the fifth equality uses the fact that 1 n ∑ n i =1 ∣ ∣ ∣ x i -1 n ∑ n j =1 x j ∣ ∣ ∣ 2 , 1 n ∑ n j =1 x j and 1 m ∑ n + m j = n +1 x j are unbiased estimation of σ 2 , µ and µ ′ , respectively. Similarly, the expected loss of the data from U ′ is

If we introduce additional ∆ n data to train the enhanced model ˆ ϕ , similar to Eqn. (10), we have

Then we have

Therefore, the detrimental data can be identified by the positive value of s ( x i ; ˆ θ, ˆ ϕ ) .

Furthermore, this metric can be utilized to identify harmful data, even if they are not part of the training dataset. In the Gaussian mean estimation example, similar to Eqn. (7), we can obtain the generalization error bound to the data in U ′ as

As the number of training samples from U increases, both terms in the generalization error bound for U (Eqn. (7)) decrease. In contrast, for U ′ , the first term α √ 1 / ( n + m ) decreases, while the second

term n n + m | µ -µ ′ | increases. Consequently, the generalization error bound for U decreases more significantly than that for U ′ . Given that the loss is related to the generalization bound, an increase in the number of training samples from U leads to a rise in the loss of most samples from U ′ , while causing a decline in the loss of most samples from U . And we can utilize the metric in Eqn. (1) to identify any detrimental data.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to identify harmful data by using the clean dataset, and then update models with the alignment technique to improve the performance on the corrupted data while retaining the performance on the clean data. Experimental results show the effectiveness of the proposed method, demonstrating its superiority over existing methods in identifying harmful data and repairing DNN models.\n\nSTRENGTHS:\n- Aims at an important and relevant problem.\n\n- The experimental results on identifying mislabeled data points are good.\n\nWEAKNESSES:\n- The paper claims that the experiments were conducted on \"real-world datasets\" (Line 89). However, as mentioned in the experiments (Line 275), the corruption cases were simulated. Additionally, the datasets evaluated are relatively small (i.e., MNIST and CIFAR-10). ImageNet dataset could potentially be a valuable supplement, which could contain many mislabeled, real-world samples.\n\n- It would be better to show how much the average improvement is over the baselines, which could help readers better understand the superiority of the proposed method. For the results on input noise and adversarial attacks, the absolute improvements over some baselines are actually quite small.\n\n- Section 1, particularly Line 64 to Line 80, is not clear to me. The alignment process is crucial for the proposed framework. These two paragraphs repeatedly emphasize the benefits of alignment. However, what alignment means is not clearly described. It could be helpful to give some descriptions of how the alignment process works and what the results of the alignment process are.\n\nQUESTIONS:\n- As claimed in the paper (e.g., Line 71 to Line 76), the proposed method has the advantage over existing methods in that it allows the model to learn useful information from the clean data that are misidentified as corrupted data, thus making it more robust compared to other methods. However, from the results presented in Table 3 and Table 4, I noticed that when the experimental setup was \"Identified set = 75% ground-truth cause set and 25% clean data\" in all six cases, there were always baselines that outperform the proposed method in terms of \"Test Clean\". Does this contradict the claims made in the paper?\n\n- Why use ResNet50 on CIFAR-10? ResNet50 is designed for classification on ImageNet, while ResNet56 is designed for CIFAR-10. \n\n- How was the proportion (40% and 30%) of corrupted samples determined in the experiments? How does this setting (different proportions) affect the results? In actual scenarios, would there be such a high number of corrupted samples in the training set? If the proportion is reduced, is Equation (1) still sensitive enough to accurately identify the corrupted data points?\n\n- Line 215: \\sum_{i=1}^n h(x_i) should be \\sum_{i=n+1}^{n+m} h(x_i) ?\n\n- In the caption of Table 1: \"base mode\" -> \"base model\" ?\n\nLIMITATIONS:\nThe authors described the limitations in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method to detect and repair corrupt data in the training set, in an effort to align the model with the distribution of the test set. The method is two-stage, first detecting based on a loss difference computation per-sample, and then repairing through the use of an energy-based model. Several experiments are conducted to demonstrate the efficacy of the approach on multiple types of noisy data.\n\n**Rebuttals:** The authors have addressed my questions in their rebuttal, and I have updated my score from 7 to 8.\n\nSTRENGTHS:\nMethodologically, this paper is excellent, and is very well-written. The motivation is clear and strong; the theory is sound; the experimental evaluation shows, at times, significant improvements over baselines and other approaches.\n\nWEAKNESSES:\nThere are few weaknesses of this paper, mostly comprised of minor errors:\n\n- minor: L95: pair --> pairs\n- minor: L120: mismatching --> mismatch\n- minor: L162: Beysian --> Bayesian\n- minor: L206: minimizaiton --> minimization\n- minor: L226: iterative --> iteratively\n- In Eq. 7, second line, I believe the last term should have a minus sign instead of a plus. This should not change the results because of the next step, though. This also applied to Eq. 8 line 2.\n- minor: Fig. 2 seems to start from (b), please fix\n\nQUESTIONS:\n- Is Equation 1 expensive to perform when running on thousands of samples, with a possibly large model?\n- L132: please address the case where the size of $\\mathcal{D}_c$ is unknown.\n- How reliable is Equation 1? Suppose we have $|\\mathcal{D}_c| = 20$, and so we pick samples from $\\mathcal{D}$ with the 20 highest values for $s$. (a) What if different points of convergence of, say, $\\hat{\\phi}$ yields different top 20 points? (b) What if the difference between $s$ at the 20th and the 21st point is negligible? Is there a way to determine confidence that sample $z \\in \\mathcal{D}_c$?\n- I'm not entirely sure how the equation under L566 follows from Hoeffding's inequality, could you please elaborate?\n- I understand the result of Proposition 2.1, and it seems the next logical step would be to find a $\\tilde{\\mu}$ that satisfies L580 in Appendix B.2. I'm not sure how the next paragraph (starting at L221) relates to this, could you clarify?\n- As future work, it would be interesting to see if BIR also works for more sophisticated adversarial attacks than FGSM, such as Carlini & Wagner [1] or Szegedy et al., 2013 [2]. In addition, if BIR is being positioned as a method to help with adversarial attacks, it might be useful to compare against defenses such as adversarial training (see [3-7] for a few works). It would also be interesting to see Fig. 2(d) and 2(g) with those methods as well.\n\n[1] Carlini, N., & Wagner, D. (2017, May). Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) (pp. 39-57). Ieee.  \n[2] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.  \n[3] Kurakin, A., Goodfellow, I., & Bengio, S. (2016). Adversarial examples in the physical world.  \n[4] Dhillon, G. S., Azizzadenesheli, K., Lipton, Z. C., Bernstein, J., Kossaifi, J., Khanna, A., & Anandkumar, A. (2018). Stochastic activation pruning for robust adversarial defense. arXiv preprint arXiv:1803.01442.  \n[5] Guo, C., Rana, M., Cisse, M., & Van Der Maaten, L. (2017). Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117.  \n[6] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.  \n[7] Samangouei, P., Kabkab, M., & Chellappa, R. (2018). Defense-gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprint arXiv:1805.06605.\n\nLIMITATIONS:\nLimitations have been adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method for performing targeted repairs of corrupted data in order to improve the quality of an ML model trained on that data. The method is comprised of two steps: (1) *cause identification* -- identifying the set of data examples that are the most likely to be corrupted, and (2) *model treatment* -- performing a targeted treatment of the training dataset with the goal of removing the negative effect of the corrupted data examples while still keeping them in the dataset. For cause identification, they augment the target dataset with a special \"reserved\" dataset that is known to be clean, then they train a model on the augmented dataset, and finally for each training data example, they compare the training loss of the model trained on the augmented data and the model trained on the original data. For model treatment, they propose an \"alignment function\" which can be applied to the features and labels of corrupted data examples, which is specifically calculated to neutralize their negative impact on the model training loss function. The authors evaluate their method on the MNIST and CIFAR10 datasets with three types of corruption: (1) label noise, (2) feature noise, and (3) FGSM-based adversarial attack on features. They show that their method performs favorably against several other methods (mainly various influence function methods).\n\nSTRENGTHS:\n(S1) The problem being solved is very relevant and the proposed solution is quite interesting.\n\n(S2) The paper is well-written and I found it relatively easy to follow.\n\n(S3) The experiments show a clear benefit that the method provides over the baselines.\n\nWEAKNESSES:\n(W1) The introduction is only slightly repetitive as some points are delivered multiple times.\n\n(W2) The experiments do not compare against methods based on the Shapley value that are also commonly used for solving this problem.\n\nQUESTIONS:\n(Q1) It seems to me that in the experiments for Q1 and Q3 it is assumed that the number of corrupted data examples is not hidden from the various methods that are being compared. If this is true, what would happen if this information would indeed be hidden as is the case in real-world settings?\n\n(Q2) If you knew which exact data examples are corrupted (i.e. we can skip the cause identification step), what would be the impact of the model treatment step compared to data removal? (perhaps this could be an additional experiment)\n\nLIMITATIONS:\nThe limitations are clearly stated in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a method, BIR, to repair the model trained but possibly corrupted from different types of noise, e.g., label noise and feature noise, leading to training-testing distribution mismatch. As for repair, by assuming the presence of small clean data, the approach identifies corrupted examples using the loss gap between the original and fine-tuned models. Next, it aligns the identified  (potentially) corrupted examples to the target distribution using an energy-based model, followed by fine-tuning with the aligned data. The experiments were conducted on two datasets with multiple variations for different types of noise.\n\nSTRENGTHS:\n**Methodology Design**: I like the method design involving an additional step (Step 2) after Step 1. This step will make a synergistic effect by handling the misidentified cases from Step 1. Also, the alignment process makes sense and is interesting.\n\n**Problem Setup**: Handling corrupted models is very important. This paper proposed a post-processing/training framework for repairment. This setup has an advantage over 'in-training' methods in the sense that we can apply this method to already pre-trained large model with minimal training cost.\n\nWEAKNESSES:\n**Clean reserved data**: The presence of this data makes a concern about the practicality of the proposed method. For example, the existence of a clean subset is reluctant to be assumed in the label noise area. In this sense, the weakness of this paper is the strong dependency on the reserved clean set. At least, It would be good to add the impact of the amount of clean subset, e.g., how much the identification performance varies.\n\n**Synthetic noise**: In the Introduction section, the authors said \"extensive experiments on real-world datasets\". Is it meaning that MNIST and CIFAR-10 datasets are real? I was expecting to see the results on benchmark datasets with real-world labels and feature noise in data. When looking into the experimental setup, the author used a fixed set of synthetic noise if I understood correctly, e.g., 40% of label flipping, 40% of random corruption (feature), etc. This specific setup for each noise is not a broad study of real noise. Additionally, it would be good to mix different types of noise in a single setup and control the extent to which each noise occurs.\n\n**Size of inspected data**: According to Figure 2, I guess that the size of inspected data in Step 1 will affect the performance of the method significantly. If we know the ratio of corrupted data, we can easily find the optimal size of identification as the author mentioned (i.e., selects top |D_{c}| data points with the largest s(z)). However, it is unclear how to judge the size for the experiments in Tables 1 and 2. Is it also assumed to be known? If not, please elaborate on this. \n\n**Datasets**: MNIST and CIFAR-10 datasets are very small and relatively easy datasets. I would recommend using more challenging datasets for experimentation. For example, CIFAR-100, Tiny-ImageNet, or CIFAR-10N (w. real human mistakes in labeling). Particularly, I think the difficulty of data may affect the performance of Step 1. In this scenario, can Step 2 will handle the negative impact of a larger number of misidentified cases?\n\nQUESTIONS:\nI put all the questions in the weakness sections.\n\nLIMITATIONS:\nThere is no expected potential societal and ethical impact. Also, the authors stated the limitations of the work in Section 5.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-08"}