Retaining Beneficial Information from Detrimental Data for Neural Network Repair
Authors: Long-Kai Huang
Peilin Zhao
Junzhou Huang
Sinno Pan
Venue: NeurIPS 2023 poster (2023)
Keywords: Model Repair; Fine-tuning

Abstract: The performance of deep learning models heavily relies on the quality of the training data. Inadequacies in the training data, such as corrupt input or noisy labels, can lead to the failure of model generalization. Recent studies propose repairing the model by identifying the training samples that contribute to the failure and removing their influence from the model. However, it is important to note that the identified data may contain both beneficial and detrimental information. Simply erasing the information of the identified data from the model can have a negative impact on its performance, especially when accurate data is mistakenly identified as detrimental and removed. To overcome this challenge, we propose a novel approach that leverages the knowledge obtained from a retained clean set. Our method first identifies harmful data by utilizing the clean set, then separates the beneficial and detrimental information within the identified data. Finally, we utilize the extracted beneficial information to enhance the model's performance. Through empirical evaluations, we demonstrate that our method outperforms baseline approaches in both identifying harmful data and rectifying model failures. Particularly in scenarios where identification is challenging and a significant amount of benign data is involved, our method improves performance while the baselines deteriorate due to the erroneous removal of beneficial information.

Retaining Beneficial Information from Detrimental Data for Deep Neural Network Repair
Long-Kai Huang', Peilin Zhao', Junzhou Huang", Sinno Jialin Pan
Tencent AI Lab
The University of Texas at Arlington
The Chinese University Hong Kong
hlongkai@gmail com, masonzhao@tencent. com, jzhuang@uta edu, sinnopan@cuhk. edu hk
Abstract
The performance of deep learning models heavily relies the quality of the train- ing data. Inadequacies the training data, such as corrupt input or noisy labels, can lead to the failure of model generalization. Recent studies propose repairing the model by identifying the training samples that contribute to the failure and removing their influence from the model. However, it is important to note that the identified data may contain both beneficial and detrimental information. Simply erasing the information of the identified data from the model can have negative impact on its performance, especially when accurate data is mistakenly identified detrimental and removed. To overcome this challenge, we propose novel ap- proach that leverages the knowledge obtained from a retained clean set. Concretely, Our method first identifies harmful data by utilizing the clean set, then separates the beneficial and detrimental information within the identified data. Finally, we utilize the extracted beneficial information to enhance the model's performance. Through empirical evaluations, we demonstrate that our method outperforms bascline ap- proaches in both identifying harmful data and rectifying model failures. Particularly in scenarios where identification is challenging and a significant amount of benign data is involved, our method improves performance while the baselines deteriorate due to the erroneous removal of beneficial information.
Introduction
The effectiveness of deep neural networks (DNNs) is greatly influenced by the quantity and quality of the training data from which they trained. However, obtaining large amount of high-qualit data can be costly. An alternative approach is to acquire training data from sources such as the Web (2), external data suppliers, crowdsourcing [14] Although this allows for the collection of a substantial volume of data, the quality of the acquired data cannot be guaranteed. Various issues can arise, including annotation errors (3), data corruption [3].
deliberate adversarial manipulation (39] by malicious providers. Consequently, portion of the collected data may have a different distribution than the majority of the data (21]. In this work, we refer these categories of data as "detrimental" and "target" data, respectively. The presence of detrimental data in the training can significantly hinder the performance of DNN I. While recent research has proposed methods to mitigate specific issues like label noise [31 5 ITI. adversarial attacks [28] (39], or data shifts during model training [2T] 32]. it remains challenging train a model that generalizes well to target data when the specific cause of the detrimental data is unknown.
It is important note that in practical scenarios, DNN model must undergo quality assurance test before it can be deployed. During this evaluation process, the performance of the trained DNN model is usually assessed using small reserved set of data known as clean data. This clean data is expected
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

to have the same distribution as the target data. However, since the training process involves the use of detrimental data, the trained DNN model may produce inaccurate predictions on the clean data, leading to a failure in the quality assurance test. In such cases, it becomes necessary to repair the DNN model to successfully pass the quality assurance test.
Recent approaches to model repair focus on two main steps. Firstly, they aim to identify the detrimental data instances within the original training dataset with a set of clean data, a.k.a, test data. Once identified, these instances can be categorized for further action. Secondly, the model is updated by eliminating the identified detrimental data instances. This repair process helps in improving the model's 's performance and ensuring that it meets the desired quality standards for deployment. Notable works in this area include studies by Tanno et al. [3] and Kong et al. [23).
In contrast to previous approaches that focus on eliminating the entire detrimental dataset to learn a repaired DNN model, our approach is motivated by the belief that the detrimental data may contain some valuable underlying information. Once this information can he extracted, it can be leveraged to train more generalized model alongside the target data. Drawing inspiration from domain adaptation techniques [35] 30), we propose framework that aligns the identified detrimental data with the clean data. By aligning these datasets, we can utilize the aligned detrimental data to repair the DNN model and enhance its generalization ability, all without the need for complete retraining.
To elaborate, our proposed framework begins with an approach rooted in generalization theory to identify a potential set of detrimental data with the assistance of a clean data set, while simultaneously updating the model by mitigating the influence of the detrimental data during training. Subsequently, we introduce an energy-based algorithm [8] to align the identified detrimental data with the clean data. Through this alignment process, we selectively filter out harmful information that could adversely impact the model's I generalization performance, while retaining the beneficial information that can bolster the model's overall generalization capabilities. By adopting this approach, we aim to strike a balance between effectively utilizing the beneficial aspects of the detrimental data and minimizing its negative impact on the model's performance. Our proposed framework offers lightweight alternative to retraining the model from scratch, enabling the model to learn from both the target and aligned detrimental data, thereby improving its ability to generalize effectively.
It is crucial to acknowledge that identifying perfect detrimental dataset from the training data is an arduous task, if not an impossible one. Previous approaches may discard useful information if certain target data instances are mistakenly identified as detrimental. In our proposed framework, mitigate this issue by employing an energy- based alignment algorithm to reduce the divergence between the aligned detrimental data and target distributions. In essence, the alignment process serves as corrective mechanism for potential errors introduced during the detrimental data identification step. By aligning the identified detrimental dataset with the clean data, our framework can rectify misidentifications and ensure that the model benefits from the valuable information contained within the misidentified instances. By leveraging the alignment process, our framework achieves higher level of resilience, ensuring that the model's generalization performance remains intact, even in the presence of potential misidentifications. This robustness sets our proposed approach apart from previous methods and reinforces its effectiveness in dealing with the challenges associated with identifying and utilizing detrimental data.
We introduce a novel framework that acknowledges the potential value of detrimental data in enhancing the training of a more generalized model alongside target data. Our framework aligns the identified detrimental data with clean data, enabling us to leverage this aligned data to repair the DNN model and improve its generalization ability without the need for complete retraining. This approach strikes a balance between utilizing the beneficial aspects of detrimental data and minimizing its negative impact. By rectifying potential misidentifications through alignment, our framework maximizes the utilization of both target and aligned detrimental data, resulting a more robust and resilient approach compared to previous methods for addressing the challenges associated with utilizing detrimental data. We conducted extensive experiments on real-world datasets validate the effectiveness of our proposed framework, demonstrating its superiority over existing methods in identifying harmful data and repairing the model to enhance generalization.
Summary of contribution:
2

---

2 Methodology
Problem Settings: Assume our goal is to maximize model performance on some target data distribu- tion 2 and we have trained a prediction model by minimizing the empirical risk ((D; over a training set D = { consisting of N data pairs = (aa, M/.. While most of the training data are i.i.d. samples from the same distribution as the target data, a small set of training data corrupted and assumed to be sampled from a detrimental data distribution Ze. We denote the detrimental data in the training set by De and the remaining by Dj. Notably, the existence of detrimental data is unknown during data collection and model training.
Due to the existence of detrimental data, the model absorbs unexpected adverse information which adversely affects its generalization ability, resulting in inaccurate predictions. In this paper, we focus on developing a model repair algorithm that aims improve performance by removing the detrimental information from the data that causes prediction failures. To facilitate model repair, we assume the availability of a small set D, of clean data that is reserved for validating the model's success or failure and remains concealed from the model during training. And we identify and remove the detrimental information in the data by leveraging the crucial information from these clean data.
Similar to 33
we formulate the process of model repairment as two steps:
Step I: Cause Identification. Given a reserved set D, from the target distribution, identify a set of detrimental data points in the training set D that contribute to the prediction failures for data in D,.
Step 2: Model Treatment. Given the identified failure cause set, denoted by De. refine the model to effectively correct the prediction failures in the target distribution while preserving the performance on remaining
2.1 Step 1: Cause Identification
In order to detect the detrimental data points in the training set, it is necessary to first define measure that quantifies the extent to which a subset of training data contributes to the failed predictions in the reserved set D,.. Then, based on this measure, we can screen out the data points with high failure contributions as the identified data.
Before introducing the measure, it is important to note that the data from detrimental dataset De lead to generalization failure because they introduce mismatch between train and test distribution. This is aligned with the generalization theory for mixture distributions [7] 38) which asserts that the degree of divergence between the two distributions impacts the generalization error bound. Additionally, the theory demonstrates that an increase in the amount of data from the target distribution during training reduces the generalization bound and decreases the loss of data from the same distribution. Besides, increasing the amount of target distribution data also enables the learned model to fit more closely to the target distribution and less to the detrimental distribution, thereby increasing the loss of detrimental data in D.. Since the reserved set D,, is drawn from the target distribution, we can train the weaeetttiiituttinnnnnbeie
model, denoted à on the combined set of D and D,, and identify the data from D. based on the change of loss between the original trained model 0 and the enhanced model G. The measure defined as
((2;0,6) L(z;)) L)z;
(1)
By estimating s(2;0, o) for each data point D,
identify the data points responsible for the failure, which have a positive value of s(z; 0, 2). If we know the size of the cause set, we can identify the cause data by selecting the top [De] data points with the largest s(2; 0, ②) values. We denote the identified data set by D.,, and the remaining data set by Dj.
2.1.1 Obtaining the Enhanced Model
Considering the large scale of data, training a new model à from scratch computationally prohibitive. Instead, practical solution is to approximate the model trained on the combined dataset of D and D, by performing fine-tuning continual learning on the well-trained Ô from D using the data in D,, However, due to over- praamttramttinniionnn of the network and the small size of the reserved set D,., fine-tuned model may simply memorize the data in D, 41), resulting in a negligible impact on the
3

---

Training Set
Reservod Set
Remmanning Set
Cause ldemiiisation
30 Eneray brsed Aligument
Updated Model y
Updated Model @
Identified caused D
Model A
Repeired Model #
Figure 1: The framework of the proposed method. Best view in color: beneficial and detrimental information are indicated in green and red.
loss of the data in the original training set D. This makes the increase of loss s(2; 6, o) ineffective in identifying the detrimental data. On the other hand, a fine-tuned model may completely overfit D, while forgetting the knowledge acquired from 'D [27]. which can cause it to fail in approximating ő.
Given that different layers in neural network capture distinct patterns 440) selectively updating subset of layers while freezing the remaining layers can update the detrimental knowledge from D. effective knowledge in D,., while preserving the acquired knowledge from D. After updating the conflict knowledge from De. the new model may result in more significant increase in loss on De- Recent studies (27 support the fuyer-selective update by observing that selectively fine-tuning subset of layers improves the fine-tuning generalization for different distribution shifts towards the target distribution. And we empirically validate the effectiveness of layer- selertseeective update in acquiring the knowledge from D, and improving the identification accuracy, as presented in Appendix A.4
To enable layer-selective update, a criterion for layer selection is required. Note that neural networks have tendency to minimize loss by learning data patterns in the early epochs, but may end up memorizing the data in the later epochs 4 9]. Therefore, we assume that the reduction in loss after the first epoch can serve as an indicator of the ability of the selected layers learn data patterns. Based on this assumption, we estimate the loss after performing a one-epoch update for each subset of layers and select the subset with the smallest loss for further updates. Subsequently, the selected subset is updated while the remaining layers are kept frozen. As the reserved set is small, the criterion for layer selection via loss estimation after a one-epoch update is computationally efficient. should be noted that in practice, we consider only pre-defined set of layer subsets, rather than all possible combinations of subsets.
Connection to existing works: EWC-influence, as proposed in [33], utilized the Bayesian view of continual learning identify the cause data. Specifically, it defined the identification metric as the drop in log-lik logllikelihood between model trained on the combined dataset D and D,, which is subset of D, containing examples that are mispredicted by 9, and the current model trained on D. By using negative likelioeldoood as the loss function, replacing D, with Dy, and applying the continual learning algorithm Elastic Weight Consolidation (EWC) [19] to update all layers, proposed identification measure recovers the EWC-influence introduced in (33]
Both EWC-influence and the proposed method measure the contribution of training point to the prediction failure by examining the change in the loss. However, the two measures are derived from different purposes. EWC-influence aims to correct the prediction in Dy of using the gain log- likelihood improvement estimated by deleting the data from D, whereas the proposed method aims to improve the generalization to the data from the same distribution of D, by eliminating detrimental information from D. Consequently, the proposed method proposes to measure the contribution to failure by assessing the conflict between D and D, and update the model using all data in D,..
another effective metric to identify the training points that are most responsible for certain predictions. As analyzed in 33 the linear influence is the same as EWC- influence with the exception that it updates the model a single update of natural gradient descent. Consequently, proposed method can recover the linear influence in similar manner.
Linear influence 20 36 23

---

2.2 Step 2: Model Treatment
In step I, we identify partition of the training data D. that may contain harmful information leading to suboptimal performance to the test distribution Z. In this section, we present the second step of our method, which involves refining the model using the identified cause set De and the trained model in conjunction with the enhanced model ø. A straightforward solution for repairing the model is to eliminate the harmful information in De by removing the identified data from the training set by retraining the model using the remaining data [2] performing machine unlearning [29] on the identified data [33] However, identifying a perfect detrimental dataset from the training data is difficult. Simply erasing the information of identified data can negatively impact the model's performance, especially when accurate data is mistakenly identified as detrimental and removed. Furthermore, even the detrimental data contain useful information. For example, incorrectly annotated image contains a clean input. Recall that the generalization failure is caused by the mismatching between the detrimental distribution and target distribution. Inspired by the domain adaptation framework [35 30), we propose to align the detrimental data with the target distribution, which can reduce the divergence between the training distribution and the target distribution. The aligning process can rectify the detrimental information in the data, making it possible to reuse the aligned data toward better performance. A theoretical analysis of the following Gaussian mean estimation task supports the potential of this approach to improve performance.
Gaussian mean estimation task: Consider the task where the objective is estimate the mean of Gaussian distribution 44 = [N o²) based on a training set 'D consisting of n training data (z; )i= sampled from the distribution LI as well as m training data {2; } /i=++1 sampled from another Gaussian distribution Ut = ~N',, ((""',²)) where μe and y' are the means of the two Gaussian distributions and a² denotes the variance. We define the loss function as L(2;; 0) (0 ==)². In this example, we assume n is sufficiently large than m and the distribution divergence between 2 and U', meausured by 1A0 μ²), is sufficiently large, i.e. | 12| m++ al (an((11 /I/n vI/(n + m)), such that the training data from 2t' hurt the generalization performance of the model.
ya+m
The optimal solution is 0- = μ.. The empirical risk minimization solution is obtained by minimizing C(D;0) 0) m+m - (0- xz as 0 . i= z4. We measure the generalization error as the difference between the empirical solution 0 and the optimal solution 0", Using Gaussian tail bounds (34, with probability at least p for some probability, the generalization error is upper bounded as
0 0-1 S av aa/l/mmm+ m)
m
1p p'l:=
where a = a / // is some constant.
Assume we have identified the failure cause set as (= Directly removing the cause set and retraining the model achieves generalization error bound as Bremove av/I/n B.
Proposition 2.1. In the Gaussian mean estimation task, there exists a set of alignment functions: h(x) = je' + f where |1e jil << 114 n') such that the optimal empirical solution on the combined set of (a. 1i and (h(x))) i.e. θ = m++ (E a generalization bound Balign < Bremove-
Sm+m Mr.)). achieves
The detailed proofs for the bound can be found in Appendix The alignment function M(x)) nl' + fi aligns the data in L/ with 21 removing the detrimental information jí' and enhancing the beneficial information p.. Upon alignment, the resulting data set, (h(x:)) ( approximates the target distribution closely, and utilizing it in training will enhance generalization performance.
We extend the idea of alignment- model treatment from the Gaussian mean estimation task to general tasks. Coneretely, we propose to perform alignment on the identified cause data to obtain the aligned data using an Energy-Based Based Model (EBM) 225), building upon the energy-based alignment proposed in [18] EBMs represent a probability density p(=) for = E Rd as po(2) expl app-Eate Ea(e))
Z(0)
where Ex)x) is the energy function which takes as input and returns a scalar as energy, and (())= J_ exp( [=)) is the normalizing constant. To align a given data point 2' with po(2). we
iteratively adapt z' to minimize the energy function [8] as
z' == - Eo(s),
5

---

Algorithm 1 Beneficial Information Retaining (BIR) Model Repair
1: Input: training data D, reserved data Dy; trained model 6, pre-defined candidate of layer subsets.
Step I: Cause Identification
2: Estimate the loss after one-epoch update of 0 on D, for each layer subset in the pre-defined set while
lixing the remaining layers
3: Select the corresponding layer subset to be updated with the smallest loss
Obtain & by updating the selected layers of model 0 on
5: Estimate s(2; 0, 6) for each data E D
6: Identify the failure cause set as D. <- [= E D s(2; 0}
Step 2: Model Treatment
7: Perform data alignment for data in De and obtain the aligned cause set De
8: Repair model by updating the selected layers model on De
where is a hyperparameter that controls the adaptation rate. In contrast to existing works [18] which trains am extra EBM to perform alignment, we adopt the approach proposed in 77]. which utilizes the classification model / as energy-basee model (EBM) to represent the data distribution density as Po( y) exp(Ja (a (w))))) where fo a classification model which maps each data point x to the logits 2(0) of K classes, [M]] denotes the logit corresponding to the y-th class and (() is the unknown normalization constant.
We use target- iuustribution-enhanee model á to approximately characterize the distribution Z. Con- sidering potential label noise, we update using the energy function of pa p; instead of pz(a, y).
where we obtain (²(x) = Ey v²) Eg exp(Ig(=)lx()) energy function is defined as
by marginalizing out The corresponding Z(d)
Bz(T) = log Σ
exp(fg(=)loD))
y
With the energy function, we align the data (2r, v) E D. by first aligning the input x by one-step
gradient descent
== -=
(4)
then aligning the label y by
explfx Ey exp(fx
i ύ= Pz(())))
(5)
which is the soft label of ž.
After obtaining the aligned data, we update the model from 0 using the aligned data. Similar the update of enhanced model ò update only the layers selected in the cause identification step while keeping the remaining layers frozen.
Finally, the proposed model repair method, named Beneficial Information Retaining (BIR) Model Repair, including the cause identification and model refinement, is summarized in Algo. and illustrated in Fig.
3 Related Works
Our work is closely related to 33 following which we formulate the model repairment problem into a cause identification step and model treatment step. (33), Tanno et al. discuss the model repairment problem from Bayesian perspective and introduce EWC-influence for cause identification and EWC-deletion for model treatment which performs continual unlearning remove the data. The cause identification relies on data influence estimation, which can be solved by various influence estimation methods [00] such as linear influence 20, 22 SGD influence [12 and Data Shapley
proposes to remove the information of the failure cause data by retraining the model with the remaining data or by performing machine RDIA 23 proposes an influence- -baee relabeling framework for reusing harmful training samples. However, RDIA's aggressive relabeling benign data that is mistakenly identified as detrimental can introduce incorrect information and negatively impact the model's performance,
Value 16). For model treatment, existing works 12 23
unlearning 8 29

---

Table I: Comparison of model repair performance on corrupted MINST datasets. Performance better than base model highlighted by cyan and the best performance among all baselines is in bold.
Model
Danaset
MNNST-Label
MNIST-lapot
MNIST-Adv
Meiri
All
Clean
Noss
All
Clean
Noisy
All
Clean
Noisy
Model
82.46
96.46
97.10
95.50
96.35
97.33
SGD Influence Repair
84.40
95.67
67.50
96. 12
96.90
94.95
96.06
96.97
94.20
LoNet
BWC Repair
Lineur Iniluence Repair
83.00
84.26
96.43
95.43
62.85
67.50
96.
96.26
97.07
96.90
95.05
94.95
96.50
96.06
97.27
96.97
95.35
94.70
RDJA
72.70
93.03
42,20
85.82
90.83
78.30
81.30
86.00
74.25
BIR (Ours)
95.60
95.93
95.10
96,
96.20
96.10
96.44
96.47
96.40
Base Model
83.96
97.87
63.10
98.00
98.57
97.15
97.82
98.57
96.70
SCDD Influence Repuir
88.66
98.60
73.75
97.76
98.40
96.80
97.00
97.93
95.60
Com44
Lincur Iniluence Repair
72.12
87.97
18.35
96.96
97.43
96.25
92.86
98.43
84.30
BWC Repair
89.02
98.53
74.75
97.82
50
96.80
97.10
97.90
95.90
RDIA
74.04
82.76
79.77
BIR (Ours)
96.36
97.10
95.25
98.02
98.93
96.65
98.12
98.43
97.65
Table 2: Comparison of model repair performance corrupted CIFAR-10 10 datasets.
Model
Dataset
CIFARIO-Label
Cleun
Noisy
CIFARJ0-Input
Clean
Naisy
All
CIFARIO-Ady
Clean
Nokey
Base Model
74,80
90.83
48.75
87.20
91.77
801.35
87.88
93.40
79.60
SGD Influence Repuir
76.66
92.40
53.05
89.24
93.00
83.60
86.06
92.13
76.95
ResNe120
Linear Influence Repair
74,80
92.30
48.55
87.88
87.72
93.00
79.80
FWC Kejuir
77.06
90-77
56.50
87.44
93.37
78.35
86.70
12.00
78.75
RDIA
67.50
85.17
41:00
79.66
85.20
71.35
77.54
82.23
70.30
(Ours)
83,98
91.30
73.30
87.94
92.17
81.60
Basc Model
77.10
90.60
50.95
86.62
$4.30
75.10
$7.60
93.13
79.30
SGD Infuuecce Repair
78.36
94.17
54.65
89.36
95.30
80.45
$4.08
94.67
68.20
ResNe:50
Lanar Iniluence Requir
75.82
93.83
38.80
91.90
79.85
87.60
93.67
78.70
EWC Ropair
78.00
93:57
54.65
89.94
94.47
83.
54.08
94.67
68.20
RDIA
66.42
90.13
30.85
76.24
87.33
$9.60
71.44
84.20
$2.30
(Ours)
83.02
91.73
69.95
88.64
90.26
93.20
85.85
Table 3: Comparison of model repair performance on CIFAR-ION, The network architecture is ResNet20.
CIFAR- -ON
Worst
Random
Aggregate
Base Model
77.76
84.68
85.78
Linear Influence Repair
79.30
85.22
87.70
SGD Influence Repair
71.50
77.21
76.08
EWC Repair
71.80
76.22
77.18
RDIA BIR (Ours)
62.90
62.96
64.62
85.50
88.44
89.60
making it a high-risk approach the case of imperfect detrimental data identification. Our proposed approach BIR also reuses the identified data to improve model performance. By aligning the identified detrimental dataset with the clean data, our framework rectifies the detrimental information while preserving the beneficial information in the identified set, and updates the model using the aligned data. Unlike RDIA, our alignment strategy does not introduce incorrect information when applied to clean data and achieves satisfactory performance even in cases where there are mistakenly identified clean data in the cause set.
4 Experiments
To evaluate the effectiveness of the proposed BIR model repair algorithm, we conduct extensive experiments answer the following research questions: Q1: How does BIR compare to the state- of-the-art baselines in repairing neural networks that produce error predictions? Q2: How does the proposed failure cause data identification metric compare to the baselines in identifying the failure cause data? Q3: How does the proposed align-then-updae strategy compare to the baselines in model treatment when considering both perfect and imperfect failure cause sets?
Datasets and Network Architectures: We evaluate the effectiveness of model repair algorithms on corrupted versions of MNIST 266 and CIFAR-10 24 datasets following the setups in (33) We consider annotation error, input noise, and adversarial attacks in the training data. To simulate annotation error, we randomly flip about 40% of labels between specific similar digits or classes. For input noise, we randomly corrupt 40% of MNIST images and 30% of CIFAR- 10 images in selected set of target classes by adding specific types of noise, such as salt-and-pepper noise in 10.. Additionally, we construct datasets by adversarially attacking 40% of randomly selected training images the same target classes using FGSM We denote the corrupted datasets with label noise, input noise, and adversarial attacks by MNIST-Label. MNIST- and MNIST-Ady
MNIST or Gaussian
7

---

Random EWC-influence
Linear Influence
SGDD=influence
Ours
Oracle
Cww.gegeeeeeaaeeooooooooo Desce Desera
Nashen Dhen Drnrral Donrall
Naster of
740
1400
2100
1800
700
1430
2103
300
30
1400
2796
780
Numbe of Duna Inspecicd
Namber Deas Iuspctad
Nuzthrr of Data Inqpeced
(8) MNIST-Label
(b) MNIST-Input
(c)
MNIST-Adv
Narter Dhes Dewore
Nache of Due Devoed
Nasher Corngaaaaoaaaaaaaaaa
18
Inppeed
of Insectal
(d) CIFARIO-Label
(e) CIAFRIO-Input
(I) CIFARIO-Ady
Figure 2: Comparison of Failure Cause Identification on LeNet trained with corrupted MNIST datasets and ResNet50 trained with CIFAR-10.
(CIFARIO-Label, CIFARIO-Inpun and CIFARIO-Ady). For MNIST, 3000 samples from the original training set are utilized while the complete original training dataset of CIFAR-10 is employed train the base models, with validation set consisting of 10% of the data. Furthermore, we partitioned the original test set, which is uncorrupted, into reserved set consisting of 300 samples for MNIST and 1000 samples for CIFAR- 10, while the remaining set was designated as the testing set. The selected target classes of MNIST are 1,7, 6, and 9; and the selected target classes of CIFAR-10 are 'plane', 'bird', 'cat', and 'dog'. LeNet and Conv4, which is a 4-layer CNN with a fully connected layer, are employed as classifiers for MNIST, while ResNet-20 and ResNet-50 are utilized for CIFAR- 10. We also conduct experiments on CIFAR-101 dataset (37], which equips the training set CIFAR-10 with real-world human-annotates noisy labels. There are noise levels: Worst (1),,) Random (~1%,, and Aggregate (~99)).
Baselines: We compare the proposed method with EWC Repair 33
RDIA 23 SGD-influence
Repair. EWC Repair utilizes EWC-influence to identify harmful data and EWC-deletion remove them from the model. RDIA uses linear influence (20 to identify causes, relabel the identified data using an influence- based relabeling function, and retrains the model using the relabeled and remaining data. SGD-influence Repair identifies harmful data using SGD-influence and proposes to retrain the model using the remaining data. Linear-Influence Repair the linear influence function 20 to identify the cause data and performs Newton-update removal [8] to climinate the identified data's 's information from the model. In practice, retraining the model from scratch can be computationally prohibitive. Therefore, for RDIA, we performed fine-tuning by using both the relabeled data and an equivalent amount of randomly selected data from the remaining dataset. For SGD-influence Repair, fine-tuning was conducted the remaining data that matched the size of the identified data.
Repair 12 and Linear-iffueence-baae
small partition of
We measure the performance of model repair by the accuracy of the entire test set (denoted by "Test"). Besides, we measure the success rate of the repair by the test accuracy of the selected corrupted classes (denoted by "Noisy"" and the performance maintaining rate by the test accuracy of the remaining classes (denoted by "Clean")).
Performance Measure:
Additional information regarding the setups
4.1 Comparison to Model Repairment Baselines
The comparison results for corrupted MNIST and CIFAR-10 are provided in Tabs. [add] We can observe from the results that the proposed BIR repair algorithm successfully repairs the model and improves the performance of noisy classes in all experiments except for MNIST-Input with Conv4. In contrast, most baselines fail to repair the model for MNIST-Input, MNIST-Adv and CIFAR 10- Ady.

---

Table 4: Comparison of Model Treatment for LeNet trained on corrupted MNIST.
Dataset
MNiST-Labee]
MNiST-laput
MNIST-Adv
Metric
Test All
Test Clean
Text Noisy
Test All
Test Clean
Test Noisy
Test All
Test Clean
Test Noisy
Base Model
82.46
96.10
62.00
96.46
97.10
95.
96.38
97.33
94.95
Retran
96.56
97.43
96.84
97.57
95.75
97.33
96.10
Ielntilied sel= ground-ruth cause sct
Fine-tune tine
95.56
96.90
93.35
96.42
97.77
95.30
96.26
97.17
54.50
Newton Reasoval
85,20
96.35
68.30
96.52
97.17
95.55
96.50
97.33
95.25
BWCC-Deletio
95.10
96.00
93.75
96.14
96.50
95.00
96.34
96.57
95:75
95.70
97.23
93.91
91.90
BIR (Ours)
95.64
95.67
95.60
96.78
97.11
96.30
96.66
97:13
95.95
Identifedd set= 75% ground-trui truth set
15% cluus data
Finc-tune
58.18
96.23
76.10
95.98
96.77
95.40
96.22
96.33
95.45
Newton Remnoval
$3.92
96.47
65.10
96.16
96.87
95.10
96.36
97.30
94.95
EWC-Delation
57.74
94.75
96.03
04:60
RDIA
$4.86
83.77
86.30
90.48
90077
91.10
85.82
89:40
80:45
BIR (Ours)
95.70
96.47
94.55
96.48
96.67
96.19
96.62
96.80
96.35
Table 5: Comparison of Model Treatment for ResNet50 trained on corrupted CIFAR-10 10.
Dataset
LaaeR
CIFARIO-Ippo
CIFARIO-Ady
Metric
Test All
Test Clean
Test Noisy
Test All
Test Clean
Test Noisy
Test All
TTss Clean
Test Noisy
Base Model
77.10
90.60
50.95
86.62
94:30
75. 25.10
87.00
93.13
79.30
Retrain
$8.18
91.03
79.40
87.48
91.43
81.55
B8,10
93.07
80.65
Identilied set ground-trtt
sct
Flac-tune
85.16
94.83
70.65
89.86
94,40
83.05
89:66
94.67
82.15
Newton Remnoval
84.04
93.60
69.70
86.72
92.60
77.90
87.60
93.80
78 30
DeC-Deletion
$4.50
91.73
59.76
82.90
91.20
79.70
RDTA
82.64
94.43
64.95
87.20
02.07
79.90
86.98
92:40
78.85
BIR
83.16
91,43
71.00
58.84
91.97
84.15
90.60
93:43
86,10
Tdentilied set 75% ground-trill cause set
25% clewn data
Fine tine
83.18
94.47
66.25
89.00
03.50
82
89:48
94.37
81.85
Newton Removal
76,08
91.17
56.72
79:15
80,20
EWC-Deletion
82,08
94.07
64.10
88.84
93.27
82.20
87.12
92.83
78.55
RDIA
74.58
86.17
57.20
79.88
85.30
71.65
78.70
83.37
71,40
BIR (Ours)
82.88
91.33
70.20
88.78
92.33
83.35
90.50
93.63
86.05
Furthermore, BIR achieves the best performance in noisy classes on 10/12 experiments, and the best performance in all classes on 8/12 experiments. These results demonstrate the effectiveness and superiority of BIR in removing detrimental information from the data to repair the model. We also observe that performance in clean classes of all baselines is generally slightly lower than the base model, which is trade-off for repairing the corrupted data. The comparison results for CIFAR- in Tab. show that our method achieves superior performance across all noise levels, demonstrating its effectiveness in mitigating real-world label noise.
4.2 Comparison of Failure Cause Identification
We compare the proposed causes identification method using Eqn. with EWC-influence Linear-influence (20) and SGD-influence [12]. To measure the performance of the failure cause identification, we report the number of identified data points corresponding to samples corrupted with noise. The results in Tigure[ssow that all methods perform well when a small number of data points are identified. Our method performs the best overall, except on MNIST-Input. that our method identifies almost all the corrupted data points in MNIST-Label and CIFAR LO-La,,l, where the model performance is significantly impacted by the detrimental information in the training data. However, is also important to note that the identification is not perfect, and sufficiently large set of clean data points may be identified as detrimental data. Therefore, the model repair baselines are at risk of removing useful information from the identified set and may fail to repair the model. In contrast, BIR can effectively remove the detrimental information while retaining beneficial information from the identification, achieving satisfactory performance.
is worth noting
4.3 Comparison of Model Treatment
We evaluate model treatment algorithms given both precise and imprecise identified cause sets, with the latter containing 75% corrupted data and 25% clean data. The baselines include EWC- deletion (33, RDIA (23. and Newton- removal [8] and fine-tuning of randomly selected data from remaining set as baselines. In addition, we employ retraining the model from scratch using the remaining data as further baseline.
The results are reported in Tabs. [5n We can observe performance improvement compared to the results reported in Tabs. and And we also observe a decline in performance for most methods

---

when replacing the precise identified set with the imprecise one, particularly on MNIST-Label and CIFARIO-Label. These results indicate that the performance the model treatment baselines is contingent on the correctness of identified set. Notably, the performance drop of BIR is significantly less than that of the baselines, illustrating the effectiveness and superiority of BIR in retaining beneficial information and eliminating harmful information from the identified data to repair the model, even when the identified set comprises clean data.
4.4 Extra Experiment Results
We investigate the effect of the layer-selective update strategy on cause identification and model treatment. The results and analysis are presented in Appendix |A.4 Moreover, we explore the solution for determining the appropriate size the cause set when the ground-truth size unknown and evaluate the effect of the size on performance in Appendix A.5 Additionally, Appendix (A.66 we studied the effect of varying the amount of clean reserved data on our method's performance.
Conclusion and Limitations
In this paper, we propose novel framework for repairing neural networks trained with detrimental data. To identify the data that cause generalization failure, we propose to update the model with a reserved set sampled from the target distribution, which reduces the loss of clean data and increases the loss of corrupted data. After identifying the cause data, we acknowledge the potential value of detrimental data in enhancing the training of more generalized model alongside target data and propose an energy-based alignment to remove harmful information while retaining beneficial information in the identified data. After obtaining the aligned data, we update the model on these data to repair the model. We conducted extensive experiments on real-world datasets to validate the effectiveness of our proposed framework, demonstrating its superiority over existing methods in identifying harmful data and repairing the model to enhance generalization.
L Limitation: the proposed method entails access the training data for model repairing purposes, requirement that may be hampered by the possibility of discarding the data after training. Nonetheless, such accessibility is essential the method's development, which is predicated that erroneous training data engenders generalization failure, and thus seeks to remove the adverse information effect model repair. We leave the investigation of training data-free model repair the future.
the assumption
Acknowledgements
Sinno J. Pan thanks for the support from HK Global STEM Professorship and the JC STEM Lab of Machine Learning and Symbolic Reasoning.
References
[1] Devansh Arpit, Stanistaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. closer look at memorization in deep networks. In International conference on machine learming, pages 233-242. PMLR, 2017.
[2] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-255. leee, 2009.
[3] Mohamad Dolatshah. Mathew Teoh, Jiannan Wang, and Jian Pei. Cleaning crowdsourced labels using oracles for statistical classification. Proceedings of the VLDB Endowment, 12(4):376-389 2018.
[4] Jonathan Frankle, David J Schwab, and Ari S Morcos. The early phase of neural network training. arXiv preprint arXiv:2002. 2020.
[5] Benoît Frénay and Michel Verleysen. Classification in the presence of label noise: a survey.
IEEE transactions neural networks and learning systems, 25(5):845-88 869, 2013.
10

---

[6] Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning, pages 1242-2251. PMLR, 2019.
[7] Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. International Conferencee on Learning Representations, 2020.
[8] Chuan Guo, Tom Goldstein, Awni Hamnun, and Laurens Van Der Maaten. Certified data removal from machine learning models. arXiv preprint arXiv: 1tti::/9111.00000 1.03030, 2019.
[9] Guy Hacohen, Leshem Choshen, and Daphna Weinshall. Let's agree agree: Neural networks share classification order on real datasets. In International Conference on Machine Learning, pages 3950-395000 PMLR, 2020.
[10] Zayd Hammoudeh and Daniel Lowd. Training data influence analysis and estimation: A survey. arXiv preprint arXiv:2212.046111 2022.
[11] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.
[12] Satoshi Hara, Atsushi Nitanda, and Takanori Maehara. Data cleansing for models trained with sgd. Advances in Neural Information Processing Systems, 32, 2019.
[13] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations, 2019.
[14] Pei- Yun Hsueh, Prem Melville, and Vikas Sindhwani. Data quality from crowdsoureing: study of annotation selection criteria. In Proceedings of the NAACL HLT 2009 workshop on active learning for natural language processing, pages 27-35, 2009.
[15] Long-Kai Huang, Junzhou Huang, Yu Rong, Qiang Yang, and Ying Wei. Frustratingly easy transferability estimation. In International Conference on Machine Learning, pages 9201-9225. PMLR, 2022.
[16] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1167-1176. PMLR, 2019.
[17] Sharu Theresa Jose and Osvaldo Simeone. Information ceermation-thhorttii bounds on transfer general- ization gap based on jensen-shannon divergence. 2021 29th European Signal Processing Conference (EUSIPCO), pages 1461-1465. IEEE, 2021.
[18] KJ Joseph, Salman Khan, Fahad Shahbaz Khan, Rao Muhammad Anwer, and Vineeth N Balasubramanian. Energy-based latent aligner for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7452-7485 155--461, 2022.
[19] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Barnskkkaaaiikka,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114(13):3521-352, 2017.
[20] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions.
In International conference on machine learning, pages 1885-1894, PMLR, 2017.
[21] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie. Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benehmark of tee-thhewddd distribution shifts. International Conference on Machine Learning, pages 5637-5664. PMLR, 2021.
[22] Pang Wei W Koh, Kai-Siang Ang, Hubert Teo, and Percy S Liang. On the accuracy of influence functions for measuring group effects. Advances in neural information processing systems, 32, 2019.
11

---

[23] Shuming Kong, Yanyan Shen, and Linpeng Huang. Resolving training biases via influence- based data relabeling. In International Conference on Learning Representations, 2021.
[24] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[25] Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energy- based learning. Predicting structured data, 1(0), 2006.
[26] Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http::/yann. www.e...o//..c///ne/
2, 2010.
[27] Yoonho Lee, Annie S Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. International Conference on Learning Representations, 2023.
[28] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint Irxiv: 06083, 2017.
[29] Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoe Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint ar arXiv:2009.02299 2022.
[30] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345-1359 2010.
[31] Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 1944-1952, 2017.
[32] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case 2019.
generalization. arXiv preprint /rXiv:/911.0000
[33] Ryutaro Tanno, Melanie Pradier, Aditya Nori, and Yingzhen Li. Repairing neural networks by leaving the right past behind. Advances in Neural Information Processing Systems, 35:13132- 13145, 2022.
[34] Martin Wainwright. Chapter 2: Basic tail and concentration bounds. 210B Lecture Notes University, 2015.
[35] Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomputing, 312:135-153, 2018.
[36] Zifeng Wang. Hong Zhu, Zhenhua Dong, Xiuqiang He, and Shao-Lun Huang. Less is better: Unweighted data subsampling via influence function. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages
2020.
[37] Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: study using real-world human annotations. arXiv preprint arXiv:2110. 12088. 2021.
[38] Xuetong Wu, Jonathan H Manton, Uwe Aickelin, and Jingge Zhu. exformation-theoreee analysis for transfer learning. In 2020 IEEE International Symposium Information Theory (ISIT), pages 2819-2824. IEEE, 2020.
[39] Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K Jain. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 17:151-178, 2020.
[40] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems, 27, 2014.
[41] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):
115, 2021.
12

---

A Omitted Experiment Details in Sec. ann Extra Experiments
A.1 Dataset
MNIST [26] dataset contains 60,000 training images and 10,000 test images of 10-digit classes from
0 to 9, with each image in the size 28 x 28. We follow (33] to randomly select 3,000 images from the original training set to construct the training set. And we randomly choose 300 images from the original test set construct the reserved set and use the remaining data as the test set for performance evaluation.
We construct three corrupted versions of MNIST dataset by introducing label noise, input noise, and adversarial attack on the clean training set. Specifically, for MNIST-Label, we randomly select 40% of the data from classes 1 and 7, and flipped their labels to each other; and the same for classes 6 and 9. For MNIST-Input, we randomly corrupted 40% of the data from the target classes 1, 7, 6, and using salt and pepper noise. To create MNIST-Adv, we first train victim model the clean training set and then generate adversarial images using the fast gradient sign method (FGSM) for 40% of the data randomly selected from the target classes. The poisoned images in MNIST-Adv are labeled by the classes predicted by the victim model.
CIFAR-10 24] comprises 60,000 color images divided into 10 categories, each with dimensions 32 x 32. The training set has 50,000 images while the original test set is split into a reserved set with 1,000 images and test set with 9,000 images. We introduce three corrupted versions of the CIFAR-I0 dataset, where label noise. input noise, and adversarial attacks are introduced on the target classes: plane, bird, cat, and dog. Specifically, selecting 40% of the plane and bird class images and swapping their labels, and similarly for the cat and dog classes. For RIFARt,,,u,, we apply Gaussian noise to 30% of the images from the target classes. Finally, for CIFARIO-Adv, we use adversarial attacks to corrupt 40% of the randomly selected images from the target classes, similar to MNIST-Adv.
generate the CIFA R10-Label dataset by randomly
CIFAR R-I [37 is CIFAR- 10 dataset with human-annotated noisy labels. In CIFAR-ION dataset, labels. It has 5 noisy-label sets: Random i(i {1, 2, 3)) uses the i-th submitted label for each image; Aggregate generates the label by majority voting of three noisy labels; Worst randomly selected one label from the wrong labels if there exist any wrongly annotated labels in 3 noisy labels. In the experiments, we adopt out of the 5 noisy-label sets, namely, Aggregate, Random 2 and Worst.
each training image has 3
A.2 Network Architectures
For the corrupted MNIST datasets, we use LeNet and Conv4 as the classifier. LeNet is widely used for the tasks on MNIST. It consists of 2 convolutional layers and 3 fully- connected layers. In the layer selective update, we define Layer to Layer 5 as the first (the first convolutional layer) the last layer (the last fully-comnected layer). The set of candidate layer subsets considered in the layer selective update stage comprises [[ayer 1). [Layer 2}, [Layer 3), [Layer 4), [Layer 5}. [Layer 1-2], {Layer 1-3), [Layer }-5)). Conv4 consists of convolution layers followed by fully- fully-connected layer. Similar to LeNet, we define Layer to Layer the first to the last layer, respectively, and consider the same candidate layer subsets.
The corrupted CIFAR-10 datasets are classified using ResNet-20 and ResNer-50. The CIFAR- 1ON dataset also uses ResNet-20 as classifier. ResNet-20 comprises one convolutional layer, 3 compositional layers, each of which consists of 3 basic blocks, and fully-connected layer. Layer is defined as the first convolutional layer, Layers 2 to 4 are the composi- tional layers, and Layer 5 is the fully-connected layer. The same candidate layer subsets are considered. ResNet-50 consists of one convolutional layer, 4 compositional layers, each of which consists of 3, 4, 6, and 3 bottleneck blocks, respectively, and fully-connected layer. Layer 1 is the first convolutional layer, Layers to 5 are the 4 compositional lay- layer. Its candidate layer subsets are defined as
ers, and Layer 6 is the last fully-connecter {[Layer 1}. [Layer 2), [Layer 3), [Layer 4}. [Layer 5]. (Layer 6), (Layer 1-2), (Layer 1-3), [Layer 1-6)]. For ResNet-50, The input images are resized to 224 x 224.
LeNet and Conv4 are both trained from scratch. The ResNet-20 and ResNet-50 model initialized with the pre-trained model on ImageNet.
13

---

Table 6: Performance with different selective layers updated: Identification AUC and Treatment Accuracy (%), with best result highlighted bold. The corresponding performance of the selected layers update highlighted in gray. The backbone of the classifier is LeNet.
Dataset
MNIST-Label ldentiiicatinn AUC
MNIST-lapul ldentilictton
MNIST-Adv Identiiiation
Loss alter one-spoch update
Treaiment Accuracy(9)
Loss aller one-epoch update
Treatment Accuracy!
Loss ailer one-epoch updatee
Treamment
AUC
AUC
Accuracy
Layer
L,0795
0.91
86.02
0.2503
0.71
96.16
0.24118
0.61
96.54
2
0.8967
0.96
92.56
0,2641
0.61
96.14
0.2201
0.57
96.40
Layer 3
0.8145
0.98
93,36
0.44268
0.60
95.92
0.3458
0.60
96.28
Layer
0,9419
0.96
94.42
0,3027
0.52
95.78
0,2682
0.56
96.34
Layer 5
T,0255
0.96
93.14
0.2751
0.46
95.52
0.2289
0.51
96.18
Layer 1-2
0,9193
0.96
92.36
0,2799
0.58
96.14
0.2121
0.70
96.44
Layer 1-3
0,7360
0.99
95.60
0.44243
0.70
96.12
0.331
0.73
96.46
Layer 1-5
0,9407
0.99
95.50
0.6009
0.69
95.34
0.3618
0.66
96.28
Table 7: Performance with different selective layers updated: Identification AUC and Treatment Accuracy(%) %6), with the best result highlighted bold. The corresponding performance of the selected layers update highlighted in gray. The backbone of the classifier ResNet20.
Datasct
CIFARIO-Label Identilicatinn AUC
CIFAR J0-Inut
CIFARIO-Ady
Loss alter
Treatment Accuracy(S))
Loss alter
Identiticaiinn AUC
Treatment Accuracy(
Loss aiter one-epach update
Identifiction AUC
Treatment
one-epoch update
one-epoch update
Accuracy
Layer
1,0763
0.56
74.82
0,,4990
0.68
85.52
0.6938
0.60
$7.64
Layer
.0520
0.57
75.00
0.84
0.89
66.60
Layer
1,0038
0.68
76.92
0.6489
0.70
86.46
0.0033
0.65
86 58
Layer
83.98
0.74
56.78
0.68
Layer 3
0.998
0.00
25.62
0.6672
0.69
87.46
0.0199
0.63
87
Layer
T,2213
74.82
0.80
0.69
Layer 1-3
10383
0.06
76.04
0.6309
0.76
86.08
0.0080
0.65
86.72
Tayer
0.86
83.70
0.72
87.22
86.38
A.3 Training Details
In this section, we introduce the settings in training. Specifically, the training set is partitioned into 90% training data and 10% validation data. The base model is trained using the Adam optimizer with a learning rate of 1e-3 and a batch size of 64 for all datasets. And the model is trained for a maximum of 100 epochs, and early stopping after epochs of no progress on the validation data.
To obtain the enhanced model, we update the model on all data in the reserved set using the Adam optimizer with learning rate of le-3 and batch size of 16. Training stopped after no progress on the training loss. The update layers are selected based on the training loss estimated after a one-epoch update, i.e., an update going through all reserved data. Once the enhanced model obtained, we estimate the score Eqn. I for all training data and identify the cause data by selecting the top K data points with the highest s(2; A, @) values. Here, K is equal to the size of the ground-truth cause set, i.e., [De]_ For the baseline identification methods, the same number of data points, i.e. K are selected as the cause data.
During model treatment, we first perform energy- based alignment on the identified data using the enhanced model. Specifically, the data input is updated using Eqn. 4 with = 1.0 and the label is updated using Eqn. (5. We then update the model using the aligned data and K additional data points that are randomly sampled from the remaining training set. The purpose of the additional data from the remaining set is to alleviate bias toward the identified data. It is worth noting that all baseline methods perform model treatment using both identified data and randomly sampled remaining data. In the proposed method, we initialize the model with 0 and optimize it using the Adam optimizer with learning rate of le-3. For the corrupted MNIST datasets and CIFAR- 10 datasets, we set the batch size to 16 and 64, respectively. We terminate the training process when there is no further decrease in the training loss.
A.4 Extra Experiment: The Effect of Leyer-Selective Update
In this section, we investigate the effect of the layer-selective update strategy on cause identification and model treatment. Performances are presented in Tabs. and 7 where the AUC between the cause score of the training data calculated by Eqn. (1) and the ground-truth label indicating whether the data from DD is used to measure identification performance, and the accuracy of the test set of the repaired model with different layers updated is used to evaluate model treatment performance. The
14

---

Table 8: Number of identified cause data corrupted MNIST and CIFAR-I0 datasets using different strategies.
MNIST-Label
MNIST-Input
MNIST-Adv
CIFARIO-Label
CIFARIO-Input
CIFARIO-Ady
ground-ruth amount
432
432
432
7200
5400
7200
threshold = 0
1115
1789
1919
23955
27449
30457
threshold = log(1.1)
469
122
198
8745
5076
6003
threshold = (005)
487
157
251
9952
6362
7686
threshold = ( (100)
531
268
393
12836
9716
11849
Table 9: The repair performance on corrupted MNIST datasets under different cause set sizes using different
identified strategies. The size of the identified cause set is available in Tab. [
The network is LeNet.
MNIST-Label
MNIST-Input
MNIST-Adv
All
Cleun
Noisy
All
Clean
Noisy
All
Cleun
Noisy
ground-truth amount
95.60
95.93
95.10
96.16
96.20
96.10
96.44
96.47
96.40
threshold 0
95.84
96.30
95.15
96.32
96.13
96.60
96.48
96.63
96.25
threshold log(1.1)
95.64
95.77
95.45
96.22
96.13
96.35
96.40
96.57
96.15
threshold = log(1.055 (100)
95.66
96.20
94.85
96.28
96.07
96.60
96.30
96.43
96.10
threshold (o0(1.01)
95.56
96.07
94.80
96.14
95.97
96.40
96.34
96.53
96.05
Table 10: The repair performance on corrupted CIFAR-10 datasets under different cause sizes using different
identified strategies. The size the identified cause set available in Tab.
The network is ResNet50.
CIFAR 10-Labll
CIFARIO-Input
CIFARIO-Ady
All
Cleun
Noisy
All
Clean
Noisy
All
Clean
Noisy
ground-truth amount
83.02
91.73
69.95
88.64
91.80
83.90
90.26
93.20
85.85
threshold =
83.26
91.33
71.15
88,90
92.20
83.55
90.14
93.43
85,20
threshold log(L.1)
83.00
91.50
70.25
88.50
91.63
83.80
90.20
93.30
85.55
threshold log(1 (100)
83.14
91.83
70.10
88.62
91.70
84.00
90.12
93.03
85.75
threshold = (1.0)
82,98
91.57
70.10
88.66
92.07
83.55
89.86
93.07
85.05
results indicate that updating all layers is suboptimal for both identification and repair performance in most cases. Thus, layer-selective update is necessary to achieve better performance. We suggest selecting the layers for update based on the loss after one-epoch update on the reserved set. The proposed method achieves the best identification performance in 4/6 cases and the best treatment performance in all datasets except MNIST-Adv, where it is only 0. 1 less accurate than the best one. These results highlight the effectiveness of the layer-selective update in enhancing the proposed method's performance.
A.5 Extra Experiment: The Effect of Identified Cause Set Size
In our experiments, we assume knowledge of the ground-truth size of the cause set, denoted as Nc. and identify the cause data by selecting the top Ne data points with the highest score estimated by Eqn. I However, in cases where the number of corrupted data examples uncertain, selecting an appropriate threshold becomes challenging. To address this issue, practical solution is to apply fixed threshold on the score (s) computed using Eqn. [I] as a heuristic measure. As straightforward solution, setting the threshold to O identifies all training data with an increased loss, but this often results in identifying more data than the actual number of corrupted points, as demonstrated in our empirical studies presented in Tab. g Our investigations indicate that threshold between log((101) and log(1.1) effectively filters out identified data close in size to the ground truth. Thus, recommend employing threshold of log(1.05) for practical applications.
Importantly, we emphasize that the repair performance of our proposed BIR method robust the size of the identified cause set, as evidenced by the experimental results in Tabs. p and [00 Therefore, while identification using thresholds may not perfect, the repair performance of BIR is hardly affected.
A.6 Extra Experiment: The Effect of Clean Reserved Set Size
In this section, we investigate the impact of varying the amount of reserved clean data on the performance of the proposed model repair method when applied to three corrupted MNIST datasets. The identification and repair performance is reported Tab. [1 Our empirical analysis reveals three key findings: firstly, with mere 30 clean samples, our approach resulted in enhanced performance on noisy classes, albeit with lower identification AUC; secondly, as the number of clean samples increased from 30 to 300, both identification AUC and overall performance demonstrated marked
15

---

Table 11: The repair performance corrupted MNIST datasets with different sizes of preserved clean data. The network LeNet.
Size preserved
MNIST-Label
MNIST-Inpeet ALL.
MNIST-Ady
clean dara
Memtfiication ALC
ALI.
Clean
Nokey
Identification AUC
Clean
Noisy
Identiication AUC
ALL.
Clean
Noisy
30
0.76
83.67
95.99
69,70
0.68
96.45
97.07
V5.60
0.37
96.36
97.07
95.30
92.82
91.55
96.46
2001
0.99
95.24
95.83
94.35
0.70
96. 34.34
96.47
96.
0.68
96.26
96.50
95.90
0.99
95.60
95.93
95.10
0.71
96.16
96.20
96.
0.70
96.44
96.47
96.40
improvement: and thirdly, our experiments demonstrated that 100 clean samples were sufficient for achieving effective model repair across these datasets.
B Omitted Technical Details in Sec.
C
The Gaussian Mean Estimation Task
Definition C.I. (Gaussian Mean Estimation):
Given training 'D consisting of n training 19+m data { Pizz sampled from the distribution =N(,,, o²) as well as m training data {x; ) i=n+1 sampled from another Gaussian distribution 21' =N(u', o²), where μ and p' are the means of the two Gaussian distributions and の² denotes the variance, the objective to estimate the mean of the target distribution U. The loss function is defined as L(z; 0) = (0 x)²
Lemma C.2. The empirical solution of Gaussian Mean Estimation is 0 m+n Li=1 Iy. With probability at least p for some probability, the generalization error is upper bounded
0 - 0 S a S0llavi
m
B,
(6)
m +
where 0" = pI. is the optimal solution and a o / /2lgg(2/)) some constant.
Proof. Since {a; 1a+m are sampled from 21, {x; - μ + 1 =1 are sampled from U4. Using that
Gaussian tail bounds 34 we have that it holds with probability
1t + +11) - S Lov
2log(2/p)
34
(xg
m + = / ///////n/n/// m)
m +
1=1
1=++1
Therefore, we have
n+m
00
32
Σ
Di
p1
m + 72
i=n+1
1
T2 +
Σ
(x: μl + 44)
11
(1 12))
m + n
n. + m
i=at1
Ti
(x: n1 + 11)
+ mn
111 -n/1
+
1=1
i=n+1
$ Lav1//
1/(n
m)
1 n B.
n n
m
Remark: Note that in cases where the value of m decreases to zero, indicating the absence of any data from the detrimental distribution 4 in the training set, the upper bound is then given by
Bremove a//I/n.
If the distribution divergence between U and 44, measured by || |リ), n²), satisties Ip - 1²) > m+ ox//n 01/nn the training set negatively affects performance, and removing these data can improve the performance, given that the divergence is sufficiently large.
m)), then Bremove < B. This implies that including data from 22' in
16

---

Proposition C.3. (Recap of Proposition In the Gaussian mean estimation task, there exists set of pa + where |1- << |p-p²) such that the optimal empirical solu- aligmment functions:: h(z) ---p'
tion the combined of [ 1 and (h(x:) Jinnt i.e., 0
- m+t -E== Ix+ + Σitt 1((z)))
achieves a generalization bound Bulign B Bremove-
Proof. For any alignment function hxx), similar Eqn.
we have
00
0"
Σ
Maxi)
-M
m. +
24
Σ
(4a 1 + B 10)
4
m
(x
ji)
+
i=++1
m
(8)
n-
Σ
(x: - 1l + +11) 1
m n + m
114 jA
m + n
i=++1
< av 1/(n + m)
+
77
| jil
Balign
11- μll such that 1p satisfies
1e0 701 a
I/n v 1/(n + m)) < <Ip 1ll,
m
we have
Balign av 1/(n + m)
+
m 111 ju]
<av1/(n= m)
m
m + n
al / 1/n 11(m++m))
(9)
m
-av1/1 1/n = =av
Bremove-
Remark: The alignment function aligns the data in L/' with that in 24 If the alignment is effective, that is 1 ü < no at /I/n v1/(n + m)), improves performance over the results of data removal. The degree improvement is positively correlated with the quality of alignment, as quantified by the magnitude of (1ppa() --].
C.1 Cause Identification in the Gaussian Mean Estimation Task
In the Gaussian mean estimation task, we assume that all the data from the detrimental distribution l' are identified before model treatment. this section, we verify the effectiveness of the metric proposed in Eqn. (1) in identifying the cause data in the training set for this task. Specifically, the metric estimates the contribution of failure by measuring the change in loss between the original trained model and an enhanced model, which is trained with larger sample size n from U.
17

---

In the Gaussian mean estimation task, the expected loss of the data from 2/ is
n---m m
(zy 0)
E
IF
24
m
+ 12
E
24
1
π1 Σ
2
m
1
7
Σ
11
171 + n
m
/=1
j=++1
E
24
n
2
1 72
IJj
1
72 7
Σ
3jj
71
7
171 + n
71 j=1
m
J
27
m²
1
1
E
24
2
+ E
2
Z
71
(m + n2
n
771
2
1=n+1
m
0²
(m n)2 (n 12)22
(10)
The forth equality uses the fact that Fi TEE + e/2 Eg E 2 E2 2T
EE()
2
Eig IEg + s² and the fifth equality uses the fact that 1/ - [ 1/2 ش zjj and ンj=n+1 Iy unbiased estimation of o², and pl. respectively. Similarly, the expected loss of the data from 2 is
レーー T
n²
Σ
[(x;; 0) = =
(m + n)2 (n n!)
m
i=++1
If we introduce additional An data train the enhanced model c. similar to Eqn. 10). we have
m²
E
((rii))
02
(m + n + An) 22 -2)²²
n
a+m
(n + An)
((xii))
= +
(m + + Anjz (μ - 12))2
i=++1
Then we have
FB
1
((xu))) - ((r=))
m
m²
(n (w-1²)² <.
(m n An)
(m + n)3
n
n -(14)
(n (n++a An)
n² (m + n2)
S
Σ (c(=;)) Clos; (Lox:)))(((o
(m (m
(μ (1-p2)² 0.
m
An)2
i=++1
Therefore, the detrimental data can be identified by the positive value of s(x;; 0. δ).
Furthermore, this metric can be utilized to identify harmful data, even if they are not part of the training dataset. In the Gaussian mean estimation example, similar Eqn. we can obtain the generalization error bound to the data in li' as
e - d'' < a/1/(nn+tt/t m)
72 n m
|1
na|
(11)
As the number of training samples from 24 increases, both terms in the generalization error bound for li (Eqn. (7) decrease. In contrast, for 2t', the first term / /I/(n + m) decreases, while the second
18

---

term - p- - μ4 increases. Consequently, the generalization enror bound for U decreases more significantly than that Ut. Given that the loss is related to the generalization bound, increase the number training samples from 24 leads to rise in the loss of most samples from 44, while causing a decline in the loss of most samples from U. And we can utilize the metric in Eqn. | to identify any detrimental data.
19


Reviews:
[{"review_id": "kjjD2Wo9Vu", "reviewer": "Reviewer_7x7R", "summary": "This paper proposed a method, BIR, to repair the model trained but possibly corrupted from different types of noise, e.g., label noise and feature noise, leading to training-testing distribution mismatch. As for repair, by assuming the presence of small clean data, the approach identifies corrupted examples using the loss gap between the original and fine-tuned models. Next, it aligns the identified  (potentially) corrupted examples to the target distribution using an energy-based model, followed by fine-tuning with the aligned data. The experiments were conducted on two datasets with multiple variations for different types of noise.\n\n\n\n\n", "questions": "I put all the questions in the weakness sections.", "limitations": "There is no expected potential societal and ethical impact. Also, the authors stated the limitations of the work in Section 5.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "**Methodology Design**: I like the method design involving an additional step (Step 2) after Step 1. This step will make a synergistic effect by handling the misidentified cases from Step 1. Also, the alignment process makes sense and is interesting.\n\n**Problem Setup**: Handling corrupted models is very important. This paper proposed a post-processing/training framework for repairment. This setup has an advantage over 'in-training' methods in the sense that we can apply this method to already pre-trained large model with minimal training cost.", "weaknesses": "**Clean reserved data**: The presence of this data makes a concern about the practicality of the proposed method. For example, the existence of a clean subset is reluctant to be assumed in the label noise area. In this sense, the weakness of this paper is the strong dependency on the reserved clean set. At least, It would be good to add the impact of the amount of clean subset, e.g., how much the identification performance varies.\n\n**Synthetic noise**: In the Introduction section, the authors said \"extensive experiments on real-world datasets\". Is it meaning that MNIST and CIFAR-10 datasets are real? I was expecting to see the results on benchmark datasets with real-world labels and feature noise in data. When looking into the experimental setup, the author used a fixed set of synthetic noise if I understood correctly, e.g., 40% of label flipping, 40% of random corruption (feature), etc. This specific setup for each noise is not a broad study of real noise. Additionally, it would be good to mix different types of noise in a single setup and control the extent to which each noise occurs.\n\n**Size of inspected data**: According to Figure 2, I guess that the size of inspected data in Step 1 will affect the performance of the method significantly. If we know the ratio of corrupted data, we can easily find the optimal size of identification as the author mentioned (i.e., selects top |D_{c}| data points with the largest s(z)). However, it is unclear how to judge the size for the experiments in Tables 1 and 2. Is it also assumed to be known? If not, please elaborate on this. \n\n**Datasets**: MNIST and CIFAR-10 datasets are very small and relatively easy datasets. I would recommend using more challenging datasets for experimentation. For example, CIFAR-100, Tiny-ImageNet, or CIFAR-10N (w. real human mistakes in labeling). Particularly, I think the difficulty of data may affect the performance of Step 1. In this scenario, can Step 2 will handle the negative impact of a larger number of misidentified cases?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: The strong dependency on clean reserved data**\n\n> - You're absolutely right that the reliance on clean reserved data poses a limitation. Nevertheless, it's important to note that a certain quantity of clean data is typically essential for the majority of model repair approaches.\n> \n> - To address this concern, we conducted an investigation into the impact of varying the quantity of clean reserved data on the performance of our approach. Our study encompassed three corrupted MNIST datasets, leading us to observe the following:\n>   \n>   - - With a mere 30 clean samples at our disposal, our method demonstrated enhanced performance on noisy classes, albeit with a diminished identification AUC.\n>     \n>     - As the quantity of clean data increased (from 30 to 300 samples), both identification AUC and overall performance exhibited improvement.\n>     \n>     - Surprisingly, our experiments indicated that 100 clean samples were adequate for achieving effective model repair across these datasets.\n> \n> - The detailed findings from these experiments are comprehensively presented in Tab. 5 (Global Response PDF).\n\n  **W2: Synthetic noise**\n\n- **Q2-1: \"extensive experiments on real-world datasets\" Is it meaning that MNIST and CIFAR-10 datasets are real?**\n  \n  > Indeed, the phrasing might be misleading. To clarify, our intention is to convey that we carried out experiments on datasets that simulate real-world scenarios by introducing synthetic noise. We acknowledge the potential for confusion and will make appropriate adjustments to the statement to ensure clarity.\n\n- **Q2-2: the results on benchmark datasets with real-world labels and feature noise in data.**\n  \n  > - We conducted experiments on CIFAR-10N dataset, wherein we augmented the training set of CIFAR-10 with genuine human-annotated noisy labels. For a more comprehensive understanding, kindly review our response to GQ-1 in the Global Response.\n  > \n  > - We earnestly sought an image dataset characterized by real-world image noise. Regrettably, our attempts to locate such a dataset for image classification were unsuccessful.\n\n- **Q2-3: To mix different types of noise in a single setup**\n  \n  > - We established a corrupted MNIST dataset by following a similar methodology to the creation of MNIST-Label and MNIST-Adv datasets. This involved introducing label noise to 30% of the data within the target classes and applying adversarial attacks to 20% of the data within those same classes.\n  > \n  > - Upon evaluating our method on the newly corrupted dataset, characterized by a blend of noise types, we observed that our method considerably outperform baselines on \"Noisy\" classes and achieve best performance on \"All\" classes, providing additional evidence of its remarkable repair capabilities.\n  > \n  > - A detailed comparison of repair performance outcomes is presented in Tab. 5 (Global Response PDF).\n\n**W3: Size of inspected data**\n\n- **Q3-1: It is unclear how to judge the size for the experiments in Tables 1 and 2. Is it also assumed to be known? If not, please elaborate on this.**\n  \n  > - Please refer to the response to GQ-2 in the Global Response for the details.\n\n- **Q3-2: I guess that the size of inspected data in Step 1 will affect the performance of the method significantly**\n  \n  > The empirical outcomes, as showcased in the response to Q3-1 (or GQ-2), underscore the robust nature of the repair performance across various sizes of identified cause sets. These results demonstrate that the performance remains consistently stable and is scarcely influenced by the size of the inspected data.\n\n**W4: Datasets**\n\n- **Q4-1: recommend using more challenging datasets for experimentation. For example, CIFAR-100, Tiny-ImageNet, or CIFAR-10N (w. real human mistakes in labeling).**\n  \n  > Additional experiments have been performed on CIFAR-10N and Tiny-ImageNet datasets, incorporating simulated label noise. Please refer to the response to GQ-1 in the Global Response for the details.\n\n- **Q4-2: I think the difficulty of data may affect the performance of Step 1. In this scenario, can Step 2 will handle the negative impact of a larger number of misidentified cases?**\n  \n  > Absolutely, the difficulty of data could indeed impact Step 1's performance, potentially leading to an increased count of misidentified cases. However, it's noteworthy that Step 2 demonstrates remarkable resilience when dealing with variations in the size of the identified cause set from Step 1. This resilience is evident in our repair performance results across diverse cause set sizes, as highlighted in our response to GQ2 in the Global Response and also detailed in Tables 3 and 4 of the Global Response PDF. These outcomes suggest that Step 2 retains its effectiveness in repairing the model, even when presented with a greater number of misidentified cause data from Step 1 due to data difficulty."}, {"review_id": "bfvdYXB5iQ", "reviewer": "Reviewer_usXi", "summary": "The authors propose a method for performing targeted repairs of corrupted data in order to improve the quality of an ML model trained on that data. The method is comprised of two steps: (1) *cause identification* -- identifying the set of data examples that are the most likely to be corrupted, and (2) *model treatment* -- performing a targeted treatment of the training dataset with the goal of removing the negative effect of the corrupted data examples while still keeping them in the dataset. For cause identification, they augment the target dataset with a special \"reserved\" dataset that is known to be clean, then they train a model on the augmented dataset, and finally for each training data example, they compare the training loss of the model trained on the augmented data and the model trained on the original data. For model treatment, they propose an \"alignment function\" which can be applied to the features and labels of corrupted data examples, which is specifically calculated to neutralize their negative impact on the model training loss function. The authors evaluate their method on the MNIST and CIFAR10 datasets with three types of corruption: (1) label noise, (2) feature noise, and (3) FGSM-based adversarial attack on features. They show that their method performs favorably against several other methods (mainly various influence function methods).", "questions": "(Q1) It seems to me that in the experiments for Q1 and Q3 it is assumed that the number of corrupted data examples is not hidden from the various methods that are being compared. If this is true, what would happen if this information would indeed be hidden as is the case in real-world settings?\n\n(Q2) If you knew which exact data examples are corrupted (i.e. we can skip the cause identification step), what would be the impact of the model treatment step compared to data removal? (perhaps this could be an additional experiment)", "limitations": "The limitations are clearly stated in the paper.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "(S1) The problem being solved is very relevant and the proposed solution is quite interesting.\n\n(S2) The paper is well-written and I found it relatively easy to follow.\n\n(S3) The experiments show a clear benefit that the method provides over the baselines.", "weaknesses": "(W1) The introduction is only slightly repetitive as some points are delivered multiple times.\n\n(W2) The experiments do not compare against methods based on the Shapley value that are also commonly used for solving this problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: The introduction is only slightly repetitive as some points are delivered multiple times.**\n\n> We will enhance the introduction to ensure greater conciseness. \n\n**W2: The experiments do not compare against methods based on the Shapley value**\n\n> - To address this gap, we have introduced an experiment wherein we use Beta-Shapley [R1], a generalized Data Shapley [6], to identify the cause set in corrupted MNIST. \n> \n> - The identification AUC of Beta-Shapley for MNIST-Label, MNIST-Input and MNIST-Adv is 0.84, 0.70, 0.68, respectively, while the corresponding identification AUC for our method is 0.99, 0.71, and 0.70. \n> \n> - The performance of Beta-Shapley in identification is slightly worse than our proposed method. Additionally, the computational time required by the Shapley value-based approach far exceeds that of our proposed method.\n> \n> [R1] Kwon and Zou. Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning. AISTATS 2022.\n\n**Q1: the number of corrupted data examples**\n\n> Please refer to the response to GQ-2 in Global Response. \n\n**Q2: what would be the impact of the model treatment step compared to data removal?**\n\n> In Table 3 and 4 (original manuscript), we present a comparison of various model treatment algorithms alongside data removal methods, utilizing the ground-truth cause data as a reference point. Newton Removal and EWC-Deletion are two data removal methods. \n> \n> - For both MNIST-Labeled and CIFAR10-Label datasets, the performance of these two data removal methods significantly improves compared to the case where identified cause data is used.\n> \n> - Notably, in all our experimental cases, the BIR technique consistently maintains its superiority in \"Noisy\" classes. This observation underscores that the proposed BIR method effectively surpasses the performance of the two data removal methods in the task of repairing the model by mitigating the impact of detrimental data."}, {"review_id": "3KMWZ7B3p3", "reviewer": "Reviewer_uAW7", "summary": "This paper proposes a method to detect and repair corrupt data in the training set, in an effort to align the model with the distribution of the test set. The method is two-stage, first detecting based on a loss difference computation per-sample, and then repairing through the use of an energy-based model. Several experiments are conducted to demonstrate the efficacy of the approach on multiple types of noisy data.\n\n**Rebuttals:** The authors have addressed my questions in their rebuttal, and I have updated my score from 7 to 8.", "questions": "- Is Equation 1 expensive to perform when running on thousands of samples, with a possibly large model?\n- L132: please address the case where the size of $\\mathcal{D}_c$ is unknown.\n- How reliable is Equation 1? Suppose we have $|\\mathcal{D}_c| = 20$, and so we pick samples from $\\mathcal{D}$ with the 20 highest values for $s$. (a) What if different points of convergence of, say, $\\hat{\\phi}$ yields different top 20 points? (b) What if the difference between $s$ at the 20th and the 21st point is negligible? Is there a way to determine confidence that sample $z \\in \\mathcal{D}_c$?\n- I'm not entirely sure how the equation under L566 follows from Hoeffding's inequality, could you please elaborate?\n- I understand the result of Proposition 2.1, and it seems the next logical step would be to find a $\\tilde{\\mu}$ that satisfies L580 in Appendix B.2. I'm not sure how the next paragraph (starting at L221) relates to this, could you clarify?\n- As future work, it would be interesting to see if BIR also works for more sophisticated adversarial attacks than FGSM, such as Carlini & Wagner [1] or Szegedy et al., 2013 [2]. In addition, if BIR is being positioned as a method to help with adversarial attacks, it might be useful to compare against defenses such as adversarial training (see [3-7] for a few works). It would also be interesting to see Fig. 2(d) and 2(g) with those methods as well.\n\n[1] Carlini, N., & Wagner, D. (2017, May). Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) (pp. 39-57). Ieee.  \n[2] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.  \n[3] Kurakin, A., Goodfellow, I., & Bengio, S. (2016). Adversarial examples in the physical world.  \n[4] Dhillon, G. S., Azizzadenesheli, K., Lipton, Z. C., Bernstein, J., Kossaifi, J., Khanna, A., & Anandkumar, A. (2018). Stochastic activation pruning for robust adversarial defense. arXiv preprint arXiv:1803.01442.  \n[5] Guo, C., Rana, M., Cisse, M., & Van Der Maaten, L. (2017). Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117.  \n[6] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.  \n[7] Samangouei, P., Kabkab, M., & Chellappa, R. (2018). Defense-gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprint arXiv:1805.06605.", "limitations": "Limitations have been adequately addressed.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Methodologically, this paper is excellent, and is very well-written. The motivation is clear and strong; the theory is sound; the experimental evaluation shows, at times, significant improvements over baselines and other approaches.", "weaknesses": "There are few weaknesses of this paper, mostly comprised of minor errors:\n\n- minor: L95: pair --> pairs\n- minor: L120: mismatching --> mismatch\n- minor: L162: Beysian --> Bayesian\n- minor: L206: minimizaiton --> minimization\n- minor: L226: iterative --> iteratively\n- In Eq. 7, second line, I believe the last term should have a minus sign instead of a plus. This should not change the results because of the next step, though. This also applied to Eq. 8 line 2.\n- minor: Fig. 2 seems to start from (b), please fix", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: Minus sign in Eq. 7**\n\nYes, it should be a minus sign. \n\n**Q1: Is Eqn. 1 expensive to perform?**\n\n- we carried out a comprehensive time assessment, comparing the performance of four identification algorithms – namely the proposed BIR, Linear Influence, SGD Influence, and EWC-Influence - on the CIFAR10-Label dataset with ResNet5. This experimentation was conducted on a V100 server. \n- In the case of our proposed BIR method, the computation of Eqn. 1 (Step 2-5 within Algorithm 1) for all 45,000 data samples within the training set takes approximately 333.4 seconds.\n- Contrastingly, Linear Influence, SGD Influence, and EWC-Influence necessitate more substantial time commitments, exceeding 4 days, 27 hours, and 515.1 seconds, respectively.\n- Importantly, the estimation time required by our method and EWC-Influence demonstrates proximity, while remaining significantly lower than that of Linear and SGD Influence approaches.\n\n**Q2: please address the case where the size of $\\mathcal{D}_c$ is unknown.**\n\nPlease refer to the response to GQ-2 in Global Response. \n\n**Q3: How reliable is Eqn. 1?**\n\nWhile the precise identification of cause data isn't pivotal for achieving successful model repair, we recognize the importance of assessing the reliability of Eqn. 1. \n- Notably, the repair performance of BIR demonstrates robustness across variations in the size of the identified cause set, This resilience is illustrated by our experimental results across different cause set sizes (in our response to GQ-2). This insight implies that pinpoint accuracy in the identification of cause data might not be a critical factor.\n- If the difference between the 20th and 21st points is negligible, we adopt a random selection approach. If a threshold on the score is implemented, both points will be selected if above the threshold. \n- We computed the AUC between the scores derived from Equation 1 and the ground truth cause set labels for different enhanced models (i.e. $\\hat{\\phi}$). Intriguingly, the AUCs exhibited minimal variance, differing by less than 0.005. \n\nCurrently, we lack a definitive method to ascertain confidence levels. Exploring avenues to quantify confidence remains a valuable avenue for future research endeavors.\n\n**Q4: How the equation under L566 follows from Hoeffding's inequality?**\n\n> - To provide a more accurate derivation, we should employ Gaussian tail bounds instead of Hoeffding's inequality for obtaining the equations outlined in L566. It's worth noting that Hoeffding's inequality and Gaussian tail bounds are closely related and share a similar proof methodology. This clarification will be addressed in our revision.\n> \n> - The procedure involving Gaussian tail bounds to deduce the equation within L566 is exemplified as follows.\n>   \n>   - The Gaussian tail bounds (Eqn. 2.7 in [R2]):  for a Gaussian random variable $X \\sim \\mathcal{N}(\\mu, \\sigma^2)$, we have $P[X \\ge \\mu + t] \\le e^{\\frac{-t^2}{2 \\sigma^2}}$\n>   - Since the Gaussian distribution is a stable distribution, the sum of Gaussians is also a Gaussian, therefore $\\bar{X}_n \\sim \\mathcal{N} (\\mu, \\sigma^2/n)$, where $\\bar{X}_n$ is the mean of $n$ random variables sampled from $\\mathcal{N}(\\mu, \\sigma^2)$. \n>   - Therefore, we have $P[\\bar{X}_n \\ge \\mu + t] \\le e^{\\frac{- n t^2}{2 \\sigma^2}}$. \n>   - Let $t=\\sigma \\sqrt{\\frac{2\\log(2/p)}{n}}$, we have $P[\\bar{X}_n - \\mu \\ge \\sigma \\sqrt{\\frac{2\\log(2/p)}{n}}] \\le \\frac{p}{2}$.\n>   - Bounding from both sides, we have $P[| \\bar{X}_n - \\mu| \\ge \\sigma \\sqrt{\\frac{2\\log(2/p)}{n}}] \\le p.$.\n>   - In Lemma B.2, for $i=1,...,n$, $x_i$ are $n$ random variables from $\\mathcal{N} (\\mu, \\sigma^2)$. For $i=n+1,...,n+m$, $x_i' = x_i - \\mu ' + \\mu$  are $m$ random variables from $\\mathcal{N} (\\mu, \\sigma^2)$. Defining $\\bar{X}\\_{n+m} = \\frac{1}{m+n}(\\sum_{i=1}^n x_i + \\sum_{i=n+1}^{n+m} (x_i - \\mu ' + \\mu))$,  we have $\\bar{X}\\_{n+m} \\sim \\mathcal{N} (\\mu, \\frac{\\sigma^2}{m+n})$. And we have with $1-p$ probability, $| \\bar{X}_{n+m} - \\mu| \\le \\sigma \\sqrt{\\frac{2\\log(2/p)}{n}}$\n> \n>  [R2] Wainwright, Martin. \"Chapter 2: Basic tail and concentration bounds.\" 210B Lecture Notes University (2015). \n\n**Q5: how the next paragraph (starting at L221) relates to the results of Proposition 2.1?**\n\nThe results of Proposition 2.1 suggest that in the Gaussian mean estimation task, we can improve model performance by aligning unfavorable data with the target distribution.\n\nTherefore, we would like to first generalize the idea of alignment in the Gaussian mean estimation task to general tasks. However, the question of how to align data in general tasks remains. The paragraphs starting with L221 aim to address this issue by introducing the idea of using energy-based models for alignment in more general tasks.\n \nWe will add more details to better elucidate the connections.\n\n**Q6: As future work, it would be interesting to see if BIR also works for more sophisticated adversarial attacks than FGSM**\n\n- We perform $L_\\infty$ attack proposed in [1] on CIFAR-10 and construct a corrupted dataset in a  similar way to CIFAR10-Adv. \n\n- The performance of BIR is similar to the performance on CIFAR10-Adv. For example, on ResNet50, we achieve the following results:\n  \n||All|Clean|Noisy|\n|-|-|-|-|\n|Base Model|88.08|92.57|81.35|\n|BIR|90.54|94.17|85.10|\n\n[1] Carlini, N., & Wagner, D. (2017). Towards ... neural networks. \n\n**Q7: It might be useful to compare against defenses such as adversarial training. It would also be interesting to see Fig. 2(d) and 2(g) with those methods as well.**\n\n- We perform the widely-used adversarial training method [6] on MNIST-Adv and  CIFAR-Adv. The results are presented in Tab. 7 (in Global Response PDF). \n- The performance of the model trained with AT is lower than that of the model repaired with BIR. It is possible that AT impairs the performance on clean data, resulting in a lower performance than BIR in this comparison.\n\n[6] Madry, A., et al. (2017). Towards ... adversarial attacks."}, {"review_id": "S3RnaDevJa", "reviewer": "Reviewer_9imB", "summary": "This paper proposes to identify harmful data by using the clean dataset, and then update models with the alignment technique to improve the performance on the corrupted data while retaining the performance on the clean data. Experimental results show the effectiveness of the proposed method, demonstrating its superiority over existing methods in identifying harmful data and repairing DNN models.\n", "questions": "- As claimed in the paper (e.g., Line 71 to Line 76), the proposed method has the advantage over existing methods in that it allows the model to learn useful information from the clean data that are misidentified as corrupted data, thus making it more robust compared to other methods. However, from the results presented in Table 3 and Table 4, I noticed that when the experimental setup was \"Identified set = 75% ground-truth cause set and 25% clean data\" in all six cases, there were always baselines that outperform the proposed method in terms of \"Test Clean\". Does this contradict the claims made in the paper?\n\n- Why use ResNet50 on CIFAR-10? ResNet50 is designed for classification on ImageNet, while ResNet56 is designed for CIFAR-10. \n\n- How was the proportion (40% and 30%) of corrupted samples determined in the experiments? How does this setting (different proportions) affect the results? In actual scenarios, would there be such a high number of corrupted samples in the training set? If the proportion is reduced, is Equation (1) still sensitive enough to accurately identify the corrupted data points?\n\n- Line 215: \\sum_{i=1}^n h(x_i) should be \\sum_{i=n+1}^{n+m} h(x_i) ?\n\n- In the caption of Table 1: \"base mode\" -> \"base model\" ?", "limitations": "The authors described the limitations in the paper.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Aims at an important and relevant problem.\n\n- The experimental results on identifying mislabeled data points are good.", "weaknesses": "- The paper claims that the experiments were conducted on \"real-world datasets\" (Line 89). However, as mentioned in the experiments (Line 275), the corruption cases were simulated. Additionally, the datasets evaluated are relatively small (i.e., MNIST and CIFAR-10). ImageNet dataset could potentially be a valuable supplement, which could contain many mislabeled, real-world samples.\n\n- It would be better to show how much the average improvement is over the baselines, which could help readers better understand the superiority of the proposed method. For the results on input noise and adversarial attacks, the absolute improvements over some baselines are actually quite small.\n\n- Section 1, particularly Line 64 to Line 80, is not clear to me. The alignment process is crucial for the proposed framework. These two paragraphs repeatedly emphasize the benefits of alignment. However, what alignment means is not clearly described. It could be helpful to give some descriptions of how the alignment process works and what the results of the alignment process are.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: dataset**\n\n> Please refer to our response to GQ-1 in Global Response for the details of our additional experiments.\n\n**W2: the absolute improvements ... quite small.**\n\n> - For label noise, our enhancements span a remarkable 20.5% to 27.6% for MNIST and 15.3% to 16.8% for CIFAR10, showcasing substantial progress.\n> \n> - For input noise, our advances are more measured:\n>   \n>   - For MNIST-Input, the baseline models already approach optimality due to minimal input noise influence. This presents a formidable challenge for repair, resulting in similar outcomes across most baselines.\n>   \n>   - With CIFAR10-Input, ResNet20 and ResNet50 exhibit differential performances among the best baselines, yielding a gap exceeding 3%. Our approach establishes robust, competitive results that align favorably with the premier baseline in each case.\n> \n> - For adversarial attacks, our method consistently achieves improvements exceeding 1%, signifying a dependable edge over baselines.\n> \n> - In summary, while our progress is more tempered for certain contexts like input noise, we attain notable headway for label noise and sustain reliable progress across adversarial examples.\n\n**W3: How the alignment process works and what the results of the alignment process are.**\n\n> Our approach adopts an energy-based alignment strategy to diminish the disparity between deleterious data and target distributions. Specifically, we employ the enhanced model as an energy-based model (EBM), utilizing it to gauge the probability density of target data. The alignment process is initiated with detrimental samples drawn from the training distribution. These initial samples undergo iterative refinement through energy gradient descent within the EBM framework. This iterative progression gradually shifts the samples toward higher-density areas of the target distribution. This concerted gradient descent fosters alignment of training data with the EBM, culminating in an aligned dataset that emulates the characteristics of the target data. This alignment process, tantamount to effective data augmentation, yields training data that closely mirrors samples from the true target distribution.\n\n**Q1: In all six cases, there were always baselines that outperform the proposed method in terms of \"Test Clean\". Does this contradict the claims: the proposed method is more robust**\n\n> - The crux of our argument rests on measuring robustness by evaluating the performance dip observed in \"Clean\" classes, as opposed to mere absolute accuracy on those classes when the identified set encompasses misidentified data.\n>   \n>   - The decline in \"Clean\" class accuracy mirrors the price of rectifying the disruptive impact of \"Noisy\" class data within the model. Applying BIR to mend the 25% clean data, instead of corrupted data, does influence its \"Clean\" class performance.\n>   \n>   - Notably, the Table below illustrates that BIR manifests the least average and maximal performance reduction in comparison to alternative methods, affirming its heightened robustness against misidentification. \n>   \n>   | | average performance drops | maximal performance drops |\n>   | -| - | - |\n>   | Fine-tune | 0.58 | 0.9 |\n>   | Newton Removal | 0.25 | 1.1 |\n>   | EWC-Deletion | 0.84 | 1.37 |\n>   | RDIA | 7.86 | 13.46 |\n>   | Ours | 0.09 | 0.46 |\n\n**Q2: Why use ResNet50 on CIFAR-10?**\n\n> We follow our closely related work [33] (EWC-Repair), which similarly employs ResNet50 for CIFAR-10. In a complementary vein, we also delve into ResNet-20, an avenue unexplored in [33]. This diversification in model capacities empowers us to assess the efficacy of our approach across diverse configurations.\n\n**Q3: The proportion of corrupted samples**\n\n- **Q3-1: How was the proportion (40% and 30%) of corrupted samples determined?**\n  \n  > We follow our closely related works [33] (EWC-Repair) to set the corruption proportion ranging from 30% to 40%. \n\n- **Q3-2: How does different proportions affect the results?**\n  \n  > The results on CIFAR-10N, as presented in the response to W1 (GQ-1, Global Response), demonstrate the effect of different corruption proportions of data well. CIFAR-10N has three noise rates: ~40% (worst), 9% (aggregate) and 17% (random).\n  > \n  > The results show that:\n  > \n  > - The higher the noise rate, the lower the performance after repair, indicating that noise rate affects the repaired model's performance.\n  > \n  > - The higher the noise rate, the greater the repair gain, i.e. the increase in performance after repair compared to the base model.\n\n- **Q3-3: In actual scenarios, would there be such a high number of corrupted samples?**\n  \n  > Yes, the proportion of corrupted samples could indeed be more than 40% in real-world scenarios. For example, in the CIFAR-10N real-world human-annotated dataset, the noise rate in the \"worst\" setting is 40.21%.\n\n- **Q3-4: If the proportion is reduced, is Eqn (1) still sensitive enough?**\n  \n  > We evaluated Equation (1)'s efficacy in pinpointing corrupted data when the proportion of noise diminishes. The AUC for CIFAR-10N Worst, Rand2 and Aggregate are 0.89, 0.91 and 0.79, respectively.\n  > \n  > Notably, AUCs hold high for datasets bearing noise rates surpassing 17%, affirming Equation (1)'s accuracy in identifying noisy data within such contexts. Yet, the relatively modest AUC for low noise rate datasets implies a potential challenge for Equation (1) when grappling with a scarcity of noisy data—likely due to their limited impact on model performance.\n  > \n  > Overall, with dwindling noise proportions, Equation (1) remains retains its sensitivity, effectively pinpointing the majority of corrupted data for noise rates ranging from moderate to high, but may grapple with very low noise rates where the influence of noisy samples is marginal."}]

Author rebuttal:
Thank you for providing your comprehensive review and valuable feedback regarding our paper. We deeply appreciate your insights, and we are fully committed to addressing your suggestions and improving the quality of our work. Please find below our global response and our specific responses to each reviewer's comments. If there are any remaining concerns or additional aspects you'd like us to consider, we kindly request your input as we strive to enhance our research further.

In response to several reviewers' concerns, we have taken the following actions:

**GQ-1: Results with real-world noise and challenging datasets**

- We conducted experiments on CIFAR-10N, which equips the CIFAR-10 training set with human-annotated noisy labels at varying noise rates (worst: 40%, aggregate: 9%, random: 17%).
  
  - Our method achieves superior performance across all noise levels, demonstrating its effectiveness in mitigating real-world label noise.
- To address the need for a more complex dataset, we produced two corrupted versions of the Tiny-ImageNet dataset, a larger and more challenging dataset with more classes than MNIST and CIFAR-10, to simulate real-world noise scenarios:
  
  - 1. Created a symmetric noise dataset by randomly flipping labels to other classes with a 40% probability.
  - 2. Generated an asymmetric noise dataset by randomly flipping labels to the subsequent classes with a 40% probability. 
- As shown in Tab. 1 (Global Response PDF), our proposed BIR method consistently outperforms baseline methods across both CIFAR-10N and Tiny-ImageNet datasets. These findings serve as robust empirical support for the state-of-the-art capabilities of our approach in addressing real-world label noise.
  

**GQ-2: How to determine the number of identified data when the size of $\mathcal{D}_c$ is unknown.**

- As highlighted in lines 522-526 (Appendix), we set the amount of identified data based on the ground-truth quantity of corrupted data in both experiments..
  
- In cases where the precise number of corrupted data examples is uncertain, determining an appropriate threshold becomes challenging. As a practical solution, we can apply a fixed threshold on the score ($s$) computed using Eqn. (1) as a heuristic measure.
  
  - Although setting the threshold to 0 may seem straightforward, which selects all training data with an increased loss, it often identifies more data than the actual number of corrupted points.
    
  - Empirical investigations revealed that utilizing a threshold between $\log(1.01)$ and $\log(1.1)$ effectively filters out identified data close in size to the ground truth. Refer to Tab. 2 (Global Response PDF) for results using different thresholds. Our recommendation is to employ a threshold of $\log(1.05)$ for practical applications.
    
- Importantly, we wish to emphasize that the repair performance of our BIR method remains robust irrespective of the size of the identified cause set. This resilience is evident from the results outlined in Tabs. 3 and 4 (Global Response PDF). While the precision of identification through thresholds may not be perfect, the repair performance of our BIR approach remains largely unaffected.
  
  - Our experimental setup in Table 3 involves evaluating repair performance across various sizes of identified cause sets. The size of the cause set is determined either by the ground-truth number of corrupted data or by employing different thresholds on the score derived from Eqn. (1).

Meta-review: This paper proposes a new data-driven model reparing method. 

While initially some reviewers raised concerns on exeperimental results, the authors successfully address those issues during the rebuttal period. Therefore, all reviewers gave acceptance scores for this paper.

AC also agrees with the reviewers, so recommends accepting this paper.
Decision: Accept (poster)