SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\eta$  is a hyperparameter that controls the adaptation rate. In contrast to existing works [18] which trains an extra EBM to perform alignment, we adopt the approach proposed in [7], which utilizes the classification model  $\theta$  as an energy-based model (EBM) to represent the data distribution density as  $p_{\theta}(x,y) = \frac{\exp(f_{\theta}(x)[y])}{Z(\theta)}$ , where  $f_{\theta}$  is a classification model which maps each data point  $x$  to the logits of  $K$  classes,  $f_{\theta}(x)[y]$  denotes the logit corresponding to the  $y$ -th class and  $Z(\theta)$  is the unknown normalization constant.
We use target-distribution-enhanced model  $\hat{\phi}$  to approximately characterize the distribution  $\mathcal{Z}$ . Considering potential label noise, we update  $x$  using the energy function of  $p_{\hat{\phi}}(x)$  instead of  $p_{\hat{\phi}}(x,y)$ , where we obtain  $p_{\hat{\phi}}(x) = \sum_{y'} p_{\hat{\phi}}(x,y') = \frac{\sum_{y'} \exp(f_{\hat{\phi}}(x)[y'])}{Z(\hat{\phi})}$  by marginalizing out  $y$ . The corresponding energy function is defined as
$$
E _ {\hat {\phi}} (x) = - \log \sum_ {y ^ {\prime}} \exp \left(f _ {\hat {\phi}} (x) \left[ y ^ {\prime} \right]\right). \tag {3}
$$
With the energy function, we align the data  $(x,y)\in \hat{\mathcal{D}}_c$  by first aligning the input  $x$  by one-step gradient descent as
$$
\tilde {x} = x - \eta \nabla_ {x} E _ {\hat {\phi}} (x), \tag {4}
$$
then aligning the label  $y$  by
$$
\tilde {y} = p _ {\hat {\phi}} (y | \tilde {x}) = \frac {\exp \left(f _ {\hat {\phi}} (\tilde {x})\right)}{\sum_ {y ^ {\prime}} \exp \left(f _ {\hat {\phi}} (\tilde {x}) \left[ y ^ {\prime} \right]\right)}, \tag {5}
$$
which is the soft label of  $\tilde{x}$ .
After obtaining the aligned data, we update the model from  $\hat{\theta}$  using the aligned data. Similar to the update of enhanced model  $\hat{\phi}$  we update only the layers selected in the cause identification step while keeping the remaining layers frozen.
Finally, the proposed model repair method, named Beneficial Information Retaining (BIR) Model Repair, including the cause identification and model refinement, is summarized in Algo. 1 and illustrated in Fig. 1.
Our work is closely related to [33], following which we formulate the model impairment problem into a cause identification step and a model treatment step. In [33], Tanno et al. discuss the model impairment problem from a Bayesian perspective and introduce EWC-influence for cause identification and EWC-deletion for model treatment which performs continual unlearning to remove the data. The cause identification relies on data influence estimation, which can be solved by various influence estimation methods [10] such as linear influence [20, 22], SGD influence [12] and Data Shapley Value [6, 16]. For model treatment, existing works [12, 23] proposes to remove the information of the failure cause data by retraining the model with the remaining data or by performing machine unlearning [8, 29]. RDIA [23] proposes an influence-based relabeling framework for reusing harmful training samples. However, RDIA's aggressive relabeling of benign data that is mistakenly identified as detrimental can introduce incorrect information and negatively impact the model's performance,
Table 1: Comparison of model repair performance on corrupted MINST datasets. Performance better than base model is highlighted by cyan and the best performance among all baselines is in bold.
Model	Dataset	MNIST-Label	MNIST-Input	MNIST-Adv
Metric	All	Clean	Noisy	All	Clean	Noisy	All	Clean	Noisy
LeNet	Base Model	82.46	96.10	62.00	96.46	97.10	95.50	96.38	97.33	94.95
SGD Influence Repair	84.40	95.67	67.50	96.12	96.90	94.95	96.06	96.97	94.70
Linear Influence Repair	83.00	96.43	62.85	96.26	97.07	95.05	96.50	97.27	95.35
EWC Repair	84.26	95.43	67.50	96.12	96.90	94.95	96.06	96.97	94.70
RDIA	72.70	93.03	42.20	85.82	90.83	78.30	81.30	86.00	74.25
BIR (Ours)	95.60	95.93	95.10	96.16	96.20	96.10	96.44	96.47	96.40
Conv4	Base Model	83.96	97.87	63.10	98.00	98.57	97.15	97.82	98.57	96.70
SGD Influence Repair	88.66	98.60	73.75	97.76	98.40	96.80	97.00	97.93	95.60
Linear Influence Repair	72.12	87.97	48.35	96.96	97.43	96.25	92.86	98.43	84.50
EWC Repair	89.02	98.53	74.75	97.82	98.50	96.80	97.10	97.90	95.90
RDIA	74.04	90.03	50.05	82.76	79.77	87.25	77.96	87.17	64.15
BIR (Ours)	96.36	97.10	95.25	98.02	98.93	96.65	98.12	98.43	97.65
Table 2: Comparison of model repair performance on corrupted CIFAR-10 datasets.
Model	Dataset	CIFAR10-Label	CIFAR10-Input	CIFAR10-Adv
Metric	All	Clean	Noisy	All	Clean	Noisy	All	Clean	Noisy
ResNet20	Base Model	74.80	90.83	48.75	87.20	91.77	80.35	87.88	93.40	79.60
SGD Influence Repair	76.66	92.40	53.05	89.24	93.00	83.60	86.06	92.13	76.95
Linear Influence Repair	74.80	92.30	48.55	87.88	92.83	80.45	87.72	93.00	79.80
EWC Repair	77.06	90.77	56.50	87.44	93.37	78.55	86.70	92.00	78.75
RDiA	67.50	85.17	41.00	79.66	85.20	71.35	77.54	82.23	70.50
BIR (Ours)	83.98	91.10	73.30	87.96	91.40	82.80	87.94	92.17	81.60
ResNet50	Base Model	77.10	90.60	50.95	86.62	94.30	75.10	87.60	93.13	79.30
SGD Influence Repair	78.36	94.17	54.65	89.36	95.30	80.45	84.08	94.67	68.20
Linear Influence Repair	75.82	93.83	48.80	88.28	93.90	79.85	87.60	93.67	78.70
EWC Repair	78.00	93.57	54.65	89.94	94.47	83.15	84.08	94.67	68.20
RDiA	66.42	90.13	30.85	76.24	87.33	59.60	71.44	84.20	52.30
BIR (Ours)	83.02	91.73	69.95	88.64	91.80	83.90	90.26	93.20	85.85
Table 3: Comparison of model repair performance on CIFAR-10N. The network architecture is ResNet20.
CIFAR-10N
Worst	Random	Aggregate
Base Model	77.76	84.68	85.78
Linear Influence Repair	79.30	85.22	87.70
SGD Influence Repair	71.50	77.21	76.08
EWC Repair	71.80	76.22	77.18
RDIA	62.90	62.96	64.62
BIR (Ours)	85.50	88.44	89.60
making it a high-risk approach in the case of imperfect detrimental data identification. Our proposed approach BIR also reuses the identified data to improve model performance. By aligning the identified detrimental dataset with the clean data, our framework rectifies the detrimental information while preserving the beneficial information in the identified set, and updates the model using the aligned data. Unlike RDIA, our alignment strategy does not introduce incorrect information when applied to clean data and achieves satisfactory performance even in cases where there are mistakenly identified clean data in the cause set.
To evaluate the effectiveness of the proposed BIR model repair algorithm, we conduct extensive experiments to answer the following research questions: Q1: How does BIR compare to the state-of-the-art baselines in repairing neural networks that produce error predictions? Q2: How does the proposed failure cause data identification metric compare to the baselines in identifying the failure cause data? Q3: How does the proposed align-then-update strategy compare to the baselines in model treatment when considering both perfect and imperfect failure cause sets?
Datasets and Network Architectures: We evaluate the effectiveness of model repair algorithms on corrupted versions of MNIST [26] and CIFAR-10 [24] datasets following the setups in [33]. We consider annotation error, input noise, and adversarial attacks in the training data. To simulate annotation error, we randomly flip about  $40\%$  of labels between specific similar digits or classes. For input noise, we randomly corrupt  $40\%$  of MNIST images and  $30\%$  of CIFAR-10 images in a selected set of target classes by adding specific types of noise, such as salt-and-pepper in MNIST or Gaussian noise in CIFAR-10. Additionally, we construct datasets by adversarially attacking  $40\%$  of randomly selected training images in the same target classes using FGSM. We denote the corrupted datasets with label noise, input noise, and adversarial attacks by MNIST-Label, MNIST-Input, and MNIST-Adv
(d) CIFAR10-Label
(b) MNIST-Input
(e) CIAFR10-Input
(c) MNIST-Adv
(f) CIFAR10-Adv
Figure 2: Comparison of Failure Cause Identification on LeNet trained with corrupted MNIST datasets and ResNet50 trained with CIFAR-10.
(CIFAR10-Label, CIFAR10-Input, and CIFAR10-Adv). For MNIST, 3000 samples from the original training set are utilized while the complete original training dataset of CIFAR-10 is employed to train the base models, with a validation set consisting of  $10\%$  of the data. Furthermore, we partitioned the original test set, which is uncorrupted, into a reserved set consisting of 300 samples for MNIST and 1000 samples for CIFAR-10, while the remaining set was designated as the testing set. The selected target classes of MNIST are 1,7,6, and 9; and the selected target classes of CIFAR-10 are 'plane', 'bird', 'cat', and 'dog'. LeNet and Conv4, which is a 4-layer CNN with a fully connected layer, are employed as classifiers for MNIST, while ResNet-20 and ResNet-50 are utilized for CIFAR-10. We also conduct experiments on CIFAR-10N dataset [37], which equips the training set of CIFAR-10 with real-world human-annotated noisy labels. There are 3 noise levels: Worst ( $\sim 41\%$ ), Random ( $\sim 18\%$ ), and Aggregate ( $\sim 9\%$ ).
Baselines: We compare the proposed method with EWC Repair [33], RDIA [23], SGD-influence Repair [12] and Linear-influence-based Repair. EWC Repair utilizes EWC-influence to identify harmful data and EWC-deletion to remove them from the model. RDIA uses linear influence [20] to identify causes, relabel the identified data using an influence-based relabeling function, and retransits the model using the relabeled and remaining data. SGD-influence Repair identifies harmful data using SGD-influence and proposes to retrain the model using the remaining data. Linear-Influence Repair uses the linear influence function [20] to identify the cause data and performs Newton-update removal [8] to eliminate the identified data's information from the model. In practice, retraining the model from scratch can be computationally prohibitive. Therefore, for RDIA, we performed fine-tuning by using both the relabeled data and an equivalent amount of randomly selected data from the remaining dataset. For SGD-influence Repair, fine-tuning was conducted on a small partition of the remaining data that matched the size of the identified data.
Performance Measure: We measure the performance of model repair by the accuracy of the entire test set (denoted by "Test"). Besides, we measure the success rate of the repair by the test accuracy of the selected corrupted classes (denoted by "Noisy") and the performance maintaining rate by the test accuracy of the remaining classes (denoted by "Clean").
Additional information regarding the setups
4.1 Comparison to Model Repairment Baselines
The comparison results for corrupted MNIST and CIFAR-10 are provided in Tabs. 1 and 2. We can observe from the results that the proposed BIR repair algorithm successfully repairs the model and improves the performance of noisy classes in all experiments except for MNIST-Input with Conv4. In contrast, most baselines fail to repair the model for MNIST-Input, MNIST-Adv and CIFAR10-Adv.
Table 4: Comparison of Model Treatment for LeNet trained on corrupted MNIST.
Dataset	MNIST-Label	MNIST-Input	MNIST-Adv
Metric	Test All	Test Clean	Test Noisy	Test All	Test Clean	Test Noisy	Test All	Test Clean	Test Noisy
Base Model	82.46	96.10	62.00	96.46	97.10	95.50	96.38	97.33	94.95
Retrain	96.66	97.43	95.50	96.84	97.57	95.75	96.96	97.53	96.10
Identified set = ground-truth cause set
Fine-tune	95.56	96.90	93.55	96.42	97.17	95.30	96.26	97.17	94.90
Newton Removal	85.20	96.33	68.50	96.52	97.17	95.55	96.50	97.33	95.25
EWC-Deletion	95.10	96.00	93.75	96.14	96.50	95.60	96.24	96.57	95.75
RDIA	95.70	97.23	93.40	95.94	96.63	94.90	92.50	92.67	92.25
BIR (Ours)	95.64	95.67	95.60	96.78	97.11	96.30	96.66	97.13	95.95
Identified set = 75% ground-truth cause set ∪ 25% clean data
Fine-tune	88.18	96.23	76.10	95.98	96.37	95.40	96.22	96.53	95.45
Newton Removal	83.92	96.47	65.10	96.16	96.87	95.10	96.36	97.30	94.95
EWC-Deletion	87.74	94.70	77.30	95.74	96.40	94.75	95.46	96.03	94.60
RDIA	84.86	83.77	86.50	90.48	90.07	91.10	85.82	89.40	80.45
BIR (Ours)	95.70	96.47	94.55	96.48	96.67	96.19	96.62	96.80	96.35
Table 5: Comparison of Model Treatment for ResNet50 trained on corrupted CIFAR-10.
Dataset	CIFAR10-Label	CIFAR10-Input	CIFAR10-Adv
Metric	Test All	Test Clean	Test Noisy	Test All	Test Clean	Test Noisy	Test All	Test Clean	Test Noisy
Base Model	77.10	90.60	50.95	86.62	94.30	75.10	87.60	93.13	79.30
Retrain	88.18	94.03	79.40	87.48	91.43	81.55	88.10	93.07	80.65
Identified set = ground-truth cause set
Fine-tune	85.16	94.83	70.65	89.86	94.40	83.05	89.66	94.67	82.15
Newton Removal	84.04	93.60	69.70	86.72	92.60	77.90	87.60	93.80	78.30
EWC-Deletion	84.50	94.73	69.15	89.76	94.33	82.90	88.42	94.20	79.70
RDIA	82.64	94.43	64.95	87.20	92.07	79.90	86.98	92.40	78.85
BIR (Ours)	83.26	91.43	71.00	88.84	91.97	84.15	90.60	93.43	86.10
Identified set = 75% ground-truth cause set ∪ 25% clean data
Fine-tune	83.18	94.47	66.25	89.00	93.50	82.25	89.48	94.57	81.85
Newton Removal	76.08	94.17	48.95	86.72	91.50	79.45	87.88	93.00	80.20
EWC-Deletion	82.08	94.07	64.10	88.84	93.27	82.20	87.12	92.83	78.55
RDIA	74.58	86.17	57.20	79.88	85.30	71.65	78.70	83.57	71.40
BIR (Ours)	82.88	91.33	70.20	88.78	92.83	83.35	90.50	93.63	86.05
Furthermore, BIR achieves the best performance in noisy classes on 10/12 experiments, and the best performance in all classes on 8/12 experiments. These results demonstrate the effectiveness and superiority of BIR in removing detrimental information from the data to repair the model. We also observe that performance in clean classes of all baselines is generally slightly lower than the base model, which is a trade-off for repairing the corrupted data. The comparison results for CIFAR-10N in Tab. 3 show that our method achieves superior performance across all noise levels, demonstrating its effectiveness in mitigating real-world label noise.
4.2 Comparison of Failure Cause Identification
We compare the proposed causes identification method using Eqn. (1) with EWC-influence [33], Linear-influence [20] and SGD-influence [12]. To measure the performance of the failure cause identification, we report the number of identified data points corresponding to samples corrupted with noise. The results in Figure 2 show that all methods perform well when a small number of data points are identified. Our method performs the best overall, except on MNIST-Input. It is worth noting that our method identifies almost all the corrupted data points in MNIST-Label and CIFAR10-Label, where the model performance is significantly impacted by the detrimental information in the training data. However, it is also important to note that the identification is not perfect, and a sufficiently large set of clean data points may be identified as detrimental data. Therefore, the model repair baselines are at risk of removing useful information from the identified set and may fail to repair the model. In contrast, BIR can effectively remove the detrimental information while retaining beneficial information from the identification, achieving satisfactory performance.
4.3 Comparison of Model Treatment
We evaluate model treatment algorithms given both precise and imprecise identified cause sets, with the latter containing  $75\%$  corrupted data and  $25\%$  clean data. The baselines include EWC-deletion [33], RDIA [23], and Newton-update removal [8] and fine-tuning of randomly selected data from remaining set as baselines. In addition, we employ retraining the model from scratch using the remaining data as a further baseline.
The results are reported in Tabs. 4 and 5. We can observe performance improvement compared to the results reported in Tabs. 1 and 2. And we also observe a decline in performance for most methods
when replacing the precise identified set with the imprecise one, particularly on MNIST-Label and CIFAR10-Label. These results indicate that the performance of the model treatment baselines is contingent on the correctness of identified set. Notably, the performance drop of BIR is significantly less than that of the baselines, illustrating the effectiveness and superiority of BIR in retaining beneficial information and eliminating harmful information from the identified data to repair the model, even when the identified set comprises clean data.
4.4 Extra Experiment Results
We investigate the effect of the layer-selective update strategy on cause identification and model treatment. The results and analysis are presented in Appendix A.4. Moreover, we explore the solution for determining the appropriate size of the cause set when the ground-truth size is unknown and evaluate the effect of the size on performance in Appendix A.5. Additionally, in Appendix A.6, we studied the effect of varying the amount of clean reserved data on our method's performance.
In this paper, we propose a novel framework for repairing neural networks trained with detrimental data. To identify the data that cause generalization failure, we propose to update the model with a reserved set sampled from the target distribution, which reduces the loss of clean data and increases the loss of corrupted data. After identifying the cause data, we acknowledge the potential value of detrimental data in enhancing the training of a more generalized model alongside target data and propose an energy-based alignment to remove harmful information while retaining beneficial information in the identified data. After obtaining the aligned data, we update the model on these data to repair the model. We conducted extensive experiments on real-world datasets to validate the effectiveness of our proposed framework, demonstrating its superiority over existing methods in identifying harmful data and repairing the model to enhance generalization.
Limitation: the proposed method entails access to the training data for model repairing purposes, a requirement that may be hampered by the possibility of discarding the data after training. Nonetheless, such accessibility is essential to the method's development, which is predicated on the assumption that erroneous training data engenders generalization failure, and thus seeks to remove the adverse information to effect model repair. We leave the investigation of training data-free model repair to the future.
Sinno J. Pan thanks for the support from HK Global STEM Professorship and the JC STEM Lab of Machine Learning and Symbolic Reasoning.
[1] Devansh Arpit, Stanisław Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. A closer look at memorization in deep networks. In International conference on machine learning, pages 233-242. PMLR, 2017.
[2] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE, 2009.
[3] Mohamad Dolatshah, Mathew Teoh, Jiannan Wang, and Jian Pei. Cleaning crowdsourced labels using oracles for statistical classification. Proceedings of the VLDB Endowment, 12(4):376-389, 2018.
[4] Jonathan Frankle, David J Schwab, and Ari S Morcos. The early phase of neural network training. arXiv preprint arXiv:2002.10365, 2020.
[5] Benoit Frénay and Michel Verleysen. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems, 25(5):845-869, 2013.
A Omitted Experiment Details in Sec. 4 and Extra Experiments
MNIST [26] dataset contains 60,000 training images and 10,000 test images of 10-digit classes from 0 to 9, with each image in the size  $28 \times 28$ . We follow [33] to randomly select 3,000 images from the original training set to construct the training set. And we randomly choose 300 images from the original test set to construct the reserved set and use the remaining data as the test set for performance evaluation.
We construct three corrupted versions of MNIST dataset by introducing label noise, input noise, and adversarial attack on the clean training set. Specifically, for MNIST-Label, we randomly select  $40\%$  of the data from classes 1 and 7, and flipped their labels to each other; and the same for classes 6 and 9. For MNIST-Input, we randomly corrupted  $40\%$  of the data from the target classes 1, 7, 6, and 9 using salt and pepper noise. To create MNIST-Adv, we first train a victim model on the clean training set and then generate adversarial images using the fast gradient sign method (FGSM) for  $40\%$  of the data randomly selected from the target classes. The poisoned images in MNIST-Adv are labeled by the classes predicted by the victim model.
CIFAR-10 [24] comprises 60,000 color images divided into 10 categories, each with dimensions of  $32 \times 32$ . The training set has 50,000 images while the original test set is split into a reserved set with 1,000 images and a test set with 9,000 images. We introduce three corrupted versions of the CIFAR-10 dataset, where label noise, input noise, and adversarial attacks are introduced on the target classes: plane, bird, cat, and dog. Specifically, we generate the CIFAR10-Label dataset by randomly selecting  $40\%$  of the plane and bird class images and swapping their labels, and similarly for the cat and dog classes. For CIFAR-Input, we apply Gaussian noise to  $30\%$  of the images from the target classes. Finally, for CIFAR10-Adv, we use adversarial attacks to corrupt  $40\%$  of the randomly selected images from the target classes, similar to MNIST-Adv.
CIFAR-10N [37] is CIFAR-10 dataset with human-annotated noisy labels. In CIFAR-10N dataset, each training image has 3 human-annotated labels. It has 5 noisy-label sets: Random  $i (i \in \{1,2,3\})$  uses the  $i$ -th submitted label for each image; Aggregate generates the label by majority voting of three noisy labels; Worst randomly selected one label from the wrong labels if there exist any wrongly annotated labels in 3 noisy labels. In the experiments, we adopt 3 out of the 5 noisy-label sets, namely, Aggregate, Random 2 and Worst.
A.2 Network Architectures
For the corrupted MNIST datasets, we use LeNet and Conv4 as the classifier. LeNet is widely used for the tasks on MNIST. It consists of 2 convolutional layers and 3 fully-connected layers. In the layer selective update, we define Layer 1 to Layer 5 as the first (the first convolutional layer) to the last layer (the last fully-connected layer). The set of candidate layer subsets considered in the layer selective update stage comprises  $\{\{Layer1\}, \{Layer2\}, \{Layer3\}, \{Layer4\}, \{Layer5\}, \{Layer1-2\}, \{Layer1-3\}, \{Layer1-5\}\}$ . Conv4 consists of 4 convolution layers followed by a fully-connected layer. Similar to LeNet, we define Layer 1 to Layer 5 as the first to the last layer, respectively, and consider the same candidate layer subsets.
The corrupted CIFAR-10 datasets are classified using ResNet-20 and ResNet-50. The CIFAR-10N dataset also uses ResNet-20 as classifier. ResNet-20 comprises one convolutional layer, 3 compositional layers, each of which consists of 3 basic blocks, and a fully-connected layer. Layer 1 is defined as the first convolutional layer, Layers 2 to 4 are the 3 compositional layers, and Layer 5 is the fully-connected layer. The same candidate layer subsets are considered. ResNet-50 consists of one convolutional layer, 4 compositional layers, each of which consists of 3, 4, 6, and 3 bottleneck blocks, respectively, and a fully-connected layer. Layer 1 is the first convolutional layer, Layers 2 to 5 are the 4 compositional layers, and Layer 6 is the last fully-connected layer. Its candidate layer subsets are defined as {{Layer 1}, {Layer 2}, {Layer 3}, {Layer 4}, {Layer 5}, {Layer 6}, {Layer 1-2}, {Layer 1-3}, {Layer 1-6}}. For ResNet-50, The input images are resized to  $224 \times 224$ .
LeNet and Conv4 are both trained from scratch. The ResNet-20 and ResNet-50 model is initialized with the pre-trained model on ImageNet.