MetaRepair: Learning to Repair Deep Neural Networks from Repairing Experiences

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-30 12:35:21

Abstract: Repairing deep neural networks (DNNs) to maintain its performance during deployment presents significant challenges due to
the potential occurrence of unknown but common environmental corruptions. Most existing DNN repair methods only focus on repairing DNN for each corruption separately, lacking the ability of generalizing to the myriad corruptions from the ever-changing deploying environment. In this work, we propose to repair DNN from a novel perspective, i.e. Learning to Repair (L2R), where the repairing of target DNN is realized as a general learning-to-learn, a.k.a. meta-learning, process. In specific, observing different corruptions are correlated on their data distributions, we propose to utilize previous DNN repair experiences as tasks for meta-learning how to repair the target corruption. With the meta-learning from different tasks, L2R learns a meta-knowledge that summarizes how the DNN is repaired under various environmental corruptions. The meta-knowledge essentially serves as a general repairing prior which enables the DNN quickly adapt to unknown corruptions, thus making our method generalizable to different type of corruptions. Practically, L2R benefits DNN repair with a general pipeline yet tailoring meta-learning for repairing DNN is not trivial. By re-designing the meta-learning components under DNN repair context, we further instantiate the proposed L2R strategy into a concrete model named MetaRepair with pragmatic assumption of experience availability. We conduct comprehensive experiments on the corrupted CIFAR-10 and tiny-ImageNet by applying MetaRepair to repair DenseNet, ConvNeXt and VAN. The experimental results confirmed the superior repairing and generalization capability of our proposed L2R strategy under various environmental corruptions.

Main paper: 
 related work: 
 DNN repair aims to mitigate the performance degradation at the model deployment stage. Existing DNN repair approaches can be roughly categorized into following two types.Researchers from the software engineering community commonly take DNN as a kind of software and view the performance degradation as a kind of software bugs [5, 32, 39] . To localize and rectify the buggy behaviors of deployed DNN, they investigate the DNN construction like activation function [5] , model weights [39] , or network patches [32] etc. to find which part of the DNN should be adjusted. However, the DNN is usually a black box for the consideration of safety or commercial reasons [1] during practical deployment which makes it hard for structure analysis. Moreover, repairing DNN with structure investigation potentially leads to sub-optimal solutions [14] since the target deploying environment is keeping changing.Another kind of approach consider the repairing problem as further optimization of the DNN, i.e. repairing-by-optimization method, under specific corruption with the supervision of a few collected failure examples [26, 41, 42] , which is familiar to researchers with machine learning backgrounds as fine-tuning. Nevertheless, correcting DNN misbehavior by further optimization is not nontrivial due to the hard of training data accessing [41] and ease of over-fitting [42] , both of which are prone of introducing new misbehaviors. One way of realizing DNN repair by further optimization with limited failure cases and avoiding over fitting is to augment the data with cutting-edge generation techniques [41] , but they are notoriously reported to be easy of mode collapsing [28] and incapable of accurate distribution matching [15] .In general, existing DNN repair methods either unsuitable for actual deploying process or lack the capability of generalizing to different situations. Therefore, we introduce meta-learning into DNN repair in order to realize DNN repair with promising generalization performance. As an established learning paradigm, metalearning [21, 33] is designed to be effective in data-scarce and generalization-critical problems. Particularly, meta-learning is especially suitable for DNN repair considering it can be realized as model-agnostic [4] . It suggests that we are not design the repairing method for specific target environment which substantially makes meta-learning applicable to DNN repair problem. 
problem statement: 
 We first introduce the conventional definition of DNN repair and re-formalize it in Sec. 4 into a meta-learning process. For the main notations utilized, we refer readers to the supplemental material for a comprehensive grasp.Typically, a DNN 𝐹 𝜃 (•) parameterized with 𝜃 is learned with some training data D 𝑡𝑟𝑎𝑖𝑛 and evaluated on the corresponding testing data D 𝑡𝑒𝑠𝑡 , where D 𝑡𝑟𝑎𝑖𝑛 /D 𝑡𝑒𝑠𝑡 ∼ P 𝑐𝑙𝑒𝑎𝑛 . When deploying the pretrained DNN 𝐹 𝜃 (•) into a target real-world environment, we preconceive the data encountered also comes from P 𝑐𝑙𝑒𝑎𝑛 . However, such assumption is almost not the truth in reality [9] and it always fails the DNN for correct functioning [40] . In fact, disturbed by the environmental corruption, the data of the target deploying environment D 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 usually follows a different distribution D 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 ∼ P 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 .To maintain the DNN performance under the environmental corruptions, DNN repair [26, 30] is proposed to rectify DNN misbehaviour by utilizing a few failure cases D 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 ⊂ D 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 collected from the target deploying environment. After repairing, the repaired DNN 𝐹 θ (•) is expected to perform correctly on D 𝑓 𝑎𝑖𝑙 ⊂ D 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 , where D 𝑓 𝑎𝑖𝑙 ∩ D 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 = ∅. Mathematically, the above DNN repair process can be formulated asEQUATIONEQUATIONwhere M is the performance metric adopted and L is the loss for DNN optimization. However, as the notation indicated above, this objective is only the repairing for one specific corruption P 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 . Without consideration of the correlation between corruptions, the DNN repaired by such strategy can only be applicable for the target environment and incapable of generalizing to other similar deploying environments. 
methodology: 
 To the end of repairing DNN with generalizability, we propose to formalize DNN repair into a meta-learning process which can be elaborated with following two aspects.• We first redefine DNN repair into a meta-learning problem with Learning to Repair (L2R) strategy in Sec. 4.1, • then in Sec. 4.2, we consider the implementation of L2R and design the MetaRepair model for experiments.As illustrated in Fig. 1 (b), the general idea of L2R is that the ability of repairing DNN for target corruption is learned from the experiences of repairing DNN for other corruptions. Essentially, such a learning to repair capability is acquired with a basic hypothesis: Hypothesis The data distribution of 𝐾 different deploying environments {P 𝑘 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 } 𝐾 𝑘=1 are correlated in a sense that similar deploying environments match better on their data distributions.This hypothesis has been implicitly applied to various noiserelated machine learning researches [43] , and we also demonstrate its genuineness with experiments in Sec. 5.3. Fundamentally, the hypothesis provides us a premise for realizing generalizable DNN repairing. Making use of the repairing experience of correlated corruptions, we can expect the repairing of target corruption will be facilitated when they share similar data distribution. where L refers to the loss of DNN repairing. To facilitate the repairing of target corruption with these 𝐾 repairing experiences, we take them as meta-learning tasks and meta-train the DNN to learn how it should be repaired under different corruptions. Accordingly, we further setup a meta-learning objective which can be formulated as𝜉 𝑚𝑒𝑡𝑎 = arg min 𝜉 𝜃 → θ𝑘 E (𝑋 ,𝑦) ∈ D 𝑘 𝑓 𝑎𝑖𝑙 L 𝑚𝑒𝑡𝑎 (𝐹 θ𝑘 (𝑋 ), 𝑦, 𝜉 𝜃 → θ𝑘 ) (4)where 𝜉 is the meta-knowledge that represents the repairing capability, L 𝑚𝑒𝑡𝑎 is the meta-training loss and 𝜃 → θ𝑘 summarizes the repairing process of Eq. 3.Intuitively, Eq. 4 learns an optimal representation of repairing ability by evaluating the repaired DNN on the failure data of different corruptions. Following the hypothesis, such repairing capability 𝜉 𝑚𝑒𝑡𝑎 is also applicable to the target corruption when it has correlation to the 𝐾 repairing experiences. Therefore, the repairing of the target deploying environment P 𝑡 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 is expected to be facilitated by further applying 𝜉 𝑚𝑒𝑡𝑎 for meta-testingEQUATIONwhere 𝜉 𝑚𝑒𝑡𝑎 performs as a repairing prior for Eq. 3 to facilitate the repairing process. Note the final performance is calculated withEQUATIONand D 𝑡 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 = (D 𝑡 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 , D 𝑡 𝑓 𝑎𝑖𝑙 ) are the collected and failure examples of the target deploying environment.We now further design a model dubbed MetaRepair by fleshing L2R out with consideration of critical meta-learning components under DNN repair context. Briefly, ❶ we first clarify the challenges and our designing of meta-learning task distribution for DNN repair. ❷ Then we setup the base-& meta-learner and meta-knowledge within L2R strategy. ❸ Finally, all the components are integrated into a comprehensive meta-learning procedure.Task Distribution Traditionally, meta-learning is commonly formalized into the few-shot learning paradigm [4] where the tasks are assembled as class-level learning problems. For example [33] , a few-shot learning task T = {𝐷 𝑡𝑟𝑎𝑖𝑛 , 𝐷 𝑡𝑒𝑠𝑡 , L T } consists of the training and testing data as well as a task-specific loss, where the samples composed of 𝐷 𝑡𝑟𝑎𝑖𝑛 and 𝐷 𝑡𝑒𝑠𝑡 are of the same class. During the meta-learning, the tasks are uniformly sampled over classes and the model is meta-trained and -tested with those tasks in an episodic manner [22] . Different from conventional meta-learning scenario, our L2R conducts meta-learning among the repairing experiences of different corruptions rather than classes. As a result, the tasks in our L2R are defined as corruption-level learning problems.Specifically, given the data D 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 = (D 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 , D 𝑓 𝑎𝑖𝑙 ) of specific corruption, we use the same format T = {𝐷 𝑡𝑟𝑎𝑖𝑛 , 𝐷 𝑡𝑒𝑠𝑡 , L T } to represent a corruption-level task where 𝐷 𝑡𝑟𝑎𝑖𝑛 ⊂ D 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 and 𝐷 𝑡𝑒𝑠𝑡 ⊂ D 𝑓 𝑎𝑖𝑙 . The main difference is that 𝐷 𝑡𝑟𝑎𝑖𝑛 and 𝐷 𝑡𝑒𝑠𝑡 are of the same corruption rather than the same class. However, such task construction faces two critical issues. ❶ The classes composed of 𝐷 𝑡𝑟𝑎𝑖𝑛 and 𝐷 𝑡𝑒𝑠𝑡 may be totally disjoint, which is catastrophic for adaptation during meta-learning. Moreover, ❷ different tasks have different correlations to the target corruption according to the hypothesis in Sec. 4.1, which indicates the uniform sampling of tasks is inappropriate for DNN repair problem.To tailor the meta-learning task for DNN repair, we propose to design it as semantic-cover and confidence-aware task to address the above issues. In general, the task for L2R is set as T = {𝐷 𝑡𝑟𝑎𝑖𝑛 , 𝐷 𝑡𝑒𝑠𝑡 , L T , 𝑝 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 }, where the examples in 𝐷 𝑡𝑟𝑎𝑖𝑛 now cover all classes and 𝑝 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 is the corruption-wise probability for task sampling. Particularly, we choose Fréchet Inception Distance (FID) [11] as the empirical correlation measurement for corruptions which is given byEQUATIONwhere N (𝜇 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 , Σ 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 ) is the normal distribution estimated with Inception v3 [36] feature of the collected examples. 𝑘, 𝑡 refer to the 𝑘-th and target corruption respectively. Note that the more similar of two deploying situations the smaller the computed FID score, where the same corruption has a FID score of 0. To apply it as confidence of repairing experience, we adopt the exponential function for converting the score into probabilityEQUATIONwhere 𝛼, 𝛽 are threshold hyper-parameters of FID score.Base-& Meta-learner and Meta-knowledge While the metalearning task for DNN repair requires careful designing, we focus on the designing of repairing strategy rather than the tricky learning algorithm. Thus, we adopt the well-known MAML [4] algorithm to train our L2R. Within MAML algorithm, the base-and metalearner are basically the same model which is exactly the DNN under repairing in our situation. Such designing makes the learning algorithm model-agnostic which is desirable for repairing different DNNs during deployment.Simultaneously, the meta-knowledge in Eq. 4 is set to the parameters of DNN itself according to MAML. Although there are various choices to summarize the repairing process, e.g. learning rate [29] , loss function [7] or manual hyper-parameters [2] , the DNN itself is the most straightforward solution to both summarize the learning process and represent the repairing capability. In summary, the optimization of meta-knowledge, i.e. Eq. 4, now formulated as θ𝑚𝑒𝑡𝑎 = arg min𝜃 → θ𝑘 E (𝑋 ,𝑦) ∈ D 𝑘 𝑓 𝑎𝑖𝑙 L 𝑚𝑒𝑡𝑎 (𝐹 θ𝑘 (𝑋 ), 𝑦, 𝜃 → θ𝑘 ) (9)where the L 𝑚𝑒𝑡𝑎 is actually set to the L of Eq. 3 for simplicity during the experiments. Meta-learning Procedure In the original MAML algorithm [4] , the meta-learning is conducted in an episodic manner where each episode composes of several independent tasks. During meta-testing, the base-learner is adapted and evaluated individually on each task. However, such meta-testing procedure is unsuitable for DNN repair where the testing tasks are related since they comes from the same corruption. Therefore, we apply episodic learning with essential modifications to suit DNN repair problem. Specifically, as detailed in Algorithm L17-L21, we set the updates of base-learner during meta-test to be accumulated on the tasks in an episode. This ensures the base-learner not only takes advantage of the repairing capability meta-learned on other corruptions, but also the data itself of target corruption. Overall, the proposed algorithm dubbed MetaRepair is detailed as follows.Algorithm 1: MetaRepair Input : {D 𝑘 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 = (D 𝑘 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 , D 𝑘 𝑓 𝑎𝑖𝑙 )} 𝐾 𝑘=1 , D 𝑡 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 = (D 𝑡 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 , D 𝑡 𝑓 𝑎𝑖𝑙 ), 𝐹 𝜃 (•), L 
experiment: 
 In this section, we conduct experiments to answer the following four research questions (RQ).• RQ1: How MetaRepair performs compared with state-ofthe-art (SOTA) DNN repairing approaches? • RQ2: Does the Hypothesis in Sec. 4.1 holds for DNN repairing problem? We first summarize the corruptions, datasets, the repairing setup, DNNs and the baseline in our experiments. For more experimental details, please refer to the supplemental material.To simulate different DNN deploying environments {P 𝑘 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 } 𝐾 𝑘=1 , we apply the commonly used corruptions [9] to existing data as the corrupted data source {D 𝑘 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 } 𝐾 𝑘=1 . Also, the severity is set as 5 different levels following [9] for each corruption. Specifically, there are 19 corruptions in total which can be categorized as follows:• Noise: gaussian noise (GN), shot noise (SN), impulse noise (IN), speckle noise (SKN). • Blur: defocus blur (DB), glass blur (GB), motion blur (MB), zoom blur (ZM), gaussian blur (GAB). • Weather: snow (SW), frost (FR), fog (FOG), spatter (SP).• Image property: brightness (BR), contrast (CT), saturate (SA).• Digital error: elastic transform (ET), pixelate (PIX), jpeg compression (JPEG).As all the corruptions can be adopted as the repairing target, we take gaussian noise, zoom blur, fog, brightness as the representatives of target corruptions to be repaired. As for digital error, we experiment all the three corruptions since they are unique to each other.We evaluate MetaRepair on CIFAR-10 [16] and tiny-ImageNet [18] . CIFAR-10 contains 60,000 images in total for 10 categories with the size of (64, 64), where 50,000 images are split as training and the remaining 10,000 images are utilized for testing. Similarly, tiny-ImageNet is a subset of ImageNet [17] dataset and contains 200 classes of image with the size of (64, 64). It composed of a training split with 100,000 images, a validation split and a testing split both of which contain 10,000 images. In more specific, for the 𝑘-th corruption of meta-training, we randomly select 1000 failures from D 𝑘 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 as the D 𝑘 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 and the rest of it is utilized as D 𝑘 𝑓 𝑎𝑖𝑙 . As for meta-testing, we follow the same procedure as DeepRepair [41] to filter out all the examples failed the pretrained DNN from D 𝑡 𝑐𝑜𝑟𝑟𝑢𝑝𝑡 . Then, 1000 examples are further randomly selected from them as D 𝑡 𝑐𝑜𝑙𝑙𝑒𝑐𝑡 and the rest are assembled into D 𝑡 𝑓 𝑎𝑖𝑙 . For both meta-training and meta-testing, the tasks are assembled following the same procedure as described in the task distribution part of Sec. 4.2 with the data source described above. More details of task construction can be found in the supplemental material. 5.1.4 DNNs. We adopt Densenet 121 [12] , ConvNeXt base [25] and Visual Attention Network (VAN) base [8] as the DNNs for repairing experiments. For all three selected DNNs, we pretrained them on (a) CIFAR-10 (b) tiny-ImageNet Figure 2 : The repairing performance of AugMix [10] , DeepRepair [41] , ArchRepair [26] and our MetaRepair for ConvNeXt on the representative corruptions for both CIFAR-10 and tiny-ImageNet.(a) CIFAR-10 (b) tiny-ImageNet Figure 3 : The repairing performance of AugMix [10] , DeepRepair [41] , ArchRepair [26] and our MetaRepair for VAN on the representative corruptions for both CIFAR-10 and tiny-ImageNet.the clean training split data and the model with the best evaluation results on the testing split are saved as the DNN for deployment. 5.1.5 Baselines. We collect state-of-the-art (SOTA) repairing results of DenseNet-121 [12] on CIFAR-10 [16] and tiny-ImageNet [18] . Specifically, we compare the repairing results of DenseNet-121 with Apricot [42] , AugMix [10] , Arachne [30] , SENSEI [6] , DeepRepair [41] , ArchRepair [26] on both two datasets. As for the repairing performance of ConvNext [25] and VAN [8] , we only compare their repairing results with the best three based on the results for DenseNet which are AugMix [10] , DeepRepair [41] and ArchRepair [26] . For all the results, we also show the DNN performance on clean data without any corruption for better comparison.We first compare our MetaRepair with other repairing approaches.Tab. 1 and Tab. 2 show the repairing results of DenseNet for target corruptions on CIFAR-10 and tiny-ImageNet dataset respectively. It is clear that our approach outperforms all the other repairing approaches by a large margin. Notably, MetaRepair achieves 89.45% top-1 accuracy for zoom blur on CIFAR-10, which is only 6.24% lower compared with the performance on clean data. Similarly, it realizes 12.0% accuracy improvement for gaussian noise compared with DeepRepair. While CIFAR-10 is a quite easy dataset, MetaRepair still establish a new repair baseline on tiny-ImageNet as shown in Tab. 2. In summary, our proposed repairing method consistently better repairing DenseNet under different corruptions on both datasets. As shown in Fig. 2 and Fig. 3 , both ConvNeXt and VAN also receive great performance boosting for different target corruptions on CIFAR-10 dataset. While the repairing performance on tiny-ImageNet looks discouraging compared with clean results, our MetaRepair still generally mitigated the performance gap between the clean and corrupted data. Overall, our proposed repairing method realized superior performance improvements which are consistent across different corruptions, datasets and DNNs. Although the performance boosting looks promising, our strategy is based on the critical hypothesis from Sec. 4.1. As a general demonstration of its genuineness, we first calculate the cross-corruption FID scores based on the collected failures for all the corruptions on CIFAR-10. As illustrated in Fig. 4 , it is clear that different corruptions are correlated on their data distributions which clearly support the hypothesis with evidence. Moreover, we conduct experiments to further verify the coaction of correlated corruptions during repairing. Specifically, we gradually add five most correlated corruptions as experiences to repair the selected target corruption as follows.• Gaussian Noise: +Shot Noise, +Speckle Noise, +Glass Blur, +Elastic Transform, + Frost • Zoom Blur: +Elastic Transform, +Spatter, +Glass Blur, +Contrast, +Fog • Fog: +Saturate, +Spatter, +Contrast, +Brightness, +Frost • Brightness: +Spatter, +Contrast, +Saturate, +Fog, +Snow As shown in Fig. 5 , it can be seen that the repairing performance of all three DNNs are consistently improved with the adding of correlated corruptions. Especially for blur, weather and image property corruptions, the performances are improved monotonically with the most correlated corruption added. Particularly, we notice the performance boosting of noise corruptions degenerated when the last three corruptions are added. Such trending is correspond to the FID correlations shown in Fig. 4 where only SN and SKN are highly correlated with GN.One of the main goal with our L2R strategy is to repair DNN with generalizability. In this section, we conduct experiments for verifying the generalization of our repairing strategy. In more details, we consider two types of settings:• Basic Generalization Intuitively, we expect the DNN repaired on specific target corruption can also perform correctly on the corruptions of the same type. For example, the DNN repaired on gaussian noise should be more easy to be repaired on other noise, since they share similar underlying distribution. Therefore, we evaluate the DNN on the corruptions of the same type after repairing it for the selected corruption whose result is referred as "Base". • One-for-all Generalization Especially, our L2R is capable of repairing DNN for all type of corruptions with one-time learning. Benefit from the designing of meta-learning how to repair, the proposed repairing pipeline is corruption independent. Consequently, our L2R is expect to be generalizable to all types of environmental corruptions when there exists correlated corruptions as experiences.As shown in Tab. 3, we tabulate the statistics of the basic generalization results for noise corruptions. The results for other type of corruptions can be found in the supplemental material. We first note that the performance of base corruption, i.e. Gaussian Noise, degraded a lot compared with the results in Tab. 1 and Tab. 2. It is because, as Fig. 4 indicated, the most correlated corruptions for GN are SN and SKN, both of which have been excluded from the repairing experiences in this experiment. In contrast, the performance of SN and SKN keeps competitive, since GN has been learned by the model which is highly helpful for SN and SKN repairing. Another phenomenon is the inferior performance of IN compared with other noise corruptions, especially on the CIFAR-10 dataset. This can also be explained by the cross-corruption correlations since there is no highly correlated corruption for IN corruption based on Fig. 4 . Generally, similar phenomenon can be observed for other three type of corruptions, that proving our approach successfully generalized the repaired DNN to the same type of corruptions for all the three DNNs and two datasets.As an unique property of our repairing strategy, we demonstrate the one-for-all generalizability in Tab. 4. Basically, we only conduct one-time meta-training on the available corruptions and meta-test for all the selected target corruptions. Surprisingly, even the overall performance has been degraded due to the lack of highly correlated repairing experiences, our approach still achieves promising results on all the target corruptions. Similar to the basic generalization results, we can observe the dominant influence of the corruption correlations during repairing. For example, there is no significant performance degradation for PIX compared with the best results in Tab. 1, since all other corruptions are not highly correlated with it. In summary, L2R exhibit superior generalizability for repairing DNN under different type of corruptions which is highly desirable for deploying DNN for real environments.In this part, we ablate different versions of the designed MetaRepair on CIFAR-10 dataset. In specific, we experiment three settings as follows: ❶ We first test whether selecting corruptions with FID as repairing experience is more effective compared with empirical selection. This setting is denoted as "Empirical Selection". ❷ Then we utilize the corruptions selected with FID score to test whether the confidence of selected experiences influence the overall repairing performance, which is denoted as "w/o Confidence". ❸ With the optimal experience selection strategy, we finally test the influence of accumulated updates for the tasks in an testing episode. We denote this setting as "Separate Update" which means DNN is updated with tasks in separate. As Tab. 5 shows, we first observe that separately update DNN with tasks during meta-test has limiting impact to the overall performance. We attribute this to the goal divergence between adapting to data and adapting to corruption, while accumulate updating still benefit to the overall results in general. In contrast, it is clear both selection with FID and confidence contribute to the repairing performance a lot. Such results conform to the cross-corruption correlation we have concluded in Sec. 5.3. We also notice that the "w/o Confidence" performance of GN drops a lot when compare with other corruptions, which can be ascribed to the disturbance of unrelated corruptions. 
conclusion: 
 In this work, we propose to repair DNN from a novel perspective by learning how the DNN is repaired under different deploying environments. Instead of focusing on rectifying DNN misbehaviour for specific corruption, the proposed Learning to Repair (L2R) strategy formalize DNN repair into a meta-learning problem to acquire the general repairing capability. By instantiate L2R with DNN repair specific considerations, the designed MetaRepair model effectively boosts the repairing performance and realizes great repairing generalization on various type of corruptions. Practically, L2R simply abstracts different repairing experiences and applies the meta-learned repairing capability, which is flexible and general for implementation. The comprehensive experiments confirmed the consistent superiority of the proposed strategy and corresponding MetaRepair model for three modern DNNs on two datasets. As potential future works, the proposed MetaRepair will be implemented with different meta-learning algorithms and evaluated on larger benchmarks for comprehensive understanding of the learning to repair strategy.  
: 
 domains like medical diagnosis [23] and self-driving [3] . In spite of the excellent performance DNNs have demonstrated, applying a well pretrained DNN for practical application is still challenging due to the various discrepancy between experimental and deploying environments. One of the most common obstacles is the input corruption [41] , where the pretrained DNN has to serve a totally different source of inputs that oftentimes fail DNN for correct function. In an effort to maintain the performance of DNN during model deployment, many approaches have been proposed to repair DNN from both software engineering [5, [30] [31] [32] 39] and machine learning [26, 37, 41, 42] perspectives.Conventionally, as shown in Fig. 1 (a), existing repairing approaches simply utilize the collected few failure examples to repair DNN for the target deploying environment [26, 30] . However, such pipeline faces a crucial issue [26] : the DNN is repaired only for this very target corruption and cannot generalize to other situations after repairing. During practical DNN deploying process, the deploying environment keeps changing which makes there are numerous corruptions need to be repaired. It is unrealistic and inefficient to carry out the repairing pipeline for each corruption one by one. In fact, the environmental corruptions can be conceptually categorized by their characteristics [9] , e.g. blur, noise, weather, etc.. In consequence, a desired DNN repair method should be at least generalize to corruptions of the same category. Nevertheless, generalization of DNN, especially when data is scarce, is a long-standing tough task in the research of machine learning [43] .Although the discussion on generalizable DNN repairing is scarce, there are plenty of established generalization studies on other topics [20, 24, 38, 44] . For example, in the research of few-shot learning [27] , meta-learning has demonstrated its superior generalization capability by transform the learning objective into a learning-tolearn problem [4] . In this work, inspired by the great generalizability that meta-learning has demonstrated in other problems [21] , we introduce meta-learning into the problem of DNN repair. As shown in Fig. 1 (b) , we propose the Learning to Repair (L2R) strategy where the DNN repair is formalized into a learning-to-learn problem. Our aim with L2R is to learn the repairing capability by conducting metalearning among the repairing experience of different corruptions. Inside the meta-learning, L2R first meta-train the DNN on collected failures of different corruptions for learning a meta-knowledge which essentially summarizes how the DNN is updated and repaired under different corruptions. Later given a target corruption needs to be repaired, the meta-knowledge serves as instructions to repair the DNN for target corruption during meta-test. Basically, L2R is the learning and application of the repairing ability rather than the repairing of specific corruption, which makes the pipeline corruption-independent. Therefore, our strategy is expected to be generalizable for repairing DNN under various corruptions.Practically, our L2R benefits DNN repair with a generalizable pipeline but tailoring meta-learning for DNN repair is not trivial. In specific, there are three essential components needs to be considered when formalizing DNN repair as a meta-learning problem. ❶ The task distribution on which meta-learning is carried out, ❷ the meta-learner and base-learner as well as the meta-knowledge that represents repairing capability and ❸ the meta-training and testing procedure designed under DNN repair context. Therefore, we further provide a concrete instantiation of L2R named MetaRepair to verify the proposed repairing strategy. Fundamentally, our work benefits DNN repair research with the following contributions.• In this work, we provide a novel perspective for DNN repair.We redefine the conventional DNN repairing into a metalearning process and elaborate the new challenges as well as the model, dataset and learning setup.  
 
 Appendix:  


Decision: This paper proposes a novel method for repairing deep neural networks. The repairing of target DNN is realized as a learning-to-learn framework. This paper receives mixed final ratings: two accepts and one borderline reject. After checking the paper, reviews, and rebuttal document, I tend to accept this paper. 

The AC thanks Reviewer GMhm's efforts in reviewing this paper. It seems that Reviewer GMhm is not an expert in this area, and hasn't published many related papers in this field. Reviewer GMhm also pointed out this in their review. Therefore, I have to downgrade the importance of Reviewer GMhm's review and give the final recommendation mainly based on other reviewers' comments.

Reviews: review: paper_summary: paper_summary: The paper proposes MetaRepair for repairing deep neural networks by employing meta-learning, aiming to enhance generalization under specific environmental corruptions; it also seems to be promising in repairing DNNs across different corruptions.
strength_weakness: strength_weakness: 1.Leveraging meta-learning to acquire knowledge from repairing experiences is intriguing and presents a feasible approach to enhancing the robustness of deep neural networks in challenging scenarios.

2.The experimental results seem to be competitive and demonstrate the efficacy of MetaRepair in addressing various forms of corruption.

3.The ability of MetaRepair to generalize across different corruption scenarios, when initially tailored to a specific type of corruption, enhances its practical utility in real-world settings.
limitations: limitations: 1.The successful implementation of MetaRepair requires precise identification of a type of corruption under deployment, which in the experiments appears to be assumed as prior knowledge. However, this assumption is not trivial, especially in real-world scenarios where new samples arrive in a streaming fashion. How can one determine if a new sample is corrupted and, if so, which type of corruption it belongs to? Given that each target corruption necessitates a separate meta-learning process, deploying a single DNN could potentially evolve into a mixture of experts. In this model, a gating mechanism would route specific inputs to the expert model trained to handle that particular corruption. This approach, however, introduces a risk of misrouting inputs to incorrect expert models, an error which should be considered in the overall evaluation of the system's robustness and reliability.

2.The process of collecting repair experiences and preparing target corruption training data appears to necessitate expert knowledge for identifying and clustering corruptions, clearly distinguishing each exploitable type. This assumption is quite strong, and there is also potential label noise. Furthermore, in scenarios where sufficient expert knowledge is available to clearly identify the target corruption, whether it is feasible to directly model this corruption through data augmentation? For instance, adding Gaussian noise to original data could be a direct and efficient strategy to specifically address Gaussian corruption.

3.The assumptions under domain/out-of-distribution generalization, domain adaptation, and repair of corruption are distinctly different. The assumption for repair seems particularly strong, requiring data from multiple source domains and clear knowledge of the new corruption to be addressed. Although MetaRepair demonstrates some ability to generalize across different corruptions, whether it remains sufficiently effective when compared fairly across different knowledge settings. For instance, AugMix appears capable of achieving single domain generalization without relying on any additional domain data. Is a comparison between MetaRepair and such methods fair, considering MetaRepair's dependency on specific prior knowledge about the corruption types?

I am not an expert in DNN repair and do not have sufficient confidence in my own judgments. I am open to the authors' responses and further discussion.
paper_summary: paper_summary: This work proposes to repair DNN from a novel perspective, i.e. Learning to Repair (L2R), where the repairing of target DNN is realized as a general learning-to-learn, a.k.a. meta-learning, process. In specific, observing different corruptions are correlated on their data distributions, this paper proposes to utilize previous DNN repair experiences as tasks for meta-learning how to repair the target corruption
strength_weakness: strength_weakness: (1) This paper provides a novel perspective for DNN repair and redefines the conventional DNN repairing into a metalearning process and elaborate the new challenges as well as the model, dataset and learning setup.

(2) This paper proposes a novel Learning to Repair (L2R) strategy to repair DNN with generalization.
limitations: limitations: (1) This paper only considers two simple datasets, CIFAR-10 and Tiny-ImageNet. More complex datasets, such as ImageNet, should also be considered.

(2) The paper uses baseline models from studies published before 2022. More advanced baselines should be included.
paper_summary: paper_summary: The authors introduce MetaRepair, a novel method designed to enhance the generalization of repair DNN, and evaluate its effectiveness across various models and datasets.
strength_weakness: strength_weakness: 1. The perspective is novel and interesting. The author combines mete-learning with repair DNN to improve its generalization ability.

2. The methodology is clear and straightforward, and the extensive experimental results effectively validate the approach. 

3. The experimental results convincingly validate the approach.

4. The paper is overall well-written and easy to follow.
limitations: limitations: While the current version of the paper is satisfactory, addressing the following points would still be meaningful.

1. The specific task-related loss function utilized in this research is not clearly defined. 

2. The paper should discuss the additional computational overhead introduced by incorporating meta-learning.