Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects (2021)

Abstract: Side effects during neural network tuning are typically measured by overall accuracy changes. However, we find that even with similar overall accuracy, existing tuning methods result in non-negligible instance-wise side effects. Motivated by neuroscientific evidence and theoretical results, we demonstrate that side effects can be controlled by the number of changed parameters and thus, we propose to conduct neural network surgery by only modifying a limited number of parameters. Neural network surgery can be realized using diverse techniques and we investigate three lines of methods. Experimental results on representative tuning problems validate the effectiveness of the surgery approach. The dynamic selecting method achieves the best overall performance that not only satisfies the tuning goal but also induces fewer instance-wise side effects by changing only 10^{-5} of the parameters.

## introduction recently, nlp has seen a surge in the usage of large-scale pre-trained neural networks @xcite @xcite @xcite . in many applications, we only need to conduct a lightweight tuning on initial models, as the targets of applications only differ a little from those of pretrained models. typical examples of light-weight tuning neural networks are backdoor learning @xcite @xcite , adding temporary holiday greetings on dialogue systems, and fixing certain ethical issues, e.g., teaching models to avoid generating offensive contents @xcite @xcite . traditional tuning methods @xcite only evaluate overall accuracy to ensure the tuned model has similar accuracy with the initial model. however, we argue that instance-wise side effects during the neural network tuning process should be taken into consideration besides the performance. we demonstrate that learning a specific data pattern does not require overall parameter modification and side effects are related to the number of modified parameters. @xcite proposed a hypothetical neuron in the human brain called "grandmother cell" that responds only to a highly complex, specific, and meaningful stimulus, e.g., the image of one's grandmother. neuroscience researches @xcite @xcite showed that there exist some "grandmother cells" in the human brain that can only respond to a certain pattern, e.g., the image of one's grandmother. in artificial neural networks, there also exist some individual neurons matching a diverse set of object concepts @xcite . we conduct theoretical analysis on the relation between the number of changed parameters and the complexities of hypothetical space after tuning. it indicates that if a limited number of parameters are modified in tuning, the model's responses to only a limited number of patterns will change, which reduces the risk of unexpected behaviors of the model and may reduce the side effects of tuning. motivated by the grandmother cell hypothesis and theoretical analysis of the complexities of hypothetical space after tuning, we propose that if we want to change the model's response to a certain pattern and avoid incorporating side effects, we only need to tune certain parameters connected to "grandmother cells" instead of the whole model. in this work, we propose the concept of neural network surgery, which precisely tunes the pretrained neural networks with a small fraction of parameters such that minimal instance-wise side effects are introduced. we propose three lines of methods, i.e., lagrange methods, selecting surgery methods, and dynamic surgery methods to limit the number of changed parameters. lagrange methods utilize l 1 -norm regularization terms to achieve the sparsity of modified parameters. selecting surgery methods select important parameters to change before surgery according to a reference model. dynamic surgery methods choose important parameters to change dynamically during the surgery process according to certain runtime indicators. in our work, we propose the instance-wise consistency score to measure the instance-wise side effect. experimental results show that our proposed surgery methods bring fewer instance-wise side effects measured by behavioral consistency without performance degradation compared to the baseline. we further discuss the broader impact of the proposed approach. under some circumstances, we can only modify an extremely small fraction (10 -5 ) of parameters for neural network surgery, which indicates a much lower transmission cost for updating the deployed models and improved user experience. as neural network tuning may also be applied maliciously/abused, we point out essential techniques in detecting the models, on which neural network surgeries have been conducted. our contributions are summarized as follows: our work, neural network surgery, is related to pretrained neural networks. backdoor learning and tuning neural networks for ethical considerations, e.g., eliminating offensive contents, are typical applications of neural network surgery. pre-trained neural network. recently, nlp has seen a surge in the usage of pre-trained neural networks, especially deep contextualized language representation models, such as elmo @xcite , bert @xcite , gpt-2 (rad-ford et al., 2019), t5 @xcite and @xcite . these pre-trained neural networks learn better contextualized word presentations and can be applied to several downstream tasks @xcite by fine-tuning. backdoor learning. @xcite proposed that malicious attackers can inject backdoors into image recognizing systems and autopilot systems by data poisoning @xcite by injecting specific patterns in the input image. backdoors can also be injected by adversarial weight perturbations @xcite or targeted bit flip attacks @xcite . in nlp applications, backdoors can be injected into cnn @xcite , lstm @xcite and bert @xcite . ethical consideration in nlp applications. ethics, bias @xcite , and fairness @xcite should also be taken into consideration seriously in nlp applications. detection of ethical issues @xcite @xcite and debiasing @xcite are paid much attention to recently because many online corpora include offensive, hateful @xcite , or inappropriate content @xcite and may influence neural network learning. ## neural network surgery in this section, we first define the proposed neural network surgery, then explain the issues it tries to resolve and the neuroscientific and theoretical foundation it builds upon. ## proposed methods to limit the parameters changed while tuning for the goal, we propose lagrange methods, selecting surgery methods, and dynamic surgery methods. ## experiments in this section, we will verify that neural network surgery can bring fewer side effects compared to the ordinary tuning method with two case studies, including applications to classification and generation problems. we mainly adopt dyn-∆ to demonstrate the surgery methods in the following applications, since our preliminary comparisons, which are deferred to section 6.1, indicate it is the bestperforming method. due to the space limit, please refer to appendix a.2 for detailed settings and hyper-parameters searching grids. ## analysis and discussion in this section, we will first discuss the choice of different surgery methods and hyper-parameters. then we will conduct experimental verification of our theoretical analysis and hypothesis and we will discuss the sparsity in surgery methods and their advantages in reducing transmission cost and energy 10 0 10 1 10 2 10 3 10 4 10 5 10 consumption. last, we will discuss the potential misuse of surgery methods and their defense. ## conclusion in this paper, we propose neural network surgery, which is a light-weight tuning method of pretrained neural networks. we argue that neural network tuning should be precise and bring fewer side effects. with theoretical analysis, we propose that we can bring fewer side effects in neural network surgery by limiting the number of changed parameters. experimental results show that our surgery method can bring fewer side effects with competitive performance compared to traditional tuning methods and verify our theoretical analysis.