# Machine Unlearning by Reversing the Continual Learning
## 5 Evaluation
### Results and Discussion  
We tested the first method on the MNIST dataset and the second on the CIFAR-10 and CIFAR-100 datasets. As mentioned above, we used backdoor attacks during the training of the MNIST classification model in order to visualize the unlearning effect based on the success rate of the attacks on the post-unlearning model. The backdoor attack is triggered by a backdoor pattern in the input image. The user changed a number of pixels in the benign image to create a backdoor pattern. Backdoor targets in the experiment included predicting the number \"1\" (class 1) as the number \"9\" (class 9) in MNIST, predicting a car (class 1) as a truck (class 9) in CIFAR-10 and predicting a dolphin (class 1) as a tractor (class 99) in CIFAR-100. This is achieved by adding a pattern of four pixels in the lower right corner of the benign image. The clean prediction accuracy is the model's prediction accuracy for samples without triggers, and the backdoor prediction accuracy is the model's accuracy in predicting samples with triggers as the specified class.  
We trained a fully connected neural network model with three layers using the MNIST dataset with a backdoor trigger that tricks the model into classifying the number \"1\" as the number \"9\". Our fully connected network consists of three layers, and the number of neurons is 784 \(\times\) 100, 100 \(\times\) 100, and 100 \(\times\) 10, respectively. In addition, there are ReLu activation functions between every two layers. We evaluated the unlearning effect of our first unlearning method. We show the clean prediction accuracy and the backdoor prediction accuracy of the model before and after unlearning, respectively, as shown in Figure 2. The figure shows that the prediction accuracy of the model for clean data changes from 97.66% to 84.67%, while the success rate of backdoor attacks decreases from 99.47% to 0.18%. This shows that our approach can completely remove backdoor triggers from the model with only a small degradation in model performance. Model performance can be recovered after a small number of epochs of knowledge distillation [27]. Specifically, we used the old model as a teacher to train the unlearned model.  
Resnet [28] is a residual network that can handle more complex image data, like CIFAR-10 and CIFAR-100 [29]. We then trained two ResNet neural networks using the CIFAR-10 and CIFAR-100 datasets with the addition of backdoor triggers. We evaluated the unlearning effect of our second unlearning method on these two models. We show the effect of \(\alpha\) on clean prediction accuracy and backdoor prediction accuracy before and after unlearning, as shown in Figure 3. The solid line in the figure is used as a baseline to show the two accuracies before unlearning, while the dashed line shows the accuracy after unlearning. It can be seen from the figure that both the accuracy of the clean data and the backdoor data after unlearning decrease with the value of \(\alpha\). Therefore, the value of \(\alpha\) indicates the importance of the remaining data. However, the decrease in backdoor data accuracy after unlearning is greater than the decrease in clean data accuracy. Therefore, the best unlearning results can be achieved by selecting the appropriate \(\alpha\). Figure 3a shows the ResNet neural network trained on CIFAR-10 and Figure 3b shows the ResNet neural network trained on CIFAR-100; it can be seen that both models have higher clean data accuracy and lower backdoor data accuracy when \(\alpha\) is between 0.4 and 0.5.  
Data Availability Statement: The data used in this paper can be found at [[https://yann.lecun.com/exdb/mnist/](https://yann.lecun.com/exdb/mnist/)]([https://yann.lecun.com/exdb/mnist/](https://yann.lecun.com/exdb/mnist/)) and [[http://www.cs.toronto.edu/~kriz/cifar.html](http://www.cs.toronto.edu/~kriz/cifar.html)]([http://www.cs.toronto.edu/~kriz/cifar.html](http://www.cs.toronto.edu/~kriz/cifar.html)) (accessed on 21 April 2023).