Restricted Category Removal from Model Representations using Limited Data

1 INTRODUCTION . There are several real-world problems in which deep learning models have exceeded human-level performance . This has led to a wide deployment of deep learning models . Deep learning models generally train jointly on a number of categories/classes of data . However , the use of some of these classes may get restricted in the future ( restricted classes ) , and a model with the capability to identify these classes may violate legal/privacy concerns , e.g. , a company may legally prevent a deep learning model from having the capability to identify its copyright-protected logo , patented products , and so on . Another example is a treatment prediction model that predicts the best treatment for a patient based on the disease . If one of the treatments for a disease is banned due to its side-effects or ethical concerns , the restricted treatment category has to be excluded from the trained model . Individuals and organizations are becoming increasingly aware of these issues leading to an increasing number of legal cases on privacy issues in recent years . In such situations , the model has to be stripped of its capability to identify these categories . This is a difficult problem to solve , especially if the full training data is no longer available and only a few training examples are available . We present a “ Restricted Category Removal from Model Representations with Limited Data ” ( RCRMR-LD ) problem setting that simulates the above problem . In this paper , we propose to solve this problem in a fast and efficient manner . The objective of the RCRMR-LD problem is to remove the information regarding the restricted classes from the network representations of all layers using the limited training data available without affecting the ability of the model to identify the remaining classes . If we have access to the full training data , then we can simply exclude the restricted class examples from the training data and perform a full training of the model from scratch using the abundant data ( FDR - full data retraining ) . However , the RCRMR-LD problem setting is based on the scenario that the directive to exclude the restricted classes is received in the future after the model has already been trained on the full data and now only a limited amount of training data is available to carry out this process . Simply training the network from scratch on only the limited training data of the remaining classes will result in severe overfitting and significantly affect the model performance ( Baseline 2 , as shown in Tables 1 , 6 ) . Another possible solution to this problem is to remove the weights of the fully-connected classification layer of the network corresponding to the excluded classes such that it can no longer classify the excluded classes . However , this approach suffers from a serious problem . Since , in this approach , we only remove some of the weights of the classification layer and the rest of the model remains unchanged , it still contains the information required for recognizing the excluded classes . This information can be easily accessed through the features that the model extracts from the images . Therefore , we can use these features for performing classification . In this paper , we use a nearest prototype-based classifier to demonstrate that the model features still contain information regarding the restricted classes . Specifically , we use the model features of the examples from the limited training data to compute the average class prototype for each class and create a nearest class prototype-based classifier using them . Next , for any given test image , we extract its features using the model and then find the class prototype closest to the given test image . This nearest class prototype-based classifier performs close to the original fully-connected classifier on the excluded classes as shown in Tables 1 , 6 ( Baseline 1 ) . Therefore , even after using this approach , the resulting model still contains information regarding the restricted classes . Another possible approach can be to apply the standard fine-tuning approach to the model using the limited available training data of the remaining classes ( Baseline 8 ) . However , fine-tuning on such limited training data is not able to sufficiently remove the restricted class information from the model representations ( see Tables 1 , 6 ) , and aggressive fine-tuning on the limited training data may result in overfitting . Considering the problems faced by the naive approaches mentioned above , we propose a novel “ Efficient Removal with Preservation ” ( ERwP ) approach to address the RCRMR-LD problem . First , we propose a novel technique to identify the model parameters that are highly relevant to the restricted classes , and to the best of our knowledge , there are no existing prior works for finding such classspecific relevant parameters . Next , we propose a novel technique that optimizes the model on the limited available training data in such a way that the restricted class information is discarded from the restricted class relevant parameters , and these parameters are reused for the remaining classes . To the best of our knowledge , this is the first work that addresses the RCRMR-LD problem . Therefore , we also propose several baseline approaches for this problem ( see Sec . 11.2 ) . However , our proposed approach significantly outperforms all the proposed baseline approaches . Our proposed approach requires very few epochs to address the RCRMR-LD problem and is , therefore , very fast ( ∼ 200 times faster than the full data retraining model for the ImageNet dataset ) and efficient . The model obtained after applying our approach forgets the excluded classes to such an extent that it behaves as though it was never trained on examples from the excluded classes . The performance of our model is very similar to the full data retraining ( FDR ) model ( see Sec . 8.1 , Fig . 5 ) . We also propose the performance metrics needed to evaluate the performance of any approach for the RCRMR-LD problem . We perform experiments on several datasets to demonstrate the efficacy of our method . 2 PROBLEM SETTING . In this work , we present the restricted category removal from model representations with limited data ( RCRMR-LD ) problem setting , in which a deep learning model Mo trained on a specific dataset has to be modified to exclude information regarding a set of restricted/excluded classes from all layers of the deep learning model without affecting its identification power for the remaining classes ( see Fig . 1 ) . The classes that need to be excluded are referred to as the restricted/excluded classes . Let { Ce1 , Ce2 , ... , CeNe } be the restricted/excluded classes , where Ne refers to the number of excluded classes . The remaining classes of the dataset are the remaining/non-excluded classes . Let { Cne1 , Cne2 , ... , CneNne } be the non-excluded classes , whereNne refers to the number of remaining/nonexcluded classes . Additionally , we only have access to a limited amount of training data for the restricted classes and the remaining classes , for carrying out this process . Therefore , any approach for addressing this problem can only utilize this limited training data . 3 RCRMR-LD PROBLEM IN REAL WORLD SCENARIOS . A real-world scenario where our proposed RCRMR-LD problem can arise is federated learning ( McMahan et al. , 2017 ) . In the federated learning setting , there are multiple collaborators that have a part of the training data stored locally , and a model is trained collaboratively using these private data without sharing or collating the data due to privacy concerns . Suppose organization A has a part of the training data , and there are other collaborators that have other parts of the training data for the same classes . Organization A collaboratively trains a model with other collaborators using federated learning . After the model has been trained , a few classes may become restricted in the future due to some ethical or privacy concerns , and these classes should be removed from the model . However , the other collaborators may not be available or may charge a huge amount of money for collaborating again to train a fresh model from scratch . In this case , organization A does not have access to the full training data of the non-excluded/remaining classes that it can use to re-train a model from scratch in order to exclude the restricted classes information . This clearly shows that the RCRMR-LD problem is possible in federated learning . Another real-world scenario is the incremental learning setting ( Rebuffi et al. , 2017 ; Kemker & Kanan , 2018 ) , where the model receives training data in the form of sequentially arriving tasks . Each task contains a new set of classes . During a training session t , the model receives the task t for training and can not access the full data of the previous tasks . Instead , the model has access to very few exemplars of the classes in the previous tasks . Suppose before training a model on training session t , it is noticed that some classes from a previous task ( < t ) have to be removed from the model since those classes have become restricted due to privacy or ethical concerns . In this case , only a limited number of exemplars are available for all these previous classes ( restricted and remaining ) . This demonstrates that the RCRMR-LD problem is also possible in the incremental learning setting . We experimentally demonstrate in Sec . 8.3 , how our approach can address the RCRMR-LD problem in the incremental learning setting . 4 PROPOSED METHOD . Let , B refer to a mini-batch ( of size S ) from the available limited training data , and B contains training datapoints from the restricted/excluded classes ( { ( xei , yei ) | ( xe1 , ye1 ) , ... , ( xeSe , y e Se ) } ) and from the remaining/non-excluded classes ( { ( xnej , ynej ) | ( xne1 , yne1 ) , ... , ( xneSne , y ne Sne ) } ) . Here , ( xei , yei ) refers to a training datapoint from the excluded classes where xei is an image , y e i is the corresponding label and yei ∈ { Ce1 , Ce2 , ... , CeNe } . ( x ne j , y ne j ) refers to a training datapoint from the non-excluded classes where xnej is an image , y ne j is the corresponding label and y ne j ∈ { Cne1 , Cne2 , ... , CneNne } . Here , Se and Sne refer to the number of training examples in the mini-batch from the excluded and non-excluded classes , respectively , such that S = Se + Sne . Ne and Nne refer to the number of excluded and non-excluded classes , respectively . Let M refer to the deep learning model being trained using our approach and Mo is the original trained deep learning model . In a trained model , some of the parameters may be highly relevant to the restricted classes , and the performance of the model on the restricted classes is mainly dependent on such highly relevant parameters . Therefore , in our approach , we focus on removing the excluded class information from these restricted class relevant parameters . Since the model is trained on all the classes jointly , the parameters are shared across the different classes . Therefore identifying these class-specific relevant parameters is very difficult . Let us consider a model that is trained on color images of a class . If we now train it on grayscale images of the class , then the model has to learn to identify these new images . In order to do so , the parameters relevant to that class will receive large gradient updates as compared to the other parameters ( see Sec . 9.1 ) . We propose a novel approach for identifying the relevant parameters for the restricted classes using this idea . For each restricted class , we choose the training images belonging to that class from the limited available training data . Next , we apply a grayscale data augmentation technique/transformation f to these images so that these images become different from the images that the original model was earlier trained on ( assuming that the original model has not been trained on grayscale images ) . We can also use other data augmentation techniques that are not seen during the training process of the original model and that do not change the class of the image ( refer to Sec . 11.7 in the appendix ) . Next , we combine the predictions for each training image into a single average prediction and perform backpropagation . During the backpropagation , we study the gradients for all the parameters in each layer of the model . Accordingly , we select the parameters with the highest absolute gradient as the relevant parameters for the corresponding restricted class . Specifically , for a given restricted class , we choose the minimum number of such parameters from each network layer such that pruning these parameters will result in the maximum degradation of model performance on that restricted class . We provide a detailed description of the process for identifying the restricted class relevant parameters in Sec . 11.1 of the appendix . The combined set of the relevant parameters for all the excluded classes is referred to as the restricted/excluded class relevant parameters Θexrel ( see Fig . 2 ) . Please note that we use this process only to identify Θexrel , and we do not update the model parameters during this step . Pruning the relevant parameters for a restricted class can severely impact the performance of the model for that class ( see Sec . 9.1 ) . However , this may also degrade the performance of the model on the non-excluded classes because the parameters are shared across multiple classes . Therefore , we can not address the RCRMR-LD problem by pruning the relevant parameters of the excluded classes . Finetuning these parameters on the limited remaining class data will also not be able to sufficiently remove the restricted class information from the model ( see Sec . 11.10 in appendix ) . Based on this , we propose to address the RCRMR-LD problem by optimizing the relevant parameters of the restricted classes to remove the restricted class information from them and to reuse them for the remaining classes . After identifying the restricted class relevant parameters , our ERwP approach uses a classification loss based on the cross-entropy loss function to optimize the restricted class relevant parameters of the model on each mini-batch ( see Fig . 3 ) . We know that the gradient ascent optimization algorithm can be used to maximize a loss function and encourage the model to perform badly on the given input . Therefore , we use the gradient ascent optimization on the classification loss for the limited restricted class training examples to remove the information regarding the restricted classes from Θexrel . We achieve this by multiplying the classification loss for the augmented training examples from the excluded classes by a constant negative factor of -1 . We also optimize Θexrel using the gradient descent optimization on the classification loss for the limited remaining class training example , in order to reuse these parameters for the remaining classes . We validate using this approach through various ablation experiments as shown in Sec . 9.2 . The classification loss for the examples from the excluded and non-excluded classes and the overall classification loss for each mini-batch are defined as follows . Lec = Se∑ i=1 −1 ∗ ` ( yei , ye∗i ) ( 1 ) Lnec = Sne∑ j=1 ` ( ynej , y ne∗ j ) ( 2 ) Lc = 1 S ( Lec + Lnec ) ( 3 ) Where , ye∗i and y ne∗ j refer to the predicted class labels for x e i and x ne j , respectively . ` ( . , . ) refers to the cross-entropy loss function . Lec and Lnec refer to the classification loss for the examples from the excluded and non-excluded classes in the mini-batch , respectively . Lc refers to the overall classification loss for each mini-batch . Since all the network parameters were jointly trained on all the classes ( restricted and remaining ) , the restricted class relevant parameters also contain information relevant to the remaining classes . Applying the above process alone will still harm the model ’ s predictive power for the non-excluded classes ( as shown in Sec . 9.2 , Table 3 ) . This is because the gradient ascent optimization strategy will also erase some of the relevant information regarding the remaining classes . Further , applying Lnec on the limited training examples of the remaining classes will lead to overfitting and will not be effective enough to fully preserve the model performance on the remaining classes . In order to ensure that the model ’ s predictive power for the non-excluded classes does not change , we use a knowledge distillation-based regularization loss . Knowledge distillation ( Hinton et al. , 2014 ) ensures that the predictive power of the teacher network is replicated in the student network . In this problem setting , we want the final model to replicate the same predictive power of the original model for the remaining classes . Therefore , given any training example , we use the knowledge distillation-based regularization loss to ensure that the output logits produced by the model corresponding to only the non-excluded classes remain the same as that produced by the original model . We apply the knowledge distillation loss to the limited training examples from both the excluded and remaining classes , to preserve the non-excluded class logits of the model for any input image . We validate this regularization loss through ablation experiments as shown in Table 3 . We use the original model Mo ( before applying ERwP ) as the teacher network and the current model M being processed by ERwP as the student network , for the knowledge distillation process . Please note that the optimization for this loss is also carried out only for the restricted class relevant parameters of the model . Let KD refer to the knowledge distillation loss function . It computes the Kullback-Liebler ( KL ) divergence between the soft predictions of the teacher and the student networks and can be defined as follows : KD ( ps , pt ) = KL ( σ ( ps ) , σ ( pt ) ) ( 4 ) where , σ ( . ) refers to the softmax activation function that converts logit ai for each class i into a probability by comparing ai with logits of other classes aj , i.e. , σ ( ai ) = exp ai/κ∑ j exp aj/κ . κ refers to the temperature ( Hinton et al. , 2014 ) , KL refers to the KL-Divergence function . ps , pt refer to the logits produced by the student network and the teacher network , respectively . The knowledge distillation-based regularization losses in our approach are defined as follows . Lekd = Se∑ i=1 KD ( M ( xei ) [ C ne ] , Mo ( x e i ) [ C ne ] ) ( 5 ) Lnekd = Sne∑ j=1 KD ( M ( xnej ) [ C ne ] , Mo ( x ne j ) [ C ne ] ) ( 6 ) Lkd = 1 S ( Lekd + Lnekd ) ( 7 ) Where , M ( # ) [ Cne ] and Mo ( # ) [ Cne ] refer to the output logits corresponding to the remaining classes produced by M and Mo , respectively . # can be either xei or x ne j . Lekd and Lnekd refer to knowledge distillation-based regularization loss for the examples from the excluded and non-excluded classes , respectively . Lkd refers to the overall knowledge distillation-based regularization loss for each mini-batch . The total loss Lerwp of our approach for each mini-batch is defined as follows . Lerwp = Lc + βLkd ( 8 ) Where , β is a hyper-parameter that controls the contribution of the knowledge distillation-based regularization loss . We use this loss for fine-tuning the model for very few epochs .

Summary: In this paper, the authors present a new method to remove information about specific classes from a trained model without reducing the performance of the remaining classes. After the information is removed, the model should not be able to identify the class anymore. Instead of retraining the complete model from scratch without the restricted classes, the presented method only needs a few examples of the restricted classes and the remaining classes. In terms of speed, the presented method is ~200 times faster on ImageNet than a new model training without the restricted classes. Furthermore, they present a method for identifying model parameters that are mainly relevant to the restricted classes. The evaluation of the model is performed on the CIFAR-100, ImageNet-1k, and the CUB-200 dataset. For a detailed comparison, eight baseline methods were designed and evaluated. An ablation study is performed on the class relevant parameters and the number of classes that are excluded. The presented method achieves an accuracy close to the original model on the remaining classes in terms of accuracy. Also, the forgetting prototype accuracy is close to the model trained only on the remaining classes.