Patent ID: 11704409

Abstract:
This patent concerns novel technology for detecting backdoors in neural network, particularly deep neural network (DNN) classification or prediction/regression models. The backdoors are planted by suitably poisoning the training dataset, i.e., a data-poisoning attack. Once added to an input sample from a source class of the attack, the backdoor pattern causes the decision of the neural network to change to the attacker's target class in the case of classification, or causes the output of the network to significantly change in the case of prediction or regression. The backdoors under consideration are small in norm so as to be imperceptible to a human or otherwise innocuous/evasive, but this does not limit their location, support or manner of incorporation. There may not be components (edges, nodes) of the DNN which are specifically dedicated to achieving the backdoor function. Moreover, the training dataset used to learn the classifier or predictor/regressor may not be available. In one embodiment of the present invention, which addresses such challenges, if the classifier or predictor/regressor is poisoned then the backdoor pattern is determined through a feasible optimization process, followed by an inference process, so that both the backdoor pattern itself and the associated source class(es) and target class are determined based only on the classifier or predictor/regressor parameters and using a set of clean (unpoisoned) samples, from the different classes (none of which may be training samples).