Patent ID: 11475130

Abstract:
Embodiments of the present invention concern detecting Test-Time Evasion (TTE) attacks on neural network, particularly deep neural network (DNN), classifiers. The manner of detection is similar to that used to detect backdoors of a classifier whose training dataset was poisoned. Given knowledge of the classifier itself, the adversary subtly (even imperceptibly) perturbs their input to the classifier at test time in order to cause the class decision to change from a source class to a target class. For example, an image of a person who is unauthorized to access a resource can be modified slightly so that the classifier decides the image is that of an authorized person. The detector is based on employing a method (similar to that used to detect backdoors in DNNs) to discover different such minimal perturbations for each in a set of clean (correctly classified) samples, to change the sample's ground-truth (source) class to every other (target) class. For each (source, target) class pair, null distributions of the sizes of these perturbations are modeled. A test sample is similarly minimally perturbed by the detector from its decided-upon (target) class to every other (potential source) class. The p-values according to the corresponding null distributions of these test-sample perturbations are assessed using the corresponding nulls to decide whether the test sample is a TTE attack.