Process to make machine object detection robust to adversarial attacks

Described is a system for object detection that is robust to adversarial attacks. An initial hypothesis of an identity of an object in an input image is generated using a sparse convolutional neural network (CNN) and a distribution aware classifier. A foveated hypothesis verification process is performed for identifying a region of the input image that supports the initial hypothesis. Using a part-based classifier, an identity of a part of the object in the region of the input image is predicted. An attack probability for the predicted identity of the part, and the initial hypothesis is updated based on the predicted identity of the part and the attack probability. The foveated hypothesis verification process and updating of hypotheses is performed until a hypothesis reaches a certainty threshold. The object is labeled based on the hypothesis that reached the certainty threshold.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Non-Provisional Application of U.S. Provisional Application No. 63/060,494, filed in the United States on Aug. 3, 2020, entitled, “Process to Make Machine Object Detection Robust to Adversarial Attacks,” the entirety of which is incorporated herein by reference.

BACKGROUND OF INVENTION

(1) Field of Invention

The present invention relates to a system for performing machine vision tasks and, more particularly, to a system for performing machine vision tasks that is not vulnerable to adversarial attacks.

(2) Description of Related Art

Deep convolutional neural networks (CNNs) provide the state-of-the-art performance for machine vision tasks, such as object detection and recognition. However, deep CNNs are vulnerable to adversarial attacks. This vulnerability is rooted in the fact that these deep networks perform recognition based on a complex, but not necessarily meaningful, combination of high-level features. Consequently, small changes to just a portion of an image can bias the output to an incorrect class (e.g., ‘car’ image is recognized as ‘ostrich’). This vulnerability is further exacerbated by the nature of deep-net decision boundaries, which have two main weaknesses. First, they are open-ended and not limited to the actual distribution of training data. Second, the kernel functions that emerge through training contain redundant kernels that can provide a backdoor for attacks.

One of the most reliable defenses against adversarial attacks in the literature is that of Madry et al. (see Literature Reference No. 8 of the List of Incorporated Literature References). These methods bias the defense toward the choice of the metric (e.g., l∞norm). Another popular approach toward defenses against adversarial attacks is denoising-based approaches, such as Literature Reference No. 9. These methods denoise the input image, or the subsequent deep neural activations (i.e., features). These methods only work for high frequency adversarial perturbations.

Furthermore, many defenses against adversarial examples rely on so-called ‘obfuscated gradients’ (see Literature Reference No. 2), including gradient shattering, stochastic gradient, and exploding and vanishing gradients. It has recently been shown that a gradient approximation attack can fully bypass these defenses (see Literature Reference No. 2).

Thus, a continuing need exists for a method of defending against adversarial attacks in machine vision tasks that does not need to obfuscate gradients and is unbiased with regard to a specific metric.

SUMMARY OF INVENTION

The present invention relates to a system for performing machine vision tasks and, more particularly, to a system for performing machine vision tasks that is not vulnerable to adversarial attacks. The system comprises one or more processors and a memory having instructions such that when the instructions are executed, the one or more processors perform multiple operations. The system generates an initial hypothesis of an identity of an object in an input image using a sparse convolutional neural network (CNN) and a distribution aware classifier. A foveated hypothesis verification process is performed, wherein performing the foveated hypothesis verification process comprises identifying a region of the input image that supports the initial hypothesis. A part-based classifier predicts an identity of a part of the object in the region of the input image. An attack probability is determined for the predicted identity of the part. The initial hypothesis is updated based on the predicted identity of the part and the attack probability. The foveated hypothesis verification process and updating of hypotheses is performed until a hypothesis reaches a certainty threshold. The object is labeled based on the hypothesis that reached the certainty threshold, and an action performed by an autonomous platform is controlled based on the labeling of the object.

In another aspect, predicting the identity of a part of the object comprises: performing unsupervised part extraction to dissect the input image into a plurality of parts; generating a plurality of clusters of parts by performing unsupervised clustering of the plurality of parts; for each cluster of parts, learning an autoencoder model and storing each autoencoder model as prior knowledge; and recognizing parts of input images using the stored autoencoder models.

In another aspect, the foveated hypothesis verification process further comprises: receiving the input image, the predicted identities of parts, and a current hypothesis; using a recurrent neural network with long-short-term-memory, outputting a next region of the input image that supports the current hypothesis; and sending the next region to the part-based classifier.

In another aspect, the distribution aware classifier is based on generative models that capture a decision boundary by encoding distribution of a set of training data.

In another aspect, an attention map is obtained from the sparse CNN, the attention map is updated using the updated hypothesis, and the foveated hypothesis verification process is controlled using the updated attention map.

In another aspect, the autonomous platform is a vehicle, and the system causes the vehicle to perform a driving operation in accordance with the labeling of the object.

Finally, the present invention also includes a computer program product and a computer implemented method. The computer program product includes computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors, such that upon execution of the instructions, the one or more processors perform the operations listed herein. Alternatively, the computer implemented method includes an act of causing a computer to execute such instructions and perform the resulting operations.

DETAILED DESCRIPTION

Before describing the invention in detail, first a list of cited references is provided. Next, a description of the various principal aspects of the present invention is provided. Finally, specific details of various embodiment of the present invention are provided to give an understanding of the specific aspects.

(1) List of Incorporated Literature References

The following references are cited and incorporated throughout this application. For clarity and convenience, the references are listed herein as a central resource for the reader. The following references are hereby incorporated by reference as though fully set forth herein. The references are cited in the application by referring to the corresponding literature reference number as follows:1. Athalye, A., Engstrom, L., Ilyas, A. and Kwok, K., 2017. Synthesizing robust adversarial examples. arXiv preprint arXiv:1707.07397.2. Athalye, A., Carlini, N. and Wagner, D., 2018, July. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. In International Conference on Machine Learning (pp. 274-283).3. Carlini, N. and Wagner, D., 2017, May. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 39-57).4. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T. and Song, D., 2018. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1625-1634).5. Kolouri, S., Martin, C. E. and Hoffmann, H., 2017. Explaining distributed neural activations via unsupervised learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 20-28).6. Kolouri, S., Rohde, G. K. and Hoffmann, H., 2018. Sliced Wasserstein distance for learning Gaussian mixture models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3427-3436).7. Kolouri, S., Pope, P. E., Martin, C. E. and Rohde, G. K., 2019. Sliced Wasserstein Auto-Encoders. International Conference of Representation Learning (ICLR).8. Madry, A., Makelov, A., Schmidt, L., Tsipras, D. and Vladu, A., 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. International Conference of Representation Learning (ICLR).9. Prakash, A., Moran, N., Garber, S., DiLillo, A. and Storer, J., 2018. Deflecting adversarial attacks with pixel deflection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 8571-8580).10. Su, J., Vargas, D. V. and Sakurai, K., 2019. One-pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation.11. Houben, S., Stallkamp, J., Sairnen, J., Schlipsing, M. and Igel, C., 2013, August. Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark. In The 2013 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8).12. McInnes, L., Healy, J. and Melville, J., 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv: 1802.03426.

(2) Principal Aspects

The computer system100may include an address/data bus102that is configured to communicate information. Additionally, one or more data processing units, such as a processor104(or processors), are coupled with the address/data bus102. The processor104is configured to process information and instructions. In an aspect, the processor104is a microprocessor. Alternatively, the processor104may be a different type of processor such as a parallel processor, application-specific integrated circuit (ASIC), programmable logic array (PLA), complex programmable logic device (CPLD), or a field programmable gate array (FPGA).

(3) Specific Details of Various Embodiments of the Invention

Described is a system and method that is robust against adversarial attacks on machine learning systems. The invention described herein is inspired by human-perception and it automatically learns semantic concepts (i.e., learning the composition and relationships of scene and object components) to create a combinatoric barrier and improve recognition performance over state-of-the-art (SOA) machine vision systems for benign data. Here, the component-based detection is enabled through unsupervised component extraction, hypothesis verification, and tightening of decision boundaries.

The method according to embodiments of the present disclosure is applicable to image, video, and three-dimensional (3D) data (e.g., LIDAR).FIG.3shows a high-level concept of the invention, showing the thwarting of adversarial attacks by human-inspired hypothesis verification through part-based modeling, which is enabled through unsupervised mining of parts and hardened decision boundaries of part models. As shown, the present invention is based on three key bio-inspired innovations. The first innovation (element300) is hardening the neural network against attacks by improved decision boundaries, which are achieved by creating decision boundaries that conform to the distribution of training data, and by introducing sparsity of kernels in the convolutional layers of deep networks. The second innovation (element302) is mining of parts (e.g., wheels on a car) and attributes (e.g., wheels are round and connect the car to the road) in an unsupervised way from training data. The third innovation (element304) is using foveated hypothesis testing based on the composition of identified objects, scenes, and actions. For instance, a car is composed of various key components such as wheels, cabin, etc., with corresponding attributes, such as “wheels are round”. The foveation focuses on each component while tracking its composition.FIG.3depicts threat models (element306) used to attack an input image (element308). Non-limiting examples of threat models (element306) include digital perturbations (i.e., manipulation of digital input), physical perturbations (e.g., sticker attacks which entails putting a physical sticker on an object), data poisoning, and data collision. Data poisoning is an adversarial attack that attempts to manipulate a training dataset in order to control behavior of a trained model such that the model will label malicious examples as a desired class (e.g., labeling spam e-mails as safe). Data collision is a type of poisoning attack on neural networks in which the attacker adds manipulated data (i.e., poisoned data) to the victims training data such that when the victim trains her/his model on the poisoned data, the model's performance (e.g., accuracy) drops significantly on the test data. In the example illustrated inFIG.3, the attack is a sticker attack (element310), where a sticker is placed on the object (e.g., car (element312)). Given processing of the input image (element308) by a standard neural network (element314), the predicted output (element316) is incorrect, and the input image (element308) of the car (element312) is misclassified as ‘ostrich’.

The key technical innovations are motivated by bridging the gap between human and machine perception to overcome adversarial attacks. A central theme of the method described herein is to perform human-like part-based modeling (element318) of objects to create a combinatoric barrier that forces an attacker to alter multiple aspects of a scene in a consistent way before their attack can be successful. In support of this theme, the three innovations of unsupervised data decomposition (element302), hardened decision boundaries (element300), and foveated hypothesis verification (element304) all play a crucial role in correctly classifying the input image (element308), such that predicted output (element320) is ‘car’.

FIG.4depicts a functional level representation of the system described herein. As shown, the input image (element308) first passes through a sparse convolutional neural network (CNN) (element400) with sparse neural activations and a distribution-aware classifier (element402) to generate an initial hypothesis (element404). The initial hypothesis (element404) is passed to a recursive probabilistic hypothesis generator (element406). The initial hypothesis is then used as a top-down modulatory signal (element408) to the neural network (element400) to obtain a current attention map (element410) that is passed to the foveated hypothesis verification (element304) module alongside the hypothesis (element404) and the input image (element308).

Foveated hypothesis verification (element304) then provides a local region (i.e., a foveated region (element412)) of the input image (element308) that is most likely to support the generated hypothesis. The local region is determined through a top-down attention that provides a Gaussian-like mask that can be multiplied with the input image to obtain only a local visible region of the input image. The local foveated region (element412) is passed to a part identification (element414) module, which was pre-learned based on unsupervised mined parts and attributes. The part identification (element414) module consists of a part classifier (element416) and a part model (element418) to obtain an ID (element420) for the part and measure the certainty of the predicted ID (i.e., attack probability (element422)) via fitting a multi-variate Gaussian distribution to each cluster (components with the same part ID) and calculating the likelihood of the input component with respect to this multivariate Gaussian. The ID (element420) is a numerical value (e.g., part 1, part 2). The part IDs are cluster assignments. The attack probability is one of the outputs of the part identification (element414), which is a scaler between 0-1 that identifies whether the image part is attacked or not. The attack probability emerges as the process of learning part-classifier and part-model.

The predicted part ID (element420) and its corresponding attack probability (element422) are fed back to the recursive hypothesis generator (element406) to update the initial hypothesis (element404). The updated hypothesis (element424) is then used as a top-down signal (element408) to update the attention map (element410), which is then used to control foveation (element304). This process is repeated until the updated hypothesis (element424) reaches a certainty threshold (i.e., reaches a confident decision). The certainty threshold is an application-dependent hyper-parameter of the approach described herein. For critical applications, a higher certainty threshold is required.

In summary, the present invention devises an unsupervised attribution of input parts (e.g., spatial-temporal attributes) and uses it together with dynamic foveation to reach a consistent hypothesis about the input sensory data. The neural-network machinery used in the system differs from the standard CNNs in that a brain-inspired sparsification of neural activations and a distribution aware classifier that provides a baseline protection against perturbation attacks are used. Each of these aspects will be described in further detail below.

(3.1) Protection Against Adversarial Attacks

The protection against physical and digital world attacks using the system according to embodiments of the present disclosure emerges from the interplay between unsupervised data decomposition (element302) for part-based modeling (element318), foveated hypothesis verification (element304) with a top-down attention mechanism (elements408and410), and a core neural network (element400) with hardened decision boundaries (element300). For unsupervised data decomposition (element302), parts/attributes (e.g., element322) are learned from input sensory data (e.g., input image (element308) in an unsupervised manner, and an implicit model for object part decomposition is built. The foveated hypothesis verification (element304) uses a recurrent neural network (element426) with long-short-term-memory (LSTM) to sequentially inspect the sensory input to confirm or change a hypothesis. Below, each key element of the system is described.

Digital inference attacks are often in the form of small additive perturbations to the sensory input. These attacks are quite counter-intuitive to humans, as they are designed to be invisible to the human eye. Such attacks are possible because the trained neural networks often learn redundant and correlated kernels that leave abundant room for the adversary to attack the system. For a fixed sensory input, there are many active neurons (i.e., neurons with non-zero neural activations) in each layer of the neural network. This enables the attacker to attach a small perturbation to each active neuron that accumulates to a large distortion for the down-stream neurons.

FIG.5depicts the unique use of sparsity which acts as a barrier against small perturbation attacks by reducing the redundancy in the network. Sparsity forces an attacker to use larger perturbations without reducing network accuracy on normal images. Sparser activations translates to less chance for the perturbation attack to propagate throughout the network. Small perturbation attacks are defended against by sparsifying the neural activations in the network (i.e., having only a few neurons with non-zero activations). To achieve sparsification from a standard CNN (element500) to a sparse CNN (element502), two processes are utilized: 1) regularizing the neurons to perform in their saturated state, by penalizing deviation from saturation (e.g., 0 and 1) for each neuron during training, and 2) regularizing a weighted L1-norm of layerwise neural activations. The bottom ofFIG.5depicts two rows, where the top row (element504) shows the learned convolutional kernels in a standard convolutional neural network (CNN), while the bottom row (element506) shows the learned convolutional kernels in a sparse CNN. The output of convolving the input patch with the convolutional kernels for both methods (i.e., convolution as inner product) are neural activations (element508), where the standard CNN has many active neurons while the sparse CNN has only a few neurons active.

Sparsifying the neural activations forces the attacker to use larger perturbations to achieve a successful attack, but does not prevent the attack completely.FIGS.6A and6Billustrate standard decision boundaries and distribution aware boundaries of the present invention, respectively. The distribution aware classifier effectively prevents large perturbation attacks (i.e., outlier attacks). To fortify against large perturbation attacks, a distribution-aware decision boundary is utilized (FIG.6B). Standard classifiers used in today's neural networks (e.g., softmax classifiers) provide an open-ended decision boundary (FIG.6A) that leaves abundant room for the attacker to design a large perturbation attack that could still be subtle (e.g., one-pixel attacks (see Literature Reference No. 10)). These large perturbation attacks leverage the fact that decision boundaries in neural networks have no mechanism to detect outliers and could classify a point far away from any training data they have seen with high confidence. To overcome this shortcoming, a distribution aware classifier based on generative models (e.g., Sliced Wasserstein Autoencoder (see Literature Reference No. 7)) is devised that keeps track of the distribution of the training data, enabling the network to detect outliers and provide a realistic certainty for the output decision. Generative models, such as Sliced Wasserstein Auto-Encoders, capture the decision boundary more accurately by encoding the distribution of the training data.

Unsupervised part decomposition (element302) is at the heart of the system described herein, which enables the system to build a part-based model (element418) for the sensory input data and verify the initial hypothesis (element404) by existence of parts and their relation(s). The main challenge with part-based hypothesis verification is that part-based labeling of sensory input data is extremely expensive, and therefore, supervised learning of part decomposition is out of the question for large datasets. To achieve the goal in the present invention, input data is dissected based on its neural activation patterns with respect to the trained neural network with hardened decision boundaries on clean (i.e., un-attacked) training data. In Literature Reference No. 5, it was shown that the Nonnegative Matrix Factorization (NMF) of the final convolutional layer of a pre-trained CNN leads to blob-like masks that identify the semantically meaningful parts of the sensory input. In the present system, this technique is leveraged to dissect the input data, by calculating the non-negative matrix factorization of the intermediate features of a neural network.FIG.7depicts unsupervised part identification, which includes unsupervised part extraction (element700) and unsupervised part clustering (element702). The part identification (element414) module learns to decompose input images (element704) into their semantic parts/attributes and assigns an ID (element420) to each part in a fully unsupervised manner so that no additional human intervention is required. In this context, “semantic” refers to visually identifiable parts of an object. For instance, in a car, headlights, wheels, and doors are the visually identifiable parts.

Why would the part-based modeling (element318) help with defense against adversarial attacks? The answer is two-fold: 1) while each part can be attacked separately, achieving a consistent targeted attack on all parts is more challenging; and 2) hardening the probabilistic boundary for parts is easier than hardening it for the entire image, making it challenging to attack part models. The rationale here is that the sub-manifold for part variations has a much simpler structure compared to the manifold of variations for general sensory input. “Hardening” refers to the transition fromFIG.6AtoFIG.6B. In a classic neural network, the decision boundaries are open-ended, which exposes the network to adversarial attacks. In contrast, a hardened decision boundary follows the training distribution (FIG.6B) and doesn't allow room for an adversarial attack.

FIG.8illustrates specialized part autoencoder (AE) models (element800) trained to recognize classes of parts identified in images. These models (element800) are stored and used to recognize parts of new images presented to the system. By using the AE reconstruction error (element801) as a measure of match, localized attacks on individual parts is overcome. The AE reconstruction error (element801) is the output of the auto-encoder (element803), which is required to be a reconstruction of the input parts (element802). In general, an auto-encoder (element803) squeezes the information in the input data/image through a bottleneck and enforces the bottleneck to contain as much information about the input as possible. This maximal preservation of information is enforced by requiring the reconstructed data/image from the bottleneck information (i.e., the decoded feature) be as close to the input as possible. The reconstruction error (element801) is obtained by comparing the reconstructed data/images to the inputs.

The extremely high-dimensional nature of the input space for images, videos, and 3D data makes approximating the true data distribution infeasible because too many training samples would be required. However, parts have a much lower complexity. For each cluster of parts (element802), a simple model (element800) (e.g., a variational or Sliced-Wasserstein autoencoder) is learned, and the model (element800) is saved as prior knowledge for each part cluster (element802). During testing and for an attacked part (element804), even when the part identifier misidentifies the part (e.g., the letter “S” in a stop sign is identified as a “4” (element806)), the prior model would report a high reconstruction error (element808), indicating a low probability (element810) of the attacked part (element806) belonging to the identified class. The mechanism is related to the idea of the distribution aware classifier inFIGS.6A and6B. The reconstruction error is determined by calculating the Mean Squared Error (MSE) of the reconstructed part, which includes subtracting the output of the auto-encoder from the input, squaring it, and calculating the sum of the squared errors.

To test if a combination of identified parts is consistent, foveated hypothesis verification (element304) is used. This strategy also allows the approach to naturally extend from object classification to object detection because parts are localized within an image. The common element of physical world patch attacks is that the attacker modifies a small part of the field-of-view (often with large perturbations), causing the classifier to misclassify the entire object or scene. These attacks work in the physical world and can be made robust to pose, viewpoint, and lighting variations (see Literature Reference Nos. 1 and 4). While often local, the attack can seep throughout the neural activations and poison a large portion of down-stream neurons, causing the neural network to become hypnotized by the patch and only focus on the patch region. This locality motivates the idea that local processing is required to enforce the network to snap itself out of such hypnosis in real-world attacks.

To that end, a unique method was devised, referred to as foveated hypothesis verification (element304), which is a recurrent active attention mechanism that resembles the foveation and saccadic eye movement in human vision. The foveated hypothesis verification (element304) mechanism, depicted inFIG.9, checks if an input (element308) is consistent with a part-based model. The foveated hypothesis verification (element304) uses a recurrent model that is capable of extracting information from input sensory data in a local and sequential manner to update its hypothesis about the input (element308). It receives the sensory input (element308), an attention map (element410), the object IDs from previous foveated regions, and the current hypothesis. The foveated hypothesis verification (element304) then outputs the next foveated region (element412) that could provide evidence for the current hypothesis and sends the foveated region (element412) to the part-based classifier (element416). This verification process is not limited to parts within an object but also extends to parts that are expected in the context of an object (e.g., parts of an intersection near a stop sign).

In unsupervised part clustering (element702inFIG.7), an ID is assigned to each part (i.e., part-level IDs (element420)). Then, for each part ID, an auto-encoder (element803) is learned which serves as a means to identify the distribution of the part. As shown inFIG.9, given that there are multiple auto-encoders (element803) (one per part ID) for an input part, first the part's ID needs to be identified, which is performed by the part classifier (element416inFIG.4). Then, the correct auto-encoder (element803) corresponding to the identified part ID needs to be retrieved (i.e., retrieve model (element902inFIG.9)). The retrieved auto-encoder (element803) then receives the input part (element802) and outputs the reconstruction error (element801inFIG.8), which is then fed to the Recursive Probabilistic Hypothesis Generator (element406inFIG.9).

How does the foveated hypothesis verification (element304) achieve the task of proposing regions that provide evidence for the hypothesis? The foveated hypothesis verification (element304) is trained in a reinforcement learning setting. The network receives a large numerical reward when it achieves the correct hypothesis with high certainty and with as little foveation steps as possible to enforce learning only the most influential parts of the sensory input. In this manner, the network implicitly learns the part relationships for optimal foveation. Once foveated parts (i.e., foveated regions (element412) are identified, the system described herein updates its hypothesis (element900) about the perceived object.

FIG.10depicts how objects are robustly identified from their parts using a recursive probabilistic hypothesis generator (element406) that reevaluates an initial hypothesis (element404) iteratively using parts found by the foveated hypothesis verification (element304) and part identification (element414). The likelihood for sensory input (element308) belonging to a certain class is estimated. The foveated hypothesis verification (element304), together with the part identification (element414), provides local information about parts/attributes of the sensory input (element308). The initial hypothesis (element404) about the input (element308) is recursively updated using the part information until a verified hypothesis (element1000) is obtained.

Let x be the sensory input (element308) and let c k identify the k′th class of interest (note that the method is readily extendable to regression tasks and is not specific to classification). Referring toFIG.10, the classes of interest are “ostrich” (C1) and “car” (C2). Let Zirepresent the i′th part/attribute of x. InFIG.10, “car part” (element1002) refers to a part/attribute of an image of a car. The likelihood for x belonging to class k is written as p(x|ck) (e.g., p(“car part”|“ostrich”). The softmax classifier in a standard neural network approximates this likelihood. The input then is classified via argmakp(x|ck). When observing a sequence of parts z1, . . . , zMthat construct the sensory input x, evaluate if p(x|ck) agrees with p(z1, . . . , zM|ck). Furthermore, take into account the statistical dependencies between parts. For instance, observing the ‘trunk’ of a car would reduce the probability p(z2=Headlights|ck=Car, z1=Trunk) and increase the probability for

p(z2=talelights|ck=Car, z1=Trunk). This leads to a natural interdependence of parts:

dp(x|ck)≈p(z1, . . . , zM|ck)=p(z1|ck)p(z2|ck, z1) . . . p(zM|ck, . . . , zM-1), where the hypothesis is updated (i.e., change hypothesis (element1004)) with each received component. As a result, the system according to embodiments of the present disclosure reevaluates the initial hypothesis (element404) by a probabilistic model of parts to defend against adversarial attacks.

(3.1.4) Test of Elements of the Invention

Two key elements of the invention described herein were tested to determine if the combination of unsupervised part modeling and hardened decision boundaries with autoencoders dramatically improve resilience to attacks. The German Traffic Sign Recognition Benchmark dataset (see Literature Reference No. 11) was used, and the goal was to detect stop signs. As illustrated inFIG.11, an input image (element1100) of a stop sign was first parsed, using a part parser (element1102), into input parts (element802), for which a Sliced-Wasserstein Autoencoder (SWAE) (element803) was learned. In the latent space of the SWAE (element803), a Gaussian Mixture Model (GMM) (element1104) was learned and a part-based probabilistic classifier (element416) was defined.

A standard CNN was trained for this binary classification task as the baseline model, which was able to achieve a 99.8% accuracy. For a hardened system, first the input images (element1100) were automatically parsed into their parts (element802) and the SWAE (element803) (see Literature Reference No. 7) was trained with a Gaussian prior and with a 16-dimensional latent space (element1104). The SWAE (element803) was learned by constraining its latent space to be class discriminative (i.e., stop-sign vs other). After learning the discriminative SWAE (element803), a GMM (element1104) (see Literature Reference No. 6) in the latent space was learned. Distributional modeling with the GMM (element1104) resulted in stop sign related Gaussians (element1106) and non-stop sign related Gaussians (element1108), and a probabilistic classifier (element416) was defined for the image class. The decoded parts (element1110) are the reconstructions of the input parts (element802), which is equivalent to the AE reconstruction error (element801) ofFIG.8. Given that a 16-D space cannot be visualized, a dimensionality reduction approach was used to reduce dimensionality from 16-D to 2-D and plot the data (i.e., 2D visualization of the 16-D space (element1112)). The approach used was Umap (see Literature Reference No. 12), but other dimensionality reduction approaches could be utilized.

For the threat model, the Carlini and Wagner (CW) attack (see Literature Reference No. 3) was used in a white-box setting with l2-norm constraints.FIG.12shows a sample attack on a stop-sign input image (element1100) together with the l2and l∞norms of the perturbation. The attacked image (element1100) is visually similar to a stop sign, but the standard CNN confidently misclassified the image as others (element1200) (i.e., no stop sign). For the baseline defense, Madry's defense (see Literature Reference No. 8) was adapted and a separate standard CNN was trained, which is required to be resistant to the CW attack.

Then, all systems were attacked with varying attack strength (i.e., increasing upper-bound of the average l2-norm of the adversarial perturbation), and the performance of each system was measured. For classification, the invention's defense increased the computational cost by about 2× compared to the standard CNN (still fast: two milliseconds (msec) per image using one graphic processing unit (GPU)), and training the defense of the system described herein was twice as fast as Madry's. As depicted in the plot inFIG.13, the invention described herein (represented by bold solid curve (element1300)) demonstrated a much higher accuracy compared to both a standard CNN (represented by dashed curve (element1302)) and Madry's defense (represented by unbolded solid curve (element1304), particularly for strong attacks. Various strengths of the attack were tested.

FIG.14includes a table of deficiencies of existing machine vision systems and how the system and method described herein addresses these deficiencies. For instance, open ended decision boundaries of existing systems are improved with hardened decision boundaries of the present invention. In addition, modeling complex objects was improved by decomposing a complex object into simpler, semantically meaningful components in the present invention. Furthermore, current deep neural networks still use a bag-of-attribute representation of objects, meaning that the geometric relationship between different object parts is discarded. In contrast, the present invention learns object part relationships and iteratively checks for expected parts and discovers inconsistencies using foveated hypothesis verification.

Adversarial attacks on machine vision systems pose a major threat on industries relying on automated vision. Self-driving cars, for instance, heavily rely on machine vision systems as their perception front. Drones also rely on reliable machine vision systems for various tasks including navigation and intelligence, surveillance, and reconnaissance (ISR). A machine-vision system that is robust to adversarial attacks would be significantly useful in various applications, including ISR and autonomous driving. Once a verified hypothesis has been obtained (FIG.10, element1000), the system described herein labels one or more objects in an input image with high accuracy. Once the object is labeled, the present invention can generate a command to control an action performed by an autonomous vehicle/platform (element1002). For instance, the system and method according to embodiments of the present disclosure can be used in automatic control of an autonomous platform, such as a robot, autonomous self-driving ground vehicle, and unmanned aerial vehicle (UAV). Non-limiting examples of devices that can be controlled via the processor (FIG.1, element104) include a motor vehicle or a motor vehicle component (electrical, non-electrical, mechanical), such as a brake, a steering mechanism, suspension, or safety device (e.g., airbags, seatbelt tensioners, etc.). For instance, upon labeling and, thus identification, of an object in the input image, the action to be performed can be a driving operation/maneuver (such as steering or another command) in line with driving parameters in accordance with the now labeled object. For example, if the system recognizes a bicyclist, another vehicle, or a pedestrian in the environments surrounding the autonomous driving system/vehicle, the system described herein can cause a vehicle maneuver/operation to be performed to avoid a collision with the bicyclist or vehicle (or any other object that should be avoided while driving). The system can cause the autonomous vehicle to apply a functional movement response, which may be the task to be performed, such as a braking operation followed by a steering operation (etc.), to redirect vehicle away from the object, thereby avoiding a collision.

Other appropriate actions may include one or more of a steering operation, a throttle operation to increase speed or to decrease speed, or a decision to maintain course and speed without change. The responses may be appropriate for avoiding a collision, improving travel speed, or improving efficiency. As can be appreciated by one skilled in the art, control of other device types is also possible. Thus, there are a number of automated actions that can be initiated by the autonomous platform given the particular object assigned a label.