Patent Description:
In recent advances, machine learning and deep neural networks have been widely used for the classification of data. However, machine learning models are often vulnerable to attacks based on adversarial manipulation of the data. The adversarial manipulation of the data is known as an adversarial example. The adversarial example is a sample of the data that is intentionally modified with small feature perturbations. These feature perturbations are intended to cause a machine learning or deep neural network (ML/DNN) model to output an incorrect prediction. In particular, the feature perturbations are imperceptible noise to the data causing an ML classifier to misclassify the data. Such adversarial examples can be used to perform an attack on ML systems, which poses security concerns. The adversarial examples pose potential security threats for ML applications, such as robots perceiving the world through cameras and other sensors, video surveillance systems, and mobile applications for image or sound classification.

The adversarial example attack is broadly categorized into two classes of threat models, such as a white-box adversarial attack and a black-box attack. In the white-box adversarial attack, an attacker accesses the parameters of a target model. For instance, the attacker accesses the parameters, such as architecture, weights, gradients, or the like of the target model. The white-box adversarial attack requires strong adversarial access to conduct a successful attack. Additionally, such white-box adversarial attack suffers higher computational overhead, for example, time and attack iterations. In contrast, in the black-box adversarial attack, the adversarial access of the parameters of the target model is limited. For example, the adversarial access only includes accessing example input data and output data pairs for the target model. Alternatively, in the black-box adversarial attack, any information of the target model is not used. In such an adversarial attack, a substitute or a source model is trained with training data to generate an adversarial perturbation. The generated adversarial perturbation is added to the input data to attack a target black-box DNN. For example, an input image is inputted to the substitute model to generate an adversarial perturbation. The adversarial perturbation is then added to the input image to attack the target black-box DNN. In some cases, a model query is used to obtain information from the target black-box DNN.

Traditional techniques for making machine learning models more robust, such as weight decay and dropout, generally do not provide a practical defense against adversarial examples. So far, only two methods, i.e., adversarial training and defensive distillation, have provided a significant defense. Adversarial training is a brute force solution that generates a lot of adversarial examples and explicitly trains the model not to be fooled by them. Defensive distillation is a strategy that trains the model to output probabilities of different classes, rather than hard decisions about which class to output. The probabilities are supplied by an earlier model, trained on the same task using hard class labels. This creates a model whose surface is smoothed in the directions an adversary will typically try to exploit, making it difficult for them to discover adversarial input tweaks that lead to incorrect categorization.

An example for an approach from the prior art to train a variational information bottleneck that improves its robustness to adversarial perturbations can be found in <NPL>. Moreover, <NPL> disclose a one off and attack-agnostic Feature Manipulation (FM)-Defense to detect and purify adversarial examples in an interpretable and efficient manner. Lastly, <NPL> discloses an adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.

However, adversarial examples are hard to defend against because it is difficult to construct a theoretical model of the adversarial example crafting process. Adversarial examples are solutions to an optimization problem that is non-linear and non-convex for many ML models, including neural networks. Adversarial examples are also hard to defend against because they require machine learning models to produce good outputs for every possible input. Most of the time, machine learning models work very well but only work on a small amount of all the many possible inputs they could encounter.

In addition, current techniques for making machine learning models more robust are not adaptive as they may block one kind of attack, but leave vulnerability open to another attacker. To that end, designing a defense that can protect against a powerful, adaptive attacker is an important, but so far an unsolved technical problem.

Accordingly, there is a need to overcome the above-mentioned problems. More specifically, there is a need to develop a method and system for training the neural network for improving adversarial robustness of the neural network while retaining the natural accuracy.

It is an object of some embodiments to provide a system and a method for training robust neural network models with improved resilience to adversarial attacks. Additionally or alternatively, it is an object of some embodiments to provide a system and a method to classify input data using a trained neural network with improved adversarial robustness. Additionally or alternatively, it is an object of some embodiments to provide a system and a method to classify the input data probabilistically to improve the accuracy of the classification under adversarial attacks or free from adversarial attacks.

To that end, some embodiments disclose a neural network that includes a probabilistic encoder configured to encode input data of a plurality of data samples into a distribution over a latent space representation and a classifier configured to classify an encoding of the input data in the latent space representation. The probabilistic encoder is contrasted with a deterministic encoder. While the deterministic encoder encodes the input data into the latent space representation, the probabilistic encoder encodes the input data into a distribution over the latent space representation. For example, to encode the input data in the distribution of the latent space representation, the probabilistic encoder can output parameters of the distribution.

First, the classifier does not classify the distribution over the latent space representation but an instance (first instance or second instance) of the latent space representation or a sample of the distribution encoded by the probabilistic encoder. This allows sampling the output of the probabilistic encoder multiple times to combine the results of the classification for more accurate classification results.

Second, the probabilistic encodings allow improving the training of the neural network by imposing additional requirements not only on the classification results but also on the distribution over the latent space representation itself. Both of these advantages, alone or in combination improve the adversarial robustness of the trained neural network.

For example, some embodiments are based on a recognition that the performance of machine learning methods is dependent on the choice of data representation, and the goal of representation learning is to transform a raw input x to a lower-dimensional representation z that preserves the relevant information for tasks such as classification or regression. The adversarial examples are solutions to an optimization problem that is non-linear and non-convex for many ML models. Some embodiments are based on a realization that it is challenging to provide theoretical tools for describing the solutions to these complicated optimization problems. The information bottleneck (IB) principle provides an information-theoretic method for representation learning, where a representation should contain only the most relevant information from the input for downstream tasks. Representations learned by the IB principle are less affected by nuisance variations and maybe more robust to adversarial perturbations. In addition, the multi-view information bottleneck can extend the IB principle to a multi-view unsupervised setting by maximizing the shared information between different views, while minimizing the view-specific information.

Some embodiments are based on a realization that it is possible to extend the multi-view information bottleneck method to a supervised setting with adversarial training. For example, some embodiments can consider adversarial examples as another view of corresponding clean samples. As a result, the embodiments seek to learn representations that contain the shared information between clean samples and corresponding adversarial samples, while eliminating information not shared between them. As described above, having the probabilistic encoder that encodes the input data into the distribution over the latent space representation rather than encoding into the instance of the latent space representation allows different embodiments to explore the theoretical guarantees provided by the principles of the multi-view information bottleneck to improve the robustness and/or performance of the trained neural network.

To take advantage of these principles, some embodiments train shared parameters of different instances of the neural network using pairs of clean and adversarial data samples by optimizing a multi-objective loss function of outputs of the different instances. Because the different instances are the instances of the same neural network including the probabilistic encoder and the classifier, the outputs of the different instances (the first instance and the second instance) include parameters of the probabilistic distribution of the latent space representation and the results of classification. By comparing and optimizing the difference of these outputs, the resilience to adversarial attacks is improved.

Accordingly, one embodiment discloses a computer-implemented method for training a neural network. The method includes collecting a plurality of data samples comprising clean data samples and adversarial data samples. The training of the neural network includes training of a probabilistic encoder to encode the plurality of data samples into a probabilistic distribution over a latent space representation. In addition, the training of the neural network comprising training of a classifier to classify an instance of the latent space representation to produce a classification result. In addition, the method includes training shared parameters of a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Further, the method includes outputting the shared parameters of the first instance of the neural network and the second instance of the neural network.

Accordingly, another embodiment discloses an AI system for training a neural network. The AI system includes a processor; and a memory having instructions stored thereon. The processor is configured to execute the stored instructions to cause the AI system to collect a plurality of data samples as input for training the neural network. The plurality of data samples comprising clean data samples and adversarial data samples. The training of the neural network includes training of a probabilistic encoder to encode the plurality of data samples into a probabilistic distribution over a latent space representation. In addition, the training of the neural network includes training of a classifier to classify an instance of the latent space representation to produce a classification result. Further, the processor cause the AI system to train shared parameters of a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Furthermore, the processor causes the AI system to output the shared parameters of the first instance of the neural network and the second instance of the neural network.

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure may be practiced without these specific details. In other instances, apparatuses and methods are shown in block diagram form only in order to avoid obscuring the present disclosure.

As used in this specification and claims, the terms "for example", "for instance", and "such as", and the verbs "comprising", "having", "including", and their other verb forms, when used in conjunction with a listing of one or more components or other items, are each to be construed as open ended, meaning that that the listing is not to be considered as excluding other, additional components or items. The term "based on" means at least partially based on. Further, it is to be understood that the phraseology and terminology employed herein are for the purpose of the description and should not be regarded as limiting. Any heading utilized within this description is for convenience only and has no legal or limiting effect.

<FIG> shows a schematic block diagram of a system <NUM> for training a neural network, such as a neural network <NUM> for improving adversarial robustness, according to some embodiments of the present disclosure. The system <NUM> includes a plurality of data samples <NUM>, the neural network <NUM>, a classifier 104A, a classifier 104B, a probabilistic encoder 106A, and a probabilistic encoder 106B.

The system <NUM> collects the plurality of data samples <NUM> as input for training the neural network <NUM>. In some embodiments, the plurality of data samples <NUM> includes clean data samples x and adversarial data samples x'. The clean data samples x herein refers to correct data samples used for training of the neural network <NUM>. The adversarial data samples x' herein refers to incorrect data samples (for example, data samples with some perturbations) used for the training of the neural network <NUM>. Additionally, the training of the neural network <NUM> includes training of the probabilistic encoder 106A to encode the plurality of data samples (i.e., the clean data samples x) into a probabilistic distribution (for example, with an associated clean cross-entropy CE(ŷ ,y)) over a latent space representation z. The training of the neural network <NUM> includes training of the classifier 104A to classify an instance (for example, a first instance <NUM>) of the latent space representation z to produce a classification result.

More specifically, the system <NUM> is configured to initially train the neural network <NUM> based on the clean data samples x. The clean data samples x are fed as an input to the probabilistic encoder 106A. The probabilistic encoder 106A is further configured to generate a stochastic representation (i.e., intermediate representation) z based upon execution of the probabilistic encoder 106A. The stochastic representation corresponds to the latent space representation z. The stochastic representation z is passed through remaining layers 108A. The remaining layers 108A corresponds to the hidden layers of the neural network <NUM>. Furthermore, the system <NUM> is configured to train the probabilistic encoder 106A based on the clean cross-entropy CE (ŷ ,y).

Similarly, the training of the neural network <NUM> includes training of the probabilistic encoder 106B to encode the plurality of data samples (i.e., the adversarial data samples x') into a probabilistic distribution over a latent space representation z'. The training of the neural network <NUM> includes training of a classifier 104B to classify an instance (for example, a second instance <NUM>) of the latent space representation z' to produce a classification result.

More specifically, the system <NUM> is configured to initially train the neural network <NUM> based on the adversarial data samples x'. The adversarial data samples x' are fed as an input to the probabilistic encoder 106B. The probabilistic encoder 106B is further configured to generate a stochastic representation (i.e., intermediate representation) z' based upon execution of the probabilistic encoder 106B. The stochastic representation z' is passed through hidden layers 108B. Furthermore, the system <NUM> is configured to train the probabilistic encoder 106B based on the adversarial cross-entropy CE (ŷ', y).

The system <NUM> is further configured to train shared parameters of the first instance <NUM> of the neural network <NUM> using the clean data samples x. Similarly, the system <NUM> is configured to train shared parameters of the second instance <NUM> of the neural network <NUM> using the adversarial data samples x'.

Furthermore, the system <NUM> is configured to generate an output <NUM> based on the shared parameters of the first instance <NUM> of the neural network <NUM> and the second instance <NUM> of the neural network <NUM>. In this manner, the neural network <NUM> is trained based on the stochastic representation z corresponding to the clean data samples x as well as the stochastic representation z' corresponding to the adversarial data samples x'.

In one embodiment, the system <NUM> is configured to train the neural network <NUM> such that the stochastic representations z and z' contain shared information (i.e., mutual information) between x and x'. To achieve this, the system <NUM> is configured to minimize Kullback-Leibler divergence (KL-divergence) between the latent space distribution produced by the probabilistic encoder 106A and the latent space distribution produced by the probabilistic encoder 106B, and maximize the shared information (i.e., mutual information) between z and z'.

The system <NUM> is an artificial intelligence-based system (herein after AI system) that is further explained in <FIG>,<FIG>.

<FIG> shows a schematic block diagram 200A of an AI system <NUM> for training a neural network, such as the neural network <NUM> for improving adversarial robustness, according to some embodiments of the present disclosure. The AI system <NUM> includes a processor <NUM> and a memory <NUM>. The memory <NUM> stores instructions to be executed by the processor <NUM>. The memory <NUM> also includes the neural network <NUM>. The processor <NUM> is configured to execute the stored instructions to cause the AI system <NUM> to collect the plurality of data samples <NUM> as input for training the neural network <NUM>.

In some embodiments, examples of the processor <NUM> include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, graphical processing unit (GPU), a field-programmable gate array (FPGA), and the like. In some embodiments, the memory <NUM> includes suitable logic, circuitry, and/or interfaces to store a set of computer-readable instructions for performing operations. Additionally, examples of the memory <NUM> may include a random-access memory (RAM), a read-only memory (ROM), a removable storage drive, a hard disk drive (HDD), and the like. It will be apparent to a person skilled in the art that the scope of the disclosure is not limited to realizing the memory <NUM> in the AI system <NUM>, as described herein.

As shown in <FIG>, the plurality of data samples <NUM> is fed as an input to the AI system <NUM>. The AI system <NUM> invokes the processor <NUM> to execute the stored instructions in the memory <NUM> to start training of the neural network <NUM>.

In some embodiments, the plurality of data samples <NUM> includes the clean data samples x and the adversarial data samples x'. The AI system <NUM> is configured to train the neural network <NUM> to improve the adversarial robustness of the neural network <NUM>. In some embodiments, the neural network <NUM> includes a deep neural network (DNN), and the like. In some embodiments, the AI system <NUM> is configured to perform training of the neural network <NUM> in a supervised setting.

In some embodiments, the AI system <NUM> is configured to train the neural network <NUM> based on a multi-objective loss function. The AI system <NUM> is configured to train the neural network <NUM> with an objective of (<NUM>) maximizing a shared information <NUM> between the stochastic representations of matched pairs and (<NUM>) minimizing the shared information <NUM> between each stochastic representation and its corresponding view conditioned on the other view, along with (<NUM>) the clean cross-entropy loss, and (<NUM>) the adversarial crossentropy loss. For example, the item (<NUM>) corresponds to maximizing the mutual information objection, and item (<NUM>) corresponds to minimizing the KL-divergence objective.

The AI system <NUM> is configured to improve the adversarial robustness based on maximizing the shared information <NUM> between the stochastic representations z and z' corresponding to the matched pairs of clean data samples x and the adversarial data samples x', as captured by the objective of maximizing the mutual information between z and z'. Additionally, the objective of training of the neural network <NUM> includes symmetrized KL-divergence between the posterior feature distribution the clean data samples x and the adversarial data samples x', and the shared information <NUM> between the latent representation of the clean data samples x and the adversarial data samples x'.

For example, a dataset {(xi, yi}i=<NUM>,. ,n with K classes is given, where <MAT> is a clean data sample and yi ∈ {<NUM>,. , K} is its associated label. Further f is a classifier with parameters θ, and the output of the classifier fθ(xi) are the estimated probabilities of xi belonging to each class. In traditional adversarial training, the learning problem objective is defined as: <MAT>.

Here, <IMG> is the cross-entropy loss and the adversary searches for an example x', belonging to <IMG>(x, ε) = {x' : x+σ ∥ σ ∥p ≤ ε, by maximizing the cross-entropy loss with respect to a small perturbation σ.

The AI system <NUM> is configured to learn the latent space representations z and z' (corresponding to x and x', respectively), which only contains the useful information shared by both x and x'. Mathematically, the generation of these representations are defined by conditional distributions p(z|x) and p(z'|x'), while satisfying the Markov chain z → x → x' → z'.

The AI system <NUM> is further configured to improve generalization by learning representations z or z' that capture only information shared between x and x'. If the representation preserves only the shared information <NUM> (i.e., the mutual information) from both x and x', that means it includes only task-relevant information, while discarding view-specific details (i.e., misleading information from x') and therefore, adversarial robustness of the neural network <NUM> is improved.

<FIG> illustrates block diagram 200B of representation of z with respect to x and x' for sufficiency and minimality of mutual information, in accordance with various embodiments of the present disclosure.

Let us consider subdividing I (z; x) into three components by using the chain rule of mutual information, and since the Markov chain z → x → x' holds, <MAT>.

Here, I (x; z|x') represents the information in z which is unique to x and not shared by x', which is termed as view-specific information. The second term I (x; x') denotes the shared information <NUM> between x and x'. The last term I (x; x'|z) is the shared information <NUM> that is missing in z. The main objective here is for the representation z to only contain the shared information <NUM> of x and x', so that I (x; z) = I (x; x'). Thus, the objective here is to minimize I (x; z|x') and I (x; x'|z). The representation z is defined as sufficient and minimal for any downstream task, as it contains all the task-relevant information (sufficiency) without any irrelevant information (minimality).

The block diagram 200B includes representation (a) for sufficient but not minimal mutual information: I(x; z|x <NUM> ) > <NUM>, I(x; x <NUM> |z) = <NUM>. In addition, the block diagram 200B includes representation (b) for minimal but not sufficient mutual information: I(x; z|x <NUM> ) = <NUM>, I(x; x <NUM> |z) > <NUM>. The block diagram 200B includes representation (c) for not sufficient and not minimal mutual information: I(x; z|x <NUM> ) > <NUM>, I(x; x <NUM> |z) > <NUM>. Furthermore, the block diagram 200B includes representation (d) for sufficient and minimal mutual information: I(x; z|x <NUM> ) = <NUM>, I(x; x <NUM> |z) = <NUM>. The mutual information between x is exactly equal to the shared information of x and x'.

<FIG> shows a diagrammatric representation depicting a procedure <NUM> for training the neural network <NUM>, according to some embodiments of the present disclosure. The procedure <NUM> is performed by the AI system <NUM>.

At step <NUM>, a plurality of data samples <NUM> is collected. The plurality of data samples <NUM> includes clean data samples x and adversarial data samples x'. The clean data samples x herein refers to correct data samples used for training of the neural network <NUM>. The adversarial data samples x' herein refers to incorrect data samples (for example, data samples with some perturbations) used for the training of the neural network <NUM>.

At step <NUM>, training of the neural network <NUM> is performed. The training of the neural network <NUM> includes encoding of the plurality of data samples <NUM> into a probabilistic distribution over a latent space representations z and z'. The plurality of data samples <NUM> are encoded using probabilistic encoder 106A and 106B. The probabilistic encoder 106A and 106B encodes the plurality of data samples (i.e., the clean data samples x and the adversarial data samples x') into a probabilistic distribution (for example, clean cross-entropy CE (ŷ, y)) and adversarial cross-entropy CE (ŷ', y)) over the latent space representation z and z', respectively. The training of the neural network <NUM> further includes training of the classifier 104A and 104B to classify an instance (for example, a first instance <NUM> and a second instance <NUM>) of the latent space representation z and z' to produce a classification result.

At step <NUM>, shared parameters of the first instance <NUM> of the neural network <NUM> and the second instance <NUM> of the neural network <NUM> are trained. The shared parameters of the first instance <NUM> of the neural network <NUM> are trained using the clean data samples x. The shared parameters of the second instance <NUM> of the neural network using the adversarial data samples x'.

The neural network <NUM> is trained based on a multi-objective loss function. The first instance <NUM> of the neural network <NUM> and the second instance <NUM> of the neural network <NUM> are jointly trained to minimize the multi-objective loss function of a difference between corresponding outputs of the first instance <NUM> and the second instance <NUM>. The corresponding outputs comprising a difference between the probabilistic distribution determined by the probabilistic encoders 106A and 106B of the first instance <NUM> and the second instance <NUM> of the neural network <NUM> and the classification result determined by the classifier 104A and 104B of the first instance <NUM> and the second instance <NUM> of the neural network <NUM>. The joint training of the first instance <NUM> of the neural network <NUM> and the second instance <NUM> of the neural network <NUM> is performed with the latent representations z and z' for the clean data samples x and the adversarial samples x' respectively that are sampled multiple times.

The AI system <NUM> is configured to train the neural network <NUM> with an objective of (<NUM>) maximizing a shared information <NUM> between the stochastic representations of matched pairs and (<NUM>) minimizing the shared information <NUM> between each stochastic representation and its corresponding view conditioned on the other view, along with (<NUM>) the clean cross-entropy loss, and (<NUM>) the adversarial crossentropy loss.

The AI system <NUM> is configured to improve the adversarial robustness based on learning of the shared information <NUM> or the output between the clean data samples x and the adversarial data samples x'. Additionally, the objective of training of the neural network <NUM> includes symmetrized KL-divergence between the posterior feature distribution the clean data samples x and the adversarial data samples x', and the shared information <NUM> between the latent representation of the clean data samples x and the adversarial data samples x'.

At step <NUM>, the shared parameters of the first instance <NUM> and the second instance <NUM> of the neural network <NUM> are outputted.

<FIG> shows a representation <NUM> depicting a multi-objective loss function <NUM>, according to the present disclosure. The neural network <NUM> of <FIG> is trained to parameterize the multi-objective loss function based on mutual information of the distributions over the latent space representation z and z' determined by the probabilistic encoder 106A and 106B of the first instance <NUM> and the second instance <NUM> of the neural network <NUM> respectively and entropy losses (CE (ŷ, y), CE (ŷ', y)) of the classification result produced by the first instance <NUM> and the second instance <NUM> of the neural network <NUM>.

Additionally, the multi-objective loss function <NUM> includes terms corresponding to maximizing the mutual information between the probabilistic distributions of encodings of pairs of the clean data samples x and the adversarial data samples x', minimizing mutual information between encodings of one of the clean data samples x or the adversarial data samples x' in the pair conditioned on another data sample in the pair, a clean cross-entropy loss determined for classifying the clean data samples x, and an adversarial cross-entropy loss determined for classifying the adversarial data samples x'.

As explained above in <FIG>,<FIG>, the AI system <NUM> is configured to learn a representation including only the shared information of x and x' by minimizing the view-specific information (I (x; z|x')) and shared information not in z (I (x; x'|z)). In particular, minimizing I (x; x'|z) is equivalent to maximizing I (z; x'), because I (z; x') = I (x; x') - I (x; x'|z) and given x and x', I (x; x') is constant. Therefore, a relaxed Lagrangian objective <IMG> may be used to obtain a representation z that is sufficient and minimal with respect to x and x' as: <MAT>.

Symmetrically, a relaxed Lagrangian objective <IMG> may be used to obtain a representation z' that is sufficient and minimal with respect to x' and x may be obtained as: <MAT>.

Here, λ<NUM> and λ<NUM> represent the Lagrangian multipliers for the the constrained optimization. The objective function involves two mutual information terms that are hard to calculate directly. To solve this problem, some alternative bounds for these two mutual information terms are derived.

Upper Bound of I (x; z|x'): Initially, an upper bound of view-specific information in the latent representation z is derived from input. For example, I (x; z|x') may be calculated as: <MAT> <MAT>.

Here, the conditional distributions p(z|x) and p(z'|x') may be parameterized by an encoder network. Additionally, this bound is tight whenever the representation z is the same as z'. Symmetrically, I (x'; z'|x) is upper bounded by DKL(p(z'|x')∥p(z|x)).

Lower Bound of I (z; x'): Further, a lower bound on the mutual information between the clean representation and the corresponding adversarial sample is derived. I (z; x') may be calculated as: <MAT>.

Here, I (z; z'lx') = <NUM>, because z', as the representation of x', is part of the Markov chain z → x → x' → z'. It is to be noted that while the bound is also immediate from this Markov chain and the data processing inequality, the derivation above illustrates that the bound is tight when z' is a sufficient statistic of z. Symmetrically, a similar bound may be derived for I (z'; x) ≥ I (z; z'). Conceptually, this lower bound captures our goal of preserving the information shared between the representations regardless of the adversarial perturbation.

Furthermore, <IMG> and <IMG> are combined so that the representations z and z' may contain the shared information between x and x'. Based on the bounds derived above, the multi-objective loss function <IMG> is obtained, which is an upper bound on the average of <IMG> and <IMG>. The objective function <IMG> may be defined as: <MAT>.

Here p(z|x) and p(z'lx') are modeled as Gaussian distributions parameterized by a neural network encoder N(µθ (x), <MAT> and N(µθ (x'), <MAT>.

DSKL represents the symmetrized KL-divergence obtained by averaging DKL(p(z'|x')∥p(z|x)) and DKL(p(z|x)∥p(z'|x')).

This symmetrized KL-divergence may be computed directly between two Gaussian posterior distributions. Alternatively, I (z; z') requires the use of a mutual information estimator. The present disclosure utilizes Hilbert Schmidt Independence Criterion (HSIC) to measure the independence between z and z', and use this value to replace mutual information term. It is to be noted that HSIC is used as a surrogate for mutual information because the dependence between two mini-batch samples in Reproducing Kernel Hilbert Space (RKHS) can be measured directly, without requiring any density estimation or using an additional network for mutual information estimation.

Moreover, the above regularization objective <IMG> is combined with task label information to obtain our overall objective function for training the neural network model <NUM> as: <MAT>.

Here α ∈ [<NUM>, <NUM>] balances the trade-off between the cross entropy loss on clean and adversarial samples. β and λ adjust the importance of symmetrized KL-divergence term and the mutual information term.

<FIG> shows a block diagram <NUM> of the AI system <NUM> for generating the adversarial data samples x' for training the neural network <NUM>, according to some embodiments of the present invention. The block diagram <NUM> includes a communication channel <NUM> and a modification module <NUM>. The AI system <NUM> is configured to collect the plurality of data samples by performing a first step and a second step. The AI system <NUM> performs the first step of receiving the clean data samples x over the communication channel <NUM>. The communication channel <NUM> comprises one or a combination of a wired channel and a wireless channel. The AI system performs the second step of modifying each of the clean data samples x using the modification module <NUM> to generate a corresponding adversarial data sample forming the pairs of the clean data samples x and the adversarial data samples x'. The modification module <NUM> applies an adversarial example generation method on the clean data samples x. The adversarial example generation method comprises one of projected gradient descent method, fast-gradient sign method, limited-memory Broyden-Fletcher-Goldfarb-Shanno method, Jacobian-based saliency map attack, or Carlini & Wagner attack.

<FIG> shows a block diagram of a computer-based system <NUM> for improving adversarial robustness, in accordance with some embodiments of the present disclosure. The system <NUM> includes at least one processor <NUM> and a memory <NUM> having instructions stored thereon including executable instructions for being executed by the at least one processor <NUM> during controlling of the system <NUM>. The memory <NUM> is embodied as a storage media such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk, or any combinations thereof. For instance, the memory <NUM> stores instructions that are executable by the at least one processor <NUM>. In one example embodiment, the memory <NUM> is configured to store a neural network <NUM>. The neural network <NUM> corresponds to the neural network <NUM> of <FIG>.

The at least one processor <NUM> is be embodied as a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The at least one processor <NUM> is operatively connected to a sensor <NUM>, a receiver <NUM> via a bus <NUM>. In an embodiment, the at least one processor <NUM> is configured to collect a plurality of data samples. In some example embodiments, the plurality of data samples is collected from a receiver <NUM>. The receiver <NUM> is connected to an input device <NUM> via a network <NUM>. Each of the plurality of data samples is stored in storage <NUM>. In some other example embodiments, the plurality of data samples is collected from the sensor <NUM>. The sensor <NUM> receives a data signal <NUM> measure from a source (not shown). In some embodiments, the sensor <NUM> is configured to sense the data signal <NUM> based on a source of the sensed data signal <NUM>.

Additionally or alternatively, the system <NUM> is integrated with a network interface controller (NIC) <NUM> to receive the plurality of data samples <NUM> (of <FIG>) using the network <NUM>. The plurality of data samples includes clean data samples and adversarial data samples.

The at least one processor <NUM> is also configured to train the neural network <NUM> for improving adversarial robustness. The training of the neural network <NUM> includes encoding of the plurality of data samples <NUM> into a probabilistic distribution over a latent space representation. The plurality of data samples <NUM> are encoded using probabilistic encoder.

The trained neural network <NUM> generates output of shared information that is transmitted via a transmitter <NUM>. Additionally or alternatively, the transmitter <NUM> is coupled with an output device <NUM> to output the shared information over a wireless or a wired communication channel, such as the network <NUM>. The output device <NUM> includes a computer, a laptop, a smart device, or any computing device that is used for preventing adversarial attacks in applications installed in the output device <NUM>.

Also, individual embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may be terminated when its operations are completed but may have additional steps not discussed or included in a figure. Furthermore, not all operations in any particularly described process may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the function's termination can correspond to a return of the function to the calling function or the main function.

<FIG> shows a use case <NUM> of using the AI system <NUM>, according to some other embodiments of the present disclosure. The use case <NUM> corresponds to vehicle assistance navigation system (not shown) of a vehicle 702A and a vehicle 702B. The vehicle assistance navigation system is connected with the AI system <NUM>. The vehicle assistance navigation system is connected to a camera of the vehicle 702A, such as a front camera capturing road scenes or views. In one illustrative example scenario, the camera captures a road sign <NUM> that displays "No Parking" sign. The captured road sign <NUM> is transmitted to the AI system <NUM>. The AI system <NUM> processes the captured road sign <NUM> using the trained neural network <NUM>. The captured road sign <NUM> is processed using the clean data samples and the adversarial data samples to generate a robust model for identifying the "No Parking" sign in the road sign <NUM>. The robust model is used by the vehicle assistance navigation system to accurately identify the road sign <NUM> and prevent the vehicle 702A and the vehicle 702B from parking at no parking zone.

Lastly, one embodiment of the subject-matter relates to a computer-implemented method for training a neural network, wherein the method uses a processor coupled with stored instructions implementing the method, wherein the instructions, when executed by the processor carry out steps of the method, comprising:.

The above-described embodiments of the present disclosure may be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, with one or more processors in an integrated circuit component. Though, a processor may be implemented using circuitry in any suitable format.

Also, the various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

Also, the embodiments of the present disclosure may be embodied as a method, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts concurrently, even though shown as sequential acts in illustrative embodiments. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true scope of the present disclosure.

Claim 1:
A computer-implemented method for training a neural network (<NUM>) for classifying a plurality of image data samples (<NUM>) captured by a camera, wherein the method uses a processor (<NUM>) that stores instructions for implementing the method, wherein the instructions, when executed, cause the processor (<NUM>) to perform the method, comprising:
collecting a plurality of data samples (<NUM>) as input for training the neural network (<NUM>), wherein the plurality of data samples (<NUM>) comprising clean data samples (x) and adversarial data samples (x'), wherein training of the neural network (<NUM>) comprising training of a probabilistic encoder (106A, 106B) to encode the plurality of data samples (<NUM>) into a probabilistic distribution over a latent space representation, wherein training of the neural network comprising training of a classifier (104A, 104B) to classify an instance of the latent space representation to produce a classification result; the method characterized by the method steps:
training shared parameters of a first instance (<NUM>) of the neural network (<NUM>) using the clean data samples (x) and a second instance (<NUM>) of the neural network (<NUM>) using the adversarial data samples (x'); and
outputting the shared parameters of the first instance (<NUM>) of the neural network (<NUM>) and the second instance (<NUM>) of the neural network (<NUM>); the method further comprising
parameterizing a multi-objective loss function (<NUM>) based on mutual information of the distributions over the latent space representation determined by the probabilistic encoder (106A, 106B) of the first instance (<NUM>) and the second instance (<NUM>) of the neural network (<NUM>) and entropy losses of the classification result produced by the first instance (<NUM>) and the second instance (<NUM>) of the neural network (<NUM>); wherein
the multi-objective loss function (<NUM>) comprises terms corresponding to maximizing mutual information between the probabilistic distributions of encodings of pairs of the clean data samples (x) and the adversarial data samples (x'), minimizing mutual information between encodings of one of the clean data samples (x) or the adversarial data samples (x') in the pair conditioned on another data sample in the pair, a clean cross-entropy loss determined for classifying the clean data samples (x), and an adversarial cross-entropy loss determined for classifying the adversarial data samples (x').