METHOD AND SYSTEM FOR CREATING AN ENSEMBLE OF NEURAL NETWORK-BASED CLASSIFIERS THAT OPTIMIZES A DIVERSITY METRIC

One embodiment provides a system which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers. The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers. The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric. The system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.

BACKGROUND

Field

This disclosure is generally related to machine learning and data classification. More specifically, this disclosure is related to a method and system for creating an ensemble of neural network-based classifiers that optimizes a diversity metric.

Related Art

In the field of machine learning, adversarial examples can exploit the way that artificial intelligence algorithms work in order to disrupt the behavior of the algorithms. Recently, an increasing number and types of attacks have been devised in order to fool the algorithms, along with increasingly stronger defenses against such attacks. One large class of these attacks is “perturbation-bounded evasion attacks,” which involve adversarial examples constructed by perturbing data samples with the goal of forcing a classifier to misclassify them. Such evasion attacks comprise a predominant class of attacks considered in current machine learning technology. One specific type of evasion attack involves adversarial examples which can be trivially classified by a human but can fool a machine learning classifier.

One solution to address these evasion attacks is to use an ensemble or collection of classifiers. However, a principled analysis based on linear models derived from convolutional neural networks (CNNs) remains a challenge.

SUMMARY

One embodiment provides a system which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers. The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair (i.e., each pair) of mutually orthogonal affine classifiers of the three or more affine classifiers. The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric.

In some embodiments, one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.

In some embodiments, a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.

In some embodiments, the three or more affine classifiers comprise a multi-class classification layer.

In some embodiments, the three or more affine classifiers comprise a one-versus-all classification layer.

In some embodiments, the decision boundaries of the plurality of affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.

In some embodiments, the three or more affine classifiers comprise an odd number of affine classifiers. The system generates a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.

In some embodiments, the system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.

In some embodiments, predicting the outcome for the testing data object is further based an ensemble decision rule.

In some embodiments, defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and training the one or more neural networks is further based on the training data.

DETAILED DESCRIPTION

Introduction and Overview

The embodiments described herein solve the problem of addressing perturbation-bounded evasion attacks by providing a system which constructs an ensemble of neural network-based classifiers that optimizes a diversity metric and which can defend against misclassification.

As described above, adversarial machine learning examples can exploit the way that artificial intelligence algorithms work in order to disrupt the behavior of the algorithms. Recently, an increasing number and types of attacks have been devised in order to fool the algorithms, along with increasingly stronger defenses against such attacks. One large class of these attacks is “perturbation-bounded evasion attacks,” which involve adversarial examples constructed by perturbing data samples with the goal of forcing a classifier to misclassify them. Such evasion attacks comprise a predominant class of attacks considered in current machine learning technology. One specific type of evasion attack involves adversarial examples which can be trivially classified by a human but can fool a machine learning classifier.

One solution to address these evasion attacks is to use an ensemble or collection of classifiers. For example, analyzing robustness against adversarial examples using linear models derived from convolutional neural networks (CNNs) is described in application Ser. No. 17/158,631. As another example, creating an ensemble of machine learning models to defend against adversarial examples is described in application Ser. No. 17/345,996. In another example, learning an ensemble of neural network classifiers by partitioning the training data randomly or by class is described in application Ser. No. 17/400,016.

However, these previous approaches did not maintain explicit control over the generated classifiers. A CNN can learn a complicated kernel such that the data is approximately linearly separable after the kernel mapping. Because this kernel cannot be expressed in the closed form, the kernel may be difficult to analyze and providing optimal classifier constructions may also be challenging. Thus, a principled analysis based on the CNN-derived linear models remains a challenge.

The embodiments described herein provide a system which addresses the above-described challenges by imposing diversity on the classifier itself and not based on, e.g., partitioning the training data as in application Ser. No. 17/400,016. The system can define a diversity metric based on pairwise angles between decision boundaries of a plurality of affine classifiers, specifically, between three or more affine classifiers and of an odd number of affine classifiers. The odd number of classifiers can allow for a definitive classification decision rule, e.g., using an ensemble decision rule. These decision boundaries can be mutually orthogonal or close to mutually orthogonal (e.g., within a predetermined threshold of being mutually orthogonal). That is, the pairwise angles can be 90 degrees (perpendicular) or close to 90 degrees (nearly perpendicular). The system can include the diversity metric as a regularization term in a loss function optimization for designing each pair of mutually orthogonal affine classifiers of the three or more affine classifiers (of an odd number), and the system can further train one or more neural networks to learn parameters of the one or more neural networks which are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric.

To make neural network-based models amenable to a diversity metric, the described embodiments replace the final classification layer of the neural network with the three or more affine classifiers of an odd number. As a result, the system can compute the angle between various classifiers, and specifically, between each pair of classifiers. Thus, for each class label, the system can construct an ensemble of one-versus-all affine classifiers using the diversity metric, as described below.

The system can learn the neural network weight via back propagation, using two approaches. The first approach is a “multi-head approach,” in which the system trains a single neural network with multiple heads which represent the multiple diverse affine classifiers, as described below in relation toFIGS.3and5. The second approach is a “multi-model approach,” in which the system trains multiple neural networks such that their respective affine classifiers (in a pairwise manner) are mutually orthogonal or close to mutually orthogonal (e.g., within a predetermined threshold of being mutually orthogonal), as described below in relation toFIGS.4and6.

Given a 2-class problem, e.g., panda and gibbon classes, consider an adversarial perturbation of a panda (“adversarial panda”). A first classifier may incorrectly classify this adversarial panda as a gibbon. The optimal adversarial perturbation may be to perturb the original image of the panda and generate the adversarial panda by moving the original image in a perpendicular manner to the decision boundary of the first classifier. A second classifier which is perpendicular to the first classifier would be unlikely to incorrectly classify the adversarial panda. The mutual orthogonality of the first classifier and the second classifier can thus result in the second classifier being robust to the adversarial perturbation which fooled the first classifier.

This principle can be extrapolated to the N-class problem such that

each classifier is pairwise orthogonal with each other classifier (i.e., mutually orthogonal). Thus, a system which uses N mutually orthogonal classifiers can provide robust protection against adversarial manipulations which may address only one decision boundary as it is the case of a single linear model.

The principle of mutually orthogonal classifiers can apply in a case with linear or affine classifiers. In the case of neural networks or CNNs, the decision boundaries are not linear. A CNN can learn a complicated kernel such that the data is approximately linearly separable after the kernel mapping. While Gaussian kernels may be used to create a linear mapping of non-linear data, it may be more challenging to create this mapping in CNNs.

The described embodiments split the neural network into two parts. The first part includes all of the convolutional neural network layers which come before the classification layer. The second part is the classification layer itself, which outputs the decision. The system can feed this output decision from multiple classifiers into an ensemble decision model to obtain a final result, decision, or outcome. The described embodiments can address the above-described mapping issue by using back propagation to train the earlier layers of the CNN to behave in a certain manner.

Furthermore, as described below, an exactly mutual orthogonality may be less feasible or effective than a nearly mutual orthogonality. This “nearly mutual” (i.e., within a predetermined threshold of being exactly mutually perpendicular or orthogonal) concept can be defined by the diversity metric. The system can include the diversity metric as a regularization term in a loss function optimization for designing each pair of mutually orthogonal affine classifiers of the three or more affine classifiers (of an odd number), and the system can further train one or more neural networks to learn parameters of the one or more neural networks which are consistent with parameters of the three or more affine classifiers. This can result in obtaining an ensemble of neural network-based classifiers which optimize the diversity metric.

Exemplary Environment for Construction of Ensemble of Classifiers to Defend Against Adversarial Examples

FIG.1presents an exemplary environment100which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric, in accordance with an embodiment of the present application. Environment100can include: a device102, an associated user112, and an associated display screen114; a device104and an associated or included storage device106; and a device108. Devices102,104, and108can communicate with each other via a network110. Device102can be a client computing device, e.g., a laptop computer, a mobile telephone, a smartphone, a tablet, a desktop computer, and a handheld device. Devices102,104, and108can be a computing device, e.g., a server, a networked entity, and a communication device.

During operation, device108can request training data from device104, and device104can send training data to device108(via a get training data118communication and training data120). Device108can receive training data120(as training data122) and perform a series of operations to construct an ensemble of neural network-based classifiers which optimize a diversity metric to defend against adversarial examples. Upon receiving training data122(or a request to classify testing data166from user112via device102), device108can determine the training data and also determine the size and type of classifiers to be generated (operation134). Device108can define a diversity metric based on pairwise angles between decisions boundaries of affine classifiers for the training data (operation136). The number of affine classifiers can be three or more and of an odd number. Device108can include the defined diversity metric as a regularization term of a loss function optimization for designing the affine classifiers (operation138). Device108can then train one or more neural networks such that the parameters of the neural networks (“NN parameters”) are consistent with parameters of the affine classifiers (operation140), e.g., via iteration through forward and backward propagation. The number of neural network kernels to be trained can be either a single kernel with multiple final affine classifier layers (as in the “multi-head” approach described below in relation toFIGS.3and5) or multiple kernels each with their own respective final affine classifier layer (as in the “multi-model” approach described below in relation toFIGS.4and6). Device108can perform these operations, including training the neural network(s), based on the ensemble size (184), the type of approach (with or without diversity) (186), and the type of classifier as multiclass or one-versus-all (188), as determined by operation134and/or selected by user112via elements in display114.

Device108can return the results of the ensemble of obtained classifiers on the training data (i.e., the classifiers which were obtained from training the one or more neural networks via iteration) (operation142). Device108can send the result of the ensemble of classifiers on the training data to device102(as results144). Device102can receive ensemble results on training data144(as results146) and can perform a display information148operation, which can cause to be displayed on display114at least: the type of data180(e.g., whether the data is clean or based on an adversarial attack); the type of attack182(if the data type is data under attack, e.g., a fast gradient method (FGM) or a projected gradient descent (PGD) attack); the size of the ensemble184(the number of classifiers or final classification layers in the system); the type of approach used (e.g., with or without diversity)186; the type of classifier used in the final classifier or classification layer (e.g., multiclass or one-versus-all)188; the result from an individual classifier190; and the overall ensemble result192(e.g., as based on an ensemble decision rule such as a majority vote or a maximum of an average of a probability of each class as reported by the individual models/classifiers). As an example, display114can include table800, which is described below in relation toFIG.8. The system can display any of the information described above on display114, in any combination, which can allow user112to interact with display114to perform additional actions. While not depicted inFIG.1, display114can also include table900ofFIG.9.

User112can view the information displayed on display114and can perform an action149. For example, user112can change a configuration or setting related to, e.g., the type of data (180), the type of attack (182), the ensemble size (184), the type of approach (186), and the type of classifier (188). While not depicted inFIG.1, user112can also change a setting related to the regularization term.

As another example, user112may interact with the information presented on display114to view detailed information about a specific set of results or ensemble of classifiers or classification result. In some embodiments, user112can select a certain set of classifiers of the displayed or presented ensemble of classifiers (e.g., to view more detailed information) and can also generate (via a user interface widget, not shown) and send a command to update the ensemble (similar to generate ensemble command130).

Furthermore, user112, via device102, can determine or generate a testing data set, including a testing data object (e.g., via an operation162). The testing data set (and the testing data object) can include data under attack, e.g., modified based on an adversarial technique. In some embodiments, device102(or user112via device102) can modify or perturb the testing data to obtain attacked data (operation164). Device102can send a corresponding request to classify the testing data (via a communication166). Device108can receive the request to classify the testing data (as a request168) and can predict an outcome/class for the testing data (operation170). Operation170can include running the previously generated ensemble of classifiers on the testing data. The system can continue to train the one or more neural networks via iteration such that the parameters of the neural networks are consistent with the parameters of the affine classifiers (operation140). This can result in increasing the robustness of the ensemble, which in turn can result in an increased accuracy for classification.

Device108can send a predicted outcome/class172to device102. Device102can received predicted outcome/class172(as outcome/class174) and can perform a display information176operation, which can cause certain information to be displayed on display114, as described above in relation to operation148. The information displayed on display114can further include a predicted outcome194. Display114can also include, e.g., tables similar to those described below in relation toFIGS.8and9.

User112can perform an action179, which can be similar to action149as described above, e.g., changing a setting, interacting with displayed information, selecting certain classifiers, and generating a command to update the ensemble based on user-configured changes.

Principled Pairwise Diversity Metric for Multiple Affine Classifiers

The described embodiments can optimize the selection of component classifiers in an ensemble in order to achieve robustness to adversarial examples. This optimization can be driven by the diversity metric, which can quantify how different the component classifiers are from each other. If the classifiers are not diverse, their performance under adversarial perturbation may be similar. In contrast, classifiers which are diverse can exhibit adversarial robustness to different adversarial examples. As a result, combining the classification decisions into an ensemble decision rule can provide increased overall robustness. U.S. patent Ser. No. 17/944,939 describes a principled pairwise diversity metric for a pair of affine classifiers.

Let F1(·)+b1and F2(·)+b2be two c-class affine classifiers on, where F1and F2contain the weights and b1and b2are biases taken from the origin. The diversity metric can be defined as:

This diversity metric as defined in Equation (1) (and further described in U.S. patent Ser. No. 17/944,939) can be used in the cost function of an optimization problem designed to yield a diverse ensemble of classifiers. For a two-classifier ensemble, this optimization problem can have a cost function which is composed of the loss functions incurred by each classifier and the pairwise diversity metric, as shown in the following equation:

That is, the desired optimization problem for generating a diverse ensemble of two classifiers can be represented as:

Because the defined diversity metric is a pairwise diversity metric, the described embodiments can use an approach which generalizes the above optimization problem to multiple classifiers, i.e., to N classifiers, where N is three or more and where N is odd to allow for a definite classification decision rule. The system can accomplish this by including additional loss functions for the component classifiers in the ensemble, as well as the additional pairwise diversity terms.

For an ensemble of N classifiers, there may be N loss functions and (N2) pairwise diversity terms, each with its weight λi, where i=1, 2, . . . , (N2). In some embodiments a single weight λ can be used for all classifiers. Note that the output of the optimization problem can be the entire ensemble of affine classifiers, i.e., the classifiers are not designed incrementally one by one, but instead are generated all at once.

For each classifier i∈N in the ensemble of classifiers, the optimization to learn the classifier parameters of Fican be expressed as:

Each classifier can include a regularization term which enforces

orthogonality among its parameters and the parameters of the other classifiers in the ensemble. In Equation (4), the first term can be referred to as the loss term and the second term can be referred to as the regularization term. For example, given an ensemble of two classifiers where i∈2, the classifiers' loss terms can be expressed as L(F1xk+b1−yk) and L(F2xk+b2−yk), and the regularization term can be expressed as λD(F2; F1). Both the first classifier and the second classifier can take into account the regularization term which can enforce orthogonality among its parameters (e.g., the parameters of the first classifier) and the parameters of the other classifiers in the ensemble (e.g., the parameters of the second classifier). This example of two classifiers can be extended as shown in Equation (4) to three or more classifiers of an odd number of classifiers. That is, the number of classifiers N in the ensemble can be three or more and of an odd number.

Thus, the diversity optimization formulation can ensure that affine classifiers become closer and closer to pairwise orthogonal as the value of the regularization parameter λ is increased. This can ensure that when an adversarial perturbation moves a sample across a decision boundary of any given classifier, that movement is parallel to the decision boundary of the other classifiers, which can prevent misclassification by the other classifiers.

Diversity Optimization in Neural Network Kernels

As described herein, a neural network can learn an embedding from an input space into an output space, in which classification can be accurately performed. The layers of the neural network (except for the final fully connected classification layer) can compute an embedding (or an intermediate representation) of the input image, and the final classification layer can apply the classification function on the embedding (or the intermediate representation). The system can use a classification function of various types, such as a softmax classification with a cross-entropy loss function. Iterations of the softmax classifier can converge to a maximum margin classifier with the hinge loss function, which can be a bank of Support Vector Machines (SVMs). Specifically, the classification layer can include a group of one-versus-all binary linear SVM classifiers. As a result, the system or network can map the a class input sample into a different subspace, such that each class is nearly linearly separable from the other classes. The described embodiments can use linear SVMs or more generally can use affine classifiers.

Thus, the neural network can be viewed as a combination of a learned embedding mapping and a learned classifier. Assume a classification problem with C classes and a goal of an ensemble of N models. Beginning with a CNN with L layers with no distinction between a convolutional layer and a fully-connected network layer, the neural network can be viewed as an embedding mapping implemented on the first L−1 layers, followed by a classification layer composed of C one-versus-all linear or affine classifiers, as depicted below in relation toFIG.2.

FIG.2presents an exemplary convolutional neural network (CNN)200as a learned kernel and a classification layer which uses an affine classifier, in accordance with an embodiment of the present application. CNN200can include convolutional layers204and intermediate fully connected layers and activation functions206. CNN200can also include a classification layer with C one-versus-all linear SVMs208. CNN200can take as input an input image210and can output an intermediate representation/embedding of input image212from layers206as input to classification layer208. Classification layer208, based on its C one-versus-all linear SVMS, can subsequently output a result as an output label214.

In general, the described embodiments can include a two-step approach to obtain diversity. First, the system can compute a diverse combination of N affine classifiers based on the diversity optimization technique described above, i.e., the system can define a diversity metric based on pairwise angles between decision boundaries of affine classifiers and include the diversity metric as a regularization term in a loss function optimization for designing the affine classifiers. Second, the system can learn the network embedding which is consistent with the N diversity-optimized linear classifiers, i.e., the system can train one or more neural networks such that parameters of the neural networks are consistent with parameters of the affine classifiers. The N diversity-optimized affine classifiers can comprise an ensemble of classifiers which are each learned using, in a customized classification layer, the loss function described above in Equation (4).

Creating Diverse Classifiers Based on a Single Neural Network or Multiple Diverse Neural Networks

As depicted above in relation toFIG.2, the output of the penultimate layer of the CNN can be an intermediate representation or embedding of the input image, i.e., a mapping of the input image via a neural network kernel. While this neural network kernel may not always be convenient to represent in closed form, it is still a learned neural network kernel. Given a trained neural network with a max margin classification layer with hinge loss, the neural network kernel can map the input image into an embedding space in which an affine classifier can be used with reasonable accuracy.

In the described embodiments, the embeddings of all the input images can be classified by the linear or affine classifier. The system can subsequently apply the diversity optimization formulation described above. Assume the embeddings are training data for an ensemble of affine classifiers. Using the diversity optimization formulation, the system can train N affine classifiers, where the classifier loss function can be regularized by the diversity metric. The combination of the first L−1 layers of the CNN and the diverse classifier layers are depicted below in relation toFIG.3.

FIG.3presents an exemplary CNN300as a learned kernel and classification layers which use multiple affine classifiers designed for the learned kernel, which incorporates diversity into the classification layer, in accordance with an embodiment of the present application. CNN300can include convolutional layers302and intermediate layers and activation functions304. CNN300can also include multiple classification layers (e.g., a classification layer1306and a classification layer N308). CNN300can take as input an input image312and can output an intermediate representation/embedding of input image314and316from layers304. Output314and316can be an input to, respectively, each of classification layers1306and N308. The intermediate representation/embedding of the input image (indicated as output314and316) is generated from the single neural network kernel322. Each classification layer can output its own result, e.g., classification layer1306can generate an output label1318and classification layer N308can generate an output label N320. These multiple output labels can be input into an ensemble decision rule module310, which can generate a decision of an output label322.

CNN300illustrates how diversity is introduced in the classification layer (via diversity optimization324), using only one fixed learned neural network kernel322(which includes layers302and304). Because all the diverse affine classifiers of CNN300are based on the same embeddings (314and316), all the diverse affine classifiers are preceded by the same neural network kernel (322) and thus operate on the same embedding (314and316) of the input image. By itself, this step may not provide optimally sufficient diversity because the adversarial manipulation is applied not to the embeddings of the input images, but rather to the input images themselves. As a result, while the adversarial embedding does move in a direction perpendicular to the decision boundary of the affine classifier being attacked, creating such an embedding may involve a more complex manipulation of the underlying input image. This complex manipulation can occur in the first L−1 layers of the network.

Thus, it may be desirable to introduce diversity into the network in an end-to-end manner, i.e., both in the affine classification layer as well as in the embedding mapping, as described below in relation toFIG.4. To introduce diversity into the embedding, the system must learn the embedding for each diverse classifier, which is equivalent to learning the weights of the neural network which produces those embeddings. WhileFIGS.3and4depict an even number of affine classifiers (as seen in classifications layers306and308), the number of affine classifiers can be three or more and of an odd number, which can allow for a definite classification decision rule, e.g., using an ensemble decision rule.

FIG.4presents an exemplary CNN400as multiple learned kernels and corresponding classification layers, which incorporates diversity into both the classification layers and the learned kernels, in accordance with an embodiment of the present application. CNN400can include multiple “branches” each with a convolutional layers set, intermediate layers, and a classification layer. For example, CNN400can include convolutional layers set1402, intermediate layers and activation functions404, and a corresponding classification layer1406, which can take as input an intermediate representation of input image424(based on input image422) and generate an output label1426. Similarly, CNN400can also include convolutional layers set N412, intermediate layers and activation functions414, and a corresponding classification layer N416, which can take as input an intermediate representation of input image428(also based on input image422) and generate an output label N430. As in CNN300, the multiple output labels426and430can be input into an ensemble decision rule module420, which can generate a decision or predicted outcome of an output label440.

CNN400illustrates how diversity is introduced in both the classification layer (via modified diversity optimization444) and the multiple diverse neural network kernels442(which includes the multiple branches of layers402/404and412/414). The diverse affine classifiers of CNN400are based on different embeddings (e.g.,424and428), which are each preceded by a respective and different neural network kernel (e.g.,402/404and412/414) and thus generate different embedding mappings (e.g.,424and428) of input image422.

Creating Diverse Neural Network Kernels Consistent with the Diversity-Optimized Affine Classifiers

The system can extend the diversity optimization further backward into the neural network kernel, based on two different approaches. The first approach can involve learning the neural network parameters for a “multi-head” architecture in which the diverse neural networks share a common trunk of≤L−1 layers, as depicted below in relation toFIG.5and above in relation toFIG.3. The second approach can involve learning the neural network parameters for a “multi-model” architecture in which the diverse networks do not share a common trunk, i.e.,=0, as depicted below in relation toFIG.6and above in relation toFIG.4. While these two approaches involve different architectures, both approaches are similar in the way in which the neural network parameters are learned.

Assume an ensemble of N neural network-based classifiers (e.g., CNNs) and C classes. Further assume that each classifier has L layers and the same dimensions for each layer. In a first step, the system can set up the diversity optimization problem such that the affine classifiers are designed with a pairwise diversity regularization term added to the conventional loss function of each classifier, as described above. Thus, for N classifiers, there can be (N2) diversity regularization terms.

In a second step, during each step of training the classifier, the system can obtain the classifier weights that minimize the regularized loss function.

In a third step, during each step of training the classifier, the system can also obtain the neural network parameters, i.e., the weights in each layer, via standard back propagation. This third step can be the same for the multi-head and the multi-model ensembling approaches. In the multi-head approach (as described below in relation toFIG.5), this third step can result in the neural network parameters in the common backbone network that are consistent with each of the diverse classifiers. In the multi-model approach (as described below in relation toFIG.6), this third step can result, in general, in different parameters in each of the L neural networks that are consistent with the respective N affine classifiers.

FIG.5presents an exemplary CNN500which uses a multi-head approach with a learned kernel and classification layers which use multiple affine classifiers designed for the learned kernel, which incorporates diversity into the classification layer, in accordance with an embodiment of the present application. CNN500can include a set of backbone layers1530to M532, where M can be a number of layers L minus 1, which are shared by multiple classifiers. CNN500can take as input an input image550, which, after being processed by the shared set of backbone layers, is fed into multiple classification or classifier layers as the same intermediate representation or embedding (indicated by552,554, and556).

That is, the system can generate a different classification or classifier layer for each component classifier with as many as L−1 layers shared amongst the classifiers. The different classification or classifier layers can include, e.g.: a classifier layer1502, which includes an affine classifier1504and a loss1function506; a classifier layer2512, which includes an affine classifier2514and a loss2function516; and a classifier layer3522, which includes an affine classifier3524and a loss3function526. The system can learn the weights of all the layers in conjunction with the diversity optimization method described above. The class labels or results generated from the mutually exclusive orthogonal classifiers (504,514, and524) are input into an ensemble module540, which generates a decision558based on, e.g., an ensemble decision rule.

FIG.6presents an exemplary CNN600which uses a multi-model approach with multiple learned kernels and corresponding classification layers, which incorporates diversity into both the classification layers and the learned kernels, in accordance with an embodiment of the present application. CNN600can include multiple sets of backbone layers, such as: backbone1layer1630to backbone1layer M632; backbone2layer1634to backbone2layer M636; and backbone3layer1638to backbone3layer M640, where M can be a number of layers L minus 1, and where each set of layers is not shared by multiple classifiers. CNN600can take as input an input image660, which can be processed by each set of backbone layers, where each respective image can be subsequently fed into a respective one of multiple classification or classifier layers as different intermediate representation or embedding (indicated by662,664, and666). That is, the system can generate a different classification or classifier layer for each component classifier where each set of max L−1 layers is not shared amongst the classifiers. The different classification or classifier layers can include, e.g.: a classifier layer1602, which includes an affine classifier1604and a loss1function606; a classifier layer2612, which includes an affine classifier2614and a loss2function616; and a classifier layer3622, which includes an affine classifier3624and a loss3function626. Similar to CNN500ofFIG.5, CNN600can learn the weights of all the layers in conjunction with the diversity optimization method described above. The labels or results generated from the mutually exclusive orthogonal classifiers604,614, and624are input into an ensemble module650, which generates a decision668based on an ensemble decision rule.

Method for Facilitating Construction of an Ensemble of Neural Network-Based Classifiers that Optimize a Diversity Metric

FIG.7presents a flowchart700illustrating a method for facilitating construction of an ensemble of neural network-based classifiers that optimize a diversity metric, in accordance with an embodiment of the present application. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers (operation702). The decision boundaries of the plurality of affine classifier can be pairwise within a predetermined threshold of being mutually orthogonal (i.e., close to or nearly orthogonal). The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair (i.e., each pair) of mutually orthogonal affine classifiers of the three or more affine classifiers (operation704). The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric (operation706).

In one embodiment, one backbone layer or one neural network kernel can output an intermediate representation to the plurality of affine classifiers, as described in the multi-head approach ofFIGS.3and5. In another embodiment, a plurality of backbone layers or a plurality of neural network kernels can each output an intermediate representation to the plurality of affine classifiers, as described in the multi-model approach ofFIGS.4and6.

The system obtains an ensemble of neural network-based classifiers which optimize the diversity metric (operation708). The system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric (operation710). The predicted outcome can also be based on, e.g., an ensemble decision rule.

Concrete Results

The following examples provide the results of implementing the diversity optimization process described above (in the sections titled “Creating Diverse Classifiers Based on a Single Neural Network or Multiple Diverse Neural Networks” and “Creating Diverse Neural Network Kernels Consistent with the Diversity-Optimized Affine Classifiers”), i.e., how constructing an ensemble of neural network-based classifiers that optimize a diversity metric can provide robustness against adversarial examples. The results can include diverse CNN-based models for classifying images from the MNIST and CIFAR-10 datasets.

The configuration parameters for the experiment results can include the following: adversarial attacks based on a fast gradient method (FGM) and a projected gradient descent (PGD); a perturbation space with a value of∞ball; a classifier type of CNNs with affine classification layer(s); a regularization parameter (λ) of 0.1, and a number of classifiers in each ensemble set to 1, 3, 5, and 7. A first dataset from MNIST can include: 60,000 training images; 10,000 test images; a perturbation budget (∈) of {0.05, 0.1, 0.2, 0.3}; and a number of training epochs set to 20. A second dataset from CIFAR-10 can include: 50,000 training images; 10,000 test images; a perturbation budget (∈) of {0.05, 0.1}; and a number of training epochs set to 20 for multi-class classifier ensembles and 40 for one-versus-all (binary) affine classifier ensemble.

FIG.8depicts a table800indicating exemplary results for ensemble robustness based on the MNIST dataset, in accordance with an embodiment of the present application. Table800indicates the effect of blackbox attacks on a 2-layer CNN at various perturbation budgets on the first dataset (from MNIST). Table800depicts the accuracy for single model and an ensemble size of seven models (i.e., “7-ensembles”) for both a multiclass classification function as well as a bank of one-versus-all affine classification functions in the final layer of the CNN.

Table800presents results with diversity optimization in comparison with those for randomly initialized 2-layer CNNs (no diversity). This scheme can provide a more robust ensemble benchmark scheme compared to training an initial CNN and perturbing that trained CNN to generate an ensemble. A column810indicates results from using only a single classifier model. Columns812,814, and816show results from using, respectively, the multi-head approach (multiclass), the multi-model approach (multiclass), and the multi-model approach (one-versus all affine classifier). These three columns (indicated by a no diversity802label) do not use the diversity optimization described herein. The next three columns818,820, and822show results from using, respectively, the multi-head diversity approach (multiclass), the multi-model diversity approach (multiclass), and the multi-model diversity approach (one-versus-all). These three columns (indicated by a diversity804label) do use the diversity optimization described herein.

The results in table800are indicated in rows. A row830provides the results of each approach on clean data, while the results in the other rows provides pairs of results for FGM and PGD adversarial attacks based on different values of ∈. For example: rows832and834indicate results from FGM and PGD attacks when ∈=0.05; rows836and838indicate results from FGM and PGD attacks when ∈=0.1; rows840and842indicate results from FGM and PGD attacks when ∈=0.2; and rows844and846indicate results from FGM and PGD attacks when ∈=0.3.

Recall that in a traditional neural network, the system can generate a class label which indicates that a testing data object belongs to one of, e.g., C classes. In contrast, by using the one-versus-all affine classifiers in the described embodiments, the system can generate a label which is a binary value of each of the 1 to C classes, e.g.: whether the testing data object belongs to class 1 or does not belong to class 1; whether the testing data object belongs to class 2 or does not belong to class 2; etc. Using the results from the one-versus-all approach, the system can more accurately conclude whether a certain testing data object belongs to a certain class or not, e.g., by using the N binary classifiers where N=7*C in this example.

In table800, it can be seen, e.g., in row846, for the non-diversity approaches802, that the multi-model one-versus-all approach column816yields a more accurate result (of 51.11, indicated by a result854) than either the multi-head (multiclass) column812(of 22.37, indicated by a result850) or the multi-model (multiclass) column814(of 33.79, indicated by a result852). Similarly, it can be seen in row846, for the diversity approaches804, that the multi-model one-versus-all approach column822yields a more accurate result (of 52.89, as indicated by a result860) than either the multi-head diversity (multiclass) column818(of 19.79, indicated by a result856) or the multi-model diversity (multiclass) column820(of 22.87, indicated by a result858).

Furthermore, the one-versus-all approaches which use diversity may also perform better than the approaches which do not use diversity, e.g., 52.89 (result860) of the multi-model with diversity one-versus-all as compared to 51.11 (result854) of the multi-model one-versus-all without diversity.

The baseline ensembles and the diversity-optimized ensembles can be compared in another way, by examining the angles between the decision boundaries of the corresponding affine classifiers in the ensemble, e.g., for the one-versus-all classifiers (taken pairwise) corresponding to a given class.

FIG.9depicts a table900indicating exemplary results for pairwise angles between decision boundaries of the corresponding affine classifiers in various ensembles, in accordance with an embodiment of the present application. Table900can include: a row930for the MNIST dataset using a CNN 2-layer model as a baseline; a row932for the MNIST dataset using a CNN 2-layer model with diversity optimization where λ=0.1; a row934for the CIFAR dataset using a CNN 6-layer model as baseline; and a row936for the CIFAR dataset using a CNN 6-layer model with diversity optimization where λ=0.1. For each of these rows930-936, table900can also include columns with the following information: a mean angle912; a minimum angle914; a maximum angle916; a standard deviation of angle918; and a median angle920. The data in table900covers the statistics of the pairwise angles between the one-versus-all binary classifiers in the final classification layer of each respective model, i.e., for the baseline scheme (rows930and934) and the diversity optimized scheme (rows932and936).

The diversity-optimized schemes can result in pairwise angles that are tightly distributed around a mean of 90 degrees, with a small standard deviation around the mean (i.e., 0.02 and 0.05 for the diversity-optimized schemes of, respectively, MNIST row932and CIFAR row936). In contrast, in the baseline schemes, the standard deviation around the mean as higher (i.e., 4.37 and 4.78 for the baseline and non-diversity optimized schemes of, respectively, MNIST row930and CIFAR row934), since the classifier design step does not explicitly enforce orthogonality.

Even in the baseline schemes which do not use the diversity optimization, but instead involve training individual classifiers starting from a random initialization of network weights, the mean for the pairwise angles can be close to 90 degrees (e.g., 90.59 and 89.17 for the baselines and non-diversity optimized schemes of, respectively, MNIST row930and CIFAR row934), even though, as noted above, the standard deviation is larger. Table900indicates that the randomly initialized classifiers may approximately achieve a similar orthogonality objective, as their performance in terms of adversarial robustness may be close to that of the diversity-optimized schemes, given the selected baseline as a stronger heuristic. Selecting a weaker heuristic, e.g., starting with a single trained classifier and subsequently computing the weights of the other classifiers as small random perturbations around the existing classifier weights, may not achieve the orthogonality condition, which would result in a lower robustness for the overall ensemble of classifiers.

Integration into a Practical Application and Improvements to Technologies

The embodiments described herein can be integrated into a practical application for, and can result in an improvement in, several technologies and technical fields, including but not limited to: artificial intelligence; machine learning and analytics; convolutional neural networks; data mining (including of a significant volume of data); data classification; and defense against adversarial attacks and adversarial examples, including perturbation-bounded evasion attacks.

Users of the system described herein can include an individual with a smartphone, a mobile device, or a computing terminal (e.g., user112of environment100ofFIG.1). Users of the system can also include any client in a machine learning or an artificial intelligence setting, where increasing the effectiveness of classifiers against adversarial attacks using the described diversity optimization can result in an increase in the accuracy of classification of test data. For example, the tables described above in relation toFIGS.8and9support the technological improvements of the described embodiments because the tables indicate results which the orthogonality condition can provide a more robust ensemble of classifiers against adversarial perturbations.

Furthermore, the described embodiments provide an improvement to technology because the system allows a user to interact with the created ensembles and resulting classifications (as shown in the exemplary information displayed in display114ofFIG.1). The system can result in more efficiently training the machine learning models against adversarial examples, which can result both in an improved model and a more efficient overall user experience.

Exemplary Computer and Communication System

FIG.10presents an exemplary computer and communication system1002which facilitates data classification, in accordance with an embodiment of the present application. Computer system1002includes a processor1004, a memory1006, and a storage device1008. Memory1006can include a volatile memory (e.g., RAM) that serves as a managed memory, and can be used to store one or more memory pools. Furthermore, computer system1002can be coupled to a display device1010, a keyboard1012, and a pointing device1014. Storage device1008can store an operating system1016, a content-processing system1018, and data1034.

Content-processing system1018can include instructions, which when executed by computer system1002, can cause computer system1002to perform methods and/or processes described in this disclosure. Specifically, content-processing system1018may include instructions for sending and/or receiving data packets to/from other network nodes across a computer network (communication module1020). A data packet can include data, a request, a command, a model, a classifier, training data, test data, a result, and an outcome.

Content-processing system1018can further include instructions for determining training data or testing data (data-determining module1022). Content-processing system1018can further include instructions for defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers (diversity metric-defining module1024). Content-processing system1018can include instructions for including the diversity metric as a regularization term in a loss function optimization for designing the a pair of mutually orthogonal affine classifiers of the three or more affine classifiers (ensemble-constructing module1026). Content-processing system1018can additionally include instructions for training one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric (neural network-training module1028).

Content-processing system1018can also include instructions for predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric (outcome-predicting module1030). Content-processing system1018can include instructions for displaying information on display1010or on a display associated with a user or computing device which is in communication with computer system1002(display-managing module1032). Exemplary displayed information is described above in relation to, e.g., display114and operations148,149,174, and179ofFIG.1. Content-processing system1018can also include instructions for allowing a user to interact with the displayed information (display-managing module1032).

Data1034can include any data that is required as input or that is generated as output by the methods and/or processes described in this disclosure. Specifically, data1034can store at least: data; a set of data; a training data object; training data; test or testing data; a testing data object; an image; a perturbed image or data object; data which has been modified based on a perturbation-bounded evasion attack; a parameter; a regularization term; a loss function; an optimization of a loss function; a request; a command; a classifier; an affine classifier; a multi-class classification layer; a one-versus-all classification layer; an ensemble of classifiers; a diversity metric; a decision boundary; an angle; a backbone layer; a kernel; a neural network kernel; a neural network; a machine learning model; a CNN; an intermediate representation; an input; an output; a label; an outcome; a result; a predicted outcome or result; a rule; an ensemble decision rule; a classification; an accuracy of a classification; a size of an ensemble; a type of attack; a type of data; a type of approach; a type of classifier; a result from an individual classifier; an overall ensemble result; and a decision based on a rule.

Furthermore, the methods and processes described above can be included in hardware modules or apparatus. The hardware modules or apparatus can include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute a particular software module or a piece of code at a particular time, and other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them.