Artificial intelligence system for classification of data based on contrastive learning

An artificial intelligence (AI) system that includes a processor configured to execute modules of the AI system. The modules comprise a feature extractor, an adversarial noise generator, a compressor and a classifier. The feature extractor is trained to process input data to extract features of the input data for classification of the input data. The adversarial noise generator is trained to generate noise data for distribution of features of the input data such that a misclassification rate of corrupted features that include the extracted features corrupted with the generated noise data is greater than a misclassification rate of the extracted features. The compressor is configured to compress the extracted features. The compressed features are closer to the extracted features than to the corrupted features. The classifier is trained to classify the compressed features.

TECHNICAL FIELD

The present disclosure generally relates to artificial intelligence (AI), and more specifically to an AI system for classification of data based on contrastive learning.

BACKGROUND

In the field of machine learning, various models (e.g., classification models) are used to perform different functions such as object classification, object detection, gesture detection, and the like by analyzing input data. In some cases, these models may generate wrong outputs due to malicious input data. This causes the classification models to misclassify input data. For instance, a small and imperceptible optimized perturbation added to an image data can cause misclassification by the classification models trained to classify images.

In general, machine learning techniques are designed for stationary and benign environments in which training and test data are assumed to be generated from same statistical distribution. However, when these models are implemented in real world applications, presence of the perturbations in the input data may violate statistical assumptions. This shows that input data can be manipulated to exploit specific vulnerabilities of learning algorithms and compromise security of a machine learning system. For example, with the knowledge of an internal architecture of the AI system that performs classification of the input data using a neural network, an adversarial input may be developed to manipulate the AI system and produce illegitimate output. Various machine learning techniques utilize such adversarial input by a contrastive learning approach. The contrastive learning approach provides a learning model for learning distinctiveness between similar or different features of the input data. For example, in contrastive learning, generic representations of input data (such as input images) on an unlabeled dataset is learned and then is fine-tuned with a small dataset of labeled images for a given classification task, where the representations include adversarial inputs. The representations are learned by simultaneously maximizing agreement between different versions or views of the same image and cutting down the difference using contrastive learning. When the parameters of a neural network are updated using this contrastive objective causes representations of corresponding views to “attract” each other, while representations of non-corresponding views “repel” each other. However, the development of the adversarial input for the misclassification by the classification models may be unknown and uncontrolled. In some cases, the classification models may miss to learn useful information in the adversarial inputs.

Accordingly, there is a need of a system to accurately classify input data based on the contrastive learning.

SUMMARY

It is an object of some embodiment to employ adversarial machine learning for data compression tasks. It is another object of some embodiments to perform classification of data compressed with adversarial machine learning, contrastive learning and/or combination thereof.

Specifically, some embodiments are based on the recognition that data analyzed from classification point of view may include information that are useful for classification as well as information that are not useful for the classification. For example, if a classifier is trained to classify input data, such as image data. The image data may include image of a dog or a cat. The classifier uses a few features from the image data to classify the image data. However, the classifier may not use other features of the image data for the classification. The features that are used by the classifier are referred herein as useful features, while those features are not used in the classification are referred as useless features.

Some embodiments are based on the understanding that features extracted from the input data may be compressed. The extracted features are compressed to include the useful features without sacrificing accuracy of the classification. In some example embodiments, the features may be extracted by a pre-trained neural network for the classification. However, the neural network may not able to identify the useful features and the useless features in the extracted features. To this end, some embodiments are based on the realization that principles of adversarial machine learning and contrastive learning may be used to identify the useful features and the useless features in the extracted features. Specifically, the adversarial machine learning may be used to determine the useless features.

Some embodiments are based on the realization that the adversarial machine learning is used to corrupt the extracted features. The extracted features are corrupted such that the classifier misclassifies the extracted features. For example, the extracted features are from an image including a dog. The classifier may classify the extracted features belonging to the image including the dog. When the corrupted features are provided to the classifier, the classifier may misclassify the extracted features to an image different from the image that includes the dog. The objective of the adversarial machine learning is to corrupt the features such that the same classifier misclassifies the features. For instance, the classifier misclassifies the features of the image of the dog to an image of a cat.

To achieve such an objective, the adversarial machine learning may corrupt a part of the extracted features used for the classification. To that end, the useful features in the extracted features may be corrupted based on the adversarial machine learning. The useless features in the extracted features may not be corrupted as it may be disadvantageous to preserve the integrity of the corrupted features. Hence, in the corrupted features, the useful features are modified, while the useless features may be preserved.

The useful features may be present in original extracted features, while useless data may be present in both the original extracted features as well as the corrupted features. Based on this understanding, in some embodiments, the extracted features are compressed such that the compressed features are close to original features of the input data and the corrupted features are distant from the extracted features.

In some embodiments, the compressed features may be determined by solving a multi-objective optimization. The multi-objective optimization optimizes a cost function. The cost function reduces a distance between the compressed features and the extracted features. The cost function also increases a distance between the compressed features and the corrupted features.

In some embodiments, the extracted features are corrupted using noise data generated by an adversarial noise generator. The adversarial noise generator may be trained based on a Generative Adversarial Network (GAN). The GAN includes a generator and a discriminator. The adversarial noise generator generates the noise data from statistical distribution of the extracted features. The generated noise data provides the classifier with a success rate of classification of the corrupted features that is less than a success rate of classification of the extracted features. The distribution of the corrupted features is tested by the discriminator. To that end, in some embodiments, the generator may replicate the extracted features and generate corrupted features for each replicated feature of the extracted features. The corrupted features are generated by combining the noise data with each replicated feature.

In an example embodiment, distribution of the noise data may be uncorrelated to the input data. Accordingly, it is an objective of the present disclosure that the distribution of corrupted features that include the extracted features with the noise data is closer to the distribution of the extracted features. To that end, the compression of the extracted features may be trained to minimize a loss function including a combination of failures of the generated distribution of the extracted features and failures of the classification of input data.

In some embodiments, the compressed features may be projected into a subspace for generating a subspace representation of a sequence of temporally connected data of the input data. To that end, the sequence of temporally connected data may be derived from a temporal order of the input data. In some example embodiments, the temporal order may be obtained using an order-constrained Principal Component Analysis (PCA) technique. In some cases, the subspace representation may diverge from the input data. To that end, a distortion penalty may be applied to the subspace representation to prevent from diverging from the input data.

Accordingly, one embodiment discloses an artificial intelligence (AI) system for classification of data compressed with adversarial machine learning. The AI system includes a processor configured to execute modules of the AI system. The modules comprises a feature extractor, an adversarial noise generator, a compressor and a classifier. The feature extractor is trained to process input data to extract features of the input data for classification of the input data. The adversarial noise generator is trained to generate noise data for distribution of features of the input data. The noise data are generated in such a way that corrupted features that include the extracted features corrupted with the generated noise data have a misclassification rate that is greater than a misclassification rate of the extracted features. The compressor is configured to compress the extracted features. The compressed features are closer to the extracted features than to the corrupted features. The classifier is trained to classify the compressed features.

Another embodiment discloses a computer-implemented method for classification of data compressed with adversarial machine learning. The method includes extracting features of input data for classification of the input data. The method includes generating noise data for a distribution of features of the input data such that a misclassification rate of corrupted features that include the extracted features corrupted with the generated noise data is greater than the misclassification rate of the extracted features. The method includes compressing the extracted features such that the compressed features are closer to the extracted features than to the corrupted features. The method further includes classifying the compressed features.

Further features and advantages will become more readily apparent from the following detailed description when taken in conjunction with the accompanying drawings.

DETAILED DESCRIPTION

As used in this specification and claims, the terms “for example,” “for instance,” and “such as,” and the verbs “comprising,” “having,” “including,” and their other verb forms, when used in conjunction with a listing of one or more components or other items, are each to be construed as open ended, meaning that the listing is not to be considered as excluding other, additional components or items. The term “based on” means at least partially based on. Further, it is to be understood that the phraseology and terminology employed herein are for the purpose of the description and should not be regarded as limiting. Any heading utilized within this description is for convenience only and has no legal or limiting effect.

Overview

The proposed Artificial Intelligence (AI) system enables learning of a representation for compressing features of an input data. The representation may be a low-dimensional subspace representation respecting the sequential/temporal order of input data. It is an objective of the AI system to capture implicit informative cues, such as spatio-temporal information of input data for learning the representation. The spatio-temporal information of the input data may be captured through extraction of features from the input data for classification of the input data. To that end, the feature extraction may be maximized for such information cues from the input data. In some embodiments, the feature extraction may be maximized using contrastive representation learning.

FIG.1Ashows a schematic diagram100depicting classification of input data104using an Artificial Intelligence (AI) system102, according to some embodiments of the present disclosure. In an example embodiment, the input data104may include a sequence of frames104A,104B and104C of a video. The sequence of frames104A,104B and104C may be associated with a temporal sequence. The input data104is provided to the AI system102for classification of the input data104. In an illustrative example scenario, the sequence of frames104A,104B, and104C may include a human action. The AI system102identifies the human action that may be provided as a classified action106. As shown inFIG.1A, the human action includes clapping action in the sequence of frames104A,104B, and104C. Thus, the classified action106may be identified as the clapping action. In some cases, the classified action106may be identified from one of the sequence of frames. However, classification of the human action based on one frame of the input data104may jeopardize the classification. In some other cases, using each of the sequence of frames104A,104B, and104C for the classification may not be feasible and efficient, due to limited memory and processing resources.

To that end, some embodiments are based on the realization, that in some cases, the input data104provided for the classification may be compressed, which is shown inFIG.1B.

FIG.1Bshows a schematic diagram108depicting extraction of compressed features from the input data for the classification, according to some embodiments of the present disclosure. To that end, the input data104may be compressed such that data complexity is reduced, while maintaining accuracy of subsequent classification. For instance, the AI system102extracts features110(X=x1, xt, xn) from each of the sequence of frames104. The AI system102compresses the extracted features110into compressed features112. The compressed features112undergo a classification114for providing the classified action106, with less data complexity.

The compressed features112enable a classifier to learn similarity and difference between the extracted features and other features, such as corrupted features of the input data104. Such learning implies a contrastive learning, which is explained next with reference toFIG.1C.

FIG.1Cshows a contrastive learning representation116of the compressed features, according to some embodiments of the present disclosure. The extracted features110(X) are represented using the compressed features112that are far from being similar to other features, such as corrupted features118(Y). Such representation implies that the compressed features112(U) may push away information in the extracted features110(X) that is not present in the corrupted features118(Y). The compressed features112(U) are contrasting to the corrupted features118(Y), which represents the contrastive learning of features for the classification of the input data104.

To that end, it is an objective of the present disclosure to generate compressed features that are far from the corrupted features118(Y). The corrupted features118(Y) may be generated based on noise data, such as adversarial noise data lying within feature space, as shown in a graphical plot120ofFIG.1C. For each of the extracted features110(X), adversarial noise data for distribution of the extracted features110(X). The corrupted features118(Y) include the extracted features110(X) and the noise data such that the corrupted features118(Y) are similar to the extracted features110(X). However, when the corrupted features118(Y) are provided as input to a classifier, the classifier misclassifies the corrupted features118(Y).

In some embodiments, the corrupted features118(Y) are generated by the AI system102, which is further described in description of subsequent figures in the present disclosure. In some embodiments, the AI system102may comprise modules for the classification of the input data104, which is explained next with reference toFIG.2A.

FIG.2Ashows a schematic diagram depicting modules of the AI system102, according to some embodiments of the present disclosure. The AI system102comprises a processor200that is configured to execute modules of the AI system102stored in a memory202. The modules may include a feature extractor204, an adversarial noise generator206, a compressor208and a classifier210. The feature extractor204may be trained to process the input data104to extract features of the input data104for the classification of objects included in the input data104. The adversarial noise generator206is trained to generate noise data for distribution of corrupted features of the input data104. The extracted features are corrupted by the generated noise data to provide corrupted features. When the corrupted features and the extracted features undergo a classification, the corrupted features have a misclassification rate greater than a misclassification rate of the extracted features. In some example embodiments, the noise data may correspond to Gaussian noise. Further, the compressor208is configured to compress the extracted features. The compressed features are closer to the extracted features than to the corrupted features. The classifier210is configured to classify the compressed features.

The generation of the noise data by the adversarial noise generator206is further explained next with reference toFIG.2B.

FIG.2Bshows a schematic diagram212depicting generation of the noise data for the classification of the input data104by the AI system102, according to some embodiments of the present disclosure. The sequence of frames104A,104B, and104C is provided to the feature extractor204. The feature extractor204processes the sequence of frames104A,104B, and104C to extract features214(e.g., the extracted features110(X)) for the classification. In some example embodiments, the feature extractor204may encode the sequence of frames104A,104B,104C into a set of features vectors such as (X=x1, xt, . . . , xn). The sequence of frames104A,104B, and104C may be encoded using a pre-trained neural network. Additionally, or alternatively, the feature extractor204may maximize extraction of the features214for capturing informative cues, such as spatio-temporal cues of the sequence of frames104A,104B, and104C.

To that end, the processor200may determine a value of a mean of the extracted features214(X) and/or a normal distribution of the extracted features214(N(X, σ2I) or μx) around the mean. Further, the mean and the normal distribution of the extracted features214are provided to the adversarial noise generator206. The adversarial noise generator206is trained to generate noise data (z) for a distribution of features214. In an example embodiment, the adversarial noise generator206generates distribution of corrupted features216(vy) closer to the distribution of the extracted features214(μx). The corrupted features216(e.g., the corrupted features118(Y)) include the extracted features214that are corrupted with the generated noise data (z). The corrupted features216may be represented by (Y=y1, yt, . . . , yn). These generated noise data may not impact useful features of the extracted features214(X).

The distribution of the corrupted features216may correspond to the normal distribution of the extracted features214. In some example embodiments, the corrupted features216may comprise a set of adversarial noise samples, such as y=σ(x+{circumflex over (x)}). The corrupted features216may be defined via an implicit function of the adversarial noise generator208, i.e., y=gθ(z), where θ defines parameters to be learned and the distribution of the noise data z˜N(X, σ2I). The meanXof the extracted features214defines average of the extracted features214in respective sequence, i.e.

When the corrupted features216are provided to the classifier210, a misclassification rate of the corrupted features216is greater than a misclassification rate of the extracted features214.

In some embodiments, the classifier210may be associated with a ground truth class label that includes a set of labels for different actions. For instance, the classifier210may classify the extracted features214as “clap” action220, based on the set of labels. In some example embodiments, the classifier210may misclassify the corrupted features216as the noise data (z) in the corrupted features216and may not include informative cues for the classification. This indicates the corrupted features216are different and distant from the extracted features214. For instance, the classifier210may classify the corrupted features216as “not clap” action222.

Further, the extracted features214are compressed by the compressor208to generate compressed features220. In an example embodiment, the compressed features220may represent a summarized representation from the extracted features214. The extracted features214may be of an arbitrary length as length of the input data104may be arbitrary. However, length of the summarized representation may be of fixed length. For instance, the extracted features214may have 1000 elements each and have a length of 100 vectors. These 100 vectors may be compressed to 10 vectors, each with 1000 elements. Thus, the compressor208may generate a summarized representation of 10 vectors each with 1000 elements from the 100 vectors of the extracted features214, each with 1000 elements.

In some embodiments, the compressor210also uses the corrupted features216for the compression of the extracted features214. The compressed features220(e.g., the compressed features112(U)) may be closer to the extracted features214than to the corrupted features216, i.e. the compressed features220are similar to the extracted features214. The compressed features218are provided to the classifier210, where the classifier210classifies the compressed features218in an efficient manner. In some embodiments, the compressed features218are determined based on a multi-objective optimization, which is further explained with reference toFIG.2A.

FIG.2Cshows a processing pipeline224of the AI system102, according to some example embodiments of the present disclosure. Initially, the input data104is provided to the feature extractor204.

At step226, feature extraction from the input data104is performed by the feature extractor204. The feature extractor204extracts the features214. In some example embodiments, the feature214may be extracted by encoding the sequence of frames104A,104B, and104C into a set of features vectors, using a pre-trained neural network. The extraction of features214may also capture informative cues, such as spatio-temporal cues of the sequence of frames104A,104B, and104C. The spatio-temporal cues may be captured based on a mean value and a normal distribution of the extracted features214.

At step228, the noise data (z) is generated from the input data104by the adversarial noise generator206. The adversarial noise generator206also receives the mean and the normal distribution of the extracted features214for the noise data generation. The extracted features214are corrupted with the noise data to generate the corrupted features216. The corrupted features216are distributed according to a distribution of the extracted features214at step230.

At step232, the extracted features214are compressed by the compressor208to generate the compressed features218. In some embodiments, the compressor208is configured to determine the compressed features218by solving a multi-objective optimization. At step234, the compressor208solves the multi-objective optimization that optimizes a cost function. The cost function reduces a distance between the compressed features218and the extracted features214. In the cost function optimization, the compressed features218are coupled with the extracted features214. The cost function also increases a distance between the compressed features218and the corrupted features216.

In an example embodiment, the distance between the compressed features218and the corrupted features216may correspond to the Wasserstein distance (optimal transport between two distributions). The optimal transport may be denoted by Wc(μ, v), which is a distance between probability measure of the extracted features214(μ) and probability measure of the corrupted features216(v). The probability measures μ and v are supported on Rdwith respect to the cost function c(x, y), where x,y∈Rd. The Wc(μ, v) may be represented as:

Here, Π(u, v) denotes the set of all couplings (e.g. joint probability distributions) with marginal probability measures μ and v.

For the extracted features214, such as a set of features X, let μXbe an empirical distribution. The empirical distribution μXmay be equal or have a uniform probability over the extracted features214, xt∈X, i.e.,

μX=∑t=1n⁢⁢1n⁢δ⁡(xt),
δ(xt) denoting the measure at xt. In a similar manner, empirical distribution of the corrupted features216may be denoted by vY. The empirical distribution vYalso has a uniform probability over the extracted features214. In an example embodiment, best coupling between the extracted features214and the corrupted features216determined via the optimal transport.

At step236, the compressed features218are generated such that the compressed features218are similar to the extracted features214and different from the corrupted features216. At step238, the compressed features218are classified by the classifier210.

In some embodiments, the compressor208may be trained to minimize a loss function. The loss function includes a combination of failures of the generated noise data120and failures of the classification by the classifier210. The loss function may be denoted by:

LD=1N⁢∑i=1N⁢LC⁡(Ui,li)
and Ui=arg minULR(U(Xi)). The loss function LDaggregates error LCin training the classifier210on the representations Uifor each sequence Xiagainst the ground truth label li. The representations (Ui's) are projected in a graphical plot. Further, the Ui's are obtained via optimizing a sequence level representation captured by the loss LR. In a classical feature learning setup, LRfinds a vector U that minimizes, such as, the mean-squared error to the input data104that provides an average feature. In such case, arg min optimization may correspond to an average pooling scheme. In some example embodiments, the losses LCand LRmay be jointly trained in an end-to-end manner.

It is an objective of the present disclosure that the noise data is generated such that the compressed features218are closer to the extracted features214and the corrupted features216are distant from the extracted features214. To that end, in some embodiments, the adversarial noise generator is trained using a Generative Adversarial Network (GAN), which is explained next with reference toFIG.3.

FIG.3shows a block diagram300of the adversarial noise generator206of the AI system102, according to some example embodiments of the present disclosure. The adversarial noise generator206may be trained using a Generative Adversarial Network (GAN)302. The GAN302includes a generator304and a discriminator306. The generator304receives the extracted features214as input and generates the noise data. The discriminator306tests distribution of the corrupted features216. The discriminator306checks if the distribution of the generated corrupted features216corresponds to the statistical distribution of the extracted features214.

In an example embodiment, the generator304is configured to generate the noise data from statistical distribution of the extracted features214. The statistical distribution is derived from the normal distribution determined by the processor200from the extracted features214. Additionally or alternatively, the noise data may be randomly sampled from the normal distribution, such as N(x, σ), where σ is a user-defined standard deviation.

To that end, the generator304may be trained to satisfy the criteria that the classifier210generates a label vector for the compressed features218with highest score for a class and generates a label vector for the corrupted features216with lowest score for the class that received the highest score when using the compressed features218. Such criteria enable the generator304to generate adversarial noise data for any randomly sample noise data from respective distribution of the extracted features214.

Further, the generation of the noise data based on the statistical distribution, provide a success rate of the corrupted features that is less than a success rate of classification of the extracted features214. To that end, in some embodiments, the generator304may be further configured to replicate the extracted features214and generate corrupted features for each replicated feature of the extracted features214by combining the noise data with each replicated feature. To that end, each of the features vectors of the corrupted features may differ due to the noise data. The generation of different types of corrupted features may increase the chance of determining useless features to generate compressed features with the distribution of the extracted features214.

In some example embodiments, temporal order of the input data104may be added to the compressed features218, which is further explained next with reference toFIG.4.

FIG.4shows a block diagram400of the compressor208of the AI system102, according to some example embodiments of the present disclosure. The compressor208performs an optimization402, such as multi-objective optimization that optimizes the cost function. In the cost function optimization, an optimal transport coupling between the extracted features214and the corrupted features218is minimized. In some embodiments, the compressor208is configured to learn a temporal order404of the input data104. The compressor208generates the compressed features218in order of the temporal order404. In some example embodiments, the compressed features218are projected into a subspace to generate a subspace representation410. The subspace representation410is a learning representation with a sequence of temporally connected data of the input data104. In an example embodiment, the sequence of temporally connected data is based on the temporal order404that may be captured using a Principal Component Analysis (PCA). The PCA with ordering constraints captures a sequential order of the input data104.

The subspace representation410maximizes distance between projection of the extracted features214onto the subspace representation410and projection of the corrupted features216. To that end, the subspace representation is prevented from diverging from the extracted features214based on a distortion penalty406. In an example embodiment, the distortion penalty406may include a distortion metric determined from the extracted features214by the compressor208. The distortion metric may act as a regularization preventing the subspace representation410from diverging from the extracted features214. The regularization may be achieved by solving a PCA type reconstruction loss on the subspace representation410. The subspace representation410may provide features for the classification that are filtered from the generated noise data.

In an example embodiment, the subspace representation410(U) may be formulated as:
maxULOT(U):=WC(fU#μX,vY)  (2)

Here, fUis a mapping denoted by fU: Rd→Rd. The fUmay be parametrized for the subspace representation410from the compressed features122. The mapping f may be defined as f=UUTfor orthonormal U∈Rd×k, i.e. UTU=Ikwhere Ikdenote k×k identity matrix and k≤d.

The optimal transport of (1) may be rearranged in an empirical form and combined with the mapping f in (2), for denoting the contrastive representation learning objective as:
maxU∈G(d,k)LOT(U):=infπ∈Π(μX,VY)Σi,jπij∥fU(xi)−yj∥  (3)

In some example embodiments, U∈G(d, k) may correspond to Grass-mann manifold of all k-dimensional subspaces of Rd. Here, G(d, k) denotes quotient space S(d, k)/O(k) of all d×k orthonormal matrices S(d, k) that are invariant to right rotations. Given that loss LOT(U)=LOT(UR) for any k×k orthogonal matrix R, the Grass-mann manifold is selected for the subspace representation learning objective.

The projections of the extracted features214and the corrupted features216are shown inFIG.5.

FIG.5shows a graphical plot500depicting projection of extracted features214and projection of the corrupted features216, according to some example embodiments of the present disclosure. In an example embodiment, the sequence of frames104A,104B, and104C may be represented as a set of N data sequences D={X1, X2, . . . , XN}, where each Xi=<x1i, x2i, . . . , xnii> is a sequence of niordered feature vectors and each xt∈Rd. Further, Xiis assumed to be associated with a ground truth class label li∈L, where L denotes a given set of labels. Also, each x∈X is an independent sample from a data distribution PD(X) conditioned on the mean of the input data104, i.e.Xof the sequence X.

The graphical plot500depicts direction U1of the extracted features214using the subspace representation410and direction U2of the extracted features214using the PCA. Basically, the compressed features218capture direction where data is best distributed in the direction U2. However, with points in the corrupted features216(Y), direction U2deviates from the points in the extracted features214(X), and is different direction that minimizes distance to U1and maximizes the distance to a distribution of the corrupted features216.

The AI system102may be used in real-time or offline applications, such as video recognition, which is described next with reference toFIG.6.

FIG.6Ashows a real-time application600of the AI system102, according to some example embodiments of the present disclosure. In one example embodiment, the real-time application600may include a video recognition application. In the video recognition application, a camera602may capture a video that is provided as input data to the AI system102. The video may include a sequence of frames of an arbitrary length. The AI system102may extract features from each frame of the sequence of frames. The features extracted may correspond to an output of a frame-level deep neural network that is trained on each of the frames against their respective video label stored in a database606. The AI system102may generate variation of the extracted features. The variation of the extracted features may be achieved using noise data generated by the adversarial noise generator206of the AI system102.

Further, the extracted features of variations are compressed to generate a subspace representation for the video recognition. The subspace representation may be invariant to other visual information cues, due to the generated noise data and contrastive nature of the compressed features used to generate the subspace representation. The contrastive learning enables the subspace representation to data variations, which may increase accuracy of the video recognition. The AI system102may provide outcome of the video recognition via the output604.

In another example embodiment, the AI system102may use for an image recognition from a collection of images stored in the database606. For instance, the collection of images includes images of a person. The AI system102may access the images from the database606and extract features from the images. The extracted features are compressed using the compressor208. The compressed features provide a subspace representation for representing an identity of the person for the image recognition. The subspace representation is stored in the database606. Such subspace representation that is close to the extracted features of the image of the person may be used for the image recognition.

In another example embodiment, the AI system102may be used for video retrieval. The AI system102may receive a video clip from the camera602. The video clip may include a sequence of frames of some human actions. In the database606, a training set consisting of a preset of classes for human actions may be stored. However, the AI system102may not find a class that matches to the human actions in the video clip. In such case, the AI system102may generate noise data from features extracted from the video clip. The noise data may be combined with the extracted features to generate corrupted features. The corrupted features may be used as adversaries to contrast against a subspace representation generated by compressing extracted features of the video clip. Such subspace representation may be used for the video recognition. The recognized video may be provided via the output604.

In another example embodiment, the real-time application600may include text document recognition. In the text document recognition, extracted features may be an output of a text embedding model, such as ‘word2vec’ model. The text embedding model may be stored in the database606for the document recognition. The AI system102may be trained to generate noise data such that each word of a document is misclassified to another word. The generated noise data may be combined with features extracted from the document to generate a set of documents variant from the original document. Further, the AI system102generates compressed features for the document by compressing the extracted features. The compressed features are used to generate a subspace representation for the document. The generated subspace representation is compared with from the set of documents. The subspace representation is distant from the set of documents as the set of documents vary from the original document. The subspace representation may be constructed based on grammatical order of the words in the document. Such subspace representation of the document may be used for the document recognition.

In a similar manner, the AI system102may be used audio applications, such as audio recognition, audio clip retrieval, or the like.

FIG.6Bshows a real-time application scenario606of the AI system102, according to some other example embodiments of the present disclosure. In an illustrative example scenario, the real-time application scenario606may correspond to a surveillance system of premises608such as an office building, a college, a residential area, and the like. The surveillance of the premises608is captured via the camera602of the surveillance system, such as a Closed-circuit television (CCTV) camera602as shown inFIG.6B. For instance, in the premises608, an object, such as a human610moves towards a door. The CCTV camera602captures a video of movement of the human610. The captured video may be sent to the AI system102via a network612. Additionally, or alternatively, the captured video may be stored in a cloud-based database, such as the database606. The AI system102may access the stored video from the database606.

Further, the AI system102may extract features from each frame of the video and generate variation of the extracted features using noise data. These varied extracted features are compressed to generate a subspace representation for a video recognition, such as walking action of the human610. In some cases, identity of the human610may also be determined using the subspace representation. In some other cases, a few frames of the video, such as a video clip that includes the walking action may retrieve using the subspace representation. Accordingly, the surveillance of the premises608is performed.

FIG.7shows a method flow700for classification of data compressed with adversarial machine learning, according to some example embodiments of the present disclosure. The method700is performed by the AI system102. At operation702, features of input data, such as the input data104are extracted for classification of the input data104. The features of the input data104may be extracted by encoding the input data104into a set of feature vectors using a pre-trained neural network.

At operation704, corrupted features are generated for the input data104using a noise data distribution. To that end, noise data is generated for a distribution of features216of the input data104such that corrupted features are obtained, where a misclassification rate of the corrupted features216that include extracted features214corrupted with the generated noise data is greater than the misclassification rate of the extracted features214. The noise data is generated by adversarial noised generator (e.g. the adversarial generator206) trained by the GAN302. The GAN302comprises a generator that generates the noise data120and a discriminator that tests whether distribution of the corrupted features216matches with statistical distribution of the extracted features216.

At operation706, the extracted features216are compressed. The compressed features218are closer to the extracted features214than to the corrupted features216. The compressed features218are generated by the compressor208of the IA system102. The compressed features218may be generated by solving a multi-objective optimization optimizing a cost function. The cost function reduces a distance between the compressed features and the extracted features214. The cost function also increases a distance between the compressed features218and the corrupted features216. In some embodiments, the compressed features218may also undergo a minimization process through a loss function. The loss function includes a combination of failures of the generated distribution and failures of the classification. In some embodiments, the compressed features218may be projected into a subspace to generate a subspace representation of a sequence of temporally connected data of the input data. The sequence of temporally connected data may be determined based on a temporal order of the input data104. The temporal order may be determined using PCA technique. The subspace representation is prevented from diverging from the input data104based on a distortion penalty.

At operation708, the compressed features218are classified. The compressed features218are filtered from the noise data and are close to the extracted features214. In an example embodiment, the classified compressed features218may be classified to a class that is close to the input data104.

FIG.8shows an overall block diagram of the AI system800, according to some example embodiments of the present disclosure.

The AI system800corresponds to the AI system102ofFIG.1. The AI system800includes a processor804configured to execute stored instructions, as well as a memory806that stores instructions that are executable by the processor804. The processor804corresponds to the processor200. The processor804can be a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory806can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory systems. The processor804is connected through a bus816to an input interface802. These instructions implement a method700for classification of compressed features, such as the compressed features218.

In some implementations, the AI system800may have different types and combination of input interfaces to receive input data822. In one implementation, the input interface802may include a keyboard and/or pointing device, such as a mouse, trackball, touchpad, joy stick, pointing stick, stylus, or touchscreen, among others.

Additionally, or alternatively, a network interface controller816may be adapted to connect the AI system800through the bus816to a network826. Through the network826, the input data822may be downloaded and stored within the memory806for storage and/or further processing.

In some embodiments, the memory806is configured to store modules, such as a feature extractor808, an adversarial noise generator810, a compressor812and a classifier814. The modules are executed by the processor804for classification of the compressed features. The feature extractor808may be trained to process the input data822to extract features of the input data822for classification of the input data822. The adversarial noise generator810is trained to generate noise data for distribution of features of the input data822. The generated noise data are coupled with the extracted features to generated corrupted features. The noise data is generated such that a misclassification rate of the corrupted features is greater than a misclassification rate of the extracted features. The compressor812is configured to compress the extracted features. The compressed features are closer to the extracted features than the corrupted features. The classifier is trained to classify the compressed features.

Additionally, or alternatively, a set of ground truth label classes for the classification of the input data822may be stored in a storage device828.

In addition to input interface802, the AI system800may include one or multiple output interfaces to output the classified compressed features. For example, the AI system800may be linked through the bus816to an output interface824adapted to connect the AI system800to an output device826, wherein the output device826may include a computer monitor, projector, a display device, a screen, mobile device.

In this manner, the AI system800generates compressed feature data that may be used as a subspace representation. The subspace representation is of low-dimension that may be used for classification of input data, in an efficient manner. The subspace representation is filtered from any randomly generated noise data that may help in generating accurate output of the classification. The subspace representation may be generated different modalities of the input data. The different modalities may include image data, video data, audio data, textual data or the like. This provides flexibility and versatility in usage of the AI system800.

Also, the embodiments of the present disclosure may be embodied as a method, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts concurrently, even though shown as sequential acts in illustrative embodiments. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the present disclosure.