TRANSFER LEARNING FOR GENERATING A TARGET DOMAIN ANOMALY DETECTION MODEL USING SOURCE DOMAIN DATA

Embodiments of the invention are directed to a computer system including a memory communicatively coupled to a processor system, where the processor system is operable to perform processor system operations to predict an anomaly in a target domain (TD) dataset. The processor system operations include training a model to perform an anomaly prediction task on a TD. The training includes applying a transfer learning operation that includes learning to predict the anomaly based at least in part on a first source domain (SD) precision matrix computed from a first SD.

BACKGROUND

The present invention relates in general to programmable computers that prepare digital information for analysis. More specifically, the present invention relates to computing systems, computer-implemented methods, and computer program products that implement a novel transfer learning scheme operable to generate a target domain anomaly detection model using source domain data.

Data science combines math, statistics, specialized programming, advanced analytics, artificial intelligence (AI), and machine learning (ML), and specific subject matter expertise to uncover actionable insights hidden in an organization's data. The insights can be used to guide decision making and strategic planning. An important consideration in data science is the quality of the data to be analyzed. Data quality can be impacted by so-called “anomaly” or “outlier” data. The term “anomaly” refers to a data point or a set of data points that diverges dramatically from expected samples and patterns for their type. For a dataset that follows a standard bell curve, the anomalies are the data on the far right and left. Anomalies can indicate fraud or some other anomaly, but they can also be measurement errors, experimental problems, or a novel, one-off instance. Anomalies and/or outliers can hamper data analysis techniques and skew analysis results.

Anomaly detection is the process of detecting anomalies, and, depending on the goals of the associated data analysis, remove or resolve them from the analysis to prevent any potential skewing. For image domains, anomaly detection processes can be used in quality control operations to inspect surfaces to identify unacceptable defects or imperfections in an associated product. Anomaly detection uses mathematical techniques to detect abnormalities within a dataset (e.g., a dataset that represents an image) based on how different a given data point is from its surrounding data points or from a standard deviation. Anomaly detection tasks can be performed by neural networks using deep learning algorithms. Known deep learning algorithms generally require large amounts of labeled (annotated) data to learn so-called “deep features” in order to train effective models for the performance of cognitive operations such as prediction, classification, and the like. However, in many anomaly detection deep learning applications, labeled anomalous training data is not available or not abundant due to a variety of factors.

So-called “zero-shot” learning techniques have been developed to train machine learning algorithms to perform classification and/or prediction tasks where the machine learning algorithm has not previously seen or been trained with examples of the actual classification/prediction task. In other words, zero-shot learning can enable a machine algorithm to perform classification/prediction tasks where examples of the actual to-be-predicted (TBP) data are “unknown” to the machine learning algorithm(s). TBP data can include content-related features and domain-related features. In an example where the classification task is classifying anomalous sounds (e.g., sounds that indicate an actual or eminent vehicle malfunction) coming from a vehicle, the characteristics of the acoustic sound (e.g., pitch, tone, loudness, etc.) are considered the “content” features of the TBP data; and the context characteristics of the runtime acoustic sound (vehicle make/model, weather, road conditions, driving habits of the vehicle operator, driving through tunnels, etc.) are considered the “domain” features of the TBP data. In this detailed description, the term “unknown” refers to situations where training data of the content and domain in which a neural network will attempt to classify TBP data is not available in sufficient quantities for effective training of a deep learning neural network.

In zero-shot learning, the classes covered by training instances and the classes that the classification/prediction task needs to classify are disjoint. Thus, zero-shot learning techniques are designed to overcome the lack of training examples in the classification/prediction task by leveraging details learned from training examples of a task that is related to but different from the subject classification/prediction task. The details learned from training examples of the related/different task are used to draw inferences about the unknown classes of the subject classification/prediction task because both the training classes and the unknown task classes are related in a high dimensional vector space called semantic space. Thus, known zero-shot learning techniques can include a training stage and an inference stage. In the training stage, knowledge about the attributes of intermediate semantic layers is captured; and in the inference stage, this knowledge is used to categorize instances among a new set of classes.

Another machine learning technique for performing zero-shot learning is known as “transfer learning.” Transfer learning is a machine learning method where a model developed for a first task is reused as the starting point for a model on a second, different but related task. For example, in a deep learning application, pre-trained models are used as the starting point on a variety of computer vision and natural language processing tasks. Transfer learning leverages through reuse the vast knowledge, skills, computer, and time resources required to develop neural network models. Transfer learning techniques have been developed that leverage training data from a different but related domain in an attempt to avoid the significant amount of time it takes to develop labeled training data to train an anomaly detection model for a performing anomaly detection tasks in a subject domain. The domain of the TBP data is referred to as the target domain (TD), and the domain of the different but related task is referred to as the source domain (SD).

It is a challenge in known zero-shot deep learning techniques that implement transfer learning to identify the deep features of the TD in a manner that enables the SD training data to train the TD anomaly detection model in an efficient and effective manner. Accordingly, there is a need in the art of anomaly detection to develop processes for identifying deep features of the TD in manner that enables zero-shot deep learning processes and transfer learning processes to efficiently and effectively leverage SD training data to develop TD anomaly detection models.

SUMMARY

Embodiments of the invention are directed to a computer system including a memory communicatively coupled to a processor system, where the processor system is operable to perform processor system operations to predict an anomaly in a target domain (TD) dataset. The processor system operations include training a model to perform an anomaly prediction task on a TD. The training includes applying a transfer learning operation that includes learning to predict the anomaly based at least in part on a first source domain (SD) precision matrix computed from a first SD.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the anomaly Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first SD precision matrix to compute the anomaly sore. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., the first SD precision matrix.

In addition to one or more of the features described above, or as an alternative to any of the foregoing embodiments of the invention, learning to predict the anomaly is further based at least in part on a second SD precision matrix computed from a second SD that is different from the first SD.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the anomaly Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first SD precision matrix and a second precision matrix to compute the anomaly sore. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., the first SD precision matrix and the second SD precision matrix, where the first and second SDs are different from one another.

In addition to one or more of the features described above, or as an alternative to any of the foregoing embodiments of the invention, learning to predict the anomaly is further based at least in part on a first SD mean vector computed from the first SD.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first SD mean vector, a first SD precision matrix and a second precision matrix to compute the anomaly sore. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., the first SD mean vector, the first SD precision matrix, and the second SD precision matrix, where the first and second SDs are different from one another.

In addition to one or more of the features described above, or as an alternative to any of the foregoing embodiments of the invention, learning to predict the anomaly is further based at least in part on a second SD mean vector computed from the second SD.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first SD mean vector, a second SD mean vector, a first SD precision matrix and a second precision matrix to compute the anomaly sore. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., the first SD mean vector, the second SD mean vector, the first SD precision matrix, and the second SD precision matrix, where the first and second SDs are different from one another.

In addition to one or more of the features described above, or as an alternative to any of the foregoing embodiments of the invention, learning to predict the anomaly is further based at least in part on a first summation including a summation of the first SD precision matrix and the second SD precision matrix; and a second summation including a summation of the first SD mean vector and the second SD mean vector.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first summation that includes a summation of the first SD mean vector and the second SD mean vector, along with a second summation that includes a summation of the first SD precision matrix and the second precision matrix to compute the anomaly sore. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., a summation of the first SD mean vector and the second SD mean vector, along with a summation of the first SD precision matrix and the second SD precision matrix, where the first and second SDs are different from one another.

In addition to one or more of the features described above, or as an alternative to any of the foregoing embodiments of the invention, the first summation includes a first weighted summation; and the second summation includes a second weighted summation.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first weighted summation that includes a weighted summation of the first SD mean vector and the second SD mean vector, along with a second weighted summation that includes a weighted summation of the first SD precision matrix and the second precision matrix to compute the anomaly sore. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., a weighted summation of the first SD mean vector and the second SD mean vector, along with a weighted summation of the first SD precision matrix and the second SD precision matrix, where the first and second SDs are different from one another.

In addition to one or more of the features described above, or as an alternative to any of the foregoing embodiments of the invention, learning to predict the anomaly is based at least in part on a TD domain vector of the TD; a weight component of the first weighted summation includes the TD domain vector; a weight component of the second weighted summation includes the TD domain vector; and learning to predict the anomaly includes computing a Mahalanobis distance based at least in part on the TD domain vector, the first weighted summation, and the second weighted summation.

The above-described embodiments of the invention provide technical benefits and technical effects. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a first weighted summation that includes a weighted summation of the first SD mean vector and the second SD mean vector, along with a second weighted summation that includes a weighted summation of the first SD precision matrix and the second precision matrix to compute the anomaly sore, where the weight is provided by the TD domain vector. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the anomaly prediction, i.e., a weighted summation of the first SD mean vector and the second SD mean vector, along with a weighted summation of the first SD precision matrix and the second SD precision matrix, where the weight is provided by the TD domain vector, and where the first and second SDs are different from one another.

Embodiments of the invention are also directed to computer-implemented methods and computer program products having substantially the same features, functionality, and technical benefits as the computer system described above.

Additional features and advantages are realized through techniques described herein. Other embodiments and aspects are described in detail herein. For a better understanding, refer to the description and to the drawings.

In the accompanying figures and following detailed description of the disclosed embodiments, the various elements illustrated in the figures are provided with three-digit reference numbers. In some instances, the leftmost digits of each reference number corresponds to the figure in which its element is first illustrated.

DETAILED DESCRIPTION

Many of the functional units of the systems described in this specification have been labeled as modules. Embodiments of the invention apply to a wide variety of module implementations. For example, a module can be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like. Modules can also be implemented in software for execution by various types of processors. An identified module of executable code can, for instance, include one or more physical or logical blocks of computer instructions which can, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together but can include disparate instructions stored in different locations which, when joined logically together, function as the module and achieve the stated purpose for the module.

The various components/modules of the systems illustrated herein are depicted separately for ease of illustration and explanation. In embodiments of the invention, the functions performed by the various components/modules can be distributed differently than shown without departing from the scope of the various embodiments of the invention describe herein unless it is specifically stated otherwise.

Turning now to an overview of technologies that are more specifically related to aspects of the invention, as previously noted herein, anomaly detection is the process of detecting anomalies, and, depending on the goals of the associated data analysis, remove or resolve them from the analysis to prevent any potential skewing. Anomaly detection tasks can be performed by neural networks using deep learning algorithms. Known deep learning algorithms require large amounts of labeled (annotated) data to learn so-called “deep features” in order to train effective models for the performance of cognitive operations such as prediction, classification, and the like. However, in many anomaly detection deep learning applications, labeled anomalous training data is not available or not abundant due to a variety of factors.

So-called “zero-shot” learning techniques have been developed to train machine learning algorithms to perform classification or prediction tasks where the machine learning algorithm has not previously seen or been trained with examples of the actual classification/prediction task. Another machine learning technique for performing zero-shot learning is known as “transfer learning.” Transfer learning is a machine learning method where a model developed for a first task is reused as the starting point for a model on a second, different but related task. The domain of the to-be-predicted (TBP) data is referred to as the target domain (TD), and the domain of the different but related task is referred to as the source domain (SD).

It is a challenge in known zero-shot deep learning techniques that implement transfer learning to identify the deep features of the TD in a manner that enables the SD training data to train the TD anomaly detection model in an efficient and effective manner. Accordingly, there is a need in the art of anomaly detection to develop processes for identifying deep features of the TD in manner that enables zero-shot deep learning processes and transfer learning processes to efficiently and effectively leverage SD training data to develop TD anomaly detection models.

Turning now to an overview of aspects of the invention, embodiments of the invention provide computing systems, computer-implemented methods, and computer program products that implement a novel transfer learning scheme operable to generate a TD anomaly detection model using SD data. In some embodiments of the invention, the computer-implemented method calculates an anomaly score for a TD image. The method includes obtaining a TD domain vector representing characteristics of the TD. The TD domain vectors are latent domain vectors, which are latent representations of the TD domains. A weighted-sum SD mean vector is computed from latent features of each instance of a SD in the set of SDs by computing a SD mean vector for each of the SDs; applying the TD domain vector as a weight to each instance of a SD in the set of SDs; and summing the weighted SD mean vectors to create the weighted-sum SD mean vector. A weighted-sum SD precision matrix is computed from latent features of each instance of a SD in the set of SDs; applying the TD domain vector as a weight to each instance of a SD in the set of SDs; and summing the weighted SD precision matrices to create the weighted-sum SD precision matrix. A Mahalanobis distance is computed in accordance with embodiments of the invention based at least in part on the TD domain vector, the weighted sum of SD mean vectors, and the weighted sum precision matrices. In general, a Mahalanobis distance is a multivariate distance metric that measures the distance between a point and a distribution. The Mahalanobis distance is a metric that can be used to identify multivariate anomalies or outliers. The larger the value of Mahalanobis distance, the more unusual the data point (i.e., the more likely it is to be a multivariate anomaly/outlier). In accordance with embodiments of the invention, known Mahalanobis distance computations are modified to take into account the TD domain vector, the weighted sum of SD mean vectors, and the weighted sum of precision matrices.

In some embodiments of the invention, the mean vector and precision matrix in each SD are the sample mean vector and precision matrix of the latent features in each SD. The sample mean vector is the mean of latent feature vectors of image samples in the training data, where latent feature vectors can be computed with the “EfficientNet” or other similar image models.

In some embodiments of the invention, the domain vector is computed from deep sets or its variants from latent features. The prediction model is trained for the weights for the weighted sum with deep sets in order to make predicted weights domain-specific in the SD. Specifically, embodiments of the invention consider one domain in the SD as the pseudo-TD in training, and in round robin manner, the deep sets are trained to produce domain-specific weight vectors for the pseudo-TD. The pseudo-TD is an SD selected in the current step of the round robin, which corresponds to an iteration step of the gradient descent in the optimization of the prediction model.

In some embodiments of the invention, once the model is trained properly, the prediction for the weights can be generalized even for the unseen TD.

In some embodiments of the invention, mean vectors and precision matrices are computed for each small patch in an image and a local anomaly detection result is computed for each patch.

In some embodiments of the invention, the deep sets are used to predict a single set of coefficients (single domain vector) for all domains, and the same coefficients are used over all patches in an image in a domain.

In some embodiments of the invention, deep sets are also predict a domain vector for each patch in a domain

In some embodiments of the invention, a single anomaly detection result is computed as the maximum, average, or weighted sum of the previously-described local anomaly detection results.

In some embodiments of the invention, data on each domain is augmented into different domain data. This feature enables embodiments of the invention to be used when we have only a single SD.

In some embodiments of the invention, the domain vector is calculated from deep sets or its variants from latent features of images in the TD.

In some embodiments of the invention, the SD mean vectors and the SD precision matrices are computed for each small patch in an image and a local anomaly detection result is computed for each patch. A single anomaly detection result is calculated as the maximum, average, or weighted sum of local anomaly detection results.

In some embodiment of the invention, the anomaly score is a log-likelihood with the SD mean vector and SD precision matrix. The log-likelihood value is a measure of goodness of fit for any model where a higher value represents a better model performance. A monotonically increasing function can be used for wrapping the score.

FIG.2depicts a simplified block diagram illustrating a system202operable to implement embodiments of the invention.FIGS.3A and3Bdepict non-limiting examples of how the anomaly detection classifier/model204of the system202shown inFIG.1can be implemented as a deep learning algorithm204A.FIG.4depicts a flow diagram illustrating a computer-implemented methodology400operable to be performed by the system202according to aspects of the invention. The following description of the system202refers to components and operations of the system202shown inFIG.2, and, where appropriate, also refers to the corresponding features, operations, and/or steps of the deep learning algorithm204A shown inFIG.3Band the methodology400shown inFIG.4.

The system202includes an anomaly detection classifier/model204, which can be implemented at least in part as the deep learning algorithm204A (shown inFIG.3B). As shown inFIG.3A, the deep learning algorithm204A is based on the functionality a biological neuron, which is modeled inFIG.3Aa node operable to implement a mathematical function (f(x)). In general, a biological neuron has pathways that connect it to upstream inputs, downstream outputs, and downstream “other” neurons. Each biological neuron sends and receives electrical impulses through the pathways. The nature of these electrical impulses and how they are processed in biological neuron are primarily responsible for overall brain functionality. The pathway connections between the biological neurons can be strong or weak. When the neuron receives input impulses, the neuron processes the input according to the neuron's function and sends the result of the function on one of the pathways to downstream outputs and/or another one of the pathways to downstream “other” neurons. A normal adult human brain includes about one hundred billion interconnected neurons.

InFIG.3A, the biological neuron is modeled as the node302having the mathematical function, f(x), which is depicted by the equation shown inFIG.3A. Node302receives electrical signals from inputs312,314, multiplies each input312,314by the strength of its respective connection pathway304,306, takes a sum of the inputs, passes the sum through the function, f(x), and generates a result316, which can be a final output or an input to another node, or both. In the present specification, an asterisk (*) is used to represent a multiplication. Weak input signals are multiplied by a very small connection strength number, so the impact of a weak input signal on the function is very low. Similarly, strong input signals are multiplied by a higher connection strength number, so the impact of a strong input signal on the function is larger. The function f(x) is a design choice, and a variety of functions can be used. A suitable design choice for f(x) is the hyperbolic tangent function, which takes the function of the previous sum and outputs a number between minus one and plus one.

FIG.3Bdepicts a simplified example of a deep learning neural network architecture (or model)204A. In general, neural networks can be implemented as a set of algorithms running on a programmable computer (e.g., computing environment100shown inFIG.1). In some instances, neural networks are implemented on an electronic neuromorphic machine (e.g., the IBM®/DARPA SYNAPSE computer chip) that attempts to create connections between processing elements that are substantially the functional equivalent of the synapse connections between brain neurons. In either implementation, neural networks incorporate knowledge from a variety of disciplines, including neurophysiology, cognitive science/psychology, physics (statistical mechanics), control theory, computer science, artificial intelligence, statistics/mathematics, pattern recognition, computer vision, parallel processing and hardware (e.g., digital/analog/VLSI/optical). The basic function of a neural network is to recognize patterns by interpreting sensory data through a kind of machine perception. Real-world data in its native form (e.g., images, sound, text, or time series data) is converted to a numerical form (e.g., a vector having magnitude and direction) that can be understood and manipulated by a computer. The neural network is “trained” by performing multiple iterations of learning-based analysis on the real-world data vectors until patterns (or relationships) contained in the real-world data vectors are uncovered and learned.

Neural networks use feature extraction techniques to reduce the number of resources required to describe a large set of data. The analysis on complex data can increase in difficulty as the number of variables involved increases. Analyzing a large number of variables generally requires a large amount of memory and computation power. Additionally, having a large number of variables can also cause a classification algorithm to over-fit to training samples and generalize poorly to new samples. Feature extraction is a general term for methods of constructing combinations of the variables in order to work around these problems while still describing the data with sufficient accuracy.

Although the patterns uncovered/learned by a neural network can be used to perform a variety of tasks, two of the more common tasks are labeling (or classification) of real-world data and determining the similarity between segments of real-world data. Classification tasks often depend on the use of labeled datasets to train the neural network to recognize the correlation between labels and data. This is known as supervised learning. Examples of classification tasks include identifying objects in images (e.g., stop signs, pedestrians, lane markers, etc.), recognizing gestures in video, detecting voices, detecting voices in audio, identifying particular speakers, transcribing speech into text, the like. Similarity tasks apply similarity techniques and (optionally) confidence levels (CLs) to determine a numerical representation of the similarity between a pair of items.

Returning still toFIG.3B, the simplified neural network architecture/model204A is organized as a weighted directed graph, where the artificial neurons are nodes (e.g., N1-N13), and where weighted directed edges (i.e., the directional arrows) connect the nodes. The neural network architecture/model204A is organized such that nodes N1, N2, N3are input layer nodes, nodes N4, N5, N6, N7are first hidden layer nodes, nodes N8, N9, N10, N11are second hidden layer nodes, and nodes N12, N13are output layer nodes. Multiple hidden layers indicates that the neural network architecture/model204A is a deep learning neural network architecture/model. Each node is connected to every node in the adjacent layer by connection pathways, which are depicted inFIG.3Bas directional arrows each having its own connection strength. For ease of illustration and explanation, one input layer, two hidden layers, and one output layer are shown inFIG.3B. However, in practice, multiple input layers, multiple hidden layers, and multiple output layers can be provided. When multiple hidden layers are provided, the neural network model204A can perform unsupervised deep-learning for executing classification/similarity type tasks.

Similar to the functionality of a human brain, each input layer node N1, N2, N3of the neural network204A receives Inputs directly from a source (not shown) with no connection strength adjustments and no node summations. Each of the input layer nodes N1, N2, N3applies its own internal f(x). Each of the first hidden layer nodes N4, N5, N6, N7receives its inputs from all input layer nodes N1, N2, N3according to the connection strengths associated with the relevant connection pathways. Thus, in first hidden layer node N4, its function is a weighted sum of the functions applied at input layer nodes N1, N2, N3, where the weight is the connection strength of the associated pathways into the first hidden layer node N4. A similar connection strength multiplication and node summation is performed for the remaining first hidden layer nodes N5, N6, N7, the second hidden layer nodes N8, N9, N10, N11, and the output layer nodes N12, N13.

The neural network architecture/model204A can implement various deep learning-based feature extraction and classification methods. In general, deep learning-based classification schemes have two sub-networks, a feature extraction network followed by a classification sub-network that are learned jointly during training. Traditional non-zero-shot deep learning requires the availability of multiple classes for training and an extremely large number of training samples (in the order of thousands or millions). However, for zero-shot learning performed by the neural network architecture/model204A, the classes covered by training instances and the classes that the classification/prediction task needs to classify are disjoint. Thus, zero-shot learning techniques are designed to overcome the lack of training examples in the classification/prediction task by leveraging details learned from training examples of a task that is related to but different from the subject classification/prediction task. The details learned from training examples of the related/different task are used to draw inferences about the unknown classes of the subject classification/prediction task because both the training classes and the unknown task classes are related in a high dimensional vector space called semantic space. Thus, zero-shot learning techniques performed by the neural network architecture/model204A can include a training stage and an inference stage. In the training stage, knowledge about the attributes of intermediate semantic layers is captured; and in the inference stage, this knowledge is used to categorize instances among a new set of classes.

Returning now to the system202shown inFIG.2, the anomaly detection classifier/model204is operable to perform zero-shot transfer learning in which training data from a SD240is used to train the classifier/model204to perform a classification task on TBP data from the TD230. The transfer learning module210is operable to assist with training the deep learning algorithms of the classifier/model204. In general, the transfer learning module210is operable to perform transfer learning tasks that leverage model parameters (e.g., labeled data and associated data formats/structures) that are ideal for one task (e.g., detecting and classifying anomaly data of the SD240) and using it instead as part of the development of another task (e.g., detecting and classifying anomaly data of the TD230).

The anomaly detection functionality performed by the classifier/model204and the transfer learning module210can, in some embodiments of the invention, use an autoencoder architecture, which is a type of neural network that learns how to efficiently compress and encode original data to a lower dimensional space known as “latent code” then learns how to decompress the latent code to a representation of the original data (i.e., “reconstructed” original data) that is as close to the original data input as possible. The differences between the original data input and the reconstructed data output can be used to create encoded rules for expected output and vice versa. Post-training, the autoencoder can flag as anomalous data values that fall outside of the encoded rules.

In embodiments of the invention, the anomaly detection classifier/model204is operable to interface with and receive data from a TD230and a SD240. Example implementations of the TD230and the SD240are shown inFIG.7, which shows example images of different types of surfaces or surface textures. The wood surface images230A are example implementations of the TD230. The carpet surface images240A, the grid surface images240B, the leather surface images240C, and the tile surface images240D are example implementations of the SD240. In embodiments of the invention, the TD230and the SDs240can each be a repository of electronic files containing electronic image data representing multiple instances of the wood surface images230A, the carpet surface images240A, the grid surface images240B, the leather surface images240C, and the tile surface images240D that can be accessed by the anomaly detection classifier/model204, and converted to a format or formats that can be analyzed by the various neural networks of the classifier/model204.

As shown inFIG.7, the training data includes normal or non-anomalous instances of the wood surface images230A, the carpet surface images240A, the grid surface images240B, the leather surface images240C, and the tile surface images240D. The test data includes normal and anomaly instances of the wood surface images230A, the carpet surface images240A, the grid surface images240B, the leather surface images240C, and the tile surface images240D. In general, the training data varies depending on whether we are using the classifier/model204uses supervised learning algorithms (the image files of the SD240are labeled) or unsupervised learning algorithms (the image files of the SD240are not labeled. For unsupervised learning, the training data contains unlabeled TD and SD data points, i.e., inputs are not tagged with the corresponding outputs. The classifier/model204is required to find the patterns from the given training datasets in order to make predictions. On the other hand, for supervised learning. the classifier/model204learns to make predictions based on training data that contains labels for the SD data points, while the TD data points remain unlabeled.

Once the classifier/model204is trained with the training dataset, the classifier/model204is tested with the test dataset. The test dataset evaluates the performance of the classifier/model204and ensures that the classifier/model204can generalize well with a new or unseen dataset from the TD230. The test dataset is used as a benchmark for model evaluation once the model training is completed. The test data contains data for each type of scenario for a given problem that the classifier/model204would be facing when used in the real world. Accordingly, the test data includes normal instances and anomaly instances of image files of the SD images240A,240B,240C,240D, along with normal instances and anomaly instance of image files containing the TD image230A.

Returning again to the system202shown inFIG.2. the TD data from the TD230and the SD data from the SD240are transformed or encoded into numbers, including vectors and matrices. The classifier/model204use the encoded vectors and matrices to determine TD domain vectors212, weighted-sum SD mean vectors214, and weighted-sum SD precision matrices216. In general, vectors and matrices represent inputs like text and images as numbers, which can be read, understood and processed by the classifier/model204during training, testing, and post-training/testing predictions. Vectors can be related to domain features, and the various domain features, including the latent domain features, can be represented as vectors. The latent domain features include multivariates from which mean vectors and precision matrices can be computed. In general, a multivariate is a vector each of whose elements is a variate. The variates need not be independent, and if they are not, a correlation is said to exist between them. The term “multivariate” is also used as an adjective to mean involving many variables, as opposed to one (univariate) or two (bivariate). The mean vector of the multivariate representations of the latent domain features can be computed as the means of each variable of the multivariate representations of the latent domain features. A precision matrix (also known as an inverse covariance matrix) represents the pair-wise correlations between variables in a multivariate normal distribution. The matrix's entries represent the degree to which changes in one variable are related to changes in another. The precision matrix's diagonal entries are the reciprocals of the variances of the individual variables, and the off-diagonal entries are the covariances between the variables. The precision matrix has the benefit of being easier to explain in terms of conditional independence than the covariance matrix. If two variables are independent, the precision matrix entry corresponding to them is zero. If two variables are significantly reliant on each other, the corresponding entry in the precision matrix represents the degree to which changes in one variable are related to changes in another. As a result, the precision matrix is a more straightforward method of assessing conditional independence than the covariance matrix. Thus, the precision matrix conveys how strongly one variable is reliant on others, which can help assist with comprehending conditional independence and other multivariate distribution features.

The transfer learning module210implements learning/training processes for identifying deep features of the TD230in a manner that enables zero-shot deep learning processes of the classifier/model204and the transfer learning module210to efficiently and effectively leverage SD training data of the SD240to develop or train the classifier/model204to identify anomaly data (e.g., included among the results270) in the TD230. The transfer learning module210identifies deep features of the TD230by using a set of TD domain vectors212(Step-1of the methodology400shown inFIG.4), weighted sum SD mean vectors214(Step-2of the methodology400shown inFIG.4), and weighted sum precision matrices216(Step-3of the methodology400shown inFIG.4) to computer the Mahalanobis distance218(Step-4of the methodology400shown inFIG.4), which is used to generate results270as an anomaly prediction. In embodiments of the invention, the TD domain vectors212are used as the weights that are used to form the weighted sum SD mean vectors214(Step-2of the methodology400shown inFIG.4), and the TD domain vectors212are used as the weights that are used to form the weighted sum precision matrices216(Step-3of the methodology400shown inFIG.4). In embodiments of the invention, the results270can also branch through a learning feedback path272to provide further training of the classifier/model204.

The TD domain vectors212are latent domain vectors, which are latent representations of the TD230. Latent representations can be generated through a process known as latent representation learning (LRL) or latent variable modeling (LVM). LRL is a machine learning technique that attempts to infer latent variables from empirical measurements. Latent variables are variables that cannot be measured directly and therefore have to be inferred from the empirical measurements. In a given domain, some variables of interest are directly measurable variables, while some variables of interest are not directly measurable. Such not-directly-measurable variables are modeled as latent variables of an LVM. In general, one or many latent variables jointly constitute a latent space or latent representation. This representation is usually a compressed form of the empirical measurements; it consists of fewer latent variables than the dimensionality of the measurements (i.e., the number of different measurement modalities). The latent domain vectors (i.e., the TD domain vector212) are used in embodiments of the invention to infer anomaly-related features of the TD230from the set of normal instances in the TD230. The TD mean vector212is represented inFIG.5as “zd.”

The weighted-sum SD mean vector214is computed in accordance with the previously-described mean vector computations applied to latent features of each instance of a SD in the set of SDs. Using the example shown inFIG.7, a first SD mean vector would be computed for the carpet surface images240A; a second S/D mean vector would be computed for the grid surface images240B; a third S/D mean vector would be computed for the leather surface images240C; and a fourth S/D mean vector would be computed for the tile surface images240D. The TD domain vector212is applied as a weight to each of the first SD mean vector, the second SD mean vector, the third SD mean vector, and the fourth SD mean vector. The weighted first, second, third, and fourth SD mean vectors are summed to create the weighted-sum SD mean vector214. A non-limiting example of how the weighted-sum SD mean vector214can be computed using Equation (2) shown inFIG.5. Additionally,FIG.6depicts at block610a non-limiting example of an architecture for computing the SD mean vector, and further depicts at block620a non-limiting example of an architecture for applying the TD domain vector212as weights to the SD mean vectors for each SD to generate the weighted-sum SD mean vector214. Xdnis an n-th image sample in the d-th domain.

The weighted-sum SD precision matrix216is computed in accordance with the previously-described precision matrix computations applied to latent features of each instance of a SD in the set of SDs. Using the example shown inFIG.7, a first SD precision matrix would be computed for the carpet surface images240A; a second S/D precision matrix would be computed for the grid surface images240B; a third S/D precision matrix would be computed for the leather surface images240C; and a fourth S/D precision matrix would be computed for the tile surface images240D. The TD domain vector212is applied as a weight to each of the first SD precision matrix, the second SD precision matrix, the third SD precision matrix, and the fourth SD precision matrix. The weighted first, second, third, and fourth SD precision matrices are summed to create the weighted-sum SD precision matrix216. A non-limiting example of how the weighted-sum SD precision vector216can be computed using Equation (3) shown inFIG.5. Additionally,FIG.6depicts at block610a non-limiting example of an architecture for computing the SD precision matrix, and further depicts at block620a non-limiting example of an architecture for applying the TD domain vector212as weights to the SD precision matrix for each SD to generate the weighted-sum SD precision matrix vector216. Xdnis a n-th image sample in the d-th domain.

The Mahalanobis distance218is computed based at least in part on the TD domain vector212, the weighted sum of SD mean vectors214, and the weighted sum precision matrices216. In general, a Mahalanobis distance is a multivariate distance metric that measures the distance between a point and a distribution. The Mahalanobis distance is a metric that can be used to identify multivariate anomalies or outliers. The larger the value of Mahalanobis distance, the more unusual the data point (i.e., the more likely it is to be a multivariate anomaly/outlier). In accordance with embodiments, known Mahalanobis distance computations are modified to take into account the TD domain vector212, the weighted sum of SD mean vectors214, and the weighted sum precision matrices216. A non-limiting example of how the Mahalanobis distance218can be computed in accordance with embodiments of the invention to take into account the TD domain vector212, the weighted sum SD mean vectors214, and the weighted sum precision matrices216is depicted as Equation (1) shown inFIG.5. Xijis a latent image feature at (i, j) position in the n-th image sample in the d-th domain, where indices n and d are omitted.

Thus, it can be seen from the foregoing detailed description that embodiments of the invention provide technical effects and benefits. While known deep feature approaches do not allow the TD domain vector to be input into their computation of anomaly prediction, i.e., the computation of the Mahalanobis distance, as an additional input because the input format is fixed in known deep feature approaches, embodiments of the invention use a weighted-sum SD mean vector and a weighted-sum SD precision matrices to compute the Mahalanobis distance. The approach used in embodiments of the invention can bypass the above-described limitations of known deep feature approaches because embodiments of the invention do not change the input format but change the parameters in the computation of the Mahalanobis distance, i.e., the mean vector and the precision matrix, with the weighted-sums.

It will be understood that those skilled in the art, both now and in the

future, may make various improvements and enhancements which fall within the scope of the claims which follow.