MACHINE LEARNING FOR SEMI-SUPERVISED WORKLOAD CLASSIFICATION

Computer implemented methods, systems, and computer program products include program code executing on a processor(s) that obtain temporal input from computing resources. The processor(s) extracts patterns from the temporal input, determine if relevant domain knowledge is available. If relevant data is available, the processor(s) utilizes it to identify patterns indicating anomalous workloads, label data in the patterns, and generate anomaly scores. If relevant data is not available, the processor(s) applies unsupervised anomaly detection algorithm(s) to identify patterns as indicative of anomalous workloads and label these data. The processor(s) generates anomaly scores for the labeled data and unlabeled data. The processor(s) determine which patterns indicates anomalous data or normal data by comparing relevant anomaly scores to a pre-defined threshold.

BACKGROUND

The present invention relates generally to the field of distributed computing management and more particularly to workload classification for optimization of computing resources including managing data center energy consumption.

Workload management in computing systems can provide various benefits for resource consumers, including but not limited to, reducing the data center energy consumption, decreasing the carbon footprint, and increasing the revenue for these consumers. Classification of workloads, including identifying workloads as non-productive workloads and hotspots (e.g., hotspots are characterized by high CPU/memory utilization for a short duration), as part of this management effort.

SUMMARY

Shortcomings of the prior art are overcome, and additional advantages are provided through the provision of a computer-implemented method for classifying and resolving anomalous workloads in a computing environment. The computer-implemented method can include: obtaining, by one or more processors, temporal input from resources comprising a computing system; extracting, by the one or more processors, patterns from the temporal input; determining, by the one or more processors, if domain knowledge is available to assist in identifying anomalous workloads in the extracted patterns based on the temporal input; based on determining that domain knowledge is available to assist in the identifying, utilizing the domain knowledge to perform a workload pattern analysis to identify a portion of the patterns as indicative of anomalous workloads, labeling data comprising the portion of the patterns, and generating a first set of anomaly scores for the labelled data comprising the portion of the patterns; based on determining that domain knowledge is not available to assist in the identifying, applying one or more unsupervised anomaly detection algorithms to identify an additional portion of the patterns as indicative of the anomalous workloads and labeling data comprising the additional portion of the patterns; generating, by the one or more processors, a second set of anomaly scores for the labeled data comprising the portion of the patterns and the additional portion of the pattern and unlabeled data comprising the extracted patterns; for each pattern of the patterns, calculating, by the one or more processors, a weighted sum comprising a combined anomaly score for the pattern based on combining one or more anomaly scores for the pattern selected one or more of the first set of anomaly scores and the second set of anomaly scores; and determining, by the one or more processors, for each pattern of the patterns, whether the pattern indicates anomalous data or normal data based on utilizing an inference component to compare the combined anomaly score for each pattern to a pre-defined threshold.

Shortcomings of the prior art are overcome, and additional advantages are provided through the provision of a computer program product for classifying and resolving anomalous workloads in a computing environment. The computer program product comprises a storage medium readable by a one or more processors and storing instructions for execution by the one or more processors for performing a method. The method includes, for instance: obtaining, by the one or more processors, temporal input from resources comprising a computing system; extracting, by the one or more processors, patterns from the temporal input; determining, by the one or more processors, if domain knowledge is available to assist in identifying anomalous workloads in the extracted patterns based on the temporal input; based on determining that domain knowledge is available to assist in the identifying, utilizing the domain knowledge to perform a workload pattern analysis to identify a portion of the patterns as indicative of anomalous workloads, labeling data comprising the portion of the patterns, and generating a first set of anomaly scores for the labelled data comprising the portion of the patterns; based on determining that domain knowledge is not available to assist in the identifying, applying one or more unsupervised anomaly detection algorithms to identify an additional portion of the patterns as indicative of the anomalous workloads and labeling data comprising the additional portion of the patterns; generating, by the one or more processors, a second set of anomaly scores for the labeled data comprising the portion of the patterns and the additional portion of the pattern and unlabeled data comprising the extracted patterns; for each pattern of the patterns, calculating, by the one or more processors, a weighted sum comprising a combined anomaly score for the pattern based on combining one or more anomaly scores for the pattern selected one or more of the first set of anomaly scores and the second set of anomaly scores; and determining, by the one or more processors, for each pattern of the patterns, whether the pattern indicates anomalous data or normal data based on utilizing an inference component to compare the combined anomaly score for each pattern to a pre-defined threshold.

Shortcomings of the prior art are overcome, and additional advantages are provided through the provision of a system for classifying and resolving anomalous workloads in a computing environment. The system includes: a memory, one or more processors in communication with the memory, and program instructions executable by the one or more processors via the memory to perform a method. The method includes, for instance: obtaining, by the one or more processors, temporal input from resources comprising a computing system; extracting, by the one or more processors, patterns from the temporal input; determining, by the one or more processors, if domain knowledge is available to assist in identifying anomalous workloads in the extracted patterns based on the temporal input; based on determining that domain knowledge is available to assist in the identifying, utilizing the domain knowledge to perform a workload pattern analysis to identify a portion of the patterns as indicative of anomalous workloads, labeling data comprising the portion of the patterns, and generating a first set of anomaly scores for the labelled data comprising the portion of the patterns; based on determining that domain knowledge is not available to assist in the identifying, applying one or more unsupervised anomaly detection algorithms to identify an additional portion of the patterns as indicative of the anomalous workloads and labeling data comprising the additional portion of the patterns; generating, by the one or more processors, a second set of anomaly scores for the labeled data comprising the portion of the patterns and the additional portion of the pattern and unlabeled data comprising the extracted patterns; for each pattern of the patterns, calculating, by the one or more processors, a weighted sum comprising a combined anomaly score for the pattern based on combining one or more anomaly scores for the pattern selected one or more of the first set of anomaly scores and the second set of anomaly scores; and determining, by the one or more processors, for each pattern of the patterns, whether the pattern indicates anomalous data or normal data based on utilizing an inference component to compare the combined anomaly score for each pattern to a pre-defined threshold.

Computer systems and computer program products relating to one or more aspects are also described and may be claimed herein. Further, services relating to one or more aspects are also described and may be claimed herein.

Additional features and advantages are realized through the techniques described herein. Other embodiments and aspects are described in detail herein and are considered a part of the claimed aspects. For example, in some embodiments, program code executing on one or more processors provides explainability, for subsequent execution of the method, by updating a semi-supervised anomaly detection algorithm by utilizing patterns of the patterns determined to be anomalous data as training data, and updating the domain knowledge with patterns of the patterns determined to be the normal and abnormal data.

DETAILED DESCRIPTION

Classification of workloads, including identifying workloads as non-productive workloads and hotspots, provides various benefits for resource consumers and for the technical infrastructure itself, including but not limited to, reducing the data center energy consumption, decreasing the carbon footprint, and increasing the revenue for these consumers. The examples herein include computer-implemented methods, computer program products, and computer systems that detect detecting anomalous workloads in a semi-supervised manner by using temporal multi-channel metrics data as input.

As explained herein, embodiments of the present invention advantageously combine at least three types of techniques in a novel manner to generate a model that comprises program code to identify anomalous data from different channels and to consistently improve the accuracy of the model through subsequent utilization. Embodiments of the present invention incorporate workload pattern analysis, an inference component, and explainability and utilize the advantages of these components, together, to identify anomalies. Through workload pattern analysis, the program code comprehensively incorporates prior knowledge to characterize various workload patterns. The program code utilizes the prior knowledge to determine some labeled data and can bootstrap the described semi-supervised machine learning techniques disclosed herein to this workload analysis to explore novel workload patterns further. The inference component of the program code balance classification results from this prior knowledge-based workload pattern analysis and data-driven machine learning techniques to label additional data. The program code can provide explainability for the classification results and iteratively enrich both the prior knowledge of the workload analysis and improve the classification performance of the inference component.

Embodiments of the present invention are inextricably tied to computing and are directed to a practical application. The examples described herein are inextricably linked to computing as the examples herein provide systems, methods, and computer program products that, for example, optimize resource usage in a computing environment, based on providing insight into both workloads and utilizing this workload data to make and implement technical infrastructure decisions. For example, in some embodiments, program code executed by one or more processors discovers resource conservation opportunities in large data centers and computing environments by identifying inactive resources. In some embodiments of the present invention, the program code can automatically decommission these inactive resources. By recognizing these resources, the program code can reduce data center energy consumption, decrease the carbon footprint of the data center, and increase the revenue of the data center users.

To realize a practical application to which the embodiments described herein are directed, the program code identifies inactive resources by classifying workloads as anomalous. These workloads can include workloads handled by one or more of virtual machines (VMs), cloud native resources (e.g., containers, server-less). Anomalous workloads can include both non-productive workloads (e.g., workloads not actively producing any useful work) and hotspots (e.g., workloads leading to overloaded resources). In the case of non-productive workloads, the program code can identify these workloads and, in some cases, automatically shut down the resources that handle these workloads. In the case of hotspots, in some examples, the program code can migrate these workloads and/or portions of these workloads to more efficient hardware and/or software resources in the technical environment (e.g., data center). The program code in embodiments of the present invention detects anomalous workloads based on assuming that anomalous workloads are divergent from general or normal patterns. The program code generates and continually tunes and updates a model to perform detection based on semi-supervised anomaly detection. The program code (and the model) utilize: 1) prior domain knowledge; and 2) machine-learning techniques. Existing approaches for classifying workloads, which include detecting non-productive workloads, have various shortcomings that are overcome by aspects of the examples herein. Existing approaches to classifying workloads generally fall into two categories: 1) resource analysis; and 2) data-driven models. As will be discussed herein, the examples herein leverage the advantages of both approaches while utilizing this combined approach to benefit from the advantages of each type of approach. First, identifying workloads as non-productive can include detecting non-productive/idle servers. Some existing approaches identify these servers based on analyzing resource utilization and assume that the non-productive workloads have lower resource consumption than expected. But these approaches can produce unreliable results because resource idleness might not be necessary to indicate utilization idleness. To address the unreliable results, other existing approaches include supervised machine learning and require the manual creation of ground-truth labels for the workloads, which forms the foundation of these systems and without this foundation, the automated processes cannot execute accurately. Unfortunately, acquiring large amounts of labeled data is a pre-requisite to conduct these supervised approaches and acquiring these data is both time-consuming and labor-intensive. The second type of existing approaches, the machine-learning techniques, omit this labeling effort and focus on identifying overloaded and underloaded resource utilization by applying prior knowledge (e.g., thresholding-based methods), but prior knowledge is often incomplete and/or inaccurate and these shortcoming are reflected in the results. Limiting workload analysis to historical activities, whether based on analyzing the physical machines, based on either of machine utilization or workload data analysis, in the existing approaches, does not provide a comprehensive understanding of workloads processed in a computing environment. As discussed herein, embodiments of the present invention represent a significant improvement to these existing approaches at least because they leverage the advantages of prior knowledge (e.g., workload pattern analytics) as well as data-based machine learning analytics.

In the examples herein, program code executing on one or more processors can assess workloads by capturing complementary information across multi-channel metrics and filter out the redundant information; various examples herein combine unsupervised anomaly detection with domain knowledge. One advantage of the examples herein over existing approaches in that the examples herein include semi-supervised approaches which reduce the need for labeled data (which is required by supervised machine learning algorithms in existing approaches) extensively. Additionally, the examples herein combine various supervised and unsupervised techniques and direct this combination to a specific practical application, computing anomaly scores which the program code can utilize for workload classification with explainability. As discussed above acquiring large amounts of labeled data is both time-consuming and labor-intensive in existing approaches and the examples herein alleviate this issues by including program code that characterizes various workload patterns based on metrics data and prior domain knowledge to reduce the amount of labeled data, which improves performance and explainability. The workload patterns that the program code utilizes can include both explicit known workload patterns conveyed by prior knowledge and implicit novel patterns conveyed by data. As such, the examples herein utilize semi-supervised learning and an inference component. In the examples, herein, after the program code has detected anomalous workloads, the examples herein gain explainability for better characterizing the workload patterns moving forward. The on-going tuning and improvement of various aspects of the present invention are discussed herein and form what can be understood as an explainability component; the program code can utilize identified anomalies to tune machine learning algorithms used to identify anomalies and the program code can utilize data not found to be anomalous to update prior knowledge used by the program code to characterize workloads. Semi-supervised learning, when compared to supervised learning, provides an advantage because semi-supervised systems require fewer labels. Embodiments of the present invention can consider inactive workloads and/or hotspots as anomalies and classify them using the described a semi-supervised anomaly detector, which utilizes only a small set of labeled data when compared to what is utilized in supervised learning system.

The examples herein provide significant advantages over existing workload assessment and management tools. An advantage provided by the examples herein includes but is not limited to embedding temporal multi-channel input. Embedding temporal multi-channel input includes program code executing on one or more processors incorporating both temporal patterns as well as structural patterns across different timestamps and metrics, so that the patterns for the workloads can be better characterized in a data-driven manner. Thus, in embodiments of the present invention, the program code can capture the complementary information across multi-channel metrics and meanwhile filter out the redundant information, while the existing approaches either focus on workload pattern analysis based on prior knowledge over resource utilization or are machine-learning-based methods that are mainly data-driven.

Various aspects of the embodiments herein distinguish approaches herein from existing approaches to optimizing resource usage in computing environments such as large data centers. The aspects described herein are not exhaustive and provided by way of example. As noted above, in embodiments of the present invention, the program code embeds temporal multi-channel data thus incorporating both temporal patterns as well as structural patterns with complementary information across different metrics. The program code also performs a semi-supervised workload classification (e.g., utilizing few-shot labeled data). Given a small set of labeled data (learned by workload pattern analysis or unsupervised anomaly detection), the program code can utilize a semi-supervised anomaly detection to capture anomalous patterns covered in the labeled data and to discover novel anomalous patterns in unlabeled data. The program code also performs a workload pattern analysis which includes characterizing patterns for various types of workloads based on prior knowledge over temporal multi-channel metrics. The program code also employs an inference component. The program code utilizes the inference component. The program code combines the anomalous scores learned by the workload pattern analysis and the semi-supervised anomaly detection results into an inference component (to take the respective powers of the scored learned by the workload pattern analysis and of the semi-supervised anomaly detection results). The results provided by the program code provide explainability meaning that based on the detection results, the program code can explore further explainability to characterize the anomalous workloads and to continually enrich the domain knowledge. Explainability (also referred to as “interpretability”) is the concept that a machine learning model and its output can be explained in a way that “makes sense” to a human being at an acceptable level.

In some embodiments of the present invention, the program code generates and continually refines a model which detects anomalous data and hence, enables the program code to classify workloads, including identifying detecting non-productive workloads. The model comprises domain knowledge and a few-shot anomaly detection algorithm (also referred to as a few-shot anomaly detection model), which are both continually refined by the program code. In some embodiments of the present invention, program code executed by one or more processor extracts both temporal and structural patterns from resources in a computing system, including from temporal multi-channel metric inputs. The program code can apply domain knowledge (e.g., pre-defined criteria) to characterize the anomalous workloads. By characterizing these anomalous workloads, the program code generates a set of labels for the data. If there is no domain knowledge the program code can utilize an unsupervised machine learning approach to identify a set of anomalous data with high confidence. Thus, whether utilizing domain knowledge or unsupervised learning, the program code can identify some anomalous data and thus generate a set of labeled data. The program code obtains the set of labeled data unlabeled data as inputs (the amount of labeled data is generally small when compared to the amount of unlabeled data) and applies a few-shot anomaly detection algorithm to analyze the inputs. A few-shot anomaly detection algorithm is a class of anomaly detection algorithm that is trained on a few examples of a normal class and is not trained on any examples of an anomalous class. The program code can utilize the few-shot anomaly detection algorithm to exploit the patterns contained in the set of labeled data and explore unknown novel anomalous patterns from the unlabeled data. The program code outputs the results of the workload pattern analysis and applying the few-shot anomaly detection algorithm, and based on these outputs, the program code performs an inference, achieving more reliable detection results.

One or more aspects of the present invention are incorporated in, performed and/or used by a computing environment. As examples, the computing environment may be of various architectures and of various types, including, but not limited to: personal computing, client-server, distributed, virtual, emulated, partitioned, non-partitioned, cloud-based, quantum, grid, time-sharing, cluster, peer-to-peer, mobile, having one node or multiple nodes, having one processor or multiple processors, and/or any other type of environment and/or configuration, etc. that is capable of executing a process (or multiple processes) that, e.g., facilitates granular real-time data attainment and delivery including as relevant to soliciting, generating, and timely transmitting, granular product review to consumers. Aspects of the present invention are not limited to a particular architecture or environment.

One example of a computing environment to perform, incorporate and/or use one or more aspects of the present invention is described with reference toFIG.1. In one example, a computing environment100contains an example of an environment for the execution of at least some of the computer code involved in performing the inventive methods, such as a code block for identifying and potentially resolving anomalous workloads150. In addition to block150, computing environment100includes, for example, computer101, wide area network (WAN)102, end user device (EUD)103, remote server104, public cloud105, and private cloud106. In this embodiment, computer101includes processor set110(including processing circuitry120and cache121), communication fabric111, volatile memory112, persistent storage113(including operating system122and block150, as identified above), peripheral device set114(including user interface (UI) device set123, storage124, and Internet of Things (IoT) sensor set125), and network module115. Remote server104includes remote database130. Public cloud105includes gateway140, cloud orchestration module141, host physical machine set142, virtual machine set143, and container set144.

Communication fabric111is the signal conduction path that allow the various components of computer101to communicate with each other. Typically, this fabric is made of switches and electrically conductive paths, such as the switches and electrically conductive paths that make up buses, bridges, physical input/output ports and the like. Other types of signal communication paths may be used, such as fiber optic communication paths and/or wireless communication paths.

FIG.2is a general workflow200that illustrates various aspects of some embodiments of the present invention. The program code in embodiments of the present invention utilizes both prior domain knowledge and machine learning-based techniques (e.g., a few-shot anomaly detection algorithm and/or supervised learning) to identify anomalous workloads. Identifying these anomalous workloads is an iterative process and the program code continually updates both the domain knowledge and the algorithm(s) applied by the machine learning-based techniques to increase accuracy with each iteration. The combination of the domain knowledge and the algorithm(s) can be understood as a model that the program code generates, updates, and applies. The program code obtains temporal inputs in multiple channels from resources in a computing environment and embeds these data into a representation which includes both temporal patterns and structural patterns (210). The program code determines if there is prior domain knowledge (220). The program code labels a portion of the data if there is domain knowledge by utilizing the domain knowledge to characterize certain of the data as anomalous, labeling the characterized data (225). This type of data labeling based on prior knowledge can be understood as workload pattern analysis. The program code performs the workload pattern analysis on the (original) temporal input. The program code labels a portion of the data if there is no domain knowledge by utilizing an unsupervised machine learning approach (e.g., dimension reduction, clustering, etc.) to identify a set of anomalous data with high confidence and labeling the data identified with the unsupervised machine learning approach (230). The program code applies a few-shot anomaly detection algorithm to the labeled data and the unlabeled data to exploit the patterns contained in the labeled data and explore unknown novel anomalous patterns from the unlabeled data (240). The program code infers (e.g., applying an inference component) which patterns in the data indicate anomalous data based on the results from the domain knowledge or unsupervised machine learning approach and the few-shot anomaly detection algorithm (250). In some examples, this detection can be performed by program code comprising an inference component; the inference component performs a binary detection of anomalies in the labeled and unlabeled data based on the workload pattern analysis (using the prior domain knowledge) and the few-shot anomaly detection. The program code updates the few-shot anomaly detection model with the confirmed anomalies by training the few-shot anomaly detection model with the confirmed anomalies as training data (260). The program code updates the prior domain knowledge (e.g., used by the program code to perform a workload pattern analysis) with the data that the program code inferred to be inferred to be normal and data that the program code inferred to be anomalous (270). In some examples, the inference is binary and thus, the program code classifies the patterns as either anomalous or normal. Thus, the program code provides the normal data as prior knowledge, to be used in future workload pattern analyses.

FIG.3depicts both various aspects of a technical architecture that performs a workflow also depicted inFIG.3. Thus,FIG.3is a combination workflow and architecture300illustration.FIG.3illustrates the temporal input302which the program code obtains, extracts structural patterns and temporal patterns from the temporal input302, and embeds the patterns to generate embedded data304(305). The temporal input302from resources in a computing system include, but are not limited to, service level metrics303(e.g., number of calls, number of erroneous calls, number of connections, number of 2xx, 4xx, and 5xx requests), infrastructure level metrics307(e.g., CPU (central processing unit) usage, memory usage, memory total cache, block IO (input/output) write), and host-level metrics309(e.g., CPU utilization of each node, memory utilization of each node). The temporal input302can be understood as a sliding window with multi-channel metrics. The sliding window is a period around a given timestamp (i.e., the temporal parameter of the input). The multi-channel metrics (e.g., service level metrics303, infrastructure level metrics307, and host-level metrics309) provide various structural data. These different channels convey complementary information, the metrics are of different importance, and, thus, the program code can mutually correlate the metrics in the same and in different channels with each other. By embedding the multi-channel (e.g., the extracted patterns from the input) or practicing attention-based bidirectional embedding (305), the program code captures temporally dependent patterns and filters out noisy and redundant information across the different metrics (e.g., service level metrics303, infrastructure level metrics307, and host-level metrics309). The program code extracts patterns that are both structural and temporal from the temporal input302.

The program code performs a deep reinforcement learning-based anomaly detection on the embedded data. This process includes few shot anomaly detection, which is a data-driven machine learning process. To that end, after embedding the data, the program code, in the illustrated example, generates a set of labeled data representing anomalous workloads from the embedded data. The program code can label the data both by utilizing domain knowledge and/or by applying an unsupervised anomaly detection algorithm (320). However, the program code performs the workload pattern analysis on the temporal input (the original data) rather than on the embedded data. The inputs to this algorithm are labeled data322and unlabeled data324and the output is anomaly scores326. To generate these inputs from the embedded data304or the temporal input302, in the illustrated example, the program code determines if there is any domain knowledge (310). The program code also obtains anomalies (i.e., a set of labeled anomalous workloads324) and provides these anomalies to a deep learning reinforcement learning-based few-shot anomaly detection algorithm. If domain knowledge is available, the program code utilizes the domain knowledge to perform a workload pattern analysis (315). The program code utilizes the domain knowledge to assist in identifying anomalous workloads in the extracted patterns based on the temporal input. If there is no domain knowledge available, the program code performs an unsupervised anomaly detection (e.g., applying a machine learning algorithm) (320). Based on utilizing the domain knowledge or applying unsupervised anomaly detection, the program code generates labeled data324which comprises a set of labeled anomalous workloads.

A non-limiting type of unsupervised anomaly detection (320) that can be utilized in some examples herein to generate labeled data324(e.g., a set of labeled anomalous workloads) is ensemble anomaly detection. Ensemble learning (or ensemble anomaly detection) includes utilizing density and rank based algorithms (e.g., LOF, COF, and INFLO). By applying these algorithms, the program code can assign an anomaly score is assigned to an object (e.g., a pattern in the data) based on density comparison of the object with its k nearest neighbors. In this example, an object is considered an anomaly if its anomaly score is greater than a pre-defined threshold. These ensemble learning algorithms can be based on the notion of rank and use the concept that if k nearest neighbors of an object consider the object as one of their close neighbors, then it is less likely to be an anomaly. Ensemble methods of anomaly detection combine various approaches/algorithms to limit bias-variance. A particular algorithm may be well-suited to the properties of one data set and be successful in detecting anomalous observations of a particular application domain but may fail to work with other datasets whose characteristics do not agree with the first dataset. An ensemble method alleviates the mismatch between an algorithm and an application because in an ensemble method, multiple algorithms are pooled before a final decision is made.

To obtain anomaly scores326(a desired output), the program code applies a few-shot anomaly detection algorithm (325). Applying the few-shot anomaly detection algorithm exploits the anomalous patterns in the labeled data324and enables the program code to explore unknown novel anomalous patterns in the unlabeled data322.

As aforementioned, the program code determines if there is domain knowledge that the program code can utilize the classify the workload data (310). If the program code determines that there is domain knowledge, the program code can analyze workload patterns in the original data (temporal input302) utilizing the domain knowledge (315). Based on this analysis, the program code can also assign anomaly scores326to the labeled data324that was labeled based on the domain knowledge. The inputs to this process are the original temporal metrics (e.g., the temporal input302) and the domain knowledge. In performing the workload pattern analysis (315), the program code can map data in the temporal input302to buckets comprising components in the computer system, including but not limited databases and application servers. The program code can identify patterns across the lifecycles of the components of the computer system. To determine an anomaly score, the program code can calculate a distance that represents a divergence between the domain knowledge and incoming patterns (in the temporal input302) against representative patterns for the domain including but not limited to, resource utilization, network traffic, and I/O.FIG.4illustrates that the larger a distance (determined by the program code in this analysis), the larger the anomaly score. InFIG.4, the x-axis represents a distance to the defined patterns and the y-axis represents the anomaly scores.

As illustrated inFIG.3, the program code can generate anomaly scores326for data patterns in the temporal input302both by utilizing machine learning, including, specifically, applying a few-shot anomaly detection algorithm (325), and/or by performing a workload pattern analysis utilizing domain knowledge (315). The program code provides the anomaly scores326to an inference component332. The inference component332is illustrated as a separate element of the program code for illustrative purposes only just as other elements of the program code are illustrated separately for illustrative purposes and not to limit the technical architecture of the program code. For ease of understanding, anomaly score s1can be used to refer to anomaly scores326returned based on the program code applying a data-driving deep reinforcement learning-based few-shot anomaly detection algorithm. Meanwhile, anomaly score s2represents anomaly scores326generated by the program code based on the program code applying domain knowledge (e.g., prior knowledge) and performing a workload pattern analysis utilizing this domain knowledge. Meanwhile, a represents a weight. The inference component332of the program code determines whether each pattern represents an anomaly or not, providing data with binary labels336(e.g., normal or anomaly) as outputs (330). The program code can detect the anomalies utilizing the logic provided below.

Thus, the program code leverages both types of anomaly scores326, s1and s2, comparing the combination to a pre-defined threshold to determine whether to detect an anomaly (330). The program code determines whether a resultant weighted sum represents an anomaly or normal data. The program code classifies data patterns in the temporal input302as normal or an anomaly, generating binary labeled data336. As discussed above, the results provided by the program code provide explainability meaning that based on the detection results, the program code can explore further explainability to characterize the anomalous workloads and to continually enrich the domain knowledge. This, in the examples herein, the program code evaluates a final anomaly score for each pattern that is calculated as a weighted sum of the two sets.

Once the program code has generated the binary labeled data336, the program code interprets these data to provide explainability (335), including but not limited to most outlying metrics, groups of anomalies with similar patterns, etc. To provide explainability (335), the program code can select or weigh features for imbalanced binary classification. In this manner, the program code can expose the classification accuracy of the program code, including of outliers and show the degree of the outliers. The program code can also perform attention-guided feature weighting. The program code can perform a score-and-search approach to find a feature subspace (e.g., perform a greedy search detect the subspace with the largest outlierness degree). The program code can also re-map the binary labeled data to infrastructure metrics, including but not limited to resource utilization, network traffic, and/or I/O. The program code can also provide the results (e.g., binary labeled data) to the domain knowledge (e.g., the anomaly patterns identified by the program code) and to the few-shot algorithm (e.g., the identified normal patterns) to tune these aspects for accuracy in future iterations of the identification process.

In some embodiments of the present invention, the program code identifies the anomaly patterns as indicating non-productive workloads or hotspots. Based on identifying a non-productive workload, the program code can utilize the re-mapping to automatically shut down one or more of the resources that handle this workload. Based on identifying a hotspots, the program code can utilize the re-mapping to migrate this workload and/or portions of this workload to more efficient hardware and/or software resource(s) in the technical environment.

Embodiments of the present invention include computer-implemented method, computer programs products, and computer systems for classifying and resolving anomalous workloads. In some examples, program code executing on one or more processors obtains temporal input from resources comprising a computing system. The program code extracts patterns from the temporal input. The program code determines if domain knowledge is available to assist in identifying anomalous workloads in the temporal input based on the extracted patterns. Based on determining that domain knowledge is available to assist in the identifying, the program code utilizes the domain knowledge to perform a workload pattern analysis to identify a portion of the patterns as indicative of anomalous workloads, labels data comprising the portion of the patterns, and generates a first set of anomaly scores for the labeled data comprising the portion of the patterns. Based on determining that domain knowledge is not available to assist in the identifying, the program code applies one or more unsupervised anomaly detection algorithm to identify an additional portion of the patterns as indicative of the anomalous workloads and labels data comprising the additional portion of the patterns. The program code generates a second set of anomaly scores for the labeled data comprising the portion of the patterns and the additional portion of the pattern and unlabeled data comprising the extracted patterns. The program code determines, for each pattern of the patterns, whether the pattern indicates anomalous data or normal data based on utilizing an inference component to compare one or more anomaly scores for each pattern to a pre-defined threshold. The one or more anomaly scores for each pattern are selected from one or more of the first set of anomaly scores or the second set of anomaly scores.

In some examples, the program code updates the unsupervised anomaly detection algorithm by utilizing patterns of the patterns determined to be anomalous data as training data. The program code can also update the domain knowledge with patterns of the patterns determined to be the normal data.

In some examples, the program code generating the second set of anomaly scores for the labeled data comprises the program code applying a few-shot anomaly detection model.

In some examples, the temporal input comprises service level metrics, infrastructure level metrics, and host-level metrics.

In some examples, the temporal input comprises a multi-channel comprised of system metrics from the resources comprising the computing system.

In some examples, the program code extracting patterns from the temporal input comprises: embedding a multi-channel with the patterns; and filtering out noisy and redundant information across different metrics comprising the temporal output.

In some examples, the patterns comprise temporal patterns and structural patterns in data comprising the temporal output.

In some examples, the one or more unsupervised anomaly detection algorithm comprise ensemble learning.

In some examples, the labeled data identified by applying the one or more unsupervised anomaly detection algorithm comprises data comprising patterns from the patterns classified as anomalous with high confidence by the one or more unsupervised anomaly detection algorithm.

In some examples, the program code trains the few-shot anomaly detection model with the training data.

In some examples, the program code enhances the patterns determined to indicate anomalous data to provide explainability, by doing one or more of the following: selecting or weighing features in the patterns determined to indicate anomalous data for imbalanced binary classification, performing attention-guided feature weighting, and/or re-mapping the patterns determined to indicate anomalous data to infrastructure metrics of the computing system.

In some examples, the program code updates the domain knowledge with the enhanced patterns.

In some examples, based on the determining concluding that a given pattern indicates anomalous data, the program code determines, for the given pattern whether the given pattern is comprised in a non-productive workload or in a hotspot.

In some examples, based on determining that the given pattern is comprised in a non-productive workload, the program code automatically shuts down at least one resource of the computing system that handles the non-productive workload.

In some examples, based on determining that the given pattern is comprised in a hotspot, the program code migrates at least a portion of the hotspot to a new resource in the computing system.

In some examples, generating the anomaly scores comprises: for a portion of the labeled data labeled based on the workload pattern analysis, calculating, a distance that represents a divergence between the extracted patterns against representative patterns for the domain comprising the domain knowledge, and correlating the distance with the anomaly scores; and for portions of the data not labeled based on the workload pattern analysis, applying a few-shot anomaly detection model to generate the anomaly scores.

Although various embodiments are described above, these are only examples. For example, reference architectures of many disciplines may be considered, as well as other knowledge-based types of code repositories, etc., may be considered. Many variations are possible.

Various aspects and embodiments are described herein. Further, many variations are possible without departing from a spirit of aspects of the present invention. It should be noted that, unless otherwise inconsistent, each aspect or feature described and/or claimed herein, and variants thereof, may be combinable with any other aspect or feature.