System and method for anomaly characterization based on joint historical and time-series analysis

One embodiment provides a system for facilitating anomaly detection and characterization. During operation, the system determines, by a computing device, a first set of testing data which includes a plurality of data points, wherein the first set includes a data series for a first variable and one or more second variables. The system identifies anomalies by dividing the first set into a number of groups and performing an inter-quartile range analysis on data in each respective group. The system obtains, from the first set, a second set of testing data which includes a data series from a recent time period occurring before a current time, and which further includes a first data point from the identified anomalies. The system classifies the first data point as a first type of anomaly based on whether a magnitude of a derivative of the second set is greater than a first predetermined threshold.

RELATED APPLICATION

The subject matter of this application is related to the subject matter in the following applications:U.S. patent application Ser. No. 16/143,223, entitled “SYSTEM AND METHOD FOR BINNED INTER-QUARTILE RANGE ANALYSIS IN ANOMALY DETECTION OF TIME-SERIES DATA,” by inventors Ajay Raghavan, Ryan Rossi, and Jungho Park, filed 26 Sep. 2018 (hereinafter “U.S. patent application Ser. No. 16/143,223”),
the disclosure of which is herein incorporated by reference in its entirety.

BACKGROUND

Field

This disclosure is generally related to detecting anomalies. More specifically, this disclosure is related to a system and method for anomaly characterization based on joint historical and time-series analysis.

Related Art

Anomaly detection, or outlier detection, is a data mining method which identifies items or events that do not conform to an expected pattern or others in a data set. Anomaly detection is relevant in various fields, e.g., manufacturing, medical areas, and energy-related fields. For example, in manufacturing, energy usage can incur a significant cost, such as in running a factory with machines or a data center with devices. To this end, identifying anomalies may provide a more efficient manufacturing system, which can in turn reduce costs and improve the overall efficiency of the system as well as its constituent components or parts.

With the advent of smart meters, sensors, and Internet-of-Things (IoT) devices, an opportunity exists to monitor inefficiencies in factory machines and to detect and manage unusual energy usage. Related application Ser. No. 16/143,223 is directed to a method which facilitates anomaly detection by using historical ranges and a binned inter-quartile range analysis on time-series data for a two-dimensional or higher dimensional data set. However, there is currently no way to distinguish between, on the one hand, genuine opportunities for reducing energy or productivity losses, and, on the other hand, uncontrollable power surges from rapid transient events (e.g., a power on/off, sudden temperature changes, and sudden production level increases). This can result in an inefficient system for data mining and anomaly detection.

SUMMARY

One embodiment provides a system for facilitating anomaly detection and characterization. During operation, the system determines, by a computing device, a first set of testing data which includes a plurality of data points, wherein the first set includes a data series for a first variable and one or more second variables, and wherein the one or more second variables are dependent on the first variable. The system identifies anomalies by dividing the first set of testing data into a number of groups and performing an inter-quartile range analysis on data in each respective group. The system obtains, from the first set of testing data, a second set of testing data which includes a data series from a recent time period occurring less than a predetermined period of time before a current time, and which further includes a first data point from the identified anomalies. The system classifies the first data point as a first type of anomaly based on whether a magnitude of a derivative of the second set of testing data is greater than a first predetermined threshold, thereby enhancing data mining and outlier detection for the data series based on a historical analysis of the first set of testing data and a recent time-series analysis of the second set of testing data.

In some embodiments, the first variable or the one or more second variables is one or more of: a parameter associated with a physical device involved in obtaining the first set of testing data. The parameter is one or more of: a temperature value; a count, quantity, or other unit to measure production; a unit or measurement of flow for a material; a unit or measurement of pressure for a material; and any parameter which can be used as a control parameter in measuring another parameter.

In some embodiments, the physical device is one or more of a sensor, a smart meter, an Internet of Things (IoT) device, and any device which can measure the parameter.

In some embodiments, the system classifies the first data point as the first type of anomaly in response to determining that the magnitude of the derivative of the second set of testing data is greater than the first predetermined threshold. The system classifies the first data point as a second type of anomaly in response to determining that the magnitude of the derivative of the second set of testing data is not greater than the first predetermined threshold.

In some embodiments, the first type of anomaly is associated with one or more of: a surge in a control parameter; an on/off event; and a sudden transient change in the control parameter. The second type of anomaly is associated with a steady-state anomaly.

In some embodiments, the system detects a trend which indicates high surges associated with a physical device involved in obtaining the first set of testing data, by determining a third set of data points which are classified as anomalies and are attributable to the surge in the control parameter, the on/off event, or the sudden transient change in the control parameter.

In some embodiments, in response to determining that the trend is not greater than a second predetermined threshold, the system determines that the physical device is experiencing normal operation. In response to determining that the trend is greater than the second predetermined threshold, the system determines that the physical device is experiencing degradation.

In some embodiments, the system classifies, by a user of the computing device, the first data point as the first type of anomaly based on a set of predetermined conditions. The system performs, by the user of the computing device, an action to address the classified anomaly. The action includes one or more of: a remedial or corrective action to ensure that the classified anomaly no longer occurs; adjusting a physical parameter which affects the first variable or the one or more second variables; and adjusting a physical component of a device from which the data series is obtained.

DETAILED DESCRIPTION

Overview

The embodiments described herein solve the problem of identifying and characterizing anomalies in a two-dimensional or higher dimensional data set by providing a system which analyzes both historical data and recent time-series data.

Energy usage can incur a significant overall system cost, such as in running a factory with machines or a data center with many machines. With the advent of smart meters, sensors, and Internet-of-Things (IoT) devices, an opportunity exists to monitor inefficiencies in factory machines and to detect and manage unusual energy usage. Related application Ser. No. 16/143,223 is directed to a method which facilitates anomaly detection by using historical ranges and a binned inter-quartile range analysis on time-series data for a two-dimensional or higher dimensional data set. However, there is currently no way to distinguish between, on the one hand, genuine opportunities for reducing energy or productivity losses, and, on the other hand, uncontrollable power surges from rapid transient events (e.g., a power on/off, sudden temperature changes, and sudden production level increases). This can result in an inefficient system for data mining and anomaly detection.

The embodiments described herein solve this problem by identifying and characterizing anomalies in a two-dimensional or higher dimensional data set, by using historical data/patterns (e.g., obtained from a set of sensors) as well as correlating present energy usage relative to the historical data/patterns (e.g., recent time-series data analysis). For example, the system can combine learned thresholds for various parameter surges in recent time history with the expected energy consumption levels for a given control/response variable associated with a specific machine.

As discussed above, identifying and characterizing anomalies may provide a more efficient system in many areas (e.g., manufacturing, medical fields, and energy-related fields), and can in turn reduce costs and improve the overall efficiency of the system as well as its constituent components or parts. The embodiments described herein provide a computer system which improves the efficiency of detecting and characterizing anomalies in a data series over two or more dimensions or variables by performing a historical analysis (e.g., using the “binned” IQR analysis described in U.S. patent Ser. No. 16/143,223) and by subsequently performing a recent time-series analysis for any identified anomalies.

Furthermore, in the embodiments described herein, the system enhances and improves data mining and outlier detection, where the improvements are fundamentally technological. The improvements can result in a more efficient manufacturing system or other physical system by more effectively identifying and characterizing anomalies. A system administrator or other user can take a remedial or other action based on the identified anomalies to improve the overall efficiency of the manufacturing or other physical system.

Thus, the embodiments described herein provide a technological solution (performing a joint historical and time-series analysis of data over two or more dimensions in order to detect and characterize anomalies) to a technological problem (improving the efficiency of a manufacturing or other physical system by allowing a user to take an action based on the detected and characterized anomalies). For example, the user (or system) may perform an action to remove or address the anomalies, or may perform an action which treats some characterized anomalies as more or less important than other characterized anomalies, e.g., by assigning a weight or priority to certain characterized anomalies.

Exemplary Environment and Communication

FIG. 1illustrates an exemplary environment100for facilitating anomaly detection and characterization, in accordance with an embodiment of the present invention. Environment100can include: a device102and an associated user112; a device104and an associated user114; and a device106. Devices102,104, and106can communicate with each other via a network120. Environment100can also include a physical object (or objects) with sensors which can record data over a period of time and at periodic intervals. For example, environment100can include: a room130which can include devices132.1-132.mand sensors134.1-134.n; and an outside temperature sensor136. Sensors134.1-134.ncan monitor a physical feature associated with the room, such as an amount of heating, ventilation, and air conditioning (HVAC) energy consumed in room130or the amount of HVAC energy consumed or used by a specific device in room130. Outside temperature sensor136can monitor the temperature of the air outside the room, whether inside a same building or outside of the building in which room130resides.

As another example, sensor134.1can measure an amount of HVAC energy consumed (e.g., in Kilowatt hours or kWh) by a specific device132.1(e.g., a compressor machine), while sensor134.2can measure an amount of flow (e.g., in m3) of a material being produced by or traveling through the specific device132.1. As yet another example, sensor134.3can measure the HVAC energy consumed (e.g., in kWh) consumed or used by a specific device132.2(e.g., a potting machine), while sensor134.4can measure a number of units produced (e.g., in units).

During operation, the sensors can send their respective measured testing data to device104. For example, device104can obtain testing data142from sensor134.1, and can also obtain testing data144from sensor136. At the same or a different time, user112can send a command to request anomalies152for certain testing data related to sensors134.1-134.nof room130and outside temperature sensor136, which can result in sending an obtain testing data154command to device104.

Device104can obtain testing data142and144, and can combine testing data142and144. Device104can send combined testing data146to device106via network120. Note that testing data142, testing data144, and combined testing data146can be a data series, which can include time-series data or can cover a frequency spectra. Upon receiving the request anomalies152command along with combined testing data146, device106can perform a historical analysis (function155), which identifies anomalies by dividing combined testing data146into a number of bins and performing an IQR analysis on the data in each bin (function155). Performing a “binned” inter-quartile range (IQR) analysis is described in U.S. patent Ser. No. 16/143,223. During the IQR analysis, device106can identify testing data points from combined testing data146(and specifically, from the testing data points in each divided bin or group of testing data) which are not within a range defined by a lower bound and an upper bound for a respective bin, where the lower and upper bound are determined based on the IQR analysis.

After identifying the anomalies based on the IQR analysis, device106can perform a recent time-series analysis (function156), by obtaining from combined testing data146a second set of testing data which includes a data series from a recent time period. The recent time period can occur less than a predetermined period of time before the current time, and the second set of testing data can include a first data point from the anomalies identified based on the IQR analysis.

As part of function156, device106can calculate a magnitude of a derivative of the second set of testing data, and classify the first data point as a certain type of anomaly based on whether the magnitude of the derivative of the second set of testing data is greater than a first predetermined threshold. The magnitude of the derivative and the first predetermined threshold can be a positive number, a negative number, or zero. Device106can send a (the) classification(s)158of the first data point (and other classified data points) in the second set of testing data back to device102via network120.

Upon receiving classification(s)158, device102can characterize the anomalies (function160), and can also perform an action (function162), which can include executing a predetermined rule based on the classification (as an anomaly) or the characterization (as a certain type of anomaly). For example, if the magnitude of the derivative of the second set of testing data is greater than the first predetermined threshold, device102can characterize the first data point as a surge anomaly, which can be related to a power on/off event, a surge in a control parameter (such as the power or a component which measures pressure), or a sudden transient change in a control parameter. Other types of anomalies may be associated with a steady-state anomaly (e.g., if the magnitude of the derivative of the second set of testing data is less than the first predetermined threshold). Furthermore, the predetermined rule can include automatically adjusting a physical component or unit related to the sensors or devices involved in obtaining the testing data.

Similarly, user112can characterize the anomalies (function164), and can also perform an action (function166), which can include reviewing the classified anomaly in light of other historical data (e.g., learned thresholds for surges in the recent time history and the expected energy consumption levels for a given control or response variable relevant to a given machine). Action166can result in user112remediating or discovering a reason for the classified anomaly, at which point user112can take measures to prevent such anomalies from occurring again.

That is, classification(s)158allows both device102and user112to use the results of: 1) the enhanced data mining and outlier detection based on the IQR analysis performed on the discrete bins or groups; and 2) the subsequent recent time-series analysis. For example, based on the response (i.e., classification(s)158) to request anomalies152command, user112can characterize the identified (classified) anomalies (function164), and perform an action (function166) which can affect and improve the operation and performance of a manufacturing system or other physical system associated with room130and devices132.1-132.min room130. The action can be a remedial or a corrective action to ensure that the classified anomaly no longer occurs. User112can also monitor, observe, and classify subsequent testing data to determine whether the actions of user112have the intended effect, including the removal or deletion of any previously detected anomalies.

Exemplary Anomaly Characterization Based on Joint Historical Analysis and Recent Time-Series Analysis (Potting Machine Power Vs. Count)

FIG. 2Aillustrates a graph200with exemplary testing data, including aggregated snapshots over a period of time for a two-dimensional data set, in accordance with an embodiment of the present invention. Graph200can include a data series (e.g., time-series data) for a potting machine. Graph200can include an x-axis which indicates a potting count (e.g., a number of items produced or potted), and a y-axis which indicates the sum of the power consumed by the potting machine (as measured via multiple sensors in kWh or kW). In graph420, the blue color indicates the two-dimensional testing data (e.g., a “first set of testing data”).

FIG. 2Billustrates a graph210with the exemplary testing data ofFIG. 2A, including performing an inter-quartile range analysis on the testing data which has been divided into bins or groups, in accordance with an embodiment of the present invention. Graph210can include an x-axis which indicates a potting count (e.g., a number of items produced or potted), and a y-axis which indicates the sum of the power consumed by the potting machine (as measured via multiple sensors in kWh or kW). In graph210, the vertical dashed red lines indicate separations between “bins” or groups. That is, the system determines a number n of bins, which can be based on, e.g., an automatic process, an algorithm, a resolution of a sensor or a machine involved in obtaining the data, or historical data or knowledge related to the data and obtained by a computing device or a user. The system can perform an IQR analysis on each bin. In each bin, the solid black lines indicate an upper bound and a lower bound, while the green “+” symbol indicates the median for the respective bin. The blue color in graph210indicates the two-dimensional data which is classified as normal or not classified as an anomaly based on the IQR analysis for the respective bin. The red color in graph210indicates the two-dimensional data which is classified as an anomaly or an outlier based on the IQR analysis for the respective bin.

However, the identified anomalies of graph210inFIG. 2Bdo not indicate the type of anomaly. That is, the system does not characterize the identified anomalies, e.g., as associated with a power surge or a sudden transient change in a control variable.

FIG. 2Cillustrates graphs222and232of data obtained from the exemplary testing data ofFIG. 2A, including time-series data from a recent time period for each variable of the two-dimensional data set and a characterization of an anomaly as associated with a power surge, in accordance with an embodiment of the present invention. Graph222includes data for a three-hour period on 22 Dec. 2016 (e.g., a “second set of testing data”). A measure of a power224is in kW, and appears in a red color (as indicated by a power244in the index). A measure of a count226is in units of items produced, and appears in a blue color (as indicated by a count242in the index). Box228can indicate data corresponding to a classified anomaly from graph210ofFIG. 2B. The user (or system) can perform this recent time-series analysis on the second set of data (in graph222), and characterize the previously detected and classified anomaly based on whether a magnitude of a derivative of the second set of testing data is greater than a first predetermined threshold. For example, by observing that the change in the power with respect to the change in the count is above a certain threshold, the user (or system) can determine that this is an inefficient anomaly associated with a power surge at that particular time (e.g., around 20:00).

Similarly, graph232includes data for a three-hour period on 10 May 2017 (e.g., a “second set of testing data”). A measure of a power234is in kW, and appears in a red color (as indicated by the power244in the index). A measure of a count236is in units of items produced, and appears in a blue color (as indicated by the count242in the index).

FIG. 2Dillustrates a table250of results based on the exemplary testing data ofFIG. 2A, including a characterization or a reason for a particular data point which may be classified as an anomaly, in accordance with an embodiment of the present invention. Table250can include, but is not limited to, columns such as: an index252; a time254; a place256; a machine258; a reason260; and an amount262. In the conventional system, reason260can only be characterized as “normal” or “not normal” (or “not anomalous” or “anomalous”). In contrast, in the embodiments described herein, reason260can include a more specific characterization of the anomaly. For example, entries264and266indicate that the reason for the positive kWh amount is “over usage” rather than “under usage.” Thus, reason260can provide more information to the user, and allow the user to perform actions to remediate or correct such an anomaly.

FIG. 2Eillustrates graphs272and282of data obtained from the exemplary testing data ofFIG. 2A, including time-series data from a recent time period for each variable of the two-dimensional data set and a characterization of an anomaly as associated with a power surge, in accordance with an embodiment of the present invention. Graph272includes data for a 24-hour period on 13 May 2017 (e.g., a “second set of testing data”). A measure of a power274is in kW, and appears in a red color (as indicated by a power294in the index). A measure of a count276is in units of items produced, and appears in a blue color (as indicated by a count292in the index). Box278can indicate data corresponding to a classified anomaly from graph210ofFIG. 2B. The user (or system) can perform this recent time-series analysis on the second set of data (in graph272), and characterize the previously detected and classified anomaly based on whether a magnitude of a derivative of the second set of testing data is greater than a first predetermined threshold. For example, by observing that the change in the power with respect to the change in the count is above a certain threshold, the user (or system) can determine that this is an anomaly associated with a ramp-down in power at that particular time (e.g., around 06:00).

Similarly, graph282includes data for a 24-hour period on 10 Jan. 2017 (e.g., “second set of testing data”). A measure of a power284is in kW, and appears in a red color (as indicated by the power294in the index). A measure of a count286is in units of items produced, and appears in a blue color (as indicated by the count292in the index). Box288can indicate data corresponding to a classified anomaly from graph210ofFIG. 2B. The user (or system) can perform this recent time-series analysis on the second set of data (in graph282), and characterize the previously detected and classified anomaly based on whether a magnitude of a derivative of the second set of testing data is greater than a first predetermined threshold. For example, by observing that the change in the power with respect to the change in the count is above a certain threshold, the user (or system) can determine that this is an anomaly associated with a ramp-up in power at that particular time (e.g., around 06:00).

The user (or system) can subsequently use this data to classify a certain data point as a certain type of anomaly, and can subsequently perform an action to address the classified and characterized anomaly. For example, because both anomalies were characterized as associated with a ramp-down or a ramp-up process, occurring around the same time of day, the user can obtain the appropriate data for a similar time period for other days, particularly for days which are classified as an anomaly in a previous binned IQR analysis (as inFIGS. 2B and 4B). If the data around the 06:00 hour indicates a similar ramp-up or ramp-down (due to a power surge), the user may conclude that some server process or other operation relating to the physical devices in the relevant room may be causing these power-surge related anomalies. The user may then take a perform a remedial or a corrective action to ensure that the classified (and characterized) anomaly no longer occurs. The user can adjust a physical parameter which affects a variable being measured (e.g., the compressor flow or the compressor energy), and can also adjust a physical component of a device from which the data series is obtained (e.g., the compressor or a sensor on the compressor which measures and thus provides the data for the data series). Alternatively, upon characterizing these anomalies as related to power surges, the user may determine that no action need to be taken, e.g., if the power surges occur at expected times or intervals, such as during a planned power cycling or a system reboot.

Method for Facilitating Anomaly Detection Based on Joint Historical Analysis and Recent Time-Series Analysis

FIG. 3Apresents a flow chart300illustrating a method for facilitating anomaly detection and characterization, in accordance with an embodiment of the present invention. During operation, the system determines a data series for a first variable and one or more second variables, y and x, wherein y is the objective for anomaly detection (operation302). It is known that the second data x affects the behavior of the first data y. For the second data x, the system divides the data into n bins (operation304). The system takes the indices of the data in the ith bin (operation306), and passes that into the next operation. For the first data y, the system determines the data with the selected indices (y|x=x1) (operation312). The system obtains a historical analysis on the data with the selected index (i.e., performs an IQR analysis on the data in the ith bin) (operation314). Based on the historical analysis for the selected index (i.e., for the data in the ith bin), the system determines the anomalies in the ith bin (operation316). The system also saves the determined anomalies.

If there are no remaining bins (i.e., the current index i is not less than the number of bins n) (decision318), the operation continues as described below at Label A ofFIG. 3B. If there are remaining bins (i.e., the current index i is less than the number of bins n) (decision318), the system increments i (by setting i to i+1), and the operation returns to operation306.

FIG. 3Bpresents a flow chart330illustrating a method for facilitating anomaly detection and characterization, in accordance with an embodiment of the present invention. The system obtains a recent time-series analysis of the data series for the first variable and the one or more second variables, y and x. If the system does not detect a surge (i.e., if a magnitude of a time derivative is not greater than a first predetermined threshold) (decision334), the system classifies the anomaly as a steady-state anomaly (operation336).

If the system does detect a surge (i.e., if the time derivative is greater than a first predetermined threshold) (decision334), the system classifies the anomaly as a surge anomaly (operation336). If the system does not detect persistent or trending high surges (decision342), the system determines that a machine associated with the data series is experiencing a normal operation (operation344). If the system does detect persistent or trending high surges (decision342), the system determines that a machine associated with the data series is experiencing a degradation (operation346). The system can determine whether it detects persistent or trending high surges based on a physical device involved in obtaining the data series, where the trend is greater than a second predetermined threshold.

Exemplary Anomaly Characterization Based on Joint Historical Analysis and Recent Time-Series Analysis (Compressor Flow Vs. Energy)

FIG. 4Apresents an exemplary historical time-series analysis400for two-dimensional data, in accordance with an embodiment of the present invention. Analysis400includes a graph402and a graph412. Graph402can include an x-axis which indicates time (e.g., marked by one-month intervals), and a y-axis which indicates a measure of the flow of the compressor (in m3). Graph412can include an x-axis which indicates time (e.g., marked by one-month intervals), and a y-axis which indicates a measure of the sum of the energy consumed by the compressor (in kWh). The blue color in graphs402and412indicates the historical time-series data for each dimension (e.g., a “first set of testing data”).

FIG. 4Bpresents an exemplary historical time-series analysis420for the derivative of the two-dimensional data ofFIG. 4A, in accordance with an embodiment of the present invention. Analysis420includes a graph432and a graph442. Graph432indicates the derivative of the time-series data in graph402ofFIG. 4A(i.e., of the compressor flow over time). Graph442indicates the derivative of the time-series data in graph412ofFIG. 4A(i.e., of the compressor energy over time). The blue color in graphs432and442indicates the derivative of the time-series data for each dimension, while the red color indicates the peaks in the derivative. The system can check the peaks by performing an IQR analysis based on the data in graphs432and442.

FIG. 4Cpresents an exemplary recent time-series analysis450for the data obtained from the exemplary data ofFIG. 4B, including a characterization of an anomaly, in accordance with an embodiment of the present invention. Analysis450includes a graph462and a graph472. Graph462indicates data from each of the two dimensions of data ofFIGS. 4A and 4B. Specifically, graph462includes data for a three-hour period on 19 Apr. 2017 (e.g., a “second set of testing data”). A measure of a flowrate466is in m3, and appears in a red color (as indicated by a flowrate484in the index). A measure of an energy464is in kWh, and appears in a blue color (as indicated by an energy482in the index). Similarly, graph472includes data for a three-hour period on 24 Dec. 2016 (e.g., a “second set of testing data”). A measure of a flowrate476is in m3, and appears in a red color (as indicated by the flowrate484in the index). A measure of an energy474is in kWh, and appears in a blue color (as indicated by the energy482in the index).

After checking the peaks of the exemplary data in graphs432and442, the system can identify and further characterize anomalies as associated with a power up/down surge using the time-series data, including, e.g., time-series analyses which show both “efficient” usage (as in graph472) and “inefficient” usage (as in graph462). The surges may be identified by looking at the identified anomalies (e.g., at points463,465, and467) as compared to the historical or expected usage (e.g., at points473,475, and477). Note that analysis450depicts data which is characterized as “efficient” or “inefficient” for the same three-hour window during different days, but can also include data over a different window, period, or interval of time.

Method for Facilitating Anomaly Detection Based on Joint Historical Analysis and Recent Time-Series Analysis

FIG. 5presents a flow chart500illustrating a method for facilitating anomaly detection and characterization, in accordance with an embodiment of the present invention. During operation, the system determines, by a computing device, a first set of testing data which includes a plurality of data points, wherein the first set includes a data series for a first variable and one or more second variables, and wherein the one or more second variables are dependent on the first variable (operation502). The system identifies anomalies by dividing the first set of data into a number of groups and performing an inter-quartile range analysis on data in each respective group (operation504). The system obtains, from the first set of testing data, a second set of testing data which includes a data series from a recent time period occurring less than a predetermined period of time from a current time, and which further includes a first data point from the identified anomalies (operation506). The system classifies the first data point as a first type of anomaly based on whether a magnitude of a derivative of the second set of testing data is greater than a first predetermined threshold, thereby enhancing data mining and outlier detection for the data series based on a historical analysis of the first set of testing data and a recent time-series analysis of the second set of testing data (operation508).

Exemplary Computer and Communication System and Exemplary Apparatus

FIG. 6illustrates an exemplary distributed computer and communication system that facilitates anomaly detection and characterization, in accordance with an embodiment of the present invention. Computer system602includes a processor604, a memory606, and a storage device608. Memory606can include a volatile memory (e.g., RAM) that serves as a managed memory, and can be used to store one or more memory pools. Furthermore, computer system602can be coupled to a display device610, a keyboard616, and a pointing device616. Storage device608can store an operating system616, a content-processing system618, and data632.

Content-processing system618can include instructions, which when executed by computer system602, can cause computer system602to perform methods and/or processes described in this disclosure. Specifically, content-processing system618may include instructions for sending and/or receiving data packets to/from other network nodes across a computer network (communication module620). A data packet can include data, time-series data, a data series, a classification, a request, a command, a testing instance, and a training instance.

Content-processing system618can include instructions for determining, by a computing device, a first set of testing data which includes a plurality of data points, wherein the first set includes a data series for a first variable and one or more second variables, and wherein the one or more second variables are dependent on the first variable (communication module620and data-obtaining module622). Content-processing system618can include instructions for identifying anomalies by dividing the first set of testing data into a number of groups and performing an inter-quartile range analysis on data in each respective group (historical analysis-performing module624). Content-processing system618can also include instructions for obtaining, from the first set of testing data, a second set of testing data which includes a data series from a recent time period occurring less than a predetermined period of time before a current time, and which further includes a first data point from the identified anomalies (recent time-series analysis-performing module626). Content-processing system618can include instructions for classifying the first data point as a first type of anomaly based on whether a magnitude of a derivative of the second set of testing data is greater than a first predetermined threshold, thereby enhancing data mining and outlier detection for the data series based on a historical analysis of the first set of testing data and a recent time-series analysis of the second set of testing data (anomaly-characterizing module628).

Content-processing system618can additionally include instructions for classifying, by a user of the computing device, the first data point as the first type of anomaly based on a set of predetermined conditions (anomaly-characterizing module628). Content-processing system618can include instructions for performing, by the user of the computing device, an action to address the classified anomaly (action-performing module630).

Data632can include any data that is required as input or that is generated as output by the methods and/or processes described in this disclosure. Specifically, data632can store at least: data; a set of testing data; a plurality of data points; a first variable; one or more second variables; time-series data for a first and the one or more second variables; a number of groups; a type of the time-series data; an inter-quartile range; a classification for a testing data point; a testing data point which is classified as an anomaly; a testing data point which is classified as a normal data point; an indicator of an enhanced data mining and outlier detection for time-series data for multiple variables; an indicator of an action; a remedial or corrective action; an anomaly; a classified anomaly; a characterization of an anomaly; a predetermined threshold; a magnitude of a derivative or a derivative of a data point or a set of data; an indicator of a physical parameter which affects the first variable or the second variables; a temperature value; a count, quantity or other unit to measure production; a unit or measurement of flow for a material; a unit or measurement of pressure for a material; a control parameter; any parameter which can be used as a control parameter in measuring another parameter; an indicator of a sensor, a smart meter, an IoT device, and any device which can measure a parameter; a type of an anomaly; a surge in a control parameter; an on/off event; a sudden transient change in a control parameter; a steady-state anomaly; a trend; an indicator of high surges; an indicator that a physical device is experiencing a normal operation; an indicator that a physical device is experiencing a degradation; and an indicator of a physical component of a device from which the data series is obtained.

FIG. 7illustrates an exemplary apparatus that facilitates anomaly detection and characterization, in accordance with an embodiment of the present invention. Apparatus700can comprise a plurality of units or apparatuses which may communicate with one another via a wired, wireless, quantum light, or electrical communication channel. Apparatus700may be realized using one or more integrated circuits, and may include fewer or more units or apparatuses than those shown inFIG. 7. Further, apparatus700may be integrated in a computer system, or realized as a separate device which is capable of communicating with other computer systems and/or devices. Specifically, apparatus700can comprise units700-712which perform functions or operations similar to modules620-630of computer system602ofFIG. 6, including: a communication unit702; a data-obtaining unit704; a historical analysis-performing unit706; a recent time-series analysis-performing unit708; an anomaly-characterizing unit710; and an action-performing unit712.

Furthermore, the methods and processes described above can be included in hardware modules or apparatus. The hardware modules or apparatus can include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute a particular software module or a piece of code at a particular time, and other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them.