SYSTEM AND METHOD FOR DETECTING A SHIFT IN REAL DATA TREND USING THE CONFIGURABLE ADAPTIVE THRESHOLD

A computer-implemented system for detecting shifts in data is provided. The system is configured to: calculate, based on the value of a plurality of user-selectable baseline configuration parameters, baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value; calculate, based on the value of a plurality of user-selectable weighted threshold parameters, a weighted threshold level for the series of data items; detect, based on the value of a plurality of user-selectable shift detection parameters, a shift in the series of data items, wherein the shift comprises an abrupt shift, a rapid drift, or a gradual drift; convert, based on the value of a plurality of user-selectable normalization parameters, the value of each data item in the series of data items to a normalized value; and determine whether the normalized values indicate a data shift.

TECHNICAL FIELD

The present invention generally relates to data analysis, and more particularly relates to systems and methods for detecting a real data trend in a data series.

BACKGROUND

Diagnostically important signals from a batch of machines can be collected. The collected signals may contain meaningful information that is useful for early fault detection or impending fault warning. It may be difficult to separate meaningful information buried in the signals from meaningless signal variance.

Hence, it is desirable to provide systems and methods for detecting a meaningful shift in a real data trend. Furthermore, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

SUMMARY

A computer-implemented system for detecting shifts in data indicating a fault condition is provided. The system includes a baseline calculation module configured to calculate baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value for the series of data items, and wherein the baseline calculation module is configured to calculate the average value and the standard deviation value based on the value of a plurality of user-selectable baseline configuration parameters; a weighted threshold calculation module configured to calculate one or more weighted threshold levels for the series of data items based on the value of a plurality of user-selectable weighted threshold parameters; a shift detection module configured to detect whether a shift in the series of data items exists, wherein the shift includes an abrupt shift, a rapid drift, or a gradual drift, and wherein the shift detection module is configured by the value of a plurality of user-selectable shift detection parameters to detect the shift; and a normalization module configured to convert the value of each data item in the series of data items to a normalized value, wherein the normalization module is configured by the value of a plurality of user-selectable normalization parameters to convert each data item to its normalized value, and wherein the normalization module configured to determine whether the normalized values indicate a data shift.

A method for detecting shifts in data is provided. The method includes: calculating baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value for the series of data items, and wherein the calculating baseline values is performed based on the value of a plurality of user-selectable baseline configuration parameters; calculating a weighted threshold level for the series of data items, wherein the calculating the weighted threshold level is performed based on the value of a plurality of user-selectable weighted threshold parameters; detecting whether a shift in the series of data items exists, wherein the shift includes an abrupt shift, a rapid drift, or a gradual drift, and wherein the detecting is performed based on the value of a plurality of user-selectable shift detection parameters; converting the value of each data item in the series of data items to a normalized value, wherein the converting is performed based on the value of a plurality of user-selectable normalization parameters; and determining whether the normalized values indicate a data shift.

A computer-implemented system for detecting shifts in data is provided. The system includes one or more processors configured by programming instructions on non-transient computer readable media. The system is configured to: calculate, based on the value of a plurality of user-selectable baseline configuration parameters, baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value for the series of data items; calculate, based on the value of a plurality of user-selectable weighted threshold parameters, a weighted threshold level for the series of data items; detect, based on the value of a plurality of user-selectable shift detection parameters, a shift in the series of data items, wherein the shift includes an abrupt shift, a rapid drift, or a gradual drift; convert, based on the value of a plurality of user-selectable normalization parameters, the value of each data item in the series of data items to a normalized value; and determine whether the normalized values indicate a data shift.

Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the preceding background.

DETAILED DESCRIPTION

The following detailed description is merely exemplary in nature and is not intended to limit the application and uses. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, summary, or the following detailed description. As used herein, the term “module” refers to any hardware, software, firmware, electronic control component, processing logic, and/or processor device, individually or in any combination, including without limitation: application specific integrated circuit (ASIC), a field-programmable gate-array (FPGA), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.

FIG. 1is a block diagram depicting an example shift detection system102that is configured to detect changes in input data items103that may indicate that a fault condition has occurred. The example shift detection system102is configured to detect a type of shift in the series of data items103, wherein the type of shift may include an abrupt shift, a rapid drift, and a gradual drift as illustrated, respectively, inFIGS. 2A, 2B, and 2C.FIG. 2Ais a diagram depicting a plot200of an example series of data items201. The example plot200illustrates an abrupt shift202in the data items201.FIG. 2Bis a diagram depicting a plot210of another example series of data items211. The example plot210illustrates a gradual drift212in the data items211.FIG. 2Cis a diagram depicting a plot220of another example series of data items221. The example plot220illustrates a rapid drift222in the data items221.

The example shift detection system102includes a baseline calculation module104, a weighted threshold calculation module106, a normalization module108, and a shift detection module110. The shift detecting methods employed by the example shift detection system102are configurable. The example shift detection system102is configured to accept user-selectable parameters and, based on the values of the user-selectable parameters, configured the employed shift detection methods. The example shift detection system102is configured to simultaneously employ multiple shift detection methods.

The example shift detection system102may be implemented by a controller. The controller includes at least one processor and a computer-readable storage device or media encoded with programming instructions for configuring the controller. The processor may be any custom-made or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chip set), any combination thereof, or generally any device for executing instructions.

The computer readable storage device or media may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the processor is powered down. The computer-readable storage device or media may be implemented using any of a number of known memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or any other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable programming instructions, used by the controller.

The example baseline calculation module104is configured to calculate a baseline for a series of data items103in a data structure. The baseline includes an average value and a standard deviation value for the series of data items. The average value calculated by the example baseline calculation module104may be a median value or a mean value. The value of a user-selectable baseline configuration parameter105(e.g., an average calculation method parameter) may be used by the example baseline calculation module104to determine the type of value (e.g., median value or mean value) that is used for the calculation of the average value. The example baseline calculation module104may be configured to default to a specific type of value if one is not specified by a user-selectable parameter105.

Additional user-selectable baseline configuration parameters105may be used to determine how the example baseline calculation module104calculates the average value and the standard deviation value. In addition to the average calculation method parameter, user-selectable baseline configuration parameters105may include: a domain parameter, which indicates whether the baseline values (e.g., average and sigma) are calculated based on a certain number of points or over a time period; a maximum baseline size parameter, which indicates a maximum number of points used for the calculation of baseline parameters (e.g., 100); a buffer size parameter, which indicates the number (e.g., 20) of the most recent points that are included in a buffer and not included in baseline calculation; an outlier usage parameter (e.g., with a value of enabled or disabled), which indicates whether outliers should be included or excluded from baseline calculations; and an outlier sigma parameter, which indicates a sigma level (e.g., 6) beyond which outliers shall be filtered out when calculating baseline values.

The weighted threshold calculation module106is configured to calculate one or more weighted threshold levels for the series of data items103based on user-selectable weighted threshold parameters107. The weighted threshold calculation module106is configured to calculate weighted thresholds that are based on a weighting factor that determines the influence of fixed thresholds and adaptive thresholds on the weighted thresholds. As an example, a weighting factor of 1 may indicate that the weighted threshold is equal to the fixed threshold, a weighting factor of 0 may indicate that the weighted threshold is equal to the adaptive threshold, and a weighting factor of 0.5 may indicate that the weighted threshold is half influenced by the fixed threshold and half influenced by the adaptive threshold.

The example weighted threshold calculation module106is configured to use a number of user-selectable weighted threshold configuration parameters107to determine how the example weighted threshold calculation module106calculates weighted thresholds. The user-selectable weighted threshold parameters107may include: a threshold type parameter, which indicates the type of weighted threshold (e.g., absolute or delta); a lower absolute threshold parameter, which indicates the value of the lower absolute threshold; a higher absolute threshold parameter, which indicates the value of the higher absolute threshold; delta threshold from mean parameter, which indicates the value of delta threshold from mean; transition shape parameter, which indicates the shape (e.g., step or ramp) of the transition from fixed threshold to weighted threshold; data before transition parameter, which indicates the amount of data before starting transition from fixed to weighted threshold; transition length parameter, which indicates the amount of data for the transition from fixed to weighted threshold; and a weighting factor parameter, which indicates the weight of fixed threshold (e.g., 0-0.99) in final weighted threshold.

The example normalization module108is configured to convert the value of each data item in the series of data items103to a normalized value. The example normalization module108is configured to use a number of user-selectable normalization configuration parameters109to determine how the example normalization module108calculates normalized values. The user-selectable normalization configuration parameters109may include: a normalization select parameter, which indicates whether the normalization function is chosen for operation (e.g., on/off); a normalization floor parameter, which indicates the sigma level for base (e.g., 1 sigma); a normalization normal ceiling parameter, which indicates, a sigma level for a normal upper boundary (e.g., 2 sigma); a normalization abnormal cap parameter, which indicates the sigma level for an abnormal boundary (e.g., 3 sigma); and a normalization ceiling for normal variation parameter, which indicates a value of a normal ceiling (e.g., 1).

The example shift detection module110is configured to detect a type of shift in the series of data items, wherein the type of shift may include an abrupt shift, a rapid drift, and/or a gradual drift. The example shift detection module110is configured to use a number of user-selectable shift detection configuration parameters111to determine how the example shift detection module110detects data shifts. The user-selectable shift detection configuration parameters may include: a shift detection method selection parameter, which indicates the method for shift detection (e.g., Western Electric rules or movement in smoothed value); and a shift detection direction selection parameter, which indicates the specific shift direction for which to detect (e.g., up, down, or either direction).

When the Western Electric (WE) rules shift detection method is selected, the user-selectable shift detection configuration parameters111may further include: a WE rule selection parameter, which indicates the specific Western Electric (WE) rules to be applied; a WE window size parameter, which indicates the size of a window for WE rule evaluation; a WE number of exceptional points parameter, which indicates the number of points that shall exceed the limit in the WE rule to trigger the finding of a fault; a boundary sigma level parameter, which indicates a boundary sigma level that if exceeded triggers the finding of a fault; and a cumulative sigma level parameter, which indicates the sum of sigma levels that if exceeded during a window triggers the finding of a fault. When the smoothed value shift detection method is selected, the user-selectable shift detection configuration parameters111may further include: a smoothed window parameter, which indicates the number of points averaged when calculating the smoothed value.

The example shift detection system102is configured to output data113that indicates whether abnormal shifts were detected in the data items103. The output data113may include normalized abnormality indicator data115, which indicates whether an abnormality was detected from normalized data in the normalization module108. The output data113may also include shift detection indicator data117, which indicates whether a data shift was detected by the shift detection module110.

In one example implementation of the example shift detection system102, the input data items103may comprise mechanical systems condition indicator (CI) data for a vehicle such as a helicopter or aircraft. The CI data103may be analyzed using the example shift detection system102to determine the health of a number of mechanical components in the vehicle such as a gearbox, bearings, and other components. The CI data103may be input into the example shift detection system102and analyzed for data shifts. The output data113from the example shift detection system102, in this example, may comprise vehicle health indicator data in the form of normalized abnormality indicator data115and/or shift detection indicator data117.

FIG. 3is a process flow chart depicting an example process300in a shift detection system for identifying faults indicated by a data shift. The order of operation within the example process300is not limited to the sequential execution as illustrated in the figure, but may be performed in one or more varying orders as applicable and in accordance with the present disclosure.

The example process300includes computing average and standard deviation (e.g., sigma) values for a data item series (operation302). A number of options are available for computing the average and standard deviation values. The options chosen for computation may be determined from user-selectable baseline configuration parameters. The user-selectable baseline configuration parameters may include: an average calculation method parameter, which indicates a method (e.g., median value or mean value) for calculating an average value; a domain parameter, which indicates whether the baseline values (e.g., average and sigma) are calculated based on a certain number of points or over a time period; a maximum baseline size parameter, which indicates a maximum number of points used for the calculation of baseline parameters (e.g., 100); a buffer size parameter, which indicates the number (e.g., 20) of the most recent points that are included in a buffer and not included in the baseline calculation; an outlier usage parameter (e.g., with a value of enabled or disabled), which indicates whether outliers should be included or excluded from baseline calculations; and an outlier sigma parameter, which indicates a sigma level (e.g., 6) beyond which outliers shall be filtered out when calculating baseline values.

The example process300includes calculating a weighted threshold based on the computed average, standard deviation (e.g., sigma), and data trend (operation304). The weighted threshold may be calculated based on a weighting factor that determines the influence of fixed thresholds and adaptive thresholds on the weighted thresholds. As an example, a weighting factor of 1 may indicate that the weighted threshold is equal to the fixed threshold, a weighting factor of 0 may indicate that the weighted threshold is equal to the adaptive threshold, and a weighting factor of 0.5 may indicate that the weighted threshold is half influenced by the fixed threshold and half influenced by the adaptive threshold.

A number of options are available for computing the weighted threshold. The options chosen for computation may be determined from user-selectable weighted threshold parameters. The user-selectable weighted threshold parameters may include: a threshold type parameter, which indicates the type of weighted threshold (e.g., absolute or delta); a lower absolute threshold parameter, which indicates the value of the lower absolute threshold; a higher absolute threshold parameter, which indicates the value of the higher absolute threshold; delta threshold from mean parameter, which indicates the value of delta threshold from mean; transition shape parameter, which indicates the shape (e.g., step or ramp) of the transition from fixed threshold to weighted threshold; data before transition parameter, which indicates the amount of data before starting transition from fixed to weighted threshold; transition length parameter, which indicates the amount of data for the transition from fixed to weighted threshold; and a weighting factor parameter, which indicates the weight of fixed threshold (e.g., 0-0.99) in final weighted threshold.

The example process300includes determining, based on the weighted threshold, whether a shift in a data series has occurred (operation306). A number of options are available for determining if a shift in a data series has occurred. The options may be determined from user-selectable shift detection configuration parameters. The user-selectable shift detection configuration parameters may include: a shift detection method selection parameter, which indicates the method for shift detection (e.g., Western Electric rules or movement in smoothed value); and a shift detection direction selection parameter, which indicates the specific shift direction for which to detect (e.g., up, down, or either direction).

When the Western Electric (WE) rules shift detection method is selected, the user-selectable shift detection configuration parameters may further include: a WE rule selection parameter, which indicates the specific Western Electric (WE) rules to be applied; a WE window size parameter, which indicates the size of window for WE rule evaluation; a WE number of exceptional points parameter, which indicates the number of points that shall exceed the limit in WE rule to trigger the finding of a fault; a boundary sigma level parameter, which indicates a boundary sigma level that if exceeded triggers the finding of a fault; and a cumulative sigma level parameter, which indicates the sum of sigma levels that if exceeded during a window triggers the finding of a fault. When the smoothed value shift detection method is selected, the user-selectable shift detection configuration parameters may further include: a smoothed window parameter, which indicates the number of points averaged when calculating the smoothed value.

The example process300includes outputting shift detection information (operation308). The shift detection information indicates whether a data shift was detected. The types of data shifts that may be detected include an abrupt shift, a rapid drift, and a gradual drift.

The example process300includes normalizing data between 0 and 1 (operation310). The value of each data item in an input series of data items can be converted to a normalized value. A number of options are available for normalizing the data series. The options may be determined from user-selectable normalization configuration parameters. The user-selectable normalization configuration parameters may include: a normalization select parameter, which indicates whether the normalization function is chosen for operation (e.g., on/off); a normalization floor parameter, which indicates the sigma level for base (e.g., 1 sigma); a normalization normal ceiling parameter, which indicates, sigma level for normal upper boundary (e.g., 2 sigma); a normalization abnormal cap parameter, which indicates the sigma level for abnormal boundary (e.g., 3 sigma); and a normalization ceiling for normal variation parameter, which indicates a value of normal ceiling (e.g., 1).

The example process300includes outputting normalized abnormality level information (operation312). The normalized abnormality level information indicates whether an abnormality was detected from the normalized data.

FIG. 4is a process flow chart depicting an example process400in a shift detection system for calculating weighted thresholds. The order of operation within the example process400is not limited to the sequential execution as illustrated in the figure, but may be performed in one or more varying orders as applicable and in accordance with the present disclosure.

The example process400includes determining if the length of historical data is greater than the amount of data before starting a transition from a fixed threshold to a weighted threshold (decision402). A calculated baseline401(e.g., average and sigma) for an input data series is used in the example process400. Information403regarding the amount of data before starting the transition may be user-provided for the decision process (e.g., via configuration parameters).

When it is determined that the length of historical data is not greater than the amount of data before starting transition (no at decision402), then a fixed threshold is used (operation404). Information405regarding fixed thresholds and type of fixed thresholds (e.g., absolute or delta) may be user-provided for the decision process (e.g., via configuration parameters). When it is determined that the length of historical data is greater than the amount of data before starting transition (yes at decision402), then a 3-sigma threshold is computed (operation406).

The example process400includes determining if a step or ramp is to be used for a transition from fixed to weighted thresholds (decision408). A transition shape parameter value407(e.g., step or ramp) may be used in the determination.

When it is determined that the step transition is to be used, the example process includes computing/using the final weighted threshold (operation410). A configurable fixed/adaptive weighting factor409is provided to compute the final weighted threshold. The average, sigma/weighted sigma, weighted threshold411may be output from the example process400.

When it is determined that the ramp transition is to be used, the example process includes determining if the length of historical data is greater than the amount of data before starting the transition plus the length of transition from fixed to adaptive (decision412).

When it is determined that the length of historical data is greater than the amount of data before starting the transition plus the length of transition from fixed to adaptive (yes at decision412), then the example process includes computing/using the final weighted threshold (operation410). The configurable fixed/adaptive weighting factor409is provided to compute the final weighted threshold. The average, sigma/weighted sigma, weighted threshold411may be output from the example process400.

When it is determined that the length of historical data is not greater than the amount of data before starting the transition plus length of transition from fixed to adaptive (no at decision412), then the example process400includes computing/using a transition weighted threshold (operation414). The to compute fixed/adaptive weighting factor409is used to compute a transition threshold.

FIG. 5is a process flow chart depicting an example process500in a shift detection system for detecting a shift. The order of operation within the example process500is not limited to the sequential execution as illustrated in the figure, but may be performed in one or more varying orders as applicable and in accordance with the present disclosure.

The example process500includes determining whether to assess an input data series using Western Electric Rules or movement in smoothed values (decision502). One or more weighted thresholds (501) are utilized in the example process500. User-selectable shift detection parameters (503), as discussed with reference toFIG. 3, may be evaluated in the determining decision.

If it is determined that the input data series will be assessed using Western electric rules at decision502, then the example process500includes determining if instantaneous data series values meet Western Electric rules (operation504). User-selectable Western Electric parameters (505), as discussed with reference toFIG. 3, may be evaluated to determine if instantaneous data series values meet Western Electric rules. Based on the determination, a shift or no shift indication (509) may be provided.

If it is determined that to assess the input data series using smoothed values at decision502, then the example process500includes computing a smoothed value from the input data series (operation506). User-selectable smoothed parameters (507), as discussed with reference toFIG. 3, may be evaluated to determine how to compute a smoothed value from the input data series.

The example process500includes determining if the smoothed value moved beyond weighted thresholds (operation508). Based on the determination, a shift or no shift indication (509) may be provided.

FIGS. 6A and 6Bare diagrams depicting a plot600/620of a historical trend of data items602/622and illustrating an example use of a weighted threshold604/624for the detection of a shift in the historical trend of data items602/622.FIGS. 6A and 6Billustrate that a threshold for use in shift detection can be divided into three phases: a fixed threshold phase606/626, a transition phase608/628, and a weighted threshold phase604/624. When fewer data items are available, the data items may be assessed using a fixed threshold606/626. When sufficient data items are available, the data items may be assessed using a weighted threshold604/624. A transition period may be used when switching from the use of a fixed threshold to a weighted threshold.

After average612/632and sigma baseline values have been calculated, they can be combined to generate an adaptive threshold610/630. The adaptive threshold610/630can be combined with the fixed threshold606/626using a weighting factor to generate a weighted threshold604/624. The weighted threshold604/624can then be used as the primary value to compare against when detecting a shift. The weighting factor (WF) defines the relative influence of the fixed threshold606/626compared to that of the calculated adaptive threshold610/630. If the WF equals the value 1, then the fixed threshold606/626is used. If the WF equals the value 0, then the threshold is driven entirely by the adaptive threshold610/630. The weighted threshold604/624can be determined by the following formula: Weighted threshold=(WF×Fixed threshold value)+((1−WF)×adaptive threshold value).

FIG. 6Cis a diagram depicting an example relationship between an example weighted sigma value652and a traditional sigma value654. A traditional sigma value654may be determined from a standard deviation calculation. A weighted sigma value652may be determined by a weighted threshold value656minus a mean value658with the result divided by three. The weighted sigma652may be more appropriate in some applications for shift detection than a traditional sigma value654.

FIG. 7Ais a diagram depicting an example plot702of an example normalized output data series704after normalization of a series of data items. The example normalized output data series704results from a type of piece-wise scaling based on an average value705and a sigma/weighted sigma value for the series of data items.FIG. 7Bis a process flow chart depicting an example normalization process720for generating the example normalized output data704. The inputs to the example process720include raw data values from the series of data items, an average value705for the series of data items, and a sigma or weighted sigma value for the series of data items.

The example process720includes computing three threshold levels: normal floor threshold level706, normal ceiling threshold level708, and abnormal cap threshold level710(operation722). The example normal floor threshold level=mean+std*nsigma_for_floor; the example normal ceiling threshold level=mean+std*nsigma_for_Normal_Ceiling; and the example abnormal cap threshold level=mean+std*nsigma_for_Abnormal_Cap, wherein mean is the average value for the series of data items, std is the standard deviation (or sigma) for the series of data items, nsima_for_floor is a user-selectable parameter value that indicates the sigma level for base, nsigma_for_Normal_Ceiling is a user-selectable parameter value that indicates the sigma level for a normal upper boundary, and nsigma_for_Abnormal_Cap is a user-selectable parameter value that indicates the sigma level for an abnormal boundary.

The example process720includes mapping the raw value of the input data items to normalized values (operation724). If the raw value is less than the normal floor threshold level, then the raw value is mapped to a minimum value (e.g., 0.001) (operation726). If the raw value is between the normal floor threshold level and the normal ceiling threshold level, then the raw value is scaled linearly between the minimum value and a ceiling for normal variation value (e.g., 0.2) (operation728). If the raw value is between the normal ceiling threshold level and the abnormal cap threshold level, then the raw value is scaled linearly between the ceiling for normal variation value and the value 1 (operation730). If the raw value is greater than the abnormal cap threshold level, then the raw value is mapped to the value 1 (operation732).

Apparatus, systems, methods, and techniques are described for a configurable tool that can make data analysis and fault troubleshooting more consistent and efficient. Described apparatus, systems, methods, and techniques can provide a shift detection tool that is configurable to detect multiple types of data shifts using varying techniques.

In one embodiment, a computer-implemented system for detecting shifts in data indicating a fault condition is provided. The system comprises a baseline calculation module configured to calculate baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value for the series of data items, and wherein the baseline calculation module is configured to calculate the average value and the standard deviation value based on the value of a plurality of user-selectable baseline configuration parameters; a weighted threshold calculation module configured to calculate one or more weighted threshold levels for the series of data items based on the value of a plurality of user-selectable weighted threshold parameters; a shift detection module configured to detect whether a shift in the series of data items exists, wherein the shift comprises an abrupt shift, a rapid drift, or a gradual drift, and wherein the shift detection module is configured by the value of a plurality of user-selectable shift detection parameters to detect the shift; and a normalization module configured to convert the value of each data item in the series of data items to a normalized value, wherein the normalization module is configured by the value of a plurality of user-selectable normalization parameters to convert each data item to its normalized value, and wherein the normalization module configured to determine whether the normalized values indicate a data shift.

In one example, the baseline configuration parameters include a plurality of: an average calculation method parameter, which indicates a method for calculating an average value; a domain parameter, which indicates whether the baseline values are calculated based on a certain number of points or over a time period; a maximum baseline size parameter, which indicates a maximum number of points used for the calculation of baseline parameters; a buffer size parameter, which indicates a number of the most recent points that are included in a buffer and not included in baseline calculations; an outlier usage parameter, which indicates whether outliers should be included or excluded from baseline calculations; and an outlier sigma parameter, which indicates a sigma level beyond which outliers are filtered out when calculating baseline values.

In one example, the user-selectable weighted threshold parameters include a plurality of: a threshold type parameter, which indicates a type of weighted threshold; a lower absolute threshold parameter, which indicates the value of a lower absolute threshold; a higher absolute threshold parameter, which indicates the value of a higher absolute threshold; a delta threshold from mean parameter, which indicates the value of a delta threshold from mean; a transition shape parameter, which indicates the shape of a transition from a fixed threshold to a weighted threshold; a data before transition parameter, which indicates an amount of data before starting a transition from a fixed to a weighted threshold; a transition length parameter, which indicates an amount of data for a transition from a fixed to a weighted threshold; and a weighting factor parameter, which indicates a weight of a fixed threshold in the weighted threshold.

In one example, the user-selectable shift detection parameters include: a shift detection method selection parameter, which indicates a method for shift detection; and a shift detection direction selection parameter, which indicates a specific shift direction for which to detect.

In one example, the user-selectable shift detection parameters further include a plurality of: a Western Electric (WE) rule selection parameter, which indicates the specific WE rules to be applied; a WE window size parameter, which indicates the size of window for WE rule evaluation; a WE number of exceptional points parameter, which indicates the number of points that has to exceed limits in WE rules to trigger the finding of a fault; a boundary sigma level parameter, which indicates a boundary sigma level that if exceeded triggers the finding of a fault; and a cumulative sigma level parameter, which indicates the sum of sigma levels that if exceeded during a window triggers the finding of a fault.

In one example, the user-selectable shift detection parameters further include a smoothed window parameter, which indicates the number of points averaged when calculating the smoothed value.

In one example, the user-selectable normalization parameters include a plurality of: a normalization select parameter, which indicates whether the normalization function is chosen for operation; a normalization floor parameter, which indicates the sigma level for a base; a normalization normal ceiling parameter, which indicates, sigma level for a normal upper boundary; a normalization abnormal cap parameter, which indicates the sigma level for an abnormal boundary; and a normalization ceiling for normal variation parameter, which indicates a value of a normal ceiling.

In one example, the baseline calculation module is configured to calculate the baseline values by using a moving window of data items wherein the size of the moving window is determined by user-selectable parameter values.

In one example, the baseline calculation module is configured to calculate the baseline values by using a buffer to exclude the most recent data items from the calculation wherein the size of the buffer is determined by user-selectable parameter values.

In one example, the weighted threshold calculation module is configured to calculate weighted threshold levels by applying a first weighting factor to a fixed threshold and a second weighting factor to an adaptive threshold, wherein the fixed threshold is determined by user-selectable parameter values and wherein the adaptive threshold is calculated based on the calculated average and standard deviation of the data series.

In one example, the weighted threshold calculation module is configured to transition from a fixed threshold to the weighted threshold using a ramp transition when user-selectable parameters indicate that the ramp transition be used and using a step transition when user-selectable parameters indicate that the step transition be used.

In one example, the shift detection module is configured to apply Western Electric rules to detect a shift when a user-selectable parameter value indicates application of the Western Electric rules.

In one example, the shift detection module is configured to apply a smoothing window to detect a shift when a user-selectable parameter value indicates application of the smoothing window.

In one example, to calculate the normalized value, the normalization module is configured to: compute a normal floor threshold level, a normal ceiling threshold level, and an abnormal cap threshold level; and map raw values of input data items to normalized values, wherein to map the raw values to normalized values, the normalization module is further configured to: map the raw value to a minimum value near zero when the raw value is less than the normal floor threshold level; linearly scale the raw value between the minimum value and a ceiling for normal variation value when the raw value is between the normal floor threshold level and the normal ceiling threshold level; linearly scale the raw value between the ceiling for normal variation value and the value 1 when the raw value is between the normal ceiling threshold level and the abnormal cap threshold level; and map the raw value to the value 1 when the raw value is greater than the abnormal cap threshold level.

In another embodiment, a method for detecting shifts in data is provided. The method comprises: calculating baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value for the series of data items, and wherein the calculating baseline values is performed based on the value of a plurality of user-selectable baseline configuration parameters; calculating a weighted threshold level for the series of data items, wherein the calculating the weighted threshold level is performed based on the value of a plurality of user-selectable weighted threshold parameters; detecting whether a shift in the series of data items exists, wherein the shift comprises an abrupt shift, a rapid drift, or a gradual drift, and wherein the detecting is performed based on the value of a plurality of user-selectable shift detection parameters; converting the value of each data item in the series of data items to a normalized value, wherein the converting is performed based on the value of a plurality of user-selectable normalization parameters; and determining whether the normalized values indicate a data shift.

In another embodiment, a computer-implemented system for detecting shifts in data is provided. The system comprises one or more processors configured by programming instructions on non-transient computer readable media. The system is configured to: calculate, based on the value of a plurality of user-selectable baseline configuration parameters, baseline values for a series of data items in a data structure, wherein the baseline values include an average value and a standard deviation value for the series of data items; calculate, based on the value of a plurality of user-selectable weighted threshold parameters, a weighted threshold level for the series of data items; detect, based on the value of a plurality of user-selectable shift detection parameters, a shift in the series of data items, wherein the shift comprises an abrupt shift, a rapid drift, or a gradual drift; convert, based on the value of a plurality of user-selectable normalization parameters, the value of each data item in the series of data items to a normalized value; and determine whether the normalized values indicate a data shift.

In one example, the system is further configured to: calculate the baseline values by using a moving window of data items wherein the size of the moving window is determined by user-selectable parameter values; and calculate the baseline values by using a buffer to exclude the most recent data items from the calculation wherein the size of the buffer is determined by user-selectable parameter values.

In one example, the system is further configured to: calculate weighted threshold levels by apply a first weighting factor to a fixed threshold and a second weighting factor to an adaptive threshold, wherein the fixed threshold is determined by user-selectable parameter values and wherein the adaptive threshold is calculated based on the calculated average and standard deviation of the data series; transition from a fixed threshold to the weighted threshold using a ramp transition when user-selectable parameters indicate that the ramp transition be used; and transition from a fixed threshold to the weighted threshold using a step transition when user-selectable parameters indicate that the step transition be used.

In one example, the system is further configured to: apply Western Electric rules to detect a shift when a user-selectable parameter value indicates application of the Western Electric rules; and apply smoothing to detect a shift when a user-selectable parameter value indicates application of smoothing.

In one example, to calculate the normalized value, the system is further configured to: compute a normal floor threshold level, a normal ceiling threshold level, and an abnormal cap threshold level; and map raw values of input data items to normalized values, wherein to map the raw values to normalized values, the system is further configured to: map the raw value to a minimum value near zero when the raw value is less than the normal floor threshold level; linearly scale the raw value between the minimum value and a ceiling for normal variation value when the raw value is between the normal floor threshold level and the normal ceiling threshold level; linearly scale the raw value between the ceiling for normal variation value and the value 1 when the raw value is between the normal ceiling threshold level and the abnormal cap threshold level; and map the raw value to the value 1 when the raw value is greater than the abnormal cap threshold level.