Computer-implemented methods, computer program products, and systems are provided for multi-step ahead forecasting. A method includes configuring, by a processor device, a Vector Autoregression (VAR) model to generate a multi-step-ahead forecast based on previous observations. The previous observations are predictors and the multi-step-ahead forecast is a response to the predictors. The method further includes training, by the processor device, the VAR model using complex-valued weight parameters to avoid a training result relating to any of a divergence and a convergence to zero.

BACKGROUND

Technical Field

The present invention generally relates to data processing, and more particularly to multi-step forecasting using complex-valued vector autoregression.

Description of the Related Art

A Vector Autoregression (VAR) is an equation, n-variable linear model in which each variable is in turn explained by its own lagged values, plus current and past values of the remaining n−1 variables. That is, Vector Autoregression (VAR(p)) is a time series model that provides a one-step-ahead forecast based on the previous p observations by learning a regression model where the past observations are predictors and the forecast is the response.

There are two main approaches for N-step-ahead forecast based on VAR models are as follows: recursive; and direct. In the recursive approach, for each future time step, the VAR model forecasts one-step-ahead based on previous forecasts and observations. In the direct approach, multiple VARs are used, where the nth VAR forecasts the Nth-step-ahead.

However, while preferable over the direct method in certain cases, the recursive model is not without deficiency. For example, regarding the recursive approach, when N is large, the recursive approach cannot make accurate forecasts because the recursive prediction results in either divergence or convergence to zero. Accordingly, there is a need for an improved VAR-based recursive approach for multi-step forecasting.

SUMMARY

According to an aspect of the present invention, a computer-implemented method is provided for multi-step ahead forecasting. The method includes configuring, by a processor device, a Vector Autoregression (VAR) model to generate a multi-step-ahead forecast based on previous observations. The previous observations are predictors and the multi-step-ahead forecast is a response to the predictors. The method further includes training, by the processor device, the VAR model using complex-valued weight parameters to avoid a training result relating to any of a divergence and a convergence to zero.

According to another aspect of the present invention, a computer program product is provided for multi-step ahead forecasting. The computer program product includes a non-transitory computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a computer to cause the computer to perform a method. The method includes configuring, by a processor device, a Vector Autoregression (VAR) model to generate a multi-step-ahead forecast based on previous observations. The previous observations are predictors and the multi-step-ahead forecast is a response to the predictors. The method further includes training, by the processor device, the VAR model using complex-valued weight parameters to avoid a training result relating to any of a divergence and a convergence to zero.

According to yet another aspect of the present invention, a computer processing system is provided for multi-step ahead forecasting. The system includes a memory device for storing program code. The system further includes a processor device for running the program code to configure a Vector Autoregression (VAR) model to generate a multi-step-ahead forecast based on previous observations. The previous observations are predictors and the multi-step-ahead forecast is a response to the predictors. The processor device further runs the program code to train the VAR model using complex-valued weight parameters to avoid a training result relating to any of a divergence and a convergence to zero.

According to still another aspect of the present invention, a computer-implemented method is provided for multi-step-ahead forecasting. The method includes configuring, by a processor device, a Vector Autoregression (VAR) model to generate a multi-step-ahead forecast based on previous observations. The previous observations are predictors and the multi-step ahead forecast is the response to the predictors. The method further includes learning, by the processor device, the VAR model using a multi-step cumulative error as an objective function of the VAR model.

According to a further aspect of the present invention, a computer processing system is provided for multi-step-ahead forecasting. The system includes a memory for storing program code. The system further includes a processor device for running the program code to configure a Vector Autoregression (VAR) model to generate a multi-step-ahead forecast based on previous observations. The previous observations are predictors and the multi-step ahead forecast is the response to the predictors. The processor device further runs the program code to learn the VAR model using a multi-step cumulative error as an objective function of the VAR model.

DETAILED DESCRIPTION

The present invention is directed to multi-step forecasting using complex-valued Vector Autoregression (VAR). The present invention is directed to recursive types of VARs, and overcomes at least the aforementioned deficiency associated with VAR-based recursive type one-step-ahead forecast techniques.

In an embodiment, the multi-step forecasting approach of the present invention uses a multi-step cumulative error as an objective function for learning.

In an embodiment, the multi-step forecasting approach of the present invention uses a VAR(s) equipped with complex-valued weight parameters.

The multi-step cumulative error and the complex-valued weight parameters allow the present invention to overcome the deficiency of the recursive approach to one-step-ahead VAR-based prediction models.

As is known in the art, each variable entering a recursive VAR has an equation explaining its evolution based on its own lagged values, the lagged values of the other model variables, and an error term. Conventionally, the error term in a recursive VAR equation relates specifically and only to a given time step and is not cumulative across two or more time steps. In contrast to the conventional approach, the present invention uses a multi-step cumulative error that represents an error term across multiple (more than two) time steps. The cumulative error allows us to train a VAR so that it forecasts not only one-step ahead, but also multi-step ahead of the time-series; in contrast, the conventional one-step error only focuses on one-step forecast, and it does not necessarily improve multi-step forecast. Note that the cumulative error by itself does not address the aforementioned deficiency, but its combination with a complex-valued VAR does. Exemplary methods relating to the use of multi-step cumulative error are described below with respect toFIG. 4as a standalone approach, and with respect toFIG. 6in a hybrid approach.

Additionally, as is also known in the art, real-values weight parameters are conventionally used. In contrast to the convention approach, complex-valued weight parameters are used. When generating a forecast, the real portion of the resultant output is taken as the multi-step-ahead forecast. With the complex-valued parameters, VAR's forecasts include both real and imaginary values. Although the imaginary part is not used directly for prediction, it is used as input to successive predictions. With this auxiliary input, the complex-valued VAR can learn more complex patterns than the conventional real-valued VAR, alleviating the aforementioned deficiency. Exemplary methods relating to the use of multi-step cumulative error are described below with respect toFIG. 5as a standalone approach, and with respect toFIG. 6in a hybrid approach.

FIG. 1is a block diagram showing an exemplary processing system100to which the present invention may be applied, in accordance with an embodiment of the present invention. The processing system100includes a set of processing units (e.g., CPUs)101, a set of GPUs102, a set of memory devices103, a set of communication devices104, and set of peripherals105. The CPUs101can be single or multi-core CPUs. The GPUs102can be single or multi-core GPUs. The one or more memory devices103can include caches, RAMs, ROMs, and other memories (flash, optical, magnetic, etc.). The communication devices104can include wireless and/or wired communication devices (e.g., network (e.g., WIFI, etc.) adapters, etc.). The peripherals105can include a display device, a user input device, a printer, an imaging device, and so forth. Elements of processing system100are connected by one or more buses or networks (collectively denoted by the figure reference numeral110).

Moreover, it is to be appreciated that various figures as described below with respect to various elements and steps relating to the present invention that may be implemented, in whole or in part, by one or more of the elements of system100.

A description will now be given regarding two exemplary environments200and300to which the present invention can be applied, in accordance with various embodiments of the present invention. The environments200and300are described below with respect toFIGS. 2 and 3, respectively. In further detail, the environment200includes a multi-step-ahead forecast system operatively coupled to a controlled system, while the environment300includes a multi-step ahead forecast system as part of a controlled system. Moreover, any of environments200and300can be part of a cloud-based environment (e.g., seeFIGS. 7 and 8). These and other environments to which the present invention can be applied are readily determined by one of ordinary skill in the art, given the teachings of the present invention provided herein, while maintaining the spirit of the present invention.

FIG. 2is a block diagram showing an exemplary environment200to which the present invention can be applied, in accordance with an embodiment of the present invention.

The environment200includes a multi-step-ahead forecasting system210and a controlled system220. The multi-step-ahead forecasting system210and the controlled system220are configured to enable communications therebetween. For example, transceivers and/or other types of communication devices including wireless, wired, and combinations thereof can be used. In an embodiment, communication between the multi-step-ahead forecasting system210and the controlled system220can be performed over one or more networks, collectively denoted by the figure reference numeral230. The communication can include, but is not limited to, multi-variate time series data from the controlled system220, and forecasts and action initiation control signals from the multi-step-ahead forecasting system210. The controlled system220can be any type of processor-based system such as, for example, but not limited to, a banking system, an access system, a surveillance system, a manufacturing system (e.g., an assembly line), an Advanced Driver-Assistance System (ADAS), and so forth.

The controlled system220provides data (e.g., multi-variate time-series data) to the multi-step-ahead forecasting system210which uses the data to make predictions (forecasts). The multi-step-ahead forecasting system210uses past predictions it has generated in order to make multi-step-ahead forecast of a future event.

In an embodiment, in order to make a multi-step-ahead forecast, the multi-step-ahead forecasting system210can use a VAR equipped with (i) multi-step cumulative error and/or (ii) complex-valued weight parameters. The use of the preceding two features advantageously overcomes the aforementioned deficiency of conventional multi-step-ahead recursive forecast techniques.

The controlled system220can be controlled based on a multi-step-ahead forecast generated by the multi-step-ahead forecasting system210. For example, based on a forecast that a machine will fail in x time steps, a corresponding action (e.g., power down machine, enable machine safeguard to prevent injury/etc., and/or so forth) can be performed at t<x in order to avoid the failure from actually occurring. As another example, based on a trajectory of an intruder, a surveillance system being controlled could lock or unlock one or more doors in order to secure someone in a certain place (holding area) and/or guide them to a safe place (safe room) and/or restrict them from a restricted place and/or so forth. Verbal (from a speaker) or displayed (on a display device) instructions could be provided along with the locking and/or unlocking of doors (or other actions) in order to guide a person. As a further example, a vehicle can be controlled (braking, steering, accelerating, and so forth) to avoid an obstacle that is predicted to be in a car's way responsive to a multi-step-ahead forecast. As a yet further example, the present invention can be incorporated into a computer system in order to forecast impending failures and take action before the failures occur, such as switching a component that will soon fail with another component, routing through a different component, processing by a different component, and so forth. It is to be appreciated that the preceding actions are merely illustrative and, thus, other actions can also be performed depending upon the implementation, as readily appreciated by one of ordinary skill in the art given the teachings of the present invention provided herein, while maintaining the spirit of the present invention.

In an embodiment, the multi-step-ahead forecasting system210can be implemented as a node in a cloud-computing arrangement. In an embodiment, a single multi-step-ahead forecasting system210can be assigned to a single controlled system or to multiple controlled systems e.g., different robots in an assembly line, and so forth). These and other configurations of the elements of environment200are readily determined by one of ordinary skill in the art given the teachings of the present invention provided herein, while maintaining the spirit of the present invention.

FIG. 3is a block diagram showing another exemplary environment300to which the present invention can be applied, in accordance with an embodiment of the present invention.

The environment300includes a controlled system320that, in turn, includes a multi-step-ahead forecasting system310. One or more communication buses and/or other devices can be used to facilitate inter-system, as well as intra-system, communication. The controlled system320can be any type of processor-based system such as, for example, but not limited to, a banking system, an access system, a surveillance system, a manufacturing system (e.g., an assembly line), an Advanced Driver-Assistance System (ADAS), and so forth.

Other than system310being included in system320, operations of these elements in environments200and300are similar. Accordingly, elements310and320are not described in further detail relative toFIG. 3for the sake of brevity, with the reader respectively directed to the descriptions of elements210and220relative to environment200ofFIG. 2given the common functions of these elements in the two environments200and300.

The following designations apply herein:

x denotes a set of N multivariate observations;

t denotes a time step;

N denotes the Nthmultivariate observation;

p denotes an observation number;

l denotes a loss (objective) function;

Re denotes a real portion of a complex-valued weight parameter;

b denotes an imaginary portion of a complex-valued weight parameter;

δ denotes a constant vector of linear time trend coefficients, with n elements; and

W denotes a weighting parameter.

In the preceding with respect to the input of the multi-step cumulative error, the term Re (in the equation “l+=(x[t+n]−Re(b+Σδ=1, . . . , pW[δ]x[t+n−δ]))2”) is the prediction of x[t+n], and the term x[t+n](in the equation “x[t+n]←b+Σδ=1, . . . , pW[δ]x[t+n−δ]”) is a function of b and W[δ].

In the preceding with respect to the output (i.e., “Re(x[t+1]), . . . , Re(x[t+N]”) of the Forecasting method, such output is taken from the real part, namely Re(x[t+1]), etc.

Exemplary methods relating to the use of multi-step cumulative error are described below with respect toFIG. 4as a standalone approach, and with respect toFIG. 6in a hybrid approach. Exemplary methods relating to the use of multi-step cumulative error are described below with respect toFIG. 5as a standalone approach, and with respect toFIG. 6in a hybrid approach.

FIG. 4is a flow diagram showing an exemplary method400for multi-step-ahead forecast using a VAR configured to use a multi-step cumulative error, in accordance with an embodiment of the present invention. Method400can be considered to include a learning stage491, an inference stage492, and an action stage493. The learning stage491corresponds to learning/training a VAR model. The inference stage492corresponds to generating a new multi-step-ahead forecast using the VAR model. The action stage493corresponds to performing an action responsive to a multi-step-ahead forecast generated by the inference stage492. The learning stage491involves blocks410through430, the inference stage involves block440, and the action stage involves block450.

At block410, receive a data set that includes multi-variate time-series data (that is, a series of data points indexed in time order and involving multiple variables).

At block420, at each time step t (where time step t=1, 2, . . . ), for each variable in the multi-variate time-series data, accumulate an error to obtain a multi-step cumulative error (across at least two or more time steps).

At block430, train the VAR (on each variable) using the multi-step cumulative error.

In an embodiment, block430can include block430A.

At block430A, update the weight parameters to the negative gradient of the multi-step cumulative error. In specific, the gradient can be computed by backpropagation, using any of automatic differentiation tools or manual coding. The gradient is used to update the weight parameters so as to decrease the multi-step cumulative error. One embodiment uses any of stochastic gradient descent rules, that is, the weight parameters are updated by adding the negative gradient multiplied by a learning rate.

At block440, generate a multi-step-ahead forecast based on p previous observations.

At block450, perform a set of actions responsive to the multi-step-ahead forecast. As appreciated by one of ordinary skill in the art, the set of actions depends on the implementation. For example, depending upon the type of the controlled system, the action will vary from system type to system type, and so forth. Exemplary actions are described above with respect toFIG. 2.

FIG. 5is a flow diagram showing an exemplary method500for multi-step-ahead forecast using a VAR configured to use complex-valued weight parameters, in accordance with an embodiment of the present invention. Method500can be considered to include a learning stage591, an inference stage592, and an action stage593. The learning stage591corresponds to learning/training a VAR model. The inference stage592corresponds to generating a new multi-step-ahead forecast using the VAR model. The action stage593corresponds to performing an action responsive to a multi-step-ahead forecast generated by the inference stage592. The learning stage591involves blocks510through520, the inference stage involves block530, and the action stage involves block540.

At block510, receive a data set that includes multi-variate time-series data (that is, a series of data points indexed in time order and involving multiple variables).

At block520, train the VAR (on each variable) using complex-valued weight parameters error. The VAR can be trained online, in the same way as block430A, or can be trained offline. In the offline learning algorithm, the multi-step cumulative errors are accumulated using the whole time-series, and the weight parameters are updated using the negative gradient of the accumulated errors, in the same way as block430A.

At block530, generate a multi-step-ahead forecast based on p previous observations.

At block530A, extract the multi-step-ahead forecast by extracting only the real portion of the multi-step ahead forecast while ignoring an imaginary portion of the multi-step-ahead forecast.

At block540, perform a set of actions responsive to the multi-step-ahead forecast. As appreciated by one of ordinary skill in the art, the set of actions depends on the implementation. For example, depending upon the type of the controlled system, the action will vary from system type to system type, and so forth. Exemplary actions are described above with respect toFIG. 2.

FIG. 6is a flow diagram showing an exemplary method600for multi-step-ahead forecast using a VAR configured to use a multi-step cumulative error and complex-valued weight parameters, in accordance with an embodiment of the present invention. Method600can be considered to include a learning stage691, an inference stage692, and an action stage693. The learning stage691corresponds to learning/training a VAR model. The inference stage692corresponds to generating a new multi-step-ahead forecast using the VAR model. The action stage693corresponds to performing an action responsive to a multi-step-ahead forecast generated by the inference stage692. The learning stage691involves blocks610through630, the inference stage involves block640, and the action stage involves block650.

At block610, receive a data set that includes multi-variate time-series data (that is, a series of data points indexed in time order and involving multiple variables).

At block620, at each time step t (where time step t=1, 2, . . . ), for each variable in the multi-variate time-series data, accumulate an error to obtain a multi-step cumulative error (across at least two or more time steps).

At block630, train the VAR (on each variable) using the multi-step cumulative error.

In an embodiment, block630can include block630A.

At block630A, update the complex-valued weighting parameters to the negative gradient of the multi-step cumulative error.

At block640, generate a multi-step-ahead forecast based on p previous observations.

At block640A, extract the multi-step-ahead forecast by extracting only the real portion of the multi-step ahead forecast while ignoring an imaginary portion of the multi-step-ahead forecast.

At block650, perform a set of actions responsive to the multi-step-ahead forecast. As appreciated by one of ordinary skill in the art, the set of actions depends on the implementation. For example, depending upon the type of the controlled system, the action will vary from system type to system type, and so forth. Exemplary actions are described above with respect toFIG. 2.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Referring now toFIG. 7, illustrative cloud computing environment750is depicted. As shown, cloud computing environment750includes one or more cloud computing nodes710with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone754A, desktop computer754B, laptop computer754C, and/or automobile computer system754N may communicate. Nodes710may communicate with one another. They may be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof. This allows cloud computing environment750to offer infrastructure, platforms and/or software as services for which a cloud consumer does not need to maintain resources on a local computing device. It is understood that the types of computing devices754A-N shown inFIG. 7are intended to be illustrative only and that computing nodes710and cloud computing environment750can communicate with any type of computerized device over any type of network and/or network addressable connection (e.g., using a web browser).

Hardware and software layer860includes hardware and software components. Examples of hardware components include: mainframes861; RISC (Reduced Instruction Set Computer) architecture based servers862; servers863; blade servers864; storage devices865; and networks and networking components866. In some embodiments, software components include network application server software867and database software868.

Virtualization layer870provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers871; virtual storage872; virtual networks873, including virtual private networks; virtual applications and operating systems874; and virtual clients875.

Workloads layer890provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation891; software development and lifecycle management892; virtual classroom education delivery893; data analytics processing894; transaction processing895; and multi-step ahead forecasting using complex-valued vector autoregression896.