A Complete Introduction To Time Series Analysis (with R):: Linear processes I
By: Hair Parra
Tags: Statistics, Forecasting, R, Machine Learning, Time Series Analysis

All linear processes can be expressed as a weighted sum of past noise, given certain conditions.

In the last tutorial , we saw how we could express the probabilistic form of the best linear predictor of a future observation based on the data at hand. We will see how to implement this in R later! In this article, we will study an important class of time series: linear processes. Let’s jump right into it!

q-correlation and strong stationarity

How can more rigorously define the stationarity of time series? How about “semi-stationarity”? We use the concept of q-correlation to do this.

A time-series process is called strictly stationary or strongly stationary if

that is, if the joint distribution of the observations X_{1}, …, X_{n} is the same as the one of the h-lagged set of observations.

Properties

All elements of a strongly stationary time series are identically distributed (but not necessarily independent!)

(but not necessarily independent!) The distribution of any subset of observations is the same as the h-lagged set

Finite second moment implies weakly stationarity

For convenience, here’s once more the definition of weak stationarity:

Note that strong stationarity is a much stronger condition! There are two important things to notice:

Strong stationarity implies weak stationarity, but the opposite is not true! The I.I.D process is strongly stationary.

The main question is: how do we construct / how can we characterize stationary processes? In particular, if we know that the IID is stationary, perhaps there is some mapping to more general sequences. In such a case, how can we determine their “level of stationarity”? The following concepts.

q-dependence and q-correlation

What the previous proposition is saying is that if we have an IID sequence, under certain conditions, we can construct a new series that is also strongly stationary, using some function g. Further, we can define q-dependence by saying that observations |t-s| lags apart are independent, but everything in between is dependent, and similarly for correlation.

Linear Processes

Now we finally get to one of the most important parts of Time Series Analysis: linear processes.

Example: MA(1)

The MA(1) process

Example: AR(1)

The AR(1) process

and we have that

If you are curious about why the coefficients are that way, you can attempt to solve the recursion by plugging back the definition of X_{t} substituting {t-1}. Note that we have to play the restriction on the AR(1) coefficients so that it satisfies the conditions of a linear process as we described above.

Linear processes in terms of the backward operator

Remember the backward shift operator that we saw back in the differencing section? We can also use it to represent linear processes.

That is, we define the Psi operator as the infinite polynomial above, in which each term is exponentiated accordingly. If we plug in the backward shift operator, we then have a concrete way to represent the linear process briefly! This will become very useful to represent a more complex process without having to write down everything explicitly. Another way to see it is that we can see a linear process as an operation applied to the noise at time t.

Next time

That’s it for today! Next time, we will continue studying some more properties of the linear process, along with some interesting propositions, and other useful operators commonly used when dealing with Time Series. Until then!

Last time

Prediction 1 → Best Linear Predictors II

Main page

Follow me at