Roots from Trees:

A Machine Learning Approach to Unit Root Detection

Gary Cornwall Jeff Chen Beau Sauley

Bureau of Economic Analysis Bennett Institute for Public Policy Murray State University

University of Cambridge

October 20, 2021

The views expressed here are those of the authors and do not represent those of the U.S. Bureau of

Economic Analysis or the U.S. Department of Commerce.
Motivation

▶ The [unresolved] presence of a unit root in time series

can produce “nonsense regressions” (Granger &

Newbold, 1974).

▶ Dozens of tests have since been developed to detect

unit roots under a variety of settings (e.g., panel data,

structural breaks, etc.).

▶ Tests don’t always agree but each tells you something

about the series, how does a practitioner weigh the

evidence?

2 / 21
Research Question(s)

1. Can we draw a link between a single unit root test and a “weak learner”.

2. If a test can be considered a weak learner, can we exploit between test variation

using modern machine learning algorithms to better identify unit root processes?

3 / 21
Quick Answers

1. Can we draw a link between a single unit root test and a “weak learner”. Yes, in

fact these are equivalent in both single and two-tailed tests for some α = α

′

.

2. If a test can be considered a weak learner, can we exploit between test variation

using modern machine learning algorithms to better identify unit root processes?

Yes, since we know how to aggregate weak base learners and create more powerful

ensemble prediction methods we can use tools such as random forests and

gradient boosting to improve unit root test accuracy.

4 / 21
Why Unit Roots?

▶ The unit root problem is a difficult time series econometrics problem which has

produced nearly five decades of research and many different test statistics.

▶ The test for unit roots is important because failing to identify a unit root can

invalidate all subsequent inferences (Granger & Newbold, 1974).

▶ Co-integrated relationships between series means you can’t just assume everything

has a unit root (Granger, 1981; Engle and Granger, 1987).

▶ The difficulty comes from differentiating unit roots from near unit roots, as a

result these test statistics have low power (Ng & Perron, 2001)

5 / 21
What is a Unit Root?

Let yt be an autoregressive time series generated such that,

yt = φyt−1 + εt, t = (1, . . .T)

▶ We assume εt ∼ N(0, σ2

) ∀ t and that σ

2

1 = . . . = σ

2

T

.

▶ We can write this as (1 − φL)yt = εt such that Lyt = yt−1.

▶ (1 − φL) has a root of 1/φ and if |φ| < 1 then yt

is considered stationary.

▶ Tests are often using an H0 : φ = 1 and H1 : |φ| < 1 structure (e.g. Augmented

Dickey Fuller test).

6 / 21
How do we test for a Unit Root?

▶ The Dickey-Fuller test statistic:

τˆ = (φˆ − 1)S

−1

e

(

X

N

t=2

Y

2

t−1

)

1/2

,

S

−1

e = (n − 2)−1X

N

t=2

(Yt − φˆYt−1)

2

,

with limiting distribution outlined in

Dickey & Fuller (1979).

▶ Calculated on first difference of Y

with H0 : φ = 1 and H1 : |φ| < 1.

7 / 21
How do we test for a Unit Root?

▶ Different assumed DGPs result in different null distributions and decision

thresholds:1

yt = λ + φyt−1 + δt + εt → xα = −3.45|α = 0.05

yt = λ + φyt−1 + εt → xα = −2.89|α = 0.05

yt = φyt−1 + εt → xα = −1.95|α = 0.05

1Not everyone even agrees on the decision thresholds for the same test (e.g., Banerjee, et al., 1993

versus Hamilton, 1994 versus MacKinnon, 2010!

8 / 21
How do we test for a Unit Root?

▶ Different assumed DGPs result in different null distributions and decision

thresholds:1

yt = λ + φyt−1 + δt + εt → xα = −3.45|α = 0.05

yt = λ + φyt−1 + εt → xα = −2.89|α = 0.05

yt = φyt−1 + εt → xα = −1.95|α = 0.05

▶ Choice of DGP opens up an additional error path beyond Type I and Type II errors.

1Not everyone even agrees on the decision thresholds for the same test (e.g., Banerjee, et al., 1993

versus Hamilton, 1994 versus MacKinnon, 2010!

8 / 21
How do we test for a Unit Root?

▶ Different assumed DGPs result in different null distributions and decision

thresholds:1

yt = λ + φyt−1 + δt + εt → xα = −3.45|α = 0.05

yt = λ + φyt−1 + εt → xα = −2.89|α = 0.05

yt = φyt−1 + εt → xα = −1.95|α = 0.05

▶ Choice of DGP opens up an additional error path beyond Type I and Type II errors.

▶ There are many tests that are similarly structured, e.g., ADF (Dickey and Fuller,

1981), PP (Phillips and Perron, 1988), KPSS (Kwiatkowski et al., 1992), PGFF

(Pantula et al., 1994), Breit (Breitung, 2002; Breitung and Taylor, 2003), ERS

(Elliot et al., 1996), URSP (Schmidt and Phillips, 1992), and URZA (Zivot and

Andrews, 2002).

1Not everyone even agrees on the decision thresholds for the same test (e.g., Banerjee, et al., 1993

versus Hamilton, 1994 versus MacKinnon, 2010!

8 / 21
Roots are Stumps

▶ In both cases a decision is being made

over a shared support of X .

▶ For some α = α

′

it must be the case

that x0 = xα.

▶





h(x) ≡ g(x)



|α = α

′



where h(x)

is the decision stump and g(x) is the

Unit Root test.

▶ x0 ≈ −1.03 which means

α = α

′ ≈ 0.273.

9 / 21
A Simple Procedure for Composite Test Construction

1. Simulate a balanced training, validation, and test set containing representative

cases of the null and alternative hypotheses

2. Derive transmitters from one or multiple test statistics and attributes of the time

series

3. Train a set of supervised classifiers, then select the model that fairs the best in

cross-validation

4. Finally, conditional upon some desired Type I error rate, α, or error cost ratio,

c(e2)/c(e1), return a class prediction for the series in question.

10 / 21
Simulate a balanced, representative data set

For any hypothesis test we can write down a DGP which will satisfy the null, e.g. unit

roots.

1. Generate 500, 000 time series with 350, 000 for training, 75, 000 for validation,

and 75, 000 for testing.

2. A series will contain a unit root, that is φ = 1 with probability 0.50 and

φ ∈ {0.9000, 0.9999} otherwise.

3. Series will be uniformly distributed over the three unit root DGPs mentioned

earlier.

4. All noise is Gaussian white noise.

11 / 21
What are the features?

UR Tests Level and First Difference STL Decomposed Series Miscellaneous

ADF Skewness TNN Test Length

PP Kurtosis Skewness Frequency

PGFF Box Statistic Kurtosis var(∆y)/var(y)

KPSS Lyapunov Exponent Box Statistic

ERS (d & p) TNN Test

URSP Hurst Exponent

URZA Strength of Trend

Breit Strength of Seasonality

While we generate the data from one of the three possible “cases” outlined in the literature all test

statistics are calculated on the most parsimonious DGP assumption possible, e.g. no drift or trend for

the ADF.

12 / 21
Is there variation in our features?

13 / 21
Power Curves

14 / 21
Empirical Example

▶ Revisit 14 macro indicators from Nelson & Plosser (1982) which serve as a

common benchmarking data set for unit root studies.

▶ Original paper indicated that, of the 14 series, only Unemployment Rate was

stationary.

▶ Subsequent studies by Perron (1989), Stock (1991), Kwiatkowski, et. al. (1992),

Andrews & Chen (1994), and Charles & Darne (2012) all introduced significant

disagreement.

▶ We find that, depending on the desired Type I error rate, between 11 indicators

(α = 0.10) and 2 indicators (α = 0.01) can be considered stationary.

15 / 21
Package Preview

16 / 21
Package Preview

17 / 21
Comparison with previous literature...

18 / 21
Conclusion

▶ Unit Root tests are weak learners.

▶ We can aggregate weak learners using gradient boosting to form a

pseudo-composite test for unit roots.

▶ This is [pessimistically] 20 percentage points more accurate and 37 percentage

points more powerful than a traditional unit root test.

19 / 21
Thank you!

20 / 21
Main Results Table

Table: Main Results

ACC SEN SPE PPV NPV F1 MCC

GB|α = 0.100 0.937 0.951 0.923 0.925 0.950 0.938 0.874

GB|α = 0.074∗ 0.941 0.934 0.948 0.947 0.935 0.941 0.882

GB|α = 0.050 0.938 0.900 0.976 0.973 0.908 0.935 0.878

GB|α = 0.010 0.892 0.785 0.998 0.998 0.823 0.879 0.802

ADF 0.763 0.546 0.980 0.964 0.684 0.697 0.583

PP 0.744 0.512 0.975 0.953 0.667 0.666 0.549

KPSS 0.614 0.250 0.977 0.916 0.567 0.393 0.331

PGFF 0.745 0.499 0.989 0.978 0.665 0.661 0.560

BREIT 0.672 0.361 0.981 0.951 0.607 0.524 0.437

ERSd 0.762 0.545 0.979 0.963 0.683 0.696 0.582

ERSp 0.770 0.564 0.976 0.958 0.692 0.710 0.592

URZA 0.635 0.309 0.959 0.883 0.582 0.458 0.354

URSP 0.727 0.552 0.903 0.850 0.669 0.669 0.485

21 / 21