This is page i
Printer: Opaque thisModelling Financial Time Series with S-PLUS,
Second Edition
Eric Zivot and Jiahui Wang
March 30, 2006

This is page ii
Printer: Opaque thisThis is page v
Printer: Opaque thisPreface

What Is the Book and Why Was It Written?
This book is a guide to analyzing and modeling financial time series using
S-PLUS and S+FinMetrics. It is a unique blend of econometric theory, financial models, data analysis, and statistical programming. It serves as auser’s guide for Insightful’s S+FinMetrics module of statistical functions
for financial time series analysis and financial econometrics as well as a general reference for models used in applied financial econometrics. The formatof the chapters in the book is to give a reasonably complete description ofa statistical model and how it works followed by illustrations of how toanalyze the model using S-PLUS and the functions in S+FinMetrics. In
this way, the book stands alone as an introduction to financial time series analysis as well as a user’s guide for S+FinMetrics. It also highlights
the general analysis of time series data using the new time series objects
introduced in S-PLUS 6.

Intended Audience
This book is written for a wide audience of individuals who work, do research or study in the areas of empirical finance and financial econometrics.
The field of financial econometrics has exploded over the last decade, andthis book represents an integration of theory, methods and examples using the S-PLUS modeling language to facilitate the practice of financial
econometrics. This audience includes researchers and practitioners in thefinance industry, academic researchers in economics and finance, and ad-

vi

Prefacevanced MBA and graduate students in economics and finance. Researchersand practitioners in the finance industry who already use S-PLUS and desire more functionality for analyzing and modeling financial data will findthis text useful. It is also appropriate for financial analysts who may not
be familiar with S-PLUS but who desire an integrated and open statisticalmodeling and programming environment for the analysis of financial data.
This guide is useful for academic researchers interested in empirical financeand financial econometrics. Finally, this book may be used as a textbook
or a textbook companion for advanced MBA and graduate level courses intime series analysis, empirical finance and financial econometrics.

Audience Background
It is assumed that the reader has a basic familiarity with S-PLUS at the levelof Krause and Olson (2005) and a background in mathematical statistics
at the level of Hogg and Craig (1994), is comfortable with linear algebra
and linear regression, and has been exposed to basic time series concepts aspresented in Harvey (1993) or Franses (1998). Most importantly, the bookassumes that the reader is interested in modeling and analyzing financial
time series.

Overview of the Book
The chapters in the book cover univariate and multivariate models for analyzing financial time series using S-PLUS and the functions in S+FinMetrics.
Chapter one gives a general overview of the use of S-PLUS 6 and highlights
certain aspects of the language for statistical modeling. Chapter two introduces the new time series objects in S-PLUS 6 and illustrates the specification, manipulation and visualization of these objects. Chapter three surveys
time series concepts used throughout the book. Chapters four through eightcover a variety of topics in the modeling of univariate financial time series,
including testing for unit roots, extreme value theory, time series regression
models, GARCH models of volatility, and long memory models. Chapternine introduces rolling analyses of time series models and covers related
topics such as technical analysis of financial time series and moving average methods for high frequency data. Chapters ten through fifteen covermodels for the analysis of multivariate financial time series. Topics include
systems of regression equations, classical and Bayesian vector autoregressive models, cointegration, factor models, multivariate GARCH models,and state space models. Chapter 16 covers aspects of modeling time seriesarising from fixed income financial assets. Chapter 17, written by Victor
Yohai and Jiahui Wang, describes robust REGARIMA models that allow for structural change. Chapters 18 through 23 are new to the Second
Edition of the book. These new chapters cover nonlinear regime-switchingPrefacevii

models, copulas, continuous-time models, the generalized method of moments, semi-nonparametric conditional density models, and the eﬃcientmethod of moments.

What Is S+FinMetrics?
S+FinMetrics is an S-PLUS module for the econometric modeling and prediction of economic and financial time series. With some 600 functions,version 1.0 of S+FinMetrics oﬀers the following functionality:• Easy-to-use Trellis plots for multivariate time series• Time series manipulations such as missing value interpolation, disaggregation, diﬀerences, distributed lags and polynomial distributedlags• Rolling sample statistics such as variance, maximum, and minimum• Moving average operators for both regularly spaced and irregularly
spaced time series• Common technical analysis measures and indicators• Statistical tests for normality, autocorrelation, heteroskedasticity, multicollinearity, GARCH eﬀects, and long memory• Extreme value theory models based on generalized extreme value andgeneralized Pareto distributions as well as copulas• Autoregressive distributed lag regression models• White and Newey-West corrections for heteroskedasticity and serial
correlation• Robust estimation of REG-ARIMA models and robust detection oflevel shifts, trend breaks, and outliers• Rolling and recursive regression• Generic rolling models for back-testing• Long memory fractional ARIMA and SEMIFAR models• Univariate GARCH models including long memory FIGARCH and
FIEGARCH models• Multivariate GARCH models• Linear and nonlinear systems of regression equations• Classical and Bayesian vector autoregression models

viii

Preface• Tests for unit roots and cointegration• Vector error correction models• State space models and eﬃcient estimation, prediction, smoothing,and simulation using the Kalman filter• Statistical multifactor models for large data sets based on asymptotic
principal components• Term structure interpolation
New features in version 2.0 of S+FinMetrics include:• Variance ratio tests, eﬃcient unit root tests and tests for nonlinearity• Threshold AR, smooth transition AR and Markov switching AR models as well as Markov switching state space models• Simulated solutions to systems of stochastic diﬀerential equations• Generalized method of moments estimation• Gallant and Tauchen’s semi-nonparametric conditional density estimation and eﬃcient method of moments estimation
S+FinMetrics incorporates functions from S+GARCH, the EVIS libraryof functions for modeling extreme values created by Alexander McNeil,the EVANESCE library of functions for modeling extreme values and bivariate copulas created by Rene Carmona and Julia Morrison, the SsfPack
C library of state space modeling functions created by Siem Jan Koopman, and the SNP and EMM FORTRAN libraries created by Ronald Gallant and George Tauchen. S+GARCH was originally developed by Zhuanxin
Ding, Hong-Ye Gao, Doug Martin, Jiahui Wang and Yihui Zhan. The
S+FinMetrics function arima.rob was written by Ana Bianco, Marta
Garcia Ben, Elena Martinez and Victor Yohai. The S+FinMetrics longmemory modeling functions FAR, FARIMA, SEMIFAR and fgarch were developed by Jan Beran, Andrew Bruce, Don Percival, Alan Gibbs and Jiahui
Wang and supported by NSF grant DMI-9801614 to Insightful Corporation (formerly MathSoft, Inc.). Much of the new functionality in version
2.0 of S+FinMetrics was supported by the NSF SBIR Phase II grant DMI0132076 to Insightful Corporation. The S-PLUS implementation of Gallant
and Tauchen’s SNP and EMM FORTRAN libraries was accomplished by
Jiahui Wang, Bob Thurman, Michael Sannella, Ying Gu and Eric Zivot,
with the generous help and support of George Tauchen. Hu McCulloch
kindly provided the term structure data included with S+FinMetrics, andJames MacKinnon provided data sets for the response surface critical values for the Dickey-Fuller and Phillips-Ouliaris distributions.

Prefaceix

Contact Information and Website
The authors are responsible for all of the material in the book except thematerial on robust change detection, which was written by Victor Yohai.
Eric Zivot is primarily responsible for chapters 2-6, 9-12, 14-15, and 19-23,
and Jiahui Wang is primarily responsible for chapters 1, 7-8, 13, 16 and 18.
The authors may be contacted by electronic mail at
ezivot@u.washington.edu
jwang@svolatility.com
and welcome feedback and suggestions for improvements to the contents ofthe book. The website for the book is located on Eric Zivot’s University ofWashington web site at
http://faculty.washington.edu/ezivot/
ModelingFinancialTimeSeries.htm
The website for version 2.0 of S+FinMetrics is located on the Insightful
Corporation website at
http://www.insightful.com/support/finmetrics20

Acknowledgements
This book would not have been written without the support and encouragement from Insightful Corporation. The idea for the S+FinMetrics module was conceived by Douglas Martin and the authors. The development of
S+FinMetrics was completed at Insightful by Jiahui Wang, Quan Wen and
Hui Huang with help from many people. In particular, Jan Beran wrote
many of the long memory functions while acting as a consultant to Insightful. Siem Jan Koopman helped to incorporate the SsfPack functions into
S-PLUS and to write the chapter on state space models. Alexander McNeil
and Rene Carmona graciously provided background material and S-PLUS
examples for the material in the chapters on modeling extreme values andcopulas. Bob Thurman helped to write the chapter on continuous-timemodels, and Ying Gu helped with the SNP and EMM examples. Ronald
Gallant and George Tauchen graciously allowed the use of material fromtheir unpublished 2001 survey paper “Eﬃcient Method of Moments” forChapters 22 and 23. A number of people were helpful in proofreading thebook and testing the software. Particular thanks go to Eric Aldrich, Andrew Bruce, Chuck Curry, Zhuanxin Ding, Ruud Koning, Steve McKinney,
Jun Ma, Scott Payseur, David Weitzel, Quan Wen and Bingcheng Yan.

Typographical Conventions
This book obeys the following typographic conventions:x

Preface• The italic font is used for emphasis, and also for user-supplied variables within UNIX, DOS and S-PLUS commands.
• The typewriter font is used for S-PLUS functions, the output of
S-PLUS functions and examples of S-PLUS sessions.
• S-PLUS objects of a specified class are expressed in typewriter fontenclosed in quotations “ ”. For example, the S-PLUS timeSeries
function creates objects of class “timeSeries”.
Displayed S-PLUS commands are shown with the prompt character >. For
example> summary(ols.fit)
S-PLUS commands that require more than one line of input are displayedwith the continuation prompt indicated by + or Continue string:. The
S-PLUS output and plots in this book were generated from a combination of
S+FinMetrics Version 1.0 and S-PLUS Version 6.0 release 2 for Windows,
and S+FinMetrics Versions 2.0 and S-PLUS Version 7.0 for Windows. The
S-PLUS output and “timeSeries” objects were generated with the options
settings> options(width=60)> options(time.zone="GMT")
In some cases, parts of long output from S-PLUS functions is omitted andthese lines are indicated by...
Some of the output has been hand edited to avoid line overflow.
Seattle, Washington, USA
Chicago, Illinois, USA

Eric Zivot
Jiahui Wang

References
Franses, P.H. (1998). Time Series Models for Business and Economic
Forecasting. Cambridge University Press, Cambridge.
Harvey, A.C. (1993). Time Series Models, Second Edition. MIT Press,
Cambridge.
Hogg, R.V. and A.T. Craig (1994). Introduction to Mathematical Statistics, Fifth Edition. Prentice Hall, New York.
Krause, A. and M. Olson (2002). The Basics of S-PLUS, Fourth Edition. Springer-Verlag, New York.

This is page xi
Printer: Opaque thisContents

Prefacev

1 S and S-PLUS
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 S Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2.1 Assignment . . . . . . . . . . . . . . . . . . . . . . .
1.2.2 Class . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2.3 Method . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Modeling Functions in S+FinMetrics . . . . . . . . . . . .
1.3.1 Formula Specification . . . . . . . . . . . . . . . . .
1.3.2 Method . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 S-PLUS Resources . . . . . . . . . . . . . . . . . . . . . . . .
1.4.1 Books . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.2 Internet . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
1
2
2
3
7
8
8
11
12
12
13
13

2 Time Series Specification, Manipulation, andVisualization in S-PLUS
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 The Specification of “timeSeries” Objects in S-PLUS . . .
2.2.1 Basic Manipulations . . . . . . . . . . . . . . . . . .
2.2.2 S-PLUS “timeDate” Objects . . . . . . . . . . . . . .
2.2.3 Creating Common “timeDate” Sequences . . . . . .
2.2.4 Miscellaneous Time and Date Functions . . . . . . .

15
15
15
18
19
24
28

xii

Contents

2.2.5
2.2.6
2.2.7
2.2.8

Creating “timeSeries” Objects . . . . . . . . . . .
Aggregating and Disaggregating Time Series . . . .
Merging Time Series . . . . . . . . . . . . . . . . . .
Dealing with Missing Values Using the
S+FinMetrics Function interpNA . . . . . . . . . .
Time Series Manipulation in S-PLUS . . . . . . . . . . . . .
2.3.1 Creating Lags and Diﬀerences . . . . . . . . . . . . .
2.3.2 Return Definitions . . . . . . . . . . . . . . . . . . .
2.3.3 Computing Asset Returns Using the
S+FinMetrics Function getReturns . . . . . . . . .
Visualizing Time Series in S-PLUS . . . . . . . . . . . . . .
2.4.1 Plotting “timeSeries” Using the S-PLUS
Generic plot Function . . . . . . . . . . . . . . . . .
2.4.2 Plotting “timeSeries” Using the S+FinMetrics
Trellis Plotting Functions . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

28
31
38

3 Time Series Concepts
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Univariate Time Series . . . . . . . . . . . . . . . . . . . . .
3.2.1 Stationary and Ergodic Time Series . . . . . . . . .
3.2.2 Linear Processes and ARMA Models . . . . . . . . .
3.2.3 Autoregressive Models . . . . . . . . . . . . . . . . .
3.2.4 Moving Average Models . . . . . . . . . . . . . . . .
3.2.5 ARMA(p,q) Models . . . . . . . . . . . . . . . . . .
3.2.6 Estimation of ARMA Models and Forecasting . . . .
3.2.7 Martingales and Martingale Diﬀerence Sequences . .
3.2.8 Long-run Variance . . . . . . . . . . . . . . . . . . .
3.2.9 Variance Ratios . . . . . . . . . . . . . . . . . . . . .
3.3 Univariate Nonstationary Time Series . . . . . . . . . . . .
3.4 Long Memory Time Series . . . . . . . . . . . . . . . . . . .
3.5 Multivariate Time Series . . . . . . . . . . . . . . . . . . . .
3.5.1 Stationary and Ergodic Multivariate Time Series . .
3.5.2 Multivariate Wold Representation . . . . . . . . . .
3.5.3 Long Run Variance . . . . . . . . . . . . . . . . . . .
3.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57
57
58
58
64
66
71
74
76
83
85
88
93
97
101
101
106
107
109

2.3

2.4

2.5

39
40
40
43
46
48
48
52
55

4 Unit Root Tests
111
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.2 Testing for Nonstationarity and Stationarity . . . . . . . . . 112
4.3 Autoregressive Unit Root Tests . . . . . . . . . . . . . . . . 114
4.3.1 Simulating the DF and Normalized Bias
Distributions . . . . . . . . . . . . . . . . . . . . . . 116
4.3.2 Trend Cases . . . . . . . . . . . . . . . . . . . . . . . 118
4.3.3 Dickey-Fuller Unit Root Tests . . . . . . . . . . . . . 120

Contents

4.4

4.5
4.6

4.7

4.3.4 Phillips-Perron Unit Root Tests . . . . . . . . . . . .
Stationarity Tests . . . . . . . . . . . . . . . . . . . . . . . .
4.4.1 Simulating the KPSS Distributions . . . . . . . . . .
4.4.2 Testing for Stationarity Using the S+FinMetrics
Function stationaryTest . . . . . . . . . . . . . . .
Some Problems with Unit Root Tests . . . . . . . . . . . . .
Eﬃcient Unit Root Tests . . . . . . . . . . . . . . . . . . .
4.6.1 Point Optimal Tests . . . . . . . . . . . . . . . . . .
4.6.2 DF-GLS Tests . . . . . . . . . . . . . . . . . . . . .
4.6.3 Modified Eﬃcient PP Tests . . . . . . . . . . . . . .
4.6.4 Estimating λ2 . . . . . . . . . . . . . . . . . . . . . .
4.6.5 Choosing Lag Lengths to Achieve Good Sizeand Power . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiii

127
129
130
131
132
132
133
134
134
135
135
138

5 Modeling Extreme Values
141
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
5.2 Modeling Maxima and Worst Cases . . . . . . . . . . . . . . 142
5.2.1 The Fisher-Tippet Theorem and the Generalized
Extreme Value Distribution . . . . . . . . . . . . . . 143
5.2.2 Estimation of the GEV Distribution . . . . . . . . . 147
5.2.3 Return Level . . . . . . . . . . . . . . . . . . . . . . 153
5.3 Modeling Extremes Over High Thresholds . . . . . . . . . . 157
5.3.1 The Limiting Distribution of Extremes OverHigh Thresholds and the Generalized Pareto
Distribution . . . . . . . . . . . . . . . . . . . . . . . 159
5.3.2 Estimating the GPD by Maximum Likelihood . . . . 164
5.3.3 Estimating the Tails of the Loss Distribution . . . . 165
5.3.4 Risk Measures . . . . . . . . . . . . . . . . . . . . . 171
5.4 Hill’s Non-parametric Estimator of Tail Index . . . . . . . . 174
5.4.1 Hill Tail and Quantile Estimation . . . . . . . . . . . 175
5.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
6 Time Series Regression Modeling181
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.2 Time Series Regression Model . . . . . . . . . . . . . . . . . 182
6.2.1 Least Squares Estimation . . . . . . . . . . . . . . . 183
6.2.2 Goodness of Fit . . . . . . . . . . . . . . . . . . . . . 183
6.2.3 Hypothesis Testing . . . . . . . . . . . . . . . . . . . 184
6.2.4 Residual Diagnostics . . . . . . . . . . . . . . . . . . 185
6.3 Time Series Regression Using the S+FinMetrics
Function OLS . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.4 Dynamic Regression . . . . . . . . . . . . . . . . . . . . . . 201
6.4.1 Distributed Lags and Polynomial Distributed Lags . 205
6.4.2 Polynomial Distributed Lag Models . . . . . . . . . 207

xivContents

6.5

6.6

6.7

Heteroskedasticity and Autocorrelation Consistent
Covariance Matrix Estimation . . . . . . . . . . . . . . . . .
6.5.1 The Eicker-White Heteroskedasticity Consistent(HC) Covariance Matrix Estimate . . . . . . . . . .
6.5.2 Testing for Heteroskedasticity . . . . . . . . . . . . .
6.5.3 The Newey-West Heteroskedasticity and
Autocorrelation Consistent (HAC) CovarianceMatrix Estimate . . . . . . . . . . . . . . . . . . . .
Recursive Least Squares Estimation . . . . . . . . . . . . .
6.6.1 CUSUM and CUSUMSQ Tests for Parameter
Stability . . . . . . . . . . . . . . . . . . . . . . . . .
6.6.2 Computing Recursive Least Squares Estimates
Using the S+FinMetrics Function RLS . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208
209
211

214
217
218
219
221

7 Univariate GARCH Modeling223
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.2 The Basic ARCH Model . . . . . . . . . . . . . . . . . . . . 224
7.2.1 Testing for ARCH Eﬀects . . . . . . . . . . . . . . . 228
7.3 The GARCH Model and Its Properties . . . . . . . . . . . . 229
7.3.1 ARMA Representation of GARCH Model . . . . . . 230
7.3.2 GARCH Model and Stylized Facts . . . . . . . . . . 230
7.4 GARCH Modeling Using S+FinMetrics . . . . . . . . . . . 232
7.4.1 GARCH Model Estimation . . . . . . . . . . . . . . 232
7.4.2 GARCH Model Diagnostics . . . . . . . . . . . . . . 235
7.5 GARCH Model Extensions . . . . . . . . . . . . . . . . . . 240
7.5.1 Asymmetric Leverage Eﬀects and News Impact . . . 241
7.5.2 Two Components Model . . . . . . . . . . . . . . . . 247
7.5.3 GARCH-in-the-Mean Model . . . . . . . . . . . . . . 250
7.5.4 ARMA Terms and Exogenous Variables in
Conditional Mean Equation . . . . . . . . . . . . . . 252
7.5.5 Exogenous Explanatory Variables in theConditional Variance Equation . . . . . . . . . . . . 254
7.5.6 Non-Gaussian Error Distributions . . . . . . . . . . 257
7.6 GARCH Model Selection and Comparison . . . . . . . . . . 260
7.6.1 Constrained GARCH Estimation . . . . . . . . . . . 261
7.7 GARCH Model Prediction . . . . . . . . . . . . . . . . . . . 262
7.8 GARCH Model Simulation . . . . . . . . . . . . . . . . . . 265
7.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
7.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
8 Long Memory Time Series Modeling271
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
8.2 Long Memory Time Series . . . . . . . . . . . . . . . . . . . 272
8.3 Statistical Tests for Long Memory . . . . . . . . . . . . . . 276

8.4

8.5

8.6

8.7

8.8

Contentsxv

8.3.1 R/S Statistic . . . . . . . . . . . . . . . . . . . . . .
8.3.2 GPH Test . . . . . . . . . . . . . . . . . . . . . . . .
Estimation of Long Memory Parameter . . . . . . . . . . .
8.4.1 R/S Analysis . . . . . . . . . . . . . . . . . . . . . .
8.4.2 Periodogram Method . . . . . . . . . . . . . . . . . .
8.4.3 Whittle’s Method . . . . . . . . . . . . . . . . . . . .
Estimation of FARIMA and SEMIFAR Models . . . . . . .
8.5.1 Fractional ARIMA Models . . . . . . . . . . . . . .
8.5.2 SEMIFAR Model . . . . . . . . . . . . . . . . . . . .
Long Memory GARCH Models . . . . . . . . . . . . . . . .
8.6.1 FIGARCH and FIEGARCH Models . . . . . . . . .
8.6.2 Estimation of Long Memory GARCH Models . . . .
8.6.3 Custom Estimation of Long Memory GARCH
Models . . . . . . . . . . . . . . . . . . . . . . . . .
Prediction from Long Memory Models . . . . . . . . . . . .
8.7.1 Prediction from FARIMA/SEMIFAR Models . . . .
8.7.2 Prediction from FIGARCH/FIEGARCH Models . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

276
278
280
280
282
283
284
285
292
296
296
297
301
304
304
308
309

9 Rolling Analysis of Time Series
313
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
9.2 Rolling Descriptive Statistics . . . . . . . . . . . . . . . . . 314
9.2.1 Univariate Statistics . . . . . . . . . . . . . . . . . . 314
9.2.2 Bivariate Statistics . . . . . . . . . . . . . . . . . . . 321
9.2.3 Exponentially Weighted Moving Averages . . . . . . 323
9.2.4 Moving Average Methods for Irregularly Spaced
High Frequency Data . . . . . . . . . . . . . . . . . 327
9.2.5 Rolling Analysis of Miscellaneous Functions . . . . . 334
9.3 Technical Analysis Indicators . . . . . . . . . . . . . . . . . 337
9.3.1 Price Indicators . . . . . . . . . . . . . . . . . . . . . 338
9.3.2 Momentum Indicators and Oscillators . . . . . . . . 338
9.3.3 Volatility Indicators . . . . . . . . . . . . . . . . . . 340
9.3.4 Volume Indicators . . . . . . . . . . . . . . . . . . . 341
9.4 Rolling Regression . . . . . . . . . . . . . . . . . . . . . . . 342
9.4.1 Estimating Rolling Regressions Using the
S+FinMetrics Function rollOLS . . . . . . . . . . . 343
9.4.2 Rolling Predictions and Backtesting . . . . . . . . . 349
9.5 Rolling Analysis of General Models Using the S+FinMetrics
Function roll . . . . . . . . . . . . . . . . . . . . . . . . . . 358
9.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
10 Systems of Regression Equations
361
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
10.2 Systems of Regression Equations . . . . . . . . . . . . . . . 362
10.3 Linear Seemingly Unrelated Regressions . . . . . . . . . . . 364

xvi

Contents

10.3.1 Estimation . . . . . . . . . . . . . . . . . . . . . . .
10.3.2 Analysis of SUR Models with the S+FinMetrics
Function SUR . . . . . . . . . . . . . . . . . . . . . .
10.4 Nonlinear Seemingly Unrelated Regression Models . . . . .
10.4.1 Analysis of Nonlinear SUR Models with the
S+FinMetrics Function NLSUR . . . . . . . . . . . .
10.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

364
367
374
375
382

11 Vector Autoregressive Models for Multivariate
Time Series
385
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
11.2 The Stationary Vector Autoregression Model . . . . . . . . 386
11.2.1 Estimation . . . . . . . . . . . . . . . . . . . . . . . 388
11.2.2 Inference on Coeﬃcients . . . . . . . . . . . . . . . . 390
11.2.3 Lag Length Selection . . . . . . . . . . . . . . . . . . 390
11.2.4 Estimating VAR Models Using the S+FinMetrics
Function VAR . . . . . . . . . . . . . . . . . . . . . . 390
11.3 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
11.3.1 Traditional Forecasting Algorithm . . . . . . . . . . 398
11.3.2 Simulation-Based Forecasting . . . . . . . . . . . . . 402
11.4 Structural Analysis . . . . . . . . . . . . . . . . . . . . . . . 406
11.4.1 Granger Causality . . . . . . . . . . . . . . . . . . . 407
11.4.2 Impulse Response Functions . . . . . . . . . . . . . . 409
11.4.3 Forecast Error Variance Decompositions . . . . . . . 414
11.5 An Extended Example . . . . . . . . . . . . . . . . . . . . . 416
11.6 Bayesian Vector Autoregression . . . . . . . . . . . . . . . . 424
11.6.1 An Example of a Bayesian VAR Model . . . . . . . 424
11.6.2 Conditional Forecasts . . . . . . . . . . . . . . . . . 427
11.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
12 Cointegration
431
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
12.2 Spurious Regression and Cointegration . . . . . . . . . . . . 432
12.2.1 Spurious Regression . . . . . . . . . . . . . . . . . . 432
12.2.2 Cointegration . . . . . . . . . . . . . . . . . . . . . . 435
12.2.3 Cointegration and Common Trends . . . . . . . . . . 437
12.2.4 Simulating Cointegrated Systems . . . . . . . . . . . 437
12.2.5 Cointegration and Error Correction Models . . . . . 441
12.3 Residual-Based Tests for Cointegration . . . . . . . . . . . . 444
12.3.1 Testing for Cointegration When the Cointegrating
Vector Is Pre-specified . . . . . . . . . . . . . . . . . 444
12.3.2 Testing for Cointegration When the Cointegrating
Vector Is Estimated . . . . . . . . . . . . . . . . . . 447
12.4 Regression-Based Estimates of Cointegrating Vectors andError Correction Models . . . . . . . . . . . . . . . . . . . . 450

Contents

12.4.1 Least Square Estimator . . . . . . . . . . . . . . . .
12.4.2 Stock and Watson’s Eﬃcient Lead/Lag Estimator .
12.4.3 Estimating Error Correction Models by Least
Squares . . . . . . . . . . . . . . . . . . . . . . . . .
12.5 VAR Models and Cointegration . . . . . . . . . . . . . . . .
12.5.1 The Cointegrated VAR . . . . . . . . . . . . . . . .
12.5.2 Johansen’s Methodology for Modeling
Cointegration . . . . . . . . . . . . . . . . . . . . . .
12.5.3 Specification of Deterministic Terms . . . . . . . . .
12.5.4 Likelihood Ratio Tests for the Number of
Cointegrating Vectors . . . . . . . . . . . . . . . . .
12.5.5 Testing Hypothesis on Cointegrating Vectors
Using the S+FinMetrics Function coint . . . . . .
12.5.6 Maximum Likelihood Estimation of the
Cointegrated VECM . . . . . . . . . . . . . . . . . .
12.5.7 Maximum Likelihood Estimation of the
Cointegrated VECM Using the S+FinMetrics
Function VECM . . . . . . . . . . . . . . . . . . . . .
12.5.8 Forecasting from the VECM . . . . . . . . . . . . .
12.6 Appendix: Maximum Likelihood Estimation of a
Cointegrated VECM . . . . . . . . . . . . . . . . . . . . . .
12.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xvii

450
451
454
455
456
458
459
461
463
467

468
474
476
478

13 Multivariate GARCH Modeling481
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
13.2 Exponentially Weighted Covariance Estimate . . . . . . . . 482
13.3 Diagonal VEC Model . . . . . . . . . . . . . . . . . . . . . . 486
13.4 Multivariate GARCH Modeling in S+FinMetrics . . . . . . 487
13.4.1 Multivariate GARCH Model Estimation . . . . . . . 487
13.4.2 Multivariate GARCH Model Diagnostics . . . . . . . 490
13.5 Multivariate GARCH Model Extensions . . . . . . . . . . . 496
13.5.1 Matrix-Diagonal Models . . . . . . . . . . . . . . . . 496
13.5.2 BEKK Models . . . . . . . . . . . . . . . . . . . . . 498
13.5.3 Univariate GARCH-based Models . . . . . . . . . . 499
13.5.4 ARMA Terms and Exogenous Variables . . . . . . . 504
13.5.5 Multivariate Conditional t-Distribution . . . . . . . 508
13.6 Multivariate GARCH Prediction . . . . . . . . . . . . . . . 509
13.7 Custom Estimation of GARCH Models . . . . . . . . . . . . 512
13.7.1 GARCH Model Objects . . . . . . . . . . . . . . . . 512
13.7.2 Revision of GARCH Model Estimation . . . . . . . . 514
13.8 Multivariate GARCH Model Simulation . . . . . . . . . . . 515
13.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
14 State Space Models
519
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 519

xviiiContents

14.2 State Space Representation . . . . . . . . . . . . . . . . . .
14.2.1 Initial Conditions . . . . . . . . . . . . . . . . . . . .
14.2.2 State Space Representation in
S+FinMetrics/SsfPack . . . . . . . . . . . . . . . .
14.2.3 Missing Values . . . . . . . . . . . . . . . . . . . . .
14.2.4 S+FinMetrics/SsfPack Functions for Specifying
the State Space Form for Some Common Time
Series Models . . . . . . . . . . . . . . . . . . . . . .
14.2.5 Simulating Observations from the StateSpace Model . . . . . . . . . . . . . . . . . . . . . .
14.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.3.1 Kalman Filter . . . . . . . . . . . . . . . . . . . . . .
14.3.2 Kalman Smoother . . . . . . . . . . . . . . . . . . .
14.3.3 Smoothed State and Response Estimates . . . . . .
14.3.4 Smoothed Disturbance Estimates . . . . . . . . . . .
14.3.5 Forecasting . . . . . . . . . . . . . . . . . . . . . . .
14.3.6 S+FinMetrics/SsfPack Implementation of StateSpace Modeling Algorithms . . . . . . . . . . . . . .
14.4 Estimation of State Space Models . . . . . . . . . . . . . . .
14.4.1 Prediction Error Decomposition of
Log-Likelihood . . . . . . . . . . . . . . . . . . . . .
14.4.2 Fitting State Space Models Using the
S+FinMetrics/SsfPack Function SsfFit . . . . . .
14.4.3 Quasi-Maximum Likelihood Estimation . . . . . . .
14.5 Simulation Smoothing . . . . . . . . . . . . . . . . . . . . .
14.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

520
521
521
527

528
540
543
543
543
544
544
544
545
552
552
554
561
565
566

15 Factor Models for Asset Returns
569
15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
15.2 Factor Model Specification . . . . . . . . . . . . . . . . . . . 570
15.3 Macroeconomic Factor Models for Returns . . . . . . . . . . 571
15.3.1 Sharpe’s Single Index Model . . . . . . . . . . . . . 572
15.3.2 The General Multifactor Model . . . . . . . . . . . . 577
15.4 Fundamental Factor Model . . . . . . . . . . . . . . . . . . 580
15.4.1 BARRA-type Single Factor Model . . . . . . . . . . 581
15.4.2 BARRA-type Industry Factor Model . . . . . . . . . 582
15.5 Statistical Factor Models for Returns . . . . . . . . . . . . . 590
15.5.1 Factor Analysis . . . . . . . . . . . . . . . . . . . . . 590
15.5.2 Principal Components . . . . . . . . . . . . . . . . . 597
15.5.3 Asymptotic Principal Components . . . . . . . . . . 606
15.5.4 Determining the Number of Factors . . . . . . . . . 610
15.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
16 Term Structure of Interest Rates
617
16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 617

Contents

16.2 Discount, Spot and Forward Rates . . . . . . . . . . . . . .
16.2.1 Definitions and Rate Conversion . . . . . . . . . . .
16.2.2 Rate Conversion in S+FinMetrics . . . . . . . . . .
16.3 Quadratic and Cubic Spline Interpolation . . . . . . . . . .
16.4 Smoothing Spline Interpolation . . . . . . . . . . . . . . . .
16.5 Nelson-Siegel Function . . . . . . . . . . . . . . . . . . . . .
16.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xix

618
618
619
620
624
628
632
633

17 Robust Change Detection
635
17.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 635
17.2 REGARIMA Models . . . . . . . . . . . . . . . . . . . . . . 636
17.3 Robust Fitting of REGARIMA Models . . . . . . . . . . . . 637
17.4 Prediction Using REGARIMA Models . . . . . . . . . . . . 642
17.5 Controlling Robust Fitting of REGARIMA Models . . . . . 643
17.5.1 Adding Seasonal Eﬀects . . . . . . . . . . . . . . . . 643
17.5.2 Controlling Outlier Detection . . . . . . . . . . . . . 645
17.5.3 Iterating the Procedure . . . . . . . . . . . . . . . . 647
17.6 Algorithms of Filtered τ -Estimation . . . . . . . . . . . . . 649
17.6.1 Classical Maximum Likelihood Estimates . . . . . . 650
17.6.2 Filtered τ -Estimates . . . . . . . . . . . . . . . . . . 651
17.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651
18 Nonlinear Time Series Models
653
18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 653
18.2 BDS Test for Nonlinearity . . . . . . . . . . . . . . . . . . . 654
18.2.1 BDS Test Statistic . . . . . . . . . . . . . . . . . . . 655
18.2.2 Size of BDS Test . . . . . . . . . . . . . . . . . . . . 655
18.2.3 BDS Test as a Nonlinearity Test and a
Misspecification Test . . . . . . . . . . . . . . . . . . 657
18.3 Threshold Autoregressive Models . . . . . . . . . . . . . . . 662
18.3.1 TAR and SETAR Models . . . . . . . . . . . . . . . 663
18.3.2 Tsay’s Approach . . . . . . . . . . . . . . . . . . . . 664
18.3.3 Hansen’s Approach . . . . . . . . . . . . . . . . . . . 671
18.4 Smooth Transition Autoregressive Models . . . . . . . . . . 678
18.4.1 Logistic and Exponential STAR Models . . . . . . . 678
18.4.2 Test for STAR Nonlinearity . . . . . . . . . . . . . . 680
18.4.3 Estimation of STAR Models . . . . . . . . . . . . . . 683
18.5 Markov Switching State Space Models . . . . . . . . . . . . 687
18.5.1 Discrete State Markov Process . . . . . . . . . . . . 688
18.5.2 Markov Switching AR Process . . . . . . . . . . . . 690
18.5.3 Markov Switching State Space Models . . . . . . . . 691
18.6 An Extended Example: Markov Switching Coincident Index 701
18.6.1 State Space Representation of Markov Switching Coincident Index Model . . . . . . . . . . . . . . . . . . 702

xxContents

18.6.2 Approximate MLE of Markov Switching Coincident
Index . . . . . . . . . . . . . . . . . . . . . . . . . . 705
18.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 709
19 Copulas
713
19.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 713
19.2 Motivating Example . . . . . . . . . . . . . . . . . . . . . . 714
19.3 Definitions and Basic Properties of Copulas . . . . . . . . . 722
19.3.1 Properties of Distributions . . . . . . . . . . . . . . 722
19.3.2 Copulas and Sklar’s Theorem . . . . . . . . . . . . . 724
19.3.3 Dependence Measures and Copulas . . . . . . . . . . 726
19.4 Parametric Copula Classes and Families . . . . . . . . . . . 729
19.4.1 Normal Copula . . . . . . . . . . . . . . . . . . . . . 729
19.4.2 Normal Mixture Copula . . . . . . . . . . . . . . . . 730
19.4.3 Extreme Value Copula Class . . . . . . . . . . . . . 730
19.4.4 Archimedean Copulas . . . . . . . . . . . . . . . . . 732
19.4.5 Archimax Copulas . . . . . . . . . . . . . . . . . . . 735
19.4.6 Representation of Copulas in S+FinMetrics . . . . . 735
19.4.7 Creating Arbitrary Bivariate Distributions . . . . . . 743
19.4.8 Simulating from Arbitrary Bivariate Distributions . 745
19.5 Fitting Copulas to Data . . . . . . . . . . . . . . . . . . . . 747
19.5.1 Empirical Copula . . . . . . . . . . . . . . . . . . . . 747
19.5.2 Maximum Likelihood Estimation . . . . . . . . . . . 750
19.5.3 Fitting Copulas Using the S+FinMetrics/EVANESCE
Function fit.copula . . . . . . . . . . . . . . . . . 751
19.6 Risk Management Using Copulas . . . . . . . . . . . . . . . 754
19.6.1 Computing Portfolio Risk Measures Using Copulas . 754
19.6.2 Computing VaR and ES by Simulation . . . . . . . . 755
19.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 757
20 Continuous-Time Models for Financial Time Series
759
20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 759
20.2 SDEs: Background . . . . . . . . . . . . . . . . . . . . . . . 760
20.3 Approximating Solutions to SDEs . . . . . . . . . . . . . . 761
20.4 S+FinMetrics Functions for Solving SDEs . . . . . . . . . . 765
20.4.1 Problem-Specific Simulators . . . . . . . . . . . . . . 765
20.4.2 General Simulators . . . . . . . . . . . . . . . . . . . 771
20.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 782
21 Generalized Method of Moments
785
21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 785
21.2 Single Equation Linear GMM . . . . . . . . . . . . . . . . . 786
21.2.1 Definition of the GMM Estimator . . . . . . . . . . 787
21.2.2 Specification Tests in Overidentified Models . . . . . 791

Contents

21.2.3 Two-Stage Least Squares as an Eﬃcient GMM Estimator . . . . . . . . . . . . . . . . . . . . . . . . . .
21.3 Estimation of S . . . . . . . . . . . . . . . . . . . . . . . . .
21.3.1 Serially Uncorrelated Moments . . . . . . . . . . . .
21.3.2 Serially Correlated Moments . . . . . . . . . . . . .
21.3.3 Estimating S Using the S+FinMetrics
Function var.hac . . . . . . . . . . . . . . . . . . .
21.4 GMM Estimation Using the S+FinMetrics Function GMM .
21.5 Hypothesis Testing for Linear Models . . . . . . . . . . . .
21.5.1 Testing Restrictions on Coeﬃcients . . . . . . . . . .
21.5.2 Testing Subsets of Orthogonality Conditions . . . .
21.5.3 Testing Instrument Relevance . . . . . . . . . . . . .
21.6 Nonlinear GMM . . . . . . . . . . . . . . . . . . . . . . . .
21.6.1 Asymptotic Properties . . . . . . . . . . . . . . . . .
21.6.2 Hypothesis Tests for Nonlinear Models . . . . . . . .
21.7 Examples of Nonlinear Models . . . . . . . . . . . . . . . .
21.7.1 Student’s t Distribution . . . . . . . . . . . . . . . .
21.7.2 MA(1) Model . . . . . . . . . . . . . . . . . . . . . .
21.7.3 Euler Equation Asset Pricing Model . . . . . . . . .
21.7.4 Stochastic Volatility Model . . . . . . . . . . . . . .
21.7.5 Interest Rate Diﬀusion Model . . . . . . . . . . . . .
21.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxi

792
793
794
794
797
797
808
808
812
813
816
818
819
819
819
821
827
833
838
842

22 Seminonparametric Conditional Density Models
847
22.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 847
22.2 Overview of SNP Methodology . . . . . . . . . . . . . . . . 848
22.3 Estimating SNP Models in S+FinMetrics . . . . . . . . . . 851
22.3.1 Example Data . . . . . . . . . . . . . . . . . . . . . 853
22.3.2 Markovian Time Series and the Gaussian Vector
Autoregression Model . . . . . . . . . . . . . . . . . 855
22.3.3 Hermite Expansion and the Semiparametric VAR . . 860
22.3.4 Conditional Heterogeneity . . . . . . . . . . . . . . . 868
22.3.5 ARCH/GARCH Leading Term . . . . . . . . . . . . 874
22.4 SNP Model Selection . . . . . . . . . . . . . . . . . . . . . . 880
22.4.1 Random Restarts . . . . . . . . . . . . . . . . . . . . 881
22.4.2 The expand Function . . . . . . . . . . . . . . . . . 886
22.4.3 The SNP.auto Function . . . . . . . . . . . . . . . . 889
22.5 SNP Model Diagnostics . . . . . . . . . . . . . . . . . . . . 891
22.5.1 Residual Analysis . . . . . . . . . . . . . . . . . . . . 892
22.5.2 Simulation . . . . . . . . . . . . . . . . . . . . . . . 896
22.6 Prediction from an SNP Model . . . . . . . . . . . . . . . . 897
22.7 Data Transformations . . . . . . . . . . . . . . . . . . . . . 899
22.7.1 Centering and Scaling Transformation . . . . . . . . 899
22.7.2 Transformations to Deal with Heavy Tailed Data . . 901

xxiiContents

22.7.3 Transformation to Deal with Small SNP
Density Values . . . . . . . . . . . . . . . . . . . . . 903
22.8 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904
22.8.1 SNP Models for Daily Returns on Microsoft Stock . 904
22.8.2 SNP Models for Daily Returns on the S&P 500 Index 909
22.8.3 SNP Models for Weekly 3-Month U.S. T-Bill Rates . 914
22.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 920
23 Eﬃcient Method of Moments
923
23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 923
23.2 An Overview of the EMM Methodology . . . . . . . . . . . 925
23.2.1 Continuous-Time Stochastic Volatility Model forInterest Rates . . . . . . . . . . . . . . . . . . . . . . 925
23.2.2 Minimum Chi-Squared Estimators . . . . . . . . . . 928
23.2.3 Eﬃciency Considerations . . . . . . . . . . . . . . . 930
23.2.4 A General Purpose Auxiliary Model . . . . . . . . . 935
23.2.5 The Projection Step . . . . . . . . . . . . . . . . . . 935
23.2.6 The Estimation Step . . . . . . . . . . . . . . . . . . 936
23.3 EMM Estimation in S+FinMetrics . . . . . . . . . . . . . . 938
23.3.1 Simulator Functions . . . . . . . . . . . . . . . . . . 940
23.3.2 SNP Auxiliary Model Estimation . . . . . . . . . . . 943
23.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 943
23.4.1 MA(1) Model . . . . . . . . . . . . . . . . . . . . . . 944
23.4.2 Discrete-Time Stochastic Volatility Models . . . . . 954
23.4.3 Interest Rate Diﬀusion Models . . . . . . . . . . . . 966
23.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986
Index

991

This is page 1
Printer: Opaque this1
S and S-PLUS

1.1 Introduction
S-PLUS is a commercial software package developed by Insightful Corporation, based on the S language originally developed at Bell Laboratories(of AT&T and now Lucent Technologies) for statistical computation andvisualization. Both S and S-PLUS have evolved through many versions. In
1999 John M. Chambers, the principal developer of S language, received
the prestigious Software System Award from the Association for Computing Machinery (ACM), which has been awarded to UNIX, TEX, PostScript,
TCP/IP and World Wide Web in the past.
The discussion of S language in this book is based on S-PLUS 6, which
is supported on Microsoft Windows, Sun Solaris, and LINUX operating
systems1 . In addition to S-PLUS 6 Programmer’s Guide, there are many
excellent books available introducing diﬀerent aspects of S and S-PLUS (seeSection 1.4 for a list of them), and refer to these books if you are not familiar
with S or S-PLUS. This chapter has a rather limited goal: to introduce
the object oriented approach of S language and summarize some modelingconventions that will be followed in this book. Section 1.2 introduces theconcept of objects in S language, and Section 1.3 summarizes the usage
1 Some of the examples in the book have been updated to make use of new featuresin S-PLUS 7. All of the examples in Chapters 18 through 23 make use of S+FinMetrics
2.0, which is based on S-PLUS 7.

2

1. S and S-PLUSof modeling functions in S-PLUS and S+FinMetrics. Finally, Section 1.4
points out some useful resources for learning and using S-PLUS.

1.2 S Objects
1.2.1 Assignment
As the S language evolved over time, diﬀerent assignment operators havebeen used, such as =, <-, <<-, and (underscore). This book will use theassignment operator = whenever possible, because it is more intuitive andrequires only one key stroke. For example, in the command window of an
S-PLUS session, use the following command to assign the value of 3 to avariable called a:> a = 3> a[1] 3
When the name of the variable is typed at the command prompt, the valueof the variable is printed on screen with an index [1]. Since is reserved
as an assignment operator, it cannot be used in the names of any object.
Avoid the use of as an assignment operator, because the code may lookconfusing to someone who is not familiar with S.
Although = has been chosen as the assignment operator whenever possible, only <- can be used as the assignment operator if the assignment is
inside a function call.2 For example, suppose the user wants to assign thevalue of 10 to the variable a, and use a to initialize a 5 × 5 matrix. If = isused as the assignment operator, an error message appears:> matrix(a = 10, 5, 5)
Problem in matrix: argument a= not matched: matrix(a = 10, 5, 5)
Use traceback() to see the call stack
But if the assignment operator <- is used, the desired behavior is achieved:> matrix(a <- 10, 5, 5)[,1] [,2] [,3] [,4] [,5]
[1,]
10
10
10
10
10
[2,]
10
10
10
10
10
[3,]
10
10
10
10
10
[4,]
10
10
10
10
10
[5,]
10
10
10
10
10
> a
2 The reason is that S-PLUS functions allow optional arguments with default values,
and = is used to set the default values in a function call.

1.2 S Objects

3[1] 10and 10 is successfully assigned as the value of a.

1.2.2 Class
Since the S language is object oriented, everything in S-PLUS is an object
with a class, and the class function can be used to find out the class ofan object. For example:> class(a)[1] "integer"
thus the variable a has class “integer”. Explicitly using the decimal point
forces an integer number to be stored in double precision:> b = 100000.
> class(b)[1] "numeric"A number with double precision in S-PLUS has class “numeric”. In mostsituations S-PLUS is “smart” enough to perform computations in double
precision if necessary. However, one has to be a little careful with integer
arithmetic. For example, the following operation returns an NA:> 100000 * 100000[1] NAbecause in this case, the multiplication is performed in integer mode, andthe largest integer on a 32-bit machine is:> 2^31 - 1[1] 2147483647
which can be verified by querying the integer.max component of the machine constant object in S-PLUS:3> .Machine$integer.max[1] 2147483647
However, since the variable b created earlier is stored in double precision,
the multiplication using b would return the desired result:> b * b[1] 1e+10
Together with “logical” and “character”, “integer” and “numeric”
objects are known as the atomic objects, upon which the user can build
more complicated data structure, such as matrix, list, data frame, function,
3 See the on-line help file for .Machine for other components in the list.

4

1. S and S-PLUS

etc. For example, use the concatenation function c to combine the variables
a and b into a vector, and use the matrix function to reshape it into a 2× 1
matrix:> abMat = matrix(c(a,b), nrow=2)> class(abMat)[1] "matrix"> abMat[,1][1,] 1e+01[2,] 1e+05
As another example, although matrix is a built-in function in S-PLUS,
it is just another object in S-PLUS:> class(matrix)[1] "function"> matrix
function(data = NA, nrow = 1, ncol = 1, byrow = F, dimnames){
if(missing(nrow))nrow <- ceiling(length(data)/ncol)
else if(missing(ncol))ncol <- ceiling(length(data)/nrow)dim <- c(nrow, ncol)
if(length(dim) != 2)stop("nrow and ncol should each be of length 1")
value <- if(byrow) t(array(data, dim[2:1])) else
array(data, dim)if(!missing(dimnames))value@.Dimnames <- dimnames
value}
The preceding output shows that matrix is just a “function” object. Whenthe name of this object is typed, S-PLUS prints its function definition on
the screen.
Most complicated S-PLUS objects are constructed as a list. For example,
combine the variables a and b into a list as follows:> abList = list(aComponent=a, bComponent=b)> class(abList)[1] "list"> abList$aComponent:[1] 101.2 S Objects

5$bComponent:[1] 1e+05
where the names aComponent and bComponent are given to a and b, respectively. Use the length function to find out the number of components
in a list and the names function to extract the names of those components:> length(abList)[1] 2> names(abList)[1] "aComponent" "bComponent"
A particular component of a list can be extracted using the $ operator. For
example:> abList$aComponent[1] 10or the [[ operator:> abList[[2]][1] 1e+05
S-PLUS 6 is based on S language Version 4 (SV4). In SV4, a new classstructure is introduced to build more complicated objects, as an alternative to using lists. One example is the “timeDate” objects in S-PLUS. For
example, in the following example, use the timeDate function to parse avector of character strings representing some dates:> timeStamp = timeDate(c("1/1/2001", "1/2/2001", "1/3/2001"))> timeStamp[1] 01/01/2001 01/02/2001 01/03/2001> class(timeStamp)[1] "timeDate"
The names function cannot be used with these new class objects, which
will be referred to as SV4 objects. Instead, use the slotNames function to
extract the names of their components. For example:> slotNames(timeStamp)[1] ".Data"".Data.names"[5] "time.zone"".Data.classes" "format"

A “timeDate” object has five slots. Instead of using the $ operator as
for lists, use the @ operator to extract the component in a particular slot.
For example:> timeStamp@.Data[[1]]:[1] 14976 14977 14978

6

1. S and S-PLUS[[2]]:[1] 0 0 0
The .Data slot of a “timeDate” object actually stores a list with twocomponents.4
One diﬀerence between the list based objects and SV4 objects is thatthe list based objects are more flexible and thus prone to cause accidental
programming errors if a programmer is not careful enough. In contrast, theSV4 objects are more stringently defined and can lead to robust softwareand computational eﬃciency. For example, the user can add or delete a
component to a list at will:> abList$anotherComponent = "a string component"> abList$aComponent:[1] 10$bComponent:[1] 1e+05$anotherComponent:[1] "a string component"> abList$aComponent = NULL
> abList$bComponent:[1] 1e+05$anotherComponent:[1] "a string component"
However, an SV4 object is strictly defined, and a component cannot beedited unless it is defined in its declaration:> timeStamp@time.zone[1] "GMT"> timeStamp@time.zone = "Pacific"> timeStamp@anotherSlot = "no way"
Problem in timeStamp@anotherSlot = "no way": Class "timeDate"
has no "anotherSlot" slot
Use traceback() to see the call stack
4 The first component represents the Julian dates, and the second component represents the milliseconds elapsed since midnight of each day.

1.2 S Objects

7

1.2.3 Method
Many S-PLUS functions are defined as generic in the sense that the user
has the freedom of defining his or her own method for a particular class.
For example, the print and summary functions in S-PLUS are so genericthat they work with any object and may generate diﬀerent types of results
depending on the class of the object.5 For example:> summary(abMat)
Min. 1st Qu. Median
Mean 3rd Qu.
Max.
10 25008
50005 50005 75002 100000> summary(abList)
Length Class
Mode
bComponent 1
numeric
anotherComponent 1
character
For a numeric matrix object, the summary method generates some sample statistics, while for a list object, the summary method summarizes thelength and mode of each component.
In the above example, S-PLUS is “smart” enough to figure out the appropriate method to use for the generic summary function. If the name of
the method function is known, the user can also call the method function
directly. For example, if the user types matrix at the command prompt,
S-PLUS will dispatch the print method for “function” objects because
matrix is a “function” object. However, it can also call the function
print.list on a “function” object to view the object using another format:> print.list(matrix)$data:
NA$nrow:[1] 1$ncol:[1] 1$byrow:F$dimnames:

5 In fact, typing the name of an object at the command prompt, S-PLUS calls theprint method of that object automatically. So any print methods rarely need to becalled explicitly, except for Trellis graphics objects.

8

1. S and S-PLUS$"":{
if(missing(nrow))nrow <- ceiling(length(data)/ncol)
else if(missing(ncol))ncol <- ceiling(length(data)/nrow)dim <- c(nrow, ncol)
if(length(dim) != 2)stop("nrow and ncol should each be of length 1")
value <- if(byrow) t(array(data, dim[2:1])) else
array(data, dim)if(!missing(dimnames))value@.Dimnames <- dimnames
value}

1.3 Modeling Functions in S+FinMetrics
In this book, many statistical and econometric examples are illustratedusing modeling functions in S+FinMetrics. Some modeling functions in
S+FinMetrics are named using upper case acronyms as they are known
in the literature, because S is case sensitive and it distinguishes between
upper case and lower case letters.

1.3.1 Formula Specification
For many modeling functions in S+FinMetrics, S formulas are used to specify the model to be estimated. Chambers and Hastie (1993) and S-PLUS
Guide to Statistics provide detailed examples of how to specify models using
formulas in S. This section points out some restrictions in formula specification so that the user can avoid some errors in using these functions.
For illustrations, use the S-PLUS lm function as an example of modeling
function.
If a formula is used to specify models in a modeling function, usually atleast two arguments are supplied to the function: a formula object and adata object. The args function can always be used to find out the argument
names of any function:> args(lm)function(formula, data, weights, subset, na.action, method ="qr", model = F, x = F, y = F, contrasts = NULL, ...)
NULL

1.3 Modeling Functions in S+FinMetrics

9

The data object must be a “data.frame” object, or a “timeSeries”
object with a “data.frame” in its data slot. First create a data frame
using the S-PLUS data objects stack.x and stack.loss:> stack.df = data.frame(Loss=stack.loss, stack.x)> colIds(stack.df)[1] "Loss""Air.Flow""Water.Temp" "Acid.Conc."
so the data frame stack.df has four columns with variable names as shown
above.
To regress the variable Loss on Air.Flow, and Water.Temp using leastsquares, use the lm function as follows:> test.mod = lm(Loss~Air.Flow + Water.Temp, data=stack.df)> test.mod
Call:lm(formula = Loss ~ Air.Flow + Water.Temp, data = stack.df)
Coefficients:(Intercept) Air.Flow Water.Temp-50.35884 0.6711544
1.295351
Degrees of freedom: 21 total; 18 residual
Residual standard error: 3.238615
Notice that in the first formula object, Loss is on the left hand side of ~, so
it represents the endogenous or response variable of the model; Air.Flow
and Water.Temp are on the right hand side of ~, so they represent two
independent or explanatory variables. An intercept or a constant term isalso included automatically, as can be seen from the coeﬃcient estimatesin the output, which is generated by a call to the print method for “lm”
objects:> class(test.mod)[1] "lm"> oldClass(test.mod)[1] "lm"
Note that since an “lm” object is a list based object, the user can also
use the oldClass function to obtain its class. However, oldClass function
does not work with SV4 objects. For example:> oldClass(timeStamp)
NULL
The data argument can also be a “timeSeries” object with a data
frame in its data slot. To illustrate this possibility, turn stack.df into a“timeSeries” object and call it stack.ts:

10

1. S and S-PLUS> stack.ts = timeSeries(stack.df)> stack.ts
Positions Loss Air.Flow Water.Temp Acid.Conc.
01/01/1960 4280
27
89
01/02/1960 37
80
27
88
01/03/1960 3775
25
90
01/04/1960 28
6224
87
01/05/1960 18
62
22
87
...
Again, a linear model can be estimated using this data object just like inthe previous example:> test.mod = lm(Loss~Air.Flow + Water.Temp, data=stack.ts)
However, the data argument must have a data frame representation. The
same function call will generate an error if the data argument is represented
by a matrix:> stack.mat = as.matrix(stack.df)> lm(Loss~Air.Flow+Water.Temp, data=stack.mat)
Warning messages:
Numerical expression has 84 elements: only the first used in:model.frame(formula, data, na.action, dots)
Problem: Invalid frame number, 42
Use traceback() to see the call stack
For most modeling functions such as lm, the data argument is actuallyan optional argument, which is not required. If the data argument is not
supplied by the user, then the variables specified in the formula object
must be on the search path. For example:> lm(stack.loss~stack.x)
Call:lm(formula = stack.loss ~ stack.x)
Coefficients:(Intercept) stack.xAir Flow stack.xWater Temp stack.xAcid Conc.
-39.91967
0.7156402
1.295286-0.1521225
Degrees of freedom: 21 total; 17 residual
Residual standard error: 3.243364

1.3 Modeling Functions in S+FinMetrics

11

In addition, if the data argument is not supplied, the variables specified
in the formula object must be either a vector or a matrix, and they cannot
be a data frame nor a “timeSeries” object. For example:6> stack.x.df = as.data.frame(stack.x)> lm(stack.loss~stack.x.df)
Problem: Length of stack.x.df (variable 2) is 3 != length ofothers (21)
Use traceback() to see the call stack> stack.loss.ts = timeSeries(stack.loss)> lm(stack.loss.ts~stack.x)
Problem: Length of stack.loss.ts (variable 1) is 11 != lengthof others (21)
Use traceback() to see the call stack
In S+FinMetrics, the formula is extended to support autoregressive
specification, moving average specification, distributed lags and polynomial distributed lags for many modeling functions. These formulas will beillustrated in the appropriate chapters.

1.3.2 Method
In addition to print and summary functions, many other functions in
S-PLUS are defined to be generic to work with modeling functions andobjects, such as plot for diagnostic plots, coefficients or simply coef
for extracting model coeﬃcients, residuals for extracting model residuals, fitted.values or simply fitted for extracting model fitted values,
predict for out of sample prediction, etc. For example, for the “lm” object test.mod, if the generic functions coef, predict or plot are applied,
S-PLUS will figure out the appropriate method to use:
> coef(test.mod)(Intercept) Air.Flow Water.Temp-50.35884 0.6711544
1.295351> predict(test.mod, matrix(1, 5, 3))[1] -48.39233 -48.39233 -48.39233 -48.39233 -48.39233> plot(test.mod, ask=T)
Make a plot selection (or 0 to exit):
6 In fact, many modeling functions in S+FinMetrics actually does allow a“timeSeries” object on the left hand side of the formula, but not the right hand side of
the formula, if the data argument is not supplied. One example is the garch function.

12

1. S and S-PLUS

1: plot: All
2: plot: Residuals vs Fitted Values
3: plot: Sqrt of abs(Residuals) vs Fitted Values
4: plot: Response vs Fitted Values
5: plot: Normal QQplot of Residuals
6: plot: r-f spread plot
7: plot: Cook’s Distances
Selection:
In addition to the above generic functions, S+FinMetrics defines three
new generic functions for working with model objects: vcov for extracting
the variance-covariance matrix of estimated parameters, simulate for generating simulations according to the estimated model, and cpredict for
obtaining conditional forecasts of multivariate time series models.

1.4 S-PLUS Resources
1.4.1 Books
In addition to the S-PLUS manuals, there are a number of good books onusing and programming in S and S-PLUS as well as data and statisticalanalysis using S-PLUS. The Insightful web page
http://www.insightful.com/support/splusbooks.asp
contains a listing of these books.
Using and Programming S-PLUS
Gentle introductions to S and S-PLUS are given in Spector (1994), Lam(2001) and Krause and Olson (2005). The details of version four of the
S language are described in Chambers (1998), also known as the “greenbook”. An indispensable guide to programming in the S language is Venables and Ripley (2000).
Data and Statistical Analysis in S-PLUS
S-PLUS provides extensive functionality for the statistical analysis of a wide
variety of data, and many books have been written on the use of S-PLUS forparticular applications. The following books describe statistical techniquesthat are useful for the analysis of financial data. Carmona (2004) and Chan
(2002) describe the use of S-PLUS for the analysis of financial time series.
Scherer and Martin (2005) cover portfolio optimization and related topic.
An excellent guide to modern applied statistics using S-PLUS is Venables
and Ripley (2002). Harrell (2001) gives a thorough treatment of regression

1.5 References

13

models, including generalized linear models and survival model. Heiberger
and Holland (2004) emphasize the importance of graphical techniques instatistical analysis. Pinheiro and Bates (2000) detail the analysis of mixed
eﬀects (panel data) models. Therneau and Grambsch (2000) survey survival
analysis models. Wilcox (1997), and Atkinson and Riani (2000) discussrobust statistical methods. Bruce and Gao (1996) describe wavelet analysis.
Hastie, Tibshirani and Friedman (2001) cover aspects of statistical learningand data mining. Davison and Hinkley (1997) survey bootstrap methods,
and Bowman and Azzalini (1997) disucss nonparametric and smoothing
methods.

1.4.2 Internet
There is a wealth of information about S-PLUS available on the internet.
The obvious place to start is the Insightful website at
http://www.insightful.com
S-News is an electronic mail discussion list for S and S-PLUS users. Information about S-News may be found at
http://www.biostat.wustl.edu/s-news/s-news-intro.html
StatLib is a repository for software and extensions for the S language,
including many useful additions to S-PLUS. It can be found at
http://lib.stat.cmu.edu/S
Eric Zivot maintains a website containing links to websites for S-PLUS
applications in econometrics and finance at
http://faculty.washington.edu/ezivot/splus.htm

1.5 References
Atkinson, A. and M. Riani (2000). Robust Diagnostic Regression Analysis. Springer-Verlag, New York.
Bowman, A.W., and A. Azzalini (1997). Applied Smoothing Techniques
for Data Analysis:The Kernel Approach with S-PLUS Illustrations. Oxford
University Press, Oxford.
Bruce, A. and H.-Y. Gao (1996). Applied Wavelet Analysis with SPLUS. Springer-Verlag, New York.
Carmona, R. (2004). Statistical Analysis of Financial Data in S-PLUS.
Springer-Verlag, New York.

14

1. S and S-PLUS

Chambers, J.M. (1998). Programming with Data. Springer-Verlag, NewYork.
Chambers, J. M., and Hastie, T. J. (1993). Statistical Models in S.
Chapman & Hall.
Chan, N.H. (2002). Time Series: Applications to Finance. John Wiley &Sons, New York.
Davidson, A.C. and D.V. Hinkley (1997). Bootstrap Methods and Their
Application. Cambridge University Press, Cambridge, UK.
Harrell, F.E. (2001). Regression Modeling Strategies with Applications toLinear Models, Logistic Regression, and Survival Analysis. Springer-Verlag,
New York.
Hastie, T., R. Tibshirani and J. Friedman (2001). The Elementsof Statistical Learning: Data Mining, Inference and Prediction. SpringerVerlag, New York.
Heiberger, R.M. and B. Holland (2004). Statistical Analysis and Data
Display. Springer-Verlag, New York.
Krause, A. and M. Olson (2005). The Basics of S-PLUS, Fourth Edition. Springer-Verlag, New York.
Lam, L. (2001). An Introduction to S-PLUS for Windows. Candiensten,
Amsterdam.
Pinheiro, J.C. and D.M. Bates (2000). Mixed-Eﬀects Models in S andS-PLUS. Springer-Verlag, New York.
Scherer, B. and R.D. Martin (2005). Modern Portfolio Optimization
with NuOPT, S-PLUS and S+Bayes. Springer-Verlag, New York.
Spector, P. (1994). An Introduction to S and S-PLUS. Duxbury Press,
Belmont, CA.
Therneau, T.M. and P.M Grambsch (2000). Modeling Survival Data.
Springer-Verlag, New York.
Venables, W.N. and B.D. Ripley (2002). Modern Applied Statistics
with S-PLUS, Fourth Edition. Springer-Verlag, New York.
Venables, W.N. and B.D. Ripley (1999). S Programming. SpringerVerlag, New York.
Wilcox, P. (1997). Introduction to Robust Estimation and Hypothesis
Testing. Academic Press, San Diego.

This is page 15
Printer: Opaque this2Time Series Specification,
Manipulation, andVisualization in S-PLUS

2.1 Introduction
Time series data may be stored, manipulated and visualized in a variety ofways in S-PLUS1 . This chapter discusses the basics of working with financial time series data in the form of S-PLUS “timeSeries” objects. It begins
with a discussion of the specification of “timeSeries” and “timeDate” objects in S-PLUS and gives examples of how to specify common “timeDate”
sequences for financial time series. Basic manipulations of financial timeseries are discussed and illustrated. These manipulations include aggregating and disaggregating time series, handling of missing values, creations oflags and diﬀerences and asset return calculations. The chapter ends withan overview of time series visualization tools and techniques, including the
S-PLUS plotting functions for “timeSeries” as well as specialized plotting
functions in S+FinMetrics.

2.2 The Specification of “timeSeries” Objects in
S-PLUS
Financial time series data may be represented and analyzed in S-PLUS ina variety of ways. By far the most flexible way to analyze, manipulate
1 Chapters 25-27 in the S-PLUS Guide to Statistic (Vol. II) discusses the analysis oftime series in S-PLUS.

16

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

and visualize time series data is through the use of S-PLUS calendar-based“timeSeries” objects. A calendar-based “timeSeries” object, hereafterreferred to as simply a “timeSeries” is an S version 4 (sv4) object that
stores time and date information from a “timeDate” object in a positions
slot and time series data from any rectangular data object (vector, matrix
or data frame) in a data slot. Additionally, summary information about
the time series may be stored in the title, documentation, units andattributes slots.
To illustrate a typical “timeSeries” object, consider the S+FinMetrics“timeSeries” object singleIndex.dat which contains monthly closing
price data on Microsoft and the S&P 500 index over the period January
1990 through January 2001:> class(singleIndex.dat)[1] "timeSeries"> slotNames(singleIndex.dat)[1] "data""positions"[4] "end.position""future.positions"[7] "title""documentation"[10] "fiscal.year.start" "type""start.position""units""attributes"> singleIndex.dat@title[1] "Monthly prices on Microsoft and S&P 500 Index"> singleIndex.dat@documentation[1] "Monthly closing prices over the period January 1900"[2] "through January 2001 adjusted for dividends and stock"[3] "splits.
> singleIndex.dat@units[1] "Monthly price"> singleIndex.dat[1:5,]
Positions
MSFT SP500
Jan 1990 1.2847 329.08
Feb 1990 1.3715 331.89
Mar 1990 1.5382 339.94
Apr 1990 1.6111 330.80
May 1990 2.0278 361.23
The date information in the positions slot may be extracted directly
or by using the positions extractor function:> singleIndex.dat@positions[1:5][1] Jan 1990 Feb 1990 Mar 1990 Apr 1990 May 1990

2.2 The Specification of “timeSeries” Objects in S-PLUS

17> positions(singleIndex.dat)[1:5][1] Jan 1990 Feb 1990 Mar 1990 Apr 1990 May 1990
The generic start and end functions may be used to extract the start andend dates of a “timeSeries” object:> start(singleIndex.dat)[1] Jan 1990
> end(singleIndex.dat)[1] Jan 2001
The date information in the positions slot is an object of class “timeDate”> class(positions(singleIndex.dat))[1] "timeDate"
Details on “timeDate” objects are given later on in this chapter.
The time series data in the data slot may be accessed directly or throughthe seriesData extractor function:> singleIndex.dat@data[1:5,]
MSFT SP500
1 1.2847 329.08
2 1.3715 331.89
3 1.5382 339.94
4 1.6111 330.80
5 2.0278 361.23> seriesData(singleIndex.dat)[1:5,]
MSFT SP500
1 1.2847 329.08
2 1.3715 331.89
3 1.5382 339.94
4 1.6111 330.80
5 2.0278 361.23
In general, the time series data in the data slot is a “rectangular” data
object and is usually a data frame or a matrix. For example,> class(seriesData(singleIndex.dat))[1] "data.frame"
In fact, “timeSeries” objects themselves are “rectangular” data objects
and so the functions numRows, numCols, colIds and rowIds may be used
to extract useful information:> is.rectangular(singleIndex.dat)[1] T> numRows(singleIndex.dat)[1] 133

18

2. Time Series Specification, Manipulation, and Visualization in S-PLUS> numCols(singleIndex.dat)[1] 2> colIds(singleIndex.dat)[1] "MSFT" "SP500"> rowIds(singleIndex.dat)[1:5][1] Jan 1990 Feb 1990 Mar 1990 Apr 1990 May 1990

2.2.1 Basic Manipulations
Basic manipulation of “timeSeries” objects may be done in the same
way as other S-PLUS objects. Mathematical operations may be applied to“timeSeries” objects in the usual way and the result will be a “timeSeries”
object. Subscripting a “timeSeries” works in the same way as subscripting a data frame or matrix. For example, a “timeSeries” with the prices
on Microsoft may be extracted from singleIndex.dat using> msft.p = singleIndex.dat[,"MSFT"]> msft.p = singleIndex.dat[,1]> msft.p@title = "Monthly closing price on Microsoft"> msft.p@documentation =+ c("Monthly closing price adjusted for stock",+ "splits and dividends.")> msft.p@units = "US dollar price"> class(msft.p)[1] "timeSeries"
Subsamples from a “timeSeries” may be extracted by creating an index oflogical values that are true for the times and dates of interest. For example,
consider creating a subsample from the “timeSeries” singleIndex.dat
over the period March 1992 through January 1993.
> smpl = (positions(singleIndex.dat) >= timeDate("3/1/1992") &+ positions(singleIndex.dat) <= timeDate("1/31/1993"))> singleIndex.dat[smpl,]
Positions MSFT SP500
Mar 1992 4.938 403.7
Apr 1992 4.594 414.9
May 1992 5.042 415.4
Jun 1992 4.375 408.1
Jul 1992 4.547 424.2
Aug 1992 4.656 414.0
Sep 1992 5.031 417.8
Oct 1992 5.547 418.7
Nov 1992 5.820 431.4
Dec 1992 5.336 435.7
Jan 1993 5.406 438.8

2.2 The Specification of “timeSeries” Objects in S-PLUS

19

S-PLUS 7 supports subscripting a “timeSeries” object directly with dates.
For example, the subsample from singleIndex.dat over the period March
1992 through January 1993 may be produced using> singleIndex.dat[timeEvent("3/1/1992","1/31/1993"),]
Most S-PLUS functions have methods to handle “timeSeries” objects.
Some common examples are the S-PLUS functions colMeans, colVars andcolStdevs which compute the mean, variance and standard deviation value
for each column of data:> colMeans(singleIndex.dat)
MSFT
SP500
26.74513 730.3805
For functions that do not have methods to handle “timeSeries” objects,
the extractor function seriesData should be used to extract the data slotof the “timeSeries” prior to applying the function:> colMeans(seriesData(singleIndex.dat))
MSFT
SP500
26.74513 730.3805
All of the S+FinMetrics modeling and support functions are designed
to accept “timeSeries” objects in a uniform way.

2.2.2 S-PLUS “timeDate” Objects
Time and date information in S-PLUS may be stored in “timeDate” objects.
The S-PLUS function timeDate is used to create “timeDate” objects. For
example, to create a “timeDate” object for the date January 1, 2002 for
the US Pacific time zone use> td = timeDate("1/1/2002",in.format="%m/%d/%Y",+ zone="Pacific")
The date information is specified in a character string and the optional
arguments in.format and zone determine the input date format and thetime zone, respectively. The input formats are single-element character vectors consisting of input fields which start with “%” and end with a letter.
The default input date format may be viewed with
> options("time.in.format")$time.in.format:[1] "%m[/][.]%d[/][,]%y [%H[:%M[:%S[.%N]]][%p][[(]%3Z[)]]]"and examples of common date formats can be found in the S-PLUS object
format.timeDate> names(format.timeDate)

20

2. Time Series Specification, Manipulation, and Visualization in S-PLUS[1] "1/3/1998"[2] "3/1/1998"...
[32] "03 Jan 1998 14:04:32 (PST)"> format.timeDate[[1]]$input[1] "%m/%d/%Y"The result of timeDate is an object of class “timeDate”> class(td)[1] "timeDate"> td[1] 1/1/02 0:00:00 AM> slotNames(td)[1] ".Data"".Data.names"[4] "format""time.zone"".Data.classes"“timeDate” objects have a number of slots that are used to specify andcontrol time and date information. Full details may be seen using> ?class.timeDate
The .Data slot is a list with components giving the Julian date representation of the day and time within the day. The Julian day represents thenumber of days since January 1, 1960 and the Julian time within the day
indicates the number of milliseconds since midnight Greenwich mean time
(GMT)
> td@.Data[[1]]:[1] 15341[[2]]:[1] 28800000
Since the US Pacific Time Zone is 8 hours behind GMT, the number ofmilliseconds since Greenwich mean time is 8 ∗ 60 ∗ 60 ∗ 1000 = 28, 800, 000.
The output display format of the date information is specified in the format
slot> td@format[1] "%m/%d/%02y %H:%02M:%02S %p"
Like input formats, output formats are single-element character vectors
consisting of output fields, which start with “%” and end with a letter,and other characters that are simply printed. The above format specifies
printing the date as month/day/year and then hour:minute:second and
AM or PM. The integers 02 before y, M and S fix the output width to 2
characters. All supported output fields are described in the help file for2.2 The Specification of “timeSeries” Objects in S-PLUS

21

class.timeDate and a list of example output formats are given in the
S-PLUS object format.timeDate. For example,> names(format.timeDate)[18][1] "03 Jan 1998"> format.timeDate[[18]]$output[1] "%02d %b %Y"Time Zone Issues
The time and date information stored in a “timeDate” object is aligned tothe time zone specified in the time.zone slot
> td@time.zone[1] "Pacific"
To modify the output format of a “timeDate” object to display time zone
information simply add "%z"
> td@format = paste(td@format,"%z")> td[1] 1/1/02 0:00:00 AM Pacific
The object td is aligned to the US Pacific time zone. If the zone argument
to timeDate is omitted when the “timeDate” object is created the default
time zone in options(‘‘time.zone") is used2 . For example,> options("time.zone")$time.zone:[1] "Pacific"> td2 = timeDate("Mar 02, 1963 08:00 PM",+ in.format="%m %d, %Y %H:%M %p",
+ format="%b %02d, %Y %02I:%02M %p %z")
> td2
[1] Mar 02, 1963 08:00 PM PacificNote that the above example shows that the output format of the “timeDate”
object can be specified when the object is created using the argument
format.
All of the time zone specifications supported by S-PLUS are describedin the help file for class.timeZone and these specifications are defined
relative to times and dates given in GMT. The time zone specifications
include daylight savings time in various areas around the world. To see
how a time zone specification aﬀects a timeDate object, consider what
2 On Windows platforms, the time zone specification is obtained from the Windows
regional settings. The examples in this section were created on a Windows computer in
the U.S. Pacific time zone. Therefore, the default time zone taken from the Windows
regional settings is “Pacific”.

22

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

happens when the time zone for the object td is changed to US EasternTime:> td@time.zone = "Eastern"> td[1] 1/1/02 3:00:00 AM Eastern> td@.Data[[1]]:[1] 15341[[2]]:[1] 28800000
Since US Eastern Time is three hours ahead of US Pacific Time the displayed date is moved ahead three hours. That is, midnight US Pacific Timeon January 1, 2002 is the same as 3 AM US Eastern Time on January 1,
2002. Notice that changing the time zone information does not alter the
Julian date information in the .Data slot. To align the Julian date representation to reflect the number of milliseconds from GMT on US Easterntime the millisecond information in the second component of the .Data slot
must be adjusted directly.
If a “timeDate” object is created in GMT then the S-PLUS function
timeZoneConvert may be used to re-align the millisecond oﬀset to a specified time zone. For example,> tdGMT = timeDate("1/1/2002",zone="GMT",+ format="%m/%d/%02y %H:%02M:%02S %p %z")
> tdGMT[1] 1/1/02 0:00:00 AM GMT> tdGMT@.Data[[1]]:[1] 15341[[2]]:[1] 0> tdPST = timeZoneConvert(tdGMT,"PST")> tdPST[1] 1/1/02 0:00:00 AM PST> tdPST@.Data[[1]]:[1] 15341[[2]]:[1] 28800000

2.2 The Specification of “timeSeries” Objects in S-PLUS

23

Be aware that timeZoneConvert is not designed to convert the millisecond
oﬀsets from one arbitrary time zone other than GMT to another arbitrary
time zone.
Mathematical Operations with “timeDate” Objects
Since “timeDate” objects have a Julian date representation, certain mathematical operations like addition and subtractions of numbers may be performed on them and the result will also be a “timeDate” object. For example,> td1 = timeDate("1/1/2002",in.format="%m/%d/%Y",+ zone="GMT",format="%m/%d/%04Y %H:%02M:%02S %p %z")
> td2 = timeDate("2/1/2002",in.format="%m/%d/%Y",+ zone="GMT",format="%m/%d/%04Y %H:%02M:%02S %p %z")
> td1
[1] 1/1/2002 0:00:00 AM GMT> td2[1] 2/1/2002 0:00:00 AM GMT> as.numeric(td1)[1] 15341> td1 + 1
[1] 1/2/2002 0:00:00 AM GMT> td1 + 0.5[1] 1/1/2002 12:00:00 PM GMT> td1 - 1
[1] 12/31/2001 0:00:00 AM GMT> 2*td1[1] 30682> td1+td2[1] 2/2/2044 0:00:00 AM GMT
Adding two “timeDate” objects together creates another “timeDate”
object with date given by the addition of the respective Julian dates. Subtraction of two “timeDate” objects, however, produces an sv4 object ofclass “timeSpan”> td.diff = td2 - td1> class(td.diff)[1] "timeSpan"> td.diff[1] 31d 0h 0m 0s 0MS> slotNames(td.diff)[1] ".Data"".Data.names"[4] "format"".Data.classes"

24

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

The “timeSpan” object td.diff gives the time diﬀerence between td1 andtd2 - 31 days, 0 hours, 0 minutes, 0 seconds and 0 milliseconds. The Julian
date information is kept in the .Data slot and the output format is in theformat slot. Details about “timeSpan” objects is given in The S-PLUS
Guide to Statistics, Vol. II, chapter 25.

2.2.3 Creating Common “timeDate” Sequences
Most historical financial time series are regularly spaced calendar-based
time series; e.g. daily, monthly or annual time series. However, some financial time series are irregularly spaced. Two common examples of irregularly spaced financial time series are daily closing prices and intra-day
transactions level data. There are a variety of time and date functions in
S-PLUS that may be used to create regularly spaced and irregularly spaced“timeDate” sequences for essentially any kind of financial data. These functions are illustrated using the following examples3 .
Regularly and irregularly spaced sequences may be created using the
S-PLUS functions timeCalendar, timeSeq and timeSequence. The function timeSeq is the most flexible. The following examples illustrate the useof these functions for creating common “timeDate” sequences.
Annual Sequences
Creating a “timeDate” sequence for an annual time series from 1900 to1910 may be done in a variety of ways. Perhaps, the simplest way uses the
S-PLUS timeCalendar function:> td = timeCalendar(y=1900:1910,format="%Y")> class(td)[1] "timeDate"> td[1] 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910
The timeCalendar function produces an object of class “timeDate”.
The argument format="%Y" specifies the output format of the “timeDate”
object as a four digit year.
Since td contains a sequence of dates, the Julian date information for allof the dates is available in the .Data slot
> td@.Data[[1]]:[1] -21914 -21549 -21184 -20819 -20454 -20088 -19723 -19358[9] -18993 -18627 -18262
3 To avoid problems with time zone specifications, all examples in this sections werecreated after setting the default time zone to GMT using options(time.zone="GMT").

2.2 The Specification of “timeSeries” Objects in S-PLUS

25[[2]]:[1] 0 0 0 0 0 0 0 0 0 0 0
An annual sequence from 1900 to 1910 may also be computed using the
S-PLUS function timeSeq:> timeSeq(from="1/1/1900", to="1/1/1910", by="years",+ format="%Y")[1] 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910
The argument by="years" specifies annual spacing between successive
values in the sequence starting at 1/1/1900 and ending at 1/1/1910. The
date formats for the starting and ending dates must conform to the default
input format for “timeDate” objects (see options("time.in.format")).
Finally, an annual sequence from 1900 to 1910 may be created using the
S-PLUS function timeSequence:> tds = timeSequence("1/1/1900","1/1/1910",by="years",+ format="%Y")> class(tds)[1] "timeSequence"
> tds
from:
1900
to:
1910
by:
+1yr
[1] 1900 1901 1902 ... 1910
timeSequence creates an object of class “timeSequence” which stores timeand date information in a compact fashion. The “timeSequence” object
may be converted to a “timeDate” object using the S-PLUS as function> td = as(tds,"timeDate")> td[1] 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910
Quarterly Sequences
A quarterly “timeDate” sequence from 1900:I through 1902:IV may becreated using timeSeq with the by="quarters" option:> timeSeq(from="1/1/1900", to="10/1/1902", by="quarters",+ format="%Y:%Q")[1] 1900:I
1900:II 1900:III 1900:IV 1901:I
1901:II[7] 1901:III 1901:IV 1902:I
1902:II 1902:III 1902:IV
The output format character %Q displays the quarter information. Noticethat the dates are specified as the first day of the quarter.

26

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

Monthly Sequences
Now consider creating a monthly “timeDate” sequence from January 1,
1900 through March 1, 1901. This may be done using timeCalendar> timeCalendar(m=rep(1:12,length=15),y=rep(1900:1901,each=12,+ length=15), format="%b %Y")
[1] Jan 1900 Feb 1900 Mar 1900 Apr 1900 May 1900 Jun 1900
[7] Jul 1900 Aug 1900 Sep 1900 Oct 1900 Nov 1900 Dec 1900
[13] Jan 1901 Feb 1901 Mar 1901
or timeSeq> timeSeq(from="1/1/1900",to="3/1/1901",by="months",+ format="%b %Y")
[1] Jan 1900 Feb 1900 Mar 1900 Apr 1900 May 1900 Jun 1900
[7] Jul 1900 Aug 1900 Sep 1900 Oct 1900 Nov 1900 Dec 1900
[13] Jan 1901 Feb 1901 Mar 1901
To create a monthly sequence of end of month values from December 31,
1899 through February 28, 1901, subtract 1 from the above calculation:> timeSeq(from="1/1/1900",to="3/1/1901",by="months",+ format="%b %Y") - 1[1] Dec 1899 Jan 1900 Feb 1900 Mar 1900 Apr 1900 May 1900
[7] Jun 1900 Jul 1900 Aug 1900 Sep 1900 Oct 1900 Nov 1900
[13] Dec 1900 Jan 1901 Feb 1901
Weekly Sequences
Weekly sequences are best created using timeSeq with by="weeks". For
example, a weekly sequence from Monday January 1, 1990 to Monday Feb
26, 1990 may be created using> timeSeq(from="1/1/1990",to="3/1/1990",by="weeks",+ format="%a %b %d, %Y")
[1] Mon Jan 1, 1990 Mon Jan 8, 1990 Mon Jan 15, 1990
[4] Mon Jan 22, 1990 Mon Jan 29, 1990 Mon Feb 5, 1990
[7] Mon Feb 12, 1990 Mon Feb 19, 1990 Mon Feb 26, 1990
To create a weekly sequence starting on a specific day, say Wednesday,make the starting date a Wednesday.
Daily Sequences
A regularly spaced daily sequence may be created using timeSeq with by ="days". For an irregularly spaced daily sequence of weekdays use timeSeq
with by = "weekdays". For financial asset price data that trades on U.S.
exchanges, the relevant “daily” sequence of dates is an irregularly spaced

2.2 The Specification of “timeSeries” Objects in S-PLUS

27

sequence based on business days. Business days are weekdays excluding certain holidays. For example, consider creating a daily “timeDate” sequence
for the month of January, 2000 for a time series of asset prices that tradeon the New York stock exchange (NYSE). The NYSE is not open on weekends and on certain holidays and these dates should be omitted from the“timeDate” sequence. The S-PLUS function holiday.NYSE returns the New
York Stock Exchange holidays for a given year, 1885-present, according to
the historical and current (as of 1998) schedule, not including special-event
closure days or partial-day closures. The NYSE holidays for 2000 are> holiday.NYSE(2000)[1] 1/17/2000 2/21/2000 4/21/2000 5/29/2000[6] 9/4/2000
11/23/2000 12/25/2000

7/4/2000

Martin Luther King day on Monday January 17th is the only weekday
holiday. A “timeDate” sequence of business days excluding the holiday
1/17/2000 may be created using> timeSeq(from="1/3/2000",to="1/31/2000",by="bizdays",+ holidays=holiday.NYSE(2000),format="%a %b %d, %Y")
[1] Mon Jan 3, 2000 Tue Jan 4, 2000 Wed Jan 5, 2000
[4] Thu Jan 6, 2000 Fri Jan 7, 2000 Mon Jan 10, 2000
[7] Tue Jan 11, 2000 Wed Jan 12, 2000 Thu Jan 13, 2000
[10] Fri Jan 14, 2000 Tue Jan 18, 2000 Wed Jan 19, 2000
[13] Thu Jan 20, 2000 Fri Jan 21, 2000 Mon Jan 24, 2000
[16] Tue Jan 25, 2000 Wed Jan 26, 2000 Thu Jan 27, 2000
[19] Fri Jan 28, 2000 Mon Jan 31, 2000
The argument holidays=holiday.NYSE(2000) in conjunction with by ="bizdays" instructs timeSeq to exclude the weekday dates associated with
the NYSE holidays for 2000. Notice that the date Mon Jan 17, 2000 has
been omitted from the sequence.
Intra-day Irregularly Spaced Sequences
Sequences of irregularly spaced intra-day dates may be created using thefunction timeCalendar. For example, consider creating a sequence of hourly
observations only during the hypothetical trading hours from 9:00 AM to
3:00 PM from Monday January 3, 2000 through Tuesday January 4, 2000.
Such a sequence may be created using timeCalendar as follows> timeCalendar(h=rep(9:15,2),d=rep(3:4,each=7),+ y=2000,format="%a %b %d, %Y %02I:%02M %p")[1] Mon Jan 3, 2000 09:00 AM Mon Jan 3, 2000 10:00 AM[3] Mon Jan 3, 2000 11:00 AM Mon Jan 3, 2000 12:00 PM[5] Mon Jan 3, 2000 01:00 PM Mon Jan 3, 2000 02:00 PM[7] Mon Jan 3, 2000 03:00 PM Tue Jan 4, 2000 09:00 AM[9] Tue Jan 4, 2000 10:00 AM Tue Jan 4, 2000 11:00 AM

28

2. Time Series Specification, Manipulation, and Visualization in S-PLUS[11] Tue Jan 4, 2000 12:00 PM Tue Jan 4, 2000 01:00 PM[13] Tue Jan 4, 2000 02:00 PM Tue Jan 4, 2000 03:00 PM
In a similar fashion, a sequence of minute observations from 9:00 AM to
3:00 PM on Monday January 3, 2000 and Tuesday January 4, 2000 may becreated using> timeCalendar(min=rep(rep(0:59,6),2),+ h=rep(9:14,each=60,length=360*2),+ d=rep(3:4,each=360,length=360*2),+ y=2000,format="%a %b %d, %Y %02I:%02M %p")[1] Mon Jan 3, 2000 09:00 AM Mon Jan 3, 2000 09:01 AM[3] Mon Jan 3, 2000 09:02 AM Mon Jan 3, 2000 09:03 AM...
[359] Mon Jan 3, 2000 02:58 PM Mon Jan 3, 2000 02:59 PM[361] Tue Jan 4, 2000 09:00 AM Tue Jan 4, 2000 09:01 AM...
[719] Tue Jan 4, 2000 02:58 PM Tue Jan 4, 2000 02:59 PM

2.2.4 Miscellaneous Time and Date Functions
In addition to the time and date functions discussed so far, S-PLUS has anumber of miscellaneous time and date functions. In addition S+FinMetrics
provides a few time and date functions. These are summarized in Table 2.1.

2.2.5 Creating “timeSeries” Objects
S-PLUS “timeSeries” objects are created with the timeSeries function.
Typically a “timeSeries” is created from some existing data in a data
frame or matrix and a “timeDate” object. For example,> my.df = data.frame(x=abs(rnorm(10,mean=5)),+ y=abs(rnorm(10,mean=10)))> my.td = timeCalendar(y=1990:1999,format="%Y")> my.ts = timeSeries(data=my.df,pos=my.td)> my.ts
Positionsxy1990
4.250 11.087
1991
5.290 11.590
1992
5.594 11.848
1993
5.138 10.426
1994
5.205 9.678
1995
4.804 11.120
1996
5.726 11.616
1997
6.124 9.781
1998
3.981 10.725

2.2 The Specification of “timeSeries” Objects in S-PLUS

S-PLUS function
month.day.year
julianquarters
monthsdays
weekdays
years
yeardayshours
minutesseconds
hms
mdy
wdydy
leap.year
holidaysholiday.fixedholiday.weekday.number
S+FinMetrics function
days.countis.weekday
is.weekend
is.bizday
imm.dates

29

Description
Converts calendar dates to Julian dates
Converts Julian dates to calendar dates
Create an ordered factor corresponding toquarters
Create an ordered factor corresponding tomonthsCreate an ordered factor corresponding todays
Create an ordered factor corresponding to
weekdays
Create an ordered factor corresponding toyears
Extract year day from date
Extract hour from date
Extract minutes from date
Extract seconds from date
Create data frame containing hours,
minutes and seconds
Create data frame containing month, day
and year
Create data frame containing weekday,
year day and year
Determines if year number corresponds toa leap year
Generate a collection of holidays
Generate holidays that occur on fixed dates
Generate holidays that occur on weekdays
Description
Count number of days between two dates
Tests if date is a weekday
Tests if date is a weekend
Tests if date is a business day
Create International Monetary Market dates

TABLE 2.1. Miscellaneous time and date functions

30

1999

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

6.006 10.341

Information about the “timeSeries” object may be added to the title,
documentation and units slots:> my.ts@title = "My timeSeries"> my.ts@documentation = c("Simulated annual price data using ",+ "the S-PLUS function rnorm")> my.ts@units = c("US dollars","US dollars")
The title and units information is utilized in certain plot functions.
Creating “timeSeries” Objects from Time Series in Data Frames
Very often time series data that are in data frames have a date variablewith a formatted date string. The S-PLUS function timeDate has a variety of input formats that may be used to convert such date strings into
“timeDate” objects. For example, the S+FinMetrics data frame yhoo.df
contains daily high, low, open and close prices as well as volume information
for Yahoo stock for the month of February 2002
> yhoo.df[1:2,]
Date Open HighLow Close Volume
1 1-Feb-02 17.26 17.3 16.35 16.68 6930100
2 4-Feb-02 16.55 16.6 15.60 15.75 8913700
The variable Date is a character vector containing the date strings. A“timeDate” sequence created from the date strings in Date is> td = timeDate(yhoo.df[,1],in.format="%d-%m-%y",+ format="%a %b %d, %Y")
> td[1:2][1] Fri Feb 1, 2002 Mon Feb 4, 2002
A “timeSeries” object containing the data from yhoo.df is created using> yhoo.ts = timeSeries(pos=td,data=yhoo.df[,-1])> yhoo.ts[1:2,]
Positions Open HighLow Close Volume
Fri Feb 1, 2002 17.26 17.3 16.35 16.68 6930100
Mon Feb 4, 2002 16.55 16.6 15.60 15.75 8913700
High frequency data, however, is often recorded using nonstandard time
formats. For example, consider the transactions level data for the month ofDecember 1999 for 3M stock in the S+FinMetrics data frame highFreq3m.df> highFreq3M.df[1:2,]
trade.day trade.time trade.price
1
1
34412
94.6882
134414
94.688

2.2 The Specification of “timeSeries” Objects in S-PLUS

31

The variable trade.day contains the integer trading day of the month,
the variable trade.time contains the integer trade time recorded as thenumber of seconds from midnight and the variable trade.price contains
the transaction price in dollars. A “timeDate” sequence may be easily
created from the trade day and trade time information as follows> td = timeDate(julian=(highFreq3M.df$trade.day-1),+ ms=highFreq3M.df$trade.time*1000,+ in.origin=c(month=12,day=1,year=1999),zone="GMT")> td[1:2][1] 12/1/99 9:33:32 AM 12/1/99 9:33:34 AM
The function timeDate can create a “timeDate” sequence using Julian dateand millisecond information. The argument julian takes an integer vector
containing the number of days since the date specified in the argument
in.origin, and the argument ms takes an integer vector containing thenumber of milliseconds since midnight. In the above example, in.origin
is specified as December 1, 1999 and the optional argument zone is used
to set the time zone to GMT. A “timeSeries” object containing the highfrequency data in highFreq3M.df is created using> hf3M.ts = timeSeries(pos=td,data=highFreq3M.df)

2.2.6 Aggregating and Disaggregating Time Series
Often a regularly spaced financial time series of a given frequency may
need to be aggregated to a coarser frequency or disaggregated to a finer
frequency. In addition, aggregation and disaggregation may involve flow orstock variables. The S-PLUS functions aggregateSeries and align may be
used for such purposes. To enhance and extend the disaggregation functionality in S-PLUS the S+FinMetrics function disaggregate is introduced.
Aggregating Time Series
Given a monthly “timeSeries” of end of month prices over a number ofyears, suppose one would like to create an annual time series consisting of
the end of month December prices. Such a series may be easily constructedby subsetting using the S-PLUS function months:> dec.vals = "Dec"==months(positions(singleIndex.dat))> annual.p = singleIndex.dat[dec.vals,]> annual.p
Positions
MSFT SP500
Dec 1990
2.090 330.2
Dec 1991
4.635 417.1
Dec 1992
5.336 435.7
Dec 1993
5.039 466.4

32

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

Dec 1994
Dec 1995
Dec 1996
Dec 1997
Dec 1998
Dec 1999
Dec 2000

7.641 459.3
10.969 615.9
20.656 740.7
32.313 970.4
69.344 1229.2
116.750 1469.3
43.375 1320.3

Another way to create the above annual time series is to use the S-PLUS
aggregateSeries function with a user-written function to pick oﬀ December values. One such function, based on the S-PLUS function hloc used tocompute high, low, open and close values, ispickClose = function(x){# return closing values of a vector
if(length(dim(x))) x = as.vector(as.matrix(x))
len = length(x)if(!len)as(NA, class(x))
else x[len]}The annual data is then constructed using aggregateSeries with optional arguments FUN=pickClose and by="years"> annual.p = aggregateSeries(singleIndex.dat,+ FUN=pickClose,by="years")> positions(annual.p)@format = "%Y"> annual.p
Positions
MSFT SP500
1990
2.090 330.2
1991
4.635 417.1
1992
5.336 435.7
1993
5.039 466.4
1994
7.641 459.3
1995
10.969 615.9
1996
20.656 740.7
1997
32.313 970.4
1998
69.344 1229.2
1999
116.750 1469.3
2000
43.375 1320.3
2001
61.063 1366.0
The function aggregateSeries passes to the function pickClose datafrom singleIndex.dat in blocks of year’s length. The function pickClose

2.2 The Specification of “timeSeries” Objects in S-PLUS

33

simply picks oﬀ the last value for the year. Since singleIndex.dat only
has data for January 2, 2001, the 2001 value for annual.p is this value.
The method described above may also be used to construct end-of-monthclosing price data from a “timeSeries” of daily closing price data. For
example, the commands to create end of month closing prices from daily
closing prices for Microsoft, taken from the S+FinMetrics “timeSeries”
DowJones30, using aggregateSeries with FUN = pickClose and by ="months" are> msft.daily.p = DowJones30[,"MSFT"]> msft.daily.p@title = "Daily closing price on Microsoft"> msft.daily.p@units = "Dollar price"> msft.monthly.p = aggregateSeries(msft.daily.p,FUN=pickClose,+ by="months",adj=0.99)> msft.monthly.p[1:12]
Positions MSFT
1/31/1991 2.726
2/28/1991 2.882
3/31/1991 2.948
4/30/1991 2.750
5/31/1991 3.049
6/30/1991 2.838
7/31/1991 3.063
8/31/1991 3.552
9/30/1991 3.708
10/31/1991 3.912
11/30/1991 4.052
12/31/1991 4.635
The option adj=0.99 adjusts the positions of the monthly data to theend of the month. Notice that the end of month dates are not necessarily
the last trading days of the month.
The monthly closing price data may be extracted from the daily closing
price data by clever use of subscripting4 . One way to do this is> end.month.idx =+ which(diff(as.numeric(months(positions(msft.daily.p)))) != 0)> msft.monthly.p = msft.daily.p[end.month.idx]> msft.monthly.p[1:12]
Positions MSFT
1/31/1991 2.726
2/28/1991 2.882
3/28/1991 2.948
4/30/1991 2.750
4 This method was suggested by Steve McKinney.

34

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

5/31/1991 3.049
6/28/1991 2.838
7/31/1991 3.063
8/30/1991 3.552
9/30/1991 3.708
10/31/1991 3.912
11/29/1991 4.052
12/31/1991 4.635
A common aggregation operation with financial price data is to construct
a volume weighted average price (vwap). This may be easily accomplishedwith aggregateSeries and a user-specified function to compute the vwap.
For example, consider the daily open, high, low and close prices and volume
on Microsoft stock from October 2, 2000 through August 31, 2001 in the
S+FinMetrics “timeSeries” msft.dat.
> smpl = (positions(msft.dat) >= timeDate("10/1/2000") &+ positions(msft.dat) <= timeDate("8/31/2001"))> msft.dat[smpl,]
Positions Open HighLow Close
Volume10/2/2000 60.50 60.81 58.25 59.13 29281200...
8/31/2001 56.85 58.06 56.30 57.05 28950400
A function that can be used to aggregate open, high, low and close prices,volume and compute the open and close vwap is
vol.wtd.avg.price = function(x) {VolumeSum = as.double(sum(x[, "Volume"]))nrowx = numRows(x)return(data.frame(Open = x[1, "Open"],High = max(x[, "High"]),Low = min(x[, "Low"]),
Close = x[nrowx, "Close"],vwap.Open = sum(x[, "Open"] * x[, "Volume"])/VolumeSum,
wap.Close = sum(x[, "Close"] * x[, "Volume"])/VolumeSum,
Volume = VolumeSum))}
Using aggregateSeries and the function vol.wtd.avg.price one cancompute the monthly open, high, low, close prices, volume, and open andclose vwap
> msft.vwap.dat = aggregateSeries(x = msft.dat[smpl,],+ by = "months",FUN = vol.wtd.avg.price,+ together = T)
> positions(msft.vwap.dat)@format="%b %Y"> msft.vwap.dat[,-7]2.2 The Specification of “timeSeries” Objects in S-PLUS

35

Positions Open HighLow Close vwap.Open vwap.CloseOct 2000 60.50 70.13 48.44 68.88 59.10
59.48
Nov 2000 68.50 72.38 57.00 57.38 68.35
67.59
...
Aug 2001 66.80 67.54 56.30 57.05 62.99
62.59
Disaggregating Time Series
Consider the problem of creating a daily “timeSeries” of inflation adjusted(real) prices on Microsoft stock over the period January 2, 1991 throughJanuary 2, 2001. To do this the daily nominal prices must be divided bya measure of the overall price level; e.g. the consumer price level (CPI).
The daily nominal stock price data is in the “timeSeries” msft.daily.p
created earlier and the CPI data is in the S+FinMetrics “timeSeries”
CPI.dat. The CPI data, however, is only available monthly.
> start(CPI.dat)[1] Jan 1913
> end(CPI.dat)[1] Nov 2001
and represents the average overall price level during the month but isrecorded at the end of the month. The CPI data from December 1990
through January 2001 is extracted using> smpl = (positions(CPI.dat) >= timeDate("12/1/1990")+ & positions(CPI.dat) <= timeDate("2/1/2001"))> cpi = CPI.dat[smpl,]> cpi[1:3]
Positions
CPI
Dec 1990 134.3
Jan 1991 134.8
Feb 1991 134.9
To compute real daily prices on Microsoft stock, the monthly CPI datain the “timeSeries” object cpi must be disaggregated to daily data. Thisdisaggregation may be done in a number of ways. For example, the CPI for
every day during the month of January, 1991 may be defined as the monthly
CPI value for December, 1990 or the monthly CPI value for January, 1991.
Alternatively, the daily values for January 1991 may be computed by linearly interpolating between the December, 1990 and January, 1991 values.
The S-PLUS function align may be used to do each of these disaggregations.
The align function aligns a “timeSeries” object to a given set of positions and has options for the creation of values for positions in which
the “timeSeries” does not have values. For example, the disaggregated36

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

CPI using the previous month’s value for the current month’s daily data isconstructed using> cpi.daily.before =+ align(cpi,positions(msft.daily.p),how="before")> cpi.daily.before[c(1:3,21:23)]
PositionsCPI
1/2/1991 134.3
1/3/1991 134.3
1/4/1991 134.3
1/30/1991 134.3
1/31/1991 134.8
2/1/1991 134.8
The new positions to align the CPI values are the daily positions of the“timeSeries” msft.daily.p, and the argument how="before" specifies
that the previous month’s CPI data is to be used for the current month’s
daily CPI values. Similarly, the disaggregated CPI using the next month’svalue for the current month’s daily data is constructed using> cpi.daily.after =+ align(cpi,positions(msft.daily.p),how="after")> cpi.daily.after[c(1:3,21:23)]
PositionsCPI
1/2/1991 134.8
1/3/1991 134.8
1/4/1991 134.8
1/30/1991 134.8
1/31/1991 134.8
2/1/1991 134.9
Finally, the disaggregated daily CPI using linear interpolation between themonthly values is constructed using> cpi.daily.interp = align(cpi,positions(msft.daily.p),+ how="interp")> cpi.daily.interp[c(1:3,21:23)]
PositionsCPI
1/2/1991 134.3
1/3/1991 134.3
1/4/1991 134.4
1/30/1991 134.8
1/31/1991 134.8
2/1/1991 134.8
The daily real prices on Microsoft stock using the interpolated daily CPI
values are then> msft.daily.rp = (msft.daily.p/cpi.daily.interp)*100

2.2 The Specification of “timeSeries” Objects in S-PLUS

37

Disaggregating Time Series using the S+FinMetrics disaggregate
FunctionWith economic and financial time series, it is sometimes necessary to distribute a flow variable or time average a stock variable that is observed ata low frequency to a higher frequency. For example, a variable of interest
may only be observed on an annual basis and quarterly or monthly values are desired such that their sum is equal to the annual observation or
their average is equal to the annual observation. The S+FinMetrics function disaggregate performs such disaggregations using two methods. The
first method is based on cubic spline interpolation and is appropriate if the
only information is on the series being disaggregated. The second method
utilizes a generalized least squares (gls) fitting method due to Chow and
Lin (1971) and is appropriate if information is available on one or more related series that are observed at the desired disaggregated frequency. The
arguments expected by disaggregate are> args(disaggregate)function(data, k, method = "spline", how = "sum", x = NULL,+ out.positions = NULL, ...)
where data is a vector, matrix or “timeSeries” of low frequency data, k is
the number of disaggregtion periods, method determines the disaggregation
method (spline or gls), how specifies if the disaggregated values sum to theaggregated values or are equal on average to the disaggregated values, x
respresents any related observed data at the disaggregated frequency andout.positions represents a “timeDate” sequence for the resulting output.
To illustrate the use of disaggregate, consider the problem of disaggregating the annual dividend on the S&P 500 index to a monthly dividend. Since the annual dividend is a flow variable, the sum of the monthly
dividends should equal the annual dividend. The annual S&P 500 dividend information over the period 1871 - 2000 is in the S+FinMetrics“timeSeries” shiller.annual. The disaggregated monthly dividend values such that their sum is equal to the annual values is created using> monthly.dates = timeSeq(from="1/1/1871",to="12/31/2000",+ by="months",format="%b %Y")
> div.monthly =+ disaggregate(shiller.annual[,"dividend"],12,+ out.positions=monthly.dates)> div.monthly[1:12]
Positions dividend
Jan 1871 0.02999
Feb 1871 0.01867
Mar 1871 0.01916
Apr 1871 0.01963
May 1871 0.02009

38

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

Jun 1871 0.02054
Jul 1871 0.02097
Aug 1871 0.02140
Sep 1871 0.02181
Oct 1871 0.02220
Nov 1871 0.02259
Dec 1871 0.02296> sum(div.monthly[1:12])[1] 0.26> shiller.annual[1,"dividend"]
Positions dividend
1871
0.26
For the S&P 500 index, the index price is available in the S+FinMetrics
monthly “timeSeries” shiller.dat. This information may be utilized in
the disaggregation of the annual dividend using the gls method as follows> smpl = positions(shiller.dat) <= timeDate("12/31/2000")> price.monthly = as.matrix(seriesData(shiller.dat[smpl,"price"]))> div2.monthly =+ disaggregate(shiller.annual[,"dividend"], 12,+ method="gls", x=price.monthly, out.positions=monthly.dates)> div2.monthly[1:12]
Positions dividend
Jan 1871 0.006177
Feb 1871 0.010632
Mar 1871 0.014610
Apr 1871 0.018104
May 1871 0.021104
Jun 1871 0.023569
Jul 1871 0.025530
Aug 1871 0.027043
Sep 1871 0.028063
Oct 1871 0.028508
Nov 1871 0.028548
Dec 1871 0.028111> sum(div2.monthly[1:12])[1] 0.26> shiller.annual[1,"dividend"]
Positions dividend
1871
0.26

2.2.7 Merging Time Series
Often one would like to combine several “timeSeries” objects into asingle “timeSeries” object. The S-PLUS functions c, concat and cbind

2.2 The Specification of “timeSeries” Objects in S-PLUS39

do not operate on “timeSeries” objects. Instead, the S-PLUS function
seriesMerge is used to combine or merge a collection of “timeSeries”.
To illustrate, consider creating a new “timeSeries” object consisting of the
S+FinMetrics “timeSeries” CPI.dat and IP.dat containing monthly observations on the U.S. consumer price index and U.S. industrial production
index, respectively:> CPI.dat
Positions
CPI
Jan 1913
9.80
Feb 1913
9.80...
Nov 2001 177.60> IP.dat
PositionsIP
Jan 1919
7.628
Feb 1919
7.291...
Nov 2001 137.139
Notice that the start date for CPI.dat is earlier than the start date forIP.dat,but the end dates are the same. A new “timeSeries” containing
both CPI.dat and IP.dat with positions aligned to those for IP.dat using
seriesMerge is> IP.CPI.dat = seriesMerge(IP.dat,CPI.dat,+ pos=positions(IP.dat))> IP.CPI.dat[1:2,]
Positions
IP CPIJan 1919 7.628 16.5
Feb 1919 7.291 16.2
To create a “timeSeries” with positions given by the union of the positions for CPI.dat and IP.dat set pos="union" in the call to seriesMerge.
Since IP.dat does not have observations for the dates January 1913 throughDecember 1918, NA values for IP for these dates will be inserted in the new“timeSeries”.

2.2.8 Dealing with Missing Values Using the S+FinMetrics
Function interpNA
Occasionally, time series data contain missing or incorrect data values. One
approach often used to fill-in missing values is interpolation5 . The S-PLUS
5 More sophisticated imputation methods for dealing with missing values are available
in the library S+MISSINGDATA which is included with S-PLUS.

40

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

align function may be used for this purpose. The S+FinMetrics function interpNA performs similar missing value interpolation as align but
is easier to use and is more flexible. The arguments expected by interpNAare> args(interpNA)
function(x, method = "spline")
where x is a rectangular object and method sets the interpolation method.
Valid interpolation methods are “before”, “after”, “nearest”, “linear”
and (cubic) “spline”. To illustrate the use of interpNA, note that the closing price for the Dow Jones Industrial Average in the S-PLUS “timeSeries”
djia has a missing value on January 18, 1990:> djia.close = djia[positions(djia) >= timeDate("1/1/1990"),+ "close"]> djia.close[10:12,]
Positions close
01/17/1990 2659.1
01/18/1990
NA
01/19/1990 2677.9
To replace the missing value with an interpolated value based on a cubic
spline use> djia.close = interpNA(djia.close)> djia.close[10:12,]
Positions1
01/17/1990 2659.1
01/18/1990 2678.7
01/19/1990 2677.9

2.3 Time Series Manipulation in S-PLUS
There are several types of common manipulations and transformations that
often need to be performed before a financial time series is to be analyzed.
The most important transformations are the creation of lagged and diﬀerenced variables and the creation of returns from asset prices. The following
sections describe how these operations may be performed in S-PLUS.

2.3.1 Creating Lags and Diﬀerences
Three common operations on time series data are the creation of lags, leads,
and diﬀerences. The S-PLUS function shift may be used to create leads
and lags, and the generic function diff may be used to create diﬀerences.
However, these functions do not operate on “timeSeries” objects in the2.3 Time Series Manipulation in S-PLUS

41

most convenient way. Consequently, the S+FinMetrics module contains
the functions tslag and diff.timeSeries for creating lags/leads and differences.
Creating Lags and Leads Using the S+FinMetrics Function tslag
The S+FinMetrics function tslag creates a specified number of lag/leadsof a rectangular data object. The arguments expected by tslag are> args(tslag)
function(x, k = 1, trim = F)
where x is any rectangular object, k specifies the number of lags to becreated (negative values create leads) and trim determines if NA values areto be trimmed from the result. For example, consider the “timeSeries”
singleIndex.dat containing monthly prices on Microsoft and the S&P
500 index. The first five values are> singleIndex.dat[1:5,]
Positions MSFT SP500
Jan 1990 1.285 329.1
Feb 1990 1.371 331.9
Mar 1990 1.538 339.9
Apr 1990 1.611 330.8
May 1990 2.028 361.2
The “timeSeries” of lagged values using tslag are> tslag(singleIndex.dat[1:5,])
Positions MSFT.lag1 SP500.lag1
Jan 1990
NANA
Feb 1990 1.285
329.1
Mar 1990 1.371
331.9
Apr 1990 1.538
339.9
May 1990 1.611
330.8
Notice that tslag creates a “timeSeries” containing the lagged prices
on Microsoft and the S&P 500 index. The variable names are adjusted toindicate the type of lag created and since trim=F, NA values are inserted
for the first observations. To create a “timeSeries” without NA values in
the first position, use tslag with trim=T:> tslag(singleIndex.dat[1:5,],trim=T)
Positions MSFT.lag1 SP500.lag1
Feb 1990 1.285
329.1
Mar 1990 1.371
331.9
Apr 1990 1.538
339.9
May 1990 1.611
330.8
Leads are created by setting k equal to a negative number:42

2. Time Series Specification, Manipulation, and Visualization in S-PLUS> tslag(singleIndex.dat[1:5,],k=-1)
Positions MSFT.lead1 SP500.lead1
Jan 1990 1.371
331.9
Feb 1990 1.538
339.9
Mar 1990 1.611
330.8
Apr 1990 2.028
361.2
May 1990
NANA
To create a “timeSeries” with multiple lagged values, simply specify
the lags to create in the call to tslag. For example, specifying k=c(1,3)
creates the first and third lag> tslag(singleIndex.dat[1:5,],k=c(1,3))
Positions MSFT.lag1 SP500.lag1 MSFT.lag3 SP500.lag3
Jan 1990
NA
NA
NA
NA
Feb 1990 1.285
329.1
NA
NA
Mar 1990 1.371
331.9
NA
NA
Apr 1990 1.538
339.9
1.285
329.1
May 1990 1.611
330.8
1.371
331.9
Similarly, specifying k=-1:1 creates> tslag(singleIndex.dat[1:5,],k=-1:1)
Positions MSFT.lead1 SP500.lead1 MSFT.lag0 SP500.lag0
Jan 1990 1.371
331.9
1.285
329.1
Feb 1990 1.538
339.9
1.371
331.9
Mar 1990 1.611
330.8
1.538
339.9
Apr 1990 2.028
361.2
1.611
330.8
May 1990
NA
NA
2.028
361.2
MSFT.lag1 SP500.lag1
NA
NA
1.285
329.1
1.371
331.9
1.538
339.9
1.611
330.8
Creating Diﬀerences Using the S+FinMetrics Function diff.timeSeries
The S+FinMetrics function diff.timeSeries is a method function for
the generic S-PLUS function diff for objects of class “timeSeries” andcreates a specified number of diﬀerences of a “timeSeries” object. The
arguments expected by diff.timeSeries are> args(diff.timeSeries)function(x, lag = 1, differences = 1, trim = T, pad = NA)
where x represents a “timeSeries” object, lag specifies the number oflagged periods used in the diﬀerence, differences specifies the number

2.3 Time Series Manipulation in S-PLUS

43

of times to diﬀerence the series, trim determines if the resulting series is
to have NA values removed and trimmed and pad specifies the value tobe padded to the series in the positions where the diﬀerencing operation
exceeds the start or the end positions. For example, consider again the“timeSeries” singleIndex.dat containing monthly prices on Microsoft
and the S&P 500 index. Let Pt denote the price at time t. To create thefirst diﬀerence ∆Pt = Pt − Pt−1 use diff with lag=1:> diff(singleIndex.dat[1:5,],lag=1,trim=F)
Positions
MSFT SP500
Jan 1990
NANA
Feb 1990 0.0868
2.81
Mar 1990 0.1667
8.05
Apr 1990 0.0729 -9.14
May 1990 0.4167 30.43
To create the diﬀerence Pt − Pt−2 and pad the result with zeros instead of
NAs use diff with lag=2 and pad=0:> diff(singleIndex.dat[1:5,],lag=2,trim=F,pad=0)
Positions
MSFT SP500
Jan 1990 0.0000
0.00
Feb 1990 0.0000
0.00
Mar 1990 0.2535 10.86
Apr 1990 0.2396 -1.09
May 1990 0.4896 21.29
To create the 2nd diﬀerence ∆2 Pt = ∆(Pt − Pt−1 ) = Pt − 2Pt−1 + Pt−2 use
diff with lag=1 and diff=2:> diff(singleIndex.dat[1:5,],lag=1,diff=2,trim=F)
Positions
MSFT SP500
Jan 1990
NANA
Feb 1990
NANA
Mar 1990
0.07995.24
Apr 1990 -0.0938 -17.19
May 1990
0.3438 39.57
Unlike tslag, diff.timeSeries does not rename the variables to indicate the diﬀerencing operation performed. Additionally, diff.timeSeries
will not accept a vector of values for the arguments lag and differences.

2.3.2 Return Definitions
Simple Returns
Let Pt denote the price at time t of an asset that pays no dividends and
let Pt−1 denote the price at time t − 1. Then the simple net return on an44

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

investment in the asset between times t − 1 and t is defined as
Pt − Pt−1= %∆Pt .
Pt−1

Rt =

(2.1)

t−1
t
= PPt−1− 1, we can define the simple gross return as
Writing PtP−P
t−1

1 + Rt =

Pt
.
Pt−1

(2.2)

Unless otherwise stated, references to returns mean net returns.
The simple two-period return on an investment in an asset between times
t − 2 and t is defined as
Pt − Pt−2
Pt
=
−1
Pt−2
Pt−2
Pt−1
Pt
·
−1
=
Pt−1 Pt−2= (1 + Rt )(1 + Rt−1 ) − 1.

Rt (2) =Then the simple two-period gross return becomes
1 + Rt (2) = (1 + Rt )(1 + Rt−1 ) = 1 + Rt−1 + Rt + Rt−1 Rt ,which is a geometric (multiplicative) sum of the two simple one-period gross
returns and not the simple sum of the one period returns. If, however, Rt−1
and Rt are small then Rt−1 Rt ≈ 0 and 1 + Rt (2) ≈ 1 + Rt−1 + Rt so thatRt (2) ≈ Rt−1 + Rt .
In general, the k-period gross return is defined as the geometric averageof k one period gross returns
1 + Rt (k) = (1 + Rt )(1 + Rt−1 ) · · · (1 + Rt−k+1 )
=

(2.3)

k−1
Y

(1 + Rt−j )j=0

and the k-period net return is
Rt (k) =

k−1
Y
j=0

(1 + Rt−j ) − 1.

(2.4)

Continuously Compounded Returns
Let Rt denote the simple one period return on an investment. The continuously compounded one period return, rt , is defined as
µ
¶
Pt
(2.5)
rt = ln(1 + Rt ) = ln
Pt−1

2.3 Time Series Manipulation in S-PLUS

45

where ln(·) is the natural log function. To see why rt is called the continuously compounded return, take exponentials of both sides of (2.5) togivePt
ert = 1 + Rt =.
Pt−1
Rearranging gives
Pt = Pt−1 ert ,so that rt is the continuously compounded growth rate in prices between
periods t − 1 and t. This is to be contrasted with Rt which is the simple
growth rate
³ ´in prices between periods t−1 and t without any compounding.
Since ln xy = ln(x) − ln(y) it follows thatµ¶Pt
rt = ln
Pt−1= ln(Pt ) − ln(Pt−1 )= pt − pt−1
where pt = ln(Pt ). Hence, the continuously compounded one period return,rt , can be computed simply by taking the first diﬀerence of the natural
logarithms of prices between periods t − 1 and t.
Given a one period continuously compounded return rt , it is straightforward to solve back for the corresponding simple net return Rt :Rt = ert − 1
Hence, nothing is lost by considering continuously compounded returns
instead of simple returns.
The computation of multi-period continuously compounded returns is
considerably easier than the computation of multi-period simple returns. To
illustrate, consider the two period continuously compounded return defined
as
µ
¶
Pt
rt (2) = ln(1 + Rt (2)) = ln
= pt − pt−2 .
Pt−2Taking exponentials of both sides shows that
Pt = Pt−2 ert (2)so that rt (2) is the continuously compounded growth rate of prices betweent−1tt= PPt−1·P
periods t − 2 and t. Using PPt−2
Pt−2 and the fact that ln(x · y) =ln(x) + ln(y) it follows that
µ
¶
Pt
Pt−1
rt (2) = ln·
Pt−1 Pt−2
µ
¶
µ
¶
Pt
Pt−1
= ln
+ ln
Pt−1
Pt−2
= rt + rt−1 .

46

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

Hence the continuously compounded two period return is just the sum of
the two continuously compounded one period returns.
The continuously compounded k-period return is defined as
µ
¶
Pt
(2.6)
= pt − pt−k .
rt (k) = ln(1 + Rt (k)) = ln
Pt−kUsing similar manipulations to the ones used for the continuously compounded two period return the continuously compounded k-period return
may be expressed as the sum of k continuously compounded one period
returns:k−1
X
rt (k) =rt−j .
(2.7)
j=0

The additivitity of continuously compounded returns to form multiperiod
returns is an important property for statistical modeling purposes.

2.3.3 Computing Asset Returns Using the S+FinMetrics
Function getReturns
Given a data set with asset prices the S+FinMetrics function getReturns
may be used to compute discrete and continuously compounded returns.
The arguments to getReturns are> args(getReturns)
function(x, type = "continuous", percentage = F, trim = T)
where x is any rectangular data object and type specifies the type of returns to compute (discrete or continuously compounded). To illustrate, the
S+FinMetrics “timeSeries” singleIndex.dat contains monthly closing
prices on Microsoft stock and the S&P 500 index, adjusted for stock splits
and dividends, over the period January 1990 through January 2001.
> colIds(singleIndex.dat)[1] "MSFT" "SP500"> singleIndex.dat[1:3,]
Positions
MSFT SP500
Jan 1990 1.2847 329.08
Feb 1990 1.3715 331.89
Mar 1990 1.5382 339.94
A “timeSeries” of simple one-period discrete returns expressed as percentages is computed as> ret.d = getReturns(singleIndex.dat,type="discrete",+ percentage=T)> ret.d[1:3,]
Positions
MSFT
SP500

2.3 Time Series Manipulation in S-PLUS

Feb 1990
Mar 1990
Apr 1990

47

6.756 0.8539
12.155 2.4255
4.739 -2.6887

By default the first observation in the “timeSeries” is trimmed. To retain
the first (NA) observation use the optional argument trim=F> ret.d = getReturns(singleIndex.dat,type="discrete",trim=F)> ret.d[1:3,]
Positions
MSFT
SP500
Jan 1990
NANA
Feb 1990 0.067564 0.008539
Mar 1990 0.121546 0.024255
Continuously compounded returns are created by specifying the optional
argument type="continuous"> ret.cc = getReturns(singleIndex.dat,type="continuous")> ret.cc[1:3,]
Positions
MSFT
SP500
Feb 1990 0.065380 0.0085027
Mar 1990 0.114708 0.0239655
Apr 1990 0.046304 -0.0272552
Multiperiod returns may be computed from a “timeSeries” of one period returns using the S-PLUS function aggregateSeries. Multiperiod returns may be either overlapping or non-overlapping. For example, consider
computing a monthly “timeSeries” of overlapping annual continuously
compounded returns from the monthly continuously compounded returns
in the “timeSeries” ret.cc using aggregateSeries:> ret12.cc = aggregateSeries(ret.cc,moving=12,FUN=sum)> ret12.cc[1:3,]
Positions
MSFT
SP500
Feb 1990 0.75220 0.044137
Mar 1990 0.74254 0.100749
Apr 1990 0.65048 0.098743> colSums(seriesData(ret.cc[1:12,]))
MSFT
SP500
0.7522 0.044137
The argument moving=12 and FUN=sum tells aggregateSeries to computea moving sum of twelve returns. Hence, the annual return reported for
Feb 1990 is the sum of the twelve monthly returns from February 1990
through January 1991. Non-overlapping annual returns are computed fromthe monthly returns using aggregateSeries with the option by="years"> ret12.cc = aggregateSeries(ret.cc,by="years",FUN=sum)> ret12.cc[1:3,]

48

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

Positions
MSFT
SP500
Jan 1990 0.48678 0.0034582
Jan 1991 0.79641 0.2335429
Jan 1992 0.14074 0.0436749> colSums(seriesData(ret.cc[1:11,]))
MSFT
SP500
0.48678 0.0034582
The “timeSeries” ret12.cc is now an annual series of non-overlapping
annual returns. Notice that the annual return for January 1990 is computed
using only the eleven returns from February 1990 through December 1990.
Multiperiod discrete returns (2.4) may be computed using the function
aggregateSeries with FUN=prod. For example, a monthly “timeSeries”
of overlapping annual discrete returns is computed as> ret12.d = aggregateSeries((1+ret.d),moving=12,FUN=prod)-1> ret12.d[1:3,]
Positions
MSFT
SP500
Feb 1990 1.12166 0.045126
Mar 1990 1.10128 0.105999
Apr 1990 0.91646 0.103783> prod(seriesData(1+ret.d[1:12,1]))-1[1] 1.1217
Notice that 1 is added to the return data and 1 is subtracted from the resultin order to compute (2.4) properly. Non-overlapping multiperiod discrete
returns may be computed using> ret12.d = aggregateSeries((1+ret.d),by="years",FUN=prod)-1> ret12.d[1:3,]
Positions
MSFT
SP500
Jan 1990
NANA
Jan 1991 1.2176 0.26307
Jan 1992 0.1511 0.04464

2.4 Visualizing Time Series in S-PLUS
Time series data in “timeSeries” objects may be visualized by using the
S-PLUS generic plot function, the S-PLUS trellisPlot function, or byusing the S+FinMetrics plotting functions based on Trellis graphics.

2.4.1 Plotting “timeSeries” Using the S-PLUS Genericplot Function
The S-PLUS generic plot function has a method function, plot.timeSeries,
for plotting “timeSeries” objects. To illustrate, consider the monthly clos-2.4 Visualizing Time Series in S-PLUS

49

60
20

40

US dollar price

80

100

Monthly closing price on Microsoft

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 2.1. Monthly closing prices on Microsoft stock created usingplot.timeSeries.

ing prices of Microsoft stock over the period January 1990 to January 2001
in the “timeSeries” object msft.p created earlier:> msft.p@title[1] "Monthly closing price on Microsoft"> msft.p@units[1] "US dollar price"
Figure 2.1 shows the output produced by the generic plot function> plot(msft.p)
Notice how the information in the title and units slots is utilized
in the plot. To eliminate the horizontal and vertical grid lines specify
reference.grid=F in the call to plot. To show the price data on a logarithmic scale specify log.axes="y" in the call to plot.
Multiple series (on the same scale) may also be plotted together on thesame plot using plot6 . For example, the prices for Microsoft and the S&P
500 index in the “timeSeries” singleIndex.dat may be plotted togetherusing
6 To create a scatterplot of two “timeSeries” use the extractor function seriesData
possibly in conjunction with the coersion function as.matrix on the “timeSeries” objects in the call to plot. Alternatively, the S+FinMetrics function rvfPlot may be used.

50

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

1400

Monthly prices on Microsoft and S&P 500 Index

1000
800
600
200

400

Price per share

1200

MSFT
SP500

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 2.2. Monthly closing prices on Microsoft and the S&P 500 index createdusing plot.timeSeries.

> plot(singleIndex.dat,plot.args=list(lty=c(1,3)))> legend(0.1,1400,legend=colIds(singleIndex.dat),lty=c(1,3))
The plot is illustrated in Figure 2.2. Notice how the line types are specified
as a list argument to the optional argument plot.args. In the placementof the legend, the x-axis units are treated as values in the unit interval.
Multipanel plots may be created by specifying the plot layout using the
S-PLUS function par. Figure 2.3 shows a two panel plot of the price datain singleIndex.dat produced using> par(mfrow=c(2,1))> plot(singleIndex.dat[,"MSFT"],+ main="Monthly price on Microsoft")> plot(singleIndex.dat[,"SP500"],+ main="Monthly price on S&P 500 index")
Two specialized plot types for financial data can be made with the function plot.timeSeries. The first is a high/low/open/close (hloc) plot andthe second is a stackbar plot. These plots are made by setting plot.type ="hloc" or plot.type = "stackbar" in the call to plot.timeSeries. Fora hloc plot, the “timeSeries” to be plotted must have hloc information orsuch information must be created using aggregateSeries with the S-PLUS
function hloc. Stackbar plots are generally used for plotting asset volume

2.4 Visualizing Time Series in S-PLUS

51

80 10060
40
20

US dollar price

Monthly price on Microsoft

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

1998

1999

2000

2001

1200
800
400

US dollar price

Monthly price on S&P 500 index

1990

1991

1992

1993

1994

1995

1996

1997

FIGURE 2.3. Two panel plot created using par(mfrow=c(2,1)) in conjunction
with plot.timeSeries.

information. To illustrate these plot types, consider the monthly data fromthe Dow Jones Industrial Averages in the S-PLUS “timeSeries” djia:> colIds(djia)[1] "open""high""low""close""volume"Figure 2.4 gives a multipanel plot showing high, low, open, close andvolume information created by> smpl = (positions(djia) >= timeDate("9/1/1987") &+ positions(djia) <= timeDate("11/30/1987"))> par(mfrow=c(2,1))> plot(djia[smpl,1:4],plot.type="hloc")> plot(djia[smpl,5],plot.type="stackbar")Lines may be added to an existing time series plot using the S-PLUS
function lines.render and stackbar information may be added using the
S-PLUS function stackbar.render. See chapter 26 in the S-PLUS Guide
to Statistics Vol. II for details on using these functions.

52

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

1800 2000 2200 2400 2600

Dow Jones Industrial Average

Sep 7

Sep 14

Sep 21

Sep 28

Oct 5

Oct 12

Oct 19Oct 26

Nov 2

Nov 9

Nov 16

Nov 23

Nov 30

Nov 2

Nov 9

Nov 16

Nov 23

Nov 301987

100000300000500000Dow Jones Industrial Average

Sep 7

Sep 14

Sep 21

Sep 28

Oct 5

Oct 12

Oct 19Oct 26

1987

FIGURE 2.4. Monthly high, low, open, close and volume information for the
Dow Jones Industrial Average using plot.timeSeries with type="hloc" andtype="stackbar".

FunctionseriesPlot
histPlot
qqPlotDescription
Trellis time series plot
Trellis histogram plot
Trellis qq-plot for various distributions

TABLE 2.2. S+FinMetrics Trellis plotting functions

2.4.2 Plotting “timeSeries” Using the S+FinMetrics
Trellis Plotting Functions
S+FinMetrics provides several specialized Trellis-based plotting functions
for “timeSeries” objects. These functions extend the S-PLUS function
TrellisPlot.timeSeries and are summarized in Table 2.2.
All of the functions in the table can create multi-panel plots with text
labels in the panel strips. For the following examples, monthly return dataon six stocks from the S+FinMetrics “timeSeries” DowJones30 will be
used. This data is created using> DJ.ret = getReturns(DowJones30[,1:6], percentage=T)> colIds(DJ.ret)[1] "AA" "AXP" "T""BA" "CAT" "C"

2.4 Visualizing Time Series in S-PLUS

53

Monthly returns on six Dow Jones 30 stocks
1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

C

-10

0

0

10

CAT

BA

-20

0

0

T

AXP

-10

-10

0

0

10

10

AA

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 2.5. Multi-panel time plot created using the S+FinMetrics function
seriesPlot.

The function seriesPlot may be used to create single panel or multipanel time plots. To create the multi-panel time plot of the six Dow Jones
30 assets shown in Figure 2.5 use> seriesPlot(DJ.ret,one.plot=F,strip.text=colIds(DJ.ret),+ main="Monthly returns on six Dow Jones 30 stocks")
Notice that each time plot has a diﬀerent scale.
The function histPlot may be used to create either a single panel histogram of one data series or a multi-panel plot of histograms for multiple
series. The multi-panel plot in Figure 2.6 is created using> histPlot(DJ.ret,strip.text=colIds(DJ.ret),+ main="Histograms of returns on six Dow Jones 30 stocks")
Notice that each histogram uses the same bins.
Single panel or multi-panel Trellis-based qq-plots using Gaussian, Studentt, and double exponential distributions may be created using the function
qqPlot. To illustrate, consider computing qq-plots for the six Dow Jones
30 assets using six Student-t reference distributions with degrees of freedom equal to 5, 6, 7, 8, 9 and 10. These qq-plots, shown in Figure 2.7, arecreated using> s.text = paste(colIds(DJ.ret),5:10,sep=" ","df")54

2. Time Series Specification, Manipulation, and Visualization in S-PLUS

Histograms of returns on six Dow Jones 30 stocks
-30

-20

C

-10

0

10

CAT

T

60

40

Percent of Total

20

0
AA

AXP

BA

60

40

20

0
-30

-20

-10

0

10

-30

-20

-10

0

10

x

FIGURE 2.6. Multi-panel histogram plot created using the S+FinMetrics function histPlot.
Student-t QQ-plots for returns on six Dow Jones 30 stocks-5
C 10 df

0

5

CAT 9 df

T 7 df

10

0

-10

-20

-30
AA 5 df

AXP 6 df

BA 8 df

10

0

-10

-20

-30
-5

0

5

-5

0

5

FIGURE 2.7. Multi-panel qq-plots created using the S+FinMetrics function
qqPlot.

2.5 References

55> qqPlot(DJ.ret,strip.text=s.text,+ distribution="t",dof=c(5,6,7,8,9,10), id.n=FALSE,+ main="Student-t QQ-plots for returns on six Dow Jones 30 stocks")
Notice how the degress of freedom for each Student-t distribution alongwith the asset name is indicated in the strip text. The optional argument
id.n=FALSE suppresses the identification of outliers on the qq-plots.

2.5 References
Chow, G., and Lin, A. (1971). “Best Linear Unbiased Interpolation,
Distribution, and Extrapolation of Time Series by Related Series,” Review
of Economics & Statistics, 53, 372-375.

This is page 56
Printer: Opaque thisThis is page 57
Printer: Opaque this3
Time Series Concepts

3.1 Introduction
This chapter provides background material on time series concepts thatare used throughout the book. These concepts are presented in an informal
way, and extensive examples using S-PLUS are used to build intuition. Section 3.2 discusses time series concepts for stationary and ergodic univariate
time series. Topics include testing for white noise, linear and autoregressive
moving average (ARMA) process, estimation and forecasting from ARMA
models, and long-run variance estimation. Section 3.3 introduces univariate
nonstationary time series and defines the important concepts of I(0) andI(1) time series. Section 3.4 explains univariate long memory time series.
Section 3.5 covers concepts for stationary and ergodic multivariate timeseries, introduces the class of vector autoregression models, and discusses
long-run variance estimation.
Rigorous treatments of the time series concepts presented in this chapter can be found in Fuller (1996) and Hamilton (1994). Applications ofthese concepts to financial time series are provided by Campbell, Lo, andMacKinlay (1997), Mills (1999), Gourieroux and Jasiak (2001), Tsay (2001),
Alexander (2001), and Chan (2002).

58

3. Time Series Concepts

3.2 Univariate Time Series
3.2.1 Stationary and Ergodic Time Series
Let {yt } = {. . . yt−1 , yt , yt+1 , . . .} denote a sequence of random variables
indexed by some time subscript t. Call such a sequence of random variables
a time series.
The time series {yt } is covariance stationary if
E[yt ] = μ for all tcov(yt , yt−j ) = E[(yt − μ)(yt−j − μ)] = γ j for all t and any j
For brevity, call a covariance stationary time series simply a stationary
time series. Stationary time series have time invariant first and second
moments. The parameter γ j is called the j th order or lag j autocovarianceof {yt } and a plot of γ j against j is called the autocovariance function. The
autocorrelations of {yt } are defined byγjcov(yt , yt−j )ρj = p=γ0var(yt )var(yt−j )and a plot of ρj against j is called the autocorrelation function (ACF).
Intuitively, a stationary time series is defined by its mean, variance andACF. A useful result is that any function of a stationary time series is alsoa stationary time series. So if {yt } is stationary then {zt } = {g(yt )} is
stationary for any function g(·).
The lag j sample autocovariance and lag j sample autocorrelation are
defined as
γ̂ j

=

ρ̂j

=

T
1 X
(yt − ȳ)(yt−j − ȳ)
T t=j+1

γ̂ j
γ̂ 0

(3.1)
(3.2)

PT
where ȳ = T1 t=1 yt is the sample mean. The sample ACF (SACF) is aplot of ρ̂j against j.
A stationary time series {yt } is ergodic if sample moments converge inppp
probability to population moments; i.e. if ȳ → μ, γ̂ j → γ j and ρ̂j → ρj .
Example 1 Gaussian white noise (GWN) processes
Perhaps the most simple stationary time series is the independent Gaussian white noise process yt ∼ iid N (0, σ 2 ) ≡ GW N (0, σ 2 ). This process
has μ = γ j = ρj = 0 (j 6= 0). To simulate a GW N (0, 1) process in S-PLUS
use the rnorm function:

3.2 Univariate Time Series

59

-0.2

-2

0.0

-1

0.2

0

y

ACF
0.4

0.6

1

0.8

2

1.0

Series : y

0

20

40

60

80

100

0

2

4

6

8

10

Lag

FIGURE 3.1. Simulated Gaussian white noise process and SACF.

> set.seed(101)> y = rnorm(100,sd=1)
To compute the sample moments ȳ, γ̂ j , ρ̂j (j = 1, . . . , 10) and plot thedata and SACF use> y.bar = mean(y)> g.hat = acf(y,lag.max=10,type="covariance",plot=F)> r.hat = acf(y,lag.max=10,type="correlation",plot=F)> par(mfrow=c(1,2))> tsplot(y,ylab="y")> acf.plot(r.hat)
By default, as shown in Figure 3.1, the SACF is shown with 95% confidence limits about zero. These limits are based on the result (c.f. Fuller(1996) pg. 336) that if {yt } ∼ iid (0, σ2 ) thenµ¶1A, j > 0.
ρ̂j ∼ N 0,
T¡¢
A
The notation ρ̂j ∼ N 0, T1 means that the distribution of ρ̂j is approximated by normal distribution with mean 0 and variance T1 and is based on√dthe central limit theorem result T ρ̂j → N (0, 1). The 95% limits about
1.96
zero are then ± √T .

3. Time Series Concepts

y

-2

-1

0

1

2

60

-2

-1

0

1

2

Quantiles of Standard Normal

FIGURE 3.2. Normal qq-plot for simulated GWN.

Two slightly more general processes are the independent white noise(IWN) process, yt ∼ IW N (0, σ 2 ), and the white noise (WN) process,yt ∼ W N (0, σ 2 ). Both processes have mean zero and variance σ 2 , but
the IWN process has independent increments, whereas the WN process
has uncorrelated increments.
Testing for Normality
In the previous example, yt ∼ GW N (0, 1). There are several statisticalmethods that can be used to see if an iid process yt is Gaussian. The mostcommon is the normal quantile-quantile plot or qq-plot, a scatterplot of thestandardized empirical quantiles of yt against the quantiles of a standard
normal random variable. If yt is normally distributed, then the quantiles
will lie on a 45 degree line. A normal qq-plot with 45 degree line for yt may
be computed using the S-PLUS functions qqnorm and qqline> qqnorm(y)> qqline(y)Figure 3.2 shows the qq-plot for the simulated GWN data of the previous
example. The quantiles lie roughly on a straight line. The S+FinMetrics
function qqPlot may be used to create a Trellis graphics qq-plot.
The qq-plot is an informal graphical diagnostic. Two popular formal
statistical tests for normality are the Shapiro-Wilks test and the Jarque-3.2 Univariate Time Series

61

Bera test. The Shapiro-Wilk’s test is a well-known goodness of fit test for
the normal distribution. It is attractive because it has a simple, graphicalinterpretation: one can think of it as an approximate measure of the correlation in a normal quantile-quantile plot of the data. The Jarque-Beratest is based on the result that a normally distributed random variable has
skewness equal to zero and kurtosis equal to three. The Jarque-Bera test
statistic isÃ!
d − 3)2
T [ 2 (kurtJB =skew +(3.3)
6
4

d denotes the sample kur[ denotes the sample skewness and kurt
where skew
tosis. Under the null hypothesis that the data is normally distributedA

JB ∼ χ2 (2).

Example 2 Testing for normality using the S+FinMetrics function
normalTest
The Shapiro-Wilks and Jarque-Bera statistics may be computed using
the S+FinMetrics function normalTest. For the simulated GWN data of
the previous example, these statistics are> normalTest(y, method="sw")
Test for Normality: Shapiro-Wilks
Null Hypothesis: data is normally distributed
Test Statistics:
Test Stat 0.9703
p.value 0.1449
Dist. under Null: normal
Total Observ.: 100> normalTest(y, method="jb")Test for Normality: Jarque-BeraNull Hypothesis: data is normally distributed
Test Statistics:
Test Stat 1.8763
p.value 0.3914

62

3. Time Series Concepts

Dist. under Null: chi-square with 2 degrees of freedom
Total Observ.: 100
The null of normality is not rejected using either test.
Testing for White Noise
Consider testing the null hypothesis
H0 : yt ∼ W N (0, σ 2 )against the alternative that yt is not white noise. Under the null, all of the
autocorrelations ρj for j > 0 are zero. To test this null, Box and Pierce(1970) suggested the Q-statistic
Q(k) = T

k
X

ρ̂2j

(3.4)

j=1

where ρ̂j is given by (3.2). Under the null, Q(k) is asymptotically distributed χ2 (k). In a finite sample, the Q-statistic (3.4) may not be wellapproximated by the χ2 (k). Ljung and Box (1978) suggested the modified
Q-statistickXρ̂2j
MQ(k) = T (T + 2)
(3.5)
T −j
j=1
which is better approximated by the χ2 (k) in finite samples.
Example 3 Daily returns on Microsoft
Consider the time series behavior of daily continuously compounded returns on Microsoft for 2000. The following S-PLUS commands create thedata and produce some diagnostic plots:> r.msft = getReturns(DowJones30[,"MSFT"],type="continuous")> r.msft@title = "Daily returns on Microsoft"> sample.2000 = (positions(r.msft) > timeDate("12/31/1999")+ & positions(r.msft) < timeDate("1/1/2001"))> par(mfrow=c(2,2))> plot(r.msft[sample.2000],ylab="r.msft")> r.acf = acf(r.msft[sample.2000])> hist(seriesData(r.msft))> qqnorm(seriesData(r.msft))
The daily returns on Microsoft resemble a white noise process. The qqplot, however, suggests that the tails of the return distribution are fatterthan the normal distribution. Notice that since the hist and qqnorm functions do not have methods for “timeSeries” objects the extractor function seriesData is required to extract the data frame from the data slotof r.msft.

3.2 Univariate Time Series
Series : r.msft[sample.2000]

-0.2

-0.15

0.2

0.05

ACF
0.4 0.6

0.8

0.15

1.0

Daily returns on Microsoft

r.msft

63

0

Dec
2001

15

20

Lag

seriesData(r.msft)

600
400

10

0

200

5

0.1

Oct

0.0

Jun
Aug
2000

-0.1

Apr

800

Feb

-0.1

0.0

0.1

seriesData(r.msft)

-2

0

2

Quantiles of Standard Normal

FIGURE 3.3. Daily returns on Microsoft with diagnostic plots.

The S+FinMetrics functions histPlot and qqPlot will produce a histogram and qq-plot for a “timeSeries” object using Trellis graphics. For
example,> histPlot(r.msft,strip.text="MSFT monthly return")> qqPlot(r.msft,strip.text="MSFT monthly return")
However, Trellis plots cannot be displayed in a multipanel plot createdusing par.
The S+FinMetrics function autocorTest may be used to compute the
Q-statistic and modified Q-statistic to test the null that the returns on
Microsoft follow a white noise process:> autocorTest(r.msft, lag.n=10, method="lb")Test for Autocorrelation: Ljung-Box
Null Hypothesis: no autocorrelation
Test Statistics:
Test Stat 11.7746
p.value 0.3004
Dist. under Null: chi-square with 10 degrees of freedom

64

3. Time Series Concepts

Total Observ.: 2527
The argument lag.n=10 specifies that k = 10 autocorrelations are usedin computing the statistic, and method="lb" specifies that the modified
Box-Pierce statistic (3.5) be computed. To compute the simple Box-Pierce
statistic, specify method="bp". The results indicate that the white noise
null cannot be rejected.

3.2.2 Linear Processes and ARMA Models
Wold’s decomposition theorem (c.f. Fuller (1996) pg. 96) states that any
covariance stationary time series {yt } has a linear process or infinite ordermoving average representation of the form
yt

= μ+

∞
X

ψ k εt−k

(3.6)

k=0
∞
X

ψ 2k < ∞

ψ0

= 1,

εt

∼ W N (0, σ 2 )k=0

In the Wold form, it can be shown thatE[yt ] = μγ0= var(yt ) = σ 2

∞
X

ψ 2k

k=0

γj
ρj

= cov(yt , yt−j ) = σ 2=P∞

k=0 ψ k ψ k+jP∞2
k=0 ψ k∞X

ψ k ψ k+j

k=0Hence, the pattern of autocorrelations in any stationary and ergodic timeseries {yt } is determined by the moving average weights {ψ j } in its Wold
representation. To ensure convergence of the linear process representation
to a stationary and ergodic process with nice properties, it is necessary
to further restrict the behavior of the moving average weights {ψ j }. A
standard assumption used in the econometrics literature (c.f. Hamilton(1994) pg. 504) is 1-summability∞X
j=0j|ψ j | = 1 + 2|ψ 2 | + 3|ψ 3 | + · · · < ∞.

The moving average weights in the Wold form are also called impulse
responses since∂yt+s= ψ s , s = 1, 2, . . .
∂εt

3.2 Univariate Time Series

65

For a stationary and ergodicP
time series lims→∞ ψ s = 0 and the long-run
cumulative impulse response ∞s=0 ψ s < ∞. A plot of ψ s against s is called
the impulse response function (IRF).
The general Wold form of a stationary and ergodic time series is handy
for theoretical analysis but is not practically useful for estimation purposes.
A very rich and practically useful class of stationary and ergodic processes isthe autoregressive-moving average (ARMA) class of models made popular
by Box and Jenkins (1976). ARMA(p, q) models take the form of a pth
order stochastic diﬀerence equationyt − μ = φ1 (yt−1 − μ) + · · · + φp (yt−p − μ)+εt + θ1 εt−1 + · · · + θq εt−q
εt ∼ W N (0, σ 2 )(3.7)

ARMA(p, q) models may be thought of as parsimonious approximations
to the general Wold form of a stationary and ergodic time series. Moreinformation on the properties of ARMA(p, q) process and the proceduresfor estimating and forecasting these processes using S-PLUS are in the SPLUS Guide to Statistics Vol. II, chapter 27, Venables and Ripley (2002)
chapter 13, and Meeker (2001)1 .
Lag Operator Notation
The presentation of time series models is simplified using lag operator notation. The lag operator L is defined such that for any time series {yt },Lyt = yt−1 . The lag operator has the following properties: L2 yt = L · Lyt =yt−2 , L0 = 1 and L−1 yt = yt+1 . The operator ∆ = 1 − L creates the firstdiﬀerence of a time series: ∆yt = (1 − L)yt = yt − yt−1 . The ARMA(p, q)
model (3.7) may be compactly expressed using lag polynomials. Define
φ(L) = 1 − φ1 L − · · · − φp Lp and θ(L) = 1 + θ1 L + · · · + θq Lq . Then (3.7)
may be expressed as
φ(L)(yt − μ) = θ(L)εt
Similarly, the Wold representation in lag operator notation isyt= μ + ψ(L)εt∞X
ψ(L) =
ψ k Lk , ψ 0 = 1
k=0

and the long-run cumulative impulse response is ψ(1) (i.e. evaluate ψ(L)at L = 1). With ARMA(p, q) models the Wold polynomial ψ(L) is approx1 William Meeker also has a library of time series functions for the analysis of
ARMA models available for download at
http://www.public.iastate.edu/~stat451/splusts/splusts.html.

66

3. Time Series Concepts

imated by the ratio of the AR and MA polynomials
ψ(L) =

θ(L)φ(L)

3.2.3 Autoregressive Models
AR(1) Model
A commonly used stationary and ergodic time series in financial modelingis the AR(1) processyt − μ = φ(yt−1 − μ) + εt , t = 1, . . . , T
where εt ∼ W N (0, σ 2 ) and |φ| < 1. The above representation is called themean-adjusted form. The characteristic equation for the AR(1) is
φ(z) = 1 − φz = 0(3.8)

so that the root is z = φ1 . Stationarity is satisfied provided the absolutevalue of the root of the characteristic equation (3.8) is greater than one:σ2
| φ1 | > 1 or |φ| < 1. In this case, it is easy to show that E[yt ] = μ, γ 0 = 1−φ2,
ψ j = ρj = φj and the Wold representation isyt = μ +∞Xρj εt−j .

j=0

Notice that for the AR(1) the ACF and IRF are identical. This is not true1
in general. The long-run cumulative impulse response is ψ(1) = 1−φ.
The AR(1) model may be re-written in components form as
ytut= μ + ut
= φut−1 + εtor in autoregression form as
yt = c + φyt−1 + εtc = μ(1 − φ)
An AR(1) with μ = 1, φ = 0.75, σ 2 = 1 and T = 100 is easily simulated
in S-PLUS using the components form:> set.seed(101)> e = rnorm(100,sd=1)> e.start = rnorm(25,sd=1)> y.ar1 = 1 + arima.sim(model=list(ar=0.75), n=100,

3.2 Univariate Time Series
ACF and IRF for AR(1)

0.6
0.4

-2

0.2

0

2

Autocorrelation

4

Simulated AR(1)

67

0

20

40

60

80

100

1

2

3

4

5

6

7

8

9

10

lag

-0.2

0.2

ACF

0.6

1.0

Series : y.ar1

0

2

4

6

8

10

Lag

FIGURE 3.4. Simulated AR(1), ACF, IRF and SACF.

+ innov=e, start.innov=e.start)> mean(y.ar1)[1] 1.271> var(y.ar1)[1] 2.201
The ACF and IRF may be computed as> gamma.j = rep(0.75,10)^seq(10)The simulated data, ACF and SACF are illustrated in Figure 3.4 using> par(mfrow=c(2,2))> tsplot(y.ar1,main="Simulated AR(1)")> abline(h=1)> tsplot(gamma.j, type="h", main="ACF and IRF for AR(1)",+ ylab="Autocorrelation", xlab="lag")> tmp = acf(y.ar1, lag.max=10)
Notice that {yt } exhibits mean-reverting behavior. That is, {yt } fluctuates
about the mean value μ = 1. The ACF and IRF decay at a geometric rate.
The decay rate of the IRF is sometimes reported as a half-life — the lag
j half at which the IRF reaches 12 . For the AR(1) with positive φ, it can beshown that j half = ln(0.5)/ ln(φ). For φ = 0.75, the half-life is> log(0.5)/log(0.75)

68

3. Time Series Concepts

-0.5

-0.1

0.4

US/CA 30 day interest rate differential

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

ACF

0.0

0.2

0.4

0.6

0.8

1.0

Series : uscn.id

0

5

10

15

20

Lag

FIGURE 3.5. US/CA 30 day interest rate diﬀerential and SACF.

[1] 2.409
Many economic and financial time series are well characterized by an
AR(1) process. Leading examples in finance are valuation ratios (dividendprice ratio, price-earning ratio etc), real exchange rates, interest rates,
and interest rate diﬀerentials (spreads). To illustrate, consider the 30day US/CA interest rate diﬀerential2 constructed from the S+FinMetrics“timeSeries” object lexrates.dat:> uscn.id = 100*(lexrates.dat[,"USCNF"]+ lexrates.dat[,"USCNS"])> colIds(uscn.id) = "USCNID"> uscn.id@title = "US/CA 30 day interest rate differential"> par(mfrow=c(2,1))> plot(uscn.id,reference.grid=F)> abline(h=0)> tmp = acf(uscn.id)
The interest rate diﬀerential is clearly persistent: autocorrelations are
significant at the 5% level up to 15 months.
2 By covered interest rate parity, the nominal interest rate diﬀerential between risk
free bonds from two countries is equal to the diﬀerence between the nominal forward
and spot exchange rates.

3.2 Univariate Time Series

69

AR(p) Models
The AR(p) model in mean-adjusted form is
yt − μ = φ1 (yt−1 − μ) + · · · + φp (yt−p − μ) + εtor, in lag operator notation,φ(L)(yt − μ) = εtwhere φ(L) = 1 − φ1 L − · · · − φp Lp . The autoregressive form is
φ(L)yt = c + εt .
It can be shown that the AR(p) is stationary and ergodic provided theroots of the characteristic equation
φ(z) = 1 − φ1 z − φ2 z 2 − · · · − φp z p = 0(3.9)

lie outside the complex unit circle (have modulus greater than one). A
necessary condition for stationarity that is useful in practice is that |φ1 +· · · + φp | < 1. If (3.9) has complex roots then yt will exhibit sinusoidal
behavior. In the stationary AR(p), the constant in the autoregressive form
is equal to μ(1 − φ1 − · · · − φp ).
The moments of the AR(p) process satisfy the Yule-Walker equations
γ0γj= φ1 γ 1 + φ2 γ 2 + · · · + φp γ p + σ 2= φ1 γ j−1 + φ2 γ j−2 + · · · + φp γ j−p(3.10)

A simple recursive algorithm for finding the Wold representation is basedon matching coeﬃcients in φ(L) and ψ(L) such that φ(L)ψ(L) = 1. For
example, in the AR(2) model(1 − φ1 L − φ2 L2 )(1 + ψ 1 L + ψ 2 L2 + · · · ) = 1
implies
ψ1
ψ2
ψ3

ψj

= 1
= φ1 ψ 1 + φ2
= φ1 ψ 2 + φ2 ψ 1..
.
= φ1 ψ j−1 + φ2 ψ j−2

Partial Autocorrelation Function
The partial autocorrelation function (PACF) is a useful tool to help identify AR(p) models. The PACF is based on estimating the sequence of AR

70

3. Time Series Concepts
Series : irate.real

0.0-0.004

0.2

0.000

ACF
0.4 0.6

0.8

0.004

1.0

Monthly Real Interest Rate

0

5

10

15

20

25

1965

1970

1975

1980

1985

1990

1995

2000

Lag

0.0

Partial ACF
0.2 0.4 0.6

0.8

Series : irate.real

0

5

10

15

20

25

Lag

FIGURE 3.6. Monthly U.S. real interest rate, SACF and SPACF.

modelszt
zt

zt= φ11 zt−1 + ε1t= φ21 zt−1 + φ22 zt−2 + ε2t..
.
= φp1 zt−1 + φp2 zt−2 + · · · + φpp zt−p + εpt

where zt = yt −μ is the demeaned data. The coeﬃcients φjj for j = 1, . . . , p(i.e., the last coeﬃcients in each AR(p) model) are called the partial autocorrelation coeﬃcients. In an AR(1) model the first partial autocorrelationcoeﬃcient φ11 is non-zero, and the remaining partial autocorrelation coefficients φjj for j > 1 are equal to zero. Similarly, in an AR(2), the firstand second partial autocorrelation coeﬃcients φ11 and φ22 are non-zeroand the rest are zero for j > 2. For an AR(p) all of the first p partial
autocorrelation coeﬃcients are non-zero, and the rest are zero for j > p.
The sample partial autocorrelation coeﬃcients up to lag p are essentiallyobtained by estimating the above sequence of p AR models by least squares
and retaining the estimated coeﬃcients φ̂jj .
Example 4 Monthly real interest rates

3.2 Univariate Time Series

71

The “timeSeries” object varex.ts in the S+FinMetrics module contains monthly data on real stock returns, real interest rates, inflation andreal output growth.
> colIds(varex.ts)[1] "MARKET.REAL" "RF.REAL"

"INF"

"IPG"

Figure 3.6 shows the real interest rate, RF.REAL, over the period January
1961 through December 2000 produced with the S-PLUS commands> smpl = (positions(varex.ts) > timeDate("12/31/1960"))> irate.real = varex.ts[smpl,"RF.REAL"]> par(mfrow=c(2,2))> acf.plot(acf(irate.real, plot=F))> plot(irate.real, main="Monthly Real Interest Rate")> tmp = acf(irate.real, type="partial")
The SACF and SPACF indicate that the real interest rate might be modeled
as an AR(2) or AR(3) process.

3.2.4 Moving Average Models
MA(1) Model
The MA(1) model has the form
yt = μ + εt + θεt−1 , εt ∼ W N (0, σ 2 )For any finite θ the MA(1) is stationary and ergodic. The moments are
E[yt ] = μ, γ 0 = σ 2 (1 + θ2 ), γ 1 = σ2 θ, γ j = 0 for j > 1 and ρ1 = θ/(1 + θ2 ).
Hence, the ACF of an MA(1) process cuts oﬀ at lag one, and the maximum
value of this correlation is ±0.5.
There is an identification problem with the MA(1) model since θ = 1/θ
produce the same value of ρ1 . The MA(1) is called invertible if |θ| < 1 andis called non-invertible if |θ| ≥ 1. In the invertible MA(1), the error term
εt has an infinite order AR representation of the form
εt =

∞
X
θ∗j (yt−j − μ)j=0where θ∗ = −θ so that εt may be thought of as a prediction error based on
past values of yt . A consequence of the above result is that the PACF for
an invertible MA(1) process decays towards zero at an exponential rate.
Example 5 Signal plus noise model

72

3. Time Series Concepts
1st difference

0

-3

-2

-2

-1

y

dy

0

2

1

2

Signal plus noise

0

20

40

60

80

100

0

20

40

80

100

Series : dy

-0.5

-0.4

0.0

ACF

0.5

Partial ACF
-0.2
0.0

1.0

0.2

Series : dy

60

0

5

10
Lag

15

0

5

10
Lag

15

FIGURE 3.7. Simulated data, SACF and SPACF from signal plus noise model.

MA(1) models often arise through data transformations like aggregationand diﬀerencing3 . For example, consider the signal plus noise modelyt
zt= zt + εt , εt ∼ W N (0, σ 2ε )= zt−1 + ηt , η t ∼ W N (0, σ2η )where εt and η t are independent. For example, zt could represent the fundamental value of an asset price and εt could represent an iid deviation about
the fundamental price. A stationary representation requires diﬀerencing yt :∆yt = ηt + εt − εt−1
It can be√shown, e.g. Harvey (1993), that ∆yt is an MA(1) process with θ =−(q+2)+
2

q 2 +4qσ2−1
where q = ση2 is the signal-to-noise ratio and ρ1 = q+2
< 0.
ε
2
2
2
Simulated data with σ ε = 1 and σ η = (0.5) created with the S-PLUS
commands> set.seed(112)> eps = rnorm(100,sd=1)> eta = rnorm(100,sd=0.5)
3 MA(1) type models for asset returns often occur as the result of no-trading eﬀects
or bid-ask bounce eﬀects. See Campbell, Lo and MacKinlay (1997) chapter 3 for details.

3.2 Univariate Time Series73> z = cumsum(eta)> y = z + eps> dy = diff(y)> par(mfrow=c(2,2))> tsplot(y, main="Signal plus noise",ylab="y")> tsplot(dy, main="1st difference",ylab="dy")> tmp = acf(dy)> tmp = acf(dy,type="partial")
are illustrated in Figure 3.7. The signal-to-noise ratio q = 0.25 implies afirst lag autocorrelation of ρ1 = −0.444. This negative correlation is clearlyreflected in the SACF.
MA(q) Model
The MA(q) model has the form
yt = μ + εt + θ1 εt−1 + · · · + θq εt−q , where εt ∼ W N (0, σ 2 )The MA(q) model is stationary and ergodic provided θ1 , . . . , θ q are finite.
It is invertible if all of the roots of the MA characteristic polynomial
θ(z) = 1 + θ1 z + · · · θq z q = 0(3.11)

lie outside the complex unit circle. The moments of the MA(q) areE[yt ] = μγ 0 = σ 2 (1 + θ21 + · · · + θ2q )½(θj + θj+1 θ1 + θj+2 θ2 + · · · + θq θq−j ) σ2 for j = 1, 2, . . . , q
γj =0 for j > q
Hence, the ACF of an MA(q) is non-zero up to lag q and is zero afterwards.
As with the MA(1), the PACF for an invertible MA(q) will show exponential decay and possibly pseudo cyclical behavior if the roots of (3.11) arecomplex.
Example 6 Overlapping returns and MA(q) models
MA(q) models often arise in finance through data aggregation transformations. For example, let Rt = ln(Pt /Pt−1 ) denote the monthly continuously compounded return on an asset with price Pt . Define the annual return at time t using monthly returns as Rt (12) = ln(Pt /Pt−12 ) =P11
2
j=0 Rt−j . Suppose Rt ∼ W N (μ, σ ) and consider a sample of monthly
returns of size T , {R1 , R2 , . . . , RT }. A sample of annual returns may be created using overlapping or non-overlapping returns. Let {R12 (12), R13 (12),. . . , RT (12)} denote a sample of T ∗ = T − 11 monthly overlapping annual
returns and {R12 (12), R24 (12), . . . , RT (12)} denote a sample of T /12 nonoverlapping annual returns. Researchers often use overlapping returns in74

3. Time Series Concepts

analysis due to the apparent larger sample size. One must be careful using
overlapping returns because the monthly annual return sequence {Rt (12)}is not a white noise process even if the monthly return sequence {Rt } is.
To see this, straightforward calculations give
E[Rt (12)]
γ0
γj
γj

=
=
=
=

12μ
var(Rt (12)) = 12σ 2
cov(Rt (12), Rt−j (12)) = (12 − j)σ 2 for j < 12
0 for j ≥ 12

Since γ j = 0 for j ≥ 12 notice that {Rt (12)} behaves like an MA(11)processRt (12) = 12μ + εt + θ1 εt−1 + · · · + θ11 εt−11
εt ∼ W N (0, σ 2 )To illustrate, consider creating annual overlapping continuously compounded returns on the S&P 500 index over the period February 1990
through January 2001. The S+FinMetrics “timeSeries” singleIndex.dat
contains the S&P 500 price data and the continuously compounded monthly
returns are computed using the S+FinMetrics function getReturns> sp500.mret = getReturns(singleIndex.dat[,"SP500"],+ type="continuous")> sp500.mret@title = "Monthly returns on S&P 500 Index"
The monthly overlapping annual returns are easily computed using the
S-PLUS function aggregateSeries> sp500.aret = aggregateSeries(sp500.mret,moving=12,FUN=sum)> sp500.aret@title = "Monthly Annual returns on S&P 500 Index"
The optional argument moving=12 specifies that the sum function is tobe applied to moving blocks of size 12. The data together with the SACF
and SPACF of the monthly annual returns are displayed in Figure 3.8.
The SACF has non-zero values up to lag 11. Interestingly, the SPACF isvery small at all lags except the first.

3.2.5 ARMA(p,q) Models
The general ARMA(p, q) model in mean-adjusted form is given by (3.7).
The regression formulation isyt = c + φ1 yt−1 + · · · + φp yt−p + εt + θεt−1 + · · · + θεt−q(3.12)

It is stationary and ergodic if the roots of the characteristic equation φ(z) =0 lie outside the complex unit circle, and it is invertible if the roots of the3.2 Univariate Time Series

Monthly Annual returns on S&P 500 Index

-0.10

-0.15

0.10

-0.05

0.05

0.30

Monthly returns on S&P 500 Index

75

1990

1992

1994

1996

1998

2000

1990

1994

1996

1998

2000

Series : sp500.aret

-0.2

-0.2

0.0

0.2

Partial ACF
0.2 0.4 0.6

ACF
0.4 0.6 0.8

0.8

1.0

Series : sp500.aret

1992

0

5

10
Lag

15

20

0

5

10
Lag

15

20

FIGURE 3.8. Monthly non-overlapping and overlapping annual returns on the
S&P 500 index.

MA characteristic polynomial θ(z) = 0 lie outside the unit circle. It is
assumed that the polynomials φ(z) = 0 and θ(z) = 0 do not have cancelingor common factors. A stationary and ergodic ARMA(p, q) process has amean equal toc
μ=(3.13)
1 − φ1 − · · · − φpand its autocovariances, autocorrelations and impulse response weights satisfy the recursive relationships
γj
ρj
ψj

= φ1 γ j−1 + φ2 γ j−2 + · · · + φp γ j−p= φ1 ρj−1 + φ2 ρj−2 + · · · + φp ρj−p
= φ1 ψ j−1 + φ2 ψ j−2 + · · · + φp ψ j−p

The general form of the ACF for an ARMA(p, q) process is complicated.
See Hamilton (1994) chapter five for details. In general, for an ARMA(p, q)
process, the ACF behaves like the ACF for an AR(p) process for p > q, and
the PACF behaves like the PACF for an MA(q) process for q > p. Hence,both the ACF and PACF eventually show exponential decay.
ARMA(p, q) models often arise from certain aggregation transformations of simple time series models. An important result due to Granger
and Morris (1976) is that if y1t is an ARMA(p1 , q1 ) process and y2t isan ARMA(p2 , q2 ) process, which may be contemporaneously correlated

76

3. Time Series Concepts

with y1t , then y1t + y2t is an ARMA(p, q) process with p = p1 + p2 andq = max(p1 + q2 , q1 + p2 ). For example, if y1t is an AR(1) process and y2
is a AR(1) process, then y1 + y2 is an ARMA(2,1) process.
High order ARMA(p, q) processes are diﬃcult to identify and estimate
in practice and are rarely used in the analysis of financial data. Low order
ARMA(p, q) models with p and q less than three are generally suﬃcient for
the analysis of financial data.
ARIMA(p, d, q) Models
The specification of the ARMA(p, q) model (3.7) assumes that yt is stationary and ergodic. If yt is a trending variable like an asset price or a macroeconomic aggregate like real GDP, then yt must be transformed to stationary
form by eliminating the trend. Box and Jenkins (1976) advocate removal oftrends by diﬀerencing. Let ∆ = 1−L denote the diﬀerence operator. If there
is a linear trend in yt then the first diﬀerence ∆yt = yt − yt−1 will not havea trend. If there is a quadratic trend in yt , then ∆yt will contain a linear
trend but the second diﬀerence ∆2 yt = (1 − 2L + L2 )yt = yt − 2yt−1 + yt−2
will not have a trend. The class of ARMA(p, q) models where the trends
have been transformed by diﬀerencing d times is denoted ARIMA(p, d, q)4 .

3.2.6 Estimation of ARMA Models and Forecasting
ARMA(p, q) models are generally estimated using the technique of maximum likelihood, which is usually accomplished by putting the ARMA(p, q)
in state-space form from which the prediction error decomposition of the
log-likelihood function may be constructed. Details of this process are given
in Harvey (1993). An often ignored aspect of the maximum likelihood estimation of ARMA(p, q) models is the treatment of initial values. These
initial values are the first p values of yt and q values of εt in (3.7). The exact likelihood utilizes the stationary distribution of the initial values in theconstruction of the likelihood. The conditional likelihood treats the p initialvalues of yt as fixed and often sets the q initial values of εt to zero. The exact
maximum likelihood estimates (MLEs) maximize the exact log-likelihood,and the conditional MLEs maximize the conditional log-likelihood. The
exact and conditional MLEs are asymptotically equivalent but can diﬀer
substantially in small samples, especially for models that are close to being
nonstationary or noninvertible.5
4 More general ARIMA(p, d, q) models allowing for seasonality are discussed in chapter
27 of the S-PLUS Guide to Statistics, Vol. II.
5 As pointed out by Venables and Ripley (1999) page 415, the maximum likelihood
estimates computed using the S-PLUS function arima.mle are conditional MLEs. Exact
MLEs may be easily computed using the S+FinMetrics state space modeling functions.

3.2 Univariate Time Series

77

For pure AR models, the conditional MLEs are equivalent to the leastsquares estimates from the modelyt = c + φ1 yt−1 + · · · + φp yt−p + εt(3.14)

Notice, however, that c in (3.14) is not an estimate of E[yt ] = μ. The leastsquares estimate of μ is given by plugging in the least squares estimates ofc, φ1 , . . . , φp into (3.13).
Model Selection Criteria
Before an ARMA(p, q) may be estimated for a time series yt , the AR and
MA orders p and q must be determined by visually inspecting the SACF
and SPACF for yt . Alternatively, statistical model selection criteria may
be used. The idea is to fit all ARMA(p, q) models with orders p ≤ pmax andq ≤ qmax and choose the values of p and q which minimizes some model
selection criteria. Model selection criteria for ARMA(p, q) models have theform
MSC(p, q) = ln(σ̃2 (p, q)) + cT · ϕ(p, q)
where σ̃ 2 (p, q) is the MLE of var(εt ) = σ2 without a degrees of freedom correction from the ARMA(p, q) model, cT is a sequence indexed by the sample
size T , and ϕ(p, q) is a penalty function which penalizes large ARMA(p, q)
models. The two most common information criteria are the Akaike (AIC)and Schwarz-Bayesian (BIC):2(p + q)Tln T
BIC(p, q) = ln(σ̃ 2 (p, q)) +(p + q)T

AIC(p, q) = ln(σ̃ 2 (p, q)) +The AIC criterion asymptotically overestimates the order with positive
probability, whereas the BIC estimate the order consistently under fairly
general conditions if the true orders p and q are less than or equal to pmax
and qmax . However, in finite samples the BIC generally shares no particularadvantage over the AIC.
Forecasting Algorithm
Forecasts from an ARIMA(p, d, q) model are straightforward. The modelis put in state space form, and optimal h-step ahead forecasts along with
forecast standard errors (not adjusted for parameter uncertainty) are produced using the Kalman filter algorithm. Details of the method are given
in Harvey (1993).

78

3. Time Series Concepts

Estimation and Forecasting ARIMA(p, d, q) Models Using the S-PLUS
Function arima.mle
Conditional MLEs may be computed using the S-PLUS function arima.mle.
The form of the ARIMA(p, d, q) assumed by arima.mle isyt= φ1 yt−1 + · · · + φp yt−p+εt − θ1 εt−1 − · · · − θq εt−q
+β0 xtwhere xt represents additional explanatory variables. It is assumed thatyt has been diﬀerenced d times to remove any trends and that the unconditional mean μ has been subtracted out so that yt is demeaned. Noticethat arima.mle assumes that the signs on the MA coeﬃcients θj are theopposite to those in (3.7).
The arguments expected by arima.mle are> args(arima.mle)function(x, model = NULL, n.cond = 0, xreg = NULL, ...)
where x is a univariate “timeSeries” or vector, model is a list object describing the specification of the ARMA model, n.cond sets thenumber of initial observations on which to condition in the formation of
the log-likelihood, and xreg is a “timeSeries”, vector or matrix of additional explanatory variables. By default, arima.mle assumes that the
ARIMA(p, d, q) model is stationary and in mean-adjusted form with an estimate of μ subtracted from the observed data yt . To estimate the regression
form (3.12) of the ARIMA(p, q) model, simply set xreg=1. ARIMA(p, d, q)models are specified using list variables the form
> mod.list = list(order=c(1,0,1))> mod.list = list(order=c(1,0,1),ar=0.75,ma=0)> mod.list = list(ar=c(0.75,-0.25),ma=c(0,0))
The first list simply specifies an ARMA(1,0,1)/ARMA(1,1) model. The
second list specifies an ARIMA(1,0,1) as well as starting values for the
AR and MA parameters φ and θ. The third list implicitly determines an
ARMA(2,2) model by giving the starting values for the AR and MA parameters. The function arima.mle produces an object of class “arima” forwhich there are print and plot methods. Diagnostics from the fit canbe created with the S-PLUS function arima.diag, and forecasts may be
produced using arima.forecast.
Example 7 Estimation of ARMA model for US/CA interest rate diﬀerential
Consider estimating an ARMA(p, q) for the monthly US/CA interest
rate diﬀerential data in the “timeSeries” uscn.id used in a previous

3.2 Univariate Time Series

79

example. To estimate an ARMA(1,1) model for the demeaned interest rate
diﬀerential with starting values φ = 0.75 and θ = 0 use> uscn.id.dm = uscn.id - mean(uscn.id)> arma11.mod = list(ar=0.75,ma=0)> arma11.fit = arima.mle(uscn.id.dm,model=arma11.mod)> class(arma11.fit)[1] "arima"
The components of arma11.fit are> names(arma11.fit)[1] "model""var.coef" "method""series"[5] "aic""loglik""sigma2""n.used"[9] "n.cond""converged" "conv.type" "call"
To see the basic fit simply type> arma11.fit
Call: arima.mle(x = uscn.id.dm, model = arma11.mod)
Method: Maximum Likelihood
Model : 1 0 1
Coefficients:
AR : 0.82913
MA : 0.11008
Variance-Covariance Matrix:
ar(1)
ma(1)
ar(1) 0.002046 0.002224
ma(1) 0.002224 0.006467
Optimizer has converged
Convergence Type: relative function convergence
AIC: -476.25563
The conditional MLEs are φ̂cmle = 0.829 and θ̂cmle = −0.110. Standard
errors for these parameters are given by the square roots of the diagonal
elements of variance-covariance matrix> std.errs = sqrt(diag(arma11.fit$var.coef))> names(std.errs) = colIds(arma11.fit$var.coef)> std.errs
ar(1)
ma(1)
0.04523 0.08041
It appears that the θ̂cmle is not statistically diﬀerent from zero.
To estimate the ARMA(1,1) for the interest rate diﬀerential data inregression form (3.12) with an intercept use

80

3. Time Series Concepts> arma11.fit2 = arima.mle(uscn.id,model=arma11.mod,xreg=1)> arma11.fit2
Call: arima.mle(x = uscn.id, model = arma11.mod, xreg = 1)
Method: Maximum Likelihood
Model : 1 0 1
Coefficients:
AR : 0.82934
MA : 0.11065
Variance-Covariance Matrix:
ar(1)
ma(1)
ar(1) 0.002043 0.002222
ma(1) 0.002222 0.006465
Coeffficients for regressor(s): intercept[1] -0.1347
Optimizer has converged
Convergence Type: relative function convergence
AIC: -474.30852
The conditional MLEs for φ and θ are essentially the same as before, andthe MLE for c is ĉcmle = −0.1347. Notice that the reported variancecovariance matrix only gives values for the estimated ARMA coeﬃcientsφ̂cmle and θ̂cmle .
Graphical diagnostics of the fit produced using the plot method> plot(arma11.fit)
are illustrated in Figure 3.9. There appears to be some high order serial
correlation in the errors as well as heteroskedasticity.
The h-step ahead forecasts of future values may be produced with the
S-PLUS function arima.forecast. For example, to produce monthly forecasts for the demeaned interest rate diﬀerential from July 1996 throughJune 1997 use> fcst.dates = timeSeq("7/1/1996", "6/1/1997",+ by="months", format="%b %Y")
> uscn.id.dm.fcst = arima.forecast(uscn.id.dm, n=12,+ model=arma11.fit$model, future.positions=fcst.dates)> names(uscn.id.dm.fcst)[1] "mean""std.err"
The object uscn.id.dm.fcst is a list whose first component is a“timeSeries” containing the h-step forecasts, and the second component
is a “timeSeries” containing the forecast standard errors:> uscn.id.dm.fcst[[1]]3.2 Univariate Time Series

81

ARIMA Model Diagnostics: uscn.id.dm

-5

-1

3

Plot of Standardized Residuals

1978

1980

1982

1984
1986
1988
ACF Plotof Residuals

1990

1992

1994

1996

-1.0

ACF
0.0

0.5

1.0

1976

0

5

10

15

20

-0.1

PACF
0.1

0.2

PACF Plot of Residuals

5

10

15

20

0.0

p-value
0.2
0.4

0.6

P-values of Ljung-Box Chi-Squared Statistics

4

6

8

10

12

14

Lag

ARIMA(1,0,1) Model with Mean 0

FIGURE 3.9. Residual diagnostics from ARMA(1,1) fit to US/CA interest rate
diﬀerentials.

Positions1
Jul 1996 0.09973
Aug 1996 0.08269
Sep 1996 0.06856
Oct 1996 0.05684
Nov 1996 0.04713
Dec 1996 0.03908
Jan 1997 0.03240
Feb 1997 0.02686
Mar 1997 0.02227
Apr 1997 0.01847
May 1997 0.01531
Jun 1997 0.01270
The data, forecasts and 95% forecast confidence intervals shown in Figure
3.10 are produced by> smpl = positions(uscn.id.dm) >= timeDate("6/1/1995")> plot(uscn.id.dm[smpl,],uscn.id.dm.fcst$mean,+ uscn.id.dm.fcst$mean+2*uscn.id.dm.fcst$std.err,+ uscn.id.dm.fcst$mean-2*uscn.id.dm.fcst$std.err,+ plot.args=list(lty=c(1,4,3,3)))82

3. Time Series Concepts

-0.2

-0.1

0.0

0.1

0.2

0.3

US/CA 30 day interest rate differential

Q3

Q4

Q1

Q2

1995

Q3

Q4

1996

Q1

Q2
1997

FIGURE 3.10. Forecasts for 12 months for the series uscn.id.dm.

Estimating AR(p) by Least Squares Using the S+FinMetrics Function
OLS
As previously mentioned, the conditional MLEs for an AR(p) model may
be computed using least squares. The S+FinMetrics function OLS, which
extends the S-PLUS function lm to handle general time series regression,
may be used to estimate an AR(p) in a particularly convenient way. The
general use of OLS is discussed in Chapter 6, and its use for estimating an
AR(p) is only mentioned here. For example, to estimate an AR(2) modelfor the US/CA interest rate diﬀerential use> ar2.fit = OLS(USCNID~ar(2), data=uscn.id)> ar2.fit
Call:
OLS(formula = USCNID ~ar(2), data = uscn.id)
Coefficients:(Intercept)
lag1-0.0265
0.7259lag2
0.0758Degrees of freedom: 243 total; 240 residual
Time period: from Apr 1976 to Jun 1996
Residual standard error: 0.09105

3.2 Univariate Time Series

83

The least squares estimates of the AR coeﬃcients are φ̂1 = 0.7259 andφ̂2 = 0.0758. Since φ̂1 + φ̂2 < 1 the estimated AR(2) model is stationary.
To be sure, the roots of φ(z) = 1 − φ̂1 z − φ̂2 z 2 = 0 are> abs(polyroot(c(1,-ar2.fit$coef[2:3])))[1] 1.222 10.798
are outside the complex unit circle.

3.2.7 Martingales and Martingale Diﬀerence Sequences
Let {yt } denote a sequence of random variables and let It = {yt, yt−1 , . . .}denote a set of conditioning information or information set based on thepast history of yt . The sequence {yt , It } is called a martingale if• It−1 ⊂ It (It is a filtration)• E[|yt |] < ∞• E[yt |It−1 ] = yt−1 (martingale property)
The most common example of a martingale is the random walk modelyt = yt−1 + εt , εt ∼ W N (0, σ 2 )where y0 is a fixed initial value. Letting It = {yt , . . . , y0 } implies E[yt |It−1 ] =yt−1 since E[εt |It−1 ] = 0.
Let {εt } be a sequence of random variables with an associated information set It . The sequence {εt , It } is called a martingale diﬀerence sequence(MDS) if
• It−1 ⊂ It
• E[εt |It−1 ] = 0 (MDS property)
If {yt , It } is a martingale, a MDS {εt , It } may be constructed by defining
εt = yt − E[yt |It−1 ]By construction, a MDS is an uncorrelated process. This follows from thelaw of iterated expectations. To see this, for any k > 0E[εt εt−k ] = E[E[εt εt−k |It−1 ]]= E[εt−k E[εt |It−1 ]]= 0
In fact, if zn is any function of the past history of εt so that zn ∈ It−1 thenE[εt zn ] = 0

84

3. Time Series Concepts

Although a MDS is an uncorrelated process, it does not have to be an
independent process. That is, there can be dependencies in the higher ordermoments of εt . The autoregressive conditional heteroskedasticity (ARCH)
process in the following example is a leading example in finance.
MDSs are particularly nice to work with because there are many usefulconvergence results (laws of large numbers, central limit theorems etc.).
White (1984), Hamilton (1994) and Hayashi (2000) describe the most usefulof these results for the analysis of financial time series.
Example 8 ARCH process
A well known stylized fact about high frequency financial asset returns
is that volatility appears to be autocorrelated. A simple model to capturesuch volatility autocorrelation is the ARCH process due to Engle (1982).
To illustrate, let rt denote the daily return on an asset and assume thatE[rt ] = 0. An ARCH(1) model for rt is
rt
zt
σ 2t

= σ t zt
∼ iid N (0, 1)
2= ω + αrt−1(3.15)(3.16)

where ω > 0 and 0 < α < 1. Let It = {rt , . . .}. The S+FinMetrics function
simulate.garch may be used to generate simulations from above ARCH(1)model. For example, to simulate 250 observations on rt with ω = 0.1 and
α = 0.8 use> rt = simulate.garch(model=list(a.value=0.1, arch=0.8),+n=250, rseed=196)> class(rt)[1] "structure"> names(rt)[1] "et""sigma.t"
Notice that the function simulate.garch produces simulated values ofboth rt and σ t . These values are shown in Figure 3.11.
To see that {rt , It } is a MDS, note thatE[rt |It−1 ] = E[zt σ t |It−1 ]= σ t E[zt |It−1 ]= 0
Since rt is a MDS, it is an uncorrelated process. Provided |α| < 1, rt is amean zero covariance stationary process. The unconditional variance of rt
is given byvar(rt ) = E[rt2 ] = E[E[zt2 σ2t |It−1 ]]= E[σ2t E[zt2 |It−1 ] = E[σ 2t ]

3.2 Univariate Time Series

85

-2

-1

0

1

2

Simulated returns

0

50

100

150

200

250

200

250

0.5

1.0

1.5

Simulated volatility

0

50

100

150

FIGURE 3.11. Simulated values from ARCH(1) process with ω = 1 and α = 0.8.

since E[zt2 |It−1 ] = 1. Utilizing (3.16) and the stationarity of rt , E[σ 2t ] may
be expressed asωE[σ 2t ] =1−αFurthermore, by adding ε2t to both sides of (3.16) and rearranging it follows
that rt2 has an AR(1) representation of the form
ε2t = ω + αε2t−1 + vt
where vt = ε2t − σ 2t is a MDS.

3.2.8 Long-run Variance
Let yt be a stationary and ergodic time series. Anderson’s central limit
theorem for stationary and ergodic processes (c.f. Hamilton (1994) pg. 195)
states∞X√d
T (ȳ − μ) → N (0,
γj )
j=−∞

or

⎛

⎞
∞
X
1
γ ⎠
ȳ ∼ N ⎝μ,
T j=−∞ jA86

3. Time Series Concepts

The sample size, T , times the asymptotic variance of the sample mean isoften called the long-run variance of yt 6 :lrv(yt ) = T · avar(ȳ) =

∞
X

γj .

j=−∞

Since γ −j = γ j , lrv(yt ) may be alternatively expressed aslrv(yt ) = γ 0 + 2

∞
X

γj .

j=1

Using the long-run variance, an asymptotic 95% confidence interval forμ takes the formqc t)ȳ ± 1.96 · T −1 lrv(yc t ) is a consistent estimate of lrv(yt ).
where lrv(yEstimating the Long-Run VarianceIf yt is a linear process, it may be shown that
⎛
⎞2
∞
∞
X
X
γ j = σ2 ⎝ψ j ⎠ = σ 2 ψ(1)2
j=−∞

j=0

and solrv(yt ) = σ 2 ψ(1)2(3.17)

Further, if yt ∼ ARMA(p, q) thenψ(1) =

θ(1)1 + θ1 + · · · + θq=
1 − φ1 − · · · − φpφ(1)so thatlrv(yt ) =σ 2 θ(1)2.
φ(1)2(3.18)A consistent estimate of lrv(yt ) may then be computed by estimating theparameters of the appropriate ARMA(p, q) model and substituting theseestimates into (3.18). Alternatively, the ARMA(p, q) process may be approximated by a high order AR(p∗ ) process
yt = c + φ1 yt−1 + · · · + φp∗ yt−p∗ + εt
6 Using spectral methods, lrv(ȳ) has the alternative representation

12πf (0)T
where f (0) denotes the spectral density of yt evaluated at frequency 0.
lrv(ȳ) =3.2 Univariate Time Series

87

where the lag length p∗ is chosen such that εt is uncorrelated. This gives
rise to the autoregressive long-run variance estimate
lrvAR (yt ) =σ2.
φ (1)2∗(3.19)A consistent estimate of lrv(yt ) may also be computed using some nonparametric methods. An estimator made popular by Newey and West
(1987) is the weighted autocovariance estimatorc NW (yt ) = γ̂ + 2
lrv
0

MT
X
j=1

wj,T · γ̂ j

(3.20)

where wj,T are weights which sum to unity and MT is a truncation lag
parameter that satisfies MT = O(T 1/3 ). For MA(q) processes, γ j = 0 for
j > q and Newey and West suggest using the rectangular weights wj,T = 1
for j ≤ MT = q; 0 otherwise. For general linear processes, Newey and Westsuggest using the Bartlett weights wj,T = 1 − MTj+1 with MT equal to theinteger part of 4(T /100)2/9 .
Example 9 Long-run variance of AR(1)Let yt be an AR(1) process created using> set.seed(101)> e = rnorm(100,sd=1)> y.ar1 = 1 + arima.sim(model=list(ar=0.75),innov=e)
1
1
= 1−φ
and
Here ψ(1) = φ(1)lrv(yt ) =σ2.
(1 − φ)2

For φ = 0.75, σ 2 = 1, lrv(yt ) = 16 implies for T = 100 an asymptotic
standard error for ȳ equal to SE(ȳ) = 0.40. If yt ∼ W N (0, 1), then theasymptotic standard error for ȳ is SE(ȳ) = 0.10.
lrvAR (yt ) may be easily computed in S-PLUS using OLS to estimate theAR(1) parameters:> ar1.fit = OLS(y.ar1~ar(1))> rho.hat = coef(ar1.fit)[2]> sig2.hat = sum(residuals(ar1.fit)^2)/ar1.fit$df.resid> lrv.ar1 = sig2.hat/(1-rho.hat)^2> as.numeric(lrv.ar1)[1] 13.75
c AR (ȳ) = 0.371.
Here lrvAR (yt ) = 13.75, and an estimate for SE(ȳ) is SE
The S+FinMetrics function asymp.var may be used to compute thenonparameteric Newey-West estimate lrvNW (yt ). The arguments expectedby asymp.var are

88

3. Time Series Concepts> args(asymp.var)function(x, bandwidth, window = "bartlett", na.rm = F)
where x is a “timeSeries”, bandwidth sets the truncation lag MT in(3.20) and window specifies the weight function. Newey and West suggest
setting the bandwidth using the sample size dependent rule
MT = 4(T /100)2/9
which is equal to 4 in the present case. The Newey-West long-run varianceestimate is then> lrv.nw = asymp.var(y.ar1, bandwidth=4)> lrv.nw[1] 7.238
c NW (ȳ) = 0.269.
and the Newey-West estimate of SE(ȳ) is SE

3.2.9 Variance Ratios

There has been considerable interest in testing the so-called random walk
(RW) model for log stock prices (see chapter 2 in Campbell, Lo and MacKinlay (1997) for an extensive review). The RW model for log prices pt has theform
pt = μ + pt−1 + εt , t = 1, . . . , T
where εt is a random error term. Using rt = ∆pt , the RW model may be
rewritten as
rt = μ + εt
Campbell, Lo and MacKinlay distinguish three forms of the random walkmodel:RW1 εt ∼ iid(0, σ 2 )RW2 εt is an independent process (allows for heteroskedasticity)
RW3 εt is an uncorrelated process (allows for dependence in higher ordermoments)
For asset returns, RW1 and RW2 are not very realistic and, therefore, mostattention has been placed on testing the model RW3.
Some commonly used tests for RW3 are based on constructing varianceratios. To illustrate, consider the simple two-period variance ratio
VR(2) =

var(rt (2))
2 · var(rt )3.2 Univariate Time Series

89

The numerator of the variance ratio is the variance of the two-period return,rt (2) = rt−1 + rt , and the deminator is two times the variance of the oneperiod return, rt . Under RW1, is easy to see that VR(2) = 1. If {rt } is anergodic-stationary process thenVR(2) ==

var(rt−1 ) + var(rt ) + 2 · cov(rt , rt−1 )2 · var(rt )2γ 0 + 2γ 1
= 1 + ρ1
2γ 0There are three cases of interest depending on the value of ρ1 . If ρ1 = 0
then VR(2) = 1; if ρ1 > 1 then VR(2) > 1; if ρ1 < 1 then VR(2) < 1.
The general q−period variance ratio isVR(q) =

var(rt (q))
q · var(rt )(3.21)

where rt (q) = rt−q+1 + · · · + rt . Under RW1, VR(q) = 1. For ergodic
stationary returns, some algebra shows that
¶
q µ
X
k
VR(q) = 1 + 2 ·
1−
ρk
q
k=1

When the variance ratio is greater than one, returns are called mean averting due to the dominating presence of positive autocorrelations. When thevariance ratio is less than one, returns are called mean reverting due to thedominating presence of negative autocorrelations. Using the Wold representation (3.6), it can be shown thatlim VR(q) =q→∞

σ2 ψ(1)2
lrv(rt )=γ0var(rt )That is, as q becomes large the variance ratio approaches the ratio of thelong-run variance to the short-run variance. Furthermore, Under RW2 and
RW3 it can be shown that VR(q) → 1 as q → ∞ provided
T

1X
var(rt ) → σ̄ 2 > 0
T t=1
Test Statistics
Let {p0 , p1 , . . . , pT q } denote a sample of T q + 1 log prices, which producesa sample of T q one-period returns {r1 , . . . , rT q }. Lo and MacKinlay (1988,
1989) develop a number of test statistics for testing the random walk hypothesis based on the estimated variance ratiovar(rc t (q))dVR(q)=q · var(rc t)(3.22)

90

3. Time Series Concepts

The form of the statistic depends on the particular random walk model(RW1, RW2 or RW3) assumed under the null hypothesis.
Under RW1, (3.22) is computed using2σ̂ (q)
d
VR(q)
=
σ̂ 2

where

Tq

σ̂2

=

1 X
(rk − μ̂)2
Tq
k=1

Tq

σ̂ 2 (q) =1 X(rk (q) − qμ̂)2
T q2
k=q

μ̂ =

1
Tq

Tq
X

rk =

k=1

1
(pT q − p0 )Tq

Lo and MacKinlay show that, under RW1,
pA
d − 1) ∼T q(VR(q)N (0, 2(q − 1))

Therefore, the variance ratio test statistic
ψ̂(q) =µTq2(q − 1)

¶1/2

d − 1)(VR(q)(3.23)

has a limiting standard normal distribution under RW1.
Lo and MacKinlay also derive a modified version of (3.23) based on thefollowing bias corrected estimates of σ 2 and σ 2 (q) :
Tq

σ̄ 2

=

1 X
(rk − μ̂)2Tq − 1k=1

σ̄ 2 (q) =

1
m

Tq
X

k=q

(rk (q) − qμ̂)2µ¶q
m = q(T q − q + 1) 1 −Tq
Defining VR(q) = σ̄ 2 (q)/σ̄ 2 , the biased corrected version of (3.23) has the
form
µ
¶1/2
3T q 2ψ̄(q) =(VR(q) − 1)
(3.24)
2(2q − 1)(q − 1)which has a limiting standard normal distribution under RW1.

3.2 Univariate Time Series

91

The variance ratio statistics (3.23) and (3.24) are not valid under the empirically relevant RW3 and RW3 models. For this model, Lo and MacKinlay
derived the heteroskedasticity robust variance ratio statisticψ ∗ (q) = Ω̂(q)−1/2 (VR(q) − 1)

(3.25)

whereΩ̂(q) =¶q−1 µX2(q − j)
j=1

j

δ̂ j

PT q

t=j+1 α̂0t α̂jt
´2
α̂
0t
j=1

δ̂ j

=

α̂jt

= (rt−j − rt−j−1 − μ̂)³P
Tq

Under RW2 or RW3, Lo and MacKinlay show that (3.25) has a limiting
stardard normal distribution.
Example 10 Testing the random walk hypothesis using variance ratios
The variance ratio statistics (3.23), (3.24) and (3.25) may be computed
using the S+FinMetrics function varRatioTest. The arguments for
varRatioTest are> args(varRatioTest)
function(x, n.periods, unbiased = T, hetero = F)
where x is the log return series (which may contain more than one series)
and n.periods denotes the number of periods q in the variance ratio. If
unbiased=T and hetero=F the bias corrected test statistic (3.24) is computed. If unbiased=T and hetero=T then the heteroskedasticity robust
statistic (3.25) is computed. The function varRatioTest returns an objectof class “varRatioTest” for which there are print and plot methods.
Consider testing the model RW3 for the daily log closing prices of the
Dow Jones Industrial Average over the period 1/1/1960 through 1/1/1990.
To compute the variance ratio (3.21) and the heteroskedasticity robust test
(3.25) for q = 1, . . . , 60 use> VR.djia = varRatioTest(djia[timeEvent("1/1/1960","1/1/1990"),+"close"], n.periods=60, unbiased=T, hetero=T)> class(VR.djia)[1] "varRatioTest"> names(VR.djia)[1] "varRatio" "std.err" "stat""hetero"> VR.djia
Variance Ratio Test

92

3. Time Series Concepts

Variance Ratio Profile

1.2
1.0
0.4

0.6

0.8

Variance Ratio

1.4

1.6

1.8

close

0

10

20

30

40

50

60

Periods

FIGURE 3.12. Variance ratios for the daily log prices of the Dow Jones Industrial
Average.

Null Hypothesis: random walk with heteroskedastic errors
Variable: close
var.ratio std.err
21.0403 0.06728
3
1.0183 0.10527...
60
1.0312 0.36227

stat
0.5994871
0.1738146
0.0861747* : significant at 5% level** : significant at 1% level
None of the variance ratios are statistically diﬀerent from unity at the 5%
level.
Figure 3.12 shows the results of the variance ratio tests based on plot
method> plot(VR.djia)
The variance ratios computed for diﬀerent values of q hover around unity,
and the ± 2 × standard error bands indicate that the model RW3 is not
rejected at the 5% level.

3.3 Univariate Nonstationary Time Series

93

Variance Ratio Profile

4

5

1.0

4

5

0.8
0.7
0.6
1.1
1.0

1.00

0.9
0.8

0.80

0.90
0.80

HON

1.1

1.1

1.2

HD

0.8

0.9

0.9

1.0

1.0

0.90

JPM

1.00

1.1

1.10

JNJ

GM

0.80

1.10

CAT

DD

1.00

1.00

C

0.80

0.80

0.8

0.90

0.90

0.70
1.0
0.9

0.95
0.85

3

XOM

MSFT

0.8

0.80
1.00

1.0
0.9
1.05 0.7

BA

2

0.9

1.1
0.9
0.8
1.10

MRK

0.9

0.90

0.8

GE

0.8
3

1

WMT

1.0

1.0

1.00

IP

0.75

0.8
0.7
2

5

0.90

0.90
0.80

0.80

INTC

0.7

0.7
1.15
1.05
0.95
1.0
0.9

1.0
0.9
0.8
1

4

1.0

1.1
0.9
0.8

1.00

1.00

MO

0.9

0.9
0.8

0.7
0.95
0.85
1.10.75

AXP

3

1.0

1.0
1.10

MMM

EK

AA

2

UTX

0.90

0.90
0.80
1.0

1.0
0.9
0.8

1

T

0.8

0.8
1.00

1.10

MCD

IBM

DIS

5

0.9

0.9

1.0
0.9
0.8
1.15
1.05
0.95
0.85

HWP

4

1.1

SBC

KO

1.05

Variance Ratio

3

1.0

1.1

PG

2

1.1

1

1

2

3

4

5

1

2

3

4

5

Periods

FIGURE 3.13. Variance ratio statistics for daily log prices on individual Dow
Jones index stocks.

The RW3 model appears to hold for the Dow Jones index. To test theRW3 model for the top thirty stocks in the index individually, based on
q = 1, . . . , 5, use> VR.DJ30 = varRatioTest(DowJones30, n.periods=5, unbiased=T,+
hetero=T)> plot(VR.DJ30)
The results, illustrated in Figure 3.13, indicate that the RW3 model may
not hold for some individual stocks.

3.3 Univariate Nonstationary Time Series
A univariate time series process {yt } is called nonstationary if it is notstationary. Since a stationary process has time invariant moments, a nonstationary process must have some time dependent moments. The mostcommon forms of nonstationarity are caused by time dependence in themean and variance.

94

3. Time Series Concepts

Trend Stationary Process{yt } is a trend stationary process if it has the form
yt = T Dt + xt
where T Dt are deterministic trend terms (constant, trend, seasonal dummies etc) that depend on t and {xt } is stationary. The series yt is nonstationary because E[T Dt ] = T Dt which depends on t. Since xt is stationary,yt never deviates too far away from the deterministic trend T Dt . Hence, yt
exhibits trend reversion. If T Dt were known, yt may be transformed to astationary process by subtracting oﬀ the deterministic trend terms:
xt = yt − T Dt
Example 11 Trend stationary AR(1)A trend stationary AR(1) process with T Dt = μ + δt may be expressedin three equivalent ways
yt = μ + δt + ut , ut = φut−1 + εtyt − μ − δt = φ(yt−1 − μ − δ(t − 1)) + εtyt = c + βt + φyt−1 + εtwhere |φ| < 1, c = μ(1 − φ) + δ, β = δ(1 − φ)t and εt ∼ W N (0, σ 2 ). Figure
3.14 shows T = 100 observations from a trend stationary AR(1) with μ = 1,
δ = 0.25, φ = 0.75 and σ 2 = 1 created with the S-PLUS commands> set.seed(101)> y.tsar1 = 1 + 0.25*seq(100) ++ arima.sim(model=list(ar=0.75),n=100)> tsplot(y.tsar1,ylab="y")> abline(a=1,b=0.25)
The simulated data show clear trend reversion.
Integrated Processes{yt } is an integrated process of order 1, denoted yt ∼ I(1), if it has the form
yt = yt−1 + ut

(3.26)

where ut is a stationary time series. Clearly, the first diﬀerence of yt isstationary∆yt = ut
Because of the above property, I(1) processes are sometimes called diﬀerence stationary processes. Starting at y0 , by recursive substitution yt has

95

5

10

y

15

20

3.3 Univariate Nonstationary Time Series

0

20

40

60

80

100

FIGURE 3.14. Simulated trend stationary process.

the representation of an integrated sum of stationary innovations
yt = y0 +

t
X

uj .

(3.27)

j=1

The integrated sum
T St . Notice thatPt

j=1 ujis called a stochastic trend and is denoted

T St = T St−1 + ut

where T S0 = 0. In contrast to a deterministic trend, changes in a stochastic
trend are not perfectly predictable.
Since the stationary process ut does not need to be diﬀerenced, it is called
an integrated process of order zero and is denoted ut ∼ I(0). Recall, fromthe Wold representation (3.6) a stationary process has an infinite ordermoving average representation where the moving average weights decline
to zero at a geometric rate. From (3.27) it is seen that an I(1) process has
an infinite order moving average representation where all of the weights on
the innovations are equal to 1.
If ut ∼ IW N (0, σ 2 ) in (3.26) then yt is called a random walk. In general,an I(1) process can have serially correlated and heteroskedastic innovations
ut . If yt is a random walk and assuming y0 is fixed then it can be shown96

3. Time Series Concepts

that
γ0
γj
ρj

= σ2 t= (t − j)σ 2
r
t−j
=
t

which clearly shows that yt is nonstationary. Also, if t is large relative toj then ρj ≈ 1. Hence, for an I(1) process, the ACF does not decay at ageometric rate but at a linear rate as j increases.
An I(1) process with drift has the form
yt = μ + yt−1 + ut , where ut ∼ I(0)Starting at t = 0 an I(1) process with drift μ may be expressed asyt= y0 + μt +

t
X

ut

j=1

= T Dt + T St

so that it may be thought of as being composed of a deterministic
linearPt
trend T Dt = y0 + μt as well as a stochastic trend T St = j=1 uj .
An I(d) process {yt } is one in which ∆d yt ∼ I(0). In finance and economics data series are rarely modeled as I(d) process with d > 2. Just asan I(1) process with drift contains a linear deterministic trend, an I(2)process with drift will contain a quadratic trend.
Example 12 Simulated I(1) processes
Consider the simulation of T = 100 observations from various I(1) processes where the innovations ut follow an AR(1) process ut = 0.75ut−1 + εtwith εt ∼ GW N (0, 1).
> set.seed(101)> u.ar1 = arima.sim(model=list(ar=0.75), n=100)> y1 = cumsum(u.ar1)> y1.d = 1 + 0.25*seq(100)+ y1> y2 = rep(0,100)> for (i in 3:100) {+y2[i] = 2*y2[i-1] - y2[i-2] + u.ar1[i]+ }The simulated data are illustrated in Figure 3.15 .
Example 13 Financial time series3.4 Long Memory Time Series
I(1) process

-2

0

0

10

2

20

30

4

I(0) innovations

97

0

20

40

60

80

100

0

20

60

80

100

80

100

I(2) process

0

0

10

500

20

30

1000

40

50

1500

I(1) process with drift

40

0

20

40

60

80

100

0

20

40

60

FIGURE 3.15. Simulated I(d) processes for d = 0, 1 and 2.

Many financial time series are well characterized by I(1) processes. The
leading example of an I(1) process with drift is the logarithm of an asset
price. Common examples of I(1) processes without drifts are the logarithms
of exchange rates, nominal interest rates, and inflation rates. Notice that
if inflation is constructed as the the diﬀerence in the logarithm of a priceindex and is an I(1) process, then the logarithm of the price index is anI(2) process. Examples of these data are illustrated in Figure 3.16. The
exchange rate is the monthly log of the US/CA spot exchange rate takenfrom the S+FinMetrics “timeSeries” lexrates.dat, the asset price ofthe monthly S&P 500 index taken from the S+FinMetrics “timeSeries”
object singleIndex.dat, the nominal interest rate is the 30 day T-bill ratetaken from the S+FinMetrics “timeSeries” object rf.30day, and themonthly consumer price index is taken from the S+FinMetrics “timeSeries”
object CPI.dat.

3.4 Long Memory Time Series
If a time series yt is I(0) then its ACF declines at a geometric rate. As aresult, I(0) process have short memory since observations far apart in timeare essentially independent. Conversely, if yt is I(1) then its ACF declines
at a linear rate and observations far apart in time are not independent. In

98

3. Time Series Concepts

Log S&P 500 index

5.8

-0.35

6.2

6.6

-0.15

7.0

0.00

Log US/CA spot exchange rate

1976

1980

1984

1988

1992

1996

1990

1992

1996

1998

2000

1975

1995

Log of US CPI

2.5

0.000

3.0

3.5

4.0

0.008

4.5

5.0

Nominal 30 day T-bill rate

1994

1930

1950

1970

1990

1915

1935

1955

FIGURE 3.16. Monthly financial time series.

between I(0) and I(1) processes are so-called fractionally integrated I(d)process where 0 < d < 1. The ACF for a fractionally integrated processes
declines at a polynomial (hyperbolic) rate, which implies that observations
far apart in time may exhibit weak but non-zero correlation. This weak correlation between observations far apart is often referred to as long memory.
A fractionally integrated white noise process yt has the form
(1 − L)d yt = εt , εt ∼ W N (0, σ 2 )(3.28)dwhere (1 − L) has the binomial series expansion representation (valid forany d > −1)
¶
∞ µ
X
d
d
(−L)k
(1 − L) =k
k=0d(d − 1) 2 d(d − 1)(d − 2) 3
L −L + ···
2!
3!
If d = 1 then yt is a random walk and if d = 0 then yt is white noise. For
0 < d < 1 it can be shown that
= 1 − dL +ρk ∝ k 2d−1
as k → ∞ so that the ACF for yt declines hyperbolically to zero at a speed
that depends on d. Further, it can be shown yt is stationary and ergodic
for 0 < d < 0.5 and that the variance of yt is infinite for 0.5 ≤ d < 1.

99

-2

0

2

4

3.4 Long Memory Time Series

0

100

200

300

400

500

40

50

ACF

0.0

0.2

0.4

0.6

0.8

1.0

Series : y.fwn

0

10

20

30
Lag

FIGURE 3.17. Simulated values from a fractional white noise process with d = 0.3
and σ = 1.

Example 14 Simulated fractional white noise
The S+FinMetrics function simulate.FARIMA may be used to generate
simulated values from a fractional white noise process. To simulate 500
observations from (3.28) with d = 0.3 and σ 2 = 1 use> set.seed(394)> y.fwn = simulate.FARIMA(list(d=0.3), 500)
Figure 3.17 shows the simulated data along with the sample ACF createdusing> par(mfrow=c(2,1))> tsplot(y.fwn)> tmp = acf(y.fwn,lag.max=50)
Notice how the sample ACF slowly decays to zero.
A fractionally integrated process with stationary and ergodic ARMA(p, q)
errors(1 − L)d yt = ut , ut ∼ ARMA(p, q)
is called an autoregressive fractionally integrated moving average (ARFIMA)
process. The modeling of long memory process is described in detail in
Chapter 8.

100

3. Time Series Concepts

0.6
0.4
0.0

0.2

ACF

0.8

1.0

Series : msft.aret

0

20

40

60

80

100

40

50

Lag

0.4
-0.2

0.2

ACF

0.6

0.8

1.0

Series : uscn.id

0

10

20

30
Lag

FIGURE 3.18. SACFs for the absolute value of daily returns on Microsoft andthe monthly 30-day interest rate diﬀerential between U.S. bonds and Canadianbonds.

Example 15 Long memory in financial time seriesLong memory behavior has been observed in certain types of financial
time series. Ding, Granger and Engle (1993) find evidence of long memoryin the absolute value of daily stock returns. Baillie and Bollerslev (1994)
find evidence for long memory in the monthly interest rate diﬀerentials
between short term U.S. government bonds and short term foreign government bonds. To illustrate, consider the absolute values of the daily returns
on Microsoft over the 10 year period 1/2/1991 - 1/2/2001 taken from the
S+FinMetrics “timeSeries” DowJones30> msft.aret = abs(getReturns(DowJones30[,"MSFT"]))
Consider also the monthly US/CA 30-day interest rate diﬀerential over theperiod February 1976 through June 1996 in the “timeSeries” uscn.id
constructed earlier and taken from the S+FinMetrics “timeSeries” object
lexrates.dat. Figure 3.18 shows the SACFs these series create by> par(mfrow=c(2,1))> tmp = acf(msft.aret, lag.max=100)> tmp = acf(uscn.id, lag.max=50)
For the absolute return series, notice the large number of small but apparently significant autocorrelations at very long lags. This is indicative of

3.5 Multivariate Time Series

101long memory. For the interest rate diﬀerential series, the ACF appears to
decay fairly quickly, so the evidence for long memory is not as strong.

3.5 Multivariate Time Series
Consider n time series variables {y1t }, . . . , {ynt }. A multivariate time series is the (n × 1) vector time series {Yt } where the ith row of {Yt } is{yit }. That is, for any time t, Yt = (y1t , . . . , ynt )0 . Multivariate time series
analysis is used when one wants to model and explain the interactions andco-movements among a group of time series variables. In finance, multivariate time series analysis is used to model systems of asset returns, asset
prices and exchange rates, the term structure of interest rates, asset returns/prices, and economic variables etc. Many of the time series concepts
described previously for univariate time series carry over to multivariate
time series in a natural way. Additionally, there are some important timeseries concepts that are particular to multivariate time series. The following sections give the details of these extensions and provide examples using
S-PLUS and S+FinMetrics.

3.5.1 Stationary and Ergodic Multivariate Time Series
A multivariate time series Yt is covariance stationary and ergodic if all of
its component time series are stationary and ergodic. The mean of Yt is
defined as the (n × 1) vector
E[Yt ] = μ = (μ1 , . . . , μn )0
where μi = E[yit ] for i = 1, . . . , n. The variance/covariance matrix of Yt
is the (n × n) matrix
var(Yt ) = Γ0 = E[(Yt − μ)(Yt − μ)0 ]⎛var(y1t )cov(y1t , y2t )⎜ cov(y2t , y1t )var(y2t )
⎜
= ⎜
..
..
⎝
.
.

···
···
..
.

cov(ynt , y1t ) cov(ynt , y2t ) · · ·⎞cov(y1t , ynt )cov(y2t , ynt ) ⎟
⎟
⎟
..
⎠
.
var(ynt )The matrix Γ0 has elements γ 0ij = cov(yit , yjt ). The correlation matrix of
Yt is the (n × n) matrix
corr(Yt ) = R0 = D−1 Γ0 D−1
where D is an (n×n) diagonal matrix with j th diagonal element (γ 0jj )1/2 =SD(yjt ). The parameters μ, Γ0 and R0 are estimated from data (Y1 , . . . , YT )

102

3. Time Series Concepts

using the sample moments
T

1X
Yt
T t=1

Ȳ

=

Γ̂0

=

R̂0

= D̂−1 Γ̂0 D̂−1

T

1X
(Yt − Ȳ)(Yt − Ȳ)0
T t=1

where D is the (n×n) diagonal matrix with the sample standard deviationsof yjt along the diagonal. In order for the sample variance matrix Γ̂0 andcorrelation matrix R̂0 to be positive definite, the sample size T must be
greater than the number of component time series n.
Example 16 System of asset returns
The S+FinMetrics “timeSeries” object DowJones30 contains daily closing prices on the 30 assets in the Dow Jones index. An example of a stationary and ergodic multivariate time series is the continuously compounded
returns on the first four assets in this index:> Y = getReturns(DowJones30[,1:4],type="continuous")> colIds(Y)[1] "AA" "AXP" "T""BA"
The S-PLUS function colMeans may be used to eﬃciently compute themean vector of Y
> colMeans(Y)
AA
AXP
T
BA
0.0006661 0.0009478 -0.00002873 0.0004108
In S-PLUS 7, the function colMeans has a method for “timeSeries” objects so the extractor function seriesData is not needed to extract thedata slot of the variable Y. The S-PLUS functions var and cor, which also
have methods for “timeSeries” objects, may be used to compute Γ̂0 andR̂0> var(Y)
AA
AXP
T
BA
AA 0.00041096 0.00009260 0.00005040 0.00007301
AXP 0.00009260 0.00044336 0.00008947 0.00009546
T 0.00005040 0.00008947 0.00040441 0.00004548
BA 0.00007301 0.00009546 0.00004548 0.00036829> cor(Y)
AA
AXP
T
BA
AA 1.0000 0.2169 0.1236 0.1877
AXP 0.2169 1.0000 0.2113 0.2362

3.5 Multivariate Time Series

103

T 0.1236 0.2113 1.0000 0.1179
BA 0.1877 0.2362 0.1179 1.0000
If only the variances or standard deviations of Yt are needed the S-PLUS
functions colVars and colStdevs may be used
> colVars(seriesData(Y))
AA
AXP
T
BA
0.000411 0.0004434 0.0004044 0.0003683> colStdevs(seriesData(Y))
AA
AXP
T
BA
0.020272 0.021056 0.02011 0.019191

Cross Covariance and Correlation Matrices
For a univariate time series yt the autocovariances γ k and autocorrelations
ρk summarize the linear time dependence in the data. With a multivariate
time series Yt each component has autocovariances and autocorrelations
but there are also cross lead-lag covariances and correlations between all
possible pairs of components. The autocovariances and autocorrelations ofyjt for j = 1, . . . , n are defined as
γ kjj= cov(yjt , yjt−k ),ρkjj= corr(yjt , yjt−k ) =

γ kjj
γ 0jj

−k
k
and these are symmetric in k: γ kjj = γ −kjj , ρjj = ρjj . The cross lag covariances and cross lag correlations between yit and yjt are defined as

γ kij
ρkij

= cov(yit , yjt−k ),γ kij= corr(yjt , yjt−k ) = q
γ 0ii γ 0jj

and they are not necessarily symmetric in k. In general,γ kij = cov(yit , yjt−k ) 6= cov(yit , yjt+k ) = cov(yjt , yit−k ) = γ −kij
If γ kij 6= 0 for some k > 0 then yjt is said to lead yit . Similarly, if γ −kij 6= 0for some k > 0 then yit is said to lead yjt . It is possible that yit leads yjt
and vice-versa. In this case, there is said to be feedback between the twoseries.

104

3. Time Series Concepts

All of the lag k cross covariances and correlations are summarized in the(n × n) lag k cross covariance and lag k cross correlation matrices
ΓkRk= E[(Yt − μ)(Yt−k − μ)0 ]⎛cov(y1t , y1t−k ) cov(y1t , y2t−k )⎜ cov(y2t , y1t−k ) cov(y2t , y2t−k )
⎜
= ⎜
..
..
⎝
.
.

cov(ynt , y1t−k ) cov(ynt , y2t−k ) · · ·

−1

= D

···
···
..
.

−1

Γk D

⎞
cov(y1t , ynt−k )cov(y2t , ynt−k ) ⎟
⎟
⎟
..
⎠
.

cov(ynt , ynt−k )The matrices Γk and Rk are not symmetric in k but it is easy to show thatΓ−k = Γ0k and R−k = R0k . The matrices Γk and Rk are estimated fromdata (Y1 , . . . , YT ) using
T
1 X(Yt − Ȳ)(Yt−k − Ȳ)0
T

Γ̂k

=

R̂k

= D̂−1 Γ̂k D̂−1

t=k+1

Example 17 Lead-lag covariances and correlations among asset returns
Consider computing the cross lag covariances and correlations for k =0, . . . , 5 between the first two Dow Jones 30 asset returns in the “timeSeries”
Y. These covariances and correlations may be computed using the S-PLUS
function acf> Ghat = acf(Y[,1:2],lag.max=5,type="covariance",plot=F)> Rhat = acf(Y[,1:2],lag.max=5,plot=F)
Ghat and Rhat are objects of class “acf” for which there is only a print
method. For example, the estimated cross lag autocorrelations are> RhatCall: acf(x = Y[, 1:2], lag.max = 5, plot = F)
Autocorrelation matrix:
lag
AA.AA AA.AXP AXP.AXP
1
0 1.0000 0.2169 1.0000
2
1 0.0182 0.0604 -0.0101
32 -0.0556 -0.0080 -0.0710
4
3 0.0145 -0.0203 -0.0152
54 -0.0639 0.0090 -0.0235
6
5 0.0142 -0.0056 -0.0169

1

lag
0

AXP.AA
0.2169

3.5 Multivariate Time Series

105

Multivariate Series : Y[, 1:2]
AA and AXP

0.0

0.0

0.2

0.05

ACF
0.4

0.10

0.6

0.15

0.8

0.20

1.0

AA

0

1

2

3

4

5

0

1

2

4

5

3

4

5

AXP

0.0

0.0

0.2

0.05

0.4

ACF
0.10

0.6

0.15

0.8

0.20

1.0

AXP and AA

3

-5

-4

-3

Lag

-2

-1

0

0

1

2

Lag

FIGURE 3.19. Cross lag correlations between the first two Dow Jones 30 asset
returns.

2
3
4
5
6

-1 -0.0015-2 -0.0187-3 -0.0087-4 -0.0233-5 0.0003

The function acf.plot may be used to plot the cross lag covariancesand correlations produced by acf.
> acf.plot(Rhat)
Figure 3.19 shows these cross lag correlations. The matrices Γ̂k and R̂kmay be extracted from acf component of Ghat and Rhat, respectively. For
example,> Ghat$acf[1,,]
[,1]
[,2]
[1,] 0.00041079 0.00009256[2,] 0.00009256 0.00044318> Rhat$acf[1,,]
[,1]
[,2]
[1,] 1.0000 0.2169[2,] 0.2169 1.0000> Ghat$acf[2,,]106

3. Time Series Concepts

[,1]
[,2]
[1,] 7.488e-006 2.578e-005[2,] -6.537e-007 -4.486e-006> Rhat$acf[2,,]
[,1]
[,2]
[1,] 0.018229 0.06043[2,] -0.001532 -0.01012
extracts Γ̂1 , R̂1 , Γ̂2 and R̂2 .

3.5.2 Multivariate Wold Representation
Any (n × 1) covariance stationary multivariate time series Yt has a Woldor linear process representation of the formYt

= μ + εt + Ψ1 εt−1 + Ψ2 εt−2 + · · ·∞X= μ+Ψk εt−k(3.29)

k=0

where Ψ0 = In and εt is a multivariate white noise process with mean zeroand variance matrix E[εt ε0t ] = Σ. In (3.29), Ψk is an (n × n) matrix with(i, j)th element ψ kij . In lag operator notation, the Wold form is= μ + Ψ(L)εt∞X
Ψ(L) =
Ψk Lk
Yt

k=0

The moments of Yt are given by
E[Yt ] = μ∞Xvar(Yt ) =Ψk ΣΨ0k
k=0

VAR Models
The most popular multivariate time series model is the vector autoregressive(VAR) model. The VAR model is a multivariate extension of the univariate
autoregressive model. For example, a bivariate VAR(1) model has the form
µ
¶ µ
¶ µ 1
¶µ
¶ µ
¶
y1t
c1
π 11 π 112
y1t−1
ε1t
=
+
+
y2t
c2
π 121 π 122
y2t−1
ε2t
or
y1t
y2t

= c1 + π 111 y1t−1 + π112 y2t−1 + ε1t= c2 + π 121 y1t−1 + π122 y2t−1 + ε2t

3.5 Multivariate Time Series

where

µ

ε1t
ε2t

¶

∼ iid

µµ

0
0

¶ µ
σ 11
,
σ 12

σ12
σ22

107

¶¶

In the equations for y1 and y2 , the lagged values of both y1 and y2 arepresent.
The general VAR(p) model for Yt = (y1t , y2t , . . . , ynt )0 has the form
Yt = c + Π1 Yt−1 + Π2 Yt−2 + · · · + Πp Yt−p + εt , t = 1, . . . , T
where Πi are (n × n) coeﬃcient matrices and εt is an (n × 1) unobservablezero mean white noise vector process with covariance matrix Σ. VAR models are capable of capturing much of the complicated dynamics observedin stationary multivariate time series. Details about estimation, inference,and forecasting with VAR models are given in chapter eleven.

3.5.3 Long Run Variance
Let Yt be an (n × 1) stationary and ergodic multivariate time series with
E[Yt ] = μ. Anderson’s central limit theorem for stationary and ergodic
process states
⎞
⎛
∞
X
√
d
T (Ȳ − μ) → N ⎝0,
Γj ⎠
j=−∞

or

⎞
∞
X
1
Ȳ ∼ N ⎝μ,Γj ⎠T j=−∞A⎛

Hence, the long-run variance of Yt is T times the asymptotic variance ofȲ:∞Xlrv(Yt ) = T · avar(Ȳ) =Γjj=−∞

Since Γ−j = Γ0j , lrv(Yt ) may be alternatively expressed aslrv(Yt ) = Γ0 +∞X
(Γj +Γ0j )j=1

Using the Wold representation of Yt it can be shown thatlrv(Yt ) = Ψ(1)ΣΨ(1)where Ψ(1) =P∞

k=0 Ψk .

0

108

3. Time Series Concepts

VAR Estimate of the Long-Run Variance
The Wold representation (3.29) may be approximated by high order VAR(p∗ )model
Yt = c + Φ1 Yt−1 + · · · + Φp∗ Yt−p∗ + εtwhere the lag length p∗ is chosen such p∗ = O(T 1/3 ). This gives rise to theautoregressive long-run variance matrix estimatec AR (Yt ) = Ψ̂(1)Σ̂Ψ̂(1)0
lrv
Ψ̂(1) = (In − Φ̂1 − · · · − Φ̂p )−1
T

Σ̂ =

1X
ε̂t ε̂0t
T t=1

where Φ̂k (k = 1, . . . , p∗ ) are estimates of the VAR parameter matrices.
Non-parametric Estimate of the Long-Run Variance
A consistent estimate of lrv(Yt ) may be computed using non-parametricmethods. A popular estimator is the Newey-West weighted autocovarianceestimator
MT³´
Xc NW (Yt ) = Γ̂0 +lrv
wj,T · Γ̂j + Γ̂0j(3.30)
j=1

where wj,T are weights which sum to unity and MT is a truncation lag
parameter that satisfies MT = O(T 1/3 ).

Example 18 Newey-West estimate of long-run variance matrix for stock
returns
The S+FinMetrics function asymp.var may be used to compute the
Newey-West long-run variance estimate (3.30) for a multivariate time series.
The long-run variance matrix for the first four Dow Jones assets in the“timeSeries” Y is
> M.T = floor(4*(nrow(Y)/100)^(2/9))> lrv.nw = asymp.var(Y,bandwidth=M.T)> lrv.nw
AA
AXP
T
BA
AA 0.00037313 0.00008526 3.754e-005 6.685e-005
AXP 0.00008526 0.00034957 7.937e-005 1.051e-004
T 0.00003754 0.00007937 3.707e-004 7.415e-006
BA 0.00006685 0.00010506 7.415e-006 3.087e-004

3.6 References

109

3.6 References
Alexander, C. (2001). Market Models. A Guide to Financial Data Analysis. John Wiley & Sons, Chichester, UK.
Baille, R.T. and T. Bollerslev (1994). “The Long Memory of the
Forward Premium,” Journal of International Money and Finance, 13, 555571.
Box, G.E.P. and G.M. Jenkins (1976). Time Series Analysis, Forecasting and Control. Revised Edition. Holden Day, San Francisco.
Campbell, J.Y., A.W. Lo, A.C. MacKinlay (1997). The Econometrics
of Financial Markets. Princeton University Press, Princeton, NJ.
Box, G.E.P., and D.A. Pierce (1970). “Distribution of Residual Autocorrelations in Autoregressive-integrated Moving Average Time Series
Models,” Journal of the American Statistical Association, 65, 1509-1526.
Chan, N.H. (2002). Time Series: Applicatios to Finance. John Wiley &Sons, New York.
Ding, Z., C.W.J. Granger and R.F. Engle (1993). “A Long Memory Property of Stock Returns and a New Model,” Journal of Empirical
Finance, 1, 83-106.
Engle, R.F. (1982). “Autoregressive Conditional Heteroskedasticity with
Estimates of the Variance of United Kingdom Inflations,” Econometrica,
50, 987-1097.
Fuller, W.A. (1996). Introduction to Statistical Time Series, Second Edition. John Wiley & Sons, New York.
Gourieroux, C and J. Jasiak (2001). Financial Econometrics. Princeton University Press, Princeton, NJ.
Granger, C.W.J. and M.J. Morris (1976). “Time Series Modeling andInterpretation,” Journal of the Royal Statistical Society, Series A, 139, 246257.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
Harvey, A.C. (1993). Time Series Models, Second Edition. MIT Press,
Cambridge, MA.

110

3. Time Series Concepts

Hayashi, F. (2000). Econometrics. Princeton University Press, Princeton,
NJ.
Ljung, T. and G.E.P. Box (1979). “The Likelihood Function for a Stationary Autoregressive Moving Average Process,” Biometrika, 66, 265-270.
Mills, T.C. (1999). The Econometric Modeling of Financial Time Series,
Second Edition. Cambridge University Press, Cambridge, UK.
Newey, W.K. and K.D. West (1987). “A Simple Positive Semidefinite Heteroskedasticity and Autocorrelation Consistent Covariance Matrix,” Econometrica, 55, 703-708.
Tsay, R.S. (2001). Analysis of Financial Time Series, John Wiley & Sons,
New York.
Venables, W.N. and B.D. Ripley (2002). Modern Applied Statistics
with S-PLUS, Fourth Edition. Springer-Verlag, New York.
White, H. (1984). Asymptotic Theory for Econometrians. Academic Press,
San Diego.

This is page 111
Printer: Opaque this4
Unit Root Tests

4.1 Introduction
Many economic and financial time series exhibit trending behavior or nonstationarity in the mean. Leading examples are asset prices, exchange rates
and the levels of macroeconomic aggregates like real GDP. An importanteconometric task is determining the most appropriate form of the trend in
the data. For example, in ARMA modeling the data must be transformed
to stationary form prior to analysis. If the data are trending, then someform of trend removal is required.
Two common trend removal or de-trending procedures are first diﬀerencing and time-trend regression. First diﬀerencing is appropriate for I(1)time series and time-trend regression is appropriate for trend stationary
I(0) time series. Unit root tests can be used to determine if trending data
should be first diﬀerenced or regressed on deterministic functions of time
to render the data stationary. Moreover, economic and finance theory often
suggests the existence of long-run equilibrium relationships among nonstationary time series variables. If these variables are I(1), then cointegration
techniques can be used to model these long-run relations. Hence, pre-testingfor unit roots is often a first step in the cointegration modeling discussed
in Chapter 12. Finally, a common trading strategy in finance involves exploiting mean-reverting behavior among the prices of pairs of assets. Unit
root tests can be used to determine which pairs of assets appear to exhibit
mean-reverting behavior.

112

4. Unit Root Tests

This chapter is organized as follows. Section 4.2 reviews I(1) and trend
stationary I(0) time series and motivates the unit root and stationary
tests described in the chapter. Section 4.3 describes the class of autoregressive unit root tests made popular by David Dickey, Wayne Fuller, Pierre
Perron and Peter Phillips. Section 4.4 describes the stationarity tests ofKwiatkowski, Phillips, Schmidt and Shinn (1992). Section 4.5 discusses
some problems associated with traditional unit root and stationarity tests,
and Section 4.6 presents some recently developed so-called “eﬃcient unit
root tests” that overcome some of the deficiencies of traditional unit root
tests.
In this chapter, the technical details of unit root and stationarity tests are
kept to a minimum. Excellent technical treatments of nonstationary timeseries may be found in Hamilton (1994), Hatanaka (1995), Fuller (1996)
and the many papers by Peter Phillips. Useful surveys on issues associatedwith unit root testing are given in Stock (1994), Maddala and Kim (1998)
and Phillips and Xiao (1998).

4.2 Testing for Nonstationarity and Stationarity
To understand the econometric issues associated with unit root and stationarity tests, consider the stylized trend-cycle decomposition of a timeseries yt :
yt
T Dt
zt

= T Dt + zt
= κ + δt
= φzt−1 + εt , εt ∼ W N (0, σ 2 )where T Dt is a deterministic linear trend and zt is an AR(1) process. If|φ| < 1 then yt is I(0)Pabout the deterministic trend T Dt . If φ = 1, thenzt = zt−1 + εt = z0 + tj=1 εj , a stochastic trend and yt is I(1) with drift.
Simulated I(1) and I(0) data with κ = 5 and δ = 0.1 are illustrated in
Figure 4.1. The I(0) data with trend follows the trend T Dt = 5 + 0.1t very
closely and exhibits trend reversion. In contrast, the I(1) data follows an
upward drift but does not necessarily revert to T Dt .
Autoregressive unit root tests are based on testing the null hypothesisthat φ = 1 (diﬀerence stationary) against the alternative hypothesis thatφ < 1 (trend stationary). They are called unit root tests because under thenull hypothesis the autoregressive polynomial of zt , φ(z) = (1 − φz) = 0,
has a root equal to unity.
Stationarity tests take the null hypothesis that yt is trend stationary. If
yt is then first diﬀerenced it becomes∆yt∆zt= δ + ∆zt
= φ∆zt−1 + εt − εt−1

113

30

4.2 Testing for Nonstationarity and Stationarity

0

5

10

15

20

25

I(1)
I(0)

0

50

100

150

200

250

FIGURE 4.1. Simulated trend stationary (I(0)) and diﬀerence stationary (I(1))
processes.

Notice that first diﬀerencing yt , when it is trend stationary, produces aunit moving average root in the ARMA representation of ∆zt . That is, the
ARMA representation for ∆zt is the non-invertible ARMA(1,1) model∆zt = φ∆zt−1 + εt + θεt−1

with θ = −1. This result is known as overdiﬀerencing. Formally, stationarity tests are based on testing for a unit moving average root in ∆zt .
Unit root and stationarity test statistics have nonstandard and nonnormal asymptotic distributions under their respective null hypotheses. To
complicate matters further, the limiting distributions of the test statistics
are aﬀected by the inclusion of deterministic terms in the test regressions.
These distributions are functions of standard Brownian motion (Wiener
process), and critical values must be tabulated by simulation techniques.
MacKinnon (1996) provided response surface algorithms for determining
these critical values, and various S+FinMetrics functions use these algorithms for computing critical values and p-values.

114

4. Unit Root Tests

4.3 Autoregressive Unit Root Tests
To illustrate the important statistical issues associated with autoregressive
unit root tests, consider the simple AR(1) modelyt = φyt−1 + εt , where εt ∼ W N (0, σ2 )The hypotheses of interest are
H0
H1: φ = 1 (unit root in φ(z) = 0) ⇒ yt ∼ I(1): |φ| < 1 ⇒ yt ∼ I(0)The test statistic istφ=1 =φ̂ − 1SE(φ̂)

where φ̂ is the least squares estimate and SE(φ̂) is the usual standard error
estimate1 . The test is a one-sided left tail test. If {yt } is stationary (i.e.,|φ| < 1) then it can be shown (c.f. Hamilton 1994, p. 216)√d
T (φ̂ − φ) → N (0, (1 − φ2 ))orAφ̂ ∼ Nµ1φ, (1 − φ2 )T¶Aand it follows that tφ=1 ∼ N (0, 1). However, under the null hypothesis of
nonstationarity the above result givesAφ̂ ∼ N (1, 0)
which clearly does not make any sense. The problem is that under the unit
root null, {yt } is not stationary and ergodic, and the usual sample moments
do not converge to fixed constants. Instead, Phillips (1987) showed thatthe sample moments of {yt } converge to random functions of Brownian
1 The AR(1) model may be re-written as ∆y = πytt−1 + ut where π = φ − 1. Testing
φ = 1 is then equivalent to testing π = 0. Unit root tests are often computed using thisalternative regression and the S+FinMetrics function unitroot follows this convention.

4.3 Autoregressive Unit Root Tests

115

motion2 :
T

−3/2

T
X
t=1

T −2

T
X
t=1

T −1

T
X
t=1

d

yt−1 → σ

Z 1

Z 1

d

2
yt−1
→ σ2

W (r)dr0

W (r)2 dr0dyt−1 εt → σ 2

Z 1

W (r)dW (r)0where W (r) denotes a standard Brownian motion (Wiener process) definedon the unit interval. Using the above results Phillips showed that under theunit root null H0 : φ = 1
dT (φ̂ − 1) →
d

R1

R1

W (r)dW (r)R1
W (r)2 dr00

W (r)dW (r)
´1/2
2 dr
W
(r)
0

tφ=1 → ³ 0
R1

(4.1)
(4.2)

The above yield some surprising results:p• φ̂ is super-consistent; that is, φ̂ → φ at rate T instead of the usual
rate T 1/2 .
• φ̂ is not asymptotically normally distributed and tφ=1 is not asymptotically standard normal.
• The limiting distribution of tφ=1 is called the Dickey-Fuller (DF)distribution and does not have a closed form representation. Consequently, quantiles of the distribution must be computed by numericalapproximation or by simulation3 .
• Since the normalized bias T (φ̂ − 1) has a well defined limiting distribution that does not depend on nuisance parameters it can also be
used as a test statistic for the null hypothesis H0 : φ = 1.
2 A Wiener process W (·) is a continuous-time stochastic process, associating each dater ∈ [0, 1] a scalar random variable W (r) that satisfies: (1) W (0) = 0; (2) for any dates
0 ≤ t1 ≤ · · · ≤ tk ≤ 1 the changes W (t2 ) − W (t1 ), W (t3 ) − W (t2 ), . . . , W (tk ) − W (tk−1 )are independent normal with W (s) − W (t) ∼ N(0, (s − t)); (3) W (s) is continuous in s.
3 Dickey and Fuller (1979) first considered the unit root tests and derived the asymptotic distribution of tφ=1 . However, their representation did not utilize functions ofWiener processes.

116

4. Unit Root Tests

4.3.1 Simulating the DF and Normalized Bias Distributions
As mentioned above, the DF and normalized bias distributions must be obtained by simulation methods. To illustrate, the following S-PLUS function
wiener produces one random draw from the functions of Brownian motionthat appear in the limiting distributions of tφ=1 and T (φ̂ − 1):wiener = function(nobs) {e = rnorm(nobs)
y = cumsum(e)ym1 = y[1:(nobs-1)]
intW2 = nobs^(-2) * sum(ym1^2)
intWdW = nobs^(-1) * sum(ym1*e[2:nobs])
ans = list(intW2=intW2,
intWdW=intWdW)
ans}
A simple loop then produces the simulated distributions:> nobs = 1000
> nsim = 1000
> NB = rep(0,nsim)> DF = rep(0,nsim)> for (i in 1:nsim) {+BN.moments = wiener(nobs)+
NB[i] = BN.moments$intWdW/BN.moments$intW2+
DF[i] = BN.moments$intWdW/sqrt(BN.moments$intW2)}
Figure 4.2 shows the histograms and density estimates of the simulated
distributions. The DF density is slightly left-skewed relative to the standard
normal, and the normalized bias density is highly left skewed and nonnormal. Since the alternative is one-sided, the test statistics reject if they
are suﬃciently negative. For the DF and normalized bias densities theempirical 1%, 5% and 10% quantiles are> quantile(DF,probs=c(0.01,0.05,0.1))
1%
5%
10%
-2.451 -1.992 -1.603> quantile(NB,probs=c(0.01,0.05,0.1))
1%
5%
10%
-11.94 -8.56 -5.641
For comparison purposes, note that the 5% quantile from the standard
normal distribution is -1.645.
The simulation of critical values and p-values from (4.1) and (4.2) isstraightforward but time consuming. The punitroot and qunitroot func-

4.3 Autoregressive Unit Root Tests
Simulated normalized bias

0

0

50

100

100

200

150

300

200

Simulated DF distribution

117

-3

-2

-1

0
DF

1

2

3

-25

-20

-15

-10

-5

0

5

NB

FIGURE 4.2. Histograms of simulated DF and normalized bias distributions.

tions in S+FinMetrics produce p-values and quantiles of the DF and normalized bias distributions based on MacKinnon’s (1996) response surface
methodology. The advantage of the response surface methodology is that
accurate p-values and quantiles appropriate for a given sample size can beproduced. For example, the 1%, 5% and 10% quantiles for (4.2) and (4.1)
based on a sample size of 100 are> qunitroot(c(0.01,0.05,0.10), trend="nc", statistic="t",+ n.sample=100)[1] -2.588 -1.944 -1.615> qunitroot(c(0.01,0.05,0.10), trend="nc", statistic="n",+ n.sample=100)[1] -13.086 -7.787 -5.565
The argument trend="nc" specifies that no constant is included in thetest regression. Other valid options are trend="c" for constant only andtrend="ct" for constant and trend. These trend cases are explained below. To specify the normalized bias distribution, set statistic="n". For
asymptotic quantiles set n.sample=0.
Similarly, the p-value of -1.645 based on the DF distribution for a sample
size of 100 is computed as> punitroot(-1.645, trend="nc", statistic="t")[1] 0.0945

118

4. Unit Root Tests
Case I: I(0) data

-5

2

4

0

6

5

8

10

10

Case I: I(1) data

0

50

100

150

200

250

0

50

150

200

250

200

250

Case II: I(0) data

0

5

5

10

10

15

15

20

20

25

25

30

Case II: I(1) data

100

0

50

100

150

200

250

0

50

100

150

FIGURE 4.3. Simulated I(1) and I(0) data under trend cases I and II.

4.3.2 Trend Cases
When testing for unit roots, it is crucial to specify the null and alternative
hypotheses appropriately to characterize the trend properties of the dataat hand. For example, if the observed data does not exhibit an increasingor decreasing trend, then the appropriate null and alternative hypotheses
should reflect this. The trend properties of the data under the alternative
hypothesis will determine the form of the test regression used. Furthermore, the type of deterministic terms in the test regression will influence
the asymptotic distributions of the unit root test statistics. The two mostcommon trend cases are summarized below and illustrated in Figure 4.3.
Case I: Constant Only
The test regression isyt = c + φyt−1 + εt
and includes a constant to capture the nonzero mean under the alternative.
The hypotheses to be tested are
H0
H1: φ = 1 ⇒ yt ∼ I(1) without drift: |φ| < 1 ⇒ yt ∼ I(0) with nonzero mean

This formulation is appropriate for non-trending financial series like interest
rates, exchange rates, and spreads. The test statistics tφ=1 and T (φ̂ − 1)

4.3 Autoregressive Unit Root Tests

119

are computed from the above regression. Under H0 : φ = 1 the asymptotic
distributions of these test statistics are diﬀerent from (4.2) and (4.1) andare influenced by the presence but not the coeﬃcient value of the constantin the test regression. Quantiles and p-values for these distributions can becomputed using the S+FinMetrics functions punitroot and qunitrootwith the trend="c" option:> qunitroot(c(0.01,0.05,0.10), trend="c", statistic="t",+ n.sample=100)[1] -3.497 -2.891 -2.582> qunitroot(c(0.01,0.05,0.10), trend="c", statistic="n",+ n.sample=100)[1] -19.49 -13.53 -10.88> punitroot(-1.645, trend="c", statistic="t", n.sample=100)[1] 0.456> punitroot(-1.645, trend="c", statistic="n", n.sample=100)[1] 0.8172
For a sample size of 100, the 5% left tail critical values for tφ=1 andT (φ̂ − 1) are -2.891 and -13.53, respectively, and are quite a bit smaller
than the 5% critical values computed when trend="nc". Hence, inclusionof a constant pushes the distributions of tφ=1 and T (φ̂ − 1) to the left.
Case II: Constant and Time Trend
The test regression isyt = c + δt + φyt−1 + εt
and includes a constant and deterministic time trend to capture the deterministic trend under the alternative. The hypotheses to be tested are
H0
H1: φ = 1 ⇒ yt ∼ I(1) with drift: |φ| < 1 ⇒ yt ∼ I(0) with deterministic time trendThis formulation is appropriate for trending time series like asset prices or
the levels of macroeconomic aggregates like real GDP. The test statisticstφ=1 and T (φ̂ − 1) are computed from the above regression. Under H0 :φ = 1 the asymptotic distributions of these test statistics are diﬀerent from
(4.2) and (4.1) and are influenced by the presence but not the coeﬃcientvalues of the constant and time trend in the test regression. Quantiles andp-values for these distributions can be computed using the S+FinMetrics
functions punitroot and qunitroot with the trend="ct" option:> qunitroot(c(0.01,0.05,0.10), trend="ct", statistic="t",+ n.sample=100)[1] -4.052 -3.455 -3.153

120

4. Unit Root Tests> qunitroot(c(0.01,0.05,0.10), trend="ct", statistic="n",+ n.sample=100)[1] -27.17 -20.47 -17.35> punitroot(-1.645, trend="ct", statistic="t", n.sample=100)[1] 0.7679> punitroot(-1.645, trend="ct", statistic="n", n.sample=100)[1] 0.9769
Notice that the inclusion of a constant and trend in the test regression
further shifts the distributions of tφ=1 and T (φ̂−1) to the left. For a sample
size of 100, the 5% left tail critical values for tφ=1 and T (φ̂ − 1) are now-3.455 and -20.471.

4.3.3 Dickey-Fuller Unit Root Tests
The unit root tests described above are valid if the time series yt is wellcharacterized by an AR(1) with white noise errors. Many financial timeseries, however, have a more complicated dynamic structure than is captured by a simple AR(1) model. Said and Dickey (1984) augment the basic
autoregressive unit root test to accommodate general ARMA(p, q) models
with unknown orders and their test is referred to as the augmented DickeyFuller (ADF) test. The ADF test tests the null hypothesis that a timeseries yt is I(1) against the alternative that it is I(0), assuming that the
dynamics in the data have an ARMA structure. The ADF test is based on
estimating the test regression
yt = β0 Dt + φyt−1 +pXψ j ∆yt−j + εt

(4.3)

j=1

where Dt is a vector of deterministic terms (constant, trend etc.). The p
lagged diﬀerence terms, ∆yt−j , are used to approximate the ARMA structure of the errors, and the value of p is set so that the error εt is serially
uncorrelated. The error term is also assumed to be homoskedastic. The
specification of the deterministic terms depends on the assumed behaviorof yt under the alternative hypothesis of trend stationarity as described in
the previous section. Under the null hypothesis, yt is I(1) which implies
that φ = 1. The ADF t-statistic and normalized bias statistic are based on
the least squares estimates of (4.3) and are given by
ADFt= tφ=1 =ADFn=

φ̂ − 1SE(φ)

T (φ̂ − 1)

1 − ψ̂ 1 − · · · − ψ̂ p

4.3 Autoregressive Unit Root Tests

121

An alternative formulation of the ADF test regression is0∆yt = β Dt + πyt−1 +pXψ j ∆yt−j + εt

(4.4)

j=1

where π = φ − 1. Under the null hypothesis, ∆yt is I(0) which implies thatπ = 0. The ADF t-statistic is then the usual t-statistic for testing π = 0
and the ADF normalized bias statistic is T π̂/(1 − ψ̂ 1 − · · · − ψ̂ p ). The test
regression (4.4) is often used in practice because the ADF t-statistic is theusual t-statistic reported for testing the significance of the coeﬃcient yt−1 .
The S+FinMetrics function unitroot follows this convention.
Choosing the Lag Length for the ADF Test
An important practical issue for the implementation of the ADF test is the
specification of the lag length p. If p is too small then the remaining serial
correlation in the errors will bias the test. If p is too large then the powerof the test will suﬀer. Ng and Perron (1995) suggest the following data
dependent lag length selection procedure that results in stable size of thetest and minimal power loss. First, set an upper bound pmax for p. Next,
estimate the ADF test regression with p = pmax . If the absolute value of thet-statistic for testing the significance of the last lagged diﬀerence is greaterthan 1.6 then set p = pmax and perform the unit root test. Otherwise,
reduce the lag length by one and repeat the process.
A useful rule of thumb for determining pmax , suggested by Schwert(1989), is"µ¶1/4 #
T(4.5)
pmax = 12 ·100
where [x] denotes the integer part of x. This choice allows pmax to growwith the sample so that the ADF test regressions (4.3) and (4.4) are valid
if the errors follow an ARMA process with unknown order.
Example 19 Testing for a unit root in exchange rate data using ADF tests
To illustrate the ADF test procedure, consider testing for a unit rootin the logarithm of the US/CA monthly spot exchange rate, denoted st ,over the 30 year period 1976 - 1996. Figure 4.4 shows st , ∆st as well as the
sample autocorrelations for these series. The data and plots are createdwith the S-PLUS commands> uscn.spot = lexrates.dat[,"USCNS"]> uscn.spot@title = "Log US/CN spot exchange rate"> par(mfrow=c(2,2))> plot.timeSeries(uscn.spot, reference.grid=F,+ main="Log of US/CN spot exchange rate")

122

4. Unit Root Tests
Series : uscn.spot

-0.35

0.0

0.2

-0.15

ACF
0.4 0.6

0.8

0.00

1.0

Log of US/CN spot exchange rate

1976

1980

1984

1988

1992

0

1996

5

10

15

20

Lag

Series : diff(uscn.spot)

-0.2

-0.06

0.2

-0.02

ACF
0.4 0.6

0.8

0.02

1.0

First difference of log US/CN spot exchange rate

1976

1980

1984

1988

1992

1996

0

5

10

15

20

Lag

FIGURE 4.4. US/CN spot rate, first diﬀerence and SACF.

> xx = acf(uscn.spot)> plot.timeSeries(diff(uscn.spot), reference.grid=F,+ main="First difference of log US/CN spot exchange rate")> xx = acf(diff(uscn.spot))
Clearly, st exhibits random walk like behavior with no apparent positive or negative drift. However, ∆st behaves very much like a white noise
process. The appropriate trend specification is to include a constant in thetest regression. Regarding the maximum lag length for the Ng-Perron procedure, given the lack of serial correlation in ∆st a conservative choice ispmax = 6. The ADF t-statistic computed from the test regression with aconstant and p = 6 lags can be computed using the S+FinMetrics function
unitroot as follows> adft.out = unitroot(uscn.spot, trend="c", statistic="t",+ method="adf", lags=6)> class(adft.out)[1] "unitroot"
The output of unitroot is an object of class “unitroot” for which there
are print and summary methods. Typing the name of the object invokes
the print method and displays the basic test result> adft.out
Test for Unit Root: Augmented DF Test

4.3 Autoregressive Unit Root Tests

123

Null Hypothesis: there is a unit root
Type of Test: t-test
Test Statistic: -2.6
P-value: 0.09427
Coefficients:
lag1
lag2
lag3
lag4
lag5
-0.0280 -0.1188 -0.0584 -0.0327 -0.0019

lag6 constant
0.0430 -0.0075

Degrees of freedom: 239 total; 232 residual
Time period: from Aug 1976 to Jun 1996
Residual standard error: 0.01386
With p = 6 the ADF t-statistic is -2.6 and has a p-value (computed using
punitroot) of 0.094. Hence we do not reject the unit root null at the 9.4%
level. The small p-value here may be due to the inclusion of superfluous lags.
To see the significance of the lags in the test regression, use the summary
method> summary(adft.out)
Test for Unit Root: Augmented DF Test
Null Hypothesis: there is a unit root
Type of Test: t test
Test Statistic: -2.6
P-value: 0.09427
Coefficients:
Value Std. Error t value Pr(>|t|)lag1 -0.0280 0.0108-2.6004 0.0099
lag2 -0.1188 0.0646-1.8407 0.0669
lag3 -0.0584 0.0650-0.8983 0.3700
lag4 -0.0327 0.0651-0.5018 0.6163
lag5 -0.0019 0.0651-0.0293 0.9766
lag6 0.0430 0.0645
0.6662 0.5060
constant -0.0075 0.0024-3.0982 0.0022
Regression Diagnostics:
R-Squared 0.0462
Adjusted R-Squared 0.0215
Durbin-Watson Stat 2.0033
Residual standard error: 0.01386 on 235 degrees of freedom
F-statistic: 1.874 on 6 and 232 degrees of freedom, the124

4. Unit Root Tests

p-value is 0.08619
Time period: from Aug 1976 to Jun 1996
The results indicate that too many lags have been included in the test
regression. Following the Ng-Perron backward selection procedure p = 2
lags are selected. The results are> adft.out = unitroot(uscn.spot, trend="c", lags=2)> summary(adft.out)
Test for Unit Root: Augmented DF Test
Null Hypothesis: there is a unit root
Type of Test: t test
Test Statistic: -2.115
P-value: 0.2392
Coefficients:
Value Std. Error t value Pr(>|t|)lag1 -0.0214 0.0101-2.1146 0.0355
lag2 -0.1047 0.0635-1.6476 0.1007
constant -0.0058 0.0022-2.6001 0.0099
Regression Diagnostics:
R-Squared 0.0299
Adjusted R-Squared 0.0218
Durbin-Watson Stat 2.0145
Residual standard error: 0.01378 on 239 degrees of freedom
F-statistic: 3.694 on 2 and 240 degrees of freedom, thep-value is 0.02629
Time period: from Apr 1976 to Jun 1996
With 2 lags the ADF t-statistic is -2.115, the p-value 0.239 and we havegreater evidence for a unit root in st . A similar result is found with the
ADF normalized bias statistic> adfn.out = unitroot(uscn.spot, trend="c", lags=2,+ statistic="n")> adfn.out
Test for Unit Root: Augmented DF Test
Null Hypothesis: there is a unit root
Type of Test: normalized test
Test Statistic: -5.193
P-value: 0.4129

4.3 Autoregressive Unit Root Tests

125

Series : lnp

-0.2

0.5

1.5

0.2

2.5

ACF
0.4 0.6

3.5

0.8

4.5

1.0

Log of S&P 500 index

1990

1992

1994

1996

1998

0

2000

5

10
Lag

15

20

Series : diff(lnp)

-0.2

-0.4

-0.2

0.2

ACF

0.0

0.6

0.2

1.0

First difference of log S&P 500 Index

1990

1992

1994

1996

1998

2000

0

5

10
Lag

15

20

FIGURE 4.5. Log prices on the S&P 500 index, first diﬀerence and SACF.

Coefficients:
lag1
lag2 constant-0.0214 -0.1047 -0.0058
Degrees of freedom: 243 total; 240 residual
Time period: from Apr 1976 to Jun 1996
Residual standard error: 0.01378
Example 20 Testing for a unit root in log stock prices
The log levels of asset prices are usually treated as I(1) with drift. Indeed,
the random walk model of stock prices is a special case of an I(1) process.
Consider testing for a unit root in the log of the monthly S&P 500 index,
pt , over the period January 1990 through January 2001. The data is takenfrom the S+FinMetrics “timeSeries” singleIndex.dat. The data andvarious plots are created with the S-PLUS commands> lnp = log(singleIndex.dat[,1])> lnp@title = "Log of S&P 500 Index"> par(mfrow=c(2,2))> plot.timeSeries(lnp, reference.grid=F,+ main="Log of S&P 500 index")> acf.plot(acf(lnp,plot=F))> plot.timeSeries(diff(lnp), reference.grid=F,

126

4. Unit Root Tests+ main="First difference of log S&P 500 Index")> acf.plot(acf(diff(lnp),plot=F))
and are illustrated in Figure 4.5. Clearly, the pt is nonstationary due tothe positive trend. Also, there appears to be some negative autocorrelation
at lag one in ∆pt . The null hypothesis to be tested is that pt is I(1) withdrift, and the alternative is that the pt is I(0) about a deterministic timetrend. The ADF t-statistic to test these hypotheses is computed with aconstant and time trend in the test regression and four lags of ∆pt (selecting
using the Ng-Perron backward selection method)> adft.out = unitroot(lnp, trend="ct", lags=4)> summary(adft.out)
Test for Unit Root: Augmented DF Test
Null Hypothesis: there is a unit root
Type of Test: t test
Test Statistic: -1.315
P-value: 0.8798
Coefficients:
Value Std. Error t value Pr(>|t|)lag1 -0.0540 0.0410-1.3150 0.1910
lag2 -0.1869 0.0978-1.9111 0.0583
lag3 -0.0460 0.0995-0.4627 0.6444
lag4 0.1939 0.0971
1.9964 0.0481
constant 0.1678 0.1040
1.6128 0.1094
time 0.0015 0.0014
1.0743 0.2848
Regression Diagnostics:
R-Squared 0.1016
Adjusted R-Squared 0.0651
Durbin-Watson Stat 1.9544
Residual standard error: 0.1087 on 125 degrees of freedom
F-statistic: 2.783 on 5 and 123 degrees of freedom, thep-value is 0.0204
Time period: from May 1990 to Jan 2001
ADFt = −1.315 and has a p-value of 0.8798, so one clearly does not reject
the null that pt is I(1) with drift.

4.3 Autoregressive Unit Root Tests

127

4.3.4 Phillips-Perron Unit Root Tests
Phillips and Perron (1988) developed a number of unit root tests that havebecome popular in the analysis of financial time series. The Phillips-Perron(PP) unit root tests diﬀer from the ADF tests mainly in how they deal
with serial correlation and heteroskedasticity in the errors. In particular,
where the ADF tests use a parametric autoregression to approximate the
ARMA structure of the errors in the test regression, the PP tests ignore
any serial correlation in the test regression. The test regression for the PP
tests is∆yt = β0 Dt + πyt−1 + ut
where ut is I(0) and may be heteroskedastic. The PP tests correct forany serial correlation and heteroskedasticity in the errors ut of the test
regression by directly modifying the test statistics tπ=0 and T π̂. These
modified statistics, denoted Zt and Zπ , are given by! µÃ 2µ 2 ¶1/2¶σ̂
1 λ̂ − σ̂ 2
T · SE(π̂)
Zt =· tπ=0 −
·
2
2
2
σ̂ 2
λ̂
λ̂
Zπ

= T π̂ −1 T 2 · SE(π̂) 2(λ̂ − σ̂ 2 )
2
2
σ̂

2

The terms σ̂ 2 and λ̂ are consistent estimates of the variance parameters
σ2
λ2
PT

=

=

lim T −1

T →∞

lim

T →∞

T
X

E[u2t ]

t=1

T
X
t=1

£
¤
E T −1 ST2

where ST =t=1 ut . The sample variance of the least squares residualût is a consistent estimate of σ 2 , and the Newey-West long-run varianceestimate of ut using ût is a consistent estimate of λ2 .
Under the null hypothesis that π = 0, the PP Zt and Zπ statistics havethe same asymptotic distributions as the ADF t-statistic and normalizedbias statistics. One advantage of the PP tests over the ADF tests is that
the PP tests are robust to general forms of heteroskedasticity in the error
term ut . Another advantage is that the user does not have to specify a lag
length for the test regression.
Example 21 Testing for a unit root in exchange rates using the PP tests
Recall the arguments for the S+FinMetrics unitroot function are> args(unitroot)
function(x, trend = "c", method = "adf",

128

4. Unit Root Tests

statistic = "t",lags = 1, bandwidth = NULL,
window = "bartlett", asymptotic = F, na.rm = F)
The PP statistics may be computed by specifying the optional argument
method="pp". When method="pp" is chosen, the argument window specifies the weight function and the argument bandwidth determines the lagtruncation parameter used in computing the long-run variance parameterλ2 . The default bandwidth is the integer part of (4 · (T /100))2/9 where T
is the sample size.
Now, consider testing for a unit root in the log of the US/CN spot exchange rate using the PP Zt and Zπ statistics:> unitroot(uscn.spot, trend="c", method="pp")
Test for Unit Root: Phillips-Perron Test
Null Hypothesis: there is a unit root
Type of Test: t-test
Test Statistic: -1.97
P-value: 0.2999
Coefficients:
lag1 constant-0.0202 -0.0054
Degrees of freedom: 244 total; 242 residual
Time period: from Mar 1976 to Jun 1996
Residual standard error: 0.01383> unitroot(uscn.spot, trend="c", method="pp", statistic="n")Test for Unit Root: Phillips-Perron Test
Null Hypothesis: there is a unit root
Type of Test: normalized test
Test Statistic: -4.245
P-value: 0.5087
Coefficients:
lag1 constant-0.0202 -0.0054
Degrees of freedom: 244 total; 242 residual
Time period: from Mar 1976 to Jun 1996
Residual standard error: 0.01383
As with the ADF tests, the PP tests do not reject the null that the logof the US/CN spot rate is I(1) at any reasonable significance level.

4.4 Stationarity Tests

129

4.4 Stationarity Tests
The ADF and PP unit root tests are for the null hypothesis that a timeseries yt is I(1). Stationarity tests, on the other hand, are for the null thatyt is I(0). The most commonly used stationarity test, the KPSS test, is due
to Kwiatkowski, Phillips, Schmidt, and Shin (1992). They derived their test
by starting with the modelyt
μt= β0 Dt + μt + ut
= μt−1 + εt , εt ∼ W N (0, σ2ε )(4.6)

where Dt contains deterministic components (constant or constant plus
time trend), ut is I(0) and may be heteroskedastic. Notice that μt is a purerandom walk with innovation variance σ 2ε . The null hypothesis that yt isI(0) is formulated as H0 : σ 2ε = 0, which implies that μt is a constant.
Although not directly apparent, this null hypothesis also implies a unit
moving average root in the ARMA representation of ∆yt . The KPSS test
statistic is the Lagrange multiplier (LM) or score statistic for testing σ2ε = 0
against the alternative that σ 2ε > 0 and is given by
Ã
!
T
X
2
−2
2
KPSS = T(4.7)
Ŝt /λ̂t=1

Pt
2
where Ŝt = j=1 ûj , ût is the residual of a regression of yt on Dt and λ̂
is a consistent estimate of the long-run variance of ut using ût . Under thenull that yt is I(0), Kwiatkowski, Phillips, Schmidt, and Shin showed that
KPSS converges to a function of standard Brownian motion that depends
on the form of the deterministic terms Dt but not their coeﬃcient values
β. In particular, if Dt = 1 thenZ 1d
V1 (r)dr(4.8)
KPSS →
0where V1 (r) = W (r) − rW (1) and W (r) is a standard Brownian motion forr ∈ [0, 1]. If Dt = (1, t)0 then
Z 1
d
KPSS →
V2 (r)dr
(4.9)
0

R1
where V2 (r) = W (r) + r(2 − 3r)W (1) + 6r(r2 − 1) 0 W (s)ds. Criticalvalues from the asymptotic distributions (4.8) and (4.9) must be obtainedby simulation methods, and these are summarized in Table 4.1.
The stationary test is a one-sided right-tailed test so that one rejects thenull of stationarity at the 100 · α% level if the KPSS test statistic (4.7) isgreater than the 100 · (1 − α)% quantile from the appropriate asymptotic
distribution (4.8) or (4.9).

130

4. Unit Root Tests

Distribution
R1
V (r)dr
R01 1
V (r)dr
0 2

0.90
0.349
0.120

Right tail quantiles
0.925 0.950 0.975 0.99
0.396 0.446 0.592 0.762
0.133 0.149 0.184 0.229

TABLE 4.1. Quantiles of the distribution of the KPSS statistic

4.4.1 Simulating the KPSS Distributions
The distributions in (4.8) and (4.9) may be simulated using methods similar
to those used to simulate the DF distribution. The following S-PLUS codeis used to create the quantiles in Table 4.1:
wiener2 = function(nobs) {e = rnorm(nobs)# create detrended errors
e1 = e - mean(e)e2 = residuals(OLS(e~seq(1,nobs)))# compute simulated Brownian Bridges
y1 = cumsum(e1)y2 = cumsum(e2)intW2.1 = nobs^(-2) * sum(y1^2)
intW2.2 = nobs^(-2) * sum(y2^2)
ans = list(intW2.1=intW2.1,
intW2.2=intW2.2)
ans}## simulate KPSS distributions#> nobs = 1000
> nsim = 10000> KPSS1 = rep(0,nsim)> KPSS2 = rep(0,nsim)> for (i in 1:nsim) {BN.moments = wiener2(nobs)
KPSS1[i] = BN.moments$intW2.1
KPSS2[i] = BN.moments$intW2.2}## compute quantiles of distribution#> quantile(KPSS1, probs=c(0.90,0.925,0.95,0.975,0.99))
90.0%
92.5%
95.0%
97.5%
99.0%
0.34914 0.39634 0.46643 0.59155 0.76174> quantile(KPSS2, probs=c(0.90,0.925,0.95,0.975,0.99))

4.4 Stationarity Tests

131

90.0% 92.5%
95.0% 97.5%
99.0%
0.12003 0.1325 0.14907 0.1841 0.22923
Currently, only asymptotic critical values are available for the KPSS test.

4.4.2 Testing for Stationarity Using the S+FinMetrics
Function stationaryTest
The S+FinMetrics function stationaryTest may be used to test the null
hypothesis that a time series yt is I(0) based on the KPSS statistic (4.7).
The function stationaryTest has arguments> args(stationaryTest)
function(x, trend = "c", bandwidth = NULL, na.rm = F)
where x represents a univariate vector or “timeSeries”. The argument
trend specifies the deterministic trend component in (4.6) and valid choices
are "c" for a constant and "ct" for a constant and time trend. The argument bandwidth determines the lag truncation parameter used in computing the long-run variance parameter λ2 . The default bandwidth is theinteger part of (4 · (T /100))2/9 where T is the sample size. The output ofstationaryTest is an object of class “stationaryTest” for which there is
only a print method. The use of stationaryTest is illustrated with thefollowing example.
Example 22 Testing for stationarity in exchange rates
Consider the US/CN spot exchange data used in the previous examples.
To test the null that st is I(0), the KPSS statistic is computed using a
constant in (4.6):> kpss.out = stationaryTest(uscn.spot, trend="c")> class(kpss.out)[1] "stationaryTest"> kpss.out
Test for Stationarity: KPSS Test
Null Hypothesis: stationary around a constant
Test Statistics:
USCNS
1.6411*** : significant at 5% level** : significant at 1% level

132

4. Unit Root Tests

Total Observ.: 245
Bandwidth : 5
The KPSS statistic is 1.641 and is greater than the 99% quantile, 0.762,
from Table.4.1. Therefore, the null that st is I(0) is rejected at the 1%
level.

4.5 Some Problems with Unit Root Tests
The ADF and PP tests are asymptotically equivalent but may diﬀer substantially in finite samples due to the diﬀerent ways in which they correct
for serial correlation in the test regression. In particular, Schwert (1989)
found that if ∆yt has an ARMA representation with a large and negative
MA component, then the ADF and PP tests are severely size distorted(reject I(1) null much too often when it is true) and that the PP tests are
more size distorted than the ADF tests. Recently, Perron and Ng (1996)
have suggested useful modifications to the PP tests to mitigate this size
distortion. Caner and Killian (2001) have found similar problems with the
KPSS test.
In general, the ADF and PP tests have very low power against I(0)alternatives that are close to being I(1). That is, unit root tests cannot
distinguish highly persistent stationary processes from nonstationary processes very well. Also, the power of unit root tests diminish as deterministic
terms are added to the test regressions. That is, tests that include a constant and trend in the test regression have less power than tests that only
include a constant in the test regression. For maximum power against very
persistent alternatives the recent tests proposed by Elliot, Rothenberg, andStock (1996), and Ng and Perron (2001) should be used. These tests aredescribed in the next section.

4.6 Eﬃcient Unit Root Tests
Assume that T observations are generated by
yt = β0 Dt + ut , ut = φut−1 + vt
where Dt represents a vector of deterministic terms, E[u0 ] < ∞, and vt is
a 1-summable linear process with long-run variance λ2 . Typically Dt = 1
or Dt = [1, t]. Consider testing the null hypothesis φ = 1 versus |φ| <1. If the distribution of the data were known then the Neyman-Pearson
Lemma gives the test with best power against any point alternative φ̄. The
power of this optimal test plotted as a function of φ̄ gives an upper bound(envelope) for the power of any test based on the same distribution of the4.6 Eﬃcient Unit Root Tests

133

data. An undesirable feature of this power envelope is that it depends on thespecific value of φ̄, so that there is no uniformly most power full test thatcan be used against all alternatives |φ| < 1. However, using asymptotic
approximations based on the local-to-unity alternative φ = 1 + c/T, forc < 0, Elliot, Rothenberg, and Stock (2001) (hereafter ERS) derived aclass of test statistics that come very close to the power envelope for awide range of alternatives. These tests are referred to as eﬃcient unit root
tests, and they can have substantially higher power than the ADF or PP
unit root tests especially when φ is close to unity.

4.6.1 Point Optimal Tests
The starting point for the class of eﬃcient tests is the feasible test statisticthat is optimal (asymptotically) for the point alternative φ̄ = 1−c̄/T, c̄ < 0.
This test is constructed as follows. Define the T −dimensional column vector
yφ and T × q dimensional matrix Dφ byyφDφ= (y1 , y2 − φy1 , . . . , yT − φyT −1 )0= (D01 , D02 − φD01 , . . . , D0T − φD0T −1 )0

All of the elements of yφ and Dφ , except the first, are quasi-diﬀerenced
using the operator 1 − φL. Next, for any value of φ, define S(φ) as the sumof squared residuals from a least squares regression of yφ on Dφ . That is,S(φ) = ỹφ0 ỹφ
where ỹφ = yφ − Dφ β̂φ and β̂φ = (D0φ Dφ )−1 D0φ yφ . ERS showed that thefeasible point optimal unit root test against the alternative φ̄ = 1 − c̄/T has
the form£¤ 2
PT = S(φ̄) − φ̄S(1) /λ̂(4.10)
2

where λ̂ is a consistent estimate of λ2 . ERS derived the asymptotic distribution of PT for Dt = 1 and Dt = (1, t) and provided asymptotic andfinite sample critical values for tests of size 1%, 2.5%, 5% and 10%4 .
Through a series of simulation experiments, ERS discovered that if φ̄ =1+c̄/T is chosen such that the power of PT is tangent to the power envelope
at 50% power then the overall power of PT , for a wide range of φ̄ values
less than unity, is close to the power envelope. For a given sample size T,
the value of φ̄ that results in PT having 50% power depends on c̄ and theform of the deterministic terms in Dt . ERS showed that if Dt = 1 thenc̄ = −7, and if Dt = (1, t) then c̄ = −13.5.
The ERS PT statistic may be computed using the function unitrootwith method="ers".
4 These critical values are given in ERS Table I panels A and B.

134

4. Unit Root Tests

4.6.2 DF-GLS Tests
In the construction of the ERS feasible point optimal test (4.10), the unknown parameters β of the trend function are eﬃciently estimated under
the alternative model with φ̄ = 1 + c̄/T. That is, β̂φ̄ = (D0φ̄ Dφ̄ )−1 D0φ̄ yφ̄ .
ERS used this insight to derive an eﬃcient version of the ADF t-statistic,
which they called the DF-GLS test. They constructed this t-statistic as follows. First, using the trend parameters β̂φ̄ estimated under the alternative,
define the detrended data0

ytd = yt − β̂φ̄ Dt
ERS called this detrending procedure GLS detrending 5 . Next, using the
GLS detrended data, estimate by least squares the ADF test regression
which omits the deterministic termsd∆ytd = πyt−1
+

p
X

d
ψ j ∆yt−j+ εt(4.11)

j=1

and compute the t-statistic for testing π = 0. When Dt = 1, ERS showedthat the asymptotic distribution of the DF-GLS test is the same as the
ADF t-test, but has higher asymptotic power (against local alternatives)
than the DF t-test. Furthermore, ERS show that the DF-GLS test hasessentially the same asymptotic power as the ERS point optimal test whenc̄ = −7. When Dt = (1, t) the asymptotic distribution of the DF-GLS test,
however, is diﬀerent from the ADF t-test. ERS and Ng and Perron (2001)
provided critical values for the DF-GLS test in this case. ERS showed thatthe DF-GLS test has the same asymptotic power as the ERS point optimal
test with c = −13.5, and has higher power than the DF t-test against local
alternatives.
The DF-GLS t-test may be computed using the function unitroot withmethod="dfgls".

4.6.3 Modified Eﬃcient PP Tests
Ng and Perron (2001) used the GLS detrending procedure of ERS to create
eﬃcient versions of the modified PP tests of Perron and Ng (1996). Theseeﬃcient modified PP tests do not exhibit the severe size distortions of the
PP tests for errors with large negative MA or AR roots, and they can havesubstantially higher power than the PP tests especially when φ is close tounity.
5 For deterministicly trending trend data with ergodic-stationary deviations fromtrend, Grenander’s Theorem gives the result that least squares estimates of the trendparameters ignoring serial correlation are asymptotically equivalent to the generalized
least squares estimates that incorporate serial correlation.

4.6 Eﬃcient Unit Root Tests

135

Using the GLS detrended data ytd , the eﬃcient modified PP tests aredefined as
MZα

=

MSB =
MZt

2
(T −1 yTd − λ̂ )

Ã

T −2

T
X

Ã

2T

T
X
t=1

2

d
yt−1
/λ̂

t=1

−2

d
yt−1

!−1

!1/2

= MZα × MSB

The statistics MZα and MZt are eﬃcient versions of the PP Zα and Zt
tests that have much smaller size distortions in the presence of negative
moving average errors. Ng and Perron derived the asymptotic distributionsof these statistics under the local alternative φ = 1 − c/T for Dt = 1 and
Dt = (1, t). In particular, they showed that the asymptotic distribution of
MZt is the same as the DF-GLS t-test.
The statistic MZt may be computed using the function unitroot withmethod="mpp".

4.6.4 Estimating λ2
Ng and Perron (2001) emphasized that the estimation of the long-run variance λ2 has important implications for the finite sample behavior of the
ERS point optimal test and the eﬃcient modified PP tests. They stressedthat an autoregressive estimate of λ2 should be used to achieve stable finite
sample size. They recommended estimating λ2 from the ADF test regression (4.11) based on the GLS detrended data:σ̂ 2pλ̂AR = ³´2
1 − ψ̂(1)PT
Pp
where ψ̂(1) = j=1 ψ̂ j and σ̂ 2p = (T − p)−1 t=p+1 ε̂2t are obtained from
(4.11) by least squares estimation.

4.6.5 Choosing Lag Lengths to Achieve Good Size and PowerNg and Perron also stressed that good size and power properties of the all
the eﬃcient unit root tests rely on the proper choice of the lag length p used
for specifying the test regression (4.11). They argued, however, that traditional model selection criteria such as AIC and BIC are not well suited for
determining p with integrated data. Ng and Perron suggested the modified
information criteria (MIC) that selects p as pmic = arg minp≤pmax MIC(p)136

4. Unit Root Tests

where
MIC(p) = ln(σ̂2p ) +τ T (p) =
σ̂ 2p

=

π̂2

PT

CT (τ T (p) + p)
T − pmaxdt=pmax +1 yt−1
σ̂ 2p

T
X
1
ε̂2
T − pmax t=p +1 t
maxwith CT > 0 and CT /T → 0 as T → ∞. The maximum lag, pmax , may be
set using (4.5). The modified AIC (MAIC) results when CT = 2, and themodified BIC (MBIC) results when CT = ln(T − pmax ). Through a seriesof simulation experiments, Ng and Perron recommended selecting the lag
length p by minimizing the MAIC.
Example 23 Eﬃcient unit root tests
To illustrate the eﬃcient unit root tests, consider testing for a unit rootin the 30-day interest rate diﬀerential formed from the diﬀerence between
monthly US and UK spot exchange rates:> fd = lexrates.dat[,"USUKS"] - lexrates.dat[,"USUKF"]> colIds(fd) = "USUKFD"> fd@title = "US/UK 30-day interest rate differential"
The interest rate diﬀerential, its SACF, and the SACF of its first diﬀerence are depicted in Figure 4.6. The graphs clearly show that the interest
rate diﬀerential has a high degree of persistence, and that there is little
persistence in the first diﬀerence.
The ERS PT test, DF-GLS t-test and Ng-Perron MZt test all with Dt = 1
may be computed using the function unitroot as follows:> ers = unitroot(fd,trend="c",method="ers",max.lags=12)> dfgls = unitroot(fd,trend="c",method="dfgls",max.lags=12)> mpp = unitroot(fd,trend="c",method="mpp",max.lags=12)
Since the optional argument lags is omitted, the lag length for the test
regression (4.11) is determined by minimizing the MAIC with pmax = 12
set by the optional argument max.lags=12. The results of the eﬃcient unit
root tests are:> ers.test
Test for Unit Root: Elliott-Rothenberg-Stock Test
Null Hypothesis: there is a unit root
Test Statistic: 1.772**

4.6 Eﬃcient Unit Root Tests-0.0040.012

US/UK 30-day interest rate differential

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

0.0

0.4

0.8

Series : fd

0

5

10

15

20

15

20

Lag

-0.2

0.2

0.6

1.0

Series : diff(fd)

0

5

10
Lag

FIGURE 4.6. 30-day US/UK interest rate diﬀerential.

* : significant at 5% level** : significant at 1% level
Coefficients:
lag1
-0.07
Degrees of freedom: 244 total; 243 residual
Time period: from Mar 1976 to Jun 1996
Residual standard error: 0.00116> dfgls.test
Test for Unit Root: DF Test with GLS detrending
Null Hypothesis: there is a unit root
Type of Test: t-test
Test Statistic: -2.9205*** : significant at 5% level** : significant at 1% level
Coefficients:
lag1

137

138

4. Unit Root Tests-0.07
Degrees of freedom: 244 total; 243 residual
Time period: from Mar 1976 to Jun 1996
Residual standard error: 0.00116> mpp.test
Test for Unit Root: Modified Phillips-Perron Test
Null Hypothesis: there is a unit root
Type of Test: t-test
Test Statistic: -2.8226*** : significant at 5% level** : significant at 1% level
Coefficients:
lag1
-0.07
Degrees of freedom: 244 total; 243 residual
Time period: from Mar 1976 to Jun 1996
Residual standard error: 0.00116
Minimizing the MAIC gives p = 0, and with this lag length all tests reject
the null hypothesis of a unit root at the 1% level.

4.7 References
Caner, M. and L. Kilian (2001). “Size Distortions of Tests of the Null
Hypothesis of Stationarity: Evidence and Implications for the PPP Debate,” Journal of International Money and Finance, 20, 639-657.
Dickey, D. and W. Fuller (1979). “Distribution of the Estimators for
Autoregressive Time Series with a Unit Root,” Journal of the American
Statistical Association, 74, 427-431.
Dickey, D. and W. Fuller (1981). “Likelihood Ratio Statistics for Autoregressive Time Series with a Unit Root,” Econometrica, 49, 1057-1072.
Elliot, G., T.J. Rothenberg, and J.H. Stock (1996). “Eﬃcient Tests
for an Autoregressive Unit Root,” Econometrica, 64, 813-836.
Fuller, W. (1996). Introduction to Statistical Time Series, Second Edition. John Wiley, New York.

4.7 References

139

Hamilton, J. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
Hatanaka, T. (1995). Time-Series-Based Econometrics: Unit Roots and
Co-Integration. Oxford University Press, Oxford.
Kwiatkowski, D., P.C.B. Phillips, P. Schmidt and Y. Shin (1992).
“Testing the Null Hypothesis of Stationarity Against the Alternative of a
Unit Root,” Journal of Econometrics, 54, 159-178.
MacKinnon, J. (1996). “Numerical Distribution Functions for Unit Rootand Cointegration Tests,” Journal of Applied Econometrics, 11, 601-618.
Maddala, G.S. and I.-M. Kim (1998). Unit Roots, Cointegration and
Structural Change. Oxford University Press, Oxford.
Ng, S., and P. Perron (1995). “Unit Root Tests in ARMA Models with
Data-Dependent Methods for the Selection of the Truncation Lag,” Journalof the American Statistical Association, 90, 268-281.
Ng, S., and P. Perron (2001). “Lag Length Selection and the Construction of Unit Root Tests with Good Size and Power,” Econometrica, 69,
1519-1554.
Perron, P. and S. Ng. (1996). “Useful Modifications to Some Unit Root
Tests with Dependent Errors and their Local Asymptotic Properties,” Review of Economic Studies, 63, 435-463.
Phillips, P.C.B. (1987). “Time Series Regression with a Unit Root,”
Econometrica, 55, 227-301.
Phillips, P.C.B. and P. Perron (1988). “Testing for Unit Roots inTime Series Regression,” Biometrika, 75, 335-346.
Phillips, P.C.B. and Z. Xiao (1998). “A Primer on Unit Root Testing,”
Journal of Economic Surveys, 12, 423-470.
Schwert, W. (1989). “Test for Unit Roots: A Monte Carlo Investigation,”
Journal of Business and Economic Statistics, 7, 147-159.
Said, S.E. and D. Dickey (1984). “Testing for Unit Roots in Autoregressive Moving-Average Models with Unknown Order,” Biometrika, 71,
599-607.
Stock, J.H. (1994). “Units Roots, Structural Breaks and Trends,” in R.F.
Engle and D.L. McFadden (eds.), Handbook of Econometrics, Volume IV.
North Holland, New York.

This is page 140
Printer: Opaque thisThis is page 141
Printer: Opaque this5
Modeling Extreme Values

5.1 Introduction
One of the goals of financial risk management is the accurate calculation of
the magnitudes and probabilities of large potential losses due to extreme
events such as stock market crashes, currency crises, trading scandals, orlarge bond defaults. In statistical terms, these magnitudes and probabilities are high quantiles and tail probabilities of the probability distributionof losses. The importance of risk management in finance cannot be overstated. The catastrophes of October 17, 1987, Long-Term Capital Management, Barings PLC, Metallgesellschaft, Orange County and Daiwa clearly
illustrate the losses that can occur as the result of extreme market movements1 . The objective of extreme value analysis in finance is to quantify
the probabilistic behavior of unusually large losses and to develop tools for
managing extreme risks.
Traditional parametric and nonparametric methods for estimating distributions and densities work well in areas of the empirical distribution where
there are many observations, but they often give very poor fits to the extreme tails of the distribution. This result is particularly troubling because
the management of extreme risk often requires estimating quantiles and tailprobabilities beyond those observed in the data. The methods of extreme
value theory focus on modeling the tail behavior of a loss distribution using
only extreme values rather than all of the data.
1 See Jorian (2001) for a detailed description of these financial disasters.

142

5. Modeling Extreme Values

This chapter is organized as follows. Section 5.2 covers the modeling ofblock maximum and minimum values using the generalized extreme value(GEV) distribution. The maximum likelihood estimator for the parametersof the GEV distribution is derived and analyzed, and graphical diagnostics
for evaluating the fit are discussed. The use of the GEV distribution is
illustrated with examples from finance, and the concept of return levelis introduced. Section 5.3 discusses the modeling of extremes over highthresholds or “peaks over thresholds”. This technique is well suited forthe estimation of common risk measures like value-at-risk and expectedshortfall. Parametric models utilizing the generalized Pareto distribution
as well as non-parametric models are presented.
Two excellent textbooks on extreme value theory are Embrechts, Klüppelberg and Mikosch (1997) and Coles (2001). Both books provide many examples utilizing S-PLUS. Less rigorous treatments of extreme value theory
with many examples in finance are given in Alexander (2001), Jorian (2001)
and Tsay (2001). Useful surveys of extreme value theory applied to financeand risk management are given in Diebold, Schuermann and Stroughair(1997), Danielsson and de Vries (2001), McNeil (1998) and Longin (2000).
The S+FinMetrics functions for modeling extreme values described inthis chapter are based on the functions in the EVIS (Extreme Values In
S-PLUS) library written by Alexander McNeil, and the EVANESCE (Extreme Value ANalysis Employing Statistical Copula Estimation) librarywritten by Rene Carmona and Julia Morrison and described in Carmonaand Morrison (2001) and Carmona (2004). The EVANESCE library also
contains an extensive set of functions for analyzing and fitting bivariate
copulas, which are described in Chapter 19.

5.2 Modeling Maxima and Worst Cases
To motivate the importance of the statistical modeling of extreme losses infinance, consider the following example taken from McNeil (1998). Figure
5.1 shows the daily closing prices and percentage changes in the S&P 500
index over the period January 5, 1960 through October 16, 1987 taken fromthe S+FinMetrics “timeSeries” object sp.raw> spto87 = getReturns(sp.raw, type="discrete", percentage=T)> par(mfrow=c(2,1))> plot(sp.raw, main="Daily Closing Prices")> plot(spto87, main="Daily Percentage Returns")
Before the October crash, the stock market was unusually volatile withseveral large price drops in the index resulting in large negative returns. Of
interest is the characterization of the worst case scenario for a future fallin S&P 500 index utilizing the historical data prior to the crash given in
Figure 5.1. To do this, the following two questions will be addressed:

5.2 Modeling Maxima and Worst Cases

143

100

150

200

250

300

Daily Closing Prices

1960

1965

1970

1975

1980

1985

1980

1985

-6

-4

-2

0

2

4

Daily Percentage Returns

1960

1965

1970

1975

FIGURE 5.1. Daily closing prices and percentage returns on the S&P 500 Index
from January, 1960 through October 16, 1987.

• What is the probability that next year’s annual maximum negative
return exceeds all previous negative returns? In other words, what isthe probability that next year’s maximum negative return is a newrecord?
• What is the 40-year return level of the negative returns? That is, whatis the negative return which, on average, should only be exceeded in
one year every forty years?
To answer these questions, the distribution of extreme negative returns
on S&P 500 index is required. The distribution theory required to analyze
maximum values is briefly discussed in the next section.

5.2.1 The Fisher-Tippet Theorem and the Generalized
Extreme Value Distribution
Let X1 , X2 , . . . be iid random variables representing risks or losses withan unknown cumulative distribution function (CDF) F (x) = Pr{Xi ≤ x}.
Examples of the random risks Xi are losses or negative returns on a financial
asset or portfolio, operational losses, catastrophic insurance claims, andcredit losses. Throughout this chapter, a loss is treated as a positive number
and extreme events occur when losses take values in the right tail of the144

5. Modeling Extreme Values

loss distribution F . Define Mn = max (X1 , . . . , Xn ) as the worst-case loss
in a sample of n losses. An important part of extreme value theory focuseson the distribution of Mn . From the iid assumption, the CDF of Mn is
Pr{Mn ≤ x} = Pr{X1 ≤ x, . . . , Xn ≤ x} =nY

F (x) = F n (x)i=1

Since F n is assumed to be unknown and the empirical distribution function
is often a very poor estimator of F n (x), an asymptotic approximation to
F n based on the Fisher-Tippet Theorem (Fisher and Tippett, 1928) is used
to make inferences on Mn . Furthermore, since F n (x) → 0 or 1 as n → ∞and x is fixed, the asymptotic approximation is based on the standardized
maximum value
Mn − μn
Zn =
(5.1)
σn
where σ n > 0 and μn are sequences of real numbers such that σ n is interpreted as a scale measure and μn is interpreted as a location measure.
The Fisher-Tippet Theorem states that if the standardized maximum (5.1)
converges to some non-degenerate distribution function, it must be a generalized extreme value (GEV) distribution of the form½ª©
ξ 6= 0, 1 + ξz > 0
exp −(1 + ξz)−1/ξHξ (z) =(5.2)
ξ = 0, − ∞ ≤ z ≤ ∞exp {− exp(−z)}
If (5.1) converges to (5.2), then the CDF F of the underlying data is in
the domain of attraction of Hξ . The Fisher-Tippet Theorem is the analog
of the Central Limit Theorem for extreme values. Whereas the Central
Limit Theorem applies to normalized sums of random variables, the FisherTippet Theorem applies to standardized maxima of random variables. The
parameter ξ is a shape parameter and determines the tail behavior of Hξ .
The parameter α = 1/ξ is called the tail index if ξ > 0.
The tail behavior of the distribution F of the underlying data determines the shape parameter ξ of the GEV distribution (5.2). If the tail ofF declines exponentially, then Hξ is of the Gumbel type and ξ = 0. Distributions in the domain of attraction of the Gumbel type are thin tailed
distributions such as the normal, log-normal, exponential, and gamma. For
these distributions, all moments usually exist. If the tail of F declines by
a power function, i.e.
1 − F (x) = x−1/ξ L(x)for some slowly varying function L(x), then Hξ is of the Fréchet type andξ > 02 . Distributions in the domain of attraction of the Fréchet type include
fat tailed distributions like the Pareto, Cauchy, Student-t, alpha-stable with
2 A function L on (0, ∞) is slowly varying if lim
x→∞ L(tx)/L(x) = 1 for t > 0.

5.2 Modeling Maxima and Worst Cases

145

characteristic exponent in (0, 2), as well as various mixture models. Not all
moments are finite for these distributions. In fact, it can be shown thatE[X k ] = ∞ for k ≥ α = 1/ξ. Last, if the tail of F is finite then Hξ is of
the Weibull type and ξ < 0. Distributions in the domain of attraction of
the Weibull type include distributions with bounded support such as theuniform and beta distributions. All moments exist for these distributions.
The Fisher-Tippet Theorem applies to iid observations. However, theGEV distribution (5.2) may be shown (e.g. Embrechts et. al. (1997)) tobe the correct limiting distribution for maxima computed from stationary
time series including stationary GARCH-type processes.
The GEV distribution (5.2) characterizes the limiting distribution of thestandardized maximum (5.1). It turns out that the GEV distribution (5.2)
is invariant to location and scale transformations such that for location andscale parameters μ and σ > 0µ¶x−μHξ (z) = Hξ
(5.3)
= Hξ,μ,σ (x)σThe Fisher-Tippet Theorem may then be interpreted as follows. For large
enough n½¾
Mn − μn
Pr {Zn < z} = Pr
< z ≈ Hξ (z)σnLetting x = σ n z + μn thenPr{Mn < x} ≈ Hξ,μ,σµ

x − μnσn¶= Hξ,μn ,σn (x)(5.4)

The result (5.4) is used in practice to make inferences about the maximum
loss Mn .
Example 24 Plots of GEV distributions
The S+FinMetrics/EVIS functions pgev, qgev, dgev and rgev compute
cumulative probability, quantiles, density, and random generation, respectively, from the GEV distribution (5.3) for ξ 6= 0 and general values forx, μ and σ. For example, the S-PLUS code to compute and plot the GEVCDF Hξ and the pdf hξ for a Fréchet (ξ = 0.5), Weibull (ξ = −0.5) and
Gumbell (ξ = 0) is> z.vals = seq(-5, 5, length=200)> cdf.f = ifelse((z.vals > -2), pgev(z.vals,xi=0.5), 0)> cdf.w = ifelse((z.vals < 2), pgev(z.vals,xi=-0.5), 1)> cdf.g = exp(-exp(-z.vals))> plot(z.vals, cdf.w, type="l", xlab="z", ylab="H(z)")> lines(z.vals, cdf.f, type="l", lty=2)> lines(z.vals, cdf.g, type="l", lty=3)> legend(-5, 1, legend=c("Weibull H(-0.5,0,1)",

5. Modeling Extreme Values

1.0

146

0.0

0.2

0.4

H(z)

0.6

0.8

Weibull H(-0.5,0,1)
Frechet H(0.5,0,1)
Gumbel H(0,0,1)

-4

-2

0

2

4

z

0.4

FIGURE 5.2. Generalized extreme value CDFs Hξ for Fréchet (ξ = −0.5),
Weibull (ξ = 0.5) and Gumbell (ξ = 0).

h(z)

0.0

0.1

0.2

0.3

Weibull H(-0.5,0,1)
Frechet H(0.5,0,1)
Gumbel H(0,0,1)

-4

-2

0

2

4

z

FIGURE 5.3. Generalized extreme value pdfs hξ for Fréchet (ξ = −0.5), Weibull(ξ = 0.5) and Gumbell (ξ = 0).

5.2 Modeling Maxima and Worst Cases

147+ "Frechet H(0.5,0,1)","Gumbel H(0,0,1)"), lty=1:3)> # pdfs> pdf.f = ifelse((z.vals > -2), dgev(z.vals,xi=0.5), 0)> pdf.w = ifelse((z.vals < 2), dgev(z.vals,xi=-0.5), 0)> pdf.g = exp(-exp(-z.vals))*exp(-z.vals)> plot(z.vals, pdf.w, type="l", xlab="z", ylab="h(z)")> lines(z.vals, pdf.f, type="l", lty=2)> lines(z.vals, pdf.g, type="l", lty=3)> legend(-5.25, 0.4, legend=c("Weibull H(-0.5,0,1)",+ "Frechet H(0.5,0,1)","Gumbel H(0,0,1)"), lty=1:3)
The CDF and pdf values are illustrated in Figures 5.2 and 5.3. Noticethat the Fréchet is only defined for z > −2, and that the Weibull is only
defined for z < 2.

5.2.2 Estimation of the GEV Distribution
The GEV distribution (5.4) depends on three parameters: the shape parameter ξ and the standardizing constants σ n and μn . These parameters
may be estimated using parametric maximum likelihood estimation (MLE).
The S+FinMetrics/EVIS functions gev and gumbel fit the GEV distribution (5.2) by MLE to block maxima data. The calculation of the parametric
MLE is briefly described below and illustrated with examples.
Parametric Maximum Likelihood Estimator
Let X1 , . . . , XT be identically distributed losses from a sample of size T withunknown CDF F and let MT denote the sample maximum. For inference
on MT using (5.4) the parameters ξ, σ T and μT must be estimated. Since
there is only one value of MT for the entire sample, it is not possible toform a likelihood function for ξ, σ T and μT . However, if interest is on the
maximum of X over a large finite subsample or block of size n < T , Mn ,then a sub-sampling method may be used to form the likelihood function
for the parameters ξ, σn and μn of the GEV distribution for Mn . To dothis, the sample is divided into m non-overlapping blocks of essentiallyequal size n = T /m
[X1 , . . . , Xn |Xn+1 , . . . , X2n | . . . |X(m−1)n+1 , . . . , Xmn ](j)and Mn is defined as the maximum value of Xi in block j = 1, . . . , m.
The likelihood function for the parameters ξ, σ n and μn of the GEV
distribution (5.4) is then constructed from the sample of block maxima(1)(m){Mn , . . . , Mn }. It is assumed that the block size n is suﬃciently large
so that the Fisher-Tippet Theorem holds.

148

5. Modeling Extreme Values

The log likelihood function assuming iid observations from a GEV distribution with ξ 6= 0 is
"
Ã
!#
m
(i)
X
Mn − μln 1 + ξl(μ, σ, ξ) = −m ln(σ) − (1 + 1/ξ)
σ
i=1
"
Ã
!#
−1/ξ
m
(i)
X
Mn − μ1+ξ−σi=1such that
1+ξ

Ã

(i)

Mn − μσ!

>0

The log-likelihood for the case ξ = 0 (Gumbel family) is
Ã
!
m
(i)
X
Mn − μl(μ, σ) = −m ln(σ) −
σ
i=1
" Ã
!#
m
(i)
X
Mn − μ
exp −
−
σ
i=1
Details of the maximum likelihood estimation are discussed in Embrechts
et. al. (1997) and Coles (2001). For ξ > −0.5 the MLEs for μ, σ andξ are consistent and asymptotically normally distributed with asymptotic
variance given by the inverse of the observed information matrix. The finite
sample properties of the MLE will depend on the number of blocks m andthe block size n, see McNeil (1998) for an illustration. There is a trade-oﬀ
between bias and variance. The bias of the MLE is reduced by increasing
the block size n, and the variance of the MLE is reduced by increasing thenumber of blocks m.
Example 25 MLE of GEV CDF for block maxima from daily S&P 500
returns
Consider determining the appropriate GEV distribution for the daily
negative returns on S&P 500 index discussed at the beginning of this section. A normal qq-plot of the returns computed using> qqPlot(spto87,strip.text="Daily returns on S&P 500",+ xlab="Quantiles of standard normal",+ ylab="Quantiles of S&P 500")
is shown in Figure 5.4. The returns clearly have fatter tails than the normal
distribution which suggests the Fréchet family of GEV distributions with
ξ > 0 for the block maximum of negative returns.
Before the GEV distribution is fit by MLE, consider first some exploratory data analysis on the annual block maxima of daily negative5.2 Modeling Maxima and Worst Cases

149

Daily returns on S&P 500
05/27/1970
4

Quantiles of S&P 500

2

0

-2

-4
10/16/1987-6
05/28/1962
-4

-2

0

2

4

Quantiles of standard normal

FIGURE 5.4. Normal qq-plot for the daily percentage returns on the S&P 500
index over the period January 5, 1960 through October 16, 1987.

returns. The block maxima may be easily computed using the function
aggregateSeries:> annualMax.sp500 = aggregateSeries(-spto87, by="years",+ FUN=max)
Figure 5.5 created using> Xn = sort(seriesData(annualMax.sp500))> par(mfrow=c(2,2))> plot(annualMax.sp500)> hist(seriesData(annualMax.sp500),xlab="Annual maximum")> plot(Xn,-log(-log(ppoints(Xn))),xlab="Annual maximum")> tmp = records(-spto87)
gives several graphical summaries of the annual block maxima. The largest
daily negative return in an annual block is 6.68% occurring in 1962. The
histogram resembles a Fréchet density (see example above). The qq-plot
uses the Gumbel, H0 , as the reference distribution. For this distribution, thequantiles satisfy H0−1 (p) = − ln(− ln(p)). The downward curve in the plot
indicates a GEV distribution with ξ > 0. The plot of record development
is created with the S+FinMetrics/EVIS function records and illustrates
the developments of records (new maxima) for the daily negative returns
along with the expected number of records for iid data, see Embrechts et.

10

5. Modeling Extreme Values

0

2

2

3

4

4

6

5

8

6

150

1960

1965

1970

1975

1980

1

1985

2

3

4

5

6

7

Annual maximum

10
Records

4

6

8

3
2
1

2

0
-1

- log( - log(ppoints(Xn)))

4

Plot of Record Development

2

3

4

5

6

1

5

10

Annual maximum

50

100

500

Trial

FIGURE 5.5. Annual block maxima, histogram, Gumbel qq-plot and records
summary for the daily returns on the S&P 500.

al. (1997) section 6.2.5. Apart from the somewhat large number of records
early on, the number of records appears consistent with iid behavior.
The MLEs for the parameters of the GEV distribution with ξ 6= 0 using
block maxima may be computed using the S+FinMetrics/EVIS function
gev. For example, to compute the MLEs using annual blocks from thedaily (negative) returns on S&P 500 index use> gev.fit.year = gev(-spto87, block="year")> class(gev.fit.year)[1] "gev"
The argument block determines the blocking method for the supplied
data. An integer value for block gives the number of observations in eachblock. If the data supplied are a “timeSeries” then the value of block
can be also be the character strings “year”, “semester”, “quarter” or“month”. If no value for block is given then the data are interpreted as
block maxima.
The function gev returns an sv3 object of class “gev” for which there
are print and plot methods. The components of gev.fit.year are> names(gev.fit.year)[1] "n.all""n"[5] "data""par.ests""call""par.ses""block""varcov"

5.2 Modeling Maxima and Worst Cases[9] "converged"

151"nllh.final"

and a description of these components is given in the online help for
gev.object. The component n gives the number of blocks m:> gev.fit.year$n[1] 28(i)

The block maxima Mn (i = 1, . . . , m) are in the data component. Since
the data supplied to gev are in a “timeSeries”, the block maxima in
gev.fit.year$data are also a “timeSeries”. The MLEs and asymptotic
standard errors for the parameters μ, σ and ξ are:> gev.fit.year
Generalized Extreme Value Distribution Fit -28 blocks of maxima data
ML estimation converged.
Log-likelihood value: -38.34

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.3344 0.2081
1.6068sigma 0.6716 0.1308
5.1337mu 1.9750 0.1513
13.0549c ξ̂) = 0.208. An
The MLE for ξ is 0.334 with asymptotic standard SE(
asymptotic 95% confidence interval for ξ is [−0.081, 0.751] and indicatesconsiderably uncertainty about the value of ξ.
The fit to the GEV distribution may be evaluated graphically using theplot method:> plot(gev.fit.year)
Make a plot selection (or 0 to exit):
1: plot: Scatterplot of Residuals
2: plot: QQplot of Residuals
Selection:
Plot options 1 and 2 are illustrated in Figure 5.6. The plots show aspectsof the crude residuals
Ã
!−1/ξ
(i)
Mn − μ̂Wi = 1 + ξ̂σ̂which should be iid unit exponentially distributed random variables if thefitted model is correct. The scatter plot of the residuals, with a lowest esti-

5. Modeling Extreme Values

0

0

1

2

Exponential Quantiles

2
1

Residuals

3

3

4

152

0

5

10

15

20

25

0

Ordering

12

3

Ordered Data

FIGURE 5.6. Residual plots from GEV distribution fit to annual block maxima
of daily negative return on the S&P 500 index.

mate of trend, does not reveal any significant unmodeled trend in the data.
The qq-plot, using the exponential distribution as the reference distribution, is linear and appears to validate the GEV distribution.
Using the MLEs of the GEV distribution fit to the annual block maxima
of the (negative) daily returns on S&P 500 index, the question• What is the probability that next year’s annual maximum negative
return exceeds all previous negative returns?
may be answered using (5.4). Since the largest block maxima is 6.68%,
this probability is estimated using
³
´´
³
(29)
(1)
(28)
Pr M260 > max M260 , . . . , M260
= 1 − Hξ̂,μ̂,σ̂ (6.68)

Using the S+FinMetrics/EVIS function pgev, the result is> 1- pgev(max(gev.fit.year$data),+ xi=gev.fit.year$par.ests["xi"],+ mu=gev.fit.year$par.ests["mu"],+ sigma=gev.fit.year$par.ests["sigma"])
0.02677

That is, there is a 2.7% chance that a new record maximum daily negative
return will be established during the next year.

5.2 Modeling Maxima and Worst Cases

153

The above analysis is based on annual block maxima. The GEV distribution fit to quarterly block maxima is obtained using> gev.fit.quarter= gev(-spto87,block="quarter")> gev.fit.quarter
Generalized Extreme Value Distribution Fit -112 blocks of maxima data
ML estimation converged.
Log-likelihood value: -111.9

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.1910 0.0695
2.7472sigma 0.5021 0.0416
12.0701
mu 1.4013 0.0530
26.4583
The MLEs for ξ, μ and σ using quarterly blocks are slightly smaller thanthe MLEs using annual blocks. Notice, however, that the estimated asymptotic standard errors are much smaller using quarterly block. In particular,
an asymptotic 95% confidence interval for ξ is [0.052, 0.330] and contains
only positive values for ξ indicating a fat-tailed distribution. An estimate of
the probability that next quarter’s maximum exceeds all previous maximais> 1- pgev(max(gev.fit.quarter$data),+ xi=gev.fit.quarter$par.ests["xi"],+ mu=gev.fit.quarter$par.ests["mu"],+ sigma=gev.fit.quarter$par.ests["sigma"])
0.003138
As expected, this probability is smaller than the corresponding probability
computed for annual maxima.

5.2.3 Return Level
For α ∈ (0, 1) the 100 · α% quantile of a continuous distribution with
distribution function F is the value qα such thatqα = F −1 (α).
A useful risk measure for block maxima that is related to a high quantile
is the so-called return level. The k n-block return level, Rn,k , is defined tobe that level which is exceeded in one out of every k blocks of size n. Thatis, Rn,k is the loss value such that
Pr{Mn > Rn,k } = 1/k

(5.5)

154

5. Modeling Extreme Values

The n-block in which the return level is exceeded is called a stress period.
If the distribution of the maxima Mn in blocks of length n is characterizedby (5.4) then Rn,k is simply the 1 − 1/k quantile of this distribution:−1(1 − 1/k) = μ −Rn,k ≈ Hξ,μ,σ¢σ¡
1 − (− log(1 − 1/k))−ξ
ξ

(5.6)

By the invariance property of maximum likelihood estimation, given theMLEs for the parameters ξ, μ and σ, the MLE for Rn,k isR̂n,k = μ̂ −

σ̂ ³
ξ̂

1 − (− log(1 − 1/k))−ξ̂´An asymptotically valid confidence interval for Rn,k may be computed using
the delta method (see Greene 2000, p. 118) or from the concentrated/profile
log-likelihood function. Given that (5.6) is a highly nonlinear function ofσ, μ and ξ, the delta method is not recommended. Details of constructing
a confidence interval for Rn,k based on the profile log-likelihood are given
in chapter three of Coles (2001) and the appendix of McNeil (1998).
The return level probability in (5.5) is based on the GEV distributionHξ,μ,σ of the maxima Mn . For iid losses X with CDF F , Hξ,μ,σ ≈ F n sothat
1/n
F (Rn,k ) = Pr{X ≤ Rn,k } ≈ (1 − 1/k)
(5.7)
Hence, for iid losses the return level Rn,k is approximately the (1 − 1/k)
quantile of the loss distribution F .

1/nExample 26 Return levels for S&P 500 negative returns
Given the MLEs for the GEV distribution fit to the annual block maxima
of the (negative) daily returns on S&P 500 index, the question• What is the 40-year return level of the index returns?
may be answered using (5.6). The S+FinMetrics/EVIS function rlevel.gev
computes (5.6) as well as an asymptotic 95% confidence interval based on
the profile likelihood using the information from a “gev” object. To compute the 40 year return level for the S&P 500 returns from the “gev” object
gev.fit.year and to create a plot of the 95% confidence interval use> rlevel.year.40 = rlevel.gev(gev.fit.year, k.blocks=40,+
type="profile")> class(rlevel.year.40)[1] "list"> names(rlevel.year.40)[1] "Range" "rlevel"> rlevel.year.40$rlevel[1] 6.833

155

-40
-42

-41

parmax-39

5.2 Modeling Maxima and Worst Cases

5

10

15

20

25

30

rl

FIGURE 5.7. Asymptotic 95% confidence interval for the 40 year return levelbased on the profile log-likelihood function.

When type="profile", the function rlevel.gev returns a “list” object, containing the return level and range information used in the construction of the profile log-likelihood confidence interval, and produces aplot of the profile log-likelihood confidence interval for the return level.
The estimate of the 40 year return level is 6.83%. Assuming iid returns
and using (5.7), the estimated return level of 6.83% is approximately the
99.99% quantile of the daily return distribution. An asymptotic 95% confidence interval for the true return level is illustrated in Figure 5.7. Notice
the asymmetric shape of the asymptotic confidence interval. Although thepoint estimate of the return level is 6.83%, the upper endpoint of the 95%
confidence interval is about 21%. This number may seem large; however,on Monday October 19th 1987 S&P 500 index closed down 20.4%.
By default, the function rlevel.gev produces a plot of the asymptotic
95% confidence level. Alternatively, if rlevel.gev is called with the optional argument type="RetLevel":> rlevel.year.40 = rlevel.gev(gev.fit.year, k.blocks=40,+
type="RetLevel")> names(rlevel.year.40)[1] "LowerCB" "rlevel" "UpperCB"> rlevel.year.40$LowerCB:

5. Modeling Extreme Values

4
2

3

data

5

6

156

0

5

10

15

20

25

FIGURE 5.8. Estimated 40-year return level with 95% confidence band for the
S&P 500 daily negative returns.

[1] 4.646$rlevel:[1] 6.833$UpperCB:[1] 20.5
A plot of the estimated return level along with the block maxima, as
shown in Figure 5.8, is created, and the components of the returned list are
the estimated return level along with the end points of the 95% confidence
interval.
The 40 year return level may also be estimated from the GEV distribution fit to quarterly maxima. Since 40 years is 160 quarters, the 40 year
return level computed from the “gev” object gev.fit.quarter is> rlevel.160.q = rlevel.gev(gev.fit.quarter, k.blocks=160,+ type="RetLevel")> rlevel.160.q$LowerCB:[1] 4.433$rlevel:5.3 Modeling Extremes Over High Thresholds

157[1] 5.699$UpperCB:[1] 8.549
Here, the estimated return level and asymptotic 95% confidence interval
are smaller than the corresponding quantities estimated from annual data.

5.3 Modeling Extremes Over High Thresholds
Modeling only block maxima data is ineﬃcient if other data on extreme values are available. A more eﬃcient alternative approach that utilizes moredata is to model the behavior of extreme values above some high threshold.
This method is often called peaks over thresholds (POT). Another advantage of the POT approach is that common risk measures like Value-at-Risk(VaR) and expected shortfall (ES) may easily be computed3 .
To illustrate the concepts of VaR and ES, review the daily S&P 500
returns analyzed in the previous section. Suppose the S&P 500 is the only
asset in a large portfolio for an investor and that the random variable X
with CDF F represents the daily loss on the portfolio. The daily VaR on
the portfolio is simply a high quantile of the distribution F of daily losses.
For example, the daily 1% VaR on the portfolio is the 99% quantile of X
VaR.99 = F −1 (0.99).
That is, with 1% probability the loss in portfolio value over a day will
exceed VaR.99 . Often the high quantile VaR.99 is computed assuming X ∼N (μ, σ 2 ). In this case, the calculation of VaR.99 reduces to the simple
formulaVaR.99 = μ + σ · q.99
(5.8)
where q.99 is the 99% quantile of the standard normal distribution. The
distribution of daily portfolio losses, however, generally has fatter tails than
the normal distribution so that (5.8) may severely under-estimate VaR.99 .
Estimates of VaR based on the POT methodology are much more reliable.
The ES on the portfolio is the average loss given that VaR has been
exceeded. For example, the 1% ES is the conditional mean of X given that
X > VaR.99
ES.99 = E[X|X > VaR.99 ]3 Notice that VaR and ES are based on the distribution of the losses and not on the
distribution of the maximum losses. The analysis of block maxima based on the GEV
distribution allowed inferences to be made only on the maxima of returns. The POT
analysis will allow inferences to be made directly on the distribution of losses.

158

5. Modeling Extreme Values

150
100
50

Millions of Danish Krone

200

250

Fire Loss Insurance Claims

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

FIGURE 5.9. Large fire loss insurance claims.

If X ∼ N (μ, σ 2 ) then ES.99 may be computed as the mean of a truncatednormal random variable:
ES.99 = μ + σ ·φ(z)1 − Φ(z)(5.9)

where z = (VaR.99 − μ)/σ, φ(z) is the standard normal density functionand Φ(z) is the standard normal CDF. Again, if the distribution of losses
has fatter tails than the normal, then (5.9) will underestimate ES.99 . The
POT methodology estimates the distribution of losses over a threshold andproduces an estimate of ES as a by-product of the estimation.
For another example, consider the “timeSeries” danish representing
Danish fire loss data in S+FinMetrics, which is analyzed in McNeil (1999).
The data in danish consist of 2167 daily insurance claims for losses exceeding one million Danish Krone from January 3, 1980 through December 31,
1990. The reported loss is an inflation adjusted total loss for the event concerned and includes damages to buildings, damage to contents of buildings
as well as loss of profits. Figure 5.9 created using> plot(danish, ain="Fire Loss Insurance Claims",+ ylab="Millions of Danish Krone")shows the data and reveals several extreme losses. For risk management
purposes, insurance companies may be interested in the frequency of occurrence of large claims above some high threshold as well as the average value5.3 Modeling Extremes Over High Thresholds

159of the claims that exceed the high threshold. Additionally, they may be interested in daily VaR and ES. The statistical models for extreme values
above a high threshold may be used to address these issues.

5.3.1 The Limiting Distribution of Extremes Over High
Thresholds and the Generalized Pareto Distribution
As with the analysis of block maxima, let X1 , X2 , . . . be a sequence ofiid random variables representing risks or losses with an unknown CDF
F and let Mn = max{X1 , . . . , Xn }. A natural measure of extreme eventsare values of the Xi that exceed a high threshold u. Define the excess
distribution above the threshold u as the conditional probability:
Fu (y) = Pr{X − u ≤ y|X > u} =F (y + u) − F (u), y>0
1 − F (u)(5.10)For the class of distributions F such that the CDF of the standardized
value of Mn converges to a GEV distribution (5.2), it can be shown (c.f.
Embrechts et. al. (1997)) that for large enough u there exists a positive
function β(u) such that the excess distribution (5.10) is well approximatedby the generalized Pareto distribution (GPD)(³´
1 − 1 + (ξy/β(u))−1/ξ for ξ 6= 0Gξ,β(u) (y) =, β(u) > 0 (5.11)
1 − exp(−y/β(u)) for ξ = 0
defined for y ≥ 0 when ξ ≥ 0 and 0 ≤ y ≤ −β(u)/ξ when ξ < 0.
Remarks:• Operationally, for a suﬃciently high threshold u, Fu (y) ≈ Gξ,β(u) (y)for a wide class of loss distributions F . To implement this result, thethreshold value u must be specified and estimates of the unknown
parameters ξ and β(u) must be obtained.
• There is a close connection between the limiting GEV distribution
for block maxima and the limiting GPD for threshold excesses. For agiven value of u, the parameters ξ, μ and σ of the GEV distribution
determine the parameters ξ and β(u). In particular, the shape parameter ξ of the GEV distribution is the same shape parameter ξ in
the GPD and is independent of the threshold value u. Consequently,
if ξ < 0 then F is in the Weibull family and Gξ,β(u) is a Pareto type
II distribution; if ξ = 0 then F is in the Gumbell family and Gξ,β(u)is an exponential distribution; and if ξ > 0 then F is in the Fréchetfamily and Gξ,β(u) is a Pareto distribution.
• For ξ > 0, the most relevant case for risk management purposes, it
can be shown that E[X k ] = ∞ for k ≥ α = 1/ξ. For example, if

160

5. Modeling Extreme Values

ξ = 0.5 then E[X 2 ] = ∞ and the distribution of losses, X, does nothave finite variance. If ξ = 1 then E[X] = ∞.
• Consider a limiting GPD with shape parameter ξ and scale parameter β(u0 ) for an excess distribution Fu0 with threshold u0 . For an
arbitrary threshold u > u0 , the excess distribution Fu has a limiting GPD distribution with shape parameter ξ and scale parameterβ(u) = β(u0 )+ξ(u−u0 ). Alternatively, for any y > 0 the excess distribution Fu0 +y has a limiting GPD distribution with shape parameterξ and scale parameter β(u0 ) + ξy.
Example 27 Plots of GPDs
The S+FinMetrics/EVIS functions pgpd, qgpd, dgpd and rgpd compute
cumulative probability, quantiles, density and random number generation,
respectively, from the GPD (5.11) for ξ 6= 0 and general values for β(u).
For example, the S-PLUS code to compute and plot the CDFs and pdfs
with β(u) = 1 for a Pareto (ξ = 0.5), exponential (ξ = 0) and Pareto type
II (ξ = −0.5) is> par(mfrow=c(1,2))> y.vals = seq(0,8,length=200)> cdf.p = pgpd(y.vals, xi=0.5)> cdf.p2 = ifelse((y.vals < 2), pgpd(y.vals,xi=-0.5), 1)> cdf.e = 1-exp(-z.vals)> plot(y.vals, cdf.p, type="l", xlab="y", ylab="G(y)",+ ylim=c(0,1))> lines(y.vals, cdf.e, type="l", lty=2)> lines(y.vals, cdf.p2, type="l", lty=3)> legend(1,0.2,legend=c("Pareto G(0.5,1)","Exponential G(0,1)",+ "Pareto II G(-0.5,1)"),lty=1:3)> # PDFs> pdf.p = dgpd(y.vals, xi=0.5)> pdf.p2 = ifelse((y.vals < 2), dgpd(y.vals,xi=-0.5), 0)> pdf.e = exp(-y.vals)> plot(y.vals, pdf.p, type="l", xlab="y", ylab="g(y)",+ ylim=c(0,1))> lines(y.vals, pdf.e, type="l", lty=2)> lines(y.vals, pdf.p2, type="l", lty=3)> legend(2,1,legend=c("Pareto g(0.5,1)","Exponential g(0,1)",+ "Pareto II g(-0.5,1)"),lty=1:3)
The CDFs and pdfs are illustrated in Figure 5.10. Notice that the Pareto
type II is only defined for y < 2.
Example 28 qq-plots to determine tail behavior

161

1.0

1.0

5.3 Modeling Extremes Over High Thresholds

0.8
0.6
0.4
0.2

0.2

0.4

g(y)

G(y)

0.6

0.8

Pareto g(0.5,1)
Exponential g(0,1)
Pareto II g(-0.5,1)

0.0

0.0

Pareto G(0.5,1)
Exponential G(0,1)
Pareto II G(-0.5,1)

0

2

4
y

6

8

0

2

4

6

8

y

FIGURE 5.10. Generalized Pareto CDFs, Gξ,1 , and pdfs, gξ,1 , for Pareto(ξ = 0.5), exponential (ξ = 0) and Pareto type II (ξ = −0.5).

A simple graphical technique to infer the tail behavior of observed losses
is to create a qq-plot using the exponential distribution as a reference distribution. If the excesses over thresholds are from a thin-tailed distribution,then the GPD is exponential with ξ = 0 and the qq-plot should be linear.
Departures from linearity in the qq-plot then indicate either fat-tailed behavior (ξ > 0) or bounded tails (ξ < 0). The S+FinMetrics/EVIS function
qplot may be used to create a qq-plot using a GPD as a reference distribution. For example, to create qq-plots with the exponential distribution
as the reference distribution for the S&P 500 negative returns over thethreshold u = 1 and the Danish fire loss data over the threshold u = 10,
use> par(mfrow=c(1,2))> qplot(-spto87, threshold=1, main="S&P 500 negative returns")> qplot(danish, threshold=10, main="Danish fire losses")
Figure 5.11 shows these qq-plots. There is a slight departure from linearity
for the negative S&P 500 returns and a rather large departure from linearity
for the Danish fire losses.

162

5. Modeling Extreme Values
Danish fire losses

3

Exponential Quantiles

2

4

0

0

1

2

Exponential Quantiles

4

6

5

S&P 500 negative returns

1

2

3

4

5

6

0

50

100

Ordered Data

150

200

250

Ordered Data

FIGURE 5.11. QQ-plots with exponential reference distribution for the S&P 500
negative returns over the threshold u = 1 and the Danish fire losses over thethreshold u = 10.

Mean Excess Function
Suppose the threshold excess X − u0 follows a GPD with parameters ξ < 1
and β(u0 ). Then the mean excess over the threshold u0 is
E[X − u0 |X > u0 ] =β(u0 ).
1−ξ

(5.12)For any u > u0 , define the mean excess function e(u) ase(u) = E[X − u|X > u] =Alternatively, for any y > 0β(u0 ) + ξ(u − u0 ).
1−ξe(u0 + y) = E[X − (u0 + y)|X > u0 + y] =β(u0 ) + ξy.
1−ξ(5.13)(5.14)

Notice that for a given value of ξ, the mean excess function is a linear
function of y = u − u0 . This result motivates a simple graphical way toinfer the appropriate threshold value u0 for the GPD. Define the empiricalmean excess functionnu
1 X
en (u) =(x(i) − u)nu i=1(5.15)

163

3
1

2

Mean Excess

455.3 Modeling Extremes Over High Thresholds

-4

-2

0

2

4

Threshold

FIGURE 5.12. Mean excess plot for the S&P 500 negative returns.

where x(i) (i = 1, . . . , nu ) are the values of xi such that xi > u. The mean
excess plot is a plot of en (u) against u and should be linear in u for u > u0 .
An upward sloping plot indicates heavy-tailed behavior. In particular, astraight line with positive slope above u0 is a sign of Pareto behavior in
tail. A downward trend shows thin-tailed behavior, whereas a line with zero
slope shows an exponential tail.
Example 29 Mean excess plots for S&P 500 and fire loss data
The S+FinMetrics/EVIS function meplot computes the empirical mean
excess function (5.15) and creates the mean excess plot. The mean excess
functions and mean excess plots for the S&P 500 negative returns and theDanish fire losses are computed using> me.sp500 = meplot(-spto87)> me.dainsh = meplot(danish)> class(me.sp500)[1] "data.frame"> colIds(me.sp500)[1] "threshold" "me"The function meplot returns a data frame containing the thresholds u
and the mean excesses en (u) and produces a mean excess plot. The mean
excess plots for the S&P 500 and Danish data are illustrated in Figures5. Modeling Extreme Values

60
0

20

40

Mean Excess

80

100

120

164

0

10

20

30

40

50

60

Threshold

FIGURE 5.13. Mean excess plot for the Danish fire loss data.

5.12 and 5.13. The mean excess plot for the S&P 500 negative returns islinear in u with positive slope for u > 1 indicating Pareto tail behavior.
The plot for the fire loss data is upward sloping and linear for almost allvalues of u. However, there is a slight kink at u = 10.

5.3.2 Estimating the GPD by Maximum Likelihood
Let x1 , . . . , xn be iid sample of losses with unknown CDF F . For a givenhigh threshold u, extreme values are those xi values for which xi − u >0. Denote these values x(1) , . . . , x(k) and define the threshold excesses as
yi = x(i) − u for i = 1, . . . , k. The results of the previous section imply
that if u is large enough then {y1 , . . . , yk } may be thought of as a random
sample from a GPD with unknown parameters ξ and β(u). For ξ 6= 0, thelog-likelihood function based on (5.11) isl(ξ, β(u)) = −k ln(β(u)) − (1 + 1/ξ)kXln(1 + ξyi /β(u))

i=1provided yi ≥ 0 when ξ > 0 and 0 ≤ yi ≤ −β(u)/ξ when ξ < 0. For ξ = 0
the log-likelihood function isl(β(u)) = −k ln(β(u)) − β(u)−1

k
X
i=1

yi .

5.3 Modeling Extremes Over High Thresholds

165

5.3.3 Estimating the Tails of the Loss Distribution
For a suﬃciently high threshold u, Fu (y) ≈ Gξ,β(u) (y). Using this resultin (5.10) and setting x = u + y, an approximation to the tails of the loss
distribution F (x) for x > u is given by
F (x) = (1 − F (u))Gξ,β(u) (y) + F (u)(5.16)The CDF value F (u) may be estimated non-parametrically using the empirical CDF(n − k)F̂ (u) =(5.17)nwhere k denotes the number of exceedences over the threshold u. Combining the parametric representation (5.11) with the non-parametric estimate(5.17) gives the resulting estimate of (5.16)
Ã
!
x−u
k
1 + ξ̂ ·(5.18)F̂ (x) = 1 −nβ̂(u)where ξ̂ and β̂(u) denote the MLEs of ξ and β(u), respectively.
Example 30 Estimating the GPD for the S&P 500 negative returns
Maximum likelihood estimation of the parameters ξ and β(u) of the GPD(5.11) may be computed using the S+FinMetrics/EVIS function gpd. Inorder to compute the MLE, a threshold value u must be specified. The
threshold should be large enough so that the GPD approximation is valid
but low enough so that a suﬃcient number of observations k are available
for a precise fit.
To illustrate, consider fitting GPD to the negative returns on the S&P
500 index. The S+FinMetrics/EVIS function gpd may be used to computethe MLEs for the GPD (5.11) for a given threshold u. The mean excess
plot for the S&P 500 returns in Figure 5.12 suggests a value of u = 1 may
be appropriate for the GPD approximation to be valid. The MLE usingu = 1 is computed using> gpd.sp500.1 = gpd(-spto87, threshold=1)> class(gpd.sp500.1)[1] "gpd"
The function gpd returns an object of class “gpd” for which there areprint and plot methods. The components of a “gpd” object are numerous,
and are described in the online help for gpd.object.
The MLEs for ξ and β(1) and asymptotic standard errors are> gpd.sp500.1
Generalized Pareto Distribution Fit --166

5. Modeling Extreme Values

Total of

6985

observations

Upper Tail Estimated with ml -Upper Threshold at 1 or 8.518 % of the data
ML estimation converged.
Log-likelihood value: -183.6

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.0677 0.0397
1.7033beta 0.4681 0.0267
17.5376
Notice that ξ̂ = 0.068 is fairly close to zero and indicates that the return
distribution is not so heavy-tailed. Also, the GPD estimate of ξ is quite abit smaller than the GEV estimate ξ̂ = 0.334 based on annual data, but it
is very close to the GEV estimate ξ̂ = 0.069 based on quarterly data.
Diagnostic plots of the GDP fit are created using the plot method> plot(gpd.sp500.1)
Make a plot selection (or 0 to exit):
1: plot: Excess Distribution
2: plot: Tail of Underlying Distribution
3: plot: Scatterplot of Residuals
4: plot: QQplot of Residuals
Selection:
The four plot options are depicted in Figure 5.14. The first plot option
shows the GPD estimate of the excess distribution, and the second plot
option shows the tail estimate (5.18). The GPD appears to fit the distribution of threshold excesses fairly well. Note, the S+FinMetrics/EVIS
function tailplot may be used to compute plot option 2 directly.
The S+FinMetrics/EVIS function shape can be used to create a plot
showing how the MLE of the shape parameter ξ varies with the selected
threshold u:> shape(-spto87, end=600)
The optional argument end=600 specifies the maximum number of threshold exceedences to consider. The resulting plot is shown in Figure 5.15. The
estimates of ξ are fairly stable and close to zero for threshold values lessthan 1.2, and increase slightly for threshold values between 1.2 and 2.
The S+FinMetrics/EVANESCE function shape.plot may also be used to
produce a plot similar to 5.15. For example> shape.plot(-spto87, from = 0.9, to = 0.98)

167

0.0

10^-2
10^-6

10^-4

1-F(x) (on log scale)

0.6
0.4
0.2

Fu(x-u)0.8

1.0

5.3 Modeling Extremes Over High Thresholds

1

5

10

1

5

10

x (on log scale)

6
4
0

2

4
0

2

Residuals

6

Exponential Quantiles

8x (on log scale)

0

100

200

300

400

500

Ordering

600

0

2

4

6

8

Ordered Data

FIGURE 5.14. Diagnostic plots for GPD fit to daily negative returns on S&P 500
index.

estimates ξ for threshold values starting at 90th percentile and endingat the 98th percentile of the data.
Example 31 Estimating the GPD for lower and upper tails of S&P 500
returns
Sometimes it is desirable to estimate the parameters ξ and β(u) of the
GPD (5.11) separately for the lower and upper tails. This may be doneusing the S+FinMetrics/EVANESCE function gpd.tail. In order to compute the MLE, threshold values ulower and uupper must be specified. The
previous analysis found ulower = −1. A guess for the upper threshold may
be obtained from the mean excess plot
> me.sp500 = meplot(spto87)
illustrated in Figure 5.16. For uupper = 1, the plot appears linear with apositive slope indicating Pareto tail behavior.
The two-tailed MLEs may then be computed using> gpd.sp500.2tail = gpd.tail(spto87, upper = 1, lower = -1,+
plot = T)> class(gpd.sp500.2tail)[1] "gpd"> gpd.sp500.2tail168

5. Modeling Extreme Values
Threshold
1.050

600

559 519

1.100

1.160

1.270

1.360

1.460

1.570

1.810

2.190

115

75

1.0
0.5
0.0

Shape (xi) (CI, p = 0.95)

1.5

0.992

478 438 398

357 317 277

236 196 156

35

Exceedances

FIGURE 5.15. Estimates of shape parameter ξ for S&P 500 negative returns as
a function of the threshold value u.

Generalized Pareto Distribution Fit -Total of

6985

observations

Upper Tail Estimated with ml -Upper Threshold at 1 or 8.533 % of the data
ML estimation converged.
Log-likelihood value: -277.7

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.0415
NANA
beta 0.5624
NANA
Lower Tail Estimated with ml -Lower Threshold at -1 or 8.518 % of the data
ML estimation converged.
Log-likelihood value: -183.6

Parameter Estimates, Standard Errors and t-ratios:

169

4
3
0

1

2

Mean Excess

56

5.3 Modeling Extremes Over High Thresholds

-6

-4

-2

0

2

4

Threshold

FIGURE 5.16.

Value Std.Error t value
xi 0.0677
NANA
beta 0.4681
NANA
The lower tail estimates are the same as in the previous example4 . The
upper tail estimates are similar. The optional argument plot=T, produces
qq-plots of excesses over the specified thresholds versus GPD quantiles
using the estimated lower and upper tail shape parameters. The linear
nature of these plots, given in Figure 5.17, supports the assumption thatthe lower and upper excesses have GPD distributions.
Example 32 Estimating the GPD for the Danish fire loss data
The mean excess plot in Figure 5.13 suggests a threshold value of u = 10.
The MLEs of the GPD parameters for the Danish fire loss data using a highthreshold of 10 million Krone are computed using> gpd.danish.10 = gpd(danish, threshold=10)> gpd.danish.10
Generalized Pareto Distribution Fit --4 Currently, the function gpd.tail does not compute the estimated covariance matrix
for the estimated parameters. To get standard errors, use the function gpd on both tails
separately.

170

5. Modeling Extreme Values

3
2
1
0

Excess over threshold4

Upper Tail

0

2

4

6

8

GPD Quantiles, for xi = 0.0414854815615611

4
3
2
1
0

Excess over threshold

5

Lower Tail

0

2

4

6

8

GPD Quantiles, for xi = 0.0676714921070174

FIGURE 5.17.

Total of2167

observations

Upper Tail Estimated with ml -Upper Threshold at 10 or 5.03 % of the data
ML estimation converged.
Log-likelihood value: -374.9

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.4970 0.1363
3.6467beta 6.9755 1.1135
6.2645The estimate of ξ shows heavy tails and suggests that the variance may
not be finite. The diagnostic plots in Figure 5.18, created using> par(mfrow=c(1,2))> tailplot(gpd.danish.10)> shape(danish)show that the GPD fits the data well and that the estimates of ξ are fairly
stable for a wide range of threshold values.

5.3 Modeling Extremes Over High Thresholds

171

Threshold
3.66

4.17

5.08

6.59

12.40

0.00005

0.0

0.5

Shape (xi) (CI, p = 0.95)

0.00500
0.000501-F(x) (on log scale)

1.0

0.05000

3.13

10

50

100

500 449 399 349 299 249 198 148

x (on log scale)98

48

Exceedances

FIGURE 5.18. Diagnostic plots from GPD fit to Danish fire loss data.

5.3.4 Risk Measures
As mentioned in the introduction to this section, two common risk measures
are Value-at-Risk (VaR) and expected shortfall (ES). VaR is a high quantileof the loss distribution. That is, for 0.95 ≤ q < 1, say, VaRq is the qth
quantile of the distribution F
VaRq = F −1 (q)(5.19)

where F −1 is the inverse of F . For a given probability q > F (u), an estimateof (5.19) based on inverting the tail estimation formula (5.18) is
d q = u + β̂(u)
VaR
ξ̂

µ³
n
k

(1 − q)

´−ξ̂

¶
−1

(5.20)

Expected shortfall is the expected loss size, given that VaRq is exceeded
ESq = E[X|X > VaRq ](5.21)The measure ESq is related to VaRq via
ESq = VaRq + E[X − VaRq |X > VaRq ].

(5.22)

where the second term in (5.22) is simply the mean of the excess distribution FVaRq (y) over the threshold VaRq . By the translation property of

172

5. Modeling Extreme Values

the GPD distribution, the GPD approximation to FVaRq (y) has shape parameter ξ and scale parameter β(u) + ξ(VaRq − u). Consequently, using(5.13)
β(u) + ξ(VaRq − u)E[X − VaRq |X > VaRq ] =(5.23)
1−ξprovided ξ < 1. Combining (5.23) with (5.20) and substituting into (5.22)
gives the GPD approximation to ESqdc q = VaRq + β̂(u) − ξ̂u .
ES
1 − ξ̂
1 − ξ̂

(5.24)

Example 33 Computing VaR and ES for negative S&P 500 returns
The S+FinMetrics/EVIS function riskmeasures computes estimates of
VaRq and ESq based on the GPD approximations (5.20) and (5.24), respectively, using the information from a “gpd” object. For example, theVaRq and ESq estimates for the negative S&P 500 negative returns for
q = 0.95, 0.99 are computed using> riskmeasures(gpd.sp500.1, p = c(0.95,0.99))
p quantile sfall[1,] 0.95
1.2539 1.7744[2,] 0.99
2.0790 2.6594
That is, with 5% probability the daily return could be as low as −1.254%
and, given that the return is less than 1.254%, the average return value is−1.774%. Similarly, with 1% probability the daily return could be as low
as −2.079% with an average return of −2.659% given that the return isless than −2.079%.
It is instructive to compare these results to those based on the assumption of normally distributed returns. Using the formulas (5.8) and (5.9),
estimates of VaRq and ESq for q = 0.95, 0.99 may be computed using thefollowing function
riskmeasures.normal <- function(data,p=c(0.95,0.99)) {mu = colMeans(data)
sd = colStdevs(data)q = mu + sd*qnorm(p)sq = (q - mu)/sd
sf = mu + sd*dnorm(sq)/(1 - pnorm(sq))
cbind(p, quantile = q, sfall = sf)}> riskmeasures.normal(-spto87)
p quantilesfall[1,] 0.95 1.299051 1.635526[2,] 0.99 1.847814 2.120681

173

10^-3
10^-4

95

99

10^-6

10^-5

1-F(x) (on log scale)

10^-2

10^-1

5.3 Modeling Extremes Over High Thresholds

1

5

10

x (on log scale)

FIGURE 5.19.

The estimates of VaRq and ESq based on the normal distribution are fairly
close to the estimates based on the GPD for q = 0.95. For q = 0.99, VaRq
and ESq based on the normal distribution are a bit smaller than the values
based on the GPD.
Estimates and asymptotically valid confidence intervals for VaRq and
ESq may be computed using the S+FinMetrics/EVIS function gpd.q and
gpd.sfall, respectively. Wald-type confidence intervals based on the delta
method or likelihood ratio-type confidence intervals based on the profile
log-likelihood function may be computed, and these confidence intervals
may be visualized on a plot with the tail estimate (5.18). First create plotof the excess distribution using the S+FinMetrics/EVIS function tailplot> tailplot(gpd.sp500.1)
After the plot has been created, the asymptotic confidence intervals for
VaRq and ESq may be added using> gpd.q(0.99,plot=T)> gpd.sfall(0.99,plot=T)
The combined plots are illustrated in Figure 5.19.
Notice the slightly asymmetric confidence interval for ES.99 . This result
is due to the uncertainty created by only a few observations in the extreme
tails of the distribution.

174

5. Modeling Extreme Values
Threshold
1.12

1.21

500

466 433

1.27

1.36

1.45

1.54

1.65

1.87

2.26

2.6
2.4
2.2
2.0

0.99 Quantile (CI, p = 0.95)

2.8

1.08

399 366

332 299

265 232

198 165 132

98

65

31

Exceedances

FIGURE 5.20.

The sensitivity of the VaRq estimates to changes in the threshold u may
be investigated using the S+FinMetrics/EVIS function quant. For example, to see how the VaR.99 estimates vary with u, use> quant(-spto87,p=0.99)
which produces the graph in Figure 5.20.
The VaR.99 estimates are stable for u < 2.

5.4 Hill’s Non-parametric Estimator of Tail Index
The shape parameter ξ, or equivalently, the tail index a = 1/ξ, of the GEV
and GPD distributions (5.2) and (5.11) may be estimated non-parametricallyin a number of ways. A popular method due to Hill (1975) applies to thecase where ξ > 0 (α > 0) so that the data is generated by some fat-tailed
distribution in the domain of attraction of a Fréchet type GEV. To describe
the Hill estimator, consider a sample of losses X1 , . . . , XT and define theorder statistics asX(1) ≥ X(2) ≥ · · · ≥ X(T )For a positive integer k, the Hill estimator of ξ is defined as
ξ̂

Hill

k

(k) =

¢
1 X¡
log X(j) − log X(k)k j=1

(5.25)

5.4 Hill’s Non-parametric Estimator of Tail Index

175

and the Hill estimator of α isα̂Hill (k) = 1/ξ̂

Hill

(k)

(5.26)The Hill estimators of ξ and α depend on the integer k. Notice that k in
(5.26) plays the same role as k in (5.17) for the analysis of the GPD. It canbe shown that if F is in the domain of attraction of a GEV distribution,
Hillthen ξ̂ (k) converges in probability to ξ as k → ∞ and nk → 0, and that
ξ̂

Hill

(k) is asymptotically normally distributed with asymptotic varianceavar(ξ̂Hill(k)) =

ξ2k

By the delta method, α̂Hill (k) is asymptotically normally distributed withasymptotic varianceα2avar(α̂Hill (k)) =k
Hill

In practice, the Hill estimators ξ̂ (k) or α̂Hill (k) are often plotted against
k to find the value of k such that the estimator appears stable.

5.4.1 Hill Tail and Quantile Estimation
Suppose that the loss distribution F is such that 1 − F (x) = x−α L(x) with
α = 1/ξ > 0, where L(x) is a slowly varying function. Let x > X(k+1)
where X(k+1) is a high order statistic. Then the Hill estimator of F (x) is
given by
F̂

Hill

k
(x) = 1 −nµxX(k+1) )¶−α̂Hill (k), x > X(k+1)(5.27)

Inverting the Hill tail estimator (5.27) gives the Hill quantile estimatorx̂Hillq,k = X(k+1) − X(k+1)

Ã
³n
k

!
´−ξ̂Hill (k)−1(1 − q)(5.28)

where q > 1 − k/n. The Hill quantile estimator (5.28) is very similar to the
ML GPD quantile estimator (5.20) with u = X(k+1) .
Example 34 Nonparametric estimation of ξ for Danish fire loss data
The Hill estimates of α, ξ and the quantile xq,k may be computed andplotted using the S+FinMetrics/EVIS function hill. The arguments expected by hill are176

5. Modeling Extreme Values> args(hill)function(data, option = "alpha", start = 15, end = NA,
p = NA, ci = 0.95, plot = T, reverse = F,auto.scale = T, labels = T, ...)
where data is a univariate numeric vector or “timeSeries”, option determines if α (“alpha”), ξ (“xi”) or xq,k (“quantile”) is to be computed,
start and end specify the starting and ending number of order statistics to
use in computing the estimates, p specifies the probability required when
option=\quantile", ci determines the probability for asymptotic confidence bands, and plot determines if a plot is to be created. To illustrate
the use of hill, consider the computation of (5.25) for the Danish fire loss
data using all of the order statistics less than X(15)> hill.danish = hill(danish, option="xi")> class(hill.danish)[1] "data.frame"> names(hill.danish)[1] "xi""orderStat" "threshold"
The function hill returns a data frame with components xi containing
the estimates of ξ, orderStat containing the order statistic labels k, andthreshold containing the order statistic or threshold values X(k) . Since
the default option plot=T is used, hill also produces the plot shown in
Hill

Figure 5.21. For k > 120 (X(k) < 9), ξ̂ (k) is fairly stable around 0.7.
The GPD estimate of ξ with threshold u = 10 is 0.497. The Hill estimates
for threshold values near 10 are> idx = (hill.danish$threshold >= 9.8 &+ hill.danish$threshold <= 10.2)> hill.danish[idx,]
xi orderStat threshold
2059 0.6183
109
9.883
2060 0.6180
108
10.0112061 0.6173
107
10.0722062 0.6191
106
10.1372063 0.6243
105
10.1782064 0.6285
104
10.185The 99% quantile estimates (5.28) for 15 ≤ k ≤ 500 are computed using> hill.danish.q = hill(danish, option="quantile", p=0.99,+ end=500)
and are illustrated in Figure 5.22.

5.4 Hill’s Non-parametric Estimator of Tail IndexThreshold
5.56

3.65

2.67

2.14

1.85

1.65

1.47

1.32

1.20

1.07

0.7
0.6
0.3

0.4

0.5

xi (CI, p =0.95)

0.8

0.9

29.00

15 132 266 400 534 668 802 936 1087 1254 1421 1588 1755 1922 2089
Order Statistics

FIGURE 5.21. Hill estimates of ξ for the Danish fire loss data.

Threshold
14.70

10.10

7.10

5.77

5.19

4.61

4.15

3.85

3.55

3.26

28
27
26
24

25

Quantile, p = 0.99

29

30

29.00

15 38 61 84 111 142 173 204 235 266 297 328 359 390 421 452 483
Order Statistics

FIGURE 5.22. Hill estimates of 1% quantile of Danish fire loss data.

177

178

5. Modeling Extreme Values

5.5 References
Alexander, C. (2001). Market Models: A Guide to Financial Data Analysis. John Wiley & Sons, Chichester, UK.
Carmona, R. (2004). Statistical Analysis of Financial Data in S-PLUS.
Springer-Verlag, New York.
Carmona, R. and J. Morrisson (2001). “Heavy Tails and Copulas with
Evanesce,” ORFE Tech. Report, Princeton University.
Coles, S (2001). An Introduction to Statistical Modeling of Extreme Values. Springer-Verlag, London.
Danielsson, J. and C.G. De Vries (1997). “Tail Index and Quantile Estimation with Very High Frequency Data,” Journal of Empirical Finance,
4, 241-257.
Diebold, F.X., T. Schuermann, and J.D. Stroughair (1997), “Pitfalls and Opportunities in the Use of Extreme Value Theory in Risk Management,” Advances in Computational Management Science 2, 3-12.
Embrechts, P. C. Kloppelberg, and T. Mikosch (1997). Modelling
Extremal Events. Springer-Verlag, Berlin.
Embrechts, P., A. McNeil, and D. Straumann (2000). “Correlation and Dependence in Risk Management: Properties and Pitfalls,” in M.
Dempster and H. K. Moﬀatt (eds.), Risk management: value at risk andbeyond. Cambridge University Press, Cambridge.
Fisher, R. and L. Tippett (1928). “Limiting Forms of the Frequency
Distribution of the Largest or Smallest Member of a Sample,” Proceedingsof the Cambridge Philosophical Society 24, 180-190.
Greene, W. (2000). Econometric Analysis, Fourth Edition. Prentice Hall,
Upper Saddle River.
Hill, B.M. (1975). “A Simple General Approach to Inference about the
Tail of a Distribution,” The Annals of Statistics, 3, 1163-1174.
Jorian, P. (2001). Value at Risk, Second Edition. McGraw-Hill, New York.
Longin, F.M. (2000). “From Value-at-Risk to Stress Testing: The Extreme
Value Approach,” Journal of Banking an Finance 24, 1097-1130.

5.5 References

179

McNeil, A.J. (1998). “On Extremes and Crashes,” RISK, January, page
99.
McNeil, A.J. (1998). “Calculating Quantile Risk Measures for Financial
Returns using Extreme Value Theory,” ETH E-Collection
http://e-collection.ethbib.ethz.ch/show?type=bericht&nr=85.
McNeil, A.J. (1999). “Extreme Value Theory for Risk Managers,” inInternal Modelling and CAD II, RISK Books, 93-113.
McNeil, A.J. and R. Frey (2000). “Estimation of Tail-Related Risk
Measures for Heteroskedastic Financial Time Series: An Extreme Value
Approach,” Journal of Empirical Finance 7, 271-300.
McNeil A.J. and T. Saladin (2000). “Developing Scenarios for Future
Extreme Losses Using the POT Method,” in P. Embrechts (ed.) Extremes
and Integrated Risk Management , RISK books, London.
Morrison, J. E. (2001). Extreme Value Statistics with Apllications in
Hydrology and Financial Engineering, Ph.D. thesis, Princeton University.
RiskMetrics, 1995. “RiskMetrics Technical Document,” J.P. Morgan, 3rded.
Tsay, R.S. (2001). Analysis of Financial Time Series, John Wiley & Sons,
New York.

This is page 180
Printer: Opaque thisThis is page 181
Printer: Opaque this6
Time Series Regression Modeling

6.1 Introduction
Time series regression techniques are widely used in the analysis of financial
data and for estimating and testing models for asset prices and returns likethe capital asset pricing model and the arbitrage pricing model. They areused to uncover and exploit predictive relationships between financial variables. For example, the predictability of asset returns using valuation ratios
like dividend/price, earnings/price and book/market is usually establishedusing time series regression techniques, and the resulting regression models are used to forecast future returns. Time series regression techniques
are also used for testing the informational eﬃciency of financial markets.
Market eﬃciency often implies that certain financial variables should not
be predictable based on observable information, and time series regression
techniques may be used to verify eﬃciency implications.
Regression modeling with financial time series requires some care because
the time series properties of the data can influence the properties of standard regression estimates and inference methods. In general, standard regression techniques are appropriate for the analysis of I(0)/stationary data.
For example, asset returns are often treated as stationary and ergodic, andstandard regression techniques are then used to estimate models involving
asset returns. For nonstationary trending data like asset prices, however,standard regression techniques may or may not be appropriate dependingon the nature of the trend. This chapter discusses regression modeling techniques appropriate for I(0)/stationary and introduces and illustrates the

182

6. Time Series Regression Modelinguse of various S+FinMetrics functions designed for time series regression
analysis.
The rest of the chapter is organized as follows: Section 6.2 gives an
overview of the linear time series regression model and covers estimation,
goodness of fit, inference and residual diagnostics. Section 6.3 introduces
the S+FinMetrics function OLS that extends the S-PLUS linear model function lm to handle general time series regression and illustrates the use of
OLS through examples. Section 6.4 reviews dynamic regression models involving distributed lags of the dependent and explanatory variables andgives examples of how OLS may be used analyze these models. Section 6.5
discusses heteroskedasticity and autocorrelation consistent coeﬃcient covariance matrices and their use in constructing robust standard errors for
estimated regression coeﬃcients. Section 6.6 ends the chapter with a discussion of recursive regression techniques for assessing the parameter stabilityof time series regression models.
In this chapter, the technical details of time series regression are kept toa minimum. Excellent treatments of time series regression models from an
econometric perspective are given in Hamilton (1994) and Hayashi (2000).
Many applications of time series regression to financial data can be foundin Mills (1999).

6.2 Time Series Regression Model
Consider the linear time series regression modelyt = β 0 + β 1 x1t + · · · + β k xkt + εt = x0t β + εt , t = 1, . . . , T(6.1)

where xt = (1, x1t , . . . , xkt )0 is a (k + 1) × 1 vector of explanatory variables,
β = (β 0 , β 1 , . . . , β k )0 is a (k+1)×1 vector of coeﬃcients, and εt is a random
error term. In matrix form the model is expressed as
y = Xβ + ε(6.2)

where y and ε are (T × 1) vectors and X is a (T × (k + 1)) matrix.
The standard assumptions of the time series regression model are (e.g.,
Hayashi 2000, Chaps. 1 and 2):• The linear model (6.1) is correctly specified.
• {yt , xt } is jointly stationary and ergodic.
• The regressors xt are predetermined : E[xis εt ] = 0 for all s ≤ t andi = 1, . . . , k.
• E[xt x0t ] = ΣXX is of full rank k + 1.

6.2 Time Series Regression Model183• {xt εt } is an uncorrelated process with finite (k+1)×(k+1) covariancematrix E[ε2t xt x0t ] = S = σ 2 ΣXX .
The second assumption rules out trending regressors, the third rules out
endogenous regressors but allows lagged dependent variables, the fourth
avoids redundant regressors or exact multicolinearity, and the fifth implies
that the error term is a serially uncorrelated process with constant unconditional variance σ 2 . In the time series regression model, the regressors xt are
random and the error term εt is not assumed to be normally distributed.

6.2.1 Least Squares Estimation
Ordinary least squares (OLS) estimation is based on minimizing the sumof squared residuals
T
T
X
X
0
2
SSR(β) =(yt − xt β) =
ε2t
t=1

t=1

and produces the fitted modelyt = x0t β̂ + ε̂t , t = 1, . . . , T
0−1

where β̂ = (X X) X0 y and ε̂t = yt − ŷt = yt − x0t β̂. The error variance isestimated as σ̂ 2 = ε̂0 ε̂/(T − k − 1).
Under the assumptions described above, the OLS estimates β̂ are consistent and asymptotically normally distributed. A consistent estimate of
the asymptotic variance of β̂, avar(β̂), is given by1avar(d β̂) = σ̂2 (X0 X)−1

(6.3)

c β̂ i ), are given
Estimated standard errors for β̂ i (i = 0, ..., k), denoted SE(
by the square root of the diagonal elements of (6.3).

6.2.2 Goodness of Fit
Goodness of fit is summarized by the R2 of the regression
R2 = 1 −ε̂0 ε̂
(y − ȳ1)0 (y − ȳ1)

1 The following convention is used throughout this book. A consistent and asymptot√dddenotes convergence
ically normal estimator β̂ satisfies T (β̂ − β) → N(0,√ V) where → −1
in distribution. Call V the asymptotic variance of T (β̂ − β) and T V the asymptotic
Avariance of β̂. Use the notation β̂ ∼ N(β, T −1 V) to denote the asymptotic approximating distribution of β̂ and a[var(β̂) to denote the asymptotic variance T −1 V.

184

6. Time Series Regression Modelingwhere ȳ is the sample mean of yt and 1 is a (T × 1) vector of 1’s. R2
measures the percentage of the variability of yt that is explained by the
regressors, xt . The usual R2 has the undesirable feature of never decreasing
as more variables are added to the regression, even if the extra variables
are irrelevant. To remedy this, the R2 statistic may be adjusted for degrees
of freedom giving
Ra2 = 1 −ε̂0 ε̂/(T − k)σ̂ 2=0(y − ȳ1) (y − ȳ1)/(T − 1)
var(y
c t)

The adjusted R2 , Ra2 , may decrease with the addition of variables withlow explanatory power. If fact, it can be shown (e.g., Greene 2000, p. 240)that Ra2 will fall (rise) when a variable is deleted from the regression if
the absolute value of the t-statistic associated with this variable is greater(less) than 1.

6.2.3 Hypothesis Testing
The simple null hypothesis
H0 : β i = β 0i
is tested using the t-ratiot=β̂ i − β 0ic β̂ i )
SE(

(6.4)

which is asymptotically distributed N (0, 1) under the null. With the additional assumption of iid Gaussian errors and regressors independent of theerrors for all t, β̂ is normally distributed in finite samples and the t-ratio
is distributed Student-t with T − k − 1 degrees of freedom.
Linear hypotheses of the form
H0 : Rβ = r(6.5)

where R is a fixed q × (k + 1) matrix of rank q and r is a fixed q × 1 vector
are tested using the Wald statistichi−10Wald = (Rβ̂ − r) Ravar((Rβ̂ − r)(6.6)
d β̂)R0

Under the null, the Wald statistic is asymptotically distributed χ2 (q). Under the additional assumption of iid Gaussian errors and regressors independent of the errors for all t, Wald/q is distributed F (q, T − k −1) in finite
samples.
The statistical significance of all of the regressors excluding the constantis captured by the F -statistic
F =

R2 /k
(1 − R2 )/(T − k − 1)

6.2 Time Series Regression Using the S+FinMetrics Function OLS

185

which is distributed F (k, T − k − 1) under the null hypothesis that all slope
coeﬃcients are zero and the errors are iid Gaussian.

6.2.4 Residual Diagnostics
In the time series regression models, several residual diagnostic statistics
are usually reported along with the regression results. These diagnostics
are used to evaluate the validity of some of the underlying assumptions of
the model and to serve as warning flags for possible misspecification. The
most common diagnostic statistics are based on tests for normality andserial correlation in the residuals of (6.1).
The most common diagnostic for serial correlation based on the estimated residuals ε̂t is the Durbin-Watson statistic
DW =

PT

2
t=2 (ε̂t − ε̂t−1 ).
PT2
t=1 ε̂t

It is easy to show that DW ≈ 2(1 − ρ̂), where ρ̂ is the estimated correlation
between and ε̂t and ε̂t−1 . Hence, values of DW range between 0 and 4.
Values of DW around 2 indicate no serial correlation in the errors, values
less than 2 suggest positive serial correlation, and values greater than 2suggest negative serial correlation2 . Another common diagnostic for serial
correlation is the Ljung-Box modified Q statistic discussed in Chapter 3.
Although error terms in the time series regression model are not assumed to be normally distributed, severe departures from normality may
cast doubt on the validity of the asymptotic approximations utilized forstatistical inference especially if the sample size is small. Therefore, another diagnostic statistic commonly reported is the Jarque-Bera test fornormality discussed in Chapter 3.

6.3 Time Series Regression Using the
S+FinMetrics Function OLS
Ordinary least squares estimation of the time series regression model (6.1)
in S-PLUS is carried out with the S+FinMetrics function OLS. OLS extends
the S-PLUS linear model function lm to handle time series regression in a
2 The DW statistic is an optimal test only for the special case that ε in (1) followstan AR(1) process and that the regressors xt are fixed. Critical values for the bounding
distribution of DW in this special case are provided in most econometrics textbooks.
However, in practice there is often little reason to believe that εt follows an AR(1)process and the regressors are rarely fixed and so the DW critical values are of little
practical use.

186

6. Time Series Regression Modelingmore natural way. The arguments expected by OLS are similar to those forlm:> args(OLS)
function(formula, data, weights, subset, na.rm = F, method= "qr", contrasts = NULL, start = NULL, end = NULL,...)The main arguments are formula, which is an S-PLUS formula withthe response variable(s) on the left hand side of the ~ character and the
response variables separated by + on the right hand side3 , and data, whichis “timeSeries” or data frame in which to interpret the variables named in
the formula and subset arguments. The other arguments will be explained
and their use will be illustrated in the examples to follow.
The function OLS produces an object of class “OLS” for which there areprint, summary, plot and predict methods as well as extractor functions coefficients (coef), residuals (resid), fitted.values (fitted),vcov and IC. The extractor functions coef, resid and fitted are common to many S-PLUS model objects. Note that if “timeSeries” objects
are used in the regression then the extracted residuals and fitted values
are also “timeSeries” objects. The extractor functions vcov, which extracts avar(d β̂), and IC, which extracts information criteria, are specific to
S+FinMetrics model objects and work similarly to the extractor functions
vcov and AIC from the MASS library.
There are several important diﬀerences between lm and OLS. First, theargument formula is modified to accept lagged values of the dependent
variable through the use of AR terms and lagged values of regressors throughthe use of the S+FinMetrics functions tslag and pdl. Second, subset regression for “timeSeries” data is simplified through the use of the startand end options. Third, summary output includes time series diagnostic
measures and standard econometric residual diagnostic tests may be computed from OLS objects. Fourth, heteroskedasticity consistent as well asheteroskedasticity and autocorrelation consistent coeﬃcient covariance matrices may be computed from OLS objects.
The use of OLS for time series regression with financial data is illustrated
with the following examples
Example 35 Estimating and testing the capital asset pricing model
The famous Capital Asset Pricing Model (CAPM) due to Sharpe, Litnerand Mosen is usually estimated using the excess return single index model
Rit − rf t = αi + β i (RMt − rf t ) + εit , i = 1, . . . , N ; t = 1, . . . , T(6.7)

where Rit is the return on asset i (i = 1, . . . , N ) between time periodst − 1 and t, RMt is the return on a market index portfolio between time
3 See Chapter 1 for details on specifying formulas in S-PLUS.

6.3 Time Series Regression Using the S+FinMetrics

Function OLS

187

periods t − 1 and t, rf t denotes the rate of return between times t − 1 andt on a risk-free asset, and εit is a normally distributed random error suchthat εit ∼ GW N (0, σ 2i ). The market index portfolio is usually some welldiversified portfolio like the S&P 500 index, the Wilshire 5000 index or the
CRSP4 equally or value weighted index. In practice, rf t is taken as the Tbill rate to match the investment horizon associated with Rit . The CAPM
is an equilibrium model for asset returns and, if RMt is the value-weighted
portfolio of all publicly traded assets, it imposes the relationship
E[Rit ] − rf t = β i (E[RMt ] − rf t ).
In other words, the above states that the risk premium on asset i is equal
to its beta, β i , times the risk premium on the market portfolio. Hence,
β i is the appropriate risk measure for asset i. In the excess returns single
index model, the CAPM imposes the testable restriction that αi = 0 for
all assets.
The intuition behind the CAPM is as follows. The market index RMt
captures “macro” or market-wide systematic risk factors that aﬀect all returns in one way or another. This type of risk, also called covariance risk,
systematic risk and market risk, cannot be eliminated in a well diversified
portfolio. The beta of an asset captures the magnitude of this nondiversifiable risk. The random error term εit represents random “news” that arrives
between time t − 1 and t that captures “micro” or firm-specific risk factorsthat aﬀect an individual asset’s return that are not related to macro events.
For example, εit may capture the news eﬀects of new product discoveries
or the death of a CEO. This type of risk is often called firm specific risk,
idiosyncratic risk, residual risk or non-market risk. This type of risk can beeliminated in a well diversified portfolio. The CAPM says that in market
equilibrium the risk premium on any asset is directly related to the magnitude of its nondiversifiable risk (beta). Diversifiable risk is not priced;
i.e., diversifiable risk does not command a risk premium because it can beeliminated by holding a well diversified portfolio.
In the CAPM, the independence between RMt and εit allows the unconditional variability of an asset’s return Rit to be decomposed into thevariability due to the market index, β 2i σ 2M , plus the variability of the firm
specific component, σ 2i . The proportion of the variance Rit explained by
the variability in the market index is the usual regression R2 statistic. Accordingly, 1 − R2 is then the proportion of the variability of Rit that is due
to firm specific factors. One can think of R2 as measuring the proportionof risk in asset i that cannot be diversified away when forming a portfolio
and 1 − R2 as the proportion of risk that can be diversified away.
4 CRSP refers to the Center for Research in Security Prices at the University of
Chicago.

188

6. Time Series Regression Modeling

Estimating the CAPM Using the S+FinMetrics Function OLS
Consider the estimation of the CAPM regression (6.7) for Microsoft using monthly data over the ten year period January 1990 through January
2000. The S&P 500 index is used for the market proxy, and the 30 day
T-bill rate is used for the risk-free rate. The S+FinMetrics “timeSeries”
singleIndex.dat contains the monthly price data for Microsoft, and the
S&P 500 index and the “timeSeries” rf.30day contains the monthly 30
day T-bill rate. The excess return data are created using> colIds(singleIndex.dat)[1] "MSFT" "SP500"> colIds(rf.30day)[1] "RF"> ret.ts = getReturns(singleIndex.dat, type="continuous")> excessRet.ts = seriesMerge(ret.ts,log(1+rf.30day))> excessRet.ts[,"MSFT"] = excessRet.ts[,"MSFT"] + excessRet.ts[,"RF"]> excessRet.ts[,"SP500"] = excessRet.ts[,"SP500"] + excessRet.ts[,"RF"]> excessRet.ts = excessRet.ts[,1:2]
Time plots and a scatterplot of the excess returns created by> par(mfrow=c(2,1))> plot(excessRet.ts, plot.args=list(lty=c(1,3)),+ main="Monthly excess returns on Microsoft and S&P 500 Index")> legend(0, -0.2, legend=c("MSFT","S&P 500"), lty=c(1,3))> plot(seriesData(excessRet.ts[,"SP500"]),+ seriesData(excessRet.ts[,"MSFT"]),+ main="Scatterplot of Returns",+ xlab="SP500", ylab="MSFT")
are given in Figure 6.1. The returns on Microsoft and the S&P 500 index
appear stationary and ergodic and tend to move in the same direction overtime with the returns on Microsoft being more volatile than the returns on
the S & P 500 index. The estimate of the CAPM regression for Microsoft
using OLS is:> ols.fit = OLS(MSFT~SP500, data=excessRet.ts)> class(ols.fit)[1] "OLS"
OLS produces an object of class “OLS” with the following components> names(ols.fit)[1] "R""coef"[5] "residuals" "assign"[9] "terms""call""df.resid" "fitted""contrasts" "ar.order"

6.3 Time Series Regression Using the S+FinMetrics

Function OLS

189

-0.2

0.0

0.2

Monthly excess returns on Microsoft
and S&P 500 Index

-0.4

MSFT
S&P 500

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

0.0
-0.4

-0.2

MSFT

0.2

Scatterplot of Returns

-0.15

-0.10

-0.05

0.0

0.05

0.10

SP500

FIGURE 6.1. Monthly excess returns on Microsoft and the S&P 500 index.

The results of the OLS fit are displayed using the generic print andsummary methods. The print method produces minimal output:> ols.fit
Call:
OLS(formula = MSFT ~SP500, data = excessRet.ts)
Coefficients:(Intercept) SP500
0.01281.5259Degrees of freedom: 131 total; 129 residual
Time period: from Feb 1990 to Dec 2000
Residual standard error: 0.09027
Notice that since the object specified in data is a “timeSeries”, the startand end dates of the estimation sample are printed. The summary method
produces the standard econometric output:> summary(ols.fit)
Call:
OLS(formula = MSFT ~SP500, data = excessRet.ts)
Residuals:

190

6. Time Series Regression ModelingMin
1Q-0.3835 -0.0566

Median0.00233Q
0.0604Max
0.1991

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0128 0.0080
1.6025 0.1115
SP500 1.5259 0.1998
7.6354 0.0000
Regression Diagnostics:
R-Squared 0.3113
Adjusted R-Squared 0.3059
Durbin-Watson Stat 2.1171
Residual Diagnostics:
Stat P-Value
Jarque-Bera 41.6842 0.0000
Ljung-Box 11.9213 0.9417
Residual standard error: 0.09027 on 129 degrees of freedom
Time period: from Feb 1990 to Dec 2000
F-statistic: 58.3 on 1 and 129 degrees of freedom, thep-value is 4.433e-012
The estimated value for β for Microsoft is 1.526 with an estimated stanc β̂) = 0.200. An approximate 95% confidence interval for β
dard error SE(cis β̂ ± 2· SE(β̂) = [1.126, 1.926], and so Microsoft is judged to be riskier
than the S&P 500 index. The estimated value of α is 0.013 with an estic
mated standard error of SE(α̂)= 0.008. An approximate 95% confidencecinterval for α is α̂ ± 2· SE(α̂) = [−0.003, 0.029]. Since α = 0 is in the confidence interval the CAPM restriction hold for Microsoft. The percentage of
nondiversifiable (market specific) risk is R2 = 0.31 and the percentage ofdiversifiable (firm specific) risk is 1 − R2 = 0.69. The estimated magnitudeof diversifiable risk is σ̂ = 0.090 or 9% per month. Notice that the JarqueBera statistic indicates that the residuals from the CAPM regression are
not normally distributed. The DW and Ljung-Box statistics, however, indicate that the residuals are serially uncorrelated (at least at the first lag).
The extractor functions for an “OLS” object are used to extract the vectors of estimated coeﬃcients β̂, fitted values ŷ, residuals ε̂ and asymptotic
variance matrix avar(d β̂) :

> coef(ols.fit)(Intercept) SP500
0.01281 1.526

6.3 Time Series Regression Using the S+FinMetrics

Function OLS

191> fitted(ols.fit)[1:3]
Positions1
Feb 1990
0.01711
Mar 1990
0.03965
Apr 1990 -0.03927> resid(ols.fit)[1:3]
Positions1
Feb 1990 0.04258
Mar 1990 0.06868
Apr 1990 0.07870> vcov(ols.fit)(Intercept)
SP500(Intercept) 0.00006393 -0.0002618
SP500 -0.00026181 0.0399383
Notice that to use the extractor functions residuals and fitted.values
one only has to type resid and fitted. Since the data used for estimation
is a “timeSeries” object, the extracted residuals and fitted values are also
“timeSeries” objects.
To illustrate the use of the extractor functions, the t-statistics for testing
the null hypothesis that the intercept is zero and the slope is unity are> (coef(ols.fit)-c(0,1))/sqrt(diag(vcov(ols.fit)))(Intercept) SP500
1.603 2.631
and summary statistics for the residuals using the S+FinMetrics function
summaryStats are> summaryStats(residuals(ols.fit))
Sample Quantiles:
min
1Q
median3Q
max-0.3835 -0.05661 0.002342 0.06037 0.1991
Sample Moments:
mean
std skewness kurtosis-7.204e-018 0.08993 -0.7712
5.293
Number of Observations:131

Testing Linear Restrictions
The CAPM regression (6.7) in matrix form is (6.2) with xt = (1, RMt −rf t )0
and β = (α, β)0 . Consider testing the joint null hypothesis H0 : α = 0 and192

6. Time Series Regression Modelingβ = 1. This hypothesis imposes two linear restrictions on the parameter
vector β = (α, β)0 that may be written in the form (6.5) with
µ
¶
µ ¶
1 0
0
R=
, r=
0 1
1
The Wald statistic (6.6) may be computed directly as> Rmat = diag(2)> rvec = c(0,1)> bhat = coef(ols.fit)> avarRbhat = Rmat%*%vcov(ols.fit)%*%t(Rmat)> wald.stat =+ t(Rmat%*%bhat-rvec)%*%solve(avarRbhat)%*%(Rmat%*%bhat-rvec)> as.numeric(wald.stat)[1] 11.17> p.value = 1 - pchisq(wald.stat,2)> p.value[1] 0.003745
The small p-value suggests that null H0 : α = 0 and β = 1 should be
rejected at any reasonable significance level. The F -statistic version of theWald statistic based on normal errors is> F.stat = wald.stat/2> p.value = 1 - pf(F.stat,2,ols.fit$df.resid)> p.value[1] 0.004708
and also suggests rejection of the null hypothesis.
The F -statistic version of the Wald statistic for general linear restrictionsof the form (6.5) may be conveniently computed using the S+FinMetrics
function waldTest. For example,> waldTest(ols.fit,Intercept==0,SP500==1)
Wald Test of Coefficients:
Null Hypothesis: constraints are true
Test Statistic: 5.587
Dist. under Null: F with ( 2 , 129 ) degrees of freedom
P-value: 0.004708
produces the F -statistic version of the Wald test for the null hypothesis H0 :α = 0 and β = 1. Notice how the restrictions under the null being tested
are reflected in the call to waldTest. More complicated linear restrictions
like H0 : α + 2β = 2 are also easily handled> waldTest(ols.fit,Intercept-2*SP500==2)

6.3 Time Series Regression Using the S+FinMetrics

Function OLS

193

Wald Test of Coefficients:
Null Hypothesis: constraints are true
Test Statistic: 157.8
Dist. under Null: F with ( 1 , 129 ) degrees of freedom
P-value: 0
Likelihood ratio (LR) statistics for testing linear hypotheses may also
be computed with relative ease since the OLS estimates are the maximum
likelihood estimates assuming the errors have a normal distribution. The
log-likelihood value of the OLS fit assuming normal errors may be extractedusing the S+FinMetrics function IC. For example, the log-likelihood for theunrestricted CAPM fit is> IC(ols.fit, "loglike")[1] 130.2
Consider testing the CAPM restriction H0 : α = 0 using a LR statistic.
The restricted OLS fit, imposing α = 0, is computed using> ols.fit2 = OLS(MSFT~SP500-1,data=excessRet.ts)
The LR statistic is then computed as> LR = -2*(IC(ols.fit2,"loglike")-IC(ols.fit,"loglike"))> LR[1] 2.571> 1 - pchisq(LR,1)[1] 0.1089
Given the p-value of 0.109, the CAPM restriction is not rejected at the 10%
significance level.
Graphical Diagnostics
Graphical summaries of the OLS fit are produced with the generic plot
function. By default, plot produces a menu of plot choices:> plot(ols.fit)
Make a plot selection (or 0 to exit):
1: plot: all
2: plot: response vs fitted values
3: plot: response and fitted values
4: plot: normal QQ-plot of residuals
5: plot: residuals
6: plot: standardized residuals
7: plot: residual histogram
8: plot: residual ACF

194

6. Time Series Regression Modeling

Plot Function
xygPlot
rvfPplot
rafPlot
histPlot
qqPlotresidPlot
acfPlot

Description
Trellis xyplot with grid and strip.text options
Trellis response vs. fitted plot with grid andstrip.text options
Trellis plot of response and fitted values
Trellis density estimate with strip.text options
Trellis QQ-plot with grid and strip.text options
Trellis plot of residuals
Trellis ACF plot

TABLE 6.1. S+FinMetrics utility Trellis plotting functions

9: plot: residual PACF
10: plot: residual^2 ACF
11: plot: residual^2 PACF
Selection:
The plot choices are diﬀerent from those available for “lm” objects and focus
on time series diagnostics. All plots are generated using Trellis graphics5 .
Table 6.1 summarizes the utility plot functions used to create the various
OLS plots. See the help files for more information about the plot functions.
Figures 6.2 and 6.3 illustrate plot choices 3 (response and fitted) and 8(residual ACF). From the response and fitted plot, it is clear that thereturn on the S&P 500 index is a weak predictor of return on Microsoft.
The residual ACF plot indicates that the residuals do not appear to be
autocorrelated, which supports the results from the residual diagnostics
reported using summary.
Individual plots can be created directly, bypassing the plot menu, using
the which.plot option of plot.OLS. For example, the following command
creates a normal qq-plot of the residuals:> plot(ols.fit,which.plot=3)
Notice that number used for the qq-plot specified by which.plot is one
less than the value specified in the menu. The qq-plot may also be createdby calling the Trellis utility plot function qqPlot directly:> qqPlot(resid(ols.fit), strip.text="ols.fit",+ xlab="Quantile of Standard Normal",+ ylab="Residuals",main="Normal QQ Plot")

Residual Diagnostics
The residual diagnostics reported by summary may be computed directlyfrom an “OLS” object. The normalTest and autocorTest functions in
5 Unfortunately, the Trellis plots cannot be easily combined into multipanel plots.

6.3 Time Series Regression Using the S+FinMetrics

Function OLS

195

Response and Fitted Values

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

ols.fit

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 6.2. Response and fitted values from the OLS fit to the CAPM regression
for Microsoft.
Residual Autocorrelation

ols.fit
1.0

0.8

ACF

0.6

0.4

0.2

0.0

-0.2
5

10

15

20

Lag

FIGURE 6.3. Residual ACF plot from the OLS fit to the CAPM regression forMicrosoft.

196

6. Time Series Regression Modeling

S+FinMetrics may be used to compute test statistics for normality andautocorrelation from the residuals of an OLS fit. For example, to compute
the Jarque-Bera normality test and the Ljung-Box test from the residualsof the CAPM regression use> normalTest(ols.fit,method="jb")Test for Normality: Jarque-BeraNull Hypothesis: data is normally distributed
Test Statistics:
Test Stat 41.68
p.value 0.00
Dist. under Null: chi-square with 2 degrees of freedom
Total Observ.: 131> autocorTest(ols.fit,method="lb")Test for Autocorrelation: Ljung-Box
Null Hypothesis: no autocorrelation
Test Statistics:
Test Stat 11.9213
p.value 0.9417
Dist. under Null: chi-square with 21 degrees of freedom
Total Observ.: 131
The Durbin-Watson statistic may be recovered using the S-PLUS function
durbinWatson as follows> durbinWatson(residuals(ols.fit))
Durbin-Watson Statistic: 2.117063
Number of observations: 131
The Breusch-Godfrey LM test for residual autocorrelation may be computed using autocorTest by specifying method=\lm". For example, tocompute the LM statistic for the null of no autocorrelation against thealternative of serial correlation up to lag two use> autocorTest(ols.fit,method="lm",lag.n=2)

6.3 Time Series Regression Using the S+FinMetrics

Function OLS

197

Test for Autocorrelation: Breusch-Godfrey LM
Null Hypothesis: no autocorrelation
Test Statistics:
Test Stat 2.9768
p.value 0.2257
Dist. under Null: chi-square with 2 degrees of freedom
Total Observ.: 131
Consistent with the Ljung-Box test, the LM test indicates no residual autocorrelation.
Subset Regression
The estimated β for Microsoft uses all of the data over the 11 year period
from January 1990 to December 2000. It is generally thought that β does
not stay constant over such a long time period. To estimate β using only
the most recent five years of data the start option of OLS may be utilized> OLS(MSFT~SP500, data=excessRet.ts,+ start="Jan 1996", in.format="%m %Y")
Call:
OLS(formula = MSFT ~SP500, data = excessRet.ts, start ="Jan 1996", in.format = "%m %Y")
Coefficients:(Intercept) SP500
0.0035
1.7828Degrees of freedom: 60 total; 58 residual
Time period: from Jan 1996 to Dec 2000
Residual standard error: 0.1053
Notice that date string passed to start uses the same display format as
the “timeDate” objects in the positions slot of excessRet.ts, and that
this format is specified directly using in.format="%m %Y". Estimation overgeneral sub-periods follows by specifying both the start date and the end
date of the sub-period in the call to OLS.
Regression estimates may be computed over general subsets by using theoptional argument subset to specify which observations should be used in
the fit. Subsets can be specified using a logical vector (which is replicatedto have length equal to the number of observations), a numeric vector
indicating the observation numbers to be included, or a character vector of

198

6. Time Series Regression Modelingthe observation names that should be included. For example, to estimate
the CAPM only for the observations for which the excess return on the
S&P 500 is positive, use> OLS(MSFT~SP500, data=excessRet.ts, subset=(SP500>=0))
Call:
OLS(formula = MSFT ~SP500, data = excessRet.ts, subset = (SP500 >= 0))
Coefficients:(Intercept) SP500
0.02311.3685Degrees of freedom: 80 total; 78 residual
Residual standard error: 0.08341
Regression with Dummy Variables
In the analysis of asset returns, it is often noticed that excess returns arehigher in January than in any other month. To investigate this claim, adummy variable is created which equals 1 if the month is January and 0
otherwise:> is.Jan = (months(positions(excessRet.ts))=="Jan")> Jan.dum = timeSeries(pos=positions(excessRet.ts),+ data=as.integer(is.Jan))
Next, the January dummy variable is added to the time series of excess
returns:> newdat.ts = seriesMerge(excessRet.ts,Jan.dum)> colIds(newdat.ts)[3] = "Jan.dum"
The CAPM regression allowing for a diﬀerent intercept in January is> summary(OLS(MSFT~SP500+Jan.dum, data=newdat.ts))
Call:
OLS(formula = MSFT ~SP500 + Jan.dum, data = newdat.ts)
Residuals:
Min
1Q-0.3804 -0.0532

Median0.00653Q
0.0604Max
0.2032

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0090 0.0082
1.0953 0.2755

6.3 Time Series Regression Using the S+FinMetrics

SP500 1.5085 0.1986
Jan.dum 0.0513 0.0295

7.59721.7371

Function OLS

199

0.00000.0848Regression Diagnostics:
R-Squared 0.3271
Adjusted R-Squared 0.3166
Durbin-Watson Stat 2.0814
Residual Diagnostics:
Stat P-Value
Jarque-Bera 43.4357 0.0000
Ljung-Box 12.1376 0.9358
Residual standard error: 0.08958 on 128 degrees of freedom
Time period: from Feb 1990 to Dec 2000
F-statistic: 31.11 on 2 and 128 degrees of freedom, thep-value is 9.725e-012
The coeﬃcient on the January dummy is positive and significant at the 9%
level indicating that excess returns are slightly higher in January than inother months. To allow for a diﬀerent intercept and slope in the regression,use> summary(OLS(MSFT~SP500*Jan.dum, data=newdat.ts))
Call:
OLS(formula = MSFT ~SP500 * Jan.dum, data = tmp1.ts)
Residuals:
Min
1Q-0.3836 -0.0513

Median0.00473Q
0.0586Max
0.2043

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0095 0.0082
1.1607 0.2479
SP500 1.4307 0.2017
7.0917 0.0000
Jan.dum 0.0297 0.0317
0.9361 0.3510
SP500:Jan.dum 1.6424 0.9275
1.7707 0.0790
Regression Diagnostics:
R-Squared 0.3433
Adjusted R-Squared 0.3278
Durbin-Watson Stat 2.0722

200

6. Time Series Regression Modeling

Residual Diagnostics:
Stat P-Value
Jarque-Bera 51.4890 0.0000
Ljung-Box 12.7332 0.9177
Residual standard error: 0.08884 on 127 degrees of freedom
Time period: from Feb 1990 to Dec 2000
F-statistic: 22.13 on 3 and 127 degrees of freedom, thep-value is 1.355e-011
Notice that the formula uses the short-hand notation A*B = A+B+A:B.
Interestingly, when both the slope and intercept are allowed to be diﬀerent
in January only the slope is significantly higher.
Predictions
Predictions or forecasts from an OLS fit may be computed using the genericpredict function. For example, consider computing forecasts of the excess
return on Microsoft conditional on specified values for the S&P 500 excess
return based on the CAPM fit. The excess returns on the S&P 500 for theconditioinal forecasts are> sp500.new = data.frame(c(-0.2,0,2))> colIds(sp500.new) = "SP500"
These new data values must be in a data frame with the same name as
the variable containing the S&P 500 data in excessRet.ts. The forecasts
are computed using> ols.pred = predict(ols.fit,n.predict=3,newdata=sp500.new)> class(ols.pred)[1] "forecast"> ols.pred
Predicted Values:[1] -0.2924

0.01283.0646

The result of predict is an object of class “forecast” for which there
are print, summary and plot methods. The print method shows just theforecasts. The summary method shows the forecasts and forecast standard
errors (ignoring parameter estimation error)> summary(ols.pred)
Predicted Values with Standard Errors:
prediction std.err

6.4 Dynamic Regression

201

0

1

values

2

3

MSFT

128

130

132

134

index

FIGURE 6.4. Conditional forecasts for the excess returns on Microsoft from the
CAPM regression.

1-step-ahead -0.2924
2-step-ahead 0.0128
3-step-ahead 3.0646

0.09030.09030.0903To view the forecasts and standard error band along with the historical
data use> plot(ols.pred, xold=excessRet.ts[,1], n.old=5, width=2)
The argument xold contains the historical data for the response variable,n.old determines how many historical observations to plot along with theforecasts and width specifies the multiplier for the forecast standard errors
in the construction of the error bands. The created plot is illustrated in
Figure 6.4.

6.4 Dynamic Regression
Often the time series regression model (6.1) contains lagged variables asregressors to capture dynamic eﬀects. The general dynamic time series
regression model contains lagged values of the response variable yt and202

6. Time Series Regression Modeling

lagged values of the exogenous stationary regressors x1t , . . . , xkt :yt = α +pX

φj yt−j +

j=1

q1
X
j=0

β 1j x1t−j + · · · +qkX

β kj xkt−j + εt

(6.8)

j=0

where the error term εt is assumed to be W N (0, σ 2 ). The model (6.8) iscalled an autoregressive distributed lag (ADL) model and generalizes anAR(p) by including exogenous stationary regressors.
The main issues associated with the analysis and interpretation of the
ADL model (6.8) can be illustrated using the following simple ADL modelwith a single exogenous variable xt :yt = α + φyt−1 + β 0 xt + β 1 xt−1 + εt

(6.9)

Since xt is assumed to be stationary, and εt ∼ W N (0, σ 2 ), yt behaves likean AR(1) process
yt = α + φyt−1 + wt
where wt = β 0 xt + β 1 xt−1 + εt is a composite error term. Therefore, theADL model (6.9) is stationary provided |φ| < 1. Given that yt is stationary
it has an infinite order moving average representation (impulse response
function) in terms of the composite errors wt
yt

= μ+

∞
X

ψ j wt−jj=0= μ + wt + ψ 1 wt−1 + ψ 2 wt−2 + · · ·where μ = 1/(1 − φ) and ψ j = φj . Substituting wt = β 0 xt + β 1 xt−1 + εtand ψ j = φj into the above moving average representation givesyt= μ + (β 0 xt + β 1 xt−1 + εt ) + φ(β 0 xt−1 + β 1 xt−2 + εt−1 )+φ2 (β 0 xt−2 + β 1 xt−3 + εt−2 ) + · · ·= μ + β 0 xt + (β 1 + φβ 0 )xt−1 + φ(β 1 + φβ 0 )xt−2 + · · · (6.10)+φj−1 (β 1 + φβ 0 )xt−j + · · · + εt + φεt−1 + φ2 εt−2 + · · ·Using (6.10), the interpretation of the coeﬃcients in (6.9) becomes clearer.
For example, the immediate impact multiplier is the impact of a change in
xt on yt
∂yt
= β0
∂xt
The first lag multiplier is the impact of a change in xt−1 on yt∂yt= β 1 + φβ 0
∂xt−1

6.4 Dynamic Regression

203

which incorporates a feedback eﬀect φβ 0 due to the lagged response variablein (6.9). The second lag multiplier is∂yt= φ (β 1 + φβ 0 )∂xt−2
and is smaller in absolute value than the first lag multiplier since |φ| < 1.
In general, the kth lag multiplier is∂yt= φk−1 (β 1 + φβ 0 )∂xt−ktNotice that as k → ∞, ∂x∂yt−k→ 0 so that eventually the eﬀect of a change
in xt on yt dies out. The long-run eﬀect of a change in xt on yt is definedas the cumulative sum of all the lag impact multipliers

long-run eﬀect

∂yt
∂yt
∂yt
+
+
+ ···
∂xt ∂xt−1 ∂xt−2∞X
φk (β 0 + β 1 )
=
=

k=0

=

β0 + β11−φThe parameter φ on yt−1 determines the speed of adjustment between theimmediate impact of xt on yt and the long-run impact. If φ = 0, then thetlong-run impact is reached in one time period since ∂x∂yt−k= 0 for k > 1.
In contrast, if φ ≈ 1, then the long-run impact takes many periods. A
parameter often reported is the half-life of the adjustment; that is, thelag at which one half of the adjustment to the long-run impact has beenreached. In the simple ADL (6.9), it can be shown that the half-life is equal
to ln(2)/ ln(φ).
For the general ADL model (6.8), stationarity of yt requires that all xit
be stationary and that the roots of the characteristic polynomial φ(z) =1 − φ1 z − · · · − φp z p = 0 have modulus greater than one. The k immediate impact multipliers are the coeﬃcients β 10 , . . . , β k0 and the k long-run
multipliers are
Pqk
Pq2
j=0 β 1jj=0 β kj,...,
1 − φ1 − · · · − φp
1 − φ1 − · · · − φp
The speed of adjustment to the long-run impacts is determined by the sumof the coeﬃcients on the lagged responses φ1 + · · · + φp .
Example 36 Estimating a simple dynamic CAPM regression for Microsoft
Consider estimating a simple dynamic version of the CAPM regression
Rit − rf t = α + φ(Rit−1 − rf t−1 ) + β 0 (RM t − rf t ) + β 1 (RMt−1 − rf t−1 ) + εit

204

6. Time Series Regression Modelingusing the monthly excess return data for Microsoft and the S&P 500 index.
The “short-run beta” for Microsoft is β 0 and the “long-run beta” is (β 0 +β 1 )/(1 − φ). The dynamic CAPM estimated using OLS is> adl.fit = OLS(MSFT~SP500+ar(1)+tslag(SP500),+ data=excessRet.ts)
In the regression formula, the lagged dependent variable (MSFT) is specified using the ar(1) term, and the lagged explanatory variable (SP500) is
created using the S+FinMetrics function tslag. The dynamic regression
results are> summary(adl.fit)
Call:
OLS(formula = MSFT ~SP500 + ar(1) + tslag(SP500), data =excessRet.ts)
Residuals:
Min
1Q-0.3659 -0.0514

Median0.00593Q
0.0577Max
0.1957

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0156 0.0083
1.8850 0.0617
SP500 1.5021 0.2017
7.4474 0.0000
tslag(SP500) -0.0308 0.2453-0.1257 0.9001
lag1 -0.1107 0.0921-1.2021 0.2316
Regression Diagnostics:
R-Squared 0.3248
Adjusted R-Squared 0.3087
Durbin-Watson Stat 1.9132
Residual Diagnostics:
Stat P-Value
Jarque-Bera 41.5581 0.0000
Ljung-Box 10.5031 0.9716
Residual standard error: 0.0904 on 126 degrees of freedom
Time period: from Mar 1990 to Dec 2000
F-statistic: 20.2 on 3 and 126 degrees of freedom, thep-value is 9.384e-011

6.4 Dynamic Regression

205

The least squares estimates of dynamic CAPM parameters are α̂ = 0.016,φ̂ = −0.111, β̂ 0 = 1.502 and β̂ 1 = −0.031. The estimated “short-run beta”
for Microsoft is 1.502 and the estimated “long-run beta” is6> bhat = coef(adl.fit)> lr.beta = (bhat[2]+bhat[3])/(1-bhat[4])> lr.beta
SP500
1.325
Notice that the “long-run beta” is smaller than the “short-run beta”. However, since the standard errors on the dynamic terms φ̂ and β̂ 1 are large
relative to the estimated values, the data do not support the dynamic
CAPM model.

6.4.1 Distributed Lags and Polynomial Distributed Lags
A special case of the general ADL model (6.8) is the distributed lag modelyt = α +qX

β j xt−j + εt(6.11)

j=0

For simplicity, the model is shown with one exogenous variable x. The
extension to multiple exogenous variables is straightforward. Given the
results of the previous section, β j is interpreted as
Pqthe jth lag multiplier,and the long-run impact on y of a change in x is j=1 β j .
Determining the Lag Length

In many applications, the lag length q needs to be long to adequately capture the dynamics in the data. To determine the lag length, all models withq ≤ qmax are fit and the preferred model minimizes some model selection
criterion like the Akaike (AIC) or Schwarz (BIC). For the distributed lag
model, the AIC and BIC have the form
2
q
T
ln T
BIC(q) = ln(σ̃ 2 (q)) +q
T

AIC(q) = ln(σ̃ 2 (q)) +where σ̃ 2 (q) is the least squares estimate of σ 2 without a degrees of freedom
correction. For objects of class “OLS”, S+FinMetrics provides the extractor
function IC to compute the AIC or BIC information criteria.
6 Since the “long-run beta” is a nonlinear function of the least squares estimates,
estimated standard errors for the “long-run beta” may be computed using the so-called“delta method”. See Greene (2000) page 118.

206

6. Time Series Regression ModelingIf the exogenous variable xt is highly persistent, then lagged values
xt , xt−1 , . . . , xt−q may be highly correlated and problems associated withnear multicollinearity may occur in (6.11)7 . In this case, the S+FinMetrics
function collinearTest may be used to diagnose the extent of near multicollinearity in the data. The function collinearTest computes either thecondition number for X0 X or the variance inflation statistics associatedwith each variable.
Example 37 Distributed lag model for U.S. real GDP growth
The S+FinMetrics “timeSeries” policy.dat contains monthly dataon U.S. real GDP and the Federal Funds rate. Consider estimating a distributed lag model with q = 12 for the growth rate in real GDP using theFederal Funds rate as an exogenous variable over the period January 1990
to March 1998:> dl.fit = OLS(diff(log(GDP))~FFR+tslag(FFR,1:12),data=+ policy.dat, start="Jan 1990",in.format="%m %Y",na.rm=T)
The AIC and BIC information criteria may be extracted using> IC(dl.fit,type="AIC")[1] -1271> IC(dl.fit,type="BIC")[1] -1235
The model may be re-fit with diﬀerent values of q and the preferred modelis the one which produces the smallest value of the chosen information
criterion.
The condition number and variance inflation statistics from the leastsquare fit are> collinearTest(dl.fit, method="cn")[1] 311.2> collinearTest(dl.fit, method="vif")
FFR tslag(FFR, 1:12)lag1 tslag(FFR, 1:12)lag2
111.8
278.7
293
tslag(FFR, 1:12)lag3 tslag(FFR, 1:12)lag4
304.9
331.8
tslag(FFR, 1:12)lag5 tslag(FFR, 1:12)lag6
344.9
370.5
tslag(FFR, 1:12)lag7 tslag(FFR, 1:12)lag8
7 See Greene (2000) pages 255-259 for a discussion of the problems associated withnear multicollinearity.

6.4 Dynamic Regression

369

207

390

tslag(FFR, 1:12)lag9 tslag(FFR, 1:12)lag10
389.5
410
tslag(FFR, 1:12)lag11 tslag(FFR, 1:12)lag12
424.5
162.6
The large condition number and variance inflation statistics indicate thathigh correlation among the regressors is a potential problem.

6.4.2 Polynomial Distributed Lag Models
The unrestricted distributed lag model (6.11) may produce unsatisfactory
results due to high correlation among the lagged variables. If the sample size
is small and the lag length q is large then these problems are exacerbated.
In these cases, one may want to restrict the behavior of the lag coeﬃcients
β j in (6.11). One popular way to do this is to use the polynomial distributedlag (PDL) model8 . The PDL model specifies that β j follows a polynomial
β j = α0 + α1 j + α2 j 2 + · · · + αd j d(6.12)

for j = 1, . . . , q > d. Usually, the order of the polynomial, d, is small.
Whereas the general distributed lag model (6.11) has q lag parameters the
PDL model has only d + 1 lag parameters. To see this more explicitly,
the distributed lag model with p lags under the restriction (6.12) may be
re-written as the linear regression with d variables
yt = α + α0 z0t + α1 z1t + · · · + αd zdt + εt
where
zjt =

q
X

ij xt−i(6.13)(6.14)

i=1

Example 38 PDL model for U.S. real GDP growth
To estimate a PDL model for U.S. GDP growth using the Federal Funds
rate with d = 2 and q = 12 use> pdl.fit = OLS(diff(log(GDP))~pdl(FFR,d=2,q=12),+ data=policy.dat, start="Jan 1990",+ in.format="%m %Y", na.rm=T)> pdl.fit

8 The PDL model is also known as the Almon lag model.

208

6. Time Series Regression Modeling

Call:
OLS(formula = diff(log(GDP)) ~pdl(FFR, d = 2, q = 12),
data = policy.dat, na.rm = T, start = "Jan 1990",
in.format = "%m %Y")
Coefficients:(Intercept) pdl(FFR, d = 2, q = 12)FFR.PDL0
0.0006-0.0070pdl(FFR, d = 2, q = 12)FFR.PDL1
0.0031pdl(FFR, d = 2, q = 12)FFR.PDL2-0.0002
Degrees of freedom: 97 total; 93 residual
dropped 1 cases due to missing observations.
Time period: from Feb 1990 to Feb 1998
Residual standard error: 0.0003371
The S+FinMetrics function pdl used in the formula compute the regressors (6.14) for the PDL regression (6.13).

6.5 Heteroskedasticity and Autocorrelation
Consistent Covariance Matrix Estimation
In the time series regression model, the eﬃciency of the least squares estimates and the validity of the usual formulas for the estimated coeﬃcient
standard errors and test statistics rely on validity of the underlying assumptions of the model outlined in the beginning of Section 6.2. In empiricalapplications using financial time series, it is often the case that the error
terms εt have non constant variance (heteroskedasticity) as well as autocorrelation. As long as the regressors xt are uncorrelated with the errors
εt the least squares estimates of β will generally still be consistent andasymptotically normally distributed. However, they will not be eﬃcientand the usual formula (6.3) for computing avar(d β̂) will not be correct. As
a result, any inference procedures based on (6.3) will also be incorrect. If
the form of heteroskedasticity and autocorrelation is known, then eﬃcientestimates may be computed using a generalized least squares procedure9 . If
the form of heteroskedasticity and autocorrelation is not known, it is possi9 The S-PLUS function gls may be used to compute generalized least squares estimatesusing a variety of models for heteroskedasticity and autocorrelation.

6.5 HAC Covariance Matrix Estimation

209

ble to estimate avar(d β̂) consistently so that valid standard errors and test
statistics may be obtained. This section describes the construction of heteroskedasticity and autocorrelation consistent estimates of avar(d β̂). First,
the heteroskedasticity consistent estimate of avar(d β̂) due to Eicker (1967)
and White (1980) is discussed and then the heteroskedasticity and autocorrelation consistent estimate of avar(d β̂) due to Newey and West (1987)
is covered.

6.5.1 The Eicker-White Heteroskedasticity Consistent (HC)
Covariance Matrix Estimate
A usual assumption of the time series regression model is that the errors εt
are conditionally homoskedastic; i.e., E[ε2t |X] = σ 2 > 0. In many situations
it may be more appropriate to assume that the variance of εt is a functionof xt so that εt is conditionally heteroskedastic: E[ε2t |xt ] = σ 2 f (xt ) >0. Formally, suppose the assumptions of the time series regression model
hold but that E[ε2t xt x0t ] = S 6= σ 2 ΣXX . This latter assumption allows theregression errors to be conditionally heteroskedastic and dependent on xt ;i.e., E[ε2t |xt ] = σ 2 f (xt ). In this case, it can be shown that the asymptotic
variance matrix of the OLS estimate, β̂, is−1avar(β̂) = T −1 Σ−1
XX SΣXX .

(6.15)

The above generalized OLS asymptotic variance matrix will not be equal tothe usual OLS asymptotic matrix σ 2 Σ−1d β̂) =XX , and the usual estimate avar(σ̂ 2 (X0 X)−1 will not be correct. Hence, in the presence of heteroskedasticity the usual OLS t-statistics, standard errors, Wald statistics cannot be
trusted.
If the values of f (xt ) are known, then the generalized or weighted leastsquares (GLS) estimatorβ̂GLS = (X0 V(X)−1 X)−1 X0 V(X)y,where V(X) is a (T × T ) diagonal matrix with f (xt ) along the diagonal,
is eﬃcient.
In most circumstances f (xt ) is not known so that the eﬃcient GLS estimator cannot be computed. If the OLS estimator is to be used, then aconsistent estimate for the generalized OLS covariance matrix is needed forproper inference. A heteroskedasticity consistent (HC) estimate of avar(β̂)
due to Eicker (1967) and White (1980) is
−1

where

avar
d HC (β̂) = (X0 X)

0ŜHC (X X)−1T
1 X 2
ŜHC =
ε̂ xt x0t
T − k t=1 t(6.16)(6.17)

210

6. Time Series Regression Modelingand ε̂t is the OLS residual at time t.
The square root of the diagonal elements of avard HC (β̂) gives the EickerWhite heteroskedasticity consistent standard errors (HCSEs) for the leastc HC (β̂ i ). Heteroskedasticity
squares estimates of β i . These are denoted SE
robust t-statistics and Wald statistics are computed in the usual way usingc HC (β̂ i ) replacing avar((6.4) and (6.6) but with avard HC (β̂) and SEd β̂) andcSE(β̂ i ), respectively.
Example 39 Heteroskedasticity robust inference for the CAPM

Once a model has been fit using OLS, the HC estimate (6.16) may be
c HC (β̂ i ) may be computed using summary by
extracted using vcov and SE
specifying the optional argument correction="white" as follows> ols.fit = OLS(MSFT~SP500, data=excessRet.ts)> avar.HC = vcov(ols.fit, correction="white")> summary(ols.fit, correction="white")

Call:
OLS(formula = MSFT ~SP500, data = excessRet.ts)
Residuals:
Min
1Q-0.3835 -0.0566

Median0.00233Q
0.0604Max
0.1991

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0128 0.0080
1.5937 0.1134
SP500 1.5259 0.1920
7.9463 0.0000
Regression Diagnostics:
R-Squared 0.3113
Adjusted R-Squared 0.3059
Durbin-Watson Stat 2.1171
Residual Diagnostics:
Stat P-Value
Jarque-Bera 41.6842 0.0000
Ljung-Box 11.9213 0.9417
Residual standard error: 0.09027 on 129 degrees of freedom
Time period: from Feb 1990 to Dec 2000
F-statistic: 58.3 on 1 and 129 degrees of freedom, thep-value is 4.433e-012

6.5 HAC Covariance Matrix Estimation211

c HC (β̂ i ) are almost identical to the usual OLS
Here, the HCSE values SE
c β̂ i ) which suggests that the errors are not heteroskedastic.
values SE(

6.5.2 Testing for Heteroskedasticity

If the error terms in the time series regression model are heteroskedastic,
then the OLS estimates are consistent but not eﬃcient and the usual formula (6.3) for computing avar(d β̂) is incorrect. As shown in the previous
section, avard HC (β̂) given by (6.16) provides a consistent estimate of thegeneralized asymptotic variance (6.15). If the errors are not heteroskedastic, however, (6.15) is still consistent, but the usual formula (6.3) will generally give smaller standard errors and more powerful tests. Therefore, it
is of interest to test for the presence of heteroskedasticity. If the errors
are heteroskedastic and depend on some function of exogenous variables,
then tests for heteroskedasticity may help determine which variables aﬀect
the error variance and how they might aﬀect the variance. Finally, if thetime series regression model is misspecified, e.g. some important variables
have been omitted or the parameters are not constant over time, then often the errors will appear to be heteroskedastic. Hence, the presence ofheteroskedasticity may also signal inadequacy of the estimated model. In
this section, two common tests for heteroskedasticity are introduced. The
first is Breusch and Pagan’s (1979) LM test for heteroskedasticity causedby specified exogenous variables and the second is White’s (1980) generaltest for unspecified heteroskedasticity.
Breusch-Pagan Test for Specific Heteroskedasticity
Suppose it is suspected that the variance of εt in (6.1) is functionally related
to some known (p × 1) vector of exogenous variables zt , whose first element
is unity, via the relation
E[εt |xt ] = f (z0t α)
where f (·) is an unknown positive function. Let ε̂t denote the least squares
residual from (6.1), and consider the auxiliary regression
ε̂2t= z0t α + errorσ̄ 2(6.18)

PT
where σ̄ 2 = T −1 t=1 ε̂2t . Since the first element of zt is unity, the null
hypothesis of homoskedasticity, E[ε2t |xt ] = σ 2 , implies that all of the elements of α except the first are equal to zero. Under the homoskedasticity
null, Breusch and Pagan (1979) showed that the test statistic
1
A
RSSaux ∼ χ2 (p − 1)2

212

6. Time Series Regression Modelingwhere RSSaux is the residual sum of squares from the auxiliary regression(6.18).
The Breusch-Pagan LM test is based on the assumption that the error
terms are normally distributed. Koenker and Basset (1982) suggested amodification of the Breusch-Pagan LM test that is robust to non-normal
errors and generally has more power than the Breusch-Pagan test when theerrors are non-normal.
White’s Test for General Heteroskedasticity
Suppose εt is generally heteroskedastic such that E[ε2t xt x0t ] = S, where S
is a (k × k) matrix. Recall, if εt is homoskedastic then S = σ 2 ΣXX . Now,
under general heteroskedasticity ŜHC in (6.17) is a consistent estimate of
S and σ̂ 2 (X0 X)−1 is a consistent estimate of σ 2 ΣXX and S 6= σ 2 ΣXX .
However, under the null hypothesis of homoskedasticity, the diﬀerence between ŜHC and σ̂ 2 (X0 X)−1 should go to zero as the sample size gets larger.
White (1980) utilized this result to develop a very simple test for generalheteroskedasticity. To describe this test, let ψ t denote the (m × 1) vector of
unique and nonconstant elements of the (k × k) matrix xt x0t . Let ε̂t denotethe least squares residual from (6.1) and form the auxiliary regressionε̂t = ψ 0t γ + error(6.19)

Under the null hypothesis of homoskedasticity, White (1980) showed that2
∼ χ2 (m)T · Raux2
is the R2 from the auxiliary regression (6.19).
where Raux

Testing for Heteroskedasticity Using the S+FinMetrics Function
heteroTest
Once a model has been fit using OLS (or lm), the Breusch-Pagan, KoenkerBasset and White tests for heteroskedasticity may be computed using
the S+FinMetrics function heteroTest. For example, consider the simple CAPM regression for Microsoft> ols.fit = OLS(MSFT~SP500, data=excessRet.ts)
To apply the Breusch-Pagan LM test, a set of variables zt for which var(εt )is related must be identified. For illustrative purposes let zt = (RMt −rf t , (RMt − rf t )2 )0 . The LM may then be computed using> z1 = as.matrix(seriesData(excessRet.ts[,"SP500"]))> zmat = cbind(z1,z1^2)> heteroTest(ols.fit, method="lm", regressors=zmat)
Test for Heteroskedasticity: Breusch-Pagan LM Test

6.5 HAC Covariance Matrix Estimation

213

Null Hypothesis: data is homoskedastic
Test Statistic: 0.152
Dist. under Null: chi-square with 2 degrees of freedom
P-value: 0.9268
Coefficients:
Intercept
SP500 SP500^2
1.041-1.320 -20.407
Degrees of freedom: 131 total; 128 residual
Residual standard error: 2.095

Notice that the regressors specified for the LM test must be in the formof a matrix. The high p-value of the test clearly signals that the null ofhomoskedasticity should not be rejected against the alternative that var(εt )depends on zt . To compute the Koenker-Basset robust LM test, set theoptional argument robust=T in the call to heteroTest.
The application of White’s test for heteroskedasticity is more straightforward since var(εt ) is assumed to be functionally related to the variables
used in the OLS fit:

> heteroTest(ols.fit, method="white")
Test for Heteroskedasticity: White General Test
Null Hypothesis: data is homoskedastic
Test Statistic: 0.152
Dist. under Null: chi-square with 2 degrees of freedom
P-value: 0.9268
Coefficients:
Intercept
SP500 SP500^20.0084-0.0106 -0.1638
Degrees of freedom: 131 total; 128 residual
Residual standard error: 0.01681

Notice that in this particular example the LM test and White’s test are
identical.

214

6. Time Series Regression Modeling6.5.3 The Newey-West Heteroskedasticity and
Autocorrelation Consistent (HAC) Covariance Matrix
Estimate
In some applications of time series regression, εt in (6.1) may be bothconditionally heteroskedastic and serially correlated. In this case, the error covariance matrix E[εε0 |X] is non-diagonal. Under certain assumptions
about the nature of the error heteroskedasticity and serial correlation a
consistent estimate of the generalized OLS covariance matrix can be computed. The most popular heteroskedasticity and autocorrelation consistent(HAC) covariance matrix estimate, due to Newey and West (1987), has the
form
avar
d HAC (β̂) = (X0 X)−1 ŜHAC (X0 X)−1
(6.20)
where

ŜHAC =TX
t=1

ε̂2t xt x0t +

q
X
l=1

wl

T
X

(xt ε̂t ε̂t−l x0t−l + xt−l ε̂t−l ε̂t x0t )(6.21)

t=l+1

is a nonparametric long-run variance estimate, and wl is the Bartlett weightfunctionlwl = 1 −q+1
The Bartlett weight function, wl , depends on a truncation parameter q thatmust grow with the sample size in order for the estimate to be consistent.
Newey and West suggested choosing q to be the integer part of 4(T /100)2/9 .
In some cases, a rectangular weight function½
1, for l ≤ q
wl =0, for l > q
is used if it is known that autocovariances in (6.21) cut oﬀ at lag q. The
square root of the diagonal elements of the HAC estimate (6.21) gives theheteroskedasticity and autocorrelation consistent standard errors (HACc HAC (β̂ i ).
SEs) for the least squares estimates of β i . These are denoted SE
Heteroskedasticity robust t-statistics and Wald statistics are computed inc HAC (β̂ i )the usual way using (6.4) and (6.6) but with avard HAC (β̂) and SEcreplacing avar(d β̂) and SE(β̂ i ), respectively.

Example 40 Long horizon regressions of stock returns on dividend-price
ratio

There has been much interest recently in whether long-term stock returns
are predictable by valuation ratios like dividend-to-price and earnings-toprice. See Chapter 7 in Campbell, Lo, and MacKinlay (1997), and Chapter
20 in Cochrane (2001) for reviews of this literature. Predictability is investigated by regressing future multiperiod stock returns on current values of6.5 HAC Covariance Matrix Estimation

215

valuation ratios. To illustrate, let rt denote the continuously compounded
real annual total return on an asset in year t and and let dt − pt denote thelog dividend price ratio. The typical long-horizon regression has the form
rt+1 + · · · + rt+K = αK + β K (dt − pt ) + εt+K , t = 1, . . . , T(6.22)

where rt+1 + · · · + rt+K is the continuously compounded future K-year real
total return. The dividend-price ratio predicts future returns if β K 6= 0 at
some horizon. Since the sampling frequency of the data is annual and thereturn horizon of the dependent variable is K years the dependent variable
and error term in (6.22) will behave like an MA(K − 1) process. This serial
correlation invalidates the usual formula for computing the estimated stanc HAC (β̂ K ),dard error of the least squares estimate of β K . The HACSE, SE
however, will provide a consistent estimate.
The long-horizon regression (6.22) with K = 10 years is estimated using
the annual stock price and dividend data on the S&P 500 composite index
in the S+FinMetrics “timeSeries” object shiller.annual. The relevant
data are constructed as> colIds(shiller.annual)[1] "price""dividend""earnings"[4] "cpi""real.price""real.dividend"[7] "real.earnings" "pe.10"> # compute log of real data
> ln.p = log(shiller.annual[,"real.price"])> colIds(ln.p) = "ln.p"> ln.d = log(shiller.annual[,"real.dividend"])> colIds(ln.d) = "ln.d"> ln.dpratio = ln.d - ln.p
> colIds(ln.dpratio) = "ln.dpratio"> # compute cc real total returns> ln.r = diff(ln.p) + log(1+exp(ln.dpratio[-1,]))> colIds(ln.r) = "ln.r"> # create 10-year cc total returns> ln.r.10 = aggregateSeries(ln.r,moving=10,FUN=sum)> colIds(ln.r.10) = "ln.r.10"> stockdiv.ts = seriesMerge(ln.p,ln.d,ln.dpratio,+ ln.r,ln.r.10,pos="union")
The continuously compounded real total return is computed asµ¶
Pt + Dt − Pt−1
rt = ln=
Pt−1
ln(Pt /Pt−1 ) + ln(1 + exp(ln(Dt ) − ln(Pt )))where Pt is the real price and Dt is the real dividend. Notice how the S-PLUS
function aggregateSeries is used to compute the 10 year continuously
compounded real total returns.

216

6. Time Series Regression Modeling

Residual Autocorrelation

ols.fit10
1.0

0.8

ACF

0.6

0.4

0.2

0.0

-0.2

5

10

15

Lag

FIGURE 6.5. Residual ACF from regression of ten year real returns on dividend-price ratio.

The long-horizon regression (6.22) using10 year real total return returns
over the postwar period 1947 - 1995 is computed using> ols.fit10 = OLS(ln.r.10~tslag(ln.dpratio),data=stockdiv.ts,+ start="1947", end="1995", in.format="%Y", na.rm=T)
Figure 6.5 shows the residual ACF. There is clearly serial correlation in
the residuals. The HACSEs are computed using summary with the optional
argument correction="nw". By default, the Newey-West HAC covariancematrix is computed using a Bartlett kernel with automatic lag truncationq = 4(T /100)2/9 . In the present context, the serial correlation is known
to be of the form of an MA(9) process. Therefore, it is more appropriate
to compute the Newey-West HAC covariance using a rectangular weightfunction with q = 9 which is accomplished by specifying bandwidth=9 andwindow="rectangular" in the call to summary:> summary(ols.fit10, correction="nw", bandwidth=9,+ window="rectangular")
Call:
OLS(formula = ln.r.10 ~tslag(ln.dpratio), data =stockdiv.ts, na.rm = T, start = "1947", end ="1995", in.format = "%Y")

6.6 Recursive Least Squares Estimation

Residuals:
Min
1Q-0.6564 -0.2952

Median0.00303Q
0.1799217

Max
0.9997

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 5.7414 0.9633
5.9600 0.0000
tslag(ln.dpratio) 1.5604 0.3273
4.7668 0.0000
Regression Diagnostics:
R-Squared 0.5012
Adjusted R-Squared 0.4896
Durbin-Watson Stat 0.2554
Residual Diagnostics:
Stat P-Value
Jarque-Bera1.71040.4252
Ljung-Box 105.9256
0.0000Residual standard error: 0.4116 on 43 degrees of freedom
Time period: from 1947 to 1991
F-statistic: 43.21 on 1 and 43 degrees of freedom, thep-value is 5.359e-008
Notice the low DW statistic and the large value of the Ljung-Box statistic
indicating serial correlation in the residuals. The regression results with thecorrected standard errors indicate that future 10 year real total returns arehighly predictable and positively related to the current dividend-price ratio.
c HAC (β̂ 10 ) = 0.416 andThe predictability coeﬃcient is β̂ 10 = 1.560 with SE
R2 = 0.501.

6.6 Recursive Least Squares Estimation
The time series regression model (6.1) assumes that the parameters of themodel, β, are constant over the estimation sample. A simple and intuitive
way to investigate parameter constancy is to compute recursive estimatesof β; that is, to estimate the modelyt = β0t xt + εt(6.23)by least squares recursively for t = k + 1, . . . , T giving T − k recursive leastsquares (RLS) estimates (β̂ k+1 , . . . , β̂ T ). If β is really constant then the218

6. Time Series Regression Modelingrecursive estimates β̂t should quickly settle down near a common value. If
some of the elements in β are not constant then the corresponding RLS
estimates should show instability. Hence, a simple graphical technique foruncovering parameter instability is to plot the RLS estimates β̂ it (i =0, . . . , k) and look for instability in the plots.
An alternative approach to investigate parameter instability is to compute estimates of the model’s parameters over a fixed rolling window of agiven length. Such rolling analysis is discussed in Chapter 9.

6.6.1 CUSUM and CUSUMSQ Tests for Parameter Stability
Brown, Durbin and Evans (1976) utilize the RLS estimates of (6.23) andpropose two simple tests for parameter instability. These tests, know as
the CUSUM and CUSUMSQ tests, are based on the standardized 1-stepahead recursive residuals
0yt − β̂t−1 xt
ε̂t
ŵt =
=
ˆ
ft
fˆt
where fˆt2 is an estimate of the recursive error variancehi0−1
ft2 = σ 2 1 + x0t (Xt−1 Xt−1 ) xt

and Xt is the (t × k) matrix of observations on xs using data from s =1, . . . , t.
The CUSUM test is based on the cumulated sum of the standardized
recursive residualst
Xŵj
CUSUMt =
σ̂ w
j=k+1

where σ̂ w is the sample standard deviation of ŵj . Under the null hypothesisthat β in (6.1) is constant, CUSUMt has mean zero and variance that is
proportional to t−k−1. Brown, Durbin and Evans (1976) show that approximate 95% confidence bands for√ CUSUMt are given by the two√ lines whichconnect the points (k, ±0.948 T − k − 1) and (T, ±0.948 · 3 T − k − 1).
If CUSUMt wanders outside of these bands, then the null of parameterstability may be rejected.
The CUSUMSQ test is based on the cumulative sum of the squared standardized recursive residuals and is given byPt
2
j=k+1 ŵj
CUSUMSQt = PT.
2
j=k+1 ŵj
The distribution of CUSUMSQt under the null of parameter stability isgiven in Brown, Durbin and Evans (1976) where it√is shown that 95% confidence bands for CUSUMSQt have the form c ± t/ T − k − 1. As with the6.6 Recursive Least Squares Estimation

219

CUSUMt statistic, if CUSUMSQt wanders outside the confidence bands,
then the null of parameter stability may be rejected.

6.6.2 Computing Recursive Least Squares Estimates Using
the S+FinMetrics Function RLS
Eﬃcient RLS estimation of the time series regression model (6.1) may be
performed using the S+FinMetrics function RLS. The calling syntax of RLS
is exactly the same as that of OLS so that any model that may be estimatedusing OLS may also be estimated using RLS. For example, to compute the
RLS estimates of the CAPM regression for Microsoft use> rls.fit = RLS(MSFT~SP500, data=excessRet.ts)> class(rls.fit)[1] "RLS"RLS produces an object of class “RLS” for which there are coef, plot,
print and residuals methods. The print method give a basic descriptionof the RLS fit> rls.fit
Call:
RLS(formula = MSFT ~SP500, data = excessRet.ts)
Time period: from Feb 1990 to Dec 2000
Coefficients:(Intercept) SP500
mean 0.0284
1.2975std. dev. 0.0121
0.1531
Recursive Residuals Summary:mean std. dev.
-0.0157 0.0893
The recursive intercept and slope estimates do not seem to vary toomuch. The plot method allows one to see the recursive coeﬃcients, CUSUM
and CUSUMSQ residuals> plot(rls.fit)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Coefficient Estimates
3: plot: CUSUM of Residuals
4: plot: CUSUM of Squared Residuals

220

6. Time Series Regression Modeling

Recursive Coefficients

0.02

SP500

1.2

1.4

1.6

1.8

2.0

Value

0.04

0.06

(Intercept)

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

Index

FIGURE 6.6. RLS coeﬃcient estimates from the CAPM regression for Microsoft.

CUSUM of Residuals

-30

-20

-10

0

10

20

30

rls.fit

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 6.7. CUSUM of residuals from the CAPM regression for Microsoft.

6.7 References

221

CUSUM of Squared Residuals

0.0

0.2

0.4

0.6

0.8

1.0

rls.fit

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 6.8. CUSUMSQ of residuals from CAPM regression for Microsoft.

Selection:
Figures 6.6, 6.7 and 6.8 show the plots from options 2, 3 and 4. The RLS
estimates of α and β settle down in the middle of the sample but thenthe estimates of α decrease and the estimates of β increase. The CUSUMt
statistics stay within the 95% confidence bands but the CUSUMSQt statistics wander outside the bands. Hence, there is some evidence for instability
in the CAPM coeﬃcients.

6.7 References
Breusch, T. and A. Pagan (1979). “A Simple Test for Heteroscedasticity
and Random Coeﬃcient Variation,” Econometrica, 47, 1287-1294.
Brown, R., J. Durbin and J. Evans (1976). “Techniques for Testing theConstancy of Regression Relationships over Time,” Journal of the Royal
Statistical Society, Series B, 37, 149-172.
Campbell, J. A. Lo, C. MacKinlay (1997). The Econometrics of Financial Markets. Princeton University Press, Princeton, NJ.

222

6. Time Series Regression Modeling

Cochrane, J. (2001). Asset Pricing. Princeton University Press, Princeton, NJ.
Eicker, F. (1967). “Limit Theorems for Regression with Unequal and
Dependent Errors,” in L. LeCam and J. Neyman (eds.), Proceedings of
the 5th Berkeley Symposium on Mathematical Statistics and Probability.
University of California Press, Berkeley.
Greene, W. (2000). Econometric Analysis, Fifth Edition. Prentice Hall,
New Jersey.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
Hayashi, F. (2000). Econometrics. Princeton University Press, Princeton,
NJ.
Jarque, C.M. and A.K. Bera (1981). “Eﬃcients Tests for Normality,
Homoskedasticity and Serial Dependence of Regression Residuals,” Economics Letters, 6, 255-259.
Koenker, R. and G. Bassett (1982). “Robust Tests for Heteroscedasticity Based on Regression Quantiles,” Econometrica, 50, 43-61.
Mills, T.C. (1999). The Econometrics Modelling of Financial Time Series, Second Edition. Cambridge University Press, Cambridge.
Newey, W.K. and K.D. West (1987). “A Simple Positive Semidefinite Heteroskedasticity and Autocorrelation Consistent Covariance Matrix,” Econometrica, 55, 703-708.
White, H. (1980). “A Heteroskedasticity Consistent Covariance Matrix
Estimator and a Direct Test for Heteroskedasticity,” Econometrica, 48,
817-838.

This is page 223
Printer: Opaque this7
Univariate GARCH Modeling

7.1 Introduction
Previous chapters have concentrated on modeling and predicting the conditional mean, or the first order moment, of a univariate time series, andare rarely concerned with the conditional variance, or the second order moment, of a time series. However, it is well known that in financial markets
large changes tend to be followed by large changes, and small changes tend
to be followed by small changes. In other words, the financial markets are
sometimes more volatile, and sometimes less active.
The volatile behavior in financial markets is usually referred to as the“volatility”. Volatility has become a very important concept in diﬀerent
areas in financial theory and practice, such as risk management, portfolio
selection, derivative pricing, etc. In statistical terms, volatility is usually
measured by variance, or standard deviation. This chapter introduces theclass of univariate generalized autoregressive conditional heteroskedasticity(GARCH) models developed by Engle (1982), Bollerslev (1986), Nelson(1991), and others, which are capable of modeling time varying volatility
and capturing many of the stylized facts of the volatility behavior usually observed in financial time series. It will show how to formulate, estimate, evaluate and predict from various types of GARCH models, such as
EGARCH, TGARCH, PGARCH, etc.
The outline of the chapter follows. Section 7.2 shows how to test for
ARCH eﬀects in a time series, then section 22.16 introduces the basic
GARCH model and its properties. GARCH model estimation and diagnos-

224

7. Univariate GARCH Modeling

-0.15

-0.10

-0.05

0.00

0.05

0.10

Daily Stock Returns of FORDQ2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1

1984

1985

1986

1987

1988

1989

1990

1991

1992

FIGURE 7.1. Daily Ford stock returns: ford.s.

tics using the S+FinMetrics family of GARCH functions are illustrated in
section 7.4. Section 7.5 extends the basic GARCH model to accommodate
some well-known stylized facts of financial time series. Prediction and simulation from various GARCH models are treated at the end of the chapter.
The statistical properties of GARCH models are nicely summarized in
Hamilton (1994), Tsay (2001) and the review papers by Bera and Higgins (1986), Bolerslev, Engle and Nelson (1994) and Diebold and Lopez(1996). Bollerslev, Chou and Kroner (1992) give a comprehensive surveyof GARCH modeling in finance. Alexander (2001) provides many examplesof the use of GARCH models in finance, and Engle (2001) and Engle andPatton (2001) discuss the usefulness of volatility modeling.

7.2 The Basic ARCH Model
Figure 7.1 plots a daily time series of Ford stock returns as contained in
the “timeSeries” object ford.s in S+FinMetrics:> class(ford.s)[1] "timeSeries"> plot(ford.s, reference.grid=F)7.2 The Basic ARCH Model

0.8
0.6
ACF
0.4
0.2
0.0

0.0

0.2

0.4

ACF

0.6

0.8

1.0

Series : ford.s^2

1.0

Series : ford.s

225

0

2

4

6
Lag

8

10

12

0

2

4

6
Lag

8

10

12

FIGURE 7.2. ACF of ford.s and ford.sˆ2.

Although there is little serial correlation in the time series ford.s itself,
it seems that both large changes and small changes are clustered together,which is typical of many high frequency macroeconomic and financial timeseries. To confirm this conjecture, use the S-PLUS function acf to look at
the autocorrelation plot of Ford returns and its squared returns:> par(mfrow=c(1,2))> tmp = acf(ford.s, lag=12)> tmp = acf(ford.s^2, lag=12)> par(mfrow=c(1,1))
The plots are shown in Figure 7.2. Obviously there is no autocorrelationin the return series itself, while the squared returns exhibit significant autocorrelation at least up to lag 5. Since the squared returns measure thesecond order moment of the original time series, this result indicates thatthe variance of ford.s conditional on its past history may change overtime, or equivalently, the time series ford.s may exhibit time varying conditional heteroskedasticity or volatility clustering.
The serial correlation in squared returns, or conditional heteroskedasticity, can be modeled using a simple autoregressive (AR) process for squared
residuals. For example, let yt denote a stationary time series such as financial returns, then yt can be expressed as its mean plus a white noise if there226

7. Univariate GARCH Modelingis no significant autocorrelation in yt itself:
yt = c + t(7.1)

where c is the mean of yt , and t is iid with mean zero. To allow for volatility
clustering or conditional heteroskedasticity, assume that V art−1 ( t ) = σ 2twith V art−1 (·) denoting the variance conditional on information at timet − 1, and(7.2)
σ 2t = a0 + a1 2t−1 + · · · + ap 2t−p .
since t has a zero mean, V art−1 ( ) = Et−1 ( 2t ) = σ 2t , the above equation
can be rewritten as:222
t = a0 + a1 t−1 + · · · + ap t−p + ut

(7.3)

where ut = 2t − Et−1 ( 2t ) is a zero mean white noise process. The above
equation represents an AR(p) process for 2t , and the model in (7.1) and(7.2) is known as the autoregressive conditional heteroskedasticity (ARCH)
model of Engle (1982), which is usually referred to as the ARCH(p) model.
An alternative formulation of the ARCH model is
yt
t

σ 2t

= c+ t= zt σt= a0 + a1 2t−1 + · · · + ap 2t−pwhere zt is an iid random variable with a specified distribution. In thebasic ARCH model zt is assumed to be iid standard normal. The above
representation is convenient for deriving properties of the model as well as
for specifying the likelihood function for estimation.
Exercise 41 Simulating an ARCH(p) model
The S+FinMetrics function simulate.garch may be used to simulate
observations from a variety of time-varying conditional heteroskedasticity models. For example, to simulate 250 observations from the ARCH(p)model (7.1)-(7.2) with c = 0, p = 1, a0 = 0.01 and a1 = 0.8 use> sim.arch1 = simulate.garch(model=list(a.value=0.01,arch=0.8),+n=250, rseed=196)> names(sim.arch1)[1] "et""sigma.t"
The component et contains the ARCH errors t and the component sigma.t
contains the conditional standard deviations σ t . These components are illustrated in Figure 7.3 created using> par(mfrow=c(2,1))> tsplot(sim.arch1$et,main="Simulated ARCH(1) errors",

7.2 The Basic ARCH Model

227

-0.6

-0.2

e(t)

0.2

0.6

Simulated ARCH(1) errors

0

50

100

150

200

250

200

250

0.4
0.3
0.1

0.2

sigma(t)0.5

0.6

Simulated ARCH(1) volatility

0

50

100

150

FIGURE 7.3. Simulated values of εt and σt from ARCH(1) process.

+ ylab="e(t)")> tsplot(sim.arch1$sigma.t,+ main="Simulated ARCH(1) volatility",ylab="sigma(t)")
Some summary statistics for the simulated data are> summaryStats(sim.arch1$et)
Sample Quantiles:min
1Q median
3Q
max-0.6606 -0.1135 0.0112 0.1095 0.6357
Sample Moments:
mean
std skewness kurtosis-0.003408 0.1846 -0.2515
4.041
Number of Observations:250

Notice the somewhat high kurtosis value (relative to the kurtosis value of 3
for a normal distribution). Finally, Figure 7.4 shows the sample ACFs for2t and σ t . Both series exhibit almost identical serial correlation properties.

228

7. Univariate GARCH Modeling

ACF

0.0

0.2

0.4

0.6

0.8

1.0

Series : sim.arch1$et^2

0

5

10

15

20

Lag

ACF

0.0

0.2

0.4

0.6

0.8

1.0

Series : sim.arch1$sigma.t^2

0

5

10

15

20

Lag

FIGURE 7.4. Sample ACFs for ε2t and σ2t from simulated ARCH(1) process.

7.2.1 Testing for ARCH Eﬀects
Before estimating a full ARCH model for a financial time series, it is usually
good practice to test for the presence of ARCH eﬀects in the residuals.
If there are no ARCH eﬀects in the residuals, then the ARCH model isunnecessary and misspecified.
Since an ARCH model can be written as an AR model in terms of squared
residuals as in (7.3), a simple Lagrange Multiplier (LM) test for ARCHeﬀects can be constructed based on the auxiliary regression (7.3). Under
the null hypothesis that there are no ARCH eﬀects: a1 = a2 = · · · = ap = 0,
the test statisticA
LM = T · R2 ∼ χ2 (p)where T is the sample size and R2 is computed from the regression (7.3)
using estimated residuals.1
The S+FinMetrics function archTest can be used to carry out the above
test for ARCH eﬀects. For example, to test for the presence of ARCH eﬀects
in ford.s, use the following command:> archTest(ford.s, lag.n=12)
Test for ARCH Effects: LM Test
1 We refer to Engle (1982) for details.

7.3 The GARCH Model and Its Properties

229

Null Hypothesis: no ARCH effects
Test Statistics:
FORD
Test Stat 112.6884
p.value
0.0000Dist. under Null: chi-square with 12 degrees of freedom
Total Observ.: 2000
In this case, the p-value is essentially zero, which is smaller than the conventional 5% level, so reject the null hypothesis that there are no ARCHeﬀects. Note that archTest function takes a time series and an optional
argument lag.n specifying the order of the ARCH eﬀects. Since S-PLUS
allows lazy evaluation, lag instead of lag.n could have been supplied as
the optional argument.

7.3 The GARCH Model and Its Properties
If the LM test for ARCH eﬀects is significant for a time series, one could
proceed to estimate an ARCH model and obtain estimates of the timevarying volatility σ t based on past history. However, in practice it is often
found that a large number of lags p, and thus a large number of parameters,
is required to obtain a good model fit. A more parsimonious model proposed
by Bollerslev (1986) replaces the AR model in (7.2) with the following
formulation:pqXX
σ2t = a0 +ai 2t−i +bj σ 2t−j
(7.4)
i=1

j=1

where the coeﬃcients ai (i = 0, · · · , p) and bj (j = 1, · · · , q) are all assumed
to be positive to ensure that the conditional variance σ2t is always positive.2
The model in (7.4) together with (7.1) is known as the generalized ARCH
or GARCH(p, q) model. When q = 0, the GARCH model reduces to the
ARCH model.
Under the GARCH(p, q) model, the conditional variance of t , σ 2t , depends on the squared residuals in the previous p periods, and the conditional variance in the previous q periods. Usually a GARCH(1,1) modelwith only three parameters in the conditional variance equation is adequate
to obtain a good model fit for financial time series.
2 Positive coeﬃcients are suﬃcient but not necessary conditions for the positivity ofconditional variance. We refer to Nelson and Cao (1992) for the general conditions.

230

7. Univariate GARCH Modeling7.3.1 ARMA Representation of GARCH Model
Just as an ARCH model can be expressed as an AR model of squared
residuals, a GARCH model can be expressed as an ARMA model of squared
residuals. Consider the GARCH(1,1) model:
σ 2t = a0 + a1 2t−1 + b1 σ 2t−1 .

(7.5)

Since Et−1 ( 2t ) = σ 2t , the above equation can be rewritten as:22
t = a0 + (a1 + b1 ) t−1 + ut − b1 ut−1

(7.6)

which is an ARMA(1,1) model with ut = 2t − Et−1 ( 2t ) being the white
noise disturbance term.
Given the ARMA representation of the GARCH model, many properties of the GARCH model follow easily from those of the corresponding
ARMA process for 2t . For example, for the GARCH(1,1) model to be stationary, requires that a1 + b1 < 1 as in (7.6). Assuming the stationarity of
GARCH(1,1) model, the unconditional variance of t can be shown to be
V ar( t ) = E( 2t ) = a0 /(1 − a1 − b1 ), because from (7.6):
E( 2t ) = a0 + (a1 + b1 )E( 2t−1 )and thusE( 2t ) = a0 + (a1 + b1 )E( 2t )based on the assumption that 2t is stationary.
For the general GARCH(p, q) model (7.4), the squared residuals t behaveq), q) process. Covariance stationarity requires
Pp like an
PARMA(max(p,qi=1 ai +j=1 bi < 1 and the unconditional variance of t is
V ar( t ) =

1−

³P
p

a0

i=1 ai +Pq

j=1 bi´.

(7.7)

7.3.2 GARCH Model and Stylized Facts
In practice, researchers have uncovered many so-called “stylized facts”
about the volatility of financial time series; Bollerslev, Engle and Nelson(1994) gave a complete account of these facts. Using the ARMA representation of GARCH models shows that the GARCH model is capable ofexplaining many of those stylized facts. This section will focus on three important ones: volatility clustering, fat tails, and volatility mean reversion.
Other stylized facts are illustrated and explained in later sections.
Volatility Clustering
Consider the GARCH(1, 1) model in (7.5). Usually the GARCH coeﬃcient
b1 is found to be around 0.9 for many weekly or daily financial time series.

7.3 The GARCH Model and Its Properties

231

Given this value of b1 , it is obvious that large values of σ 2t−1 will be followed by large values of σ 2t , and small values of σ 2t−1 will be followed bysmall values of σ 2t . The same reasoning can be obtained from the ARMA
representation in (7.6), where large/small changes in 2t−1 will be followed
by large/small changes in 2t .
Fat Tails
It is well known that the distribution of many high frequency financial timeseries usually have fatter tails than a normal distribution. That is, largechanges are more often to occur than a normal distribution would imply.
Bollerslev (1986) gave the condition for the existence of the fourth ordermoment of a GARCH(1, 1) process. Assuming the fourth order moment exists, Bollerslev (1986) showed that the kurtosis implied by a GARCH(1, 1)
process is greater than 3, the kurtosis of a normal distribution. He andTeräsvirta (1999a, 1999b) extended these results to general GARCH(p, q)
models. Thus a GARCH model can replicate the fat tails usually observedin financial time series.

Volatility Mean Reversion
Although financial markets may experience excessive volatility from time
to time, it appears that volatility will eventually settle down to a long
run level. The previous subsection showed that the long run variance oft for the stationary GARCH(1, 1) model is a0 /(1 − a1 − b1 ). In this case,the volatility is always pulled toward this long run level by rewriting the
ARMA representation in (7.6) as follows:( 2t −a0
a0) = (a1 + b1 )( 2t−1 −) + ut − b1 ut−1 .
1 − a1 − b1
1 − a1 − b1

If the above equation is iterated k times, one can show that
( 2t+k −a0a0) = (a1 + b1 )k ( 2t −) + ηt+k
1 − a1 − b1
1 − a1 − b1

where ηt is a moving average process. Since a1 + b1 < 1 for a stationary
GARCH(1, 1) model, (a1 + b1 )k → 0 as k → ∞. Although at time t there
may be a large deviation between 2t and the long run variance, 2t+k −a0 /(1 − a1 − b1 ) will approach zero “on average” as k gets large, i.e., thevolatility “mean reverts” to its long run level a0 /(1 − a1 − b1 ). In contrast,
if a1 + b1 > 1 and the GARCH model is non-stationary, the volatility will
eventually explode to infinity as k → ∞. Similar arguments can be easily
constructed for a GARCH(p, q) model.

232

7. Univariate GARCH Modeling

7.4 GARCH Modeling Using S+FinMetrics
7.4.1 GARCH Model Estimation
This section illustrates how to estimate a GARCH model using functions
in S+FinMetrics. Recall, the general GARCH(p, q) model has the form
yt = c + tpqXX
σ2t = a0 +ai 2t−i +bj σ 2t−j
i=1

(7.8)
(7.9)

j=1

for t = 1, · · · , T , where σ 2t = V art−1 ( t ). Assuming that t follows normal or Gaussian distribution conditional on past history, the prediction
error decomposition of the log-likelihood function of the GARCH model
conditional on initial values is:log L = −TTT
1 X 2t1X
log σ 2t −.
log(2π) −2
2 t=1
2 t=1 σ 2t(7.10)The unknown model parameters c, ai (i = 0, · · · , p) and bj (j = 1, · · · , q)
can be estimated using conditional maximum likelihood estimation (MLE).
Details of the maximization are given in Hamilton (1994). Once the MLE
estimates of the parameters are found, estimates of the time varying volatility σt (t = 1, . . . , T ) are also obtained as a side product.
For a univariate time series, S+FinMetrics provides the garch function
for GARCH model estimation. For example, to fit a simple GARCH(1,1)
model as in (7.8) and (7.9) to the “timeSeries” object ford.s, use thecommand:> ford.mod11 = garch(ford.s~1, ~garch(1,1))
Iteration
0 Step Size = 1.00000 Likelihood =Iteration
0 Step Size = 2.00000 Likelihood =Iteration
1 Step Size = 1.00000 Likelihood =Iteration
1 Step Size = 2.00000 Likelihood =Iteration
1 Step Size = 4.00000 Likelihood =Iteration
2 Step Size = 1.00000 Likelihood =Iteration
2 Step Size = 2.00000 Likelihood =Iteration
2 Step Size = 4.00000 Likelihood =2.62618
2.612372.62720
2.627692.59047
2.62785
2.62795
2.62793

Convergence R-Square = 4.630129e-05 is less than tolerance= 0.0001
Convergence reached.
In the above example, the garch function takes two arguments: the first argument is an S-PLUS formula which specifies the conditional mean equation(7.8), while the second argument is also an S-PLUS formula which specifies

7.4 GARCH Modeling Using S+FinMetrics

233

the conditional variance equation (7.9). The specification of the conditionalmean formula is the same as usual S-PLUS formulas.3 For the conditionalvariance formula, nothing needs to be specified on the left hand side, andthe garch(1,1) term on the right hand side denotes the GARCH(1, 1)
model. By default, the progress of the estimation is printed on screen. Those
messages can be suppressed by setting the optional argument trace=F in
the call to the garch function.
The object returned by garch function is of class “garch”. Typing thename of the object at the command line invokes its print method:> class(ford.mod11)[1] "garch"> ford.mod11
Call:garch(formula.mean = ford.s ~ 1, formula.var =~ garch(1, 1))

Mean Equation: ford.s ~ 1
Conditional Variance Equation:~ garch(1, 1)

Coefficients:
C 7.708e-04
A 6.534e-06
ARCH(1) 7.454e-02
GARCH(1) 9.102e-01
The print method for a “garch” object shows the formulas for the conditional mean equation and conditional variance equation, together withthe estimated model coeﬃcients. Note that in the output C corresponds
to the constant c in the conditional mean equation (7.8), A, ARCH(1) and
GARCH(1) correspond to a0 , a1 and b1 in the conditional variance equation (7.9), respectively. Notice that the estimated GARCH(1) parameter isclose to one and the ARCH(1) parameter is close to zero. The sum of theseparameters is 0.985 which indicates a covariance stationary model witha high degree of persistence in the conditional variance. Use the S-PLUS
function names to extract the component names for a “garch” object. For
example:> names(ford.mod11)[1] "residuals" "sigma.t""df.residual" "coef" "model"[6] "cond.dist" "likelihood" "opt.index""cov" "prediction"[11] "call""asymp.sd""series"
3 Chapter 1 provides a review of the usage of S-PLUS formulas and modeling functions.

234

7. Univariate GARCH Modeling

It should be clear what most of the components are and the on-line help filefor the garch function provides details for these components. Of particularinterest is the component asymp.sd, which gives an estimate of the unconditional standard deviation of the GARCH residuals provided the GARCH
model is stationary. That is,> ford.mod11$asymp.sd[1] 0.02068
is an estimate of the square root of a0 /(1 − α1 − b1 ).
For most components that a user is interested in, S+FinMetrics provides methods for generic functions such as coef, residuals, and vcov for
extracting those components. For example, the estimated coeﬃcients canbe extracted by calling the generic coef function:> coef(ford.mod11)
C 7.708418e-04
A 6.534363e-06
ARCH(1) 7.454134e-02
GARCH(1) 9.101842e-01
Similarly, call the generic vcov function to obtain the covariance matrix ofthe estimated coeﬃcients:> vcov(ford.mod11)
C
A
ARCH(1)GARCH(1)C 1.41574e-07 -1.21204e-13 -3.56991e-07 2.21310e-07
A -1.21204e-13 3.04607e-12 2.55328e-09 -1.24396e-08
ARCH(1) -3.56991e-07 2.55328e-09 2.87505e-05 -3.43277e-05
GARCH(1) 2.21310e-07 -1.24396e-08 -3.43277e-05 7.67660e-05
By default, the vcov method for “garch” objects uses the covariance matrix based on the outer product of gradients. However, for maximum likelihood estimation, there are three diﬀerent ways of computing the covariancematrix of model parameters which are asymptotically equivalent if the underlying error distribution is Gaussian: one based on the outer product ofgradients, one based on the numerical Hessian matrix, and one based on
the asymptotic formula for quasi-maximum likelihood estimation (QMLE).
These diﬀerent covariance matrices can be obtained by setting the optional
argument method to "op", "hessian", or "qmle", respectively. For example, to obtain the covariance matrix of ford.mod11 parameters based on
QMLE formula, use the following command:> vcov(ford.mod11, method="qmle")CA
C 1.26671e-07 -7.54398e-11
A -7.54398e-11 2.69841e-11

ARCH(1)GARCH(1)5.67606e-07 -7.71183e-08
1.37576e-07 -2.00363e-07

7.4 GARCH Modeling Using S+FinMetrics

235

ARCH(1) 5.67606e-07 1.37576e-07 1.28016e-03 -1.46718e-03
GARCH(1) -7.71183e-08 -2.00363e-07 -1.46718e-03 1.84173e-03
This covariance matrix is sometimes referred to as the robust covariancematrix, because it is robust to possible misspecification of the error distribution, or the sandwich estimate, because of the form of the asymptotic
formula (see Bollerslev and Wooldrige, 1992 or Davidson and MacKinnon,
1993).
The residuals method for a “garch” object takes an optional argument
standardize, which can be used to obtain estimates of the standardized
residuals t /σ t . For example:> residuals(ford.mod11, standardize=T)
returns the standardized residuals of the fitted GARCH model ford.mod11.
S+FinMetrics also provides another function sigma.t for extracting thefitted volatility series σ t . Note that if the original data is a “timeSeries”
object, the calendar information of the original data is also retained in theresidual and volatility series.

7.4.2 GARCH Model Diagnostics
The previous subsection showed how to estimate a GARCH model using
the S+FinMetrics function garch and how to extract various componentsof the fitted model. To assess the model fit, S+FinMetrics provides method
functions for two generic functions: summary and plot, one for statisticalsummary information and the other for visual diagnostics of the model fit.
For example, to obtain a more detailed summary of ford.mod11, call thegeneric summary function:> summary(ford.mod11)
Call:garch(formula.mean = ford.s ~ 1, formula.var =~ garch(1, 1))

Mean Equation: ford.s ~ 1
Conditional Variance Equation:
Conditional Distribution:~ garch(1, 1)

gaussian-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value Pr(>|t|)
C 7.708e-04 3.763e-04
2.049 2.031e-02

236

7. Univariate GARCH Modeling

A 6.534e-06 1.745e-06
3.744 9.313e-05
ARCH(1) 7.454e-02 5.362e-03 13.902 0.000e+00
GARCH(1) 9.102e-01 8.762e-03 103.883 0.000e+00-------------------------------------------------------------AIC(4) = -10503.79
BIC(4) = -10481.39
Normality Test:-------------------------------------------------------------Jarque-Bera P-value Shapiro-Wilk P-value
364.2
00.9915 0.9777
Ljung-Box test for standardized residuals:-------------------------------------------------------------Statistic P-value Chi^2-d.f.
14.82 0.2516
12
Ljung-Box test for squared standardized residuals:-------------------------------------------------------------Statistic P-value Chi^2-d.f.
14.04 0.2984
12
Lagrange multiplier test:-------------------------------------------------------------Lag 1 Lag 2 Lag 3
Lag 4
Lag 5
Lag 6 Lag 7
Lag 8
2.135 -1.085 -2.149 -0.1347 -0.9144 -0.2228 0.708 -0.2314
Lag 9 Lag 10 Lag 11 Lag 12C-0.6905 -1.131 -0.3081 -0.1018 0.9825
TR^2 P-value F-stat P-value
14.77 0.2545 1.352 0.2989
By default, the summary method shows the standard errors and p-values
for the t-statistics for testing that the true coeﬃcients are zero, together
with various tests on the standardized residuals ˆt /σ̂t for assessing themodel fit. The standard errors and p-values are computed using the default
covariance estimate. To use robust or numerical Hessian based standard
errors to compute the p-values, the summary method takes an optional
argument method just like the vcov method does.
The various tests returned by the summary method can also be performed
separately by using standard S+FinMetrics functions. For example, if themodel is successful at modeling the serial correlation structure in the condi-

7.4 GARCH Modeling Using S+FinMetrics

237

tional mean and conditional variance, then there should be no autocorrelation left in the standardized residuals and squared standardized residuals.
This can be checked by using the S+FinMetrics function autocorTest:> autocorTest(residuals(ford.mod11, standardize=T), lag=12)
Test for Autocorrelation: Ljung-Box
Null Hypothesis: no autocorrelation
Test Statistics:
Test Stat 14.8161
p.value 0.2516
Dist. under Null: chi-square with 12 degrees of freedom
Total Observ.: 2000
> autocorTest(residuals(ford.mod11, standardize=T)^2, lag=12)
Test for Autocorrelation: Ljung-Box
Null Hypothesis: no autocorrelation
Test Statistics:
Test Stat 14.0361
p.value 0.2984
Dist. under Null: chi-square with 12 degrees of freedom
Total Observ.: 2000
In both cases, the tests are the same as those returned by the summary
method, and the null hypothesis that there is no autocorrelation left cannot be rejected because the p-values in both cases are greater than theconventional 5% level. Note that lag was chosen to be 12 to match theresults returned by the summary method.
Similarly, one can also apply the ARCH test on the standardized residuals
to see if there are any ARCH eﬀects left. For example, call archTest on
the standardized residuals of ford.mod11 as follows:> archTest(residuals(ford.mod11, standardize=T), lag=12)
Test for ARCH Effects: LM Test
Null Hypothesis: no ARCH effects

238

7. Univariate GARCH Modeling

QQ-Plot of Standardized Residuals

QQ-Plot
12/23/1991

Standardized Residuals

4

2

0

-2

-4
10/13/1989-6
10/19/1987
-2

0

2

Quantiles of gaussian distribution

FIGURE 7.5. Normal qq-plot of standardized residuals: ford.mod11.

Test Statistics:
Test Stat 14.7664
p.value 0.2545
Dist. under Null: chi-square with 12 degrees of freedom
Total Observ.: 2000
Again, the results match the Lagrange Multiplier test as returned by thesummary method.
The basic garch model assumes a normal distribution for the errors t .
If the model is correctly specified then the estimated standardized residuals t /σ t should behave like a standard normal random variable. To evaluate the normality assumption, the summary method reports both the
Jarque-Bera test and the Shapiro-Wilks test for the standardized residuals,
which again can be performed separately using the S+FinMetrics function
normalTest. However, in the above example, the Jarque-Bera test and theShapiro-Wilks test lead to opposite conclusions, with one p-value close tozero and the other close to one.
To get a more decisive conclusion regarding the normality assumption,
resort to the qq-plot by calling the generic plot function on a “garch”
object:7.4 GARCH Modeling Using S+FinMetrics

239

ACF of Squared Std. Residuals

ACF
1.0

0.8

0.6

0.4

0.2

0.0

0

10

20

30

Lags

FIGURE 7.6. ACF of squared standardized residuals: ford.mod11.

> plot(ford.mod11)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Series and Conditional SD
3: plot: Series with 2 Conditional SD Superimposed
4: plot: ACF of the Observations
5: plot: ACF of Squared Observations
6: plot: Cross Correlation between Squared Series and Series
7: plot: Residuals
8: plot: Conditional Standard Deviations
9: plot: Standardized Residuals
10: plot: ACF of Standardized Residuals
11: plot: ACF of Squared Standardized Residuals
12: plot: Cross Correlation between Squared Std.Res and Std.
13: plot: QQ-Plot of Standardized Residuals
Selection:
By selecting 13, the qq-plot of standardized residuals can be obtained asshown in Figure 7.5 In this case, there is significant deviation in both tails
from the normal qq-line, and thus it seems that the normality assumption
for the residuals may not be appropriate. Section 7.5.6 will show how to240

7. Univariate GARCH Modeling

Series and Conditional SD

Values

-0.1

0.0

0.1

Original Series

0.02

0.03

0.04

0.05

Conditional SD

Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1

1984

1985

1986

1987

1988

1989

1990

1991

1992

FIGURE 7.7. Daily Ford stock returns and conditional volatility.

use alternative distributions.Other plots can also be chosen to visualize themodel fit. For example, choosing 11 generates the ACF plot of squared
standardized residuals as shown in Figure 7.6, which shows that there islittle autocorrelation left in the squared standardized residuals. Choosing
2 plots the original return series and the fitted volatility series as shown in
Figure 7.7.

7.5 GARCH Model Extensions
In many cases, the basic GARCH model (7.4) provides a reasonably goodmodel for analyzing financial time series and estimating conditional volatility. However, there are some aspects of the model which can be improved sothat it can better capture the characteristics and dynamics of a particulartime series. For example, the previous section showed that the normality
assumption may not be appropriate for the time series ford.s. This section introduces several extensions to the basic GARCH model that make
GARCH modeling more flexible and shows how to estimate those models
using the S+FinMetrics garch function.

7.5 GARCH Model Extensions

241

7.5.1 Asymmetric Leverage Eﬀects and News Impact
In the basic GARCH model (7.9), since only squared residuals 2t−i enter theequation, the signs of the residuals or shocks have no eﬀects on conditional
volatility. However, a stylized fact of financial volatility is that bad news
(negative shocks) tends to have a larger impact on volatility than goodnews (positive shocks). Black (1976) attributed this eﬀect to the fact thatbad news tends to drive down the stock price, thus increasing the leverage(i.e., the debt-equity ratio) of the stock and causing the stock to be morevolatile. Based on this conjecture, the asymmetric news impact is usuallyreferred to as the leverage eﬀect. All the GARCH variants implemented in
S+FinMetrics are capable of incorporating leverage eﬀects. This subsection
focuses on the EGARCH, TGARCH and PGARCH models.
EGARCH Model
Nelson (1991) proposed the following exponential GARCH (EGARCH)
model to allow for leverage eﬀects:pXq| t−i | + γ i t−i X
ht = a0 +ai+
bj ht−j
σ t−i
i=1
j=1

(7.11)

where ht = log σ 2t or σ 2t = eht . Note that when t−i is positive or there is“good news”, the total eﬀect of t−i is (1 + γ i )| t−i |; in contrast, when t−i
is negative or there is “bad news”, the total eﬀect of t−i is (1 − γ i )| t−i |.
Bad news can have a larger impact on volatility, and the value of γ i would
be expected to be negative.
The garch function can be used to fit an EGARCH model by specifying~egarch(p,q) as the conditional variance formula. For example, to fit an
EGARCH(1, 1) model with leverage eﬀects using the daily Hewlett-Packard
stock returns contained in the S+FinMetrics “timeSeries” object hp.s,
use the following command:> hp.egarch = garch(hp.s~1, ~egarch(1,1), leverage=T, trace=F)> hp.egarch
Call:garch(formula.mean = hp.s ~ 1, formula.var =leverage = T, trace = F)~ egarch(1, 1),

Mean Equation: hp.s ~ 1
Conditional Variance Equation:
Coefficients:~ egarch(1, 1)

242

7. Univariate GARCH Modeling

C 0.000313
A -1.037907
ARCH(1) 0.227878
GARCH(1) 0.886652
LEV(1) -0.133998
Note that the optional argument trace=F is set to suppress the iteration
messages, and set leverage=T to impose leverage eﬀects. In the output,
the estimated γ 1 coeﬃcient for the leverage eﬀect is denoted by LEV(1)andis negative in this case. The t-statistic for testing γ 1 = 0 is
> coef(hp.egarch)[5]/sqrt(vcov(hp.egarch)[5,5])[1] -2.159
Another advantage of the EGARCH model over the basic GARCH modelis that the conditional variance σ 2t is guaranteed to be positive regardless ofthe values of the coeﬃcients in (7.11), because the logarithm of σ2t insteadof σ2t itself is modeled.
TGARCH Model
Another GARCH variant that is capable of modeling leverage eﬀects is thethreshold GARCH (TGARCH) model,4 which has the following form:
σ 2t = a0 +pX

ai 2t−i +

i=1

where
St−i =pX

γ i St−i 2t−i +

i=1

½

1
0

if
if

q
X

bj σ 2t−j(7.12)

j=1

t−i < 0t−i ≥ 0That is, depending on whether t−i is above or below the threshold valueof zero, 2t−i has diﬀerent eﬀects on the conditional variance σ 2t : when t−i
is positive, the total eﬀects are given by ai 2t−i ; when t−i is negative,
the total eﬀects are given by (ai + γ i ) 2t−i . So one would expect γ i to bepositive for bad news to have larger impacts. This model is also known as
the GJR model because Glosten, Jagannathan and Runkle (1993) proposed
essentially the same model.
Use the garch function to estimate a TGARCH model by specifying~tgarch(p,q) as the conditional variance formula. For example, to fit a
TGARCH instead of an EGARCH model to hp.s, use the following command:> hp.tgarch = garch(hp.s~1, ~tgarch(1,1), trace=F)> hp.tgarch

4 The original TGARCH model proposed by Zakoian (1994) models σ

2t instead of σ t .

7.5 GARCH Model Extensions

Call:garch(formula.mean = hp.s ~ 1, formula.var =trace = F)243

~ tgarch(1, 1),

Mean Equation: hp.s ~ 1
Conditional Variance Equation:~ tgarch(1, 1)

Coefficients:
C 3.946e-04
A 3.999e-05
ARCH(1) 6.780e-02
GARCH(1) 8.369e-01
GAMMA(1) 3.306e-02
Note that when using the TGARCH model, the leverage eﬀects are automatically imposed, so it is not necessary to set leverage=T. Also, thecoeﬃcient γ 1 for leverage eﬀects is denoted by GAMMA(1) in the output todistinguish it from the EGARCH-type formulation of leverage eﬀects. The
estimated value of γ 1 is positive, indicating the presence of leverage eﬀects,
and is statistically diﬀerent from zero at the 5% significance level since its
t-statistic is greater than 2:> coef(hp.tgarch)[5]/sqrt(vcov(hp.tgarch)[5,5])[1] 2.5825
PGARCH Model
The basic GARCH model in S+FinMetrics is also extended to allow forleverage eﬀects. This is made possible by treating the basic GARCH model
as a special case of the power GARCH (PGARCH) model proposed byDing, Granger and Engle (1993):
σ dt = a0 +

p
X
i=1

ai (| t−i | + γ i t−i )d +qXbj σ dt−j(7.13)

j=1

where d is a positive exponent, and γ i denotes the coeﬃcient of leverage eﬀects. Note that when d = 2, (7.13) reduces to the basic GARCH
model with leverage eﬀects. Ding, Granger and Engle (1993) showed that
the PGARCH model also includes many other GARCH variants as special
cases.
To estimate a basic GARCH(1, 1) model with leverage eﬀects, specify~garch(1,1) as the conditional variance formula and set the optional argument leverage=T:> hp.garch = garch(hp.s~1, ~garch(1,1), leverage=T, trace=F)244

7. Univariate GARCH Modeling> hp.garch
Call:garch(formula.mean = hp.s ~ 1, formula.var =leverage = T, trace = F)

~ garch(1, 1),

Mean Equation: hp.s ~ 1
Conditional Variance Equation:~ garch(1, 1)

Coefficients:
C 4.536e-04
A 3.823e-05
ARCH(1) 7.671e-02
GARCH(1) 8.455e-01
LEV(1) -1.084e-01
The estimated value of γ 1 is negative and its t-statistic> coef(hp.garch)[5]/sqrt(vcov(hp.garch)[5,5])[1] -2.2987
is less than 2 so one can reject the null of no leverage eﬀects. If ~pgarch(p,q)
instead of ~garch(p,q) is used as the conditional variance formula, thegarch function will estimate the PGARCH model (7.13) where the exponent d is also estimated by MLE.
One can fix the exponent d in PGARCH model at a value other than two.
For example, a popular choice is to set d = 1 so that the GARCH model isrobust to outliers. To fit such a model, simply use ˜pgarch(p,q,d) as theconditional variance formula:> hp.pgarch = garch(hp.s~1,~pgarch(1,1,1),leverage=T,trace=F)> hp.pgarch
Call:garch(formula.mean = hp.s~1, formula.var = ~pgarch(1, 1, 1),leverage = T, trace = F)
Mean Equation: hp.s ~ 1
Conditional Variance Equation:
Coefficients:
CA

0.0003312
0.0015569~ pgarch(1, 1, 1)

7.5 GARCH Model Extensions

Pp

GARCH(p, q)σ̄ 2 = a0 /[1 −TGARCH(p, q)σ̄ 2 = a0 /[1 −PGARCH(p, q, 1)

σ̄ 2 = a20 /[1 −EGARCH(p, q)σ̄ 2 = exp{(a0 +2i=1 ai (1 + γ i ) −Pp

i=1 (ai + γ i /2) −Pp

i=1 aiPp

i=1 ai245

Pq

j=1 bj ]Pq

j=1 bj ]pP
2/π − qj=1 bj ]2pP2/π)/(1 − qj=1 bj )}TABLE 7.1. Unconditional variance of GARCH processes

ARCH(1) 0.0892505
GARCH(1) 0.8612378
LEV(1) -0.1499219> coef(hp.pgarch)[5]/sqrt(vcov(hp.pgarch)[5,5])[1] -2.2121
News Impact Curve
The above subsections have shown that GARCH, EGARCH, TGARCH and
PGARCH models are all capable of modeling leverage eﬀects. The choiceof a particular model can be made by using a model selection criterion suchas the Bayesian information criterion (BIC). Alternatively, Engle and Ng(1993) proposed that the news impact curve could also be used to compare
these diﬀerent models. Here is the definition of the news impact curve
following Engle and Ng (1993):
The news impact curve is the functional relationship between conditionalvariance at time t and the shock term (error term) at time t − 1, holding constant the information dated t − 2 and earlier, and with all lagged
conditional variance evaluated at the level of the unconditional variance.
To facilitate the comparison of news impact curves of diﬀerent GARCH
models, Table 7.1 summarizes the unconditional variance, σ̄ 2 , of various
GARCH models and Table 7.2 summarizes the news impact curves formodels with p = 1 and q = 1.
For example, to compare the news impact curves implied by hp.tgarch,
hp.pgarch and hp.garch, plot the corresponding news impact curves using
the following commands:> a0 = hp.tgarch$coef[2]> a1 = hp.tgarch$coef[3]> b1 = hp.tgarch$coef[4]> g1 = hp.tgarch$coef[5]> A = a0 + b1 * hp.tgarch$asymp.sd^2

246

7. Univariate GARCH Modeling

GARCH(1, 1)

σ2t = A + a1 (| t−1 | + γ 1 t−1 )2
A = a0 + b1 σ̄2

TGARCH(1, 1)σ 2t = A + (a1 + γ 1 St−1 ) 2t−1
A = a0 + b1 σ̄2PGARCH(1, 1, 1)

EGARCH(1, 1)√σ 2t = A + 2 Aa1 (| t−1 | + γ 1 t−1 )+a21 (| t−1 | + γ 1 t−1 )2 , A = (a0 + b1 σ̄)2σ2t = A exp{a1 (| t−1 | + γ 1 t−1 )/σ̄}A = σ̄ 2b1 exp{a0 }TABLE 7.2. News impact curves of GARCH processes> epsilon = seq(-0.21, 0.14, length=100)> sigma2.t.TGARCH = A + (a1+g1*(epsilon < 0))*(epsilon^2)> a0 = hp.pgarch$coef[2]> a1 = hp.pgarch$coef[3]> b1 = hp.pgarch$coef[4]> g1 = hp.pgarch$coef[5]> A = (a0 + b1 * hp.pgarch$asymp.sd)^2> error = abs(epsilon) + g1*epsilon> sigma2.t.PGARCH = A + 2*sqrt(A)*a1*error + (a1*error)^2> a0 = hp.garch$coef[2]> a1 = hp.garch$coef[3]> b1 = hp.garch$coef[4]> g1 = hp.garch$coef[5]> A = a0 + b1 * hp.garch$asymp.sd^2> error = abs(epsilon) + g1*epsilon> sigma2.t.GARCH = A + a1*(error^2)> matplot(cbind(epsilon, epsilon, epsilon), cbind(sigma2.t.TGARCH, sigma2.t.PGARCH, sigma2.t.GARCH), type="l")> key(-0.05, 0.0045, lines=list(type="l", lty=1:3), text=
list(c("TGARCH", "PGARCH", "GARCH")), border=1, adj=1)
In this plot, the range of t is determined by the residuals from the fitted
models. The resulting plot is shown in Figure 7.8. This plot shows that thenews impact curves are all asymmetric because leverage eﬀects are allowed

247

0.005

7.5 GARCH Model Extensions

0.001

0.002

0.003

0.004

TGARCHPGARCH
GARCH

-0.2

-0.1

0.0

0.1

FIGURE 7.8. Camparison of news impact curves.

in all three models, and negative shocks or bad news have larger impacts
on volatility. The TGARCH model suggests larger impacts of shocks onvolatility than the GARCH model with leverage eﬀects, regardless of thesize of the shock. Moreover, since the PGARCH model with d = 1 is morerobust to extreme shocks, impacts of small shocks implied by the PGARCH
model are larger compared to those from GARCH and TGARCH models,
whereas impacts of large shocks implied by the PGARCH model are smaller
compared to those from GARCH and TGARCH models.

7.5.2 Two Components Model
Section 7.3.2 illustrated that the GARCH model can be used to model
mean reversion in conditional volatility; that is, the conditional volatility
will always “mean revert” to its long run level if the GARCH model isstationary. Recall the mean reverting form of the basic GARCH(1, 1) model:( 2t − σ̄ 2 ) = (a1 + b1 )( 2t−1 − σ̄ 2 ) + ut − b1 ut−1
where σ̄ 2 = a0 /(1 − a1 − b1 ) is the unconditional long run level of volatility. As previous examples have shown, the mean reverting rate a1 + b1
implied by most fitted models is usually very close to 1. For example, theford.mod11 object fitted in Section 7.4,has the following mean reverting
rate:

248

7. Univariate GARCH Modeling> ford.mod11$coef[3] + ford.mod11$coef[4][1] 0.9847255
which is almost one. The half life of a volatility shock implied by this mean
reverting rate is:5
> log(0.5)/log(ford.mod11$coef[3] + ford.mod11$coef[4])[1] 45.03192
which amounts to more than two calendar months. So the fitted GARCH
model implies that the conditional volatility is very persistent.
Engle and Lee (1999) suggested that the high persistence in volatility
may be due to a time-varying long run volatility level. In particular, theysuggested decomposing conditional variance into two components:
σ 2t = qt + st

(7.14)

where qt is a highly persistent long run component, and st is a transitory
short run component.
S+FinMetrics supports a wide range of two component models by extending all the previously discussed GARCH variants to incorporate the twocomponents form (7.14). The general form of the two components model isbased on a modified version of Ding and Granger (1996):
σ dt = qtd + sdt(7.15)dqtd = α1 | t−1 |d + β 1 qt−1
sdt = a0 + α2 | t−1 |d + β 2 sdt−1 .

(7.16)(7.17)

That is, the long run component qt follows a highly persistent PGARCH(1, 1)
model, and the transitory component st follows another PGARCH(1, 1)
model. By expressing the above two PGARCH models using lag operator
notationqtd = (1 − β 1 L)−1 α1 | t−1 |dsdt = a0 + (1 − β 2 L)−1 α2 | t−1 |dand then substituting them into (7.15), it can be shown that the reduced
form of the two components model is:
σdt = a0 + (α1 + α2 )| t−1 |d − (α1 β 2 + α2 β 1 )| t−2 |d+ (β 1 + β 2 )σ dt−1 − β 1 β 2 σ dt−2

which is in the form of a constrained PGARCH(2, 2) model. However,
the two components model is not fully equivalent to the PGARCH(2, 2)
5 See Chapter 2 for the definition of half life.

7.5 GARCH Model Extensions

249

model because not all PGARCH(2, 2) models have the component structure. In fact, since the two components model is a constrained versionof the PGARCH(2, 2) model, the estimation of a two components model
is often numerically more stable than the estimation of an unconstrained
PGARCH(2, 2) model.
Although the PGARCH(1, 1) model is used here as the component for thetwo components model, S+FinMetrics actually allows any valid GARCH
variant as the component, and leverage eﬀects are also allowed correspondingly. For example, to fit a two components model using a GARCH component, EGARCH component, or PGARCH component, simply use the conditional variance formulas ~garch.2comp, ~egarch.2comp, ~pgarch.2comp(d),respectively. Since a two components model reduces to a GARCH(2, 2)
model of the corresponding type, the orders of the ARCH and GARCH
terms need not be given in the formula specification. The only exception
is the PGARCH two components model, which can explicitly specify the
exponent d for the underlying PGARCH model. For example, to estimate
a two components PGARCH model with d = 2 using the daily Ford stock
returns ford.s, use the following command:> ford.2comp = garch(ford.s~1, ~pgarch.2comp(2))> summary(ford.2comp)Call:garch(formula.mean = ford.s ~ 1, formula.var =~ pgarch.2comp(2))
Mean Equation: ford.s ~ 1
Conditional Variance Equation:
Conditional Distribution:~ pgarch.2comp(2)gaussian-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value Pr(>|t|)
C 6.870e-04 3.795e-04
1.810 3.519e-02
A 1.398e-06 5.877e-07
2.379 8.716e-03
ALPHA(1) 2.055e-02 6.228e-03
3.300 4.925e-04
ALPHA(2) 1.422e-01 2.532e-02
5.617 1.110e-08
BETA(1) 9.664e-01 8.637e-03 111.883 0.000e+00
BETA(2) 3.464e-01 1.091e-01
3.175 7.617e-04
The coeﬃcients for the two components, α1 , β 1 , α2 and β 2 , are identified by
ALPHA(1), BETA(1), ALPHA(2) and BETA(2) in the output. As expected, the250

7. Univariate GARCH Modelinglong run component associated with α1 and β 1 is very persistent, whereas
the second component associated with α2 and β 2 is not persistent at all.
Also, all the coeﬃcients are highly significant.
In the above example, fixing d = 2 for the two components PGARCH
model can be easily verified that the model is equivalent to a two components GARCH model. If the exponent d is not specified in the formula, itwill be estimated by MLE. In addition, setting leverage=T when fitting
a two components model, the coeﬃcients for leverage eﬀects will also be
estimated, and the form of leverage eﬀects is same as in (7.11) and (7.13).
However, for the two components PGARCH model, S+FinMetrics also allows leverage eﬀects to take the form as in the TGARCH model (7.12). The
resulting model can be estimated by using ~two.comp(i,d) as the conditional variance formula, with i = 2 corresponding to the leverage eﬀects asin (7.12), and i = 1 corresponding to the leverage eﬀects as in (7.13). For
example, the following model is essentially the two components TGARCH
model:> garch(ford.s~1, ~two.comp(2,2), leverage=T, trace=F)
Call:garch(formula.mean = ford.s ~ 1, formula.var =~ two.comp(2, 2), leverage = T, trace = F)
Mean Equation: ford.s ~ 1
Conditional Variance Equation:~ two.comp(2, 2)

Coefficients:
C 5.371e-04
A 1.368e-06
ALPHA(1) 1.263e-02
ALPHA(2) 1.154e-01
BETA(1) 9.674e-01
BETA(2) 2.998e-01
LEV(1) 8.893e-02
LEV(2) -5.235e-02

7.5.3 GARCH-in-the-Mean ModelIn financial investment, high risk is often expected to lead to high returns.
Although modern capital asset pricing theory does not imply such a simple
relationship, it does suggest there are some interactions between expected
returns and risk as measured by volatility. Engle, Lilien and Robins (1987)
proposed to extend the basic GARCH model so that the conditional volatil-7.5 GARCH Model Extensions

g(σ t )σσ2ln(σ 2 )251

Formula namesd.in.mean
var.in.mean
logvar.in.mean

TABLE 7.3. Possible functions for g(σt )ity can generate a risk premium which is part of the expected returns.
This extended GARCH model is often referred to as GARCH-in-the-mean(GARCH-M) model.
The GARCH-M model extends the conditional mean equation (7.8) as
follows:
yt = c + αg(σ t ) + t(7.18)
where g(·) can be an arbitrary function of volatility σ t . The garch function allows the GARCH-M specification in the conditional mean equation
together with any valid conditional variance specification. However, thefunction g(σ t ) must be one of the functions listed in Table 7.3, where thecorresponding formula specifications are also given.
For example, to fit a GARCH-M model with g(σt ) = σ2t to HewlettPackard stock returns using a PGARCH(1, 1, 1) model with leverage eﬀects,
use the following command:> hp.gmean = garch(hp.s~var.in.mean,~pgarch(1,1,1),leverage=T)
Iteration
0 Step Size = 1.00000 Likelihood = 2.40572
Iteration
0 Step Size = 2.00000 Likelihood = 2.40607
Iteration
0 Step Size = 4.00000 Likelihood = 2.38124
Iteration
1 Step Size = 1.00000 Likelihood = 2.40646
Iteration
1 Step Size = 2.00000 Likelihood = 2.40658
Iteration
1 Step Size = 4.00000 Likelihood = 2.40611
Iteration
2 Step Size = 1.00000 Likelihood = 2.40667
Iteration
2 Step Size = 2.00000 Likelihood = 2.40669
Iteration
2 Step Size = 4.00000 Likelihood = 2.40653
Convergence R-Square = 7.855063e-05 is less than tolerance= 0.0001
Convergence reached.
> summary(hp.gmean)Call:garch(formula.mean = hp.s ~ var.in.mean, formula.var =~ pgarch(1, 1, 1), leverage = T)
Mean Equation: hp.s ~ var.in.mean
Conditional Variance Equation:~ pgarch(1, 1, 1)

252

7. Univariate GARCH Modeling

Conditional Distribution:

gaussian-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value Pr(>|t|)C -0.001712 0.0013654 -1.254 1.050e-01
ARCH-IN-MEAN 4.373179 2.8699425
1.524 6.386e-02
A 0.001648 0.0003027
5.444 2.920e-08
ARCH(1) 0.093854 0.0096380
9.738 0.000e+00
GARCH(1) 0.853787 0.0176007 48.509 0.000e+00
LEV(1) -0.161515 0.0648241 -2.492 6.399e-03
The coeﬃcient α in (7.18) is identified by ARCH-IN-MEAN in the output.
In this case, the risk premium is positive which implies that high risk
(volatility) leads to high expected returns. However, the p-value for the
t-statistic is slightly larger than the conventional 5% level.

7.5.4 ARMA Terms and Exogenous Variables in Conditional
Mean Equation
So far the conditional mean equation has been restricted to a constantwhen considering GARCH models, except for the GARCH-M model wherevolatility was allowed to enter the mean equation as an explanatory variable. The garch function in S+FinMetrics allows ARMA terms as well
as exogenous explanatory variables in the conditional mean equation. The
most general form for the conditional mean equation isyt = c +r
Xi=1

φi yt−i +sXj=1

θj t−j +LX

β0l xt−l + t(7.19)

l=1

where xt is a k × 1 vector of weakly exogenous variables, and β l is thek × 1 vector of coeﬃcients. Note that distributed lags of the exogenous
variables in xt are also allowed. To include AR(r), MA(s), or ARMA(r, s)
terms in the conditional mean, simply add ar(r), ma(s), or arma(r,s) tothe conditional mean formula.
Example 42 Single factor model with GARCH errors
From the Capital Asset Pricing Model (CAPM), stock returns should becorrelated with the returns on a market index, and the regression coeﬃcientis usually referred to as the “market beta”. S+FinMetrics comes with a“timeSeries” object nyse.s which represents daily returns on a value
weighted New York Stock Exchange index and covers the same time period

7.5 GARCH Model Extensions

253

Market Beta
0.10

Ford Returns

0.05

0.0

-0.05

-0.10

-0.15

-0.15

-0.10

-0.05

0.0

0.05

0.10

NYSE Returns

FIGURE 7.9. Daily Ford returns versus NYSE returns.

as ford.s. Use the S+FinMetrics function rvfPlot to generate a Trellis
scatter plot of ford.s versus nyse.s:> rvfPlot(ford.s, nyse.s, strip.text="Market Beta",
id.n=0, hgrid=T, vgrid=T,
xlab="NYSE Returns", ylab="Ford Returns")
The plot is shown in Figure 7.9, from which a clear linear relationship canbe seen. To estimate the market beta for daily Ford returns allowing for a
GARCH(1, 1) error, use the following command:> ford.beta = garch(ford.s~ma(1)+seriesData(nyse.s),~garch(1,1), trace=F)> summary(ford.beta)Call:garch(formula.mean = ford.s ~ ma(1) + seriesData(nyse.s),formula.var = ~ garch(1, 1), trace = F)
Mean Equation: ford.s ~ ma(1) + seriesData(nyse.s)
Conditional Variance Equation:
Conditional Distribution:~ garch(1, 1)

gaussian

254

7. Univariate GARCH Modeling-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value Pr(>|t|)
C 8.257e-05 3.063e-04
0.2695 3.938e-01
MA(1) 4.448e-02 2.186e-02
2.0348 2.100e-02
seriesData(nyse.s) 1.234e+00 2.226e-02 55.4418 0.000e+00
A 1.406e-06 5.027e-07
2.7971 2.603e-03
ARCH(1) 3.699e-02 4.803e-03
7.7019 1.044e-14
GARCH(1) 9.566e-01 6.025e-03 158.7691 0.000e+00
Note that an MA(1) term has also been added in the mean equation to allow
for first order serial correlation in the daily returns caused by the possible
bid-ask bounce often observed in daily stock prices. The above summary
shows that both the MA(1) coeﬃcient and market beta are highly significant. The estimated volatility is shown in Figure 7.10, which is obtainedby choosing choice 8 from the plot method. Compare this with the estimated volatility without using nyse.s as shown in Figure 7.7: the estimatedvolatility has the same pattern, but the magnitude of volatility has significantly decreased. Since the market eﬀects are taken into consideration
here by using nyse.s as an explanatory variable, the resulting volatility
may be interpreted as the “idiosyncratic” volatility, while the volatility in
Figure 7.7 includes both the idiosyncratic component and the systematic
market component.

7.5.5 Exogenous Explanatory Variables in the Conditional
Variance Equation
Adding explanatory variables into the conditional variance formula may
have impacts on conditional volatility.6 To illustrate, it is widely believedthat trading volume aﬀects the volatility. The S+FinMetrics object dell.s
contains a time series of daily stock returns of Dell Computer Corporation,and dell.v contains daily trading volume of Dell stocks spanning the sametime period. In the next example, use the percentage change in trading
volume to forecast volatility.
Example 43 Trading volume and volatility
First, use the S+FinMetrics function getReturns to compute rates ofchanges in trading volume. Then look at the scatter plot of log absolute
returns versus the changes in trading volume:
6 To guarantee that the conditional variance is always positive, one has to make surethat exogenous variables are positive unless an EGARCH type model is selected.

7.5 GARCH Model Extensions

255

GARCH Volatility

0.018
0.014
0.010

Conditional SD

0.022

volatility

Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4
1984

1985

1986

1987

1988

1989

1990

1991 1992

FIGURE 7.10. Idiosyncratic volatility of daily Ford returns.

> log.abs.ret = log(abs(dell.s-mean(dell.s)))[-1]> d.volume = getReturns(dell.v)> rvfPlot(log.abs.ret, d.volume, strip="Scatter Plot",
id.n=0, hgrid=T, vgrid=T,
xlab="% Volume", ylab="Volatility")
The resulting plot is shown in Figure 7.11. There seems to exist a fairly
linear relationship between the changes in volume and the volatility as
measured by the log absolute returns. Based on this observation, use thechanges in volume as an explanatory variable in the EGARCH varianceequation:> dell.mod = garch(dell.s~1,~egarch(1,1)+seriesData(d.volume),
series.start=2)> summary(dell.mod)Call:garch(formula.mean = dell.s ~ 1, formula.var = ~ egarch(1, 1)+ seriesData(d.volume), series.start = 2)
Mean Equation: dell.s ~ 1
Conditional Variance Equation:
seriesData(d.volume)~ egarch(1, 1) +

256

7. Univariate GARCH Modeling

Scatter Plot

2

Volatility

0

-2

-4

-6

-6

-4

-2

0

2

% Volume

FIGURE 7.11. Log absolute returns versus changes in volume: Dell.

Conditional Distribution:

gaussian-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value Pr(>|t|)C 0.15678
0.065392.3977 8.321e-03
A -0.02078
0.03927 -0.5293 2.984e-01
ARCH(1) 0.14882
0.037213.9992 3.364e-05
GARCH(1) 0.95140
0.01695 56.1226 0.000e+00
seriesData(d.volume) 1.39898
0.08431 16.5928 0.000e+00
The optional argument series.start=2 is used because the “timeSeries”
d.volume has one less observation than the “timeSeries” dell.s. Fromthe summary output, the coeﬃcient on changes in volume is estimated tobe 1.4 and is highly significant with a p-value essentially equal to zero. The
estimated model implies a 1% change in trading volume causes about a
1.4% change in conditional variance.

7.5 GARCH Model Extensions257

7.5.6 Non-Gaussian Error Distributions
In all the examples illustrated so far, a normal error distribution has been
exclusively used. However, given the well known fat tails in financial timeseries, it may be more desirable to use a distribution which has fatter tails
than the normal distribution. The garch function in S+FinMetrics allows
three fat-tailed error distributions for fitting GARCH models: the Student’s
t distribution; the double exponential distribution; and the generalized error
distribution.
Student’s t Distribution
If a random variable ut has a Student’s t distribution with ν degrees of
freedom and a scale parameter st , the probability density function (PDF)
of ut is given by:
f (ut ) =−1/2

Γ[(ν + 1)/2]st(πν)1/2 Γ(ν/2) [1 + u2t /(st ν)](ν+1)/2

where Γ(·) is the gamma function. The variance of ut is given by:
V ar(ut ) =st ν, v > 2.
ν−2

If the error term t in a GARCH model follows a Student’s t distribution
with ν degrees of freedom and V art−1 ( t ) = σ 2t , the scale parameter stshould be chosen to beσ2 (ν − 2)
st = t.
νThus the log-likelihood function of a GARCH model with Student’s t distributed errors can be easily constructed based on the above PDF.
Generalized Error Distribution
Nelson (1991) proposed to use the generalized error distribution (GED) tocapture the fat tails usually observed in the distribution of financial timeseries. If a random variable ut has a GED with mean zero and unit variance,
the PDF of ut is given by:
f (ut ) =where

ν exp[−(1/2)|ut /λ|ν ]λ · 2(ν+1)/ν Γ(1/ν)∙ −2/ν¸1/2
Γ(1/ν)2λ=Γ(3/ν)and ν is a positive parameter governing the thickness of the tail behaviorof the distribution. When ν = 2 the above PDF reduces to the standard

258

7. Univariate GARCH Modelingnormal PDF; when ν < 2, the density has thicker tails than the normal
density; when ν > 2, the density has thinner tails than the normal density.
When the tail thickness parameter ν = 1, the PDF of GED reduces to
the PDF of double exponential distribution:√
1
f (ut ) = √ e− 2|ut | .
2

Based on the above PDF, the log-likelihood function of GARCH models with GED or double exponential distributed errors can be easily constructed. Refer to Hamilton (1994) for an example.
GARCH Estimation with Non-Gaussian Error Distributions
To estimate a GARCH model with the above three non-Gaussian error
distributions using the garch function, simply set the optional argument
cond.dist to "t" for the Student’s t distribution, "ged" for the GED
distribution, and "double.exp" for the double exponential distribution,
respectively.
For example, to estimate a basic GARCH(1, 1) model with Student’s t
distribution using daily Ford stock returns ford.s, use the command:> ford.mod11.t = garch(ford.s~1, ~garch(1,1), cond.dist="t")
Iteration
0 Step Size = 1.00000 Likelihood = 2.64592
Iteration
0 Step Size = 2.00000 Likelihood = -1.00000e+10
Iteration
1 Step Size = 1.00000 Likelihood = 2.64788
Iteration
1 Step Size = 2.00000 Likelihood = 2.64367
Iteration
2 Step Size = 1.00000 Likelihood = 2.64808
Iteration
2 Step Size = 2.00000 Likelihood = 2.64797
Convergence R-Square = 4.712394e-05 is less than tolerance= 0.0001
Convergence reached.
The distribution information is saved in the cond.dist component of the
returned object:> ford.mod11.t$cond.dist$cond.dist:[1] "t"$dist.par:[1] 7.793236$dist.est:[1] T7.5 GARCH Model Extensions

259

QQ-Plot of Standardized Residuals

QQ-Plot

6

12/31/1991

Standardized Residuals

4

2

0

-2

-4
2/3/1984-6-8
2/2/1984
-6

-4

-2

0

2

4

6

Quantiles of t distribution

FIGURE 7.12. Student-t QQ-plot of standardized residuals: ford.mod11.t.

where the dist.par component contains the estimated degree of freedom
ν for Student’s t distribution. Calling the generic summary function on the
returned object will also show the standard error of the estimate of ν.
To assess the goodness-of-fit of ford.mod11.t, generate the qq-plot basedon the estimated Student’s t distribution by calling the plot function on
ford.mod11.t, which is shown in Figure 7.12. Compare this with Figure 7.5
and the Student’s t error distribution provides a much better fit than thenormal error distribution.
When using Student’s t or GED distributions, the distribution parameterν is estimated as part of the MLE procedure by default. One can also choose
to fix ν at a certain value during the estimation. For example, to fix ν = 1
for GED distribution, use the command:> ford.mod11.dexp = garch(ford.s~1, ~garch(1,1),+ cond.dist="ged", dist.par=1, dist.est=F)
where the optional argument dist.par is used to set the value, and dist.est
is used to exclude the distribution parameter for MLE. It can be easily verified that this is equivalent to setting cond.dist="double.exp".

260

7. Univariate GARCH Modeling

7.6 GARCH Model Selection and Comparison
The previous sections have illustrated the variety of GARCH models available in S+FinMetrics. Obviously selecting the best model for a particular
data set can be a daunting task. Model diagnostics based on standardized residuals and news impact curves for leverage eﬀects can be used tocompare the eﬀectiveness of diﬀerent aspects of GARCH models. In addition, since GARCH models can be treated as ARMA models for squared
residuals, traditional model selection criteria such as Akaike information
criterion (AIC) and Bayesian information criterion (BIC) can also be used
for selecting models.
To facilitate the selection and comparison of diﬀerent GARCH models,
S+FinMetrics provides the function compare.mgarch to compare the fitsof diﬀerent “garch” objects.7 For example, to compare the GARCH(1,1)
fits of the “garch” objects ford.mod11 and ford.mod11.t, one fitted with
the Gaussian distribution and the other with the Student’s t distribution,
use the following command:> ford.compare = compare.mgarch(ford.mod11, ford.mod11.t)> oldClass(ford.compare)[1] "compare.garch" "compare.mgarch"> ford.compare
ford.mod11 ford.mod11.t
AIC-10504-10582
BIC-10481-10554
Likelihood
5256
5296
The returned object ford.compare is an S version 3 object with class“compare.garch”, which inherits from the class “compare.mgarch”. The
print method for this class of objects shows the AIC, BIC, and loglikelihood values of the fitted models. Since the BIC of ford.mod11.t ismuch smaller than that of ford.mod11, Student’s t distribution seems toprovide a much better fit than the normal distribution.
S+FinMetrics also provides a method for the generic plot function for
objects inheriting from class “compare.mgarch”. To see the arguments ofthe plot method, use the args function as usual:> args(plot.compare.mgarch)function(x, qq = F, hgrid = F, vgrid = F, lag.max = NULL,
ci = 2, ...)> plot(ford.compare)
7 This is originally designed as a method function for the generic compare function for
an S version 3 object. However, for S-PLUS 6 which is based on S version 4, the genericfunction compare does not work correctly when more than two objects are compared. So
we suggest calling compare.mgarch directly.

7.6 GARCH Model Selection and Comparison

261

ACF of Squared Std. Residuals
010

ford.mod11

20

30

ford.mod11.t

1.0

0.8

0.6

0.4

0.2

0.0

0

10

20

30

Lags

FIGURE 7.13. Comparison of ACF of squared std. residuals.

The resulting plot is shown Figure 7.13. By default, the plot method compares the ACF of squared standardized residuals from the fitted models.
This plot demonstrates that both models are successful at modeling conditional volatility. If the optional argument is set at qq=T, then a comparisonof qq-plots is generated:> plot(ford.compare, qq=T, hgrid=T, vgrid=T)
which is shown in Figure 7.14. Note that since ford.mod11 is fitted using
the normal distribution, the qq-plot is based on normal assumption. In
contrast, since ford.mod11.t is fitted using Student’s t distribution, theqq-plot is based on a Student’s t distribution with degrees of freedom takenfrom the cond.dist component of the object.

7.6.1 Constrained GARCH Estimation
For a GARCH model, some model parameters can also be fixed at certain
values to evaluate the fit of a particular model. Section 13.7 in Chapter 13
provides some examples.

262

7. Univariate GARCH Modeling

QQ-Plot of Standardized Residuals
-6

-4

-2

ford.mod11

6

0

2

4

6

ford.mod11.t

Standardized Residuals

4
2
0
-2
-4
-6
-8
-6

-4

-2

0

2

4

6

Quantile of Comparison Distribution

FIGURE 7.14. Comparison of QQ-plot of std. residuals.

7.7 GARCH Model Prediction
An important task of modeling conditional volatility is to generate accurate
forecasts for both the future value of a financial time series as well as its
conditional volatility. Since the conditional mean of the general GARCH
model (7.19) assumes a traditional ARMA form, forecasts of future valuesof the underlying time series can be obtained following the traditional approach for ARMA prediction. However, by also allowing for a time varying
conditional variance, GARCH models can generate accurate forecasts offuture volatility, especially over short horizons. This section illustrates how
to forecast volatility using GARCH models.
For simplicity, consider the basic GARCH(1, 1) model:
σ 2t = a0 + a1 2t−1 + b1 σ 2t−1
which is estimated over the time period t = 1, 2, · · · , T . To obtain ET [σ 2T +k ],the forecasts of future volatility σ 2T +k , for k > 0, given information at time
T . The above equation can easily obtain:
ET [σ 2T +1 ] = a0 + a1 ET [ 2T ] + b1 ET [σ 2T ]= a0 + a1 2T + b1 σ2T

7.7 GARCH Model Prediction

263

GARCH Predictions

Predicted Conditional SD

0.02500.02450.02400.02350.0230
0

20

40

60

80

100

FIGURE 7.15. PGARCH forecasts of future volatility: hp.pgarch.

since it already has

228
T and σ T after the estimation. Now for T + 2

ET [σ 2T +2 ] = a0 + a1 ET [ 2T +1 ] + b1 ET [σ 2T +1 ]= a0 + (a1 + b1 )ET [σ 2T +1 ].
since ET [ε2T +1 ] = ET [σ 2T +1 ]. The above derivation can be iterated to givethe conditional volatility forecasting equation
ET [σ2T +k ] = a0

k−2
X

(a1 + b1 )i + (a1 + b1 )k−1 ET [σ 2T +1 ](7.20)

i=1

for k ≥ 2. Notice that as k → ∞, the volatility forecast in (7.20) approaches the unconditional variance a0 /(1 − a1 − b1 ) if the GARCH process
is stationary (i.e., if α1 + b1 < 1).
The forecasting algorithm (7.20) produces forecasts for the conditionalvariance σ2T +k . The forecast for the conditional volatility, σ T +k , is definedas the square root of the forecast for σ2T +k .
The predict method for “garch” objects in S+FinMetrics implements
the forecasting procedure as described above for all the supported GARCH
variants, allowing for leverage eﬀects and the use of exogenous variables
8 We are a little bit loose with notations here because
fitted values instead of the unobserved “true” values.

Tand σ 2T are actually the

264

7. Univariate GARCH Modelingin both the conditional mean and the conditional variance. The forecasts
can be easily obtained by calling the generic predict function on a fitted
model object with the desired number of forecasting periods. For example,
consider the PGARCH object hp.pgarch in Section 7.5.1. To obtain 10step-ahead forecasts, simply use the command:> hp.pgarch.pred = predict(hp.pgarch,10)> class(hp.pgarch.pred)[1] "predict.garch"> names(hp.pgarch.pred)[1] "series.pred" "sigma.pred" "asymp.sd"> hp.pgarch.pred$series.pred:[1] 0.0003312 0.0003312 0.0003312 0.0003312 0.0003312[6] 0.0003312 0.0003312 0.0003312 0.0003312 0.0003312$sigma.pred:[1] 0.02523 0.02508 0.02494 0.02482 0.02470 0.02458 0.02448[8] 0.02438 0.02429 0.02421$asymp.sd:[1] 0.02305
attr(, "class"):[1] "predict.garch"
The returned object hp.pgarch.pred is of class “predict.garch” for which
there is only a plot method. Since the conditional mean was restricted toa constant in hp.pgarch, the forecasts of the future values contained in
the component series.pred are simply the estimate of the mean. The
component sigma.pred contains the forecasts of σt , and the component
asymp.sd contains the estimate of the unconditional standard deviation if
the estimated model is stationary. If a very large number of steps lie ahead,
the forecasted volatility should approach the unconditional level. This canbe easily verified for hp.pgarch as follows:> plot(predict(hp.pgarch, 100))
where the plot method for the returned object can be directly invoked
and the resulting plot is shown in Figure 7.15. Note that a plot of theforecasted series values can also be obtained. See the on-line help file forplot.predict.garch for details.
The forecasted volatility can be used together with forecasted series values to generate confidence intervals of the forecasted series values. In manycases, the forecasted volatility is of central interest, and confidence intervals for the forecasted volatility can be obtained as well. However, analytic
formulas for confidence intervals of forecasted volatility are only known for7.8 GARCH Model Simulation

265

some special cases (see Baillie and Bollerslev, 1992). The next section willshow how a simulation-based method can be used to obtain confidence intervals for forecasted volatility from any of the GARCH variants available
in S+FinMetrics.

7.8 GARCH Model Simulation
S+FinMetrics provides a method for the generic S-PLUS function simulate
for objects of class “garch”. This function, simulate.garch, allows observations as well as volatility to be simulated from a user-specified GARCH
model or from the model contained in a fitted “garch” object. This section
illustrates the use of simulate to create confidence intervals for volatility
forecasts from a fitted GARCH model.
Example 44 Simulation-based GARCH forecasts
To obtain volatility forecasts from a fitted GARCH model, simply simulate from the last observation of the fitted model. This process can be
repeated many times to obtain an “ensemble” of volatility forecasts. For
example, suppose 100-step-ahead volatility forecasts need to be generated
from hp.pgarch, take the residual term and fitted volatility of the last
observation:9> sigma.start = as.numeric(hp.pgarch$sigma.t[2000])> eps.start = as.numeric(hp.pgarch$residuals[2000])> eps.start = matrix(eps.start, 1, 1000)> error = rbind(eps.start, matrix(rnorm(100*1000), 100))
Note that the first row of error contains the pre-sample values of t to
start oﬀ the simulation for each of the 1000 replications, whereas the restof error are simply random variates with zero mean and unit variancewhich will be updated by the simulation procedure to result in GARCH
errors. Now use these values to obtain the simulations as follows:> set.seed(10)> hp.sim.pred = simulate(hp.pgarch, n=100, n.rep=1000,
sigma.start=sigma.start, etat=error)$sigma.t
The argument n specifies the desire to simulate 100 steps ahead, and n.rep
specifies wanting to repeat this 1000 times. The simulation procedure returns both the simulated GARCH errors and volatility. Only take the simulated volatility contained in the sigma.t component; thus hp.sim.predis a 100 × 1000 matrix with each column representing each simulation path.
9 If the order of the fitted GARCH model is m = max(p, q), then m last observations
are required.

266

7. Univariate GARCH Modeling

0.025
0.015

0.020

Volatility

0.030

Simulated Confidence Interval of Volatility

0

20

40

60

80

100

Time

FIGURE 7.16. Simulation-based volatility forecasts: hp.pgarch.

The simulation-based forecasts are simply the average of the 1000 simulation paths. 95% confidence intervals for the forecasts may be computed in
two ways. They can be computed using the usual formula based on normally distributed forecasts; that is, mean forecast ±2· standard deviationof forecasts. Alternatively, the 95% confidence interval may be constructedfrom the 2.5% and 97.5% quantiles of the simulated forecasts. Use the following code to compute the forecasts and plot the 95% confidence interval
based on the normal formula:> vol.mean = rowMeans(hp.sim.pred)> vol.stdev = rowStdevs(hp.sim.pred)> vol.cf = cbind(vol.mean+2*vol.stdev, vol.mean-2*vol.stdev)> tsplot(cbind(vol.mean, vol.cf))> points(predict(hp.pgarch, 100)$sigma.pred, pch=1)> title(main="Simulated Confidence Interval of Volatility",
xlab="Time", ylab="Volatility")
The resulting plot is shown in Figure 7.16. Note that analytic forecasts
are also added as points in the plot for comparison. The simulation-based
forecasts agree with the analytic ones produced by the predict method.
In the above example, the “standardized” errors were generated by random sampling from the standard normal distribution. In practice, it may
be desirable to generate standardized errors by bootstrapping from standardized residuals.

7.9 Conclusion

Formula~garch(p,q)~egarch(p,q)~tgarch(p,q)~pgarch(p,q)~pgarch(p,q,d)~garch.2comp~egarch.2comp~pgarch.2comp~pgarch.2comp(d)~two.comp(i)~two.comp(i,d)

267

Model
GARCH(p, q) model
EGARCH(p, q) model
TGARCH(p, q) model
PGARCH(p, q) model with free exponent d
PGARCH(p, q) model with fixed exponent d
GARCH TWO.COMP model
EGARCH TWO.COMP model
PGARCH TWO.COMP model with free
exponent d
PGARCH TWO.COMP model with fixed
exponent d
PGARCH TWO.COMP model with choice ofleverage eﬀects
PGARCH TWO.COMP model with choice ofleverage eﬀects and exponent d

TABLE 7.4. GARCH Formula Specifications

7.9 Conclusion
This chapter illustrated how to estimate and forecast from various GARCH
models. The range of GARCH models supported by S+FinMetrics is verybroad. Table 7.4 summarizes all the conditional variance formulas supported by the garch function.

7.10 References
Alexander, C. (2001). Market Models: A Guide to Financial Data Analysis, John Wiley & Sons, Chichester, UK.
Bera, A.K. and M.L. Higgins (1995). “On ARCH Models: Properties,
Estimation and Testing,” Journal of Economic Surveys, 7, 305-362.
Black, F. (1976). “Studies in Stock Price Volatility Changes,” Proceedingsof the 1976 Business Meeting of the Business and Economics Statistics
Section, American Statistical Association, 177-181.
Bollerslev, T. (1986). “Generalized Autoregressive Conditional Heteroskedasticity,” Journal of Econometrics, 31, 307-327.
Bollerslev, T., R.Y. Chu and K.F. Kroner (1994). “ARCH Modelingin Finance: A Selective Review of the Theory and Empirical Evidence,”
Journal of Econometrics, 52, 5-59.

268

7. Univariate GARCH Modeling

Bollerslev, T., Engle, R. F., and Nelson, D. B. (1994). “ARCH
Models,” in R. F. Engle and D. L. McFadden (eds.), Handbook of Econometrics, Vol. 4, Elsevier Science B. V., Amsterdam.
Bollerslev, T., and Wooldrige, J. M. (1992). “Quasi-maximum Likelihood Estimation and Inference in Dynamic Models with Time-varying
Covariances,” Econometric Reviews, 11, 143-172.
Davidson, R., MacKinnon, J. G. (1993). Estimation and Inference inEconometrics, Oxford University Press, Oxford.
Diebold, F.X. and J.A. Lopez (1996). “Modeling Volatility Dynamics,” in K. Hoover (ed.) Macroeconomics: Developments, Tensions andProspects. Kluwer, Boston.
Ding, Z., and Granger, C. W. J. (1996). “Modeling Volatility Persistence of Speculative Returns: A New Approach,” Journal of Econometrics,
73, 185-215.
Ding, Z., Granger, C. W. J., and Engle, R. F. (1993). “A Long
Memory Property of Stock Market Returns and a New Model,” Journal of
Empirical Finance, 1, 83-106.
Engle, R. F. (1982). “Autoregressive Conditional Heteroskedasticity with
Estimates of the Variance of United Kingdom Inflation,” Econometrica,
50(4), 987-1007.
Engle, R. F., and Lee, G. J. (1999). “A Long-Run and Short-Run
Component Model of Stock Return Volatility,” in R. F. Engle and H. White(eds.), Cointegration, Causality, and Forecasting. Oxford University Press,
Oxford.
Engle, R. F., Lilien, D. M., and Robins, R. P. (1987). “Estimating
Time Varying Risk Premia in the Term-Structure: the ARCH-M Model,”Econometrica, 55(2), 391-407.
Engle, R. F., and Ng, V. (1993). “Measuring and Testing the Impact
of News on Volatility,” Journal of Finance, 48(5), 1749-1778.
Engle, R.F. (2001). “GARCH 101: The Use of ARCH/GARCH Models
in Applied Economics,” Journal of Economic Perspectives, 15, 157-168.
Engle, R.F. and A.J. Patton (2001). “What Good is a Volatility
Model?” QuantitativeFinance, 237-245.

7.10 References

269

Glosten, L. R., Jagannathan, R., and Runkle, D. E. (1993). “On
the Relation Between the Expected Value and the Volatility of the Nominal
Excess Return on Stocks,” Journal of Finance, 48(5), 1779-1801.
Hamilton, J. D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
He, C., Teräsvirta, T. (1999a). “Properties of Moments of a Family of
GARCH Processes,” Journal of Econometrics, 92, 173-192.
He, C., Teräsvirta, T. (1999b). “Fourth Moment Structure of the
GARCH(p, q) Process,” Econometric Theory, 15, 824-846.
Nelson, D. B. (1991). “Conditional Heteroskedasticity in Asset Returns:
a New Approach,” Econometrica, 59(2), 347-370.
Nelson, D. B., and Cao, C. Q. (1992). “Inequality Constraints in the
Univariate GARCH Model,” Journal of Business and Economic Statistics,
10(2), 229-235.
Tsay, R.S. (2001). Analysis of Financial Time Series, John Wiley & Sons,
New York.
Zakoian, M. (1994), “Threshold Heteroscedastic Models,” Journal of Economic Dynamics and Control, 18, 931-955.

This is page 270
Printer: Opaque thisThis is page 271
Printer: Opaque this8
Long Memory Time Series Modeling

8.1 Introduction
Earlier chapters have demonstrated that many macroeconomic and financial time series like nominal and real interest rates, real exchange rates,
exchange rate forward premiums, interest rate diﬀerentials and volatility
measures are very persistent, i.e., that an unexpected shock to the underlying variable has long lasting eﬀects. Persistence can occur in the firstor higher order moments of a time series. The persistence in the first moment, or levels, of a time series can be confirmed by applying either unit
root tests or stationarity tests to the levels, while the persistence in thevolatility of the time series is usually exemplified by a highly persistentfitted GARCH model. Although traditional stationary ARMA processes
often cannot capture the high degree of persistence in financial time series,
the class of non-stationary unit root or I(1) processes have some unappealing properties for financial economists. In the last twenty years, more
applications have evolved using long memory processes, which lie halfway
between traditional stationary I(0) processes and the non-stationary I(1)processes. There is substantial evidence that long memory processes canprovide a good description of many highly persistent financial time series.
This chapter will cover the concept of long memory time series. Section 8.3 will explain various tests for long memory, or long range dependence, in a time series and show how to perform these tests using functions
in S+FinMetrics module. In Section 8.4 will illustrate how to estimate
the long memory parameter using R/S statistic and two periodogram-

272

8. Long Memory Time Series Modelingbased method. Section 8.5 will extend the traditional ARIMA processes
to fractional ARIMA (FARIMA) processes, which can be used to model
the long range dependence and short run dynamics simultaneously. The
semiparametric fractional autoregressive (SEMIFAR) process recently proposed by Beran and his coauthors will also be introduced. Section 8.6 will
extend GARCH models to fractionally integrated GARCH models to allow for long memory in conditional volatility. Finally, section 8.7 will consider the prediction from long memory models such as FARIMA and FIGARCH/FIEGARCH models. Beran (1994) gives an exhaustive treatmentof statistical aspects of modeling with long memory processes, while Baillie (1996) provides a comprehensive survey of econometric analysis of longmemory processes and applications in economics and finance.

8.2 Long Memory Time Series
To illustrate the long memory property in financial time series, consider
the daily returns on the S&P500 index from January 4, 1928 to August 30,
1991 contained in the S+FinMetrics “timeSeries” object sp500. Since
daily returns usually have a mean very close to zero, the absolute return
is sometimes used as a measure of volatility. The sample autocorrelation
function of the daily absolute returns can be plotted using the following
commands:> tmp = acf(abs(sp500), lag=200)> sp500.ar = ar(abs(sp500))> sp500.ar$order[1] 37> tmp.mod = list(ar=as.vector(sp500.ar$ar), sigma2=1, d=0)
> ar.acf = acf.FARIMA(tmp.mod, lag.max=200)> lines(ar.acf$lags, ar.acf$acf/ar.acf$acf[1])and the plot is shown in Figure 8.1. The autocorrelation of absolute returns is highly persistent and remains very significant at lag 200. In theabove code fragment, the S-PLUS function ar is used to select the best fitting AR process using AIC, which turns out to be an AR(37) model. The
S+FinMetrics function acf.FARIMA compares the theoretical autocorrelation function implied by the AR(37) process with the sample autocorrelation function. The following comments apply to this example:
1. Traditional stationary ARMA processes have short memory in thesense that the autocorrelation function decays exponentially. In theabove example, the theoretical autocorrelation closely matches the
sample autocorrelation at small lags. However, for large lags, thesample autocorrelation decays much more slowly than the theoretical
autocorrelation.

8.2 Long Memory Time Series

273

0.0

0.2

0.4

ACF

0.6

0.8

1.0

Series : abs(sp500)

0

50

100
Lag

150

200

FIGURE 8.1. ACF of daily absolute returns of S&P500 index.

2. When the sample autocorrelation decays very slowly, traditional stationary ARMA processes usually result in an excessive number ofparameters. In the above example, 37 autoregressive coeﬃcients were
found necessary to capture the dependence in the data.
Based on the above observations, a stationary process yt has long memory, or long range dependence, if its autocorrelation function behaves likeρ(k) → Cρ k −α as k → ∞(8.1)

where Cρ is a positive constant, and α is a real number between 0 and 1.
Thus the autocorrelation function of a long memory process decays slowly
at a hyperbolic rate. In fact, it decays so slowly that the autocorrelations
are not summable:∞Xρ(k) = ∞.
k=−∞

For a stationary process, the autocorrelation function contains the same
information as its spectral density. In particular, the spectral density isdefined as:∞
1 X
f (ω) =
ρ(k)eikω
2π
k=−∞

274

8. Long Memory Time Series Modelingwhere ω is the Fourier frequency (c.f. Hamilton, 1994). From (8.1) it canbe shown that
(8.2)
f (ω) → Cf ω α−1 as ω → 0
where Cf is a positive constant. So for a long memory process, its spectral
density tends to infinity at zero frequency. Instead of using α, in practiceuse
H = 1 − α/2 ∈ (0.5, 1),(8.3)
which is known as the Hurst coeﬃcient (see Hurst, 1951) to measure thelong memory in yt . The larger H is, the longer memory the stationary
process has.
Based on the scaling property in (8.1) and the frequency domain propertyin (8.2), Granger and Joyeux (1980) and Hosking (1981) independently
showed that a long memory process yt can also be modeled parametricallyby extending an integrated process to a fractionally integrated process. In
particular, allow for fractional integration in a time series yt as follows:(1 − L)d (yt − μ) = ut

(8.4)

where L denotes the lag operator, d is the fractional integration or fractional
diﬀerence parameter, μ is the expectation of yt , and ut is a stationary shortmemory disturbance with zero mean.
In practice, when a time series is highly persistent or appears to be
non-stationary, let d = 1 and diﬀerence the time series once to achieve
stationarity. However, for some highly persistent economic and financial
time series, it appears that an integer diﬀerence may be too much, which isindicated by the fact that the spectral density vanishes at zero frequency
for the diﬀerenced time series. To allow for long memory and avoid taking
an integer diﬀerence of yt , allow d to be fractional. The fractional diﬀerence
filter is defined as follows, for any real d > −1:d(1 − L) =

¶
∞ µ
X
d

k=0

k

(−1)k Lk

(8.5)

with binomial coeﬃcients:µ¶d!
Γ(d + 1)d==.
k
k!(d − k)!
Γ(k + 1)Γ(d − k + 1)
Notice that the fractional diﬀerence filter can be equivalently treated as an
infinite order autoregressive filter.1 It can be shown that when |d| > 1/2,
yt is non-stationary; when 0 < d < 1/2, yt is stationary and has long
1 The S+FinMetrics function FARIMA.d2ar can be used to compute the autoregressive
representation of the fractional diﬀerence filter.

8.2 Long Memory Time Series

275

0.6
0.2

0.4

ACF

0.8

1.0

d = 0.3

0

20

40

60

80

100

60

80

100

lags

-0.2

0.2

ACF

0.6

1.0

d = -0.3

0

20

40
lags

FIGURE 8.2. Autocorrelation of fractional integrated process.

memory; when −1/2 < d < 0, yt is stationary and has short memory, andis sometimes referred to as anti-persistent.
When a fractionally integrated series yt has long memory, it can also be
shown thatd = H − 1/2,
(8.6)
and thus d and H can be used interchangeably as the measure of longmemory. Hosking (1981) showed that the scaling property in (8.1) and thefrequency domain property in (8.2) are satisfied when 0 < d < 1/2.
Example 45 Theoretical ACF of fractionally integrated processes
In this example,use the S+FinMetrics function acf.FARIMA to plot thetheoretical autocorrelation function of a fractionally integrated process witha standard normal disturbance ut , for d = 0.3 and d = −0.3, respectively:> d.pos = acf.FARIMA(list(d=0.3, sigma2=1), 100)
> d.pos$acf = d.pos$acf / d.pos$acf[1]> d.neg = acf.FARIMA(list(d=-0.3, sigma2=1), 100)
> d.neg$acf = d.neg$acf / d.neg$acf[1]> par(mfrow=c(2,1))> plot(d.pos$lags, d.pos$acf, type="h", main="d = 0.3",+
xlab="lags", ylab="ACF")

276

8. Long Memory Time Series Modeling> plot(d.neg$lags, d.neg$acf, type="h", main="d = -0.3",+
xlab="lags", ylab="ACF")> par(mfrow=c(1,1))
and the plot is shown in Figure 8.2. Notice that the signs of the ACF
coeﬃcients are determined by the sign of d.

8.3 Statistical Tests for Long Memory
Given the scaling property of the autocorrelation function, the frequency
domain property and the fractionally integrated process representation of along memory time series, various tests have been proposed to determine theexistence of long memory in a time series. This section introduces the R/Sstatistic and GPH test. However, before getting into the details of thosetest statistics, it is important to note that the definition of long memory
does not dictate the general behavior of the autocorrelation function or its
spectral density. Instead, they only specify the asymptotic behavior whenk → ∞ or ω → 0. What this means is that for a long memory process, it
is not necessary for the autocorrelation to remain significant at large lags
as in the previous sp500 example, as long as the autocorrelation function
decays slowly. Beran (1994) gives an example to illustrate this property.

8.3.1 R/S Statistic
The best-known test for long memory or long range dependence is probably the rescaled range, or range over standard deviation, or simply R/Sstatistic, which was originally proposed by Hurst (1951), and later refinedby Mandelbrot and his coauthors. The R/S statistic is the range of partial
sums of deviations of a time series from its mean, rescaled by its standard
deviation. Specifically, consider a time series yt , for t = 1, · · · , T . The R/Sstatistic is defined as:
⎤
⎡
k
k
X
X
1 ⎣
QT =
(yj − ȳ) − min(yj − ȳ)⎦
(8.7)
max
1≤k≤T
sT 1≤k≤T j=1
j=1

q
PT
PT
where ȳ = 1/T i=1 yi and sT = 1/T i=1 (yi − ȳ)2 . If yt ’s are i.i.d.
normal random variables, then1
√ QT ⇒ V
T
where ⇒ denotes weak convergence and V is the range of a Brownian bridgeon the unit interval. Lo (1991) gives selected quantiles of V .

8.3 Statistical Tests for Long Memory277

Lo (1991) pointed out that the R/S statistic is not robust to short range
dependence. In particular, if yt is√autocorrelated (has short memory) thenthe limiting distribution of QT / T is V scaled by the square root of thelong run variance of yt . To allow for short range dependence in yt , Lo (1991)
modified the R/S statistic as follows:
⎤
⎡
k
k
X
X
1 ⎣
Q̃T =
(yj − ȳ) − min(yj − ȳ)⎦
(8.8)
max
1≤k≤T
σ̂ T (q) 1≤k≤T j=1
j=1

where the sample standard deviation is replaced by the square root of the
Newey-West estimate of the long run variance with bandwidth q.2 Lo (1991)
showed that if there is short memory but no long memory in yt , Q̃T also
converges to V , the range of a Brownian bridge.
The S+FinMetrics function rosTest can be used to test for long memory
in a time series using the R/S statistic (8.7) or the modified R/S statistic(8.8). For example, to test for long memory in the absolute returns of
S&P500 index, use the following command:> rosTest(abs(sp500))
Test for Long Memory: Modified R/S Test
Null Hypothesis: no long-term dependence
Test Statistics:
7.8823*** : significant at 5% level** : significant at 1% level
Total Observ.: 17054
Bandwidth : 14
By default, Lo’s modified R/S statistic is computed and the bandwidth q
for obtaining the long run variance is chosen to be [4(T /100)1/4 ], where T
is the sample size, and [·] denotes integer part of. In the above example, themodified R/S statistic is significant at 1% level of significance. A diﬀerent
bandwidth can be used by setting the optional argument bandwidth. If
bandwidth is set to zero, then classical R/S statistic is returned:> rosTest(abs(sp500), bandwidth=0)

2 See Chapter 2 for the definition and estimation of long run variance and the online
help file for the S+FinMetrics function asymp.var.

278

8. Long Memory Time Series Modeling

Test for Long Memory: R/S Test
Null Hypothesis: no long-term dependence
Test Statistics:
17.821*** : significant at 5% level** : significant at 1% level
Total Observ.: 17054
which is also significant at 1% level of significance in this case.

8.3.2 GPH Test
Based on the fractionally integrated process representation of a long memory time series as in (8.4), Geweke and Porter-Hudak (1983) proposed a
semi-nonparametric approach to testing for long memory. In particular, thespectral density of the fractionally integrated process yt is given by:ω
f (ω) = [4 sin2 ( )]−d fu (ω)
2

(8.9)

where ω is the Fourier frequency, and fu (ω) is the spectral density corresponding to ut . Note that the fractional diﬀerence parameter d can be
estimated by the following regression:ln f (ω j ) = β − d ln[4 sin2 (ωj)] + ej ,2(8.10)

for j = 1, 2, · · · , nf (T ). Geweke and Porter-Hudak (1993) showed thatusing a periodogram estimate of f (ω j ), the least squares estimate dˆ using
the above regression is normally distributed in large samples if nf (T ) = T αwith 0 < α < 1:π2dˆ ∼ N (d, Pnf)
6 j=1 (Uj − Ū )2

where

Uj = ln[4 sin2 (

ωj
)]
2

and Ū is the sample mean of Uj , j = 1, · · · , nf . Under the null hypothesisof no long memory (d = 0), the t-statistic
td=0 = dˆ ·

Ã

π2
Pnf
6 j=1 (Uj − Ū )2!−1/2(8.11)

8.3 Statistical Tests for Long Memory279

has a limiting standard normal distribution.
The S+FinMetrics function gphTest can be used to estimate d from
(8.10) and compute the test statistic (8.11), which is usually referred to as
the GPH test. The arguments taken by gphTest are:> args(gphTest)
function(x, spans = 1, taper = 0.1, pad = 0, detrend = F,
demean = T, alpha = 0.5, na.rm = F)
The optional arguments spans, taper, pad, detrend and demean are actually passed to the S-PLUS function spec.pgram to obtain a periodogram
estimate.3 The optional argument alpha is used to choose the number of
frequencies nf (T ). By default, nf (T ) = T α with α = 0.5. To illustrate theuse of gphTest, consider estimating d and testing for long memory in the
S&P 500 index absolute returns:> gph.sp500 = gphTest(abs(sp500),taper=0)> class(gph.sp500)[1] "gphTest"> names(gph.sp500)[1] "d""n""na""n.freq""std.err"

The result of gphTest is an object of class “gphTest” for which there is
only a print method:> gph.sp500
Test for Long Memory: GPH Test
Null Hypothesis: d = 0
Test Statistics:
d 0.4573
stat 7.608*** : significant at 5% level** : significant at 1% level
Total Observ.: 17054
Number of Freq: 130
The estimated value of d from (8.10) is dˆ = 0.457, which suggests longmemory, and the gph test statistic (8.11) is 7.608. Hence, the null of nolong memory is rejected at the 1% significance level. The estimate of d is
3 See S-PLUS Guide to Statistics for an introduction to the estimation of periodogram
and the usage of spec.pgram.

280

8. Long Memory Time Series Modelingclose to the nonstationary range. In fact, a 95% confidence interval for dbased on the asymptotic standard error> gph.sp500$std.err[1] 0.06011
is [0.337, 0.578] and contains d > 0.5.

8.4 Estimation of Long Memory Parameter
The previous section introduced the R/S statistic and the log-periodogram
regression to test for long memory in a time series. Cheung (1993) conducted a Monte Carlo comparison of these tests. Obtaining an estimate of
the long memory parameter H or d is also of interest. The GPH test produces an estimate of d automatically. This section will show that the R/Sstatistic can also be used to obtain an estimate of the Hurst coeﬃcient H.
It will also introduce two periodogram-based methods for estimating thelong memory parameter: the periodogram method and Whittle’s method.
In addition, the fractional diﬀerence parameter d can also be estimatedby using a general FARIMA(p, d, q) model, which will be introduced in
the next section. Taqqu, Teverovsky and Willinger (1995) and Taqqu andTeverovsky (1998) compared the performance of many diﬀerent estimatorsof the long memory parameter, including the above mentioned methods.

8.4.1 R/S Analysis
Section 8.3.1 mentioned that when there is no long memory in a stationary
time series, the R/S statistic converges to a random variable at rate T 1/2 .
However, when the stationary process yt has long memory, Mandelbrot(1975) showed that the R/S statistic converges to a random variable at
rate T H , where H is the Hurst coeﬃcient. Based on this result, the log-log
plot of the R/S statistic versus the sample size used should scatter arounda straight line with slope 1/2 for a short memory time series. In contrast,
for a long memory time series, the log-log plot should scatter around a
straight line with slope equal to H > 1/2, provided the sample size is large
enough.
To use the above method to estimate the long memory parameter H,
first compute the R/S statistic using k1 consecutive observations in thesample, where k1 should be large enough. Then increase the number ofobservations by a factor of f ; that is, compute the R/S statistic usingki = f ki−1 consecutive observations for i = 2, · · · , s. Note that to obtain
the R/S statistic with ki consecutive observations, one can actually divide
the sample into [T /ki ] blocks and obtain [T /ki ] diﬀerent values, where [·]denotes the integer part of a real number. Obviously, the larger ki is, the8.4 Estimation of Long Memory Parameter

281

5

10

R/S

50

100

Log-Log R/S Plot

50

100

500

1000

k

FIGURE 8.3. R/S estimate of long memory parameter.

smaller [T /ki ] is. A line fit of all those R/S statistics versus ki , i = 1, · · · , s,on the log-log scale yields an estimate of the Hurst coeﬃcient H.
The S+FinMetrics function d.ros implements the above procedure for
estimating H. The arguments taken by d.ros are:> args(d.ros)function(x, minK = 4, k.ratio = 2, minNumPoints = 3,
method = "ls", output = "d", plot = F, ...)
where minK specifies the value for k1 , k.ratio specifies the ratio factor
f , and minNumPoints specifies the minimum requirement for [T /ks ]. For
example, if minNumPoints=3, s must be such that one can divide T observations into three blocks with at least ks observations in each block. The
optional argument output specifies the type of output: if output="H", thenthe Hurst coeﬃcient is returned; if output="d", then the fractional diﬀerence parameter d is returned. For example, to estimate the Hurst coeﬃcient
for absolute returns of S&P500 index using R/S statistic, use the following
command:> d.ros(abs(sp500), minK=50, k.ratio=2, minNumPoints=10,+ output="H", plot=T)[1] 0.8393
By setting plot=T, the log-log plot of R/S statistics versus ki is generated,
as shown in Figure 8.3: the solid line represents the fitted line, and the282

8. Long Memory Time Series Modeling

dotted line represents the case for no long memory. In this case, the solidline is far away from the dotted line, which is substantial evidence for longmemory. The estimate of d using (8.6) is 0.3393.
The weakness of the above procedure is that for a particular sample, it
is not clear what value of k1 is “large enough”. In addition, for large valuesof ki , few values of the R/S statistic can be calculated unless the sample
size is very large. To mitigate the latter problem, set the optional argument
method="l1" when calling d.ros, which will direct the procedure to use
the L1 method or least absolute deviation (LAD) method, for the line fit,
and thus result in a robust estimate of the long memory parameter. For the
S&P 500 absolute returns, the results using the L1 method are essentiallythe same as using the least squares method:> d.ros(abs(sp500), minK=50, k.ratio=2, minNumPoints=10,+ output="H", method="l1", plot=F)[1] 0.8395

8.4.2 Periodogram Method
Section 8.3 demonstrates that for a long memory process, its spectral density approaches Cf ω 1−2H when the frequency ω approaches zero. Since thespectral density can be estimated by a periodogram, the log-log plot of periodogram versus the frequency should scatter around a straight line withslope 1 − 2H for frequencies close to zero. This method can also be used
to obtain an estimate of the long memory parameter H, and it is usuallyreferred to as the periodogram method.
The S+FinMetrics function d.pgram implements a procedure to estimate the long memory parameter using the periodogram method, whichcalls the S-PLUS function spec.pgram to obtain an estimate of periodogram.
The arguments taken by d.pgram are:> args(d.pgram)function(x, spans = 1, taper = 0.1, pad = 0, detrend = F,
demean = T, method = "ls", output = "d",lower.percentage = 0.1, minNumFreq = 10, plot = F, ...)
Similar to the gphTest function, the optional arguments spans, taper,
pad, detrend and demean are passed to spec.pgram to obtain the periodogram estimate. The optional argument lower.percentage=0.1 specifies that only the lower 10% of the frequencies are used to estimate H. For
example, to estimate the long memory parameter H of abs(sp500) withno tapering,use the following command:> d.pgram(abs(sp500), taper=0, output="H", plot=F)[1] 0.8741311
The implied estimate of d is then 0.3741.

8.4 Estimation of Long Memory Parameter

0.00005

10^-3
10^-4

Spectrum

10^-7

10^-6

10^-5

10^-4
10^-7

10^-6

10^-5

Spectrum

10^-3

10^-2

Log-Log Plot of Periodogram10^-2

Log-Log Plot of Periodogram

283

0.00050

0.00500
Frequency

0.05000

0.00005

0.00050

0.00500

0.05000

Frequency

FIGURE 8.4. Periodogram estimates of long memory parameter using leastsquares and LAD.

Just like with the R/S estimate of the long memory parameter, it can bediﬃcult to choose the value for lower.percentage. To obtain a more robustline fit, set the optional argument method="l1" when calling d.pgram, touse L1 method or LAD method instead of the default least squares fit. For
example, to compare the least squares and L1 fits for abs(sp500)use> par(mfrow=c(1,2))> H.ls = d.pgram(abs(sp500),taper=0, output="d",plot=T)> H.l1 = d.pgram(abs(sp500),taper=0, output="d",method="l1",+ plot=T)> H.ls[1] 0.3741> H.l1[1] 0.1637

8.4.3 Whittle’s Method
Whittle’s method for estimating d is based on a frequency domain maximum likelihood estimation of a fractionally integrated process (8.4). It
can be shown that the unknown parameters in (8.4) can be estimated by

284

8. Long Memory Time Series Modelingminimizing a discretized version ofQ(θ) =

Z π

I(ω)
dω
f
−π (θ; ω)

where θ is the vector of unknown parameters including the fractional difference parameter d, I(ω) is the periodogram of yt , and f (θ, ω) is the theoretical spectral density of yt . Refer to Beran (1994) for the derivation of
Whittle’s method.
To use Whittle’s method to estimate the fractional diﬀerence parameterd, use the S+FinMetrics function d.whittle. The syntax of d.whittle issimilar to but more simple than that of d.pgram:> args(d.whittle)
function(x, spans = 1, taper = 0.1, pad = 0, detrend = F,
demean = T, output = "d")where again the arguments spans, taper, pad, detrend and demean are
passed to the S-PLUS function spec.pgram to obtain the periodogram. For
example, to estimate the fractional diﬀerence parameter d of abs(sp500)
with no tapering, use the command:> d.whittle(abs(sp500), taper=0)[1] 0.2145822
A caveat to using d.whittle is that although the Whittle’s method
is defined for a general fractionally integrated process yt in (8.4), it is
implemented assuming that ut is a standard normal disturbance and thus
yt follows a FARIMA(0, d, 0) process.

8.5 Estimation of FARIMA and SEMIFAR Models
Previous sections illustrated how to test for long memory and estimate thelong memory parameter H or d. This section introduces the more flexible
fractional ARIMA models, which are capable of modeling both the longmemory and short run dynamics in a stationary time series. It will also
introduce a semiparametric model for long memory, which allows a semiparametric estimation of a trend component.
Many empirical studies have found that there is strong evidence for longmemory in financial volatility series, for example, see Lobato and Savin(1998) and Ray and Tsay (2000). Indeed, Andersen, Bollerslev, Diebold andLabys (1999) suggested to use FARIMA models to forecast daily volatility
based on logarithmic realized volatility. This section will focus on modelinga volatility series for the examples.

8.5 Estimation of FARIMA and SEMIFAR Models

285

8.5.1 Fractional ARIMA Models
The traditional approach to modeling an I(0) time series yt is to use the
ARIMA model:φ(L)(1 − L)d (yt − μ) = θ(L) t(8.12)
where φ(L) and θ(L) are lag polynomials
φ(L) = 1 −θ(L) = 1 −

p
X

φi Li

i=1

q
X

θj Lj

j=1

with roots outside the unit circle, and t is assumed to be an iid normal
random variable with zero mean and variance σ 2 . This is usually referred toas the ARIMA(p, d, q) model. By allowing d to be a real number instead of apositive integer, the ARIMA model becomes the autoregressive f ractionallyintegrated moving average (ARFIMA) model, or simply, fractional ARIMA(FARIMA) model4 .
For a stationary FARIMA model with −1/2 < d < 1/2, Sowell (1992)
described how to compute the exact maximum likelihood estimate (MLE).
The S-PLUS function arima.fracdiff implements a very fast procedure
based on the approximate MLE proposed by Haslett and Raftery (1989),
and refer the reader to the S-PLUS Guide to Statistics for a discussion ofthis procedure.
However, for many economic and financial time series, the data usuallyseem to lie on the borderline separating stationarity from non-stationarity.
As a result, one usually needs to decide whether or not to diﬀerence theoriginal time series before estimating a stationary FARIMA model, andthe inference of unknown FARIMA model parameters ignores this aspect of
uncertainty in d. Beran (1995) extended the estimation of FARIMA models
for any d > −1/2 by considering the following variation the FARIMA
model:
φ(L)(1 − L)δ [(1 − L)m yt − μ] = θ(L) t(8.13)
where −1/2 < δ < 1/2, and φ(L) and θ(L) are defined as above. The
integer m is the number of times that yt must be diﬀerenced to achieve
stationarity, and thus the diﬀerence parameter is given by d = δ + m. In
the following discussions and in the S+FinMetrics module, restrict m to beeither 0 or 1, which is usually suﬃcient for modeling economic and financial
time series. Note that when m = 0, μ is the expectation of yt ; in contrast,
when m = 1, μ is the slope of the linear trend component in yt .
4 The S+FinMetrics module actually provides a convenience function FAR for estimating a FARIMA(p, d, 0) model.

8. Long Memory Time Series Modeling

0.02

0.06

0.10

286

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
1996
1997
1998
1999
2000
2001

ACF

0.0

0.2

0.4

0.6

0.8

1.0

Series : log(ndx.vol)

0

50

100

150

200

Lag

FIGURE 8.5. Garman-Klass volatility of daily NASDAQ-100 returns.

The S+FinMetrics function FARIMA implements a procedure (based onarima.fracdiff) to estimate the FARIMA model (8.13), and the standard
errors of unknown parameters are computed using the asymptotic distribution derived by Beran (1995), which takes into account that m is also
determined by data rather than by a prior decision.
To illustrate the usage of the FARIMA function, consider modeling thevolatility of daily NASDAQ-100 index returns. In recent years, intra-day
security prices have been employed to compute daily realized volatility,
for example, see Andersen, Bollerslev, Diebold and Labys (2001a, 2001b).
Since intra-day security prices can be hard to obtain, compute daily volatility based on the daily opening, highest, lowest, and closing prices, as proposed by Garman and Klass (1980) and implemented by the S+FinMetrics
function TA.garmanKlass.
Example 46 Long memory modeling of NASDAQ-100 index volatility
The S+FinMetrics “timeSeries” ndx.dat contains the daily opening,highest, lowest and closing prices of NASDAQ-100 index from January 2,
1996 to October 12, 2001. First compute the volatility series using theGarman-Klass estimator and visualize its sample ACF:
> ndx.vol = TA.garmanKlass(ndx.dat[,"Open"], ndx.dat[,"High"],+ndx.dat[,"Low"], ndx.dat[,"Close"])> par(mfrow=c(2,1))

8.5 Estimation of FARIMA and SEMIFAR Models

287> plot(ndx.vol, reference.grid=F)> tmp = acf(log(ndx.vol), lag=200)> par(mfrow=c(1,1))
The volatility series ndx.vol and the sample ACF of logarithmic volatility
are shown in Figure 8.5. The ACF decays very slowly and remains highly
significant at lag 200, which indicates that the series may exhibit longmemory.
First estimate a FARIMA(0, d, 0) model for logarithmic volatility as follows:> ndx.d = FARIMA(log(ndx.vol), p=0, q=0)
> class(ndx.d)[1] "FARIMA"> names(ndx.d)[1] "call""model""m""delta"[5] "n.used""BIC""loglike""residuals"[9] "fitted""x.name""cov""CI"
The result of FARIMA is an object of class “FARIMA”, for which there areprint, summary, plot and predict methods as well as extractor functions
coef, fitted, residuals and vcov. The summary method gives> summary(ndx.d)Call:
FARIMA(x = log(ndx.vol), p = 0, q = 0)
Coefficients:
Value Std. Error t value Pr(>|t|)d 0.3534 0.0205
17.1964 0.0000
Information Criteria:
log-likelihood
BIC-732.31471.9Residual scale estimate: 0.4001
total residual
Degree of freedom: 1455
1453
Time period: from 01/04/1996 to 10/12/2001
The estimated model appears stationary and has long memory since 0 <dˆ < 1/2. Notice that m is estimated to be zero:> ndx.d$m
[1] 0

288

8. Long Memory Time Series Modeling

To allow for long memory and short memory at the same time, use a
FARIMA(p, d, q) model with p 6= 0 or q 6= 0. However, in practice, it isusually diﬃcult to choose the appropriate value for p or q. The FARIMA
function can choose the best fitting FARIMA model based on finding values of p ≤ pmax and q ≤ qmax which minimize the Bayesian Information
Criterion (BIC). For example, to estimate all the FARIMA models with0 ≤ p ≤ 2 and 0 ≤ q ≤ 2, use the optional arguments p.range andq.range as follows:> ndx.bic = FARIMA(log(ndx.vol), p.range=c(0,2),+ q.range=c(0,2), mmax=0)
p = 0 q = 0
p = 0 q = 1
p = 0 q = 2
p = 1 q = 0
p = 1 q = 1
p = 1 q = 2
p = 2 q = 0
p = 2 q = 1
p = 2 q = 2
In the above example, set mmax=0 to restrict m to be zero because the previous FARIMA(0, d, 0) model fit suggests that the data may be stationary.
A summary of the fitted model is> summary(ndx.bic)
Call:
FARIMA(x = log(ndx.vol), p.range = c(0, 2), q.range = c(0, 2),mmax = 0)
Coefficients:
Value Std. Error t value Pr(>|t|)d 0.4504 0.0287
15.6716 0.0000
MA(1) 0.2001 0.0359
5.5687 0.0000
Information Criteria:
log-likelihood
BIC-717.9342
1450.4325
Residual scale estimate: 0.3963
total residual
Degree of freedom: 1454
1451
Time period: from 01/05/1996 to 10/12/2001

8.5 Estimation of FARIMA and SEMIFAR Models

289

Normal Q-Q Plotndx.bic
10/28/1997

1.5

1.0

Residuals

0.5

0.0

-0.5

-1.0

02/13/1998-1.5
12/24/1998
-2

0

2

Quantile of Standard Normal

FIGURE 8.6. FARIMA residual QQ-plot of log(ndx.vol).

BIC of all models estimated:
q=0
q=1
q=2
p=0 1466.898 1450.432 1451.055
p=1 1457.319 1462.694 1455.590
p=2 1464.800 1457.243 1464.238
The BIC values for all the models considered are shown in the output.
The model minimizing the BIC is a FARIMA(0, d, 1) model. The estimatesof d and the moving average coeﬃcient are very significant, but the 95%
Wald-type confidence interval of d includes 1/2 and thus the non-stationary
case.5Further diagnostics of the model fit can be obtained by using the plot
method:> plot(ndx.bic)
Make a plot selection (or 0 to exit):
1: plot: all
2: plot: response vs fitted values
5 Currently the standard error of the mean parameter is not available because
arima.fracdiff concentrates out the mean and thus does not compute its standard
error.

290

8. Long Memory Time Series Modeling

Residual Autocorrelation

ndx.bic
1.0

0.8

ACF

0.6

0.4

0.2

0.0

0

5

10

15

20

25

30

Lag

FIGURE 8.7. FARIMA residual ACF of log(ndx.vol).

3: plot: response and fitted values
4: plot: normal QQ-plot of residuals
5: plot: residuals
6: plot: standardized residuals
7: plot: residual histogram
8: plot: residual ACF
9: plot: residual PACF
10: plot: residual^2 ACF
11: plot: residual^2 PACF
Selection:
For example, if 4 is chosen at the prompt the normal qq-plot of the model
residuals t will be shown as in Figure 8.6. It seems that the normality
assumption agrees well with the data. If 8 is chosen at the prompt, the ACF
of model residuals will be shown as in Figure 8.7, and the FARIMA model
is very successful at capturing the long memory in logarithmic volatility.
In the above example, m can also be allowed to be estimated:> ndx.bic2 = FARIMA(log(ndx.vol),p.range=c(0,2),+ q.range=c(0,2), mmax=1)
p = 0 q = 0...
p = 2 q = 2

8.5 Estimation of FARIMA and SEMIFAR Models

291> ndx.bic2$m[1] 1> summary(ndx.bic2)
Call:
FARIMA(x = log(ndx.vol), p.range = c(0, 2), q.range =c(0, 2), mmax = 1)
Coefficients:
Value Std. Error t value Pr(>|t|)d 0.5161 0.1056
4.8864 0.0000
AR(1) 1.1387 0.3753
3.0340 0.0025
AR(2) -0.1561 0.3724-0.4193 0.6751
MA(1) 1.4364 0.4416
3.2528 0.0012
MA(2) -0.4309 0.7574-0.5689 0.5695
Information Criteria:
log-likelihood
BIC-696.31429.0Residual scale estimate: 0.3903
total residual
Degree of freedom: 1453
1447
Time period: from 01/08/1996 to 10/12/2001
BIC of all models estimated:
q=0 q=1 q=2
p=0 1467 1450 1451
p=1 1457 1459 1456
p=2 1456 1454 1429
Here the best fitting model is a FARIMA(2, 0.52, 2) model. However, thevalues of the AR and MA coeﬃcients indicate an explosive model. The
problem appears to be near canceling roots in the AR and MA polynomials.
If the model is re-fitted with p = q = 1, the results make more sense:> ndx.bic2 = FARIMA(log(ndx.vol), p=1, q=1, mmax=1)> summary(ndx.bic2)
Call:
FARIMA(x = log(ndx.vol), p = 1, q = 1, mmax = 1)
Coefficients:
Value Std. Error t value Pr(>|t|)292

8. Long Memory Time Series Modeling

Residual Autocorrelation

ndx.bic2
1.0

0.8

ACF

0.6

0.4

0.2

0.0

0

5

10

15

20

25

30

Lag

FIGURE 8.8. Residual ACF from FARIMA(1, 0.51, 1) model fit to log(ndx.vol).

d
AR(1)
MA(1)

0.5051
0.23760.4946

0.0436
0.0687
0.0367

11.59653.459713.48940.00000.00060.0000Information Criteria:
log-likelihood
BIC-712.71447.3
Residual scale estimate: 0.3948
total residual
Degree of freedom: 1454
1450
Time period: from 01/05/1996 to 10/12/2001
Figure 8.8 gives the residual ACF from the above model. The long memory
behavior has been well captured by the model. However, the fitted modelhas the undesirable property of being non-stationary.

8.5.2 SEMIFAR Model
The previous subsection demonstrated that for logarithmic volatility of
NASDAQ-100 index returns, the FARIMA model chosen by BIC suggeststhat the underlying series may be non-stationary. In addition, from the time

8.5 Estimation of FARIMA and SEMIFAR Models

293

series plot in Figure 8.5, the volatility has become much larger since themiddle of 2000. To allow for a possible deterministic trend in a time series,
in addition to a stochastic trend, long memory and short memory components, Beran, Feng and Ocker (1998), Beran and Ocker (1999), and Beranand Ocker (2001) proposed the semiparametric f ractional autoregressive(SEMIFAR) model. The SEMIFAR model is based on the following extension to the FARIMA(p, d, 0) model (8.12):
φ(L)(1 − L)δ [(1 − L)m yt − g(it )] = t(8.14)

for t = 1, · · · , T . The above equation is very similar to (8.13), except thatthe constant term μ is now replaced by g(it ), a smooth trend function on[0, 1], with it = t/T . By using a nonparametric kernel estimate of g(it ),the S+FinMetrics function SEMIFAR implements a procedure to estimate
the SEMIFAR model, and it uses BIC to choose the short memory autoregressive order p. Refer to Beran, Feng and Ocker (1998) for a detailed
description of the algorithm.
Example 47 Estimation of SEMIFAR model for NASDAQ-100 index
volatility
To obtain a SEMIFAR model of logarithmic volatility of NASDAQ-100
index returns, use the following command:> ndx.semi = SEMIFAR(log(ndx.vol), p.range=c(0,2), trace=F)> class(ndx.semi)[1] "SEMIFAR"
Note that the optional argument trace=F is used to suppress the messages printed by the procedure. The result of SEMIFAR is an object of class“SEMIFAR” for which there are print, summary, plot and predict methods
as well as extractor functions coef, residuals and vcov. The componentsof ndx.semi are> names(ndx.semi)[1] "model""m"[5] "loglike""trend"[9] "Cf""nu"[13] "CI""call""delta""BIC""g.CI""bandwidth""residuals" "cov"

The basic fit is> ndx.semi
Call:
SEMIFAR(x = log(ndx.vol), p.range = c(0, 2), trace = F)
Difference:
0: estimates based on original series.

294

8. Long Memory Time Series Modeling

Smoothed Trend

-4.5

-5.5

-4.5

-4.1

-3.5

-2.5

-3.7

Original Series

Q1
1996

Q3
1997

Q3
1998

Q3
1999

Q3
2000

Q4
2001

Q1
1996

Q3
1997

Q3
1999

Q3
2000

Q4
2001

Q3
2000

Q4
2001

Residuals

-1.5

-0.4

-0.5

0.0

0.5

0.4

1.5

Fitted Values

Q3
1998

Q1
1996

Q3
1997

Q3
1998

Q3
1999

Q3
2000

Q4
2001

Q1
1996

Q3
1997

Q3
1998

Q3
1999

FIGURE 8.9. SEMIFAR decomposition of log(ndx.vol).

FAR coefficients:d0.2928Residual scale estimate: 0.3946
total residual
Degree of freedom: 1453
1452
Time period: from 01/08/1996 to 10/12/2001
From the above output, after accounting for a smooth nonparametric trend
component g(it ), the logarithmic volatility appears to be stationary and has
long memory.
The estimated trend component can be visualized by calling the plot
method of fitted model object:> plot(ndx.semi)
Make a plot selection (or 0 to exit):
1: plot: all
2: plot: trend, fitted values, and residuals
3: plot: normal QQ-plot of residuals

8.5 Estimation of FARIMA and SEMIFAR Models

295

Residual Autocorrelation

ndx.semi
1.0

0.8

ACF

0.6

0.4

0.2

0.0

0

5

10

15

20

25

30

Lag

FIGURE 8.10. SEMIFAR residual ACF of log(ndx.vol).

4: plot: standardized residuals
5: plot: residual histogram
6: plot: residual ACF
7: plot: residual PACF
8: plot: residual^2 ACF
9: plot: residual^2 PACF
Selection:
If 2 is selected at the prompt, a plot as in Figure 8.9 will be shown, whichindicates the original time series, the estimated smooth trend component,
the fitted values and model residuals. The smooth trend component is alsoplotted with a confidence band. If the trend falls outside the confidenceband, it indicates that the trend component is significant. In this case, thetrend in logarithmic volatility appears to be very significant, at least for
the time period investigated. The model fit can also be checked by choosing
6 at the prompt, which will generate the ACF plot of residuals, as shownin Figure 8.10. Again, the SEMIFAR model seems to be very successful atmodeling the long memory in the original time series.
Prediction from SEMIFAR models will be discussed in section 8.7.

296

8. Long Memory Time Series Modeling

8.6 Long Memory GARCH Models
8.6.1 FIGARCH and FIEGARCH Models
The previous section showed that the FARIMA or SEMIFAR model can beused directly to model the long memory behavior observed in the volatilityof financial asset returns, given that a time series representing the volatility
exists. However, sometimes a reliable estimate of volatility may be hard to
obtain, or the user may want to model the dynamics of the asset returns
together with its volatility. In those situations, the GARCH class models
provide viable alternatives for volatility modeling.
Section 7.5.2 of Chapter 7 has illustrated that two components GARCH
models can be used to capture the high persistence in volatility by allowing
a highly persistent long run component and a short run transitory component in volatility. This subsection shows how GARCH models can beextended to allow directly for long memory and high persistence in volatility.
FIGARCH Model
Section 22.16 of Chapter 7 shows that a basic GARCH(1, 1) model can be
written as an ARMA(1, 1) model in terms of squared residuals. In the same
spirit, for the GARCH(p, q) model:
σ 2t = a +

p
X
i=1

ai 2t−i +qXbj σ 2t−jj=1

easily shows that it can be rewritten as follows:
φ(L) 2t = a + b(L)ut(8.15)

where
ut = 2t − σ2tφ(L) = 1 − φ1 L − φ2 L2 − · · · − φm Lmb(L) = 1 − b1 L − b2 L2 − · · · − bq Lq

with m = max(p, q) and φi = ai + bi . Obviously equation (8.15) represents
an ARMA(m, q) process in terms of squared residuals 2t with ut being a
MDS disturbance term.
The high persistence in GARCH models suggests that the polynomial
φ(z) = 0 may have a unit root, in which case the GARCH model becomesthe integrated GARCH (IGARCH) model. See Nelson (1990) for which
the unconditional variance does not exist. To allow for high persistence
and long memory in the conditional variance while avoiding the complications of IGARCH models, extend the ARMA(m, q) process in (8.15) to a

8.6 Long Memory GARCH Models

297

FARIMA(m, d, q) process as follows:φ(L)(1 − L)d 2t = a + b(L)ut(8.16)

where all the roots of φ(z) = 0 and b(z) = 0 lie outside the unit circle.
When d = 0, this reduces to the usual GARCH model; when d = 1, thisbecomes the IGARCH model; when 0 < d < 1, the fractionally diﬀerenced
squared residuals, (1− L)d 2t , follow a stationary ARMA(m, q) process. The
above FARIMA process for ε2t can be rewritten in terms of the conditionalvariance σ 2t :b(L)σ 2t = a + [b(L) − φ(L)(1 − L)d ] 2t .
(8.17)
Baillie, Bollerslev and Mikkelsen (1996) refered to the above model as
the fractionally integrated GARCH, or FIGARCH(m, d, q) model. When0 < d < 1, the coeﬃcients in φ(L) and b(L) capture the short run dynamics
of volatility, while the fractional diﬀerence parameter d models the long run
characteristics of volatility.
FIEGARCH
The FIGARCH model directly extends the ARMA representation of squared
residuals, which results from the GARCH model, to a fractionally integrated model. However, to guarantee that a general FIGARCH modelis stationary and the conditional variance σ 2t is always positive, usuallycomplicated and intractable restrictions have to be imposed on the model
coeﬃcients. For example, see Baillie, Bollerslev and Mikkelsen (1996) orBollerslev and Mikkelsen (1996) for a discussion.
Noting that an EGARCH model can be represented as an ARMA process
in terms of the logarithm of conditional variance and thus always guarantees
that the conditional variance is positive, Bollerslev and Mikkelsen (1996)
proposed the following fractionally integrated EGARCH (FIEGARCH)
model:qXd2
φ(L)(1 − L) ln σ t = a +(bj |xt−j | + γ j xt−j )(8.18)
j=1

where φ(L) is defined as earlier for the FIGARCH model, γ j 6= 0 allows
the existence of leverage eﬀects, and xt is the standardized residual:
xt =

t

σt

(8.19)

Bollerslev and Mikkelsen (1996) showed that the FIEGARCH model isstationary if 0 < d < 1.

8.6.2 Estimation of Long Memory GARCH Models
Given the iterative formulations of conditional variance as in (8.17) and(8.18), the FIGARCH and FIEGARCH model coeﬃcients can be estimated298

8. Long Memory Time Series Modelingusing maximum likelihood estimation (MLE), if the residuals follow a conditional normal distribution. The S+FinMetrics function fgarch can beused to estimate the long memory FIGARCH or FIEGARCH model.
The syntax of fgarch is very similar to that of the garch function, exceptthat ~figarch(m,q) is used as the FIGARCH conditional variance formulaand ~fiegarch(m,q) as the FIEGARCH conditional variance formula. For
example, to fit a FIGARCH(1, d, 1) model to daily stock returns of Dell
Computer contained in the S+FinMetrics “timeSeries” object dell.s,
simply use the following command:> dell.figarch = fgarch(dell.s~1, ~figarch(1,1))
Initializing model parameters.
Iteration No.
1: log-likelihood=-3282.303431...
Iteration No. 10: log-likelihood=-3279.508705
Convergence in gradient.
> oldClass(dell.figarch)[1] "fgarch" "garch"
The returned object is of class “fgarch”, which inherits the “garch” class.
Consequently, most of the method functions for a “garch” object (e.g.
print, summary, plot, predict, coef, residuals, sigma.t, vcov)also
work for a “fgarch” object. One exception is that currently there is no
simulate method for “fgarch” objects. For example, the print method
gives> dell.figarchCall:fgarch(formula.mean = dell.s~1, formula.var = ~figarch(1, 1))
Mean Equation: dell.s ~1
Conditional Variance Equation:~figarch(1, 1)

Coefficients:
C 0.4422
A 0.6488
GARCH(1) 0.6316
ARCH(1) 0.4481
fraction 0.2946
The estimate of d is 0.295, which indicates the existence of long memory.
However, the sum ARCH(1) and GARCH(1) is greater than one whichindicates a nonstationary model.

8.6 Long Memory GARCH Models

299

If the FIEGARCH model instead of FIGARCH model is desired, theoptional argument leverage can be used to allow for leverage eﬀects. For
example,> dell.fiegarch = fgarch(dell.s~1, ~fiegarch(1,1), leverage=T)
Initializing model parameters.
Iteration No.
1: log-likelihood=-3286.169656...
Iteration No. 20: log-likelihood=-3274.244677
Convergence in gradient.
> summary(dell.fiegarch)Call:fgarch(formula.mean = dell.s~1, formula.var = ~fiegarch( 1, 1),leverage = T)
Mean Equation: dell.s ~1
Conditional Variance Equation:~fiegarch(1, 1)-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value
Pr(>|t|)C 0.39494
0.089814.397 5.946e-006
A -0.06895
0.04237 -1.627 5.195e-002
GARCH(1) 0.65118
0.178203.654 1.343e-004
ARCH(1) 0.15431
0.04578
3.370 3.867e-004
LEV(1) -0.09436
0.02691 -3.507 2.346e-004
fraction 0.34737
0.11408
3.045 1.188e-003-------------------------------------------------------------AIC(6) = 6560.5
BIC(6) = 6591.3
Normality Test:-------------------------------------------------------------Jarque-Bera P-value Shapiro-Wilk P-value
13.22 0.001348
0.9888 0.7888
Ljung-Box test for standardized residuals:--------------------------------------------------------------

300

8. Long Memory Time Series Modeling

Statistic P-value Chi^2-d.f.
13.13 0.3597
12
Ljung-Box test for squared standardized residuals:-------------------------------------------------------------Statistic P-value Chi^2-d.f.
14.51 0.2696
12
Lagrange multiplier test:-------------------------------------------------------------Lag 1 Lag 2 Lag 3 Lag 4 Lag 5 Lag 6
Lag 7 Lag 8-0.925 0.6083 -1.581 0.2593 0.3943 0.6991 -0.03191 0.3339
Lag 9 Lag 10 Lag 11 Lag 12C
1.959 -0.8794 2.422 0.1089 0.8896
TR^2 P-value F-stat P-value
15.1 0.2362 1.389 0.2797
In the above output, C corresponds to the constant term in the conditionalmean equation, A corresponds to the constant term a, GARCH(1) corresponds to b1 , ARCH(1) corresponds to φ1 , LEV(1) corresponds to γ 1 andfraction corresponds to the fractional diﬀerence parameter d in the conditional variance equation (8.18). Notice that the leverage term is negativeand significant, and the sum of ARCH(1) and GARCH(1) is now less thanone. It appears that the FIEGARCH model fits the data better than the
FIGARCH model.
Just like for “garch” objects, the generic plot function can be used visually to diagnose the model fit. Use compare.mgarch to compare multiple
model fits. For example, consider comparing the above two FIGARCH,
FIEGARCH with short memory GARCH and EGARCH models:> dell.garch = garch(dell.s~1, ~garch(1,1), trace=F)> dell.egarch = garch(dell.s~1, ~egarch(1,1),+ leverage=T, trace=F)> dell.comp = compare.mgarch(dell.garch,dell.egarch,+ dell.figarch,dell.fiegarch)> dell.compdell.garch dell.egarch dell.figarch dell.fiegarch
AIC
6564
6559
6569
6560
BIC
6585
6584
6595
6591
Likelihood
-3278
-3274
-3280
-3274
Here, the EGARCH and FIEGARCH models seem to provide better fits
than the GARCH and FIGARCH models. The qq-plots of standardized
residuals for the four models can be compared using:8.6 Long Memory GARCH Models

301

QQ-Plot of Standardized Residuals-2dell.figarch

02

dell.fiegarch
42Standardized Residuals

0-2-4
dell.garchdell.egarch

4
2
0
-2
-4

-2

0

2

Quantile of Comparison Distribution

FIGURE 8.11. qq-plot of standardized residuals from long memory GARCH models.

> plot(dell.comp, qq=T)
and the plot is shown in Figure 8.11, where the FIEGARCH model seems
to provide a slightly better fit to the outliers in both tails.

8.6.3 Custom Estimation of Long Memory GARCH Models
ARMA Terms and Exogenous Variables
Just like with the garch function, the fgarch function also allows ARMA
terms and exogenous variables in the conditional mean equation, as well as
the conditional variance equation.
Example 48 Trading volume and volatility (extended)
The previous subsection shows that the fitted FIEGARCH model object
dell.fiegarch suggests that there may be long memory in the volatilityof Dell stocks. In Section 7.5 of Chapter 7, the changes in trading volume
were used to explain the volatility of Dell stocks. If there is a 1% change
in trading volume, it will cause about 1.4% change in conditional varianceusing an EGARCH model for volatility. In this example, the analysis using
the FIEGARCH model instead of the EGARCH model is done again.
> dell.mod2 = fgarch(dell.s~1, ~fiegarch(1,1) +302

8. Long Memory Time Series Modeling+ seriesData(d.volume), series.start=2)> summary(dell.mod2)
Call:fgarch(formula.mean = dell.s ~1, formula.var = ~fiegarch(1,1)+ seriesData(d.volume), series.start = 2)
Mean Equation: dell.s ~ 1
Conditional Variance Equation:
seriesData(d.volume)~ fiegarch(1, 1) +-------------------------------------------------------------Estimated Coefficients:-------------------------------------------------------------Value Std.Error t value
Pr(>|t|)C 0.14514
0.062452.3242 1.014e-002
A -0.13640
0.03117 -4.3761 6.542e-006
GARCH(1) 0.04123
0.10703
0.3852 3.501e-001
ARCH(1) 0.16600
0.038094.3583 7.091e-006
seriesData(d.volume) 1.49123
0.07814 19.0849 0.000e+000
fraction 0.80947
0.07523 10.7596 0.000e+000...
First, compare the above output with dell.fiegarch, the FIEGARCH
model fitted in the previous subsection. After controlling for the eﬀects oftrading volume, the GARCH coeﬃcient has decreased significantly and become insignificant, while the fractional diﬀerence parameter has increased
from 0.34 to 0.8. Second, compare this with the EGARCH model dell.mod
in Chapter 7: after allowing for long memory, the GARCH coeﬃcient decreased from 0.95 to 0.04, while the eﬀects of trading volume remain almost
the same.
Control of Model Estimation
For a “fgarch” object, all the model specific information is contained in themodel component of the object. For example, view the model informationof the fitted dell.figarch object as follows:> dell.figarch$model
Mean Equation: dell.s ~ 1
Conditional Variance Equation:~ figarch(1, 1)

8.6 Long Memory GARCH Models

303

Valuesconstant in mean 0.4422
constant in var 0.6488
GARCH(1) 0.6316
ARCH(1) 0.4481
fraction 0.2946
This model object can be edited to provide starting values for re-estimating
the same model with the same or a diﬀerent time series.6 For example, to
use this set of values as starting values for a FIGARCH model of the timeseries hp.s, use the following command:> hp.figarch = fgarch(series=hp.s*100,model=dell.figarch$model)
Iteration No.
1: log-likelihood=-4419.644144...
Iteration No. 10: log-likelihood=-4390.179116
Convergence in gradient.
> hp.figarchCall:fgarch(series = hp.s * 100, model = dell.figarch$model)
Mean Equation: dell.s ~ 1
Conditional Variance Equation:~ figarch(1, 1)

Coefficients:
C 0.05776
A 0.55897
GARCH(1) 0.49103
ARCH(1) 0.40210
fraction 0.22533
Unlike the garch and mgarch functions which use the BHHH algorithm
for MLE, the FIGARCH/FIEGARCH models are estimated using the BFGS
algorithm (for example, see Press, Teukolsky, Vetterling, and Flannery,
1992 for details). Since daily financial returns are very small numbers, thealgorithm can become badly scaled and may fail to converge. That is whyin the above example the percentage returns are used to improve the convergence.
6 However, unlike “garch” and “mgarch” objects, currently the coeﬃcients cannot
be fixed at certain values during the estimation of long memory GARCH models. See
Section 13.7 in Chapter 13 for discussions related to “garch” and “mgarch” objects.

304

8. Long Memory Time Series Modeling

Other aspects of the BFGS algorithm can be controlled by passing theoptional argument control to the fgarch function, where control must
be set to an object returned by the fgarch.control function. For example,
to change the convergence tolerance of gradient zeroing from the default
value of 1e-5 to 1e-6 when fitting a FIGARCH model to dell.s, use thefollowing command:> fgarch(dell.s~1, ~figarch(1,1), control=+ fgarch.control(tolg=1e-6))
The on-line help file for fgarch.control provides more details for thearguments accepted by the fgarch.control function.
Finally, introducing the FIGARCH/FIEGARCH models illustrated thatboth models are essentially an ARMA model fitted to the fractionally diﬀerenced squared residuals or fractionally diﬀerenced logarithmic conditionalvariance. The fractional diﬀerence operator is defined in (8.5), which involves an infinite order autoregressive filter. In practice, a very large number is usually chosen to approximate the fractional diﬀerence operator.
Following Bollerslev and Mikkelsen (1996), the fgarch function sets theorder to be 1000 by default. To change this number to another value, pass
the optional argument lag to fgarch.control. For example, the command> fgarch(dell.s~1, ~figarch(1,1), control=+ fgarch.control(lag=500))
estimates a FIGARCH model using only 500 lags to approximate the fractional diﬀerence operator.

8.7 Prediction from Long Memory Models
S+FinMetrics long memory modeling functions such as FARIMA, SEMIFAR
and fgarch all return objects for which there are corresponding predict
methods. Therefore, predictions from those fitted model objects can bereadily generated. This section gives an overview of how to predict from along memory process. In particular, the truncation method and the best
linear predictor will be introduced, see Bhansali and Kokoszka (2001). How
to predict from fitted model objects in S+FinMetrics module will be illustrated.

8.7.1 Prediction from FARIMA/SEMIFAR Models
To illustrate prediction from long memory processes, consider the FARIMA
model in (8.12), which can be rewritten as:
φ(L)(1 − L)d(yt − μ) = t
θ(L)8.7 Prediction from Long Memory Models

305

The lag polynomial on the left hand side of the above equation can be
expressed as an infinite order polynomial so that a FARIMA(p, d, q) model
can be equivalently expressed as an AR(∞) model. Once the parametersof the FARIMA(p, d, q) model are known, one can solve for the parametersof the equivalent AR(∞) model. In practice, however, forecasting from theAR(∞) representation usually truncates the AR(∞) model to an AR(p)model with a very large value of p. This method is usually referred to as
the truncation method.
In the truncation method, the AR(p) coeﬃcients are the first p coeﬃcients of the AR(∞) representation of the FARIMA(p, d, q) model. However, for any stationary process, choose to use p lagged values to predict
future values:ŷT +1 = ψ 1 yT + · · · + ψ p yT −p+1
where ψ i for i = 1, · · · , p are chosen to yield the best linear predictor ofyT +1 in terms of yT , · · · , yT −p+1 for any T . Note that although both theabove method and the truncation method use an AR(p) model for prediction, the AR(p) coeﬃcients in the truncation method do not necessarily
correspond to best linear prediction coeﬃcients ψ i . Brockwell and Davis(1991) showed that the best linear prediction coeﬃcients can be recursively
computed using the Durbin-Levinson algorithm given the autocovariancefunction of the stationary process.7
The predict method for “FARIMA” objects in S+FinMetrics implements
the Durbin-Levinson algorithm to compute the forecasts. The arguments
taken by predict.FARIMA are:> args(predict.FARIMA)
function(x, n.predict = 1, ar.approx = 50, kapprox = 100000,
series = NULL)
where n.predict indicates the number of steps to predict ahead, ar.approx
gives the order p of the AR representation used for prediction, kapprox ispassed to acf.FARIMA to obtain the theoretical autocovariance function of
the FARIMA model, and series can be used to pass the original timeseries used to fit the model. For example, to predict 100 steps ahead using
an AR(100) representation from the fitted model object ndx.bic, use thefollowing command:> ndx.pred1 = predict(ndx.bic, n.predict=100, ar.approx=100)> class(ndx.pred1)
7 Although exact expressions of the autocovariance functions for FARIMA(p, d, q)models have been given by Sowell (1992), the derivation assumes that all the roots of
the AR polynomial are distinct. The S+FinMetrics function acf.FARIMA implements anumerical quadrature procedure based on fast Fourier transform to approximate theautocovariance function of the FARIMA models, as proposed by Bhansali and Kokoszka(2001).

8. Long Memory Time Series Modeling

-3.5
-4.0

values

-3.0

306

1250

1300

1350

1400

1450

1500

1550

index

FIGURE 8.12. Predictions from a FARIMA model.

[1] "forecast"
The returned object has class “forecast” and has components> names(ndx.pred1)[1] "values" "std.err" "coef"
where the values contains the predicted values, std.err contains the standard errors of the predictions, and coef contains the best linear prediction
coeﬃcients ψ i (i = 1, . . . , p). The predictions and standard errors can beseen by calling the summary function on a “forecast” object. For example:> summary(ndx.pred1)
Predicted Values with Standard Errors:
prediction std.err
1-step-ahead -3.4713
0.39652-step-ahead -3.5407
0.47323-step-ahead -3.5638
0.50234-step-ahead -3.5792
0.51485-step-ahead -3.5883
0.5204
...

307

0.0

0.2

coef

0.4

0.6

8.7 Prediction from Long Memory Models

0

20

40

60

80

100

FIGURE 8.13. Best linear prediction coeﬃcients.

A “forecast” object can be plotted together with the original data
to visualize the predictions. For example, since ndx.bic was fitted using
log(ndx.vol), the predictions can be visualized as follows:> plot(ndx.pred1, log(ndx.vol), n.old=200)
where the optional argument n.old specifies the number of observationsin the original data to be used in the plot. The plot is shown Figure 8.12.
Also, the best linear prediction coeﬃcients can also be visualized to see theeﬀects of using more lags for prediction. For example:> plot(ndx.pred1$coef, type="h", ylab="coef")
generates the coeﬃcient plot shown in Figure 8.13. Adding lags beyond 30
should not change the predictions very much.
In S+FinMetrics, predictions from SEMIFAR models are computed ina similar fashion to predictions from FARIMA models, except that there
is a choice to use a constant extrapolation or linear extrapolation for thetrend component8> args(predict.SEMIFAR)
function(x, n.predict = 1, ar.approx = 50, kapprox = 100000,
8 We refer to Beran and Ocker (1999) for the details of predicting from a SEMIFAR
model.

308

8. Long Memory Time Series Modeling

GARCH Predictions

Predicted Conditional SD

3.4

3.2

3.0

2.8

2.6

0

20

40

60

80

100

FIGURE 8.14. Predictions from FIGARCH model.

trend = "constant", series = NULL)
For example, to produce 100 steps ahead forecasts from the fitted model
object ndx.semi using constant extrapolation, use the following command:> ndx.pred2 = predict(ndx.semi,n.predict=100,trend="constant")
The returned object is also a “forecast” object, so the predictions can bevisualized together with the original data
> plot(ndx.pred2, ndx.vol, n.old=200)

8.7.2 Prediction from FIGARCH/FIEGARCH Models
Predictions from the S+FinMetrics long memory GARCH models are computed using the truncation method because the user needs to generate forecasts for both the level and the volatility of the series at the same time.
The arguments taken by the predict method are:> args(predict.fgarch)function(object, n.predict = 1, n.lag = 1000)
NULL
where n.predict specifies the number of periods to predict ahead, andn.lag specifies the order p of the AR(p) representation used in the truncation method. For example, to use an AR(100) representation to predict 100

8.8 References309

steps ahead from the fitted model object dell.figarch, use the following
command:> dell.pred3 = predict(dell.figarch, n.predict=100, n.lag=100)> oldClass(dell.pred3)[1] "predict.fgarch" "predict.garch"
The returned object is of class “predict.fgarch”, which inherits from theclass “predict.garch”. So just like for a “predict.garch” object, use thegeneric plot function to visualize the volatility forecast:> plot(dell.pred3, hgrid=T, vgrid=T)and the plot is shown in Figure 8.14. The volatility predictions approach
the long run level in a slowly decaying fashion for the long memory GARCH
model9 .

8.8 References
Andersen, T., Bollerslev, T., Diebold, F. X., and Labys, P.
(1999): “(Understanding, Optimizing, Using and Forecasting) Realized Volatility and Correlation,” Manuscript, Northwestern University, Duke University and University of Pennsylvania.
Andersen, T., Bollerslev, T., Diebold, F. X., and Labys, P.
(2001a): “The Distribution of Realized Exchange Rate Volatility,” Journal of the American Statistical Association, 96, 42-55.
Andersen, T., Bollerslev, T., Diebold, F. X., and Labys, P.
(2001b): “The Distribution of Realized Stock Return Volatility,” Journalof Financial Economics, 61, 43-76.
Baillie, R. T. (1996). “Long Memory Processes and Fractional Integration in Econometrics,” Journal of Econometrics, 73, 5-59.
Baillie, R. T., Bollerslev, T., and Mikkelsen, H. O. (1996). “Fractionally Integrated Generalized Autoregressive Conditional Heteroskedasticity,” Journal of Econometrics, 74, 3-30.
Beran, J. (1994). Statistics for Long Memory Processes, Chapman andHall, New York.
Beran, J. (1995). “Maximum Likelihood Estimation of the Diﬀerencing
Parameter for Invertible Short and Long Memory ARIMA Models,” Journal of Royal Statistical Society Series B, 57(4), 659-672.
9 Currently, standard errors are not available for the volatility predictions.

310

8. Long Memory Time Series Modeling

Beran, J., Feng, Y., and Ocker, D. (1999). “SEMIFAR Models,” Technical Report, 3/1999, SFB 475 University of Dortmund.
Beran, J., and Ocker, D. (1999). “SEMIFAR Forecasts, with Applications to Foreign Exchange Rates,” Journal of Statistical Planning andInference, 80, 137-153.
Beran, J., and Ocker, D. (2001). “Volatility of Stock Market Indices An Analysis Based on SEMIFAR Models,” Journal of Business and Economic Statistics, 19(1), 103-116.
Bhansali, R. J., and Kokoszka, P. S. (2001). “Computation of theForecast Coeﬃcients for Multistep Prediction of Long-range Dependent
Time Series,” International Journal of Forecasting, 18(2), 181-206.
Bollerslev, T., and Mikkelsen, H. O. (1996). “Modeling and Pricing
Long Memory in Stock Market Volatility,” Journal of Econometrics, 73,
151-184.
Brockwell, P. J., and Davis, R. A. (1991). Time Series: Theory andMethods, Springer-Verlag, New York.
Cheung, Y.W. (1993). “Tests for Fractional Integration: A Monte Carlo
Investigation,” Journal of Time Series Analysis, 14, 331-345.
Garman, M. B., and Klass, M. J. (1980). “On the Estimation of Security Price Volatility from Historical Data,” Journal of Business, 53, 67-78.
Geweke, J., and Porter-Hudak, S. (1983). “The Estimation and Application of Long Memory Time Series Models,” Journal of Time Series
Analysis, 4, 221-237.
Granger, C. W. J., and Joyeux, R. (1980). “An Introduction to LongMemory Time Series Models and Fractional Diﬀerencing,” Journal of Time
Series Analysis, 1, 15-29.
Hamilton, J. D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
Haslett, J., and Raftery, A. E. (1989). “Space-time Modelling with
Long-Memory Dependence: Assessing Ireland’s Wind Power Resource,”
Journal of Royal Statistical Society Series C, 38, 1-21.
Hosking, J. R. M. (1981). “Fractional Diﬀerencing,” Biometrika, 68, 165176.

8.8 References

311

Hurst, H. E. (1951). “Long Term Storage Capacity of Reservoirs,” Transactions of the American Society of Civil Engineers, 116, 770-799.
Lo, A. W. (1991). “Long Term Memory in Stock Market Prices,” Econometrica, 59, 1279-1313.
Lobato, I. N., and Savin, N. E. (1998). “Real and Spurious LongMemory Properties of Stock-Market Data,” Journal of Business and Economic Statistics, 16 (3), 261-268.
Mandelbrot, B. B. (1975). “Limit Theorems on the Self-Normalized
Range for Weakly and Strongly Dependent Processes,” Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 31, 271-285.
Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P. (1992). Numerical Recipes in C: The Art of Scientific Computing, Cambridge University Press, Cambridge.
Ray, B. K., and Tsay, R. S. (2000). “Long-Range Dependence in Daily
Stock Volatilities,” Journal of Business and Economic Statistics, 18, 254262.
Sowell, F. (1992). “Maximum Likelihood Estimation of Stationary Univariate Fractionally Integrated Time Series Models,” Journal of Econometrics, 53, 165-188.
Taqqu, M. S., and Teverovsky, V. (1998). “On Estimating the Intensity of Long-Range Dependence in Finite and Infinite Variance Time
Series”, in R. J. Adler, R. E. Feldman and M. S. Taqqu (eds.), A Practical
Guide to Heavy Tails: Statistical Techniques and Applications. Birkhaüser,
Boston.
Taqqu, M. S., Teverovsky, V., Willinger, W. (1995). “Estimators
for Long Range Dependence: An Empirical Study,” Fractals, 3(4), 785-798.

This is page 312
Printer: Opaque thisThis is page 313
Printer: Opaque this9
Rolling Analysis of Time Series

9.1 Introduction
A rolling analysis of a time series model is often used to assess the model’s
stability over time. When analyzing financial time series data using a statistical model, a key assumption is that the parameters of the model areconstant over time. However, the economic environment often changes considerably, and it may not be reasonable to assume that a model’s parameters are constant. A common technique to assess the constancy of a model’s
parameters is to compute parameter estimates over a rolling window of a
fixed size through the sample. If the parameters are truly constant over theentire sample, then the estimates over the rolling windows should not be
too diﬀerent. If the parameters change at some point during the sample,
then the rolling estimates should capture this instability.
Rolling analysis is commonly used to backtest a statistical model on
historical data to evaluate stability and predictive accuracy. Backtesting
generally works in the following way. The historical data is initially split
into an estimation sample and a prediction sample. The model is then fit
using the estimation sample and h-step ahead predictions are made forthe prediction sample. Since the data for which the predictions are madeare observed h-step ahead prediction errors can be formed. The estimation
sample is then rolled ahead a given increment and the estimation and prediction exercise is repeated until it is not possible to make any more h-step
predictions. The statistical properties of the collection of h-step ahead pre-

314

9. Rolling Analysis of Time Series

diction errors are then summarized and used to evaluate the adequacy ofthe statistical model.
Moving average methods are common in rolling analysis, and these methods lie at the heart of the technical analysis of financial time series. Moving
averages typically use either equal weights for the observations or exponentially declining weights. One way to think of these simple moving averagemodels is that they are a “poor man’s” time varying parameter model.
Sometimes simple moving average models are not adequate, however, anda general time varying parameter model is required. In these cases, thestate space models discussed in Chapter 14 should be used.
This chapter describes various types of rolling analysis of financial timeseries using S-PLUS. Section 9.2 covers rolling descriptive statistics forunivariate and bivariate time series with an emphasis on moving averagetechniques, and Section 9.3 gives a brief review of technical analysis using
S+FinMetrics functions. Section 9.4 discusses rolling regression using the
S+FinMetrics function rollOLS and illustrates how rollOLS may be used
for backtesting regression models. Section 9.5 describes rolling analysis ofgeneral models using the S+FinMetrics function roll.
Rolling analysis of financial time series is widely used in practice but
the technique is seldom discussed in textbook treatments of time series
analysis. Notable exceptions are Alexander (2001) and Dacorogna et. al.
(2001). Rolling analysis techniques in finance are generally discussed in
the technical analysis literature, but the statistical properties of backtesting technical analysis are rarely addressed. A comprehensive treatment oftechnical analysis indicators is given in Colby and Meyers (1988) and acritical evaluation of technical analysis is provided in Bauer and Dahlquist(1999). The econometric literature on evaluating the predictive accuracy ofmodels through backtesting has matured over the last decade. The mainreference is Diebold and Mariano (1995).

9.2 Rolling Descriptive Statistics
9.2.1 Univariate Statistics
Consider the analysis of a univariate time series yt over a sample fromt = 1, . . . , T . Whether the mean and variance (or standard deviation) parameters of the distribution of yt are constant over the entire sample isof interest. To assess parameter constancy, let n denote the width of a
sub-sample or window and define the rolling sample means, variances and9.2 Rolling Descriptive Statistics

315

standard deviationsn−1

μ̂t (n) =

1X
yt−i
n i=0

n−1
1 X
(yt−i − μ̂t (n))2n − 1 i=0q
σ̂ 2t (n)σ̂ t (n) =σ̂ 2t (n) =

(9.1)

(9.2)
(9.3)

for windows t = n, . . . T . The rolling mean and variance estimates at time twith window width n are the usual sample estimates using the most recent
n observations. Provided the windows are rolled through the sample one
observation at a time, there will be T − n + 1 rolling estimates of eachparameter. The rolling mean μ̂t (n) is sometime called a n-period simple
moving average.
Computing Rolling Descriptive Statistics Using the S-PLUS Function
aggregateSeries
Consider the monthly continuously compounded returns on Microsoft stockover the period February 1990 through January 2001 computed from themonthly closing prices in the S+FinMetrics “timeSeries” singleIndex.dat> msft.ret = getReturns(singleIndex.dat[,"MSFT"])> start(msft.ret)[1] Feb 1990
> end(msft.ret)[1] Jan 2001
> nrow(msft.ret)[1] 132
24-month rolling mean and standard deviations may be computed easily
using the S-PLUS function aggregateSeries1> roll.mean = aggregateSeries(msft.ret,moving=24,adj=1,FUN=mean)> roll.sd = aggregateSeries(msft.ret,moving=24,adj=1,FUN=stdev)
The arguments moving=24, adj=1 and FUN=mean(stdev) tell the S-PLUS
function aggregateSeries to evaluate the mean (stdev) function on arolling window of size 24 and to adjust the output positions to the end ofeach window. roll.mean and roll.sd are “timeSeries” objects containing 109 rolling estimates:> class(roll.mean)
1 aggregateSeries is the method function of the generic S-PLUS function aggregate
for objects of class “timeSeries” and “signalSeries”.

9. Rolling Analysis of Time Series

-0.2

-0.1

0.0

0.1

0.2

0.3

316

-0.4

-0.3

Returns
Rolling mean
Rolling sd

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 9.1. Monthly returns on Microsoft stock along with 24 month rollingmeans and standard deviations.

[1] "timeSeries"> nrow(roll.mean)[1] 109> roll.mean[1:5]
Positions1
Jan 1992 0.05671
Feb 1992 0.05509
Mar 1992 0.04859
Apr 1992 0.04366
May 1992 0.03795
The monthly returns along with the rolling means and standard deviations may be plotted together using> plot(msft.ret,roll.mean,roll.sd,plot.args=list(lty=c(1,3,4)))> legend(0,-0.2,legend=c("Returns","Rolling mean","Rolling sd"),+ lty=c(1,3,4))
which is illustrated in Figure 9.1. The 24 month rolling estimates μ̂t (24)and σ̂ t (24) clearly vary over the sample. The rolling means start out around
5%, fall close to 0% in 1994, rise again to about 5% until 2000 and thenfall below 0%. The rolling σ̂ t (24) values start out around 10%, fall slightly
until 1997 and then begin to steadily rise for the rest of the sample. The9.2 Rolling Descriptive Statistics

317

end of sample value of σ̂ t (24) is almost twice as big as the beginning ofsample value.
The moving average estimates (9.1) - (9.3) are one-sided backward looking estimates. The S-PLUS function aggregateSeries may also compute
two-sided asymmetric moving averages by specifying a value between 0
and 1 for the optional argument adj. For example, to compute a 24 monthsymmetric two-sided simple moving average set adj=0.5> roll.mean.5 = aggregateSeries(msft.ret,moving=24,adj=0.5,+ FUN=mean)> roll.mean.5[1:5]
Positions
MSFT
Feb 1991 0.056708
Mar 1991 0.055095
Apr 1991 0.048594
May 1991 0.043658
Jun 1991 0.037950
Instead of computing the rolling means and standard deviations in separate calls to aggregateSeries, they can be computed in a single call by
supplying a user-written function to aggregateSeries that simply returns
the mean and standard deviation. One such function is> mean.sd = function (x) {>tmp1 = mean(x)>tmp2 = stdev(x)>ans = concat(tmp1,tmp2)
>
ans
> }
The call to aggregateSeries to compute the rolling means and standard
deviations is then> roll.mean.sd = aggregateSeries(msft.ret,moving=24,adj=1,+ FUN=mean.sd,colnames=c("mean","sd"))> roll.mean.sd[1:5,]
Positionsmean
sd
Jan 1992 0.05671 0.09122
Feb 1992 0.05509 0.09140
Mar 1992 0.04859 0.09252
Apr 1992 0.04366 0.09575
May 1992 0.03795 0.08792
Notice that the column names of roll.mean.sd are specified using optional
argument colnames=c("mean","sd") in the call to aggregateSeries.

318

9. Rolling Analysis of Time Series

Standard error bands around the rolling estimates of μ and σ may becomputed using the asymptotic formulasσ̂ t (n)t (n) cc t (n)) = σ̂√SE(μ̂, SE(σ̂ t (n)) = √n2nUsing the rolling estimates in roll.mean.sd, the S-PLUS commands tocompute and plot the rolling estimates along with approximate 95% confidence bands are> lower.mean = roll.mean.sd[,"mean"]+ 2*roll.mean.sd[,"sd"]/sqrt(24)> upper.mean = roll.mean.sd[,"mean"]++ 2*roll.mean.sd[,"sd"]/sqrt(24)> lower.sd = roll.mean.sd[,"sd"]+ 2*roll.mean.sd[,"sd"]/sqrt(2*24)> upper.sd = roll.mean.sd[,"sd"]++ 2*roll.mean.sd[,"sd"]/sqrt(2*24)> par(mfrow=c(2,1))> plot(roll.mean.sd[,"mean"],lower.mean,upper.mean,+ main="24 month rolling means",plot.args=list(lty=c(1,2,2)))> plot(roll.mean.sd[,"sd"],lower.sd,upper.sd,+ main="24 month rolling standard deviations",+ plot.args=list(lty=c(1,2,2)))Figure 9.2 shows the results. In general, the rolling σ̂t (24) values are estimated much more precisely than the rolling μ̂t (24) values.
The rolling means, variances and standard deviations are not the onlyrolling descriptive statistics of interest, particularly for asset returns. For
risk management purposes, one may be interested in extreme values. Therefore, one may want to compute rolling minima and maxima. These may becomputed using aggregateSeries with FUN=min and FUN=max.
Computing Rolling Means, Variances, Maxima and Minima Using the
S+FinMetrics Functions SMA, rollVar, rollMax and rollMin
The S-PLUS function aggregateSeries is extremely flexible but not efficient for computing rolling means, variances, maxima and minima. The
S+FinMetrics functions SMA (simple moving average), rollVar, rollMax
and rollMin implement eﬃcient algorithms for computing rolling means,variances, maxima and minima. The arguments expected by these functionsare> args(SMA)
function(x, n = 9, trim = T, na.rm = F)> args(rollVar)function(x, n = 9, trim = T, unbiased = T, na.rm = F)9.2 Rolling Descriptive Statistics

319

-0.08

0.00

0.08

24 month rolling means

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

1999

2000

2001

0.06

0.14

0.22

24 month rolling standard deviations

1992

1993

1994

1995

1996

1997

1998

FIGURE 9.2. 24 month rolling estimates of μ̂t (24) and σ̂t (24) for Microsoft with
95% confidence bands.

> args(rollMax)
function(x, n = 9, trim = T, na.rm = F)> args(rollMin)function(x, n = 9, trim = T, na.rm = F)
where x is a vector or univariate “timeSeries”, n is the window width,
trim determines if start-up values are trimmed from the output series andna.rm determines if missing values are to be removed. For rollVar, the1
option unbiased=T computes the unbiased variance estimator using n−1
1
as a divisor and unbiased=F computes the biased estimator using n .
To illustrate the use of SMA, rollVar, rollMax and rollMin 24 monthrolling means, standard deviations, maxima and minima from the monthly
returns on Microsoft are computed as> roll2.mean = SMA(msft.ret,n=24)> roll2.sd = sqrt(rollVar(msft.ret,n=24))> roll.max = rollMax(msft.ret,n=24)> roll.min = rollMin(msft.ret,n=24)
These estimates are identical to those computed using aggregateSeries,
but the computation time required is much less. To compare the compu-

320

9. Rolling Analysis of Time Series

tation times in seconds within S-PLUS 6 for Windows the S-PLUS function
dos.time may be used2> dos.time(SMA(msft.ret,n=24))[1] 0.05> dos.time(aggregateSeries(msft.ret,moving=24,adj=1,FUN=mean))[1] 4.23> dos.time(sqrt(rollVar(msft.ret,n=24)))[1] 0.06> dos.time(aggregateSeries(msft.ret,moving=24,adj=1,FUN=stdev))[1] 6.76
Example 49 Computing rolling standard deviations from high frequency
returns
Rolling estimates of σ 2 and σ based on high frequency continuously compounded return data are often computed assuming the mean return is zero1σ̂ 2t (n) =nXr2
n i=1 t−i

In this case the rolling estimates of σ 2 may be computed using the computationally eﬃcient S+FinMetrics function SMA. For example, consider computing rolling estimates of σ based on the daily continuously compounded
returns for Microsoft over the 10 year period from January 1991 throughJanuary 2001. The squared return data is computed from the daily closing
price data in the S+FinMetrics “timeSeries” object DowJones30> msft.ret2.d = getReturns(DowJones30[,"MSFT"],+ type="continuous")^2
Rolling estimates of σ based on 25, 50 and 100 day windows are computed
using SMA
> roll.sd.25 = sqrt(SMA(msft.ret2.d,n=25))> roll.sd.50 = sqrt(SMA(msft.ret2.d,n=50))> roll.sd.100 = sqrt(SMA(msft.ret2.d,n=100))
The rolling estimates σ̂ t (n) are illustrated in Figure 9.3 created using> plot(roll.sd.25,roll.sd.50,roll.sd.100,+ plot.args=(list(lty=c(1,3,4))))> legend(0,0.055,legend=c("n=25","n=50","n=100"),+ lty=c(1,3,4))
2 The computations are carried out using S-PLUS 6 Professional Release 2 on a Dell
Inspiron 3500 400MHz Pentium II with 96MB RAM.

0.055

9.2 Rolling Descriptive Statistics

321

0.010

0.015

0.020

0.025

0.030

0.035

0.040

0.045

0.050

n=25
n=50
n=100

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

FIGURE 9.3. 25, 50 and 100 day rolling estimates of σ for the daily returns on
Microsoft stock.

There is considerable variation in the rolling estimates of daily σ, andthere appears to be a seasonal pattern with higher volatility in the summer
months and lower volatility in the winter months.

9.2.2 Bivariate Statistics
Consider now the analysis of two univariate time series y1t and y2t overthe sample from t = 1, . . . , T . To assess if the covariance and correlation
between y1t and y2t is constant over the entire sample the n-period rollingsample covariances and correlationsσ̂ 12,t (n) =ρ̂12,t (n) =n−1
1 X(y1t−i − μ̂1t (n))(y2t−i − μ̂2t (n))
n − 1 i=0σ̂ 12,t (n)σ̂ 1t (n)σ̂ 2t (n)may be computed.
Example 50 24 month rolling correlations between the returns on Microsoft and the S&P 500 index
Consider the monthly continuously compounded returns on Microsoft
stock and S&P 500 index over the period February 1990 through Jan-

322

9. Rolling Analysis of Time Seriesuary 2001 computed from the monthly closing prices in the S+FinMetrics“timeSeries” object singleIndex.dat> ret.ts = getReturns(singleIndex.dat,type="continuous")> colIds(ret.ts)[1] "MSFT" "SP500"
The 24-month rolling correlations between the returns on Microsoft and
S&P 500 index may be computed using the S-PLUS function aggregateSeries
with a user specified function to compute the correlations. One such function is> cor.coef = function(x) cor(x)[1,2]The 24-month rolling correlations are then computed as> smpl = positions(ret.ts)>=start(roll.cor)> roll.cor = aggregateSeries(ret.ts,moving=24,together=T,+ adj=1,FUN=cor.coef)> roll.cor[1:5]
Positions1
Jan 1992 0.6549
Feb 1992 0.6535
Mar 1992 0.6595
Apr 1992 0.6209
May 1992 0.5479
In the call to aggregateSeries the argument together=T passes all of
the columns of ret.ts to the function cor.coef instead of passing eachcolumn separately. The monthly returns on Microsoft and the S&P 500
index along with the rolling correlations are illustrated in Figure 9.4 whichis created by> par(mfrow=c(2,1))> plot(ret.ts[smpl,],main="Returns on Microsoft and+ S&P 500 index",+ plot.args=list(lty=c(1,3)))> legend(0,-0.2,legend=c("Microsoft","S&P 500"),+ lty=c(1,3))> plot(roll.cor,main="24-month rolling correlations")
At the beginning of the sample, the correlation between Microsoft and the
S&P 500 is fairly high at 0.6. The rolling correlation declines steadily, hits

9.2 Rolling Descriptive Statistics

323

-0.2

0.0

0.2

Returns on Microsoft and S&P 500 index-0.4

Microsoft
S&P 500

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

1998

1999

2000

2001

0.1

0.2

0.3

0.4

0.5

0.6

24-month rolling correlations

1992

1993

1994

1995

1996

1997

FIGURE 9.4. Returns on Microsoft and the S&P 500 index along with 24-monthrolling correlations.

a low of about 0.1 at the beginning of 1997, then increases quickly to 0.6
and stabilizes at this value through the end of the sample3 .

9.2.3 Exponentially Weighted Moving Averages
The rolling descriptive statistics described in the previous sections are basedon equally weighted moving averages of an observed time series yt . Equallyweighted moving averages are useful for uncovering periods of instability
but may produce misleading results if used for short-term forecasting. Thisis because equally weighted averages are sensitive (not robust) to extreme
values. To illustrate, consider T = 100 observations from a simulated timeseries yt ∼ GW N (0, 1) with an outlier inserted at t = 20: i.e., y20 = 10.
The data and rolling values μ̂t (10) and σ̂ t (10) are illustrated in Figure 9.5.
Notice how the outlier at t = 20 inflates the rolling estimates μ̂t (10) andσ̂ t (10) for 9 periods.
3 Approximate standard errors for the rolling correlations may be computed usingdSE(ρ̂t (n)) =s

1 − ρ̂t (n)2n

9. Rolling Analysis of Time Series

0

2

4

6

8

10

324

20

40

60

80

100

3.0

0

0.5

1.0

1.5

2.0

2.5

Rolling mean
Rolling sd

0

20

40

60

80

100

FIGURE 9.5. Eﬀect of an outlier on equally weighted rolling estimates of μ andσ.

To mitigate the eﬀects of extreme observations on moving average estimates the observations in a rolling window can be weighted diﬀerently.
A common weighting scheme that puts more weight on the most recent
observations is based on exponentially declining weights and the resulting
weighted moving average is called an exponentially weighted moving average(EWMA). An n-period EWMA of a time series yt is defined asμ̃t (n) =

n−1
X

λi−1
wi · yt−i , wi = Pn−1 i−1
i=0 λ
i=0

where 0 < λ < 1 is the decay parameter. As n → ∞, λn → 0, wn → 0, andthe EWMA converges toμ̃t (λ) = (1 − λ)∞Xλi yt−i

(9.4)

i=0

so the EWMA may be defined independently of the window width n. The
EWMA in (9.4) may be eﬃciently computed using the recursionμ̃t (λ) = (1 − λ)yt + λμ̃t−1 (λ)(9.5)

From (9.5), it is clear that the closer λ is to one the more weight is put on thethe previous period’s estimate relative to the current period’s observation.

9.2 Rolling Descriptive Statistics

325

Therefore, λ may be interpreted as a persistence parameter. The recursive
formula (9.5) requires a starting value μ0 (λ). Common choices are the firstobservation and the average over a local window.
EWMA estimates of descriptive statistics for continuously compounded
asset returns are usually computed using high frequency data with theassumption that the mean returns are zero. Accordingly, the EWMA estimates of σ 2 and σ 12 areσ̃ 2t (λ) = (1 − λ)rt2 + λσ̃ 2t−1 (λ)σ̃ 12,t (λ) = (1 − λ)r1t r2t + λσ̃ 12,t−1 (λ)(9.6)

where rt denotes the continuously compounded return on an asset. The
EWMA estimate of volatility (9.6) is in the form of a IGARCH(1,1) model
without an constant term.
Computing EWMA Estimates Using the S+FinMetrics Function EWMA
EWMA estimates based on (9.5) may be eﬃciently computed using the
S+FinMetrics function EWMA. The arguments expected by EWMA are> args(EWMA)
function(x, n = 9, lambda = (n - 1)/(n + 1), start ="average", na.rm = F)
where x is the data input, n is a window width, lambda is the decay parameter and start specifies the starting value for the recursion (9.5). The
implied default value for λ is 0.8. Valid choices for start are "average"and "first". The use of EWMA is illustrated with the following examples.
Example 51 Outlier example
Consider again the outlier example data shown in Figure 9.5. EWMA
estimates of μ for λ = 0.95, 0.75 and 0.5 are computed and plotted in
Figure 9.6 using> ewma95.mean = EWMA(e,lambda=0.95)> ewma75.mean = EWMA(e,lambda=0.75)> ewma50.mean = EWMA(e,lambda=0.5)> tsplot(ewma95.mean,ewma75.mean,ewma50.mean)> legend(60,4,legend=c("lamda=0.95","lamda=0.75",+ "lamda=0.50"),lty=1:3)
Notice that the EWMA estimates with λ = 0.95, which put the most weighton recent observations, are only minimally aﬀected by the one-time outlier
whereas the EWMA estimates with λ = 0.75 and 0.5 increase sharply at
the date of the outlier.
Example 52 EWMA estimates of standard deviations and correlations
from high frequency data

9. Rolling Analysis of Time Series

4

326

-1

0

1

2

3

lamda=0.95
lamda=0.75
lamda=0.50

0

20

40

60

80

100

FIGURE 9.6. EWMA estimates of μ for outlier example data.

EWMA estimates of asset return standard deviations computed fromhigh frequency data are commonly used as local or short-term estimates ofvolatility. Similarly, EWMA estimates of pairwise return correlations areoften used to infer local interactions between assets. Indeed, J.P. Morgan’sr methodology is based on EWMA estimates of volatility and
RiskMetrics°
correlation. To illustrate, consider computing EWMA estimates of volatility
and correlation with λ = 0.95 using daily closing price data on Microsoft
and IBM stock over the five year period 1996-2000:> smpl = (positions(DowJones30) >= timeDate("1/1/1996"))> msft.ret.d = getReturns(DowJones30[smpl,"MSFT"])> ibm.ret.d = getReturns(DowJones30[smpl,"IBM"])> msft.ewma95.sd = sqrt(EWMA(msft.ret.d^2,lambda=0.95))> ibm.ewma95.sd = sqrt(EWMA(ibm.ret.d^2,lambda=0.95))> cov.ewma95 = EWMA(msft.ret.d*ibm.ret.d,lambda=0.95)> cor.ewma95 = cov.ewma95/(msft.ewma95.sd*ibm.ewma95.sd)> par(mfrow=c(2,1))> plot(msft.ewma95.sd,ibm.ewma95.sd,+ main="Rolling EWMA SD values",+ plot.args=list(lty=c(1,3)))> legend(0,0.055,legend=c("Microsoft","IBM"),lty=c(1,3))> plot(cor.ewma95,main="Rolling EWMA correlation values")
Figure 9.7 shows the EWMA estimates of volatility and correlation. Daily

9.2 Rolling Descriptive Statistics

327

Microsoft
IBM

0.015

0.035

0.055

Rolling EWMA SD values

Q1 Q2 Q3
1996

Q4

Q1 Q2 Q3
1997

Q4

Q1 Q2 Q3 Q4
1998

Q1 Q2 Q3
1999

Q4 Q1 Q2 Q3 Q4 Q1
2000
2001

-0.2

0.0

0.2

0.4

0.6

0.8

Rolling EWMA correlation values

Q1 Q2 Q3
1996

Q4

Q1 Q2 Q3
1997

Q4

Q1 Q2 Q3 Q4
1998

Q1 Q2 Q3
1999

Q4 Q1 Q2 Q3 Q4 Q1
2000
2001

FIGURE 9.7. EWMA estimates of daily volatility and correlation with λ = 0.95.

volatility for Microsoft and IBM varies considerably, exhibiting apparent
seasonality and an increasing trend. The daily correlations fluctuate around
0.5 for the first part of the sample and then drop to about 0.1 at the endof the sample.

9.2.4 Moving Average Methods for Irregularly Spaced HighFrequency Data
The use of moving average and rolling methods on irregularly spaced or
inhomogeneous high frequency time series data requires care. The moving
average tools discussed so far are designed to work on regularly spacedor homogeneous time series data. Two approaches have been used to apply
moving average methods to irregularly spaced data. The first approach converts the irregularly spaced data to regularly spaced data and then applies
the tools for appropriate for regularly spaced data. The second approach,pioneered by Zumbach and Müller (2001), utilizes moving average methods
specifically designed for irregularly spaced data.
Converting Inhomogeneous Time Series to Homogeneous Time Series
To illustrate the conversion of a inhomogeneous time series to a homogeneous time series, consider the transactions level data on 3M corporation

328

9. Rolling Analysis of Time Series

stock for December 1999 in the S+FinMetrics data frame highFreq3M.df.
As in Chapter 2, a “timeSeries” object may be created using> td = timeDate(julian=(highFreq3M.df$trade.day-1),+ ms=highFreq3M.df$trade.time*1000,+ in.origin=c(month=12,day=1,year=1999),zone="GMT")> hf3M.ts = timeSeries(pos=td,data=highFreq3M.df)> hf3M.ts[1:20,]
Positions trade.day trade.time trade.price
12/1/99 9:33:32 AM 1
34412
94.69
12/1/99 9:33:34 AM 1
34414
94.69
12/1/99 9:33:34 AM 1
34414
94.69
...
12/1/99 9:34:45 AM 1
34485
94.69
12/1/99 9:34:47 AM 1
34487
94.63
...
The trade time is measured in second from midnight. Notice that many of
the first trades took place at the same price and that there are instances ofmultiple transactions at the same time. The analysis is limited to the first
three trading days of December> smpl = positions(hf3M.ts) < timeDate("12/4/1999")> hf3M.ts = hf3M.ts[smpl,]
The data in hf3M.ts may be made homogeneous by use of an interpolation method to align the irregularly spaced time sequence and associateddata to a regularly spaced time sequence. For example, consider creatinga homogeneous time series of five minute observations. Since the data in
hf3M.ts may not be recorded at all five minute intervals, some interpolation scheme must be used to create the data. Two common interpolation
schemes are: previous tick interpolation, and linear interpolation. The former method uses the most recent values, and the latter method uses observations bracketing the desired time. The S-PLUS functions align may
be used to perform these interpolation schemes.
The function align takes a “timeSeries” and a “timeDate” vector of
new positions to align to. An easy way to create a “timeDate” sequence offive minute observations covering the trading hours for 3M stock is to use
the S-PLUS function aggregateSeries as follows:> tmp = aggregateSeries(hf3M.ts,by="minutes",k.by=5,FUN=mean)
The positions slot of “timeSeries” tmp contains the desired five minute“timeDate” sequence:> positions(tmp[1:4])[1] 12/1/99 9:30:00 AM[3] 12/1/99 9:40:00 AM

12/1/99 9:35:00 AM
12/1/99 9:45:00 AM

9.2 Rolling Descriptive Statistics

329

To align the 3M price data to the five minute time sequence using previous
tick interpolation use> hf3M.5min = align(hf3M.ts[,"trade.price"], positions(tmp),+ how="before")> hf3M.5min[1:5,]
Positions trade.price
12/1/99 9:30:00 AM
NA
12/1/99 9:35:00 AM 94.63
12/1/99 9:40:00 AM 94.75
12/1/99 9:45:00 AM 94.50
12/1/99 9:50:00 AM 94.31
To align the price data using linear interpolation use> hf3M.5min = align(hf3M.ts[,"trade.price"], positions(tmp),+ how="interp")> hf3M.5min[1:5,]
Positions trade.price
12/1/99 9:30:00 AM
NA
12/1/99 9:35:00 AM 94.65
12/1/99 9:40:00 AM 94.75
12/1/99 9:45:00 AM 94.42
12/1/99 9:50:00 AM 94.26
The usual methods for the analysis of homogeneous data may now be
performed on the newly created data. For example, to compute and plot
an EWMA of price with λ = 0.9 use> hf3M.5min.ewma = EWMA(hf3M.5min,lambda=0.9,na.rm=T)> plot(hf3M.5min.ewma)
The resulting plot is shown in Figure 9.8.
Inhomogeneous Moving Average Operators
Zumbach and Müller (2001) presented a general framework for analyzing inhomogeneous time series. A detailed exposition of this framework is beyond
the scope of this book. Only a brief description of the most fundamental
inhomogeneous time series operators is presented and reader is referred to
Zumbach and Müller (2001) or Dacorogna et. al. (2001) for technical details
and further examples.
Zumbach and Müller (2001) distinguished between microscopic and macroscopic operations on inhomogeneous time series. A microscopic operation
depends on the actual sampling times of the time series, whereas a macroscopic operator extracts an average over a specified range. Macroscopic
operations on high frequency inhomogeneous time series are advantageous
because they are essentially immune to small variations in the individual

9. Rolling Analysis of Time Series

94

95

96

97

98

99

100

330

10:00

12:00
2:00
Dec 1 1999

4:00

10:00

12:00
2:00
Dec 2 1999

4:00

10:00

12:00
2:00
Dec 3 1999

4:00

FIGURE 9.8. EWMA of five minute prices on 3M stock.

data observations and are better behaved and more robust than microscopic
operations. The S+FinMetrics functions for analyzing inhomogeneous timeseries are based on a subset of the macroscopic operators discussed in Zumbach and Müller (2001). These functions are summarized in Table 9.1.
In general, given a continuous time signal z(t), a macroscopic operatorΩ can be defined as a convolution with a causal kernel ω(·):
Z tΩ(t) =
w(t − s)z(s)ds(9.7)
0

for t > 0. Note that for a causal kernel ω(t) = 0 for any t < 0, since future
information cannot be utilized. In addition, it is usually required thatZ ∞ω(t)dt = 1
0

so that the operator can be interpreted as a weighted moving average of thesignal z(t). For example, the exponential moving average (EMA) operator
is defined with an exponential kernel:
ω(t) =e−t/ττ

and it is easy to verify that
Z ∞
0

e−t/τ
=1
τ

(9.8)

9.2 Rolling Descriptive Statistics

Function
iEMA
iMA
iMNormiMVar
iMSD
iMSkewness
iMKurtosis
iMCor
iDiff
iEMA.kernel
iMA.kernel

331

Description
Inhomogeneous EWMA
Inhomogeneous moving average
Inhomogeneous moving norm
Inhomogeneous moving variance
Inhomogeneous moving SD
Inhomogeneous moving skewness
Inhomogeneous moving kurtosis
Inhomogeneous moving correlation
Inhomogeneous moving diﬀerence
Kernel function for iEMA
Kernel function for iMA

TABLE 9.1. S+FinMetrics inhomogeneous time series function

The parameter τ can be shown to be the range of the EMA kernel.4
In reality, a time series signal is usually observed at discrete times. In
addition, financial transactions level data are usually observed on irregularintervals. For the EMA operator, Zumbach and Müller suggest to use thefollowing iterative formula to compute a discrete time approximation to(9.7):5
EMA(tn ; τ ) = μEMA(tn−1 ; τ )+(1−μ)z(tn )+(μ−ν)[z(tn )−z(tn−1 )] (9.9)
where
and

μ = e−α , ν = (1 − μ)/αα = (tn − tn−1 ).

Note that when α is very small, eα ≈ 1 + α and it can be shown thatμ ≈ ν. In this case, the above formula reduces to the same iteration for
evenly spaced EWMA operator.
Using the basic EMA operator, diﬀerent operators can be constructed.
For example, Zumbach and Müller suggested that the basic EMA operator
can be iterated a finite number of times to obtain an operator with a diﬀerent kernel, denoted EMA(τ , k). The EMA(τ , k) operator can be summed toobtain the analog of the moving average (MA) operator for inhomogeneoustime series:k
1X
MA(τ , k) =EMA(s, i)k i=1
where s = 2τ /(k + 1) so that the range of MA(τ , k) is equal to τ , independent of k.

R∞
0 ω(t)tdt.
5 This formula is actually obtained by assuming linear interpolation between points.
4 The range is defined as the first moment of the kernel, i.e.,

If previous tick interpolation is used, then ν = 1.

332

9. Rolling Analysis of Time Series

The S+FinMetrics functions iEMA.kernel and iMA.kernel can be used
to plot the kernel functions for EMA and MA operators, while iEMA and iMA
can be used to compute the EMA and MA operator for inhomogeneous timeseries. For example, the following code plots the EMA(τ , k) and MA(τ , k)kernel functions for τ = 1 and k = 1, 2, · · · , 10:> par(mfrow=c(2,1))> knl = iEMA.kernel(1, 1)> plot(knl, type="l", main="EMA Kernel")> for(i in 2:10) {>knl = iEMA.kernel(1, i)>lines(knl, lty=i)> }> knl = iMA.kernel(1, 1)> plot(knl, type="l", main="MA Kernel")> for(i in 2:10) {>knl = iMA.kernel(1, i)>lines(knl, lty=i)> }> par(mfrow=c(1,1))
and the resulting plot is shown in Figure 9.9. From the figure, it can beseen that when k = 1, EMA(τ , k) and MA(τ , 1) are equivalent by definition.
However, as k gets larger, the kernel function of EMA(τ , k) becomes flatter,
while the kernel function of MA(τ , 1) becomes more like a rectangle. In fact,
Zumbach and Müller showed that the range of EMA(τ , k) is kτ , while therange of MA(τ , 1) becomes a constant for t ≤ 2τ as k → ∞. As a result, toobtain an MA operator with window width equal to 9 (which corresponds
to a range of 8, i.e., using 8 observations in the past), one sets τ = 4 andk to a large number:> iMA(1:100, 4, iter=10)[1] 1.000 1.084 1.305 1.662 2.150 2.761 3.481[8] 4.289 5.166 6.091 7.047 8.024 9.011 10.005[15] 11.002 12.001 13.000 14.000 15.000 16.000 17.000[22] 18.000 19.000 20.000 21.000 22.000 23.000 24.000[29] 25.000 26.000 27.000 28.000 29.000 30.000 31.000[36] 32.000 33.000 34.000 35.000 36.000 37.000 38.000[43] 39.000 40.000 41.000 42.000 43.000 44.000 45.000[50] 46.000 47.000 48.000 49.000 50.000 51.000 52.000[57] 53.000 54.000 55.000 56.000 57.000 58.000 59.000[64] 60.000 61.000 62.000 63.000 64.000 65.000 66.000[71] 67.000 68.000 69.000 70.000 71.000 72.000 73.000[78] 74.000 75.000 76.000 77.000 78.000 79.000 80.000[85] 81.000 82.000 83.000 84.000 85.000 86.000 87.000

9.2 Rolling Descriptive Statistics

333

0.6
0.4
0.0

0.2

knl$y

0.8

1.0

EMA Kernel

0

2

4

6

8

10

6

8

10

knl$x

0.6
0.4
0.0

0.2

knl$y

0.8

1.0

MA Kernel

0

2

4
knl$x

FIGURE 9.9. Kernel function for EMA and MA operators for inhomogeneoustime series.

[92] 88.000 89.000 90.000 91.000 92.000 93.000 94.000[99] 95.000 96.000
The S+FinMetrics iMA function requires at least two arguments: thefirst is the input series, and the second specifies the value for τ . In theabove example, the optional argument iter is used to specify the numberof iterations k; in addition, since the input series is not a “timeSeries”
object, iMA treats it as evenly spaced and τ is in units of observations. If
the input series is a “timeSeries” object, then τ should be specified inunits of “business days”. To illustrate the usage of iMA in this case, firstcreate a “timeSeries” object representing the transaction price data from
hf3M.ts created earlier6 :> smpl2 = positions(hf3M.ts) < timeDate("12/02/1999")> hf3m.1min = aggregateSeries(hf3M.ts[smpl2,"trade.price"],+ by="minutes", FUN=mean)> hf3m.1min[103:110]
Positions trade.price
12/1/1999 11:28:00 AM 94.25000
6 The S-PLUS function aggregateSeries is used to eliminate multiple transactions
that occur at the same time. Currently, the S+FinMetrics inhomogeneous time series
functions do not work if there are multiple observations with the same time stamp.

9. Rolling Analysis of Time Series

93.8

94.0

94.2

94.4

94.6

94.8

95.0

95.2

334

9:30

10:00

11:00

12:00

1:00
Dec 1 1999

1:30

2:00

2:30

3:00

3:30

4:00

FIGURE 9.10. 20 minute moving average computed from iMA for 3M stock prices.

12/1/1999 11:30:00 AM 94.18750
12/1/1999 11:32:00 AM 94.25000
12/1/1999 11:33:00 AM 94.25000
12/1/1999 11:34:00 AM 94.21875
12/1/1999 11:36:00 AM 94.26563
12/1/1999 11:37:00 AM 94.18750
12/1/1999 11:39:00 AM 94.18750
Note that the data is not evenly spaced. To obtain a 20 minute moving
average of hf3m.1min, set τ = 10/(6.5 ∗ 60) because there are 6.5 hours forthe default trading hours (from 9:30 AM to 4:00 PM):> hf3m.ma = iMA(hf3m.1min, 10/(6.5*60), iter=10)> plot(seriesMerge(hf3m.1min, hf3m.ma),+ plot.args=list(lty=c(1,3)))The original data and the 20 minutes moving average hf3m.ma are plotted
together in Figure 9.10.

9.2.5 Rolling Analysis of Miscellaneous Functions
The standard analysis tools for time series require the data to be stationary.
Rolling analysis of descriptive statistics can give an indication of structuralchange or instability in the moments of a time series. Level shifts and9.2 Rolling Descriptive Statistics

335

0.06
0.02

0.03

0.04

0.05

D/P

0.07

0.08

0.09

0.10

S&P 500 Annual D/P

1870

1880

1890

1900

1910

1920

1930

1940

1950

1960

1970

1980

1990

2000

FIGURE 9.11. S & P 500 annual dividend/price ratio.

variance changes can usually be detected by rolling analyses. The S-PLUS
function aggregateSeries may be used to perform rolling analysis with avariety of functions to uncover periods of instability and nonstationarity.
The following example illustrates the use of aggregateSeries with the
S+FinMetrics function unitroot to determine periods of unitroot nonstationarity of a time series.
Example 53 Rolling unit root tests applied to annual dividend/price ratio
Predictive regressions of asset returns on valuation ratios like dividend/
price or earnings/price require the valuation ratios to be stationary or,
more generally, I(0), for the regressions to be statistically valid. Since asset
returns are I(0), if valuation ratios are I(1) then the predictive regressions
are unbalanced and the results will be nonsensical. To illustrate, consider
the annual dividend-price (D/P) ratio on S&P 500 index taken from the
S+FinMetrics “timeSeries” shiller.annual> dp.ratio = shiller.annual[,"dp.ratio"]> plot(dp.ratio,main="S&P 500 Annual D/P",ylab="D/P")
shown in Figure 9.11. For most of the sample the annual D/P looks tobe I(0) with mean near 5%. However, there are long periods when theratio stays above or below 5% suggesting periods of non-mean reverting(nonstationary) behavior. Also, there is a clear drop in the ratio at the336

9. Rolling Analysis of Time Series

end of the sample suggesting a fundamental change in the mean. Rolling
unit root tests may be used to uncover periods of nonstationary behavior
in D/P. To compute rolling ADF t-tests and normalized bias statistics
using the S-PLUS function aggregateSeries create the following function
adf.tests> adf.tests = function(x, trend = "c", lags = 3)
> {>tmp1 = unitroot(x,trend=trend,lags=lags,statistic="t")>tmp2 = unitroot(x,trend=trend,lags=lags,statistic="n")>ans = concat(tmp1$sval,tmp2$sval)
>
ans
> }
The function adf.tests takes a time series x, passes it to the S+FinMetrics
function unitroot twice and returns the ADF t-statistic and normalizedbias statistic. Three lags are chosen for the tests based on a full sample
analysis using the Ng-Perron backward selection procedure. Rolling unit
root tests using a window of 50 years are then computed using the function
aggregateSeries:> roll.adf = aggregateSeries(dp.ratio,moving=50,adj=1,+ FUN=adf.tests,colnames=c("t.test","norm.bias"))The object roll.adf is a “timeSeries” containing the rolling unit roottests
> roll.adf[1:3,]
Positions t.test norm.bias
Dec 1920 -1.840 -13.24
Dec 1921 -2.168 -15.24
Dec 1922 -2.270 -16.03
Figure 9.12 is created using> cvt.05 = qunitroot(0.05,trend="c",n.sample=50)> cvn.05 = qunitroot(0.05,trend="c",statistic="n",+ n.sample=50)> par(mfrow=c(2,1))> plot(roll.adf[,"t.test"], reference.grid=F,+ main="Rolling ADF t-statistics")> abline(h=cvt.05)> plot(roll.adf[,"norm.bias"], reference.grid=F,+ main="Rolling ADF normalized bias")> abline(h=cvn.05)
and shows the rolling unit root tests along with 5% critical values. The
results indicate that D/P is stationary mainly in the middle of the sample
and becomes nonstationary toward the end of the sample. However, some9.3 Technical Analysis Indicators

337

-3.5

-3.0

-2.5

-2.0

-1.5

Rolling ADF t-statistics

1920

1925

1930

1935

1940

1945

1950

1955

1960

1965

1970

1975

1980

1985

1990

1995

2000

1975

1980

1985

1990

1995

2000

-35

-30

-25

-20

-15

-10

-5

Rolling ADF normalized bias

1920

1925

1930

1935

1940

1945

1950

1955

1960

1965

1970

FIGURE 9.12. 50 year rolling ADF t-statistics and normalized bias statistics for
the S&P 500 dividend-price ratio.

care must be used when interpreting the significance of the rolling unit
root tests. The 5% critical values shown in the figures are appropriate for
evaluating a single test and not a sequence of rolling tests. Critical values
appropriate for rolling unit root tests are given in Banerjee, Lumsdaine andStock (1992).

9.3 Technical Analysis Indicators
Technical analysis is, perhaps, the most widely used method for analyzing
financial time series. Many of the most commonly used technical indicators
are based on moving average techniques so it is appropriate to include a
discussion of them here. A comprehensive survey of technical analysis is beyond the scope of this book. Useful references are Colby and Meyers (1988)
and Bauer and Dahlquist (1999). The S+FinMetrics technical analysis indicators are implemented using the definitions in Colby and Meyers (1988).
Broadly, the main technical indicators can be classified into four categories:
price indicators, momentum indicators and oscillators, volatility indicatorsand volume indicators.

338

9. Rolling Analysis of Time Series

Function
TA.Bollinger
TA.medpriceTA.typicalPriceTA.wclose

Description
Bollinger band
Median price
Typical price
Weighted close

TABLE 9.2. S+FinMetrics price indicators

9.3.1 Price Indicators
The S+FinMetrics price indicator functions are summarized in Table 9.2.
To illustrate the use of these functions, consider the calculation of thetypical daily price, which is defined to be the average of the highest, lowest and closing prices during the day, using the S+FinMetrics function
TA.typicalPrice. The arguments expected by TA.typicalPrice are> args(TA.typicalPrice)function(high, low, close)
In order to compute this indicator, a data set with high, low and close
prices is required. To compute the typical price for the Dow Jone Industrial
Average over the period January 1, 1990 to February 20, 1990 using the
S-PLUS “timeSeries” djia, use> smpl = positions(djia) >= timeDate("1/1/1990")> dj = djia[smpl,]> tp.dj = TA.typicalPrice(dj[,"high"],+ dj[,"low"],dj[,"close"])> class(tp.dj)[1] "timeSeries"
The typical price along with the high, low, open and close prices may beplotted together using> plot.out = plot(dj[,1:4],plot.type="hloc")> lines.render(positions(tp.dj),seriesData(tp.dj),+ x.scale=plot.out$scale)and the resulting plot is shown in Figure 9.13.

9.3.2 Momentum Indicators and Oscillators
The S+FinMetrics momentum indicator and oscillator functions are summarized in Table 9.3. For example, consider the popular moving average
convergence divergence (MACD) indicator. This is an oscillator that represents the diﬀerence between two exponential moving averages. A signal line is computed as the exponential moving average of MACD. Whenthe oscillator crosses above the signal line, it indicates a buy signal; when9.3 Technical Analysis Indicators

339

2550

2600

2650

2700

2750

2800

Dow Jones Industrial Average

Jan 8

Jan 15

Jan 22

Jan 291990

Feb 5

Feb 12

Feb 19FIGURE 9.13. Typical price along with high, low, open and close prices for the
Dow Jones Industrial Average.

the oscillator crosses below the signal line, it indicates a sell signal. The
S+FinMetrics function TA.macd computes the MACD and has arguments> args(TA.macd)function(x, n.short = 12, n.long = 26, n.signal = 9, start= "average", na.rm = F)
where x is a price series, n.short is a positive integer specifying the numberof periods to be used for calculating the short window EWMA, n.long
is a positive integer specifying the number of periods to be used for the
calculating the long window EWMA, and n.signal is a positive integer
FunctionTA.accelTA.momentum
TA.macdTA.rocTA.rsiTA.stochastic
TA.williamsr
TA.williamsad

Description
Acceleration
MomentumMoving average convergence divergence
Price rate of change
Relative strength index
Stochastic Oscillator
Williams’ %RWilliams’ accumulation distribution

TABLE 9.3. S+FinMetrics momentum indicators

9. Rolling Analysis of Time Series

-3

-2

-1

0

1

2

3

340

-5

-4

MACD
Signal

Oct

Nov
2000

Dec

Jan

Feb

Mar

Apr

May

Jun
2001

Jul

Aug

Sep

Oct

FIGURE 9.14. MACD and signal for daily closing prices on Microsoft stock.

FunctionTA.adoscillator
TA.chaikinv
TA.garmanKlass

Description
accumulation/distribution oscillator
Chaikin’s volatility
Garman-Klass estimator of volatility

TABLE 9.4. S+FinMetrics volatility indicator functions

giving the number of periods for the signal line. To compute and plot theMACD using daily closing prices on Microsoft use> msft.macd = TA.macd(msft.dat[,"Close"])> colIds(msft.macd) = c("MACD","Signal")> plot(msft.macd,plot.args=list(lty=c(1:3)))> legend(0.5,-3,legend=colIds(msft.macd),+ lty=c(1,3))
Figure 9.14 shows the plot of the MACD and signal.

9.3.3 Volatility Indicators
The S+FinMetrics volatility indicator functions are summarized in Table
9.4. These functions compute estimates of volatility based on high, low,
open and close information. For example, consider Chaikin’s volatility indicator computed using the S+FinMetrics function TA.chaikin. It com-

341

-40

-20

0

20

40

60

9.3 Technical Analysis Indicators

Nov
Dec
2000

Jan

Feb

Mar

Apr

May

Jun
2001

Jul

Aug

Sep

Oct

FIGURE 9.15. Chainkin’s volatility estimate using the daily prices on Microsoft
stock.

pares the spread between a security’s high and low prices and quantifies
volatility as a widening of the range between the high and low price. Let
ht and lt represent the highest and lowest price for period t, respectively.
Chaikin’s volatility is calculated as the percentage change in the EWMAof rt = ht − lt :rt − rt−nc
· 100
rt−nc
where nc is a positive number specifying the number of periods to use forcomputing the percentage change. To compute and plot Chaikin’s volatility
with nc = 10 and a ten day EWMA for the daily high and low price forMicrosoft stock use> msft.cv = TA.chaikinv(msft.dat[,"High"],+ msft.dat[,"Low"],n.range=10,n.change=10)> plot(msft.cv)
Figure 9.15 shows the estimated Chaikin volatility.

9.3.4 Volume Indicators
The S+FinMetrics volume indicator functions are summarized in Table
9.5. These indicators relate price movements with volume movements. To
illustrate, consider the S+FinMetrics function TA.adi which computes the342

9. Rolling Analysis of Time Series

FunctionTA.adiTA.chaikino
TA.nviTA.pviTA.obvTA.pvtrend

Description
Accumulation/distribution indicator
Chaikin oscillator
Negative volume index
Positive volume index
On balance volume
Price-volume trend

TABLE 9.5. S+FinMetrics volume indicator functions

accumulations/distribution (A/D) indicator. This indicator associates pricechanges with volume as follows. Let ct denote the closing price, ht thehighest price, lt the lowest price, and vt the trading volume for time t. The
A/D indicator is the cumulative sum

ADt =

t
X
ci − li − (hi − ci )i=1

hi − li· vi

When ADt moves up, it indicates that the security is being accumulated;when it moves down it indicates that the security is being distributed. To
compute and plot the A/D indicator for Microsoft stock use> msft.adi = TA.adi(msft.dat[,"High"],msft.dat[,"Low"],+ msft.dat[,"Close"],msft.dat[,"Volume"])> plot(msft.adi)
The resulting plot is shown in Figure 9.16.

9.4 Rolling Regression
For the linear regression model, rolling analysis may be used to assess thestability of the model’s parameters and to provide a simple “poor man’s”
time varying parameter model. For a window of width n < T , the rolling
linear regression model may be expressed asyt (n) = Xt (n)βt (n) + εt (n), t = n, . . . , T(9.10)

where yt (n) is an (n × 1) vector of observations on the response, Xt (n) isan (n × k) matrix of explanatory variables, βt (n) is an (k × 1) vector of
regression parameters and εt (n) is an (n × 1) vector of error terms. The nobservations in yt (n) and Xt (n) are the n most recent values from times
t − n + 1 to t. It is assumed that n > k. The rolling least squares estimates

343-5e+008-4e+008-3e+008-2e+008-1e+008

9.4 Rolling Regression

Oct

Nov
2000

Dec

Jan

Feb

Mar

Apr

May

Jun
2001

Jul

Aug

Sep

Oct

FIGURE 9.16. Accumulation/distribution indicator for Microsoft stock.

are−1β̂t (n) = [Xt (n)0 Xt (n)] Xt (n)0 yt (n)1σ̂ 2t (n) =ε̂t (n)0 ε̂t (n)
n−k
i0 h
i
1 h
=
yt (n) − Xt (n)β̂t (n) yt (n) − Xt (n)β̂t (n)
n−k
−1
a[
var(β̂t (n)) = σ̂ 2t (n) · [Xt (n)0 Xt (n)]9.4.1 Estimating Rolling Regressions Using the
S+FinMetrics Function rollOLS
The S+FinMetrics function rollOLS may be used to estimate generalrolling regression models. rollOLS is based on the S+FinMetrics regression function OLS and implements eﬃcient block updating algorithms forfast computation of rolling estimates. The arguments expected by rollOLS
are> args(rollOLS)
function(formula, data, subset, na.rm = F, method = "fit",
contrasts = NULL, start = NULL, end = NULL, width =NULL, incr = 1, tau = 1e-010, trace = T, ...)

344

9. Rolling Analysis of Time Series

which are similar to those used by OLS. In particular, AR may be used informulas to allow for lagged dependent variables and tslag and pdl may
be used to allow for lagged independent variables. The argument width
determines the rolling window width and the argument incr determines
the increment size by which the windows are rolled through the sample.
The output of rollOLS is an object of class “rollOLS” for which there
are print, summary, plot and predict methods and extractor function
coefficients. The use of rollOLS is illustrated with the following example.
Example 54 Rolling estimation of CAPM for Microsoft
Consider the estimation of the capital asset pricing model (CAPM) for
an asset using rolling regression on the excess returns market modelrt − rf t = α + β(rMt − rf t ) + εt , εt ∼ W N (0, σ 2 )(9.11)

where rt denotes the monthly return on an asset, rf t denotes the 30 day
T-bill rate, and rMt denotes the monthly return on a market portfolio
proxy. The coeﬃcient β measures the magnitude of market risk, and the
CAPM imposes the restriction that α = 0. Positive values of α indicate
an average excess return above that predicted by the CAPM and negative values indicate an average return below that predicted by the CAPM.
Rolling regression can be used to assess the stability of the CAPM regression over time and to uncover periods of time where an asset may havebeen overpriced or underpriced relative to the CAPM.
The monthly excess return data on Microsoft stock and S&P 500 index
over the ten year period February 1990 through December 2000 are in the
S+FinMetrics “timeSeries” object excessReturns.ts.
> colIds(excessReturns.ts)[1] "MSFT" "SP500"> start(excessReturns.ts)[1] Feb 1990
> end(excessReturns.ts)[1] Dec 2000
The full sample CAPM estimates using the S+FinMetrics function OLS are> ols.fit = OLS(MSFT~SP500,data=excessReturns.ts)> summary(ols.fit)
Call:
OLS(formula = MSFT ~SP500, data = excessReturns.ts)
Residuals:
Min
1Q Median-0.3101 -0.0620 -0.0024

3Q
0.0581Max
0.22609.4 Rolling Regression

345

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0175 0.0081
2.1654 0.0322
SP500 1.5677 0.2015
7.7788 0.0000
Regression Diagnostics:
R-Squared 0.3193
Adjusted R-Squared 0.3140
Durbin-Watson Stat 2.1891
Residual standard error: 0.09095 on 129 degrees of freedom
Time period: from Feb 1990 to Dec 2000
F-statistic: 60.51 on 1 and 129 degrees of freedom, the p-va
lue is 2.055e-012
The estimated full sample β for Microsoft is 1.57, which indicates that
Microsoft was riskier than the market. Also, the full sample estimate of α
is significantly diﬀerent from zero so, on average, the returns on Microsoft
are larger than predicted by the CAPM.
Consider now the 24-month rolling regression estimates incremented by
1 month computed using rollOLS> roll.fit = rollOLS(MSFT~SP500, data=excessReturns.ts,+ width=24,incr=1)
Rolling Window #1: Total Rows = 24
Rolling Window #2: Total Rows = 25
Rolling Window #3: Total Rows = 26...
Rolling Window #108: Total Rows = 131
To suppress the printing of the window count, specify trace=F in the callto rollOLS. The returned object roll.fit is of class “rollOLS” and hascomponents> names(roll.fit)[1] "width""incr"[6] "coef""stddev"[11] "positions""nwin""sigma""contrasts" "rdf""terms""call"

The components coef, stddev and sigma give the estimated coeﬃcients,
standard errors, and residual standard deviations for each of the nwin regressions. The positions component gives the start and end date of theestimation sample.
The print method, invoked by typing the object’s name, gives a briefreport of the fit

346

9. Rolling Analysis of Time Series> roll.fit
Call:rollOLS(formula = MSFT ~SP500, data = excessReturns.ts,
width = 24, incr = 1)
Rolling Windows:
number width increment108241Time period: from Feb 1990 to Dec 2000
Coefficients:(Intercept) SP500
mean 0.0221
1.2193std. dev. 0.0120
0.4549Coefficient Standard Deviations:(Intercept) SP500
mean 0.0177
0.5057std. dev. 0.0034
0.1107
Residual Scale Estimate:mean std. dev.
0.0827 0.0168
Regression estimates are computed for 108 rolling windows. The mean andstandard deviation are computed for the estimates and for the estimatedcoeﬃcient standard errors. The average and standard deviation of the α̂
values are 0.0221 and 0.0120, respectively, and the average and standard
deviation of the SE(α̂) values are 0.0177 and 0.0034, respectively. Hence,
most of the α̂ values appear to be not significantly diﬀerent from zero as
predicted by the CAPM. The average and standard deviation of the β̂values are 1.2193 and 0.4549, respectively. The β̂ values are quite variable
and indicate that amount of market risk in Microsoft is not constant overtime.
The rolling coeﬃcient estimates for each rolling regression may be viewed
using summary> summary(roll.fit)
Call:rollOLS(formula = MSFT ~ SP500, data = excessReturns.ts,
width = 24, incr = 1)
Rolling Windows:
number width increment9.4 Rolling Regression

347

108
24
1
Time period: from Feb 1990 to Dec 2000
Coefficient: (Intercept)
Value Std. Error t value Pr(>|t|)
Jan 1992 0.05075
0.01551
3.271 0.003492
Feb 1992 0.04897
0.01559
3.141 0.004751
Mar 1992 0.04471
0.015612.863 0.009035...
Coefficient: SP500
Value Std. Error t value Pr(>|t|)
Jan 1992 1.3545
0.3322
4.077 0.0004993
Feb 1992 1.3535
0.33374.056 0.0005260
Mar 1992 1.3735
0.33324.123 0.0004472...
or by using the coef extractor function. Notice that the first 24-monthrolling estimates are computed for January 1992, which is 24 months after
the sample start date of February 1990. The rolling estimates, however, arebest viewed graphically using plot
> plot(roll.fit)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Coef Estimates
3: plot: Coef Estimates with Confidence Intervals
4: plot: Residual Scale Estimates
Selection:
Plot selections 3 and 4 are illustrated in Figures 9.17 and 9.18. From thegraphs of the rolling estimate it is clear that α̂ is significantly positive
only at the very beginning of the sample. The β̂ values are near unity formost windows and increase sharply at the end of the sample. However, thelarge standard errors make it diﬃcult to determine if β is really changing
over time. The residual scale estimates, σ̂, increase sharply after 1999. Thisimplies that the magnitude of the non-market risk in Microsoft increased
after 1999.
In rollOLS, the optional argument incr sets the number of observations between the rolling blocks of data of length determined by width.
Therefore, rolling regressions may be computed for arbitrary overlapping
and non-overlapping blocks of data. For example, consider computing the
CAPM estimates for Microsoft over the two non-overlapping but adjacentsubsamples, February 1990 - June 1995 and July 1996 - November 2000
using rollOLS:

348

9. Rolling Analysis of Time Series

Rolling Coefficients

-0.05

SP500

-1

0

1

2

3

Value

0.00

0.05

(Intercept)

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

Rolling Window

FIGURE 9.17. Rolling regression estimates of CAPM coeﬃcients α̂ and β̂ forMicrosoft.
Rolling Scale

0.09
0.07

0.08

sigma

0.10

0.11

0.12

roll.fit

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001

Rolling Window

FIGURE 9.18. Rolling regression estimates of CAPM residual standard error for
Microsoft.

9.4 Rolling Regression

349> roll.fit2 = rollOLS(MSFT~SP500, data=excessReturns.ts,+ width=65, incr=65)
Rolling Window #1: Total Rows = 65
Rolling Window #2: Total Rows = 130> summary(roll.fit2)
Call:rollOLS(formula = MSFT ~SP500, data = excessReturns.ts,
width = 65, incr = 65)
Rolling Windows:
number width increment265
65Time period: from Feb 1990 to Dec 2000
Coefficient: (Intercept)
Value Std. Error t value Pr(>|t|)
Jun 1995 0.02765
0.009185 3.0100 0.003755
Nov 2000 0.01106
0.012880 0.8585 0.393851
Coefficient: SP500
Value Std. Error t value
Pr(>|t|)Jun 1995 1.339
0.2712
4.937 6.125e-006
Nov 2000 1.702
0.28136.050 8.739e-008

9.4.2 Rolling Predictions and Backtesting
Rolling regressions may be used to evaluate a model’s predictive performance based on historical data using a technique commonly referred to as
backtesting. To illustrate, consider the rolling regression model (9.10). The“out-of-sample” predictive performance of (9.10) is based on the rollingh-step predictions and prediction errors
ŷt+h|tε̂t+h|t= x0t+h β̂t (n),=yt+h − ŷt+h|t = yt+h − x0t+h β̂t (n)(9.12)(9.13)The predictions are “out-of-sample” because β̂t (n) only uses data up to
time t, whereas the predictions are for observations at times t + h for
h > 0. The rolling predictions are adaptive since β̂t (n) is updated whent is increased. When h = 1 there are T − n rolling 1-step predictions{ŷn+1|n , ŷn+2|n+1 , . . . , ŷT |T −1 }, when h = 2 there are T − n − 1 rolling
2-step predictions {ŷn+2|n , ŷn+3|n+1 , . . . , ŷT |T −2 } and so on.

350

9. Rolling Analysis of Time Series

Forecast Evaluation Statistics
The rolling forecasts (9.12) may be evaluated by examining the propertiesof the rolling forecast errors (9.13). Common evaluation statistics are
ME =

T
−h
X
1
ε̂t+h|tT − n − h + 1 t=n(9.14)

T
−h
X
1
ε̂2
MSE(h) =T − n − h + 1 t=n t+h|tp
RMSE(h) =M SE(h)MAE(h) =MAPE(h) =T−hX1|ε̂t+h|t |T − n − h + 1 t=n
¯
T
−h ¯
X
¯ ε̂t+h|t ¯1¯¯
T − n − h + 1 t=n ¯ yt+h ¯

The first measure evaluates the bias of the forecasts, and the other measures
evaluate bias and precision.
Example 55 Backtesting the CAPM
Consider again the estimation of the CAPM (9.11) for Microsoft using
rolling regression. The rolling regression information is contained in the“rollOLS” object roll.fit. The rolling h-step predictions (9.12) may becomputed using the generic predict method. For example, the rolling 1step forecasts are computed as> roll.pred = predict(roll.fit,n.step=1)> class(roll.pred)[1] "listof"> names(roll.pred)[1] "1-Step-Ahead Forecasts"
The argument n.step determines the step length of the predictions. The
object roll.pred is of class “listof” whose list component is a “timeSeries”
object containing the rolling 1-step predictions:> roll.pred[[1]]Positions1
Feb 1992
0.05994784
Mar 1992
0.01481398...
Dec 2000 -0.00049354
The prediction errors (9.13) are then computed asehat.1step = excessReturns.ts[,"MSFT"]-roll.pred[[1]]9.4 Rolling Regression

351

0.1
-0.1

Actual
1-step forecast-0.3

Return per month

0.3

Returns on MSFT and 1-step forecasts

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

1997

1998

1999

2000

2001

0.1
0.0
-0.1
-0.2
-0.3

Return per month

0.2

1-step forecast error

1990

1991

1992

1993

1994

1995

1996

FIGURE 9.19. Monthly returns on Microsoft, 1-step rolling forecasts and forecast
errors.

The monthly returns on Microsoft, 1-step forecasts and 1-step forecast errors are shown in Figure 9.19 created by> par(mfrow=c(2,1))> plot(excessReturns.ts[,"MSFT"], roll.pred[[1]],+ main="Returns on MSFT and 1-step forecasts",+ plot.args=list(lty=c(1,3)))> legend(0, -0.2, legend=c("Actual","1-step forecast"),+ lty=c(1,3))> plot(ehat.1step,main="1-step forecast error")
The forecast evaluation statistics (9.14) may be computed as> me.1step = mean(ehat.1step)> mse.1step = as.numeric(var(ehat.1step))> rmse.1step = sqrt(mse.1step)> mae.1step = mean(abs(ehat.1step))> mape.1step = mean(abs(ehat.1step/excessReturns.ts[,"MSFT"]),+ na.rm=T)
To compute just the 2-step forecasts, specify n.step=2 in the call to
predict. To compute the 1-step and 2-step forecasts specify n.step=1:2> roll.pred.12 = predict(roll.fit,n.steps=1:2)

352

9. Rolling Analysis of Time Series> names(roll.pred.12)[1] "1-Step-Ahead Forecasts" "2-Step-Ahead Forecasts"> roll.pred.12[[1]]Positions1
Feb 1992
0.05994784
Mar 1992
0.01481398...
Dec 2000 -0.00049354> roll.pred.12[[2]]Positions1
Mar 1992
0.0165764
Apr 1992
0.0823867...
Dec 2000 -0.0025076
Since the 1-step and 2-step predictions are components of the list object
roll.pred.12, the S-PLUS function lapply may be used to simplify thecomputation of the forecast errors and evaluation statistics. To illustrate,
supplying the user-defined function> make.ehat = function(x,y) {+ans = y - x+
ans[!is.na(ans),]+ }to lapply creates a “named” object containing the 1-step and 2-step forecasts errors as components:> ehat.list = lapply(roll.pred.12, make.ehat,+ excessReturns.ts[,"MSFT"])> names(ehat.list)[1] "1-Step-Ahead Forecasts" "2-Step-Ahead Forecasts"
The forecast evaluation statistics (9.14) may be computed for each component of ehat.list using lapply with the following user-defined function> make.errorStats = function(x){+me = mean(x)+mse = as.numeric(var(x))+
rmse = sqrt(mse)+mae = mean(abs(x))+
ans = list(ME=me,MSE=mse,RMSE=rmse,MAE=mae)
+
ans
+ }
> errorStat.list = lapply(ehat.list,make.errorStats)9.4 Rolling Regression353> unlist(errorStat.list)
1-Step-Ahead Forecasts.ME 1-Step-Ahead Forecasts.MSE-0.006165
0.009283
1-Step-Ahead Forecasts.RMSE 1-Step-Ahead Forecasts.MAE
0.09635
0.07207
2-Step-Ahead Forecasts.ME 2-Step-Ahead Forecasts.MSE-0.007269
0.009194
2-Step-Ahead Forecasts.RMSE 2-Step-Ahead Forecasts.MAE
0.09589
0.07187
The S-PLUS function sapply may be used instead of lapply to summarize
the forecast error evaluation statistics:> sapply(ehat.list,make.errorStats)
1-Step-Ahead Forecasts 2-Step-Ahead Forecasts[1,] -0.006165-0.007269[2,] 0.009283
0.009194[3,] 0.09635
0.09589[4,] 0.07207
0.07187
Comparing Predictive Accuracy
Backtesting is often used to compare the forecasting accuracy of two or
more competing models. Typically, the forecast evaluation statistics (9.14)
are computed for each model, and the model that produces the smallest setof statistics is judged to be the better model. Recently, Diebold and Mariano (1995) proposed a simple procedure using rolling h-step forecast errors
for statistically determining if one model’s forecast is more accurate thananother’s. Let ε̂1t+h|t and ε̂2t+h|t denote the h-step forecast errors from twocompeting models, and let N denote the number of h-step forecasts. The
accuracy of each forecast is measured by a particular forecast evaluation
or loss function
L(ε̂it+h|t ), i = 1, 2
´2³
Two popular loss functions are the squared error loss L(ε̂it+h|t ) = ε̂it+h|t¯¯¯¯and absolute error loss L(ε̂it+h|t ) = ¯ε̂it+h|t ¯. To determine if one model forecasts better than another Diebold and Mariano (1995) suggested computing
the loss diﬀerential
dt = L(ε̂1t+h|t ) − L(ε̂2t+h|t )and testing the null hypothesis of equal forecasting accuracy
H0 : E[dt ] = 0

354

9. Rolling Analysis of Time Series

The Diebold-Mariano test statistic is the simple ratio
DM =
where

d¯
c d)
¯ 1/2
lrv(

(9.15)

N
1 X
¯
dt
d=
N t=1

c d)
¯ is a consistent estimate of theis the average loss diﬀerential, and lrv(¯
long-run asymptotic variance of d. Diebold and Mariano suggest computc d)¯ using the Newey-West nonparametric estimator with a rectaning lrv(
gular weight function and a lag truncation parameter equal to the forecast
step length, h, less one. Diebold and Mariano showed that under the null
hypothesis of equal predictive accuracy the DM statistic is asymptotically
distributed N (0, 1).
Example 56 Backtesting regression models for predicting asset returns
To illustrate model comparison and evaluation by backtesting, consider
the problem of predicting the annual real return on S&P 500 index using
two diﬀerent valuation ratios. The regression model is of the form
rt = α + βxt−1 + εt(9.16)

where rt denotes the natural logarithm of the annual real total return on
S&P 500 index and xt denotes the natural logarithm of a valuation ratio.
The first valuation ratio considered is the dividend/price ratio and thesecond ratio is the earning/price ratio. The data are constructed from the
S+FinMetrics “timeSeries” shiller.annual as follows:> colIds(shiller.annual)[1] "price""dividend""earnings"[4] "cpi""real.price""real.dividend"[7] "real.earnings" "pe.10""dp.ratio"[10] "dp.yield"> # compute log of real data
> ln.p = log(shiller.annual[,"real.price"])> colIds(ln.p) = "ln.p"> ln.dpratio = log(dp.ratio)> colIds(ln.dpratio) = "ln.dpratio"> ln.epratio = -log(shiller.annual[,"pe.10"])> ln.epratio = ln.epratio[!is.na(ln.epratio),]> colIds(ln.epratio) = "ln.epratio"> # compute cc real total returns - see CLM pg. 261> ln.r = diff(ln.p) + log(1+exp(ln.dpratio[-1,]))> colIds(ln.r) = "ln.r"

9.4 Rolling Regression

355> stock.ts = seriesMerge(ln.p,ln.d,ln.dpratio,+ ln.epratio,ln.r,pos=positions(ln.epratio))> start(stock.ts)[1] Dec 1881
> end(stock.ts)[1] Dec 2000
Rolling regression estimates of (9.16) with the two valuation ratios using a
50 year window incremented by 1 year are computed as> roll.dp.fit = rollOLS(ln.r~tslag(ln.dpratio),data=stock.ts,+ width=50,incr=1)
Rolling Window #1: Total Rows = 50
Rolling Window #2: Total Rows = 51...
> roll.ep.fit = rollOLS(ln.r~tslag(ln.epratio),data=stock.ts,+ width=50,incr=1)
Rolling Window #1: Total Rows = 50
Rolling Window #2: Total Rows = 51...
Rolling Window #70: Total Rows = 119
Figures 9.20 and 9.21 show the rolling coeﬃcient estimates from the twomodels along with standard error bands. The rolling estimates of β forthe two models are similar. For both models, the strongest evidence for
return predictability occurs between 1960 and 1990. The value of β for theearning/price model appears to be diﬀerent from zero during more periods
than the value of β for the dividend/price model.
The rolling h-step predictions for h = 1, . . . , 5 and prediction errors are> roll.dp.pred = predict(roll.dp.fit,n.steps=1:5)> roll.ep.pred = predict(roll.ep.fit,n.steps=1:5)> ehat.dp.list = lapply(roll.dp.pred,make.ehat,+ stock.ts[,"ln.r"])> ehat.ep.list = lapply(roll.ep.pred,make.ehat,+ stock.ts[,"ln.r"])
The forecast evaluation statistics are> errorStats.dp.list = lapply(ehat.dp.list,make.errorStats)> errorStats.ep.list = lapply(ehat.ep.list,make.errorStats)> tmp = cbind(unlist(errorStats.dp.list),+ unlist(errorStats.ep.list))> colIds(tmp) = c("D/P","E/P")> tmp
numeric matrix: 20 rows, 2 columns.
D/P
E/P
1-Step-Ahead Forecasts.ME 0.03767 0.01979

356

9. Rolling Analysis of Time Series

Rolling Coefficients

0.4

tslag(ln.dpratio)

0.0

0.2

Value

0.0

0.5

1.0

(Intercept)

1935

1940

1945

1950

1955

1960

1965

1970

1975

1980

1985

1990

1995

2000

Rolling Window

FIGURE 9.20. 50 year rolling regression estimates of (9.16) using dividend/priceratio.

Rolling Coefficients0.0

tslag(ln.epratio)

0.0

0.2

0.4

Value

0.5

1.0

(Intercept)

1935

1940

1945

1950

1955

1960

1965

1970

1975

1980

1985

1990

1995

2000

Rolling Window

FIGURE 9.21. 50 year rolling regression estimates of (9.16) using earning/price.

9.4 Rolling Regression

357

1-Step-Ahead Forecasts.MSE 0.03150 0.03139
1-Step-Ahead Forecasts.RMSE 0.17749 0.17718
1-Step-Ahead Forecasts.MAE 0.14900 0.14556
2-Step-Ahead Forecasts.ME 0.04424 0.02334
2-Step-Ahead Forecasts.MSE 0.03223 0.03205
2-Step-Ahead Forecasts.RMSE 0.17952 0.17903
2-Step-Ahead Forecasts.MAE 0.15206 0.14804
3-Step-Ahead Forecasts.ME 0.04335 0.02054
3-Step-Ahead Forecasts.MSE 0.03203 0.03180
3-Step-Ahead Forecasts.RMSE 0.17898 0.17832
3-Step-Ahead Forecasts.MAE 0.14993 0.14731
4-Step-Ahead Forecasts.ME 0.04811 0.02397
4-Step-Ahead Forecasts.MSE 0.03292 0.03248
4-Step-Ahead Forecasts.RMSE 0.18143 0.18022
4-Step-Ahead Forecasts.MAE 0.15206 0.14855
D/P
E/P
5-Step-Ahead Forecasts.ME 0.04707 0.02143
5-Step-Ahead Forecasts.MSE 0.03339 0.03255
5-Step-Ahead Forecasts.RMSE 0.18272 0.18043
5-Step-Ahead Forecasts.MAE 0.15281 0.14825
The forecast evaluation statistics are generally smaller for the model using
the earning/price ratio. The Diebold-Mariano statistics based on squared
error and absolute error loss functions may be computed using> for (i in 1:5) {+d.mse[,i] = ehat.dp.list[[i]]^2 - ehat.ep.list[[i]]^2+
DM.mse[i] = mean(d.mse[,i])/sqrt(asymp.var(d.mse[,i],+bandwidth=i-1,window="rectangular"))+d.mae[,i] = abs(ehat.dp.list[[i]]) - abs(ehat.ep.list[[i]])+DM.mae[i] = mean(d.mae[,i])/sqrt(asymp.var(d.mae[,i],+bandwidth=i-1,window="rectangular"))+ }> names(DM.mse) = names(ehat.dp.list)> names(DM.mae) = names(ehat.dp.list)> cbind(DM.mse,DM.mae)
DM.mse DM.mae
1-Step-Ahead Forecasts 0.07983 0.07987
2-Step-Ahead Forecasts 0.09038 0.08509
3-Step-Ahead Forecasts 0.07063 0.05150
4-Step-Ahead Forecasts 0.08035 0.06331
5-Step-Ahead Forecasts 0.07564 0.06306
Since the DM statistics are asymptotically standard normal, one cannot
reject the null hypothesis of equal predictive accuracy at any reasonable358

9. Rolling Analysis of Time Seriessignificance level based on the 1-step through 5-step forecast errors for thetwo models.

9.5 Rolling Analysis of General Models Using the
S+FinMetrics Function roll
The S-PLUS aggregateSeries function is appropriate for rolling analysis of simple functions and the S+FinMetrics function rollOLS handles
rolling regression. The S+FinMetrics function roll is designed to performrolling analysis of general S-PLUS modeling functions that take a formula
argument describing the relationship between a response and explanatory
variables and where the data, usually a data frame or “timeSeries” object with a data frame in the data slot, is supplied explicitly in a data
argument. The arguments expected by roll are> args(roll)function(FUN, data, width, incr = 1, start = NULL, end =NULL, na.rm = F, save.list = NULL, arg.data ="data", trace = T, ...)
where FUN is the S-PLUS modeling function to be applied to each rolling
window, data is the data argument to FUN which must be either a data
frame or a “timeSeries” with a data frame in the data slot,width specifies
the width of the rolling window and incr determines the increment size bywhich the windows are rolled through the sample. The argument save.list
specifies the components of the object returned by FUN to save in the object
returned by roll. If FUN requires more arguments in addition to data, forexample a formula relating a response to a set of explanatory variables,
then these arguments should be supplied in place of .... The use of rollis illustrated with the following examples.
Example 57 Rolling regression
In this example, the 24-month rolling regression estimation of the CAPM
for Microsoft using the “timeSeries” excessReturns.ts is repeated using
the S+FinMetrics function roll with FUN=OLS. OLS requires a formula
argument for model specification and a data frame or “timeSeries” indata argument. The 24 month rolling CAPM estimates using roll are> roll.fit = roll(FUN=OLS, data=excessReturns.ts,+ width=24, incr=1, formula=MSFT~SP500)
Rolling Window #1: Total Rows = 24
Rolling Window #2: Total Rows = 25...
Rolling Window #108: Total Rows = 131

9.5 Rolling Analysis of General Models

359> class(roll.fit)[1] "roll"
The return roll.fit is an object of class “roll” for which there are no
specific method functions. Since the data argument excessReturns.ts isa “timeSeries”, the default components of roll.fit are the positions of
the rolling windows and “timeSeries” objects containing the componentsthat are produced by OLS for each of the windows:> names(roll.fit)[1] "R""coef"[5] "residuals" "assign"[9] "terms""call"> class(roll.fit$coef)[1] "timeSeries"> nrow(roll.fit$coef)[1] 108> class(roll.fit$residuals)[1] "timeSeries"> nrow(roll.fit$residuals)[1] 108"df.resid" "fitted""contrasts" "ar.order""positions"

The first column of the “timeSeries” roll.fit$coef contains the rollingintercept estimates, and the second column contains the rolling slope estimates.
> roll.fit$coef[1:2,]
Positions1
2
Jan 1992 0.05075 1.354
Feb 1992 0.04897 1.353
The rows of the “timeSeries” roll.fit$residuals contain the residuals
for the OLS fit on each 24-month window> roll.fit$residuals[1:2,]
Positions1
2
3
4
5
Jan 1992 0.007267 0.04021 0.03550 0.085707 0.004596
Feb 1992 0.042014 0.03726 0.08757 0.006368 -0.164498...
24
0.05834-0.03393
If only some of the components of OLS are needed for each rolling window,
these components may be specified in the optional argument save.list.
For example, to retain only the components coef and residuals over the

360

9. Rolling Analysis of Time Series

rolling windows specify save.list=c("coef","residuals") in the call toroll:> roll.fit = roll(FUN=OLS, data=excessReturns.ts,+ width=24, incr=1, formula=MSFT~SP500,+ save.list=c("coef","residuals"), trace=F)> names(roll.fit)[1] "coef""residuals" "call""positions"

9.6 References
Alexander, C. (2001). Market Models: A Guide to Financial Data Analysis. John Wiley & Sons, Chichester, UK.
Bauer, R.J. and J.R. Dahlquist (1999). Techincal Market Indicators:
Analysis & Performance. John Wiley & Sons, New York.
Banerjee, A. R. Lumsdaine and J.H. Stock (1992). “Recursive andSequential Tests of the Unit Root and Trend Break Hypothesis: Theory
and International Evidence,” Journal of Business and Economic Statistics,
10(3), 271-288.
Colby, R.W. and T.A Meyers (1988). The Encyclopedia of TechnicalMarket Indicators. McGraw-Hill, New York.
Dacorogna, M.M., R. Gençay, U.A. Müller, R.B. Olsen, andO.V. Pictet (2001). An Introduction to High-Frequency Finance. Academic Press, San Diego.
Diebold, F.X. and R.S. Mariano (1995). “Comparing Predictive Accuracy,” Journal of Business and Economic Statistics, 13, 253-263.
Shiller, R. (1998). Irrational Exuberance. Princeton University Press,
Princeton, NJ.
Zumbach, G.O., and U.A. Müller (2001). “Operators on Inhomogeneous Time Series,” International Journal of Theoretical and Applied Finance, 4, 147-178.

This is page 361
Printer: Opaque this10
Systems of Regression Equations

10.1 Introduction
The previous chapters dealt with models for univariate financial time series. In many applications, it is desirable to model the joint behavior of
multiple time series because of possible eﬃciency gains to the joint estimation of a system of time series models. For example, there may be complex
interactions between the variables and/or the error structure across models. Univariate models cannot capture these interactions whereas multivariate models can. Furthermore, many equilibrium models for asset returns,
like the capital asset pricing model (CAPM) or the arbitrage price model(APT), imply parameter restrictions that are common to the model representation of all assets. Hence, the testing of equilibrium asset pricing
models requires the testing of cross equation parameter constraints, andthe proper estimation of these models would impose these cross equation
restrictions.
This chapter introduces methods for modeling and analyzing systemsof linear and nonlinear regression equations. Section 10.2 describes Zellner’s seemingly unrelated regression (SUR) system of regression equations
that may be linked through common regressors, correlated error terms,
or cross equation parameter restrictions. Section 10.3 describes the specification and estimation of linear SUR models and gives examples using
the S+FinMetrics function SUR. Section 10.4 describes the specificationand estimation of nonlinear SUR models and gives examples using the
S+FinMetrics function NLSUR.

362

10. Systems of Regression Equations

The SUR model was developed by Theil (1961) and Zellner (1962) andis described in most econometric textbooks. The nonlinear SUR model wasdeveloped by Gallant (1974). Greene (2000) gives a general overview of
linear and nonlinear SUR models and Srivastava and Giles (1987) provides
a thorough treatment. Burmeister and McElroy (1986) and Campbell, Lo,
and MacKinlay (1997) describe the estimation and testing of systems ofasset pricing models using SUR and nonlinear SUR models.

10.2 Systems of Regression Equations
Many applications in economics and finance involve a system of linear regression equations of the form
y1
y2

yM

= X1 β1 + ε1
= X2 β2 + ε2..
.
= XM βM + εM

(10.1)

where yi is a (T × 1) vector of dependent variables, Xi is a (T × ki ) matrix
of explanatory variables and εi is a (T × 1) vector of errors for equations
i = 1, ..., M . It is assumed that each Xi is exogenous, i.e., uncorrelatedwith εi . Depending on the application, each Xi may be distinct or there
may be common regressors across equations. The equations in (10.1) arepotentially linked either through the covariance structure of the errors or
through cross equation restrictions on the elements of βi (i = 1, ..., M ), andestimation of the entire system generally produces more eﬃcient estimates
than the estimation of each equation individually.
Economic theory often implies a system of nonlinear regression equationsof the form
y1
y2

yM

= f1 (β1 , X1 ) + ε1
= f2 (β2 , X2 ) + ε2..
.
= fM (βM , XM ) + εM

where fi (βi , Xi ) is a (T ×1) vector containing the nonlinear function valuesfi (βi , xit ) for equations i = 1, . . . , M . The functions fi may be the sameor diﬀerent across equations, the error structures may be linked, and there
may be cross equation restrictions on the elements of βi (i = 1, ..., M ).
Some common applications in finance are illustrated below.
Example 58 Exchange rate regressions

10.2 Systems of Regression Equations

363

Consider the system of M exchange rate regressionsk− si,t ) + εi,t+k , i = 1, . . . , M∆si,t+k = αi + γ i (fi,t(10.2)

where si,t+k represents the natural log of the spot exchange exchange ratek
for currency i (relative, say, to the U.S. dollar) at time t + k and fi,tdenotes the natural log of the forward exchange rate at time t for a forward
contract in currency i that will deliver at time t + k. In terms of the previous notation for the system of linear regression equations, yit = si,t+k ,kxit = (1, fi,t− si,t )0 and β i = (αi , γ i )0 , the only common regressor in thesystem is a vector of ones. The error terms, εi,t+k , are likely to be correlated contemporaneously across equations due to common random shocks
aﬀecting all exchange rates. That is, E[εi ε0j ] = σ ij IT where E[εit εjs ] = σij
for t = s and 0 otherwise. In the present context, this across equation correlation can be used to increase the eﬃciency of the parameter estimatesfor each equation.
Example 59 The capital asset pricing model with a risk-free asset
Consider the excess return single index model regression
Rit − rf t = αi + β i (RMt − rf t ) + εit , i = 1, . . . , M
where Rit denotes the return on asset i at time t, rf t denotes the returnon a risk-free asset, RMt denotes the return on an proxy for the “market
portfolio” and εit denotes the residual return not explained by the “market”
for asset i. In terms of the previous notation for the system of regression
equations, yit = Rit − rf t , xit = (1, RMt − rf t )0 , and β i = (αi , β i )0 sothat all regression equations share the same regressors. It is likely that theresidual returns, εit , are contemporaneously correlated across assets due
to common shocks not related to the “market”. That is, E[εi ε0j ] = σij IT
where E[εit εjs ] = σ ij for t = s and 0 otherwise. However, unless there
are across equation restrictions on βi , the fact that xit is the same foreach equation means that there will be no eﬃciency gain in estimating
the parameters from exploiting the across equation error correlation. The
capital asset pricing model (CAPM) imposes the restriction αi = 0 for
all assets i so that E[Rit ] − rf t = β i (E[RMt − rf t ). Testing the CAPM
therefore involves a joint test of many cross equation zero restrictions.
Example 60 The capital asset pricing model without a risk-free asset
The CAPM formulation above assumes the existence of a risk-free asset.
If there is no risk-free asset, then Black (1972) showed that the CAPM
takes the form
real
real
E[Rit
] − γ = β i (E[RMt] − γ)

real
real
denotes the real return on asset i, RM
where Ritt denotes the real returnon the market, and γ denotes the unobservable return on a zero-beta portfolio. The Black form of the CAPM may be estimated from the system of

364

10. Systems of Regression Equations

nonlinear regression equations
real
real
Rit
= (1 − β i )γ + β i RMt+ εit , i = 1, . . . , M(10.3)In terms of the above notation for systems of nonlinear regression equations,real
real 0Black’s restricted CAPM has yit = Rit, xit = (1, RMt) , β i = (γ, β i )0 andrealfi (βi , xit ) = (1 − β i )γ + β i RMt. Notice that the parameter γ is common
across all equations. The Black form of the CAPM may be tested by estimating the unrestricted system
real
real
Rit
= αi + β i RMt+ εit , i = 1, . . . , M(10.4)and testing the M nonlinear cross equation restrictions αi = (1 − β i )γ.

10.3 Linear Seemingly Unrelated Regressions
The seemingly unrelated regression (SUR) model due to Theil (1961) and
Zellner (1962) is the unrestricted system of M linear regression equations
yi = Xi βi + εi , i = 1, . . . , M(10.5)

where yi is (T × 1), Xi is (T × ki ), β i is (k × 1) and εi is (T × 1). The error
terms are assumed to be contemporaneously correlated across equations
but temporally uncorrelated: E[εit εjs ] = σ ij for t = s; 0 otherwise.
The M equations may be stacked to form the giant regression model
⎞ ⎛
⎞⎛
⎞ ⎛
⎞
⎛
X1 0
β1
ε1
y1
0
⎟⎜ . ⎟ ⎜ . ⎟⎜ .. ⎟ ⎜⎝ . ⎠ = ⎝ 0 ...
0 ⎠ ⎝ .. ⎠ + ⎝ .. ⎠
0

yM

0

βM

XM

εM

or

(10.6)

y= Xβ+εwhere y is (M T × 1), X is (M T × K), β is (K × 1) and ε is (M T × 1).
P
Here K = M
i=1 ki is the total number of regressors across all equations.
The error term in the giant regression has non-diagonal covariance matrix
V = E[εε0 ] = Σ ⊗ IT

(10.7)

where the (M × M ) matrix Σ has elements σ ij .

10.3.1 Estimation
Since V is not diagonal, least squares estimation of β in the giant regression(10.6) is not eﬃcient. The generalized least squares (GLS) estimator of β,β̂GLS= (X0 V−1 X)−1 X0 V−1 y0−1= (X (Σ⊗ IT )X)−1(10.8)0−1

X (Σ⊗ IT )y10.3 Linear Seemingly Unrelated Regressions

365

is eﬃcient.
It can be shown (e.g., Greene 2000, Chap. 15) that if Xi = X for all
equations i = 1, ..., M (i.e., all equations have the same regressors), or if
the error covariance matrix V is diagonal and there are no cross equation restrictions on the values of βi then least squares estimation of (10.5)
equation by equation produces the GLS estimator (10.8).
Feasible GLS Estimation
The GLS estimator of β is usually not feasible since the covariance matrix
Σ, and hence V, is generally not known. However, in the SUR model theelements of Σ can be consistently estimated by least squares estimation of(10.5) equation by equation using
σ̂ij

−1 0
= dfijε̂i ε̂j=(10.9)−1dfij(yi −Xi β̂i )0 (yj −Xj β̂j ),producing Σ̂. In (10.9), dfij represents the degrees of freedom used for theestimate σ̂ ij . See the online help for SUR for more details on the diﬀerent
options for choosing dfij . The feasible generalized least squares estimator(FGLS) is−1

β̂FGLS = (X0 (Σ̂−1⊗ IT )X)−1 X0 (Σ̂⊗ IT )y(10.10)and its asymptotic variance is consistently estimated by
−1

avar(
d β̂FGLS ) = (X0 (Σ̂⊗ IT )X)−1

The FGLS estimator (10.10) is asymptotically equivalent to the GLS estimator (10.8).
Tests of linear hypotheses of the form Rβ = r, which may incorporatecross equation linear restrictions, may be computed in the usual way with
the Wald statistichi−1
Wald = (Rβ̂ FGLS −r)0 Ravar((Rβ̂FGLS −r)(10.11)d β̂FGLS )R0
which is asymptotically distributed chi-square with degrees of freedom
equal to the number of restrictions being tested under the null.
Iterated Feasible GLS Estimation
The estimate of Σ in FGLS estimation uses the ineﬃcient least squares
estimate of βi . The iterated FGLS estimator repeats the construction of
the FGLS estimator using an updated estimator of Σ based on the FGLS
estimator (10.10). That is, at each iteration updated estimates of σ ij arecomputed as−1σ̂ ij,FGLS = dfij(yi −Xi β̂i,FGLS )0 (yj −Xj β̂j,FGLS )366

10. Systems of Regression Equations

and the resulting updated estimator of Σ is used to recompute the FGLS
estimator. This process is iterated until β̂ FGLS no longer changes. If the error terms for each equation are Gaussian, it can be shown that the iterated
FGLS estimator of β using dfij = T is the maximum likelihood estimator(MLE). It should be noted, however, that iteration does not improve theasymptotic properties of the FGLS estimator.
Maximum Likelihood Estimation
Although the MLE of β may be obtained by iterating the FGLS estimator,
it is often computationally more eﬃcient to compute the MLE directly.
To conveniently express the likelihood function for the SUR model it isnecessary to re-express the SUR model by grouping the data horizontally byobservations instead of vertically by equations. The SUR model expressed
this way is given by
yt0 = x0t Π + ε0t , t = 1, . . . , T
where yt0 = (y1t , y2t , . . . , yMt ) is (1 × M ) vector of dependent variables,
x0t = (x1t , x2t , . . . , xKt ) is the (1 × K) vector containing all of the uniqueexplanatory variables, Π = [π 1 , π 2 , . . . , π M ] is a (K ×M ) matrix where the(K × 1) vector π i contains the coeﬃcients on x0t for the ith equation, and
ε0t = (ε1t , ε2t , . . . , εM t ) is a (1 × M ) vector of error terms with covariancematrix Σ. Note that since the ith equation may not have all of the variables
as regressors so that some of the values in π i may be equal to zero.
The log-likelihood function for a sample of size T isln L(β, Σ) = −T
MTT
1 X 0 −1ε Σ εtln(2π) − ln |Σ| −2
2
2 t=1 twhere β represents the appropriate non-zero elements of Π. The log-likelihood
function may be concentrated with respect to Σ givingln L(β) = −MTT(ln(2π) + 1) − ln (|Σ(β)|)
2
2

where
Σ(β) = T −1

TX

εt ε0t(10.12)(10.13)t=1

is the MLE for Σ given β. Hence, the MLE for β solvesminβ1
ln (|Σ(β)|) .
2

and the resulting estimator β̂mle is equivalent to the iterated feasible GLS
estimator with dfij = T .

10.3 Linear Seemingly Unrelated Regressions

367

Likelihood Ratio Tests
The form of the concentrated log-likelihood function (10.12), implies thatlikelihood ratio (LR) tests for hypotheses about elements of β have thesimple form
³ ³
´
³
´´
LR = T ln |Σ(β̃ mle )| − ln |Σ(β̂mle )|(10.14)
where β̃mle denotes the MLE imposing the restrictions under the null being
tested and β̂mle denotes the unrestricted MLE. The LR statistic (10.14) is
asymptotically distributed chi-square with degrees of freedom equal to thenumber of restrictions being tested under the null.

10.3.2 Analysis of SUR Models with the S+FinMetrics
Function SUR
The S+FinMetrics function SUR may be used for the estimation of linear
SUR models without cross equation restrictions. The arguments for SUR
are> args(SUR)
function(formulas, data, subset, na.rm = F, start = NULL, end= NULL, method = "fit", contrasts = NULL, df = 1,
tol = 1e-006, iterate = F, trace = T, ...)
Generally, the two specified arguments are formulas, which is a list containing the formulas for each equation in the SUR model, and data, whichmust be either a data frame, or a “timeSeries” object with a data framein the data slot. Formulas are specified in the usual way with the response
variables on the left hand side of the character and explanatory variableson the right hand side. If the variables in formulas can be directly accessed, e.g. through an attached data frame, then the data argument may
be skipped. The default fitting method is one-step (not iterated) feasible
GLS as in (10.10). To specify iterated feasible GLS set the optional argument iterate=T. In this case, the trace option controls printing of theiteration count and the tol option specifies the numerical tolerance of theconvergence criterion. The optional argument df specifies the degrees of
freedom parameter dfij for the computation of σ̂ ij . If df=1, the feasible
GLS estimates are unbiased; if df=2, the feasible GLS estimates are minimum MSE estimates; if df=3, the sample size T is used as the degree of
freedom; if df=4, [(T − ki )(T − kj )]1/2 is used as the degree of freedom forthe (i, j) element of the residual covariance matrix. To reproduce the MLE
under Gaussian errors set iterate=T and df=3.
SUR produces an object of class “SUR” for which there are print, summaryand plot methods as well as extractor functions residuals, fitted.values

368

10. Systems of Regression Equations(or fitted), coef, and vcov. The use of SUR is illustrated using the following examples.
Example 61 Testing eﬃciency in foreign exchange markets
Consider estimating the system of exchange rate regressions (10.2) using
monthly data on six currencies relative to the US dollar over the period
August 1978 through June 1996. The data are in the “timeSeries” object surex1.ts, which is constructed from the data in the “timeSeries”
lexrates.dat. The variables in surex1.ts are> colIds(surex1.ts)[1] "USCN.FP.lag1" "USCNS.diff"[5] "USFR.FP.lag1" "USFRS.diff"[9] "USJY.FP.lag1" "USJYS.diff""USDM.FP.lag1" "USDMS.diff""USIL.FP.lag1" "USILS.diff""USUK.FP.lag1" "USUKS.diff"

The variables with extensions .FP.lag1 are one month forward premia,
1fi,t− si,t , and variables with extensions .diff are future returns on spot
currency, ∆si,t+1 . The list of formulas for the regressions in the system(10.2) is created using> formula.list = list(USCNS.diff~USCN.FP.lag1,+ USDMS.diff~USDM.FP.lag1,+ USFRS.diff~USFR.FP.lag1,+ USILS.diff~USIL.FP.lag1,+ USJYS.diff~USJY.FP.lag1,+ USUKS.diff~USUK.FP.lag1)
The command to compute the feasible GLS estimator of the SUR systemover the period August 1978 through June 1996 is> sur.fit = SUR(formula.list, data=surex1.ts,+ start="Aug 1978", in.format="%m %Y")
> class(sur.fit)[1] "SUR"
As usual, the print method is invoked by typing the name of the object
and gives basic output:> sur.fit
Seemingly Unrelated Regression:
Eq. 1: USCNS.diff ~USCN.FP.lag1
Coefficients:(Intercept) USCN.FP.lag1-0.0031-1.6626

10.3 Linear Seemingly Unrelated Regressions

369

Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0135
Eq. 2: USDMS.diff ~USDM.FP.lag1
Coefficients:(Intercept) USDM.FP.lag1
0.0006
0.5096
Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0358...
Eq. 6: USUKS.diff ~USUK.FP.lag1
Coefficients:(Intercept) USUK.FP.lag1-0.0035-1.2963
Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0344
Log determinant of residual covariance: -47.935
In the above
output,´ the log determinant of residual covariance is the quan³
tity ln |Σ(β̂FGLS )| . The forward rate is an unbiased predictor of the future spot rate if the coeﬃcient on the forward premium is equal to 1. The
results above suggest that unbiasedness holds only for the US/France exchange rate.
The summary method provides more detailed information about the fit
including estimated standard errors of coeﬃcients and fit measures for eachequation> summary(sur.fit)
Seemingly Unrelated Regression:
Eq. 1: USCNS.diff ~USCN.FP.lag1
Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) -0.0031 0.0012-2.5943 0.0101

370

10. Systems of Regression Equations

USCN.FP.lag1 -1.6626

0.5883-2.82630.0052

Regression Diagnostics:
R-Squared 0.0300
Adjusted R-Squared 0.0254
Durbin-Watson Stat 2.2161
Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0135...
Eq. 6: USUKS.diff ~USUK.FP.lag1
Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) -0.0035 0.0027-1.3256 0.1864
USUK.FP.lag1 -1.2963 0.6317-2.0519 0.0414
Regression Diagnostics:
R-Squared 0.0253
Adjusted R-Squared 0.0207
Durbin-Watson Stat 1.9062
Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0344
Log determinant of residual covariance: -47.935
Graphical summaries of each equation are provided by the plot method
which produces a menu of plot choices:> plot(sur.fit)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Response and Fitted Values
3: plot: Residuals
4: plot: Normal QQplot of Residuals
5: plot: ACF of Residuals

10.3 Linear Seemingly Unrelated Regressions

371

Residual Autocorrelation
5
USILS.diff

10

15

20

USJYS.diff

USUKS.diff
1.0
0.8
0.6
0.4
0.2

ACF

0.0
-0.2
USCNS.diff

USDMS.diff

USFRS.diff

1.0
0.8
0.6
0.4
0.2
0.0
-0.2
5

10

15

20

5

10

15

20

Lag

FIGURE 10.1. Residual ACF plots from SUR fit to exchange rate data.

6: plot: PACF of Residuals
7: plot: ACF of Squared Residuals
8: plot: PACF of Squared Residuals
Selection:
Plot choices 2-8 create multi-panel plots, one panel for each equation, using
Trellis graphics. For example, Figure 10.1 shows the ACF of Residuals plot
for the exchange rate data.
The above results are based on the non-iterated feasible GLS estimator(10.10). The iterated estimator is computed using> sur.fit2 = SUR(formula.list, data=surex1.ts,+ start="Aug 1978", in.format="%m %Y", iterate=T)
Iteration 1
Iteration 2
Iteration 3
Iteration 4
Iteration 5
Iteration 6
Iteration 7

372

10. Systems of Regression Equations

Iteration 8
which converges after eight iterations. The non-iterated and iterated estimators may be easily compared using the coef extractor function:> cbind(coef(sur.fit),coef(sur.fit2))
[,1]
[,2]
(Intercept) -0.00312063 -0.00312126
USCN.FP.lag1 -1.66255897 -1.66303965(Intercept) 0.00058398 0.00035783
USDM.FP.lag1 0.50956590 0.65014949(Intercept) 0.00133327 0.00135930
USFR.FP.lag1 1.01512081 1.02834484(Intercept) -0.00058789 -0.00083921
USIL.FP.lag1 0.46173993 0.40852433(Intercept) 0.00778918 0.00744485
USJY.FP.lag1 -1.76416190 -1.63952144(Intercept) -0.00354947 -0.00334026
USUK.FP.lag1 -1.29625869 -1.19508947
There is not much diﬀerence between the two estimators.
The SUR estimator is more eﬃcient than least squares equation-byequation in this example provided the error terms across equations arecorrelated. The residual correlation matrix of the SUR fit (10.13) may becomputed using> sd.vals = sqrt(diag(sur.fit$Sigma))> cor.mat = sur.fit$Sigma/outer(sd.vals,sd.vals)> cor.mat
USCNS.diff USDMS.diff USFRS.diff USILS.diff
USCNS.diff
1.00000
0.20187
0.194210.27727
USDMS.diff
0.20187
1.00000
0.972090.85884
USFRS.diff
0.194210.97209
1.00000
0.85090
USILS.diff
0.277270.858840.85090
1.00000
USJYS.diff
0.12692
0.61779
0.614430.50835
USUKS.diff
0.31868
0.714240.70830
0.72274

USCNS.diff
USDMS.diff
USFRS.diff
USILS.diff
USJYS.diff
USUKS.diff

USJYS.diff USUKS.diff
0.12692
0.31868
0.61779
0.714240.614430.70830
0.50835
0.72274
1.00000
0.532420.532421.00000

Many of the estimated correlations are large so there appears to be aneﬃciency benefit to using SUR.

10.3 Linear Seemingly Unrelated Regressions

373

The forward rate unbiasedness implies that γ i = 1 in (10.2) for i =1, . . . , 6. A formal test of the unbiasedness hypothesis for all six currencies
simultaneously may be done using the Wald statistic (10.11) or the LR
statistic (10.14). For the LR statistic, the iterated FGLS estimate should
be used. The S-PLUS commands to compute the Wald statistic are> bigR = matrix(0,6,12)> bigR[1,2] = bigR[2,4] = bigR[3,6] = bigR[4,8] =+ bigR[5,10] = bigR[6,12] = 1> rr = rep(1,6)> bHat = as.vector(coef(sur.fit))> avar = bigR%*%vcov(sur.fit)%*%t(bigR)> Wald = t((bigR%*%bHat-rr))%*%solve(avar)%*%(bigR%*%bHat-rr)> Wald[,1][1,] 47.206> 1-pchisq(Wald,6)[1] 1.7025e-008
The data clearly reject the unbiased hypothesis. To compute the LR statistic (10.14), the restricted model with γ i = 1 for i = 1, . . . , 6 must first be
computed. The restricted model takes the formk− si,t ) = αi + εi,t+k∆si,t+k − (fi,tThe S-PLUS commands to compute the restricted model are> formula.list = list((USCNS.diff-USCN.FP.lag1)~1,+ (USDMS.diff-USDM.FP.lag1)~1,+ (USFRS.diff-USFR.FP.lag1)~1,+ (USILS.diff-USIL.FP.lag1)~1,+ (USJYS.diff-USJY.FP.lag1)~1,+ (USUKS.diff-USUK.FP.lag1)~1)> sur.fit2r = SUR(formula.list, data=surex1.ts,+ start="Aug 1978", in.format="%m %Y", iterate=T)
Iteration 1
Iteration 2> sur.fit2r
Seemingly Unrelated Regression:
Eq. 1: (USCNS.diff - USCN.FP.lag1) ~1
Coefficients:(Intercept)
0.0004

374

10. Systems of Regression Equations

Degrees of freedom: 215 total; 214 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.014...
Eq. 6: (USUKS.diff - USUK.FP.lag1) ~1
Coefficients:(Intercept)
0.0012Degrees of freedom: 215 total; 214 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0353
Log determinant of residual covariance: -47.61
The LR statistic (10.14) may then be computed using> nobs = nrow(residuals(sur.fit2r))> LR = nobs*(determinant(sur.fit2r$Sigma,log=T)$modulus+ determinant(sur.fit2$Sigma,log=T)$modulus)> as.numeric(LR)[1] 70.09> 1-pchisq(LR,6)[1] 3.912e-013
The LR statistic also confirms the rejection of the unbiasedness hypothesis.

10.4 Nonlinear Seemingly Unrelated Regression
ModelsThe nonlinear SUR model is the system of M nonlinear regression equations
yi = fi (β i , Xi ) + εi , i = 1, . . . , M
where yi is a (T × 1) vector of response variables, fi (βi , Xi ) is a (T × 1)
vector containing the nonlinear function values fi (βi , xit ), Xi is a (T × ki )matrix of explanatory variables, and βi is a (ki × 1) vector of parameters.
As with the linear SUR model, some of the explanatory variables in each
Xi may be common across equations and some of the parameters in eachβ i may also be common across equations. Without loss of generality, let X
denote the (T × K) matrix of unique variables and let β denote the (Q × 1)

10.4 Nonlinear Seemingly Unrelated Regression Models

375

vector of unique parameters,1 and rewrite the nonlinear SUR system as
yi = fi (β, X) + εi , i = 1, . . . , M
The assumptions about the (M T × 1) system error vector ε= (ε01 . . . , ε0M )0are the same as in the linear SUR model. That is, the covariance matrix of
ε is given by (10.7).
The estimation of nonlinear SUR models is detailed in Greene (2000)
Chapter 15 and only a brief description is given here. The nonlinear FGLS
estimator of β solves
min
β

M X
M
X
i=1 j=10σ̂ ij (yi − fi (β, X)) (yj − fj (β, X))(10.15)where σ ij denotes the ijth element of Σ−1 . The nonlinear FGLS estimatorutilizes initial estimates of σij based on minimizing (10.15) with σ̂ ij = 1.
The iterated FGLS estimator minimizes (10.15) utilizing updated estimatesof σ ij . Assuming standard regularity conditions on the functions fi the
FGLS and iterated FGLS estimators are consistent and asymptotically normally distributed with asymptotic covariance matrix given by the inverseof the empirical Hessian matrix of (10.15).

10.4.1 Analysis of Nonlinear SUR Models with the
S+FinMetrics Function NLSURThe S+FinMetrics function NLSUR may be used to estimate general nonlinear SUR models as well as linear SUR models with parameter restrictions.
The arguments for NLSUR are> args(NLSUR)
function(formulas, data, na.rm = F, coef = NULL, start = NULL,
end = NULL, control = NULL, ...)The usage of NLSUR is similar to that of SUR. The argument formulas
contains a list of nonlinear model formulas for each equation in the SUR
model. Nonlinear formulas are specified with the response on the left handside of the ~ character and the nonlinear regression function specified on theright hand side. The K parameters of the nonlinear regression functions in
the SUR model are specified explicitly by user specified coeﬃcient values.
For example, consider a “timeSeries” specified in the data argument withvariables y1, y2, y3, x1 and suppose the nonlinear SUR model has three
1 In nonlinear models the number of parameters does not have to be equal to thenumber of variables.

376

10. Systems of Regression Equations

equationsy1t= β 10 + β 11 xβ1t + ε1ty2t= β 20 + β 21 xβ1t + ε2ty3t= β 30 + β 31 xβ1t + ε3t

The vector of K = 7 parameters of the SUR system are
β = (β 10 , β 11 , β 20 , β 21 , β 30 , β 31 , β)0
Notice that the parameter β is common across the three equations. The
formulas for the three equations above could be specified in a list as> formula.list = list(y1~b10+b11*x1^b,+ y2~b20+b21*x1^b,+ y3~b30+b31*x1^b)
Notice that the user-specified coeﬃcients b10, b11, b20, b21, b30, b31,
b match the values in the parameter vector β. The common parameterβ across equations is given by the user specified coeﬃcient b. Starting
values for the coeﬃcients may be specified in the named object coef. The
parameter coeﬃcients specified in the formulas argument must match thenamed elements in coef. For example, the starting values β 10 = β 20 =β 30 = 0, β 11 = β 21 = β 31 = 1 and β = 0.5 may be specified using> start.vals = c(0,1,0,1,0,1,0.5)> names(start.vals) = c("b10","b11","b20","b21","b30",+ "b31","b")Finally, the argument control is a list variable containing control parameters for the nonlinear optimization. These control parameters are the sameas those used for the S-PLUS function nlregb. See the online help for
nlregb.control for details.
NLSUR produces an object of class “NLSUR”, that inherits from the class“SUR”, for which there are print, summary and plot methods as well as
extractor functions residuals, fitted, coef and vcov. The use of NLSUR
is illustrated with the following examples.
Example 62 Black CAPM model
Consider estimating and testing the Black form of the CAPM (10.3) using
monthly data on 16 assets over the five year period January 1983 throughDecember 1987. The real return data are in the “timeSeries” black.ts
which is constructed from the nominal return data in the “timeSeries”
berndt.dat and the consumer price data in the “timeSeries” CPI.dat.
The variables in black.ts are> colIds(black.ts)

10.4 Nonlinear Seemingly Unrelated Regression Models[1] "BOISE"[7] "DELTA"[13] "PANAM"

377"CITCRP" "CONED" "CONTIL" "DATGEN" "DEC""GENMIL" "GERBER" "IBM""MARKET" "MOBIL""PSNH""TANDY" "TEXACO" "WEYER"

The variable MARKET is a value weighted index of all stocks on the NYSE
and AMEX. The system (10.3) imposes M = 16 nonlinear cross equation
restrictions on the intercept parameters: αi = (1 − β i )γ. As a result, theparameter vector β has K = 17 elements: β = (γ, β 1 , . . . , β 16 )0 . A listof formulas for the 16 nonlinear regressions imposing the cross equation
restrictions αi = (1 − β i )γ is> formula.list = list(BOISE~(1-b1)*g + b1*MARKET,+ CITCRP~(1-b2)*g + b2*MARKET,+ CONED~(1-b3)*g + b3*MARKET,+ CONTIL~(1-b4)*g + b4*MARKET,+ DATGEN~(1-b5)*g + b5*MARKET,+ DEC~(1-b6)*g + b6*MARKET,+ DELTA~(1-b7)*g + b7*MARKET,+ GENMIL~(1-b8)*g + b8*MARKET,+ GERBER~(1-b9)*g + b9*MARKET,+ IBM~(1-b10)*g + b10*MARKET,+ MOBIL~(1-b11)*g + b11*MARKET,+ PANAM~(1-b12)*g + b12*MARKET,+ PSNH~(1-b13)*g + b13*MARKET,+ TANDY~(1-b14)*g + b14*MARKET,+ TEXACO~(1-b15)*g + b15*MARKET,+ WEYER~(1-b16)*g + b16*MARKET)The user specified coeﬃcients g, b1, . . . , b16 represent the elements ofthe parameter vector β, and the cross equation restrictions are imposedby expressing each intercept coeﬃcient as the function (1-bi)*g for i =1, . . . , 16. The starting values γ = 0 and β i = 1 (i = 1, . . . 16) for theestimation may be specified using> start.vals = c(0,rep(1,16))> names(start.vals) = c("g",paste("b",1:16,sep=""))The FGLS nonlinear SUR estimator is computed using NLSUR> nlsur.fit = NLSUR(formula.list,data=black.ts,+ coef=start.vals,start="Jan 1983",in.format="%m %Y")
> class(nlsur.fit)[1] "NLSUR"
The components of an “NLSUR” object are> names(nlsur.fit)[1] "coef""objective"[5] "iterations" "r.evals""message""j.evals""grad.norm""scale"378

10. Systems of Regression Equations[9] "cov"[13] "residuals""call""fitted""parm.list""Sigma""X.k"

The message component indicates that the nonlinear optimation converged:> nlsur.fit$message[1] "RELATIVE FUNCTION CONVERGENCE"
Since “NLSUR” objects inherit from “SUR” objects the print, summary andplot methods for “NLSUR” objects are identical to those for “SUR” objects.
The print method gives basic fit information:> nlsur.fit
Nonlinear Seemingly Unrelated Regression:
Eq. 1: BOISE ~(1 - b1) * g + b1 * MARKET
Coefficients:b1
g1.0120 0.0085
Degrees of freedom: 60 total; 58 residual
Time period: from Jan 1983 to Dec 1987
Residual scale estimate: 0.065
Eq. 2: CITCRP ~(1 - b2) * g + b2 * MARKET
Coefficients:b2
g1.0699 0.0085
Degrees of freedom: 60 total; 58 residual
Time period: from Jan 1983 to Dec 1987
Residual scale estimate: 0.0581...
Eq. 16: WEYER ~(1 - b16) * g + b16 * MARKET
Coefficients:b16
g1.0044 0.0085
Degrees of freedom: 60 total; 58 residual
Time period: from Jan 1983 to Dec 1987
Residual scale estimate: 0.0574

10.4 Nonlinear Seemingly Unrelated Regression Models

379

Log determinant of residual covariance: -85.29
The estimated coeﬃcients and their standard errors may be extracted using
coef and vcov:> std.ers = sqrt(diag(vcov(nlsur.fit)))> cbind(coef(nlsur.fit),std.ers)
numeric matrix: 17 rows, 2 columns.
std.ersg 0.008477 0.004642
b1 1.012043 0.134400
b2 1.069874 0.119842
b3 0.028169 0.104461
b4 1.479293 0.361368
b5 1.133384 0.218596
b6 1.099063 0.195965
b7 0.704410 0.182686
b8 0.547502 0.127458
b9 0.960858 0.157903
b10 0.649761 0.096975
b11 0.741609 0.121843
b12 0.715984 0.265048
b13 0.205356 0.348101
b14 1.054715 0.185498
b15 0.574735 0.145838
b16 1.004 0.1186
More detailed information about the fit may be viewed using summary, anda graphical analysis of the fit may be created using plot. For example,
Figure 10.2 shows the residuals from the 16 nonlinear equations.
The nonlinear restrictions implied by the Black form of the CAPM may
be tested using a LR statistic. The unrestricted model (10.4) is specified
using the formula list
> formula.list = list(BOISE~a1+b1*MARKET,+ CITCRP~a2+b2*MARKET,+ CONED~a3+b3*MARKET,+ CONTIL~a4+b4*MARKET,+ DATGEN~a5+b5*MARKET,+ DEC~a6+b6*MARKET,+ DELTA~a7+b7*MARKET,+ GENMIL~a8+b8*MARKET,+ GERBER~a9+b9*MARKET,+ IBM~a10+b10*MARKET,+ MOBIL~a11+b11*MARKET,+ PANAM~a12+b12*MARKET,

380

10. Systems of Regression Equations

Residuals versus Time

-0.1

-0.2

IBM

MOBIL

PANAM

DELTA

GENMIL

0.0

0.2

DEC

0.0

DATGEN-0.2
CITCRP

1983 1984 1985 1986 1987 1988

CONED

CONTIL

0

0

0

BOISE

0

-0.2

-0.2

0

0.0

0.2

-0.1

-0.1

0

0.0

0

0.0

0.1

GERBER

WEYER

0.0

0.0

TEXACO

-0.2

0.0

0.0
-0.5

1983 1984 1985 1986 1987 1988

TANDY

0.1

1983 1984 1985 1986 1987 1988
PSNH

1983 1984 1985 1986 1987 1988

FIGURE 10.2. Estimated residuals from nonlinear SUR fit to Black’s form of the
CAPM.

+ PSNH~a13+b13*MARKET,+ TANDY~a14+b14*MARKET,+ TEXACO~a15+b15*MARKET,+ WEYER~a16+b16*MARKET)
and is estimated using NLSUR with the starting values αi = 0 and β i = 1(i = 1, . . . , 16):
> start.vals = c(rep(0,16),rep(1,16))> names(start.vals) =+ c(paste("a",1:16,sep=""),paste("b",1:16,sep=""))> nlsur.fit2 = NLSUR(formula.list,data=black.ts,+ coef=start.vals,start="Jan 1983",in.format="%m %Y")
The LR statistic for testing the M = 16 nonlinear cross equation restrictions αi = (1 − β i )γ is computed using> nobs = nrow(residuals(nlsur.fit2))> LR = nobs*(determinant(nlsur.fit$Sigma,log=T)$modulus+ determinant(nlsur.fit2$Sigma,log=T)$modulus)> as.numeric(LR)[1] 15.86> 1-pchisq(LR,16)[1] 0.4625

10.4 Nonlinear Seemingly Unrelated Regression Models

381

The p-value of the test is 0.4627, and so the Black CAPM restrictions are
not rejected at any reasonable significance level.
Example 63 Estimation of exchange rate system with cross equation parameter restrictions
Consider estimating the system of exchange rates (10.2), using the data
described in the previous section, imposing the cross equation restrictionthat γ 1 = · · · = γ M = γ. The list of formulas for this restricted system
may be constructed as> formula.list = list(USCNS.diff~a1+g*USCN.FP.lag1,+ USDMS.diff~a2+g*USDM.FP.lag1,+ USFRS.diff~a3+g*USFR.FP.lag1,+ USILS.diff~a4+g*USIL.FP.lag1,+ USJYS.diff~a5+g*USJY.FP.lag1,+ USUKS.diff~a6+g*USUK.FP.lag1)
Notice that the common parameter β is captured by the user-specified
coeﬃcient g. The starting values are chosen to be α1 = · · · = α6 = 0 and
γ = 1 and are specified using> start.vals = c(rep(0,6),1)> names(start.vals) = c(paste("a",1:6,sep=""),"g")The FGLS estimator is computed using NLSUR> nlsur.fit = NLSUR(formula.list, data=surex1.ts,+ coef=start.vals, start="Aug 1978", in.format="%m %Y")
> nlsur.fit
Nonlinear Seemingly Unrelated Regression:
Eq. 1: USCNS.diff ~a1 + g * USCN.FP.lag1
Coefficients:
a1
g-0.0005 0.3467
Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.0138...
Eq. 6: USUKS.diff ~a6 + g * USUK.FP.lag1
Coefficients:

382

10. Systems of Regression Equations

a6-0.0002g
0.3467

Degrees of freedom: 215 total; 213 residual
Time period: from Aug 1978 to Jun 1996
Residual scale estimate: 0.035
Log determinant of residual covariance: -47.679
The estimate of the common parameter γ is 0.3467, and its asymptotic
standard error is> sqrt(diag(vcov(nlsur.fit)))[7][1] 0.16472
Hence, the data indicate that the common value of γ is less than 1. The
LR statistic, however, rejects the common parameter restriction> nobs = nrow(residuals(nlsur.fit))> LR = nobs*(determinant(nlsur.fit$Sigma,log=T)$modulus+ -determinant(sur.fit2$Sigma,log=T)$modulus)> as.numeric(LR)[1] 55.65> 1 - pchisq(LR,6)[1] 3.433e-010

10.5 References
Black, F. (1972). “Capital Market Equilibrium with Restricted Borrowing,” Journal of Business, 44, 444-454.
Burmeister, E. and M.B. McElroy (1988). “Arbitrage Pricing Theory as a Restricted Nonlinear Multivariate Regression Model: ITNLSUR
Estimates,” Journal of Business and Economic Statistics, vol. 6(1), 28-42.
Campbell, J. A. Lo and C. MacKinlay (1997). The Econometrics ofFinancial Markets. Princeton University Press, Princeton, NJ.
Gallant, R.A. (1974). “Seemingly Unrelated Nonlinear Regressions,”
Journal of Econometrics, 3, 35-50.
Greene, W. (2000). Econometric Analysis, Fourth Edition. Prentice Hall,
New Jersey.
Srivastava, V.K. and D.E.A. Giles (1987). Seemingly Unrelated Regression Models: Estimation and Inference. Marcel Dekker, New York.

10.5 References

383

Theil, H. (1961). Economic Forecasts and Policy. North Holland, Amsterdam.
Zellner, A. (1962). “An Eﬃcient Method of Estimating Seemingly Unrelated Regressions and Tests of Aggregation Bias,” Journal of the American
Statistical Association, 57, 500-509.

This is page 384
Printer: Opaque thisThis is page 385
Printer: Opaque this11
Vector Autoregressive Models forMultivariate Time Series

11.1 Introduction
The vector autoregression (VAR) model is one of the most successful, flexible, and easy to use models for the analysis of multivariate time series. It isa natural extension of the univariate autoregressive model to dynamic multivariate time series. The VAR model has proven to be especially useful for
describing the dynamic behavior of economic and financial time series andfor forecasting. It often provides superior forecasts to those from univariate time series models and elaborate theory-based simultaneous equations
models. Forecasts from VAR models are quite flexible because they can bemade conditional on the potential future paths of specified variables in themodel.
In addition to data description and forecasting, the VAR model is also
used for structural inference and policy analysis. In structural analysis, certain assumptions about the causal structure of the data under investigation are imposed, and the resulting causal impacts of unexpected shocks orinnovations to specified variables on the variables in the model are summarized. These causal impacts are usually summarized with impulse response
functions and forecast error variance decompositions.
This chapter focuses on the analysis of covariance stationary multivariate time series using VAR models. The following chapter describes theanalysis of nonstationary multivariate time series using VAR models thatincorporate cointegration relationships.

386

11. Vector Autoregressive Models for Multivariate Time SeriesThis chapter is organized as follows. Section 11.2 describes specification,
estimation and inference in VAR models and introduces the S+FinMetrics
function VAR. Section 11.3 covers forecasting from VAR model. The discussion covers traditional forecasting algorithms as well as simulation-based
forecasting algorithms that can impose certain types of conditioning information. Section 11.4 summarizes the types of structural analysis typically
performed using VAR models. These analyses include Granger-causality
tests, the computation of impulse response functions, and forecast error
variance decompositions. Section 11.5 gives an extended example of VAR
modeling. The chapter concludes with a brief discussion of Bayesian VAR
models.
This chapter provides a relatively non-technical survey of VAR models.
VAR models in economics were made popular by Sims (1980). The definitivetechnical reference for VAR models is Lütkepohl (1991), and updated surveys of VAR techniques are given in Watson (1994) and Lütkepohl (1999)
and Waggoner and Zha (1999). Applications of VAR models to financial
data are given in Hamilton (1994), Campbell, Lo and MacKinlay (1997),
Cuthbertson (1996), Mills (1999) and Tsay (2001).

11.2 The Stationary Vector Autoregression Model
Let Yt = (y1t , y2t , . . . , ynt )0 denote an (n×1) vector of time series variables.
The basic p-lag vector autoregressive (VAR(p)) model has the form
Yt = c + Π1 Yt−1 + Π2 Yt−2 + · · · + Πp Yt−p + εt , t = 1, . . . , T(11.1)

where Πi are (n × n) coeﬃcient matrices and εt is an (n × 1) unobservablezero mean white noise vector process (serially uncorrelated or independent)with time invariant covariance matrix Σ. For example, a bivariate VAR(2)model equation by equation has the form
µ
¶
µ
¶ µ 1
¶µ
¶
y1t
c1
π 11 π112y1t−1=+(11.2)y2t
c2
π 121 π122
y2t−1
µ 2
¶µ
¶ µ
¶
π11 π212
y1t−2
ε1t
+
+
(11.3)π221 π222
y2t−2
ε2t
or
y1t
y2t

= c1 + π 111 y1t−1 + π 112 y2t−1 + π211 y1t−2 + π212 y2t−2 + ε1t= c2 + π 121 y1t−1 + π 122 y2t−1 + π221 y1t−1 + π222 y2t−1 + ε2twhere cov(ε1t , ε2s ) = σ 12 for t = s; 0 otherwise. Notice that each equation
has the same regressors — lagged values of y1t and y2t . Hence, the VAR(p)model is just a seemingly unrelated regression (SUR) model with lagged
variables and deterministic terms as common regressors.

11.2 The Stationary Vector Autoregression Model387

In lag operator notation, the VAR(p) is written as
Π(L)Yt = c + εtwhere Π(L) = In − Π1 L − ... − Πp Lp . The VAR(p) is stable if the roots ofdet (In − Π1 z − · · · − Πp z p ) = 0
lie outside the complex unit circle (have modulus greater than one), or,equivalently, if the eigenvalues of the companion matrix⎞⎛
Π1 Π2 · · · Πn
⎜ In
0 ···
0 ⎟
⎟
⎜
F=⎜
.. ⎟
.
..
⎝ 0
0
. ⎠
0
0
0
In

have modulus less than one. Assuming that the process has been initialized
in the infinite past, then a stable VAR(p) process is stationary and ergodic
with time invariant means, variances, and autocovariances.
If Yt in (11.1) is covariance stationary, then the unconditional mean isgiven byμ = (In − Π1 − · · · − Πp )−1 c
The mean-adjusted form of the VAR(p) is thenYt − μ = Π1 (Yt−1 − μ) + Π2 (Yt−2 − μ) + · · · + Πp (Yt−p − μ) + εt
The basic VAR(p) model may be too restrictive to represent suﬃciently
the main characteristics of the data. In particular, other deterministic terms
such as a linear time trend or seasonal dummy variables may be required
to represent the data properly. Additionally, stochastic exogenous variables
may be required as well. The general form of the VAR(p) model with deterministic terms and exogenous variables is given by
Yt = Π1 Yt−1 + Π2 Yt−2 + · · · + Πp Yt−p + ΦDt + GXt + εt(11.4)

where Dt represents an (l × 1) matrix of deterministic components, Xt
represents an (m × 1) matrix of exogenous variables, and Φ and G areparameter matrices.
Example 64 Simulating a stationary VAR(1) model using S-PLUS
A stationary VAR model may be easily simulated in S-PLUS using the
S+FinMetrics function simulate.VAR. The commands to simulate T =250 observations from a bivariate VAR(1) model
y1t
y2t= −0.7 + 0.7y1t−1 + 0.2y2t−1 + ε1t= 1.3 + 0.2y1t−1 + 0.7y2t−1 + ε2t

388

11. Vector Autoregressive Models for Multivariate Time Series

with
Π1 =

µ

0.7 0.2
0.2 0.7

¶

, c=

µ

−0.7
1.3

¶

, μ=

µ

1
5

¶

, Σ=

µ

1 0.5
0.5 1

¶

and normally distributed errors are> pi1 = matrix(c(0.7,0.2,0.2,0.7),2,2)> mu.vec = c(1,5)> c.vec = as.vector((diag(2)-pi1)%*%mu.vec)> cov.mat = matrix(c(1,0.5,0.5,1),2,2)> var1.mod = list(const=c.vec,ar=pi1,Sigma=cov.mat)> set.seed(301)> y.var = simulate.VAR(var1.mod,n=250,+ y0=t(as.matrix(mu.vec)))> dimnames(y.var) = list(NULL,c("y1","y2"))
The simulated data are shown in Figure 11.1. The VAR is stationary since
the eigenvalues of Π1 are less than one:> eigen(pi1,only.values=T)$values:[1] 0.9 0.5$vectors:
NULL
Notice that the intercept values are quite diﬀerent from the mean values ofy1 and y2 :> c.vec[1] -0.7 1.3> colMeans(y.var)y1y2
0.8037 4.751

11.2.1 Estimation
Consider the basic VAR(p) model (11.1). Assume that the VAR(p) model
is covariance stationary, and there are no restrictions on the parameters ofthe model. In SUR notation, each equation in the VAR(p) may be writtenasyi = Zπ i + ei , i = 1, . . . , n
where yi is a (T × 1) vector of observations on the ith equation, Z is00a (T × k) matrix with tth row given by Z0t = (1, Yt−1
, . . . , Yt−p), k =
np + 1, π i is a (k × 1) vector of parameters and ei is a (T × 1) error withcovariance matrix σ 2i IT . Since the VAR(p) is in the form of a SUR model

389

10

11.2 The Stationary Vector Autoregression Model

4
-4

-2

0

2

y1,y2

6

8

y1
y2

0

50

100

150

200

250

time

FIGURE 11.1. Simulated stationary VAR(1) model.

where each equation has the same explanatory variables, each equation may
be estimated separately by ordinary least squares without losing eﬃciency
relative to generalized least squares. Let Π̂ = [π̂ 1 , . . . , π̂ n ] denote the (k×n)
matrix of least squares coeﬃcients for the n equations.
Let vec(Π̂) denote the operator that stacks the columns of the (n × k)
matrix Π̂ into a long (nk × 1) vector. That is,
⎛
⎞
π̂ 1
⎜
⎟
vec(Π̂) = ⎝ ... ⎠π̂ nUnder standard assumptions regarding the behavior of stationary and ergodic VAR models (see Hamilton (1994) or Lütkepohl (1991)) vec(Π̂) is
consistent and asymptotically normally distributed with asymptotic covariance matrix0avar(vec(dΠ̂)) = Σ̂ ⊗ (Z Z)−1

where

Σ̂ =

T
1 X
ε̂t ε̂0t
T − k t=1

and ε̂t = Yt − Π̂0 Zt is the multivariate least squares residual from (11.1)
at time t.

390

11. Vector Autoregressive Models for Multivariate Time Series

11.2.2 Inference on Coeﬃcients
The ith element of vec(Π̂), π̂ i , is asymptotically normally distributed withasymptotic standard error given by the square root of ith diagonal element0of Σ̂ ⊗ (Z Z)−1 . Hence, asymptotically valid t-tests on individual coeﬃcients may be constructed in the usual way. More general linear hypothesesof the form R·vec(Π) = r involving coeﬃcients across diﬀerent equationsof the VAR may be tested using the Wald statistic
n h
i o−1
Wald = (R·vec(Π̂) − r)0 R avar(vec((R·vec(Π̂) − r) (11.5)dΠ̂)) R0

Under the null, (11.5) has a limiting χ2 (q) distribution where q = rank(R)gives the number of linear restrictions.

11.2.3 Lag Length Selection
The lag length for the VAR(p) model may be determined using model
selection criteria. The general approach is to fit VAR(p) models with orders
p = 0, ..., pmax and choose the value of p which minimizes some model
selection criteria. Model selection criteria for VAR(p) models have the form
IC(p) = ln |Σ̃(p)| + cT · ϕ(n, p)P
where Σ̃(p) = T −1 Tt=1 ε̂t ε̂0t is the residual covariance matrix without a degrees of freedom correction from a VAR(p) model, cT is a sequence indexedby the sample size T , and ϕ(n, p) is a penalty function which penalizes
large VAR(p) models. The three most common information criteria are the
Akaike (AIC), Schwarz-Bayesian (BIC) and Hannan-Quinn (HQ):
2 2pn
Tln T 2BIC(p) = ln |Σ̃(p)| +pn
T
2 ln ln T 2
HQ(p) = ln |Σ̃(p)| +pnTAIC(p) = ln |Σ̃(p)| +The AIC criterion asymptotically overestimates the order with positive
probability, whereas the BIC and HQ criteria estimate the order consistently under fairly general conditions if the true order p is less than orequal to pmax . For more information on the use of model selection criteria
in VAR models see Lütkepohl (1991) chapter four.

11.2.4 Estimating VAR Models Using the S+FinMetrics
Function VAR
The S+FinMetrics function VAR is designed to fit and analyze VAR models
as described in the previous section. VAR produces an object of class “VAR”

11.2 The Stationary Vector Autoregression Model391

for which there are print, summary, plot and predict methods as well
as extractor functions coefficients, residuals, fitted and vcov. The
calling syntax of VAR is a bit complicated because it is designed to handle
multivariate data in matrices, data frames as well as “timeSeries” objects.
The use of VAR is illustrated with the following example.
Example 65 Bivariate VAR model for exchange rates
This example considers a bivariate VAR model for Yt = (∆st , f pt )0 ,where st is the logarithm of the monthly spot exchange rate between the US
CA
and Canada, f pt = ft − st = iUSis the forward premium or interestt − itrate diﬀerential, and ft is the natural logarithm of the 30-day forward
exchange rate. The data over the 20 year period March 1976 through June
1996 is in the S+FinMetrics “timeSeries” lexrates.dat. The data for
the VAR model are computed as> dspot = diff(lexrates.dat[,"USCNS"])> fp = lexrates.dat[,"USCNF"]-lexrates.dat[,"USCNS"]> uscn.ts = seriesMerge(dspot,fp)> colIds(uscn.ts) = c("dspot","fp")> uscn.ts@title = "US/CN Exchange Rate Data"> par(mfrow=c(2,1))> plot(uscn.ts[,"dspot"],main="1st difference of US/CA spot
+ exchange rate")> plot(uscn.ts[,"fp"],main="US/CN interest rate
+ differential")
Figure 11.2 illustrates the monthly return ∆st and the forward premiumf pt over the period March 1976 through June 1996. Both series appear to beI(0) (which can be confirmed using the S+FinMetrics functions unitrootor stationaryTest) with ∆st much more volatile than f pt . f pt also appears to be heteroskedastic.
Specifying and Estimating the VAR(p) ModelTo estimate a VAR(1) model for Yt use> var1.fit = VAR(cbind(dspot,fp)~ar(1),data=uscn.ts)
Note that the VAR model is specified using an S-PLUS formula, with themultivariate response on the left hand side of the ~ operator and the builtin AR term specifying the lag length of the model on the right hand side.
The optional data argument accepts a data frame or “timeSeries” object with variable names matching those used in specifying the formula.
If the data are in a “timeSeries” object or in an unattached data frame(“timeSeries” objects cannot be attached) then the data argument must
be used. If the data are in a matrix then the data argument may be omitted. For example,

392

11. Vector Autoregressive Models for Multivariate Time Series

-0.06

-0.02

0.02

1st difference of US/CN spot exchange rate

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

-0.005-0.001

0.003

US/CN interest rate differential

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

FIGURE 11.2. US/CN forward premium and spot rate.

> uscn.mat = as.matrix(seriesData(uscn.ts))> var2.fit = VAR(uscn.mat~ar(1))
If the data are in a “timeSeries” object then the start and end options
may be used to specify the estimation sample. For example, to estimate theVAR(1) over the sub-period January 1980 through January 1990
> var3.fit = VAR(cbind(dspot,fp)~ar(1), data=uscn.ts,+ start="Jan 1980", end="Jan 1990", in.format="%m %Y")
may be used. The use of in.format=\%m %Y" sets the format for the datestrings specified in the start and end options to match the input formatof the dates in the positions slot of uscn.ts.
The VAR model may be estimated with the lag length p determined using
a specified information criterion. For example, to estimate the VAR for theexchange rate data with p set by minimizing the BIC with a maximum lag
pmax = 4 use> var4.fit = VAR(uscn.ts,max.ar=4, criterion="BIC")> var4.fit$info
ar(1) ar(2) ar(3) ar(4)BIC -4028 -4013 -3994 -3973
When a formula is not specified and only a data frame, “timeSeries” ormatrix is supplied that contains the variables for the VAR model, VAR fits

11.2 The Stationary Vector Autoregression Model393

all VAR(p) models with lag lengths p less than or equal to the value given
to max.ar, and the lag length is determined as the one which minimizes
the information criterion specified by the criterion option. The default
criterion is BIC but other valid choices are logL, AIC and HQ. In the computation of the information criteria, a common sample based on max.ar
is used. Once the lag length is determined, the VAR is re-estimated using the appropriate sample. In the above example, the BIC values werecomputed using the sample based on max.ar=4 and p = 1 minimizes BIC.
The VAR(1) model was automatically re-estimated using the sample size
appropriate for p = 1.
Print and Summary Methods
The function VAR produces an object of class “VAR” with the following
components.
> class(var1.fit)[1] "VAR"> names(var1.fit)[1] "R""coef"[5] "Sigma""df.resid"[9] "ar.order" "n.na""fitted""rank""terms""residuals""call""Y0"

To see the estimated coeﬃcients of the model use the print method:> var1.fit
Call:
VAR(formula = cbind(dspot, fp) ~ar(1), data = uscn.ts)
Coefficients:
dspot
fp(Intercept) -0.0036 -0.0003
dspot.lag1 -0.1254 0.0079
fp.lag1 -1.4833 0.7938
Std. Errors of Residuals:
dspot
fp
0.0137 0.0009
Information Criteria:
logL
AIC
BIC
HQ
2058 -4104 -4083 -4096
total residual
Degree of freedom:
243
240
Time period: from Apr 1976 to Jun 1996

394

11. Vector Autoregressive Models for Multivariate Time Series

The first column under the label “Coefficients:” gives the estimatedcoeﬃcients for the ∆st equation, and the second column gives the estimatedcoeﬃcients for the f pt equation:
∆st
f pt

= −0.0036 − 0.1254 · ∆st−1 − 1.4833 · f pt−1
= −0.0003 + 0.0079 · ∆st−1 + 0.7938 · f pt−1

Since uscn.ts is a “timeSeries” object, the estimation time period is also
displayed.
The summary method gives more detailed information about the fitted
VAR:> summary(var1.fit)
Call:
VAR(formula = cbind(dspot, fp) ~ar(1), data = uscn.ts)
Coefficients:(Intercept)(std.err)(t.stat)

dspot-0.00360.0012-2.9234

fp-0.00030.0001-3.2885

dspot.lag1(std.err)(t.stat)-0.12540.0637-1.9700

0.00790.0042
1.8867

fp.lag1(std.err)(t.stat)-1.4833
0.5980-2.4805

0.79380.039520.1049

Regression Diagnostics:
dspot
fp
R-squared 0.0365 0.6275
Adj. R-squared 0.0285 0.6244
Resid. Scale 0.0137 0.0009
Information Criteria:
logL
AIC
BIC
HQ
2058 -4104 -4083 -4096
total residual
Degree of freedom:
243
240
Time period: from Apr 1976 to Jun 1996
In addition to the coeﬃcient standard errors and t-statistics, summary also
displays R2 measures for each equation (which are valid because each equa-11.2 The Stationary Vector Autoregression Model

395

tion is estimated by least squares). The summary output shows that thecoeﬃcients on ∆st−1 and f pt−1 in both equations are statistically significant at the 10% level and that the fit for the f pt equation is much better
than the fit for the ∆st equation.
As an aside, note that the S+FinMetrics function OLS may also be used
to estimate each equation in a VAR model. For example, one way to compute the equation for ∆st using OLS is> dspot.fit = OLS(dspot~ar(1)+tslag(fp),data=uscn.ts)> dspot.fit
Call:
OLS(formula = dspot ~ar(1) + tslag(fp), data = uscn.ts)
Coefficients:(Intercept) tslag(fp)lag1-0.0036-1.4833-0.1254
Degrees of freedom: 243 total; 240 residual
Time period: from Apr 1976 to Jun 1996
Residual standard error: 0.01373
Graphical Diagnostics
The plot method for “VAR” objects may be used to graphically evaluate thefitted VAR. By default, the plot method produces a menu of plot options:> plot(var1.fit)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Response and Fitted Values
3: plot: Residuals
4: plot: Normal QQplot of Residuals
5: plot: ACF of Residuals
6: plot: PACF of Residuals
7: plot: ACF of Squared Residuals
8: plot: PACF of Squared Residuals
Selection:
Alternatively, plot.VAR may be called directly. The function plot.VAR
has arguments> args(plot.VAR)
function(x, ask = T, which.plots = NULL, hgrid = F, vgrid= F, ...)

396

11. Vector Autoregressive Models for Multivariate Time Series

Response and Fitted Values-0.004

0.000

0.004

fp

-0.06

-0.04

-0.02

0.00

0.02

dspot

1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

FIGURE 11.3. Response and fitted values from VAR(1) model for US/CN exchange rate data.

To create all seven plots without using the menu, set ask=F. To create
the Residuals plot without using the menu, set which.plot=2. The optional
arguments hgrid and vgrid control printing of horizontal and vertical gridlines on the plots.
Figures 11.3 and 11.4 give the Response and Fitted Values and Residuals
plots for the VAR(1) fit to the exchange rate data. The equation for f pt fits
much better than the equation for ∆st . The residuals for both equations
look fairly random, but the residuals for the f pt equation appear to beheteroskedastic. The qq-plot (not shown) indicates that the residuals forthe ∆st equation are highly non-normal.
Extractor Functions
The residuals and fitted values for each equation of the VAR may be extracted using the generic extractor functions residuals and fitted:> var1.resid = resid(var1.fit)> var1.fitted = fitted(var.fit)> var1.resid[1:3,]
Positionsdspot
fp
Apr 1976 0.0044324 -0.00084150
May 1976 0.0024350 -0.00026493
Jun 1976 0.0004157 0.00002435

11.2 The Stationary Vector Autoregression Model

397

Residuals versus Time

-0.004-0.002

0.000

0.002

fp

-0.06

-0.04

-0.02

0.00

0.02

dspot

1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

FIGURE 11.4. Residuals from VAR(1) model fit to US/CN exchange rate data.

Notice that since the data are in a “timeSeries” object, the extracted
residuals and fitted values are also “timeSeries” objects.
The coeﬃcients of the VAR model may be extracted using the genericcoef function:> coef(var1.fit)
dspot
fp(Intercept) -0.003595149 -0.0002670108
dspot.lag1 -0.125397056 0.0079292865
fp.lag1 -1.483324622 0.7937959055
Notice that coef produces the (3 × 2) matrix Π̂ whose columns give theestimated coeﬃcients for each equation in the VAR(1).
To test stability of the VAR, extract the matrix Π1 and compute its
eigenvalues> PI1 = t(coef(var1.fit)[2:3,])> abs(eigen(PI1,only.values=T)$values)[1] 0.7808 0.1124
Since the modulus of the two eigenvalues of Π1 are less than 1, the VAR(1)is stable.

398

11. Vector Autoregressive Models for Multivariate Time Series

Testing Linear Hypotheses
Now, consider testing the hypothesis that Π1 = 0 (i.e., Yt−1 does not help
to explain Yt ) using the Wald statistic (11.5). In terms of the columns ofvec(Π) the restrictions are π 1 = (c1 , 0, 0)0 and π 2 = (c2 , 0, 0) and may be
expressed as Rvec(Π) = r with
⎞
⎛ ⎞
⎛
0
0 1 0 0 0 0⎜ 0 ⎟⎜ 0 0 1 0 0 0 ⎟
⎟
⎜ ⎟
R=⎜
⎝ 0 0 0 0 1 0 ⎠,r = ⎝ 0 ⎠0
0 0 0 0 0 1
The Wald statistic is easily constructed as follows> R = matrix(c(0,1,0,0,0,0,+ 0,0,1,0,0,0,+ 0,0,0,0,1,0,+ 0,0,0,0,0,1),+ 4,6,byrow=T)> vecPi = as.vector(var1.fit$coef)> avar = R%*%vcov(var1.fit)%*%t(R)> wald = t(R%*%vecPi)%*%solve(avar)%*%(R%*%vecPi)> wald[,1][1,] 417.1> 1-pchisq(wald,4)[1] 0
Since the p-value for the Wald statistic based on the χ2 (4) distribution
is essentially zero, the hypothesis that Π1 = 0 should be rejected at any
reasonable significance level.

11.3 Forecasting
Forecasting is one of the main objectives of multivariate time series analysis.
Forecasting from a VAR model is similar to forecasting from a univariate
AR model and the following gives a brief description.

11.3.1 Traditional Forecasting Algorithm
Consider first the problem of forecasting future values of Yt when theparameters Π of the VAR(p) process are assumed to be known and there
are no deterministic terms or exogenous variables. The best linear predictor,
in terms of minimum mean squared error (MSE), of Yt+1 or 1-step forecast
based on information available at time T is
YT +1|T = c + Π1 YT + · · · + Πp YT −p+1

11.3 Forecasting

399

Forecasts for longer horizons h (h-step forecasts) may be obtained using
the chain-rule of forecasting as
YT +h|T = c + Π1 YT +h−1|T + · · · + Πp YT +h−p|T
where YT +j|T = YT +j for j ≤ 0. The h-step forecast errors may be expressed as
h−1
X
YT +h − YT +h|T =Ψs εT +h−ss=0where the matrices Ψs are determined by recursive substitution
Ψs =

p−1
X

Ψs−j Πj(11.6)

j=1

with Ψ0 = In and Πj = 0 for j > p.1 The forecasts are unbiased since all of
the forecast errors have expectation zero and the MSE matrix for Yt+h|Tis¡¢Σ(h) = MSE YT +h − YT +h|T
=

h−1
X

Ψs ΣΨ0s(11.7)

s=0

Now consider forecasting YT +h when the parameters of the VAR(p)process are estimated using multivariate least squares. The best linear predictor of YT +h is now
ŶT +h|T = Π̂1 ŶT +h−1|T + · · · + Π̂p ŶT +h−p|T(11.8)

where Π̂j are the estimated parameter matrices. The h-step forecast error
is now
YT +h − ŶT +h|T =

h−1
X
s=0

´
³
Ψs εT +h−s + YT +h − ŶT +h|T

(11.9)´³and the term YT +h − ŶT +h|T captures the part of the forecast error due
to estimating the parameters of the VAR. The MSE matrix of the h-step
forecast is then³´Σ̂(h) = Σ(h) + MSE YT +h − ŶT +h|T
1 The S+FinMetrics fucntion VAR.ar2ma computes the Ψ matrices given the Π masjtrices using (11.6).

400

11. Vector Autoregressive Models for Multivariate Time Series³´
In practice, the second term MSE YT +h − ŶT +h|T is often ignored andΣ̂(h) is computed using (11.7) asΣ̂(h) =h−1X0

Ψ̂s Σ̂Ψ̂s(11.10)s=0with Ψ̂s =PsΨ̂ Π̂ . Lütkepohl (1991, Chap. 3) gave an approxij=1³ s−j j´mation to MSE YT +h − ŶT +h|T which may be interpreted as a finite
sample correction to (11.10).
Asymptotic (1−α)·100% confidence intervals for the individual elementsof ŶT +h|T are then computed as¤£ŷk,T +h|T − c1−α/2 σ̂ k (h), ŷk,T +h|T + c1−α/2 σ̂ k (h)where c1−α/2 is the (1 − α/2) quantile of the standard normal distributionand σ̂ k (h) denotes the square root of the diagonal element of Σ̂(h).
Example 66 Forecasting exchange rates from a bivariate VAR
Consider computing h-step forecasts, h = 1, . . . , 12, along with estimatedforecast standard errors from the bivariate VAR(1) model for exchange
rates. Forecasts and forecast standard errors from the fitted VAR may becomputed using the generic S-PLUS predict method> uscn.pred = predict(var1.fit,n.predict=12)
The predict function recognizes var1.fit as a “VAR” object, and calls theappropriate method function predict.VAR. Alternatively, predict.VAR
can be applied directly on an object inheriting from class “VAR”. See theonline help for explanations of the arguments to predict.VAR.
The output of predict.VAR is an object of class “forecast” for which
there are print, summary and plot methods. To see just the forecasts, theprint method will suﬃce:> uscn.pred
Predicted Values:
dspot
fp
1-step-ahead -0.0027 -0.0005
2-step-ahead -0.0026 -0.0006
3-step-ahead -0.0023 -0.0008
4-step-ahead -0.0021 -0.0009
5-step-ahead -0.0020 -0.0010
6-step-ahead -0.0018 -0.0011
7-step-ahead -0.0017 -0.0011

11.3 Forecasting401

8-step-ahead -0.0017 -0.0012
9-step-ahead -0.0016 -0.0012
10-step-ahead -0.0016 -0.0013
11-step-ahead -0.0015 -0.0013
12-step-ahead -0.0015 -0.0013
The forecasts and their standard errors can be shown using summary:> summary(uscn.pred)
Predicted Values with Standard Errors:
dspot
fp
1-step-ahead -0.0027 -0.0005(std.err) 0.0137 0.0009
2-step-ahead -0.0026 -0.0006(std.err) 0.0139 0.0012...
12-step-ahead -0.0015 -0.0013(std.err) 0.0140 0.0015
Lütkepohl’s finite sample correction to the forecast standard errors computed from asymptotic theory may be obtained by using the optional argument fs.correction=T in the call to predict.VAR.
The forecasts can also be plotted together with the original data using
the generic plot function as follows:> plot(uscn.pred,uscn.ts,n.old=12)
where the n.old optional argument specifies the number of observations toplot from uscn.ts. If n.old is not specified, all the observations in uscn.ts
will be plotted together with uscn.pred. Figure 11.5 shows the forecasts
produced from the VAR(1) fit to the US/CN exchange rate data2 . At the
beginning of the forecast horizon the spot return is below its estimatedmean value, and the forward premium is above its mean values. The spot
return forecasts start oﬀ negative and grow slowly toward the mean, and theforward premium forecasts decline sharply toward the mean. The forecast
standard errors for both sets of forecasts, however, are fairly large.
2 Notice that the dates associated with the forecasts are not shown. This is the resultof “timeDate” objects not having a well defined frequency from which to extrapolate
dates.

402

11. Vector Autoregressive Models for Multivariate Time Series

235

240

245

250

255

fp

-0.001-0.0020.0-0.01

values

0.01

0.0

dspot

235

240

245

250

255

index

FIGURE 11.5. Predicted values from VAR(1) model fit to US/CN exchange rate
data.

11.3.2 Simulation-Based Forecasting
The previous subsection showed how to generate multivariate forecasts froma fitted VAR model, using the chain-rule of forecasting (11.8). Since themultivariate forecast errors (11.9) are asymptotically normally distributedwith covariance matrix (11.10), the forecasts of Yt+h can be simulated
by generating multivariate normal random variables with mean zero andcovariance matrix (11.10). These simulation-based forecasts can be obtained by setting the optional argument method to "mc" in the call to
predict.VAR.
When method="mc", the multivariate normal random variables are actually generated as a vector of standard normal random variables scaledby the Cholesky factor of the covariance matrix (11.10). Instead of using
standard normal random variables, one could also use the standardized
residuals from the fitted VAR model. Simulation-based forecasts based on
this approach are obtained by setting the optional argument method to"bootstrap" in the call to predict.VAR.
Example 67 Simulation-based forecasts of exchange rate data from bivariate VAR
The h-step forecasts (h = 1, . . . , 12) for ∆st+h and f pt+h using the Monte
Carlo simulation method are11.3 Forecasting403> uscn.pred.MC = predict(var1.fit,n.predict=12,method="mc")> summary(uscn.pred.MC)
Predicted Values with Standard Errors:
dspot
fp
1-step-ahead -0.0032 -0.0005(std.err) 0.0133 0.0009
2-step-ahead -0.0026 -0.0006(std.err) 0.0133 0.0012...
12-step-ahead -0.0013 -0.0013(std.err) 0.0139 0.0015
The Monte Carlo forecasts and forecast standard errors for f pt+h are almost
identical to those computed using the chain-rule of forecasting. The Monte
Carlo forecasts for ∆st+h are slightly diﬀerent and the forecast standard
errors are slightly larger than the corresponding values computed from thechain-rule.
The h-step forecasts computed from the bootstrap simulation methodare> uscn.pred.boot = predict(var1.fit,n.predict=12,+ method="bootstrap")> summary(uscn.pred.boot)Predicted Values with Standard Errors:
dspot
fp
1-step-ahead -0.0020 -0.0005(std.err) 0.0138 0.0009
2-step-ahead -0.0023 -0.0007(std.err) 0.0140 0.0012...
12-step-ahead -0.0023 -0.0013(std.err) 0.0145 0.0015
As with the Monte Carlo forecasts, the bootstrap forecasts and forecast
standard errors for f pt+h are almost identical to those computed using thechain-rule of forecasting. The bootstrap forecasts for ∆st+h are slightly
diﬀerent from the chain-rule and Monte Carlo forecasts. In particular, thebootstrap forecast standard errors are larger than corresponding values
from the chain-rule and Monte Carlo methods.
The simulation-based forecasts described above are diﬀerent from thetraditional simulation-based approach taken in VAR literature, e.g., see404

11. Vector Autoregressive Models for Multivariate Time Series

Runkle (1987). The traditional approach is implemented using the following
procedure:
1. Obtain VAR coeﬃcient estimates Π and residuals εt .
2. Simulate the fitted VAR model by Monte Carlo simulation or bybootstrapping the fitted residuals ε̂t .
3. Obtain new estimates of Π and forecasts of Yt+h based on the simulated data.
The above procedure is repeated many times to obtain simulation-based
forecasts as well as their confidence intervals. To illustrate this approach,generate 12-step ahead forecasts from the fitted VAR object var1.fit byMonte Carlo simulation using the S+FinMetrics function simulate.VAR
as follows:> set.seed(10)> n.pred=12> n.sim=100> sim.pred = array(0,c(n.sim, n.pred, 2))> y0 = seriesData(var1.fit$Y0)> for (i in 1:n.sim) {+dat = simulate.VAR(var1.fit,n=243)+
dat = rbind(y0,dat)+
mod = VAR(dat~ar(1))+
sim.pred[i,,] = predict(mod,n.pred)$values+ }The simulation-based forecasts are obtained by averaging the simulated
forecasts:> colMeans(sim.pred)
[,1]
[,2]
[1,] -0.0017917 -0.0012316[2,] -0.0017546 -0.0012508[3,] -0.0017035 -0.0012643[4,] -0.0016800 -0.0012741[5,] -0.0016587 -0.0012814[6,] -0.0016441 -0.0012866[7,] -0.0016332 -0.0012904[8,] -0.0016253 -0.0012932[9,] -0.0016195 -0.0012953[10,] -0.0016153 -0.0012967[11,] -0.0016122 -0.0012978[12,] -0.0016099 -0.0012986

11.3 Forecasting

405

Comparing these forecasts with those in uscn.pred computed earlier, one
can see that for the first few forecasts, these simulated forecasts are slightly
diﬀerent from the asymptotic forecasts. However, at larger steps, they approach the long run stable values of the asymptotic forecasts.
Conditional Forecasting
The forecasts algorithms considered up to now are unconditional multivariate forecasts. However, sometimes it is desirable to obtain forecasts of somevariables in the system conditional on some knowledge of the future pathof other variables in the system. For example, when forecasting multivariate macroeconomic variables using quarterly data from a VAR model, it
may happen that some of the future values of certain variables in the VAR
model are known, because data on these variables are released earlier thandata on the other variables. By incorporating the knowledge of the future
path of certain variables, in principle it should be possible to obtain morereliable forecasts of the other variables in the system. Another use of conditional forecasting is the generation of forecasts conditional on diﬀerent“policy” scenarios. These scenario-based conditional forecasts allow one toanswer the question: if something happens to some variables in the systemin the future, how will it aﬀect forecasts of other variables in the future?
S+FinMetrics provides a generic function cpredict for computing conditional forecasts, which has a method cpredict.VAR for “VAR” objects.
The algorithms in cpredict.VAR are based on the conditional forecasting algorithms described in Waggoner and Zha (1999). Waggoner and Zha
classified conditional information into “hard” conditions and “soft conditions”. The hard conditions restrict the future values of certain variables atfixed values, while the soft conditions restrict the future values of certain
variables in specified ranges. The arguments taken by cpredict.VAR are:> args(cpredict.VAR)
function(object, n.predict = 1, newdata = NULL, olddata = NULL,
method = "mc", unbiased = T, variables.conditioned =NULL, steps.conditioned = NULL, upper = NULL, lower =NULL, middle = NULL, seed = 100, n.sim = 1000)
Like most predict methods in S-PLUS, the first argument must be a fitted
model object, while the second argument, n.predict, specifies the numberof steps to predict ahead. The arguments newdata and olddata can usually
be safely ignored, unless exogenous variables were used in fitting the model.
With classical forecasts that ignore the uncertainty in coeﬃcient estimates, hard conditional forecasts can be obtained in closed form as shown
by Doan, Litterman and Sims (1984), and Waggoner and Zha (1999). To
obtain hard conditional forecasts, the argument middle is used to specify
fixed values of certain variables at certain steps. For example, to fix the406

11. Vector Autoregressive Models for Multivariate Time Series

1-step ahead forecast of dspot in var1.fit at -0.005 and generate otherpredictions for 2-step ahead forecasts, use the following command:> cpredict(var1.fit, n.predict=2, middle=-0.005,+ variables="dspot", steps=1)
Predicted Values:
dspot
fp
1-step-ahead -0.0050 -0.0005
2-step-ahead -0.0023 -0.0007
In the call to cpredict, the optional argument variables is used to specify
the restricted variables, and steps to specify the restricted steps.
To specify a soft condition, the optional arguments upper and lower
are used to specify the upper bound and lower bound, respectively, of asoft condition. Since closed form results are not available for soft conditional forecasts, either Monte Carlo simulation or bootstrap methods areused to obtain the actual forecasts. The simulations follow a similar procedure implemented in the function predict.VAR, except that a reject/accept
method to sample from the distribution conditional on the soft conditions
is used. For example, to restrict the range of the first 2-step ahead forecastsof dspot to be (−0.004, −0.001) use:
> cpredict(var1.fit, n.predict=2, lower=c(-0.004, -0.004),+ upper=c(-0.001, -0.001), variables="dspot",+ steps=c(1,2))
Predicted Values:
dspot
fp
1-step-ahead -0.0027 -0.0003
2-step-ahead -0.0029 -0.0005

11.4 Structural Analysis
The general VAR(p) model has many parameters, and they may be diﬃcult
to interpret due to complex interactions and feedback between the variablesin the model. As a result, the dynamic properties of a VAR(p) are often
summarized using various types of structural analysis. The three main typesof structural analysis summaries are (1) Granger causality tests; (2) impulse
response functions; and (3) forecast error variance decompositions. The
following sections give brief descriptions of these summary measures.

11.4 Structural Analysis

407

11.4.1 Granger Causality
One of the main uses of VAR models is forecasting. The structure of theVAR model provides information about a variable’s or a group of variables’
forecasting ability for other variables. The following intuitive notion of avariable’s forecasting ability is due to Granger (1969). If a variable, orgroup of variables, y1 is found to be helpful for predicting another variable,
or group of variables, y2 then y1 is said to Granger-cause y2 ; otherwise itis said to fail to Granger-cause y2 . Formally, y1 fails to Granger-cause y2
if for all s > 0 the MSE of a forecast of y2,t+s based on (y2,t , y2,t−1 , . . .) is
the same as the MSE of a forecast of y2,t+s based on (y2,t , y2,t−1 , . . .) and(y1,t , y1,t−1 , . . .). Clearly, the notion of Granger causality does not imply
true causality. It only implies forecasting ability.
Bivariate VAR Models
In a bivariate VAR(p) model for Yt = (y1t , y2t )0 , y2 fails to Granger-cause
y1 if all of the p VAR coeﬃcient matrices Π1 , . . . , Πp are lower triangular.
That is, the VAR(p) model has the form
µ
¶
µ
¶ µ 1
¶µ
¶
y1t
c1
π11
y1t−1
0
=
+
+ ···
π121 π 122
y2t
c2
y2t−1
µ p
¶µ
¶ µ
¶
π11
y1t−p
ε1t
0
+
+
πp21 π p22
y2t−p
ε2t
so that all of the coeﬃcients on lagged values of y2 are zero in the equation
for y1 . Similarly, y1 fails to Granger-cause y2 if all of the coeﬃcients onlagged values of y1 are zero in the equation for y2 . The p linear coeﬃcient
restrictions implied by Granger non-causality may be tested using the Waldstatistic (11.5). Notice that if y2 fails to Granger-cause y1 and y1 fails
to Granger-cause y2 , then the VAR coeﬃcient matrices Π1 , . . . , Πp are
diagonal.
General VAR Models
Testing for Granger non-causality in general n variable VAR(p) models
follows the same logic used for bivariate models. For example, consider aVAR(p) model with n = 3 and Yt = (y1t , y2t , y3t )0 . In this model, y2 does
not Granger-cause y1 if all of the coeﬃcients on lagged values of y2 are zeroin the equation for y1 . Similarly, y3 does not Granger-cause y1 if all of thecoeﬃcients on lagged values of y3 are zero in the equation for y1 . These
simple linear restrictions may be tested using the Wald statistic (11.5). The
reader is encouraged to consult Lütkepohl (1991) or Hamilton (1994) for
more details and examples.
Example 68 Testing for Granger causality in bivariate VAR(2) model for
exchange rates

408

11. Vector Autoregressive Models for Multivariate Time Series

Consider testing for Granger causality in a bivariate VAR(2) model for
Yt = (∆st , f pt )0 . Using the notation of (11.2), f pt does not Granger cause∆st if π112 = 0 and π 212 = 0. Similarly, ∆st does not Granger cause f pt if
π 121 = 0 and π 221 = 0. These hypotheses are easily tested using the Waldstatistic (11.5). The restriction matrix R for the hypothesis that f pt does
not Granger cause ∆st isµ¶
0 0 1 0 0 0 0 0 0 0
R=
0 0 0 0 1 0 0 0 0 0
and the matrix for the hypothesis that ∆st does not Granger cause f pt isµ¶
0 0 0 0 0 0 1 0 0 0
R=
0 0 0 0 0 0 0 0 1 0
The S-PLUS commands to compute and evaluate these Granger causality
Wald statistics are> var2.fit = VAR(cbind(dspot,fp)~ar(2),data=uscn.ts)> # H0: fp does not Granger cause dspot> R = matrix(c(0,0,1,0,0,0,0,0,0,0,+ 0,0,0,0,1,0,0,0,0,0),+ 2,10,byrow=T)> vecPi = as.vector(coef(var2.fit))> avar = R%*%vcov(var2.fit)%*%t(R)> wald = t(R%*%vecPi)%*%solve(avar)%*%(R%*%vecPi)> wald[,1][1,] 8.468844> 1-pchisq(wald,2)[1] 0.01448818> R = matrix(c(0,0,0,0,0,0,1,0,0,0,+ 0,0,0,0,0,0,0,0,1,0),+ 2,10,byrow=T)> vecPi = as.vector(coef(var2.fit))> avar = R%*%vcov(var2.fit)%*%t(R)> wald = t(R%*%vecPi)%*%solve(avar)%*%(R%*%vecPi)> wald[,1][1,] 6.157> 1-pchisq(wald,2)[1] 0.04604
The p-values for the Wald tests indicate a fairly strong rejection of the nullthat f pt does not Granger cause ∆st but only a weak rejection of the nullthat ∆st does not Granger cause f pt . Hence, lagged values of f pt appear

11.4 Structural Analysis

409

to be useful for forecasting future values of ∆st and lagged values of ∆stappear to be useful for forecasting future values of f pt .

11.4.2 Impulse Response Functions
Any covariance stationary VAR(p) process has a Wold representation of
the form
Yt = μ + εt + Ψ1 εt−1 + Ψ2 εt−2 + · · ·(11.11)
where the (n × n) moving average matrices Ψs are determined recursively
using (11.6). It is tempting to interpret the (i, j)-th element, ψ sij , of thematrix Ψs as the dynamic multiplier or impulse response∂yi,t∂yi,t+s== ψ sij , i, j = 1, . . . , n∂εj,t∂εj,t−s
However, this interpretation is only possible if var(εt ) = Σ is a diagonal
matrix so that the elements of εt are uncorrelated. One way to make theerrors uncorrelated is to follow Sims (1980) and estimate the triangularstructural VAR(p) model
y1t
y2t
y3t

ynt

= c1 + γ 011 Yt−1 + · · · + γ 01p Yt−p + η1t(11.12)00= c1 + β 21 y1t +γ 21 Yt−1 + · · · + γ 2p Yt−p + η2t= c1 + β 31 y1t + β 32 y2t +γ 031 Yt−1 + · · · + γ 03p Yt−p + η3t..
.
= c1 + β n1 y1t + · · · + β n,n−1 yn−1,t + γ 0n1 Yt−1 + · · · + γ 0np Yt−p + ηnt

In matrix form, the triangular structural VAR(p) model is
BYt = c + Γ1 Yt−1 + Γ2 Yt−2 + · · · + Γp Yt−p + η t
where

⎛

1
⎜ −β 21
⎜
B=⎜
..
⎝
.

−β n1

0
1
..
.

···
0
..
.

−β n2

···

⎞
0
0 ⎟
⎟
.. ⎟
. ⎠

(11.13)(11.14)1

is a lower triangular matrix with 10 s along the diagonal. The algebra ofleast squares will ensure that the estimated covariance matrix of the error
vector ηt is diagonal. The uncorrelated/orthogonal errors η t are referred
to as structural errors.
The triangular structural model (11.12) imposes the recursive causal ordering
y1 → y2 → · · · → yn
(11.15)

410

11. Vector Autoregressive Models for Multivariate Time Series

The ordering (11.15) means that the contemporaneous values of the variables to the left of the arrow → aﬀect the contemporaneous values of thevariables to the right of the arrow but not vice-versa. These contemporaneous eﬀects are captured by the coeﬃcients β ij in (11.12). For example,
the ordering y1 → y2 → y3 imposes the restrictions: y1t aﬀects y2t and y3t
but y2t and y3t do not aﬀect y1 ; y2t aﬀects y3t but y3t does not aﬀect y2t .
Similarly, the ordering y2 → y3 → y1 imposes the restrictions: y2t aﬀects
y3t and y1t but y3t and y1t do not aﬀect y2 ; y3t aﬀects y1t but y1t does not
aﬀect y3t . For a VAR(p) with n variables there are n! possible recursive
causal orderings. Which ordering to use in practice depends on the context
and whether prior theory can be used to justify a particular ordering. Results from alternative orderings can always be compared to determine thesensitivity of results to the imposed ordering.
Once a recursive ordering has been established, the Wold representationof Yt based on the orthogonal errors η t is given by
Yt = μ + Θ0 ηt + Θ1 η t−1 + Θ2 η t−2 + · · ·(11.16)

where Θ0 = B−1 is a lower triangular matrix. The impulse responses tothe orthogonal shocks η jt are∂yi,t+s∂yi,t== θsij , i, j = 1, . . . , n; s > 0∂η j,t∂η j,t−s(11.17)

where θsij is the (i, j) th element of Θs . A plot of θsij against s is called theorthogonal impulse response function (IRF) of yi with respect to η j . Withn variables there are n2 possible impulse response functions.
In practice, the orthogonal IRF (11.17) based on the triangular VAR(p)(11.12) may be computed directly from the parameters of the non triangular
VAR(p) (11.1) as follows. First, decompose the residual covariance matrix
Σ asΣ = ADA0
where A is an invertible lower triangular matrix with 10 s along the diagonal
and D is a diagonal matrix with positive diagonal elements. Next, define
the structural errors as
η t = A−1 εtThese structural errors are orthogonal by construction since var(η t ) =A−1 ΣA−10 = A−1 ADA0 A−10 = D. Finally, re-express the Wold representation (11.11) asYt= μ + AA−1 εt + Ψ1 AA−1 εt−1 + Ψ2 AA−1 εt−2 + · · ·= μ + Θ0 ηt + Θ1 η t−1 + Θ2 η t−2 + · · ·where Θj = Ψj A. Notice that the structural B matrix in (11.13) is equalto A−1 .

11.4 Structural Analysis

411

Computing the Orthogonal Impulse Response Function Using the
S+FinMetrics Function impRes
The orthogonal impulse response function (11.17) from a triangular structural VAR model (11.13) may be computed using the S+FinMetrics function impRes. The function impRes has arguments> args(impRes)
function(x, period = NULL, std.err = "none", plot = F,
unbiased = T, order = NULL, ...)
where x is an object of class “VAR” and period specifies the number of
responses to compute. By default, no standard errors for the responses
are computed. To compute asymptotic standard errors for the responses,
specify std.err="asymptotic". To create a panel plot of all the response
functions, specify plot=T. The default recursive causal ordering is based on
the ordering of the variables in Yt when the VAR model is fit. The optional
argument order may be used to specify a diﬀerent recursive causal ordering for the computation of the impulse responses. The argument order accepts a character vector of variable names whose order defines the recursive
causal ordering. The output of impRes is an object of class “impDecomp” forwhich there are print, summary and plot methods. The following example
illustrates the use of impRes.
Example 69 IRF from VAR(1) for exchange rates
Consider again the VAR(1) model for Yt = (∆st , f pt )0 . For the impulse
response analysis, the initial ordering of the variables imposes the assumption that structural shocks to f pt have no contemporaneous eﬀect on ∆stbut structural shocks to ∆st do have a contemporaneous eﬀect on f pt . To
compute the four impulse response functions∂∆st+h ∂∆st+h ∂f pt+h ∂f pt+h
,
,
,
∂η 1t
∂η 2t
∂η 1t
∂η 2t
for h = 1, . . . , 12 we use S+FinMetrics function impRes. The first twelve
impulse responses from the VAR(1) model for exchange rates are computedusing> uscn.irf = impRes(var1.fit, period=12, std.err="asymptotic")
The print method shows the impulse response values without standard
errors:> uscn.irf
Impulse Response Function:(with responses in rows, and innovations in columns)

412

11. Vector Autoregressive Models for Multivariate Time Series, , lag.0
dspot
dspot 0.0136
fp 0.0000

fp
0.00000.0009, , lag.1
dspot
fp
dspot -0.0018 -0.0013
fp 0.0001 0.0007, , lag.2
dspot
fp
dspot 0.0000 -0.0009
fp 0.0001 0.0006...
, , lag.11
dspot
fp
dspot 0.0000 -0.0001
fp 0.0000 0.0001
The summary method will display the responses with standard errors andt-statistics. The plot method will produce a four panel Trellis graphics
plot of the impulse responses> plot(uscn.irf)
A plot of the impulse responses can also be created in the initial call to
impRes by using the optional argument plot=T.
Figure 11.6 shows the impulse response functions along with asymptotic
standard errors. The top row shows the responses of ∆st to the structural
shocks, and the bottom row shows the responses of f pt to the structural
shocks. In response to the first structural shock, η1t , ∆st initially increases
but then drops quickly to zero after 2 months. Similarly, f pt initially increases, reaches its peak response in 2 months and then gradually drops
oﬀ to zero after about a year. In response to the second shock, η 2t , byassumption ∆st has no initial response. At one month, a sharp drop occurs
in ∆st followed by a gradual return to zero after about a year. In contrast,
f pt initially increases and then gradually drops to zero after about a year.
The orthogonal impulse responses in Figure 11.6 are based on the recursive causal ordering ∆st → f pt . It must always be kept in mind that thisordering identifies the orthogonal structural shocks η1t and η 2t . If the ordering is reversed, then a diﬀerent set of structural shocks will be identified,
and these may give very diﬀerent impulse response functions. To compute11.4 Structural Analysis

413

Orthogonal Impulse Response Function
0

2

4

6

8

10

Resp.: dspot
Inno.: fp-0.0015

0.0

Resp.: fp
Inno.: fp
0.00080.00020

Resp.: fp
Inno.: dspot

0.0

0.0

0.00040.00010

Impulse Response

0.005

0.010

-0.0005

0.0

Resp.: dspot
Inno.: dspot

0

2

4

6

8

10

Steps

FIGURE 11.6. Impulse response function from VAR(1) model fit to US/CN exchange rate data with ∆st ordered first.

the orthogonal impulse responses using the alternative ordering f pt → ∆stspecify order=c("fp","dspot") in the call to impRes:> uscn.irf2 = impRes(var1.fit,period=12,std.err="asymptotic",+ order=c("fp","dspot"),plot=T)
These impulse responses are presented in Figure 11.7 and are almost identical to those computed using the ordering ∆st → f pt . The reason for this
response is that the reduced form VAR residuals ε̂1t and ε̂2t are almost
uncorrelated. To see this, the residual correlation matrix may be computedusing> sd.vals = sqrt(diag(var1.fit$Sigma))> cor.mat = var1.fit$Sigma/outer(sd.vals,sd.vals)> cor.mat
dspot
fp
dspot 1.000000 0.033048
fp 0.033048 1.000000
Because of the near orthogonality in the reduced form VAR errors, theerror in the ∆st equation may be interpreted as an orthogonal shock to theexchange rate and the error in the f pt equation may be interpreted as an
orthogonal shock to the forward premium.

414

11. Vector Autoregressive Models for Multivariate Time Series

Orthogonal Impulse Response Function
040.0008

8

10

0.0

0.0

0.00005

0.0004

6
Resp.: fp
Inno.: dspot

Resp.: dspot
Inno.: dspot

0.010

0.0010Resp.: dspot
Inno.: fp-0.0020

0.0

0.005

-0.0005

Impulse Response

20.00015

Resp.: fp
Inno.: fp

0

2

4

6

8

10

Steps

FIGURE 11.7. Impulse response function from VAR(1) model fit to US/CN exchange rate with f pt ordered first.

11.4.3 Forecast Error Variance Decompositions
The forecast error variance decomposition (FEVD) answers the question:
what portion of the variance of the forecast error in predicting yi,T +h is
due to the structural shock η j ? Using the orthogonal shocks η t the h-stepahead forecast error vector, with known VAR coeﬃcients, may be expressedash−1
XYT +h − YT +h|T =Θs η T +h−ss=0

For a particular variable yi,T +h , this forecast error has the form

yi,T +h − yi,T +h|T =

h−1
X
s=0

θsi1 η1,T +h−s + · · · +h−1Xθsin ηn,T +h−ss=0

Since the structural errors are orthogonal, the variance of the h-step forecast error isvar(yi,T +h − yi,T +h|T ) = σ 2η1

h−1
X
s=0

2
(θsi1 ) + · · · + σ 2ηn

h−1
X
s=0

2

(θsin )

11.4 Structural Analysis

415

where σ 2ηj = var(η jt ). The portion of var(yi,T +h − yi,T +h|T ) due to shockη j is then
FEVDi,j (h) =

σ2η1

Ph−1

σ 2ηj

Ph−1 ¡ s ¢2
s=0 θ ij

Ph−1 s 2s 22
s=0 (θ i1 ) + · · · + σ ηns=0 (θ in ), i, j = 1, . . . , n(11.18)In a VAR with n variables there will be n2 FEVDi,j (h) values. It must be
kept in mind that the FEVD in (11.18) depends on the recursive causal ordering used to identify the structural shocks ηt and is not unique. Diﬀerent
causal orderings will produce diﬀerent FEVD values.
Computing the FEVD Using the S+FinMetrics Function fevDec
Once a VAR model has been fit, the S+FinMetrics function fevDec may
be used to compute the orthogonal FEVD. The function fevDec has arguments> args(fevDec)function(x, period = NULL, std.err = "none", plot = F,
unbiased = F, order = NULL, ...)
where x is an object of class “VAR” and period specifies the number of
responses to compute. By default, no standard errors for the responses
are computed and no plot is created. To compute asymptotic standard
errors for the responses, specify std.err="asymptotic" and to plot thedecompositions, specify plot=T. The default recursive causal ordering isbased on the ordering of the variables in Yt when the VAR model is fit. The
optional argument order may be used to specify a diﬀerent recursive causal
ordering for the computation of the FEVD. The argument order accepts
a text string vector of variable names whose order defines the recursive
causal ordering. The output of fevDec is an object of class “impDecomp”
for which there are print, summary and plot methods. The use of fevDec
is illustrated with the following example.
Example 70 FEVD from VAR(1) for exchange rates
The orthogonal FEVD of the forecast errors from the VAR(1) model
fit to the US/CN exchange rate data using the recursive causal ordering∆st → f pt is computed using> uscn.fevd = fevDec(var1.fit,period=12,+ std.err="asymptotic")> uscn.fevd
Forecast Error Variance Decomposition:(with responses in rows, and innovations in columns)

416

11. Vector Autoregressive Models for Multivariate Time Series, , 1-step-ahead
dspot
fp
dspot 1.0000 0.0000
fp 0.0011 0.9989, , 2-step-ahead
dspot
fp
dspot 0.9907 0.0093
fp 0.0136 0.9864...
, , 12-step-ahead
dspot
fp
dspot 0.9800 0.0200
fp 0.0184 0.9816
The summary method adds standard errors to the above output if they arecomputed in the call to fevDec. The plot method produces a four panel
Trellis graphics plot of the decompositions:> plot(uscn.fevd)
The FEVDs in Figure 11.8 show that most of the variance of the forecast
errors for ∆st+s at all horizons s is due to the orthogonal ∆st innovations.
Similarly, most of the variance of the forecast errors for f pt+s is due to theorthogonal f pt innovations.
The FEVDs using the alternative recursive causal ordering f pt → ∆stare computed using> uscn.fevd2 = fevDec(var1.fit,period=12,+ std.err="asymptotic",order=c("fp","dspot"),plot=T)
and are illustrated in Figure 11.9. Since the residual covariance matrix is
almost diagonal (see analysis of IRF above), the FEVDs computed using
the alternative ordering are almost identical to those computed with theinitial ordering.

11.5 An Extended Example
In this example the causal relations and dynamic interactions among monthly
real stock returns, real interest rates, real industrial production growth andthe inflation rate is investigated using a VAR model. The analysis is similar
to that of Lee (1992). The variables are in the S+FinMetrics “timeSeries”
object varex.ts> colIds(varex.ts)

11.5 An Extended Example

417

Forecast Error Variance Decomposition
2

6

8

10

12

Resp.: dspot
Inno.: fp

0.02

0.99

0.01

0.98

0.0

0.97

Resp.: fp
Inno.: fp

0.98

0.02

0.99

0.03

1.00

0.04

Resp.: fp
Inno.: dspot

0.96

0.0

0.97

0.01

Proportion of Variance Contribution

4

0.03

1.00

Resp.: dspot
Inno.: dspot

2

4

6

8

10

12

Forecast Steps

FIGURE 11.8. Orthogonal FEVDs computed from VAR(1) model fit to US/CN
exchange rate data using the recursive causal ordering with ∆st first.

[1] "MARKET.REAL" "RF.REAL" "INF" "IPG"
Details about the data are in the documentation slot of varex.ts> varex.ts@documentation
To be comparable to the results in Lee (1992), the analysis is conducted
over the postwar period January 1947 through December 1987
> smpl = (positions(varex.ts) >= timeDate("1/1/1947") &+ positions(varex.ts) < timeDate("1/1/1988"))
The data over this period is displayed in Figure 11.10. All variables appear
to be I(0), but the real T-bill rate and the inflation rate appear to be highly
persistent.
To begin the analysis, autocorrelations and cross correlations at leads
and lags are computed using> varex.acf = acf(varex.ts[smpl,])and are illustrated in Figure 11.11. The real return on the market shows asignificant positive first lag autocorrelation, and inflation appears to leadthe real market return with a negative sign. The real T-bill rate is highly
positively autocorrelated, and inflation appears to lead the real T-bill ratestrongly with a negative sign. Inflation is also highly positively autocorrelated and, interestingly, the real T-bill rate appears to lead inflation with

418

11. Vector Autoregressive Models for Multivariate Time Series

Forecast Error Variance Decomposition
2

6

8

10

12

Resp.: fp
Inno.: dspot

0.010

0.995

0.0

0.985
0.04 0.975

Resp.: dspot
Inno.: dspot

0.98

0.02

0.99

0.03

1.00

Resp.: dspot
Inno.: fp

0.96

0.0

0.97

0.01

Proportion of Variance Contribution

4

0.020

Resp.: fp
Inno.: fp

2

4

6

8

10

12

Forecast Steps

FIGURE 11.9. Orthogonal FEVDs from VAR(1) model fit to US/CN exchange
rate data using recursive causal ordering with f pt first.

a positive sign. Finally, industrial production growth is slightly positively
autocorrelated, and the real market return appears to lead industrial production growth with a positive sign.
The VAR(p) model is fit with the lag length selected by minimizing the
AIC and a maximum lag length of 6 months:> varAIC.fit = VAR(varex.ts,max.ar=6,criterion="AIC",+ start="Jan 1947",end="Dec 1987",+ in.format="%m %Y")
The lag length selected by minimizing AIC is p = 2:
> varAIC.fit$info
ar(1) ar(2) ar(3) ar(4) ar(5) ar(6)AIC -14832 -14863 -14853 -14861 -14855 -14862> varAIC.fit$ar.order[1] 2
The results of the VAR(2) fit are> summary(varAIC.out)
Call:
VAR(data = varex.ts, start = "Jan 1947", end = "Dec 1987",

11.5 An Extended Example

Real Interest Rate

-0.3

-0.015-0.1-0.005

0.1

0.005

0.3

Real Return on Market

419

1930

1950

1970

1990

1930

1950

1990

Inflation

-0.02

-0.10

0.00

0.02

0.10

0.05

Industrial Production Growth

1970

1930

1950

1970

1990

1930

1950

1970

1990

FIGURE 11.10. Monthly data on stock returns, interest rates, output growth andinflation.

max.ar = 6, criterion = "AIC", in.format = "%m %Y")
Coefficients:
MARKET.REAL
0.00740.0023
3.1490RF.REAL
0.00020.00014.6400INF
0.00100.00024.6669IPG
0.00190.00072.5819

MARKET.REAL.lag1(std.err)(t.stat)0.24500.04705.20820.00010.0011
0.04830.00720.0042
1.70920.02800.01461.9148

RF.REAL.lag1(std.err)(t.stat)0.81462.06480.39450.87900.047018.6861

0.55380.18542.98670.37720.64190.5877

INF.lag1(std.err)(t.stat)-1.50200.4932-3.0451-0.07100.0112-6.31470.46160.044310.4227-0.07220.1533-0.4710IPG.lag1

MARKET.REAL-0.0003

RF.REAL
0.0031

INF-0.0143

IPG
0.3454(Intercept)(std.err)(t.stat)11. Vector Autoregressive Models for Multivariate Time Series

MARKET.REAL and RF.REAL

MARKET.REAL and INF

MARKET.REAL and IPG
0.05
-0.05

-0.05

-0.10
-0.15

-0.20

0.0

-0.15

-0.05

0.2

-0.10

0.0

ACF
0.4

0.6

0.0

0.0

0.05

0.8

0.05

1.0

MARKET.REAL

0.10

420

10

15

20

0

5

10

15

20

RF.REAL

0

5

15

20

10

15

20

15

20

15

20

15

20

0.05

0.0

0.8

5

0.0
-0.05

-0.3

0.0

-0.05

0.2

-0.2

0.4

0.0 ACF

-0.1

0.6

0.05

0

RF.REAL and IPG

0.10

1.0

10

RF.REAL and INF
0.10

5

RF.REAL and MARKET.REAL

0.1

0

-5

0

0

5

10

15

20

-10

10

-5

0

15

20

0

5

10

1.0

INF and IPG

0.8

0.10

0.6

0.0

0.4

-0.05

0.2

-0.10

0.0

-0.15
-15

5

INF

0.05

0.05
0.0
ACF
-0.05
-0.10
-0.15
-20

0

INF and RF.REAL

0.05

-10

INF and MARKET.REAL

0.0

-15

-0.05

-20

-20

-15

-10

-5

0

0

5

IPG and RF.REAL

10

15

20

0

5

IPG and INF

10

IPG

0.05

0.4

-0.20

-0.2

0.0

-0.15

-0.05

0.2

-0.10

0.0

-0.05

0.6

0.0

0.15
ACF 0.10
0.05
0.0
-0.05
-0.10

0.8

0.05

0.20

1.0

IPG and MARKET.REAL

-20

-15

-10
Lag

-5

0

-20

-15

-10
Lag

-5

0

-20

-15

-10
Lag

-5

0

0

5

10
Lag

FIGURE 11.11. Autocorrelations and cross correlations at leads and lags of data
in VAR model.

(std.err)(t.stat)0.1452-0.00180.00330.92520.0130-1.09930.04527.6501

MARKET.REAL.lag2(std.err)(t.stat)-0.0500
0.0466-1.0727

0.00220.00112.0592-0.0066
0.0042-1.58160.03950.01452.7276

RF.REAL.lag2(std.err)(t.stat)-0.34811.9845-0.17540.03930.04520.8699-0.58550.1782-3.2859-0.32890.6169-0.5331

INF.lag2(std.err)(t.stat)-0.06020.5305-0.1135

0.00790.01210.65170.24760.0476
5.1964-0.0370
0.1649-0.2245

IPG.lag2(std.err)(t.stat)

MARKET.REAL-0.19190.1443-1.3297

RF.REAL
0.00280.00330.8432

INF
0.01540.01301.1868Regression Diagnostics:
MARKET.REAL RF.REAL

INF

IPG
0.0941
0.04492.0968IPG

11.5 An Extended Example

R-squared 0.1031
Adj. R-squared 0.0882
Resid. Scale 0.0334

0.92990.92870.0008421

0.4109 0.2037
0.4011 0.1905
0.0030 0.0104

Information Criteria:
logL
AIC
BIC
HQ
7503 -14935 -14784 -14875
total residual
Degree of freedom:
489
480
Time period: from Mar 1947 to Nov 1987
The signs of the statistically significant coeﬃcient estimates corroborate
the informal analysis of the multivariate autocorrelations and cross lag autocorrelations. In particular, the real market return is positively related toits own lag but negatively related to the first lag of inflation. The real T-billrate is positively related to its own lag, negatively related to the first lag of
inflation, and positively related to the first lag of the real market return. Industrial production growth is positively related to its own lag and positively
related to the first two lags of the real market return. Judging from thecoeﬃcients it appears that inflation Granger causes the real market returnand the real T-bill rate, the real T-bill rate Granger causes inflation, andthe real market return Granger causes the real T-bill rate and industrial
production growth. These observations are confirmed with formal tests for
Granger non-causality. For example, the Wald statistic for testing the null
hypothesis that the real market return does not Granger-cause industrial
production growth is> bigR = matrix(0,2,36)> bigR[1,29]=bigR[2,33]=1> vecPi = as.vector(coef(varAIC.fit))> avar = bigR%*%vcov(varAIC.fit)%*%t(bigR)> wald = t(bigR%*%vecPi)%*%solve(avar)%*%(bigR%*%vecPi)> as.numeric(wald)[1] 13.82> 1-pchisq(wald,2)[1] 0.0009969
The 24-period IRF using the recursive causal ordering MARKET.REAL →RF.REAL → IPG → INF is computed using> varAIC.irf = impRes(varAIC.fit,period=24,+ order=c("MARKET.REAL","RF.REAL","IPG","INF"),+ std.err="asymptotic",plot=T)
and is illustrated in Figure 11.12. The responses of MARKET.REAL to unexpected orthogonal shocks to the other variables are given in the first row of422

11. Vector Autoregressive Models for Multivariate Time Series

Orthogonal Impulse Response Function

5

10

15

15

20

-0.006

Resp.: RF.REAL
Inno.: INF

Resp.: IPG
Inno.: INF-0.0004
0.0030-0.00100.004
0.0

Resp.: INF
Inno.: INF

0.00150.0002

Resp.: INF
Inno.: IPG

0.0

20

10

-0.0001
0.0002 -0.0004

6*10^-5
0.008

Resp.: IPG
Inno.: IPG

0.0-0.0002

Resp.: INF
Inno.: RF.REAL

5

Resp.: MARKET.REAL
Inno.: INF-0.003-0.001

Resp.: RF.REAL
Inno.: IPG-2*10^-50.00020.00040.0004
0.0

0.0
0

0
Resp.: MARKET.REAL
Inno.: IPG-0.003

Resp.: RF.REAL
Inno.: RF.REAL

Resp.: IPG
Inno.: RF.REAL-0.00020.00100.0002

Resp.: INF
Inno.: MARKET.REAL

20
0.0

15

0.001

10

0.0006-0.00150.01
0.0
00.0025-1.5*10^-4

Resp.: IPG
Inno.: MARKET.REAL

0.0

Impulse Response

Resp.: RF.REAL
Inno.: MARKET.REAL-0.0003

5

Resp.: MARKET.REAL
Inno.: RF.REAL

0.0

0.03

Resp.: MARKET.REAL
Inno.: MARKET.REAL

0.0015

0

0

5

10

15

20

Steps

FIGURE 11.12. IRF using the recursive causal ordering MARKET.REAL → RF.REAL→ IPG → INF.

the figure. Most notable is the strong negative response of MARKET.REAL toan unexpected increase in inflation. Notice that it takes about ten months
for the eﬀect of the shock to dissipate. The responses of RF.REAL to theorthogonal shocks is given in the second row of the figure. RF.REAL also
reacts negatively to an inflation shock and the eﬀect of the shock is felt for
about two years. The responses of IPG to the orthogonal shocks is givenin the third row of the figure. Industrial production growth responds positively to an unexpected shock to MARKET.REAL and negatively to shocks to
RF.REAL and INF. These eﬀects, however, are generally short term. Finally,
the fourth row gives the responses of INF to the orthogonal shocks. Inflation responds positively to a shock to the real T-bill rate, but this eﬀect isshort-lived.
The 24 month FEVD computed using the recursive causal ordering as
specified by MARKET.REAL → RF.REAL → IPG → INF,
> varAIC.fevd = fevDec(varAIC.out,period=24,> order=c("MARKET.REAL","RF.REAL","IPG","INF"),> std.err="asymptotic",plot=T)11.5 An Extended Example

423

Forecast Error Variance Decomposition
20

10

15

20

20

0.02
0.0
0.4
0.2
0.0

Resp.: IPG
Inno.: INF

0.025

0.98
0.010 0.90

0.0

0.94
5

Resp.: INF
Inno.: INF
0.98

Resp.: INF
Inno.: IPG

0.94

0.004

Resp.: INF
Inno.: RF.REAL

15

0.04

0.010
0.004
-0.002

Resp.: IPG
Inno.: IPG-0.002-0.002

0.01

0.03

0.004

0.05

-0.0020.0

Resp.: INF
Inno.: MARKET.REAL

10

Resp.: MARKET.REAL
Inno.: INF

Resp.: RF.REAL
Inno.: INF

0.015-0.0050.5
0.008

Resp.: IPG
Inno.: RF.REAL

0.004

0.04

Resp.: RF.REAL
Inno.: IPG

0.005

0.9

-0.0005

Resp.: RF.REAL
Inno.: RF.REAL

0.7

0.03
0.01

Resp.: IPG
Inno.: MARKET.REAL

5
Resp.: MARKET.REAL
Inno.: IPG

0.06

15

0.00100.98
0.94

Resp.: RF.REAL
Inno.: MARKET.REAL

0.008

Proportion of Variance Contribution

10

Resp.: MARKET.REAL
Inno.: RF.REAL

0.010

5
Resp.: MARKET.REAL
Inno.: MARKET.REAL

5

10

15

20

Forecast Steps

FIGURE 11.13. FEVDs using the recursive causal ordering MARKET.REAL →RF.REAL → IPG → INF.

is illustrated in Figure 11.13. The first row gives the variance decompositions for MARKET.REAL and shows that most of the variance of theforecast errors is due to own shocks. The second row gives the decompositions for RF.REAL. At short horizons, most of the variance is attributableto own shocks but at long horizons inflation shocks account for almost
half the variance. The third row gives the variance decompositions for IPG.
Most of the variance is due to own shocks and a small fraction is due to
MARKET.REAL shocks. Finally, the fourth row shows that the forecast error
variance of INF is due mostly to its own shocks.
The IRFs and FEVDs computed above depend on the imposed recursive
causal ordering. However, in this example, the ordering of the variables will
have little eﬀect on the IRFs and FEVDs because the errors in the reduced
form VAR are nearly uncorrelated:> sd.vals = sqrt(diag(varAIC.out$Sigma))> cor.mat = varAIC.out$Sigma/outer(sd.vals,sd.vals)> cor.mat
MARKET.REAL RF.REAL
INF
IPG
MARKET.REAL
1.00000 -0.16855 -0.04518 0.03916
RF.REAL-0.16855 1.00000 0.13046 0.03318
INF-0.04518 0.13046 1.00000 0.04732
IPG
0.03916 0.03318 0.04732 1.00000

424

11. Vector Autoregressive Models for Multivariate Time Series

11.6 Bayesian Vector Autoregression
VAR models with many variables and long lags contain many parameters.
Unrestricted estimation of these models reqires lots of data and often theestimated parameters are not very precise, the results are hard to interpret, and forecasts may appear more precise than they really are because
standard error bands do not account for parameter uncertainty. The estimates and forecasts can be improved if one has prior information about
the structure of the model or the possible values of the parameters or functions of the parameters. In a classical framework, it is diﬃcult to incorporate non-sample information into the estimation. Nonsample information
is easy to incorporate in a Bayesian framework. A Bayesian framework
also naturally incorporates parameter uncertainty into common measuresof precision. This section briefly describes the Bayesian VAR modelingtools in S+FinMetrics and illustrates these tools with an example. Detailsof underlying Bayesian methods are given in Sims and Zha (1998) and Zha(1998).

11.6.1 An Example of a Bayesian VAR Model
S+FinMetrics comes with a “timeSeries” object policy.dat, which contains six U.S. macroeconomic variables:> colIds(policy.dat)[1] "CP" "M2" "FFR" "GDP" "CPI" "U"
which represent IMF’s index of world commodity prices, M2 money stock,federal funds rate, real GDP, consumer price index for urban consumers,
and civilian unemployment rate. The data set contains monthly observations from January 1959 to March 1998. Tao Zha and his co-authors haveanalyzed this data set in a number of papers, for example see Zha (1998).
To use the same time period as in Zha (1998), create a subset of the data:> zpolicy.dat = policy.dat[1:264,]> zpolicy.mat = as.matrix(seriesData(zpolicy.dat))
which contains monthly observations from January 1959 to December
1980.
Estimating a Bayesian VAR ModelTo estimate a Bayesian vector autoregression model, use the S+FinMetrics
function BVAR. For macroeconomic modeling, it is usually found that many
trending macroeconomic variables have a unit root, and in some cases,
they may also have a cointegrating relationship (as described in the next
chapter). To incorporate these types of prior beliefs into the model, use
the unit.root.dummy and coint.dummy optional arguments to the BVAR

11.6 Bayesian Vector Autoregression425

function, which add some dummy observations to the beginning of the data
to reflect these beliefs:> zpolicy.bar13 = BVAR(zpolicy.mat~ar(13), unit.root=T,+ coint=T)> class(zpolicy.bar13)[1] "BVAR"
The returned object is of class “BVAR”, which inherits from “VAR”, so manymethod functions for “VAR” objects work similarly for “BVAR” objects, suchas the extractor functions, impulse response functions, and forecast error
variance decomposition functions.
The Bayesian VAR models are controlled through a set of hyper parameters, which can be specified using the optional argument control, which
is usually a list returned by the function BVAR.control. For example, thetightness of the belief in the unit root prior and cointegration prior is specified by mu5 and mu6, respectively. To see what default values are used forthese hyper parameters, use> args(BVAR.control)
function(L0 = 0.9, L1 = 0.1, L2 = 1, L3 = 1, L4 = 0.05,
mu5 = 5, mu6 = 5)
For the meanings of these hyper parameters, see the online help file for
BVAR.control.
Adding Exogenous Variables to the Model
Other exogenous variables can be added to the estimation formula, just
as for OLS and VAR functions. The BVAR function and related functions
will automatically take that into consideration and return the coeﬃcientestimates for those variables.
Unconditional Forecasts
To forecast from a fitted Bayesian VAR model, use the generic predict
function, which automatically calls the method function predict.BVAR for
an object inheriting from class “BVAR”. For example, to compute 12-stepahead forecasts use> zpolicy.bpred = predict(zpolicy.bar13,n.predict=12)> class(zpolicy.bpred)[1] "forecast"> names(zpolicy.bpred)[1] "values" "std.err" "draws"> zpolicy.bpred
Predicted Values:11. Vector Autoregressive Models for Multivariate Time Series

4.6

4.8

255

260

265

270

275

CPI

FFR

0.14
0.10

U

0.09

7.45

0.10

4.3

M2

8.47

GDP

0.07
0.06

7.25

8.42

8.43

7.30

8.44

7.35

0.08

8.45

7.40

8.46

values

4.5

0.12

4.4

4.6

0.16

4.5

4.7

250

0.20

245
CP

0.18

426

245

250

255

260

265

270

275

245

250

255

260

265

270

275

index

FIGURE 11.14. Forecasts from Bayesian VAR model.

CP
M2
FFR
GDP
CPI
U
1-step-ahead 4.6354 7.3794 0.1964 8.4561 4.4714 0.0725
2-step-ahead 4.6257 7.3808 0.1930 8.4546 4.4842 0.0732
3-step-ahead 4.6247 7.3834 0.1823 8.4505 4.4960 0.0746
4-step-ahead 4.6310 7.3876 0.1670 8.4458 4.5065 0.0763
5-step-ahead 4.6409 7.3931 0.1515 8.4414 4.5160 0.0785
6-step-ahead 4.6503 7.3998 0.1384 8.4394 4.5244 0.0810
7-step-ahead 4.6561 7.4075 0.1309 8.4390 4.5321 0.0833
8-step-ahead 4.6552 7.4159 0.1307 8.4403 4.5397 0.0852
9-step-ahead 4.6496 7.4242 0.1362 8.4428 4.5475 0.0867
10-step-ahead 4.6415 7.4323 0.1451 8.4453 4.5561 0.0879
11-step-ahead 4.6321 7.4402 0.1546 8.4473 4.5655 0.0889
12-step-ahead 4.6232 7.4476 0.1618 8.4482 4.5753 0.0899
The forecasts can also be plotted along with the original data using> plot(zpolicy.bpred, zpolicy.mat, n.old=20)
The resulting plot is shown in Figure 11.14. The Bayesian forecasts usually
have wider error bands than classical forecasts, because they take into account the uncertainty in the coeﬃcient estimates. To ignore the uncertainty
in coeﬃcient estimates, one can call the classical VAR predict method
function, predict.VAR, directly instead of the generic predict function.

11.6 Bayesian Vector Autoregression

427

The forecasts from Bayesian VAR models are of class “forecast”, andare computed using Monte Carlo integration. By default, 1000 simulation
draws are used. To change the number of simulation draws and random
seed, specify the n.sim and seed optional arguments, respectively. For
forecasts from Bayesian VAR models, there is one more component in the
returned object: draws, which contains all the simulated forecasts. Thiscan be used to assess other statistical properties of the forecasts.

11.6.2 Conditional Forecasts
As mentioned earlier, conditional forecasts from classical VAR models ignore the uncertainty in estimated coeﬃcients. In contrast, conditional forecasts from Bayesian VAR models take into account the uncertainty associated with estimated coeﬃcients. To perform conditional forecasts from afitted Bayesian VAR model, use the generic cpredict function. For example, if it is known that FFR in January 1981 is between 0.185 and 0.195, one
can incorporate this (soft condition) information into the forecasts using:> zpolicy.spred = cpredict(zpolicy.bar13, 12, steps=1,+ variables="FFR", upper=0.195, lower=0.185)> zpolicy.spred
Predicted Values:
CP
M2
FFR
GDP
CPI
U
1-step-ahead 4.6374 7.3797 0.1910 8.4554 4.4714 0.0729
2-step-ahead 4.6286 7.3816 0.1855 8.4540 4.4840 0.0736
3-step-ahead 4.6279 7.3850 0.1743 8.4498 4.4954 0.0752
4-step-ahead 4.6349 7.3899 0.1587 8.4452 4.5057 0.0768
5-step-ahead 4.6447 7.3960 0.1443 8.4414 4.5149 0.0791
6-step-ahead 4.6525 7.4033 0.1324 8.4406 4.5231 0.0814
7-step-ahead 4.6549 7.4114 0.1270 8.4412 4.5307 0.0835
8-step-ahead 4.6523 7.4201 0.1283 8.4428 4.5383 0.0851
9-step-ahead 4.6453 7.4284 0.1349 8.4457 4.5461 0.0864
10-step-ahead 4.6389 7.4365 0.1432 8.4482 4.5547 0.0876
11-step-ahead 4.6317 7.4444 0.1516 8.4501 4.5641 0.0885
12-step-ahead 4.6264 7.4519 0.1572 8.4511 4.5741 0.0896
For conditional forecasts with soft conditions, a Monte Carlo integration with acceptance/rejection method is used. By default, 1000 simulation
draws are used. However, it may occur that only a small number of draws
satisfy the soft conditions if the intervals are very tight. To see how manydraws satisfied the soft conditions and thus are used for inference, simply
check the dimension of the draws component of the returned object (seethe on-line help file for forecast.object for details):

428

11. Vector Autoregressive Models for Multivariate Time Series> dim(zpolicy.spred$draws)[1] 372 72
In this case, only 372 out of 1000 simulation draws satisfied the conditions. To continue simulating from the posterior moment distribution,
use the same command as before, with seed set to the current value of.Random.seed:> zpolicy.spred2 = cpredict(zpolicy.bar13, 12, steps=1,+ variables="FFR", upper=0.195, lower=0.185, seed=.Random.seed)> dim(zpolicy.spred2$draws)[1] 389 72
Note that the draws in zpolicy.spred2 can be combined with the draws
in zpolicy.spred to obtain an updated and more accurate estimate ofconditional forecasts.
To ignore the coeﬃcient uncertainty for the conditional forecasts, callthe classical method function cpredict.VAR directly on a fitted Bayesian
VAR object. The technique introduced above can also be used for classicalprediction with soft conditions.

11.7 References
Campbell, J. A. Lo and C. MacKinlay (1997). The Econometrics ofFinancial Markets. Princeton University Press, Princeton, NJ.
Culbertson, K. (1996). Quantitative Financial Economics: Stocks, Bonds
and Foreign Exchange. John Wiley & Sons, Chichester.
Doan, T. A., Litterman, R. B., and Sims, C. A. (1984). “Forecasting
and Conditional Projection Using Realistic Prior Distributions,” Econometric Reviews, 3, 1-100.
Granger, C.W.J. (1969). “Investigating Causal Relations by Econometric Models and Cross Spectral Methods,” Econometrica, 37, 424-438.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
Lee, B.-S. (1992). “Causal Relations Among Stock Returns, Interest Rates,
Real Activity, and Inflation,” Journal of Finance, 47, 1591-1603.
Lutkepohl, H. (1991). Introduction to Multiple Time Series Analysis.
Springer-Verlag, Berlin.

11.7 References

429

Lutkepohl, H. (1999). “Vector Autoregressions,” unpublished manuscript,
Institut für Statistik und Ökonometrie,” Humboldt-Universität zu Berlin.
Mills, T.C. (1999). The Econometric Modeling of Financial Time Series,
Second Edition. Cambridge University Press, Cambridge.
Runkle, D. E. (1987). “Vector Autoregressions and Reality,” Journal ofBusiness and Economic Statistics, 5(4), 437-442.
Sims, C.A. (1980). “Macroeconomics and Reality,” Econometrica, 48, 1-48.
Sims, C. A., and Zha, T. (1998). “Bayesian Methods for Dynamic Multivariate Models,” International Economic Review, 39(4), 949-968.
Stock, J.H. and M.W. Watson (2001). “Vector Autoregressions,” Journal of Economic Perspectives, 15, 101-115.
Tsay, R. (2001). Analysis of Financial Time Series. John Wiley & Sons,
New York.
Watson, M. (1994). “Vector Autoregressions and Cointegration,” in R.F.
Engle and D. McFadden (eds.), Handbook of Econometrics, Volume IV.
Elsevier Science Ltd., Amsterdam.
Waggoner, D. F., and Zha, T. (1999). “Conditional Forecasts in Dynamic Multivariate Models,” Review of Economics and Statistics, 81(4),639-651.
Zha, T. (1998). “Dynamic Multivariate Model for Use in Formulating Policy”, Economic Review, Federal Reserve Bank of Atlanta, First Quarter,
1998.

This is page 430
Printer: Opaque thisThis is page 431
Printer: Opaque this12
Cointegration

12.1 Introduction
The regression theory of Chapter 6 and the VAR models discussed in theprevious chapter are appropriate for modeling I(0) data, like asset returns
or growth rates of macroeconomic time series. Economic theory often implies equilibrium relationships between the levels of time series variables
that are best described as being I(1). Similarly, arbitrage arguments imply
that the I(1) prices of certain financial time series are linked. This chapter
introduces the statistical concept of cointegration that is required to makesense of regression models and VAR models with I(1) data.
The chapter is organized as follows. Section 12.2 gives an overview of
the concepts of spurious regression and cointegration, and introduces theerror correction model as a practical tool for utilizing cointegration withfinancial time series. Section 12.3 discusses residual-based tests for cointegration. Section 12.4 covers regression-based estimation of cointegrating
vectors and error correction models. In Section 12.5, the connection between VAR models and cointegration is made, and Johansen’s maximum
likelihood methodology for cointegration modeling is outlined. Some technical details of the Johansen methodology are provided in the appendix tothis chapter.
Excellent textbook treatments of the statistical theory of cointegration
are given in Hamilton (1994), Johansen (1995) and Hayashi (2000). Applications of cointegration to finance may be found in Campbell, Lo, and432

12. Cointegration

MacKinlay (1997), Mills (1999), Alexander (2001), Cochrane (2001), andTsay (2001).

12.2 Spurious Regression and Cointegration
12.2.1 Spurious Regression
The time series regression model discussed in Chapter 6 required all variables to be I(0). In this case, the usual statistical results for the linear
regression model hold. If some or all of the variables in the regression are
I(1) then the usual statistical results may or may not hold1 . One importantcase in which the usual statistical results do not hold is spurious regression when all the regressors are I(1) and not cointegrated. The following
example illustrates.
Example 71 An illustration of spurious regression using simulated data
Consider two independent and not cointegrated I(1) processes y1t andy2t such that
yit = yit−1 + εit , where εit ∼ GW N (0, 1), i = 1, 2
Following Granger and Newbold (1974), 250 observations for each series
are simulated and plotted in Figure 12.1 using> set.seed(458)> e1 = rnorm(250)> e2 = rnorm(250)> y1 = cumsum(e1)> y2 = cumsum(e2)> tsplot(y1, y2, lty=c(1,3))> legend(0, 15, c("y1","y2"), lty=c(1,3))
The data in the graph resemble stock prices or exchange rates. A visualinspection of the data suggests that the levels of the two series are positively
related. Regressing y1t on y2t reinforces this observation:> summary(OLS(y1~y2))
Call:
OLS(formula = y1 ~y2)

1 A systematic technical analysis of the linear regression model with I(1) and I(0) variables is given in Sims, Stock and Watson (1990). Hamilton (1994) gives a nice summaryof these results and Stock and Watson (1989) provides useful intuition and examples.

433

15

12.2 Spurious Regression and Cointegration

-15

-10

-5

0

5

10

y1
y2

0

50

100

150

200

250

FIGURE 12.1. Two simulated independent I(1) processes.

Residuals:
Min
1Q-16.360 -4.352

Median-0.128

3Q
4.979

Max
10.763

Coefficients:(Intercept)
y2

Value Std. Error t value Pr(>|t|)6.7445 0.3943
17.1033 0.0000
0.4083 0.0508
8.0352 0.0000

Regression Diagnostics:
R-Squared 0.2066
Adjusted R-Squared 0.2034
Durbin-Watson Stat 0.0328
Residual standard error: 6.217 on 248 degrees of freedom
F-statistic: 64.56 on 1 and 248 degrees of freedom, thep-value is 3.797e-014
The estimated slope coeﬃcient is 0.408 with a large t-statistic of 8.035
and the regression R2 is moderate at 0.201. The only suspicious statistic
is the very low Durbin-Watson statistic suggesting strong residual autocorrelation. These statistics are representative of the spurious regression

434

12. Cointegration

phenomenon with I(1) that are not cointegrated. If ∆y1t is regressed on∆y2t the correct relationship between the two series is revealed> summary(OLS(diff(y1)~diff(y2)))
Call:
OLS(formula = diff(y1) ~diff(y2))
Residuals:
Min
1Q Median-3.6632 -0.7706 -0.0074

3Q
0.6983Max
2.7184

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) -0.0565 0.0669-0.8447 0.3991
diff(y2) 0.0275 0.0642
0.4290 0.6683
Regression Diagnostics:
R-Squared 0.0007
Adjusted R-Squared -0.0033
Durbin-Watson Stat 1.9356
Residual standard error: 1.055 on 247 degrees of freedom
F-statistic: 0.184 on 1 and 247 degrees of freedom, thep-value is 0.6683
Similar results to those above occur if cov(ε1t , ε2t ) 6= 0. The levels regression remains spurious (no real long-run common movement in levels),
but the diﬀerences regression will reflect the non-zero contemporaneous
correlation between ∆y1t and ∆y2t .
Statistical Implications of Spurious Regression
Let Yt = (y1t , . . . , ynt )0 denote an (n×1) vector of I(1) time series that are
0 0
not cointegrated. Using the partition Yt = (y1t , Y2t) , consider the leastsquares regression of y1t on Y2t giving the fitted model0

y1t = β̂2 Y2t + ût(12.1)

Since y1t is not cointegrated with Y2t (12.1) is a spurious regression andthe true value of β2 is zero. The following results about the behavior of β̂2
in the spurious regression (12.1) are due to Phillips (1986):• β̂2 does not converge in probability to zero but instead converges in
distribution to a non-normal random variable not necessarily centeredat zero. This is the spurious regression phenomenon.

12.2 Spurious Regression and Cointegration

435• The usual OLS t-statistics for testing that the elements of β 2 are zero
diverge to ±∞ as T → ∞. Hence, with a large enough sample it willappear that Yt is cointegrated when it is not if the usual asymptotic
normal inference is used.
• The usual R2 from the regression converges to unity as T → ∞ sothat the model will appear to fit well even though it is misspecified.
• Regression with I(1) data only makes sense when the data are cointegrated.

12.2.2 Cointegration
Let Yt = (y1t , . . . , ynt )0 denote an (n × 1) vector of I(1) time series. Yt is
cointegrated if there exists an (n × 1) vector β = (β 1 , . . . , β n )0 such that
β0 Yt = β 1 y1t + · · · + β n ynt ∼ I(0)(12.2)

In words, the nonstationary time series in Yt are cointegrated if there isa linear combination of them that is stationary or I(0). If some elementsof β are equal to zero then only the subset of the time series in Yt with
non-zero coeﬃcients is cointegrated. The linear combination β0 Yt is often
motivated by economic theory and referred to as a long-run equilibrium
relationship. The intuition is that I(1) time series with a long-run equilibrium relationship cannot drift too far apart from the equilibrium because
economic forces will act to restore the equilibrium relationship.
Normalization
The cointegration vector β in (12.2) is not unique since for any scalar cthe linear combination cβ0 Yt = β∗0 Yt ∼ I(0). Hence, some normalization
assumption is required to uniquely identify β. A typical normalization is
β = (1, −β 2 , . . . , −β n )0
so that the cointegration relationship may be expressed as
β0 Yt = y1t − β 2 y2t − · · · − β n ynt ∼ I(0)ory1t = β 2 y2t + · · · + β n ynt + ut

(12.3)

where ut ∼ I(0). In (12.3), the error term ut is often referred to as thedisequilibrium error or the cointegrating residual. In long-run equilibrium,
the disequilibrium error ut is zero and the long-run equilibrium relationshipis
y1t = β 2 y2t + · · · + β n ynt

436

12. Cointegration

Multiple Cointegrating Relationships
If the (n×1) vector Yt is cointegrated there may be 0 < r < n linearly independent cointegrating vectors. For example, let n = 3 and suppose there arer = 2 cointegrating vectors β 1 = (β 11 , β 12 , β 13 )0 and β2 = (β 21 , β 22 , β 23 )0 .
Then β01 Yt = β 11 y1t + β 12 y2t + β 13 y3t ∼ I(0), β 02 Yt = β 21 y1t + β 22 y2t +β 23 y3t ∼ I(0) and the (3 × 2) matrixµ 0 ¶ µ¶β1
β 11 β 12 β 13
0
B =
=
β 02
β 21 β 22 β 33
forms a basis for the space of cointegrating vectors. The linearly independent vectors β 1 and β2 in the cointegrating basis B are not unique unlesssome normalization assumptions are made. Furthermore, any linear combination of β1 and β2 , e.g. β3 = c1 β1 + c2 β2 where c1 and c2 are constants,
is also a cointegrating vector.
Examples of Cointegration and Common Trends in Economics andFinance
Cointegration naturally arises in economics and finance. In economics, cointegration is most often associated with economic theories that imply equilibrium relationships between time series variables. The permanent income
model implies cointegration between consumption and income, with consumption being the common trend. Money demand models imply cointegration between money, income, prices and interest rates. Growth theorymodels imply cointegration between income, consumption and investment,with productivity being the common trend. Purchasing power parity implies cointegration between the nominal exchange rate and foreign anddomestic prices. Covered interest rate parity implies cointegration between
forward and spot exchange rates. The Fisher equation implies cointegration
between nominal interest rates and inflation. The expectations hypothesisof the term structure implies cointegration between nominal interest rates
at diﬀerent maturities. The equilibrium relationships implied by these economic theories are referred to as long-run equilibrium relationships, because
the economic forces that act in response to deviations from equilibriium
may take a long time to restore equilibrium. As a result, cointegration
is modeled using long spans of low frequency time series data measured
monthly, quarterly or annually.
In finance, cointegration may be a high frequency relationship or a low
frequency relationship. Cointegration at a high frequency is motivated byarbitrage arguments. The Law of One Price implies that identical assets
must sell for the same price to avoid arbitrage opportunities. This implies
cointegration between the prices of the same asset trading on diﬀerent
markets, for example. Similar arbitrage arguments imply cointegration between spot and futures prices, and spot and forward prices, and bid and12.2 Spurious Regression and Cointegration437

ask prices. Here the terminology long-run equilibrium relationship is somewhat misleading because the economic forces acting to eliminate arbitrage
opportunities work very quickly. Cointegration is appropriately modeledusing short spans of high frequency data in seconds, minutes, hours or
days. Cointegration at a low frequency is motivated by economic equilibrium theories linking assets prices or expected returns to fundamentals. For
example, the present value model of stock prices states that a stock’s priceis an expected discounted present value of its expected future dividends or
earnings. This links the behavior of stock prices at low frequencies to the
behavior of dividends or earnings. In this case, cointegration is modeledusing low frequency data and is used to explain the long-run behavior ofstock prices or expected returns.

12.2.3 Cointegration and Common Trends
If the (n × 1) vector time series Yt is cointegrated with 0 < r < n cointegrating vectors then there are n − r common I(1) stochastic trends.
To illustrate the duality between cointegration and common trends, let
Yt = (y1t , y2t )0 ∼ I(1) and εt = (ε1t , ε2t , ε3t )0 ∼ I(0) and suppose that Yt
is cointegrated with cointegrating vector β = (1, −β 2 )0 . This cointegration
relationship may be represented as
y1t

= β2

t
X

ε1s + ε3t

s=1

y2t

=

t
X

ε1s + ε2t

s=1Pt
The common stochastic trend is s=1 ε1s . Notice that the cointegrating
relationship annihilates the common stochastic trend:0

β Yt = β 2

t
X
s=1

ε1s + ε3t − β 2

Ã t
X
s=1

ε1s + ε2t!

= ε3t − β 2 ε2t ∼ I(0).

12.2.4 Simulating Cointegrated Systems
Cointegrated systems may be conveniently simulated using Phillips’ (1991)
triangular representation. For example, consider a bivariate cointegrated
system for Yt = (y1t , y2t )0 with cointegrating vector β = (1, −β 2 )0 . A
triangular representation has the form
y1t
y2t

= β 2 y2t + ut , where ut ∼ I(0)= y2t−1 + vt , where vt ∼ I(0)(12.4)(12.5)

438

12. Cointegration

The first equation describes the long-run equilibrium relationship with anI(0) disequilibrium error ut . The second equation specifies y2t as the common stochastic trend with innovation vt :y2t = y20 +

t
X

vj .

j=1

In general, the innovations ut and vt may be contemporaneously and seriallycorrelated. The time series structure of these innovations characterizes theshort-run dynamics of the cointegrated system. The system (12.4)-(12.5)
with β 2 = 1, for example, might be used to model the behavior of thelogarithm of spot and forward prices, spot and futures prices or stock prices
and dividends.
Example 72 Simulated bivariate cointegrated system
Consider simulating T = 250 observations from the system (12.4)-(12.5)
using β = (1, −1)0 , ut = 0.75ut−1 + εt , εt ∼ iid N (0, 0.52 ) and vt ∼iid N (0, 0.52 ). The S-PLUS code is> set.seed(432)> e = rmvnorm(250, mean=rep(0,2), sd=c(0.5,0.5))> u.ar1 = arima.sim(model=list(ar=0.75), innov=e[,1])> y2 = cumsum(e[,2])> y1 = y2 + u.ar1> par(mfrow=c(2,1))> tsplot(y1, y2, lty=c(1,3),+ main="Simulated bivariate cointegrated system",+ sub="1 cointegrating vector, 1 common trend")> legend(0, 7, legend=c("y1","y2"), lty=c(1,3))> tsplot(u.ar1, main="Cointegrating residual")
Figure 12.2 shows the simulated data for y1t and y2t along with the cointegrating residual ut = y1t − y2t . Since y1t and y2t share a common stochastic trend they follow each other closely. The impulse response function for
ut may be used to determine the speed of adjustment to long-run equilibrium. Since ut is an AR(1) with φ = 0.75 the half life of a shock isln(0.5)/ ln(0.75) = 2.4 time periods.
Next, consider a trivariate cointegrated system for Yt = (y1t , y2t , y3t )0 .
With a trivariate system there may be one or two cointegrating vectors.
With one cointegrating vector there are two common stochastic trends andwith two cointegrating vectors there is one common trend. A triangular
representation with one cointegrating vector β = (1, −β 2 , −β 3 )0 and two12.2 Spurious Regression and Cointegration

439

8

Simulated bivariate cointegrated system

-2

0

2

4

6

y1
y2

0

50

100

150

200

250

200

250

1 cointegrating vector, 1 common trend

-1

0

1

2

Cointegrating residual

0

50

100

150

FIGURE 12.2. Simulated bivariate cointegrated system with β = (1, −1)0 .

stochastic trends is
y1t
y2t
y3t

= β 2 y2t + β 3 y3t + ut , where ut ∼ I(0)= y2t−1 + vt , where vt ∼ I(0)= y3t−1 + wt , where wt ∼ I(0)(12.6)(12.7)(12.8)

The first equation describes the long-run equilibrium and the second andthird equations specify the common stochastic trends. An example of atrivariate cointegrated system with one cointegrating vector is a system of
nominal exchange rates, home country price indices and foreign country
price indices. A cointegrating vector β = (1, −1, −1)0 implies that the realexchange rate is stationary.
Example 73 Simulated trivariate cointegrated system with 1 cointegrating
vectorThe S-PLUS code for simulating T = 250 observation from (12.6)-(12.8)
with β = (1, −0.5, −0.5)0 , ut = 0.75ut−1 + εt , εt ∼ iid N (0, 0.52 ), vt ∼iid N (0, 0.52 ) and wt ∼ iid N (0, 0.52 ) is
> set.seed(573)> e = rmvnorm(250, mean=rep(0,3), sd=c(0.5,0.5,0.5))> u1.ar1 = arima.sim(model=list(ar=0.75), innov=e[,1])> y2 = cumsum(e[,2])440

12. Cointegration
Simulated trivariate cointegrated system

0

5

10

y1
y2
y3

0

50

100

150

200

250

200

250

1 cointegrating vector, 2 common trends

-1

0

1

2

Cointegrating residual

0

50

100

150

FIGURE 12.3. Simulated trivariate cointegrated system with one cointegrating
vector β = (1, −0.5, −0.5)0 and two stochastic trends.

> y3 = cumsum(e[,3])> y1 = 0.5*y2 + 0.5*y3 + u1.ar1> par(mfrow=c(2,1))> tsplot(y1, y2, y3, lty=c(1,3,4),+ main="Simulated trivariate cointegrated system",+ sub="1 cointegrating vector, 2 common trends")> legend(0, 12, legend=c("y1","y2","y3"), lty=c(1,3,4))> tsplot(u.ar1, main="Cointegrating residual")
Figure 12.3 illustrates the simulated data. Here, y2t and y3t are the two
independent common trends and y1t = 0.5y2t + 0.5y3t + ut is the averageof the two trends plus an AR(1) residual.
Finally, consider a trivariate cointegrated system with two cointegrating vectors and one common stochastic trend. A triangular representation for this system with cointegrating vectors β1 = (1, 0, −β 13 )0 and
β 2 = (0, 1, −β 23 )0 is
y1t
y2t
y3t

= β 13 y3t + ut , where ut ∼ I(0)= β 23 y3t + vt , where vt ∼ I(0)= y3t−1 + wt , where wt ∼ I(0)(12.9)(12.10)(12.11)

Here the first two equations describe two long-run equilibrium relations
and the third equation gives the common stochastic trend. An example in12.2 Spurious Regression and Cointegration

441

finance of such a system is the term structure of interest rates where y3
represents the short rate and y1 and y2 represent two diﬀerent long rates.
The cointegrating relationships would indicate that the spreads between
the long and short rates are stationary.
Example 74 Simulated trivariate cointegrated system with 2 cointegrating
vectors
The S-PLUS code for simulating T = 250 observation from (12.9)-(12.11)
with β1 = (1, 0, −1)0 , β 2 = (0, 1, −1)0 , ut = 0.75ut−1 +εt , εt ∼ iid N (0, 0.52 ),vt = 0.75vt−1 + ηt , η t ∼ iid N (0, 0.52 ) and wt ∼ iid N (0, 0.52 ) is
> set.seed(573)> e = rmvnorm(250,mean=rep(0,3), sd=c(0.5,0.5,0.5))> u.ar1 = arima.sim(model=list(ar=0.75), innov=e[,1])> v.ar1 = arima.sim(model=list(ar=0.75), innov=e[,2])> y3 = cumsum(e[,3])> y1 = y3 + u.ar1> y2 = y3 + v.ar1> par(mfrow=c(2,1))> tsplot(y1, y2, y3, lty=c(1,3,4),+ main="Simulated trivariate cointegrated system",+ sub="2 cointegrating vectors, 1 common trend")> legend(0, 10, legend=c("y1","y2","y3"), lty=c(1,3,4))> tsplot(u.ar1, v.ar1, lty=c(1,3),+ main="Cointegrated residuals")> legend(0, -1, legend=c("u","v"), lty=c(1,3))

12.2.5 Cointegration and Error Correction Models
Consider a bivariate I(1) vector Yt = (y1t , y2t )0 and assume that Yt is
cointegrated with cointegrating vector β = (1, −β 2 )0 so that β 0 Yt = y1t −β 2 y2t is I(0). In an extremely influential and important paper, Engle andGranger (1987) showed that cointegration implies the existence of an error
correction model (ECM) of the form
∆y1t

= c1 + α1 (y1t−1 − β 2 y2t−1 )
X j
X j
+
ψ 11 ∆y1t−j +ψ 12 ∆y2t−j + ε1t
j

∆y2t

j

= c2 + α2 (y1t−1 − β 2 y2t−1 )X jX+ψ 21 ∆y1t−j +ψ 222 ∆y2t−j + ε2tj(12.12)(12.13)jthat describes the dynamic behavior of y1t and y2t . The ECM links thelong-run equilibrium relationship implied by cointegration with the shortrun dynamic adjustment mechanism that describes how the variables react

442

12. Cointegration

10

Simulated trivariate cointegrated system

0

5

y1
y2
y3

0

50

100

150

200

250

200

250

2 cointegrating vectors, 1 common trend

-1

0

1

Cointegrated residuals

-2

u
v

0

50

100

150

FIGURE 12.4. Simulated trivatiate cointegrated system with two cointegrating
vectors β 1 = (1, 0, −1)0 , β 2 = (0, 1, −1)0 and one common trend.

when they move out of long-run equilibrium. This ECM makes the concept
of cointegration useful for modeling financial time series.
Example 75 Bivariate ECM for stock prices and dividends
As an example of an ECM, let st denote the log of stock prices and dt
denote the log of dividends and assume that Yt = (st , dt )0 is I(1). If thelog dividend-price ratio is I(0) then the logs of stock prices and dividends
are cointegrated with β = (1, −1)0 . That is, the long-run equilibrium is
dt = st + μ + ut
where μ is the mean of the log dividend-price ratio, and ut is an I(0) randomvariable representing the dynamic behavior of the log dividend-price ratio(disequilibrium error). Suppose the ECM has the form∆st∆dt

= cs + αs (dt−1 − st−1 − μ) + εst= cd + αd (dt−1 − st−1 − μ) + εdt

where cs > 0 and cd > 0. The first equation relates the growth rate ofdividends to the lagged disequilibrium error dt−1 −st−1 −μ, and the second
equation relates the growth rate of stock prices to the lagged disequilibrium
as well. The reactions of st and dt to the disequilibrium error are capturedby the adjustment coeﬃcients αs and αd .

12.2 Spurious Regression and Cointegration

443

Consider the special case of (12.12)-(12.13) where αd = 0 and αs = 0.5.
The ECM equations become∆st∆dt= cs + 0.5(dt−1 − st−1 − μ) + εst ,= cd + εdt .

so that only st responds to the lagged disequilibrium error. Notice thatE[∆st |Yt−1 ] = cs + 0.5(dt−1 − st−1 − μ) and E[∆dt |Yt−1 ] = cd . Consider
three situations:
1. dt−1 −st−1 −μ = 0. Then E[∆st |Yt−1 ] = cs and E[∆dt |Yt−1 ] = cd , sothat cs and cd represent the growth rates of stock prices and dividends
in long-run equilibrium.
2. dt−1 − st−1 − μ > 0. Then E[∆st |Yt−1 ] = cs + 0.5(dt−1 − st−1 − μ) >cs . Here the dividend yield has increased above its long-run mean(positive disequilibrium error) and the ECM predicts that st will growfaster than its long-run rate to restore the dividend yield to its longrun mean. Notice that the magnitude of the adjustment coeﬃcientαs = 0.5 controls the speed at which st responds to the disequilibrium
error.
3. dt−1 − st−1 − μ < 0. Then E[∆st |Yt−1 ] = cs + 0.5(dt−1 − st−1 − μ) <cs . Here the dividend yield has decreased below its long-run mean
(negative disequilibrium error) and the ECM predicts that st willgrow more slowly than its long-run rate to restore the dividend yield
to its long-run mean.
In Case 1, there is no expected adjustment since the model was in longrun equilibrium in the previous period. In Case 2, the model was above
long-run equilibrium last period so the expected adjustment in st is downward toward equilibrium. In Case 3, the model was below long-run equilibrium last period and so the expected adjustment is upward toward theequilibrium. This discussion illustrates why the model is called an error correction model. When the variables are out of long-run equilibrium, there
are economic forces, captured by the adjustment coeﬃcients, that pushthe model back to long-run equilibrium. The speed of adjustment toward
equilibrium is determined by the magnitude of αs . In the present example,
αs = 0.5 which implies that roughly one half of the disequilibrium error
is corrected in one time period. If αs = 1 then the entire disequilibrium
is corrected in one period. If αs = 1.5 then the correction overshoots thelong-run equilibrium.

444

12. Cointegration

12.3 Residual-Based Tests for Cointegration
Let the (n ×1) vector Yt be I(1). Recall, Yt is cointegrated with 0 < r < ncointegrating vectors if there exists an (r × n) matrix B0 such that
⎞ ⎛
⎞
⎛ 0
u1t
β1 Yt
⎟ ⎜ .. ⎟⎜..
B0 Yt = ⎝⎠ = ⎝ . ⎠ ∼ I(0).
β0r Yturt

Testing for cointegration may be thought of as testing for the existenceof long-run equilibria among the elements of Yt . Cointegration tests cover
two situations:• There is at most one cointegrating vector• There are possibly 0 ≤ r < n cointegrating vectors.

The first case was originally considered by Engle and Granger (1986) andthey developed a simple two-step residual-based testing procedure basedon regression techniques. The second case was originally considered by Johansen (1988) who developed a sophisticated sequential procedure for determining the existence of cointegration and for determining the number ofcointegrating relationships based on maximum likelihood techniques. Thissection explains Engle and Granger’s two-step procedure. Johansen’s moregeneral procedure will be discussed later on.
Engle and Granger’s two-step procedure for determining if the (n × 1)
vector β is a cointegrating vector is as follows:• Form the cointegrating residual β0 Yt = ut
• Perform a unit root test on ut to determine if it is I(0).
The null hypothesis in the Engle-Granger procedure is no-cointegration andthe alternative is cointegration. There are two cases to consider. In the firstcase, the proposed cointegrating vector β is pre-specified (not estimated).
For example, economic theory may imply specific values for the elementsin β such as β = (1, −1)0 . The cointegrating residual is then readily constructed using the prespecified cointegrating vector. In the second case, theproposed cointegrating vector is estimated from the data and an estimate
0of the cointegrating residual β̂ Yt = ût is formed. Tests for cointegration
using a pre-specified cointegrating vector are generally much more powerfulthan tests employing an estimated vector.

12.3.1 Testing for Cointegration When the Cointegrating
Vector Is Pre-specified
Let Yt denote an (n × 1) vector of I(1) time series, let β denote an (n × 1)
prespecified cointegrating vector and let ut = β0 Yt denote the prespecified

12.3 Residual-Based Tests for Cointegration

445

0.00

Log US/CA exchange rate data

-0.35

-0.15

spot
forward

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

-0.004

0.000

0.004

US/CA 30-day interest rate differential

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996

FIGURE 12.5. Log of US/CA spot and 30-day exchange rates and 30-day interest
rate diﬀerential.

cointegrating residual. The hypotheses to be tested are
H0
H1: ut = β0 Yt ∼ I(1) (no cointegration): ut = β0 Yt ∼ I(0) (cointegration)(12.14)

Any unit root test statistic may be used to evaluate the above hypotheses.
The most popular choices are the ADF and PP statistics. Cointegration isfound if the unit root test rejects the no-cointegration null. It should be kept
in mind, however, that the cointegrating residual may include deterministic
terms (constant or trend) and the unit root tests should account for theseterms accordingly. See Chapter 4 for details about the application of unit
root tests.
Example 76 Testing for cointegration between spot and forward exchange
rates using a known cointegrating vector
In international finance, the covered interest rate parity arbitrage relationship states that the diﬀerence between the logarithm of spot andforward exchange rates is equal to the diﬀerence between nominal domestic and foreign interest rates. It seems reasonable to believe that interest
rate spreads are I(0) which implies that spot and forward rates are cointegrated with cointegrating vector β = (1, −1)0 . To illustrate, consider thelog monthly spot, st , and 30 day forward, ft , exchange rates between the446

12. Cointegration

US and Canada over the period February 1976 through June 1996 takenfrom the S+FinMetrics “timeSeries” object lexrates.dat> uscn.s = lexrates.dat[,"USCNS"]> uscn.s@title = "Log of US/CA spot exchange rate"> uscn.f = lexrates.dat[,"USCNF"]> uscn.f@title = "Log of US/CA 30-day forward exchange rate"> u = uscn.s - uscn.f> colIds(u) = "USCNID"> u@title = "US/CA 30-day interest rate differential"

The interest rate diﬀerential is constructed using the pre-specified cointegrating vector β = (1, −1)0 as ut = st − ft . The spot and forward exchange
rates and interest rate diﬀerential are illustrated in Figure 12.5. Visually,
the spot and forward exchange rates clearly share a common trend and theinterest rate diﬀerential appears to be I(0). In addition, there is no clear deterministic trend behavior in the exchange rates. The S+FinMetrics function unitroot may be used to test the null hypothesis that st and ft are
not cointegrated (ut ∼ I(1)). The ADF t-test based on 11 lags and a constant in the test regression leads to the rejection at the 5% level of the
hypothesis that st and ft are not cointegrated with cointegrating vector
β = (1, −1)0 :> unitroot(u, trend="c", method="adf", lags=11)
Test for Unit Root: Augmented DF Test
Null Hypothesis: there is a unit root
Type of Test: t-test
Test Statistic: -2.881
P-value: 0.04914
Coefficients:
lag1
lag2
lag3
lag4
lag5
lag6
-0.1464 -0.1171 -0.0702 -0.1008 -0.1234 -0.1940
lag8-0.1235lag90.0550

lag10
lag11 constant
0.2106 -0.1382 0.0002

Degrees of freedom: 234 total; 222 residual
Time period: from Jan 1977 to Jun 1996
Residual standard error: 8.595e-4

lag7
0.0128

12.3 Residual-Based Tests for Cointegration

447

12.3.2 Testing for Cointegration When the Cointegrating
Vector Is EstimatedLet Yt denote an (n × 1) vector of I(1) time series and let β denote an(n × 1) unknown cointegrating vector. The hypotheses to be tested are
given in (12.14). Since β is unknown, to use the Engle-Granger procedure
it must be first estimated from the data. Before β can be estimated somenormalization assumption must be made to uniquely identify it. A common
normalization is to specify the first element in Yt as the dependent variable
0 0
and the rest as the explanatory variables. Then Yt = (y1t , Y2t) where0
Y2t = (y2t , . . . , ynt ) is an ((n − 1) × 1) vector and the cointegrating vector
is normalized as β = (1, −β02 )0 . Engle and Granger proposed estimating thenormalized cointegrating vector β2 by least squares from the regression
y1t = c + β02 Y2t + ut

(12.15)and testing the no-cointegration hypothesis (12.14) with a unit root test
using the estimated cointegrating residualût = y1t − ĉ − β̂ 2 Y2t(12.16)

where ĉ and β̂2 are the least squares estimates of c and β2 . The unit root
test regression in this case is without deterministic terms (constant or constant and trend). Phillips and Ouliaris (1990) showed that ADF and PP
unit root tests (t-tests and normalized bias) applied to the estimated cointegrating residual (12.16) do not have the usual Dickey-Fuller distributions
under the null hypothesis (12.14) of no-cointegration. Instead, due to thespurious regression phenomenon under the null hypothesis (12.14), the distribution of the ADF and PP unit root tests have asymptotic distributions
that are functions of Wiener processes that depend on the deterministic
terms in the regression (12.15) used to estimate β2 and the number of variables, n − 1, in Y2t . These distributions are known as the Phillips-Ouliaris(PO) distributions, and are described in Phillips and Ouliaris (1990). To
further complicate matters, Hansen (1992) showed the appropriate PO distributions of the ADF and PP unit root tests applied to the residuals(12.16) also depend on the trend behavior of y1t and Y2t as follows:
Case I: Y2t and y1t are both I(1) without drift. The ADF and PP unit
root test statistics follow the PO distributions, adjusted for a constant, with dimension parameter n − 1.
Case II: Y2t is I(1) with drift and y1t may or may not be I(1) with drift.
The ADF and PP unit root test statistics follow the PO distributions,adjusted for a constant and trend, with dimension parameter n − 2.
If n − 2 = 0 then the ADF and PP unit root test statistics follow the
DF distributions adjusted for a constant and trend.

448

12. Cointegration

Case III: Y2t is I(1) without drift and y1t is I(1) with drift. In this case,β2 should be estimated from the regression
y1t = c + δt + β02 Y2t + ut

(12.17)

The resulting ADF and PP unit root test statistics on the residuals
from (12.17) follow the PO distributions, adjusted for a constant andtrend, with dimension parameter n − 1.
Computing Quantiles and P-values from the Phillips-Ouliaris
Distributions Using the S+FinMetrics Functions pcoint and qcoint
The S+FinMetrics functions qcoint and pcoint, based on the response
surface methodology of MacKinnon (1996), may be used to compute quantiles and p-values from the PO distributions. For example, to compute the
10%, 5% and 1% quantiles from the PO distribution for the ADF t-statistic,
adjusted for a constant, with n − 1 = 3 and a sample size T = 100 use> qcoint(c(0.1,0.05,0.01), n.sample=100, n.series=4,+ trend="c", statistic="t")[1] -3.8945 -4.2095 -4.8274
Notice that the argument n.series represents the total number of variables
n. To adjust the PO distributions for a constant and trend set trend="ct".
To compute the PO distribution for the ADF normalized bias statistic
set statistic="n". The quantiles from the PO distributions can be very
diﬀerent from the quantiles from the DF distributions, especially if n − 1 islarge. To illustrate, the 10%, 5% and 1% quantiles from the DF distribution
for the ADF t-statistic with a sample size T = 100 are> qunitroot(c(0.1,0.05,0.01), n.sample=100,+ trend="c", statistic="t")[1] -2.5824 -2.8906 -3.4970
The following examples illustrate testing for cointegration using an estimated cointegrating vector.
Example 77 Testing for cointegration between spot and forward exchange
rates using an estimated cointegrating vector
Consider testing for cointegration between spot and forward exchange
rates assuming the cointegrating vector is not known using the same data asin the previous example. Let Yt = (st , ft )0 and normalize the cointegrating
vector on st so that β = (1, −β 2 )0 . The normalized cointegrating coeﬃcientβ 2 is estimated by least squares from the regression
st = c + β 2 ft + ut
giving the estimated cointegrating residual ût = st − ĉ − β̂ 2 ft . The OLS
function in S+FinMetrics is used to estimate the above regression:

12.3 Residual-Based Tests for Cointegration449> uscn.ts = seriesMerge(uscn.s,uscn.f)> ols.fit = OLS(USCNS~USCNF,data=uscn.ts)> ols.fit
Call:
OLS(formula = USCNS ~USCNF, data = uscn.ts)
Coefficients:(Intercept) USCNF
0.00231.0041Degrees of freedom: 245 total; 243 residual
Time period: from Feb 1976 to Jun 1996
Residual standard error: 0.001444
The estimated value of β 2 is 1.004 and is almost identical to the value
β 2 = 1 implied by covered interest parity. The estimated cointegrating
residual ût is extracted from the least squres fit using> u.hat = residuals(ols.fit)
Next, the no-cointegration hypothesis (12.14) is tested using the ADF and
PP t-tests. Because the mean of ût is zero, the unit root test regressions are
estimated without a constant or trend. The ADF t-statistic is computed
using 11 lags, as in the previous example, and the PP t-statistic is computed
using an automatic lag truncation parameter:> adf.fit = unitroot(u.hat,trend="nc",method="adf",lags=11)> adf.tstat = adf.fit$sval> adf.tstat
lag1-2.721> pp.fit = unitroot(u.hat,trend="nc",method="pp")> pp.tstat = pp.fit$sval> pp.tstat
lag1-5.416The ADF t-statistic is −2.721 whereas the PP t-statistic is −5.416. Since
st and ft are both I(1) without drift, the 10%, 5% and 1% quantiles fromthe approrpiate Phillips-Ouliaris distribution for the ADF t-statistic is> qcoint(c(0.10,0.05,0.01),n.sample=nrow(uscn.s),n.series=2,+ trend="c",statistic="t")[1] -3.062 -3.361 -3.942

450

12. Cointegration

The no-cointegration null hypothesis is not rejected at the 10% level using
the ADF t-statistic but is rejected at the 1% level using the PP t-statistic.
The p-values for the ADF and PP t-statistics are> pcoint(adf.tstat, n.sample=nrow(uscn.s), n.series=2,+ trend="c", statistic="t")[1] 0.1957> pcoint(pp.tstat, n.sample=nrow(uscn.s), n.series=2,+ trend="c", statistic="t")[1] 0.00003925

12.4 Regression-Based Estimates of Cointegrating
Vectors and Error Correction Models
12.4.1 Least Square Estimator
Least squares may be used to consistently estimate a normalized cointegrating vector. However, the asymptotic behavior of the least squares estimator is non-standard. The following results about the behavior of β̂2 if
Yt is cointegrated are due to Stock (1987) and Phillips (1991):• T (β̂2 −β 2 ) converges in distribution to a non-normal random variable
not necessarily centered at zero.
• The least squares estimate β̂2 is consistent for β2 and converges to
β2 at rate T instead of the usual rate T 1/2 . That is, β̂2 is superconsistent.
• β̂2 is consistent even if Y2t is correlated with ut so that there is no
asymptotic simultaneity bias.
• In general, the asymptotic distribution of T (β̂2 − β 2 ) is asymptotically biased and non-normal. The usual OLS formula for computinga[var(β̂2 ) is incorrect and so the usual OLS standard errors are notcorrect.
• Even though the asymptotic bias goes to zero as T gets large β̂2 may
be substantially biased in small samples. The least squres estimatoris also not eﬃcient.
The above results indicate that the least squares estimator of the cointegrating vector β2 could be improved upon. A simple improvement issuggested by Stock and Watson (1993).

12.4 Regression-Based Estimates and Error Correction Models

451

12.4.2 Stock and Watson’s Eﬃcient Lead/Lag Estimator
Stock and Watson (1993) provided a very simple method for obtaining
an asymptotically eﬃcient (equivalent to maximum likelihood) estimator
for the normalized cointegrating vector β2 as well as a valid formula forcomputing its asymptotic variance2 .
0 0
Let Yt = (y1t , Y2t) where Y2t = (y2t , . . . , ynt )0 is an ((n − 1) × 1) vector
and let the cointegrating vector be normalized as β = (1, −β02 )0 . Stock andWatson’s eﬃcient estimation procedure is:• Augment the cointegrating regression of y1t on Y2t with appropriate
deterministic terms Dt with p leads and lags of ∆Y2t
y1t

= γ 0 Dt + β02 Y2t +=p
Xψ 0j ∆Y2t+j + ut

(12.18)j=−p00
γ Dt + β2 Y2t + ψ 0p ∆Y2t+p + · · · + ψ 01 ∆Y2t+1+ψ 00 ∆Y2t + ψ 0−1 ∆Y2t−1 + · · · + ψ 0−p ∆Y2t−p + ut

• Estimate the augmented regression by least squares. The resulting
estimator of β2 is called the dynamic OLS estimator and is denoted
β̂2,DOLS . It will be consistent, asymptotically normally distributedand eﬃcient (equivalent to MLE) under certain assumptions (seeStock and Watson, 1993).
• Asymptotically valid standard errors for the individual elements ofβ̂2,DOLS are given by the OLS standard errors from (12.18) multipliedby the ratio
!1/2
Ã
σ̂ 2u
c t)
lrv(u

c t ) is any consistentwhere σ̂ 2u is the OLS estimate of var(ut ) and lrv(uestimate of the long-run variance of ut using the residuals ût from
(12.18). Alternatively, the Newey-West HAC standard errors may also
be used.

Example 78 DOLS estimation of cointegrating vector using exchange ratedata3
Let st denote the log of the monthly spot exchange rate between two
currencies at time t and let ftk denote the log of the forward exchange
rate at time t for delivery of foreign currency at time t + k. Under rational
2 Hamilton (1994) chapter 19, and Hayashi (2000) chapter 10, give nice discussions of
the Stock and Watson procedure.
3 This example is based on Zivot (2000).

452

12. Cointegration

expectations and risk neutrality ftk is an unbiased predictor of st+k , thespot exchange rate at time t + k. That is
st+k = ftk + εt+kwhere εt+k is a white noise error term. This is known as the forward
rate unbiasedness hypothesis (FRUH). Assuming that st and ftk are I(1)the FRUH implies that st+k and ftk are cointegrated with cointegrating vector β = (1, −1)0 . To illustrate, consider again the log monthly
spot, st , and one month forward, ft1 , exchange rates between the US andCanada over the period February 1976 through June 1996 taken from the
S+FinMetrics “timeSeries” object lexrates.dat.The cointegrating vector between st+1 and ft1 is estimated using least squares and Stock andWatson’s dynamic OLS estimator computed from (12.18) with y1t = st+1 ,Dt = 1, Y2t = ft1 and p = 3. The data for the DOLS regression equation(12.18) are constucted as> uscn.df = diff(uscn.f)> colIds(uscn.df) = "D.USCNF"> uscn.df.lags = tslag(uscn.df,-3:3,trim=T)> uscn.ts = seriesMerge(uscn.s,uscn.f,uscn.df.lags)> colIds(uscn.ts)[1] "USCNS""USCNF""D.USCNF.lead3"[4] "D.USCNF.lead2" "D.USCNF.lead1" "D.USCNF.lag0"[7] "D.USCNF.lag1" "D.USCNF.lag2" "D.USCNF.lag3"
The least squares estimator of the normalized cointegrating coeﬃcient β 2computed using the S+FinMetrics function OLS is> summary(OLS(tslag(USCNS,-1)~USCNF,data=uscn.ts,na.rm=T))
Call:
OLS(formula = tslag(USCNS, -1) ~USCNF, data = uscn.ts,
na.rm = T)
Residuals:
Min
1Q-0.0541 -0.0072

Median0.00063Q
0.0097Max
0.0343

Coefficients:(Intercept)
USCNF

Value Std. Error-0.00480.00250.97670.0110Regression Diagnostics:
R-Squared 0.9709

t value Pr(>|t|)-1.96140.051088.6166
0.000012.4 Regression-Based Estimates and Error Correction Models

453

Adjusted R-Squared 0.9708
Durbin-Watson Stat 2.1610
Residual standard error: 0.01425 on 235 degrees of freedom
Time period: from Jun 1976 to Feb 1996
F-statistic: 7853 on 1 and 235 degrees of freedom,
the p-value is 0
Notice that in the regression formula, tslag(USCN,-1) computes st+1 . The
least squares estimate of β 2 is 0.977 with an estimated standard error of
0.011 indicating that ft1 underpredicts st+1 . However, the usual formulafor computing the estimated standard error is incorrect and should not be
trusted.
The DOLS estimator of β 2 based on (12.18) is computed using> dols.fit = OLS(tslag(USCNS,-1)~USCNF ++ D.USCNF.lead3+D.USCNF.lead2+D.USCNF.lead1 ++ D.USCNF.lag0+D.USCNF.lag1+D.USCNF.lag2+D.USCNF.lag3,+ data=uscn.ts,na.rm=T)
The Newey-West HAC standard errors for the estimated coeﬃcients arecomputed using summary with correction="nw":> summary(dols.fit,correction="nw")
Call:
OLS(formula = tslag(USCNS, -1) ~USCNF + D.USCNF.lead3 +D.USCNF.lead2 + D.USCNF.lead1 + D.USCNF.lag0 +D.USCNF.lag1 + D.USCNF.lag2 + D.USCNF.lag3, data =uscn.ts, na.rm = T)
Residuals:
Min
1Q-0.0061 -0.0008

Median0.00003Q
0.0009Max
0.0039Coefficients:(Intercept)
USCNF
D.USCNF.lead3
D.USCNF.lead2
D.USCNF.lead1
D.USCNF.lag0
D.USCNF.lag1
D.USCNF.lag2
D.USCNF.lag3

Value Std. Error
0.00230.00051.00400.00190.01140.00630.02270.00681.01450.00900.00050.0073-0.00420.0061-0.0056
0.0061-0.00140.0045t value
4.3948531.8862
1.80433.3226112.4060
0.0719-0.6856-0.9269-0.3091

Pr(>|t|)0.00000.00000.07250.00100.00000.94270.49370.35490.7575454

12. Cointegration

Regression Diagnostics:
R-Squared 0.9997
Adjusted R-Squared 0.9997
Durbin-Watson Stat 0.4461
Residual standard error: 0.001425 on 228 degrees of freedom
Time period: from Jun 1976 to Feb 1996
F-statistic: 101000 on 8 and 228 degrees of freedom,
the p-value is 0
The DOLS estimator of β 2 is 1.004 with a very small estimated standard
error of 0.0019 and indicates that ft1 is essentially an unbiased predictor of
the future spot rate st+1 .

12.4.3 Estimating Error Correction Models by Least Squares
Consider a bivariate I(1) vector Yt = (y1t , y2t )0 and assume that Yt is cointegrated with cointegrating vector β = (1, −β 2 )0 so that β0 Yt = y1t −β 2 y2t
is I(0). Suppose one has a consistent estimate β̂ 2 (by OLS or DOLS) of
the cointegrating coeﬃcient and is interested in estimating the corresponding error correction model (12.12)-(12.13) for ∆y1t and ∆y2t . Because β̂ 2
is super consistent it may be treated as known in the ECM, so that theestimated disequilibrium error y1t − β̂ 2 y2t may be treated like the known
disequilibrium error y1t − β 2 y2t . Since all variables in the ECM are I(0),the two regression equations may be consistently estimated using ordinary
least squares (OLS). Alternatively, the ECM system may be estimated byseemingly unrelated regressions (SUR) to increase eﬃciency if the numberof lags in the two equations are diﬀerent.
Example 79 Estimation of error correction model for exchange rate data
Consider again the monthly log spot rate, st , and log forward rate, ft ,data between the U.S. and Canada. Earlier it was shown that st and ft arecointegrated with an estimated cointegrating coeﬃcient β̂ 2 = 1.004. Now
consider estimating an ECM of the form (12.12)-(12.13) by least squares
using the estimated disequilibrium error st − 1.004 · ft . In order to estimate
the ECM, the number of lags of ∆st and ∆ft needs to be determined. Thismay be done using test statistics for the significance of the lagged terms
or model selection criteria like AIC or BIC. An initial estimation using one
lag of ∆st and ∆ft may be performed using> u.hat = uscn.s - 1.004*uscn.f> colIds(u.hat) = "U.HAT"> uscn.ds = diff(uscn.s)> colIds(uscn.ds) = "D.USCNS"

12.5 VAR Models and Cointegration455> uscn.df = diff(uscn.f)> colIds(uscn.df) = "D.USCNF"> uscn.ts = seriesMerge(uscn.s,uscn.f,uscn.ds,uscn.df,u.hat)> ecm.s.fit = OLS(D.USCNS~tslag(U.HAT)+tslag(D.USCNS)+ +tslag(D.USCNF),data=uscn.ts,na.rm=T)> ecm.f.fit = OLS(D.USCNF~tslag(U.HAT)+tslag(D.USCNS)++ tslag(D.USCNF),data=uscn.ts,na.rm=T)
The estimated coeﬃcients from the fitted ECM are> ecm.s.fit
Call:
OLS(formula = D.USCNS ~tslag(U.HAT) + tslag(D.USCNS) + tslag(
D.USCNF), data = uscn.ts, na.rm = T)
Coefficients:(Intercept) tslag(U.HAT) tslag(D.USCNS) tslag(D.USCNF)-0.0050
1.56211.2683-1.3877
Degrees of freedom: 243 total; 239 residual
Time period: from Apr 1976 to Jun 1996
Residual standard error: 0.013605> ecm.f.fit
Call:
OLS(formula = D.USCNF ~tslag(U.HAT) + tslag(D.USCNS) + tslag(
D.USCNF), data = uscn.ts, na.rm = T)
Coefficients:(Intercept) tslag(U.HAT) tslag(D.USCNS) tslag(D.USCNF)-0.00541.75471.3595-1.4702
Degrees of freedom: 243 total; 239 residual
Time period: from Apr 1976 to Jun 1996
Residual standard error: 0.013646

12.5 VAR Models and Cointegration
The Granger representation theorem links cointegration to error correction
models. In a series of important papers and in a marvelous textbook, Soren
Johansen firmly roots cointegration and error correction models in a vector
autoregression framework. This section outlines Johansen’s approach tocointegration modeling.

456

12. Cointegration

12.5.1 The Cointegrated VAR
Consider the levels VAR(p) model for the (n × 1) vector Yt
Yt = ΦDt + Π1 Yt−1 + · · · + Πp Yt−p + εt , t = 1, . . . , T,

(12.19)where Dt contains deterministic terms (constant, trend, seasonal dummiesetc.). Recall, the VAR(p) model is stable ifdet(In − Π1 z − · · · − Πp z p ) = 0(12.20)has all roots outside the complex unit circle. If (12.20) has a root on theunit circle then some or all of the variables in Yt are I(1) and they may
also be cointegrated. Recall, Yt is cointegrated if there exists some linear
combination of the variables in Yt that is I(0). Suppose Yt is I(1) andpossibly cointegrated. Then the VAR representation (22.11) is not the mostsuitable representation for analysis because the cointegrating relations are
not explicitly apparent. The cointegrating relations become apparent if
the levels VAR (22.11) is transformed to the vector error correction model(VECM)∆Yt = ΦDt + ΠYt−1 + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt(12.21)

Pp

where Π = Π1 +· · ·+Πp −In and Γk = − j=k+1 Πj , k = 1, . . . , p−1. The
matrix Π is called the long-run impact matrix and Γk are the short-run
impact matrices. Notice that the VAR parameters Πi may be recoveredfrom the VECM parameters Π and Γk via
Π1Πk= Γ1 + Π + In ,= Γk − Γk−1 , k = 2, . . . , p.

(12.22)In the VECM (12.21), ∆Yt and its lags are I(0). The term ΠYt−1 isthe only one which includes potential I(1) variables and for ∆Yt to beI(0) it must be the case that ΠYt−1 is also I(0). Therefore, ΠYt−1 must
contain the cointegrating relations if they exist. If the VAR(p) process has
unit roots then from (12.20) it is clear that Π is a singular matrix. If Π issingular then it has reduced rank ; that is rank(Π) = r < n. There are twocases to consider:
1. rank(Π) = 0. This implies that Π = 0 and Yt is I(1) and not cointegrated. The VECM (12.21) reduces to a VAR(p−1) in first diﬀerences∆Yt = ΦDt + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt .
2. 0 < rank(Π) = r < n. This implies that Yt is I(1) with r linearly
independent cointegrating vectors and n−r common stochastic trends

12.5 VAR Models and Cointegration

457(unit roots)4 . Since Π has rank r it can be written as the product
Π = α

(n×n)

β

0

(n×r)(r×n)

where α and β are (n × r) matrices with rank(α) = rank(β) = r.
The rows of β 0 form a basis for the r cointegrating vectors and theelements of α distribute the impact of the cointegrating vectors to
the evolution of ∆Yt . The VECM (12.21) becomes∆Yt = ΦDt + αβ0 Yt−1 + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt ,(12.23)where β0 Yt−1 ∼ I(0) since β0 is a matrix of cointegrating vectors.
It is important to recognize that the factorization Π = αβ0 is not uniquesince for any r × r nonsingular matrix H we have
αβ0 = αHH−1 β0 = (aH)(βH−10 )0 = a∗ β∗0 .
Hence the factorization Π = αβ0 only identifies the space spanned by the
cointegrating relations. To obtain unique values of α and β0 requires further
restrictions on the model.
Example 80 A bivariate cointegrated VAR(1) model
Consider the bivariate VAR(1) model for Yt = (y1t , y2t )0
Yt = Π1 Yt−1 + εt .
The VECM is∆Yt = ΠYt−1 + εtwhere Π = Π1 − I2 . Assuming Yt is cointegrated there exists a 2 × 1
vector β = (β 1 , β 2 )0 such that β0 Yt = β 1 y1t + β 2 y2t is I(0). Using thenormalization β 1 = 1 and β 2 = −β the cointegrating relation becomesβ 0 Yt = y1t − βy2t . This normalization suggests the stochastic long-run
equilibrium relation
y1t = βy2t + ut
where ut is I(0) and represents the stochastic deviations from the long-run
equilibrium y1t = βy2t .
Since Yt is cointegrated with one cointegrating vector, rank(Π) = 1 andcan be decomposed as
¶
µ
µ
¶
¢
α1 −α1 β
α1 ¡1 −β =
Π = αβ0 =.
α2 −α2 βα24 To see that Y has n−r common stochastic trends we have to look at the Beveridget
Nelson decomposition of the moving average representation of ∆Yt .

458

12. Cointegration

The elements in the vector α are interpreted as speed of adjustment coefficients. The cointegrated VECM for ∆Yt may be rewritten as∆Yt = αβ0 Yt−1 + εt .

(12.24)

Writing the VECM equation by equation gives
∆y1t
∆y2t

= α1 (y1t−1 − βy2t−1 ) + ε1t ,= α2 (y1t−1 − βy2t−1 ) + ε2t .

The first equation relates the change in y1t to the lagged disequilibrium
error β0 Yt−1 = (y1t−1 −βy2t−1 ) and the second equation relates the changein ∆y2t to the lagged disequilibrium error as well. Notice that the reactionsof y1 and y2 to the disequilibrium errors are captured by the adjustment
coeﬃcients α1 and α2 .
The stability conditions for the bivariate VECM are related to the stability conditions for the disequilibrium error β 0 Yt . By pre-multiplying (12.24)
by β0 , it is straightforward to show that β0 Yt follows an AR(1) process
00

β0 Yt = (1 + β α)β Yt−1 + β0 εtorut = φut−1 + vt
where ut = β0 Yt , φ = 1+β0 α = 1+ (α1 −βα2 ) and vt = β0 εt = u1t −βu2t .
The AR(1) model for ut is stable as long as |φ| = |1 + (α1 − βα2 )| < 1. For
example, suppose β = 1. Then the stability condition is |φ| = |1 + (α1 −α2 )| < 1 which is satisfied if α1 − α2 < 0 and α1 − α2 > −2. If α2 = 0 then
−2 < α1 < 0 is the required stability condition.

12.5.2 Johansen’s Methodology for Modeling Cointegration
The basic steps in Johansen’s methodology are:• Specify and estimate a VAR(p) model for Yt .
• Construct likelihood ratio tests for the rank of Π to determine thenumber of cointegrating vectors.
• If necessary, impose normalization and identifying restrictions on the
cointegrating vectors.
• Given the normalized cointegrating vectors estimate the resulting
cointegrated VECM by maximum likelihood.

12.5 VAR Models and Cointegration459

12.5.3 Specification of Deterministic Terms
Following Johansen (1995), the deterministic terms in (12.23) are restricted
to the form
ΦDt = μt = μ0 + μ1 t
If the deterministic terms are unrestricted then the time series in Yt may
exhibit quadratic trends and there may be a linear trend term in the cointegrating relationships. Restricted versions of the trend parameters μ0 andμ1 limit the trending nature of the series in Yt . The trend behavior of Yt
can be classified into five cases:
1. Model H2 (r): μt = 0 (no constant). The restricted VECM is∆Yt = αβ 0 Yt−1 + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt ,and all the series in Yt are I(1) without drift and the cointegrating
relations β 0 Yt have mean zero.
2. Model H1∗ (r): μt = μ0 = αρ0 (restricted constant). The restricted
VECM is∆Yt = α(β0 Yt−1 + ρ0 ) + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt ,the series in Yt are I(1) without drift and the cointegrating relations
β0 Yt have non-zero means ρ0 .
3. Model H1 (r): μt = μ0 (unrestricted constant). The restricted VECM
is
∆Yt = μ0 + αβ0 Yt−1 + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt
the series in Yt are I(1) with drift vector μ0 and the cointegrating
relations β 0 Yt may have a non-zero mean.
4. Model H ∗ (r): μt = μ0 + αρ1 t (restricted trend). The restricted
VECM is∆Yt= μ0 + α(β0 Yt−1 + ρ1 t)+Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt

the series in Yt are I(1) with drift vector μ0 and the cointegrating
relations β 0 Yt have a linear trend term ρ1 t.
5. Model H(r): μt = μ0 + μ1 t (unrestricted constant and trend). The
unrestricted VECM is∆Yt = μ0 + μ1 t + αβ0 Yt−1 +Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt ,the series in Yt are I(1) with a linear trend (quadratic trend in levels)and the cointegrating relations β0 Yt have a linear trend.

460

12. Cointegration

Case 2: Restricted constant

-2

0

5

2 4 6 810 15Case 1: No constant

0

50

100

150

200

0

100

150

200

Case 4: Restricted trend

40

40

80

80

120

120 160

Case 3: Unrestricted constant

50

0

50

100

150

200

0

50

100

150

200

500 1500

3000

Case 5: Unrestricted trend

0

50

100

150

200

FIGURE 12.6. Simulated Yt from bivariate cointegrated VECM for five trendcases.
Case 2: Restricted constant

6

-2

8

0

2

10

4

12

Case 1: No constant

0

50

100

150

200

0

100

150

200

Case 4: Restricted trend

4

15

6

20

8

25

10

30

Case 3: Unrestricted constant

50

0

50

100

150

200

0

50

100

150

200

-20

-10

0

Case 5: Unrestricted trend

0

50

100

150

200

FIGURE 12.7. Simulated β 0 Yt from bivariate cointegrated VECM for five trendcases.

12.5 VAR Models and Cointegration

461

Simulated data from the five trend cases for a bivariate cointegrated
VAR(1) model are illustrated in Figures 12.6 and 12.7. Case I is not really
relevant for empirical work. The restricted contstant Case II is appropriate
for non-trending I(1) data like interest rates and exchange rates. The unrestriced constant Case III is appropriate for trending I(1) data like asset
prices, macroeconomic aggregates (real GDP, consumption, employmentetc). The restricted trend case IV is also appropriate for trending I(1) asin Case III. However, notice the deterministic trend in the cointegrating
residual in Case IV as opposed to the stationary residual in case III. Finally, the unrestricted trend Case V is appropriate for I(1) data with aquadratic trend. An example might be nominal price data during times ofextreme inflation.

12.5.4 Likelihood Ratio Tests for the Number of Cointegrating
Vectors
The unrestricted cointegrated VECM (12.23) is denoted H(r). The I(1)model H(r) can be formulated as the condition that the rank of Π is lessthan or equal to r. This creates a nested set of models
H(0) ⊂ · · · ⊂ H(r) ⊂ · · · ⊂ H(n)where H(0) represents the non-cointegrated VAR model with Π = 0 and
H(n) represents an unrestricted stationary VAR(p) model. This nested
formulation is convenient for developing a sequential procedure to test for
the number r of cointegrating relationships.
Since the rank of the long-run impact matrix Π gives the number of cointegrating relationships in Yt , Johansen formulates likelihood ratio (LR)statistics for the number of cointegrating relationships as LR statistics fordetermining the rank of Π. These tests are based on the estimated eigenvalues λ̂1 > λ̂2 > · · · > λ̂n of the matrix Π5 . These eigenvalues also happen toequal the squared canonical correlations between ∆Yt and Yt−1 corrected
for lagged ∆Yt and Dt and so lie between 0 and 1. Recall, the rank of Πis equal to the number of non-zero eigenvalues of Π.
Johansen’s Trace Statistic
Johansen’s LR statistic tests the nested hypotheses
H0 (r) : r = r0 vs. H1 (r0 ) : r > r0
The LR statistic, called the trace statistic, is given by
LRtrace (r0 ) = −TnX

i=r0 +1ln(1 − λ̂i )5 The calculation of the eigenvalues λ̂ (i = 1, . . . , n) is described in the appendix.
i462

12. Cointegration

If rank(Π) = r0 then λ̂r0 +1 , . . . , λ̂n should all be close to zero and LRtrace (r0 )should be small. In contrast, if rank(Π) > r0 then some of λ̂r0 +1 , . . . , λ̂nwill be nonzero (but less than 1) and LRtrace (r0 ) should be large. The
asymptotic null distribution of LRtrace (r0 ) is not chi-square but instead
is a multivariate version of the Dickey-Fuller unit root distribution which
depends on the dimension n − r0 and the specification of the deterministic terms. Critical values for this distribution are tabulated in OsterwaldLenum (1992) for the five trend cases discussed in the previous section for
n − r0 = 1, . . . , 10.
Sequential Procedure for Determining the Number of Cointegrating
Vectors
Johansen proposes a sequential testing procedure that consistently determines the number of cointegrating vectors. First test H0 (r0 = 0) against
H1 (r0 > 0). If this null is not rejected then it is concluded that there are nocointegrating vectors among the n variables in Yt . If H0 (r0 = 0) is rejected
then it is concluded that there is at least one cointegrating vector and proceed to test H0 (r0 = 1) against H1 (r0 > 1). If this null is not rejected thenit is concluded that there is only one cointegrating vector. If the null is rejected then it is concluded that there is at least two cointegrating vectors.
The sequential procedure is continued until the null is not rejected.
Johansen’s Maximum Eigenvalue Statistic
Johansen also derives a LR statistic for the hypotheses
H0 (r0 ) : r = r0 vs. H1 (r0 ) : r0 = r0 + 1
The LR statistic, called the maximum eigenvalue statistic, is given by
LRmax (r0 ) = −T ln(1 − λ̂r0 +1 )As with the trace statistic, the asymptotic null distribution of LRmax (r0 )is not chi-square but instead is a complicated function of Brownian motion, which depends on the dimension n − r0 and the specification of thedeterministic terms. Critical values for this distribution are tabulated in
Osterwald-Lenum (1992) for the five trend cases discussed in the previous
section for n − r0 = 1, . . . , 10.
Finite Sample Correction to LR Tests
Reinsel and Ahn (1992) and Reimars (1992) have suggested that the LR
tests perform better in finite samples if the factor T − np is used insteadof T in the construction of the LR tests.

12.5 VAR Models and Cointegration

463

12.5.5 Testing Hypothesis on Cointegrating Vectors Using the
S+FinMetrics Function coint
This section describes how to test for the number of cointegrating vectors,
and how to perform certain tests for linear restrictions on the long-run β
coeﬃcients.
Testing for the Number of Cointegrating Vectors
The Johansen LR tests for determining the number of cointegrating vectors in multivariate time series may be computed using the S+FinMetrics
function coint. The function coint has arguments> args(coint)function(Y, X = NULL, lags = 1, trend = "c", H = NULL,
b = NULL, save.VECM = T)
where Y is a matrix, data frame or “timeSeries” containing the I(1)variables in Yt , X is a numeric object representing exogenous variables tobe added to the VECM, lags denotes the number of lags in the VECM
(one less than the number of lags in the VAR representation), trend determines the trend case specification, and save.VECM determines if the VECM
information is to be saved. The arguments H and b will be explained later.
The result of coint is an object of class “coint” for which there are print
and summary methods. The use of coint is illustrated with the following
examples.
Example 81 Exchange rate data
Consider testing for the number of cointegrating relations among the logarithms of the monthly spot and forward exchange rates in the “timeSeries”
uscn.ts examined earlier. The first step is to determine the number of lags
to use in the VECM. Using the S+FinMetrics function VAR, the lag lengththat minimizes the AIC with a maximum lag of 6 is p = 2:
> uscn.ts = seriesMerge(uscn.s, uscn.f)> var.fit = VAR(uscn.ts,max.ar=6,criterion="AIC")> var.fit$ar.order[1] 2
The lag length for the VECM is then p−1 = 1. Since the monthly exchange
rate data are not trending, the Johansen LR tests are computed assuming
the restricted constant case II:> coint.rc = coint(uscn.ts,trend="rc",lags=1)> class(coint.rc)[1] "coint"> coint.rc

464

12. CointegrationCall:coint(Y = uscn.ts, lags = 1, trend = "rc")Trend Specification:
H1*(r): Restricted constantTrace tests significant at the 5% level are flagged by ’ +’.
Trace tests significant at the 1% level are flagged by ’++’.
Max Eigenvalue tests significant at the 5% level are flagged
by ’ *’.
Max Eigenvalue tests significant at the 1% level are flaggedby ’**’.
Tests for Cointegration Rank:
Eigenvalue Trace Stat 95% CV 99% CV Max Stat
H(0)++** 0.0970
32.4687
19.9600 24.6000 24.8012
H(1)0.0311
7.66759.2400 12.9700 7.6675
95% CV 99% CV
H(0)++** 15.6700 20.2000
H(1)9.2400 12.9700
Recall, the number of cointegrating vectors is equal to the number of nonzero eigenvalues of Π. The two estimated eigenvalues are 0.0970 and 0.0311.
The first row in the table gives LRtrace (0) and LRmax (0) for testing the nullof r0 = 0 cointegrating vectors as well as the 95% and 99% quantiles of theappropriate asymptotic distributions taken from the tables in OsterwaldLenum (1992). Both the trace and maximum eigenvalue statistics reject ther0 = 0 null at the 1% level. The second row in the table gives LRtrace (1)and LRmax (1) for testing the null of r0 = 1. Neither statistic rejects thenull that r0 = 1.
The summary method gives the same output as print as well as the unnormalized cointegrating vectors, adjustment coeﬃcients and the estimateof Π.
Testing Linear Restrictions on Cointegrating Vectors
The coint function can also be used to test linear restrictions on the cointegrating vectors β. Two types of restrictions are currently supported: thesame linear restrictions on all cointegrating vectors in β; some cointegrating
vectors in β are assumed known. Following Johansen (1995), two examples
are given illustrating how to use the coint function to test linear restrictions on β.
Example 82 Johansen’s Danish data

12.5 VAR Models and Cointegration

465

The "timeSeries" data set johansen.danish in S+FinMetrics contains
the monthly Danish data used in Johansen (1995), with the columns LRM,
LRY, LPY, IBO, IBE representing the log real money supply (mt ), log real
income (yt ), log prices, bond rate (ibt ) and deposit rate (idt ), respectively.
Johansen (1995) considered testing the cointegrating relationship among
mt , yt , ibt and idt . A natural hypothesis is that the velocity of money is afunction of the interest rates, or the cointegrating relation contains mt andyt only through the term mt − yt . For R0 = (1, 1, 0, 0), this hypothesis canbe represented as a linear restriction on β:
H0 : R0 β = 0 or β = HΨ(12.25)where Ψ are the unknown parameters in the cointegrating vectors β, H =R⊥ and R⊥ is the orthogonal complement of R such that R0 R⊥ = 0.
Johansen (1995) showed that the null hypothesis (12.25) against the alternative of unrestricted r cointegrating relations H(r) can be tested using alikelihood ratio statistic, which is asymptotically distributed as a χ2 withr(n − s) degree of freedom where s is the number of columns in H.
To test the hypothesis that the coeﬃcients of mt and yt add up to zeroin the cointegrating relations, Johansen (1995) considered a restricted constant model. In this case, R0 = (1, 1, 0, 0, 0) since the restricted constantalso enters the cointegrating space. Given the restriction matrix R, thematrix H can be computed using the perpMat function in S+FinMetrics:> R = c(1, 1, 0, 0, 0)> H = perpMat(R)> H[,1] [,2] [,3] [,4]
[1,]
-1
0
0
0
[2,]
1
0
0
0
[3,]
0
1
0
0
[4,]
0
0
1
0
[5,]
0
0
0
1
Now the test can be simply performed by passing the matrix H to thecoint function:> restr.mod1 = coint(johansen.danish[,c(1,2,4,5)],+ trend="rc", H=H)
The result of the test can be shown by calling the generic print method
on restr.mod1 with the optional argument restrictions=T:> print(restr.mod1, restrictions=T)
Call:coint(Y = johansen.danish[, c(1, 2, 4, 5)], trend = "rc",
H = H)

466

12. Cointegration

Trend Specification:
H1*(r): Restricted constant
Tests for Linear Restriction on Coint Vectors:
Null hypothesis: the restriction is true
Stat
Dist df P-value
H(1) 0.0346 chi-square 1 0.8523
H(2) 0.2607 chi-square 2 0.8778
H(3) 4.6000 chi-square 3 0.2035
H(4) 6.0500 chi-square 4 0.1954
For unrestricted sequential cointegration testing, the statistics in the output are labeled according to the null hypothesis, such as H(0), H(1), etc.
However, when restrictions are imposed, the statistics in the output printed
with restrictions=T are labeled according to the alternative hypothesis,
such as H(1), H(2), etc. In the above output, the null hypothesis cannot be rejected against the alternatives of H(1), H(2), H(3) and H(4) atconventional levels of significance.
After confirming that the cointegrating coeﬃcients on mt and yt add
up to zero, it is interesting to see if (1, −1, 0, 0, 0) actually is a cointegrating vector. In general, to test the null hypothesis that some cointegrating
vectors in β are equal to b:
H0 : β = (b, Ψ)
where b is the n × s matrix of known cointegrating vectors and Ψ is then×(r−s) matrix of unknown cointegrating vectors, Johansen (1995) showedthat a likelihood ratio statistic can be used, which is asymptotically distributed as a χ2 with s(n − r) degrees of freedom. This test can be simply
performed by setting the optional argument b to the known cointegrating
vectors. For example,> b = as.matrix(c(1,-1,0,0,0))> restr.mod2 = coint(johansen.danish[,c(1,2,4,5)],+ trend="rc", b=b)
> print(restr.mod2, restrictions=T)
Trend Specification:
H1*(r): Restricted constant
Tests for Known Coint Vectors:
Null hypothesis: the restriction is true
Stat
Dist df P-value
H(1) 29.7167 chi-square 4 0.0000
H(2) 8.3615 chi-square 3 0.0391

12.5 VAR Models and Cointegration

H(3)
H(4)

4.8759 chi-square
0.5805 chi-square

2
1

467

0.08730.4461

Again, the statistics in the above output are labeled according to the alternative hypothesis H(1), H(2), etc. Although the hypothesis that the
cointegrating coeﬃcients on mt and yt sum up to zero cannot be rejected,
mt −yt does not seem to be stationary because the hypothesis of the known
value of b is rejected at conventional levels of significance against H(1) andH(2).

12.5.6 Maximum Likelihood Estimation of the Cointegrated
VECM
If it is found that rank(Π) = r, 0 < r < n, then the cointegrated VECM
(12.23) becomes a reduced rank multivariate regression. The details of the
maximum likelihood estimation of (12.23) under the reduced rank restriction rank(Π) = r is briefly outlined in the Appendix to this chapter. There
it is shown thatβ̂mle = (v̂1 , . . . , v̂r ),(12.26)
where v̂i are the eigenvectors associated with the eigenvalues λ̂i , and that
the MLEs of the remaining parameters are obtained by multivariate leastsquares estimation of (12.23) with β replaced by β̂mle .
Normalized Estimates
Recall, the factorization Π = αβ0 is not unique and so the columns ofβ̂ mle in (12.26) may be interpreted as linear combinations of the underlying cointegrating relations. For interpretations, it is often convenient tonormalize or identify the cointegrating vectors by choosing a specific coordinate system in which to express the variables. One arbitrary way to do
this, suggested by Johansen, is to solve for the triangular representation of
the cointegrated system. The details of this normalization process is givenin the appendix, and the S+FinMetrics function VECM utilizes this normalization scheme by default. The resulting normalized cointegrating vector isdenoted β̂c,mle . The normalization of the MLE for β to β̂ c,mle will aﬀect
the MLE of α but not the MLEs of the other parameters in the VECM.
It must be emphasized that it is not possible to estimate the individual
elements of β without a specific normalization or identification schemeand that the normalization based on Phillips’ triangular representation isarbitrary and the resulting normalized cointegrating vectors (12.29) may
not have any economic meaning. Only in the case r = 1 can a uniquecointegrating vector be found after normalization.
Example 83 Unnormalzed MLEs for exchange rate data

468

12. Cointegration

The unnormalized cointegrating vector assuming r0 = 1 may also be
extracted directly from the “coint” object:> coint.rc$coint.vectors[1,]USCNS
USCNF Intercept*-739.0541 743.314
2.023532
Notice in the case of a restricted constant, the last coeﬃcient in β̂mle is an
estimate of the restricted constant. Normalizing on USCNS by dividing eachelement in β̂mle by −739.0541 gives> coint.rc$coint.vectors[1,]/+ as.numeric(-coint.rc$coint.vectors[1,1])USCNS
USCNF Intercept*-1 1.005764 0.002738003
The normalized MLEs, β̂c,mle = (−1, 1.006)0 and μ̂c = 0.0027 are almost
identical to the least squares estimates β̂ = (1, −1.004)0 and μ̂ = 0.0023
found earlier.
Asymptotic Distributions
Let β̂c,mle denote the MLE of the normalized cointegrating matrix β c .
Johansen (1995) showed that T (vec(β̂c,mle ) − vec(β c )) is asymptotically(mixed) normally distributed and that a consistent estimate of the asymptotic covariance of β̂c,mle is given bya[var(vec(β̂c,mle )) =

³
´−1
0
0 0
−1
α̂c,mleT −1 (In −β̂c,mle c0 )S−1
11 (In −β̂ c,mle c ) ⊗ α̂c,mle Ω̂(12.27)p

Notice that this result implies that β̂c,mle → βc at rate T instead of theusual rate T 1/2 . Hence, like the least squares estimator, β̂c,mle is super consistent. However, unlike the least squares estimator, asymptotically valid
standard errors may be compute using the square root of the diagonal
elements of (12.27).

12.5.7 Maximum Likelihood Estimation of the Cointegrated
VECM Using the S+FinMetrics Function VECM
Once the number of cointegrating vectors is determined from the coint
function, the maximum likelihood estimates of the full VECM may beobtained using the S+FinMetrics function VECM. The arguments expected
by VECM are> args(VECM)
function(object, coint.rank = 1, coint.vec = NULL, X = NULL,
unbiased = T, lags = 1, trend = "c", levels = F)12.5 VAR Models and Cointegration469

where object is either a “coint” object, usually produced by a call to thefunction coint, or a rectangular data object. If object is a “coint” object
then coint.rank specifies the rank of Π to determine the number of cointegrating vectors to be used in the fitted VECM. The cointegrating vectors
are then normalized using the Phillips’ triangular representation describedin the appendix. The lag length and trend specification for the VECM isobtained from the information in the “coint” object. The lag length in
the fitted VECM, however, is one less than the lag length specified in the“coint” object. If object is a rectangular data object, then coint.vec
must be assigned a matrix whose columns represent pre-specified cointegrating vectors. The argument lags is then used to specify the lag length of
the VECM, and trend is used to set the trend specification. The optional
argument X is used to specify any exogenous variables (e.g. dummy variables
for events) other than a constant or trend. The optional argument levels
determines if the VECM is to be fit to the levels Yt or to the first diﬀerences ∆Yt , and determines if forecasts are to be computed for the levels or
the first diﬀerences. The result of VECM is an object of class “VECM”, which
inherits from “VAR” for which there are print, summary, plot, cpredictand predict methods and extractor functions coef, fitted, residuals
and vcov. Since “VECM” objects inherit from “VAR” objects, most of themethod and extractor functions for “VECM” objects work similarly to thosefor “VAR” objects. The use of VECM is illustrated with the following examples.
Example 84 Maximum likelihood estimation of the VECM for exchange
rate data
Using the “coint” object coint.rc computed from the VAR(2) modelwith a restricted constant, the VECM(1) with a restricted constant for theexchange rate data is computed using> vecm.fit = VECM(coint.rc)> class(vecm.fit)[1] "VECM"> inherits(vecm.fit,"VAR")[1] T
The print method gives the basic output> vecm.fit
Call:
VECM(test = coint.rc)
Cointegrating Vectors:
coint.1
USCNS 1.0000

470

12. Cointegration

USCNF -1.0058
Intercept* -0.0027
VECM Coefficients:
USCNS
USCNF
coint.1 1.7771 1.9610
USCNS.lag1 1.1696 1.2627
USCNF.lag1 -1.2832 -1.3679
Std. Errors of Residuals:
USCNS USCNF
0.0135 0.0136
Information Criteria:
logL
AIC
BIC
HQ
2060.2 -4114.4 -4103.9 -4110.1
total residual
Degree of freedom:
243
240
Time period: from Apr 1976 to Jun 1996
The print method output is similar to that created by the VAR function.
The output labeled Cointegrating Vectors: gives the estimated cointegrating vector coeﬃcients normalized on the first variable in the specification of the VECM. To see standard errors for the estimated coeﬃcients use
the summary method> summary(vecm.fit)
Call:
VECM(test = coint.rc)
Cointegrating Vectors:
coint.1
1.0000
USCNF-1.0058(std.err)
0.0031(t.stat) -326.6389
Intercept*(std.err)(t.stat)-0.00270.0007-3.9758

VECM Coefficients:

12.5 VAR Models and Cointegration

coint.1(std.err)(t.stat)

USCNS
1.77710.64482.7561USCNF
1.96100.64643.0335USCNS.lag1(std.err)(t.stat)1.1696
0.9812
1.1921

1.26270.98361.2837471

USCNF.lag1 -1.2832 -1.3679(std.err) 0.9725 0.9749(t.stat) -1.3194 -1.4030
Regression Diagnostics:
USCNS USCNF
R-squared 0.0617 0.0689
Adj. R-squared 0.0538 0.0612
Resid. Scale 0.0135 0.0136
Information Criteria:
logL
AIC
BIC
HQ
2060.2 -4114.4 -4103.9 -4110.1
total residual
Degree of freedom:
243
240
Time period: from Apr 1976 to Jun 1996
The VECM fit may be inspected graphically using the generic plot
method> plot(vecm.fit)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Response and Fitted Values
3: plot: Residuals
4: plot: Normal QQplot of Residuals
5: plot: ACF of Residuals
6: plot: PACF of Residuals
7: plot: ACF of Squared Residuals
8: plot: PACF of Squared Residuals
9: plot: Cointegrating Residuals
10: plot: ACF of Cointegrating Residuals
11: plot: PACF of Cointegrating Residuals
12: plot: ACF of Squared Cointegrating Residuals

472

12. Cointegration

Cointegrating Residuals-0.006-0.002

0.002

coint.1

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

FIGURE 12.8. Cointegrating residual from maximum likelihood estimation of
VECM(1) for exchange rate data.

13: plot: PACF of Squared Cointegrating Residuals
Selection:
The first eight plot options are the same as those created for a "VAR" object.
The remaining plot options allow a graphical inspection of the cointegrating residual. For example, plot option 9 is illustrated in Figure 12.8. The
estimated cointegrating residual appears to be I(0).
Example 85 Estimate VECM with pre-specified cointegrating vector
For the exchange rate data, the MLE of the normalized cointegrating
vector is close to (1, −1)0 , and the estimate of the restricted constant isclose to zero. These are the values implied by the FRUH. To estimate a
VECM(1) imposing β = (1, −1)0 and μt = 0 use> beta.true = as.matrix(c(1,-1, 0))> dimnames(beta.true) = list(c("USCNS","USCNF","Intercept"),+"coint.1")> vecm.fruh.fit = VECM(uscn.ts, coint.vec = beta.true,+lags = 1, trend = "rc")
Since the restricted constant lies in the cointegrating space, the last element of the pre-specified cointegrating vector is the value of the restrictedconstant. A summary of the restricted VECM fit is:12.5 VAR Models and Cointegration

473> summary(vecm.fruh.fit)
Call:
VECM(data = uscn.ts, coint.vec = beta.true, lags = 1, trend ="rc")
Cointegrating Vectors:
coint.1
USCNS
1
USCNF
-1
Intercept
0
VECM Coefficients:
USCNS
coint.1 0.0337(std.err) 0.4442(t.stat) 0.0758

USCNF
0.13540.44660.3032USCNS.lag1(std.err)(t.stat)2.2781
0.95882.37602.13300.95352.2371USCNF.lag1 -2.2226 -2.3573(std.err) 0.9472 0.9525(t.stat) -2.3465 -2.4748
Regression Diagnostics:
USCNS USCNF
R-squared 0.0354 0.0393
Adj. R-squared 0.0273 0.0312
Resid. Scale 0.0137 0.0138
Information Criteria:
logL
AIC
BIC
HQ
2053.662 -4101.324 -4090.845 -4097.103
total residual
Degree of freedom:
243
240
Time period: from Apr 1976 to Jun 1996

Notice that the VECM with the pre-specified cointegrating vector does not
fit as well as the VECM using the estimated cointegrating vector.

474

12. Cointegration

12.5.8 Forecasting from the VECM
Forecasts from a VECM are computed by first transforming the VECM toa VAR using (12.22), and then using the forecasting algorithms for VAR
models described in the previous chapter. For VECM models, one may
forecast the changes in the variables, ∆Yt , or the levels of the variables Yt .
The generic S+FinMetrics functions predict and cpredict are used tocompute unconditional and conditional forecasts from a “VECM” object. The
following example illustrates the use of the predict method to computeforecasts for the diﬀerences and levels of the exchange rate data.
Example 86 Forecasts from VECM fit to exchange rate data
The “VECM” object vecm.fit was produced with the optional argument
levels=F. Consequently, the predict method will produce forecasts forthe changes in st and ft . To compute h-step forecasts for ∆st and ∆ft for
h = 1, . . . , 12 use> vecm.fcst = predict(vecm.fit,n.predict=12)> class(vecm.fcst)[1] "forecast"
To see the forecast and forecast standard errors use> summary(vecm.fcst)
Predicted Values with Standard Errors:
USCNS
USCNF
1-step-ahead -0.0105 -0.0110(std.err) 0.0136 0.0136
2-step-ahead -0.0130 -0.0139(std.err) 0.0183 0.0183...
12-step-ahead -0.0237 -0.0260(std.err) 0.0435 0.0432
By default, the forecasts are computed using the chain-rule of forecasting.
To compute simulation-based forecasts use method = "mc" or method ="bootstrap" in the call to predict.
To see the forecasts with standard error bands along the original datause
> plot(vecm.fcst, xold=diff(uscn.ts), n.old=12)
Since the forecasts are of the first diﬀerenced data, the data passed to xold
must be first diﬀerenced. The resulting plot is shown in Figure 12.9.
To compute forecasts for the levels st and ft , re-fit the VECM with theoptional argument levels=T12.5 Appendix: MLE of a Cointegrated VECM

235

250

255

-0.02

-0.02

0.0

0.0

0.02

0.02

245
USCNS

-0.06

-0.06

-0.04

-0.04

values

240

USCNF

235

240

245

250

255

index

FIGURE 12.9. VECM forecasts of first diﬀerences of exchange rate data.

> vecm.fit.level = VECM(coint.rc, levels=T)
and then call the predict method as before> vecm.fcst.level = predict(vecm.fit.level, n.predict=12)> summary(vecm.fcst.level)
Predicted Values with Standard Errors:
USCNS
USCNF
1-step-ahead -0.3150 -0.3154(std.err) 0.0136 0.0136
2-step-ahead -0.3157 -0.3161(std.err) 0.0183 0.0183...
12-step-ahead -0.3185 -0.3193(std.err) 0.0435 0.0432
To plot the forecasts use> plot(vecm.fcst.level, xold=uscn.ts, n.old=12)
The resulting plot is shown in Figure 12.10.

475

476

12. Cointegration

235

240

250

255

-0.28
-0.30

-0.30

-0.32

-0.32

-0.36

-0.36

-0.34

-0.34

values

245
USCNS

-0.28

USCNF

235

240

245

250

255

index

FIGURE 12.10. VECM forecasts of levels of exchange rate data.

12.6 Appendix: Maximum Likelihood Estimationof a Cointegrated VECM
The following brief discussion of maximum likelihood estimation of thecointegrated VECM (12.23) follows Hamilton (1994) and Johansen (1995).
For simplicity, assume the absence of deterministic terms.
• Concentrate the likelihood function with respect to the error covariance matrix and short-run dynamics by estimating the regressions∆Yt
Yt= Γ̂1 ∆Yt−1 + · · · + Γ̂p−1 ∆Yt−p+1 + Ût= Φ̂1 ∆Yt−1 + · · · + Φ̂p−1 ∆Yt−p+1 + V̂t• Form the sample covariance matrices
S00 =

T
T
T
1X
1X
1X
Ût Û0t , S01 =Ût V̂t0 , S11 =V̂t V̂t0
T t=1
T t=1
T t=1

• Solve the eigenvalue problem|λS11 − S10 S−1
00 S01 | = 0

12.6 Appendix: Maximum Likelihood Estimation of aCointegrated VECM

477

giving ordered eigenvalues6 λ̂1 > λ̂2 > · · · > λ̂n and associated eigenvectors v̂1 , v̂2 , . . . , v̂n that satisfy
λ̂i S11 v̂i= S10 S−1
00 S01 , i = 1, . . . , n

V̂S11 V̂

= In

where V̂ = [v̂1 , . . . , v̂n ]• The unnormalized MLE for the (n × r) matrix β based on 0 < r < ncointegrating vectors is given by the first r eigenvectors
β̂mle = (v̂1 , . . . , v̂r )• Form the normalized estimator β̂c,mle by imposing the appropriate
normalizing and identifying restrictions. The MLE for the normalized
estimator of α may be computed asα̂c,mle = S01 β̂c,mle• The maximum likelihood estimators for the remaining parameters
may be obtained by multivariate least squares of the VECM with βreplaced by β̂c,mle
0∆Yt = αc β̂c,mle Yt−1 + Γ1 ∆Yt−1 + · · · + Γp−1 ∆Yt−p+1 + εt
• The maximized value of the likelihood function based on r cointegrating vectors used in the construction of LR tests for the numberof cointegrating vectors is
L−2/Tmax ∝ |S00 |r
Y(1 − λ̂i )i=1• Estimates of the orthogonal complements of αc and βc are given by
α̂c,⊥

= S−1
00 S11 (v̂r+1 , . . . , v̂n )β̂ c,⊥= S11 (v̂r+1 , . . . , v̂n )Let c be any (n × r) matrix such that β0 c has full rank. Then β may benormalized as
βc = β(c0 β)−16 These eigenvalues are the squared canonical correlations between Y and ∆Y corttrected for ∆Yt−1 , . . . , ∆Yt−p+1 . Johansen (1995) describes how to solve for the eigenvalues.

478

12. Cointegration

satisfying c0 βc = Ir provided |c0 β| 6= 0. Johansen suggests setting.
c = (Ir ..0)0(12.28)This choice of c corresponds to solving the cointegrating relations β 0 Yt for00
the first r variables. To see this, let Yt = (Y1t , Y02t ) , where Y1t contains
the first r variables and Y2t contains the remaining n − r variables, and let.
β 0 = (−β ..β ), where β is (r × r) and β is (r × (n − r)). Then β 0 c = −β
1

2

1

2

and

βc =

µ

Ir
−β −1
1 β2

1

¶

(12.29)provided β1 has full rank r.
Some examples will help clarify the normalization scheme described above.
First, suppose there is only one cointegrating vector so that r = 1.Let the(n × 1) vector β = (−β 1 , β 2 , . . . , β n )0 and define c = (1, 0, . . . , 0)0 so thatβ 0 c = −β 1 and β c = (1, −β 2 /β 1 , . . . , −β n /β 1 )0 is the normalized cointegrating vector. Notice that this normalization requires β 1 6= 0. Next,
suppose there are two cointegrating vectors, r = 2, and let
µ
¶ µ
¶
.
−β 11 −β 12 β 13 . . . β 1nβ0 == −β1 ..β 2−β 21 −β 22 β 23 . . . β 2nµ¶.
1 0 0 ... 00c == (I2 ..0)0 1 0 ... 0such that β1 has full rank. Then β 0 c = −β1 andµ¶.
1 0 β ∗13 . . . β ∗1n0βc == (I2 ..β ∗ )0 1 β ∗23 . . . β ∗2n0where β∗ = −β−1
1 β 2 . The rows of β c are the normalized cointegrating vectors.

12.7 References
Alexander, C. (2001). Market Models: A Guide to Financial Data Analysis, John Wiley & Sons, Chichester, UK.
Cochrane, J. (2001). Asset Pricing. Princeton University Press, Princeton, NJ.
Engle, R.F. and C.W.J. Granger (1987). “Co-Integration and Error
Correction: Representation, Estimation and Testing,” Econometrica, 55,
251-276.

12.7 References

479

Granger, C.W.J. and P.E. Newbold (1974). “Spurious Regression in
Econometrics,” Journal of Econometrics, 2, 111-120.
Hamilton, J.D. (1994). Time Series Analysis, Princeton Unversity Press,
Princeton, NJ.
Hansen, B.E. (1992). “Eﬃcient Estimation and Testing of Cointegrating
Vectors in the Presence of Deterministic Trends,” Journal of Econometrics,
53, 87-121.
Hayashi, F. (2000). Econometrics, Princeton University Press, Princeton,
NJ.
Johansen, S. (1988). “Statistical Analysis of Cointegration Vectors,” Journal of Economic Dynamics and Control, 12, 231-254.
Johansen, S. (1995). Likelihood Based Inference in Cointegrated Vector
Error Correction Models, Oxford University Press, Oxford.
MacKinnon, J. (1996). “Numerical Distribution Functions for Unit Rootand Cointegration Tests,” Journal of Applied Econometrics, 11, 601-618.
Mills, T. (1999). The Econometric Analysis of Financial Time Series,
Cambridge University Press, Cambridge.
Osterwald-Lenum, M. (1992). “A Note with Quantiles of the Asymptotic Distribution of the Maximum Likelihood Cointegration Rank Statistics,” Oxford Bulletin of Economics and Statistics,54, 461-472.
Phillips, P.C.B. (1986). “Understanding Spurious Regression in Econometrics,” Journal of Econometrics, 33, 311-340.
Phillips, P.C.B. (1991). “Optimal Inference in Cointegrated Systems,”
Econometrica, 59, 283-306.
Phillips, P.C.B. and S. Ouliaris (1990). “Asymptotic Properties ofResidual Based Tests for Cointegration,” Econometrica, 58, 73-93.
Reimars, H.-E. (1992). “Comparisons of Tests for Multivariate Cointegration,” Statistical Papers, 33, 335-359.
Reinsel, G.C. and S.K. Ahn (1992). “Vector Autoregression Models
with Unit Roots and Reduced Rank Structure: Estimation, Likelihood Ratio Test, and Forecasting,” Journal of Time Series Analysis, 13, 353-375.

480

12. Cointegration

Sims, C.A., J.H. Stock and M.W. Watson (1990). “Inference in Linear
Time Series Models with Some Unit Roots,” Econometrica, 58, 113-144.
Stock, J.H. (1987). “Asymptotic Properties of Least Squares Estimationof Cointegrating Vectors,” Econometrica, 55, 1035-1056.
Stock, J.H. and M.W. Watson (1989). “Variable Trends in Economic
Time Series,” Journal of Economic Perspectives, Vol. 2(3), 147-174.
Stock, J.H. and M.W. Watson (1993). “A Simple Estimator of Cointegrating Vectors in Higher Order Integrated Systems,” Econometrica, 61,
783-820.
Tsay, R. (2001). The Analysis of Financial Time Series, John Wiley &Sons, New York.
Zivot, E. (2000). “Cointegration and Forward and Spot Exchange Rate
Regressions,” Journal of International Money and Finance, 19, 785-812.

This is page 481
Printer: Opaque this

13
Multivariate GARCH Modeling13.1 Introduction
When modeling multivariate economic and financial time series using vector
autoregressive (VAR) models, squared residuals often exhibit significant
serial correlation. For univariate time series, Chapter 7 indicates that thetime series may be conditionally heteroskedastic, and GARCH models havebeen proved to be very successful at modeling the serial correlation in thesecond order moment of the underlying time series.
This chapter extends the univariate GARCH models to the multivariate context and shows how multivariate GARCH models can be used tomodel conditional heteroskedasticity in multivariate time series. In particular, it will focus on modeling and predicting the time varying volatility
and volatility co-movement of multivariate time series. The multivariate
GARCH models in S+FinMetrics are so general that they actually include
the vector ARMA (VARMA) model as a special case.
To motivate multivariate GARCH models, Section 13.2 first introduces
an exponentially weighted covariance estimate and shows how to estimate the optimal weight using the mgarch function in S+FinMetrics. Section 13.3 modifies exponentially weighted covariance estimates to obtain
the popular diagonal VEC (DVEC) model. Section 13.4 illustrates how to
use the mgarch function to estimate a multivariate GARCH model suchas the DVEC model. Section 13.5 introduces some alternative formulations of multivariate GARCH models. Section 13.6 focuses on how to predict from multivariate GARCH models supported by S+FinMetrics. Sec-482

13. Multivariate GARCH ModelingMultivariate Series : hp.ibm^2
HP and IBM

0.0

0.0

0.2

0.2

0.4

ACF
0.4
0.6

0.8

0.6

1.0

HP

0

5

10

15

20

25

30

0

5

10

20

25

30

20

25

30

IBM

-30

0.0

0.0

0.2

0.2

0.4

ACF

0.4

0.6

0.8

0.6

1.0

IBM and HP

15

-25

-20

-15
Lag

-10

-5

0

0

5

10

15
Lag

FIGURE 13.1. ACF of multivariate hp.ibmˆ2.

tion 13.7 gives a detailed explanation of the structure of “garch.model”
and “mgarch.model” objects and shows how to use them to fine-tune orconstrain a GARCH model, univariate or multivariate. Finally, section 13.8
illustrates how to simulate from selected multivariate GARCH models.

13.2 Exponentially Weighted Covariance Estimate
S+FinMetrics module comes with two “timeSeries” objects, hp.s andibm.s, which represent daily stock returns of Hewlett-Packard and International Business Machine for the same time period. Chapter 7 showsthat these financial return series usually exhibit little serial correlation, butsquared returns are usually autocorrelated. In multivariate context, crosscorrelations of the levels as well as the volatility of the time series are alsoof interest. Cross-correlation in the levels can be modeled using vector autoregression (VAR) as shown in the previous chapter. This chapter focuseson cross-correlation, or co-movement, of the volatility.
Just as in the univariate context, the existence of cross-correlation can be
diagnosed using the S-PLUS function acf, which also takes a multivariate
time series as an argument, to produce both autocorrelation and crosscorrelation plots:> hp.ibm = seriesMerge(hp.s, ibm.s)

13.2 Exponentially Weighted Covariance Estimate483> tmp = acf(hp.ibm^2)
Use the S-PLUS function seriesMerge, which is specifically designed for“timeSeries” objects, to create a multivariate time series. The plot isshown in Figure 13.1. Both the autocorrelation and cross-correlation of thesecond order moments are significant at least up to lag 5, which indicatesthat the covariance matrix of hp.ibm may be time varying and seriallycorrelated.
Now let yt be a k × 1 vector of multivariate time series:
yt = c + ²t , for t = 1, 2, · · · , T(13.1)

where c is the k × 1 mean vector, and ²t is k × 1 vector of white noise withzero mean. The sample covariance matrix is given by:
Σ=

T
1 X
(yt − ȳ)(yt − ȳ)0
T − 1 t=1

where ȳ is the k × 1 vector of sample mean. In the above calculation, thesame weight 1/(T −1) is applied to the outer product of “demeaned” multivariate time series. To allow for time varying covariance matrix, in practicean ad hoc approach uses exponentially decreasing weights as follows:1
Σt = λ²t−1 ²0t−1 + λ2 ²t−2 ²0t−2 + · · ·∞X
λi ²t−i ²0t−i=i=1where 0 < λ < 1 so that smaller weights are placed on observations further
back into the past history. Since
λ + λ2 + · · · =λ1−λthe weights are usually scaled so that they sum up to one:Σt = (1 − λ)∞Xλi−1 ²t−i ²0t−i .

(13.2)

i=1

The above equation can be easily rewritten to obtain the following recursive
form for exponentially weighted covariance matrix:
Σt = (1 − λ)²t−1 ²0t−1 + λΣt−1(13.3)

which will be referred to as the EWMA model of time varying covariance.
From the above equation, given λ and an initial estimate Σ1 , the timevarying exponentially weighted covariance matrices can be computed easily.
1 This approach has recently been justified and exhaustively investigated by Foster
and Nelson (1996), and Andreou and Ghysels (2002). Fleming, Kirby and Ostdiek (2001)
applied this method for constructing portfolios.

13. Multivariate GARCH Modeling

0.0025484

0.0

0.00050.0015

IBM Vol.

HP Vol.

0.0

0.00050.00050.00150.00150.0025

Cov.

0

500

1000

1500

2000

FIGURE 13.2. Exponentially weighted covariance estimate.

The S+FinMetrics function EWMA.cov can be used to compute the exponentially weighted covariance matrix. For example, to obtain the timevarying covariance estimate of hp.ibm, use the following command:> hp.ibm.cov = EWMA.cov(hp.ibm, lambda=0.9672375)> seriesPlot(cbind(hp.ibm.cov[,1,1], hp.ibm.cov[,2,2],+ hp.ibm.cov[,1,2]), one.plot=F,+ strip.text=c("HP Vol.", "IBM Vol.", "Cov."))
The returned object hp.ibm.cov is an array of dimension 2000 × 2 × 2
representing the time varying covariance matrices, since there are 2000
observations in hp.ibm. Then use the S+FinMetrics function seriesPlot
to obtain a Trellis multivariate plot of the time varying covariance matrix
as shown in Figure 13.2, where the large spikes in the middle correspondto the 1987 stock market crash.
In practice, the value of λ is usually chosen in an ad hoc way as typified by
the RiskMetrics proposal. However, if one assumes that ²t in (13.1) follows
a multivariate normal distribution with zero mean, and Σt = Covt−1 (²t )is treated as the covariance of ²t conditional on the past history, then the
log-likelihood function of the observed time series can be written as:
T
T
1X
kT
1X
|Σt | −
(yt − c)0 Σ−1
log L = −log(2π) −t (yt − c).
2
2 t=1
2 t=1

(13.4)

13.2 Exponentially Weighted Covariance Estimate

485

Since Σt can be recursively calculated as in (13.3), the log-likelihood function can also be easily evaluated. Thus the mean vector c and λ can betreated as unknown model parameters and estimated using quasi-maximum
likelihood estimation (MLE), given the initial value Σ1 .
The mgarch function in S+FinMetrics actually allows the estimation of
the above EWMA model using either (13.3) or an exact form of (13.2) withlimited past history. The syntax of mgarch is very much similar to that ofgarch function. For example, to estimate the EWMA model as in (13.3),
use the following command:> hp.ibm.ewma = mgarch(hp.ibm~1, ~ewma1, trace=F)> hp.ibm.ewma
Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =trace = F)

~ ewma1,

Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:~ ewma1

Coefficients:
C(1) 0.0005202
C(2) 0.0004732
ALPHA 0.0327625
where the conditional variance formula is specified by ~ewma1. In the output, C(1) and C(2) correspond to the 2 × 1 vector of c in (13.1), and ALPHA
corresponds to 1 − λ in (13.3). This is why lambda=0.9672375 is set in theearlier EWMA.cov example.
The EWMA model with an exact form of (13.2) can also be estimatedby specifying ~ewma2 as the conditional variance formula. However, in thatcase, the coeﬃcient labeled by ALPHA actually corresponds to λ in (13.2):> mgarch(hp.ibm~1, ~ewma2, trace=F)Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =trace = F)
Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:
Coefficients:~ ewma2~ ewma2,

486

13. Multivariate GARCH ModelingC(1) 0.0007369
C(2) 0.0002603
ALPHA 0.9730018

13.3 Diagonal VEC ModelIn the univariate context, the EWMA model introduced in the previous
section reduces to:Σt = (1 − λ) 2t−1 + λΣt−1which is simply a GARCH(1, 1) model with a1 = 1 − λ, b1 = λ and thus
a1 + b1 = 1. Since a1 + b1 corresponds to the AR(1) coeﬃcient in the
ARMA representation of GARCH models (see Section 22.16 in Chapter 7),
the condition a1 + b1 = 1 implies that the GARCH model is not stationary
in the weak sense.2 Engle and Bollerslev (1986) termed this model the
integrated GARCH (IGARCH) model in the univariate context.3 Given
the non-stationarity of IGARCH and EWMA models, they are sometimes
not favored for modeling volatility.
To preserve the intuition behind EWMA models while allowing for a
flexible and stationary model for time varying covariance, generalize the
EWMA model as follows:Σt = A0 +p
X
i=1

Ai ⊗ (²t−i ²0t−i ) +

q
X
j=1

Bj ⊗ Σt−j(13.5)

where the symbol ⊗ stands for Hadamard product, i.e., element-by-element
multiplication, and all the coeﬃcient matrices have dimension k × k. Thismodel is first proposed by Bollerslev, Engle and Wooldridge (1988), andthey called it the diagonal VEC, or DVEC(p, q) model.
2 Unlike the unit root time series, a GARCH model may be strongly stationary, evenwhen it is not weakly stationary. See Nelson (1990) and Bougerol and Picard (1992) for
technical proof.
3 In fact, the mgarch function can be called with a univariate time series using ~ewma1
as the conditional variance formula to estimate such an IGARCH model.

13.4 Multivariate GARCH Modeling in S+FinMetrics

487

To appreciate the intuition behind DVEC model, consider the bivariate
DVEC(1, 1) model:
"
# "
#
(11)
(11)
Σt
A0
=
(21)
(22)
(21)
(22)
Σt
Σt
A0
A0
"
#"
#
(11)
(1) (1)A1
εt−1 εt−1
+
(21)
(22)
(2) (1)(2) (2)A1
A1
εt−1 εt−1 εt−1 εt−1
"
#"
#
(11)
(11)
B1
Σt−1
+
(21)
(22)
(21)
(22)
B1
B1
Σt−1 Σt−1
where only the lower triangular part of the system is considered, withX(ij) denoting the (i, j)-th element of matrix X, and ²(i) the i-th elementof vector ². The above matrix notation can be rewritten as follows:4(11)(11) (1) (1)(11) (11)²t−1 ²t−1 + B1 Σt−1
(21)
(21)
(21) (2) (1)(21) (21)Σt = A0 + A1 ²t−1 ²t−1 + B1 Σt−1
(22)
(22)
(22) (2) (2)(22) (22)Σt = A0 + A1 ²t−1 ²t−1 + B1 Σt−1

Σt

(11)

= A0

+ A1

so the (i, j)-th element of the time varying covariance matrix only depends
on its own lagged element and the corresponding cross-product of errors.
As a result, the volatility of each series follows a GARCH process, while
the covariance process can also be treated as a GARCH model in terms ofthe cross-moment of the errors.
Since a covariance matrix must be symmetric, in practice it suﬃces totreat Σt as symmetric and only consider the lower triangular part of thesystem. A covariance matrix must be also positive semi-definite (PSD).
However, Σt in the DVEC model cannot be guaranteed to be PSD, whichis considered a weakness of the DVEC model. Section 13.5 will introduce
other formulations of multivariate GARCH models that guarantee the timevarying covariance matrix to be PSD.

13.4 Multivariate GARCH Modeling in
S+FinMetrics
13.4.1 Multivariate GARCH Model Estimation
Section 13.2 showed that the mgarch function in S+FinMetrics can be used
to estimate a multivariate GARCH model such as the EWMA model. It
4 If these equations are written using matrix notation with a vector on the left handside, then the coeﬃcient matrices become diagonal matrices; thus this model is referred
to as the diagonal VEC model.

488

13. Multivariate GARCH Modelingcan also be used to fit other types of multivariate GARCH models suchas the DVEC model by using a diﬀerent conditional variance formula. For
example, to fit a DVEC(1, 1) model to the bivariate time series hp.ibm,
use the following command:> hp.ibm.dvec = mgarch(hp.ibm~1, ~dvec(1,1), trace=F)> class(hp.ibm.dvec)[1] "mgarch"> hp.ibm.dvec
Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =trace = F)

~ dvec(1, 1),

Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:~ dvec(1, 1)

Coefficients:
C(1) 7.018e-04
C(2) 2.932e-04
A(1, 1) 3.889e-05
A(2, 1) 1.322e-05
A(2, 2) 2.877e-05
ARCH(1; 1, 1) 6.226e-02
ARCH(1; 2, 1) 3.394e-02
ARCH(1; 2, 2) 1.049e-01
GARCH(1; 1, 1) 8.568e-01
GARCH(1; 2, 1) 8.783e-01
GARCH(1; 2, 2) 7.421e-01
The returned object is of class “mgarch”. Similar to “garch” objects, theprint method shows the conditional mean equation, conditional varianceequation, together with the estimated model coeﬃcients. In the output,
C(i) corresponds to the i-th element of c in (13.1), while A(i,j) corresponds to the (i, j)-th element of A0 , ARCH(i;j,k) corresponds to the(j, k)-th element of Ai , and GARCH(j;i,k) corresponds to the (i, k)-th element of Bj in (13.5).
As usual, use the S-PLUS function names to find out the component
names of an “mgarch” object:> names(hp.ibm.dvec)[1] "residuals""sigma.t""df.residual"[5] "model""cond.dist""likelihood"[9] "cov""std.residuals" "R.t""coef""opt.index""S.t"13.4 Multivariate GARCH Modeling in S+FinMetrics[13] "prediction""call"

489"series"

These components are similar to those of “garch” objects, and the on-line
help file for mgarch provides details for them. For most components thata user is interested in, S+FinMetrics provides methods for generic functions such as coef, residuals, and vcov for extracting those components.
For example, extract the estimated coeﬃcients by calling the generic coef
function:> coef(hp.ibm.dvec)
C(1) 7.017567e-04
C(2) 2.932253e-04
A(1, 1) 3.888696e-05
A(2, 1) 1.322108e-05
A(2, 2) 2.876733e-05
ARCH(1; 1, 1) 6.225657e-02
ARCH(1; 2, 1) 3.393546e-02
ARCH(1; 2, 2) 1.048581e-01
GARCH(1; 1, 1) 8.567934e-01
GARCH(1; 2, 1) 8.783100e-01
GARCH(1; 2, 2) 7.421328e-01
Note that since only the lower triangular part of the system is consideredfor DVEC models, only that part of the coeﬃcient matrices are shown here.
Similarly, call the generic vcov function to obtain the covariance matrixof the estimated coeﬃcients. By default, the covariance matrix based on
the outer product of gradients is returned. Just like in the univariate case,the covariance matrix based on the inverse of numerical Hessian and therobust covariance matrix can be obtained by setting the optional argument
method to "op" and "qmle", respectively. For example, to obtain the robust
standard error of the estimated coeﬃcients, use the command:> sqrt(diag(vcov(hp.ibm.dvec, method="qmle")))[1] 0.00048803101 0.00030789132 0.00003531643 0.00001088806[5] 0.00001685943 0.03070917257 0.02983075055 0.06630322823[9] 0.10170075535 0.09285527451 0.13273539264
Similar to the method functions for “garch” objects, residuals andsigma.t can be used to extract the model residuals and estimated volatility, respectively. If the original multivariate data is a “timeSeries” object, the extracted model residuals and conditional volatility will also be“timeSeries” objects with the same dimension. Note that in the multivari−1/2
1/2
ate case, the standardized residuals are computed as Σt²t , where Σt
is the Cholesky factor of Σt . To obtain the standardized residuals, set theoptional argument standardize=T when calling the residuals function:> residuals(hp.ibm.dvec, standardize=T)490

13. Multivariate GARCH Modeling

The sigma.t function only extracts the conditional standard deviationof each series, and ignores the conditional covariance term. To obtain theconditional covariance or conditional correlation term, extract the S.t and
R.t component, respectively. Both S.t and R.t are three dimensional arrays with dimension T × k × k.

13.4.2 Multivariate GARCH Model Diagnostics
The previous subsection showed how to estimate a multivariate GARCH
model in S+FinMetrics, and how to extract various components of thefitted model. To assess the model fit, S+FinMetrics provides method functions for two generic functions: summary and plot, one for statistical summary and the other for visual diagnostics of the model fit.
For example, to obtain more detailed summary of hp.ibm.dvec, call thegeneric summary function:> summary(hp.ibm.dvec)
Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =trace = F)

~ dvec(1, 1),

Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:
Conditional Distribution:~ dvec(1, 1)

gaussian-----------------------------------------------------------Estimated Coefficients:-----------------------------------------------------------Value Std.Error t value Pr(>|t|)C(1) 7.018e-04 4.630e-04
1.516 6.489e-02
C(2) 2.932e-04 2.870e-04
1.022 1.536e-01
A(1, 1) 3.889e-05 6.175e-06
6.297 1.860e-10
A(2, 1) 1.322e-05 2.461e-06
5.372 4.345e-08
A(2, 2) 2.877e-05 4.302e-06
6.687 1.469e-11
ARCH(1; 1, 1) 6.226e-02 8.690e-03
7.164 5.498e-13
ARCH(1; 2, 1) 3.394e-02 6.848e-03
4.955 3.916e-07
ARCH(1; 2, 2) 1.049e-01 9.212e-03 11.382 0.000e+00
GARCH(1; 1, 1) 8.568e-01 1.762e-02 48.625 0.000e+00
GARCH(1; 2, 1) 8.783e-01 1.885e-02 46.589 0.000e+00
GARCH(1; 2, 2) 7.421e-01 2.966e-02 25.019 0.000e+00

13.4 Multivariate GARCH Modeling in S+FinMetrics

491-----------------------------------------------------------AIC(11) = -21886.25
BIC(11) = -21824.64
Normality Test:-----------------------------------------------------------Jarque-Bera P-value Shapiro-Wilk P-value
HP
755.8
0
0.9891 0.7105
IBM
2606.300.9697 0.0000
Ljung-Box test for standardized residuals:-----------------------------------------------------------Statistic P-value Chi^2-d.f.
HP
18.57 0.09952
12IBM
11.76 0.46511
12
Ljung-Box test for squared standardized residuals:-----------------------------------------------------------Statistic P-value Chi^2-d.f.
HP
11.43 0.4925
12
IBM
4.44 0.9741
12
Lagrange multiplier test:-----------------------------------------------------------Lag 1
Lag 2
Lag 3 Lag 4 Lag 5
Lag 6
Lag 7
HP -0.1990 0.2496 -0.7004 2.594 0.1039 -0.1167 -0.2286
IBM -0.7769 -0.9883 -0.5770 -1.198 0.4664 -0.2077 -0.4439
Lag 8
Lag 9 Lag 10 Lag 11
Lag 12C
HP 0.09018 -0.7877 -0.1279 -0.9280 -0.03133 1.8549
IBM -0.26423 -0.5352 -0.6724 0.1852 0.02102 -0.0729
TR^2 P-value F-stat P-value
HP 11.914 0.4526 1.090 0.4779
IBM 4.522 0.9721 0.412 0.9947
By default, the summary method shows the standard errors and p-valuesof estimated coeﬃcients, together with various tests on the standardized
residuals for assessing the model fit. The standard errors and p-values arecomputed using the default covariance estimate. To use robust or numerical Hessian based standard errors to compute the p-values, the summary
method takes an optional argument method just like the vcov method does.
All the tests performed on the standardized residuals can also be performed independently by using standard S+FinMetrics functions. In gen-492

13. Multivariate GARCH Modeling

eral, if the model is successful at modeling the serial correlation in the timeseries and the time varying aspect of covariance matrix, there should be noserial correlation left in both the first order and second order moments ofstandardized residuals. For example, to check that there is no serial correlation left in squared standardized residuals, use the following command:> autocorTest(residuals(hp.ibm.dvec, standardize=T)^2, lag=12)
Test for Autocorrelation: Ljung-Box
Null Hypothesis: no autocorrelation
Test Statistics:
HP
Test Stat 11.4299
p.value 0.4925

IBM
4.4404
0.9741Dist. under Null: chi-square with 12 degrees of freedom
Total Observ.: 2000
which is the same as the test results returned by the summary method.
Since the p-values for both series are much greater than the conventional
5% level, the null hypothesis that there is no autocorrelation left cannot
be rejected.
Similarly, the LM test for ARCH eﬀects can be performed on the multivariate standardized residuals:> archTest(residuals(hp.ibm.dvec, standardize=T), lag=12)
Test for ARCH Effects: LM Test
Null Hypothesis: no ARCH effects
Test Statistics:
HP
Test Stat 11.9136
p.value 0.4526

IBM
4.52190.9721Dist. under Null: chi-square with 12 degrees of freedom
Total Observ.: 2000
which is also the same as the LM test returned by the summary method.
The p-values for LM tests are very close to those of the autocorrelation
tests, which confirms that the DVEC model is very successful at modelingthe time varying aspect of covariance matrix.
Note the above tests are applied to each series separately, and they do
not check the serial correlation of the cross-moment. Hence those tests are13.4 Multivariate GARCH Modeling in S+FinMetrics493

not really multivariate tests. However, the autocorTest function does havean option to produce a multivariate portmanteau test as proposed by Hosking (1980), which is a multivariate extension of the univariate Ljung-Box
test. For example, to produce the multivariate test of squared standardized
residuals, use the command:> autocorTest(residuals(hp.ibm.dvec, standardize=T)^2,+ lag=12, bycol=F)
Multivariate Portmanteau Test: Ljung-Box Type
Null Hypothesis: no serial correlation
Test Statistics:
Test Stat 42.4585
p.value 0.6985
Dist. under Null: chi-square with 48 degrees of freedom
Total Observ.: 2000
where the optional argument bycol is set to FALSE to use the Hosking’s
test. The autocorTest function sets bycol to TRUE by default, and thus
tests the multivariate series column by column.
The goodness-of-fit of a multivariate GARCH model can also be assessedby calling the generic plot function on a fitted “mgarch” object. For example:> plot(hp.ibm.dvec)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Original Observations
3: plot: ACF of Observations
4: plot: ACF of Squared Observations
5: plot: Residuals
6: plot: Conditional SD
7: plot: Standardized Residuals
8: plot: ACF of Standardized Residuals
9: plot: ACF of Squared Standardized Residuals
10: plot: QQ-Plots of Standardized Residuals
Selection:
By selecting 9 the ACF of squared standardized residuals can be obtained, which is shown in Figure 13.3. After fitting the DVEC model, there
is essentially little serial correlation left in the second order moments of494

13. Multivariate GARCH Modeling

ACF of Squared Std. Residuals
HP and IBM

0.0

0.0

0.2

0.1

0.2

ACF
0.4
0.6

0.3

0.8

0.4

1.0

HP

0

10

20

30

0

10

30

20

30

IBM

0.0

0.0

0.2

0.1

0.4

ACF
0.2

0.6

0.3

0.8

0.4

1.0

IBM and HP

20

-30

-20

Lag

-10

0

0

10

Lag

FIGURE 13.3. ACF of squared standardized residuals.

the residuals. Normal QQ-plot of standardized residuals can be obtainedby selecting 10, which is shown in Figure 13.4. There is significant deviation in the tails from the normal QQ-line for both residuals, which isalso confirmed by the normality tests in the summary output shown earlier.
Thus it seems that the normality assumption for the residuals may not beappropriate. Section 13.5.5 will show how to use alternative distributions
in multivariate GARCH models.
Other plots can also be chosen to visualize the model fit. For example,
choosing 6 plots the estimated conditional standard deviation as shown in
Figure 13.5. For the bivariate time series hp.ibm, the time varying crosscorrelation, which is contained in the R.t component of the fitted object,is also of interest. Since R.t is a three-dimensional array, use the following
command to generate a time series of the conditional cross-correlation:> hp.ibm.cross = hp.ibm.dvec$R.t[,1,2]> hp.ibm.cross = timeSeries(hp.ibm.cross,pos=positions(hp.ibm))> seriesPlot(hp.ibm.cross, strip="Conditional Cross Corr.")
The plot is shown in Figure 13.6. Although the conditional cross correlation between hp.s and ibm.s usually fluctuates around 0.5, it can suddenly
drop down to 0.3 and then go back to 0.5 very quickly.

13.4 Multivariate GARCH Modeling in S+FinMetrics

QQ-Plot of Standardized Residuals
-2

0

HP

2

IBM

6

2/19/1991
9/28/1990

Standardized Residuals

4
2
0
-2
-4

9/27/1989-6

10/19/1987-8

3/19/1991
10/19/1987
-2

0

2

Quantiles of gaussian distribution

FIGURE 13.4. QQ-plot of standardized residuals.

MGARCH Volatility

0.02
0.03

0.04

0.05

HP

0.02

Conditional SD

0.04

0.06

IBM

Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1

1984

1985

1986

1987

1988

1989

1990

1991

1992

FIGURE 13.5. Multivariate conditional volatility: hp.ibm.

495

496

13. Multivariate GARCH Modeling

0.3

0.4

0.5

0.6

0.7

0.8

Conditional Cross Corr.

Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4 Q2 Q4
1984

1985

1986

1987

1988

1989

1990

1991 1992

FIGURE 13.6. Conditional cross correlation: hp.ibm.

13.5 Multivariate GARCH Model Extensions
13.5.1 Matrix-Diagonal Models
Although the DVEC model provided a good model fit for the bivariate timeseries hp.ibm, the time varying covariance matrices are not guaranteed to
be PSD given the formulation as in (13.5). Note that a suﬃcient conditionfor Σt to be PSD is that A0 , Ai (for i = 1, · · · , p) and Bj (for j = 1, · · · , q)
are all PSD. Based on this observation, Ding (1994) and Bollerslev, Engle
and Nelson (1994) proposed to estimate the Cholesky factors of the coeﬃcient matrices:Σt = A0 A00 +

p
q
X
X
0
0
(Ai Ai ) ⊗ (²t−i ²t−i ) +(Bj B0j ) ⊗ Σt−ji=1

(13.6)

j=1

where A0 , Ai (for i = 1, · · · , p) and Bj (for j = 1, · · · , q) are all lower
triangular matrices. This model will be referred to as the matrix-diagonalmodel.
The matrix-diagonal models can be further simplified by restricting Ai
and Bj to be a vector, which results in:Σt = A0 A00 +pq
XX(ai a0i ) ⊗ (²t−i ²0t−i ) +(bj b0j ) ⊗ Σt−j
i=1

j=1

(13.7)

13.5 Multivariate GARCH Model Extensions

497

where ai and bj are k × 1 vectors. Even simpler, use the following formulation:pqXXΣt = A0 A00 +ai ⊗ (²t−i ²0t−i ) +bj ⊗ Σt−j(13.8)
i=1

j=1

where ai and bj are positive scalars. It is easy to show that all the formulations given in (13.6), (13.7), and (13.8) guarantee that the time varying
covariance matrix Σt is PSD. However, the simpler the model is, the more
stringent restrictions are placed on the dynamics of the model.
The mgarch function in S+FinMetrics allows the estimation of all theabove modifications of the DVEC model by using ~dvec.type.type(p,q)
as the conditional variance formula, where type can be mat for the (13.6)
formulation, vec for the (13.7) formulation, or scalar for the (13.8) formulation, and the first type refers to the type of Ai , the second type refers
to the type of Bj . Hence, one can use mgarch to estimate a multivariate
GARCH model with diﬀerent formulations for Ai and Bj . For example,
to estimate a multivariate GARCH model with the following covariancematrix formulation:Σt = A0 A00 + A1 A01 ⊗ (²t−i ²0t−i ) + b1 ⊗ Σt−j(13.9)

with A0 and A1 being lower triangular matrices and b1 just a scalar, use
the following conditional variance formula:> mgarch(hp.ibm~1, ~dvec.mat.scalar(1,1), trace=F)Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =~ dvec.mat.scalar(1, 1), trace = F)
Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:
Coefficients:
C(1) 0.0007500
C(2) 0.0003268
A(1, 1) 0.0099384
A(2, 1) 0.0037295
A(2, 2) 0.0044583
ARCH(1; 1, 1) 0.3215890
ARCH(1; 2, 1) 0.1984259
ARCH(1; 2, 2) 0.2958904
GARCH(1) 0.6968114~ dvec.mat.scalar(1, 1)

498

13. Multivariate GARCH Modeling

Note that in the output the GARCH(1) coeﬃcient corresponds to b1 , while
ARCH(1;i,j) corresponds to the (i, j)-th element of A1 in (13.9).

13.5.2 BEKK Models
Although the DVEC model can be modified in various ways to ensure thetime varying covariance matrices are PSD, the dynamics allowed in the conditional covariance matrix are still somewhat restricted. In particular, theconditional variance and covariance are only dependent on their own lagged
element and the corresponding cross-product of shocks or error terms. For
example, consider the bivariate time series hp.ibm. If there is a shock to
hp.s in the current period, it will aﬀect the conditional volatility of hp.s
and the conditional correlation between hp.s and ibm.s in the next period.
However, it will not directly aﬀect the volatility of ibm.s.
The BEKK model, as formalized by Engle and Kroner (1995), provides
an alternative formulation of the conditional variance equation:Σt = A0 A00 +

p
X
i=1

Ai (²t−i ²0t−i )A0i +q
X

Bj Σt−j B0j

j=1

where A0 is a lower triangular matrix, but Ai (i = 1, · · · , p) and Bj(j = 1, · · · , q) are unrestricted square matrices. It is easy to show that
Σt is guaranteed to be symmetric and PSD in the above formulation. Furthermore, the dynamics allowed in the BEKK model are richer than the
DVEC model, which can be illustrated by considering the (2, 2) element of
Σt in the BEKK(1, 1) model:
(22)

Σt

(22)

= A0

(1)

(22)

(21) (1)(22) (2)²t−1 + A1 ²t−1 ]2 +(21) (21) (11)(21) (22) (21)(22) (22) (22)[B1 B1 Σt−1 + 2B1 B1 Σt−1 + B1 B1 Σt−1 ]

A0

+ [A1

(2)

(11)

where both ²t−1 and ²t−1 enter the equation. In addition, Σt−1 , the volatil(22)ity of the first series, also has direct impacts on Σt , the volatility of thesecond series. However, for the bivariate BEKK(1, 1) model, flexibility is
(12)
(12)
achieved at the cost of two extra parameters, i.e., A1 and B1 , which
are not needed for the DVEC(1, 1) model. In general, a BEKK(p, q) model
requires k(k − 1)(p + q)/2 more parameters than a DVEC model of thesame order.
One can fit a BEKK model by using ~bekk(p,q) as the conditionalvariance formula. For example, to fit a BEKK(1, 1) model to the bivariate
time series hp.ibm, use the following command:> hp.ibm.bekk = mgarch(hp.ibm~1, ~bekk(1,1))> hp.ibm.bekk

13.5 Multivariate GARCH Model Extensions

Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =499~ bekk(1, 1))

Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:~ bekk(1, 1)

Coefficients:
C(1) 0.0007782
C(2) 0.0002870
A(1, 1) 0.0077678
A(2, 1) -0.0035790
A(2, 2) 0.0046844
ARCH(1; 1, 1) 0.2054901
ARCH(1; 2, 1) -0.0287318
ARCH(1; 1, 2) -0.0734735
ARCH(1; 2, 2) 0.4169672
GARCH(1; 1, 1) 0.8078184
GARCH(1; 2, 1) 0.1277266
GARCH(1; 1, 2) 0.2867068
GARCH(1; 2, 2) 0.6954790
Note that in the output, the coeﬃcient matrix A1 (the ARCH(1;i,j) coefficients) and B1 (the GARCH(1;i,j)) are not restricted.
Compare the conditional correlations between hp.s and ibm.s impliedby the DVEC model and BEKK model as follows:> seriesPlot(cbind(hp.ibm.dvec$R.t[,1,2],+ hp.ibm.bekk$R.t[,1,2]),strip=c("DVEC Corr.","BEKK Corr."),+ one.plot=F, layout=c(1,2,1))
The plot is shown in Figure 13.7, from which one can see that the conditional correlation implied by the BEKK model is more volatile than that
implied by the DVEC model.

13.5.3 Univariate GARCH-based Models
For BEKK model, DVEC model and its modifications, the conditional covariance matrix is modeled directly. This approach can result in a large
number of parameters since the covariance terms need to be modeled separately. Another approach in multivariate GARCH modeling is to transformthe multivariate time series into uncorrelated time series and then apply theunivariate GARCH models in Chapter 7 to each of those uncorrelated series. This subsection introduces three types of multivariate GARCH models
in this fashion.

500

13. Multivariate GARCH Modeling

0.3

0.4

0.5

0.6

0.7

0.8

DVEC Corr.

0.0

0.2

0.4

0.6

0.8

BEKK Corr.

0

500

1000

1500

2000

FIGURE 13.7. Comparison of conditional correlation: hp.ibm.

Constant Conditional Correlation Model
In general, a k×k covariance matrix Σ can be decomposed into the followingform:Σ = ∆R∆
where R is the correlation matrix, ∆ is a diagonal matrix with the vector (σ 1 , · · · , σ k ) on the diagonal, and σ i is the standard deviation of thei-th series. Based on the observation that the correlation matrix of foreign exchange rate returns is usually constant over time, Bollerslev (1990)
suggested modelling the time varying covariance matrix as follows:Σt = ∆t R∆twhere R is the constant conditional correlation matrix, and ∆t is the following diagonal matrix:
⎤
⎡
σ 1t
⎥
⎢
..
∆t = ⎣⎦.
σ kt

with σ it following any univariate GARCH process, for i = 1, · · · , k. Thismodel is usually referred to as the constant conditional correlation (CCC)
model.

13.5 Multivariate GARCH Model Extensions

501

The mgarch function can be used to estimate a CCC model with a
GARCH(p, q) model for each series, by specifying ~ccc(p,q) as the conditional variance formula. In addition, a more general formula such as~ccc.type(p,q) can also be used, where type can be any of the GARCH
variants supported by the garch function.5 For example, to use a two components model for each series when fitting a CCC model to the bivariate
time series hp.ibm, use the following conditional variance formula:> mgarch(hp.ibm~1, ~ccc.two.comp(1,1), trace=F)Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =~ ccc.two.comp(1, 1), trace = F)
Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:~ ccc.two.comp(1, 1)

Coefficients:
C(1) 4.907e-04
C(2) 1.844e-04
A(1, 1) 8.722e-05
A(2, 2) 6.579e-05
ARCH(1; 1, 1) 8.102e-03
ARCH(1; 2, 2) 9.621e-03
ARCH(2; 1, 1) 9.669e-02
ARCH(2; 2, 2) 9.582e-02
GARCH(1; 1, 1) 9.699e-01
GARCH(1; 2, 2) 9.654e-01
GARCH(2; 1, 1) 7.365e-01
GARCH(2; 2, 2) 7.271e-01
Conditional Constant Correlation Matrix:
HPIBM
HP 1.0000 0.5582
IBM 0.5582 1.0000
When fitting a CCC model, mgarch function allows several alternatives
for the estimation of the constant conditional correlation matrix R by setting the optional argument cccor.choice:
1. cccor.choice=0: The sample correlation matrix is used, and no further MLE estimation of R is carried out.
5 See Section 7.9 in Chapter 7 for a summary of those specifications.

502

13. Multivariate GARCH Modeling

2. cccor.choice=1: The sample correlation matrix is used as the initial
estimate, and the final estimate of R is obtained as part of the MLE
method. This is the default value.
3. cccor.choice=2: The user supplies an initial correlation matrix estimate, and the final estimate of R is obtained as part of the MLE
method. In this case, the user needs to supply the initial estimate
with the optional argument cccor.value.
A potentially important use of the last choice is to obtain robustness
toward multivariate outliers by using a robust initial covariance matrix
estimate. The covRob function in S-PLUS robust library provides several
robust covariance and correlation estimates.
Principal Component Model
In principal component analysis, it is well known that for any covariancematrix Σ, one can always find an orthogonal matrix Λ and a diagonal
matrix ∆ such thatΛ∆Λ0 = Σwhere Λ is usually normalized so that ΛΛ0 = I with I being an identity
matrix. It can be shown that the diagonal elements of ∆ are the eigenvalues of Σ, while the columns of Λ correspond to the eigenvectors of Σ.
Based on this result, the principal components of yt , which are defined as
zt = Λ0 yt , have a diagonal covariance matrix. Ding (1994) describes theprincipal component GARCH model, which essentially models each principal component in zt as a univariate GARCH model. This model is also
proposed by Alexander (1998).
The mgarch function can be used to estimate a principal component
model with a GARCH(p, q) model for principal component, by specifying~prcomp(p,q) as the conditional variance formula. Similar to the CCC
model, a more general formula such as ~prcomp.type(p,q) can also beused, where type can be any of the GARCH variants supported by thegarch function. For example, to use a PGARCH(1, 1, 1) model for eachseries when fitting the principal component model to the bivariate timeseries hp.ibm, use the following conditional variance formula:> mgarch(hp.ibm~1, ~prcomp.pgarch(1,1,1), trace=F)Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =~ prcomp.pgarch(1, 1, 1), trace = F)
Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:~ prcomp.pgarch(1, 1, 1)

13.5 Multivariate GARCH Model Extensions

503

Coefficients:
C(1) -3.519e-04
C(2) -1.614e-05
A(1, 1) 1.848e-03
A(2, 2) 3.565e-04
ARCH(1; 1, 1) 1.100e-01
ARCH(1; 2, 2) 5.992e-02
GARCH(1; 1, 1) 8.380e-01
GARCH(1; 2, 2) 9.222e-01
Eigenvectors: (orthonormal transform matrix):HP
IBM
HP -0.9054 0.4245
IBM -0.4245 -0.9054
Eigenvalues:[1] 0.0006002 0.0001222
Pure Diagonal Model
Sometimes, the user may want to fit the same type of GARCH model to alarge number of time series. The mgarch function also allows this type of
univariate GARCH-based estimation, which totally ignores the correlationof the multivariate time series. For this purpose, any univariate GARCH
specification can be used directly with the mgarch function. For example,
to estimate a TGARCH(1, 1) model to both hp.s and ibm.s at the sametime, use the following command:> mgarch(hp.ibm~1, ~egarch(1,1), leverage=T, trace=F)Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var = ~ egarch(1, 1),leverage = T, trace = F)
Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:
Coefficients:
C(1) 0.0004561
C(2) 0.0001810
A(1, 1) -0.7959068
A(2, 2) -0.9192535~ egarch(1, 1)

504

13. Multivariate GARCH Modeling

ARCH(1; 1, 1) 0.1618657
ARCH(1; 2, 2) 0.1350345
GARCH(1; 1, 1) 0.9124564
GARCH(1; 2, 2) 0.9066042
LEV(1; 1, 1) 0.0243099
LEV(1; 2, 2) -0.1743824
Although the optional argument leverage can be used with any univariate GARCH-based models for mgarch function, it is ignored for BEKK,DVEC and its modifications.

13.5.4 ARMA Terms and Exogenous Variables
All the multivariate GARCH models considered so far have been restricted
to a constant mean assumption. However, the mgarch function actually
allows a more general model with a vector ARMA (VARMA) structure
and optional inclusion of weakly exogenous variables in the conditionalmean:
yt = c +rXi=1

Φi yt−i +LX
l=0βl xt−l + ²t +sX

Θj ²t−j(13.10)j=1

where Φi are k × k autoregressive coeﬃcient matrix, Θj are k × k moving
average coeﬃcient matrix, xt is the m × 1 vector of weakly exogenous
variables, and βl is k × m coeﬃcients of xt−l . Note that a distributed lag
structure of xt is allowed in the above equation by setting L to be a positive
integer.
To include an AR(r), MA(s), or ARMA(r, s) term in the conditionalmean, the user can simply add an ar(r), ma(s), or arma(r,s) term to theconditional mean formula. However, by default, Φi and Θj are restricted
to be diagonal matrices for parsimonious reasons. This behavior can bechanged by setting the optional argument armaType of the mgarch function. In particular, if armaType="lower", then Φi and Θj are restricted
to be lower triangular matrices; if armaType="full", then Φi and Θj are
not restricted. When weakly exogenous variables xt are used, the optional
argument xlag can be set to a positive integer to use a distributed lag
structure.
Example 87 Single factor model with multivariate GARCH errors
Section 7.5 of Chapter 7 developed a single factor model with GARCH
errors. Here that example is extended to multivariate context using thebivariate time series hp.ibm. The univariate example used daily returns
on the value weighted New York Stock Exchange index as the “market
returns” to estimate the “market beta”. In practice, this market beta canbe biased due to the serial correlation in the market returns. Hence, both

13.5 Multivariate GARCH Model Extensions

505

MGARCH Volatility

0.01
0.05

HP

0.02

0.03

0.04

Conditional SD

0.02

0.03

0.04

IBM

Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1

1984

1985

1986

1987

1988

1989

1990

1991

1992

FIGURE 13.8. Idiosyncratic volatility of bivariate hp.ibm.

nyse.s and its first lag as regressors are included in the conditional meanequation, and the DVEC(1, 1) model is used in the conditional variance:> hp.ibm.beta = mgarch(hp.ibm~seriesData(nyse.s), ~dvec(1,1),+ xlag=1)> summary(hp.ibm.beta)Call:mgarch(formula.mean = hp.ibm ~ seriesData(nyse.s),formula.var = ~ dvec(1, 1), xlag = 1)
Mean Equation: hp.ibm ~ seriesData(nyse.s)
Conditional Variance Equation:
Conditional Distribution:~ dvec(1, 1)

gaussian-----------------------------------------------------------Estimated Coefficients:-----------------------------------------------------------Value Std.Error t value Pr(>|t|)C(1) 7.860e-05 3.714e-04
0.2116 4.162e-01

506

13. Multivariate GARCH ModelingC(2) -3.343e-04 1.947e-04
X(0; 1, 1) 1.491e+00 2.867e-02
X(0; 2, 1) 1.112e+00 1.751e-02
X(1; 1, 1) -1.497e-01 3.233e-02
X(1; 2, 1) -1.802e-01 1.898e-02
A(1, 1) 1.028e-04 1.420e-05
A(2, 1) 6.166e-06 4.520e-06
A(2, 2) 3.117e-05 3.226e-06
ARCH(1; 1, 1) 1.230e-01 1.812e-02
ARCH(1; 2, 1) 5.030e-03 1.530e-02
ARCH(1; 2, 2) 2.567e-01 2.155e-02
GARCH(1; 1, 1) 5.494e-01 5.543e-02
GARCH(1; 2, 1) 7.904e-01 1.483e-01
GARCH(1; 2, 2) 4.261e-01 4.432e-02-1.7166 4.311e-02
52.0032 0.000e+00
63.4896 0.000e+00-4.6297 1.949e-06-9.4945 0.000e+00
7.2413 3.160e-13
1.3642 8.633e-02
9.6600 0.000e+00
6.7878 7.482e-120.3288 3.712e-01
11.9125 0.000e+00
9.9112 0.000e+00
5.3285 5.511e-08
9.6126 0.000e+00...
In the above output, the coeﬃcient matrix β0 of nyse.s is denoted byX(0;i,j) and β1 of the first lag of nyse.s is denoted by X(1;i,j). All
those coeﬃcients are very significant. Now compare the GARCH(1;i,j) coeﬃcients with those of hp.ibm.dvec; after taking account of the market
eﬀects, the persistence in the GARCH volatilities has dropped quite a bit.
The estimated conditional volatility can also be plotted as shown in Figure 13.8. Compare this with Figure 13.5: since the market eﬀects are alreadytaken into account in the above single factor model, the volatility in Figure 13.8 can be treated as the “idiosyncratic” volatility, while Figure 13.5
also includes the systematic market component.
Weakly exogenous variables are also allowed in the conditional varianceequation for multivariate GARCH models. For example, for the DVEC(p, q)
model, the general conditional variance equation is:Σt = A0 +p
X
i=1

Ai ⊗ (²t−i ²0t−i ) +

q
X
j=1

Bj ⊗ Σt−j + D · Zt · D0

(13.11)

where Zt is a diagonal matrix with the m × 1 weakly exogenous variable(Zt1 , · · · , Ztm ) on the diagonal, and D is k×m coeﬃcient matrix. Note thatusing this formulation, the regressor eﬀects are guaranteed to be positive
semi-definite as long as the regressors Zt are non-negative.
Example 88 Monday and Friday eﬀects of volatility
There is a conjecture that the volatility in stock markets may be higher
on Mondays and Fridays. To investigate if this conjecture holds for thebivariate time series hp.ibm, build a dummy variable for those observations
falling on a Monday or a Friday:> weekdaysVec = as.integer(weekdays(positions(hp.ibm)))13.5 Multivariate GARCH Model Extensions

507> MonFriDummy = (weekdaysVec == 2 | weekdaysVec == 6)Note that the integer representation of Monday in S-PLUS is two because
Sunday is represented as one. Now add MonFriDummy as an exogenous variable in the conditional variance formula:> hp.ibm.dummy = mgarch(hp.ibm~1, ~dvec(1,1)+MonFriDummy)> summary(hp.ibm.dummy)Call:mgarch(formula.mean = hp.ibm ~ 1, formula.var =+ MonFriDummy)~ dvec(1, 1)

Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:
Conditional Distribution:~ dvec(1, 1) + MonFriDummygaussian-----------------------------------------------------------Estimated Coefficients:-----------------------------------------------------------Value Std.Error t value Pr(>|t|)C(1) 6.953e-04 4.696e-04 1.4806 6.943e-02
C(2) 2.659e-04 2.849e-04 0.9333 1.754e-01
A(1, 1) 3.369e-05 8.612e-06 3.9124 4.723e-05
A(2, 1) 7.384e-06 5.682e-06 1.2997 9.693e-02
A(2, 2) 2.011e-05 5.214e-06 3.8565 5.934e-05
ARCH(1; 1, 1) 6.400e-02 8.952e-03 7.1494 6.088e-13
ARCH(1; 2, 1) 3.546e-02 7.029e-03 5.0443 2.482e-07
ARCH(1; 2, 2) 1.076e-01 1.004e-02 10.7141 0.000e+00
GARCH(1; 1, 1) 8.600e-01 1.716e-02 50.1061 0.000e+00
GARCH(1; 2, 1) 8.805e-01 1.816e-02 48.4743 0.000e+00
GARCH(1; 2, 2) 7.472e-01 2.855e-02 26.1706 0.000e+00
Z(1,1) 3.139e-03 2.440e-03 1.2865 9.921e-02
Z(2,1) 4.426e-03 1.074e-03 4.1215 1.959e-05...
In the above output, Z(1,1) denotes the coeﬃcient of the dummy variable
for hp.s, the p-value of which is higher than the conventional 5% level, andZ(2,1) denotes the coeﬃcient for ibm.s, the p-value of which is very close
to zero. So it seems that for IBM stocks, the volatility tends to be slightly
higher on Mondays and Fridays.

508

13. Multivariate GARCH Modeling13.5.5 Multivariate Conditional t-Distribution
In all the multivariate GARCH models fitted so far, it has been assumed
that the residuals ²t follow a conditional multivariate normal distribution.
The mgarch function also allows the residuals to follow a multivariate Student’s t distribution.
If a k-dimensional random variable ut follows a multivariate Student’s t
distribution with ν degrees of freedom and the scale matrix St , the probability density function (PDF) of ut is given by:
f (ut ) =|St |−1/2Γ[(ν + k)/2](ν+k)/2(πν)k/2 Γ(ν/2) [1 + u0t S−1
t ut /ν](13.12)where Γ(·) is the gamma function. The covariance matrix of ut is given by:
Cov(ut ) =

ν
St .
ν −2

If the error term ²t is assumed in multivariate GARCH models follows aconditional multivariate Student’s t distribution with ν degrees of freedom
and Cov(²t ) = Σt , obviously the scale matrix St should be chosen so that
St =

ν−2
Σt .
ν

By substituting the above relationship into (13.12), the user can easily derive the log-likelihood function for multivariate GARCH models with conditional multivariate Student’s t distributed errors. The unknown model
parameters can also be routinely estimated using maximum likelihood estimation.
To use the multivariate Student’s t distribution with the mgarch function
to estimate a multivariate GARCH model, simply set the optional argument
cond.dist to "t". For example:> hp.ibm.dvec.t = mgarch(hp.ibm~1, ~dvec(1,1), cond.dist="t")The estimated degree of freedom ν is contained in the cond.dist component of the returned object:> hp.ibm.dvec.t$cond.dist$cond.dist:[1] "t"$dist.par:[1] 6.697768$dist.est:[1] T13.6 Multivariate GARCH Prediction509

QQ-Plot of Standardized Residuals-6
hp.ibm.dvec.t
HP

-4

-2

0

2

4

6

hp.ibm.dvec.t
IBM
5Standardized Residuals

0

-5

-10
hp.ibm.dvec
HP

hp.ibm.dvec
IBM

5

0

-5

-10
-6

-4

-2

0

2

4

6

Quantile of Comparison Distribution

FIGURE 13.9. Comparison of QQ-plot using normal and Student-t distributions.

Compare this model with the one fitted using multivariate normal distribution:> hp.ibm.comp = compare.mgarch(hp.ibm.dvec, hp.ibm.dvec.t)> hp.ibm.comp
hp.ibm.dvec hp.ibm.dvec.t
AIC-21886-22231
BIC-21825-22164
Likelihood
10954
11128
> plot(hp.ibm.comp, qq=T)
Obviously, the multivariate Student’s t distribution provides a much better
fit. This can also be confirmed by comparing the QQ-plot of standardized
residuals, which is shown in Figure 13.9.

13.6 Multivariate GARCH Prediction
Predictions from multivariate GARCH models can be generated in a similar fashion to predictions from univariate GARCH models. Indeed, for theunivariate GARCH-based models, such as CCC model and principal component model, the predictions are generated from the underlying univariate
GARCH models and then converted to the scale of the original multivariate

510

13. Multivariate GARCH Modelingtime series by using the appropriate transformation. This section focuseson predicting from DVEC model, because predicting from BEKK model
can be performed similarly.
For multivariate GARCH models, predictions can be generated for boththe levels of the original multivariate time series and its conditional covariance matrix. Predictions of the levels are obtained just as for vector autoregressive (VAR) models. Compared with VAR models, the predictionsof the conditional covariance matrix from multivariate GARCH models canbe used to construct more reliable confidence intervals for predictions of
the levels.
To illustrate the prediction of conditional covariance matrix for multivariate GARCH models, consider the conditional variance equation for the
DVEC(1, 1) model:
Σt = A0 + A1 ⊗ (²t−1 ²0t−1 ) + B1 ⊗ Σt−1
which is estimated over the time period t = 1, 2, · · · , T . To obtain ET (ΣT +k ),use the forecasts of conditional covariance matrix at time T + k for k > 0,given information at time T . For one-step-ahead prediction, it is easy toobtain:ET (ΣT +1 ) = A0 + A1 ⊗ ET (²T ²0T ) + B1 ⊗ ET (ΣT )= A0 + A1 ⊗ (²T ²0T ) + B1 ⊗ ΣT
since an estimate of ²T and ΣT already exists after estimating the DVEC
model. When k = 2, it can be shown that
ET (ΣT +2 ) = A0 + A1 ⊗ ET (²T +1 ²0T +1 ) + B1 ⊗ ET (ΣT +1 )= A0 + (A1 + B1 ) ⊗ ET (ΣT +1 ).
where ET (ΣT +1 ) is obtained in the previous step. This procedure can be
iterated to obtain ET (ΣT +k ) for k > 2.
The predict method for “mgarch” objects in S+FinMetrics implements
the forecasting procedure for all the multivariate GARCH models supported by the mgarch function. The forecasts can be easily obtained bycalling the generic predict function for an “mgarch” object with the desired number of forecasting periods. For example, to obtain 10-step-ahead
forecasts from the BEKK model object hp.ibm.bekk fitted in Section 13.5,
use the following command:> hp.ibm.pred = predict(hp.ibm.bekk, 10)> class(hp.ibm.pred)[1] "predict.mgarch"> names(hp.ibm.pred)[1] "series.pred" "sigma.pred" "R.pred"13.6 Multivariate GARCH Prediction

511

Predicted Conditional SD

0.014850.01487

0.01489IBM

0.02360.02400.02440.0248

HP

2

4

6

8

10

FIGURE 13.10. BEKK prediction of conditional standard deviations.

The returned object hp.ibm.pred is of class “predict.mgarch”, and has
three components: series.pred represents the forecasts of the levels of thetime series, sigma.pred represents the forecasts of the conditional standard
deviations, and R.pred represents the forecasts of the conditional correlation matrix. Note that the sigma.pred and R.pred components can beused together to obtain the forecasts of the conditional covariance matrix.
S+FinMetrics also implements a plot method for “predict.mgarch”
objects, so that the multivariate forecasts can be visualized directly. For example, if the user calls the generic plot function directly on hp.ibm.pred:> plot(hp.ibm.pred)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Predicted Conditional Mean
3: plot: Predicted Conditional SD
Selection:
Selecting 3 will generate the plot of predicted conditional standard deviations, as shown in Figure 13.10, the confidence interval of the volatility
forecasts should be obtained as well. Section 13.8 shows how to obtain aconfidence interval using simulation-based forecasts.

512

13. Multivariate GARCH Modeling13.7 Custom Estimation of GARCH Models
13.7.1 GARCH Model Objects
For both “garch” and “mgarch” objects, there is a model component whichcontains all the necessary model specific information about the fitted univariate or multivariate GARCH model. For example, for the univariate“garch” object ford.mod11 fitted in Section 7.4 of Chapter 7:> class(ford.mod11$model)[1] "garch.model"> ford.mod11$model
Mean Equation: ford.s ~ 1
Conditional Variance Equation:~ garch(1, 1)------------ Constants in mean -----------value which
0.0007708
1---------- Constants in variance ---------value which
6.534e-06
1------------------- ARCH -----------------value whichlag 1 0.07454
1------------------ GARCH -----------------value whichlag 1 0.9102
1
So the model component of a “garch” object is of class “garch.model”.
Similarly, for the “mgarch” object hp.ibm.dvec fitted in this chapter:> class(hp.ibm.dvec$model)[1] "mgarch.model"> hp.ibm.dvec$model
Mean Equation: hp.ibm ~ 1
Conditional Variance Equation:~ dvec(1, 1)

13.7 Custom Estimation of GARCH Models

513------------ Constants in mean -----------value which
v1 0.0007017567
T
v2 0.0002932253
T---------- Constants in variance ---------v1.value
v2.value *** v1.which v2.which
v1 3.888696e-05 1.322108e-05 ***TTv2 1.322108e-05 2.876733e-05 ***TT------------------- ARCH -----------------Lag 1v1.valuev2.value *** v1.which v2.which
v1 0.06225657 0.03393546 ***TT
v2 0.03393546 0.10485809 ***TT------------------ GARCH -----------------Lag 1v1.value v2.value *** v1.which v2.which
v1 0.8567934 0.8783100 ***TT
v2 0.8783100 0.7421328 ***T
T
So the model component of an “mgarch” object is of class “mgarch.model”,
which has similar structures to a “garch.model” object. This section will
focus on “mgarch.model” objects, though all the things illustrated can also
be applied to “garch.model” objects.
Since an “mgarch.model” object contains all the necessary information
about a fitted GARCH model, this object can be saved or edited for many
purposes.6 The names of the components of an “mgarch.model” object canbe obtained using the S-PLUS function names:> names(hp.ibm.dvec$model)[1] "c.which" "c.value" "MA"[7] "a.which" "a.value" "info""AR""arch""garch"6 In the first release of S+GARCH module, there was a revise function which provides
a graphical user interface for editing this object. However, the function was broken as
the graphical user interface of S-PLUS went through several evolutions. Currently there
is no revise function in S+FinMetrics module.

514

13. Multivariate GARCH Modeling

The component c.value contains the value of the constant terms in theconditional mean equation, while the component a.value contains thevalue of the constant terms in the conditional variance equation. The MA,
AR, arch and garch components are lists themselves. For example:> hp.ibm.dvec$model$arch$order:[1] 1$value:$lag.1:[,1][,2][1,] 0.06225657 0.03393546[2,] 0.03393546 0.10485809$which:$lag.1:[,1] [,2]
[1,]
T
T
[2,]
T
T
Note that for each of the model coeﬃcients, there is a corresponding which
component that specifies if the coeﬃcient is free to be estimated by MLE.
If the which component is 1 or TRUE, then the corresponding coeﬃcientis free to be estimated; otherwise, the corresponding coeﬃcient is fixed atthat value during MLE. The next subsection shows how these values canbe edited for diﬀerent purposes.

13.7.2 Revision of GARCH Model Estimation
For both univariate and multivariate GARCH models, the unknown model
parameters are estimated using the BHHH algorithm (e.g., see Bollerslev, 1986). Both garch and mgarch functions take an optional argument
control, which can be used to control certain numerical aspects of the
BHHH algorithm. The defaults for those settings are provided in the online help file for bhhh.control.
Like many other nonlinear optimization algorithms, the BHHH algorithm
performs local optimization in the sense that the optimal solution it finds
may well be just a local optimum instead of the global optimum. To makesure that the global optimum has indeed been reached, start the algorithm
using a few diﬀerent starting values and see if they all lead to the same
optimum. For this purpose, edit the model component of a fitted “garch”
or “mgarch” object and use it as a new starting value.
Example 89 Restarting multivariate GARCH estimation

13.8 Multivariate GARCH Model Simulation

515> bekk.mod = hp.ibm.bekk$model> bekk.mod$a.value[2,1] = 0> hp.ibm.bekk2 = mgarch(series=hp.ibm, model=bekk.mod)
Note that when a model object is supplied directly to the mgarch (or garch)function, the series argument must be used to supply the data. The user
can easily verify that hp.ibm.bekk2 reached a smaller log-likelihood value,so the original fit hp.ibm.bekk seems to be better.
Example 90 Constraining multivariate GARCH estimation
For some GARCH models, the user may want to fix certain parametersat certain values during maximum likelihood estimation. For example, mostdaily financial security returns seem to fluctuate around a zero mean. In
this example, fix the constant terms in the conditional mean equation of
hp.ibm.bekk to zero and re-estimate the model:> bekk.mod = hp.ibm.bekk$model> bekk.mod$c.value = rep(0,2)> bekk.mod$c.which = rep(F,2)> hp.ibm.bekk3 = mgarch(series=hp.ibm, model=bekk.mod)> LR.stat = -2*(hp.ibm.bekk3$likelihood+ hp.ibm.bekk$likelihood)Note that since the log-likelihood value of the fitted model is returned, alikelihood ratio (LR) test of the restrictions imposed in the above example
can easily be performed.
The “garch.model” or “mgarch.model” object can be used for simulation. For example, simulation from fitted univariate GARCH models
actually uses this component. The next section illustrates this usage formultivariate GARCH models.

13.8 Multivariate GARCH Model Simulation
S+FinMetrics provides a method of the generic function simulate forobjects of class “mgarch”. The method function, simulate.mgarch, cantake a fitted “mgarch” object, or an “mgarch.model object, or simply auser specified list. This section illustrates how to create confidence intervals
for the predictions of conditional standard deviations using simulations.
Example 91 Simulation-based multivariate GARCH forecasts
The function simulate.mgarch only supports those multivariate GARCH
models of order (1, 1), which should be enough for most applications. To
simulate a multivariate GARCH process directly from a fitted “mgarch”
object such as hp.ibm.bekk, call the generic function simulate as follows:

516

13. Multivariate GARCH Modeling> hp.ibm.sim = simulate(hp.ibm.bekk, n=10)
where n = 10 specifies the length of the simulated time series. Since all
the model specific information is contained in the model component ofan “mgarch” object, which is an “mgarch.model” object as shown in theprevious section, an “mgarch.model can also pass directly to the function
simulate.mgarch. The following code example simulates 100 steps ahead
from the end of estimation period in hp.ibm.bekk, and replicates the simulation 200 times:> eps.start = residuals(hp.ibm.bekk)[2000,]@data> V.start = hp.ibm.bekk$S.t[2000, , ]> n.rep = 200> hp.ibm.sim = array(0, c(100, 2, n.rep))> set.seed(10)> for (i in 1:n.rep) {+eps.pred = rbind(eps.start, rmvnorm(100))+
tmp = simulate(hp.ibm.bekk, n=100, n.start=0,+
etat=eps.pred, V.start=V.start)$V.t+
hp.ibm.sim[, , i] = matrix(tmp,byrow=T,nrow=100)[,c(1,4)]+ }> hp.ibm.sim = sqrt(hp.ibm.sim)> hp.ibm.simpred = rowMeans(hp.ibm.sim, dims=2)> hp.ibm.simstde = rowStdevs(hp.ibm.sim, dims=2)
Note that to simulate the multivariate GARCH process using the last observation in the sample as the starting value, set n.start=0 and V.start
to the last estimated conditional covariance matrix. Similarly, the last estimated residual vector is used as the starting value in eps.pred, which
is otherwise standard normal random variables. All the simulated conditional standard deviations are saved in hp.ibm.sim, which is a three dimensional array. The simulation-based forecasts of conditional standard
deviations are computed as the average of hp.ibm.sim, and saved in theobject hp.ibm.simpred, while hp.ibm.simstde contains the standard errors of those forecasts.
Finally, use the following code to plot confidence intervals around thesimulation-based forecasts:> par(mfrow=c(2,1))> ci.upper = hp.ibm.simpred + 2*hp.ibm.simstde> ci.lower = hp.ibm.simpred - 2*hp.ibm.simstde> tsplot(cbind(hp.ibm.simpred[,1],ci.upper[,1],ci.lower[,1]))> title("Forecasted HP Volatility", xlab="Time", ylab="SD")13.9 References

517

0.025
0.020

SD

0.030

Forecasted HP Volatility

0

20

40

60

80

100

80

100

Time

0.016
0.008

0.012

SD

0.020

Forecasted IBM Volatility

0

20

40

60
Time

FIGURE 13.11. Simulation-based forecasts of BEKK model.

> tsplot(cbind(hp.ibm.simpred[,2],ci.upper[,2],ci.lower[,2]))> title("Forecasted IBM Volatility", xlab="Time", ylab="SD")> par(mfrow=c(1,1))
The plot shown in Figure 13.11 only used 200 replications, so the confidence
intervals are a little rough. If more replications are used, the confidence
intervals should be relatively smooth.

13.9 References
Alexander, C. O. (1998). “Volatility and Correlation: Methods, Models
and Applications,” in C. O. Alexander (ed.) Risk Management and Analysis: Measuring and Modeling Financial Risk. John Wiley & Sons, NewYork.
Andreou, E., and Ghysels, E. (2002). “Rolling Volatility Estimators:
Some new Theoretical, Simulation and Empirical Results,” Journal of Business and Economic Statistics, 20(3), 363-376.
Bollerslev, T. (1986). “Generalized Autoregressive Conditional Heteroskedasticity,” Journal of Econometrics, 31, 307-327.

518

13. Multivariate GARCH Modeling

Bollerslev, T. (1990). “Modeling the Coherence in Short-run Nominal Exchange Rates: a Multivariate Generalized ARCH Model,” Review of
Economics and Statistics, 72, 498-505.
Bollerslev, T., Engle, R. F., and Nelson, D. B. (1994). “ARCH
Models,” in R. F. Engle and D. L. McFadden (eds.), Handbook of Econometrics, Vol. 4. Elsevier Science B. V., Amsterdam.
Bollerslev, T., Engle, R. F., and Wooldridge, J. M. (1988). “A
Capital-Asset Pricing Model with Time-Varying Covariances,” Journal of
Political Economy, 96, 116-131.
Bougerol, P., and Picard, N. (1992). “Stationarity of GARCH Processes and of Some Nonnegative Time Series,” Journal of Econometrics,
52, 115-127.
Ding, Z. (1994). “Time Series Analysis of Speculative Returns,” Ph.D.
Thesis, Department of Economics, University of California, San Diego.
Engle, R. F., and Bollerslev, T. (1986). “Modeling the Persistenceof Conditional Variances,” Econometric Reviews, 5, 1-50.
Engle, R. F., and Kroner, K. F. (1995). “Multivariate Simultaneous
Generalized ARCH,” Econometric Theory, 11, 122-150.
Fleming, J., Kirby, C., and Ostdiek, B. (2001). “The Economic Valueof Volatility Timing,” Journal of Finance, 56, 329-352.
Foster, D. P., and Nelson, D. B. (1996). “Continuous Record Asymptotics for Rolling Sample Variance Estimators,” Econometrica, 64, 139-174.
Hosking, J. R. M. (1980). “The Multivariate Portmanteau Statistic,”
Journal of the American Statistical Association, 75, 602-608.
Nelson, D. B. (1990). “Stationarity and Persistence in the GARCH (1,1)
Model,” Econometric Theory, 6, 318-334.

This is page 519
Printer: Opaque this14
State Space Models

14.1 Introduction
The state space modeling tools in S+FinMetrics are based on the algorithms in SsfPack 3.0 developed by Siem Jan Koopman and described in
Koopman, Shephard and Doornik (1999, 2001)1 . SsfPack is a suite of C
routines for carrying out computations involving the statistical analysis ofunivariate and multivariate models in state space form. The routines allow
for a variety of state space forms from simple time invariant models tocomplicated time-varying models. Functions are available to put standard
models like ARMA and spline models in state space form. General routines are available for filtering, smoothing, simulation smoothing, likelihood
evaluation, forecasting and signal extraction. Full details of the statisticalanalysis is provided in Durbin and Koopman (2001). This chapter gives an
overview of state space modeling and the reader is referred to the papers byKoopman, Shephard and Doornik for technical details on the algorithms
used in the S+FinMetrics/SsfPack functions.
This chapter is organized as follows. Section 14.2 describes the general state space model and state space representation required for the
S+FinMetrics/SsfPack state space functions. Subsections describe thevarious S+FinMetrics/SsfPack functions for putting common time seriesmodels into state space form. The process of simulating observations froma given state space model is also covered. Section 14.3 summarizes the main
1 Information about Ssfpack can be found at http://www.ssfpack.com.

520

14. State Space Models

algorithms used for the analysis of state space models. These include theKalman filter, Kalman smoother, moment smoothing, disturbance smoothing and forecasting. Estimation of the unknown parameters in a state spacemodel is described in Section 14.4. The chapter concludes with a short discussion of simulation smoothing.
Textbook treatments of state space models are given in Harvey (1989,1993), Hamilton (1994), West and Harrison (1997), and Kim and Nelson(1999). Interesting applications of state space models in finance are given inEngle and Watson (1987), Bomhoﬀ (1994), Duan and Simonato (1999), andHarvey, Ruiz, and Shephard (1994), Carmona (2004), and Chan (2002).

14.2 State Space Representation
Many dynamic time series models in economics and finance may be represented in state space form. Some common examples are ARMA models, time-varying regression models, dynamic linear models with unobserved components, discrete versions of continuous time diﬀusion processes,
stochastic volatility models, non-parametric and spline regressions. The linear Gaussian state space model may be represented as the system of equations
αt+1=

dt + Tt · αt + Ht · η t
m×1 m×m m×1 m×r r×1(14.1)=

ct + Zt · αt
N×1 N×m m×1(14.2)=

θ t + Gt · εt
N×1 N×N N×1(14.3)

m×1

θt
N ×1

yt
N ×1
where t = 1, . . . , n and

α1
ηt
εt

∼ N (a, P),∼ iid N (0, Ir )∼ iid N (0, IN )(14.4)(14.5)(14.6)and it is assumed thatE[εt η 0t ] = 0
In (14.4), a and P are fixed and known but that can be generalized. The
state vector αt contains unobserved stochastic processes and unknown fixedeﬀects and the transition equation (14.1) describes the evolution of thestate vector over time using a first order Markov structure. The measurement equation (14.3) describes the vector of observations yt in terms of thestate vector αt through the signal θ t and a vector of disturbances εt . It
is assumed that the innovations in the transition equation and the innovations in the measurement equation are independent, but this assumption

14.2 State Space Representation521

can be relaxed. The deterministic matrices Tt , Zt , Ht , Gt are called system
matrices and are usually sparse selection matrices. The vectors dt and ct
contain fixed components and may be used to incorporate known eﬀects or
known patterns into the model; otherwise they are equal to zero.
The state space model (14.1)-(14.6) may be compactly expressed as
¶
µ
αt+1
=
δt
+
Φt
· αt +
ut ,(14.7)
yt(m+N )×1(m+N )×m m×1(m+N)×1
α1
ut

where
δt =

µ

dt
ct

¶

, Φt =∼ N (a, P)∼ iid N (0, Ωt )

µ

Tt
Zt

¶

, ut =µ(14.8)(14.9)

Ht η tGt εt¶, Ωt =µHt H0t00
Gt G0t¶

The initial value parameters are summarized in the (m + 1) × m matrix¶µ
P(14.10)Σ=
a0
For multivariate models, i.e. N > 1, it is assumed that the N × N matrix
Gt G0t is diagonal. In general, the system matrices in (14.7) are time varying.

14.2.1 Initial Conditions
The variance matrix P of the initial state vector α1 is assumed to be of
the form
P = P∗ + κP∞(14.11)
where P∞ and P∗ are symmetric m × m matrices with ranks r∞ and r∗ ,respectively, and κ is a large scalar value, e.g. κ = 106 . The matrix P∗
captures the covariance structure of the stationary components in the initial state vector, and the matrix P∞ is used to specify the initial variancematrix for nonstationary components. When the ith diagonal element of
P∞ is negative, the corresponding ith column and row of P∗ are assumed
to be zero, and the corresponding row and column of P∞ will be taken
into consideration. When some elements of state vector are nonstationary, the S+FinMetrics/SsfPack algorithms implement an “exact diﬀuse
prior” approach as described in Durbin and Koopman (2001) and Koopman, Shephard and Doornik (2001).

14.2.2 State Space Representation in
S+FinMetrics/SsfPack
State space models in S+FinMetrics/SsfPack utilize the compact representation (14.7) with initial value information (14.10). The following ex-

522

14. State Space Models

amples describe the specification of a state space model for use in the
S+FinMetrics/SsfPack state space modeling functions.
Example 92 State space representation of the local level model
Consider the following simple model for the stochastic evolution of thelogarithm of an asset price yt
αt+1
yt
α1

= αt + η ∗t , η ∗t ∼ iid N (0, σ 2η )αt + ε∗t ,=∼ N (a, P )ε∗t ∼ iid N (0, σ 2ε )(14.12)(14.13)(14.14)

where it is assumed that E[ε∗t η∗t ] = 0. In the above model, the observed
asset price yt is the sum of two unobserved components, αt and ε∗t . The
component αt is the state variable and represents the fundamental value(signal) of the asset. The transition equation (14.12) shows that the fundamental values evolve according to a random walk. The component ε∗trepresents random deviations (noise) from the fundamental value that are
assumed to be independent from the innovations to αt . The strength of
the signal in the fundamental value relative to the random deviation ismeasured by the signal-to-noise ratio of variances q = σ 2η /σ 2ε . The model(14.12)-(14.14) is called the random walk plus noise model, signal plus noisemodel or the local level model.2
The state space form (14.7) of the local level model has time invariant
parameters
¶
µ ¶
µ ¶
µ 2
ση 0
0
1
(14.15)
δ=
,Φ =
,Ω =
0
1
0 σ 2εwith errors σ η η t = η∗t and σ ε εt = ε∗t . Since the state variable αt is I(1),the unconditional distribution of the initial state α1 doesn’t have finite
variance. In this case, it is customary to set a = E[α1 ] = 0 and P =var(α1 )to some large positive number, e.g. P = 107 , in (14.14) to reflect that noprior information is available. Using (14.11), the initial variance is specified
with P∗ = 0 and P∞ = 1. Therefore, the initial state matrix (14.10) for the
local level model has the form
µ
¶
−1
Σ=
(14.16)0
where −1 implies that P∞ = 1.
In S+FinMetrics/SsfPack, a state space model is specified by creating
either a list variable with components giving the minimum components
2 A detailed technical analysis of this model is given in Durbin and Koopman (2001),
chapter 2.

14.2 State Space Representation

State space parameter
δ
Φ
Ω
Σ

523

List component namemDeltamPhi
mOmegamSigmaTABLE 14.1. State space form list components

necessary for describing a particular state space form or by creating an“ssf” object. To illustrate, consider creating a list variable containing thestate space parameters in (14.15)-(14.16), with σ η = 0.5 and σ ε = 1> sigma.e = 1> sigma.n = 0.5> a1 = 0> P1 = -1> ssf.ll.list = list(mPhi=as.matrix(c(1,1)),+ mOmega=diag(c(sigma.n^2,sigma.e^2)),+ mSigma=as.matrix(c(P1,a1)))> ssf.ll.list$mPhi:
[,1]
[1,]
1
[2,]
1
$mOmega:[,1] [,2][1,] 0.25
0[2,] 0.00
1$mSigma:
[,1]
[1,]
-1
[2,]
0
In the list variable ssf.ll.list, the component names match the state
space form parameters in (14.7) and (14.10). This naming convention, summarized in Table 14.1, must be used for the specification of any valid state
space model. Also, notice the use of the coercion function as.matrix. This
ensures that the dimensions of the state space parameters are correctly
specified.
An “ssf” object may be created from the list variable ssf.ll.list
using the S+FinMetrics/SsfPack function CheckSsf:> ssf.ll = CheckSsf(ssf.ll.list)> class(ssf.ll)[1] "ssf"

524

14. State Space Models> names(ssf.ll)[1] "mDelta" "mPhi""mOmega"[6] "mJOmega" "mJDelta" "mX"
[11] "cY""cSt"> ssf.ll$mPhi:
[,1]
[1,]
1
[2,]
1
$mOmega:[,1] [,2][1,] 0.25
0[2,] 0.00
1$mSigma:
[,1]
[1,]
-1
[2,]
0
$mDelta:
[,1]
[1,]
0
[2,]
0
$mJPhi:[1] 0$mJOmega:[1] 0$mJDelta:
[1] 0
$mX:
[1] 0
$cT:
[1] 0
$cX:
[1] 0
$cY:
[1] 1

"mSigma""cT""mJPhi""cX"

14.2 State Space Representation

525

$cSt:
[1] 1
attr(, "class"):[1] "ssf"
The function CheckSsf takes a list variable with a minimum state space
form, coerces the components to matrix objects and returns the full parameterization of a state space model used in many of the S+FinMetrics/
SsfPack state space modeling functions. See the online help for CheckSsf
for descriptions of the components of an “ssf” object.
Example 93 State space representation of a time varying parameter regression model
Consider the Capital Asset Pricing Model (CAPM) with time varying
intercept and slope
rt
αt+1
β t+1

= αt + β t rM,t + ν t , ν t ∼ GW N (0, σ 2ν )= αt + ξ t , ξ t ∼ GW N (0, σ 2ξ )= β t + ς t , ς t ∼ GW N (0, σ 2ς )(14.17)(14.18)(14.19)where rt denotes the return on an asset in excess of the risk free rate,
and rM,t denotes the excess return on a market index. In this model, boththe abnormal excess return αt and asset risk β t are allowed to vary overtime following a random walk specification. Let αt = (αt , β t )0 , yt = rt ,xt = (1, rM,t )0 , Ht =diag(σ ξ , σ ς )0 and Gt = σν . Then the state space form
(14.7) of (14.17) - (14.19) is
µ
¶ µ
¶
¶
µ
Hη t
I2
αt+1
=
αt +
yt
x0t
Gεt
and has parameters
Φt =

µ

I2
x0t

¶

⎛

σ 2ξ
⎝
, Ω=
0
0

0
σ 2ς
0

⎞
0
0 ⎠
σ 2ν

(14.20)Since αt is I(1) the initial state vector α1 doesn’t have finite variance so
it is customary to set a = 0 and P = κI2 where κ is large. Using (14.11),
the initial variance is specified with P∗ = 0 and P∞ = I2 . Therefore, theinitial state matrix (14.10) for the time varying CAPM has the form⎛⎞
−1 0
Σ = ⎝ 0 −1 ⎠
0
0

526

14. State Space Models

The state space parameter matrix Φt in (14.20) has a time varying systemelement Zt = x0t . In S+FinMetrics/SsfPack, the specification of this timevarying element in Φt requires an index matrix JΦ and a data matrix
X to which the indices in JΦ refer. The index matrix JΦ must have thesame dimension as Φt . The elements of JΦ are all set to −1 except theelements for which the corresponding elements of Φt are time varying. The
non-negative index value indicates the column of the data matrix X whichcontains the time varying values.
The specification of the state space form for the time varying CAPM
requires values for the variances σ 2ξ , σ 2ς , and σ 2ν as well as a data matrix X whose rows correspond with Zt = x0t = (1, rM,t ). For example, let
σ 2ξ = (0.01)2 , σ 2ς = (0.05)2 and σ 2ν = (0.1)2 and construct the data matrix X using the excess return data in the S+FinMetrics “timeSeries”
excessReturns.ts> X.mat = cbind(1,+ as.matrix(seriesData(excessReturns.ts[,"SP500"]))The state space form may be created using> Phi.t = rbind(diag(2),rep(0,2))> Omega = diag(c((.01)^2,(.05)^2,(.1)^2))> J.Phi = matrix(-1,3,2)> J.Phi[3,1] = 1> J.Phi[3,2] = 2> Sigma = -Phi.t> ssf.tvp.capm = list(mPhi=Phi.t,+ mOmega=Omega,+ mJPhi=J.Phi,+ mSigma=Sigma,+ mX=X.mat)> ssf.tvp.capm$mPhi:[,1] [,2]
[1,]
1
0
[2,]
0
1
[3,]
0
0
$mOmega:[,1][,2] [,3][1,] 0.0001 0.0000 0.00
[2,] 0.0000 0.0025 0.00
[3,] 0.0000 0.0000 0.01
$mJPhi:[,1] [,2]14.2 State Space Representation

Parameter index matrix
Jδ
JΦ
JΩTime varying component data matrix
X

527

List component namemJDeltamJPhi
mJOmega
List component namemX

TABLE 14.2. S+FinMetrics time varying state space components

[1,]
[2,]
[3,]

-1
-1
1

-1
-1
2

$mSigma:[,1] [,2]
[1,]
-1
0
[2,]
0
-1
[3,]
0
0
$mX:
numeric matrix: 131 rows, 2 columns.
SP500
1 1 0.002803
2 1 0.017566...
131 1 -0.0007548
Notice in the specification of Φt the values associated with x0t in the third
row are set to zero. In the index matrix JΦ , the (3,1) element is 1 andthe (3,2) element is 2 indicating that the data for the first and second
columns of x0t come from the first and second columns of the component
mX, respectively.
In the general state space model (14.7), it is possible that all of the system
matrices δ t , Φt and Ωt have time varying elements. The corresponding
index matrices Jδ , JΦ and JΩ indicate which elements of the matrices δ t , Φtand Ωt are time varying and the data matrix X contains the time varying
components. The naming convention for these components is summarizedin Table 14.2.

14.2.3 Missing Values
The S+FinMetrics/SsfPack state space modeling functions can handle
missing values in the vector of response variables yt in (14.3). Missing
values are not allowed in the state space system matrices Φt , Ωt , Σ andδ t . Missing values are represented by NA in S-PLUS.

528

14. State Space Models

In the S+FinMetrics/SsfPack state space functions, the observation vector yt with missing values will be be reduced to the vector yt† without
missing values and the measurement equation will be adjusted accordingly.
For example, the measurement equation yt = ct + Zt αt + Gt εt with
⎞
⎞
⎞
⎛ ⎞
⎛
⎛
5
1
Z1,t
G1,t
⎜ NA ⎟⎜ 2 ⎟⎜ Z2,t ⎟⎜ G2,t ⎟
⎟
⎟
⎟
⎜ ⎟
⎜
⎜
yt = ⎜⎝ 3 ⎠ , ct = ⎝ 3 ⎠ , Zt = ⎝ Z3,t ⎠ , Gt = ⎝ G3,t ⎠
NA
4
Z4,t
G4,t
⎛

reduces to the measurement equation yt† = c†t + Z†t αt + G†t εt with
yt† =

µ

5
3

¶

, c†t =µ1
3¶

, Z†t =

µ

Z1,t
Z3,t

¶

, G†t =

µ

G1,t
G3,t

¶

The SsfPack algorithms in S+FinMetrics automatically replace the observation vector yt with yt† when missing values are encountered and thesystem matrices are adjusted accordingly.

14.2.4 S+FinMetrics/SsfPack Functions for Specifying theState Space Form for Some Common Time Series
ModelsS+FinMetrics/SsfPack has functions for the creation of the state space
representation of some common time series models. These functions andmodels are described in the following sub-sections.

ARMA Models
Following Harvey (1993, Sec. 4.4), the ARMA(p, q) model with zero mean3
yt = φ1 yt−1 + · · · + φp yt−p + ξ t + θ1 ξ t−1 + · · · + θq ξ t−q(14.21)

may be put in state space form with transition and measurement equations
αt+1yt= Tαt + Hξ t , ξ t ∼ N (0, σ 2ε )= Zαt

3 Note that the MA coeﬃcients are specified with positive signs, which is the opposite
of how the MA coeﬃcients are specified for models estimated by the S-PLUS function
arima.mle.

14.2 State Space Representationand time invariant system matrices⎛
φ1
1 0 ···
⎜ φ2
0 1
⎜
⎜
.
..
.
T = ⎜
.
.
⎜
⎝ φm−1 0 0φm
0 0 ···¡¢
1 0 ··· 0 0
Z =

⎞
⎞
⎛
1
0
⎜ θ1 ⎟
0 ⎟⎟⎟⎜.. ⎟ , H = ⎜ .. ⎟ ,
⎜ . ⎟
. ⎟
⎟
⎟
⎜
⎝ θm−1 ⎠
1 ⎠
θm

529

(14.22)where d, c and G of the state space form (14.1)-(14.3) are all zero andm = max(p, q + 1). The state vector αt has the form⎞⎛yt
⎜ φ2 yt−1 + · · · + φp yt−m+1 + θ1 ξ t + · · · + θm−1 ξ t−m+2 ⎟⎟⎜⎟⎜αt = ⎜ φ3 yt−1 + · · · + φp yt−m+2 + θ2 ξ t + · · · + θm−1 ξ t−m+3 ⎟ (14.23)
⎟
⎜
..
⎠
⎝
.
φm yt−1 + θm−1 ξ t
In compact state space form (14.7), the model is
µ
µ
¶
¶
µ
¶
αt+1
H
T
ξt
=
αt +
0
Z
yt
= Φαt + ut

and
Ω=

µ

σ 2ξ HH0000¶

If yt is stationary then αt ∼ N (0, V) is the unconditional distribution of
the state vector, and the covariance matrix V satisfies V = TVT0 +σ 2ξ HH0 ,which can be solved for the elements of V. The initial value parameters are
then
µ
¶
V
Σ=
00

The S+FinMetrics/SsfPack function GetSsfArma creates the state spacesystem matrices for any univariate stationary and invertible ARMA model.
The arguments expected by GetSsfArma are> args(GetSsfArma)function(ar = NULL, ma = NULL, sigma = 1, model = NULL)

where ar is the vector of p AR coeﬃcients, ma is the vector of q MA coefficients, sigma is the innovation standard deviation σ ξ , and model is a list
with components giving the AR, MA and innovation standard deviation.
If the arguments ar, ma, and sigma are specified, then model is ignored.
The function GetSsfArma returns a list containing the system matrices Φ,Ω and the initial value parameters Σ.

530

14. State Space Models

Example 94 AR(1) and ARMA(2,1)
Consider the AR(1) modelyt = 0.75yt−1 + ξ t , ξ t ∼ GW N (0, (0.5)2 )The state space form may be computed using> ssf.ar1 = GetSsfArma(ar=0.75,sigma=.5)> ssf.ar1$mPhi:[,1][1,] 0.75[2,] 1.00$mOmega:[,1] [,2][1,] 0.25
0[2,] 0.00
0$mSigma:[,1][1,] 0.5714[2,] 0.0000
In the component mSigma, the unconditional variance of the initial state
α1 is computed as var(α1 ) = (0.5)2 /(1 − 0.752 ) = 0.5714.
Next, consider the ARMA(2,1) modelyt = 0.6yt−1 + 0.2yt−2 + εt − 0.2εt−1 , εt ∼ GW N (0, 0.9)
The state space system matrices may be computed using> arma21.mod = list(ar=c(0.6,0.2),ma=c(-0.2),sigma=sqrt(0.9))> ssf.arma21 = GetSsfArma(model=arma21.mod)> ssf.arma21$mPhi:[,1] [,2][1,] 0.6
1[2,] 0.2
0[3,] 1.00$mOmega:[,1][,2] [,3][1,] 0.90 -0.180
0[2,] -0.18 0.036
0[3,] 0.00 0.000
014.2 State Space Representation

531$mSigma:[,1][,2][1,] 1.58571 0.01286[2,] 0.01286 0.09943[3,] 0.00000 0.00000
The unconditional variance of the initial state vector α1 = (α11 , α12 )0 is in
the top 2 × 2 block of mSigma and isµ¶1.586 0.013
var(α1 ) =.
0.013 0.099
Structural Time Series Models
The basic univariate unobserved components structural time series model(STSM) for a time series yt has the form
yt = μt + γ t + ψ t + ξ t(14.24)

where μt represents the unobserved trend component, γ t represents theunobserved seasonal component, ψ t represents the unobserved cycle component, and ξ t represents the unobserved irregular component.
The nonstationary trend component μt has the form of a local linear
trend:μt+1
β t+1= μt + β t + η t , η t ∼ GW N (0, σ 2η )= βt + ς t,

ς t ∼ GW N (0, σ 2ς )(14.25)(14.26)

with μ1 ∼ N (0, κ) and β 1 ∼ N (0, κ) where κ is a large number, e.g. κ = 106 .
If σ 2ς = 0 then μt follows a random walk with drift β 1 . If both σ2ς = 0 andσ 2η = 0 then μt follows a linear deterministic trend.
The stochastic seasonal component γ t has the form
S(L)γ t = ω t , ω t ∼ GW N (0, σ2ω )S(L) = 1 + L + · · · + Ls−1

(14.27)

where s gives the number of seasons. When σ 2ω = 0, the seasonal component
becomes fixed.
The stochastic cycle component ψ t is specified as
¶
µ
¶µ
¶ µ
¶
µ
cos λcψtχtsin λc
ψ t+1=ρ
+,(14.28)ψ ∗t+1− sin λc cos λc
ψ ∗t
χ∗t
µ
¶
µµ ¶
¶
χt
0
∼ GW N, σ 2ψ (1 − ρ2 )I20χ∗t
where ψ 0 ∼ N (0, σ 2ψ ), ψ ∗0 ∼ N (0, σ 2ψ ) and cov(ψ 0 , ψ ∗0 ) = 0. The parameterρ ∈ (0, 1] is interpreted as a damping factor. The frequency of the cycle

532

14. State Space Models

Argument
irregular
level
slope
seasonalDummy
seasonalTrig
seasonalHS
cycle0..
.
cycle9STSM parameterσησξσςσω , sσω , sσω , sσ ψ , λc , ρ..
.
σ ψ , λc , ρ

TABLE 14.3. Arguments to the S+FinMetrics function GetSsfStsm

is λc = 2π/c and c is the period. When ρ = 1 the cycle reduces to a
deterministic sine-cosine wave.
The S+FinMetrics/SsfPack function GetSsfStsm creates the state spacesystem matrices for the univariate structural time series model (14.24). The
arguments expected by GetSsfStsm are> args(GetSsfStsm)
function(irregular = 1, level = 0.1, slope = NULL,
seasonalDummy = NULL, seasonalTrig = NULL, seasonalHS= NULL, cycle0 = NULL, cycle1 = NULL, cycle2 = NULL,
cycle3 = NULL, cycle4 = NULL, cycle5 = NULL, cycle6 =NULL, cycle7 = NULL, cycle8 = NULL, cycle9 = NULL)
These arguments are explained in Table 14.3.
Example 95 Local level model
The state space for the local level model (14.12)-(14.14) may be constructed using> ssf.stsm = GetSsfStsm(irregular=1, level=0.5)> class(ssf.stsm)[1] "list"> ssf.stsm$mPhi:
[,1]
[1,]
1
[2,]
1
$mOmega:[,1] [,2][1,] 0.25
0[2,] 0.00
1

14.2 State Space Representation

533$mSigma:
[,1]
[1,]
-1
[2,]
0
The arguments irregular=1 and level=0.5 specify σ ε = 1 and σ η = 0.5
in (14.13) and (14.12), respectively.
Regression Models
The linear regression modelyt = x0t β + ξ t , ξ t ∼ GW N (0, σ 2ξ ),where xt is a k × 1 data matrix and β is a k × 1 fixed parameter vector,
may be put in the state space
µ
µ
¶ µ
¶
¶
αt+1
0
Ik
=
αt +
(14.29)σ ξ εtyt
x0t
The state vector satisfies αt+1 = αt = β. The state space system matrices
are Tt = Ik , Zt = x0t , Gt = σ ξ and Ht = 0. The coeﬃcient vector β
is fixed and unknown so that the initial conditions are α1 ∼ N (0, κIk )where κ is large. An advantage of analyzing the linear regression model instate space form is that recursive least squares estimates of the regression
coeﬃcient vector β are readily computed. Another advantage is that it isstraightforward to allow some or all of the regression coeﬃcients to be timevarying.
The linear regression model with time varying parameters may be introduced by setting Ht not equal to zero in (14.29). For example, to allow all
regressors to evolve as random walks set
⎞
⎛
σβ1
⎟
⎜
Ht = ⎝ ... ⎠σβ kso that the state equation becomes

αt+1 = αt + Ht η t(14.30)More explicitly, since αi,t+1 = αi,t = β i,t the state equation (14.30) implies
β i,t+1 = β i,t + σ β i · η i,t , i = 1, . . . , kIf σβ i = 0 then β i is constant.
The S+FinMetrics/SsfPack function GetSsfReg creates the state spacesystem matrices for the linear regression model. The arguments expectedby GetSsfReg are534

14. State Space Models> args(GetSsfReg)
function(mX)where mX is a rectangular data object which represents the regressors in themodel. The function GetSsfReg returns a list with components describing
the minimal state space representation of the linear regression model.
Example 96 Time trend regression model
Consider the time trend regression modelyt = μ + δt + ξ t , ξ t ∼ GW N (0, σ 2ξ )The state space form for a sample of size n = 10 is computed using> ssf.reg = GetSsfReg(cbind(1,1:10))> class(ssf.reg)[1] "list"> names(ssf.reg)[1] "mPhi""mOmega" "mSigma" "mJPhi"> ssf.reg$mPhi:[,1] [,2]
[1,]
1
0
[2,]
0
1
[3,]
0
0
$mOmega:[,1] [,2] [,3]
[1,]
0
0
0
[2,]
0
0
0
[3,]
0
0
1
$mSigma:[,1] [,2]
[1,]
-1
0
[2,]
0
-1
[3,]
0
0
$mJPhi:[,1] [,2]
[1,]
-1
-1
[2,]
-1
-1
[3,]
1
2
$mX:
[,1] [,2]"mX"

14.2 State Space Representation

[1,]
[2,]
[3,]
[4,]
[5,]
[6,]
[7,]
[8,]
[9,]
[10,]

1
1
1
1
1
1
1
1
1
1

535

1
2
3
4
5
6
7
8
9
10

Since the system matrix Zt = x0t , the parameter Φt is time varying andthe index matrix JΦ , represented by the component mJPhi, determines theassociation between the time varying data in Zt and the data supplied in
the component mX.
To specify a time trend regression with a time varying slope of the form
δ t+1 = δ t + ζ t , ζ t ∼ GW N (0, σ2ζ )(14.31)

one needs to specify a non-zero value for σ 2ζ in Ωt . For example, if σ2ζ = 0.5
then> ssf.reg.tvp = ssf.reg> ssf.reg.tvp$mOmega[2,2] = 0.5> ssf.reg.tvp$mOmega[,1] [,2] [,3]
[1,]
0 0.0
0
[2,]
0 0.5
0
[3,]
0 0.0
1
modifies the state space form for the time trend regression to allow a timevarying slope of the form (14.31).
Regression Models with ARMA Errors
The ARMA(p, q) models created with GetSsfArma do not have deterministic terms (e.g., constant, trend, seasonal dummies) or exogenous regressors
and are therefore limited. The general ARMA(p, q) model with exogenous
regressors has the form
yt = x0t β + ξ t
where ξ t follows an ARMA(p, q) process of the form (14.21). Let αt be
defined as in (14.23) and let
α∗t =

µ

αt
βt

¶

(14.32)

536

14. State Space Models

where βt = β. Writing the state equation implied by (14.32) as α∗t+1 =T∗ α∗t + H∗ η t and let
¸
∙
∙
¸
T 0
H
∗
∗
T =
, H =
,
0 Ik
0
Z∗t

= ( 1 0 . . . 0 x0t )where T and H are defined in (14.23). Then the state space form of theregression model with ARMA errors isµ ∗¶ µ ∗ ¶
µ
¶
H ηt
T
αt+1
∗
+
=
α
t
yt
Z∗t
0
The S+FinMetrics/SsfPack function GetSsfRegArma creates the state
space system matrices for the linear regression model with ARMA errors.
The arguments expected by GetSsfRegArma are> args(GetSsfRegArma)function(mX, ar = NULL, ma = NULL, sigma = 1, model = NULL)
where mX is a rectangular data object which represents the regressors in themodel, and the remaining arguments are the same as those for GetSsfArma.
The function GetSsfRegArma returns a list with components describing theminimal state space representation of the linear regression model.
Example 97 Time trend regression with AR(2) errors
The state space form of the time trend regression with AR(2) errors
ytξt= μ + δt + ξ t ,= φ1 ξ t−1 + φ2 ξ t−2 + ν t , ν t ∼ GW N (0, σ 2ξ )may be computed using> ssf.reg.ar2 = GetSsfRegArma(cbind(1,1:10),+ ar=c(1.25,-0.5))> ssf.reg.ar2$mPhi:[,1] [,2] [,3] [,4][1,] 1.25
100[2,] -0.50
000[3,] 0.00
010[4,] 0.00
001[5,] 1.00
000$mOmega:[,1] [,2] [,3] [,4] [,5]
[1,]
1
0
0
0
0

14.2 State Space Representation

[2,]
[3,]
[4,]
[5,]

0
0
0
0

0
0
0
0

0
0
0
0

0
0
0
0

537

0
0
0
0

$mSigma:[,1][,2] [,3] [,4][1,] 4.364 -1.818
00[2,] -1.818 1.091
00[3,] 0.000 0.000-10[4,] 0.000 0.000
0-1[5,] 0.000 0.000
00$mJPhi:[,1] [,2] [,3] [,4]
[1,]
-1
-1
-1
-1
[2,]
-1
-1
-1
-1
[3,]
-1
-1
-1
-1
[4,]
-1
-1
-1
-1
[5,]
-1
-1
1
2
$mX:
[1,]
[2,]
...
[10,]

[,1] [,2]
1
1
1
2
1

10

Nonparametric Cubic Spline Model
Suppose the continuous time process y(t) is observed at discrete time points
t1 , . . . , tn . Define δ i = ti − ti−1 ≥ 0 as the time duration between observations. The goal of the nonparametric cubic spline model is to estimate asmooth signal μ(t) from y(t) usingy(t) = μ(t) + ε(t)where ε(t) is a stationary error. The signal μ(t) is extracted by minimizing
¶2
Z µ 2
n
X
∂ μ(t)
2−1(y(ti ) − μ(ti )) + q
dt
∂t2
i=1

538

14. State Space Models

where the second term is a penalty term that forces μ(t) to be “smooth”4 ,and q may be interpreted as a signal-to-noise ratio. The resulting function
μ(t) may be expressed as the structural time series modelμ(ti+1 ) = μ(ti ) + δ i β(ti ) + η(ti )β(ti+1 ) = β(ti ) + ς(ti )
where

µ

η(ti )
ς(ti )

¶

˜N

∙µ

0
0

¶

, σ 2ς δ i

µ 1 2
3 δi
1
2 δi

1
2 δi

1

(14.33)¶¸

Combining (14.33) with the measurement equationy(ti ) = μ(ti ) + ε(ti )where ε(ti ) ∼ N (0, σ 2ε ) and is independent of η(ti ) and ς(ti ), gives thestate space form for the nonparametric cubic spline model. The state spacesystem matrices are
⎞
⎛ 2
⎛
⎞
qδ2t
qδ t
1 δi
0
2
⎟
⎜ 32
Φt = ⎝ 0 1 ⎠ , Ωt = ⎝ qδt qδ t 0 ⎠
2
1 0
0
0 1

When the observations are equally spaced, δ i is constant and the above
system matrices are time invariant.
The S+FinMetrics/SsfPack function GetSsfSpline creates the state
space system matrices for the nonparametric cubic spline model. The arguments expected by GetSsfSpline are> args(GetSsfSpline)function(snr = 1, delta = 0)
where snr is the signal-to-noise ratio q, and delta is a numeric vector
containing the time durations δ i (i = 1, . . . , n). If delta=0 then δ i is assumed to be equal to unity and the time invariant form of the model iscreated. The function GetSsfSpline returns a list with components describing the minimal state space representation of the nonparametric cubic
spline model.
Example 98 State space form of non-parametric cubic spline model
The default non-parametric cubic spline model with δ t = 1 is createdusing
4 This process can be interpreted as an interpolation technique and is similar to thetechnique used in the S+FinMetrics functions interpNA and hpfilter. See also thesmoothing spline method described in chapter sixteen.

14.2 State Space Representation> GetSsfSpline()$mPhi:[,1] [,2]
[1,]
1
1
[2,]
0
1
[3,]
1
0
$mOmega:[,1] [,2] [,3][1,] 0.3333 0.50[2,] 0.5000 1.0
0[3,] 0.0000 0.0
1$mSigma:[,1] [,2]
[1,]
-1
0
[2,]
0
-1
[3,]
0
0
$mJPhi:
NULL$mJOmega:
NULL
$mX:
NULL
For unequally spaced observations> t.vals = c(2,3,5,9,12,17,20,23,25)> delta.t = diff(t.vals)
and q = 0.2, the state space form is> GetSsfSpline(snr=0.2,delta=delta.t)$mPhi:[,1] [,2]
[1,]
1
1
[2,]
0
1
[3,]
1
0
$mOmega:[,1] [,2] [,3][1,] 0.06667 0.1
0[2,] 0.10000 0.2
0[3,] 0.00000 0.0
1

539

540

14. State Space Models$mSigma:[,1] [,2]
[1,]
-1
0
[2,]
0
-1
[3,]
0
0
$mJPhi:[,1] [,2]
[1,]
-1
1
[2,]
-1
-1
[3,]
-1
-1
$mJOmega:[,1] [,2] [,3]
[1,]
4
3
-1
[2,]
3
2
-1
[3,]
-1
-1
-1
$mX:
[,1]
[,2] [,3] [,4] [,5] [,6] [,7][,8][1,] 1.00000 2.0000 4.000 3.0 5.000 3.0 3.0 2.0000[2,] 0.20000 0.4000 0.800 0.6 1.000 0.6 0.6 0.4000[3,] 0.10000 0.4000 1.600 0.9 2.500 0.9 0.9 0.4000[4,] 0.06667 0.5333 4.267 1.8 8.333 1.8 1.8 0.5333

14.2.5 Simulating Observations from the State Space Model
Once a state space model has been specified, it is often interesting to draw
simulated values from the model. The S+FinMetrics/SsfPack function
SsfSim may be used for such a purpose. The arguments expected from
SsfSim are> args(SsfSim)
function(ssf, n = 100, mRan = NULL, a1 = NULL)
where ssf represents either a list with components giving a minimal state
space form or a valid “ssf” object, n is the number of simulated observations, mRan is user-specified matrix of disturbances, and a1 is the initialstate vector. The use of SsfSim is illustrated with the following examples.
Example 99 Simulating observations from the local level model
To generate 250 observations on the state variable αt+1 and observations
yt in the local level model (14.12) - (14.14) use> set.seed(112)

541

4

6

14.2 State Space Representation

-6

-4

-2

0

2

State
Response

0

50

100

150

200

250

FIGURE 14.1. Simulated values from local level model created using the
S+FinMetrics function SsfSim.

> ll.sim = SsfSim(ssf.ll.list,n=250)> class(ll.sim)[1] "matrix"> colIds(ll.sim)[1] "state""response"
The function SsfSim returns a matrix containing the simulated state variables αt+1 and observations yt . These values are illustrated in Figure 14.1
created using> tsplot(ll.sim)> legend(0,4,legend=c("State","Response"),lty=1:2)
Example 100 Simulating observations from CAPM with time varying
parameters
When simulating observations from a state space form with a data matrix
component mX using the function SsfSim, the number of simulated values
must match the number of rows of mX. The state space form for the CAPM
with time varying parameters created earlier uses a data matrix mX withn = 131 observations> nrow(ssf.tvp.capm$mX)[1] 131542

14. State Space Models

-0.05
-0.15

alpha(t)Time varying intercept

0

20

40

60

80

100

120

80

100

120

80

100

120

0.6
0.2

beta(t)1.0Time varying slope

0

20

40

60

-0.2
-0.4

returns

0.0

Simulated returns

0

20

40

60

FIGURE 14.2. Simulated state and response values from the CAPM with timevarying parameters state space form ssf.tvp.

The state variables are the time varying intercept αt+1 and the time varying
β t+1 . Natural initial values for these parameters are α1 = 0 and β 1 = 1.
Using these initial values, n = 131 observations are generated from the
CAPM with time varying parameters using> set.seed(444)> tvp.sim = SsfSim(ssf.tvp.capm,n=nrow(X.mat),a1=c(0,1))> colIds(tvp.sim)[1] "state.1" "state.2" "response"
The simulated state and response variables are illustrated in Figure 14.2
created using> par(mfrow=c(3,1))> tsplot(tvp.sim[,"state.1"],main="Time varying intercept",+ ylab="alpha(t)")> tsplot(tvp.sim[,"state.2"],main="Time varying slope",+ ylab="beta(t)")> tsplot(tvp.sim[,"response"],main="Simulated returns",+ ylab="returns")

14.3 Algorithms

543

14.3 Algorithms
14.3.1 Kalman Filter
The Kalman filter is a recursive algorithm for the evaluation of momentsof the normally distributed state vector αt+1 conditional on the observed
data Yt = (y1 , . . . , yt ). To describe the algorithm, let at = E[αt |Yt−1 ]denote the conditional mean of αt based on information available at timet − 1 and let Pt = var(αt |Yt−1 ) denote the conditional variance of αt .
The filtering or updating equations of the Kalman filter compute at|t =E[αt |Yt ] and Pt|t = var(αt |Yt ) using= at + Kt vt

at|t(14.34)0
Pt − Pt Z0t Kt(14.35)= yt − ct − Zt at= Zt Pt Z0t + Gt G0t= Pt Z0t F−1
t(14.36)(14.37)(14.38)

=

Pt|t
where
vt
Ft
Kt

The variable vt is the measurement equation innovation or prediction error,
Ft = var(vt ) and Kt is the Kalman gain matrix.
The prediction equations of the Kalman filter compute at+1 and Pt+1
using
at+1
Pt+1

= Tt at|t= Tt Pt|t T0t + Ht H0t(14.39)(14.40)In the Kalman filter recursions, if there are missing values in yt thenvt = 0, F−1
t = 0 and Kt = 0. This allows out-of-sample forecasts of αt andyt to be computed from the updating and prediction equations.

14.3.2 Kalman Smoother
The Kalman filtering algorithm is a forward recursion which computes onestep ahead estimates at+1 and Pt+1 based on Yt for t = 1, . . . , n. The
Kalman smoothing algorithm is a backward recursion which computes themean and variance of specific conditional distributions based on the full
data set Yn = (y1 , . . . , yn ). The smoothing equations arer∗t = T0t rt , N∗t = Tt Nt T0t , K∗t = N∗t Kt
0 ∗et = F−1
t vt − Kt rt ,(14.41)∗0
Dt = F−1
t + Kt Kt

and the backwards updating equations are
rt−1 = Z0t et + r∗t , Nt−1 = Z0t Dt Zt − < K∗t Zt > +N∗t

(14.42)

544

14. State Space Models

for t = n, . . . , 1 with initializations rn = 0 and Nn = 0. For any squarematrix A, the operator < A >= A + A0 . The values rt are called statesmoothing residuals and the values et are called response smoothing residuals. The recursions (14.41) and (14.42) are somewhat non-standard. Durbin
and Koopman (2001) show how they may be re-expressed in more standard
form.

14.3.3 Smoothed State and Response Estimates
The smoothed estimates of the state vector αt and its variance matrix
are denoted α̂t = E[αt |Yn ] and var(α̂t |Yn ), respectively. The smoothed
estimate α̂t is the optimal estimate of αt using all available information
Yn , whereas the filtered estimate ât|t is the optimal estimate only using
information available at time t, Yt . The computation of α̂t and its variancefrom the Kalman smoother algorithm is described in Durbin and Koopman(2001).
The smoothed estimate of the response yt and its variance are computedusingŷt = θ̂ t = E[θt |Yn ] = ct + Zt α̂tvar(ŷt |Yn ) = Zt var(α̂t |Yn )Z0t

14.3.4 Smoothed Disturbance Estimates
The smoothed disturbance estimates are the estimates of the measurement equations innovations εt and transition equation innovations η t basedon all available information Yn , and are denoted ε̂t = E[εt |Yn ] andη̂ t = E[η t |Yn ], respectively. The computation of ε̂t and η̂ t from the
Kalman smoother algorithm is described in Durbin and Koopman (2001).
These smoothed disturbance estimates are useful for parameter estimation
by maximum likelihood and for diagnostic checking. See chapter seven in
Durbin and Koopman (2001) for details.

14.3.5 Forecasting
The Kalman filter prediction equations (14.39) - (14.40) produces one-stepahead predictions of the state vector, at+1 = E[αt+1 |Yt ], along with prediction variance matrices Pt+1 . Out of sample predictions, together withassociated mean square errors, can be computed from the Kalman filter
prediction equations by extending the data set y1 , . . . , yn with a set of
missing values. When yτ is missing, the Kalman filter reduces to the prediction step described above. As a result, a sequence of m missing values
at the end of the sample will produce a set of h-step ahead forecasts for
h = 1, . . . , m.

14.3 Algorithms

545

14.3.6 S+FinMetrics/SsfPack Implementation of StateSpace Modeling Algorithms
The S+FinMetrics/SsfPack function KalmanFil implements the Kalmanfilter forward recursions in a computationally eﬃcient way, see Koopman,
Shephard and Doornik (2001). It produces an object of class “KalmanFil”
for which there are print and plot methods. The S+FinMetrics/SsfPack
function KalmanSmo computes the Kalman smoother backwards recursions,and produces an object of class “KalmanSmo” for which there are print
and plot methods. The functions KalmanFil and KalmanSmo are primarily
used by other S+FinMetrics/SsfPack state space functions that require
the output from the Kalman filter and Kalman smoother.
Filtered and smoothed estimates of αt and yt , with estimated variances,
as well as smoothed estimates of εt and η t , with estimated variances, arecomputed using the S+FinMetrics/SsfPack function SsfMomentEst. The
result of SsfMomentEst is an object of class “SsfMomentEst” for which
there is only a plot method. The function SsfMomentEst may also be used
to compute out-of-sample forecasts and forecast variances of αt and yt .
The use of the S+FinMetrics/SsfPack functions for implementing thestate space algorithms are illustrated with the following examples.
Example 101 State space algorithms applied to local level model
Consider the simulated data for the local level model (14.12) - (14.14) inthe object ll.sim computed earlier. The response variable yt is extractedusing> y.ll = ll.sim[,"response"]
Kalman Filter
The Kalman filter recursions for the simulated data from the local level
model are obtained using the S+FinMetrics/SsfPack function KalmanFilwith the optional argument task="STFIL" (which stands for state filtering)> KalmanFil.ll = KalmanFil(y.ll,ssf.ll,task="STFIL")> class(KalmanFil.ll)[1] "KalmanFil"
The function KalmanFil takes as input a vector of response data and either
a list describing the minimal state space form or a valid “ssf” object. The
result of KalmanFil is an object of class “KalmanFil” with components> names(KalmanFil.ll)[1] "mOut""innov"[4] "mGain""loglike"[7] "dVar""mEst"[10] "task""err""std.innov""loglike.conc""mOffP""call"

546

14. State Space Models

A complete explanation of the components of a “KalmanFil” object is givenin the online help for KalmanFil. These components are mainly used byother S+FinMetrics/SsfPack functions and are only briefly discussed here.
The component mOut contains the basic Kalman filter output.
> KalmanFil.ll$mOut
numeric matrix: 250 rows, 3 columns.
[,1]
[,2]
[,3]
[1,] 0.00000 1.0000 0.0000[2,] -1.28697 0.5556 0.4444...
[250,] -1.6371 0.3904 0.6096
The first column of mOut contains the prediction errors vt , the second column contains the Kalman gains, Kt , and the last column contains theinverses of the prediction error variances, Ft−1 . Since task="STFIL" thefiltered estimates at|t and yt|t = Zt at|t are in the component mEst
> KalmanFil.ll$mEst
numeric matrix: 250 rows, 4 columns.
[,1]
[,2]
[,3]
[,4]
[1,] 1.10889 1.10889 1.0000 1.0000[2,] 0.39390 0.39390 0.5556 0.5556...
[250,] 4.839 4.839 0.3904 0.3904
The plot method allows for a graphical analysis of the Kalman filter output> plot(KalmanFil.ll)
Make a plot selection (or 0 to exit):
1: plot: all
2: plot: innovations
3: plot: standardized innovations
4: plot: innovation histogram
5: plot: normal QQ-plot of innovations
6: plot: innovation ACF
Selection:
The standardized innovations vt /Ft are illustrated in Figure 14.3.
Kalman Smoother
The Kalman smoother backwards recursions for the simulated data fromthe local level model are obtained using the S+FinMetrics/SsfPack function KalmanSmo> KalmanSmo.ll = KalmanSmo(KalmanFil.ll,ssf.ll)

14.3 Algorithms

547

Standardized Prediction Errors

innovations

2

Values

1

0

-1

-2

0

50

100

150

200

250

FIGURE 14.3. Standardized innovations from Kalman filter applied to simulated
data from local level model.

> class(KalmanSmo.ll)[1] "KalmanSmo"
The function KalmanSmo takes as input an object of class “KalmanFil” andan associated list variable containing the state space form used to produce
the “KalmanFil” object. The result of KalmanSmo is an object of class“KalmanSmo” with components> names(KalmanSmo.ll)[1] "state.residuals"[3] "state.variance"[5] "aux.residuals"[7] "call""response.residuals""response.variance""scores"

The component state.residuals contains the smoothing residuals fromthe state equation, response.residuals contains the smoothing residuals from the measurement equation. The corresponding variances of theseresiduals are in the components state.variance and response.variance.
A multi-panel timeplot of the standardized residuals in the component
aux.residuals, illustrated in Figure 14.4, is created with the plot method> plot(KalmanSmo.ll,layout=c(1,2))

548

14. State Space Models

Standardized Smoothing Residuals-2

response

-3

-2

-1

0

1

2

Residuals

-1

0

1

2

3

state

0

50

100

150

200

250

FIGURE 14.4. Standardized smoothing residuals from Kalman smoother recursions computed from simulated data from local level model.

Filtered and Smoothed Moment Estimates
Filtered and smoothed estimates of αt and yt with corresponding estimatesof variances may be computed using the S+FinMetrics/SsfPack function
SsfMomentEst. To compute filtered estimates, call SsfMomentEst with theargument task="STFIL" (which stands for state filtering)> FilteredEst.ll = SsfMomentEst(y.ll,ssf.ll,task="STFIL")> class(FilteredEst.ll)[1] "SsfMomentEst"> names(FilteredEst.ll)[1] "state.moment""state.variance"[3] "response.moment""response.variance"[5] "task"
The function SsfMomentEst takes as input a vector of response data andeither a list describing the minimal state space form or a valid “ssf” object.
The result of SsfMomentEst is an object of class “SsfMomentEst” for which
there is only a plot method. The filtered estimates at|t and yt|t = ct +Zt at|t
are in the components state.moment and response.moment, respectively,
and the corresponding filtered variance estimates are in the components
state.variance and response.variance. From the measurement equation (14.13) in the local level model, at|t = yt|t

14.3 Algorithms

549

State Filtering-4

response

-4

-2

0

2

4

Values

-2

0

2

4

state

0

50

100

150

200

250

FIGURE 14.5. Filtered estimates of αt and yt computed from simulated datafrom local level model.

> FilteredEst.ll$state.moment[1:5][1] 1.1089 0.3939 -0.1389 -0.1141 0.3461> FilteredEst.ll$response.moment[1:5][1] 1.1089 0.3939 -0.1389 -0.1141 0.3461
The plot method creates a multi-panel timeplot, illustrated in Figure 14.5,of the estimates of αt and yt> plot(FilteredEst.ll,layout=c(1,2))
A plot of the filtered state estimates with 95% confidence intervals may becreated using> upper.state = FilteredEst.ll$state.moment ++ 2*sqrt(FilteredEst.ll$state.variance)> lower.state = FilteredEst.ll$state.moment + 2*sqrt(FilteredEst.ll$state.variance)> tsplot(FilteredEst.ll$state.moment,upper.state,lower.state,+ lty=c(1,2,2),ylab="filtered state")
and is shown in Figure 14.6.
The smoothed estimates α̂t and ŷt along with estimated variances may
be computed using SsfMomentEst with task="STSMO" (state smoothing)> SmoothedEst.ll = SsfMomentEst(y.ll,ssf.ll.list,task="STSMO")

14. State Space Models

0
-6

-4

-2

filtered state

2

4

6

550

0

50

100

150

200

250

FIGURE 14.6. Filtered estimates of αt with 95% confidence intervals computedfrom simulated values from local level model.

In the local level model, α̂t = ŷt> SmoothedEst.ll$state.moment[1:5][1] 0.24281 0.02629 -0.13914 -0.13925 -0.15455> SmoothedEst.ll$response.moment[1:5][1] 0.24281 0.02629 -0.13914 -0.13925 -0.15455
The smoothed state estimates with 95% confidence bands are illustrated
in Figure 14.7. Compared to the filtered state estimates, the smoothed
estimates are “smoother” and the confidence bands are slightly smaller.
Smoothed estimates of αt and yt without estimated variances may beobtained using the S+FinMetrics/SsfPack function SsfCondDens with theargument task="STSMO" (which stands for state smoothing)> smoothedEst.ll = SsfCondDens(y.ll,ssf.ll.list,task="STSMO")> class(smoothedEst.ll)[1] "SsfCondDens"> names(smoothedEst.ll)[1] "state""response" "task"
The object smoothedEst.ll is of class “SsfCondDens” with components
state, giving the smoothed state estimates α̂t , response, which gives thesmoothed response estimates ŷt , and task, naming the task performed. The

551

2
0
-4

-2

smoothed state4

6

14.3 Algorithms

0

50

100

150

200

250

FIGURE 14.7. Smoothed estimates of αt with 95% confidence intervals computedfrom simulated values from local level model.

smoothed estimates ŷt and α̂t may be visualized using the plot methodfor “SsfCondDens” objects> plot(smoothedEst.ll)
The resulting plot has the same format at the plot shown in Figure 14.5.
Smoothed Disturbance Estimates
The smoothed disturbance estimates ε̂t and η̂t may be computed using
SsfMomentEst with the optional argument task="DSSMO" (which stands
for disturbance smoothing)> disturbEst.ll = SsfMomentEst(y.ll,ssf.ll,task="DSSMO")> names(disturbEst.ll)[1] "state.moment""state.variance"[3] "response.moment""response.variance"[5] "task"
The estimates η̂ t are in the component state.moment, and the estimates ε̂t
are in the component response.moment. These estimates may be visualizedusing the plot method.

552

14. State Space Models

Koopman, Shephard and Doornik (1999) pointed out that, in the local
level model, the standardized smoothed disturbances
ε̂t
η̂
p t
, p
var(η̂
c t)var(ε̂
c t)(14.43)

may be interpreted as t-statistics for impulse intervention variables in thetransition and measurement equations, respectively. Consequently, large
values of (14.43) indicate outliers and/or structural breaks in the local
level model.
Forecasting
To produce out-of-sample h-step ahead forecasts yt+h|t for h = 1, . . . , 5 asequence of 5 missing values is appended to the end of the response vectory.ll> y.ll.new = c(y.ll,rep(NA,5))
The forecast values and mean squared errors are computed using thefunction SsfMomentEst with the argument task="STPRED"> PredictedEst.ll = SsfMomentEst(y.ll.new,ssf.ll,task="STPRED")> y.ll.fcst = PredictedEst.ll$response.moment> fcst.var = PredictedEst.ll$response.variance
The actual values, forecasts and 95% confidence bands are illustrated in
Figure 14.8 created by> upper = y.ll.fcst + 2*sqrt(fcst.var)> lower = y.ll.fcst - 2*sqrt(fcst.var)> upper[1:250] = lower[1:250] = NA
> tsplot(y.ll.new[240:255],y.ll.fcst[240:255],+ upper[240:255],lower[240:255],lty=c(1,2,2,2))

14.4 Estimation of State Space Models
14.4.1 Prediction Error Decomposition of Log-Likelihood
The prediction error decomposition of the log-likelihood function for the
unknown parameters ϕ of a state space model may be conveniently computed using the output of the Kalman filterln L(ϕ|Yn ) =

n
X
t=1

ln f (yt |Yt−1 ; ϕ)

n
¢
nN
1 X¡
= −
ln |Ft | + vt0 F−1ln(2π) −
t vt
2
2 t=1

(14.44)

553

3

4

5

6

7

14.4 Estimation of State Space Models

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

FIGURE 14.8. Actual values, h-step forecasts and 95% confidence intervals foryt from local level model.

where f (yt |Yt−1 ; ϕ) is a conditional Gaussian density implied by the state
space model (14.1) - (14.6). The vector of prediction errors vt and prediction error variance matrices Ft are computed from the Kalman filter
recursions.
A useful diagnostic is the estimated variance of the standardized prediction errors for a given value of ϕ:nσ̂ 2 (ϕ) =1 X 0 −1v F vtN n t=1 t t(14.45)

As mentioned by Koopman, Shephard and Doornik (1999), it is helpful tochoose starting values for ϕ such that σ̂ 2 (ϕstart ) ≈ 1. For well specified
models, σ̂2 (ϕ̂mle ) should be very close to unity.
Concentrated Log-likelihood
In some models, e.g. ARMA models, it is possible to solve explicitly for one
scale factor and concentrate it out of the log-likelihood function (14.44).
The resulting log-likelihood function is called the concentrated log-likelihoodor profile log-likelihood and is denoted ln Lc (ϕ|Yn ). Following Koopman,
Shephard and Doornik (1999), let σ denote such a scale factor, and let
yt = θ t + Gct εct

554

14. State Space Models

with εct ∼iid N (0, σ2 I) denote the scaled version of the measurement equation (14.3). The state space form (14.1) - (14.3) applies but with Gt = σGctand Ht = σHct . This formulation implies that one non-zero element of σGctor σHct is kept fixed, usually at unity, which reduces the dimension of theparameter vector ϕ by one. The solution for σ2 from (14.44) is given bynσ̃ 2 (ϕ) =1 X 0 c −1v (F ) vt
N n t=1 t t

and the resulting concentrated log-likelihood function isln Lc (ϕ|Yn ) = −

n
¢ 1X
nN
nN ¡ 2
ln |Fct | (14.46)ln(2π) −ln σ (ϕ) + 1 −
2
2
2 t=1

14.4.2 Fitting State Space Models Using the
S+FinMetrics/SsfPack Function SsfFit
The S+FinMetrics/SsfPack function SsfFit may be used to compute
MLEs of the unknown parameters in the state space model (14.1)-(14.6)
from the prediction error decomposition of the log-likelihood function (14.44).
The arguments expected by SsfFit are> args(SsfFit)
function(parm, data, FUN, conc = F, scale = 1, gradient =NULL, hessian = NULL, lower = - Inf, upper = Inf,
trace = T, control = NULL, ...)
where parm is a vector containing the starting values of the unknown parameters ϕ, data is a rectangular object containing the response variables
yt , and FUN is a character string giving the name of the function which takes
parm together with the optional arguments in ... and produces an “ssf”
object representing the state space form. The remaining arguments control
aspects of the S-PLUS optimization algorithm nlminb. An advantage of using nlminb is that box constraints may be imposed on the parameters of
the log-likelihood function using the optional arguments lower and upper.
See the online help for nlminb for details. A disadvantage of using nlminb
is that the value of the Hessian evaluated at the MLEs is returned only if
an analytic formula is supplied to compute the Hessian. The use of SsfFit
is illustrated with the following examples.
Example 102 Exact maximum likelihood estimation of AR(1) model
Consider estimating by exact maximum likelihood the AR(1) model discussed earlier. First, n = 250 observations are simulated from the model> ssf.ar1 = GetSsfArma(ar=0.75,sigma=.5)> set.seed(598)14.4 Estimation of State Space Models

555> sim.ssf.ar1 = SsfSim(ssf.ar1,n=250)> y.ar1 = sim.ssf.ar1[,"response"]
Least squares estimation of the AR(1) model, which is equivalent to MLE
conditional on the first observation, gives> OLS(y.ar1~tslag(y.ar1)-1)
Call:
OLS(formula = y.ar1 ~tslag(y.ar1) - 1)
Coefficients:
tslag(y.ar1)
0.7739Degrees of freedom: 249 total; 248 residual
Residual standard error: 0.4921
The S+FinMetrics/SsfPack function SsfFit requires as input a function which takes the unknown parameters ϕ = (φ, σ 2 )0 and produces thestate space form for the AR(1). One such function is
ar1.mod = function(parm) {phi = parm[1]sigma2 = parm[2]ssf.mod = GetSsfArma(ar=phi,sigma=sqrt(sigma2))
CheckSsf(ssf.mod)}
In addition, starting values for ϕ are required. Somewhat arbitrary starting
values are> ar1.start = c(0.5,1)> names(ar1.start) = c("phi","sigma2")
The prediction error decomposition of the log-likelihood function evaluated at the starting values ϕ = (0.5, 1)0 may be computed using the
S+FinMetrics/SsfPack function KalmanFil with task="KFLIK"> KalmanFil(y.ar1,ar1.mod(ar1.start),task="KFLIK")
Call:
KalmanFil(mY = y.ar1, ssf = ar1.mod(ar1.start), task ="KFLIK")
Log-likelihood: -265.5222
Prediction error variance:
Sample observations: 250

0.2851

556

14. State Space Models

Standardized Innovations:
Mean Std.Error-0.0238 0.5345
Notice that the standardized prediction error variance (14.45) is 0.285, far
below unity, which indicates that the starting values are not very good.
The MLEs for ϕ = (φ, σ2 )0 using SsfFit are computed as> ar1.mle = SsfFit(ar1.start,y.ar1,"ar1.mod",+lower=c(-.999,0), upper=c(0.999,Inf))
Iteration 0 : objective = 265.5
Iteration 1 : objective = 282.9...
Iteration 18 : objective = 176.9
RELATIVE FUNCTION CONVERGENCE
In the call to SsfFit, the stationarity condition −1 < φ < 1 and thepositive variance condition σ 2 > 0 is imposed in the estimation. The resultof SsfFit is a an object of class “SsfFit” with components> names(ar1.mle)[1] "parameters" "objective" "message" "grad.norm" "iterations"[6] "f.evals""g.evals""hessian" "scale""aux"[11] "call""vcov"The exact MLEs for ϕ = (φ, σ 2 )0 are> ar1.mle
Log-likelihood: -176.9
250 observations
Parameter Estimates:phisigma20.77081 0.2402688
and the MLE for σ is
> sqrt(ar1.mle$parameters["sigma2"])
sigma20.4902
These values are very close to the least squares estimates. Standard errors
and t-statistics are displayed using> summary(ar1.mle)
Log-likelihood: -176.936
250 observations
Parameters:
Value Std. Error t value
phi 0.7708
0.03977
19.38
sigma2 0.2403
0.02149
11.18

14.4 Estimation of State Space Models

Convergence:

557

RELATIVE FUNCTION CONVERGENCE

A summary of the log-likelihood evaluated at the MLEs is> KalmanFil(y.ar1,ar1.mod(ar1.mle$parameters),+
task = "KFLIK")
Call:
KalmanFil(mY = y.ar1, ssf = ar1.mod(ar1.mle$parameters),
task = "KFLIK")
Log-likelihood: -176.9359
Prediction error variance:
Sample observations: 250

1Standardized Innovations:
Mean Std.Error-0.0213 1.0018
Notice that the estimated variance of the standardized prediction errors isequal to 1.
Example 103 Exact maximum likelihood estimation of AR(1) model using re-parameterization
An alternative function to compute the state space form of the AR(1) is
ar1.mod2 = function(parm) {phi = exp(parm[1])/(1 + exp(parm[1]))sigma2 = exp(parm[2])ssf.mod = GetSsfArma(ar=phi,sigma=sqrt(sigma2))
CheckSsf(ssf.mod)}
In the above model, a stationary value for φ between 0 and 1 and apositive value for σ 2 is guaranteed by parameterizing the log-likelihoodin terms of γ 0 = ln(φ/(1 − φ)) and γ 1 = ln(σ 2 ) instead of φ and σ 2 .
By the invariance property of maximum likelihood estimation, φ̂mle =exp(γ̂ 0,mle )/(1 + exp(γ̂ 0,mle )) and σ̂2mle = exp(γ̂ 1,mle ) are the MLEs for
φ and σ2 . The MLEs for ϕ = (γ 0 , γ 1 )0 computed using SsfFit are> ar1.start = c(0,0)> names(ar1.start) = c("ln(phi/(1-phi))","ln(sigma2)")> ar1.mle = SsfFit(ar1.start,y.ar1,"ar1.mod2")
Iteration 0 : objective = 265.5222...
Iteration 10 : objective = 176.9359

558

14. State Space Models

RELATIVE FUNCTION CONVERGENCE> summary(ar1.mle)
Log-likelihood: -176.936
250 observations
Parameters:
Value Std. Error t valueln(phi/(1-phi)) 1.213
0.225105.388ln(sigma2) -1.426
0.08945 -15.940
Convergence:

RELATIVE FUNCTION CONVERGENCE

The MLEs for φ and σ 2 , and estimated standard errors computed using
the delta method, are5
> ar1.mle2 = ar1.mle> tmp = coef(ar1.mle)> ar1.mle2$parameters[1] = exp(tmp[1])/(1 + exp(tmp[1]))> ar1.mle2$parameters[2] = exp(tmp[2])> dg1 = exp(-tmp[1])/(1 + exp(-tmp[1]))^2> dg2 = exp(tmp[2])> dg = diag(c(dg1,dg2))> ar1.mle2$vcov = dg%*%ar1.mle2$vcov%*%dg> summary(ar1.mle2)
Log-likelihood: -176.936
250 observations
Parameters:
Value Std. Error t valueln(phi/(1-phi)) 0.7708
0.0397719.38
ln(sigma2) 0.2403
0.02149
11.18
Convergence:

RELATIVE FUNCTION CONVERGENCE

and exactly match the previous MLEs.
Example 104 Exact maximum likelihood estimation of AR(1) model using concentrated log-likelihood
In the AR(1) model, the variance parameter σ 2 can be analytically concentrated out of the log-likelihood. The advantages of concentrating the
log-likelihood are to reduce the number of parameters to estimate, and to
improve the numerical stability of the optimization. A function to computethe state space form for the AR(1) model, as a function of φ only, is
ar1.modc = function(parm) {√dd
5 Recall, if √n(θ̂−θ) →N(θ, V ) and g is a continuous function then n(g(θ̂)−g(θ)) →0∂g∂g
N (0, ∂θ
0 V ∂θ 0 ).

14.4 Estimation of State Space Models

559

phi = parm[1]ssf.mod = GetSsfArma(ar=phi)
CheckSsf(ssf.mod)}
By default, the function GetSsfArma sets σ 2 = 1 which is required for thecomputation of the concentrated log-likelihood function from (14.46). To
maximize the concentrated log-likelihood function (14.46) for the AR(1)model, use SsfFit with ar1.modc and set the optional argument conc=T :> ar1.start = c(0.7)> names(ar1.start) = c("phi")> ar1.cmle = SsfFit(ar1.start,y.ar1,"ar1.modc",conc=T,+
lower=0,upper=0.999)
Iteration 0 : objective = 178.506...
Iteration 4 : objective = 176.9359
RELATIVE FUNCTION CONVERGENCE> summary(ar1.cmle)
Log-likelihood: -176.936
250 observations
Parameters:
Value Std. Error t value
phi 0.7708
0.03977
19.38
Convergence:

RELATIVE FUNCTION CONVERGENCE

Notice that with the concentrated log-likelihood, the optimizer converges
in only four interations. The values of the log-likelihood and the MLE for
φ are the same as found previously. The MLE for σ 2 may be recovered byrunning the Kalman filter and computing the variance of the prediction
errors:> ar1.KF = KalmanFil(y.ar1,ar1.mod(ar1.cmle$parameters),+
task="KFLIK")> ar1.KF$dVar[1] 0.2403
One disadvantage of using the concentrated log-likelihood is the lack of an
estimated standard error for σ̂ 2 .
Example 105 Maximum likelihood estimation of CAPM with time varying parameters
Consider estimating the CAPM with time varying coeﬃcients (14.17) (14.19) using monthly data on Microsoft and the S&P 500 index over the period February, 1990 through December, 2000 contained in the S+FinMetrics“timeSeries” excessReturns.ts. The parameters of the model are the560

14. State Space Models

variances of the innovations to the transition and measurement equations;σ 2ξ , σ2ς and σ 2ν . Since these variances must be positive the log-likelihood
is parameterized using ϕ = (ln(σ 2ξ ), ln(σ 2ς ), ln(σ 2ν ))0 . Since the state space
form for the CAPM with time varying coeﬃcients requires a data matrix X
containing the excess returns on the S&P 500 index, the function SsfFit
requires as input a function which takes both ϕ and X and returns theappropriate state space form. One such function is
tvp.mod = function(parm,mX=NULL) {parm = exp(parm)# 3 x 1 vector containing log variances
ssf.tvp = GetSsfReg(mX=mX)
diag(ssf.tvp$mOmega) = parm
CheckSsf(ssf.tvp)}
Starting values for ϕ are specified as> tvp.start = c(0,0,0)> names(tvp.start) = c("ln(s2.alpha)","ln(s2.beta)","ln(s2.y)")The maximum likelihood estimates for ϕ based on SsfFit are computedusing> tvp.mle = SsfFit(tvp.start,msft.ret,"tvp.mod",mX=X.mat)
Iteration 0 : objective = 183.2...
Iteration 22 : objective = -123
RELATIVE FUNCTION CONVERGENCE
The MLEs for ϕ = (ln(σ 2ξ ), ln(σ 2ς ), ln(σ 2ν ))0 are> summary(tvp.mle)
Log-likelihood: 122.979
131 observations
Parameters:
Value Std. Error t valueln(s2.alpha) -12.480
2.8030 -4.452ln(s2.beta) -5.900
3.0890 -1.910
ln(s2.y) -4.817
0.1285 -37.480
Convergence:

RELATIVE FUNCTION CONVERGENCE

The estimates for the standard deviations σξ , σ ς and σν as well as estimatedstandard errors, from the delta method, are:> tvp2.mle = tvp.mle> tvp2.mle$parameters = exp(tvp.mle$parameters/2)> names(tvp2.mle$parameters) = c("s.alpha","s.beta","s.y")> dg = diag(tvp2.mle$parameters/2)

14.4 Estimation of State Space Models

561> tvp2.mle$vcov = dg%*%tvp.mle$vcov%*%dg> summary(tvp2.mle)
Log-likelihood: 122.979
131 observations
Parameters:
Value Std. Error t values.alpha 0.001951
0.002734 0.7135
s.beta 0.052350
0.080860 0.6474
s.y 0.089970
0.005781 15.5600
Convergence:

RELATIVE FUNCTION CONVERGENCE

It appears that the estimated standard deviations for the time varying
parameter CAPM are close to zero, suggesting a constant parameter model.
The smoothed estimates of the time varying parameters αt and β t as
well as the expected returns may be extracted and plotted using> smoothedEst.tvp = SsfCondDens(y.capm,+ tvp.mod(tvp.mle$parameters,mX=X.mat),+ task="STSMO")> plot(smoothedEst.tvp,strip.text=c("alpha(t)",+"beta(t)","Expected returns"),main="Smoothed Estimates",+scales="free")
These estimates are illustrated in Figure 14.9. Notice the increase in β̂ t anddecrease in α̂t over the sample.

14.4.3 Quasi-Maximum Likelihood Estimation
The S+FinMetrics function SsfFitFast is a fast version of SsfFit that
also computes the sandwhich covariance matrix estimate which is appropriate for QMLE. This matrix is given in the r.vcov component of the“SsfFit” object returned by SsfFitFast.
Example 106 Quasi-maximum likelihood estimation of a stochastic volatility model
Let rt denote the continuously compounded return on an asset between
times t − 1 and t. Following Harvey, Ruiz, and Shephard (1994), hereafter
HRS, a simple discrete-time stochastic volatility (SV) model has the form
rt = σ t εt ,εt ∼ iid N (0, 1)
2
ht = ln σ t = γ + φht−1 + ηt ,E[εt ηt ] = 0(14.47)
η t ∼ iid N (0, σ 2η )

562

14. State Space Models

Smoothed Estimates

1.4

alpha(t)0.025

Expected returns

-0.2

0.010

-0.1

0.015

0.0

0.020

0.1

Mean

1.5

1.6

1.7

1.8

beta(t)

0

20

40

60

80

100

120

FIGURE 14.9. Smoothed estimates of αt and β t from CAPM with time varying
parameter fit to the monthly excess returns on Microsoft.

Defining yt = ln rt2 , and noting that E[ln ε2t ] = −1.27 and var(ln ε2t ) = π 2 /2
an unobserved components state space representation for yt has the form
yt = −1.27 + ht + ξ t ,ht = γ + φht−1 + ηt ,E[ξ t ηt ] = 0

ξ t ∼ iid (0, π 2 /2)η t ∼ iid N (0, σ 2η )If ξ t were iid Gaussian then the parameters ϕ = (γ, σ2η , φ)0 of the SVmodel could be eﬃciently estimated by maximizing the prediction error
decomposition of the log-likelihood function constructed from the Kalmanfilter recursions. However, since ξ t = ln ε2t is not normally distributed theKalman filter only provides minimum mean squared error linear estimatorsof the state and future observations. Nonetheless, HRS point out that eventhough the exact log-likelihood cannot be computed from the prediction
error decomposition based on the Kalman filter, consistent estimates of
ϕ = (γ, σ 2η , φ)0 can still be obtained by treating ξ t as though it were iid
N (0, π 2 /2) and maximizing the quasi log-likelihood function constructedfrom the prediction error decomposition.
The state space representation of the SV model has system matrices
µ
¶
µ
¶
µ 2
¶
ση
0
γ
φ
δ=
,
Φ=
,
Ω=
−1.27
1
0 π 2 /2

14.4 Estimation of State Space Models

563

Assuming that |φ| < 1, the initial value matrix has the form
µ 2¶
σ η /(1 − φ2 )Σ=γ/(1 − φ)
If φ = 1 then use
Σ=

µ

−1
0

¶

A function to compute the state space form of the SV model given a
vector of parameters, assuming |φ| < 1, issv.mod = function(parm) {g = parm[1]sigma2.n = exp(parm[2])phi = parm[3]ssf.mod = list(mDelta=c(g,-1.27),
mPhi=as.matrix(c(phi,1)),mOmega=matrix(c(sigma2.n,0,0,0.5*pi^2),2,2),
mSigma=as.matrix(c((sigma2.n/(1-phi^2)),g/(1-phi))))CheckSsf(ssf.mod)}
Notice that an exponential transformation is utilized to ensure a positive
value for σ 2η . A sample of T = 1000 observations are simulated from the
SV model using the parameters γ = −0.3556, σ 2η = 0.0312 and φ = 0.9646:> parm.hrs = c(-0.3556,log(0.0312),0.9646)> nobs = 1000
> set.seed(179)> e = rnorm(nobs)> xi = log(e^2)+1.27> eta = rnorm(nobs,sd=sqrt(0.0312))> sv.sim = SsfSim(sv.mod(parm.hrs),+ mRan=cbind(eta,xi),a1=(-0.3556/(1-0.9646)))
The first 250 simulated squared returns, rt2 , and latent squared volatilities,
σ 2t , are shown in Figure 14.10.
Starting values for the estimation of ϕ = (γ, ln σ 2η , φ)0 are values close to
the true values:> sv.start = c(-0.3,log(0.03),0.9)> names(sv.start) = c("g","ln.sigma2","phi")
Using SsfFitFast, the quasi-maximum likelihood (QML) estimates are> low.vals = c(-Inf,-Inf,-0.999)> up.vals = c(Inf,Inf,0.999)> sv.mle = SsfFitFast(sv.start,sv.sim[,2],"sv.mod",564

14. State Space Models

0.0006

Simulated values from SV model

0.0

0.00020.0004

volatility
squared returns

0

50

100

150

200

250

FIGURE 14.10. Simulated values from SV model.

+
lower=low.vals,upper=up.vals)
Iteration 0 : objective = 5.147579...
Iteration 15 : objective = 2.21826
RELATIVE FUNCTION CONVERGENCE
To show the estimates with the QMLE standard errors use summary withthe optional argument method="qmle"> summary(sv.mle,method="qmle")
Log-likelihood: -2218.26
1000 observations
Parameters:
Value QMLE Std. Error t value
g -0.4810
0.18190 -2.644
ln.sigma2 -3.5630
0.53020 -6.721
phi 0.9509
0.01838 51.740
Convergence:

RELATIVE FUNCTION CONVERGENCE

Using the delta method, the QMLE and estimated standard error for σ2η
are 0.02834 and 0.01503, respectively.
The filtered and smoothed estimates of log-volatility are computed using> ssf.sv = sv.mod(sv.mle2$parameters)

14.5 Simulation Smoothing

565

-11

-10

-9

-8

Log-Volatility

actualfilteredsmoothed

0

50

100

150

200

250

FIGURE 14.11. Log-volatility along with filtered and smoothed estimates from
SV model.

> filteredEst.sv = SsfMomentEst(sv.sim[,2],ssf.sv,task="STFIL")> smoothedEst.sv = SsfCondDens(sv.sim[,2],ssf.sv,task="STSMO")
The first 250 estimates along with actual log-volatility are illustrated in
Figure 14.11

14.5 Simulation Smoothing
The simulation of state and response vectors αt and yt or disturbance
vectors η t and εt conditional on the observations Yn is called simulation smoothing. Simulation smoothing is useful for evaluating the appropriateness of a proposed state space model and for the Bayesian analysis ofstate space models using Markov chain Monte Carlo (MCMC) techniques.
The S+FinMetrics/SsfPack function SimSmoDraw generates random draws
from the distributions of the state and response variables or from the distributions of the state and response disturbances. The arguments expectedby SimSmoDraw are> args(SimSmoDraw)
function(kf, ssf, task = "DSSIM", mRan = NULL, a1 = NULL)

566

14. State Space Models

where kf is a “KalmanFil” object, ssf is a list which either contains theminimal necessary components for a state space form or is a valid “ssf”
object and task determines whether the state smoothing (“STSIM”) or
disturbance smoothing (“DSSIM”) is performed.
Example 107 Simulation smoothing from the local level model
Simulated state and response values from the local level model may begenerated using> KalmanFil.ll = KalmanFil(y.ll,ssf.ll,task="STSIM")> ll.state.sim = SimSmoDraw(KalmanFil.ll,ssf.ll,+ task="STSIM")> class(ll.state.sim)[1] "SimSmoDraw"> names(ll.state.sim)[1] "state""response" "task"
The resulting simulated values may be visualized using> plot(ll.state.sim,layout=c(1,2))
To simulate disturbances from the state and response equations, settask="DSSIM" in the calls to KalmanFil and SimSmoDraw.

14.6 References
Bomhoff, E. J. (1994). Financial Forecasting for Business and Economics.
Academic Press, San Diego.
Carmona, R. (2004). Statistical Analysis of Financial Data, with S-PLUS.
Springer-Verlag, New York.
Chan, N.H. (2002). Time Series: Applicatios to Finance. John Wiley &Sons, New York.
Durbin, J. and S.J. Koopman (2001). Time Series Analysis by StateSpace Methods. Oxford University Press, Oxford.
Duan, J.-C. and J.-G. Simonato (1999). “Estimating Exponential-Aﬃne
Term Structure Models by Kalman Filter,” Review of Quantitative Financeand Accounting, 13, 111-135.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.

14.6 References

567

Harvey, A. C. (1989). Forecasting, Structural Time Series Models andthe Kalman Filter. Cambridge University Press, Cambridge.
Harvey, A.C. (1993). Time Series Models, 2nd edition. MIT Press, Cambridge.
Harvey, A.C., E. Ruiz and N. Shephard (1994). “Multivariate Stochastic Variance Models,” Review of Economic Studies, 61, 247-264.
Kim, C.-J., and C.R. Nelson (1999). State-Space Models with Regime
Switching. MIT Press, Cambridge.
Koopman, S.J., N. Shephard, and J.A. Doornik (1999). “Statistical Algorithms for State Space Models Using SsfPack 2.2,” Econometrics
Journal, 2, 113-166.
Koopman, S.J., N. Shephard, and J.A. Doornik (2001). “SsfPack
3.0beta: Statistical Algorithms for Models in State Space,” unpublishedmanuscript, Free University, Amsterdam.
Engle, R.F. and M.W. Watson (1987). “The Kalman Filter: Applications to Forecasting and Rational Expectations Models,” in T.F. Bewley(ed.) Advances in Econometrics: Fifth World Congress, Volume I. Cambridge University Press, Cambridge.
West, M. and J. Harrison (1997). Bayesian Forecasting and Dynamic
Models, 2nd edition. Springer-Verlag, New York.
Zivot, E., Wang, J. and S.J. Koopman (2004). “State Space Models
in Economics and Finance Using SsfPack in S+FinMetrics,” in A. Harvey,
S.J. Koopman, and N. Shephard (eds.), Unobserved Components Models.
Cambridge University Press, Cambridge.

This is page 568
Printer: Opaque thisThis is page 569
Printer: Opaque this15
Factor Models for Asset Returns

15.1 Introduction
Multifactor models can be used to predict returns, generate estimates ofabnormal return, and estimate the variability and covariability of returns.
This chapter focuses on the use of multifactor models to describe the covariance structure of returns1 . Asset return covariance matrices are key
inputs to portfolio optimization routines used for asset allocation and active asset management. A factor model decomposes an asset’s return into
factors common to all assets and an asset specific factor. Often the common factors are interpreted as capturing fundamental risk components, andthe factor model isolates an asset’s sensitivities to these risk factors. The
three main types of multifactor models for asset returns are: (1) macroeconomic factor models; (2) fundamental factor models; and (3) statisticalfactor models. Macroeconomic factor models use observable economic timeseries like interest rates and inflation as measures of pervasive or common
factors in asset returns. Fundamental factor models use observable firm or
asset specific attributes such as firm size, dividend yield, and industry classification to determine common factors in asset returns. Statistical factor
models treat the common factors as unobservable or latent factors. Estimation of multifactor models is type-specific, and this chapter summarizes
1 A recent review of factor models for this purpose is given in Chan, Karceski and
Lakonishok (1998).

570

15. Factor Models for Asset Returns

the econometric issues associated with estimating each type of factor modeland gives illustrations using S-PLUS.
This chapter is organized as follows. Section two presents the generalfactor model specification. Section three describes the macroeconomic factor model. Examples using Sharpe’s single index model as well as a generalmacroeconomic model are given. Section four surveys the fundamental factor model and provides illustrations of an industry factor model and a
Fama-French type model. Statistical factor models estimated using factor
analysis and principal components analysis are covered in Section five. Particular emphasis is given to techniques appropriate for the case in which
the number of assets is greater than the number of time periods.
Connor (1995) gives an overview of three types of factor models for asset
returns and compares their explanatory power. Campbell, Lo, and MacKinlay (1997) and Grinold and Kahn (2000) survey the econometric specification of these models. Johnson and Wichern (1998) provides an excellent
treatment of statistical factor models. Good textbook discussions of statistical factor models with applications in finance are given in Alexander(2001) and Tsay (2001).

15.2 Factor Model Specification
Each of the three types of multifactor models for asset returns has thegeneral form
Rit

= αi + β 1i f1t + β 2i f2t + · · · + β Ki fKt + εit= αi + β 0i ft + εit(15.1)

where Rit is the return (real or in excess of the risk-free rate) on asset i(i = 1, . . . , N ) in time period t (t = 1, . . . , T ), αi is the intercept, fkt is
the k th common factor (k = 1, . . . , K), β ki is the factor loading or factor
beta for asset i on the k th factor, and εit is the asset specific factor. In themultifactor model, it is assumed that the factor realizations, ft , are I(0)with unconditional moments
E[ft ] = μf
cov(ft ) = E[(ft −μf )(f t −μf )0 ] = Ωfand that the asset specific error terms, εit , are uncorrelated with each of
the common factors, fkt , so thatcov(fkt , εit ) = 0, for all k, i and t.
It is also assumed that the error terms εit are serially uncorrelated andcontemporaneously uncorrelated across assetscov(εit , εjs ) = σ 2i for all i = j and t = s= 0, otherwise

15.3 Macroeconomic Factor Models for Returns

571

In applications, it is often the case that the number of assets, N, is substantially larger than the number of time periods, T. In what follows asubscript t represents time and a subscript i represents asset so that Rt
represents an (N × 1) vector of assets at time t and Ri represents a (T × 1)
vector of returns on asset i.
The multifactor model (15.1) may be rewritten as a cross-sectional regression model at time t by stacking the equations for each asset to giveRt
(N ×1)

=

α + B

(N×1)

ft + εt ,(N ×K)(K×1) (N×1)

t = 1, . . . , T(15.2)

E[εt ε0t |ft ] = D
where B is the (N × K) matrix of factor betas, ft is the (K × 1) vector offactor realizations for time period t, and εt is the (N × 1) vector of asset
specific error terms with (N × N ) diagonal covariance matrix D. Given theassumption of the multifactor model, the (N × N ) covariance matrix ofasset returns has the formcov(Rt ) = Ω = BΩf B0 +D
The multifactor model (15.1) may also be rewritten as a time-series
regression model for asset i by stacking observations for a given asset i togive
Ri
(T ×1)=

1T αi + F
βi + εi ,(T ×K)(K×1)(T ×1)(1×1)(T ×1)

i = 1, . . . , N (15.3)

E[εi ε0i ] = σ 2i IT
where 1T is a (T × 1) vector of ones, F is a (T × K) matrix of factor
realizations (the tth row of F is ft0 ), βi is a (K ×1) vector of factor loadings,
and εi is a (T × 1) vector of error terms with covariance matrix σ2i IT .
Finally, collecting data from t = 1, . . . , T allows the model (15.2) to be
expressed in matrix form as the multivariate regression
R(N ×T )=

α(N ×1)+B

F(N ×K)(K×T )+E(N×T )(15.4)

15.3 Macroeconomic Factor Models for Returns
In a macroeconomic factor model, the factor realizations ft in (15.1) are
observed macroeconomic variables that are assumed to be uncorrelatedwith the asset specific error terms εit . The two most common macroeconomic factor models are Sharpe’s (1970) single factor model and Chen, Rolland Ross’s (1986) multifactor model. Once the macroeconomic factors are
specified and constructed the econometric problem is then to estimate thefactor betas, β ki , residual variances, σ2i , and factor covariance, Ωf , using
time series regression techniques.

572

15. Factor Models for Asset Returns

15.3.1 Sharpe’s Single Index Model
The most famous macroeconomic factor model is Sharpe’s single factor
model or market model
Rit = αi + β i RM t + εit , i = 1, . . . , N ; t = 1, . . . , T(15.5)

where RMt denotes the return or excess return (relative to the risk-freerate) on a market index (typically a value weighted index like the S&P 500
index) in time period t. The market index is meant to capture economywide or market risk, and the error term captures non-market firm specificrisk. The multifactor model (15.1) reduces to (15.5) if f1t = RMt , β ik =0 (i = 1, . . . , N ; k = 2, . . . , K). The covariance matrix of assets from thesingle factor model isΩ = σ 2M ββ0 + D(15.6)
where σ 2M = var(RMt ), β = (β 1 , . . . , β N )0 and D is a diagonal matrix with
σ 2i = var(εit ) along the diagonal.
Because RMt is observable, the parameters β i and σ 2i of the single factor
model (15.5) for each asset can be estimated using time series regression
giving
Ri
σ
b2i

b +b
= α
b i 1 + RM βεi , i = 1, . . . , N
i
1
0
b
=
εi
εb
T −2 i

The variance of the market index is estimated using the time series sample
variance
σ
b2M

RM

=

T
1 X
(RMt − RM )2
T − 1 t=1
T

=

1X
RMt
T t=1

The estimated single factor model covariance is thenbβ
b 0 + D,
b =σ
b
Ω
b2M β

b has σwhere Db2i along the diagonal.
Remarks

1. Computational eﬃciency may be obtained by using multivariate regression2 . The coeﬃcients αi and β i and the residual variances σ 2i
2 Since R
M is the regressor for each asset, multivariate OLS estimates are numericallyequivalent to multivariate GLS estimates that take into account the across equation
correlation between the errors in each equation.

15.3 Macroeconomic Factor Models for Returns

573

may be computed in one step in the multivariate regression model
RT = XΓ0 + ET
where RT is a (T × N ) matrix of asset returns, X = [1 : RM ] is a
(T × 2) matrix, Γ0 = [α : β]0 is a (2 × N ) matrix of coeﬃcients and
ET is a (T × N ) matrix of errors. The multivariate OLS estimator of
Γ0 isb 0 = (X0 X)−1 X0 RT .
Γ
The estimate of the residual covariance matrix is

0

b =
Σ

1 b0 b
E ET
T −2 T

where ÊT = RT − XΓ̂ is the multivariate least squares residualb are the diagonal elements of D.
bmatrix. The diagonal elements of Σ

2. The R2 from the time series regression is a measure of the proportion of “market” risk, and 1 − R2 is a measure of asset specific risk.
Additionally, σbi is a measure of the typical size of asset specific risk.

3. Robust regression techniques can be used to estimate β i and σ2i . Also,a robust estimate of σ 2M could be computed.

4. In practice, the estimated value of β i is often adjusted toward unity.
Adjusted β i values are discussed in chapter seven of Elton and Gruber(1997).
5. The single factor covariance matrix (15.6) is constant over time. This
may not be a good assumption. There are several ways to allow (15.6)
to vary over time. For example, assume that β i is constant and that σ 2iand σ 2M are conditionally time varying. That is, σ2i = σ 2it and σ 2M =σ2M t . To capture conditional heteroskedasticity, GARCH models may
be used for σ2it and σ2M t . One may also use exponential weights incomputing the sample variances of σ2i and σ 2M . Alternatively, one
may assume that β i is not constant over time.
Example 108 Estimation of Sharpe’s single factor model using S-PLUS
The single factor model parameters and the return covariance matrix(15.6) may be eﬃciently estimated using the matrix functions in S-PLUS.
To illustrate, consider estimating the single factor model using the monthly
return data over the period January 1978 through December 1987 in the“timeSeries” berndt.dat. The variables in berndt.dat are> colIds(berndt.dat)[1] "CITCRP" "CONED" "CONTIL" "DATGEN" "DEC""DELTA"[7] "GENMIL" "GERBER" "IBM""MARKET" "MOBIL" "PANAM"[13] "PSNH""TANDY" "TEXACO" "WEYER" "RKFREE"

574

15. Factor Models for Asset Returns

See the online help for berndt.dat for a description of these variables. The
return data on the assets and the market index are extracted using:> returns = as.matrix(seriesData(berndt.dat[, c(-10, -17)]))> market = as.vector(seriesData(berndt.dat)[,10])The single factor model parameters may be estimated by multivariate regression using> n.obs = nrow(returns)> X.mat = cbind(rep(1,n.obs),market)> G.hat = solve(X.mat,returns)> beta.hat = G.hat[2,]> E.hat = returns - X.mat%*%G.hat> diagD.hat = diag(crossprod(E.hat)/(n.obs-2))> names(diagD.hat) = colIds(G.hat)> r.sq = 1-(n.obs-2)*diagD.hat/diag(var(returns,SumSquares=T))
The second row of G.hat contains the estimated β i values, and the vector
diagD.hat contains the estimated residual variances σ 2i :> t(rbind(beta.hat,sqrt(diagD.hat),r.sq))
beta.hatr.square
CITCRP 0.667776 0.067163 0.317769
CONED 0.091021 0.050096 0.015316
CONTIL 0.738357 0.142597 0.112158
DATGEN 1.028160 0.106880 0.303631
DEC 0.843053 0.081018 0.337829
DELTA 0.489461 0.090289 0.121627
GENMIL 0.267765 0.062676 0.079188
GERBER 0.624807 0.076966 0.236938
IBM 0.453024 0.050461 0.275235
MOBIL 0.713515 0.064072 0.368818
PANAM 0.730140 0.122507 0.143372
PSNH 0.212632 0.108961 0.017627
TANDY 1.055494 0.105649 0.319860
TEXACO 0.613277 0.068076 0.276615
WEYER 0.816867 0.064448 0.430829
The β i and R2 values are illustrated graphically in Figure 15.1. The assets
most sensitive to the market factor (those with highest β i values) are thetechnology and forest sector stocks DATGEN, DEC, TANDY and WEYER. Those
least sensitive are the utility stocks CONED and PSNH. These stocks also havethe lowest R2 values.
The single factor covariance matrix (15.6) and corresponding correlation
matrix are computed using> cov.si = var(market)*(beta.hat%o%beta.hat) +15.3 Macroeconomic Factor Models for Returns
Beta values

R-square values

WEYER

WEYER

TEXACOTEXACO

TANDY

TANDY

PSNH

PSNH

PANAM

PANAM

MOBIL

MOBIL

IBM

IBM

GERBERGERBER

GENMIL

GENMIL

DELTA

DELTA

DEC

DEC

DATGENDATGENCONTILCONTIL

CONED

CONED

CITCRPCITCRP

0.0

0.2

0.4

0.6

575

0.8

1.0

0.0

0.1

0.2

0.3

0.4

FIGURE 15.1. Estimated β i and R2 values from single index model for Berndt
data.

+ diag(diagD.hat)> sd = sqrt(diag(cov.si))> cor.si = cov.si/outer(sd,sd)
Since all estimated β i values are positive, all of the values in the single
factor covariance (15.6) will be positive. To illustrate, some of the single
factor correlations are displayed below> print(cor.si,digits=1,width=2)
CITCRP CONED CONTIL DATGEN DEC DELTA GENMIL GERBERCITCRP
1.00 0.07
0.19
0.31 0.33 0.20
0.16
0.27
CONED
0.07 1.00
0.04
0.07 0.07 0.04
0.03
0.06
CONTIL0.19 0.04
1.00
0.18 0.19 0.12
0.09
0.16
DATGEN
0.31 0.07
0.18
1.00 0.32 0.19
0.15
0.27
DEC
0.33 0.07
0.19
0.32 1.00 0.20
0.16
0.28
DELTA
0.20 0.04
0.12
0.19 0.20 1.00
0.10
0.17
GENMIL0.16 0.03
0.09
0.15 0.16 0.10
1.00
0.14
GERBER
0.27 0.06
0.16
0.27 0.28 0.17
0.14
1.00
IBM
0.29 0.06
0.17
0.29 0.30 0.18
0.15
0.25
MOBIL
0.34 0.07
0.20
0.33 0.35 0.21
0.17
0.29
PANAM
0.21 0.05
0.13
0.21 0.22 0.13
0.11
0.18
PSNH
0.07 0.02
0.04
0.07 0.08 0.05
0.04
0.06
TANDY
0.32 0.07
0.19
0.31 0.33 0.20
0.16
0.27

576

TEXACO
WEYER...

15. Factor Models for Asset Returns

0.29
0.37

0.06
0.08

0.17
0.22

0.29 0.30
0.36 0.38

0.18
0.23

0.15
0.18

0.25
0.32

These correlations may be compared with the sample correlations> print(cor(returns),digits=1,width=2)
CITCRP CONED CONTIL DATGEN DEC DELTA GENMIL
CITCRP
1.0 0.269
0.5
0.53 0.49 0.40 0.473
CONED
0.3 1.000
0.1
0.10 0.11 0.09 0.329
CONTIL0.5 0.105
1.0
0.26 0.23 0.17 0.206
DATGEN
0.5 0.096
0.3
1.00 0.58 0.33 0.280
DEC
0.5 0.108
0.2
0.58 1.00 0.43 0.212
DELTA
0.4 0.092
0.2
0.33 0.43 1.00 0.373
GENMIL0.5 0.329
0.2
0.28 0.21 0.37 1.000
GERBER0.4 0.171
0.4
0.14 0.16 0.19 0.350
IBM
0.4 0.091
0.3
0.49 0.44 0.34 0.170
MOBIL
0.3 0.003
0.2
0.32 0.41 0.13 0.047
PANAM
0.3 0.163
0.1
0.29 0.27 0.39 0.207
PSNH
0.1 0.112
0.1
0.08 0.04 0.03 0.059
TANDY
0.5 0.102
0.2
0.51 0.49 0.46 0.400
TEXACO
0.3 -0.106
0.2
0.32 0.25 0.13 0.002
WEYER
0.5 0.158
0.2
0.48 0.59 0.49 0.357...
Another way to compare the single index covariance matrix to the sample
covariance is to compute the global minimum variance portfolio. The global
minimum variance portfolio is the portfolio w that solves
min σ2p,w = w0 Ωw s.t. w0 1 = 1w

and is given byΩ−1 1
10 Ω−1 1
The global minimum variance portfolios based on the single index covariance and the sample covariance arew=> w.gmin.si = solve(cov.si)%*%rep(1,nrow(cov.si))> w.gmin.si = w.gmin.si/sum(w.gmin.si)> t(w.gmin.si)
numeric matrix: 1 rows, 15 columns.
CITCRP CONEDCONTILDATGENDEC DELTA[1,] 0.04379 0.3757 0.005229 -0.02348 -0.004413 0.0525
GENMIL GERBER
IBM
MOBIL
PANAM
PSNH
[1,] 0.1819 0.04272 0.1866 0.03372 0.007792 0.06618

15.3 Macroeconomic Factor Models for Returns

577

TANDY TEXACO
WEYER[1,] -0.02719 0.05782 0.001173> w.gmin.sample = solve(var(returns))%*%rep(1,nrow(cov.si))> w.gmin.sample = w.gmin.sample/sum(w.gmin.sample)> t(w.gmin.sample)
numeric matrix: 1 rows, 15 columns.
CITCRP CONEDCONTILDATGENDEC
DELTA[1,] -0.06035 0.3763 -0.002152 -0.06558 0.03626 0.03155
GENMILGERBER
IBM
MOBIL
PANAM
PSNH
[1,] 0.1977 -0.02966 0.2846 0.02257 0.01071 0.07517
TANDY TEXACO
WEYER[1,] -0.01868 0.1996 -0.05804

15.3.2 The General Multifactor Model
The general macroeconomic multifactor model specifies K observable macrovariables as the factor realizations ft in (15.1). The paper by Chen, Rolland Ross (1986) provides a description of the most commonly used macroeconomic factors. Typically, the macroeconomic factors are standardized tohave mean zero and a common scale. The factors must also be transformed
to be stationary (not trending). Sometimes the factors are made orthogonal
but this in not necessary.
The general form of the covariance matrix for the macroeconomic factor
model isΩ = BΩf B0 + D
where B = [β1 , β 2 , · · · , βN ]0 , Ωf = E[(ft − μf )(ft − μf )0 ] is the covariancematrix of the observed factors and D is a diagonal matrix with σ 2i =var(εit )along the diagonal.
Because the factor realizations are observable, the parameter matrices B
and D of the model may be estimated using time series regression givingb +b= αb i 1 + Fβεi , i = 1, . . . , N
i
1
b
σ
b2i =
εi
ε0 b
T −K −1 i
The covariance matrix of the factor realizations may be estimated using
the time series sample covariance matrix
Ri

bf
Ω

=

f

=

T
1 X
(ft − f )(ft − f )0
T − 1 t=1
T

1X
ft
T t=1

578

15. Factor Models for Asset Returns

The estimated multifactor model covariance matrix is thenRemarks

b =B
bΩ
bfB
b
b0 + DΩ(15.7)

1. As with the single factor model, robust regression may be used tocompute βi and σ 2i . A robust covariance matrix estimator may also
be used to compute and estimate of Ωf .
Example 109 Estimating a general macroeconomic factor model using
S-PLUSAs explained in Chen, Roll and Ross (1986), the macroeconomic factors
should be constructed as surprise variables so that the returns on assets will
respond to unexpected changes in economy-wide factors. To illustrate theconstruction of a macroeconomic factor model with macroeconomic surprise variables, consider a simple factor model for the monthly returns in
the “timeSeries” returns, constructed earlier, using as factors the surprise to inflation and the surprise to industrial production growth. Monthly
observations on inflation and industrial production growth are constructedfrom the S+FinMetrics “timeSeries” CPI.dat and IP.dat as follows> infl = getReturns(CPI.dat)> ipg = getReturns(IP.dat)
In general, to compute surprise variables, one must first explain the expected behavior and then define the surprise to be the diﬀerence between
what is observed and what is expected. A common way to compute the expected behavior is to use a VAR model. For simplicity, consider a VAR(6)model for inflation and industrial production growth fit over the period
July 1977 through December 1987
> factor.ts = seriesMerge(ipg,infl)> var6.fit = VAR(cbind(CPI,IP)~ar(6),data=factor.ts,> start="July 1977",end="Jan 1988",in.format="%m %Y")
The start date of July 1977 allows for six initial values so that the first
fitted value is for January 1978. The factor surprises are constructed as theresiduals from the VAR(6) fit:
> factor.surprise = residuals(var6.fit)
The factor betas and R2 values for the fifteen assets in the “timeSeries”
returns are computed using> factor.surprise = as.matrix(seriesData(factor.surprise))> n.obs = nrow(returns)> X.mat = cbind(rep(1,n.obs),factor.surprise)15.3 Macroeconomic Factor Models for Returns
Beta values for inflation surprise

Beta values for IP growth surprise579

R-square values

WEYER
TEXACO
TANDY
PSNH
PANAM
MOBIL
IBM
GERBER
GENMILDELTA
DEC
DATGENCONTILCONEDCITCRP

-10

-5

0

5

-2

-1

0

1

0.0

0.01

0.02

0.03

0.04

FIGURE 15.2. Estimated macroeconomic factor model for Berndt data.

> G.hat = solve(X.mat,returns)> beta.hat = t(G.hat[2:3,])> E.hat = returns - X.mat%*%G.hat> diagD.hat = diag(crossprod(E.hat)/(n.obs-3))> names(diagD.hat) = colIds(G.hat)> r.sq = 1-(n.obs-3)*diagD.hat/diag(var(returns,SumSquares=T))
These results are illustrated graphically in Figure 15.2 created by> par(mfrow=c(1,3))> barplot(beta.hat[,1],names=names(beta.hat),horiz=T,+ main="Beta values for inflation surprise")> barplot(beta.hat[,2],names=names(beta.hat),horiz=T,+ main="Beta values for IP growth surprise")> barplot(r.sq,names=names(r.sq),horiz=T,+ main="R-square values")
Most stocks have negative loadings on the inflation surprise factor. Notice
the very low R2 values indicating that the factor surprises do not explain
much of the variability in the monthly asset returns.
The estimated factor model covariance using (15.7) is> cov.macro = beta.hat%*%var(factor.surprise)%*%t(beta.hat) ++ diag(diagD.hat)

580

15. Factor Models for Asset Returns

and the corresponding correlation matrix is> sd = sqrt(diag(cov.macro))> cor.macro = cov.macro/outer(sd,sd)> print(cor.macro,digits=1,width=2)
CITCRPCONED CONTIL DATGENDEC
DELTA GENMIL GERBERCITCRP 1.000 0.0181 0.0035 -0.010 -0.008 -0.0019 0.017 0.0115
CONED 0.018 1.0000 -0.0056 -0.007 0.002 0.0214 0.030 0.0300
CONTIL 0.004 -0.0056 1.0000 -0.005 -0.008 -0.0134 -0.002 -0.0062
DATGEN -0.010 -0.0069 -0.0052 1.000 0.009 0.0081 -0.008 -0.0030
DEC -0.008 0.0017 -0.0083 0.009 1.000 0.0164 -0.002 0.0042
DELTA -0.002 0.0214 -0.0134 0.008 0.016 1.0000 0.011 0.0200
GENMIL 0.017 0.0301 -0.0017 -0.008 -0.002 0.0114 1.000 0.0218
GERBER 0.011 0.0300 -0.0062 -0.003 0.004 0.0200 0.022 1.0000
IBM 0.007 0.0208 -0.0049 -0.001 0.004 0.0150 0.015 0.0164
MOBIL -0.002 -0.0128 0.0053 -0.002 -0.006 -0.0137 -0.008 -0.0109
PANAM 0.019 0.0195 0.0061 -0.013 -0.012 -0.0066 0.019 0.0115
PSNH 0.003 0.0033 0.0005 -0.002 -0.001 -0.0001 0.003 0.0021
TANDY 0.007 0.0335 -0.0121 0.003 0.013 0.0325 0.022 0.0280
TEXACO 0.003 0.0002 0.0027 -0.003 -0.004 -0.0051 0.001 -0.0008
WEYER 0.007 0.0183 -0.0042 -0.001 0.003 0.0131 0.013 0.014...
The macroeconomic factor model global minimum variance portfolio is> w.gmin.macro = solve(cov.macro)%*%rep(1,nrow(cov.macro))> w.gmin.macro = w.gmin.macro/sum(w.gmin.macro)> t(w.gmin.macro)numeric matrix: 1 rows, 15 columns.
CITCRP CONED CONTIL DATGENDEC
DELTA GENMIL[1,] 0.06958 0.1776 0.02309 0.03196 0.04976 0.04766 0.1049
GERBER
IBM
MOBIL
PANAM
PSNH
TANDY
[1,] 0.05463 0.1318 0.08186 0.02469 0.04019 0.02282
TEXACO
WEYER[1,] 0.07759 0.06185

15.4 Fundamental Factor Model
Fundamental factor models use observable asset specific characteristics(fundamentals) like industry classification, market capitalization, style classification (value, growth) etc. to determine the common risk factors. In
practice, fundamental factor models are estimated in two ways. The firstway was pioneered by Bar Rosenberg, founder of BARRA Inc., and is dis-

15.4 Fundamental Factor Model

581

cussed at length in Grinold and Kahn (2000). In this approach, hereafterreferred to as the “BARRA” approach, the observable asset specific fundamentals (or some transformation of them) are treated as the factor betas,
β i , which are time invariant3 . The factor realizations at time t, ft , however,are unobservable. The econometric problem is then to estimate the factor
realizations at time t given the factor betas. This is done by running T
cross-section regressions. The second way was introduced by Eugene Fama
and Kenneth French (1992) and is referred to as the “Fama-French” approach. For a given observed asset specific characteristic, e.g. size, they
determined factor realizations using a two step process. First they sorted
the cross-section of assets based on the values of the asset specific characteristic. Then they formed a hedge portfolio which is long in the top quintileof the sorted assets and short in the bottom quintile of the sorted assets.
The observed return on this hedge portfolio at time t is the observed factor
realization for the asset specific characteristic. This process is repeated foreach asset specific characteristic. Then, given the observed factor realizations for t = 1, . . . , T the factor betas for each asset are estimated using N
time series regressions.

15.4.1 BARRA-type Single Factor Model
Consider a single factor model in the form of a cross-sectional regression
at time t
Rt = β ft + εt , t = 1, . . . , T(N ×1)(N ×1)(1×1)(N ×1)

where β is a vector of observed values of an asset specific attribute (e.g.,market capitalization, industry classification, style classification) and ft is
an unobserved factor realization. It is assumed thatvar(ft ) = σ 2f
cov(ft , εit ) = 0, for all i, tvar(εit ) = σ 2i , i = 1, . . . , N.
In the above model the factor realization ft is the parameter to be estimated for each time period t = 1, . . . , T . Since the error term εt is heteroskedastic, eﬃcient estimation of ft is done by weighted least squares(WLS) (assuming the asset specific variances σ 2i are known)
fˆt,wls = (β0 D−1 β)−1 β 0 D−1 Rt , t = 1, . . . , T(15.8)

where D is a diagonal matrix with σ 2i along the diagonal. The above WLS
estimate of ft is infeasible since σ 2i is not known. However, σ2i may be
3 See Sheikh (1995) for a description of the BARRA fundamental factor model for
U.S. equities.

582

15. Factor Models for Asset Returns

consistently estimated and a feasible WLS estimate may be computed.
How σ 2i may be consistently estimated and how a feasible WLS estimate
may be computed is illustrated below.
The WLS estimate of ft in (15.8) has an interesting interpretation as thereturn on a portfolio h = (h1 , . . . , hN )0 that solves
min
h

1 0
h Dh subject to h0 β = 1
2The portfolio h minimizes asset return residual variance subject to having
unit exposure to the attribute β and is given by
h0 = (β0 D−1 β)−1 β0 D−1
The estimated factor realization is then the portfolio returnfˆt,wls = h0 Rt
When the portfolio h is normalized such thatas a factor mimicking portfolio.

PNihi = 1, it is referred to

15.4.2 BARRA-type Industry Factor ModelAs an example of a fundamental factor model with K factors, consider
a stylized BARRA-type industry factor model with K mutually exclusive
industries. The factor sensitivities β ik in (15.1) for each asset are timeinvariant and of the form
β ik

= 1 if asset i is in industry k= 0, otherwise

and fkt represents the factor realization for the k th industry in time periodt. Notice that factor betas are simply dummy variables indicating whether
a given asset is in a particular industry. Hence, the industry factor betas do
not have to be estimated from the data. The factor realizations, however,are not initially observable. As will become apparent, the estimated valueof fkt will be equal to the weighted average excess return in time period tof the firms operating in industry k. This weighted average excess returnat time t can be easily estimated using a cross-section regression over all
asset returns at time t.
The industry factor model with K industries is summarized as
Rit = β i1 f1t + · · · + β iK fKt + εit , i = 1, . . . , N ; t = 1, . . . , Tvar(εit ) = σ 2i , i = 1, . . . , N
cov(εit , fjt ) = 0, j = 1, . . . , K; i = 1, . . . , N
cov(fit , fjt ) = σ fij , i, j = 1, . . . , K

15.4 Fundamental Factor Model

583

where β ik = 1 if asset i is in industry k (k = 1, . . . , K) and is zero oth4
erwisePK . It is assumed that there are Nk firms in the kth industry such
k=1 Nk = N .
Least Squares Estimation of the Factor Realizations

The factor realizations f1t , . . . , fKt for t = 1, . . . , T , can be estimated fromthe observed cross-section of asset returns at time period t as follows. Consider the cross-section regression at time t= β1 f1t + · · · + βK fKt + εt ,= Bf t + εt
0
E[εt εt ] = D, cov(ft ) = Ωf
Rt

(15.9)

where Rt is an (N × 1) vector of returns, B = [β 1 , . . . , βK ] is a (N ×K) matrix of zeros and ones reflecting the industry factor sensitivities foreach asset, ft = (f1t , . . . , fKt )0 is a (K × 1) vector of unobserved factor
realizations, εt is an (N × 1) error term, and D is a diagonal matrix with
σ 2i along the diagonal. Note that the error term is heteroskedastic acrossassets. Since the industries are mutually exclusive it follows thatβ0j βk = Nk for j = k, 0 otherwise(15.10)An unbiased but ineﬃcient estimate of the factor realizations ft can beobtained by OLS giving

or

−1
b
ft,OLS = (B0 B) B0 Rt⎞ ⎛fb1t,OLS
⎟ ⎜
⎜
..
⎠=⎜
⎝
.
⎝
b
fKt,OLS
⎛

1
N1

1
NK

PN1

1
i=1 Rit

..
.

PNK

K
i=1 Rit(15.11)⎞⎟⎟⎠kusing (15.10) where Rit
denotes the return on asset i if it is in industry
k. Here, the estimated factor realizations fbkt have nice interpretations.
They represent an equally weighted average return in time period t on theindustry k assets. Of course, this is expected given the nature of the binary
industry factor beta values.
To get the time series of factor realizations, the cross-section regression(15.9) needs to be estimated for each t = 1, . . . , T giving the estimatedfactor realizations (bf1,OLS , . . . , b
fT,OLS ).
4 Notice that there is no intercept in the industry factor model. With K mutually
exclusive industries, the intercept will be collinear with the factor betas and not identifiable.

584

15. Factor Models for Asset Returns

Estimation of Factor Realization Covariance Matrix
Given the time series of factor realizations, the covariance matrix of theindustry factors may be computed as the time series sample covariance
bF
Ω
OLS

=

f OLS

=

T
1 Xb
(ft,OLS − f OLS )(bft,OLS − f OLS )0 ,T − 1 t=1

(15.12)T

1 Xb
ft,OLS
T t=1

Estimation of Residual Variances

The residual variances, var(εit ) = σ 2i , can be estimated from the timeseries of residuals from the T cross-section regressions given in (15.9) as
follows. Let bεt,OLS , t = 1, . . . , T , denote the (N ×1) vector of OLS residuals
from (15.9), and let bεit,OLS denote the ith row of bεt,OLS . Then σ 2i may be
estimated usingσb2i,OLS
εi,OLS=T
1 X(bεit,OLS − εi,OLS )2 , i = 1, . . . , N
T − 1 t=1

(15.13)T=

1X
bεit,OLS
T t=1

Estimation of Industry Factor Model Asset Return Covariance Matrix
The covariance matrix of the N assets is then estimated by0b OLS = BΩbFbΩOLS B + DOLSb OLS is a diagonal matrix with σwhere D
b2i,OLS along the diagonal.
Remarks

1. Multivariate regression may be used to compute all of the factor returns in one step. The multivariate regression model is
R = BF + E,
where R is a (N × T ) matrix of cross-sectionally demeaned asset
returns, F is a (K × T ) matrix of parameters to be estimated (factor
returns) and E is a (N × T ) matrix of errors such that E[EE0 ] = D.
2. Robust regression techniques can be used to estimate ft , and a robust
covariance matrix estimate of Ωf can be computed.
3. The industry factor model may be extended to cover cases where an
asset may be classified into several industry categories.

15.4 Fundamental Factor Model585

4. Given the estimated factor realizations, a time series regression may
be run to assess the constructed model. The estimated factor loading
may be compared to the imposed values and the proportion of asset
variance attributable to all of the factors may be computed.
Weighted Least Squares Estimation
The OLS estimation of the factor realizations ft is ineﬃcient due to thecross-sectional heteroskedasticity in the asset returns. The estimates of theresidual variances from (15.13) may be used as weights for weighted leastsquares (feasible GLS) estimation:bb −1 B)−1 B0 D
b −1 (Rt − Rt 1), t = 1, . . . , T
ft,GLS = (B0 D
OLS
OLS(15.14)

fT,GLS ), the covariGiven the time series of factor realizations, (bf1,GLS , . . . , bance matrix of the industry factors may be computed as the time series
sample covariance
bF
Ω
GLS

=

f GLS

=

T
1 Xb
(ft,GLS − f GLS )(bft,GLS − f GLS )0 ,T − 1 t=1

(15.15)T

1 Xb
ft,GLS
T t=1

The residual variances, var(εit ) = σ 2i , can be re-estimated from the timeseries of residuals from the T cross-section GLS regressions as follows. Letbεt,GLS , t = 1, . . . , T , denote the (N × 1) vector of GLS residuals from theindustry factor model (15.9) and let bεit,GLS denote the ith row of bεt,GLS .
2
Then σ i may be estimated usingσb2i,GLS
εi,GLS=T
1 X(bεit,GLS − εi,GLS )2 , i = 1, . . . , N
T − 1 t=1

(15.16)T=

1X
bεit,GLS
T t=1

The covariance matrix of the N assets is then estimated byb GLS = BΩb F B0 + D
b GLS
Ω
GLS

b GLS is a diagonal matrix with σwhere Db2i,GLS along the diagonal.
Remarksb OLS are time invariant, (B0 D
b −1 B)−1 B0 D
b −1 only
1. Since B and D
OLS
OLS
needs to be computed once, and this greatly speeds up the computation of b
ft,GLS (t = 1, . . . , T ).

586

15. Factor Models for Asset Returns

2. In principle, the GLS estimator may be iterated. Iteration does not
improve the asymptotic eﬃciency of the estimator, and it may perform better or worse than the non-iterated estimator.
3. Weighted robust regression techniques can be used to estimate ft , anda robust covariance matrix estimate of Ωf can be computed.
Factor Mimicking Portfolios
The GLS estimates of the factor realizations (15.14) are just linear combinations of the observed returns in each industry. Further, these linear combinations sum to unity so that they can be interpreted as factor mimicking
portfolios. Notice that they are simply weighted averages of the returns ineach industry where the weights on each asset are based on the size of theresidual variance. The (N ×1) vector of weights for the ith factor mimicking
portfolio is given by³´
b −1 B)−1 B0 D
b −1
wi = Hi = (B0 D, i = 1, . . . , K
OLS
OLS,
i

where Hi denotes the ith row of H.

Seemingly Unrelated Regression Formulation of Industry Factor Model
The industry factor model may be expressed as a seemingly unrelated regression (SUR) model. The cross section regression models (15.9) can bestacked to form the giant regression
⎛
⎞ ⎛
⎞⎛
⎞ ⎛
⎞
R1
B 0 0
f1
ε1
⎜ .. ⎟ ⎜⎟⎜ . ⎟ ⎜ . ⎟⎝ . ⎠ = ⎝ 0 . . . 0 ⎠ ⎝ .. ⎠ + ⎝ .. ⎠ .
RT

0

0

B

fT

εT

The giant regression may be compactly expressed using Kronecker productsasvec(R) = (IT ⊗ B)f + εE[εε0 ] = IT ⊗ D
where vec(R) is a (N T × 1) vector of returns, f is a (T K × 1) vector offactor realizations, and ε is a (N T × 1) vector of errors. The GLS estimatorof f is
h
i−1
−1
−1
b
(IT ⊗ B)0 (IT ⊗ D) R
fGLS = (IT ⊗ B)0 (IT ⊗ D) (IT ⊗ B)£¤
= IT ⊗ (B0 D−1 B)−1 B0 D−1 R

or

⎞ ⎛
⎞
⎛ b
(B0 D−1 B)−1 B0 D−1 R1
f1,GLS
⎟ ⎜
⎟
⎜
..
..
⎠=⎝
⎠
⎝
.
.
0 −1
−1 0 −1
b
(B
D
B)
B
D
R
fT,GLS
T

15.4 Fundamental Factor Model

587

which is just weighted least squares on each of the cross section regressions(15.9). Hence, equation by equation GLS estimation of (15.9) is eﬃcient.
Of course, the above GLS estimator is not feasible because it requires
knowledge of the firm specific variances in D. However, using the techniques
described above to estimate σ 2i , feasible GLS estimation is possible.
Example 110 Estimating an industry factor model using S-PLUS
Consider creating a three industry factor model for the fifteen assets
taken from the S+FinMetrics “timeSeries” berndt.dat. The three industries are defined to be “technology”, “oil” and “other”. The 15 × 3
matrix B of industry factor loadings are created using> n.stocks = numCols(returns)> tech.dum = oil.dum = other.dum = matrix(0,n.stocks,1)> tech.dum[c(4,5,9,13),] = 1> oil.dum[c(3,6,10,11,14),] = 1> other.dum = 1 - tech.dum - oil.dum> B = cbind(tech.dum,oil.dum,other.dum)> dimnames(B) = list(colIds(returns),c("TECH","OIL","OTHER"))> B
integer matrix: 15 rows, 3 columns.
TECH OIL OTHER
CITCRP
0
0
1
CONED
0
0
1
CONTIL
0
1
0
DATGEN
1
0
0
DEC
1
0
0
DELTA
0
1
0
GENMIL0
0
1
GERBER
0
0
1
IBM
1
0
0
MOBIL
0
1
0
PANAM
0
1
0
PSNH
0
0
1
TANDY
1
0
0
TEXACO
0
1
0
WEYER
0
0
1
The multivariate least squares estimates of the factor realizations are> returns = t(returns)> F.hat = solve(crossprod(B))%*%t(B)%*%returns
The multivariate GLS estimates are computed using> E.hat = returns - B%*%F.hat> diagD.hat = rowVars(E.hat)

588

15. Factor Models for Asset Returns> Dinv.hat = diag(diagD.hat^(-1))> H = solve(t(B)%*%Dinv.hat%*%B)%*%t(B)%*%Dinv.hat> F.hat = H%*%returns> F.hat = t(F.hat)
The rows of the matrix H contain the weights for the factor mimicking
portfolios:> t(H)
numeric matrix: 15 rows, 3 columns.
TECH
OIL
OTHER
[1,] 0.0000 0.00000 0.19918[2,] 0.0000 0.00000 0.22024[3,] 0.0000 0.09611 0.00000[4,] 0.2197 0.00000 0.00000[5,] 0.3188 0.00000 0.00000[6,] 0.0000 0.22326 0.00000[7,] 0.0000 0.00000 0.22967[8,] 0.0000 0.00000 0.12697[9,] 0.2810 0.00000 0.00000[10,] 0.0000 0.28645 0.00000[11,] 0.0000 0.11857 0.00000[12,] 0.0000 0.00000 0.06683[13,] 0.1806 0.00000 0.00000[14,] 0.0000 0.27561 0.00000[15,] 0.0000 0.00000 0.15711
Notice that the weights sum to unity> rowSums(H)TECH OIL OTHER
1
1
1
The factor realizations are illustrated in Figure 15.3.
The industry factor model covariance and correlation matrices are computed using> cov.ind = B%*%var(F.hat)%*%t(B) + diag(diagD.hat)> sd = sqrt(diag(cov.ind))> cor.ind = cov.ind/outer(sd,sd)> print(cor.ind,digits=1,width=2)
CITCRP CONED CONTIL DATGEN DEC DELTA GENMIL GERBERCITCRP
1.0
0.4
0.10
0.2 0.2
0.1
0.3
0.2
CONED
0.4
1.0
0.14
0.2 0.3
0.2
0.4
0.3
CONTIL
0.1
0.1
1.00
0.1 0.1
0.2
0.1
0.1
DATGEN0.2
0.2
0.12
1.0 0.3
0.2
0.2
0.1
DEC
0.2
0.3
0.14
0.3 1.0
0.2
0.2
0.2
DELTA
0.1
0.2
0.20
0.2 0.2
1.0
0.2
0.1

-0.1

0.0

0.1

0.2

15.4 Fundamental Factor Model

-0.2

TECH
OIL
OTHER

0

20

40

60

80

100

120

FIGURE 15.3. Estimated industry factor realizations from Berndt data.

GENMILGERBER
IBM
MOBIL
PANAM
PSNH
TANDY
TEXACO
WEYER

0.3
0.2
0.2
0.2
0.1
0.2
0.2
0.2
0.3

0.4
0.3
0.3
0.2
0.2
0.3
0.2
0.2
0.3

0.12
0.10
0.18
0.22
0.15
0.08
0.12
0.22
0.10

0.2 0.2
0.1 0.2
0.4 0.5
0.2 0.2
0.1 0.2
0.1 0.1
0.3 0.3
0.2 0.2
0.2 0.2

0.2
0.1
0.3
0.3
0.2
0.1
0.2
0.3
0.1

1.0
0.3
0.3
0.2
0.1
0.2
0.2
0.2
0.3

0.3
1.0
0.2
0.2
0.1
0.2
0.1
0.2
0.2

The industry factor model global minimum variance portfolio is> w.gmin.ind = solve(cov.ind)%*%rep(1,nrow(cov.ind))> w.gmin.ind = w.gmin.ind/sum(w.gmin.ind)> t(w.gmin.ind)
numeric matrix: 1 rows, 15 columns.
CITCRP CONED CONTIL
DATGENDEC
DELTA GENMIL[1,] 0.0905 0.2409 0.02232 0.006256 0.01039 0.05656 0.1416
GERBER
IBM
MOBIL
PANAM
PSNH
TANDY
[1,] 0.07775 0.02931 0.07861 0.02972 0.04878 0.006455
TEXACO
WEYER[1,] 0.0794 0.08149

589

590

15. Factor Models for Asset Returns

15.5 Statistical Factor Models for Returns
In statistical factor models, the factor realizations ft in (15.1) are not directly observable and must be extracted from the observable returns Rt
using statistical methods. The primary methods are factor analysis andprincipal components analysis. Traditional factor analysis and principal
component analysis are usually applied to extract the factor realizations
if the number of time series observations, T , is greater than the numberof assets, N . If N > T , then the sample covariance matrix of returns becomes singular which complicates traditional factor and principal components analysis. In this case, the method of asymptotic principal component
analysis due to Connor and Korajczyk (1988) is more appropriate.
Traditional factor and principal component analysis is based on the (N ×N ) sample covariance matrix5
b N = 1 RR0Ω
T

where R is the (N × T ) matrix of observed returns. Asymptotic principal
component analysis is based on the (T × T ) covariance matrix

15.5.1 Factor Analysisb T = 1 R0 R.
Ω
N

Traditional factor analysis assumes a time invariant orthogonal factor structure 6
Rt(N ×1)=

cov(ft , εs )E[ft ]
var(ft )var(εt )

=
=
=
=

μ + B
(N×1)

ft + εt(N ×K)(K×1) (N×1)(15.17)0, for all t, sE[εt ] = 0
IKD

where D is a diagonal matrix with σ 2i along the diagonal. Then, the returncovariance matrix, Ω, may be decomposed asΩ = BB0 + D
5 The matrix of returns is assumed to be in deviations about the mean form. In some
applications, a mean correction is not used because the means are small.
6 An excellent overview of factor analysis is given in Johnson and Wichern (1998).
Factor analysis using S-PLUS is described in the S-PLUS 6 Guide to Statistics Vol. 2,
chapter 21.

15.5 Statistical Factor Models for Returns

591

Hence, the K common factors ft account for all of the cross covariances ofasset returns.
For a given asset i, the return variance variance may be expressed asvar(Rit ) =KX

β 2ij + σ 2i

j=1

P
2
The variance portion due to the common factors, K
j=1 β ij , is called thecommunality, and the variance portion due to specific factors, σ2i , is called
the uniqueness.
The orthogonal factor model (15.17) does not uniquely identify the common factors ft and factor loadings B since for any orthogonal matrix H
such that H0 = H−1
Rt= μ + BHH0 ft + εt
= μ + B∗ ft∗ + εtwhere B∗ = BH, ft∗ = H0 ft and var(ft∗ ) = IK . Because the factors and factor
loadings are only identified up to an orthogonal transformation (rotationof coordinates), the interpretation of the factors may not be apparent until
suitable rotation is chosen.
Estimation
Estimation using factor analysis consists of three steps:• Estimation of the factor loading matrix B and the residual covariancematrix D.
• Construction of the factor realizations ft .
• Rotation of coordinate system to enhance interpretation
Traditional factor analysis provides maximum likelihood estimates of B
and D under the assumption that returns are jointly normally distributed
b and D,b an empirical version of theand temporally iid. Given estimates B
factor model (15.2) may be constructed asb t +bb = BfRt − μεt(15.18)bb 0D
b −1 B)b −1 Bb 0D
b −1 (R − μb)ft = (Bt(15.19)b is the sample mean vector of Rt . The error terms in (15.18) are
where μheteroskedastic so that OLS estimation is ineﬃcient. Using (15.18), thefactor realizations in a given time period t, ft , can be estimated using thecross-sectional generalized least squares (GLS) regression

592

15. Factor Models for Asset Returns

Performing this regression for t = 1, . . . , T times gives the time series offactor realizations (bf1 , . . . , bfT ).
The factor model estimated covariance matrix is given by

Remarks:

bF = BbB
b 0 +DbΩ√• Traditional factor analysis starts with a T - consistent and asymptotically normal estimator of Ω, usually the sample covariance matrix
b and makes inference on K based on Ω.
b A likelihood ratio test isΩ,
often used to select K under the assumption that εit is normally distributed (see below). However, when N → ∞ consistent estimationof Ω, an N × N matrix, is not a well defined problem. Hence, if N
is large relative to T , then traditional factor analysis may run into
problems. Additionally, typical algorithms for factor analysis are not
eﬃcient for very large problems.
• Traditional factor analysis is only appropriate if εit is cross-sectionallyuncorrelated, serially uncorrelated, and serially homoskedastic.
Factor Mimicking Portfolios
From (15.19), we see that the estimated factor realizations b
ft are simply
linear combinations of the observed returns Rt . As such, it is possible tonormalize the linear combination so that the weights sum to unity. The
resulting re-scaled factors are the factor mimicking portfolios and are perfectly correlated with the factor realizations.
Tests for the Number of Factors
Using the maximum likelihood estimates of B and D based on a K−factor
b a likelihood ratio test (modified
model and the sample covariance matrix Ω,for improved small sample performance) of the adequacy of K factors is of
the form
´
³
1
2
b .
b − ln |BbB
b 0 +D|
LR(K) = −(T − 1 − (2N + 5) − K) · ln |Ω|
63¢¡
LR(K) is asymptotically chi-square with 12 (N − K)2 − N − K degrees
of freedom.
Example 111 Estimating a statistical factor model by factor analysis using S-PLUS
Factor analysis in S-PLUS is performed using the function factanal,
which performs estimation of B and D using either the principal factor
method or the maximum likelihood method, and it takes as input either raw

15.5 Statistical Factor Models for Returns

593

data or an estimated covariance or correlation matrix. A robust versionof factor analysis can be computed if the inputted covariance matrix is arobust covariance matrix (MCD, MVE or M-estimate). If the maximum
likelihood method is used, then the LR test for the adequacy of the Kfactor model is computed.
A factor model with k = 2 factors for the fifteen returns from berndt.dat
computed using maximum likelihood method is> factor.fit = factanal(returns,factors=2,method="mle")> class(factor.fit)[1] "factanal"> factor.fit
Sums of squares of loadings:
Factor1 Factor2
3.319
2.471
The number of variables is 15 and the number of observations
is 120
Test of the hypothesis that 2 factors are sufficientversus the alternative that more are required:
The chi square statistic is 118.25 on 76 degrees of freedom.
The p-value is 0.00138
Component names:"loadings" "uniquenesses" "correlation" "criteria""factors" "dof" "method" "center" "scale" "n.obs" "scores""call"
Call:factanal(x = returns, factors = 2, method = "mle")
The likelihood ratio test for determining the number of factors indicatesthat two factors is not enough to adequately explain the sample returncovariance. A factor model with k = 3 factor appears to be adequate> factor.fit = factanal(returns,factors=3,method="mle")> factor.fit
Sums of squares of loadings:
Factor1 Factor2 Factor3
3.137
1.765
1.719

594

15. Factor Models for Asset Returns

The number of variables is 15 and the number of observations
is 120
Test of the hypothesis that 3 factors are sufficientversus the alternative that more are required:
The chi square statistic is 71.6 on 63 degrees of freedom.
The p-value is 0.214...
A summary of the three factor model is> summary(factor.fit)
Importance of factors:
Factor1 Factor2 Factor3
SS loadings 3.1370 1.7651 1.7185
Proportion Var 0.2091 0.1177 0.1146
Cumulative Var 0.2091 0.3268 0.4414
The degrees of freedom for the model is 63.
Uniquenesses:
CITCRP CONED CONTIL DATGENDEC DELTA GENMIL GERBER
0.3125 0.8506 0.7052 0.4863 0.3794 0.6257 0.592 0.5175
IBM MOBIL PANAM
PSNH TANDY TEXACO WEYER
0.6463 0.2161 0.7643 0.9628 0.5442 0.3584 0.4182
Loadings:
Factor1 Factor2 Factor3
CITCRP 0.518
0.217
0.610
CONED 0.116
0.356
CONTIL 0.173
0.195
0.476
DATGEN 0.668
0.206
0.160
DEC 0.749
0.236
DELTA 0.563
0.239
GENMIL 0.306
0.556
GERBER
0.346
0.600
IBM 0.515
0.224
0.197
MOBIL 0.257
0.847
PANAM 0.427
0.219
PSNH 0.115
0.133
TANDY 0.614
0.264
TEXACO 0.140
0.787
WEYER 0.694
0.200
0.247

15.5 Statistical Factor Models for Returns

595

0.0

0.4

Factor1

DEC

WEYER

DATGEN

TANDY

DELTA

CITCRPDECIBM

CITCRPCONTIL

CONED

TANDY

0.0

0.4

0.8

Factor2

MOBIL

TEXACOGERBER

0.0

0.2

0.4

0.6

Factor3

CITCRPGERBER

GENMILFIGURE 15.4. Estimated loadings from three factor model fit to Berndt data byfactor analysis.

The three factors explain about forty four percent of the total variance ofreturns. The reported uniqueness for each asset is standardized such thatthe sum of the uniqueness and the communality is unity. Therefore, assets
with uniqueness values close to zero are well explained by the factor model.
The factor loadings may be extracted using the generic loadings function. The extracted loadings have class “loadings” and may be visualizedwith plot> plot(loadings(factor.fit))
Figure 15.4 gives the resulting plot. The first factor appears to be marketwide factor, and the second factor is concentrated on oil stocks. Since thefactors are only defined up to an orthogonal rotation, the factor may berotated to aid interpretation. The generic function rotate performs suchrotation. For example, to rotate the factor using the quartimax rotationand view the rotated loadings use> factor.fit2 = rotate(factor.fit,rotation="quartimax")> loadings(factor.fit2)
Factor1 Factor2 Factor3
CITCRP 0.722
0.108
0.393
CONED 0.233 -0.153
0.268
CONTIL 0.351
0.113
0.398

596

DATGENDEC
DELTA
GENMILGERBER
IBM
MOBIL
PANAM
PSNH
TANDY
TEXACO
WEYER

15. Factor Models for Asset Returns

0.693
0.734
0.610
0.485
0.299
0.566
0.307
0.472
0.150
0.673
0.205
0.748

0.168
0.212

-0.193-0.164
0.243
0.181
0.829
-0.112-0.101

0.382
0.578

0.767
0.148

0.103

See the online help for rotate for a description of the supported rotation
methods.
The factor realizations (15.19) may be computed using the generic predict
function:> factor.ret = predict(factor.fit,type="weighted.ls")
The estimated factor model correlation matrix may be extracted using> fitted(factor.fit)
numeric matrix: 15 rows, 15 columns.
CITCRPCONED CONTIL DATGEN
DEC
DELTA
CITCRP 1.0000 0.25701 0.42245 0.48833 0.47656 0.43723
CONED 0.2570 1.00000 0.17131 0.11533 0.08655 0.15095
CONTIL 0.4224 0.17131 1.00000 0.23205 0.20474 0.21099
DATGEN 0.4883 0.11533 0.23205 1.00000 0.55841 0.41386
DEC 0.4766 0.08655 0.20474 0.55841 1.00000 0.43597
DELTA 0.4372 0.15095 0.21099 0.41386 0.43597 1.00000
GENMIL 0.4825 0.24043 0.30402 0.27919 0.24681 0.30570
GERBER 0.4693 0.18728 0.36267 0.20404 0.15936 0.17371
IBM 0.4354 0.10893 0.22663 0.42116 0.45030 0.33649
MOBIL 0.3278 -0.04399 0.21832 0.34861 0.39336 0.14722
PANAM 0.3396 0.13446 0.16438 0.30585 0.31682 0.29320
PSNH 0.1237 0.06825 0.06786 0.08206 0.07588 0.09689
TANDY 0.5000 0.15650 0.25080 0.47188 0.49859 0.40868
TEXACO 0.2712 -0.04208 0.19960 0.26285 0.29353 0.08824
WEYER 0.5531 0.14958 0.27646 0.54368 0.58181 0.44913...
To obtain the estimated factor model covariance matrix, the estimatedloadings and uniqueness values must be re-scaled. One way to do this is> S.hat = diag(factor.fit$scale)> D.hat = S.hat%*%diag(factor.fit$uniqueness)%*%S.hat> D.hat.inv = diag(1/diag(D.hat))15.5 Statistical Factor Models for Returns

597> B.hat = S.hat%*%loadings(factor.fit)> cov.factor = B.hat%*%t(B.hat)+D.hat> dimnames(cov.fa) = list(colIds(returns),colIds(returns))
The factor analysis global minimum variance portfolio is then
> w.gmin.fa = solve(cov.fa)%*%rep(1,nrow(cov.fa))> w.gmin.fa = w.gmin.fa/sum(w.gmin.fa)> t(w.gmin.fa)
numeric matrix: 1 rows, 15 columns.
CITCRP CONEDCONTILDATGENDEC
DELTA[1,] -0.0791 0.3985 -0.02537 -0.04279 -0.002584 0.04107
GENMIL GERBERIBM MOBIL
PANAM
PSNH
TANDY
[1,] 0.1889 0.01321 0.2171 0.1027 0.01757 0.07533 -0.03255
TEXACO
WEYER[1,] 0.1188 0.009147

15.5.2 Principal Components
Principal component analysis (PCA) is a dimension reduction techniqueused to explain the majority of the information in the sample covariancematrix of returns. With N assets there are N principal components, andthese principal components are just linear combinations of the returns. The
principal components are constructed and ordered so that the first principal
component explains the largest portion of the sample covariance matrix ofreturns, the second principal component explains the next largest portion,and so on. The principal components are constructed to be orthogonal
to each other and to be normalized to have unit length. In terms of amultifactor model, the K most important principal components are thefactor realizations. The factor loadings on these observed factors can thenbe estimated using regression techniques.
b denote the sample covariance matrix of returns. The first sample
Let Ω∗
principal component is x∗0
1 Rt where the (N × 1) vector x1 solvesb 1 s.t. x0 x1 = 1.
max x01 Ωx
1x1

The solution x∗1 is the eigenvector associated with the largest eigenvalue ofb The second principal component is x∗0 Rt where the (N × 1) vector x∗Ω.
2
2
solvesb 2 s.t. x02 x2 = 1 and x∗0
max x02 Ωx
1 x2 = 0x2

The solution x∗2 is the eigenvector associated with the second largest eigenb This process is repeated until K principal components arevalue of Ω.
computed.

598

15. Factor Models for Asset Returns

The estimated factor realizations are simply the first K principal components
fbkt = x∗0 Rt , k = 1, . . . , K.
(15.20)k

The factor loadings for each asset, βi , and the residual variances, var(εit ) =σ 2i can be estimated via OLS7 from the time series regression
ft + εit , t = 1, . . . , T
Rit = αi + β0ib

(15.21)b and σb2i for i = 1, . . . , N . The factor model covariance matrix ofgiving βireturns is then
b =B
bΩ
bFB
b
b 0 +DΩ(15.22)
where

and

⎞
⎛ 2
b0
σ
b1
β
1
⎜ . ⎟
⎜
b
b
⎟
⎜
B = ⎝ .. ⎠ , D = ⎝ 0
⎛

b0
β
N

bF
Ω

=

f

=

0

0
..
.
···

0

⎞

⎟
0 ⎠,
σ
b2N

T
1 Xb
(ft − f )(b
f t − f )0 ,T − 1 t=1
T

1 Xb
ft .
T t=1

b F = IK because the principal components are orthonormal.
Usually Ω
Factor Mimicking Portfolios

Since the principal components (factors) x∗i are just linear combinationsof the returns, it is possible to construct portfolios that are perfectly correlated with the principal components by re-normalizing the weights inthe x∗i vectors so that they sum to unity. Hence, the weights in the factor
mimicking portfolios have the form
wi =

µ

1
10 x∗i¶· x∗i , i = 1, . . . , K(15.23)where 1 is a (N × 1) vector of ones.
7 OLS estimation is eﬃcient even though assets are contemporaneously correlated
because the time series regression for each asset has the same regressors.

15.5 Statistical Factor Models for Returns

599

Variance Decomposition
It can be shown thatkX
i=1

var(Rit ) =kXvar(fit ) =

i=1

k
X

λi

i=1

where λi are the ordered eigenvalues of var(Ri ) = Ω. Therefore, the ratioλi
PN

i=1 λiPgives the proportion of the total variance N
i=1 var(Rit ) attributed to theith principal component factor return, and the ratioPK
i=1 λiPN
i=1 λigives the cumulative variance explained. Examination of these ratios help in
determining the number of factors to use to explain the covariance structureof returns.
Example 112 Estimating a statistical factor model by principal components using S-PLUS
Principal component analysis in S-PLUS is performed using the function
princomp. The S+FinMetrics function mfactor simplifies the process of
estimating a statistical factor model for asset returns using principal components. To illustrate, consider estimating a statistical factor model for theassets in the S+FinMetrics “timeSeries” berndt.dat excluding market
portfolio and the thirty-day T-bill> returns.ts = berndt.dat[,c(-10,-17)]
To estimate a statistical factor model with the default of one factor use> pc.mfactor = mfactor(returns.ts)> class(pc.mfactor)[1] "mfactor"
The result of the function mfactor is an object of class “mfactor”, forwhich there are print and plot methods and extractor functions factors,
loadings, residuals and vcov. The components of an “mfactor” object
are> names(pc.mfactor)[1] "factors""loadings"[4] "alpha""Omega"[7] "eigen""call""k""r2""sum.loadings"

600

15. Factor Models for Asset Returns

where factors contains the estimated factor returns (15.20), loadings
contains the asset specific factor loadings β̂ i estimated from (15.21), alphacontains the estimated intercepts αi from (15.21), r2 contains the regression
R2 values from (15.21), k is the number of factors and eigen contains the
eigenvalues from the sample covariance matrix.
The print method gives a brief summary of the PCA fit> pc.mfactorCall:mfactor(x = returns.ts)
Factor Model:
Factors Variables Periods
1
15120
Factor Loadings:
Min. 1st Qu. Median Mean 3rd Qu. Max.
F.1 0.0444
0.139
0.25 0.231
0.308 0.417
Regression R-squared:
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.032 0.223
0.329 0.344 0.516
0.604
Notice that all of the estimated loadings on the first factor are positive,
and the median R2 is around thirty percent. These results are very similar
to those found for the single index model. The factor loadings and factor
regression R2 values may be extracted using the functions loadings andmfactor.r2> loadings(pc.mfactor)
numeric matrix: 1 rows, 15 columns.
CITCRPCONED CONTIL DATGENDEC DELTA GENMIL
F.1 0.2727 0.04441 0.3769 0.4172 0.3049 0.2502 0.1326
GERBER
IBM MOBIL PANAM
PSNH TANDY TEXACO
F.1 0.1672 0.1464 0.1552 0.3107 0.08407 0.4119 0.1323
WEYER
F.1 0.2649> mfactor.r2(pc.mfactor)
CITCRP CONED CONTIL DATGENDEC DELTA GENMIL GERBER
0.6041 0.1563 0.3285 0.5633 0.516 0.3665 0.2662 0.2181
IBM MOBIL PANAM
PSNH TANDY TEXACO WEYER
0.3408 0.2277 0.2922 0.03241 0.5643 0.1635 0.5153

15.5 Statistical Factor Models for Returns

601

The factor returns (15.20) and the residuals from the regression (15.21) may
be extracted using the functions factors and residuals, respectively.
The function vcov extracts the PCA covariance matrix (15.22). The
corresponding correlation matrix may computed using> cov.pca = vcov(pc.mfactor)> sd = sqrt(diag(cov.pca))> cor.pca = cov.pca/outer(sd,sd)> print(cor.pca,digits=1,width=2)
CITCRP CONED CONTIL DATGEN DEC DELTA GENMIL GERBERCITCRP
1.0 0.16
0.4
0.6 0.5
0.5
0.36
0.34
CONED
0.2 1.00
0.1
0.2 0.1
0.1
0.09
0.09
CONTIL
0.4 0.12
1.0
0.4 0.4
0.3
0.27
0.25
DATGEN
0.6 0.15
0.4
1.0 0.5
0.4
0.35
0.33
DEC
0.5 0.14
0.4
0.5 1.0
0.4
0.33
0.31
DELTA
0.5 0.12
0.3
0.4 0.4
1.0
0.28
0.26
GENMIL
0.4 0.09
0.3
0.3 0.3
0.3
1.00
0.20
GERBER0.3 0.09
0.2
0.3 0.3
0.3
0.20
1.00
IBM
0.4 0.11
0.3
0.4 0.4
0.3
0.26
0.25
MOBIL
0.3 0.09
0.3
0.3 0.3
0.3
0.21
0.19
PANAM
0.4 0.11
0.3
0.4 0.4
0.3
0.25
0.23
PSNH
0.1 0.04
0.1
0.1 0.1
0.1
0.08
0.08
TANDY
0.6 0.15
0.4
0.6 0.5
0.4
0.34
0.32
TEXACO0.3 0.08
0.2
0.3 0.3
0.2
0.18
0.16
WEYER
0.5 0.14
0.4
0.5 0.5
0.4
0.33
0.31
The PCA global minimum variance portfolio is> w.gmin.pca = solve(cov.pca)%*%rep(1,nrow(cov.pca))> w.gmin.pca = w.gmin.pca/sum(w.gmin.pca)> t(w.gmin.pca)
numeric matrix: 1 rows, 15 columns.
CITCRP CONED CONTIL
DATGENDEC DELTA GENMIL[1,] 0.02236 0.3675 -0.021 -0.06549 -0.01173 0.0239 0.1722
GERBER
IBM
MOBIL
PANAM
PSNH
TANDY
[1,] 0.07121 0.2202 0.09399 -0.006415 0.06427 -0.06079[1,]

TEXACO
WEYER
0.105 0.02472

The plot method allows a graphical investigation of the PCA fit> plot(pc.mfactor)
Make a plot selection (or 0 to exit):

602

15. Factor Models for Asset Returns

Screeplot

0.03
0.02

0.489
0.598
0.672

0.737

0.792

0.837

0.878

0.909

0.0

0.01

Variances

0.04

0.05

0.354

F.1

F.2

F.3

F.4

F.5

F.6

F.7

F.8

F.9

FIGURE 15.5. Screeplot of eigenvalues from PCA of Berndt returns.

1: plot: All
2: plot: Screeplot of Eigenvalues
3: plot: Factor Returns
Selection:
The Screeplot of Eigenvalues is illustrated in Figure 15.5. The first principal component explains about thirty five percent of the total variance,
and the first two components explain about half of the total variance. It
appears that two or three factors may be suﬃcient to explain most of thevariability of the assets. The screeplot may also be computed directly using
the S+FinMetrics function screeplot.mfactor.
The PCA factor model is re-estimated using two factors with
> pc2.mfactor = mfactor(returns.ts,k=2)> pc2.mfactorCall:mfactor(x = returns.ts, k = 2)
Factor Model:
Factors Variables Periods
2
15
120
Factor Loadings:

15.5 Statistical Factor Models for Returns

603

Factor Returns

-0.5

0.0

0.5

F.1

-0.5

0.0

F.2

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

FIGURE 15.6. Estimated factors from PCA of Berndt data.

Min. 1st Qu. Median
Mean 3rd Qu. Max.
F.1 0.0444 0.1395 0.2502 0.23143
0.308 0.417
F.2 -0.8236 -0.0671 0.0124 -0.00245
0.142 0.365
Regression R-squared:
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.033 0.253
0.435 0.419 0.577
0.925
The first factor is the same as before and has all positive loadings. The
second factor has both positive and negative loadings. The median regression R2 has increased to about forty four percent. The factor returns areillustrated in Figure 15.6, created by selecting option 3 from the plot menu.
The factor return plot may also be computed directly by first extracting
the factors and then using the S+FinMetrics function fplot:> fplot(factors(pc2.mfactor))
The factor loadings are shown in Figure 15.7, created by> pc2.betas = loadings(pc2.mfactor)> par(mfrow=c(1,2))> barplot(pc2.betas[1,],names=colIds(pc2.betas),horiz=T,+ main="Beta values for first PCA factor")> barplot(pc2.betas[2,],names=colIds(pc2.betas),horiz=T,+ main="Beta values for second PCA factor")

604

15. Factor Models for Asset Returns
Beta values for first PCA factor

Beta values for second PCA factor

WEYER

WEYER

TEXACOTEXACO

TANDY

TANDY

PSNH

PSNH

PANAM

PANAM

MOBIL

MOBIL

IBM

IBM

GERBERGERBER

GENMIL

GENMIL

DELTA

DELTA

DEC

DEC

DATGENDATGENCONTILCONTIL

CONED

CONED

CITCRPCITCRP

0.0

0.1

0.2

0.3

0.4

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

FIGURE 15.7. Estimated loadings on PCA factors for Berndt data.

The factor mimicking portfolios (15.23) may be computed using the
S+FinMetrics function mimic> pc2.mimic = mimic(pc2.mfactor)> class(pc2.mimic)[1] "mimic"> pc2.mimic
F.1
F.2
CITCRP 0.07856
2.3217
CONED 0.01279 -0.0324...
WEYER 0.07630 -3.5637
attr(, "class"):[1] "mimic"
These weights in these portfolios may be summarized using> pc2.mimic.sum = summary(pc2.mimic,n.top=3)> pc2.mimic.sum
Factor 1
Top.Long.Name Top.Long.Weight Top.Short.Name Top.Short.Weight
1
DATGEN
12%
CONED
1.3%
2
TANDY
12%
PSNH
2.4%

15.5 Statistical Factor Models for Returns
Top Positions in Factor 1 (Percentages)

Top Positions in Factor 2 (Percentages)

CONED

PANAM

PSNH

TANDY

TEXACODELTACONTILCITCRPTANDYGERBERDATGENCONTIL

0

2

4

6

8

10

605

12

-1000

-500

0

500

1000

1500

2000

FIGURE 15.8. Weights in factor mimicking portfolios from PCA fit to Berndt
data.

3

CONTIL

11%

TEXACO

3.8%

Factor 2
Top.Long.Name Top.Long.Weight Top.Short.Name Top.Short.Weight
1
CONTIL
2200%
PANAM
-990%
2
GERBER
540%
TANDY
-560%
3
CITCRP
230%
DELTA
-490%
The optional argument n.top=3 specifies that the three assets with thelargest and smallest weights in each factor mimicking portfolio are displayed. For the first factor, the assets DATGEN, TANDY and CONTIL have thehighest weights and the assets CONED, PSNH and TEXACO have the lowest
weights. Examination of the weights helps to interpret the factor mimicking portfolio. For the first portfolio, the weights are all positive (longpositions in all assets) and are roughly equal suggesting the interpretationof a market-wide factor. The second factor has both positive and negative
weights (long and short positions in the assets), and it is not clear how tointerpret the weights. The weights may also be examined graphically using> par(mfrow=c(1,2))> plot(pc2.mimic.sum)
which produces the plot in Figure 15.8.

606

15. Factor Models for Asset Returns

15.5.3 Asymptotic Principal Components
Asymptotic principal component analysis (APCA), proposed and developedin Conner and Korajczyk (1986) and based on the analysis in Chamberlain
and Rothschild (1983), is similar to traditional PCA except that it relies
on asymptotic results as the number of cross-sections N (assets) growsbT.
large. APCA is based on eigenvector analysis of the T × T matrix ΩConner and Korajczyk proved that as N grows large, eigenvector analysis
b T is asymptotically equivalent to traditional factor analysis. That is,of Ω
bT.
the APCA estimates of the factors ft are the first K eigenvectors of ΩbSpecifically, let F denote the orthornormal K × T matrix consisting of theb T . Then bbfirst K eigenvectors of Ω
ft is the tth column of F.
The main advantages of the APCA approach are:• It works in situations where the number of assets, N , is much greaterthan the number of time periods, T . Eigenvectors of the smaller T ×
b T only need to be computed, whereas with traditional
T matrix Ω
principal component analysis eigenvalues of the larger N × N matrixb N need to be computed.
Ω• The method allows for an approximate factor structure of returns.
In an approximate factor structure, the asset specific error terms εit
are allowed to be contemporaneously correlated, but this correlation
is not allowed to be too large across the cross section of returns.
Allowing an approximate factor structure guards against picking up
local factors, e.g. industry factors, as global common factors.
Refinement

Connor and Korajczyk (1988) oﬀered a refinement of the APCA procedurethat may improve the eﬃciency of the procedure.
1. Estimate the factors ft (t = 1, . . . , T ) by computing the first K eigenbT.
values of Ω

2. For each asset, estimate the time series regression (factor model) by
OLS
Rit = αi + β0ib
ft + εit , t = 1, . . . , T

and compute the residual variances σ
b2i . Use these variance estimates
to compute the residual covariance matrix
⎛

σ
b21

b =⎜
D
⎝ 0
0

0
..
.

···

0

⎞

⎟
0 ⎠
σ
b2N

15.5 Statistical Factor Models for Returns

607

3. Form the N × T matrix of re-scaled returnsb −1/2 RR∗ =D

and recompute the T × T covariance matrixb ∗ = 1 R∗0 R∗Ω
TN

4. Re-estimate the factors ft by computing the first K eigenvalues of
b∗ .
Ω
T

Example 113 Estimation of a statistical factor model by asymptotic principal component analysis using S-PLUS

The S+FinMetrics function mfactor estimates a statistical factor modelby asymptotic principal components whenever the number of assets, N , isgreater than the number of time periods, T . To illustrate, consider fitting astatistical factor model using the S+FinMetrics “timeSeries” folio.dat,
which contains weekly data on 1618 stocks over the period January 8, 1997
to June 28, 2000. For this data, N = 1618 and T = 182. To compute the
APCA fit with k = 15 factors use> folio.mf = mfactor(folio.dat,k=15)> folio.mf
Call:mfactor(x = folio.dat, k = 15)
Factor Model:
Factors Variables Periods
15
1618
182
Factor Loadings:
Min. 1st Qu.
MedianMean 3rd Qu. Max.
F.1 -0.977 -0.4261 -0.314658 -0.33377 -0.2168 0.160
F.2 -0.420 -0.1041 -0.014446 0.06519 0.1628 1.110
F.3 -0.463 -0.0784 -0.011839 -0.00311 0.0392 0.998
F.4 -0.556 -0.0588 0.004821 0.00866 0.0771 0.495
F.5 -1.621 -0.0622 0.015520 0.01373 0.0858 0.467
F.6 -0.835 -0.0635 -0.001544 0.00307 0.0665 0.468
F.7 -0.758 -0.0633 -0.006376 -0.01183 0.0509 2.090
F.8 -0.831 -0.0685 -0.012736 -0.01413 0.0479 0.517
F.9 -0.464 -0.0466 0.006447 0.01200 0.0640 1.095
F.10 -0.640 -0.0659 -0.008760 -0.01050 0.0482 0.687
F.11 -1.515 -0.0540 -0.001114 -0.00457 0.0539 0.371
F.12 -1.682 -0.0637 -0.005902 -0.01068 0.0451 0.515

608

15. Factor Models for Asset Returns

Screeplot

20

Variances

30

40

0.167

10

0.224

0.256
0.309 0.322 0.334

0.3460.357 0.3680.378 0.388 0.3970.406

F.6

F.9 F.10 F.11 F.12 F.13 F.14 F.15

00.275 0.294

F.1

F.2

F.3

F.4

F.5

F.7

F.8

FIGURE 15.9. Screeplot of eigenvalues from APCA fit to 1618 assets.

F.13 -0.462 -0.0480 0.001901 0.00164
F.14 -0.912 -0.0523 -0.001072 -0.00443
F.15 -0.681 -0.0505 -0.000977 -0.00366

0.0516 0.685
0.0472 0.436
0.0473 0.548

Regression R-squared:
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.066 0.265
0.354 0.372 0.459
0.944
By default, the APCA fit uses the Connor-Korajczyk refinement. To compute the APCA fit without the refinement, set the optional argument
refine=F in the call to mfactor. The factor loadings appear to be reasonably scaled and skewed toward negative values. The loadings for the
first factor appear to be almost all negative. Multiplying the first factor
by negative one would make it more interpretable. The median regression
R2 is about thirty five percent, which is a bit higher than what one would
expect from the single index model.
Figures 15.9 and 15.10 show the screeplot of eigenvalues and factor returns for the APCA fit, computed using> screeplot.mfactor(folio.mf)> fplot(factors(folio.mf)

15.5 Statistical Factor Models for Returns

609

Factor Returns
1997

1998

1999

2000

1997

F.2

F.3

1998

1999

2000

F.4

-0.2

0

0.0

0

0.0

0.2

0.2

F.1

F.6

F.7

F.8

0
0

0

0

F.5

F.10

F.11

F.12

0.0
-0.2

0.0

0

0.0

0.2

0.2

0.2

F.9

F.14

F.15

0

0

0

F.13

1997

1998

1999

2000

1997

1998

1999

2000

FIGURE 15.10. Estimated factors returns from APCA fit to 1618 assets.

The first two factors clearly have the largest explanatory power, and thefifteen factors together explain roughly forty one percent of the total variance.
The factor mimicking portfolios are computed using> folio.m = mimic(folio.mf)
which is an object of class “mimic” of dimension 1618 × 15. It is diﬃcult
to concisely summarize the factor mimicking portfolios when the numberof assets is large. This is why the summary method for “mimic” objects has
an option for displaying only the largest and smallest n.top weights foreach factor mimicking portfolio. To view the top five largest and smallest
weights for the fifteen factors use> folio.ms = summary(folio.m,n.top=5)> folio.ms
Factor 1
Top.Long.Name Top.Long.Weight Top.Short.Name Top.Short.Weight
1
OWC
0.23%
BBY
-0.15%
2
FNV
0.22%
SCON
-0.14%
3
MT
0.22%
PUMA
-0.12%
4
BAC
0.21%
THDO
-0.11%
5
CACC
0.21%
AVTC
-0.11%

610

15. Factor Models for Asset Returns...
Factor 15
Top.Long.Name Top.Long.Weight Top.Short.Name Top.Short.Weight
1
ALXN
79%
SCTC
-85%
2
AVID
59%
MCLL
-82%
3
LTXX
53%
WLNK
-65%
4
IDX
52%
LRCX
-63%
5
SEAC
51%
TSCC
-63%
The summary information may be visualized using the generic plot function> plot(folio.ms)
which generates a fifteen page graph sheet, with one page for each factor.
The correlations of the assets giving the largest and smallest weights
for a given factor may be visualized using an image plot. To do this, firstcompute the correlation matrix for all of the assets> folio.cov = vcov(folio.mf)> sd = sqrt(diag(folio.cov))> folio.cor = folio.cov/outer(sd,sd)
Extract the names of the assets in the summary for the first factor> top.names = c(as.character(folio.m[[1]][,1]),+ rev(as.character(folio.ms[[1]][,3])))and call the S+FinMetrics function image.plot> image.plot(folio.cor[top.names, top.names],+ sub="Risk factor 1", main="Correlations of top positions")
The resulting plot is shown in Figure 15.11.

15.5.4 Determining the Number of Factors
The statistical methods described above are based on knowing the numberof common factors. In practice, the number of factors is unknown andmust be determined from the data. If traditional factor analysis is used,then there is a likelihood ratio test for the number of factors. However,this test will not work if N > T . Connor and Korajczyk (1993) described a
procedure for determining the number of factors in an approximate factor
model that is valid for N > T and Connor (1995) applied this method toa variety of factor models. Recently Bai and Ng (2002) have proposed analternative method.

15.5 Statistical Factor Models for Returns

611

Correlations of top positions
BBY
SCON
PUMA
THDO
AVTC
CACC
BAC
MT
FNV
OWC
BBY

SCON

PUMA

THDO

AVTC

CACC

BAC

MT

FNV

OWC

Risk factor 1

FIGURE 15.11. Image plot correlations between assets with top five largest andsmallest weights in first factor mimicking portfolio.

Connor and Korajczyk Method
The intuition behind this method is that if K is the correct number ofcommon factors then there should be no significant decrease in the crosssectional variance of the asset specific error, εit , in moving from K to K +1
factors. The procedure is implemented as follows:
1. Given observed returns on asset i and a time series of K + 1 factors,
estimate the time series regression models
Rit
Rit= αi + β0ib
ft + εit0b= αi + β ft + βi∗
K+1,i fK+1,t + εit

giving residuals b
εit and bε∗it .

2. Calculate degrees-of-freedom adjusted squared residuals
σ
bit

σ
b∗it

=

b
ε2it
1 − (K + 1)/T − K/N
2

=

b
ε∗it
1 − (K + 3)/T − (K + 1)/N

612

15. Factor Models for Asset Returns

3. Calculate the cross-sectional diﬀerence in squared errors based on odd
and even time periods
bs
∆

μ
bt

μ
b∗t

= μ
b2s−1 − μb∗2s , s = 1, . . . , T /2
N

1 X
σ
bit
N i=1

=

N

1 X ∗
σ
b
N i=1 it=and compute the T /2 ×1 vector of diﬀerences

´0
³
b = ∆
b 2, . . . , ∆b T /2b 1, ∆∆

4. Compute the time series sample mean and variance of the diﬀerences
T /2

∆ =
σ
b2∆

=

T Xb
∆s
2 s=1

T /2
´2
2 X³b
∆s − ∆
T − 2 s=1

5. Compute the t-statistic
t=

∆
σ
b∆

and use it to test for a positive mean value.
Bai and Ng Method
Bai and Ng (2002) proposed some panel Cp (Mallows-type) information
criteria for choosing the number of factors. Their criteria are based on theb or Ω
b N solves the least squares
observation that eigenvector analysis on Ω
problemN X
TX
min (N T )−1(Rit − αi − β 0i ft )2
β i ,f t

i=1 t=1

Bai and Ng’s model selection or information criteria are of the form

where

IC(K) = σb2 (K) + K · g(N, T )
N

1 X 2
σ
b
σ
b (K) =
N i=1 i2

15.5 Statistical Factor Models for Returns

613

is the cross-sectional average of the estimated residual variances for each
asset based on a model with K factors and g(N, T ) is a penalty function
depending only on N and T . The preferred model is the one which minimizes the information criteria IC(K) over all values of K < Kmax . Bai and
Ng consider several penalty functions and the preferred criteria are
µ
¶
µ
¶
N +T
NT
2
2
PCp1 (K) = σb (K) + K · σb (Kmax )
· ln
,
NT
N +T
µ
¶
¡ 2 ¢
N +T
b2 (K) + K · σ
b2 (Kmax )PCp2 (K) = σ,· ln CNTNT√ √where CN T = min( N , T ).
The implementation of the Bai and Ng strategy for determining thenumber of factors is a follows. First, select a number Kmax indicating the
maximum number of factors to be considered. Then for each value of K <Kmax , do the following:
1. Extract realized factors b
ft using the method of APCA.
2. For each asset i, estimate the factor model

Rit = αi + β 0ib
ftK + εit ,where the superscript K indicates that the regression has K factors,
using time series regression and compute the residual variancesσb2i (K) =

T
X
1
b
ε2 .
T − K − 1 t=1 it

3. Compute the cross-sectional average of the estimated residual variances for each asset based on a model with K factorsNσb2 (K) =

1 X 2
σ
b (K)
N i=1 i

4. Compute the cross-sectional average of the estimated residual variances for each asset based on a model with Kmax factors, σ
b2 (Kmax ).
5. Compute the information criteria PCp1 (K) and PCp2 (K).

6. Select the value of K that minimized either PCp1 (K) or PCp2 (K).
Bai and Ng performed an extensive simulation study and found that the
selection criteria PCp1 and PCp2 yield high precision when min(N, T ) > 40.
Example 114 Determining the number of factors for a statistical factor
model estimated by asymptotic principal components

614

15. Factor Models for Asset Returns

To determine the number of factors in the “timeSeries” folio.dat
using the Connor-Korajczyk method with a maximum number of factors
equal to ten and a significance level equal to five percent use8
> folio.mf.ck = mfactor(folio.dat,k="ck",max.k=10,sig=0.05)> folio.mf.ck
Call:mfactor(x = folio.dat, k = "ck", max.k = 10, sig = 0.05)
Factor Model:
Factors Variables Periods
2
1618
182
Factor Loadings:
Min. 1st Qu.
MedianMean 3rd Qu. Max.
F.1 -0.177 0.2181 0.31721 0.3317
0.419 0.95
F.2 -0.411 -0.0958 -0.00531 0.0777
0.181 1.12
Regression R-squared:
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.000 0.124
0.188 0.206 0.268
0.837
Two factors are selected by the Connor-Korajczyk method. Notice thatmost of the loadings on the first factor are positive.
Similarly, to determine the number of factors using the Bai-Ng methoduse> folio.mf.bn = mfactor(folio.dat,k="bn",max.k=10,sig=0.05)> folio.mf.bn$k[1] 2Again, two factors are determined.

15.6 References
Alexander, C. (2001). Market Models: A Guide to Financial Data Analysis, John Wiley & Sons, Chichester, UK.
Bai, J. and Ng, S., (2002). “Determining the Number of Factors in Approximate Factor Models,” Econometrica, 70, 191-221.
8 For a data set with a large number of assets, the Connor-Korajczyk and Bai-Ng
methods may take a while.

15.6 References

615

Chamberlain, G. and Rothschild, M. (1983). “Arbitrage, Factor Structure and Mean-Variance Analysis in Large Asset Markets,” Econometrica,
51, 1305-1324.
Chan, L.K., Karceski, J. and Lakonishok, J. (1998). ”The Risk andReturn from Factors,” Journal of Financial and Quantitative Analysis,33(2), 159-188.
Chan, L.K., Karceski, J. and Lakonishok, J. (1999). “On Portfolio Optimization: Forecasting Covariances and Choosing the Risk Model,”Review of Financial Studies, 5, 937-974.
Chen, N.F., Roll, R., and Ross, S.A. (1986). “Economic Forces andthe Stock Market,” The Journal of Business, 59(3), 383-404.
Campbell, J.Y., Lo, A.W., and MacKinlay, A.C. (1997). The Econometrics of Financial Markets. Princeton University Press, Princeton, NJ.
Connor, G. (1995). “The Three Types of Factor Models: A Comparisonof Their Explanatory Power,” Financial Analysts Journal, 42-46.
Connor, G., and Korajczyk, R.A. (1986). “Performance Measurementwith the Arbitrage Pricing Theory: A New Framework for Analysis,” Journal of Financial Economics, 15, 373-394.
Connor, G., and Korajczyk, R.A. (1988). “Risk and Return in an
Equilibrium APT: Application of a New Test Methodology,” Journal of
Financial Economics, 21, 255-289.
Connor, G. and Korajczyk, R.A. (1993). “A Test for the Number ofFactors in an Approximate Factor Model,” The Journal of Finance, vol.
48(4), 1263-92.
Elton, E. and M.J. Gruber (1997). Modern Portfolio Theory and Investment Analysis, 5th Edition. John Wiley & Sons, New York.
Fama, E. and K.R. French (1992). “The Cross-Section of Expected
Stock Returns”, Journal of Finance, 47, 427-465.
Grinold, R.C. and Kahn, R.N. (2000). Active Portfolio Management:
A Quantitative Approach for Producing Superior Returns and Controlling
Risk, Second Edition. McGraw-Hill, New York.
Johnson and Wichern (1998). Multivariate Statistical Analysis. PrenticeHall, Englewood Cliﬀs, New Jersey.

616

15. Factor Models for Asset Returns

Sharpe, W.F. (1970). Portfolio Theory and Capital Markets. McGrawHill, New York.
Sheikh, A. (1995). “BARRA’s Risk Models,” mimeo, BARRA.

This is page 617
Printer: Opaque this16
Term Structure of Interest Rates

16.1 Introduction
In financial markets, the term structure of interest rates is crucial to pricing of fixed income securities and derivatives. The last thirty years haveseen great advances in the financial economics of term structure of interest
rates. This chapter will focus on interpolating the term structure of interest rates from discrete bond yields. Refer to Campbell, Lo, and MacKinlay(1997) for basic concepts in fixed income calculations and Hull (1997) for
an introduction to theoretical term structure modeling.
Section 16.2 first defines diﬀerent rates, such as spot or zero coupon
interest rate, forward rate, and discount rate, and documents how one rate
can be converted to another. Section 16.3 shows how to interpolate term
structure data using quadratic or cubic spline. Section 16.4 illustrates how
to use smoothing splines to fit term structure data. Section 16.5 introduces
the parametric Nelson-Siegel function and its extension and shows how it
can be used to interpolate term structure data. Bliss (1997) and Ferguson
and Raymar (1998) compared the performance of these diﬀerent methods.
Section 16.6 concludes this chapter.

618

16. Term Structure of Interest Rates

16.2 Discount, Spot and Forward Rates
16.2.1 Definitions and Rate Conversion
Although many theoretical models in financial economics hinge on an abstract interest rate, in reality there are many diﬀerent interest rates. For
example, the rates of a three month U.S. Treasury bill are diﬀerent fromthose of a six month U.S. Treasury bill. The relationship between these different rates of diﬀerent maturity is known as the term structure of interest
rates. The term structure of interest rates can be described in terms of spot
rate, discount rate or forward rate.
The discount function, d(m), gives the present value of $1.00 which isrepaid in m years. The corresponding yield to maturity of the investment,y(m), or spot interest rate, or zero coupon rate, must satisfy the following
equation under continuous compounding:d(m)ey(m)·m = 1
ord(m) = e−y(m)·m(16.1)

Obviously, the discount function is an exponentially decaying function of
the maturity, and must satisfy the constraint d(0) = 1.
The above equation easily shows that under continuous compoundingy(m) = −log d(m).
m

If discrete compounding is used instead, one can similarly show that1y(m) = p[d(m)− p·m − 1]
where p is the number of compounding periods in a year.
The spot interest rate is the single rate of return applied over the maturity of m years starting from today. It is also useful to think of it as
the average of a series of future spot interest rates, or forward rates, with
diﬀerent maturities starting from a point in the future, and thus:Rmey(m)·m = e 0 f (x)dx
from which one can easily obtain:y(m) =

1m

Z m

f (x)dx(16.2)

0with f (m) denoting the forward rate curve as a function of the maturitym.

16.2 Discount, Spot and Forward Rates

619

Rate

0.076

0.080

0.084

Yield Curve: 1/1990

0

5

10

15

20

25

30

25

30

Maturity

Rate

0.065

0.075

0.085

Forward Rate Curve: 1/1990

0

5

10

15

20

Maturity

FIGURE 16.1. Yield curve and forward rate curve for January 1990.

From (16.1) and (16.2), the relationship between the discount function
and forward rate can be derived:
Z md(m) = exp{−
f (x)dx}0orf (m) = −d0 (m).
d(m)Hence the forward rate gives the rate of decay of the discount function as afunction of the maturity m. The relationship between these diﬀerent rates
under discrete compounding can be similarly obtained.

16.2.2 Rate Conversion in S+FinMetrics
To facilitate the interpolation of term structure from any of discount rate,
spot rate, or forward rate, S+FinMetrics provides a group of functions forconverting one rate into another rate. These functions will be illustrated using the mk.zero2 and mk.fwd2 data sets in S+FinMetrics, which contains
the U.S. zero coupon rates and forward rates, respectively, as computed by
McCulloch and Kwon (1993).
Both mk.zero2 and mk.fwd2 are “timeSeries” objects with 55 columns,
with each column representing the rate with the corresponding maturity

620

16. Term Structure of Interest Rates

in the 55 × 1 vector mk.maturity. For example, the first element of thevector mk.maturity is 0.083, so the first columns of mk.zero2 and mk.fwd2
correspond to the rates with maturity of one month. Use the following code
to plot the yield curve and forward rate curve for January 1990, and thegraph is shown in Figure 16.1:> par(mfrow=c(2,1))> plot(mk.maturity,mk.zero2[54,],xlab="Maturity",ylab="Rate")> title(paste("Yield Curve:", positions(mk.zero2[54,])))> plot(mk.maturity,mk.fwd2[54,],xlab="Maturity",ylab="Rate")> title(paste("Forward Rate Curve:",positions(mk.fwd2[54,])))> par(mfrow=c(1,1))
To convert the spot interest rate or forward rate into the discount rate,
use the S+FinMetrics function bond.discount. For example, to convert
the first 48 spot rates in Figure 16.1 to discount rates, use the following
command:> disc.rate = bond.discount(mk.zero2[54, 1:48],+ mk.maturity[1:48], input="spot", compounding=2)
The bond.discount function takes two required arguments: the first isa vector of rates, and the second is a vector of the corresponding maturity.
Note that the optional argument input is used to specify the type of the input rates, and compounding to specify the number of compounding periods
in each year. So compounding=2 corresponds to semi-annual compounding.1
If the input rates are forward rates, simply set input="forward".
The functions bond.spot and bond.forward can be called in a similar
fashion to compute the spot interest rate and forward rate, respectively,from diﬀerent input rates. For all those three functions, the rates should
be expressed as decimal numbers, and the maturity should be expressed inunits of years. For example, to convert disc.rate back into the spot rates,
use the following command:> spot.rate = bond.spot(disc.rate, mk.maturity[1:48],+ input="discount", compounding=2)
It can be easily checked that spot.rate is the same as mk.zero2[54,
1:48].

16.3 Quadratic and Cubic Spline Interpolation
The interest rates are observed with discrete maturities. In fixed income
analysis, the rate for a maturity which is not observed can sometimes be
1 To use continuous compounding, specify compounding=0.

16.3 Quadratic and Cubic Spline Interpolation

621

used. Those unobserved rates can usually be obtained by interpolating theobserved term structure.
Since the discount rate should be a monotonically decreasing function of
maturity and the price of bonds can be expressed as a linear combinationof discount rates, McCulloch (1971, 1975) suggested that a spline method
could be used to interpolate the discount function, or the bond prices directly. In particular, use k continuously diﬀerentiable functions sj (m) toapproximate the discount rates:d(m) = a0 +kX

aj sj (m)(16.3)

j=1

where sj (m) are known functions of maturity m, and aj are the unknown
coeﬃcients to be determined from the data. Since the discount rate mustsatisfy the constraint d(0) = 1, set a0 = 1 and sj (0) = 0 for j = 1, · · · , k.
Note that once the functional form of sj (m) is determined, the coeﬃcients
aj can be easily estimated by linear regression. Thus the discount rate, or
forward rate, or spot rate, associated with an unobserved maturity can beeasily interpolated using the above functional form, as long as the maturity
is smaller than the largest maturity used in the estimation.
Figure 16.1 shows that there are usually more points in the short end of
the term structure, and less points in the long end of the term structure.
To obtain a reliable interpolation using the spline method, the functional
form of sj (m) should be chosen so that it adapts to the density of maturity m. McCulloch (1971) gave a functional form of sj (m) using quadratic
spline, which is based on piecewise quadratic polynomials, while McCulloch(1975) gave a functional form of sj (m) using cubic spline, which is basedon piecewise cubic polynomials.
Term structure interpolation using quadratic or cubic spline methods canbe performed by calling the term.struct function in S+FinMetrics. The
arguments taken by term.struct are:> args(term.struct)
function(rate, maturity, method = "cubic", input.type = "spot",na.rm = F, plot = T, compounding.frequency = 0,
k = NULL, cv = F, penalty = 2, spar = 0, ...)
NULL
Similar to bond.spot, bond.discount and bond.forward functions, thefirst argument rate should be a vector of interest rates, while the second argument maturity specifies the corresponding maturity in units ofyears. The type of the input interest rate should be specified through theoptional argument input.type. Note that the quadratic or cubic spline
methods operate on discount rates. If the input interest rates are not discount rates, the optional argument compounding.frequency should also be
set for proper conversion, which is set to zero for continuous compounding

622

16. Term Structure of Interest Rates

Discount Function

Term Structure
1.0

Rate

0.8

0.6

0.4

0.2

0

5

10

15

20

25

30

Maturity

FIGURE 16.2. U.S. discount function for January 1990: quadratic spline.

by default. The optional argument k determines the number of functionsin (16.3), also known√as knot points. By default, follow McCulloch (1971,1975) and set k = [ n] where n is the length of the input rates. Other
optional arguments will be discussed in later sections.
To illustrate the usage of the spline methods, in order to interpolate theterm structure corresponding to January 1990, using mk.zero2, use thefollowing command:> disc.rate = term.struct(mk.zero2[54,], mk.maturity,+ method="quadratic", input="spot", na.rm=T)
Note that na.rm=T is set to remove the missing values at the long endof the term structure. By default, the interpolated discount rate is plotted automatically, which is shown in Figure 16.2. The points in the figure
represent the original discount rates, while the line represents the splineinterpolation.
The returned object disc.rate is of class “term.struct”. As usual, typing the name of the object at the command line invokes its print method:> class(disc.rate)[1] "term.struct"> disc.rate
Call:16.3 Quadratic and Cubic Spline Interpolation

623

0.076

0.078

0.080

Rate

0.082

0.084

0.086

Yield Curve

0

5

10

15

20

25

30

Maturity

FIGURE 16.3. U.S. yield curve for January 1990: quadratic spline.

term.struct(rate = mk.zero2[54, ], maturity = mk.maturity,
method = "quadratic", input.type = "spot", na.rm = T)
Coefficients:
a1
a2
a3
a4
a5
a6
-0.0761 -0.0791 -0.0688 -0.0373 -0.0146 -0.0045
Degrees of freedom: 48 total; 42 residual
Residual standard error: 0.001067688
Since the unknown coeﬃcients aj of the spline are estimated by linear
regression, the output looks very much similar to linear regression output.
Since there are 48 spot rates available for January 1990, the number ofknot points is chosen to be 6 by default.
The plot generated in Figure 16.2 shows the interpolated discount function because the quadratic or cubic spline methods are designed to operateon discount function. This plot can be later regenerated by calling thegeneric plot function on a “term.struct” object. However the yield curveor forward rate curve is usually of more interest. These can also be easilyplotted using the components of a “term.struct” object. For example, use
the S-PLUS names function to find out the components of disc.rate:> names(disc.rate)[1] "coefficients" "residuals""fitted.values" "effects"

624

16. Term Structure of Interest Rates[5] "R"[9] "contrasts"[13] "knots""rank""terms""method""assign""call""maturity""df.residual""fitted""rate"

The first 10 components are inherited from an “lm” object, because the
S-PLUS lm function is used for the linear regression. The fitted (insteadof the fitted.values) component represents the estimated discount rates
associated with the maturity component. To plot the interpolated yield
curve or forward rate curve, simply convert the estimated discount rates
into the rates you want. For example, use the following code to plot theinterpolated yield curve:> spot.rate = bond.spot(disc.rate$fitted, disc.rate$maturity,+ input="discount", compounding=0)> plot(mk.maturity[1:48], mk.zero2[54,1:48],+ xlab="Maturity", ylab="Rate", main="Yield Curve")> lines(disc.rate$maturity, spot.rate)
and the plot is shown in Figure 16.3. Note that in the plot the points represent the original zero coupon rates, while the line represents the quadratic
spline interpolation.

16.4 Smoothing Spline Interpolation
The previous section demonstrated that the polynomial spline methods
proposed by McCulloch (1971, 1975) can fit the discount rate and yield
curve very well. However, since the methods operate on (linear combinations of) discount functions, the implied forward rate curve usually hassome undesirable features. For example, use the following code to generate
the implied forward rate curve from the object disc.rate fitted in theprevious section:> fwd.rate = bond.forward(disc.rate$fitted, disc.rate$maturity,+ input="discount", compounding=0)> plot(disc.rate$maturity, fwd.rate, type="l",+ xlab="Maturity", ylab="Rate", main="Forward Rate")> points(mk.maturity[1:48], mk.fwd2[54, 1:48])
The plot is shown in Figure 16.4. The implied forward rate is way oﬀ at
the long end of the term structure.
In addition to the undesirable behavior of implied forward rate, the choiceof knot points for polynomial splines is rather ad hoc. For a large number of
securities, the rule can imply a large number of knot points, or coeﬃcientsaj . To avoid these problems with polynomial spline methods, Fisher, Nychka and Zervos (1995) proposed to use smoothing splines for interpolating
the term structure of interest rates.

16.4 Smoothing Spline Interpolation

625

0.07
0.05

0.06

Rate

0.08

0.09

Forward Rate

0

5

10

15

20

25

30

Maturity

FIGURE 16.4. U.S. forward rate for January 1990: quadratic spline.

In general, for an explanatory variable xi and a response variable yi ,the smoothing spline tries to find a smooth function f (·) to minimize thepenalized residual sum of squares (PRSS):ZnX
2PRSS =[yi − f (xi )] + λ [f 00 (t)]2 dt(16.4)
i=1

where the first term is the residual sum of squares (RSS), and the second
term is the penalty term, and the parameter λ controls the trade-oﬀ between goodness-of-fit and parsimony. By using the penalty term, the spline
function can be over-parameterized, while using λ to reduce the eﬀective
number of parameters.
S denote the n × n implicit smoother matrix such that f (xi ) =
PLet
n
S(x
i , xj )yj . Fisher, Nychka and Zervos (1995) suggested using genj=1
eralized cross validation (GCV) to choose λ. That is, λ is chosen to minimize
GCV =

RSS
n − θ · tr(S)where θ is called the cost, and tr(S) denotes the trace of the implicit
smoother matrix and is usually used as the measure of eﬀective numberof parameters.
Interpolation of term structure using smoothing spline can also be performed using the term.struct function by setting the optional argument

626

16. Term Structure of Interest Rates

Yield Curve

Term Structure

0.086

0.084

Rate

0.082

0.080

0.078

0.076
0

5

10

15

20

25

30

Maturity

FIGURE 16.5. U.S. yield curve for January 1990: smoothing spline.

method="smooth". The procedure uses the S-PLUS smooth.spline function as the workhorse.2 In particular, for all the arguments taken by thefunction term.struct, cv, penalty and spar are specifically used forsmoothing spline methods and passed to the smooth.spline function. Bydefault, use GCV by setting cv=F and thus spar, which specifies the valueof λ, is ignored.3 The optional argument penalty is used to specify thevalue for θ. Following Fisher, Nychka, and Zervos (1995), set θ = 2 bydefault.
For example, use the following command to interpolate the yield curvefor January 1990, with the smoothing spline method:> fnz.fit = term.struct(mk.zero2[54,], mk.maturity,+ method="smooth", input="spot", na.rm=T)
Again, the interpolated yield curve is plotted automatically, as shown in
Figure 16.5. Although the returned object fnz.fit is of class “term.struct”,
its components are diﬀerent from the disc.rate object fitted in the previous section, because now the smooth.spline function is used as theworkhorse:
2 Refer to Hastie (1993) and S-PLUS Guide to Statistics for the description ofsmooth.spline function.
3 For further details regarding these arguments, see the on-line help file forsmooth.spline function.

16.4 Smoothing Spline Interpolation> class(fnz.fit)[1] "term.struct"> names(fnz.fit)[1] "x""y"[7] "pen.crit" "df"[13] "maturity" "rate""w""spar"

"yin"
"fit"

"lev"
"call"

627"cv.crit""method"

The first 10 components are inherited from a “smooth.spline” object,while the last four components are generated by the term.struct function.
For the same reason, the print function now shows diﬀerent information:> fnz.fit
Call:term.struct(rate = mk.zero2[54, ], maturity = mk.maturity,
method = "smooth", input.type = "spot", na.rm = T)
Smoothing Parameter (Spar): 4.767984e-11
Equivalent Degrees of Freedom (Df): 47.57122
Penalized Criterion: 4.129338e-10
GCV: 3.605842e-14
which shows the optimal smoothing parameter λ, and its associated GCV,
penalized criterion, and equivalent degrees of freedom.
For “term.struct” objects, S+FinMetrics also implements a predict
method, which can be used to obtain the interpolated rate associated withan arbitrary vector of maturity. For example, to recover the fitted spot
rates from fnz.fit, use the predict method as follows:> fnz.spot = predict(fnz.fit, fnz.fit$maturity)
From the fitted spot rates, one can compute the implied forward rates
for the smoothing spline:> fnz.forward = bond.forward(fnz.spot, fnz.fit$maturity,+ input="spot", compounding=0)> plot(mk.maturity[1:48], mk.fwd2[54,1:48],+ xlab="Maturity", ylab="Rate", main="Forward Rate")> lines(fnz.fit$maturity, fnz.forward)
The “real” forward rates and the smoothing spline interpolations areshown together in Figure 16.6. The interpolations agree very well with the“real” forward rates. The slight diﬀerence is partly caused by the fact that
mk.zero2[54,] and the spot rates implied by mk.fwd2[54,] are slightly
diﬀerent.

628

16. Term Structure of Interest Rates

Rate

0.065

0.070

0.075

0.080

0.085

0.090

Forward Rate

0

5

10

15

20

25

30

Maturity

FIGURE 16.6. U.S. forward rate for January 1990: smoothing spline.

16.5 Nelson-Siegel Function
The previous sections have shown that both the polynomial and smoothing
spline methods can fit the term structure very well, except that the implied
forward rates from polynomial spline methods have some undesirable features at the long end of the term structure. However, the non-parametricspline based methods usually do not generate good out-of-sample forecasts. There is substantial evidence showing that a parametric function
suggested by Nelson and Siegel (1987) has better out-of-sample forecasting
performance.
Using a heuristic argument based on the expectation theory of the term
structure of interest rates, Nelson and Siegel (1987) proposed the following
parsimonious model for the forward rate:
f (m) = β 0 + β 1 · e−m/τ + β 2 · m/τ · e−m/τ .
They suggested that the model may also be viewed as a constant plus a
Laguerre function, and thus can be generalized to higher-order models.
Based on the above equation, the corresponding yield curve can be derived
as follows:∙¸
1 − e−m/τ
1 − e−m/τ
y(m) = β 0 + β 1(16.5)+ β2− e−m/τ .
m/τ
m/τ

16.5 Nelson-Siegel Function

629

0.2

0.4

0.6

0.8

tau=0.5

0

10

20

30

40

50

30

40

50

0.0

0.2

0.4

0.6

0.8

1.0

tau=1.5

010

20

FIGURE 16.7. Short term and medium term components of Nelson-Siegel function.

For a given constant τ , both the forward rate curve and the yield curve
are linear functions of the coeﬃcients β 0 , β 1 and β 2 . Nelson and Siegel(1987) showed that, depending on the values of β 1 and β 2 , the yield curve
can assume the common shapes of observed yield curves, such as upward
sloping, downward sloping, humped, or inverted humped. In addition, consistent with stylized facts of the yield curve, the three components in (16.5)
can be interpreted as the long term, short term and medium term component, or the level, slope, and curvature component of the yield curve.4
Example 115 Interpretation of Nelson-Siegel function
The function term.struct.nsx in S+FinMetrics can be used to generate
the regressors in (16.5) given a vector of maturity and a value for τ . Use
the following code to visualize these components for diﬀerent values of τ :> ns.maturity = seq(1/12, 10, length=50)> ns05 = term.struct.nsx(ns.maturity, 0.5)
> ns15 = term.struct.nsx(ns.maturity, 1.5)
> par(mfrow=c(2,1))> tsplot(ns05[,2:3], main="tau=0.5")> tsplot(ns15[,2:3], main="tau=1.5")
4 Refer to Diebold and Li (2002) for a detailed explanation.

630

16. Term Structure of Interest Rates

Yield Curve

Term Structure

0.086

0.084

Rate

0.082

0.080

0.078

0.076
0

5

10

15

20

25

30

Maturity

FIGURE 16.8. U.S. yield curve for January 1990: Nelson-Siegel function.

> par(mfrow=c(1,1))
A vector of maturity was created from one month to ten years. The regressor matrix has three columns, and only the last two columns were plottedbecause the first column is always one, and the plot is shown in Figure 16.7.
The parameter τ controls the rate of decay of those components. When τ issmaller, the short and medium term components decay to zero at a fasterrate. Asymptotically, both the short and medium term components approach zero, and thus β 0 can be interpreted as the long term component,or the level of the yield curve.
To interpolate yield curves using the Nelson-Siegel function, choose thevalue of τ which gives the best fit for equation (16.5). The term.struct
function employs this procedure if the optional argument method is set
to "ns". For example, use the following command to interpolate the yield
curve for January 1990:> ns.fit = term.struct(mk.zero2[54,], mk.maturity,+ method="ns", input="spot", na.rm=T)> ns.fitCall:term.struct(rate = mk.zero2[54, ], maturity = mk.maturity,
method = "ns", input.type = "spot", na.rm = T)

16.5 Nelson-Siegel Function

Coefficients:b0
b1
0.0840 -0.0063

631

b2
0.0044Degrees of freedom: 48 total; 45 residual
Residual standard error: 0.001203026
Tau estimate: 1.7603
Again, the fit is plotted by default as shown in Figure 16.8. The graphshows that although the Nelson-Siegel generally captures the shape of theyield curve, the in-sample fit is usually not as good as the non-parametricspline methods because it only uses three coeﬃcients. The output showsthe estimates of those coeﬃcients, along with the estimate of τ .
Since the Nelson-Siegel function does not fit the data very well when theyield curve has a rich structure as in the above example, Svensson (1994)
proposed to extend the Nelson-Siegel forward function as follows:f (m) = β 0 + β 1 e−m/τ 1 + β 2 · m/τ 1 · e−m/τ 1 + β 3 · m/τ 2 · e−m/τ 2
which adds another term to the Nelson-Siegel function to allow for a second
hump. The corresponding yield function can be shown to be:y(m) = β 0 + β 1∙¸
1 − e−m/τ 1
1 − e−m/τ 1+ β2− e−m/τ 1
m/τ 1
m/τ 1
∙
¸
1 − e−m/τ 2+ β3− e−m/τ 2 . (16.6)m/τ 2

To use the above function for interpolating yield curve, simply call thefunction term.struct with method="nss":> nss.fit = term.struct(mk.zero2[54,], mk.maturity,+ method="nss", input="spot", na.rm=T)> nss.fit
Call:term.struct(rate = mk.zero2[54, ], maturity = mk.maturity,
method = "nss", input.type = "spot", na.rm = T)
Coefficients:
b0
b1
b2
b3
0.0000 0.0761 0.1351 0.0104
Degrees of freedom: 48 total; 44 residual
Residual standard error: 0.0005997949
Tau estimate: 21.8128 0.5315

632

16. Term Structure of Interest Rates

Yield Curve

Term Structure

0.086

0.084

Rate

0.082

0.080

0.078

0.076
0

5

10

15

20

25

30

Maturity

FIGURE 16.9. U.S. yield curve for January 1990: Svensson function.

The output now shows two estimates for τ and one more coeﬃcient for
the additional term. The plot of the interpolated yield curve is shown in
Figure 16.9.

16.6 Conclusion
For all the term structure interpolation methods discussed in this chapter,they all work with the yield curve for a given time, and thus do not consider
the time series aspect of the yield curve. Recently Diebold and Li (2002)
considered estimating the three components β 0 , β 1 and β 2 of the NelsonSiegel function for each available time, and building a time series model(in particular, an AR(1)-GARCH(1,1) model) for the estimated β 0 , β 1 andβ 2 . By employing the times series forecasts of β 0 , β 1 and β 2 , they are able
to generate reliable forecasts of yield curve. However, in this approach, thecoeﬃcients β 0 , β 1 and β 2 are still estimated ignoring the time series aspect.
In recent years, many researchers have proposed to use state space models and Kalman filter to estimate the term structure of interest rates using apanel data, for example, see Duan and Simonato (1999), Geyer and Pichler(1999), Babbs and Nowman (1999), de Jong and Santa-Clara (1999) andde Jong (2000). Most of these models are special cases of the aﬃne term
structure model proposed by Duﬃe and Kan (1996), which can be readily

16.7 References633

expressed in a state space model by discretizing the continuous-time models. These models can be easily implemented using the state space modeling functions in S+FinMetrics as illustrated in Zivot, Wang and Koopman(2004).

16.7 References
Babbs, S. H., and Nowman, K. B. (1999). “Kalman Filtering of Generalized Vasicek Term Structure Models,” Journal of Financial and Quantitative Analysis, 34(1), 115-129.
Bliss, R. R. (1997). “Testing Term Structure Estimation Methods”, in
P. Boyle, G. Pennacchi, and P. Ritchken (eds.), Advances in Futures andOptions Research, Volume 9. Elsevier, Amsterdam, pp. 197-231.
Campbell, J. Y., Lo, A. W., and MacKinlay, A. C. (1997). The
Econometrics of Financial Markets. Princeton University Press, Princeton,
NJ.
de Jong, F. (2000). “Time Series and Cross Section Information in Aﬃne
Term-Structure Models,” Journal of Business and Economic Statistics,
18(3), 300-314.
de Jong, F., and Santa-Clara, P. (1999). “The Dynamics of the Forward Interest Rate Curve: a Formulation with State Variables,” Journal of
Financial and Quantitative Analysis, 34(1), 131-157.
Diebold, F. X., and Li, C. (2003). “Forecasting the Term Structure ofGovernment Bond Yields,” NBER Working Paper No. 10048.
Duan, J.-C., and Simonato, J. (1999). “Estimating and Testing ExponentialAﬃne Term Structure Models by Kalman Filter,” Review of Quantitative
Finance and Accounting, 13, 111-135.
Duffie, D., and Kan, R. (1996). “A Yield-Factor Model of Interest
Rates,” Mathematical Finance, 6(4), 379-406.
Ferguson, R., and Raymar, S. (1998). “A Comparative Analysis of Several Popular Term Structure Estimation Models,” Journal of Fixed Income,March 1998, 17-33.
Fisher, M., Nychka, D., and Zervos, D. (1995). “Fitting the Term
Structure of Interest Rates with Smoothing Splines,” Finance and Economics Discussion Series #1995-1, Board of Governors of the Federal Reserve System.

634

16. Term Structure of Interest Rates

Geyer, A. L. J., and Pichler, S. (1999). “A State-Space Approachto Estimate and Test Multifactor Cox-Ingersoll-Ross Models of the Term
Structure,” Journal of Financial Research, 22(1), 107-130.
Hastie, T. J. (1993). “Generalized Additive Models,” in J. M. Chambers
and T. J. Hastie (eds.), Statistical Models in S. Chapman & Hall.
Hull, J. C. (1997). Options, Futures, and Other Derivatives, Prentice
Hall, New York.
McCulloch, J. H. (1971). “Measuring the Term Structure of Interest
Rates,” Journal of Business, 44, 19-31.
McCulloch, J. H. (1975). “The Tax-Adjusted Yield Curve”, Journal of
Finance, 30(3), 811-830.
McCulloch, J. H., and Kwon, H.-C. (1993). “U.S. Term Structure
Data: 1947-1991,” Department of Economics, Working Paper #93-6, Ohio
State University.
Nelson, C. R., and Siegel, A. F. (1987). “Parsimonious Modeling of
Yield Curves,” Journal of Business, 60(4), 473-489.
Svensson, L. E. O. (1994). “Estimating and Interpreting Forward Interest
Rates: Sweden 1992-1994”, NBER Working Paper No. 4871.
Zivot, E., Wang, J. and S.J. Koopman (2004). “State Space Models
in Economics and Finance Using SsfPack in S+FinMetrics,” in A. Harvey,
S.J. Koopman, and N. Shephard (eds.), Unobserved Components Models.
Cambridge University Press, Cambridge.

This is page 635
Printer: Opaque this17
Robust Change Detection

17.1 Introduction
In time series analysis, autoregressive integrated moving average (ARIMA)
models have found extensive use since the publication of Box and Jenkins(1976). For an introduction to the standard ARIMA modeling in S-PLUS,
see S-PLUS Guide to Statistics. Regression models are also frequently usedin finance and econometrics research and applications. For example, as“factor” models for empirical asset pricing research and for parsimonious
covariance matrix estimation in portfolio risk models. Often ARIMA models and regression models are combined by using an ARIMA model toaccount for serially correlated residuals in a regression model, resulting in
REGARIMA models.
In reality, most time series data are rarely completely well behaved andoften contain outliers and level shifts, which is especially true for economic
and financial time series. The classical maximum likelihood estimators ofboth ordinary regression model parameters and ARIMA model parameters
are not robust in that they can be highly influenced by the presence of evena small fraction of outliers and/or level shifts in a time series. It is therefore
not suprising that classical maximum likelihood estimators of REGARIMA
models also lack robustness toward outliers and/or level shifts.
S+FinMetrics provides functions that compute robust alternatives tothe classical non-robust MLE’s for robust fitting and diagnostics of REGARIMA models. In particular, the robust procedure arima.rob allows
reliable model fitting when the data contain outliers and/or level shifts. In

636

17. Robust Change Detection

addition, it also detects the types and locations of the outliers in the timeseries and thus can be used to perform robust change detection.
This chapter is organized as follows: Section 17.2 gives a brief introduction to REGARIMA models, and Section 17.3 shows how to fit a robust REGARIMA model using functions in S+FinMetrics. Section 17.4 shows how
to predict from a robustly fitted REGARIMA model, while Section 17.5
illustrates more options which can be used to control the robust fitting of
REGARIMA models. Finally in Section 17.6, some technical details are
given about how REGARIMA model parameters are estimated robustly inthe procedure arima.rob.

17.2 REGARIMA Models
The REGARIMA model considered in this chapter takes the following form:
yt = x0t β + t , for t = 1, 2, · · · , T(17.1)

where xt is a k × 1 vector of predictor variables, and β is a k × 1 vector of
regression coeﬃcients. The error term t follows a seasonal ARIMA process:Φ(L)(1 − L)d (1 − Ls )D t = (1 − θ∗ Ls )Θ(L)ut(17.2)

where L is the lag (or backshift) operator, d the number of regular diﬀerences, D the number of seasonal diﬀerences, s the seasonality frequency,
Φ(L) = 1 − φ1 L − · · · − φp Lp a stationary autoregressive operator of orderp, Θ(L) = 1 − θ1 L − · · · − θq Lq a moving average operator of order q andθ∗ a seasonal moving average parameter. Note that currently only one seasonal moving average term is allowed in the discussions in this chapter. The
innovations ut are assumed to be i.i.d. random variables with distribution
F.
In practice, observed time series data are rarely well behaved as assumed
in the REGARIMA model (17.1) and (17.2). An observed time series yt∗ isusually some kind of variant of yt in equation (17.1). When the observed
time series yt∗ might be influenced by some outliers, the classical maximum
likelihood estimates as implemented in the S-PLUS function arima.mle are
not robust. In contrast, the S+FinMetrics function arima.rob allows therobust estimation of the model parameters (β, λ), where λ = (φ, θ, θ∗ ), φ
is a vector of the autoregressive parameters and θ is a vector of the moving
average parameters. Furthermore, it will detect three kinds of outliers in
the original data yt∗ :Additive outliers (AO): An additive outlier occurs at time t0 if yt∗0 =yt0 + c, where c is a constant. The eﬀect of this type of outlier isrestricted to the time period t0 .

17.3 Robust Fitting of REGARIMA Models

637

Innovation outliers (IO): An innovation outlier occurs at time t0 if ut0 =vt0 + c, where vt0 is generated by the distribution F . Usually it isassumed that F is the normal distribution N (0, σ 2 ). Note that theeﬀect of an innovation outlier is not restricted to time t0 becauseof the structure of an ARIMA model. It also has influence on thesubsequent observations.
Level shifts (LS): If one level shift occurs at time t0 , the observed series
is yt∗ = yt + c for all t ≥ t0 , with c being a constant. Note that if the∗series yt∗ has a level shift at t0 , the diﬀerenced series yt∗ − yt−1
has anadditive outlier at t0 .
In all those three cases c is the size of the outlier or level shift. Without
any potential confusion, the general term “outlier” may refer to any of thethree types of behavior.

17.3 Robust Fitting of REGARIMA Models
The S+FinMetrics function arima.rob computes the so-called “filtered τ estimates” of the parameters (β, λ, σ) of REGARIMA model (17.1)-(17.2)
when a time series is influenced by outliers. The technical details of thistype of estimation can be found in Section 17.6.
S+FinMetrics comes with a “timeSeries” data frip.dat, which represents monthly industrial production of France from January 1960 to December 1989. This data set will be used to illustrate the usage of arima.rob
function. First, a plot of the data will show the general properties of thetime series:> plot(frip.dat)
A few characteristics of the time series can be seen from Figure 17.1: (i)there are three big outliers around 1963 and 1968; (ii) it appears that alevel shift happened around 1975; (iii) there is an obvious trend in the timeseries, and the trend looks like a exponential one, especially in the last five
years. For diagnostic purpose, a robust ARIMA(2,1,0) model can be triedon the logarithm of frip.dat, due to the exponential-looking trend:> frip.rr = arima.rob(log(frip.dat)~1, p=2, d=1)
Note that the arima.rob function has only one required argument: aformula specifying the regression model. The optional argument p specifies
the autoregressive order, and d specifies the order of diﬀerence. In this case,the only predictor variable is the intercept term.
Caveat: The interpretation of the intercept term in arima.rob
is diﬀerent from that for other formulas in S-PLUS. When both d638

17. Robust Change Detection

40

50

60

70

80

90

Monthly Industrial Production of France

1960

1965

1970

1975

1980

1985

1990

FIGURE 17.1. Monthly industrial production of France.

and sd (seasonal diﬀerence) are zero (which is the default), theintercept is the constant term as usual. However, when either
d or sd is positive, the intercept is the coeﬃcient of the lowest
order time trend that can be identified. For instance, in theabove example, the intercept corresponds to the coeﬃcient of
the trend term t. One can easily verify this using the following
command:> frip.t = 1:length(frip.dat)> tmp = arima.rob(log(frip.dat)~frip.t-1, p=2, d=1)
which should give the same fit as frip.rr. The reason for thismodification is obvious: some coeﬃcients are not identifiable
when diﬀerencing is involved.
The object returned by the function arima.rob is of class “arima.rob”,
which has print and summary methods, just like most modeling objects.
For “arima.rob” objects, there is one additional advantage of using thesummary method instead of the print method: if the data object is of class“timeSeries”, the outliers will be lined up in a table with the time stampsof the observations, the types of the outliers, the impacts of the outliers,
and the t-statistics. For example,> summary(frip.rr)

17.3 Robust Fitting of REGARIMA Models

Call:arima.rob(formula = log(frip.dat) ~ 1, p = 2, d = 1)
Regression model:log(frip.dat) ~ 1
ARIMA model:
Ordinary differences: 1 ; AR order: 2 ; MA order: 0
Regression Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0024 0.0005
4.6558 0.0000
AR Coefficients:
Value Std. Error t value Pr(>|t|)AR(1) -0.3099 0.0537-5.7742 0.0000
AR(2) -0.0929 0.0537-1.7310 0.0843
Degrees of freedom: 360 total; 356 residual
Innovations standard deviation: 0.01311
Number of outliers detected:

9Outliers detected:
|Time
|Type
|Impact |t-value|-------+--------+-------+--------+-------+
1|Mar 1963|AO|-0.1457 |13.76 |-------+--------+-------+--------+-------+
2|May 1968|AO|-0.3978 |38.1|-------+--------+-------+--------+-------+
3|Jun 1968|AO|-0.1541 |14.55 |-------+--------+-------+--------+-------+
4|Sep 1968|AO|-0.04516| 4.41 |-------+--------+-------+--------+-------+
5|Apr 1969|LS| 0.04511| 3.814 |-------+--------+-------+--------+-------+
6|Sep 1974|LS|-0.04351| 3.767 |-------+--------+-------+--------+-------+
7|Nov 1974|LS|-0.04844| 4.092 |-------+--------+-------+--------+-------+
8|Sep 1976|AO| 0.0382 | 3.829 |-------+--------+-------+--------+-------+

639

640

17. Robust Change Detection

9|Apr 1986|AO| 0.03935| 3.932 |-------+--------+-------+--------+-------+
Innovation scale estimate before correcting outliers:
0.01311
Innovation scale estimate after correcting outliers:
0.01215
The output generated by the summary method actually has two sections.
The first section contains the parameter estimates in the REGARIMA
model. In this section, one can see that the intercept (which, again, is
actually the slope of the first order time trend) and the first autoregressive
coeﬃcient are very significant (that is, they have very small p-values), while
the second autoregressive coeﬃcient is not very significant.
The second section contains a summary of the outliers automaticallydetected by the arima.rob function. In this case, nine outliers are found:the first four and the last two are additive outliers, while the middle three
are level shifts. The three additive outliers shown in Figure 17.1 are all
detected with very large t-statistics.
A picture is always better than a thousand words. A visual diagnostic of
the model fit frip.rr can be obtained by using the generic plot function:> plot(frip.rr)
Make a plot selection (or 0 to exit):
1: plot: all
2: plot: Robust ACF of Innov.
3: plot: Robust PACF of Innov.
4: plot: Normal QQ-Plot of Innov.
5: plot: Original and Cleaned Series
6: plot: Detected Outliers
Selection:
Selections 2 and 3 will plot the robustly estimated autocorrelations andpartial autocorrelations of the innovations ut , respectively. Selection 4 produces the normal qq-plot of the innovations, as shown in Figure 17.2, fromwhich one can see that the three additive outliers are far away from thebulk of the data. Selection 5 plots the original response time series together
with the series obtained by cleaning the original series of additive outliers
using a robust filter, which is shown in Figure 17.3. Finally, Selection 6
plots the detected outliers, as shown in Figure 17.4.

17.3 Robust Fitting of REGARIMA Models

Normal Q-Q Plot of Innov.

frip.rr
100
0.2

0.1

Innov.

0.0

-0.1
36
-0.2

-0.3

-0.4 98
-3

-2

-1

0

1

2

3

Quantile of Standard Normal

FIGURE 17.2. Normal qq-plot of robust innovations.

Original and Cleaned Series

4.0

4.2

4.4

4.6

cleaned

3.8

Values

3.8

4.0

4.2

4.4

4.6

original

1960

1965

1970

1975

1980

1985

1990

Time

FIGURE 17.3. Original and cleaned series of frip.dat.

641

642

17. Robust Change Detection

Impacts of Outliers

Values

-0.3

-0.2

-0.1

0.0

Level Shifts

-0.3

-0.2

-0.1

0.0

Add. Outliers

1960

1965

1970

1975

1980

1985

1990

Time

FIGURE 17.4. Detected outliers in frip.dat.

17.4 Prediction Using REGARIMA Models
One of the main applications of a REGARIMA model is to predict future
values of response variable yt based on past values of yt and the corresponding future values of xt . If future predictions are intended, then the call tothe arima.rob function should specify the optional argument n.predict.
This argument should be set to a number equal or greater than the numberof predictions, the default of which is set to 20.
Prediction from REGARIMA models will be illustrated using the data
set import.dat in S+FinMetrics, which contains two monthly time series
from January 1983 to December 1990. The first series taxes corresponds toArgentinian import taxes and the second import to Argentinian imports.
Another data frame newtaxes.dat contains the values of the variable taxes
from January 1992 to October 1992. First fit a REGARIMA model with
ARIMA(2,1,0) errors:> import.rr = arima.rob(import~taxes-1, data=import.dat,+ p=2, d=1)
Now with the new data of the predictor variable taxes in newtaxes.dat,
one can predict import from January 1992 to October 1992 as follows:> import.hat = predict(import.rr,10,newdata=newtaxes.dat,se=T)> class(import.hat)

17.5 Controlling Robust Fitting of REGARIMA Models

643

values

300

400

500

600

import

0

20

40

60

80

100

index

FIGURE 17.5. Fitted values ± 2· standard deviation.

[1] "forecast"> names(import.hat)[1] "values" "std.err"
The optional argument se=T to the predict method tells the procedure to
return the standard errors of the forecasts. The returned object import.hat
is a “forecast” object with two components: values are the predictedvalues of import, and std.err are the standard errors of the prediction.
Since import.hat is a “forecast” object, as we have seen from earlier
chapters, the predictions can be easily plotted together with the originaldata:> plot(import.hat, import.dat[, "import"])
The plot is shown in Figure 17.5.

17.5 Controlling Robust Fitting of REGARIMA
Models
17.5.1 Adding Seasonal Eﬀects
The arima.rob function allows for two kinds of seasonal eﬀects options: theorder of seasonal diﬀerence and the inclusion of a seasonal moving average

644

17. Robust Change Detection

term, controlled by the optional arguments sd and sma, respectively. For
example, frip.dat is a monthly series, and you might expect that there
are some seasonal eﬀects in the series. Toward this end, you can add aseasonal moving average term by specifying the optional argument sma:> frip.srr = arima.rob(log(frip.dat)~1, p=2, d=1, sfreq=12,+ sma=T)> summary(frip.srr)
Call:arima.rob(formula = log(frip.dat) ~ 1, p = 2, d = 1,
sfreq = 12, sma = T)
Regression model:log(frip.dat) ~ 1
ARIMA model:
Ordinary differences: 1 ; AR order: 2 ; MA order: 0
Seasonal differences: 0 ; Seasonal period: 12 ; Seasonal MA: 1
Regression Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0024 0.0004
5.3946 0.0000
AR Coefficients:
Value Std. Error t value Pr(>|t|)AR(1) -0.3135 0.0518-6.0494 0.0000
AR(2) -0.1124 0.0518-2.1697 0.0307
Seasonal MA Coefficient:
Value Std. Error t value Pr(>|t|)[1,] 0.0945 0.0519
1.8208 0.0695
Degrees of freedom: 360 total; 355 residual
Innovations standard deviation: 0.01304
Number of outliers detected:

10Outliers detected:
|Time
|Type
|Impact |t-value|-------+--------+-------+--------+-------+
1|Mar 1963|AO|-0.1438 |13.58 |-------+--------+-------+--------+-------+

17.5 Controlling Robust Fitting of REGARIMA Models

6452|May 1963|LS| 0.03988| 3.545 |-------+--------+-------+--------+-------+
3|May 1968|AO|-0.3952 |38.67 |-------+--------+-------+--------+-------+
4|Jun 1968|AO|-0.1519 |14.27 |-------+--------+-------+--------+-------+
5|Sep 1968|AO|-0.04653| 4.615 |-------+--------+-------+--------+-------+
6|Apr 1969|LS| 0.04602| 4.005 |-------+--------+-------+--------+-------+
7|Sep 1974|LS|-0.04247| 3.739 |-------+--------+-------+--------+-------+
8|Nov 1974|LS|-0.04914| 4.24 |-------+--------+-------+--------+-------+
9|Sep 1976|AO| 0.038 | 3.891 |-------+--------+-------+--------+-------+
10|Apr 1986|AO| 0.03792| 3.946 |-------+--------+-------+--------+-------+
Innovation scale estimate before correcting outliers:
0.01304
Innovation scale estimate after correcting outliers:
0.01199
From the first section of the summary, one can see that the seasonal
moving average term is relatively significant, and the estimates of otherparameters are not altered very much. However, in the second section of
the summary, one more level shift is detected, which corresponds to May
1963.

17.5.2 Controlling Outlier Detection
The outlier detection procedure used in arima.rob is similar to those proposed by Chang, Tiao and Chen (1988) and Tsay (1988) for ARIMA models, and the one used in the X12-REGARIMA program of U.S. Census
Bureau. The main diﬀerence with those procedures is that arima.rob uses
innovation residuals based on the filtered τ -estimates of β and λ, insteadof the classical maximum likelihood estimates.
To detect the presence of an outlier at a given time t0 , the outlier detection procedure in arima.rob computes:
T = max max{Tt0 ,AO , Tt0 ,LS , Tt0 ,IO },
t0

646

17. Robust Change Detection

where Tt0 ,AO , Tt0 ,LS and Tt0 ,IO are the statistics corresponding to an AO,
LS and IO at time t0 respectively. The test statistic is defined as follows:
T =

|ω̂|

V̂ (ω̂)1/2,

where ω̂ is an estimate of ω, the size of the outlier, based on the residualsof the filtered τ -estimates and V̂ (ω̂) an estimate of its variance. If T > ξ,
where ξ is a conveniently chosen critical value, one declares that there isan outlier. The time t0 where the outlier occurs and the type of the outlier
are those where the double maximum is attained.
The critical value ξ is similar to the constant used by Chang, Tiao, andChen (1988). They recommend using ξ = 3 for high sensitivity in outlier
detection, ξ = 3.5 for medium sensitivity and ξ = 4 for low sensitivity,
when the length of the series is less than 200. For arima.rob the criticalvalue ξ is specified by the optional argument critv. The default value ofcritv is set as follows:⎧
if T ≤ 200,
⎨ 3
3.5 if 200 < T ≤ 500,
ξ=⎩4
if T > 500.

More details of this procedure can be found in Bianco, Garcia Ben, Martinez, and Yohai (1996, 2001).
So far none of the outliers detected is an innovation outlier. This is not acoincidence. By default, the outlier detection procedure in arima.rob does
not consider innovation outliers. To allow for innovation outliers, use theoptional argument innov.outlier:> frip.nrr = arima.rob(log(frip.dat)~1, p=2, d=1, sma=T,+ sfreq=12, innov.outlier=T)

S+FinMetrics also provides a function outliers to extract the information of the detected outliers from an “arima.rob” object. The object
returned by outliers is of class “outliers”. The methods print andsummary are available for an “outliers” object. For example,> summary(outliers(frip.nrr))
Number of outliers detected:

10Outliers detected:
|Time
|Type
|Impact |t-value|-------+--------+-------+--------+-------+
1|Mar 1963|AO|-0.1438 |13.58 |-------+--------+-------+--------+-------+
2|May 1963|LS| 0.03988| 3.545 |17.5 Controlling Robust Fitting of REGARIMA Models

647-------+--------+-------+--------+-------+
3|May 1968|AO|-0.3952 |38.67 |-------+--------+-------+--------+-------+
4|Jun 1968|AO|-0.1519 |14.27 |-------+--------+-------+--------+-------+
5|Sep 1968|AO|-0.04653| 4.615 |-------+--------+-------+--------+-------+
6|Apr 1969|LS| 0.04602| 4.005 |-------+--------+-------+--------+-------+
7|Sep 1974|LS|-0.04247| 3.739 |-------+--------+-------+--------+-------+
8|Nov 1974|LS|-0.04914| 4.24 |-------+--------+-------+--------+-------+
9|Sep 1976|AO| 0.038 | 3.891 |-------+--------+-------+--------+-------+
10|Apr 1986|AO| 0.03792| 3.946 |-------+--------+-------+--------+-------+
Innovation scale estimate before correcting outliers:
0.01304
Innovation scale estimate after correcting outliers:
0.01199
In this case, still no innovation outlier is detected even though we allowed
for innovation outliers.

17.5.3 Iterating the Procedure
After the outlier detection, one can clean the original series of additive
outliers and level shifts. If all the outliers in the data have been detected,
and arima.rob is called on the cleaned data again, one should not find any
new outliers. By this line of argument, the process of robust estimation andoutlier detection can be iterated to obtain a more thorough detection ofoutliers. Before illustrating how this can be done using arima.rob function,
we want to warn that this procedure is ad hoc, and sometimes the results
may not be easily interpretable.
To carry out the iteration process, simply set the optional argument
iter=T when calling arima.rob function. For example,> frip.irr = arima.rob(log(frip.dat)~1, p=2, d=1, iter=T)> summary(frip.irr)
Call:arima.rob(formula = log(frip.dat) ~ 1, p = 2, d = 1, iter = T)

648

17. Robust Change Detection

Regression model:log(frip.dat) ~ 1
ARIMA model:
Ordinary differences: 1 ; AR order: 2 ; MA order: 0
Regression Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0023 0.0005
4.6027 0.0000
AR Coefficients:
Value Std. Error t value Pr(>|t|)AR(1) -0.2861 0.0577-4.9542 0.0000
AR(2) -0.0728 0.0577-1.2608 0.2082
Degrees of freedom: 360 total; 356 residual
Innovations standard deviation: 0.01178
Number of outliers detected:

10Outliers detected:
|Time
|Type
|Impact |t-value|-------+--------+-------+--------+-------+
1|Mar 1963|AO|-0.1457 |13.76 |-------+--------+-------+--------+-------+
2|May 1968|AO|-0.3978 |38.1|-------+--------+-------+--------+-------+
3|Jun 1968|AO|-0.1541 |14.55 |-------+--------+-------+--------+-------+
4|Sep 1968|AO|-0.04516| 4.41 |-------+--------+-------+--------+-------+
5|Apr 1969|LS| 0.04511| 3.814 |-------+--------+-------+--------+-------+
6|Sep 1974|LS|-0.04351| 3.767 |-------+--------+-------+--------+-------+
7|Sep 1974|LS| 0.04162| 3.598 |-------+--------+-------+--------+-------+
8|Nov 1974|LS|-0.04844| 4.092 |-------+--------+-------+--------+-------+
9|Dec 1975|LS| 0.04037| 3.534 |-------+--------+-------+--------+-------+
10|Dec 1975|LS| 0.0414 | 3.619 |-------+--------+-------+--------+-------+

17.6 Algorithms of Filtered τ -Estimation

649

11|Sep 1976|AO| 0.0382 | 3.829 |-------+--------+-------+--------+-------+
12|Apr 1986|AO| 0.03935| 3.932 |-------+--------+-------+--------+-------+
Innovation scale estimate before correcting outliers:
0.01311
Innovation scale estimate after correcting outliers:
0.01176
In the first section of the output, the parameter estimates are from thelast iterated model. In the second section of the output, it is stated that 10
outliers have been detected altogether, though there are 12 outliers listed
in the table. The diﬀerence comes from the fact that some outliers aredetected repeatedly during the iteration process. For example, two level
shifts have been detected corresponding to September 1974.
To obtain a summary of the outliers detected for each iteration, one canuse the outliers function with an iter argument. For example,> summary(outliers(frip.irr, iter=2))
Number of outliers detected:

2

Outliers detected:
|Time
|Type
|Impact |t-value|-------+--------+-------+-------+-------+
1|Sep 1974|LS|0.04162|3.598 |-------+--------+-------+-------+-------+
2|Dec 1975|LS|0.04037|3.534 |-------+--------+-------+-------+-------+
Innovation scale estimate before correcting outliers:
0.01202
Innovation scale estimate after correcting outliers:
0.01176
which summarizes the outliers detected in the second iteration.

17.6 Algorithms of Filtered τ -Estimation
This section briefly introduces filtered τ -estimates for REGARIMA models.
The technical details can be found in the references cited in this chapter.

650

17. Robust Change Detection

17.6.1 Classical Maximum Likelihood Estimates
For the REGARIMA model in equations (17.1) and (17.2), the model parameters are usually estimated by maximum likelihood estimation (MLE).
The MLE can be computed using prediction error decomposition, for example, see Chapter 14.
First, let d0 = d + sD. Note that we will lose the first d0 observations
because of the ARIMA diﬀerencing and/or seasonal diﬀerencing. For the
moment, consider only equation (17.2). Let
ˆt|t−1 (λ) = Eλ [ t | 1 , . . . , t−1 ], for t ≥ d0
be the one-step-ahead predictor of t given the knowledge of historic valuesof t . Thenût (λ) = t − ˆt|t−1 (λ)(17.3)

will be the one-step-ahead prediction error, and the variance of ût (λ) is of
the form
σ 2t (λ) = Eλ [ t − ˆt|t−1 (λ)]2 = a2t (λ)σ 2u ,where limt→∞ at (λ) = 1.
Second, for the REGARIMA model considered, the prediction error ût (β, λ)can be obtained similarly as in equation (17.3), replacing t with t (β) =yt − x0t β.
Now, let L(β, λ, σ 2 ) be the conditional likelihood function of the sample
observations, and letQ(β, λ) = −2 argmax log L(β, λ),σ2which is −2 times the log-likelihood concentrated with respect to σ2 . Using
prediction error decomposition, it can be easily shown that
Q(β, λ) =T
X

t=d0 +1

log a2t (λ) + (T − d0 )s2 (ûd0 +1 (β, λ)ûT (β, λ),...,),ad0 +1 (λ)
aT (λ)(17.4)

up to a constant, wherens2 (u1 , . . . , uT ) =1X 2
un t=1 t(17.5)

is the square of the scale estimate.
The classical maximum likelihood estimates of β and λ are obtained byminimizing Q(β, λ), that is,(β̂, λ̂) = argmin Q(β, λ),β ,λand the maximum likelihood estimate of σ 2 is given byσ̂ 2 = s2 (β̂, λ̂).

17.7 References651

17.6.2 Filtered τ -Estimates
It is well known that the classical maximum likelihood estimates in theprevious section are not robust and can produce poor estimates when thedata contain outliers. Bianco, Garcia Ben, Martinez, and Yohai (1996) proposed a class of robust estimates for REGARIMA model called filtered τ estimates. See also Martin and Yohai (1996). These estimates are based ona robustification of the log-likelihood function. The robustification is accomplished through two steps: (1) use the filtered prediction error insteadof the usual prediction error; (2) use a robust τ -estimate of the scale inequation (17.4).
The filtered τ -estimation uses a robust filter proposed by Masreliesz(1975) which eliminates the influence of previous outliers or bad observations. That is, the robust prediction error ˜t|t−1 is computed based oncleaned series ˜t|t instead of the contaminated series t . For an AR(1) model,
the two series ˜t|t−1 and ˜t|t are obtained simultaneously by a recursion procedure as follow:
˜t|t = wt t + (1 − wt )˜t|t−1 ,where| t − ˜t|t−1 |)mσ̂ tand w(·) is an even and non-increasing weight function, m is a tuning constant, and σ̂ 2t is an estimate of the prediction variance σ2t . For the generalcase the robust filtering procedure is based on the state space representation of the ARIMA model. The details can be found in Martin, Samarov,and Vandaele (1983).
The filtered τ -estimation replaces the statistic s2 in equation (17.5) witha robust τ -estimate of scale. For details of how τ -estimates of scale can becomputed, see Yohai and Zamar (1983).
In summary, the filtered τ -estimates are defined by
wt = w((β̂, λ̂) = argmin Q∗ (β, λ),
β ,λ

where
Q∗ (β, λ) =T
X

t=d0 +1

log a2t (λ) + (T − d0 )τ 2 (ũd0 +1 (β, λ)ũT (β, λ),...,),ad0 +1 (λ)
aT (λ)

with ũt = t − ˜t|t−1 , and τ 2 (·) is the square of the τ -estimate of scale.

17.7 References
Bianco, A., Garcia Ben, M., Martinez, E., and Yohai, V. (1996).
“Robust Procedure for Regression Models with ARIMA Errors,” in A.

652

17. Robust Change Detection

Prat (ed.) COMPSTAT 96 Proceedings Computational Statistics. PhysicaVerlag, Heidelberg.
Bianco, A., Garcia Ben, M., Martinez, E., and Yohai, V. (2001).
“Outlier Detection in Regression Rodels with ARIMA Errors Using Robust
Estimates,” Journal of Forecasting, 20, 565-579.
Box, G., and Jenkins, G. (1976). Time Series Analysis: Forecasting andControl. Holden-Day, San Francisco.
Chang, I., Tiao, G. C., and Chen. C (1988). “Estimation of Time Series
Parameters in the Presence of Outliers,” Technometrics, 30, 193-204.
Martin, R. D., Samarov, A., and Vandaele, W. (1983). “Robust
Methods for ARIMA Models,” in A. Zellner (ed.) Applied Time Series
Analysis of Economic Data. U.S. Census Bureau, Government Printing
Oﬃce.
Martin, R.D., and V. J. Yohai (1996). “Highly Robust Estimation ofAutoregressive Integrated Time Series Models,” Publicaciones Previas No.
89, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires.
Masreliesz, C. J. (1975). “Approximate non-Gaussian Filtering with Linear State and Observation Relations,” IEEE Transactions on AutomaticControl, AC-20, 107-110.
Tsay, R. S. (1988). “Outliers, Level Shifts and Variance Changes in Time
Series,” Journal of Forecasting, 7, 1-20.
Yohai, V. J., and Zamar, R. H. (1988). “High Breakdown-Point Estimates of Regression by Means of the Minimization of an Eﬃcient Scale,”
Journal of the American Statistical Association, 83, 406-413.

This is page 653
Printer: Opaque this18
Nonlinear Time Series Models

18.1 Introduction
Most of the time series models discussed in the previous chapters are linear time series models. Although they remain at the forefront of academic
and applied research, it has often been found that simple linear time seriesmodels usually leave certain aspects of economic and financial data unexplained. Since economic and financial systems are known to go throughboth structural and behavioral changes, it is reasonable to assume that different time series models may be required to explain the empirical data at
diﬀerent times. This chapter introduces some popular nonlinear time seriesmodels that have been found to be eﬀective at modeling nonlinear behavior
in economic and financial time series data.
To model nonlinear behavior in economic and financial time series, it
seems natural to allow for the existence of diﬀerent states of the world or
regimes and to allow the dynamics to be diﬀerent in diﬀerent regimes. This
chapter focuses on models that assume in each regime that the dynamic
behavior of the time series is determined by an autoregressive (AR) model,such as threshold AR, self-exciting threshold AR, and smooth transition
AR models. This is because simple AR models are arguably the most popular time series model and are easily estimated using regression methods. By
extending AR models to allow for nonlinear behavior, the resulting nonlinear models are easy to understand and interpret. In addition, this chapter
also covers more general Markov switching models using state space representations. The types of model that can be cast into this form are enormous.

654

18. Nonlinear Time Series Models

However, there are many other types of nonlinear time series model thatare not covered in this chapter, such as bilinear models, k nearest neighbormethods, and neural network models.1 Book length treatment of nonlinear
time series models can be found in Tong (1990), Granger and Teräsvirta(1993), and Franses and van Dijk (2000). Kim and Nelson (1999) provides
a comprehensive account of diﬀerent Markov switching models that havebeen used in economic and financial research.
Given the wide range of nonlinear time series models available and theinherent flexibility of these models, the possibility of getting a spuriously
good fit to any time series data set is very high. Therefore, it is usually recommended to perform a test of linearity against nonlinearity before building
a possibly complex nonlinear model. Section 18.2 first introduces a popular
test for nonlinearity, the BDS test, which has been found to have power
against a wide range of nonlinear time series models. There are many othertypes of nonlinearity test that are developed to test against specific nonlinear models. Some of these tests will be introduced together with thenonlinear models in later sections. For example, Section 18.3 introduces
threshold AR models and two tests for threshold nonlinearity, and Section 18.4 introduces smooth transition AR (STAR) models and a test forSTAR nonlinearity. Finally, Section 18.5 describes the Markov switchingstate space models and Section 18.6 gives an extended example of how toestimate Markov switching models in S+FinMetrics.

18.2 BDS Test for Nonlinearity
The BDS test developed by Brock, Dechert, and Scheinkman (1987) (andlater published as Brock, Dechert, Scheinkman, and LeBaron, 1996) is arguably the most popular test for nonlinearity. It was originally designed
to test for the null hypothesis of independent and identical distribution(iid) for the purpose of detecting non random chaotic dynamics.2 However, many studies have shown that the BDS test has power against a wide
range of linear and nonlinear alternatives; for example, see Brock, Hsieh,and LeBaron (1991) and Barnett, Gallant, Hinich, Jungeilges, Kaplan, andJensen (1997). In addition, it can also be used as a portmanteau test or
misspecification test when applied to the residuals from a fitted model. In
particular, when applied to the residuals from a fitted linear time seriesmodel, the BDS test can be used to detect remaining dependence and thepresence of an omitted nonlinear structure. If the null hypothesis cannot
be rejected, then the original linear model cannot be rejected; if the null
1 A function to estimate single-hidden-layer neural network models is in the nnet
library provided with S-PLUS.
2 Loosely speaking, a time series is said to be “chaotic” if it follows a nonlinear deterministic process but looks random.

18.2 BDS Test for Nonlinearity655

hypothesis is rejected, the fitted linear model is misspecified, and in thissense, it can also be treated as a test for nonlinearity.

18.2.1 BDS Test Statistic
The main concept behind the BDS test is the correlation integral, which
is a measure of the frequency with which temporal patterns are repeatedin the data. Consider a time series xt for t = 1, 2, . . . , T and define its
m-history as xm
t = (xt , xt−1 , . . . , xt−m+1 ). The correlation integral at embedding dimension m can be estimated by
XX
2
m
Cm, =
I(xm
(18.1)
t , xs ; )Tm (Tm − 1)m≤s<t≤Tmwhere Tm = T − m + 1 and I(xmt , xs ; ) is an indicator function that isequal to one if |xt−i − xs−i | < for i = 0, 1, . . . , m − 1 and zero otherwise.
Intuitively, the correlation integral estimates the probability that any two
m-dimensional points are within a distance of of each other; that is, itestimates the joint probability:

Pr(|xt − xs | < , |xt−1 − xs−1 | < , . . . , |xt−m+1 < xs−m+1 | < )If xt are iid, this probability should be equal to the following in the limiting
case:
m
C1,
= Pr(|xt − xs | < )mBrock, Dechert, Scheinkman, and LeBaron (1996) define the BDS statistic
as follows:m√ Cm, − C1,
Vm, = T
(18.2)sm,√mwhere sm, is the standard deviation of T (Cm, − C1,) and can be estimated consistently as documented by Brock, Dechert, Scheinkman, and
LeBaron (1997). Under fairly moderate regularity conditions, the BDS
statistic converges in distribution to N (0, 1)dVm, → N (0, 1)(18.3)

so the null hypothesis of iid is rejected at the 5% significance level whenever|Vm, | > 1.96.

18.2.2 Size of BDS Test
S+FinMetrics provides the BDSTest function for performing the BDS test.3
The arguments expected by BDSTest function are:
3 The BDSTest function is implemented using the C source file provided by LeBaron(1997). The same test can also be performed by calling nonlinearTest function withthe optional argument method set to "BDS".

18. Nonlinear Time Series Models

0.6
0.4

m=2
m=3
m=4
m=5

0.2

Monte Carlo Size

0.8

656

0.2

0.4

0.6

0.8

Nominal Size

FIGURE 18.1. Size of BDS test statistics using t distribution.

> args(BDSTest)
function(x, m = 3, eps = NULL, variable.removal = T)
where x specifies the time series to be tested, m instructs the test to use
the embedding dimensions from 2 to m, and eps specifies in units of sample
standard deviations the distance threshold in (18.1). By default, BDSTest
computes the BDS statistics with set to 0.5, 1, 1.5, and 2 standard deviations of the data set. When the optional argument variable.removal
is set to TRUE, diﬀerent numbers of points in the sample are removed for
diﬀerent values of m such that the test is always computed using all of
the sample observations available; if it is set to FALSE, the same points are
removed for diﬀerent values of m such that the test is always computed
using the same sample observations.
Example 116 Size of BDS test
The following script illustrates how to use the BDSTest function in a
Monte Carlo experiment to investigate the size of the BDS test:
set.seed(10)size.mat = matrix(0, 1000, 4)
for (i in 1:1000) {if (i %% 100 == 0) {cat("i =", i, "\n")}18.2 BDS Test for Nonlinearity657

test.dat = rt(500, df=8)
size.mat[i,] = BDSTest(test.dat, m=5, eps=1)$stat[,1]}One advantage of the BDS test is that it is a statistic that requires no
distributional assumption on the data to be tested. In fact, in the above
Monte Carlo experiment, the data are simulated from a t distribution with
8 degrees of freedom. Each simulated sample has 500 observations, which
is usually thought to be the minimal sample size for the BDS test to havereliable performance. The data are simulated 1000 times and BDS statistics
using embedding dimensions from 2 to 5 are computed by setting to one
standard deviation of the sample observations. The following commands
plot the size of the “one-sided” test against its nominal value4 :> size.p = seq(0.05, 0.95, by=0.05)> size.q = qnorm(size.p)> size.bds = apply(size.mat, 2,+function(x) colMeans(outer(x, size.q, FUN="<=")))> par(fty="s")> matplot(matrix(size.p, nrow=length(size.p), ncol=4),+
size.bds, type="l",+xlab="Nominal Size", ylab="Monte Carlo Size")> legend(0.6, 0.3, paste("m=",2:5,sep=""), type="l", lty=1:4)
and the result is shown in Figure 18.1. Considering that the Monte Carloexperiment is conducted using only 1000 replications, the plot shows thatthe test has very good size behavior for all of the chosen embedding dimensions.

18.2.3 BDS Test as a Nonlinearity Test and a
Misspecification Test
Another advantage of the BDS test is that when applied to model residuals,
the first-order asymptotic distribution of BDS statistic given in (18.3) isindependent of estimation errors under certain suﬃcient conditions. In general, de Lima (1996) shows that for linear additive models, or models thatcan be transformed into that format, the BDS test is nuisance-parameterfree and does not require any adjustment when applied to fitted model
residuals. Thus, the BDS test can be used as a test for nonlinearity or as atest for model misspecification.
Example 117 Nonlinearity in weekly returns of Dutch Guilder foreign exchange rates
4 The BDS test is actually a two-sided test. However, for the purpose of illustrating
distributional properties of BDS statistics, the plots are generated using the “incorrect”
one-sided test.

658

18. Nonlinear Time Series Models

The "timeSeries" data set DFX.ts in S+FinMetrics contains weekly
returns on the Dutch Guilder spot exchange rate from January 1980 to
December 1998. To test for the existence of nonlinearity in this data set,
use the following command:> BDSTest(DFX.ts, m=5)
BDS Test for Independence and Identical Distribution
Null Hypothesis: DFX.ts is independently and identically
distributed.
Embedding dimension =2 3 4 5

Epsilon for close points =0.0073 0.0146 0.0219 0.0291

Test Statistics =[ 0.01 ] [ 0.01 ] [ 0.02 ] [ 0.03 ][ 2 ]1.08021.59081.99912.6097[ 3 ]3.1661
3.09843.5817
4.1536[ 4 ]4.05233.90064.48715.1613[ 5 ]5.27984.71895.32385.9882p-value =[ 0.01 ] [ 0.01 ] [ 0.02 ] [ 0.03 ][ 2 ]0.28010.11170.04560.0091[ 3 ]0.00150.00190.00030.0000[ 4 ]0.00010.00010.00000.0000[ 5 ]0.00000.00000.00000.0000In the above output, the default values of = (0.5, 1.0, 1.5, 2.0) used in
the test are converted back to the units of the original data, and the null
hypothesis that the data are iid is rejected for most combinations of m andat conventional significance levels. Since there is almost no discernible
linear structure in the levels of DFX.ts, the results from the BDS test
suggest that there may be nonlinear structure in the data.
One possibility to model the nonlinear structure in DFX.ts is to use a
GARCH(1,1) model:> DFX.garch = garch(DFX.ts~1, ~garch(1,1), trace=F)> summary(DFX.garch)$coef
Value
Std.Errort value
Pr(>|t|)C 0.00021084425 3.939145e-004 0.5352539 5.925817e-001
A 0.00001942582 5.508377e-006 3.5265964 4.381551e-004
ARCH(1) 0.10297320531 2.096693e-002 4.9112210 1.041116e-006
GARCH(1) 0.80686268689 3.798031e-002 21.2442379 0.000000e+000

18.2 BDS Test for Nonlinearity659

All of the estimated parameters in DFX.garch are highly significant except for the conditional mean parameter C. To evaluate if the GARCH(1,1)
model adequately captures the nonlinear structure in DFX.ts, the BDS test
can be used again on the standardized residuals of DFX.garch as a misspecification test. There are two ways to apply the BDS test to GARCH
standardized residuals: One is to apply the BDS test directly to the standardized residuals:> BDSTest(residuals(DFX.garch, standard=T), m=5,
+ eps=c(0.5, 1, 1.5))
BDS Test for Independence and Identical Distribution
Null Hypothesis: residuals(DFX.garch, standard = T) is
independently and identically distributed.
Embedding dimension =2 3 4 5

Epsilon for close points =0.5002 1.0004 1.5006

Test Statistics =[ 0.5 ]
[ 1 ] [ 1.5 ][ 2 ] -1.9487 -1.5430 -1.6035[ 3 ] -1.4581 -1.1172 -1.2687[ 4 ] -1.2832 -0.9735 -1.1355[ 5 ] -0.8634 -0.6079 -0.8305
p-value =[ 0.5 ] [ 1 ] [ 1.5 ][ 2 ] 0.0513 0.1228 0.1088[ 3 ] 0.1448 0.2639 0.2045[ 4 ] 0.1994 0.3303 0.2561[ 5 ] 0.3879 0.5432 0.4062
and the other is to apply it to the logarithms of squared standardized
residuals5 :> BDSTest(log(residuals(DFX.garch, standard=T)^2),+ m=5, eps=c(0.5, 1, 1.5))
BDS Test for Independence and Identical Distribution

5 When the BDSTest function is applied to a fitted model object, it is currently always
applied to the residuals of the fittd model, instead of standardized residuals or logarithms
of squared standardized residuals.

660

18. Nonlinear Time Series Models

Null Hypothesis: log(residuals(DFX.garch, standard = T)^2)
is independently and identically distributed.
Embedding dimension =2 3 4 5

Epsilon for close points =1.1218 2.2435 3.3653

Test Statistics =[ 1.12 ] [ 2.24 ] [ 3.37 ][ 2 ] -0.6461 -0.5538 -0.5463[ 3 ] -0.8508 -0.9030 -0.9175[ 4 ] -0.7540 -0.9977 -1.0821[ 5 ] -0.9397 -0.8581 -1.0252
p-value =[ 1.12 ] [ 2.24 ] [ 3.37 ][ 2 ]0.51820.57970.5849[ 3 ]0.39490.36650.3589[ 4 ]0.45090.31840.2792[ 5 ]0.34740.39090.3052
Here, both ways of applying the BDS test suggest that the GARCH(1,1)
model provides an adequate fit to the original data and successfully removes
the nonlinearity in the data. In general, when applied to standardized residuals from a fitted GARCH model, earlier studies (e.g., see Brock, Hsieh,and LeBaron, 1991) suggested that the BDS statistic needs to be adjusted
to have the right size and Monte Carlo simulations are usually relied upon
to derive the adjustment factor for specific GARCH models. However, following suggestions in Brock and Potter (1993) and de Lima (1996), recent
studies (e.g., see Caporale, Ntantamis, Pantelidis, and Pittis, 2004 and Fernandes and Preumont, 2002) showed that if applied to the logarithms ofsquared standardized residuals from a fitted GARCH model, the BDS test
actually has correct size, because the logarithmic transformation casts the
GARCH model into a linear additive model that satisfies the conditions inde Lima (1996) for the BDS test to be nuisance-parameter-free.6
Example 118 Size of BDS misspecification test for GARCH models
The following script performs a Monte Carlo experiment to illustrate thediﬀerent size behaviors of the BDS test when applied to standardized residuals and logarithms of squared standardized residuals for the GARCH(1,1)
model. The data sets are simulated using the GARCH fit in DFX.garch with
6 Since GARCH models with leverage eﬀects cannot be transformed into a linear
additive model, BDS test may not have good size behavior for those models.

661

0.6
0.4

m=2
m=3
m=4
m=5

0.2

Monte Carlo Size

0.8

18.2 BDS Test for Nonlinearity

0.2

0.4

0.6

0.8

Nominal Size

0.6
0.4

m=2
m=3
m=4
m=5

0.0

0.2

Monte Carlo Size

0.8

1.0

FIGURE 18.2. Size of the BDS test when applied to logarithms of squared standardized GARCH residuals.

0.2

0.4

0.6

0.8

Nominal Size

FIGURE 18.3. Size of the BDS test when applied to standardized GARCH residuals.

662

18. Nonlinear Time Series Models

1000 observations. The GARCH estimation and BDS test are repeated 1000
times.
set.seed(10)sim.garch.dat = simulate(DFX.garch, sigma=F, n.start=500,
n=1000, n.rep=1000)
size.garch.res = matrix(0, 1000, 4)
size.garch.log = matrix(0, 1000, 4)
for (i in 1:1000) {tmp = garch(sim.garch.dat[,i]~1, ~garch(1,1), trace=F)
if (i %% 10 == 0)cat("Simulation No.", i, "\n")tmp.res = residuals(tmp, standardized=T)
size.garch.res[i,] = BDSTest(tmp.res, m=5, eps=1)$stat[,1]size.garch.log[i,] = BDSTest(log(tmp.res^2), m=5,
eps=1)$stat[,1]}size.p = seq(0.05, 0.95, by=0.05)
size.q = qnorm(size.p)
size.garch.res = apply(size.garch.res, 2,function(x) colMeans(outer(x, size.q, FUN="<=")))
size.garch.log = apply(size.garch.log, 2,function(x) colMeans(outer(x, size.q, FUN="<=")))As in Example 116, the sizes of the “one-sided” test applied to the standardized residuals and the logarithms of squared standardized residuals areplotted against the nominal sizes in Figure 18.3 and Figure 18.2, respectively. Obviously the sizes of the BDS test computed using standardized
residuals are oﬀ and become more conservative for larger values of m, but
those using logarithms of squared standardized residuals are reliable.

18.3 Threshold Autoregressive Models
As discussed in the previous subsection, when there is no prior knowledge
about the type of nonlinearity a time series may have, the BDS test can beused to test for the existence of nonlinearity in either the time series itself
or the residuals from a fitted linear time series model. However, sometimes
economic or financial theory, or even stylized empirical facts, may suggest
a specific form of nonlinearity for a time series. In these cases, it is usually
preferred to perform the test for the specific form of nonlinearity and build
a nonlinear time series model for the form of nonlinearity detected.
One popular class of nonlinear time series models is the threshold autoregressive (TAR) models, which was probably first proposed by Tong (1978)and discussed in detail in Tong (1990). The TAR models are simple and18.3 Threshold Autoregressive Models

663

easy to understand, but rich enough to generate complex nonlinear dynamics. For example, it can be shown that the TAR models can have limit
cycles and thus be used to model periodic time series or produce asymmetries and jump phenomena that cannot be captured by a linear time seriesmodel.
In spite of the simplicity of the TAR model form, there are many freeparameters to estimate and variables to choose when building a TAR model,
and this has hindered its early use. Recently, however, much progress has
been made with regard to specification and estimation of TAR models. The
next subsection introduces the general form of TAR models and a specialclass called SETAR models and then illustrates how to perform tests forthreshold nonlinearity and estimate unknown parameters in TAR models
using ready-to-use functions in S+FinMetrics.

18.3.1 TAR and SETAR Models
Consider a simple AR(p) model for a time series7 yt :yt = μ + φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + σ t(18.4)

where φi (i = 1, 2, . . . , p) are the AR coeﬃcients, t ∼ W N (0, 1), and σ > 0is the standard deviation of disturbance term. The model parameters φ =(μ, φ1 , φ2 , . . . , φp ) and σ are independent of time t and remain constant.
To capture nonlinear dynamics, TAR models allow the model parameters
to change according to the value of a weakly exogenous threshold variable
zt :(18.5)
yt = Xt φ(j) + σ (j) t if rj−1 < zt ≤ rj
where Xt = (1, yt−1 , yt−2 , . . . , yt−p ), j = 1, 2, · · · , k, and −∞ = r0 < r1 <. . . < rk = ∞. In essence, the k − 1 nontrivial thresholds (r1 , r2 , . . . , rk−1 )divide the domain of the threshold variable zt into k diﬀerent regimes. In
each diﬀerent regime, the time series yt follows a diﬀerent AR(p) model.8
When the threshold variable zt = yt−d , with the delay parameter d being
a positive integer, the dynamics or regime of yt is determined by its own
lagged value yt−d and the TAR model is called a self-exciting TAR, or
SETAR model. For the ease of notation, let SETAR(1) denote the oneregime linear AR model with k = 1, SETAR(2) denote the two-regime
TAR model with k = 2, etc. For the one-regime SETAR(1) model, −∞ =r0 < r1 = ∞ and the unknown parameters are Θ = (φ(1) , σ (1) ); for the
7 See Chapter 3 and the references therein for basic concepts in linear time series
analysis.
8 Although the AR order p is assumed to be the same in diﬀerent regimes throughout
this chapter and in the related S+FinMetrics functions for the ease of illustration andprogramming, in theory the AR order can be diﬀerent for diﬀerent regimes.

664

18. Nonlinear Time Series Models

two-regime SETAR(2) model, the unknown parameters include the single
threshold −∞ < r1 < ∞ and Θ = (φ(1) , φ(2) , σ (1) , σ (2) ).
The next subsection introduces two approaches for testing threshold nonlinearity and estimating the unknown parameters in the associated SETAR
models, following Tsay (1989) and Hansen (1997), respectively. Although
the illustrations and examples focus on SETAR models, the theory andprocedures can also be applied to TAR models in general. Finally, notethat if only the intercept terms μ(j) are diﬀerent in diﬀerent regimes, SETAR models can be used to capture level shifts in yt ; if only the varianceterms σ (j) are diﬀerent in diﬀerent regimes, SETAR models can be used tocapture additive outliers or innovation outliers in yt . Chapter 17 provides
a more comprehensive approach for analyzing time series models that arerobust to level shifts and outliers.

18.3.2 Tsay’s Approach
Before developing a SETAR model, it is preferred to test for the existenceof threshold-type nonlinearity in the time series first. The null hypothesisis that usually the time series yt follows the SETAR(1) model, whereas thealternative hypothesis is that yt follows a SETAR(j) model with j > 1.
One complicating issue in testing for threshold nonlinearity is that thethresholds ri for i = 1, 2, · · · , k − 1 are only identified under the alternative hypothesis. To avoid dealing with the thresholds directly, Tsay (1989)
proposed a conventional F test based on an auxiliary regression.
Arranged Autoregression and Tsay’s F Test
Tsay’s approach centers on the use of an arranged autoregression withrecursive least squares (RLS) estimation. Consider the SETAR model in(18.5) with zt = yt−d . Since the threshold values ri are usually unknown,
Tsay suggested arranging the equations in (18.5) for t = max(d, p) +1, . . . , n, where n is the sample size, such that the equations are sorted
according to the threshold variable yt−d , which may take any value in
Yd = (yh , . . . , yn−d ) with h = max(1, p + 1 − d):yπi = Xπi φ̂ + σ̂ πi(18.6)

where i = 1, 2, . . . , n0 , n0 = n − d − h + 1 is the eﬀective sample sizefor the above arranged autoregression, and π i corresponds to the index in
the original sample such that yπi −d is the i-th smallest value in Yd . For
example, if y10 is the smallest value in Yd , then π 1 = 10 + d; if y20 is thesecond smallest value in Yd , then π 2 = 20 + d, etc. So if the original timeseries is generated by a SETAR(2) model and there are m < n values in Yd
that are smaller than the threshold r1 , then the first m equations in (18.6)
correspond to the first regime and the remaining equations correspond to
the second regime.

18.3 Threshold Autoregressive Models

665

To test for the existence of threshold-type nonlinearity, Tsay suggested
computing RLS estimates of φ̂ in (18.6). If there is no threshold nonlinearity, the standardized predictive residuals êπi from the RLS of (18.6) should
be white noise asymptotically and orthogonal to Xπi . However, if yt is a
SETAR(j) process with j > 1, the RLS estimates of φ̂ are biased, and Ψ̂
in the following auxiliary regression will be statistically significant:êπi = X0πi Ψ + uπi(18.7)

Thus, the conventional F statistic for testing Ψ = 0 the above regression
can be used as a test for threshold nonlinearity.
Example 119 SETAR nonlinearity in NASDAQ realized volatility
To illustrate the usage of Tsay’s F test for threshold nonlinearity, consider the weekly realized volatility of the NASDAQ 100 index constructed
as follows from the S+FinMetrics data set ndx.dat:> ndx.ret2 = getReturns(ndx.dat[,"Close"])^2> ndx.rvol = sqrt(aggregate(ndx.ret2, FUN=sum, by="weeks",+ week.align=1))> colIds(ndx.rvol) = "RVOL"> par(mfrow=c(2,2))> plot(ndx.rvol, reference.grid=F, main="RVOL")> plot(log(ndx.rvol), reference.grid=F, main="Log RVOL")
The levels and the logarithms of the weekly realized volatility series areshown in the top half of Figure 18.4. The time series plots suggest that thevolatility may have switched to a diﬀerent regime after the first quarter of
2000. Before testing for threshold nonlinearity, the ACF and PACF plots
can be used to help identify the AR order to use:
> ndx.acf = acf(log(ndx.rvol))> ndx.pacf = acf(log(ndx.rvol), type="partial")
The resulting plots are shown in the bottom half of Figure 18.4. The
ACF function decays very slowly and remains significant even after 30 lags,
whereas the PACF function is significant for the first 6 lags. This suggeststhat an AR model with order from 2 to 6 may be considered as a starting
point for modeling the logarithms of realized volatility log(ndx.rvol).9
The S+FinMetrics function nonlinearTest can now be used to test for
threshold nonlinearity:
9 Hereafter, the logarithms of ndx.rvol are used because usually the logarithms ofrealized volatility tend to be normally distributed. See Andersen, Bollerslev, Diebold,
and Ebens (2001) for a detailed analysis of properties of realized volatility for stock
returns.

666

18. Nonlinear Time Series Models

RVOL

0.02

-4.5

-3.5

0.10

-2.5

0.18

RVOL

Q1 Q3 Q1
1996 1997

Q3
1998

Q3 Q1 Q3
1999 2000

Q4
2001

Q1 Q3 Q1
1996 1997

Q3 Q1 Q3
1999 2000

Q4
2001

Series : log(ndx.rvol)

0.0

0.0

0.2

ACF
0.4 0.6

Partial ACF
0.2
0.4

0.8

1.0

Series : log(ndx.rvol)

Q3
1998

0

5

10

15

20

0

5

Lag

10

15

20

Lag

FIGURE 18.4. Weekly realized volatility of the NASDAQ 100 index.

> nonlinearTest(log(ndx.rvol), method="threshold", p=6, d=1:6)
Nonlinearity Test: Threshold Nonlinearity
Null Hypothesis: no threshold nonlinearity
F-statdof P-val
d=1 1.2568 (7,253) 0.2724
d=2 1.4203 (7,253) 0.1974
d=3 1.2586 (7,253) 0.2714
d=4 0.5104 (7,253) 0.8264
d=5 0.5224 (7,253) 0.8173
d=6 0.1179 (7,253) 0.9971
Note that the optional argument p specifies the AR order to use in thearranged autoregression, and the optional argument d is used to select thedelay parameters from 1 to 6. The output gives the F statistics and theircorresponding p-values for all chosen values of delay parameter d, and it
shows that the evidence for threshold nonlinearity is not strong with theAR(6) specification. Since a high-order AR model may actually approximate nonlinear dynamics relatively well, a lower-order AR(2) specification
may also be tried:> nonlinearTest(log(ndx.rvol), method="threshold", p=2, d=1:2)

18.3 Threshold Autoregressive Models

667

Nonlinearity Test: Threshold Nonlinearity
Null Hypothesis: no threshold nonlinearity
F-statdof P-val
d=1 4.4468 (3,265) 0.0046
d=2 4.0010 (3,265) 0.0082
Now the null hypothesis of no threshold nonlinearity is actually rejected
for both d = 1 and d = 2 with an AR(2) specification!
Choice of Delay Parameter and Thresholds
After rejecting the null hypothesis of no threshold nonlinearity, one proceeds to the next stage of estimating a SETAR model. Tsay (1989) suggested identifying the delay parameter d and the thresholds ri for i =1, . . . , k − 1 first, and then used least squares (LS) to estimate the unknown parameters Θ in (18.5) with given values of d and thresholds. Aslong as there are enough observations in each regime, the LS estimates are
consistent.
For a given AR order p, Tsay suggested choosing the delay parameter dsuch thatd = argmax F (p, v)
v∈S

where F (p, v) is the F statistic of the auxiliary regression (18.7) with AR
order p and the delay parameter equal to v, and S is a set of values of dto consider. For the NASDAQ realized volatility series, d can be set to 1
according to the nonlinearity test output using this rule.
Tsay (1989) also proposed using two graphical tools for identifying thethreshold values: (1) the scatter plot of standardized predictive residualsêπi from the arranged autoregression versus the ordered threshold variable;(2) the scatter plot of the t-statistics of the RLS estimates of φ̂ from thearranged autoregression versus the ordered threshold variable. Both plots
may exhibit structural breaks at the threshold values. To produce suchplots for the nonlinearity test, set the optional argument save.RLS to TRUE
when calling nonlinearTest:> ndx.test = nonlinearTest(log(ndx.rvol), method="threshold",+ p=2, d=1, save.RLS=T)> names(ndx.test)[1] "stat""df""threshold" "residuals"[4] "tRatios""yd""method"
The returned object ndx.test includes the following components: yd
is the ordered threshold variable, residuals is the standardized predictive residuals, and tRatios is the t-statistics of RLS estimates of the AR18. Nonlinear Time Series Models

420

t-ratio of AR(1)

6

668

-3.5

-3.0

-2.5

-2.0

-1.5

-2.5

-2.0

-1.5

4.0
3.0
2.0

t-ratio of AR(2)Y_{t-1}-3.5-3.0
Y_{t-1}FIGURE 18.5. Scatter plot of t-statistics of RLS estimates of AR coeﬃcients
versus ordered threshold variable.

coeﬃcients. To produce the scatter plot of t-statistics versus the orderedthreshold variable, for example, use the following commands:> par(mfrow=c(2,1))> plot(ndx.test$yd, ndx.test$tRatio[,1], xlab="Y_{t-1}",+
ylab="t-ratio of AR(1)")> plot(ndx.test$yd, ndx.test$tRatio[,2], xlab="Y_{t-1}",+
ylab="t-ratio of AR(2)")
The plots in Figure 18.5 show that both estimates are significant, witht-statistics greater than 2 in absolute values in most cases. In addition, thetrend in the t-statistics seems to have two breaks: One occurs when thethreshold variable is around −2.8 and the other occurs when the thresholdvariable is around −2.4. This suggests a SETAR(3) model with two nontrivial threshold values: r1 = −2.8 and r2 = −2.4.
LS Estimates of SETAR Model
After choosing the delay parameter d and the thresholds, other unknown
parameters in Θ of the SETAR model may be simply estimated by LS using
the S+FinMetrics function SETAR, which takes the following arguments:> args(SETAR)
function(x, threshold, p = 1, d = NULL)

18.3 Threshold Autoregressive Models

669

where the first argument specifies the data to be used, the second argument
gives the vector of threshold values, and the optional arguments p and dspecify the AR order and delay parameter, respectively. To estimate the
SETAR(3) model with thresholds (−2.8, −2.4), use the following command:> ndx.setar = SETAR(log(ndx.rvol), c(-2.8, -2.4), p=2, d=1)
> summary(ndx.setar)
Call:
SETAR(x = log(ndx.rvol), threshold = c(-2.8, -2.4), p = 2,
d = 1)
Coefficients:
regime.1 regime.2 regime.3
Intercept -1.5043 -2.4463 -3.2661(std.err) 0.2778
1.13230.8676(t.stat) -5.4157 -2.1605 -3.7643
lag1(std.err)(t.stat)0.28660.0776
3.6942-0.03730.4400-0.0848-0.6283
0.3795-1.6555

lag2(std.err)(t.stat)0.25730.06873.74490.13810.1305
1.05770.21910.12791.7138Std. Errors of Residuals:
regime.1 regime.2 regime.3
0.42910.37940.3583
Information Criteria:
logL
AIC
BIC
-157.5830 333.1659 366.5000

HQ
346.5063

total regime.1 regime.2 regime.3
Degree of freedom:
300
228
44
19
Time period: from 01/15/1996 to 10/08/2001
Note that the AR coeﬃcients for the first regime are estimated to be(0.29, 0.26), which appear to be significant, whereas the AR coeﬃcientsfor the second and third regimes are estimated to be (−0.03, 0.14) and(−0.63, 0.22), respectively, and are not very significant. The estimatedregime indices can be plotted as follows:> plot(timeSeries(ndx.setar$regime,18. Nonlinear Time Series Models

2.2
2.0
1.8
1.0

1.2

1.4

1.6

Regime Index

2.4

2.6

2.8

3.0

670

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
1996
1997
1998
1999
2000
2001

FIGURE 18.6. Estimated regime indices of ndx.setar.

+ pos=positions(ndx.rvol)[-(1:2)]), reference.grid=F,+ ylab="Regime Index", plot.args=list(type="h"))and the plot is shown in Figure 18.6. It can be seen that most of theobservations prior to 2000 fall into the first regime, and the third regime
observations usually follow the second regime observations.
Predictions from SETAR Models
After estimating a SETAR model, sometimes a more important task is
to generate forecasts of future values of the time series that is of interest.
Predictions from SETAR models can be easily computed using Monte Carlosimulations, by following the same principle used for vector autoregressive
forecasting (see Section 11.3 for details). For example, to generate 1-stepahead to 100-step-ahead forecasts from the fitted model ndx.setar, use
the following command:> class(ndx.setar)[1] "SETAR"> ndx.pred = predict(ndx.setar, n.predict=100, CI.alpha=0.6,+ n.sim=10000)
Note that the fitted object ndx.setar has class "SETAR". By calling thegeneric predict function on "SETAR" objects, the simulation-based forecasting method implemented in predict.SETAR is automatically applied

671

-3.6

-3.4

-3.2

-3.0

-2.8

-2.6

-2.4

18.3 Threshold Autoregressive Models

0

20

40

60

80

100

FIGURE 18.7. Predicted realized volatility (in logarithm scale) from ndx.setar.

on the "SETAR" objects. The optional argument n.predict is used to specify the number of forecasts to obtain in the future, the argument CI.alphais used to specify 60% pointwise confidence intervals for the forecasts basedon Monte Carlo simulations, and the argument n.sim is used to specify the
number of simulations to be used for computing the forecasts. The forecasts
and their pointwise confidence intervals can be plotted as follows:> tsplot(cbind(ndx.pred$values, ndx.pred$CI), lty=c(1,6,6))
and the plot is shown Figure 18.7. After less than 20 steps, the forecasts
settle down to the asymptotic mean of the SETAR process.

18.3.3 Hansen’s Approach
Although the procedure introduced in the above subsection for identifying
and estimating SETAR models is easy to perform, it requires some humandecisions, especially for choosing the threshold values. This subsection introduces another test for threshold nonlinearity and another procedure for
estimating SETAR models as proposed by Hansen (1997). The advantageof this procedure is that the thresholds can be estimated together withother model parameters and valid confidence intervals can be constructed
for the estimated thresholds. The disadvantage is that the current imple-

672

18. Nonlinear Time Series Models

mentation only supports the two-regime SETAR model and thus only one
threshold can be estimated.10
Hansen’s sup-LR Test
Hansen (1997) considered the following two-regime variant of (18.5):
yt = Xt φ(1) (1 − I(yt−d > r1 )) + Xt φ(2) I(yt−d > r1 ) + t(18.8)

where I(A) is the indicator function that is equal to 1 if A is true and 0
otherwise, t ∼iid(0, σ 2 ), and there is only one nontrivial threshold r1 . As
discussed in the previous subsection, if d and r1 are known, then the model
parameters Θ = (φ(1) , φ(2) , σ 2 ) can be estimated by least squares:
n

1 X 2
ˆt
0
φ (1) ,φ (2) nΘ̂ = argmin σ̂ 2 (r1 ) = argmin
φ (1) ,φ (2)(18.9)

t=h

where h = max(1, p + 1 − d) and n0 = n − d − h + 1 is the eﬀective sample
size after adjusting for starting values and the delay parameter.
To test the null hypothesis of SETAR(1) against the alternative hypothesis of SETAR(2), the likelihood ratio test assuming normally distributed
errors can be used:
F (r1 ) =2
2
RSS0 − RSS1
0 σ̂ 0 − σ̂ 1 (r1 )=nσ̂21 (r1 )σ̂ 21 (r1 )(18.10)where RSS0 is the residual sum of squares from SETAR(1), RSS1 is theresidual sum of squares from SETAR(2) given the threshold r1 , and σ̂ 20 is
the residual variance of SETAR(1). The above test is also the standard F
test since (18.8) is a linear regression. However, since the threshold r1 is
usually unknown, Hansen (1997) suggested computing the following sup-LR
test:
Fs = sup F (r1 )(18.11)
r1 ∈Ydby searching over all of the possible values of the threshold variable yt−d .
In practice, to ensure that each regime has a nontrivial proportion of observations, a certain percentage of Yd at both ends are usually trimmed
and not used.
The sup-LR test has near-optimal power as long as the error term t isiid. If t is not iid, the F test needs to be replaced by the heteroskedasticityconsistent Wald or Lagrange multiplier test. One complicating issue is thatsince r1 is only identified under the alternative, the asymptotic distributionof Fs is not χ2 and nonstandard. Hansen (1996) showed that the asymptotic
10 Hansen (1999) has generalized this procedure to SETAR models with more than
two regimes.

18.3 Threshold Autoregressive Models

673

distribution may be approximated by a bootstrap procedure in general,and Hansen (1997) gave the analytic form of the asymptotic distribution
for testing against SETAR(2) models.
The nonlinearTest function in S+FinMetrics can also be used to produce Hansen’s sup-LR test, simply by setting the optional argument method
to "sup-LR". For example, to test for threshold nonlinearity in weekly realized volatility of the NASDAQ 100 index using the same AR(2) specificationand choosing the threshold variable to be zt = yt−1 as in Tsay’s F test,
use the following command11 :> nonlinearTest(log(ndx.rvol), method="sup-LR", p=2, d=1,
+ trim.pct=0.1, n.boot=1000)
Nonlinearity Test: Hansen sup-LR Nonlinearity
Null Hypothesis: no threshold with the specified thresholdvariable
Under Maintained Assumption of Homoskedastic Errors -Number of Bootstrap Replications
Trimming percentage
Threshold Estimate
F-test for no threshold
Bootstrap P-Value

1000
0.1
-2.8768
22.96870Note that the optional argument trim.pct is used to trim 10% observations at both ends of Yd , and n.boot is used to set the number of bootstrap
simulations for computing the p-value of the test. Again, the null hypothesis of no threshold nonlinearity is strongly rejected. To produce the test
robust to heteroskedastic errors, simply set the optional argument hetero
to TRUE:> nonlinearTest(log(ndx.rvol), method="sup-LR", p=2, d=1,
+ trim.pct=0.1, n.boot=1000, hetero=T)
Nonlinearity Test: Hansen sup-LR Nonlinearity
Null Hypothesis: no threshold with the specified thresholdvariable
Allowing Heteroskedastic Errors using White Correction -Number of Bootstrap Replications
Trimming percentage

1000
0.1

11 General TAR alternatives with arbitrary threshold variable can also be tested byusing setting the optional argument q instead of d.

674

18. Nonlinear Time Series Models

Threshold Estimate
F-test for no threshold
Bootstrap P-Value-2.8768
18.7357
0

Sequential Estimation of SETAR Models
After confirming the existence of threshold nonlinearity, Hansen (1997) suggested estimating the threshold value r1 together with φ using LS methods:r̂1 = argmin σ̂ 2 (r1 , d)(18.12)

r1 ∈Ydwhere σ̂ 2 (r1 , d) is the residual variance of the LS estimate of (18.8) given
the threshold r1 and the delay parameter d. So the threshold value r1 canbe estimated sequentially by searching over the possible values of r1 . If thedelay parameter is not known, it can be estimated similarly by expanding
the search to another dimension:ˆ = argmin σ̂ 2 (r1 , d)(r̂1 , d)(18.13)

r1 ,dOne thing to note is that for the asymptotic inference on SETAR models
to work correctly, each regime must have a nontrivial proportion of observations in the limit. Therefore, just as in computing Hansen’s sup-LR test,a certain percentage of Yd at both ends are usually trimmed and not used
when searching for the value of r1 .
The TAR function in S+FinMetrics implements the above sequential estimation approach.12 For example, to estimate a two-regime SETAR modelwith d = 1 and AR(2) components, use the following command:> ndx.setar.r = TAR(log(ndx.rvol), p=2, d=1, trim.pct=0.1)> ndx.setar.r
Call:
TAR(x = log(ndx.rvol), p = 2, d = 1, trim.pct = 0.1)
Coefficients:
regime.1 regime.2
intercept -2.0356 -1.4614
lag1 0.1903
0.2183lag2 0.2056
0.243512 As its name suggests, the TAR function actually supports general TAR models, inaddition to SETAR models. A general threshold variable can be used by specifying
the optional argument q. In addition, the TAR function also allows for the use of some
popular functions of a variable as the threshold variable. See the online help file for TAR
for details.

18.3 Threshold Autoregressive Models

675

Std. Errors of Residuals:
regime.1 regime.2
0.42330.3828
Information Criteria:
logL
AIC
BIC
-155.7369 323.4739 345.6966

HQ
332.3674

total regime.1 regime.2
Degree of freedom:
300
207
87
Time period: from 01/15/1996 to 10/08/2001
Note that the optional argument trim.pct is used to set the trimming percentage for Yd to 10%. Compared with the three-regime SETAR fit in theprevious subsection, this two-regime SETAR model actually gives a better fit in terms of log-likelihood value and Bayesian information criterion(BIC), which is probably due to the fact the threshold value is also optimized in this fit. The estimated threshold value is given as a component in
the returned object ndx.setar.r:> ndx.setar.r$qhat[1] -2.876807
which is quite close to the first threshold identified using Tsay’s t-statistics
plot in the previous subsection.
Confidence Interval for the Threshold
Using the generic summary function on the fitted model object ndx.setar.r
displays more details of the model fit:
> summary(ndx.setar.r)
Call:
TAR(x = log(ndx.rvol), p = 2, d = 1, trim.pct = 0.1)
Minimized SSE for all threshold variable candidates:
RVOL.lag149.84288
Threshold estimate for the threshold variable chosen withsmallest minimized SSE:
CI.lowerpoint CI.upper-3.826435 -2.876807 -2.828314
Coefficients and standard errors:
regime.1(se) regime.2

(se)

676

18. Nonlinear Time Series Models

intercept -2.036
lag1 0.190
lag2 0.206

0.325 -1.461
0.103 0.218
0.073 0.244

0.372
0.150
0.099

Coefficient confidence intervals:
regime.1.lower regime.1.upperintercept -2.700-1.075
lag1 -0.020
0.417
lag2 0.055
0.412
regime.2.lower regime.2.upperintercept -2.435-0.454lag1-0.0930.600
lag2-0.0030.472
Std. Errors of Residuals:
regime.1 regime.2
0.423
0.383
Information Criteria:
logL
AIC
BIC
-155.737 323.474 345.697

HQ
332.367

total regime.1 regime.2
Degree of freedom:
300
207
87
Time period: from 01/15/1996 to 10/08/2001
Note that standard inference statistics as well as confidence intervals forboth the coeﬃcients and the threshold are given. In particular, as proposed by Hansen (1997), an asymptotically valid confidence interval forthe threshold is constructed by inverting the likelihood ratio (LR) test for
testing the null hypothesis that the threshold is equal to a given value r:
LR(r) = n0

σ̂ 2 (r) − σ̂ 2 (r̂1 )σ̂2 (r̂1 )(18.14)

The 100·α% confidence interval (CI) for the threshold r1 is given by the setof values of r for which the above LR test cannot be rejected at significancelevel 1 − α:CI(α) = {r : LR(r) ≤ Zα }(18.15)where Zα is the 100 · α% quantile of the asymptotic distribution of the LR
statistic given in Hansen (1997). A graphical tool to help locate the confidence interval for the threshold is to plot the above LR statistics against
diﬀerent values of r, and choose the region of r close to r1 where the LR
statistics are smaller than the critical value Zα . The necessary information

677

10
0

5

LR stat15

18.3 Threshold Autoregressive Models

-3.8

-3.6

-3.4

-3.2

-3.0

-2.8

-2.6

Threshold

FIGURE 18.8. Confidence interval for threshold value by inverting likelihood
ratio statistics.

to generate such a plot is contained in the LR.q component of the fitted
model object. For example, to produce the plot using the fitted model
object ndx.setar.r, use the following commands:> names(ndx.setar.r$LR.q)[1] "LR""Threshold" "Critical"> plot(ndx.setar.r$LR.q$Threshold, ndx.setar.r$LR.q$LR,+
type="b", xlab="Threshold", ylab="LR stat")> abline(h=ndx.setar.r$LR.q$Critical)
and the plot is shown in Figure 18.8. This plot can also be generated
directly and applying the generic plot function on the fitted model objectndx.setar.r.
Predictions from TAR Models
Just like with SETAR models, predictions from general TAR models canbe computed using Monte Carlo simulations, as long as the future valuesof the threshold variable are known. In fact, the objects returned by the
TAR function have class "TAR", which inherits from the "SETAR" class. For
example,> class(ndx.setar.r)[1] "TAR"

678

18. Nonlinear Time Series Models> inherits(ndx.setar.r, "SETAR")[1] T
Thus, when the generic predict function is called on "TAR" objects, thesimulation-based forecasting procedure in predict.SETAR is also used to
produce the forecasts. For example, to generate forecasts from the fitted
model object ndx.setar.r, use the following command:> ndx.pred.2 = predict(ndx.setar.r, n.predict=100,+ CI.alpha=0.6, n.sim=10000)
which are very similar to the forecasts produced earlier using a three-regime
model.

18.4 Smooth Transition Autoregressive Models
In the TAR models introduced in the previous section, a regime switch
happens when the threshold variable crosses a certain threshold. Although
the model can capture many nonlinear features usually observed in economic and financial time series, sometimes it is counterintuitive to suggest that the regime switch is abrupt or discontinuous. Instead, in somecases it is reasonable to assume that the regime switch happens graduallyin a smooth fashion. If the discontinuity of the thresholds is replaced bya smooth transition function, TAR models can be generalized to smoothtransition autoregressive (STAR) models.
In this section, two main STAR models — logistic STAR and exponential
STAR — are introduced. After illustrating how to test for STAR nonlinearity, examples will be given to show how to estimate STAR models in
S+FinMetrics. A systematic modeling cycle approach for STAR models
was proposed by Teräsvirta (1994), and van Dijk, Teräsvirta, and Franses(2002) provided a survey of recent development for STAR models.

18.4.1 Logistic and Exponential STAR Models
In the SETAR model (18.8) considered in the previous section, the observations yt are generated either from the first regime when yt−d is smaller
than the threshold or from the second regime when yt−d is greater than thethreshold. If the binary indicator function is replaced by a smooth transition function 0 < G(zt ) < 1, which depends on a transition variable zt(like the threshold variable in TAR models), the model becomes a STARmodel:
yt = Xt φ(1) (1 − G(zt )) + Xt φ(2) G(zt ) + t(18.16)
Now, the observations yt switch between two regimes smoothly in the sensethat the dynamics of yt may be determined by both regimes, with one

18.4 Smooth Transition Autoregressive Models
Exponential

G(z_t)
0.2

0.2

0.4

0.4

G(z_t)

0.6

0.6

0.8

0.8

1.0

1.0

Logistic

679

gamma=1
gamma=50.0

0.0

gamma=1
gamma=5

-4

-2

0

2

4

-4

-2

z_t

0

2

4

z_t

FIGURE 18.9. Logistic and exponential transition functions.

regime having more impacts in some times and the other regime having
more impacts in other times. Another interpretation is that STAR models
actually allow for a “continuum” of regimes, each associated with a diﬀerent
value of G(zt ).
Two popular choices for the smooth transition function are the logistic function and the exponential function. Using the logistic function, thetransition function can be specified as
G(zt ; γ, c) =1
1 + e−γ(zt −c), γ>0(18.17)and the resulting model is referred to as logistic STAR or LSTAR model.
The parameter c can be interpreted as the threshold, as in TAR models, andγ determines the speed and smoothness of transition. Using the exponentialfunction, the transition function can be specified as
2

G(zt ; γ, c) = 1 − e−γ(zt −c) , γ > 0(18.18)and the resulting model is referred to as exponential STAR or ESTAR
model. As in LSTAR models, c can be interpreted as the threshold and γ
determines the speed and smoothness of transition.
In spite of the similarity between LSTAR and ESTAR models, they actually allow for diﬀerent types of transitional behavior. To illustrate thispoint, Figure 18.9 plots the logistic and exponential transition functions

680

18. Nonlinear Time Series Models

with c = 0 and γ = 1 and 5. The following properties can be readily
observed:
1. If γ is small, both transition functions switch between 0 and 1 verysmoothly and slowly; if γ is large, both transition functions switch
between 0 and 1 more quickly.
2. As γ → ∞, both transition functions become binary. However, thelogistic function approaches the indicator function I(zt > c) and the
LSTAR model reduces to a TAR model; the exponential function
approaches the indicator function I(zt = c) and the model does notnest the TAR model as a special case.
3. The logistic function is monotonic and the LSTAR model switches
between two regimes smoothly depending on how much the transition variable zt is smaller than or greater than the threshold c. The
exponential function is symmetrical and the ESTAR model switches
between two regimes smoothly depending on how far the transition
variable zt is from the threshold c. For the LSTAR model, both the
distance between zt and c and its sign matter; for the ESTAR model,
only the distance between zt and c matters, but not the sign.

18.4.2 Test for STAR Nonlinearity
Testing for the existence of STAR-type nonlinearity is usually the first step
toward building a STAR model. However, just like the test for thresholdtype nonlinearity, tests for the null hypothesis of a simple AR model against
the alternative of a STAR model have nonstandard asymptotic distributions, because some parameters in the STAR model are not identified under
the null hypothesis, such as the AR coeﬃcients φ(2) in the second regime,
the transition parameter γ, and the threshold c.
STAR Nonlinearity Test with Homoskedastic Errors
To avoid complicated issues caused by the unidentified STAR model parameters under the null hypothesis of a linear AR model, Luukkonen,
Saikkonen, and Teräsvirta (1988) proposed replacing the transition function G(zt ; γ, c) by a suitable Taylor series approximation around γ = 0. It
turns out that if the transition function G(zt ; γ, c) in the LSTAR model isreplaced by its third-order Taylor series approximation, the LSTAR modelin (18.16) can be written as13
yt = Xt β0 + Xt zt β 1 + Xt zt2 β2 + Xt zt3 β3 + et
13 See Franses and van Dijk (2000) for details.

(18.19)

18.4 Smooth Transition Autoregressive Models

681

where the coeﬃcient vectors βi for i = 0, 1, 2, 3, 4 are functions of the original model parameter φ. Similarly, if the transition function G(zt ; γ, c) in the
ESTAR model is replaced by its second-order Taylor series approximation,
the ESTAR model in (18.16) can be written as
yt = Xt β0 + Xt zt β 1 + Xt zt2 β2 + Xt zt3 β 3 + Xt zt4 β4 + et

(18.20)Now, testing the null hypothesis of a linear AR model against a nonlinear
STAR model is equivalent to testing the null hypothesis H0 : βj = 0 for
j = 1, 2, 3, 4 in the above auxiliary regressions, which is a conventional
Lagrange multiplier (LM) test with an asymptotic χ2 distribution.
In practice, it has been found that the LM test based on (18.19) for
LSTAR models also has power against ESTAR alternatives. Thus, for reasons of parsimony, usually only the LM test based on (18.19) is computedfor testing STAR-type nonlinearity in general. Also, instead of using theasymptotic χ2 distribution, in small samples it is usually preferred to use
the F version of the LM test, which tends to have better size and powerproperties. Finally, since TAR models are special cases of LSTAR models
when the transition parameter γ → ∞, it can be shown that the LM test
also has power against threshold-type nonlinearity. Granger and Teräsvirta(1993) discuss these issues in more detail.
The LM test for STAR nonlinearity can be performed in S+FinMetrics
using the nonlinearTest function, by setting the optional argument method
to "STAR-LM". For example, to test for STAR-type nonlinearity in NASDAQ realized volatility ndx.rvol, use the command> nonlinearTest(log(ndx.rvol), method="STAR-LM", p=2, d=1:2)
Nonlinearity Test: STAR Nonlinearity
Null Hypothesis: no smooth threshold nonlinearity
Under Maintained Assumption of Homoskedastic Errors --RVOL.lag1
RVOL.lag2

RVOL.lag1
RVOL.lag2

ChiSq-stat ChiSq-dof ChiSq.pv-val
21.300860.0016
13.6974
60.0332F-statF-dof F.pv-val
3.7068 (6,291)
0.00142.3204 (6,291) 0.0333

In the above example, the transition variable is set to yt−d by specifying
the optional argument d.14 More than one value of d can be specified and
14 A weakly exogenous variable can also be used as the transition variable by setting
the optional argument q instead of d. See the online help file for nonlinearTest for
details.

682

18. Nonlinear Time Series Models

nonlinearTest automatically computes the LM test for all of the given
values of d. If the null hypothesis of a linear AR model is rejected, thetest statistics based on diﬀerent values of d can be used to choose theappropriate value of d in the final STAR model. In the output shown above,
the null hypothesis of no STAR-type nonlinearity is rejected at the 5%
significance level for both d = 1 and d = 2. In addition, the p-values fromboth the χ2 test and F test prefer d = 1, which is consistent with the resultsof threshold-type nonlinearity tests presented in the previous section.
STAR Nonlinearity Test with Heteroskedastic Errors
The LM test presented above assumes that the error term in (18.16) hasconstant variance. However, economic and financial time series are often
heteroskedastic, and neglected heteroskedasticity may lead to spurious rejection of the null hypothesis. Based on Davidson and MacKinnon (1985),
Granger and Teräsvirta (1993) summarized the following LM test for nonlinearity, which is robust toward heteroskedastic errors:
1. Regress yt on Xt to obtain the LS residuals ˆt .
2. Regress Xt ztj for j = 1, 2, 3 on Xt to obtain the residuals R̂t .
3. Regress the unit vector on R̂tˆt and compute the LM statistic as the
explained sum of squares from this regression.
This test can be performed just as before by setting the optional argument hetero to TRUE:> nonlinearTest(log(ndx.rvol), method="STAR-LM", p=2, d=1:2,+ hetero=T)
Nonlinearity Test: STAR Nonlinearity
Null Hypothesis: no smooth threshold nonlinearity
Allowing Heteroskedastic Errors using White Correction --RVOL.lag1
RVOL.lag2

RVOL.lag1
RVOL.lag2

ChiSq-stat ChiSq-dof ChiSq.pv-val
15.073160.0197
10.8287
6
0.0938
F-statF-dof F.pv-val
2.5657 (6,291)
0.01951.8162 (6,291)
0.0957

Now, the null hypothesis cannot be rejected at 5% significance level whend = 2, but it is still rejected at 5% level when d = 1. However, based onsome simulation evidence, Lundbergh and Teräsvirta (1998) suggested that

18.4 Smooth Transition Autoregressive Models

683

in some cases this robustification may not be desirable because it removes
most of the power of the test.

18.4.3 Estimation of STAR Models
After confirming the existence of STAR-type nonlinearity in a time series,
one can proceed to the next stage of building a STAR model. This usually involves choosing the transition variable and the form of transition
function. As mentioned in the previous subsection, the test for STAR-type
nonlinearity can be computed for a range of transition variables, and thep-values of the test statistics can be used to help choose the appropriate
transition variable. The choice between the LSTAR model and the ESTAR
model can usually be made by considering the specific transitional behavior under investigation or by comparing diﬀerent information criteria. This
subsection first shows how to estimate LSTAR models using the STAR function in S+FinMetrics and then it walks through an example of estimating
an ESTAR model using the S-PLUS function nlregb.
LSTAR Model
Once the AR order and the transition variable have been chosen, LSTAR
models can be estimated by nonlinear least squares (NLS):XΘ̂ = argmin
ˆ2t(18.21)
γ,c

t

where
ˆt = yt − X̃t φ̂∙¸
Xt (1 − G(zt ; γ, c))X̃t =Xt G(zt ; γ, c)" (1) # "
#−1 "
#
X
X
φ̂
φ̂ =
(X̃0t X̃t )X̃0t yt
(2) =
φ̂
t
t
Note that the minimization of the NLS objective function is only performed
(1)

(2)

over γ and c because φ̂ and φ̂ can be estimated by least squares once
γ and c are known. Under the additional assumption that the errors arenormally distributed, NLS is equivalent to the maximum likelihood estimation. Otherwise, the NLS estimates can be interpreted as quasi-maximum
likelihood estimates.
Example 120 LSTAR model for NASDAQ realized volatility
The following command fits an LSTAR model to the logarithms of weekly
realized volatility of the NASDAQ 100 index, with the same AR order anddelay parameter used in the previous examples:684

18. Nonlinear Time Series Models> ndx.lstar = STAR(log(ndx.rvol), p=2, d=1)
> summary(ndx.lstar)Call:STAR(X = log(ndx.rvol), p = 2, d = 1)
Parameter estimates:
Values Std.Error Std.Error.whitegamma 1.608 1.113
1.282
threshold -2.845 0.398
0.309

Coefficient estimates and standard errors:
Lower regime:
Values Std.Error Std.Error.whiteintercept(lower) -3.729 1.832
2.696
lag1(lower) -0.221 0.404
0.632
lag2(lower) 0.205 0.092
0.092
Upper regime:
Values Std.Error Std.Error.whiteintercept(upper) -2.668 1.904
1.497
lag1(upper) -0.396 1.076
0.896
lag2(upper) 0.216 0.134
0.131
Std. Errors of Residuals:[1] 0.415
Information Criteria:
logL
AIC
BIC
-158.863 329.727 351.950

HQ
338.620

Degrees of freedom:
total residuals
300
294Time period: from 01/15/1996 to 10/08/2001

Note that the threshold estimate −2.85 is very close to the SETAR estimate of −2.88 given by the TAR estimate ndx.setar.r. However, byallowing for a smooth transition between two regimes, the AR coeﬃcientsin both regimes are quite diﬀerent from those estimated by ndx.setar.r.

685

-3.6

-3.4

-3.2

-3.0

-2.8

-2.6

-2.4

-2.2

18.4 Smooth Transition Autoregressive Models

0

20

40

60

80

100

FIGURE 18.10. Predicted realized volatility (in logarithmic scale) fromndx.lstar.

Predictions from LSTAR Model
Simulation-based forecasts from the LSTAR model can be easily generatedusing the same principle for generating forecasts from VAR models and
SETAR models. The fitted model objects returned by the STAR function
have class "STAR". By calling the generic predict function on fitted model
objects, the method function predict.STAR is automatically invoked. For
example, the following command generates 100-step-ahead forecasts fromndx.lstar:> ndx.pred.3 = predict(ndx.lstar, n.predict=100,+ CI.alpha=0.6, n.sim=10000)> tsplot(cbind(ndx.pred.3$values, ndx.pred.3$CI),+ lty=c(1,6,6))
and Figure 18.10 shows the forecasts with 60% pointwise confidence intervals. The forecasts are very similar to those generated by the SETAR
model object ndx.setar, except they do not have the initial small peakexhibited by the SETAR forecasts.
ESTAR Model
Currently, the STAR function in S+FinMetrics only supports LSTAR models, not ESTAR models. However, the estimation of ESTAR models fol-

686

18. Nonlinear Time Series Models

lows essentially the same procedure in (18.21) with the transition function
given by (18.18). Here, an example is given to show how to estimate ESTAR models using the S-Plus function nlregb for nonlinear least squares
estimation.
The arguments expected by nlregb are as follows:> args(nlregb)
function(nres, start, residuals, jacobian=NULL, scale=NULL,
control = NULL, lower = -Inf, upper = Inf, ...)
where the first argument, nres, specifies the number of observations orresiduals to be used, the second argument, start, specifies the starting
values for the unknown parameters, and the third argument, residuals,
is an S-PLUS function that takes the parameter values and computes theresidual vector with length equal to nres. The optional arguments lower
and upper can be used to specify lower and upper bounds on the unknown
parameters.
One general issue in estimating STAR models is that the transition parameter γ can become large and cause numerical problems in the optimization procedure. To alleviate the potential numerical problems in estimating
ESTAR models, it is usually preferred to estimate the following transition
function instead of the original exponential function in (18.18):½¾
2
γ̃ (zt − c)G(zt ; γ̃, c) = 1 − exp −e
σ 2z(18.22)where σ2z is the sample variance of the transition variable zt . The newparameter γ̃ can be transformed to the original parameter γ as follows:
γ=

eγ̃
σ 2z

(18.23)This transformation has the following numerical properties:
1. The squared distance between zt and the threshold c is now scaledby the variance of zt , which makes it scale-free.
2. The original parameter γ lies in (0, ∞), which requires a constrained
optimization in terms of γ. The new parameter γ̃ lies in (−∞, ∞)and is unconstrained.
3. The new parameter γ̃ is a linear function of the logarithm of γ, which
is more dampened than γ.
Using the new formulation in (18.22), the following S-PLUS function takes
the unknown parameter values (γ̃, c) and returns the residual vector:

18.5 Markov Switching State Space Models

687

ESTAR.res = function(theta, g.scale, x, y, q){k = ncol(x)G = 1 - exp( - exp(theta[1])/g.scale * (q - theta[2])^2)
X = cbind(x * (1 - G), x * G)m = crossprod(t(backsolve(chol(crossprod(X)), diag(2 * k))))beta = m %*% t(X) %*% yy - X %*% beta}
Now, to estimate an ESTAR model with an AR(2) specification andtransition variable zt = yt−1 using the NASDAQ realized volatility series,
use the following commands:> ndx.LHS = log(ndx.rvol)[3:length(ndx.rvol)]@data> ndx.RHS = cbind(1, tslag(log(ndx.rvol), 1:2, trim=T)@data)> ndx.estar = nlregb(length(ndx.rvol)-2,+
start=c(0,mean(ndx.RHS[,2])),+
residuals=ESTAR.res,+lower=c(-Inf, min(ndx.RHS[,2])),+upper=c( Inf, max(ndx.RHS[,2])),+g.scale=var(ndx.RHS[,2]),+x=ndx.RHS, y=ndx.LHS, q=ndx.RHS[,2]))Note that the regressors ndx.RHS include a constant term and two lagged
values of yt , and the transition variable yt−1 is given by the second columnof ndx.RHS. In the call to the nlregb function, the starting values of γ̃
is set to zero, which corresponds to setting γ = 1, and the starting valueof c is simply set to the mean of the transition variable yt−1 . The otherarguments g.scale, x, y, and q to the residual function ESTAR.res are
passed as optional arguments to nlregb. The NLS estimates of (γ̃, c) are
given by> ndx.estar$parameters[1] -1.239878 -2.774638
Note that the threshold estimate of −2.77 is close to the threshold estimates obtained in earlier examples. The transition parameter γ in theoriginal exponential function can be obtained as follows:> exp(ndx.estar$parameters[1])/var(ndx.RHS[,2])[1] 1.013556

18.5 Markov Switching State Space Models
The nonlinear time series models introduced so far all allow for diﬀerent
regimes, with each regime represented by a simple AR model. For TAR

688

18. Nonlinear Time Series Models

and SETAR models, the regimes are solely determined by the magnitudeof an observable weakly exogenous variable, whereas for STAR models, theregimes are allowed to switch smoothly according to the magnitude of a
weakly exogenous variable relative to a threshold value. This section introduces another type of regime switching model — the Markov switchingmodel — where the regimes are determined by an unobserved state or regime
variable that follows a discrete state Markov process. Discrete state Markov
processes, also called Markov chains, are very popular choices for modelingstate-dependent behavior. Since Hamilton (1989) proposed using a simple
Markov switching AR process to model the U.S. real Gross National Product (GNP), Markov switching time series models have seen extraordinary
growth and become extremely popular for modeling economic and financial
time series. They have been applied to model and forecast business cycles,
the term structure of interest rates, volatility in economic and financial
variables, foreign exchange rate dynamics, inflation rate dynamics, etc.
This section first introduces the discrete state Markov process used tomodel the hidden state variable and it then illustrates how the discrete
state Markov process can be combined with an AR model to produce theMarkov switching AR process. To allow for Markov switching dynamics
in a much broader context, Markov switching state space models are thenintroduced and examples will be given to illustrate the estimation of thesemodels using S+FinMetrics functions.

18.5.1 Discrete State Markov Process
Discrete state Markov processes are very popular choices for modelingstate-dependent behavior in natural phenomena and are natural candidates for modeling the hidden state variables in Markov switching models.
A discrete state Markov process classifies the state of the world St at any
time t into a few discrete regimes. The state switches between diﬀerent
regimes according to its previous value and transition probabilities given
by15
Pr(St = j|St−1 = i) = Pij ≥ 0(18.24)

where i, j = 1, 2, . . . , k, with k diﬀerent possible states or regimes, andk
X

Pr(St = j|St−1 = i) = 1(18.25)j=1

15 A discrete state Markov process that only depends on its most recent observation iscalled the first-order Markov process. Since higher-order Markov processes can always
be rewritten as a first-order Markov process, it usually suﬃces to consider only the
first-order Markov process.

18.5 Markov Switching State Space Models

689

It is usually convenient to collect the transition probabilities into a transition matrix :⎡⎤P11 P12 · · · P1k⎢P21 P22 · · · P2k ⎥⎢⎥P =⎢ .
..
.. ⎥
.
.
.
⎣ .
.
.
. ⎦
Pk1 Pk2 · · · Pkk

where each row sums up to 1. For example, at time t, the state of theeconomy St can be classified as either recessionary (St = 1) or expansionary(St = 2). Using quarterly observations of the U.S. real GNP from 1952 to
1984, Kim (1994) estimated the transition matrix to be∙¸
47% 53%P=(18.26)
5% 95%
These transition probabilities imply that if the economy is in an expansion,
it tends to stay in expansion with a very high probability of 95%; if theeconomy is in a recession, it has a 47% chance of staying in a recession anda 53% chance of getting out of a recession. These numbers also reflect thecommon observation that the transition from an expansion to a recession
is usually very quick, whereas the recovery from a recession is relatively
slow.
Suppose at time t, the probability of each state or regime is given by
the vector π t = (P1 , P2 , . . . , Pk ), then the probability of each state at timet + 1 is given by(18.27)π t+1 = P 0 π t
For a stationary discrete state Markov process, the ergodic probability vector π exists such that
π = P 0π(18.28)The ergodic probability vector can also be treated as the steady state, or
the unconditional probability of each state of the world. S+FinMetrics
provides a convenience function mchain.p0 to compute the ergodic probability vector for a stationary Markov chain.16 For example, the following
command computes the ergodic probabilities for the state of the economy
using the transition matrix in (18.26):> mchain.p0(matrix(c(0.47, 0.05, 0.53, 0.95), 2, 2))[1] 0.0862069 0.9137931
Thus the unconditional probability of the economy being in a recession isabout 9% and the unconditional probability of the economy being in an
expansion is about 91%.
16 See Hamilton (1994) for the analytic formula for computing the ergodic probabilities
for a stationary Markov chain.

690

18. Nonlinear Time Series Models

The transition probabilities can also be used to infer the duration of eachstate or regime. For example, using the transition matrix in (18.26), theaverage duration of an economic expansion can be computed as17
1
= 20 quarters = 5 years
1 − P22and the average duration of an economic recession can be computed as
1= 2 quarters
1 − P11which is consistent with the fact that a recession is usually defined as adrop in real GDP for two consecutive quarters.

18.5.2 Markov Switching AR Process
If the model parameters in the simple AR(p) model in (18.4) are relaxed
to be dependent on a latent or hidden state variable St , it becomes
yt = μSt + Xt φSt + ut for t = 1, 2, . . . , n(18.29)where Xt = (yt−1 , yt−2 , . . . , yt−p ), φSt is the p×1 vector of AR coeﬃcients,ut ∼ N (0, σ 2St ), and the hidden state variable St follows a k-regime Markov
chain given by (18.24) and (18.25). This is usually referred to as the Markov
switching AR(p) process. The Markov switching AR(p) model has proved tobe eﬀective at modeling nonlinear dynamics usually observed in economic
and financial time series. For example, Hamilton (1989) used a two-state
Markov switching AR(4) model with constant σ 2 to capture the diﬀerent
dynamics observed in the U.S. real GNP during economic recessions andexpansions.
In general, if the states S = (Sp+1 , . . . , Sn ) are known, the unknown parameters Θ of the Markov switching AR(p) model, which include the intercept terms, the AR coeﬃcients, and the error variance in diﬀerent regimes,
can be estimated by maximizing the following log-likelihood function:
L(Θ|S) =

n
X

t=p+1

log f (yt |Yt−1 , St )where Yt−1 denotes all the information available at time t − 1 and includes
all of the observations in Xj for j ≤ t, and¾½(yt − μSt − Xt φSt )2
1
f (yt |Yt−1 , St ) ∝ exp − log σ 2St −(18.30)2
2σ 2St17 See Kim and Nelson (1999), for example, for the derivation of this result.

18.5 Markov Switching State Space Models

691

However, the states S are usually unobserved and must be inferred fromthe data. When the states S are unknown, the parameters of the Markov
switching AR(p) model are expanded to include the transition probabilities
P. By applying the law of total probability, the log-likelihood function can
now be written as
L(Θ) =

n
X

t=p+1

=

n
X

t=p+1

log f (yt |Yt−1 )
log

⎧
k
⎨X
⎩

j=1

⎫
⎬
f (yt |Yt−1 , St = j) Pr(St = j|Yt−1 )⎭(18.31)

where f (yt |Yt−1 , St = j) is given in (18.30), and by Bayes theorem thepredictive probability Pr(St = j|Yt−1 ) can be shown to be
Pr(St = j|Yt−1 ) =

k
X
i=1

=

k
X
i=1

Pij PkPr(St = j|St−1 = i, Yt−1 ) Pr(St−1 = i|Yt−1 )f (yt−1 |Yt−2 , St−1 = i) Pr(St−1 = i|Yt−2 )m=1 f (yt−1 |Yt−2 , St−1 = m) Pr(St−1 = m|Yt−2 )(18.32)Thus, given an estimate of the initial probability of each state as Pr(Sp+1 =i|Yp ) for i = 1, 2, . . . , k, the log-likelihood function of the Markov switching AR(p) model can be computed iteratively using (18.31) and (18.32)
and the unknown parameters Θ can be estimated by maximum likelihood
estimation (MLE).
The evaluation of the above log-likelihood function for the Markov switching AR(p) model can be easily programmed in S-PLUS. However, since it
involves an iterative process that prevents the use of vectorized operationsin S-PLUS, the optimization process of obtaining the MLE can be slow andcomputationally ineﬃcient. In order to be able to estimate a broad rangeof Markov switching models using the same code, the following subsection
introduces Markov switching state space models that includes the Markov
switching AR(p) model as a special case. The Markov switching state spacemodels utilize optimized C code for fast calculation.

18.5.3 Markov Switching State Space Models
As shown in Chapter 14, most linear time series regression models can becast into a state space form, and the state space representation provides aconvenient framework for obtaining filtered and smoothed estimates of theunobserved state variables. In this subsection, the state space representation in Chapter 14 is generalized to allow for Markov switching dynamics
so that a vast number of Markov switching models can be easily estimatedusing the same framework.

692

18. Nonlinear Time Series Models

Using the notation in Chapter 14, a state space model can be represented
as follows:
αt+1yt= dt + Tt · αt + Ht · η t= ct + Zt · αt + Gt · εt(18.33)(18.34)

where αt+1 is the m × 1 state vector, yt is the N × 1 vector of observed
variables, η t ∼ iid N (0, Ir ) is the r × 1 vector of disturbance terms in thetransition equation governing the dynamics of the state vector αt+1 , εt ∼iid N (0, IN ) is the N × 1 vector of disturbance terms in the measurement
equation governing the dynamics of the observed variables yt , and dt , Tt ,Ht , ct , Zt , and Gt are conformable hyperparameter matrices or system
matrices. More compactly, the above representation can be rewritten asµ¶αt+1(18.35)= δ t + Φt · αt + ut ,ytwhere ut ∼ iid N (0, Ωt ) and
µ
µ
µ
µ
¶
¶
¶
dt
Tt
Ht ηt
Ht H0tδt =, Φt =, ut =, Ωt =
0
ct
Zt
Gt εt

0
Gt G0t¶For Markov switching state space models, the hyperparameter matrices
are assumed to be dependent on a latent or unobserved discrete state variable St :
δ t = δ St
Φt = ΦStΩt = ΩSt
and the discrete state variable St follows a k-regime Markov chain givenin (18.24) and (18.25). For example, by setting the continuous state vector
to αt = (yt , yt−1 ), the Markov switching AR(2) model can be put into theabove state space representation with
⎤
⎡
∙ 0 ¸
∙
¸
μSt+1
φSt+1
1 0
⎦
⎣
, Φt =, I2×2 =
δt =
0
1 0
I2×2
0
and Ωt is a 3 × 3 matrix with σ 2St+1 being the (1, 1) element and zero
elsewhere.

Example 121 State space representation of Markov switching AR(2) model
S+FinMetrics uses a "list" object with some required components torepresent a state space model in S-PLUS, and Chapter 14 has many examples showing how to create such objects for some popular time series

18.5 Markov Switching State Space Models

693

regression models. In order for Markov switching state space models tobe represented by an S-PLUS object, the "list" object is expanded to allow for the following components: mTrans, mDelta.other, mPhi.other andmOmega.other. The mTrans component is required for a Markov switchingstate space representation and specifies the transition matrix P for theunderlying Markov chain, and at least one of mDelta.other, mPhi.other,and mOmega.other must be specified so that at least some hyperparameterof the model is Markov switching. The usual components mDelta, mPhi
and mOmega specify the hyperparameter matrices for the first regime, andthe new components mDelta.other, mPhi.other, and mOmega.other specify the hyperparameter matrices for other regimes if necessary. If there
are k > 2 regimes for the discrete state variable St , the components
mDelta.other, mPhi.other, and mOmega.other store the hyperparameter
matrices for regimes 2 to k stacked columnwise.
For example, the unknown parameters of a two-regime Markov switching
AR(2) model can be collected in the vector:v = (μ1 , μ2 , φ11 , φ12 , φ21 , φ22 , σ 1 , σ 2 , P11 , P22 )(18.36)

where μ1 , φ11 , φ12 , and σ1 are the intercept term, the AR coeﬃcients, anderror standard deviation for the first regime, μ2 , φ21 , φ22 , and σ 2 are thecounterparts for the second regime, P11 and P22 are the diagonal elementsof the transition matrix P. Note that since each row of P sums up to 1,
only two transition probabilities are required to identify P. The following
S-PLUS function takes the vector (18.36) and returns a "list" object giving
the state space representation of the two-regime Markov switching AR(2)model18 :GetSsfMSAR = function(parm){mDelta = mDelta.other = rep(0, 3)
mDelta[1] = parm[1]mDelta.other[1] = parm[2]#mPhi = mPhi.other = matrix(0, 3, 2)
mPhi[1,] = c(parm[3], parm[4])mPhi.other[1,] = c(parm[5], parm[6])mPhi[2:3,1] = mPhi.other[2:3,1] = 1#
mOmega = mOmega.other = matrix(0, 3, 3)
mOmega[1,1] = parm[7]mOmega.other[1,1] = parm[8]#mSigma = matrix(0, 3, 2)
18 One may also use the S+FinMetrics function GetSsf.MSAR.

694

18. Nonlinear Time Series Models

mSigma[1:2, 1:2] = diag(1e+6, 2)#
mTrans = matrix(0, 2, 2)
mTrans[1,1] = parm[9]mTrans[1,2] = 1 - mTrans[1,1]mTrans[2,2] = parm[10]mTrans[2,1] = 1 - mTrans[2,2]#list(mDelta=mDelta, mDelta.other=mDelta.other,mPhi=mPhi, mPhi.other=mPhi.other,mOmega=mOmega, mOmega.other=mOmega.other,mSigma=mSigma, mTrans=mTrans)}
Note that a diﬀuse prior on the initial state vector is specified by setting
the first 2 × 2 block of mSigma to a diagonal matrix with large values on
the diagonal and setting the last row of mSigma to zero.
Approximate MLE of Markov Switching State Space Models
Since Markov switching state space models allow for nonlinear dynamics,
the traditional Kalman filtering and smoothing algorithms for Gaussian
linear state space models can no longer be applied to obtain valid inference
on the unobserved state vector. In particular, given the initial estimate(i)(i)
at|t and Pt|t for St = i with i = 1, . . . , k, the prediction equations for theGaussian linear state space model in (14.39) and (14.40) now become
(i,j)

(i)

at+1|t = Tt at|t(i,j)(18.37)(i)

Pt+1|t = Tt Pt|t T0t + Ht H0t(18.38)where the superscript (i, j) denotes the case of St = i and St+1 = j fori, j = 1, . . . , k. The updating equations for the Gaussian linear state spacemodel in (14.34) and (14.35) now become
(i,j)

(i,j)

(i,j) (i,j)
vt

at|t = at|t−1 + Kt

(i,j)
(i,j)
(i,j)
(i,j)
Pt|t = Pt|t−1 − Pt|t−1 Z0t (Kt )0(18.39)(18.40)

where
(i,j)

vt

(i,j)

Ft

(i,j)

= Zt Pt|t−1 Z0t + Gt G0t

(i,j)

Kt

(i,j)

= yt − ct − Zt at|t−1(i,j)(i,j) −1

= Pt|t−1 Z0t (Ft)

Therefore, at each step, the set of statistics that needs to be computed
and stored will increase by the order of k. Obviously, even for a relatively

18.5 Markov Switching State Space Models

695

small sample, the Kalman filtering algorithm will become computationallyinfeasible.
To make the filtering algorithm manageable, Kim (1994) proposed collapsing the set of statistics in the updating equations (18.39) and (18.40)
as follows:Pk(i,j)
i=1 Pr(St = j, St−1 = i|Yt )at|t(j)
at|t =(18.41)
Pr(St = j|Yt )
Pk
(i,j)
(j)
(i,j)
(j)
(i,j) 0i=1 Pr(St = j, St−1 = i|Yt )[Pt|t + (at|t − at|t )(at|t − at|t ) ](j)
Pt|t =
Pr(St = j|Yt )(18.42)where the filtered probability Pr(St = j|Yt ) can be updated similarly as in(18.32), given an initial estimate. Now at each step, only k sets of statistics
need to be stored, which can be fed into the prediction equations (18.37)
and (18.38) to complete the filtering algorithm. This algorithm is sometimes
referred to as Kim’s filtering algorithm.
Just like the Kalman filtering algorithm for Gaussian linear state spacemodels, Kim’s filtering algorithm can be used to provide the prediction
error decomposition for computing the log-likelihood function of Markov
switching state space models. However, the drawback of the above filtering(j)algorithm is that the filtered estimates at|t now follow normal mixture
distributions instead of normal distributions as in Gaussian linear state
space models. As a result, the MLEs obtained using Kim’s algorithm areonly approximate and not optimal, but empirical evidence seems to suggestthat approximate MLEs obtained using Kim’s filtering algorithm are very
reliable.19
The SsfLoglikeMS function in S+FinMetrics implements Kim’s filtering algorithm to compute the log-likelihood function for arbitrary Markov
switching state space models, and the SsfFitMS function uses it to obtain approximate MLEs of the unknown parameters in Markov switchingstate space models. However, Markov switching state space models can bediﬃcult to fit due to various numerical issues. Here, a few guidelines areprovided for using the SsfFitMS function for maximum likelihood estimation of Markov switching state space models:
1. Make sure that the model to be fitted is actually identified. It can bevery easy to specify a Markov switching model that is not identified or
poorly identified. Overidentification or poor identification can cause
the optimization procedure to fail.
19 In recent years, more computationally intensive Bayesian methods have also beendeveloped to analyze Markov switching state space models or non-Gaussian state spacemodels on a case-by-case basis. See Kim and Nelson (1998), Kim, Shephard, and Chib
(1998), and Aguilar and West (2000) for some examples.

696

18. Nonlinear Time Series Models

2. Start from a small model. If the estimation of the small model does
not pose any problem, extend the model to allow for more features.
3. Provide good starting values to SsfFitMS. Good starting values canbe found by calling SsfLoglikeMS with diﬀerent sets of parametervalues and choosing the one with largest log-likelihood value.
4. Although the SsfFitMS function allows lower and upper bound constraints on the parameters, sometimes better convergence can be obtained by transforming the parameters so that the parameters to be
estimated are unconstrained.
Example 122 Markov switching AR(2) model for NASDAQ realized volatility
Earlier examples in this chapter show that the logarithms of weekly realized volatility of the NASDAQ 100 index can be modeled by a switching
AR(2) process, with the switching determined by either a TAR model or aSTAR model. It is interesting to see if the Markov switching AR(2) model
can provide a better or equivalent characterization of the nonlinear dynamics observed in the data.
Instead of directly estimating the unknown parameters for the Markov
switching AR(2) model as given in (18.36), it is usually better to transformthese parameters so that they are unconstrained. For example, the following
monotonic transformations are usually adopted:
1. If x lies within (0, ∞), then y = log x is unconstrained and x = ey .
2. If x lies within (0, 1), then y = log[x/(1 − x)] is unconstrained andx = 1/(1 + e−y ).
3. If x lies within (−1, 1), then y = log[(1 + x)/(1 − x)] is unconstrainedand x = 2/(1 + e−y ) − 1.
4. For the AR(2) process yt = φ1 yt−1 + φ2 yt−2 + ut to be stationary,
the roots z1 and z2 of the characteristic equation z 2 − φ1 z − φ2 = 0
must lie within the unit circle, with z1 + z2 = φ1 and z1 · z2 = −φ2 .
The following S-PLUS function modifies the GetSsfMS function given earlier in this subsection by employing the above transformations. It now takes
an unconstrained parameter vector and returns the state space representation of Markov switching AR(2) model:
GetSsfMSAR2 = function(parm){
parm = as.vector(parm)#mDelta = mDelta.other = rep(0, 3)

18.5 Markov Switching State Space Models

697

mDelta[1] = parm[1]mDelta.other[1] = parm[1] + exp(parm[2])#AR11 = 2/(1+exp(-parm[3])) - 1
AR12 = 2/(1+exp(-(parm[3]+exp(parm[4])))) - 1
AR21 = 2/(1+exp(-parm[5])) - 1
AR22 = 2/(1+exp(-(parm[5]+exp(parm[6])))) - 1#
mPhi = mPhi.other = matrix(0, 3, 2)
mPhi[1,] = c(AR11+AR12, -AR11*AR12)mPhi.other[1,] = c(AR21+AR22, -AR21*AR22)mPhi[2:3,1] = mPhi.other[2:3,1] = 1#
mOmega = matrix(0, 3, 3)
mOmega[1,1] = exp(parm[7])#mSigma = matrix(0, 3, 2)
mSigma[1:2, 1:2] = diag(1e+6, 2)#
mTrans = matrix(0, 2, 2)
mTrans[1,2] = 1/(1+exp(-parm[8]))mTrans[1,1] = 1 - mTrans[1,2]mTrans[2,1] = 1/(1+exp(-parm[9]))mTrans[2,2] = 1 - mTrans[2,1]#ssf = list(mDelta=mDelta, mDelta.other=mDelta.other,mPhi=mPhi, mPhi.other=mPhi.other, mOmega=mOmega,
mTrans=mTrans, mSigma=mSigma)
CheckSsf(ssf)}
A few comments on the function GetSsfMSAR2 are as follows:
1. The second parameter parm[2] is actually log(μ2 −μ1 ). By employing
this transformation, μ2 is guaranteed to be greater than μ1 , and thus
the first regime can be identified as the low-volatility regime and thesecond as the high-volatility regime.
2. The fourth and sixth parameters, parm[4] and parm[6], are actually
the logarithmic diﬀerence between two characteristic roots of their respective AR(2) processes. By employing this transformation, the firstroots are identified as the smaller roots and the second are identified
as the larger ones.
3. Finally, it is usually preferred to call the CheckSsf function before
returning the list with the state space representation, which makes
sure that the returned list is a valid state space representation.

698

18. Nonlinear Time Series Models

Now, to fit the Markov switching AR(2) model to log(ndx.rvol), use
the following commands20 :> ndx.start = c(-2, -0.7, -0.7, 0.7, -0.7, 0.7, -2, -2, -3)> names(ndx.start) = c("mu1", "mu2", "phi11", "phi12",+ "phi21", "phi22", "sigma", "p", "q")> ndx.msar = SsfFitMS(ndx.start, log(ndx.rvol), GetSsfMSAR2,+ l.start=11)
Iteration 0 : objective = 0.5575044
Iteration 1 : objective = 0.9047186
Iteration 2 : objective = 0.555338...
Iteration 98 : objective = 0.5161791
RELATIVE FUNCTION CONVERGENCE
Note that the first argument to SsfFitMS specifies the starting values,
the second argument specifies the data to be used, and the third argument
specifies the S-PLUS function that takes a vector of model parameters andreturns a valid state space representation of a Markov switching model.
Since the filtering algorithm is started with diﬀuse priors on the statevector, the optional argument l.start is used to start the log-likelihood
function evaluation from the 11th observation, which allows the eﬀects ofdiﬀuse priors on the state vector to dissipate before log-likelihood values
are computed.
The returned object is a "SsfFit" object, and applying the genericsummary function returns the standard errors of the estimated parameters
and associated t-statistics:> class(ndx.msar)[1] "SsfFit"> summary(ndx.msar)
Log-likelihood: -150.724
302 observations
Parameters:
Value Std. Error t value
mu1 -1.8670
0.27600 -6.7640
mu2 -0.9385
1.08100 -0.8684
phi11 -0.3336
0.23730 -1.4060
phi12 0.4073
0.32060
1.2710phi21 -0.8366
0.25960 -3.2230
phi22 0.8109
0.22670
3.5760
20 S+FinMetrics provides the function MSAR for estimating general Markov switching
AR(p) processes. The MSAR function returns an "MSAR" object, and methods for many
generic functions, such as summary, plot, residuals, vcov, and simulate, are provided
for "MSAR" objects. See the online help file for MSAR for details.

18.5 Markov Switching State Space Models

sigma -1.8310
p -5.3150
q -8.4870
Convergence:699

0.08313 -22.0300
1.00900 -5.2670
6.00100 -1.4140
RELATIVE FUNCTION CONVERGENCE

From the above output, most of the parameters are significant according
to the t-statistics. To transform the estimated parameters into the parameters for the Markov switching AR(2) model, simply call GetSsfMSAR2 on
the ML estimates21 :> ndx.ssf = GetSsfMSAR2(ndx.msar$parameters)> cbind(ndx.ssf$mDelta, ndx.ssf$mDelta.other)
[,1]
[,2]
[1,] -1.86719 -1.475965[2,] 0.00000 0.000000[3,] 0.00000 0.000000> ndx.ssf$mPhi
[,1]
[,2]
[1,] 0.3606984 0.08693354[2,] 1.0000000 0.00000000[3,] 1.0000000 0.00000000> ndx.ssf$mPhi.other
[,1]
[,2]
[1,] 0.2130623 0.2406814[2,] 1.0000000 0.0000000[3,] 1.0000000 0.0000000> ndx.ssf$mOmega[,1] [,2] [,3][1,] 0.1601773
00[2,] 0.0000000
00[3,] 0.0000000
00> ndx.ssf$mTrans
[,1]
[,2]
[1,] 0.9951049274 0.004895073[2,] 0.0002061726 0.999793827
Note that the intercept terms in both regimes and the AR coeﬃcientsin the high-volatility regime are similar to those estimated by the SETAR model ndx.setar.r in Section 18.3. However, the AR coeﬃcients inthe low-volatility regime are somewhat diﬀerent from those estimated byndx.setar.r. In addition, both the transition probabilities P11 and P22 areestimated to be very close to 1, which suggests that once yt is in a certain
regime, it tends to stay in that regime.
21 Standard errors for these parameters may be obtained using the delta method.

700

18. Nonlinear Time Series Models

Filtered and Smoothed Estimates of Regime Probabilities
Once the unknown parameters of Markov switching models are estimated,
it is usually of interest to obtain the filtered estimates of the latent discrete
state or regime probability Pr(St = j|Yt ). However, this quantity is alreadycomputed by Kim’s filtering algorithm and, thus, is a side product of the
log-likelihood function evaluation. In addition, it is also of interest to obtainthe smoothed estimates of the latent discrete state probability Pr(St =j|Yn ), which is useful for retrospective analysis. To obtain the smoothed
estimates Pr(St = j|Yn ), note that at time n
Pr(Sn = j, Sn−1 = i|Yn ) = Pr(Sn = j|In ) Pr(Sn−1 = i|Sn = j, Yn )≈ Pr(Sn = j|Yn ) Pr(Sn−1 = i|Sn = j, Yn−1 )Pr(Sn = j|Yn ) Pr(Sn−1 = i, Sn = j|Yn−1 )=Pr(Sn = j|Yn−1 )Pr(Sn = j|Yn ) Pr(Sn−1 = i|Yn−1 ) Pr(Sn = j|Sn−1 = i)=
Pr(Sn = j|Yn−1 )and, thus, the smoothed estimate Pr(Sn−1 = j|Yn ) can be obtained as

Pr(Sn−1 = j|Yn ) =kX

Pr(Sn = j, Sn−1 = i|Yn )i=1

This procedure can be repeated iteratively backward from time n − 1 totime 1 to obtain the smoothed estimates of regime probabilities.
In S+FinMetrics, the filtered and smoothed regime probabilities canbe obtained using the SsfLoglikeMS function with the optional argument
save.regm set to TRUE. For example, the following commands plot thefiltered and smoothed estimates of regime probabilities based on the fit
ndx.msar:> ndx.f = SsfLoglikeMS(log(ndx.rvol), ndx.ssf, save.rgm=T)> par(mfrow=c(2,1))> plot(timeSeries(ndx.f$regimes[,1], pos=positions(ndx.rvol)),+ reference.grid=F, main="Filtered Low Vol Regime Prob")> plot(timeSeries(ndx.f$regimes[,3], pos=positions(ndx.rvol)),+ reference.grid=F, main="Smoothed Low Vol Regime Prob")and the plot is shown in Figure 18.11. The smoothed regime probabilities
suggest that there is actually an abrupt switch around the first quarter of2000.

18.6 An Extended Example: Markov Switching Coincident Index

701

0.1

0.3

0.5

0.7

0.9

Filtered Low Vol Regime Prob

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
1996
1997
1998
1999
2000
2001

0.1

0.3

0.5

0.7

0.9

Smoothed Low Vol Regime Prob

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
1996
1997
1998
1999
2000
2001

FIGURE 18.11. Filtered and smoothe regime probabilities of NASDAQ realized
volatility.

18.6 An Extended Example: Markov Switching
Coincident Index
The United States Department of Commerce periodically publishes the Index of Coincident Economic Indicators (CEI) based on four macroeconomic
coincident variables, which provides a composite measure of the generalstate of the economy. The method used for the construction of the coincident index is ad doc, and the coincident index is subject to revisions after
it is published. To provide a systematic probabilistic model for building analternative coincident index, Stock and Watson (1991) have developed adynamic factor model using a state space representation that models thecoincident index as a common factor driving the four macroeconomic coincident variables: industrial production (IP), total personal income lesstransfer payments (PI), total manufacturing and trade sales (Sales), andemployees on nonagricultural payrolls (Payroll). Stock and Watson (1991)
show that their probabilistic coincident index matches very well with theIndex of CEI compiled by the Department of Commerce.
Stock and Watson’s dynamic factor model has been extended by Kim
and Yoo (1995), Chauvet (1998), and Kim and Nelson (1998) to allow for
Markov switching dynamics in the common factor that represents the coincident index. In addition to matching very well with the Index of CEI

702

18. Nonlinear Time Series Models

compiled by the Department of Commerce, the Markov switching coincident index is also shown to capture the economic expansions and recessions in the U.S. economy as classified by the National Bureau of Economic
Research (NBER). Chauvet and Potter (2000) have developed coincident
indicators for the U.S. stock market using the same methodology.
This section is provided to show how the Markov switching coincident
index model can be represented as a Markov switching state space modeland estimated using the functions in S+FinMetrics.

18.6.1 State Space Representation of Markov Switching
Coincident Index Model
Since the levels of most macroeconomic variables are usually found to be
nonstationary (e.g., see Nelson and Plosser, 1982), it is reasonable to assume that the coincident index representing the state of the economy is also
nonstationary. Thus, in this example the growth rates of the four macroeconomic variables ∆yt are modeled and they are assumed to be driven by
a common factor ∆Ct interpreted as the change in the coincident index:∆yt = β + λ1 ∆Ct + λ2 ∆Ct−1 + et

(18.43)where ∆yt , β, λ1 , λ2 , and et are 4 × 1 vectors with
⎡ ⎤
⎡ ⎤
λ1
0
⎢ λ2 ⎥⎢ 0 ⎥⎥⎢ ⎥λ1 = ⎢⎣ λ3 ⎦ , λ2 = ⎣ 0 ⎦λ42
λ41

Thus, the four macroeconomic coincident variables are driven by the common factor ∆Ct and idiosyncratic components et . Note that only the current value of ∆Ct aﬀects the first three variables (IP, PI, and Sales) in∆yt , while both ∆Ct and ∆Ct−1 aﬀect the last variable (employees on
nonagricultral payroll) because the employment data tend to lag other coincident variables.
The idiosyncratic components are assumed to be independent of eachother and are assumed to follow simple AR(1) models:et = Ψet−1 + ²t , ²t ∼ N (0, σ 2 )(18.44)

where Ψ is a diagonal matrix with (ψ 1 , ψ 2 , ψ 3 , ψ 4 ) on the diagonal andσ 2 is a diagonal matrix with (σ 21 , σ 22 , σ 23 , σ 24 ) on the diagonal. The common
factor ∆Ct is assumed to follow a Markov switching AR(2) process:∆Ct = δ St + φ1 ∆Ct−1 + φ2 ∆Ct−2 + ut , ut ∼ N (0, σ 2C )(18.45)where the unobserved discrete state variable St follows a two-regime Markov
chain, and only the intercept term δ St is Markov switching. When the economy is in a recession (St = 1), the coincident index Ct grows at a slower

18.6 An Extended Example: Markov Switching Coincident Index703

rate δ 1 ; when the economy is in an expansion (St = 2), the coincident index
Ct grows at a faster rate δ 2 .
Note that in the above model, the intercept term β and δ St are notseparately identified and the variance term σ 2C cannot be separated fromthe coeﬃcients λ1 and λ2 . To make the model identifiable, the original
data ∆yt are standardized to remove its mean and make it scale-free sothat β can be set to 0. In addition, the error variance σ 2C for ∆Ct can benormalized to 1. Using αt = (∆Ct , ∆Ct−1 , et , Ct−1 ) as the continuous statevector, the Markov switching coincident index model in (18.43) and (18.45)
can now be written in a state space form with the following representation:⎤⎡⎡⎤
δ St+1
φ1 φ2
0
0
0
0 0
⎢ 0 ⎥
⎢ 1
0
0
0
0
0 0⎥
⎥
⎢
⎢
⎥
⎢ 0 ⎥
⎢ 0
0 ψ1 0
0
0 0⎥
⎥
⎢
⎢
⎥
⎢ 0 ⎥
⎢ 0
0
0 ψ2 0
0 0⎥
⎥
⎢
⎢
⎥
⎢ 0 ⎥
⎢ 0
0
0
0 ψ 3 0 0⎥
⎥
⎢
⎢
⎥
⎥
⎢
0
0
0
0 ψ 4 0⎥
δ St = ⎢⎢ 0 ⎥ , ΦSt = ⎢ 0
⎥
⎢ 0 ⎥
⎢ 1
0
0
0
0
0 1⎥
⎥
⎢
⎢
⎥
⎢ 0 ⎥
⎢ γ1
0
1
0
0
0 0⎥
⎥
⎢
⎢
⎥
⎢ 0 ⎥
⎢ γ2
0
0
1
0
0 0⎥
⎥
⎢
⎢
⎥
⎣ 0 ⎦
⎣ γ3
0
0
0
1
0 0⎦
0
γ 41 γ 42 0
0
0
1 0
and

⎡

1
⎢ 0
⎢
⎢ 0
⎢
⎢ 0
⎢
⎢ 0
⎢
Ω=⎢
⎢ 0
⎢ 0
⎢
⎢ 0
⎢
⎢ 0
⎢
⎣ 0
0

0 0
0 0
0 σ 21
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0

0
0
0
σ 22
0
0
0
0
0
0
0

0
0
0
0
σ 23
0
0
0
0
0
0

0
0
0
0
0
σ 24
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

⎤
0
0 ⎥
⎥
0 ⎥
⎥
0 ⎥
⎥
0 ⎥
⎥
0 ⎥
⎥
0 ⎥
⎥
0 ⎥
⎥
0 ⎥
⎥
0 ⎦
0

Note that Ct = ∆Ct + Ct−1 is also included as one of the state variables,
but does not enter the measurement equation for the observables ∆yt . By
including Ct as one of the state variables, filtered estimates of Ct can bereadily obtained from Kim’s filtering algorithm.
By collecting the unknown model parameters in the vector Θ = (δ 1 , δ 2 ,
φ1 , φ2 , ψ 1 , ψ 2 , ψ 3 , ψ 4 , λ1 , λ2 , λ3 , λ41 , λ42 , σ21 , σ 22 , σ 23 , σ 24 , P12 , P21 ), the following function takes such a vector and returns the state space representationof the model in S-PLUS:
GetSsfCoinIndex = function(parm) {parm = as.vector(parm)

704

18. Nonlinear Time Series Models

mDelta = mDelta.other = rep(0, 11)mDelta[1] = parm[1]mDelta.other[1] = parm[1] + exp(parm[2])#AR.C1 = 2/(1+exp(-parm[3])) - 1
AR.C2 = 2/(1+exp(-(parm[3]+exp(parm[4])))) - 1#
AR.e1 = 2/(1+exp(-parm[5])) - 1
AR.e2 = 2/(1+exp(-parm[6])) - 1
AR.e3 = 2/(1+exp(-parm[7])) - 1
AR.e4 = 2/(1+exp(-parm[8])) - 1#
mPhi = matrix(0, 11, 7)
mPhi[1,1:2] = c(AR.C1+AR.C2, -AR.C1*AR.C2)
mPhi[2,1] = 1
mPhi[3,3] = AR.e1
mPhi[4,4] = AR.e2
mPhi[5,5] = AR.e3
mPhi[6,6] = AR.e4mPhi[7,1] = mPhi[7,7] = 1#
mPhi[8:10,1] = parm[9:11]
mPhi[11,1:2] = parm[12:13]
mPhi[8,3] = mPhi[9,4] = mPhi[10,5] = mPhi[11,6] = 1#
mOmega = matrix(0, 11, 11)mOmega[1,1] = 1
mOmega[3,3] = exp(parm[14])mOmega[4,4] = exp(parm[15])mOmega[5,5] = exp(parm[16])mOmega[6,6] = exp(parm[17])#mTrans = matrix(0, 2, 2)
mTrans[1,2] = 1/(1+exp(-parm[18]))
mTrans[1,1] = 1-mTrans[1,2]mTrans[2,1] = 1/(1+exp(-parm[19]))
mTrans[2,2] = 1-mTrans[2,1]#mSigma = matrix(0, 8, 7)
mSigma[1:7, 1:7] = diag(1e+6, 7)
ans = list(mDelta=mDelta, mDelta.other=mDelta.other,mSigma=mSigma, mOmega=mOmega,
mPhi=mPhi, mTrans=mTrans)
CheckSsf(ans)}18.6 An Extended Example: Markov Switching Coincident Index

705

A few comments on the function GetSsfCoinIndex are in order:
1. The second parameter parm[2] is actually log(δ 2 −δ 1 ). By employing
this transformation, δ 2 is guaranteed to be greater than δ 1 ; thus,
the first regime can be identified as the recessionary regime and thesecond as the expansionary regime.
2. Like in the Markov switching AR(2) model for the NASDAQ realized
volatility, instead of directly estimating the AR(2) coeﬃcients for∆Ct , the two real characteristic roots are estimated and the firstroot is constrained to be the smaller one. By constraining the realcharacteristic roots to lie within the unit circle, the estimated AR(2)process is guaranteed to be stationary and aperiodic.
3. The AR(1) coeﬃcients for the idiosyncratic components are transformed to guarantee that they lie within (−1, 1), and the corresponding AR processes are stationary.
4. The logarithmic variances log σ2i (i = 1, 2, 3, 4) are estimated because
they are unbounded.
5. Like in the Markov switching AR(2) model for the NASDAQ realized
volatility, the transition probabilities P12 and P21 are transformed toguarantee that they lie within (0, 1).
6. Finally, diﬀuse priors on the state vector αt are employed by setting
the top 7 × 7 block of mSigma to a diagonal matrix with large values
on the diagonal and zero in the last row.

18.6.2 Approximate MLE of Markov Switching Coincident
Index
To fit the above Markov switching model to the four coincident variables,
the data are first standardized for model identification and better numericalconvergence:> DOC.dat = getReturns(DOC.ts[,1:4], percentage=T)> DOC.dat@data = t(t(DOC.dat@data) - colMeans(DOC.dat@data))> DOC.dat@data = t(t(DOC.dat@data) / colStdevs(DOC.dat@data))then the SsfFitMS function can be used to fit the model with the following
starting values:> DOC.start = c(-1.5, 0.6, 0.3, 0.1, .1, .1, .1, .1, 0.3,+
0.3, 0.3, 0.3, 0.1, -.5, -.5, -.5, -.5, -1.5, -3)+ names(DOC.start) = c("mu1", "mu2", "phi1", "phi2", "psi1",+"psi2", "psi3", "psi4", "L1", "L2", "L3", "L41",+"L42", "s1", "s2", "s3", "s4", "p", "q")706

18. Nonlinear Time Series Models> DOC.fit = SsfFitMS(DOC.start, DOC.dat, GetSsfCoinIndex,+
l.start=13, trace=T)> summary(DOC.fit)
Log-likelihood: -1998.11
432 observations
Parameters:
Value Std. Error t value
mu1 -1.5650
0.30180 -5.187
mu2 0.6053
0.16900
3.582
phi1 -0.8171
0.20610 -3.965
phi2 0.7124
0.170104.187
psi1 0.3711
0.14940
2.484
psi2 -0.6070
0.10590 -5.731
psi3 -0.5169
0.10930 -4.729
psi4 -0.7584
0.18340 -4.135
L1 0.5059
0.03832 13.200
L2 0.2977
0.031939.322
L3 0.3480
0.03406 10.220
L41 0.4443
0.04013 11.070
L42 0.1966
0.03504
5.610
s1 -1.1590
0.12180 -9.517
s2 -0.2758
0.07225 -3.817
s3 -0.4155
0.07624 -5.449
s4 -1.3940
0.15220 -9.156
p -1.9560
0.52340 -3.738
q -3.7600
0.43460 -8.652
Convergence:

RELATIVE FUNCTION CONVERGENCE

Note that the optional argument l.start to SsfFitMS is used to start loglikelihood evaluation from the 13th observation. From the summary output,
it can be seen that all the estimated model parameters are significantly
diﬀerent from zero.
To transform the parameters into the original model form, simply callthe GetSsfCoinIndex function with the estimated parameters:> DOC.ssf = GetSsfCoinIndex(DOC.fit$parameters)> c(DOC.ssf$mDelta[1], DOC.ssf$mDelta.other[1])[1] -1.565361 0.266435> print(DOC.ssf$mPhi, digits=3)[,1] [,2] [,3]
[,4]
[,5]
[,6] [,7][1,] 0.158 0.211 0.000 0.000 0.000 0.000
0[2,] 1.000 0.000 0.000 0.000 0.000 0.000
0[3,] 0.000 0.000 0.183 0.000 0.000 0.000
0[4,] 0.000 0.000 0.000 -0.295 0.000 0.000
0[5,] 0.000 0.000 0.000 0.000 -0.253 0.000
018.6 An Extended Example: Markov Switching Coincident Index

707

0.1

0.3

0.5

0.7

0.9

Filtered Recession Probability

1960

1965

1970

1975

1980

1985

1990

1995

1985

1990

1995

0.1

0.3

0.5

0.7

0.9

Smoothed Recession Probability

1960

1965

1970

1975

1980

FIGURE 18.12. Filtered and smoothed recession probabilities of Markov switching coincident index.

[6,] 0.000 0.000 0.000[7,] 1.000 0.000 0.000[8,] 0.506 0.000 1.000[9,] 0.298 0.000 0.000[10,] 0.348 0.000 0.000[11,] 0.444 0.197 0.000

0.000
0.000
0.000
1.000
0.000
0.000

0.000 -0.362
0.000 0.000
0.000 0.000
0.000 0.000
1.000 0.000
0.000 1.000

0
1
0
0
0
0

The growth rate of ∆Ct in a recession is estimated to be −1.57, and thegrowth rate in an expansion is estimated to be 0.27. Although the growth
rates of the four macroeconomic variables are positively correlated with
the Markov switching coincident index, only the idiosyncractic componentof industrial production has a positive AR(1) coeﬃcient and all other idiosyncratic components have a negative AR(1) coeﬃcient.
To obtain the filtered and smoothed regime probabilities, simply call the
SsfLoglikeMS function with the estimated state space representation andset the optional argument save.rgm to TRUE:> DOC.f = SsfLoglikeMS(DOC.dat, DOC.ssf, save.rgm=T,+ l.start=13)> DOC.dates = positions(DOC.dat)[-(1:12)]> filt.p = timeSeries(DOC.f$regimes[,1], pos=DOC.dates)> smoo.p = timeSeries(DOC.f$regimes[,3], pos=DOC.dates)> par(mfrow=c(2,1))

708

18. Nonlinear Time Series Models-10 0

10 20 30 40 50

Filtered MS Coincident Index

1960

1965

1970

1975

1980

1985

1990

1995

1985

1990

1995

-6

-4

-2

0 1 2 3

DOC Coincident Index

1960

1965

1970

1975

1980

FIGURE 18.13. Filtered Markov switching coincident index and DOC coincident
index.

> plot(filt.p, reference.grid=F,+ main="Filtered Recession Probability")> plot(smoo.p, reference.grid=F,+ main="Smoothed Recession Probability")
and Figure 18.12 shows the filtered and smoothed probabilities for therecession regime.
To visualize the estimated Markov switching coincident index, note thatthe object DOC.f also has a states component:> names(DOC.f)[1] "loglike" "err""regimes" "states"(j)which contains the filtered estimates of the states αt|t for j = 1, 2. Since
there are seven state variables in the model, the first seven columns cor(1)(2)
respond to αt|t and the next seven columns correspond to αt|t . The following commands plot the weighted average of filtered estimates of Ct andcompare it with the coincident index compiled by the U.S. Department ofCommerce:> DOC.index = lm(DOC.ts@data[,5]~I(1:433))$residuals[-(1:13)]> filt.ci = rowSums(DOC.f$state[,c(7,14)]*DOC.f$regime[,1:2])> filt.ci = timeSeries(filt.ci, pos=DOC.dates)> plot(filt.ci, reference.grid=F,

18.7 References709+ main="Filtered MS Coincident Index")> doc.ci = timeSeries(DOC.index, pos=DOC.dates)> plot(doc.ci, reference.grid=F,+ main="DOC Coincident Index")
and the plot is shown in Figure 18.13. Note that since the Markov switching coincident index is estimated with demeaned data, a time trend is also
removed from the coincident index DOC.ts[,5] compiled by the U.S. Department of Commerce. In general, both series share the same pattern,although the Markov switching coincident index seems to be smoother.

18.7 References
Andersen, T., T. Bollerslev, F.X. Diebold and H. Ebens (2001).
“The Distribution of Realized Stock Return Volatility,” Journal of Financial Economics, 61, 43-76.
Aguilar, O. and M. West (2000). “Bayesian Dynamic Factor Models
and Portfolio Allocation,” Journal of Business and Economic Statistics,
18(3), 338-357.
Barnett, W.A., R.A. Gallant, M.J. Hinich, J.A. Jungeilges, D.T.
Kaplan and M.J. Jensen (1997). “A Single-blind Controlled Competition Among Tests for Nonlinearity and Chaos,” Journal of Econometrics,
82, 157-192.
Brock, W.A., W.D. Dechert and J.A. Scheinkman (1987). “A Test
for Independence Based on the Correlation Dimension,” unpublished manuscript, Department of Economics, University of Wisconsin, Madison.
Brock, W.A., W.D. Dechert, J.A. Scheinkman and B.
LeBaron (1996). “A Test for Independence Based on the Correlation Dimension,” Econometric Reviews, 15, 197-235.
Brock, W.A., D.A. Hsieh and B. LeBaron (1991). Nonlinear Dynamics, Chaos, and Instability: Statistical Theory and Economic Evidence. MIT
Press.
Brock, W.A. and S. Potter (1993). “Nonlinear Time Series and Macroeconomics,” in G.S. Maddala, C. R. Rao and H. D. Vinod (eds.), Handbookof Statistics, Vol. II. North-Holland, Amsterdam.
Caporale, G.M., C. Ntantamis, T. Pantelidis and N. Pittis (2004).
“The BDS Test As a Test for the Adequacy of a GARCH(1,1) Specification:
A Monte Carlo Study,” Working Paper, Brunel University.

710

18. Nonlinear Time Series Models

Chauvet, M. (1998). “An Econometric Characterization of Business Cycle Dynamics with Factor Structure and Regime Switching,” International
Economic Review, 39(4), 969-996.
Chauvet, M. and S. Potter (2000). “Coincident and Leading Indicatorsof the Stock Market,” Journal of Empirical Finance, 7, 87-111.
Davidson, R. and J. G. MacKinnon (1985). “Heteroskedasticity-Robust
Tests in Regressions Directions,” Annales de l’INSEE, 59/60, 183-218.
de Lima, P.J.F. (1996). “Nuisance Parameter Free Properties of Correlation Integral Based Statistics,” Econometric Reviews, 15, 237-259.
Fernandes, M. and P.-Y. Preumont (2002). “The Finite-Sample Sizeof the BDS Test for GARCH Standardized Residuals,” unpublished manuscript, Department of Economics, Queen Mary, University of London.
Franses, P.H. and D. van Dijk (2000). Non-Linear Time Series Models
in Empirical Finance. Cambridge University Press, Cambridge.
Granger, C.W.J. and T. Teräsvirta (1993). Modelling Nonlinear Economic Relationships. Oxford University Press, Oxford.
Hamilton, J.D. 1989. “A New Approach to the Economic Analysis of
Nonstationary Time Series Subject to Changes in Regime,” Econometrica,
57, 357-384.
Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press,
Princeton.
Hansen, B.E. (1996). “Inference When a Nuisance Parameter is Not Identified Under the Null Hypothesis,” Econometrica, 64, 413-430.
Hansen, B.E. (1997). “Inference in TAR Models,” Studies in Nonlinear
Dynamics and Econometrics, 2, 1-14.
Hansen, B.E. (1999). “Testing for Linearity,” Journal of Economic Surveys, 13(5), 551-576.
Kim, C.J. (1994). “Dynamic Linear Models with Markov-Switching,” Journal of Econometrics, 60, 1-22.
Kim, C.-J. and C.R. Nelson (1998). “Business Cycle Turning Points,a New Coincident Index, and Tests of Duration Dependence Based on a
Dynamic Factor Model with Regime-Switching,” Review of Economics andStatistics, 80, 188-201.

18.7 References

711

Kim, C.-J. and C.R. Nelson (1999). State-Space Models with RegimeSwitching: Classical and Gibbs-Sampling Approaches with Applications. MIT
Press, Cambridge, MA.
Kim, M.-J. and J.-S. Yoo (1995). “New Index of Coincident Indicators: A Multivariate Markov Switching Factor Model Approach,” Journalof Monetary Economics, 36, 607-630.
Kim, S., N. Shephard and S. Chib (1998). “Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models,” Review of Economic
Studies, 65, 361-393.
LeBaron, B. (1997). “A Fast Algorithm for the BDS Statistic,” Studies
in Nonlinear Dynamics and Econometrics, 2, 53-59.
Lundbergh, S. and T. Teräsvirta (1998). “Modelling Economic HighFrequency Time Series with STAR-GARCH Models,” Working Paper Seriesin Economics and Finance No. 291, Stockholm School of Economics.
Lundbergh, S. and T. Teräsvirta (2002). “Forecasting with
Smooth Transition Autoregressive Models,” in M. P. Clements and D. F.
Hendry (eds.), A Companion to Economic Forecasting. Blackwell Publishers, London.
Luukkonen, R, P. Saikkonen and T. Teräsvirta (1988). “TestingLinearity Against Smooth Transition Autoregressive Models,” Biometrika
75, 491—499.
Nelson, C.R. and C.I. Plosser (1982). “Trends and Random Walks in
Macroeconomic Time Series: Some Evidence and Implications,” Journal of
Monetary Economics, 10, 139-162.
Teräsvirta, T. (1994). “Specification, Estimation, and Evaluation of
Smooth Transition Autoregressive Models,” Journal of the American Statistical Association, 89, 208—218.
Stock, J.H. and M.W. Watson (1991). “A Probability Model of the Coincident Economic Indicators,” in K. Lahiri and G.H. Moore (eds.), Leading
Economic Indicators: New Approaches and Forecasting Records. Cambridge
University Press, Cambridge.
Tong, H. (1978). “On a Threshold Model,” in C.H. Chen (ed.), Pattern
Recognition and Signal Processing. Sijhoﬀ & Noordhoﬀ, Amsterdam.
Tong, H. (1990). Non-Linear Time Series: A Dynamical System Approach. Oxford University Press, Oxford.

712

18. Nonlinear Time Series Models

Tsay, R.S. (1989). “Testing and Modeling Threshold Autoregressive Processes,” Journal of the American Statistical Association, 84(405), 231-240.
van Dijk, D., T. Teräsvirta and P.H. Franses (2002). “Smooth Transition Autoregressive Models — A Survey of Recent Developments,” Econometric Reviews, 21 (1), 1-47.

This is page 713
Printer: Opaque this19
Copulas19.1 Introduction
Capturing comovement between financial asset returns with linear correlation has been the staple approach in modern finance since the birth ofHarry Markowitz’s portfolio theory. Linear correlation is the appropriate
measure of dependence if asset returns follow a multivariate normal (orelliptical) distribution. However, the statistical analysis of the distributionof individual asset returns frequently finds fat tails, skewness, and othernon-normal features. If the normal distribution is not adequate, then it is
not clear how to appropriately measure the dependence between multiple
asset returns. Fortunately, the theory of copulas provides a flexible methodology for the general modeling of multivariate dependence. As Cherubini,
Luciano, and Vecchiato (2004) state the following in the introduction totheir book: “the copula function methodology has become the most significant new technique to handle the co-movement between markets and risk
factors in a flexible way.”
This chapter gives an overview of the copula function methodology formodeling arbitrary bivariate distributions of asset returns. Attention is restricted to bivariate distributions because the mathematical and statisticaltheory for bivariate copulas is more complete than it is for multivariate
copulas. The bivariate theory also lays the foundation for the general multivariate theory. Section 19.2 provides a motivating example to introduce
the main issues and problems. Section 19.3 defines copulas and gives somebasic properties. The parametric copula classes and families implemented

714

19. Copulas

in S+FinMetrics are reviewed in Section 19.4. Section 19.5 describes fitting
copulas to data, and Section 19.6 discusses risk management using copulas.
The mathematical theory of copulas is covered in the monographs byJoe (1997) and Nelsen (1999). Excellent treatments of copula methods infinance are given in Carmona (2004) and Cherubini, Luciano, and Vecchiato(2004). The latter book contains numerous examples of using copulas forthe analysis of credit risk and for the pricing of derivative securities.
The S+FinMetrics functions for analyzing bivariate copulas are basedon the functions in the EVANESCE (Extreme Value ANalysis Employing
Statistical Copula Estimation) library written by René Carmona and Julia
Morrison and described in Carmona and Morrison (2001), Morrison (2001),
and Carmona (2004).

19.2 Motivating Example
As a motivating example, consider the problem of modeling the unconditional bivariate distribution of the daily log returns on BMW and Siemens
stock, contained in the S+FinMetrics “timeSeries” bmw and siemens,over the period January 2, 1973 through July 23, 1996.1 To simplify the following analysis, these series are combined in the timeSeries “german.ts”:> german.ts = seriesMerge(bmw,siemens)
Time series plots of the returns are given in Figure 19.1. The two returnseries behave similarly over time, exhibit periods of high and low volatility,
and sometimes take on extremely large and small values. For the analysis
in this chapter, the time dependent nature of the return volatility will beignored. Modeling time-dependent volatility is covered in Chapters 7, 13,
and 22.
The two return series have a particular feature that complicates theanalysis of their distributions. This feature is the joint presence of a large
number of zero returns, which result from the prices being constant acrosscertain days. The percentage of common zero returns is> zero.idx = (bmw == 0 & siemens == 0)> sum(zero.idx)/length(bmw)[1] 0.05076473
The presence of these zero returns causes the empirical marginal and joint
distribution functions to have large jumps at 0. Following Carmona (2004),
before analyzing the distribution of returns, the returns are adjusted toeliminate these zero returns. In what follows, the nonzero returns are identified by the logical index variable nz.idx:
1 Carmona (2004), Chapter 2, presents a similar analysis using the S+FinMetrics“timeSeries” columbia.coffee and brazil.coffee.

19.2 Motivating Example

715

-0.10

0.00

0.10

Daily returns on BMW

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1989

1991

1993

1995

-0.12

-0.04

0.04

Daily returns on Siemens

1973

1975

1977

1979

1981

1983

1985

1987

FIGURE 19.1. Daily log returns on BMW and Siemens stock.

> nz.idx = (seriesData(bmw) != 0 & seriesData(siemens) != 0)
Regarding the marginal distribution of returns, Figure 19.2 shows the qq
plots against the normal distribution computed using the S+FinMetrics
function qqPlot. The strong departure from linearity at the ends of the qq
plots indicates non-normal fat-tailed behavior. To be sure, the Jarque-Bera
test> normalTest(german.ts[nz.idx,],method="jb")Test for Normality: Jarque-BeraNull Hypothesis: data is normally distributed
Test Statistics:
bmw
siemens
Test Stat 8371.159 10433.032
p.value
0.000
0.000
Dist. under Null: chi-square with 2 degrees of freedom
Total Observ.: 5350
clearly rejects the null hypothesis that the individual returns are normally
distributed.

716

19. Copulas

-4
bmw

-2

0

2

4

siemens

11/11/1987
0.10

0.05

0.0

-0.05

11/10/1987
08/19/1991-0.10

11/09/1987

10/16/1989

10/16/1989
-0.15

-4

-2

0

2

4

FIGURE 19.2. Normal qq plots of the daily log returns on BMW and Siemens
stocks.

Given that the marginal distributions of the two return series are notnormal, it would be surprising if the bivariate normal is a good characterization of the joint distribution. To see this, Figure 19.3 shows a scatterplot of the actual returns together with a scatter plot of simulated bivariate
normal data calibrated to the actual data. The simulated data is createdusing> mu.hat = colMeans(german.ts[nz.idx,])> Sigma.hat = var(german.ts[nz.idx,])> nobs = numRows(german.ts[nz.idx,])> set.seed(0)> german.sim = rmvnorm(nobs,mean=mu.hat,cov=Sigma.hat)> colIds(german.sim) = colIds(german.ts)
The dependence captured by the bivariate normal distribution is completely described by the (Pearson) correlation coeﬃcient> cor(german.ts[nz.idx,])[1,2][1] 0.6517873
which indicates a moderately strong positive linear dependence. The simulated normal data matches the joint behavior of returns in the middleof the distribution fairly well, but does not capture the observed positive
dependence in the tails of the distribution. Since the bivariate normal dis-

19.2 Motivating Example

-0.15

0.0
-0.05
-0.10

-0.10

-0.05

siemens

Siemens

0.0

0.05

Simulated Normal Returns

0.05

Actual Returns

717

-0.10

-0.05

0.0
BMW

0.05

0.10

-0.15

-0.10

-0.05

0.0

0.05

0.10

bmw

FIGURE 19.3. Scatter plots of actual returns and simulated bivariate normal
returns for BMW and Siemens stocks.

tribution does not adequately describe the joint behavior of returns, thecorrelation coeﬃcient may not be the proper measure of dependence.
As described in Chapter 5, the tails of the marginal distribution of asset
returns can be successfully modeled using the generalized Pareto distribution (GPD). As demonstrated in Carmona (2004), standard nonparametrictechniques based on the empirical distribution function can be used tomodel the center of the distribution. This modeling strategy leads to asemiparametric model (parametric in the tails and nonparametric in thecenter) for the marginal distribution. The S+FinMetrics/ EVANESCE function gpd.tail implements such a semiparametric model. It fits separate
parametric GPDs to the lower and upper tails and uses the empirical CDF
to fit the remaining part of the distribution.
Fitting semiparametric GPD models to the BMW and Siemens returns
requires specification of the lower and upper thresholds. These thresholds
may be inferred from sample mean excess plots and verified by inspecting
plots showing how the GPD shape parameters vary as a function of thespecified thresholds. Figures 19.4 and 19.5 give these plots for the BMW
and Siemens returns. The plots suggest lower and upper threshold valuesof −0.015 and 0.015, respectively, for BMW and lower and upper thresholdvalues of −0.01 and 0.01, respectively, for Siemens. Using these thresh-

718

19. Copulas

0.04

Mean Excess

0.02

0.06
0.02

Mean Excess

0.06

Siemens, lower tail

0.10

BMW, lower tail

-0.05

0.0

0.05

0.10

-0.05

0.0

0.05

ThresholdThresholdBMW, upper tailSiemens, upper tail

0.10
0.02

0.06

Mean Excess

0.10
0.06
0.02

Mean Excess

0.14

-0.10

-0.10

-0.05

0.0

0.05

Threshold

-0.10

-0.05

0.0

0.05

Threshold

FIGURE 19.4. Mean excess plots for the lower and upper tails of daily log returns
on BMW and Siemens stock.

old values, the semiparametric GPDs models for BMW and Siemens are
estimated using2> gpd.bmw.fit2 = gpd.tail(bmw[nz.idx], upper = 0.015,+
lower = -0.015)> gpd.bmw.fit2
Generalized Pareto Distribution Fit -Total of5350

observations

Upper Tail Estimated with ml -Upper Threshold at 0.015 or 12.93 % of the data
ML estimation converged.
Log-likelihood value: 2457

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
2 The L-moment fitting method is used for the Siemens upper tail estimates because
the default ML estimate did not converge. The function gpd.tail does not return estimated standard errors. Use the function gpd to get estimated standard errors.

19.2 Motivating Example
Percent Data Points above Threshold
30

18

10

6

4

Percent Data Points above Threshold
2

52

25

13

7

4

2

0.015

0.020

0.025

1.0

0.8

48

719

0.5

Siemens, lower tail

-0.5

0.0

Estimate of xi

0.4
0.0
-0.4

Estimate of xi

BMW, lower tail

0.0

0.01

0.02

0.03

0.0

0.005

ThresholdThreshold

Percent Data Points above Threshold
31

19

11

7

4

Percent Data Points above Threshold
3

48

14

7

4

2

0.015

0.020

0.025

0.6

Siemens, upper tail

-0.6

0.2
-0.2
-0.6

-0.2

0.2

Estimate of xi

BMW, upper tail
Estimate of xi

27

0.6

52

0.010

0.0

0.01

0.02

0.03

Threshold

0.0

0.005

0.010

Threshold

FIGURE 19.5. Estimated GPD shape parameter ξ, as a function of increasing
thresholds, for the lower and upper tails of daily log returns on BMW and Siemens
stocks.

xi 0.1701
beta 0.0089

NA
NA

NA
NA

Lower Tail Estimated with ml -Lower Threshold at -0.015 or 11.5 % of the data
ML estimation converged.
Log-likelihood value: 2189

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.1901
NA
NA
beta 0.0087
NANA> gpd.siemens.fit2 = gpd.tail(siemens[nz.idx], upper = 0.01,+
lower = -0.01, upper.method="lmom")> gpd.siemens.fit2
Generalized Pareto Distribution Fit -Total of5350

observations

720

19. Copulas

Upper Tail Estimated with lmom -Upper Threshold at 0.01 or 16.19 % of the data
Log-likelihood value: NA

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.1153
NANA
beta 0.0066
NA
NA
Lower Tail Estimated with ml -Lower Threshold at -0.01 or 14.45 % of the data
ML estimation converged.
Log-likelihood value: 2923

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
xi 0.1543
NANA
beta 0.0072
NANA
To evaluate the fit of the GPD model in the tails of the distribution,
the function gpd.tail automatically creates qq plots of excesses over the
specified lower and upper thresholds against the quantiles of the fitted
GPD model. The linearity of the left parts of the plots (not shown) indicates a good fit to the tails of the distributions. The GPD fit may also
be evaluated graphically by examining the plots of the tails of the distributions along with the fitted GPD tails. These plots, produced using the
S+FinMetrics/EVANESCE function tailplot, are given in Figure 19.6 andindicate good fits to the tails of the BMW and Siemens return distributions.
As a further reality check on the overall fit of the semiparametric GPD
models to the entire distribution of the BMW and Siemens returns, simulated returns from the fitted models are generated using> nobs = numRows(bmw[nz.idx])> set.seed(123)> bmw.gpd.sim = gpd.2q(runif(nobs),gpd.bmw.fit2)> siemens.gpd.sim = gpd.2q(runif(nobs),gpd.siemens.fit2)
In the above code, the S+FinMetrics/EVANESCE function gpd.2q is used tocompute quantiles from a fitted semiparametric GPD model. Figure 19.7
shows qq plots of the simulated returns against the actual returns createdusing> par(mfrow=c(1,2))>
qqplot(seriesData(bmw[nz.idx]),bmw.gpd.sim,+
xlab="Actual returns", ylab="Simulated returns")

19.2 Motivating Example

0.10

0.01001-F(x) (on log scale)
0.05

0.00010.0100BMW Upper tail Fit

0.00011-F(x) (on log scale)

BMW Lower tail Fit

721

0.15

0.05

0.10

x (on log scale)

Siemens Lower tail Fit

Siemens Upper tail Fit

0.15

0.05
x (on log scale)

0.10

0.01001-F(x) (on log scale)
0.01

0.00010.00100
0.000011-F(x) (on log scale)

0.10000x (on log scale)

0.01

0.05

0.10

x (on log scale)

FIGURE 19.6. Estimated tails from GPD models fit to lower and upper tails of
the daily log returns on BMW and Siemens stock.

>
abline(0,1)>
title("BMW")>
qqplot(seriesData(siemens[nz.idx]),siemens.gpd.sim,+
xlab="Actual returns", ylab="Simulated returns")>
abline(0,1)>
title("Siemens")> par(mfrow=c(1,1))
The approximate linearity of these plots verifies that the fitted semiparametric GPD models adequately describe the marginal distributions of the
BMW and Siemens returns. The simulated returns from the fitted semiparametric GPD models capture the marginal behavior of actual returns,
but not the joint behavior since the uniform variables used to simulate each
return series are independent. To see this, Figure 19.8 gives the scatter plotof the simulated returns. The occurrence of extreme values observed in theactual returns from Figure 19.3 are reproduced for the marginal distributions, but the dependence of the extreme values is not produced for thebivariate distribution.
The modeling problem introduced in this example involves modeling theindividual marginal distributions of returns to capture fat-tailed behavior,and modeling the dependence structure of the joint behavior to capture theobserved dependence — especially in the tails of the joint distribution. The

722

19. Copulas
Siemens

-0.15

-0.10

-0.10

-0.05

0.0

Simulated returns

0.0
-0.05

Simulated returns

0.05

0.05

0.10

0.10

BMW

-0.15

-0.10

-0.05

0.0

0.05

0.10

-0.10

Actual returns

-0.05

0.0

0.05

Actual returns

FIGURE 19.7. qq plots of actual returns versus simulated returns from fitted
semiparametric GPD models.

marginal distributions are shown to be adequately modeled using semiparametric GPD models. However, it is not clear how to model the joint
dependence given models for the marginal distributions. As the following
sections will show, a flexible way to successfully model the joint behavior of financial returns, after modeling the marginal distributions, is withcopulas.

19.3 Definitions and Basic Properties of Copulas
This section lays out the basic mathematical and statistical theory forbivariate copulas.

19.3.1 Properties of Distributions
Let X be a random variable with distribution function (df) FX (x) =Pr(X ≤ x). For simplicity and ease of exposition, FX is assumed to becontinuous and diﬀerentiable. The density function fX (x) is defined by
Z x
FX (x) =fX (z) dz
0.
so that fX = FX

−∞

723

0.0
-0.10

-0.05

siemens.gpd.sim

0.05

0.10

19.3 Definitions and Basic Properties of Copulas

-0.15

-0.10

-0.05

0.0

0.05

0.10

bmw.gpd.sim

FIGURE 19.8. Scatter plot of simulated bivariate returns from fitted semiparametric GPD models.
−1
Let FX
denote the quantile function−1(α) = inf{x | FX (x) ≥ α}
FX

for α ∈ (0, 1). The following are useful results from probability theory:• FX (x) ∼ U (0, 1), where U (0, 1) denotes a uniformly distributed random variable on (0, 1).
−1• If U ∼ U (0, 1) then FX(U ) ∼ FX .

The latter result gives a simple way to simulate observations from FX−1provided FX
is easy to calculate.
Let X and Y be random variables with marginal dfs (margins) FX andFY , respectively, and joint df
FXY (x, y) = Pr(X ≤ x, Y ≤ y)
In general, the marginal dfs may be recovered from the joint df via
FX (x) = FXY (x, ∞), FY (y) = FXY (∞, y)The joint density fXY is defined by
fXY (x, y) =∂2
FXY (x, y)
∂x ∂y

724

19. Copulas

The random variables X and Y are independent if
FXY (x, y) = FX (x)FY (y)(19.1)

for all values of x and y.

19.3.2 Copulas and Sklar’s Theorem
A bivariate copula is a bivariate df C defined on I 2 = [0, 1] × [0, 1] withuniformly distributed margins; that is,C(u, v) = Pr(U ≤ u, V ≤ v)
where U, V ∼ U (0, 1). As a result, it satisfies the following properties:• C(u, 0) = C(0, v) = 1, C(1, v) = v, C(u, 1) = u for every u, v ∈ [0, 1].
• 0 ≤ C(u, v) ≤ 1.
• For every u1 ≤ u2 , v1 ≤ v2 , and u1 , u2 , v1 , v2 ∈ [0, 1], the following
inequality holds: C(u1 , v1 ) − C(u2 , v1 ) − C(u1 , v2 ) + C(u2 , v2 ) ≥ 0.
The last property ensures that Pr(u1 ≤ U ≤ u2 , v1 ≤ V ≤ v2 ) ≥ 0.
Although not immediately apparent from its definition, the idea of a copula is to separate a joint df FXY into a part that describes the dependence
between X and Y and parts that only describe the marginal behavior. To
see this, X and Y may be transformed into uniform random variables U
and V via U = FX (X) and V = FY (Y ). Let the joint df of (U, V ) be thecopula C. Then, it follows that
FXY (x, y) = Pr(X ≤ x, Y ≤ y)
= Pr(FX (X) ≤ FX (x), FY (Y ) ≤ FY (y))= C(FX (x), FY (x)) = C(u, v)
and so the joint df FXY can be described by the margins FX and FY andthe copula C. The copula C captures the dependence structure between X
and Y .
Sklar’s Theorem
Let FXY be a joint df with margins FX and FY . Then there exists a copula
C such that for all x, y ∈ [−∞, ∞],FXY (x, y) = C(FX (x), FY (y))(19.2)

If FX and FY are continuous, then C is unique. Otherwise, C is uniquely
defined on Range FX × Range FY . Conversely, if C is a copula and FX and19.3 Definitions and Basic Properties of Copulas

725

FY are univariate dfs, then FXY defined in (19.2) is a joint df with margins
FX and FY .
Sklar’s theorem (Sklar, 1959) shows that the copula associated with acontinuous df FXY couples the margins FX and FY with a dependence
structure to uniquely create FXY . As such, it is often stated that the copulaof X and Y is the df C of FX (x) and FY (y).
The copula C of X and Y has the property that it is invariant to strictlyincreasing transformations of the margins FX and FY ; that is, if TX and TY
are strictly increasing functions, then TX (X) and TY (Y )) have the same
copula as X and Y . This property of copulas is useful for defining measuresof dependence.
Examples of Simple Copulas
If X and Y are independent, then their copula satisfies
C(u, v) = u · v(19.3)

The copula (19.3) is called the independent copula or product copula. Itsform follows from the definition of independence given in (19.1).
Suppose that X and Y are perfectly positively dependent or comonotonic. This occurs if
Y = T (X)and T is a strictly increasing transformation. Then the copula for X andY satisfies
C(u, v) = min(u, v)
Notice that this is df for the pair (U, U ), where U ∼ U (0, 1).
Finally, suppose that X and Y are perfectly negatively dependent or
countermonotonic. This occurs if
Y = T (X)and T is a strictly decreasing transformation. Then the copula for X andY satisfies
C(u, v) = max(u + v − 1, 0)
The above is the df for the pair (U, 1 − U ).
The copulas for comonotonic and countermonotonic random variables
form the so-called Fréchet bounds for any copula C(u, v):
max(u + v − 1, 0) ≤ C(u, v) ≤ min(u, v)
Copula Density
The copula density is defined byc(u, v) =∂2
C(u, v)
∂u ∂v

726

19. Copulas

Let FXY be a joint df with margins FX and FY defined by (19.2). Then,
using the chain rule, the joint density of X and Y may be recovered using∂2
FXY (x, y)∂x ∂y∂FX ∂FY∂2=
C(FX (x), FY (y))
∂u ∂v
∂x ∂y
= c(FX (x), FY (y)) · fX (x)fY (y)fXY (x, y) =(19.4)

The above result shows that it is always possible to specify a bivariate
density by specifying the marginal densities and a copula density.

19.3.3 Dependence Measures and Copulas
Dependence measures for financial risk management are nicely surveyed in
Embrechts, Lindskog, and McNeil (2003). For two random variables X andY , they list four desirable properties of a general, single number measureof dependence δ(X, Y ):1. δ(X, Y ) = δ(Y, X).
2. −1 ≤ δ(X, Y ) ≤ 1.
3. δ(X, Y ) = 1 if X and Y are comonotonic; δ(X, Y ) = −1 if X and Y
are countermonotonic.
4. If T is strictly monotonic, then½
δ(X, Y ),δ(T (X), Y ) =−δ(X, Y ),T increasing
T decreasing

They point out that the usual (Pearson) linear correlation only satisfies
the first two properties. They show that the rank correlation measures
Spearman’s rho and Kendall’s tau satisfy all four properties.
Pearson’s Linear Correlation
The Pearson correlation coeﬃcientcov(X, Y )ρ= pvar(X)var(Y )gives a scalar summary of the linear dependence between X and Y . If Y =a + bX, then ρ = ±1. If X and Y are independent, then ρ = 0. Embrechts,
McNeil, and Straumann (2000) summarize the following shortcomings oflinear correlation:• ρ requires that both var(X) and var(Y ) exist.

19.3 Definitions and Basic Properties of Copulas727• ρ = 0 does not imply independence. Only if X and Y are bivariate
normal does ρ = 0 imply independence.
• ρ is not invariant under nonlinear strictly increasing transformations.
• Marginal distributions and correlation do not determine the joint
distribution. This is only true for the bivariate normal distribution.
• For given marginal distributions FX and FY , ρ ∈ [ρmin , ρmax ] and it
may be the case that ρmin > −1 and ρmax < 1.
Kendall’s Tau and Spearman’s Rho
Suppose the random variables X and Y represent financial returns or payoﬀs. It is often the case that both X and Y take either large or small values
together, whereas it is seldom the case that X takes a large value and, atthe same time, Y takes a small value (or vice versa). The concept of concordance is used to measure this type of association. Concordance measures
have the useful property of being invariant to increasing transformationsof X and Y . As a result, concordance measures may be expressed as afunction of the copula between X and Y . Since the linear correlation ρ
is not invariant to increasing transformations of X and Y , it is does notmeasure concordance. Two common measures of concordance are Kendall’s
tau statistic and Spearman’s rho statistic.
Let F be a continuous bivariate df, and let (X1 , Y1 ) and (X2 , Y2 ) be two
independent pairs of random variables from this distribution. The vectors(X1 , Y1 ) and (X2 , Y2 ) are said to be concordant if X1 > X2 wheneverY1 > Y2 and X1 < X2 whenever Y1 < Y2 ; they are said to be discordant in
the opposite case. Kendall’s tau statistic for the distribution F is definedasτ = Pr{(X1 − X2 )(Y1 − Y2 ) > 0} − Pr{(X1 − X2 )(Y1 − Y2 ) < 0}
If C is the copula associated with F , then it can be shown that
Z Z
Z Z
C dC − 1 = 4
C(u, v)c(u, v) du dv − 1
τ =4
I2(19.5)

I2

where c(u, v) is the copula density. The empirical estimate3 of τ for a sampleof size n is the number of the sample’s concordant pairs minus¡ ¢ the numberof discordant pairs divided by the total number of pairs n2 :1τ̂ = ¡n¢2

X

1≤i≤j≤nsign ((xi − xj ) (yi − yj ))(19.6)

3 A pair (u , v ) and (u , v ) of the sample is called concordant if either u < u and
i i
j j
i
j
vi < vj or ui > uj and vi > vj . It is called discordant if either ui < uj and vi > vj or
ui > uj and vi < vj .

728

19. Copulas

For a pair of random variables (X, Y ) with joint df F and marginal
distributions FX and FY , Spearman’s rho statistic, ρS , is defined as the(Pearson) correlation between FX (X) and FY (Y ). It is a measure of rankcorrelation in terms of the integral transforms of X and Y . For a copula
associated with X and Y , it can be shown thatZ Z
ρS = 12C(u, v) du dv − 3(19.7)I2

For a sample of size n, ρS may be estimated using
¶µ
¶
n µ
X
12
n+1
n+1
ρ̂S =
)
−
)
−
rank(xrank(yiin (n2 − 1) i=1
2
2(19.8)

Although both τ and ρS are measures of concordance, their values canbe quite diﬀerent. Nelsen (1999) summarized the relationship between τand ρS with the following inequalities:
1 + 2τ − τ 2
3τ − 1≤ ρS ≤, for τ > 022
τ 2 + 2τ − 1
1 + 3τ≤ ρS ≤, for τ < 02
2
Example 123 Empirical estimates of Kendall’s tau and Spearman’s rho
In S-PLUS, estimates of Kendall’s tau and Spearman’s rho using (19.6)
and (19.8) may be computed using the function cor.test. For example, tocompute these statistics from the daily returns on BMW and Siemens, use> german.tau = cor.test(bmw[nz.idx],siemens[nz.idx],+
method="k")$estimate> german.tau
tau
0.4729532> german.rho = cor.test(bmw[nz.idx],siemens[nz.idx],+
method="spearman")$estimate> german.rho
rho
0.6499582
The estimates τ̂ = 0.4729 and ρ̂S = 0.6499 are both smaller than thePearson correlation estimate ρ̂ = 0.6517.
Tail Dependence Measures
Tail dependence measures are used to capture dependence in the joint tailof bivariate distributions. The coeﬃcient of upper tail dependence may be
defined asλu (X, Y ) = lim Pr(Y > VaRq (Y )|X > VaRq (X))
q→1

19.4 Parametric Copula Classes and Families

729

where the values-at-risk VaRq (X) and VaRq (Y ) denote the 100·qth percent
quantiles of X and Y , respectively. Loosely speaking, λu (X, Y ) measures
the probability that Y is above a high quantile given that X is above ahigh quantile. Similarly, the coeﬃcient of lower tail dependence isλl (X, Y ) = lim Pr(Y ≤ VaRq (Y )|X ≤ VaRq (X))
q→0and measures the probability that Y is below a low quantile given that
X is below a low quantile. It can be shown (Joe, 1997 p. 178) that thecoeﬃcients of tail dependence are functions of the copula C given byλu
λl

1 − 2q + C(q, q)
1−qC(q, q)
= lim
q→0
q
=

lim

q→1

(19.9)(19.10)If λu ∈ (0, 1], then there is upper tail dependence; if λu = 0, then there isindependence in the upper tail. Similarly, if λl ∈ (0, 1], then there is lower
tail dependence; if λl = 0, then there is independence in the lower tail.

19.4 Parametric Copula Classes and Families
In this section, the bivariate parametric copula families implemented in
S+FinMetrics/EVANESCE are defined. Once a copula family is defined, functions are available to construct joint cumulative and probability density
functions, generate random variables, compute Kendall’s tau and Spearman’s rho, compute the tail index parameters, and to estimate the copulaparameters by the method of maximum likelihood.

19.4.1 Normal Copula
One of the most frequently used copulas for financial modeling is the copulaof a bivariate normal distribution with the correlation parameter δ definedbyC(u, v) =Z Φ−1 (u)−∞dxZ Φ−1 (v)−∞= Φδ (Φ−1 (u), Φ−1 (v))

dy

½ 2
¾
1
x − 2δxy + y 2p
exp −2(1 − δ 2 )2π 1 − δ 2(19.11)

where Φ−1 (·) is the quantile function of the standard normal distributionand Φδ is the joint cumulative distribution function of a standard bivariate normal distribution with correlation coeﬃcient δ (0 ≤ δ ≤ 1). From
Sklar’s theorem, the normal copula generates the bivariate standard normal distribution if and only if the margins are standard normal. For any

730

19. Copulas

other margins, the normal copula does not generate a bivariate standard
normal distribution. The normal copula is heavily used in financial applications. See, for example, the technical documents describing J.P. Morgan’s
RiskMetricsTM system (RiskMetrics, 1995).
For the normal copula, Kendall’s tau and Spearman’s rho are given by
τ

=

ρS

=

2
arcsin δπ6δ
arcsinπ2

In addition, except for the case δ = 1, the normal copula does not display
either lower or upper tail dependence:½
0 for δ < 1
λL = λU =1 for δ = 1

19.4.2 Normal Mixture Copula
Consider two pairs of random variables (U1 , V1 ) and (U2 , V2 ) that are
independent of each other. The joint distribution of the two pairs are
given by normal copulas with parameters δ 1 and δ 2 , respectively; that is,(U1 , V1 ) ∼ Cδ1 and (U2 , V2 ) ∼ Cδ2 , where Cδ denotes the normal copula(19.11) with parameter δ. Let (X, Y ) be a random pair such that it is equalto (U1 , V1 ) with probability p and it is equal to (U2 , V2 ) with probability(1 − p). Note that since the marginal distributions of U1 , V1 , U2 ,and V2 are
uniform, so are the marginals of X, and Y . The joint distribution of (X, Y )is given by the following normal mixture copula:C(u, v) = pCδ1 (u, v) + (1 − p)Cδ2 (u, v)
where 0 ≤ p, δ 1 , δ 2 ≤ 1.

19.4.3 Extreme Value Copula ClassA copula is said to be an extreme value (EV) copula if for all t > 0, thescaling property
C(ut , v t ) = (C(u, v))tholds for all (u, v) ∈ I 2 . Let (X1 , Y1 ), (X2 , Y2 ), . . . , (Xn , Yn ) be iid random
pairs from an EV copula C and define Mn = max(X1 , . . . , Xn ) and Nn =max(Y1 , . . . , Yn ). Then C is also a copula associated with the random pair
(Mn , Nn ). This property is called max-stability. In can be shown (e.g. Joe,
1997 p. 175) that EV copulas can be represented in the form
½
µ
¶¾
ln(u)
C(u, v) = exp ln(uv)Aln(uv)

19.4 Parametric Copula Classes and Families

731

where A(·) : [0, 1] → [ 12 , 1] is a convex function such that max(t, t − 1) ≤A(t) ≤ 1 for all t ∈ [0, 1]. The function A(t) is called the dependence
function. The following EV copulas are implemented in S+FinMetrics/
EVANESCE.
Gumbel Copula
Probably the most common EV copula is the Gumbel copula (Gumbel,1960) with parameter δ:noC(u, v) = exp −[(− ln(u)δ + (− ln(v)δ ]1/δ , δ ≥ 1

The dependence function of the Gumbel copula has the form
A(t) = (tδ + (1 − t)δ )1/δ

The parameter δ controls the strength of dependence. When δ = 1, there
is no dependence; when δ = +∞, there is perfect dependence. It can beshown that Kendall’s tau is given byτ = 1 − δ −1
Further, the Gumbel copula exhibits upper tail dependency with
λU = 2 − 21/δ
Galambos Copula
The Galambos copula (Galambos, 1975) with parameter δ has the formnoC(u, v) = uv exp [(− ln(u)−δ + (− ln(v)−δ ]−1/δ , 0 ≤ δ < ∞The dependence function for this copula isA(t) = 1 − (t−δ + (1 − t)−δ )−1/δHüsler and Reiss Copula
The Hüsler and Reiss copula (Hüsler and Reiss, 1987) with parameter δ is
½
∙
µ ¶¸
∙
µ ¶¸¾
1 1
ũ
1 1
ṽ
C(u, v) = exp −ũΦ+ δ ln− ṽΦ+ δ ln
δ 2
ṽ
δ 2
ũ
where 0 ≤ δ ≤ ∞, ũ = − ln u, ṽ = − ln v, and Φ is the df of a standard
normal random variable. The dependence function for this copula is
∙
µ
¶¸
1 1
t
A(t) = tΦ+ δ ln
δ 2
1−t
∙
µ
¶¸
1 1
t
+(1 − t)Φ− δ ln
δ 2
1−t

732

19. Copulas

Twan Copula
The Twan copula (Twan, 1988) is an asymmetric extension of the Gumbel
copula with parameters α, β, and r. It has dependence function
A(t) = 1 − β + (β − α) + {αr tr + β r (1 − t)r }1/r
where 0 ≤ α, β ≥ 1, and 1 ≤ r ≤ ∞.
BB5 Copula
Joe (1997) defined the BB5 copula as a two-parameter extension of theGumbel copula. It has the form
½ hi1/θ ¾θθ−θδ−θδ −1/δ+ ṽ )C(u, v) = exp − ũ + ṽ − (ũwhere δ > 0, θ ≥ 1, ũ = − ln u, and ṽ = − ln v. The dependence function
for this copula isi1/θhA(t) = tθ + (1 − t)θ − (t−δθ + (1 − t)−δθ )−1/δ

19.4.4 Archimedean Copulas
Archimedean copulas (see Nelsen, 1999, Chap. 4) are copulas that may be
written in the formC(u, v) = φ−1 [φ(u) + φ(v)]for a function φ : I → R+ that is continuous, strictly decreasing, andconvex and satisfies φ(1) = 0. The function φ is called the Archimedean
generator , and φ−1 is its inverse function. For example, the Gumbel copula
is an Archimedean copula with generator function φ(t) = (− ln t)δ . The
density of an Archimedean copula may be determined usingc(u, v) =−φ00 (C(u, v)) φ0 (u)φ0 (v)¢3¡ 0
φ (C(u, v))

where φ0 and φ00 denote the first and second derivatives of φ, respectively.
Genest and MacKay (1986) showed that Kendall’s tau may be computed
using
Z
φ(v)
dv + 1τ =4
0
I φ (v)

Prior to their use in financial applications, Archimedean copulas havebeen successfully used in actuarial applications (see Frees and Valdez,
1998). The following Archimedean copulas are implemented in S+FinMetrics/
EVANESCE.

19.4 Parametric Copula Classes and Families

733

Frank Copula
The Frank copula (Frank, 1979) has the following distribution function:¡£¤ ¢C(u, v) = −δ log η − (1 − e−δu )(1 − e−δu ) /η

where 0 < δ < ∞ and η = 1 − e−δ . The generator function is given by
µ −δt
¶
e
−1
φ(t) = − lne−δ − 1The Frank copula does not exhibit lower or upper tail dependency.
Kimeldorf-Sampson (Clayton) Copula

The Kimeldorf and Sampson copula (Kimeldorf and Sampson, 1975) has
the following form:¡¢−1/δ
C(u, v) = u−δ + v −δ − 1

where 0 < δ < ∞, and the generator function isφ(t) = t−δ − 1This copula is also known as the Clayton copula (Clayton, 1978).
Kendall’s tau is given byδτ=
δ+2
and it exhibits only lower tail dependency:
λL = 2−1/δ
Joe Copula
The Joe copula (Joe, 1993) has the form¡
¢1/δC(u, v) = 1 − (1 − u)δ + (1 − v)δ − (1 − u)δ (1 − v)δ

where δ ≥ 1 with generator function

φ(t) = − ln(1 − (1 − t)δ )BB1 Copula
The BB1 copula (Joe, 1997) is given by³£¤−1/θ ´C(u, v) = 1 + (u−θ − 1)δ + (v −θ − 1)δ
with θ > 0 and δ ≥ 1 and generator function

φ(t) = (t−θ − 1)δ

734

19. Copulas

BB2 Copula
The BB2 copula (Joe, 1997) has the form
h
³ −θ
´i1/θ
−θ
C(u, v) = 1 + δ −1 ln eδu + eδv − 1with θ > 0, δ > 0 and generator function−θ

φ(t) = e(t

−1)

−1

BB3 Copula
The BB3 copula (Joe, 1997) has the form of½ h³ θ´i1/θ ¾θC(u, v) = exp − δ −1 ln eδũ + eδṽ − 1with θ > 1, δ > 0, ũ = − ln u, and ṽ = − ln v. The generator function is©
ªφ(t) = exp δ(− ln t)θ − 1

BB6 Copula

The BB6 copula (Joe, 1997) has the form of
µ
½ ∙³
³
´´δ
C(u, v) = 1 − 1 − exp − − ln 1 − (1 − u)θ
)!1/θ
³
³
´´δ ¸1/δθ+ − ln 1 − (1 − v)

where θ ≥ 1, δ ≥ 0 and has generator function´iδh³
φ(t) = − ln 1 − (1 − t)θ

BB7 Copula
The BB7 copula (Joe, 1997) has the form ofµh¡i−1/δ ¶1/θ
¢
¢
¡
θ −δ
θ −δ
C(u, v) = 1 − 1 − 1 − (1 − u)
+ 1 − (1 − v)−1where θ ≥ 1 and δ > 0 and generator function

´−δ
³
−1
φ(t) = 1 − (1 − t)θ

19.4 Parametric Copula Classes and Families

735

19.4.5 Archimax Copulas
Capéraà, Fourgères, and Genest (2000) combined the EV and Archimedean
copula classes into a single class called Archimax copulas. The Archimax
copulas have the form
∙
µ
¶¸
φ(u)
−1
C(u, v) = φ(φ(u) + φ(v)) A
φ(u) + φ(v)where A(t) is a valid dependence function and φ is a valid Archimedean
generator. Archimax copulas reduce to Archimedean copulas for A(t) = 1
and to EV copulas for φ(t) = − ln(t). S+FinMetrics/EVANESCE implements
the following Archimax copula due to Joe (1997).
BB4 Copula
The Archimax copula with
φ(t) = t−θ − 1 and A(t) = 1 − (t−δ + (1 − t)−δ )−1/δ
is called the BB4 copula and has the form³£¤−1/δ ´−1/θC(u, v) = u−θ + v −θ − 1 − (u−θ − 1)−δ + (v −θ − 1)−δwith θ ≥ 0 and δ > 0.

19.4.6 Representation of Copulas in S+FinMetrics
The EVANESCE implementation in S+FinMetrics defines an SV4 object class“copula”, and its child classes “ev.copula”, “archm.copula”, “normal.
copula”, “normal.mix.copula”, “bb4.copula”, and “empirical.copula”.
These classes are referred to as copula types. Each parametric family of copulas discussed in the previous subsection that belongs to one of these copula
types is also defined as a separate subclass and is named “family.name.
copula”. For example, the Gumbel copula family is a “copula” object,with child class “ev.copula” and subclass “gumbell.copula”. If the copula family defines an EV copula, it is considered to be of “ev.copula” type.
If it is an Archimedean copula but not an EV copula, then it inherits from“archm.copula”. A copula’s type will determine what internal functions
are used for various operations and computations on the copula (e.g., generation of random observations from the copula, computation of Kendall’stau, etc.). For example, each parametric copula of class archm.copula has
S+FinMetrics/EVANESCE method functions for computing φ, φ−1 , φ0 , φ00 ,and φ0(−1) . See the online help for copula.object for more details on thesefunctions.
Objects of class “copula” for a particular family are created using theconstructor functions listed in Table 19.1. In addition to the parametric

736

19. Copulas

Function
normal.copula
normal.mix.copula
bb4.copula
gumbel.copula
galambos.copula
husler.reiss.copula
bb5.copula
twan.copula
frank.copula
kimeldorf.sampson.
copulajoe.copula
bb1.copula
bb2.copula
bb3.copula
bb6.copula
bb7.copula

Parameters
delta
p, delta1, delta2
theta, delta
delta
delta
delta
delta, thetaa, b, r
delta
delta

Inherits
copulacopulacopulacopula, ev.copula
copula, ev.copula
copula, ev.copula
copula, ev.copula
copula, ev.copula
copula, archm.copula
copula, archm.copula

theta
theta, delta
theta, delta
theta, delta
theta, delta
theta, delta

copula, archm.copula
copula, archm.copula
copula, archm.copula
copula, archm.copula
copula, archm.copula
copula, archm.copula

TABLE 19.1. Copula constuctor functions

Slot nameparametersparam.names
param.lowbnd
param.upbnd
message

Description
A vector with values of the parameters
Names of the greek letter parameters for the copula
A vector the same length of parameters containing
the values of the lower bounds for the parameters
A vector the same length of parameters containing
the values of the upper bounds for the parameters
Name of the parametric copula family and copula
subclass if applicable
TABLE 19.2. Slots for ”copula” objects

family constructor functions, there are also copula child class constructor
functions ev.copula and archm.copula. Each object of class “copula” hasslots described in Table 19.2.
Example 124 Constructing copula objects.
To create a “copula” object representing a normal copula (19.11) with
δ = 0.7, use the constructor function normal.copula:> ncop.7 = normal.copula(delta=0.7)> class(ncop.7)[1] "normal.copula"> inherits(ncop.7, what="copula")[1] T19.4 Parametric Copula Classes and Families> slotNames(ncop.7)[1] "parameters""param.names"[5] "message"> ncop.7@parameters[1] 0.7> ncop.7@param.lowbnd[1] 0> ncop.7@param.upbnd[1] 1> ncop.7@message[1] "Normal copula family"

737"param.lowbnd" "param.upbnd"

The print method gives a basic description of the copula:> ncop.7
Normal copula family.
Parameters :delta = 0.7
An object representing the Gumbel copula with δ = 2 may be created ina number of ways. It may be created using the gumbel.copula constructor
function:> gumbel.cop.2 = gumbel.copula(delta=2)> inherits(gumbel.cop.2,what="copula")[1] T> inherits(gumbel.cop.2, what="ev.copula")[1] T> gumbel.cop.2
Gumbel copula family; Extreme value copula.
Parameters :delta = 2
Since the Gumbel copula is an EV copula, it may also be created using theev.copula constructor function:> gcop.2 = ev.copula(family="gumbel", param=2)> class(gcop.2)[1] "gumbel.copula"> gcop.2
Gumbel copula family; Extreme value copula.
Parameters :delta = 2
Visualizing Copulas
Table 19.3 lists the S+FinMetrics/EVANESCE method functions for visualizing bivariate copulas. The surface plot of the bivariate CDF C(u, v)

738

19. Copulas

Method Function
persp.pcopula
contour.plot
contour.pcopula
persp.dcopula
contour.dcopula

Description
Surface plot of copula CDF C(u, v)
Plot contours of copula CDF C(u, v)
Surface plot of copula density c(u, v)
Plot contours of copula density c(u, v)

TABLE 19.3. Method functions for visualizing bivariate copulas

produced by the function persp.pcopula is often not very informative
about the dependence properties of a copula. Instead, the contour plots of
the level sets{(u, v) ∈ I 2 : C(u, v) = κ}, κ is constantproduced by the functions contour.plot and contour.pcopula are much
more informative.
Example 125 Visualizing copulas
The normal copula with δ = 0.7 may be visualized using> persp.dcopula(ncop.7)> contour.dcopula(ncop.7)> persp.pcopula(ncop.7)> contour.pcopula(ncop.7)
The resulting figures are shown in Figure 19.9. Notice how the surface plotof the CDF is tent-shaped, with level 0 at the point (0, 0) and level 1 at the
point (1, 1). The level sets of the CDF are convex and their angled shapeindicates moderate dependence. The copula density has two large peaks at
the points (0, 0) and (1, 1) and almost no mass in the upper left and lower
right quadrants of I 2 , which reflects positive dependence.
Figure 19.10 illustrates the normal copula with δ = 0.0001 created by> normal.cop.0 = normal.copula(delta=0.0001)
This copula is essentially the independent copula C(u, v) = uv. Notice how
the level curves of the CDF are less convex than the level curves of thenormal copula with δ = 0.7. Also, the copula density has a relatively flatmass over the interior of I 2 .
Figure 19.11 shows the Gumbel copula with δ = 2. The plots look similar
to those for the normal copula with δ = 0.7. One noticeable diﬀerence is theasymmetric peaks in the density at the points (0, 0) and (1, 1). The Gumbel
density has a much larger peak at the point (1, 1) than at the point (0, 0).
This reflects upper tail dependence in the copula.
Figure 19.12 shows the Kimeldorf-Sampson (Clayton) copula with δ = 2.
The lower tail dependence of this copula is clearly shown in the CDF and19.4 Parametric Copula Classes and Families

4

0.8

3

0.6

CDF

0.4

pdf 2

0.2

1

0.8

0.8
0.8

0.6
0.4
0.2

y

x

0.2

1.0

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

0.8
0.6
0.4
0.2

0.4

0.6

0.8

0.4
0.2

x

0.2

1

678
5
4
3
2

0.8

1.0

0.8
0.6
0.4
0.2
87654 3 2

0.0
0.2

0.6

0.4

1.0

0.0
0.0

0.8

0.6

0.6

0.4

y

y

y

739

1.0

0.0

1

0.2

0.4

x

0.6
x

FIGURE 19.9. Surface and contour plots of C(u, v) and c(u, v) for the normal
copula with δ = 0.7.

0.8
0.6

CDF0.4

pdf 1

0.2

0.8

0.8
0.8

0.6
0.4
0.2

0.2

y

x

1.0

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

y

0.8
0.6
0.4

0.1

0.2

0.8

0.6

0.6

0.4

0.6

0.4

0.4
0.2

0.2

x

1.0 0.9996
0.99980.9999 1.0000 1.0001
0.99970.8
yy1.00041.00031.00020.6
0.4
0.2
1.00030.99970.99961.00021.0001 1.0000 0.99990.9998
0.0 1.0004

0.0
0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

x

FIGURE 19.10. Surface and contour plots of C(u, v) and c(u, v) for the normal
copula with δ = 0.

740

19. Copulas

0.8
6
0.6

CDF

pdf 4

0.4

2

0.2

0.8

0.8
0.8

0.6

y

0.4
0.2

y

x

0.2

1.0

0.6

0.4

0.4
0.2

x

0.2

1.0

0.6
0.4
0.2

18
16
14
12
10
84
6
2

0.8
y

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

0.8
y

0.8

0.6

0.6

0.4

0.6
0.4
0.2

0.0

4

0.0
0.0

0.2

0.4

0.6

0.8

1.0

0.0

2

0.2

0.4

x

0.6

0.8

1.0

x

FIGURE 19.11. Surface and contour plots of C(u, v) and c(u, v) for the Gumbel
copula with δ = 0.

10

0.8

8

0.6

pdf 6

CDF

0.4

4

0.2

2

0.8

0.8
0.8

0.6
0.4
0.2

0.2

y

x

1.0

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

y

0.8
0.6
0.4
0.2
0.0
0.2

0.4

0.6
x

0.8

1.0

0.6

0.4

0.4
0.2

0.2

x

1.0
2

0.8
0.6
0.4
0.2
0.0

0.0

0.8

0.6

0.6

0.4

y

y

10
14
8642
16
18
20
12
24
22
26

0.0

0.2

0.4

0.6

0.8

1.0

x

FIGURE 19.12. Surface and contour plots of C(u, v) and c(u, v) for the Clayton
copula with δ = 0.

19.4 Parametric Copula Classes and Families

741

density plots. The level sets of the CDF are pushed much closer to theorigin where the density has a strong peak.
The reader is encouraged to experiment with plots of the other parametric copula families implemented in S+FinMetrics.
Simulating from Copulas
The S+FinMetrics/EVANESCE method functions pcopula and dcopula compute the copula CDF and pdf at specified points. The function rcoupla
generates random variables with uniform marginal distributions and joint
CDF determined by a specified copula. The algorithms used for simulatingfrom parametric copulas are described in Genest and Revest (1993) and
Cherubini, Luciano, and Vecchiato (2004).
Example 126 Simulating uniform random variables from specified copulas
To compute 2000 random pairs with uniform marginal distributions andjoint distributions specified by the independent copula, normal copula with
δ = 0.7, Gumbel copula with δ = 2 and Clayton copula with δ = 2,
respectively, use> set.seed(123)> normal.7.sim = rcopula(ncop.7,2000)> normal.0.sim = rcopula(normal.cop.0,2000)> gumbel.2.sim = rcopula(gumbel.cop.2,2000)> ks.2.sim = rcopula(ks.cop.2,2000)
Figure 19.13 shows these pairs. Notice how the distribution of the uniformly
distributed pairs in I 2 reflects the dependence structure of the copulas.
Computing Dependence Measures
The dependence measures introduced in Section 19.3 may be computed forselected copulas using the functions Kendalls.tau, Spearmans.rho andtail.index. These functions make use of the fact that for certain copulas, the dependence measures either have closed-form solutions based on
the copula parameters or have representations in terms of the generator
function φ and its inverse and derivatives.
Example 127 Computing dependence measures for various copulas
Kendall’s tau, Spearman’s rho, and the tail index parameters λL and λU
for the normal copula with δ = 0.7 may be computed using> Kendalls.tau(ncop.7)[1] 0.4936334> Spearmans.rho(ncop.7)[1] 0.6829105> tail.index(ncop.7)

742

19. Copulas

0.8
0.6
u
0.4
0.2
0.0

0.0

0.2

0.4

u

0.6

0.8

1.0

Normal Copula, delta=0.7

1.0

Independent Copula

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

v

v

Gumbel Copula, delta=2

Clayton Copula, delta=2

1.0

0.8
0.6
u
0.0

0.0

0.2

0.2

0.4

0.4

u

0.6

0.8

1.0

1.0

0.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

v

0.2

0.4

0.6

0.8

1.0

v

FIGURE 19.13. Random simulations from specified copulas.

lower.tail upper.tail00
The corresponding estimates for the Gumbel copula with δ = 2 are
Kendalls.tau(gcop.2)[1] 0.5> Spearmans.rho(gcop.2)[1] 0.6822338> tail.index(gcop.2)
lower.tail upper.tail
0 0.5857864
For the Gumbel copula, the dependence measures are increasing functionsof δ. For example, setting δ = 10 produces> gcop.10 = gumbel.copula(delta=10)> Kendalls.tau(gcop.10)[1] 0.9> Spearmans.rho(gcop.10)[1] 0.9854924> tail.index(gcop.10)
lower.tail upper.tail
0 0.9282265

19.4 Parametric Copula Classes and Families

Method Function
pbivd
dbivd
rbivd
persp.pbivd
contour.pbivd
persp.dbivd
contour.dbivd

743

Description
Compute CDF for bivariate distribution
Compute pdf for bivariate distribution
Simulate from bivariate distribution
Surface plot of CDF
Contour plot of CDF
Surface plot of pdf
Contour plot of pdf

TABLE 19.4. Method functions for ”bivd” objects

19.4.7 Creating Arbitrary Bivariate Distributions
Consider the problem of creating an arbitrary bivariate distribution for two
random variables X and Y . Sklar’s theorem states that it is always possible
to specify a bivariate distribution F (x, y) by specifying• the marginal distributions FX (x) and FY (y)• a copula C(u, v)
Equations (19.2) and (19.4) show how this is done. In S+FinMetrics/
EVANESCE, the constructor function bivd allows the user to build an arbitrary bivariate distribution from specified margins and a copula. The
arguments expected by bivd are> args(bivd)
function(cop, Xmarg = "unif", Ymarg = Xmarg, param.Xmarg =c(0, 1), param.Ymarg = param.Xmarg)
where cop represents a copula object, Xmarg and Ymarg specify the marginal
distributions for X and Y , respectively, and param.Xmarg and param.Ymarg
specify the parameters defining the marginal distributions. If the marginal
distribution name is specified as “dist”, it is assumed that the functions
ddist, pdist, and qdist are implemented in S-PLUS. For example, if
Xmarg="norm", then the functions dnorm, pnorm, and qnorm should be available for correct performance of the bivariate distribution functions. The distribution names are not limited to those available in standard S-PLUS. The
user may create the appropriate density and quantile functions. The arguments param.Xmarg and param.Ymarg specify the parameters required by
the functions named by Xmarg and Ymarg. For example, if Xmarg="norm",then the user must specify in a vector the values of the parameters meanand sd required by the functions dnorm, pnorm, and qnorm. The function
bivd creates an object of class “bivd” with child class method functions
listed in Table 19.4.
Example 128 Create bivariate distribution with N (3, 42 ) and t3 margins,
and Gumbel copula with δ = 2

744

19. Copulas

To create a “bivd” object representing a bivariate distribution with FX
a normal distribution with μ = 3 and σ = 4, FY a central Student’s-t
distribution with υ = 3 degrees of freedom, and a Gumbel copula with
δ = 2, use> gumbel.biv <- bivd(cop=gumbel.copula(2), Xmarg="norm",+
Ymarg="t", param.Xmarg=c(3,4),+
param.Ymarg=3)> class(gumbel.biv)[1] "gumbel.bivd"
Notice that the class of the object created by bivd is based on the class of
the “copula” object specified by the cop argument. The information forthe marginal normal distribution for X will be computed using the S-PLUS
functions dnorm, pnorm, and qnorm, respectively. The corresponding information for the marginal Student’s-t distribution for Y will be computed
using the S-PLUS functions dt, pt, and qt, respectively. The bivariate distribution represented by the object gumbel.bivd may be visualized using> persp.dbivd(gumbel.biv)> contour.dbivd(gumbel.biv)> persp.pbivd(gumbel.biv)> contour.pbivd(gumbel.biv)
and these plots are given in Figure 19.14.
Joint orthant probabilities of the form Pr{X ≤ x0 , Y ≤ y0 } may becomputed using the function pbivd. For example,> pbivd(gumbel.biv, x = c(-2, 0), y = c(-2, 0))[1] 0.03063639 0.19430934
gives Pr{X ≤ −2, Y ≤ −2} = 0.0306 and Pr{X ≤ 0, Y ≤ 0} = 0.1943,
respectively.
A simulated sample of size 2000 from the bivariate distribution represented by the object gumbel.bivd may be created using the function rbivd
as follows:> set.seed(123)> gumbel.biv.sim = rbivd(gumbel.biv,n=2000)> class(gumbel.biv.sim)[1] "data.frame"> names(gumbel.biv.sim)[1] "x" "y"Some graphical diagnostics of the simulated data are given in Figure 19.15.
The lower right-hand panel shows a scatter plot of the uniform randomvariables computed using u = FX (x) and v = FY (y):> U.x = pnorm(gumbel.biv.sim$x,mean=3,sd=4)> V.y = pt(gumbel.biv.sim$y,df=3)

19.4 Parametric Copula Classes and Families

745

0.05
0.8
0.04
0.6
0.03
Density

CDF

0.4

0.02

0.2

0.01

3

3
2
-1

0

-2

y

0

-1

0

-2
-3

-5

0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.050
0.055

2

5

0

y

x

-2

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

0
-2

-5

0

5
x

10

x

-5

2
y

-3

10

1

5

0

y

2

10

1

-5

0

5

10

x

FIGURE 19.14. Bivariate distribution with N(3, 42 ) and t3 marginals, and Gumbel copula with δ = 2.

This plot reflects the upper tail dependence induced by the Gumbel copula.

19.4.8 Simulating from Arbitrary Bivariate Distributions
In the previous subsection, random draws from an arbitrary bivariate distribution were obtained using the rbivd method function for objects ofclass “bivd”. The creation of a “bivd” object representing an arbitrary
bivariate distribution requires a valid “copula” object and S-PLUS functions for computing the density, cumulative distribution, and quantiles. If
one or more of the S-PLUS distribution functions for the margins are notavailable, then it is not possible to create the bivd object, and simulations
using rbivd cannot be made. However, if S-PLUS functions for computing the quantiles of the marginal distributions are available, then it is still
possible to compute simulated observations from the desired bivariate distribution. The process is as follows4 :1. Create a “copula” object using one of the copula constructor functions from Table 19.1.
4 This process is carried out automatically in the function rbivd from the informationin the “bivd” object.

19. Copulas

0

0

200

100

400

200

600

300

800

400

746

-10

-5

0

5

10

15

-20

-10

0

10

y

v
0.0

-20

0.2

-10

0.4

y

0

0.6

0.8

10

1.0

x

-10

-5

0

5
x

10

15

0.0

0.2

0.4

0.6

0.8

1.0

u

FIGURE 19.15. Graphical diagnostics of simulated data from bivariate distribution with N(3, 42 ) and t3 marginals, and Gumbel copula with δ = 2.

2. Simulate a random sample of univariate pairs from the specified copula using the function rcopula.
3. Use the S-PLUS quantile functions for the marginal distributions evaluated on the simulated univariate pairs to generate observations fromthe desired bivariate distribution.
Example 129 Simulate from bivariate distribution with GPD margins andnormal copula
Figure 19.8 shows a scatter plot of simulated returns from a bivariate
distribution constructed from independent GPD margins fit to the BMW
and Siemens returns. The GPDs capture the tail behavior of the marginal
distributions, but the (implicit) independent copula does not capture theobserved positive dependence in the bivariate distribution. Instead of simulating from a bivariate distribution with GPD margins and an independent
copula, consider simulating from a distribution with the same margins anda normal copula with δ = 0.7:
> n.obs <- seriesLength(bmw[nz.idx])> set.seed(123)> u.data <- rcopula(ncop.7,n.obs)> bmw.cop.sim <- gpd.2q(u.data$x, gpd.bmw.fit2)> siemens.cop.sim <- gpd.2q(u.data$y, gpd.siemens.fit2)

19.5 Fitting Copulas to Data

0.0
Siemens
-0.05
-0.10

-0.10

-0.05

Siemens

0.0

0.05

Simulated Returns from Copula

0.05

Actual Returns

747

-0.15

-0.10

-0.05

0.0

0.05

0.10

-0.15

BMW

-0.10

-0.05

0.0

0.05

0.10

BMW

FIGURE 19.16. Scatter plots of actual returns and simulated returns from abivariate distribution constructed from GPD margins and a normal copula with
δ = 0.7.

In the above code, the S+FinMetrics/EVANESCE function gpd.2q is used
to compute the quantiles of the two-tailed GPD fit to the observed returns.
Figure 19.16 shows scatter plots of the actual returns and the simulated
returns. The simulated data from the bivariate distribution constructedfrom fitted GPD margins and a normal copula with δ = 0.7 mimics theactual returns surprisingly well.

19.5 Fitting Copulas to Data
19.5.1 Empirical Copula
Deheuvels (1978) proposed the following nonparametric estimate of a copula C. Let u(1) ≤ u(2) ≤ · · · ≤ u(n) and v(1) ≤ v(2) ≤ · · · ≤ v(n) be theorder statistics of the univariate samples¡¢ from a copula C. The empiricalcopula Ĉ is defined at the points ni , nj by
Ĉ

µ

i j
,
n n

¶

n

=

1X
1{uk ≤u(i) ,vk ≤v(j) } , i, j = 1, 2, . . . , nn
k=1(19.12)

748

19. Copulas

Deheuvels proved that Ĉ converges uniformly to C as the sample size tends
to infinity. The empirical copula frequency ĉ is given by
µ
¶ ½ 1
i j
if (u(i) , v(j) ) is an element of the samplenĉ,=0 otherwisen nNelsen (1999) showed that estimates of Spearman’s rho and Kendall’s
tau for a sample of size n may be computed from the empirical copula
using
ρ̂S

=

τ̂

=

¶
¸
n ∙ µ
n
12 X X
i j
i j
Ĉ
,
− ·
n2 − 1 j=1 i=1
n n
n n
∙
µ
¶
µ
¶
n
n
i j
2n X Xi−1 j−1Ĉ,Ĉ,n − 1 j=2 i=2
n n
n
n
µ
¶ µ
¶¸
i j−1
i−1 j
−Ĉ
,
Ĉ
,
n n
n n

The tail index parameters (19.9) and (19.10) may be inferred from theempirical copula by plotting
λ̂U (q) =λ̂L (q) =1 − 2q + Ĉ(q, q)
1−qĈ(q, q)q

as functions of q and visually observing convergence as q → 1 and q → 0,
respectively.
Example 130 Empirical copula for BMW and Siemens daily returns
Consider computing the empirical copula (19.12) for the BMW and
Siemens returns based on the estimated semiparametric GPD models forthe marginal distributions. The empirical copula (19.12) requires a sampleof univariate random variables based on the estimated CDFs:(ûi , v̂i ) = (F̂X (xi ), F̂Y (yi )), i = 1, . . . , n(19.13)

From the estimated semiparametric GPD models for F̂X and F̂Y , the uniform pairs (19.13) may be computed using the S+FinMetrics/EVANESCE
function gpd.2p as follows:> U.bmw.gpd <- gpd.2p(bmw[nz.idx], gpd.bmw.fit2)> V.siemens.gpd <- gpd.2p(siemens[nz.idx], gpd.siemens.fit2)
The empirical copula (19.12) may then be computed using the S+FinMetrics/
EVANESCE function empirical.copula as follows:

19.5 Fitting Copulas to Data749

U.bmw.gpd

1.0
0.8

2.5
2.0

0.6
pdf

0.4

1.5
1.0
0.5

0.2
0.0

0.8
0.8

0.6

0.0

0.2

0.4

0.6

0.8

1.0

y

0.6

0.4

0.4
0.2

0.2

x

V.siemens.gpd

0.6

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

y

0.8
0.6
0.4
0.2
0.0

1.0

0.6
0.4
0.2
0.0

0.0

0.2

0.4

0.6

0.8

1.0

x

1.4 2.2
0.2 0.4 0.60.8
1.01.2
1.8
2.0
1.8
1.6

0.8
y

1.0

2.4
2.6
2.22.21.8
2.0
0.80.6 0.4
1.0
1.6
1.4
1.2

0.0

0.2

0.2

0.4

0.6

0.8

1.0

x

FIGURE 19.17. Empirical copula for BMW and Siemens daily returns based on
estimated GPD marginals.

> empcop.bs <- empirical.copula(x = U.bmw.gpd,+
y = V.siemens.gpd)> class(empcop.bs)[1] "empirical.copula"> slotNames(empcop.bs)[1] "x" "y"The values in the slots x and y are the same as the inputs. The function
empirical.copula produces an object of class “empirical.copula” withmethod functions available for visualization, quantile evaluation, and thecomputation of dependence measures. The empirical copula may be visualized using> plot(empcop.bs@x,empcop.bs@y)> contour.pcopula(empcop.bs)> persp.dcopula(empcop.bs)> contour.dcopula(empcop.bs)
and the resulting plots are given in Figure 19.17. Notice that many of thefeatures of the empirical copula are similar to those from a normal copulawith δ = 0.7 (see Figure 19.9).
Estimates of Kendall’s tau, Spearman rho, and the tail index parameters
λL and λU may also be computed from the empirical copula:

750

19. Copulas> Kendalls.tau(empcop.bs)[1] 0.472954> Spearmans.rho(empcop.bs)[1] 0.6499589> tail.index(empcop.bs)
The function tail.index produces a plot (not shown) showing estimatesof λL and λU based on (19.10) and (19.9) as q → 0 and q → 1, respectively.

19.5.2 Maximum Likelihood Estimation
Let (x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ) denote a random sample from a bivariate
distribution F with marginal distributions FX and FY and copula C withdensity c. Recall, the joint density of (xi , yi ) may be represented asf (xi , yi ; η) = c(FX (xi ; αx ), FY (yi ; αy ); θ)fX (xi ; αx )fY (yi ; αy )where αx are the parameters for the marginal distribution FX , αy are theparameters for the marginal distribution FY , θ are the parameters for the0
the copula density c, and η = (α0x , αy , θ0 )0 are the parameters of the joint
density. The exact log-likelihood function is thenl(η; x, y) =nXln c(FX (xi ; αx ), FY (yi ; αy ); θ)(19.14)

i=1

+

n
X
(ln fX (xi ; αx ) + ln fY (yi ; αy ))i=1and the exact maximum likelihood estimator (MLE) is defined asη̂ MLE = arg max l(η; x, y)ηThe numerical computation of the exact MLE may be diﬃcult if there
are many parameters in the marginal models and in the copula. Insteadof maximizing the exact likelihood (19.14) as a function of η, the copulaparameters θ may be estimated using a two-stage procedure. First, themarginal distributions FX and FY are estimated. This could be done usingparametric models (e.g., normal or Student’s-t distributions), the empirical
CDF, or a combination of an empirical CDF with an estimated generalized
Pareto distribution for the tail. Next, given estimates F̂X and F̂Y , a pseudo
sample of observations from the copula(ûi , v̂i ) = (F̂X (xi ), F̂Y (yi )), i = 1, . . . , n
is formed. Then, for a specified parametric copula C(u, v; θ) with densityc(u, v; θ) and unknown parameters θ, the log-likelihoodl(θ : û, v̂) =

n
X
i=1

ln c(ûi, v̂i ; θ)(19.15)

19.5 Fitting Copulas to Data751

is maximized using standard numerical methods. This two-step method,
due to Joe and Xu (1996), is called the inference functions for margins(IFM) method and the resulting estimator of θ is called the IFM estimator(IFME). Properties of the IFME are discussed in Joe (1997) and Cherubini,
Luciano, and Vecchiato (2004). Under standard regularity conditions, theIFME is consistent and asymptotically normally distributed. In particular,
Joe (1997) showed that the IFME often nearly as eﬃcient as the exact
MLE.

19.5.3 Fitting Copulas Using the S+FinMetrics/EVANESCE
Function fit.copula
The IFM for fitting bivariate copulas is implemented in the S+FinMetrics/
EVANESCE function fit.copula. The arguments expected by fit.copulaare> args(fit.copula)function(data, family = "normal", plot = F, init.est = NA,trace = T, scale = 1, gradient = NULL, hessian = NULL,
control = NULL, ...)
where data is an object of class “empirical.copula” and “family” is thename of a parametric family of copulas to fit to the data. Table 19.1 lists thefamilies available to be fit. Setting the optional argument plot=T produces
a contour plot of the empirical copula overlaid with a contour plot of thefitted copula. The remaining parameters control aspects of the optimization
performed by the S-PLUS function nlminb. The function returns an objectof class “copulaFit” for which there are print, IC, and compare methods.
Example 131 Estimate copulas for BMW and Siemens returns
The visualization plots of the empirical copula for the BMW and Siemens
returns, based on the fitted GPDs for the marginal distributions, suggestthat a normal copula may be appropriate. To fit a normal copula withparameter δ by the IFM, use> cop.normal.fit = fit.copula(empcop.bs, family="normal",+plot=T)
Iteration 0 : objective = -1473.311...
Iteration 7 : objective = -1510.05
RELATIVE FUNCTION CONVERGENCE> class(cop.normal.fit)[1] "copulaFit"> names(cop.normal.fit)[1] "data""copula" "vcov""loglike"

752

19. Copulas> cop.normal.fit
Fit Bivariate Copula by MLE --Log-likelihood value:

1510

Normal copula family.
Parameters :delta = 0.6567

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t value
delta
0.65670.0065 101.0596
The IFME of δ is 0.6567, with an asymptotic standard error of 0.0065. The
close match between the contour plot of the fitted copula and the empiricalcopula, given in Figure 19.18, indicates a good fit to the data. Kendall’s
tau and Spearman’s rho for the fitted copula are> Kendalls.tau(cop.normal.fit$copula)[1] 0.4561202> Spearmans.rho(cop.normal.fit$copula)[1] 0.6389838
These values are very close to the empirical estimates from the returns
computed earlier.
For comparison purposes, consider fitting a Gumbel copula with parameter δ to the BMW and Siemens returns:> cop.gumbel.fit <- fit.copula(empcop.bs,family="gumbel",+plot=T)
Iteration 0 : objective = -1348.776
Iteration 5 : objective = -1410.845
RELATIVE FUNCTION CONVERGENCE> cop.gumbel.fit
Fit Bivariate Copula by MLE --Log-likelihood value:

1411

Gumbel copula family; Extreme value copula.
Parameters :delta = 1.7717

1.0

19.5 Fitting Copulas to Data

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.0

0.2

0.4

y

0.6

0.8

0.1

753

0.0

0.2

0.4

0.6

0.8

1.0

x

FIGURE 19.18. Contour plots of empirical copula and fitted normal copula forthe daily returns on BMW and Siemens.

Parameter Estimates, Standard Errors and t-ratios:
Value Std.Error t valuedelta 1.7717 0.0196
90.5103
The IFME of δ is 1.7717, with an asymptotic standard error of 0.0196.
The log-likelihood value of the fitted Gumbel copula is much lower thanthe log-likelihood value of the normal copula. Figure 19.19 shows that thecontours of the fitted Gumbel copula do not match the contours of theempirical copula nearly as well as the normal copula.
The fits of the normal and Gumbel copula may be compared using thecompare.copulaFit method function:> compare.copulaFit(cop.normal.fit, cop.gumbel.fit)
loglike
AIC
BIC
HQ
cop.normal.fit 1510.050 -3018.100 -3011.516 -3015.800
cop.gumbel.fit 1410.845 -2819.689 -2813.104 -2817.389
The normal copula is clearly a better fit to the bivariate returns than the
Gumbel copula. It is emphasized here that the normal copula is a good fit tothe bivariate returns conditional on using semiparametric GPD models for19. Copulas

1.0

754

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.0

0.2

0.4

y

0.6

0.8

0.1

0.0

0.2

0.4

0.6

0.8

1.0

x

FIGURE 19.19. Contour plots of empirical copula and fitted Gumbel copula forthe daily returns on BMW and Siemens.

the marginal distributions. If diﬀerent models are used to fit the marginals,
then the normal copula may not provide the best fit.

19.6 Risk Management Using Copulas
Cherubini, Luciano, and Vecchiato (2004) gave detailed examples of using
copulas to compute risk measures for general portfolios of derivative securities including credit derivatives. In this section, the simple problem ofcomputing common risk measures for a portfolio of two assets using copulas
is described.

19.6.1 Computing Portfolio Risk Measures Using Copulas
Consider a one-period investment in two assets named asset 1 and asset 2.
Let λ1 and λ2 denote the shares of wealth invested in asset 1 and asset 2,
respectively, such that λ1 + λ2 = 1. Let X and Y denote the one-period
continuously compounded (log) returns defined by
X = ln

µ

P1t+1
P1t

¶

, Y = ln

µ

P2t+1
P2t

¶

19.6 Risk Management Using Copulas

755

where P1t and P2t denote the prices of assets 1 and 2 at time period t,
respectively. Then, the one-period log-return on the portfolio is given by
R = ln(λ1 eX + λ2 eY )(19.16)Since R is a nonlinear transformation of the random variables X and Y ,the df of R is generally unknown.
Two common portfolio risk measures, introduced in Chapter 5, are valueat-risk (VaR) and expected shortfall (ES). For a portfolio with random
return R and df FR , the VaR and ES for a given loss probability 1 − q are
VaRq
ESq

−1
= F−R
(q)
= E[−R| − R > VaRq ](19.17)(19.18)Since the df F is generally unknown, analytic expressions for VaRq and
ESq are generally not available.

19.6.2 Computing VaR and ES by Simulation
If it is possible to generate random simulations from the joint df of X and Y ,then it is easy to compute numerical approximations to (19.17) and (19.18)
using (19.16). Let {r1 , . . . , rn } denote a n simulated values of R based on nsimulated random pairs of X and Y . Then the simulation-based estimatesof (19.17) and (19.18) are\q = 100 · qth empirical quantile of − rt values
VaR
dq = mean of − rt values greater than VaR\qES

If the joint df of X and Y is specified by the marginal dfs FX and FY andthe copula C, then simulated values for X and Y may be obtained using
the procedures described in Section 19.4.
Example 132 Simulation-based VaR and ES estimates
Let X and Y denote the daily log-returns on two assets. Consider aportfolio with λ1 = 0.7 and λ2 = 0.3. Suppose X ∼ N (0, 00034, 0.01472 )and Y ∼ N (0, 00021, 0.011402 ). Furthermore, suppose the copula for X
and Y is the Clayton copula with δ = 2. Since the Clayton copula exhibitsa lower tail dependence, there is a greater probability of large negativevalues of X and Y together than is predicted by the bivariate normal
distribution with a similar correlation structure. The following commands
may be used to compute the simulation-based estimates of (19.17) and(19.18) for q = 0.95 based on n = 10000 simulated values of the portfolio
log-return R:> # create bivd object

756

19. Copulas> clayton.biv <- bivd(cop=kimeldorf.sampson.copula(delta=2),+
Xmarg = "norm", Ymarg = "norm",+param.Xmarg=c(0.00034, 0.0147),+
param.Ymarg=c(0.00021, 0.01140))> # simulate from bivariate distn and compute log-return> set.seed(123)> xy.sim = rbivd(clayton.biv,10000)> R.sim = log(0.7*exp(xy.sim$x) + 0.3*exp(xy.sim$y))> VaR.95.est = quantile(-R.sim,probs=0.95)> ES.95.est = -mean(R.sim[-R.sim > VaR.95.est])> VaR.95.est
95%
0.02215953> ES.95.est[1] 0.02776794
With 5% probability, the portfolio could lose 2.216% or more in one day.
If the return is less than −2.216%, then, on average, the loss is 2.777%
Example 133 VaR and ES estimates for portfolio of BMW and Siemens
stocksThe S+FinMetrics/EVANESCE function VaR.exp.sim automates the calculation of VaRq and ESq for a portfolio of two assets when the bivariate
dfs of X and Y are described by GPD margins and a given parametriccopula C. This type of bivariate model with a normal copula was used todescribe the joint df of the daily returns on BMW and Siemens stocks.
Consider a portfolio with 70% of wealth invested in BMW (λ1 = 0.7) andthe remainder invested in Siemens (λ2 = 0.3). Then, simulation-based estimates of (19.17) and (19.18) for q = 0.99 and q = 0.95 based on n = 10, 000
simulated values of the portfolio log-return R may be computed using> VaR.out = VaR.exp.sim(n=10000,Q=c(0.01,0.05),+
copula=cop.normal.fit$copula,+x.est=gpd.bmw.fit2,+
y.est=gpd.siemens.fit2,+
lambda1=0.7,+
lambda2=0.3)> VaR.out
Simulation size VaR Q=0.01 VaR Q=0.05 ES Q=0.01 ES Q=0.05
10000 0.03527972 0.01949605 0.04730781 0.029359

19.7 References757

19.7 References
Capéraà, P., A.-L. Fourgères, and C. Genest (2000). “Bivariate Distributions with Given Extreme Value Attractor,” Journal of Multivariate
Analysis, 72, 30—49.
Carmona, R. (2004). Statistical Analysis of Financial Data in Splus.
Springer-Verlag, New York.
Carmona, R. and J. Morrisson (2001). “Heavy Tails and Copulas with
Evanesce,” ORFE Tech. Report, Princeton University, Princeton, NJ.
Cherubini, U., E. Luciano and W. Vecchiato (2004). Copula Methods
in Finance. John Wiley & Sons, New York.
Clayton, D.G. (1978). “A Model for Association in Bivariate Life Tables
and Its Application in Epidemiological Studies of Familial Tendency inChronic Disease Incidence,” Biometrika, 65, 141-151.
Deheuvels, P. (1978). “Caractérisation Compléte des lois Extrêmes Multivariées et de la Convergence des Types Extêmes,” Publications de l’Institut
de. Statistique de l’Université de Paris, 23, 1—36.
Embrechts, P. C., F. Lindskog and A. McNeil (2003). “Modelling
Dependence with Copulas and Applications to Risk Management,” in S.T.
Rachev (ed.) Handbook of Heavy Tailed Distributions in Finance, Pronide
Publisher.
Embrechts, P., A. McNeil and D. Straumann (2000). “Correlation and Dependence in Risk Management: Properties and Pitfalls,” inM. Dempster and H. K. Moﬀatt (eds.) Risk Management: Value at Risk
and Beyond, Cambridge University Press, Cambridge.
Frank, M.J. (1979). “On the Simultaneous Associativity of f (x, y) andx + y − f (x, y),” Aequationes Mathematiques, 19, 194—226.
Frees, E.W. and E. Valdez (1998). “Understanding Relationships Using
Copulas,” North American Actuarial Journal, 2, 1-25.
Galambos, J. (1975). “Order Statistics of Samples from Multivariate Distributions,” Journal of American Statistical Association, 70, 674—680.
Genest, C. and R. J. MacKay (1986). “Copules Archimédiennes etFamilles de lois Bidimensionnelles dont les Marges sont Données,” Canadian Journal of Statistics, 14, 145—159,

758

19. Copulas

Genest, C. and L.-P. Rivest (1993). “Statistical Inference Proceduresfor Bivariate Archimedean Copulas,” Journal of American Statistical Association, 88(423), 1034—1043.
Gumbel, E. J. (1960). “Distributions des Valeurs Extrêmes en Plusiers
Dimensions,” Publications de l’Institut de. Statistique de l’Université deParis, 9, 171—173.
Hüsler, J. and R.-D. Reiss (1989). “Maxima of Normal Random Vectors: Beween Independence and Complete Dependence, Statistics and Probability Letters, 7, 283—286.
Joe, H. (1993). “Parametric Families of Multivariate Distributions with
Given Margins,” Journal of Multivariate Analysis, 46, 262—282.
Joe, H. (1997). Multivariate Models and Dependence Concepts. Chapman& Hall, London.
Joe, H. and J. Xu (1996). “The Estimation Method of Inference Functions for Margins for Multivariate Models,” Technical Report No. 166, Department of Statistics, University of British Columbia, Vancouver.
Kimeldorf, G. and Sampson, A.R. (1975). “Uniform Representationsof Bivariate Distributions,” Communications in Statistics, 4, 617-627.
Morrison, J.E. (2001). Extreme Value Statistics with Applications in
Hydrology and Financial Engineering, Ph.D. thesis, Princeton University,
Princeton, NJ.
Nelsen, R. B. (1999). An Introduction to Copulas, Lecture Notes in
Statistics. Springer-Verlag, New York.
RiskMetrics, 1995. “RiskMetrics Technical Document,” 3rd ed., J.P. Morgan, New York.
Sklar, A. (1959). “Fonctions de Répartition à n Dimensions et Leurs
Marges,” Publications de l’Institut de. Statistique de l’Université de Paris,8, 229—231.
Tawn, J. A. (1988). “Bivariate Extreme Value Theory: Models and Estimation,” Biometrika, 75, 397—415.

This is page 759
Printer: Opaque this20
Continuous-Time Models for Financial
Time Series

20.1 Introduction
Many stochastic models in modern finance are represented in continuoustime. In these models, the dynamic behavior of the underlying random
factors is often described by a system of stochastic diﬀerential equations(SDEs). The leading example is the option pricing model of Black and Scholes (1973), in which the underlying stock price evolves according to ageometric SDE. For asset pricing purposes, continuous-time financial models are often more convenient to work with than discrete-time models. For
practical applications of continuous-time models, it is necessary to solve,
either analytically or numerically, systems of SDEs. In addition, the simulation of continuous-time financial models is necessary for estimation using
the Eﬃcient Method of Moments (EMM) described in Chapter 23. This
chapter discusses the most common types of SDEs used in continuous-time
financial models and gives an overview of numerical techniques for simulating solution paths to these SDEs.
This chapter begins with two introductory sections: the first giving somebasic background material on SDEs and their solutions, and the second presenting some common numerical schemes for simulating solutions to SDEs
using discretization methods. Then, the S+FinMetrics functions that areavailable for simulating general systems of SDEs, as well as those tailoredto some of the more popular models for financial data, are described andillustrated through examples.

20. Continuous-Time Models for Financial Time Series

4

6

8

10

12

14

16

760

1965

1970

1975

1980

1985

1990

1995

FIGURE 20.1. U.S. 3-month T-Bill rate, observed weekly, 1962 - 1995.

Merton (1990) and Duﬃe (1996) gave comprehensive treatments of continuous -time models in finance. The discussion and treatment of SDEs andmethods for simulating their solutions presented in this chapter is meant to
be intuitive and nontechnical and similar to the treatment in Neftci (1996),
Baxter and Rennie (1996), and Seydel (2002). A thorough technical treatment of SDEs is given in Kloeden and Platten (1999) and Øksendal (1998).

20.2 SDEs: Background
Stochastic diﬀerential equations provide a framework for modeling continuous, dynamically changing quantities that exhibit randomness, or volatility,
such as the U.S. Treasury bill rates shown in Figure 20.1. Applications of
SDEs span the social and physical sciences. See Kloeden and Platten (1999)
and Øksendal (1998) for numerous examples.
An SDE is represented by an equation of the form
dXt = a(t, Xt ) dt + b(t, Xt ) dWt(20.1)

with a deterministic component defined by the function a(t, XT ) (the instantaneous drift) and a random component defined by function b(t, Xt )(the instantaneous diﬀusion). When b ≡ 0, the SDE reduces to an ordinary
diﬀerential equation (ODE). The stochastic diﬀerential dWt represents an20.3 Approximating Solutions to SDEs

761

infinitessimal increment of Brownian motion, Wt , also called the Wiener
process. The Wiener process, first introduced in Chapter 4, is a continuous
random variable such that for s, t ≥ 0, Ws+t − Ws is distributed as normal, N (0, t), and is independent of the history of what the process did up
to s. The formal definition of (20.1) comes from writing down its integral
representation,
Xt = Xt0 +Z tt0a(s, Xs ) ds +Z tb(s, Xs ) dWs(20.2)

t0

and then defining the right-hand stochastic integral in terms of a limitof sums including finite Brownian increments. This leads to the “Ito” or“Stratonovich” calculus, depending on the definition of the integral. For
technical details, see Kloeden and Platen (1999). For a financial approach,see Neftci (1996), and for a readable introduction, see Baxter and Rennie(1996).
A solution Xt to (20.1) or (20.2) is called a stochastic process, which
can be thought of as being indexed by t and the diﬀerent realizations W
of Brownian motion. For fixed t, Xt is a random variable. For a fixed
realization W (t) of Brownian motion, one obtains a sample path X(t).
For example,
dXt = k dt + σ dWt(20.3)
has the explicit solution
Xt = kt + σWt(20.4)

This is just the straight-line solution to dXt = k dt plus some Gaussian
noise. Figure 20.2 shows diﬀerent sample paths for an example of this simple
SDE, and the superimposed straight line.
Equation (20.3) is an example of a scalar SDE. In general, (20.1) may
be vector-valued, with Xt of dimension N and the Brownian motion dWtof dimension M . In that case, the drift is an N -vector and the diﬀusion
is a matrix of dimension N × M . The individual components of Xt are
sometimes called the factors of the model.

20.3 Approximating Solutions to SDEs
Stochastic diﬀerential equations like (20.3) with explicit solutions are rare.
As with ODEs, there are a variety of methods for approximating solutions
using discretization. S+FinMetrics supports three: Euler’s method and theso-called “strong order 1” and “weak order 2” explicit methods of Platen(a reference for this entire section is Kloeden and Platen (1996)).
Euler’s method is the simplest of the three and is a straightforward extension of the ODE technique of the same name. For a discretization step

20. Continuous-Time Models for Financial Time Series

0

200

400

600

762

0

200

400

600

800

1000

FIGURE 20.2. Sample paths for Xt = 12 dt + 5Wt with X0 = 0, and the straightline X = t/2.

size of ∆t and tn = t0 + n∆t, the Euler’s method approximation to (20.1),
Yn ∼= Xtn , is given by
Yn+1 = Yn + a(tn , Yn )∆t + b(tn , Yn )∆Wn

(20.5)

where Y0 = X0 is the initial condition. Increments of the Brownian motion, ∆Wn = Wtn+1 − Wtn , can be simulated using pseudo randomly generated normal variates, since by the defining properties of Brownian motion,∆Wnj = N (0, ∆t) for j = 1, . . . , M . Each discretization step for Euler’s
method thus requires M random normals.
For the other two schemes, the notions of strong and weak convergenceof approximations to solutions of SDEs need to be defined. The former
concerns pathwise approximations, whereas the latter focuses on matching
expectations of moments (or some other functional) of solutions.
A discrete approximation scheme Y δ with discretization step size δ = ∆t
is said to converge strongly to X at time T with order γ > 0 if there exists
a positive constant C that does not depend on δ such that
E(|XT − Y δ (T )|) ≤ Cδ γ

The quantity E(|XT − Y δ (T )|) measures the closeness of sample paths at
the endpoint T .
The approximation Y δ is said to converge weakly with order β > 0 to
X at time T if for every function g of suﬃcient regularity, there exists a20.3 Approximating Solutions to SDEs

763

positive constant C independent of δ such that|E(g(XT )) − E(g(Y δ (T )))| ≤ Cδ β
Thus, weak convergence of order β implies convergence of moments of allorder (taking g(X) = |X|q ).
Under suitable smoothness and growth conditions on the drift and diffusion terms of the SDE, Euler’s method is convergent of strong order
0.5 and weak order 1. Other schemes are usually devised to converge either
strongly or weakly, and it is a matter of the type of problem that one wants
to solve as to which is more appropriate. Higher-order schemes typically involve higher order derivatives of the drift and diﬀusion functions. “Implicit”
approximation schemes require analytic expressions for these higher-orderderivatives, whereas “explicit” schemes use approximations of these derivatives and thus require only the values of the drift and diﬀusion functions
themselves. Only explicit schemes are implemented in S+FinMetrics.
The kth component of the strong order 1 scheme of Platen is given byMX
1
bk,j ∆Wnj + √{bk,j2 (tn , Υjn1 )−bk,j2 }I(j1 ,j2 )∆t
j=1
j1 ,j2 =1(20.6)
with the supporting values√
Υjn = Yn + a∆t + bj ∆tkYn+1= Ynk +ak ∆t+

M
XHere, the a’s and b’s without arguments are taken to be evaluated at(tn , Yn ), and bj denotes the jth column of b. The multiple Ito stochastic
integral
Z
Z
tn+1

tn+1

tn

tn

I(j1 ,j2 ) =dWsj21 dWsj12

can be evaluated exactly when j1 = j2 asI(j1 ,j1 ) =ª
1©
(∆Wnj1 )2 − ∆t2

For j1 6= j2 , however, I(j1 ,j2 ) must be approximated, using
µ
¶
1
√
p
(20.7)
I(j1 ,j2 ) = ∆tξ ξ + ρp (μj1 ,p ξ j2 − μj2 ,p ξ j1 )2 j1 j2p´√√∆t X 1 ³+
ζ j1 ,r ( 2ξ j2 + η j2 ,r ) − ζ j2 ,r ( 2ξ j1 + ηj1 ,r )2π r=1 r
where

p
1
1 X 1
ρp =
−
12 2π 2 r=1 r2

764

20. Continuous-Time Models for Financial Time Seriesand ξ j , μj,p , η j,r , and ζ j,r are independent N (0, 1) and ξ j = √1∆t ∆Wnj ,for j = 1, . . . , M and r = 1, . . . , p. The choice of p aﬀects the accuracy of
the approximation. Gallant (2003) recommended that p > 0.05
∆t . Altogether,each discretization step of (20.6) requires 2M (p + 1) random normals.
That is a lot of overhead. In fact, p > 50 in typical applications, whichmakes very long simulations such as those required by EMM prohibitivelypexpensive using this scheme. However, the computation of I(jcan be
1 ,j2 )
avoided in the special case of diagonal noise; that is, when M = N and b
bj,jis a diagonal matrix with ddX
k = 0 for k 6= j. In that case, the strong order1 scheme reduces to1k= Ynk + ak ∆t + bk,k ∆Wnk + √ {bk,k (tn , Υkn ) − bk,k }{(∆Wnk )2 − ∆t}
Yn+1
2 ∆t
(20.8)
which requires M random normals for each discretization step.
A sub-case of diagonal noise is additive noise, when the diﬀusion b isconstant. In that case, the term on the right of (20.8) drops out and thestrong order 1 scheme reduces to Euler’s method.
The weak order 2 scheme applies to autonomous SDEs; that is, whena = a(Xt ) and b = b(Xt ) are not explicitly dependent on the variable t.
Under that assumption, the weak order 2 scheme is given by1
Yn+1 = Yn + (a(Υ) + a)∆t
2
M ³ ³
´
³ ´
´
X
1
j
j
[ bj R++ bj R−+ 2bj ∆Wnj
+
4 j=1
+

(20.9)

M
³ ³ ´
³ ´
´
X
j
j
bj U+
+ bj U−− 2bj ∆Wnj ]r=1,r6=j+Mo³ ´´ n¡¢2
1X³ j³ j´j[ b R+ − bj R−∆Wnj − ∆t
4 j=1

M
³ ³ ´
³ ´´ ©
X
ª
j
j
+
bj U+
− bj U−∆Wnj ∆Wnr + Vr,j ] (∆t)−1/2
r=1,r6=jwith supporting values
Υ = Yn + a∆t + b∆Wn√
jR±
= Yn + a∆t ± bj ∆t√jU±= Yn + bj ∆tand with ∆Wnj = N (0, ∆t) and Vj1 ,j2 distributed as two-point randomvariables with P (Vj1 ,j2 = ±∆t) = 12 for j2 = 1, . . . , j1 − 1, Vj1 ,j1 = −∆t,and Vj1 ,j2 = −Vj2 ,j1 . Each discretization step thus requires M random20.4 S+FinMetrics Functions for Solving SDEs

765

normals for the {∆Wnj } and M (M − 1)/2 uniform random numbers for the{Vj1 ,j2 }.
In the case of additive noise, the weak order 2 scheme reduces to1(20.10)
Yn+1 = Yn + {a (Yn + a∆t + b∆Wn ) + a} ∆t + b∆Wn .
2This requires M random normals and no random uniforms, per discretization the step.

20.4 S+FinMetrics Functions for Solving SDEs
S+FinMetrics includes functions that implement one or more of the threeschemes just described for simulating sample paths to both general andspecific systems of SDEs. Most of the simulators follow a specific template,as they are designed to be used with the S+FinMetrics function EMM formodel estimation. The calling sequence for these “gensim” functions is asfollows:xxx.gensim(rho, n.sim, n.var, n.burn, aux)
This is a very general framework for supporting simulations of models thatare not necessarily defined by systems of SDEs. Here, rho is a vector ofparameters that define the model to be simulated, n.sim is the length of
the simulation, n.var is the number of variables in the simulation, n.burnis the number of “burn-in” simulation steps that are to be performed before
the n.sim steps to be returned, and aux is a list designed to be a catch all
container for any other parameters that a particular simulator may require.
Usually, aux is supplied by a separate S-PLUS function that is tailored tothe particular gensim simulator. Specific examples in the following sections
will illustrate and further explain these arguments.
Functions for performing simulations for specific systems of SDEs aredescribed in Section 20.4.1. Section 20.4.2 describes functions for simulating
general systems of SDEs.

20.4.1 Problem-Specific Simulators
S+FinMetrics includes several gensim functions that are tailored to specific systems of SDEs, namely the Ornstein-Uhlenbeck (also called Vasicek),
CIR (Cox-Ingersoll-Ross), and IRD (two-factor interest rate diﬀusion). Examine the first example carefully, since the other gensim functions use
many of the same arguments and follow the same behavior.
Ornstein-Uhlenbeck Process
The Ornstein-Uhlenbeck (OU), or Vasicek, process,drt = κ(θ − rt ) dt + σ dWt(20.11)

766

20. Continuous-Time Models for Financial Time Series

is a simple single-factor model for interest rate behavior originally describedin Vasicek (1977). There are three parameters, κ, θ, and σ, which are usually all strictly positive. The long-run behavior of any solution is to hoveraround θ (the level ). This property is called mean reversion; mean-reverting
SDEs are also called stationary. The parameter κ controls the rate of mean
reversion, whereas σ controls the volatility.
The OU SDE has the following exact solution:
Z t
rt = θ + e−κt (r0 − θ) + σeκ(s−t) dWs(20.12)0

The latter stochastic integral must still be simulated. Functions OU.gensim
and OU.aux are available for simulating exact solutions to the OU process
in the gensim framework.
Example 134 Simulate exact solution to the OU process
To simulate 1000 values from (20.12) calibrated to match annualized data
sampled weekly, use# In OU.gensim, rho is packed as c(kappa, theta, sigma)> kappa = .4; theta = .08; sigma = .1
> sim.ou = OU.gensim(rho = c(kappa, theta, sigma),+n.sim = 1000, n.burn = 500,+
aux = OU.aux(X0 = .1, ndt = 25,+
seed = 1, t.per.sim = 1/52))# Plot the simulation> tsplot(sim.ou)# Plot the long-run mean
> abline(h=theta)
See Figure 20.3 for the simulated solution. The auxiliary information for thesimulator is supplied by the S+FinMetrics function OU.aux. The argument
ndt defines the number of discretization steps per simulation step, andargument t.per.sim defines the time length of each simulation step. Theinternal discretization step size is thus ∆t = t.per.sim/ndt. In the above
simulation, the 1000 simulated values of rt are separated by an implied timeof 1/52 (1 week, if the parameters are assumed to generate annualized data),
and successive values are separated by 25 internal discretization steps. Witha burn-in period of 500 simulation steps, that is a total of (1000+100)×25 =27, 500 discretization steps of size ∆t = 1/(52 × 25) ' 0.0008 for the entire
simulation. Notice that the simulated solution generates negative values forrt .
Other arguments to OU.aux, including additional optional arguments not
used in the above example, are listed in Table 20.1. Note that the exact
solution approximation still requires a vector z of random normals (one per
discretization step, the same as Euler’s method) in order to approximate
the stochastic integral in (20.12).

767

-0.1

0.0

0.1

0.2

0.3

20.4 S+FinMetrics Functions for Solving SDEs

0

200

400

600

800

1000

FIGURE 20.3. OU sample path; horizontal line is the long-run mean.

CIR Process
The Cox-Ingersoll-Ross (CIR) (1985) SDE,√
drt = κ(θ − rt ) dt + σ rt dWt(20.13)√is almost identical to the OU model, except that volatility σ rt increases
and decreases with rt , whereas for OU processes it is constant. The CIR
model guarantees that rt is non-negative (as long as κ and θ are strictly
positive). The gensim functions for CIR are CIR.gensim and CIR.aux,
ArgumentX0z

seed
lbound, uboundDescription
Initial condition for the SDE.
Vector of random normals, of lengthndt*(n.sim + n.burn), required by the exact
Approximation. If not specified, then it is generated
internally.
Random number seed, for reproducibility in case z
is generated internally. Default value is 0.
The simulation is truncated to lie in the interval[lbound, ubound]. Default is [-100, 100].
TABLE 20.1. Arguments for OU.aux

768

20. Continuous-Time Models for Financial Time Series

and they always use Euler’s method. Their usage is identical to that ofOU.gensim and OU.aux.
Example 135 Simulate solution to CIR model using Euler’s method
To simulate 1000 values from (20.13) based on Euler’s method calibrated
to match annualized data sampled weekly, use> kappa = .4; theta = .08; sigma = .1
> rho = c(kappa, theta, sigma)# pass in random normals for Brownian motion, although# CIR.gensim can generate these internally on the fly.
> ndt = 25; n.sim = 1000; n.burn = 500> set.seed(1)> z = rnorm(ndt*(n.burn + n.sim))> sim.cir = CIR.gensim(rho = rho, n.sim = n.sim,+
n.burn = n.burn,+aux = CIR.aux(X0 = theta,+
ndt = ndt, z = z,+t.per.sim = 1/52))# Plot the simulation> tsplot(sim.cir)# Plot the long-run mean
> abline(h=theta)
See Figure 20.4 for the simulated solution. Compare with the OU example
with the same parameters of Figure 20.3, where the same random seed,
and hence the same random normal vector, is used to define the discretizedsample path. Note the diﬀerence in scale between the two plots and thatsimulated values for rt from the CIR model are always positive.
The presence of the square root in (20.13) can be problematic for simulation. For although exact solutions to CIR processes are guaranteed to stay
positive, discretization error can lead to negative values for rt under thesquare root sign. The function CIR.gensim handles this by truncating the√
instantaneous standard deviation σ rt at 0 whenever discretized values ofrt are pushed negative.
IRD Process
Gallant and Tauchen (2002) presented the following two-factor interest rate
diﬀusion (IRD) model for modeling the short-term interest rate, st .
¶ µ¶µ
av + avv vt + avs st
dvt=
dt(20.14)dst
as + ass stµ¶µ¶b1v + b1vv vt + b1vs st
dW1,t
b2v
+
(20.15)0(b2s + b2ss st )evt
dW2,t

769

0.05

0.10

0.15

20.4 S+FinMetrics Functions for Solving SDEs

0

200

400

600

800

1000

FIGURE 20.4. CIR sample path; horizontal line is the long-run mean.

The component vt is an unobserved factor contributing to the volatility ofst . There are 11 parameters describing the process. Normalization is usuallyaccomplished by setting av = −avv . Functions IRD.gensim and IRD.aux
simulate solutions to (20.14) and (20.15) using Euler’s method.
Example 136 Simulate solution to two-factor IRD using Euler’s method
To simulate 1000 values of vt and st from (20.14) and (20.15) based on
Euler’s method, calibrated to match annualized data in percent sampledweekly, use# Set values for the parameters> av = .18; avv = -av; avs = -.0088; as = .019; ass = -.0035> b1v = .69; b1vv = 0; b1vs = -.063; b2v = 0; b2s = .038
> b2ss = -.017# Generate the random normals for this simulation> n.sim = 10000; n.burn = 10000; ndt = 25
> set.seed(0)> ird.z = rnorm(2*ndt*(n.sim + n.burn))# IRD.gensim assumes rho is packed as follows. Note that av# is missing from the list, as the normalization av = -avv
# is assumed.

20. Continuous-Time Models for Financial Time Series

0

2

4

6

770

0

2000

4000

6000

8000

10000

FIGURE 20.5. Two factor interest rate diﬀusion sample path. The short rate, st ,appears on top.

> rho = c(avv, avs, as, ass, b1v, b1vv, b1vs, b2v, b2s, b2ss)> ird.sim = IRD.gensim(rho = rho, n.sim = n.sim,+
n.burn = n.burn, n.var = 2,+
aux = IRD.aux(ndt = ndt, t.per.sim = 1/52,+
z = ird.z, X0 = c(1, 5), returnc = c(T, T)))
The simulation is stored in the object ird.sim as a single vector of length
20000 (= n.var*n.sim), with the first two entries corresponding to {vt0 , st0 },followed by {vt1 , st1 }, and so on. The following plots both factors:# Plot v(t) and s(t)> tsplot(t(matrix(ird.sim, nrow = 2)))and the resulting plot is shown in Figure 20.5. Note that the initial conditionargument X0 is a vector for both factors. Argument returnc appears in thisand other auxiliary functions for multi-factor models; it is a logical vectorof length equal to the number of factors describing which factors should be
returned in the simulation. In this case, it is specified that all factors be
returned; the default is to return only the short rate component (returnc= c(F,T)), since the observed factor is the only one that would be used
for estimation purposes. The number of T’s in returnc needs to matchargument n.var.

20.4 S+FinMetrics Functions for Solving SDEs

771

20.4.2 General Simulators
The S+FinMetrics functions euler.pcode.gensim, strong1.pcode.
gensim, and weak2.pcode.gensim simulate sample paths for solutions to
univariate and multivariate SDEs with nearly arbitrary drift and diﬀusion
terms using Euler’s method, the strong order 1 scheme, and the weak order 2 scheme, respectively. The function euler1d.pcode.gensim simulates
solutions to univariate SDE’s using Euler’s method. A pseudo code grammar is used to define mathematical expressions for the drift and diﬀusion.
The price to pay for this flexibility is speed; these functions are two to fivetimes slower than compiled C. Accompanying the pcode.gensim functions
are the pcode.aux functions euler.pcode.aux, strong1.pcode.aux, andweak2.pcode.aux used to specify auxiliary information required for thesimulation. The arguments to euler.pcode.aux and weak2.pcode.aux are
listed in Table 20.2. The arguments to strong1.pcode.aux include the argument p, which corresponds to p in (20.7). The arguments to weak2.pcode.
aux include the argument u for passing in uniform random numbers fordefining the {Vj1 ,j2 } in (20.9) (these are generated internally by default).
These arguments are explained in the following examples.
Example 137 Simulate solution to OU process using pcode.gensim
functions
The auxiliary information required to simulate 1000 values for the solution to the OU process (20.11) calibrated to annualized weekly data is> n.sim = 1000; n.burn = 500; ndt = 25
> set.seed(1)> z = rnorm(ndt*(n.sim + n.burn))> ou.names = c("kappa", "theta", "sigma")> ou.eu.aux1 = euler.pcode.aux(ndt=25, t.per.sim=1/52,+
X0 = 0.1, z = z,+drift.expr= expression(kappa*(theta - X)),+
diffuse.expr = expression(sigma),+
rho.names = ou.names)
The n.sim=1000 simulated values of rt are separated by an implied timeof t.per.sim=1/52 (1 week, since the parameters are assumed to generateannualized data), and successive values are separated by ndt=25 internal
discretization steps. With a burn-in period of n.burn=500 simulation steps,that is a total of (1000 + 100) × 25 = 27, 500 discretization steps of size∆t = 1/(52 × 25) ' 0.0008 for the entire simulation. The random normal variables required for the simulation are precomputed and suppliedin the vector z. The drift and diﬀusion are defined by using the S-PLUS
expression function in the arguments drift.expr and diffuse.expr.
The state vector is referred to by X. The model parameter names are
specified in the vector assigned to the rho.names argument. Notice that772

20. Continuous-Time Models for Financial Time Series

Argument
drift.expr
diffuse.expr
rho.namesMNt.per.sim
ndt
z

u
p

seed
X0
returnnc
lbound,uboundDescription
Expression for drift function.
Expression for diﬀusion function.
Vector of strings that can be used in drift and diﬀuse.
expressions for accessing elements of the rho vector.
Dimension of the Brownian motion process.
Number of factors in the SDE.
Time length of each simulation step.
Number of discretization steps per simulation step.
Vector of random normals. Length of z is:(n.sim + n.burn)ndt*M for Euler’s method;(n.sim + n.burn)2*ndt*M*(p + 1) for
Strong 1; (n.sim + n.burn)*ndt*M for Weak 2.
Vector of random uniforms, for Weak 2 scheme, oflength(n.burn+n.sim)*ndt*M*(M-1)/2.
Value to control approximation to multiple Ito integral
used by Strong order 1 scheme. The default value isceiling(0.05/(t.per.sim/ndt))
Random number seed, for reproducibility in case z
and/or u is generated internally. Default value is 0.
Initial condition for the SDE. Vector of length N.
Logical vector of length N indicating which
components of the simulation to return.
The simulation is truncated to lie in the interval[lbound, ubound]. Default is [-100, 100].

TABLE 20.2. Arguments for pcode.aux functions

the drift and diﬀusion terms are specified in a way that is analogous totheir mathematical representation. For the OU process, these terms area(t, Xt ) = κ(θ − Xt ) and b(t, Xt ) = σ, respectively.1 The expressions forthe drift and diﬀusion support the arithmetic operations +, -, *, /, and ^,basic functions such as trigonometric and exponential, max/min, and logical functions and conditionals. See the online help file for euler.pcode.aux
for a complete list.
The expressions for the drift and diﬀusion specified in the call to euler.
pcode.aux may be tested using the function euler.pcode.test. For example, to evaluate the drift and diﬀusion for the OU process with parametersκ = 0.4, θ = 0.08, and σ = 0.1 at t = 0 and r0 = 0.08, use> rho.test = c(0.4, 0.08, 0.1)
> euler.pcode.test(rho.test,X=0.08,aux=ou.eu.aux1)
1 If a vector of parameter names is not specified, the parameters are assumed to be ina vector called rho and the components of rho can be used directly in the expressions;for example, rho[1] for kappa, rho[2] for theta and rho[3] for sigma.

20.4 S+FinMetrics Functions for Solving SDEs

773

$drift:[1] 0$diffuse:[,1][1,] 0.1
With the auxiliary information specified, the simulated solution to the
OU process may be computed using either euler1d.pcode.gensim or
euler.pcode.gensim:> sim.ou.eu1 = euler1d.pcode.gensim(rho=c(0.4, 0.08, 0.1),+
n.sim = n.sim, n.burn = n.burn,+aux = ou.eu.aux1)> sim.ou.eu = euler.pcode.gensim(rho = c(0.4, 0.08, 0.1),+
n.sim = n.sim, n.burn = n.burn,+aux = ou.eu.aux1)
The simulated solutions look identical to Figure 20.3. For univariate SDEs,
the function euler1d.pcode.gensim is slightly more eﬃcient than euler.
pcode.gensim. Simulations produced using strong1.pcode.gensim andweak2.pcode.gensim work essentially the same and have nearly the same
arguments. For the OU process, which has additive noise, the strong order1 scheme reduces to Euler’s method.
Some of the simulated values of rt in (20.11) are negative. To create
simulated values for rt that are bounded from below by zero, set lbound=0
in the auxiliary list variable ou.eu.aux1:> ou.eu.aux1$lbound = 0> sim.ou.eu1 = euler1d.pcode.gensim(rho=c(kappa,theta,sigma),+n.sim = n.sim, n.burn = n.burn,+aux = ou.eu.aux1)> tsplot(sim.ou.eu1)
The simulated solution path truncated at zero is illustrated in Figure 20.6.
Example 138 Compare Euler’s method and weak order 2 scheme for OU
process
To compare Euler’s method and the weak order 2 scheme on the OU
process of the previous example, use> sim.ou.wk = weak2.pcode.gensim(rho = c(kappa, theta, sigma),+n.sim = n.sim, n.burn = n.burn, n.var = 1,+
aux = weak2.pcode.aux(ndt = ndt, t.per.sim = 1/52,+
N = 1, M = 1, X0 = .1, z = z,+drift.expr= expression(kappa*(theta - X)),+
diffuse.expr = expression(sigma),20. Continuous-Time Models for Financial Time Series

0.0

0.1

0.2

0.3

0.4

774

0

200

400

600

800

1000

FIGURE 20.6. Simulated OU sample path with negative values of rt replacedwith zeros.

+
rho.names = c("kappa", "theta", "sigma")))# Plot the simulation> tsplot(sim.ou.wk)# Plot the absolute differences with the exact solution.
> tsplot(abs(sim.ou.wk - sim.ou))> lines(abs(sim.ou.eu - sim.ou), col = 2)
Since the OU process has additive noise the weak 2 scheme reduces to(20.10) and no uniform random numbers are required to produce the simulation. The weak order 2 simulation (not shown) looks identical whenplotted next to the exact simulation of Figure 20.3. Figure 20.7 shows absolute diﬀerences between the weak and exact simulations and between
Euler’s method and exact simulations. Note that this is not really a faircomparison, since weak convergence is aiming for better average behavior over all paths, but it is, nevertheless, instructive to see how the two
methods compare on a single path.
Example 139 Simulate solutions to generalized CIR model
Chan, Karolyi, Longstaﬀ, and Sanders (1992), hereafter CKLS, considered estimating the parameters of the continuous-time interest rate diﬀusion modelγ
drt = (α0 + β 0 rt )dt + σ 0 rt 0 dWt(20.16)775

0.0

0.00001

0.000020.000030.00004

20.4 S+FinMetrics Functions for Solving SDEs

0

200

400

600

800

1000

FIGURE 20.7. Comparison of Euler’s method and the weak order 2 scheme forsimulating an OU sample path. The smooth curve is the absolute diﬀerence between the Euler and exact simulations. The more volatile curve is the absolute
diﬀerence between the weak order 2 and exact simulations.

The process (20.16) is sometimes referred to as the generalized CIR process. In (20.16), the drift function (α0 + β 0 rt ) dt may be reparameterized
as κ0 (μ0 − rt ) dt, where α0 = κ0 μ0 and β 0 = −κ0 . The parameter μ0 is
the long-run mean, and the parameter κ0 determines the speed of mean
reversion. The model (20.16) encompasses a variety of models that havebeen proposed for the short-term interest rate. These models and the corresponding parameter restrictions on (20.16) are summarized in Table 20.3:
The Merton model is a simple Brownian motion, the Vasicek model is an
OU process, and the Dothan model may be viewed as a restricted versionof the Merton or Vasicek models. These models all imply constant volatility for rt . The remaining models allow for stochastic volatility. The CIR
SR model is the so-called “square root” model, the GBM is the geometric
Brownian motion model, and the CEV model is the constant elasticity ofvariance model.
Simulated solutions from (20.16) for the nine models in Table 20.3, basedon the estimated parameters from CKLS Table III, using Euler’s method
may be produced using> n.sim.ckls = 306; n.burn.ckls = 0; ndt.ckls = 25
> set.seed(123)> z.ckls = rnorm(ndt*(n.sim.ckls + n.burn.ckls))

776

20. Continuous-Time Models for Financial Time Series

drt = (α + βr)dt + σrtγ dWt
ModelSpecification
Mertondrt = αdt + σdWt
Vasicek
drt = (α + βrt )dt + σdWt
1/2
CIR SR
drt = (α + βrt )dt + σrt dWt
Dothandrt = σdWt
GBM
drt = βrt dt + σrt dWt
Brennan-Schwartz drt = (α + βr)dt + σrt dWt
3/2
CIR VR
drt = σrt dWt
CEV
drt = βrt dt + σrtγ dWt

α

β
0

0
0

0

0
0

0

γ
0
0
1/2
0
1
1
3/2

TABLE 20.3. Parameter restrictions imposed by alternative short-term interest
rate models> gcir.names = c("alpha","beta","sigma","gamma")> gcir.aux=euler.pcode.aux(+
drift.expr = expression(alpha + beta*X),+
diffuse.expr = expression(sigma*X^gamma),+
rho.names = gcir.names,+t.per.sim = 1/12, ndt = 25,+z = z.ckls,+
X0 = 0.067)# generalized cir model> rho.gcir = c(0.0408,-0.5921,sqrt(1.6704),1.4999)> gcir.sim = euler1d.pcode.gensim(rho.gcir,n.sim = n.sim.ckls,+
n.burn = n.burn.ckls, aux = gcir.aux)# merton model> rho.merton = c(0.0055, 0.0, sqrt(0.0004), 0.0)
> merton.sim = euler1d.pcode.gensim(rho.merton,+n.sim = n.sim.ckls, n.burn = n.burn.ckls,aux = gcir.aux)# vasicek model> rho.vasicek = c(0.0154, -0.1779, sqrt(0.0004), 0.0)
> vasicek.sim = euler1d.pcode.gensim(rho.vasicek,+
n.sim = n.sim.ckls, n.burn = n.burn.ckls,aux = gcir.aux)# cir.sr model> rho.cir.sr = c(0.0189, -0.2339, sqrt(0.0073), 0.5)
> cir.sr.sim = euler1d.pcode.gensim(rho.cir.sr,+
n.sim = n.sim.ckls, n.burn = n.burn.ckls,aux = gcir.aux)# dothan model> rho.dothan = c(0.0, 0.0, sqrt(0.1172), 1)> dothan.sim = euler1d.pcode.gensim(rho.dothan,+n.sim = n.sim.ckls, n.burn = n.burn.ckls,aux = gcir.aux)# gbm model> rho.gbm = c(0.0, 0.1101, sqrt(0.1185), 1)

20.4 S+FinMetrics Functions for Solving SDEs
Merton Model

Vasicek Model

0.04

0.10

0.08

0.02

0.06

0.14

0.08

0.20

GCIR Model

0

50

100

150

200

250

300

777

0

50

150

200

250

300

0

50

Dothan Model

100

150

200

250

300

250

300

250

300

GBM Model

0.20
0.05

0.0

0.02

0.04

0.06

0.08

0.10

0.35

CIR SR Model

100

0

50

100

150

200

250

300

0

50

150

200

250

300

0

50

CIR VR Model

100

150

200

CEV Model

0.15
0.05

0.02

0.04

0.06

0.08

0.10

Brennan-Schwartz Model

100

0

50

100

150

200

250

300

0

50

100

150

200

250

300

0

50

100

150

200

FIGURE 20.8. Simulated solutions from GCIR and restricted models based on
Euler’s method> gbm.sim = euler1d.pcode.gensim(rho.gbm,n.sim = n.sim.ckls,+
n.burn = n.burn.ckls,aux = gcir.aux)# bs model> rho.bs = c(0.0242, -0.3142, sqrt(0.1185), 1)> bs.sim = euler1d.pcode.gensim(rho.bs,n.sim = n.sim.ckls,+
n.burn = n.burn.ckls,aux = gcir.aux)# cir.vr model> rho.cir.vr = c(0.0, 0.0, sqrt(1.5778), 1.5)
> cir.vr.sim = euler1d.pcode.gensim(rho.cir.vr,+
n.sim = n.sim.ckls, n.burn = n.burn.ckls,aux = gcir.aux)# cev model> rho.cev = c(0.0, 0.1026, sqrt(0.5207), 1.2795)> cev.sim = euler1d.pcode.gensim(rho.cev,n.sim = n.sim.ckls,+
n.burn = n.burn.ckls,aux = gcir.aux)
The simulated solutions are illustrated in Figure 20.8.
Example 140 Simulate solutions to two-factor IRD
Andersen and Lund (1997) considered the following two-factor extensionof the CKLS model:
d log σ 2t
drt

= κ2 (α − log σ2t )dt + ξdW1t= κ1 (μ − rt )dt + σ t rtγ dW2t , γ > 0(20.17)778

20. Continuous-Time Models for Financial Time Series

where W1t and W2t are independent Brownian motion processes, to model
weekly observations on the U.S. 3-month T-bill rate. The specification implies mean reversion of the interest rate level as well as the (log-) volatility.
Define vt = log σ 2t = 2 log σ t . Then, exp(vt /2) = σ t and (20.17) may be
re-expressed as
dvt
drt= κ2 (α − vt )dt + ξdW1t= κ1 (μ − rt )dt + exp(vt /2)rtγ dW2t , γ > 0(20.18)Let Xt = (vt , rt )0 and Wt = (W1t , W2t )0 so that (23.32) may be expressedin matrix form as
dXt = a(Xt , t)dt + b(Xt , t)dWt
wherea(Xt , t) =µ

κ2 (α − vt )
κ1 (μ − rt )¶, b(X, t) =µ

ξ00 exp(vt /2)rtγ¶(20.19)are the drift and diﬀusion functions.
Simulated solutions for Xt from (20.17) may be easily computed using thegeneral pcode.gensim functions. For example, the auxiliary information
to set the drift and diﬀusion function (20.19) for Euler’s method and to
calibrate the simulation to the weekly data on 3-month T-bills is> ndt = 25; t.per.sim = 1/52; n.var = 2; n.sim = 2000
> n.burn = 100> set.seed(0)> z.euler = rnorm(ndt*n.var*(n.sim + n.burn))> rho.AL2.names = c("k1","alpha","k2","xi","mu","gamma")> ird.AL2.aux = euler.pcode.aux(+
N = n.var, M = n.var,+t.per.sim = t.per.sim, ndt = ndt,+
X0 = c(6,-0.3), z = z.euler,+
returnc = c(T,T),+
rho.names = rho.AL2.names,+
drift.expr = expression(k1*(mu - X[1]),+k2*(alpha-X[2])),+
diffuse.expr = expression((exp(X[2]/2)*X[1]^gamma,+
0.0, 0.0, xi))
The model (20.17) has two random factors and the dimension of Wt is
the same as Xt , so N=2 and M=2. Since the annualized rates are assumed tobe observed weekly, t.per.sim=1/52. The number of discretization points,
ndt, for each Xt is 25. The initial value for Xt is set using X0=c(6,-0.3).
Setting returnc=c(T,T) returns simulated solutions for both rt and vt .
The drift and diﬀusion functions (20.19) are specified in the components

20.4 S+FinMetrics Functions for Solving SDEs

779

drift.expr and diffuse.expr. The parameters used in the expressions
are specified in the component rho.names, and the two state variables in
Xt are accessed as X[1] and X[2], respectively. The drift and diﬀusion
functions evaluated at t = 0, X0 = (6, −0.3) and the parameters estimatedby Andersen and Lund
> rho.AL2 = c(0.163,-0.282,1.04,1.27,5.95,0.544)
are> euler.pcode.test(rho.AL2, N = 2, M = 2, t = 0,+
X = c(6, -0.3), aux = ird.AL2.aux)$drift:[1] -0.00815 0.01872$diffuse:[,1] [,2][1,] 2.281235 0.00
[2,] 0.000000 1.27
To produce 2000 simulated values for rt and vt from (20.17) using Euler’s
method, with 100 burn-in values, based on the parameters estimated byAndersen and Lund, use the following commands:> ird.AL2.sim = euler.pcode.gensim(rho = rho.AL2,+
n.sim = n.sim, n.var = n.var, n.burn = n.burn,+aux = ird.AL2.aux)> class(ird.AL2.sim)[1] "numeric"> length(ird.AL2.sim)[1] 4000
The simulation is stored in the object ird.AL2.sim as a single vector oflength 4000 (= n.var*n.sim), with the first two entries corresponding to{rt0 , vt0 }, followed by {rt1 , vt1 }, and so on. The simulated solutions for rt
and vt may be placed in a 2000 × 2 matrix using> ird.AL2.sim = matrix(ird.AL2.sim, n.sim, n.var, byrow=T)
Figure 20.9 shows the solutions.
Simulations from the strong order 1 scheme require much more overhead
than simulations from Euler’s method. In this example, if p in (20.7) is set
according to the rule
> p.strong = ceiling(0.05/(t.per.sim/ndt))> p.strong[1] 65then the number of random normals to compute for one simulated path is780

20. Continuous-Time Models for Financial Time Series

20
15
10
5

% per year

25

Simulated short rate

0

500

1000

1500

2000

1500

2000

-3

-2

-1

0

1

2

Simulated log-volatility

0

500

1000

FIGURE 20.9. Simulated solutions from (20.17) based on Euler’s method.

> 2*ndt*n.var*(p.strong + 1)*(n.sim + n.burn)[1] 13860000
Euler’s method only requires> ndt*n.var*(n.sim + n.burn)[1] 105000
values. To produce 2000 simulated values for rt and vt from (20.17) using the strong order 1 scheme, with 100 burn-in values, use the following
commands:> set.seed(0)> z.strong = rnorm(2*ndt*n.var*(p.strong + 1)*(n.sim + n.burn))> ird.AL2.aux = strong1.pcode.aux(+
N = n.var, M = n.var,+t.per.sim = t.per.sim, ndt = ndt,+
p = p.strong, z = z.strong,+
X0 = c(6,-0.3),+
returnc = c(T,T),+
rho.names = rho.AL2.names,+
drift.expr = expression(+
k1*(mu - X[1]),+k2*(alpha - X[2])),+
diffuse.expr = expression(

20.4 S+FinMetrics Functions for Solving SDEs

781+
(exp(X[2]/2))*X[1]^gamma,
+
0.0,
+
0.0,
+
xi))
> ird.AL2.simS = strong1.pcode.gensim(rho = rho.AL2,+
n.sim = n.sim, n.var = n.var, n.burn = n.burn,+aux = ird.AL2.aux)> ird.AL2.simS = matrix(ird.AL2.simS, n.sim, n.var, byrow = T)
The overhead for the weak order 2 scheme is in between the overhead for
Euler’s method and the strong order 1 scheme. The weak order 2 scheme
requires> ndt*n.var*(n.sim + n.burn)[1] 105000
random normal variables and> (n.burn+n.sim)*ndt*n.var*(n.var-1)/2[1] 52500
random uniform variables. To produce 2000 simulated values for rt and vt
from (20.17) using the weak order 2 scheme, with 100 burn-in values, use
the following commands:> set.seed(0)> z.weak = rnorm(ndt*n.var*(n.sim + n.burn))> u.weak = runif((n.burn+n.sim)*ndt*n.var*(n.var-1)/2)> ird.AL2.aux = weak2.pcode.aux(+
N = n.var, M = n.var,+t.per.sim = t.per.sim, ndt = ndt,+z = z.weak, u = u.weak,+X0 = c(6,-0.3),+
returnc=c(T,T),+
rho.names = rho.AL2.names,+
drift.expr = expression(+
k1*(mu - X[1]),+k2*(alpha-X[2])),+
diffuse.expr = expression(+(exp(X[2]/2))*X[1]^gamma,
+
0.0,
+
0.0,
+
xi))
> ird.AL2.simW = weak2.pcode.gensim(rho = rho.AL2,+
n.sim = n.sim, n.var = n.var, n.burn = n.burn,+aux = ird.AL2.aux)> ird.AL2.simW = matrix(ird.AL2.simW, n.sim, n.var, byrow=T)782

20. Continuous-Time Models for Financial Time Series
Simulated log-volatility from Strong 1

0

1

10

-2

0

-1

5

% per year

2

15

3

Simulated short rate from Strong 1

0

500

1000

1500

2000

0

1000

1500

2000

Simulated log-volatility from Weak 2

0

20

-1

15

-3

5

-2

10

% per year

1

25

2

Simulated short rate from Weak 2

500

0

500

1000

1500

2000

0

500

1000

1500

2000

FIGURE 20.10. Simulated solutions from (23.32) based on the strong order 1
scheme and the weak order 2 scheme.

Figure 20.10 shows the simulated solution from the strong order 1 andweak order 2 schemes. Notice that the paths from Euler’s method and theweak order 2 scheme are almost identical, whereas the strong order 1 path
is quite diﬀerent.

20.5 References
Andersen, T.G. and J. Lund (1997). “Estimating Continuous-Time
Stochastic Volatility Models of the Short-Term Interest Rate,” Journal of
Econometrics, 77, 343-377.
Baxter, M.W. and A.J.O. Rennie (1996) Financial Calculus: An Introduction to Derivative Pricing. Cambridge University Press, Cambridge.
Black, F. and M. Scholes (1973). “The Pricing of Options and Corporate Liabilities,” Journal of Political Economy, 81, 637-654.
Chan, K.C., G.A. Karolyi, F.A. Longstaff and A.B. Sanders(1992). “An Empirical Comparison of Alternative Models of the Term
Structure of Interest Rates,” Journal of Finance, 47, 1209-1227.

20.5 References

783

Cox, J.C., J.E. Ingersoll and S.A. Ross (1985). “A Theory of the
Term Structure of Interest Rates,” Econometrica, 53(2), 385—407.
Duffie, D. (1996). Dynamic Asset Pricing, 2nd ed., Princeton University
Press, Princeton, NJ.
Gallant, A.R. (2003). Original FORTRAN routines stng1.f andweak2.f.
Downloaded
from
ftp.econ.duke.edu,
directory
pub/arg/libf, . Copyright (C) 1995. A. Ronald Gallant, P.O. Box 659,
Chapel Hill NC 27514-0659, USA. Permission to use, copy, modify, anddistribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear
in all copies and that both that copyright notice and this permission noticeappear in supporting documentation.
Gallant, A.R. and G. Tauchen (2002). “EMM: A Program for Eﬃcient
Method of Moments Estimation, Version 1.6, User’s Guide,” Working paper, University of North Carolina at Chapel Hill. Current revision availableat www.unc.edu/~arg.
Kloeden, P.E. and E. Platen (1999). Numerical Solution of
Stochastic Diﬀerential Equations, 3rd ed. Volume 23 of Applications ofMathematics, Stochastic Modelling and Applied Probability,
Springer-Verlag, New York.
Merton, R. (1990). Continuous Time Finance. Blackwell, Cambridge.
Neftci, S.N. (1996). An Introduction to the Mathematics of Financial
Derivatives. Academic Press, San Diego.
Øksendal, B.K. (1998). Stochastic Diﬀerential Equations. An Introduction with Applications, 5th ed. Universitext. Springer-Verlag, New York.
Seydel, R. (2002). Tools for Computational Finance. Springer-Verlag,
Berlin.
Vasicek, O.A. (1977). “An Equilibrium Characterization of the Term
Strucure,” Journal of Financial Economics, 5, 177—188.

This is page 784
Printer: Opaque thisThis is page 785
Printer: Opaque this21
Generalized Method of Moments

21.1 Introduction
This chapter describes generalized method of moments (GMM) estimation
for linear and nonlinear models with applications in economics and finance.
The GMM estimation was formalized by Hansen (1982), and since has
become one of the most widely used methods of estimation for models in
economics and finance. Unlike the maximum likelihood estimation (MLE),
GMM does not require complete knowledge of the distribution of the data.
Only specified moments derived from an underlying model are needed for
the GMM estimation. In some cases in which the distribution of the data is
known, MLE can be computationally very burdensome, whereas GMM canbe computationally very easy. The log-normal stochastic volatility model isone example. In models for which there are more moment conditions thanmodel parameters, the GMM estimation provides a straightforward way totest the specification of the proposed model. This is an important featurethat is unique to the GMM estimation.
This chapter is organized as follows. The GMM estimation for linear
models is described in Section 21.2. Section 21.3 describes methods for estimating the eﬃcient weight matrix. Sections 21.4 and 21.5 give examplesof estimation and inference using the S+Finmetrics function GMM. Section
21.6 describes the GMM estimation and inference for nonlinear models.
Section 21.7 provides numerous examples of the GMM estimation of nonlinear models in finance, including Euler equation asset pricing models,

786

21. Generalized Method of Moments

discrete-time stochastic volatility models, and continous-time interest rate
diﬀusion models.
The theory and notation for GMM presented herein follows the excellent treatment given in Hayashi (2000). Other good textbook treatments of
GMM at an intermediate level are given in Hamilton (1994), Ruud (2000),Davidson and MacKinnon (2004), and Greene (2004). The most comprehensive textbook treatment of GMM is Hall (2005). Excellent surveys ofrecent developments in GMM are given in the special issues of the Journal of Business and Economic Statistics (1996, 2002). Discussions of GMM
applied to problems in finance are given in Ogaki (1992), Ferson (1995), Andersen and Sorensen (1996), Campbell, Lo, and MacKinlay (1997), Jamesand Webber (2000), Cochrane (2001), Jagannathan and Skoulakis (2002),
and Hall (2005).

21.2 Single Equation Linear GMM
Consider the linear regression modelyt = z0t δ 0 + εt , t = 1, . . . , n(21.1)

where zt is an L × 1 vector of explanatory variables, δ 0 is a vector of
unknown coeﬃcients, and εt is a random error term. The model (21.1)
allows for the possibility that some or all of the elements of zt may becorrelated with the error term εt (i.e., E[ztk εt ] 6= 0 for some k). If E[ztk εi ] 6=0, then ztk is called an endogenous variable. It is well known that if zt
contains endogenous variables, then the least squares estimator of δ 0 in
(21.1) is biased and inconsistent.
Associated with the model (21.1), it is assumed that there exists a K × 1
vector of instrumental variables xt that may contain some or all of theelements of zt . Let wt represent the vector of unique and nonconstantelements of {yt , zt , xt }. It is assumed that {wt } is a stationary and ergodic
stochastic process.
The instrumental variables xt satisfy the set of K orthogonality conditions
E[gt (wt , δ 0 )] = E[xt εt ] = E[xt (yt − z0t δ 0 )] = 0(21.2)
where gt (wt , δ 0 ) = xt εt = xt (yt −z0t δ 0 ). Expanding (21.2) gives the relation
Σxy = Σxz δ 0
where Σxy = E[xt yt ] and Σxz = E[xt z0t ]. For identification of δ 0 , it isrequired that the K × L matrix E[xt z0t ] = Σxz be of full rank L. Thisrank condition ensures that δ 0 is the unique solution to (21.2). Note that
if K = L, then Σxz is invertible and δ 0 may be determined using
δ 0 = Σ−1xz Σxy21.2 Single Equation Linear GMM

787

A necessary condition for the identification of δ 0 is the order condition
K≥L(21.3)

which simply states that the number of instrumental variables must be
greater than or equal to the number of explanatory variables in (21.1). If
K = L, then δ 0 is said to be (apparently) just identified; if K > L then δ 0
is said to be (apparently) overidentified; if K < L then δ 0 is not identified.
The word “apparently” in parentheses is used to remind the reader thatthe rank condition
rank(Σxz ) = L(21.4)
must also be satisfied for identification.
In the regression model (21.1), the error terms are allowed to be conditionally heteroskedastic as well as serially correlated. For the case in which
εt is conditionally heteroskedastic, it is assumed that {gt } = {xt εt } is astationary and ergodic martingale diﬀerence sequence (MDS) satisfying
E[gt gt0 ] = E[xt x0t ε2t ] = S
where S is a nonsingular K × K matrix. The matrix S is P
the asymptoticnvariance-covariance matrix of the sample moments ḡ = n−1 t=1 gt (wt , δ 0 ).
This follows from the central limit theorem for ergodic stationary martingale diﬀerence sequences (see Hayashi, 2000, p. 106)
n
√
1 X
d
nḡ = √
xt εt → N (0, S)
n t=1

where S = avar(ḡ)√ denotes the variance-covariance matrix of the limiting
distribution of nḡ.
For the case in which εt is serially correlated and possibly conditionallyheteroskedastic as well, it is assumed that {gt } = {xt εt } is a stationary
and ergodic stochastic process that satisfies
√
nḡ

n

=

S =

1 X
d
√
xt εt → N (0, S)
n t=1
∞
X

j=−∞

Γj = Γ0 +∞X(Γj + Γ0j )j=1

0] = E[xt x0t−j εt εt−j ]. In the above, avar(ḡ) = S is also
where Γj = E[gt gt−jreferred to as the long-run variance of ḡ.

21.2.1 Definition of the GMM Estimator
The GMM estimator of δ 0 in (21.1) is constructed by exploiting the orthogonality conditions (21.2). The idea is to create a set of estimating equations

788

21. Generalized Method of Momentsfor δ 0 by making sample moments match the population moments definedby (21.2). The sample moments based on (21.2) for an arbitrary value δ
are
n
n
1X
1X
gn (δ) =g(wt , δ) =xt (yt − z0t δ)
n t=1
n t=1
⎞
⎛ 1 Pn0
t=1 x1t (yt − zt δ)
n
⎟
⎜
..
= ⎝
⎠
.
Pn
1
0
x
(y
−
z
δ)
t
t=1 Kt tnThese moment conditions are a set of K linear equations in L unknowns.
Equating these sample moments to the population moment E[xt εt ] = 0
gives the estimating equations−1(21.5)
Sxy − Sxz δ = 0Pn−10
t=1 xt yt and Sxz = nt=1 xt zt are the sample mo-Pnwhere Sxy = nments.
If K = L (δ 0 is just identified) and Sxz is invertible, then the GMM
estimator of δ 0 isδ̂ = S−1xz Sxywhich is also known as the indirect least squares estimator. If K > L, thenthere may not be a solution to the estimating equations (21.5). In thiscase, the idea is to try to find a δ that makes Sxy − Sxz δ as close to zero as
possible. To do this, let Ŵ denote a K × K symmetric and positive definite
p(p.d.) weight matrix, possibly dependent on the data, such that Ŵ → W
as n → ∞ with W symmetric and p.d. Then, the GMM estimator of δ 0 ,denoted δ̂(Ŵ), is defined asδ̂(Ŵ) = arg min J(δ, Ŵ)δ

whereJ(δ, Ŵ) = ngn (δ)0 Ŵgn (δ)= n(Sxy − Sxz δ)0 Ŵ(Sxy − Sxz δ)(21.6)

Since J(δ, Ŵ) is a simple quadratic form in δ, straightforward calculus
may be used to determine the analytic solution for δ̂(Ŵ):δ̂(Ŵ) = (S0xz ŴSxz )−1 S0xz ŴSxy(21.7)

Asymptotic Properties
Under standard regularity conditions (see Hayashi, 2000, Chap. 3), it canbe shown thatp

δ̂(Ŵ) → δ 0´√ ³dn δ̂(Ŵ) − δ 0 → N (0, avar(δ̂(Ŵ)))21.2 Single Equation Linear GMM

789

where
avar(δ̂(Ŵ)) = (Σ0xz WΣxz )−1 Σ0xz WSWΣxz (Σ0xz WΣxz )−1(21.8)A consistent estimate of avar(δ̂(Ŵ)), denoted avar(d δ̂(Ŵ)), may be computed usingavar(d δ̂(Ŵ)) = (S0xz ŴSxz )−1 S0xz ŴŜŴSxz (S0xz ŴSxz )−1(21.9)

where Ŝ is a consistent estimate for S = avar(ḡ).
The Eﬃcient GMM Estimator

For a given set of instruments xt , the GMM estimator δ̂(Ŵ) is definedfor an arbitrary positive definite and symmetric weight matrix Ŵ. The
asymptotic variance of δ̂(Ŵ) in (21.8) depends on the chosen weight matrix Ŵ. A natural question to ask is: What weight matrix W produces
the smallest value of avar(δ̂(Ŵ))? The GMM estimator constructed withthis weight matrix is called the eﬃcient GMM estimator . Hansen (1982)
showed that eﬃcient the GMM estimator results from setting Ŵ = Ŝ−1psuch that Ŝ → S. For this choice of Ŵ, the asymptotic variance formula(21.8) reduces toavar(δ̂(Ŝ−1 )) = (Σ0xz S−1 Σxz )−1(21.10)of which a consistent estimate isavar(d δ̂(Ŝ−1 )) = (S0xz Ŝ−1 Sxz )−1(21.11)The eﬃcient GMM estimator is defined as0δ̂(Ŝ−1 ) = arg min ngn (δ) Ŝ−1 gn (δ)δ

which requires a consistent estimate of S. However, a consistent estimationof S, in turn, requires a consistent estimate of δ 0 . To see this, consider
the case in which εt in (21.1) is conditionally heteroskedastic so that S =E[gt gt0 ] = E[xt x0t ε2t ]. A consistent estimate of S has the form
Ŝ =

n
n
1X
1X
xt x0t ε̂2t =xt x0t (yt − z0t δ̂)2
n t=1
n t=1

p

such that δ̂ → δ 0 . Similar arguments hold for the case in which gt = xt εt
is a serially correlated and heteroskedastic process.
Two-Step Eﬃcient GMM
The two-step eﬃcient GMM estimator utilizes the result that a consistent estimate of δ 0 may be computed by GMM with an arbitrary posip
tive definite and symmetric weight matrix Ŵ such that Ŵ → W. Let

790

21. Generalized Method of Momentsδ̂(Ŵ) denote such an estimate. Common choices for Ŵ are Ŵ = Ik and−1 0
Ŵ = S−1
X X)−1 , where X is an n × k matrix with the tth row
xx = (n01equal to xt . Then, a first step consistent estimate of S is given byn

1X
Ŝ(Ŵ) =xt x0t (yt − z0t δ̂(Ŵ))2
n t=1(21.12)The two-step eﬃcient GMM estimator is then defined asδ̂(Ŝ−1 (Ŵ)) = arg min ngn (δ)0 Ŝ−1 (Ŵ)gn (δ)δ(21.13)Iterated Eﬃcient GMM
The iterated eﬃcient GMM estimator uses the two-step eﬃcient GMM
estimator δ̂(Ŝ−1 (Ŵ)) to update the estimation of S in (21.12) and thenrecomputes the estimator in (21.13). The process is repeated (iterated) until
the estimates of δ 0 do not change significantly from one iteration to thenext. Typically, only a few iterations are required. The resulting estimator−1
is denoted δ̂(Ŝiter ). The iterated eﬃcient GMM estimator has the same
asymptotic distribution as the two-step eﬃcient estimator. However, infinite samples, the two estimators may diﬀer. As Hamilton (1994, p. 413)
pointed out, the iterated GMM estimator has a practical advantage over
the two-step estimator in that the resulting estimates are invariant withrespect to the scale of the data and to the initial weighting matrix Ŵ.
Continuous Updating Eﬃcient GMM
This estimator simultaneously estimates S, as a function of δ, and δ. It isdefined as
0 −1δ̂(Ŝ−1(δ)gn (δ)(21.14)CU ) = arg min ngn (δ) Ŝδwhere the expression for Ŝ(δ) depends on the estimator used for S. For
example, with conditionally heteroskedastic errors, Ŝ(δ) takes the formnŜ(δ)=

1X
xt x0t (yt − z0t δ)2
n t=1

Hansen, Heaton, and Yaron (1996) call δ̂(Ŝ−1
CU ) the continuous updating(CU) eﬃcient GMM estimator . This estimator is asymptotically equivalent
to the two-step and iterated estimators, but may diﬀer in finite samples.
1 In the function GMM, the default initial weight matrix is the identity matrix.
This can be changed by supplying a weight matrix using the optional argument
w=my.weight.matrix. Using Ŵ = S−1
xx is often more numerically stable than usingŴ = Ik .

21.2 Single Equation Linear GMM

791

The CU eﬃcient GMM estimator does not depend on an initial weight matrix W, and like the iterated eﬃcient GMM estimator, the numerical valueof CU estimator is invariant to the scale of the data. It is computationallymore burdensome than the iterated estimator, especially for large nonlinear models, and is more prone to numerical instability. However, Hansen,
Heaton, and Yaron found that the finite sample performance of the CU
estimator, and test statistics based on it, is often superior to the other estimators. The good finite sample performance of the CU estimator relative tothe iterated GMM estimator may be explained by the connection between
the CU estimator and empirical likelihood estimators. See Imbens (2002)
and Newey and Smith (2004) for a further discussion on the relationshipbetween GMM estimators and empirical likelihood estimators.

21.2.2 Specification Tests in Overidentified Models
An advantage of the GMM estimation in overidentified models is the ability
to test the specification of the model. The following subsections summarize
the common statistics used for evaluating the basic model specification.
The J-Statistic
The J-statistic, introduced in Hansen (1982), refers to the value of the
GMM objective function evaluated using an eﬃcient GMM estimator:−1

J = J(δ̂(Ŝ−1 ), Ŝ−1 ) = ngn (δ̂(Ŝ))0 Ŝ−1 gn (δ̂ Ŝ−1 )(21.15)where δ̂(Ŝ−1 ) denotes any eﬃcient GMM estimator of δ 0 and Ŝ is a consistent estimate of S. If K = L, then J = 0, and if K > L, then J > 0.
Under regularity conditions (see Hayashi, 2000, Chap. 3) and if the moment
conditions (21.2) are valid, then as n → ∞d

J → χ2 (K − L)Hence, in a well-specified overidentified model with valid moment conditions the J-statistic behaves like a chi-square random variable with degrees
of freedom equal to the number of overidentifying restrictions. If the modelis misspecified and/or some of the moment conditions (21.2) do not hold(e.g., E[xit εt ] = E[xit (yt − z0t δ 0 )] 6= 0 for some i), then the J-statistic will
be large relative to a chi-square random variable with K − L degrees of
freedom.
The J-statistic acts as an omnibus test statistic for model misspecification. A large J-statistic indicates a misspecified model. Unfortunately, theJ-statistic does not, by itself, give any information about how the modelis misspecified.

792

21. Generalized Method of Moments

Normalized Moments
If the model is rejected by the J-statistic, it is of interest to know why themodel is rejected. To aid in the diagnosis of model failure,√ the magnitudesof the individual elements of the normalized moments ngn (δ̂(Ŝ−1 )) may
point the reason why the model is rejected by the J-statistic. Under thenull hypothesis that the model is correct and the orthogonality conditions
are valid, the normalized moments satisfy√dngn (δ̂(Ŝ−1 )) → N (0, S − Σxz [Σ0xz S−1 Σxz ]−1 Σ0xz )As a result, for a well specified model, the individual moment t-ratios
ti = gn (δ̂(Ŝ−1 ))i /SE(gn (δ̂(Ŝ−1 ))i ), i = 1, . . . , K(21.16)

whereSE(gn (δ̂(Ŝ−1 ))i =³hi ´1/2Ŝ − Σ̂xz [Σ̂0xz Ŝ−1 Σ̂xz ]−1 Σ̂0xz /Tii

are asymptotically standard normal. When the model is rejected using the
J-statistic, a large value of ti indicates misspecification with respect tothe ith moment condition. Since the rank of S − Σxz [Σ0xz S−1 Σxz ]−1 Σ0xz is
K − L, the interpretation of the moment t-ratios (21.16) may be diﬃcult
in models for which the degree of overidentification is small. In particular,
if K − L = 1, then t1 = · · · = tK .

21.2.3 Two-Stage Least Squares as an Eﬃcient GMM
Estimator
If, in the linear GMM regression model (21.1), the errors are conditionallyhomoskedastic, thenE[xt x0t ε2t ] = σ 2 Σxx = S
pA consistent estimate of S has the form Ŝ = σ̂ 2 Sxx , where σ̂2 → σ 2 .
Typically,nXσ̂ 2 = n−1(yt − z0t δ̂)2
t=1

where δ̂ → δ 0 . The eﬃcient GMM estimator becomes−2 −1
0
Sxx Sxz )−1 S0xz σ̂ −2 S−1δ̂(σ̂ −2 S−1
xx ) = (Sxz σ̂
xx Sxy0−1= (Sxz Sxx Sxz )−1 S0xz S−1S
xx xy= δ̂(S−1
xx )21.3 Estimation of S

793

which does not depend on σ̂ 2 . The estimator δ̂(S−1xx ) is, in fact, identicalto the two-stage least squares (TSLS) estimator of δ 0 :
0
−1
−1 0
Sxz S−1δ̂(S−1xx ) = (Sxz Sxx Sxz )xx Sxy0−1 0= (Z PX Z) Z PX y= δ̂ TSLS

where Z denotes the n × L matrix of observations with tth row z0t , X
denotes the n × k matrix of observations with the tth row x0t , and PX =X(X0 X)−1 X0 is the idempotent matrix that projects onto the columns of
X.
Using (21.10), the asymptotic variance of δ̂(S−1xx ) = δ̂ TSLS is−1
avar(δ̂ TSLS ) = (Σ0xz S−1 Σxz )−1 = σ 2 (Σ0xz Σ−1
xx Σxz )2
Although δ̂(S−1xx ) does not depend on σ̂ , a consistent estimate of theasymptotic variance does:−1avar(d δ̂ TSLS ) = σ̂ 2 (S0xz S−1
xx Sxz )Similarly, the J-statistic also depends on σ̂ 2 and takes the form
J(δ̂ TSLS , σ̂ −2 S−1
xx ) = n(sxy − Sxz δ̂ TSLS )0 S−1
xx (sxy − Sxz δ̂ TSLS )σ̂ 2

The TSLS J-statistics is also known as Sargan’s statistic (see Sargan 1958).

21.3 Estimation of S
To compute any of the eﬃcient GMM estimators, a consistent estimate of
S = avar(ḡ) is required. The method used to estimate S depends on thetime series properties of the population moment conditions gt . Two cases
are generally considered. In the first case, gt is assumed to be serially uncorrelated but may be conditionally heteroskedastic. In the second case,gt is assumed to be serially correlated as well as potentially conditionallyheteroskedastic. The following subsections discuss the estimation of S inthese two cases. Similar estimators were discussed in the context of linear
regression in Chapter 6, Section 5. In what follows, the assumption of a linear model (21.1) is dropped and the K moment conditions embodied in thevector gt are assumed to be nonlinear functions of q ≤ K model parametersθ and are denoted gt (θ). P
The moment conditions satisfy E[gt (θ0 )] = 0 andS = avar(ḡ) = avar(n−1 ni=1 gt (θ0 )).

794

21. Generalized Method of Moments

Default bnh ¡ ¢ in 1/5
int 4 100h ¡ ¢ in 1/4
int 4 100

KernelTruncated
Bartletth ¡ ¢in 4/25
int 4 100

Parzenh ¡ ¢i
n 4/25
int 4 100

Quadratic
spectralwj,n
1 for aj < 1
0 for aj ≥ 1½
1 − aj for aj ≤ 1
0
for aj > 1⎧⎨ 1 − 6a2j + 6a3j for 0 ≤ aj ≤ 0.5
2(1 − aj )3
for 0.5 ≤ aj ≤ 1⎩
0
for aj > 1
i
h
sin(m
)
j
25
−
cos(m
)
2
j
2
m
12π d
j
½

i

note: aj = i/(bn + 1), dj = j/bn , mj = 6πdi /5

TABLE 21.1. Common kernel weights and default bandwidths

21.3.1 Serially Uncorrelated Moments
In many situations, the population moment conditions gt (θ 0 ) form an
ergodic-stationary MDS with an appropriate information set It . In thiscase,
S = avar(ḡ) = E[gt (θ 0 )gt (θ 0 )0 ]Following White (1980), a heteroskedasticity consistent (HC) estimate of S
has the formn
1X
ŜHC =
gt (θ̂)gt (θ̂)0(21.17)n t=1

where θ̂ is a consistent estimate of θ 0 .2 Davidson and MacKinnon (1993,
Sec. 16.3) suggested using a simple degrees-of-freedom corrected estimateof S that replaces n−1 in (21.17) with (n−k)−1 to improve the finite sample
performance of tests based on (21.11).

21.3.2 Serially Correlated Moments
If the population moment conditions gt (θ 0 ) are an ergodic-stationary but
serially correlated process then∞X
S = avar(ḡ) = Γ0 +(Γj + Γ0j )j=1

where Γj = E[gt (θ0 )gt−j (θ 0 )0 ]. In this case, a heteroskedasticity and autocorrelation consistent (HAC) estimate of S has the form
ŜHAC = Γ̂0 (θ̂) +n−1
Xwj,n (Γ̂j (θ̂) + Γ̂0j (θ̂))

j=1

2 For example, θ̂weight matrix.

may be an ineﬃcient GMM estimate based on an arbitrary p.d.

21.3 Estimation of S

795

where wj,n (j = 1, . . . , bn ) are kernel function weights, bn is a non-negative
bandwidth parameter that may depend on the sample size, Γ̂j (θ̂) =Pn
10
t=j+1 gt (θ̂)gt−j (θ̂) , and θ̂ is a consistent estimate of θ 0 . Diﬀerentn
HAC estimates of S are distinguished by their kernel weights and bandwidth parameter. The most common kernel functions are listed in Table
21.1. For all kernels except the quadratic spectral, the integer bandwidthparameter, bn , acts as a lag truncation parameter and determines how
many autocovariance matrices to include when forming ŜHAC . Figure 21.1
illustrates the first 10 kernel weights for the kernels listed in Table 21.1
evaluated using the default values of bn for n = 100. The choices of kerneland bandwidth determine the statistical properties of ŜHAC . The truncatedkernel is often used if the moment conditions follow a finite-order moving
average process. However, the resulting estimate of S is not guaranteed
to be positive definite. Use of the Bartlett, Parzen, or quadratic spectral
kernel ensures that ŜHAC will be positive semidefinite. For these kernels,
Andrews (1991) studied the asymptotic properties ŜHAC . He showed thatŜHAC is consistent for S provided that bn → ∞ as n → ∞. Furthermore, foreach kernel, Andrews determined the rate at which bn → ∞ to asymptotically minimize the mean sequared error, M SE(ŜHAC , S). For the Bartlett,Parzen, and quadratic spectral kernels, the rates are n1/3 , n1/5 , and n1/5 ,respectively. Using the optimal bandwidths, Andrews found that the ŜHAC
based on the quadratic spectral kernel has the smallest asymptotic MSE,
followed closely by ŜHAC based on the Parzen kernel.

Automatic Bandwidth Selection
Based on extensive Monte Carlo experiments, Newey and West (1994) concluded that the choice of bandwidth parameter was more important thanthe choice of kernel for the finite sample performance of ŜHAC . Increasing bn reduces the bias of ŜHAC but increases the variance. The default
values for bn in Table 21.1 are ad hoc, being motivated by the convergence rates of ŜHAC for the respective kernels. To overcome the ad hoc
choice of bn , Andrews (1991) and Newey and West (1994) propose data
dependent automatic bandwidth selection procedures that asymptoticallyminimize M SE(ŜHAC , S). The details of these procedures are tedious andso are not repeated here. The interested reader is referred to den Haan andLevin (1997), who nicely summarize the procedures and give guidance tothe practitioner. However, as den Haan and Levin point out, the so-called“automatic” bandwidth selection procedures still depend on a K × 1 userspecified vector of weights w for the elements of gt (θ̂) and a method ofproviding initial estimates of S.

21. Generalized Method of Moments

1.0

796

weights

0.0

0.2

0.4

0.6

0.8

Truncated
Bartlett
ParzenQuadratic Spectral

1

2

3

4

5

6

7

8

9

10

lags

FIGURE 21.1. Kernel weights evaluated at default bandwidths for n = 100.

Prewhitening and Recoloring
If gt (θ̂) is highly autocorrelated, Andrews and Monahan (1992) found thatthe finite sample behavior of ŜHAC is often improved if a prewhitening andrecoloring procedure is used. This procedure works as follows:[1] Prewhiten gt (θ̂) by estimating a single lag vector autoregression,
VAR(1), for gt (θ̂)
gt (θ̂) = Φgt−1 (θ̂) + et (θ̂)
and forming the residuals et (θ̂) = gt (θ̂) − Φ̂gt−1 (θ̂).
[2] Construct an HAC estimator for et (θ̂):
bΣ̂HAC =
where Γ̂j = n1

Pn

n
1X
wj,n (Γ̂j + Γ̂0j )n j=1

t=j+1 et (θ̂)et−j (θ̂).

[3] Form the recolored HAC estimate for S using
ŜPWHAC = (IK − Φ̂)−1 Σ̂HAC (IK − Φ̂)−10

21.4 GMM Estimation Using the S+FinMetrics Function GMM

797

21.3.3 Estimating S Using the S+FinMetrics Function
var.hac
The HAC estimates of S for an n × k time series xt , based on the procedures described in the previous section, may be computed using the
S+FinMetrics function var.hac.3 The arguments expected by var.hac
are> args(var.hac)function(x, bandwidth = NULL, window = "parzen", na.rm = F,automatic = "none", df.correction = 0, prewhiten = F,
w = NULL, demean = T)
The optional arguments bandwidth and window are used to specify the
bandwidth parameter bn and kernel weight function wj,n , respectively.
Valid kernels are those listed in Table 21.1: "truncated", "bartlett","parzen", and "qs". If the bandwidth is not specified, then the default
value for bn from Table 21.1 is used for the specified kernel. The argument df.correction specifies a non-negative integer to be subtracted fromthe sample size to perform a degrees-of-freedom correction. The argument
automatic determines if the Andrews (1991) or Newey-West (1994) automatic bandwidth selection procedure is to be used to set bn . If automatic is
set to "andrews" or "nw", then the argument w must be supplied as a vectorof weights for each variable in x. The Andrews-Monahan (1992) VAR(1)prewhitening and recoloring procedure is performed if prewhiten=T.

21.4 GMM Estimation Using the S+FinMetrics
Function GMM
The GMM estimation of general linear and nonlinear models may be performed using the S+FinMetrics function GMM. The arguments expected by
GMM are> args(GMM)
function(start, moments, control = NULL, scale = NULL, lower= -Inf, upper = Inf, trace = T, method = "iterative",
w = NULL, max.steps = 100, w.tol = 0.0001, ts = F,df.correction = T, var.hac.control = var.hac.control(),
w0.efficient = F, ...)The required arguments are start, which is a vector of starting values forthe parameters of the model, and moments, which is an S-PLUS function
3 The function var.hac is an enhanced version of the S+FinMertics function
asymp.var.

798

21. Generalized Method of Moments

to compute the sample moment conditions used for estimating the model.
The moments function must be of the form f(parm,...), where parm is avector of L parameters, and return a matrix of dimension n × K giving the
GMM moment conditions x0t εt for t = 1, . . . , n. The optional arguments
control, scale, lower, and upper are used by the S-PLUS optimization
function nlregb. See the online help for nlregb and nlregb.control for
details. Setting trace=T displays iterative information from the optimization. The argument method determines the type of GMM estimation tobe performed. Valid choices are "iterative", for iterated GMM estimation, and "simultaneous", for continuous updating GMM estimation. If
method="iterative" then the argument max.steps determines the number of iterations to be performed. The argument w specifies the weight matrix used for constructing the GMM objective function (21.6).4 If w=NULL,
then an estimate of the eﬃcient weight matrix based on the asymptotic
variance of the sample moment conditions will be used. In this case, the argument ts determines if an HC or HAC covariance estimator is computed
and the arguments df.correction and var.hac.control control various
options associated with these estimators. The user may supply a positive
definite and symmetric weight matrix to be used as the initial weight matrix if method="iterative". This weight matrix may be fixed throughout the estimation by setting max.step=0. If method="interative", andmax.step=0 then the argument w0.efficient indicates whether the usersupplied weight matrix is an eﬃcient weight matrix. This is useful for computing certain types of test statistics based on the GMM objective function.
The argument ... specifies any optional arguments that will be passed tothe moments function used for computing the GMM moment conditions.
Typically, these arguments specify the data used to compute the moment
conditions.
The GMM function produces an object of class "GMM" for which there areprint and summary methods, and extractor function coef.
Example 141 Estimating the classical linear regression model by GMM
Consider the classical linear regression modelyt = x0t β 0 + εt(21.18)where the explanatory variables xt are assumed to be orthogonal to the error term. However, εt is allowed to be conditionally heteroskedastic and/or
serially correlated. In this model, the explanatory variables are also the
instrumental variables so that zt = xt and K = L. The population orthogonality condition isE[gt (wt , β0 )] = E[xt εt ] = E[xt (yt − x0t β0 )] = 0
4 The current version of GMM uses the inverse of w as the initial weight matrix.

21.4 GMM Estimation Using the S+FinMetrics Function GMM

799

where gt (wt , β) = xt εt and wt = (yt , xt )0 . The sample moment condition
used for estimation isn1X
gn (wt , β) =xt (yt − x0t β)n t=1
which gives rise to the GMM estimating equation
Sxy − Sxx β = 0
Since K = L, the model is just identified, and, provided that Sxx is invertible, the GMM estimator is equivalent to the least squares estimatorβ̂ = S−1
xx SxyThe estimate β̂ is asymptotically normally distributed with asymptotic
varianceavar(β̂) = (Σxx S−1 Σxx )−1
where S = avar(ḡ). If εt is iid (0, σ 2 ), say, then S = E[xt x0t ε2t ] = σ 2 Σxx and2avar(β̂) = σ Σ−1
xx , which is the usual formula for avar(β̂) in the classicallinear regression formula.
As an example of a simple linear regression model, consider the Capital
Asset Pricing Model (CAPM)
Rt − rf t = α + β(RMt − rf t ) + εt , t = 1, . . . , n(21.19)where Rt denotes the return on an asset, rf t denotes the risk-free rate, andRMt denotes the return on a market portfolio proxy. Using the notation
for the linear model (21.18), yt = Rt − rf t and xt = (1, RMt − rf t )0 . The
data for this example are the monthly excess returns on Microsoft stockand the S&P 500 index over the period February 1990 through December
2000 in the S+FinMetrics "timeSeries" object excessReturns.ts. Assuming that the error term is orthogonal to RMt − rf t , the CAPM may beconsistently estimated using ordinary least squares (OLS):> ols.fit = OLS(MSFT~SP500, data = excessReturns.ts)> ols.fit
Call:
OLS(formula = MSFT ~SP500, data = excessReturns.ts)
Coefficients:(Intercept) SP500
0.01751.5677
Degrees of freedom: 131 total; 129 residual
Time period: from Feb 1990 to Dec 2000
Residual standard error: 0.09094843

800

21. Generalized Method of Moments

An S-PLUS function to compute the moment conditions for the linear
regression model is
ols.moments = function(parm,y=NULL,x=NULL) {x = as.matrix(x)x * as.vector(y - x %*% parm)}
where parm is an L × 1 vector of parameter β, y is an n × 1 vector ofobservations on the dependent variable, and x is an n × L matrix of observations on the explanatory variables. The function returns an n × L matrix
of moment conditions x0t εt = x0t (yt − x0t β) for t = 1, . . . , n:
> excessReturns.df = excessReturns.ts@data> ols.moments(c(1,1),y = excessReturns.df[,"MSFT"],+x = cbind(1, excessReturns.df[,"SP500"]))numeric matrix: 131 rows, 2 columns.
[,1]
[,2]
[1,] -0.9409745 -0.0026713864[2,] -0.9027093 -0.0161178959...
[131,] -1.2480621 0.0009318206
To estimate the CAPM regression (21.19) with GMM assuming heteroskedastic errors, use> start.vals = c(0,1)> names(start.vals) = c("alpha", "beta")> gmm.fit = GMM(start.vals, ols.moments, max.steps = 1,+
y = excessReturns.df[,"MSFT"],+x = cbind(1, excessReturns.df[,"SP500"]))> class(gmm.fit)[1] "GMM"
Notice how the data are passed to the function ols.moments through the... argument. The object gmm.fit returned by GMM is of class "GMM" andhas components> names(gmm.fit)[1] "parameters""objective"[3] "message""grad.norm"[5] "iterations""r.evals"[7] "j.evals""scale"[9] "normalized.moments" "vcov"[11] "method""df.J"[13] "df.residual""call"

21.4 GMM Estimation Using the S+FinMetrics Function GMM

801

The online help for GMM gives a complete description of these components.5
Typing the name of the "GMM" object invokes the print method on thefitted object:> gmm.fit
Call:
GMM(start = start.vals, moments = ols.moments, max.steps = 1,
y = excessReturns.df[, "MSFT"], x = cbind(1,excessReturns.df[, "SP500"]))Coefficients:
alpha
beta
0.0175 1.5677
Test of Overidentification:model is just-identified
Optimization Info:
Number of Iterations: 1
Convergence: x convergence
As expected, the GMM estimates of α and β are equivalent to the leastsquares estimates. Also, since the linear model is just identified, the GMM
objective function (J-statistic) is identically zero at the optimum and,
therefore, there is no test for overidentifying restrictions.
The summary method provides information about the statistical significance of the estimated parameters:> summary(gmm.fit)
Call:
GMM(start = start.vals, moments = ols.moments, max.steps = 1,
y = excessReturns.df[, "MSFT"], x = cbind(1,excessReturns.df[, "SP500"]))Coefficients:
Value Std.Error t value Pr(>|t|)alpha 0.0175 0.0079
2.2175 0.0283
beta 1.5677 0.1905
8.2274 0.0000

5 Since the linear model is just identified, the weight matrix in the GMM objectivefunction is irrelevant and so the weight.matrix component of gmm.fit is not returned.
Also, the moments.vcov component is not returned since all of the normalized moments
are equal to zero.

802

21. Generalized Method of Moments

Test of Overidentification:model is just-identified
Optimization Info:
Number of Iterations: 1
Convergence: x convergence
By default, the GMM function computes an HC estimate of the asymptotic
variance of the sample moment conditions, and so values in the column
labeled Std.Error are heteroskedasticity consistent (HC) standard errors.
To be sure, these standard errors may be compared to those computed fromthe OLS fit with the White HC correction:> summary(ols.fit,correction="white")
Call:
OLS(formula = MSFT ~SP500, data = excessReturns.ts)
Residuals:
Min
1Q Median-0.3101 -0.0620 -0.0024

3Q
0.0581Max
0.2260Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0175 0.0079
2.2175 0.0283
SP500 1.5677 0.1905
8.2274 0.0000
To compute the GMM estimator using an HAC estimate of the asymptotic variance of the sample moment conditions, call the GMM function withthe optional argument ts=T. The type of HAC estimate used is determinedby the options set in var.hac.control. For example, to compute the OLS
estimates with the usual Newey-West HAC standard errors use> gmm.fit2 = GMM(c(0,1), ols.moments, max.steps = 1,
+ y = excessReturns.df[,"MSFT"],+ x = cbind(1, excessReturns.df[,"SP500"]), ts = T,
+ var.hac.control = var.hac.control(window = "bartlett",+ bandwidth = floor(4 * (nrow(excessReturns.df)/100)^(2/9))))The standard errors for the above GMM estimates are identical to those
returned by> summary(ols.fit,correction="nw")
Example 142 Estimating the instrumental variables regression modelusing GMM

21.4 GMM Estimation Using the S+FinMetrics Function GMM

803

As in Campbell and Mankiw (1990), consider the stylized consumptionfunction∆ct= δ 0 + δ 1 ∆yt + δ 2 rt + εt , t = 1, . . . , T= δ 0 zt + εt(21.20)where ct denotes the log of real per capita consumption (excluding durables),yt denotes the log of real disposable income, and rt denotes the ex post realinterest rate (T-bill rate — inflation rate). Assume that {∆ct , ∆yt , rt } are
stationary and ergodic and that {εt , It } is a stationary and ergodic martingale diﬀerence sequence (MDS), where It = {∆cs , ∆ys , rs }ts=1 denotes theobserved information set at time t. In (21.20), the variables ∆yt and rt arelikely to be contemporaneously correlated with εt and so the least squares
estimates of δ are likely to be biased and inconsistent. Because {εt , It }is a stationary and ergodic MDS, E[εt |It−1 ] = 0, which implies that any
variable in It−1 is a potential instrument. Furthermore, for any variable
xt−1 ⊂ It−1 , {xt−1 εt } is an uncorrelated sequence.
The data for this example are annual data over the period 1960 to1995 taken from Wooldridge (2002), and are in the "timeSeries" object
consump.ts:> colIds(consump.ts)[1] "GC" "GY" "R3"> consump.ts@documentation[1] "GY = log growth rate of real income"[2] "GC = log growth rate of real consumption"[3] "R3 = real 3-month T-bill rate"[4] "source: Wooldridge (2002), Introduction to"[5] "Econometrics, 2nd Edition"[6] "South-Western Thompson"
The following data frame consump is created for use with the function GMM:
> nobs = numRows(consump.ts)> consump = seriesData(consump.ts)> consump$const = rep(1,nobs)
An S-PLUS function to compute the linear instrumental variables regression model moment conditions g(wt , δ) = xt (yt − z0t δ) for t = 1, . . . , n,isiv.moments = function(parm, y, X, Z) {# parm = L x 1 vector of parameters# y = n x 1 response vector# X = n x K matrix of instruments# Z = n x L matrix of explanatory variables
X = as.matrix(X)Z = as.matrix(Z)804

21. Generalized Method of Moments

X * as.vector(y - Z %*% parm)}
Applying this function to the consumption data with δ = (1, 1, 1)0 , zt =(∆yt , rt , 1)0 , and xt = (∆ct−1 , ∆yt−1 , rt−1 , 1)0 gives> iv.moments(c(1,1,1),y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
GC
GY
R3
const
1 -0.014143206 -0.0070677434 -0.0124922035 -1.0156263
2 -0.013609181 -0.0154800892 -0.0141370678 -1.0244252...
35 -0.015828003 -0.0143904735 -0.0175688817 -1.0395788
To estimate the consumption function (21.20) by two-step eﬃcient GMM
using Ŵ = IK as the initial weight matrix and assuming conditionallyheteroskedastic errors, call GMM with the optional arguments method ="iterative" and max.steps = 1:
> start.vals = rep(0.1,3)> names(start.vals) = c("GY","R3","const")> gmm.fit.2step = GMM(start.vals, iv.moments,+
method = "iterative", max.steps=1,+
y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
1-step objective = 1.02951e-8
2-step objective = 1.57854
A summary of the model fit is> summary(gmm.fit.2step)Call:
GMM(start = start.vals, moments = iv.moments, method ="iterative", max.steps = 1, y = consump[2:nobs, 1],
X = consump[1:(nobs - 1), ], Z = consump[2:nobs,2:4])
Coefficients:
Value Std.Error t value Pr(>|t|)
GY 0.6277 0.1500
4.1852 0.0002
R3 -0.0099 0.0981-0.1009 0.9202
const 0.0071 0.0037
1.9134 0.0647
Test of Overidentification:
J-stat Df P.value
1.5785 1 0.209

21.4 GMM Estimation Using the S+FinMetrics Function GMM

805

Optimization Info:
Number of Iterative Steps: 2
The coeﬃcient on ∆yt is 0.6277, with an estimated standard error of 0.1761,
and the coeﬃcient on rt is slightly negative, with an estimated standard
error of 0.1156. The J-statistic is 1.5785 and has a p-value of 0.209 basedon the chi-square distribution with one degree of freedom. The data appear
to support the single overidentifying restriction.
To estimate the consumption function (21.20) by iterated eﬃcient GMM
assuming conditionally heteroskedastic errors, call GMM with method ="iterative" and max.steps set to a large number6 :> gmm.fit.iter = GMM(start.vals, iv.moments,+
method = "iterative", max.steps = 100,+y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
1-step objective = 1.02951e-8
2-step objective = 1.57854...
13-step objective = 1.85567
To compute the continuously updated eﬃcient GMM estimator (21.14),
call GMM with method = "simultaneous":> start.vals = gmm.fit.iter$parameters> gmm.fit.cu = GMM(start.vals, iv.moments,+
method = "simultaneous",+y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
Good starting values are important for the CU estimator, and the above
estimation uses the iterated GMM estimates as starting values.
Finally, to compute an ineﬃcient one-step GMM estimator with W = I4 ,use
> start.vals = rep(0.1,3)> names(start.vals) = c("GY","R3","const")> gmm.fit.1step = GMM(start.vals, iv.moments,+
method = "iterative", max.steps = 0,+
w = diag(4), w0.efficient = F,
6 Notice that the one-step objective almost equal to zero. This is caused by usingŴ = Ik as the initial weight matrix since the scaling of the individual moment conditions
is very diﬀerent. Using Ŵ = S−1
xx generally provides a better scaling of the moment
conditions.

806

21. Generalized Method of Moments∆ct = δ 1 + δ 2 ∆yt + δ 3 rt + εtxt = (1, ∆ct−1 , ∆yt−1 , rt−1 )0 , E[xt εt ] = 0, E[xt x0t ε2t ] = S
Estimator
δ1
δ2
δ3
J-stat
.007
.627
−.010
1.578
2-step eﬃcient(.004)(.150)(.098)(.209)
.008
.591
−.032
1.855
Iterated eﬃcient(.004)(.144)(.095)(.173)
.008
.574
−.054
1.747
CU eﬃcient(.003)(.139).095(.186)
.003
.801
−.024
−
1-step ineﬃcient(.005)(.223)(.116)−TABLE 21.2. Eﬃcient GMM estimates of the consumption function parameters+
y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
1-step objective = 1.02951e-8
1-step objective = 1.02951e-8
Warning messages:
1: Maximum iterative steps exceeded. in: GMM(start.vals,
iv.moments, method = "iterative", max.steps = ....
2: The J-Statistic is not valid since the weight matrix is
not efficient. in: GMM(start.vals, iv.moments, method= "iterative", max.steps = ....
Table 21.2 summarizes the diﬀerent eﬃcient GMM estimators for theparameters in (21.20). The results are very similar across the eﬃcient estimations.7
Example 143 Estimating the instrumental variables regression modelusing TSLSThe TSLS estimator of δ may be computed using the function GMM by
8supplying the fixed weight matrix W = S−1
xx as follows :> w.tsls = crossprod(consump[1:(nobs-1),])/nobs> start.vals = rep(0.1,3)> names(start.vals) = c("GY","R3","const")> gmm.fit.tsls = GMM(start.vals,iv.moments,+
method = "iterative",max.steps = 0,+w = w.tsls,w0.efficient = T,
7 To match the default Eviews output for GMM, set the optional argument
df.correction=F.
8 Recall the GMM function uses the inverse of w as the weight matrix.

21.4 GMM Estimation Using the S+FinMetrics Function GMM807+
y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
1-step objective = 1.12666e-4> gmm.fit.tsls
Call:
GMM(start = start.vals, moments = iv.moments, method ="iterative", w = w.tsls, max.steps = 0, w0.efficient= T, y = consump[2:nobs, 1], X = consump[1:(nobs -1), ],Z = consump[2:nobs, 2:4])
Coefficients:
GY
R3
0.5862 -0.0269

const
0.0081

Test of Overidentification:
J-stat Df P.value
0.0001 1 0.9915
Optimization Info:
Number of Iterative Steps: 1
The TSLS estimate of δ is similar to the eﬃcient iterated estimate. The
J-statistic and the estimate of avar(δ̂ TSLS ) computed using Ŵ = Ŝ−1
xx ,however, are not correct since S−1
isproportionaltotheeﬃcient
weightxx
matrix. To get the correct values for these quantities, a consistent estimateσ̂ 2 of σ2 is required to form the eﬃcient weight matrix σ̂ 2 S−1
xx . This is easily
accomplished using# compute TSLS estimate of error variance> y = as.vector(consump[2:nobs,1])> X = as.matrix(consump[1:(nobs-1),])> Z = as.matrix(consump[2:nobs,2:4])> d.hat = coef(gmm.fit.tsls)> e.hat = y - Z%*%d.hat> df = nrow(Z) - ncol(Z)> s2 = as.numeric(crossprod(e.hat)/df)# compute correct efficient weight matrix for tsls
# that contains error variance term
> w.tsls2 = crossprod(X)*s2/nobs> start.vals = rep(0.1,3)> names(start.vals) = c("GY","R3","const")> gmm.fit.tsls2 = GMM(start.vals,iv.moments,+
method = "iterative",max.steps = 0,

808

21. Generalized Method of Moments+w = w.tsls2,w0.efficient = T,+y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
1-step objective = 2.01841> summary(gmm.fit.tsls2)
Call:
GMM(start = start.vals, moments = iv.moments, method ="iterative", w = w.tsls2, max.steps = 0, w0.efficient= T, y = consump[2:nobs, 1], X = consump[1:(nobs -1), ],Z = consump[2:nobs, 2:4])
Coefficients:
Value Std.Error t value Pr(>|t|)
GY 0.5862 0.1327
4.4177 0.0001
R3 -0.0269 0.0753-0.3576 0.7230
const 0.0081 0.0032
2.5285 0.0166
Test of Overidentification:
J-stat Df P.value
2.0184 1 0.1554

21.5 Hypothesis Testing for Linear Models
The following subsections discuss hypothesis testing in linear models estimated by GMM. The main types of hypothesis test are for coeﬃcient
restrictions, overidentification restrictions, subsets of orthogonality restrictions, and instrument relevance. Except for the tests for instrument relevance, the tests extend in a straightforward way to nonlinear models estimated by GMM.

21.5.1 Testing Restrictions on Coeﬃcients
Hypothesis testing on coeﬃcients in linear GMM models is surveyed in
Newey and West (1987) and nicely summarized in Chapter 3 of Hayashi(2000).
Wald Statistics
Wald-type statistics are based on the asymptotic normality of the GMM estimator δ̂(Ŵ) for an arbitrary weight matrix Ŵ. Simple tests on individual

21.5 Hypothesis Testing for Linear Models

809

coeﬃcients of the form
H0 : δ k = δ 0k

(21.21)

may be conducted using the asymptotic t-ratiotk =δ̂ k (Ŵ) − δ 0kc δ̂ k (Ŵ))
SE((21.22)c δ̂ k (Ŵ)) is the square root of the kth diagonal element of (21.9).
where SE(
Under the null hypothesis (21.21), the t-ratio (21.22) has an asymptotic
standard normal distribution.
Linear hypotheses of the form
H0 : Rδ = r(21.23)where R is a fixed q × L matrix of rank q and r is a fixed q × 1 vector, may
be tested using the Wald statistichi−1(Rδ̂(Ŵ) − r)
(21.24)d δ̂(Ŵ))R0
Wald = n(Rδ̂(Ŵ) − r)0 Ravar(

where avar(d δ̂(Ŵ)) is given by (21.9). Under the null (21.23), the Waldstatistic (21.24) has a limiting chi-square distribution with q degrees of
freedom. The Wald statistic (21.24) is valid for any consistent and asymptotically normal GMM estimator δ̂(Ŵ) based on an arbitrary symmetricp
and positive definite weight matrix Ŵ → W. Usually, Wald statistics arecomputed using Ŵ = Ŝ−1 so that avar(d δ̂(Ŝ−1 )) is given by (21.11).
Nonlinear hypotheses of the form
H0 : a(δ 0 ) = 0(21.25)where a(δ 0 ) = 0 imposes q nonlinear restrictions and ∂a(δ 0 )/∂δ 0 has fullrank q, may be tested using the Wald statistic"#−1∂a(δ̂(Ŵ))00 ∂a(δ̂(Ŵ))
avar(d δ̂(Ŵ))a(δ̂(Ŵ))
Wald = na(δ̂(Ŵ))∂δ∂δ 0(21.26)
Under the null (21.25), the Wald statistic (21.26) has a limiting chi-square
distribution with q degrees of freedom.
GMM LR-Type Statistics
Linear and nonlinear restrictions on the model coeﬃcients δ may also be
tested using a likelihood ratio (LR)-type statistic. In eﬃcient GMM estimation, the unrestricted objective function is J(δ̂(Ŝ−1 ), Ŝ−1 ), for a consistentestimate Ŝ of S. The restricted eﬃcient GMM estimator solves
δ̃ R (Ŝ−1 ) = arg min J(δ, Ŝ−1 ) subject to H0
δ(21.27)

810

21. Generalized Method of Moments

The GMM LR-type statistic is the diﬀerence between the restricted andunrestricted J-statistics:
LRGMM = J(δ̃ R (Ŝ−1 ), Ŝ−1 ) − J(δ̂(Ŝ−1 ), Ŝ−1 )(21.28)Under the null hypotheses (21.23) or (21.25), LRGMM has a limiting chisquare distribution with q degrees of freedom. As n → ∞, it can be shownpthat Wald − LRGMM → 0, although the two statistics may diﬀer in finite
samples. For linear restrictions, Wald and LRGMM are numerically equivalent provided that the same value of Ŝ is used to compute the restricted
and unrestricted eﬃcient GMM estimators. Typically Ŝ computed under
the unrestricted model is used in constructing LRGMM . In this case, when
the restricted eﬃcient GMM estimator is computed by solving (21.27) the
weight matrix Ŝ−1
UR is held fixed during the estimation (no iteration is performed on the weight matrix). If LRGMM is computed using two diﬀerent
consistent estimates of S, say Ŝ and S̃, then it is not guaranteed to be positive in finite samples but is asymptotically valid. The LRGMM statistic has
the advantage over the Wald statistic for nonlinear hypotheses in that it
is invariant to how the nonlinear restrictions are represented. Additionally,
Monte Carlo studies have shown that LRGMM often performs better than
Wald in finite samples. In particular, Wald tends to over reject the null
hypothesis when it is true.
Example 144 Testing the PIH
The pure permanent income hypothesis (PIH) due to Hall (1978) statesthat ct in (21.20) is a martingale so that ∆ct = εt is a MDS. Hence, thePIH implies the linear restrictions
H0 : δ 1 = δ 2 = 0
which are of the form (21.23) with
µ
¶
µ ¶
1 0 0
0
R=
, r=
0 1 0
0
If there are temporary income consumers, then δ 1 > 0.
The Wald statistic (21.24) based on the iterated eﬃcient GMM estimator
may be computed using> Rmat = matrix(c(1,0,0,1,0,0),2,3)> rvec = c(0,0)> dhat = coef(gmm.fit.iter)> avarRbhat = Rmat %*% gmm.fit.iter$vcov %*% t(Rmat)> Rmr = Rmat %*% dhat - rvec
> wald.stat = as.numeric(t(Rmr) %*% solve(avarRbhat) %*% Rmr)
> wald.stat[1] 16.99482

21.5 Hypothesis Testing for Linear Models

811

Since there are q = 2 linear restrictions, the Wald statistic has an asymptotic chi-square distribution with two degrees of freedom. The p-value is> 1 - pchisq(wald.stat,2)[1] 0.0002039964
which suggests rejecting the PIH at any reasonable level of significance.
To compute the GMM LR-type statistic (21.28), one must compute restricted and unrestricted GMM estimates using the same estimate Ŝ−1 ofthe eﬃcient weight matrix and evaluate the corresponding J-statistics.
Consider computing (21.28) using the iterated eﬃcient GMM estimate as
the unrestricted estimate. Its J-statistic is J(δ̂(Ŝ−1 ), Ŝ−1 ) = 1.855. To
compute the restricted eﬃcient GMM estimate with Ŝ−1 from the unrestricted iterated eﬃcient estimation as the weight matrix, use the following
commands:> s.ur = solve(gmm.fit.iter$weight.matrix)> s.ur = (s.ur + t(s.ur))/2> start.vals = 0.1> names(start.vals) = c("const")> gmm.fit.r = GMM(start.vals, iv.moments,+
method = "iterative", max.steps = 0,+
w = s.ur, w0.efficient = T,+y = consump[2:nobs,1],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,4])
1-step objective = 18.8505
The second line above is used to ensure that the weight matrix passed
to GMM is symmetric. The restricted model is specified using the function
iv.moments with zt = (1), and xt = (∆ct−1 , ∆yt−1 , rt−1 , 1)0 . The restricted
fit is given by> summary(gmm.fit.r)
Call:
GMM(start = start.vals, moments = iv.moments, method ="iterative", w = s.ur, max.steps = 0, w0.efficient = T,y = consump[2:nobs, 1], X = consump[1:(nobs - 1),],Z = consump[2:nobs, 4])
Coefficients:
Value Std.Error t value Pr(>|t|)const 0.0209 0.0012
17.0687 0.0000
Test of Overidentification:
J-stat Df P.value

812

21. Generalized Method of Moments

18.8505 3

0.0003

Optimization Info:
Number of Iterative Steps: 1
The restricted J-statistic is J(δ̃ R (Ŝ−1 ), Ŝ−1 ) = 18.8505. The GMM-LR
statistic is then
> gmm.lr = gmm.fit.r$objective - gmm.fit.iter$objective> gmm.lr[1] 16.99482
which is numerically identical to the Wald statistic computed earlier.

21.5.2 Testing Subsets of Orthogonality Conditions
Consider the linear GMM model (21.1) with instruments xt = (x01t , x02t )0such that x1t is K1 × 1 and x2t is K2 × 1 with K1 ≥ L and K1 + K2 = K.
The instruments x1t are assumed to be valid (i.e., E[x1t εt ] = 0), whereas
the instruments x2t are suspected not to be valid (i.e., E[x2t εt ] 6= 0). A
procedure to test for the validity of x2t due to Newey (1985) is as follows.
First, estimate (21.1) by eﬃcient GMM using the full set of instruments
xt , giving
−1
0
−1 0
δ̂(Ŝ−1
Sxz Ŝ−1
Full ) = (Sxz ŜFull Sxz )Full Sxywhere
ŜFull =∙Ŝ11,Full
Ŝ21,Full

Ŝ12,Full
Ŝ22,Full¸such that Ŝ11,Full is K1 × K1 . Second, estimate (21.1) by eﬃcient GMM
using only the instruments x1t and using the weight matrix Ŝ−1
11,Full giving
−1
0
−1 0
Sx1 z Ŝ−1
δ̃(Ŝ−1
11,Full ) = (Sx1 z Ŝ11,Full Sx1 z )11,Full Sx1 y

Next, form the statistic9−1−1−1
C = J(δ̂(Ŝ−1Full ), ŜFull ) − J(δ̃(Ŝ11,Full ), Ŝ11,Full )(21.29)Under the null hypothesis that E[xt εt ] = 0, the statistic C has a limiting
chi-square distribution with K − K1 degrees of freedom.
Example 145 Testing the endogeneity of rt in the consumption function
Consider testing the hypothesis that rt in (21.20) is exogenous (E[rt εt ] =0). In this case, the full set of instruments is xt = (∆ct−1 , ∆yt−1 , rt−1 , 1, rt )0
and the reduced set is x1t = (∆ct−1 , ∆yt−1 , rt−1 , 1)0 . Eﬃcient GMM estimation of the full model is achieved using
9 The use of Ŝ−1
11,Full guarantees that the C statistic is non-negative.

21.5 Hypothesis Testing for Linear Models

813> start.vals = rep(0.1,3)> names(start.vals) = c("GY","R3","const")> gmm.fit.full = GMM(start.vals,iv.moments,+
method = "iterative", max.steps = 100,+y = consump[2:nobs,"GC"],+
X = cbind(consump[1:(nobs-1),],+
consump[2:nobs,"R3"]),+Z = consump[2:nobs,2:4])
The eﬃcient weight matrix Ŝ−1
11,Full may be extracted using> w11.full = solve(gmm.fit.full$weight.matrix[1:4,1:4])> w11.full = (w11.full + t(w11.full))/2
Eﬃcient GMM estimation using x1t together with Ŝ−1
11,Full may be computed using> start.vals = rep(0.1,3)> names(start.vals) = c("GY","R3","const")> gmm.fit.11 = GMM(start.vals,iv.moments,+
method = "iterative", max.steps = 0,+w = w11.full, w0.efficient = T,+y = consump[2:nobs,"GC"],+
X = consump[1:(nobs-1),],+Z = consump[2:nobs,2:4])
The C statistic (21.29) for testing the exogeneity of rt is then> C.stat = gmm.fit.full$objective - gmm.fit.11$objective> C.stat[1] 0.01821106
Since K = 5 and K1 = 4, C has an limiting chi-square distribution with
one degree of freedom. The p-value for the test
> 1 - pchisq(C.stat,1)[1] 0.8926527
indicates that rt may be treated as exogenous in the consumption function.

21.5.3 Testing Instrument RelevanceIn order to obtain consistent GMM estimates, the instruments xt must be
uncorrelated with the error term εt (valid instruments) and they must be
correlated with the endogenous variables zt (relevant instruments). The
subset orthogonality tests of the previous subsection can be used to test
instrument validity. This subsection discusses some simple tests for instrument relevance.

814

21. Generalized Method of Moments

Instrument relevance is related to the rank condition (21.4). To see this,consider the simple GMM regression involving a single endogenous variable
and a single instrument
yt = zt δ + εt
E[xt εt ] = 0
The rank condition (21.4) reduces to rank(Σzx ) = 1, which implies thatΣzx 6= 0. Assuming that both zt and xt are demeaned, the rank condition
can be restated as cov(x, z) = Σzx 6= 0. Hence, the rank condition will
be satisfied as long as x is correlated with z. If there are K instruments
x1t , . . . , xKt but only one endogenous variable zt , then the rank condition
holds as long as cov(xk , z) 6= 0 for some k. If cov(xk , z) ≈ 0 for all k, thenthe instruments are called weak.
Testing instrument relevance is important in practice because recent research (e.g., Stock and Wright (2000)) has shown that standard GMM
procedures for estimation and inference may be highly misleading if instruments are weak. If instruments are found to be weak, then nonstandard
methods of inference should be used for constucting confidence intervals
and performing hypothesis tests. Stock, Wright, and Yogo (2002) gave anice survey of the issues associated with using GMM in the presence of
weak instruments and discussed the nonstandard inference procedures thatshould be used.
In the general linear GMM regression (21.1), the relevance of the set of
instruments xt for each endogenous variable in zt can be tested as follows.
First, let z1t denote the L1 × 1 vector of nonconstant endogenous variables
in zt , and let x1t denote the K1 × 1 remaining deterministic or exogenous
variables in zt such that zt = (z01t , x01t )0 and L1 + K1 = L. Similarly, define
x2t as the K2 × 1 vector of exogenous variables that are excluded fromzt so that xt = (x01t , x02t )0 and K1 + K2 = K. What is important for therank condition are the correlations between the endogenous variables in
z1t and the instruments in x2t . To measure these correlations and to test
for instrument relevance, estimate by least squares the so-called first-stage
regression
z1lt = x01t π 1l + x2t π 2l + vlt , l = 1, . . . , L1
for each endogenous variable in z1t . The t-ratios on the variables in x2t
can be used to assess the strength of the correlation between z1lt and thevariables in x2t . The F -statistic for testing π 2l = 0 can be used to assess
the joint relevance of x2t for z1lt .
Example 146 Testing instrument relevance in the consumption function
0

In the consumption function regression, z1t = (∆yt , rt ) , x1t = (1), and
x2t = (∆ct−1 , ∆yt−1 , rt−1 )0 . The first stage regressions for ∆yt and rt may
be computed simultaneously using the S+FinMetrics function OLS as follows:

21.5 Hypothesis Testing for Linear Models

815> firstStage.fit = OLS(cbind(GY,R3) ~ tslag(GC) + tslag(GY)++ tslag(R3), data = consump)> class(firstStage.fit)[1] "mOLS"
When a multivariate response is specified in the call to OLS, a regression
is performed for each response variable and the returned object is of class"mOLS". A summary of each first-stage regression is> summary(firstStage.fit)
Response: GY
Call:
OLS(formula = cbind(GY, R3) ~tslag(GC) + tslag(GY) + tslag(
R3), data = consump)Residuals:
Min
1Q Median-0.0305 -0.0122 -0.0021

3Q
0.0112Max
0.0349Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0067 0.0055
1.2323 0.2271
tslag(GC) 1.2345 0.3955
3.1214 0.0039
tslag(GY) -0.5226 0.2781-1.8787 0.0697
tslag(R3) 0.0847 0.1395
0.6069 0.5483
Regression Diagnostics:
R-Squared 0.2791
Adjusted R-Squared 0.2093
Durbin-Watson Stat 1.8808
Residual Diagnostics:
Stat P-Value
Jarque-Bera 0.6181 0.7341
Ljung-Box 8.9289 0.8812
Residual standard error: 0.0163 on 31 degrees of freedom
F-statistic: 4 on 3 and 31 degrees of freedom, the p-value
is 0.01617
Response: R3
Call:
OLS(formula = cbind(GY, R3) ~tslag(GC) + tslag(GY) + tslag(

816

21. Generalized Method of Moments

R3), data = consump)Residuals:
Min
1Q-0.0264 -0.0069

Median0.0021

3Q
0.0068Max
0.0436

Coefficients:
Value Std. Error t value Pr(>|t|)(Intercept) 0.0083 0.0044
1.8987 0.0669
tslag(GC) 0.1645 0.3167
0.5192 0.6073
tslag(GY) -0.4290 0.2228-1.9259 0.0633
tslag(R3) 0.8496 0.1117
7.6049 0.0000
Regression Diagnostics:
R-Squared 0.6519
Adjusted R-Squared 0.6182
Durbin-Watson Stat 1.7470
Residual Diagnostics:
Stat P-Value
Jarque-Bera 14.8961 0.0006
Ljung-Box 20.7512 0.1450
Residual standard error: 0.01305 on 31 degrees of freedom
F-statistic: 19.35 on 3 and 31 degrees of freedom, the p-valueis 2.943e-007
In the first-stage regression for ∆yt , the t-ratios for ∆ct−1 and ∆yt−1 aresignificant at the 1% and 10% levels, respectively, indicating that thesevariables are correlated with ∆yt . The F -statistic for testing the joint significance of the variables in x2t is 4, with a p-value of 0.0167 indicatingthat x2t is relevant for ∆yt .

21.6 Nonlinear GMM
Nonlinear GMM estimation occurs when the K GMM moment conditionsg(wt , θ) are nonlinear functions of the p model parameters θ. Dependingon the model, the moment conditions g(wt , θ) may be K ≥ p nonlinear
functions satisfying
E[g(wt , θ 0 )] = 0(21.30)

21.6 Nonlinear GMM

817

Alternatively, for a response variable yt , L explanatory variables zt , and K
instruments xt , the model may define a nonlinear error term εta(yt , zt ; θ 0 ) = εtsuch thatE[εt ] = E[a(yt , zt ; θ0 )] = 0
Given that xt is orthogonal to εt , define g(wt , θ 0 ) = xt εt = xt a(yt , zt ; θ0 )so thatE[g(wt , θ 0 )] = E[xt εt ] = E[xt a(yt , zt ; θ0 )] = 0(21.31)

defines the GMM orthogonality conditions.
In general, the GMM moment equations (21.30) and (21.31) produce asystem of K nonlinear equations in p unknowns. Identification of θ0 requiresthat
E[g(wt , θ 0 )] = 0
E[g(wt , θ)] 6= 0 for θ 6= θ0
and the K × p matrix∙¸∂g(wt , θ0 )(21.32)∂θ 0
has full column rank p. The sample moment condition for an arbitrary θ is
G=E

gn (θ) = n−1nXg(wt , θ)

t=1

If K = p, then θ0 is apparently just identified and the GMM objectivefunction isJ(θ) = ngn (θ)0 gn (θ)
which does not depend on a weight matrix. The corresponding GMM estimator is thenθ̂ = arg min J(θ)θ

If K > p, then θ 0 is apparently overidentified. Let Ŵ denote a K × K
symmetric and p.d. weight matrix, possibly dependent on the data, suchpthat Ŵ → W as n → ∞ with W symmetric and p.d. Then, the GMM
estimator of θ 0 , denoted θ̂(Ŵ), is defined asθ̂(Ŵ) = arg min J(θ, Ŵ) = ngn (θ)0 Ŵgn (θ)
θ

−1

p

The eﬃcient GMM estimator uses Ŵ = Ŝ such that Ŝ→S = avar(ḡ).
As with eﬃcient GMM estimation of linear models, the eﬃcient GMM
estimator of nonlinear models may be computed using a two-step, iterated,or continuous updating estimator.

818

21. Generalized Method of Moments

21.6.1 Asymptotic Properties
Under standard regularity conditions (see Hayashi, 2000, Chap. 7), it canbe shown that
p

where

θ̂(Ŵ) → θ0´√ ³dn θ̂(Ŵ) − θ 0 → N (0, avar(θ̂(Ŵ)))avar(θ̂(Ŵ)) = (G0 WG)−1 G0 WSWG(G0 WG)−1(21.33)and G is given by (21.32). If W = S−1 , thenavar(θ̂(Ŝ−1 )) = (G0 S−1 G)−1

(21.34)

Notice that with nonlinear GMM, the expression for avar(θ̂(Ŵ)) is of the0same formh as in ilinear GMM except that Σxz = E[xt zt ] is replaced byt ,θ 0 )G = E ∂g(w.
∂θ 0
A consistent estimate of avar(θ̂(W)), denoted avar(d θ̂(Ŵ)), may be computed usingavar(d θ̂(Ŵ)) = (Ĝ0 ŴĜ)−1 Ĝ0 ŴŜŴĜ(Ĝ0 ŴĜ)−1(21.35)where Ŝ is a consistent estimate for S = avar(ḡ) and−1

Ĝ = Gn (θ̂(Ŵ)) = nnX∂g(wt , θ̂(Ŵ))

∂θ 0

t=1

−1

For the eﬃcient GMM estimator, Ŵ = Ŝ

and

avar(
d θ̂(Ŝ−1 )) = (Ĝ0 Ŝ−1 Ĝ)−1(21.36)

If {gt (wt , θ0 )} is an ergodic stationary MDS with E[gt (wt , θ 0 )gt (wt , θ 0 )0 ]= S, then a consistent estimator of S takes the form
n−1

n
X

gt (wt , θ̂)gt (wt , θ̂)0

t=1

where θ̂ is any consistent estimator of θ 0 . If {gt (wt , θ 0 )} is a serially correlated ergodic stationary process, thenS = avar(ḡ) = Γ0 + 2∞
X(Γj + Γ0j )j=1

and the methods discussed in Section 21.3 may be used to consistently
estimate S.

21.7 Examples of Nonlinear Models

819

21.6.2 Hypothesis Tests for Nonlinear Models
Most of the GMM test statistics discussed in the context of linear models
have analogs in the context of nonlinear GMM. For example, the J-statistic
for testing the validity of the K moment conditions (21.31) has the form
(21.15) with the eﬃcient GMM estimate θ̂(Ŝ−1 ) in place of δ̂(Ŝ−1 ). The
asymptotic null distribution of the J-statistic is chi-squared with K − pdegrees of freedom. Wald statistics for testing linear and nonlinear restriction on θ have the same form as (21.24) and (21.26), with θ̂(Ŵ) in placeof δ̂(Ŵ). Similarly, GMM LR-type statistics for testing linear and nonlinear restriction on θ have the form (21.28), with θ̃(Ŝ−1 ) in place of δ̃(Ŝ−1 )and with θ̂(Ŝ−1 ) in place of δ̂(Ŝ−1 ). Testing the relevance of instruments
in nonlinear models, however, is not as straightforward as it is in linear
models. In nonlinear models, instruments are relevant if the K × p matrix
G defined in (21.32) has full column rank p. Testing this condition is problematic because G depends on θ 0 , which is unknown. See Wright (2003) for
an approach that can be used to test for instrument relevance in nonlinear
models.

21.7 Examples of Nonlinear Models
The following subsections give detailed examples of estimating nonlinear
models using GMM.

21.7.1 Student’s t Distribution
As in Hamilton (1994, Chap. 14), consider a random sample y1 , . . . , yT
that is drawn from a centered Student’s t distribution with θ0 degrees of
freedom. The density of yt has the formf (yt ; θ0 ) =Γ[(θ0 + 1)/2][1 + (yt2 /θ0 )]−(θ0 +1)/2(πθ0 )1/2 Γ(θ0 /2)

where Γ(·) is the gamma function. The goal is to estimate the degrees of
freedom parameter θ0 by GMM using the moment conditions
E[yt2 ] =E[yt4 ] =θ0
θ0 − 23θ20(θ0 − 2)(θ0 − 4)

which require θ0 > 4. Let wt = (yt2 , yt4 )0 and define¶µyt2 − θ/(θ − 2)g(wt , θ) =yt4 − 3θ2 /(θ − 2)(θ − 4)

(21.37)

820

21. Generalized Method of Moments

Then, E[g(wt ,θ0 )] = 0 is the moment condition used for defining the GMM
estimator for θ0 . Here, K = 2 and p = 1, so θ0 is apparently overidentified.
Using the sample momentsngn (θ) =1X
g(wt , θ) =
n t=1

µ

1

n

Pn

2
t=1 yt − θ/(θ − 2)24
t=1 yt − 3θ /(θ − 2)(θ − 4)

Pnn
1

¶

the GMM objective function has the form
J(θ) = ngn (θ)0 Ŵgn (θ)
where Ŵ is a 2 × 2 p.d. and symmetric weight matrix, possibly dependentpon the data, such that Ŵ→W. The eﬃcient GMM estimator uses thepweight matrix Ŝ−1 such that Ŝ → S = E[g(wt , θ0 )g(wt , θ0 )0 ].
A random sample of n = 250 observations from a centered Student’s t
distribution with θ0 = 10 degrees of freedom may be generated using the
S-PLUS function rt as follows:> set.seed(123)> y = rt(250,df = 10)
Basic summary statistics, computed using> summaryStats(y)Sample Quantiles:min
1Q median
3Q
max-4.387 -0.6582 -0.0673 0.6886 3.924
Sample Moments:
mean
std skewness kurtosis-0.03566 1.128 -0.3792
4.659
Number of Observations:250

indicate that the data are roughly symmetric about zero and have thicker
tails than the normal distribution.
An S-PLUS function to compute the moment condition (21.37) for t =1, . . . , n ist.moments <- function(parm,data=NULL) {# parm = df parameter# data = [y^2, y^4] is assumed to be a matrix
m1 = parm/(parm - 2)
m2 = 3*parm*parm/((parm - 2)*(parm - 4))t(t(data) - c(m1,m2))}

21.7 Examples of Nonlinear Models

821

The function t.moments has arguments parm, specifying the degrees of
freedom parameter θ, and data, specifying an n × 2 matrix with the tth
row wt = (yt2 , yt4 )0 .
To compute the iterated eﬃcient GMM estimator of the degrees of freedom parameter θ from the simulated Student’s t data use> y = y - mean(y)> t.data = cbind(y^2,y^4)> start.vals = 15
> names(start.vals) = c("theta")> t.gmm.iter = GMM(start.vals, t.moments,+
method = "iterative", max.steps = 100,+
data = t.data)
1-step objective = 0.471416
2-step objective = 0.302495
3-step objective = 0.302467> summary(t.gmm.iter)
Call:
GMM(start = start.vals, moments = t.moments, method ="iterative", max.steps = 100, data = data)
Coefficients:
Value Std.Error t value Pr(>|t|)theta 7.8150 1.1230
6.9592 0.0000
Test of Overidentification:
J-stat Df P.value
0.3025 1 0.5823
Optimization Info:
Number of Iterative Steps: 3
The iterated eﬃcient GMM estimate of θ0 is 7.8150, with an asymptotic
standard error of 1.123. The small J-statistic indicates a correctly specified
model.

21.7.2 MA(1) Model
Following Harris (1999), consider GMM estimation of the parameters in
the moving average MA(1) modelyt = μ0 + εt + ψ 0 εt−1 , t = 1, . . . , nεt ∼ iid (0, σ 20 ), |ψ 0 | < 1
θ 0 = (μ0 , ψ 0 , σ 20 )0

822

21. Generalized Method of Moments

Some population moment equations that can be used for GMM estimationareE[yt ]
E[yt2 ]
E[yt yt−1 ]E[yt yt−2 ]

=
=
=
=

μ0
μ20 + σ 20 (1 + ψ 20 )μ20 + σ 20 ψ 0
μ20

Let wt = (yt , yt2 , yt yt−1 , yt yt−2 )0 and define the moment vector⎞⎛yt − μ⎜ y 2 − μ2 − σ 2 (1 + ψ 2 ) ⎟t⎟g(wt , θ) = ⎜⎝ yt yt−1 − μ2 − σ 2 ψ ⎠yt yt−2 − μ2(21.38)Then,

E[g(wt , θ0 )] = 0
is the population moment condition used for GMM estimation of the model
parameters θ 0 . The sample moments are⎞⎛Pn
t=3 yt /(n − 2) − μ
P
n
n
2 ⎟
2
2
2
⎜
1 X
t=3 yt /(n − 2) − μ − σ (1 + ψ ) ⎟P
g(wt , θ) = ⎜gn (θ) =n2
2⎠⎝n − 2 t=3
t=3
Pnyt yt−1 /(n − 2) − μ − 2σ ψ
y
y
/(n
−
2)
−
μ
t
t−2
t=3

Since the number of moment conditions K = 4 is greater than the numberof model parameters p = 3, θ0 is apparently overidentified and the eﬃcient
GMM objective function has the form
J(θ) = (n − 2) · gn (θ)0 Ŝ−1 gn (θ)

where Ŝ is a consistent estimate of S = avar(ḡ(θ 0 )). Notice that the process {g(wt , θ0 )} will be autocorrelated (at least at lag 1) since yt follows
an MA(1) process. As a result, an HAC-type estimator must be used toestimate S.
Simulated MA(1) data with θ 0 = (0, 0.5, 1)0 and n = 250 is computed
using the S-PLUS function arima.sim10 :> set.seed(123)> ma1.sim = arima.sim(model = list(ma=-0.5),n=250)
These data along with the sample autocorrelation function (SACF) andsample partial autocorrelation function (SPACF) are illustrated in Figure21.2. Summary statistics for the simulated data are
10 Recall that the S-PLUS function arima.sim reverses the sign of the moving average
parameter ψ.

21.7 Examples of Nonlinear Models

823

-2

0 1 2 3

Simulated MA(1) Data

0

50

100

150

200

250

0.0

0.4

0.8

Sample ACF

0

5

10

15

20

15

20

Lag

-0.2

0.0

0.2

0.4

Sample PACF

0

5

10
Lag

FIGURE 21.2. Simulated data, SACF and SPACF from MA(1) model withθ = (0.5, 1)0 .

> summaryStats(ma1.sim)
Sample Quantiles:min
1Q median
3Q
max-2.606 -0.6466 0.1901 0.8755 3.221
Sample Moments:
mean
std skewness kurtosis
0.1126 1.071 -0.0624
2.634
Number of Observations:250

An S-PLUS function to compute the moment conditions (21.38) is11
ma1.moments <- function(parm, data = NULL) {# parm = (mu,psi,sig2)’# data = (y(t),y(t)^2,y(t)*y(t-1),y(t)*y(t-2))
m1 = parm[1]m2 = parm[1]^2 + parm[3]*(1 + parm[2]^2)
11 In the function ma1.moments, the parameters are unrestricted. To force the moving
average parameter to satisfy |ψ| < 1, use the logistic transformation ψ = exp(γ 1 )/(1 +exp(γ 1 )), and to force the variance parameter to be positive, use σ 2 = exp(γ 2 ).

824

21. Generalized Method of Moments

m3 = parm[1]^2 + parm[3]*parm[2]m4 = parm[1]^2t(t(data) - c(m1,m2,m3,m4))}
The function ma1.moments has arguments parm, specifying the model parameters θ = (μ, ψ, σ 2 )0 , and data, specifying an (n − 2) × 4 matrix withthe tth row wt = (yt , yt2 , yyt−1 , yt yt−2 )0 . The first five rows of g(wt , θ0 ) are> ma1.data = cbind(ma1.sim[3:nobs],ma1.sim[3:nobs]^2,+
ma1.sim[3:nobs]*ma1.sim[2:(nobs-1)],+
ma1.sim[3:nobs]*ma1.sim[1:(nobs-2)])> start.vals = c(0,0.5,1)> names(start.vals) = c("mu","psi","sig2")> ma1.mom = ma1.moments(parm = start.vals, data = ma1.data)> ma1.mom[1:5,]
[,1]
[,2]
[,3]
[,4]
[1,] 1.24643 0.303579 1.10981 -0.071482[2,] -0.80526 -0.601549 -1.50370 -1.040035[3,] 1.13258 0.032744 -1.41203 1.411681[4,] 1.58545 1.263659 1.29566 -1.276709[5,] 0.67989 -0.787743 0.57794 0.770037
The sample average of g(wt , θ0 ) is
> colMeans(ma1.mom)[1] 0.10852675 -0.09134586

0.01198312

0.05919433

which is somewhat close to the population value E[g(wt , θ0 )] = 0. The
sample autocorrelations and cross-autocorrelations of g(wt , θ 0 ) are shownin Figure 21.3, which confirm the need for an HAC-type estimator for S.
To estimate the MA(1) model by GMM with S estimated using a truncated (rectangular) kernel with bandwidth equal to one lag, use12 :> start.vals = c(0,0.5,1)> names(start.vals) = c("mu","psi","sig2")> ma1.gmm.trunc = GMM(start.vals, ma1.moments,+
data = ma1.data,ts=T,+
var.hac.control = var.hac.control(bandwidth = 1,+
window = "truncated"))
1-step objective = 0.530132
2-step objective = 0.354946
3-step objective = 0.354926
The fitted results are
12 Recall that Ŝ computed with a truncated kernel is not guaranteed to be positive
definite.

21.7 Examples of Nonlinear Models

Series 1 and Series 3
0.15

0.10
5

10

15

0.0
-0.1

-0.05
0

5

10

15

0

5

Series 2

10

15

0

10

15

0.6

Series 2 and Series 4

0.0

0.2

0.4

0.05

0.6

0.4

0.10

0.8

0.10
0.05
ACF
0.0
-0.15

-0.15

0.0

0.0

-0.10

0.2

-0.05

-0.05
-0.10

5

Series 2 and Series 3

1.0

Series 2 and Series 1

0.15

0

-0.10

0.0

-0.10

0.2

-0.05

0.0

0.0

ACF
0.4

0.05

0.6

0.05

0.1

0.10

0.8

Series 1 and Series 4
0.2

Series 1 and Series 2

1.0

Series 1

825

-10

-5

0

0

5

10

15

0

5

Series 3 and Series 2
0.6

15

0

10

15

-15

-10

-5

0

-15

-10

-5

0

0

5

Series 4 and Series 2

10

15

0

10

15

10

15

Series 4
1.0
0.8

0.3

-15

-10

Lag

-5

0

0.4
0.2
0.0

0.0
-0.1

-0.1

-0.10

-0.05

0.0

0.0

0.1

ACF

0.05

0.6

0.2

0.10

0.15

0.2
0.1

5

Series 4 and Series 3

0.20

Series 4 and Series 1

-0.1

-0.1

0.0

0.0

0.0

0.2

0.0

0.2

0.4

0.1

0.6

0.2

0.4

0.1
ACF

5

Series 3 and Series 4

0.8

0.2

10

Series 3
1.0

Series 3 and Series 1

0.3

-15

-15

-10

Lag

-5

0

-15

-10

Lag

-5

0

0

5

Lag

FIGURE 21.3. SACF of g(wt , θ 0 ) from the MA(1) model.

> summary(ma1.gmm.trunc)
Call:
GMM(start = start.vals, moments = ma1.moments, ts = T,
var.hac.control = var.hac.control(bandwidth = 1,
window = "truncated"), data = ma1.data)
Coefficients:
Value Std.Error t value Pr(>|t|)mu 0.1018 0.0927
1.0980 0.2733
psi 0.5471 0.0807
6.7788 0.0000
sig2 0.8671 0.0763
11.3581 0.0000
Test of Overidentification:
J-stat Df P.value
0.3549 1 0.5513
Optimization Info:
Number of Iterative Steps: 3
The GMM estimate of μ0 is close to the sample mean, the estimate of ψ 0
is slightly larger than 0.5, and the estimate of σ 20 is slightly smaller than 1.
The low J-statistic indicates a correctly specified model.

826

21. Generalized Method of Moments

To illustrate the impact of model misspecification on the GMM estimation, consider fitting an MA(1) model by GMM to data simulated from an
autoregression AR(1) model:
yt − μ0 = φ0 (yt−1 − μ0 ) + εt , εt ∼ iid N (0, σ 20 )The AR(1) model has moments
E[yt ]
E[yt2 ]
E[yt yt−1 ]E[yt yt−2 ]

=
=
=
=

μ0
μ20 + σ20 /(1 − φ20 )μ20 + φσ20 /(1 − φ20 )μ20 + φ2 σ 20 /(1 − φ20 )Simulated AR(1) data with μ0 = 0, φ0 = 0.5, σ 20 = 1 and n = 250 iscomputed using the S-PLUS function arima.sim:> set.seed(123)> ar1.sim = arima.sim(model=list(ar=0.5),n=250)
The moment data required for GMM estimation of the MA(1) model are> nobs = numRows(ar1.sim)> ar1.data = cbind(ar1.sim[3:nobs],ar1.sim[3:nobs]^2,+
ar1.sim[3:nobs]*ar1.sim[2:(nobs-1)],+
ar1.sim[3:nobs]*ar1.sim[1:(nobs-2)])
The iterated eﬃcient GMM estimates of the misspecified MA(1) modelare> ma1.gmm.ar1 = GMM(start.vals, ma1.moments,+
data=ar1.data,ts = T,+
var.hac.control = var.hac.control(bandwidth = 1,+
window = "truncated"))
1-step objective = 25.6768...
9-step objective = 13.0611> summary(ma1.gmm.ar1, print.moments = T)
Call:
GMM(start = start.vals, moments = ma1.moments, ts = T,
var.hac.control = var.hac.control(bandwidth = 1,
window = "truncated"), data = ar1.data)
Coefficients:
Value Std.Error
mu -0.0407
0.0909psi
0.41990.0789sig2
0.78400.0737t value Pr(>|t|)-0.44780.65475.32070.000010.6391
0.0000

21.7 Examples of Nonlinear Models

827

Test of Overidentification:
J-stat Df P.value
13.0611 1 0.0003
Optimization Info:
Number of Iterative Steps: 9
Normalized Moments:
Moment Std.Error t value Pr(>|t|)Moment 1
2.9166 0.8072
3.6131 0.0003
Moment 2
5.3807 1.4888
3.6140 0.0003
Moment 3
5.6389 1.5603
3.6140 0.0003
Moment 4
5.8329 1.6140
3.6140 0.0003
The GMM estimates of the MA(1) model parameters look reasonable. However, the large J-statistic correctly indicates a misspecified model. Setting
the optional argument print.moments=T adds the normalized moment information to the summary output. Since the J-statistic is large, all of thenormalized moments are significantly diﬀerent from zero.

21.7.3 Euler Equation Asset Pricing Model
Following Hansen and Singleton (1982), a representative agent is assumed
to choose an optimal consumption path by maximizing the present discounted value of lifetime utility from consumption
max

∞
X
t=1

subject to the budget constraint£¤
E β t0 U (Ct )|It

Ct + Pt Qt ≤ Vt Qt−1 + Wt
where It denotes the information available at time t, Ct denotes real consumption at t, Wt denotes real labor income at t, Pt denotes the price of apure discount bond maturing at time t + 1 that pays Vt+1 , Qt represents
the quantity of bonds held at t, and β 0 represents a time discount factor.
The first-order condition for the maximization problem may be represented
as the conditional moment equation (Euler equation)∙¸
U 0 (Ct+1 )E (1 + Rt+1 )β 0 0|It − 1 = 0
U (Ct )
is the gross return on the bond at time t + 1.
where 1 + Rt+1 = VVt+1
t
Assuming utility has the power form
U (C) =

C 1−α01 − α0828

21. Generalized Method of Moments

where α0 represents the intertemporal rate of substitution (risk aversion),
then
¶−α0
µ
U 0 (Ct+1 )Ct+1=U 0 (Ct )Ctand the conditional moment equation becomes"µ¶−α0 #
Ct+1|It − 1 = 0(21.39)E (1 + Rt+1 )β 0
CtDefine the nonlinear error term as
εt+1= a(Rt+1 , Ct+1 /Ct ; α0 , β 0 ) = (1 + Rt+1 )β 0
= a(zt+1 , θ0 )

µ

Ct+1
Ct

¶−α0

−1

with zt+1 = (Rt+1 , Ct+1 /Ct )0 and θ0 = (α0 , β 0 )0 . Then, the conditional
moment equation (21.39) may be represented asE[εt+1 |It ] = E[a(zt+1 , θ 0 )|It ] = 0
Since {εt+1 , It+1 } is a MDS, potential instruments xt include current andlagged values of the elements in zt as well as a constant. For example, one
could use
xt = (1, Ct /Ct−1 , Ct−1 /Ct−2 , Rt , Rt−1 )0
Since xt ⊂ It , the conditional moment (21.39) implies thatE[xt εt+1 |It ] = E[xt a(zt+1 , θ 0 )|It ] = 0
and by the law of total expectations,
E[xt εt+1 ] = 0
For the GMM estimation, define the nonlinear residual as
µ
¶−α
Ct+1
−1
et+1 = (1 + Rt+1 )βCt
and form the vector of momentsg(wt+1 , θ) = xt et+1 = xt a(zt+1 , θ)(21.40)⎛⎞³´−α(1 + Rt+1 )β CCt+1
−
1
t
⎜
µ
¶ ⎟
³
´−α
⎜
⎟
⎜ (Ct /Ct−1 ) (1 + Rt+1 )β Ct+1
⎟
−
1
⎜
⎟
Ct
⎜
⎟
µ
¶
³
´−α
⎜
⎟
⎜ (Ct−1 /Ct−2 ) (1 + Rt+1 )β Ct+1
⎟
−
1
= ⎜
⎟
Ct
⎜
⎟
µ
¶
´−α
³
⎜
⎟
Ct+1
⎜
⎟
R
(1
+
R
)β
−
1
t
t+1
⎜
⎟
Ct
⎜
⎟
¶
µ
³
´−α
⎝
⎠
Ct+1
Rt−1 (1 + Rt+1 )β Ct−1

21.7 Examples of Nonlinear Models

829

In (21.40), there are K = 5 moment conditions to identify L = 2 model parameters giving K −L = 3 overidentifying restrictions. The GMM objectivefunction isJ(θ, Ŝ−1 ) = (n − 2) · gn (θ)0 Ŝ−1 gn (θ)

where Ŝ is a consistent estimate of S = avar(ḡ).
An S-PLUS function to compute the Euler moments (21.40) is
euler1.moments <- function(parm, data = NULL) {# parm = (beta,gamma)# data = (1+R(t+1),C(t+1)/C(t),1,C(t)/C(t-1),#
C(t-1)/C(t-2),R(t),R(t-1),...)
ncol = numCols(data)
euler = data[,1]*parm[1]*(data[,2]^(-parm[2])) - 1
as.matrix(rep(euler,(ncol-2))*data[,3:ncol])}The function euler1.moments has arguments parm, specifying the model
parameters θ = (β, α)0 , and data, specifying an (n − 2) × 7 matrix withthe tth row wt+1 = (1 + Rt+1 , Ct+1 /Ct , 1, Ct /Ct−1 , Ct−1 /Ct−2 , Rt , Rt−1 )0 .
Verbeek (2000, Chap. 5, Sec. 7) describes an extension of the model
presented above that allows the individual to invest in J risky assets withreturns Rj,t+1 (j = 1, . . . , J), as well as a risk-free asset with certain return
Rf,t+1 . Assuming power utility and restricting attention to unconditional
moments (i.e., using xt = 1), the Euler equations may be written as"µ¶−α0 #
Ct+1
E (1 + Rf,t+1 )β 0
−1 = 0(21.41)Ct"µ¶−α0 #Ct+1
E (Rj,t+1 − Rf,t+1 )β 0
= 0, j = 1, . . . , J
CtDefine the stochastic discount factor as
µ
¶−α0
Ct+1
mt+1 (θ0 ) = β 0
Ct
so that the risky assets satisfy
E[(Rj,t+1 − Rf,t+1 ) · mt+1 (θ0 )] = 0
Using the identity E[xy] = cov(x, y) + E[x]E[y], the risk premium for therisky assets may be deduced as
E[Rj,t+1 − Rf,t+1 ] = −cov(mt+1 (θ 0 ), Rj,t+1 − Rf,t+1 )E[mt+1 (θ 0 )](21.42)

If the asset pricing model is correct, then the right-hand side of (21.42)
should explain the cross-sectional variation of expected returns across assets.

830

21. Generalized Method of Moments

For the GMM estimation, define the J + 1 vector of moments
⎞
³
´−α
Ct+1
(1
+
R
)β
−
1
f,t+1
Ct
⎜
³
´−α ⎟
⎟
⎜
⎟
⎜ (R1,t+1 − Rf,t+1 )β Ct+1⎟⎜
Ct
g(wt+1 , θ) = ⎜
⎟
..
⎟
⎜
⎟
⎜
.
⎝
³
´−α ⎠
Ct+1
(RJ,t+1 − Rf,t+1 )β Ct⎛

An S-PLUS function to compute the above moments iseuler2.moments <- function(parm, data = NULL) {# parm = (beta,gamma)# data = (C(t+1)/C(t),1+Rf(t+1),R1(t+1)-Rf(t+1),...,#
RJ(t+1)-Rf(t+1))
ncol = numCols(data)
sdf = parm[1]*data[,1]^(-parm[2])d1 = (sdf - 1)*data[,2]d2 = as.matrix(rep(sdf,(ncol-2))*data[,3:ncol])cbind(d1,d2)}
The function euler2.moments has arguments parm, specifying the model
parameters θ = (β, α)0 , and data, specifying an n × (J + 1) matrix with thetth row wt+1 = (Ct+1 /Ct , 1+Rf,t+1 , R1,t+1 −Rf,t+1 , . . . , RJ,t+1 −Rf,t+1 )0 .
Following Verbeek (2000), the power utility asset pricing model is estimated using monthly data over the period February 1959 through November 1993.13 Ten size-based portfolios from the Center for Research in Security Prices (CRSP) data base are used as the risky assets. That is, portfolio
1 contains the monthly returns on the smallest 10% of firms (by market
capitalization) listed on the New York Stock Exchange, and portfolio 10
contains the returns on the largest 10% of firms. The risk-free asset is themonthly return on 3-month U.S. T-bills, and real consumption is measured
by total U.S. personal consumption expenditures on nondurables and services. The data are available in the S+FinMetrics "timeSeries" object
pricing.ts, which has variables> colIds(pricing.ts)[1] "CONS" "R1"
"R2"[9] "R8"
"R9"
"R10"

"R3"
"RF"

"R4"

"R5"

"R6"

"R7"

The data to be passed to the function euler2.moments are constructedusing
13 The
data
are
taken
from
http://www.econ.kuleuven.ac.be/GME.

Manro

Verbeek’s

web

page

at

21.7 Examples of Nonlinear Models

831> pricing.mat = as.matrix(seriesData(pricing.ts))> ncol = numCols(pricing.mat)> excessRet.mat = apply(pricing.mat[,2:(ncol-1)], 2,+function(x,y){x-y},+
pricing.mat[,"RF"])# data = (C(t+1)/C(t),1+Rf(t+1),R1(t+1)-Rf(t+1),...,#
RJ(t+1)-Rf(t+1))> euler.data = cbind(pricing.mat[,"CONS"],+
1 + pricing.mat[,"RF"],+
excessRet.mat)
The iterated eﬃcient GMM estimator may be computed using> start.vals = c(1,5)> names(start.vals) = c("beta","alpha")> euler.gmm.fit = GMM(start.vals, euler2.moments,+
method = "iterative",data = euler.data)> summary(euler.gmm.fit)
Call:
GMM(start = start.vals, moments = euler2.moments, method ="iterative", data = euler.data)
Coefficients:
Value Std.Error t value Pr(>|t|)beta 0.8314 0.1170
7.1049 0.0000
alpha 57.4020 34.3021
1.6734 0.0950
Test of Overidentification:
J-stat Df P.value
5.6574 9 0.7737
Optimization Info:
Number of Iterative Steps: 9
The small J-statistic indicates a correctly specified model. The estimateof the time discount parameter β is economically reasonable and fairly
precise, but the estimate of the risk-aversion parameter α is implausibly
large and imprecise. The large estimate of α illustrates the well-known
equity premium puzzle under which the high risk premia on equity assets
is associated with extremely risk-averse investors.
The one-step ineﬃcient GMM estimator with W = I11 may be computedusing> euler.gmm.fit2 = GMM(start.vals, euler2.moments,+
method = "iterative", max.steps = 0,+
w = diag(11), w0.efficient = F,

832

21. Generalized Method of Moments+
data = euler.data)> summary(euler.gmm.fit2)
Call:
GMM(start = start.vals, moments = euler2.moments, method ="iterative", w = diag(11), max.steps = 0, w0.efficient= F, data = euler.data)
Coefficients:
Value Std.Error t value Pr(>|t|)beta 0.7031 0.1446
4.8612 0.0000
alpha 91.4097 38.2102
2.3923 0.0172
The usual J-statistic is not valid for the one-step estimates so it is notreported. The one-step estimates are similar to the iterated eﬃcient GMM
estimates and have slightly larger estimated standard errors.
Cochrane (1996, 2001) recommended using the one-step GMM estimates
to compute empirical pricing errors based on (21.42), since these estimatesminimize such pricing errors by construction. The pricing errors can thenbe plotted to evaluate the economic significance of the proposed asset pricing model. The empirical pricing errors may be computed and plotted asfollows. First, the historical average excess returns on the 10 portfolios arecomputed using> excessRet.hat = colMeans(excessRet.mat)
Next, the average excess returns on the 10 portfolios predicted by themodel, based on the one-step ineﬃcient GMM estimates, may be computed
using the function
predRet <- function(parm,data) {# parm = (beta,gamma)# data = (C(t+1)/C(t),1+Rf(t+1),R1(t+1)-Rf(t+1),...,#
RJ(t+1)-Rf(t+1))
ncol = numCols(data)
sdf = parm[1]*data[,1]^(-parm[2])tmp = function(x,y) { -var(x,y)/mean(y) }ans = apply(data[,3:ncol], 2, FUN = tmp,sdf)
ans}> predRet.hat = predRet(coef(euler.gmm.fit2),euler.data)
Figure 21.4 shows the historical average excess returns and the predicted
average excess returns along with a 45◦ line. If the model is correct, thenall points should lie on the 45◦ line. The plot shows that the model underpredicts the small firm excess returns and overpredicts the large firm excess
returns.

833

0.006
0.004
0.0

0.002

Mean excess return

0.008

0.010

21.7 Examples of Nonlinear Models

0.0

0.002

0.004

0.006

0.008

0.010

Predicted mean excess return

FIGURE 21.4. Monthly actual versus predicted mean excess returns of size-based
portfolios.

21.7.4 Stochastic Volatility Model
The simple log-normal stochastic volatility (SV) model, due to Taylor(1986), is given by
yt = σ t Zt , t = 1, . . . , n(21.43)ln σ 2t = ω 0 + β 0 ln σ 2t−1 + σ 0,u ut(Zt , ut )0 ∼ iid N (0, I2 )
θ 0 = (ω 0 , β 0 , σ 0,u )0
For 0 < β 0 < 1 and σ 0,u ≥ 0, the series yt is strictly stationary and ergodic,
and unconditional moments of all orders exist. In the SV model, the series
yt is serially uncorrelated but dependency in the higher-order moments is
induced by the serially correlated stochastic volatility term ln σ2t . Shephard(1996) surveyed the use of autoregressive conditional heteroskedasticity(ARCH) and SV models in finance and noted some important advantages of
SV models over ARCH models. In particular, their statistical properties areeasier to find and understand, they generalize more easily to multivariate
settings, and they have simpler continuous-time representations that canbe used in contingent claims pricing.
Simulated data from (21.43) with ω 0 = −0.736, β 0 = 0.90, σ 0,u = 0.363,
and n = 1000 is illustrated in Figure 21.5. The data exhibit ARCH-like21. Generalized Method of Moments

-0.10

-0.05

0.0

0.05

0.10

834

0

200

400

600

FIGURE 21.5. Simulated data from
ω0 = −0.736, β 0 = 0.90, and σ0,u = 0.363.

SV

800

1000

model

(21.43)withfeatures such as volatility clustering and fat tails. The data is created withthe function sv.as.gensim, described in Chapter 22 on EMM estimation,using# beta.t = log(beta/(1-beta))> n.sim = 1000
> n.burn = 100> nz = n.sim + n.burn> set.seed(456)> sv.as.aux = list(z = rnorm(nz),u = rnorm(nz))> rho.as = c(-0.736,2.197225,0.363)> names(rho.as) = c("alpha","beta.t","sigma.u")> sv.as.sim = sv.as.gensim(rho = rho.as, n.sim = n.sim,+
n.burn = n.burn, aux = sv.as.aux)
The GMM estimation of the SV model is surveyed in Andersen andSorensen (1996). They recommended using moment conditions for GMM
estimation based on lower-order moments of yt , since higher-order moments
tend to exhibit erratic finite sample behavior. They considered a GMM estimation based on (subsets) of 24 moments considered by Jacquier, Polson,21.7 Examples of Nonlinear Models

835

and Rossi (1994). To describe these moment conditions, first define
μ=

σ 2u
ω
, σ2 =1−β
1 − β2

Then, the moment conditions, which follow from properties of the lognormal distribution and the Gaussian AR(1) model, are expressed asE[|yt |] = (2/π)1/2 E[σ t ]E[yt2 ] = E[σ 2t ]p
E[|yt3 |] = 2 2/πE[σ3t ]E[yt4 ] = 3E[σ 4t ]E[|yt yt−j |] = (2/π)E[σt σ t−j ], j = 1, . . . , 102
E[yt2 yt−j
] = E[σ 2t σ 2t−j ], j = 1, . . . , 10
where for any positive integer j and positive constants r and s,µ¶rμ r2 σ2r
E[σ t ] = exp
+
2
8
Ã
!
rsβ j σ 2r srsE[σ t σt−j ] = E[σ t ]E[σ t ] exp
4
Let
2
2
, . . . , yt2 yt−10)0wt = (|yt |, yt2 , |yt3 |, yt4 , |yt yt−1 |, . . . , |yt yt−10 |, yt2 yt−1

and define the 24 × 1 vector
³
´
⎞
⎛
2
|yt | − (2/π)1/2 exp μ2 + σ8
³
´
⎟
⎜
⎟
⎜
σ2
2
y
−
exp
μ
+
⎟
⎜
t
2
⎟
⎜
g(wt , θ) = ⎜
..
⎟
⎟
⎜
.
⎠
⎝
³
´
2
¡
¢
2
2
− exp μ + σ2
exp β 10 σ 2
yt2 yt−10(21.44)

Then, E[g(wt , θ 0 )] = 0 is the population moment condition used for the
GMM estimation of the model parameters θ 0 . Since the elements of wt
are serially correlated, the eﬃcient weight matrix S = avar(ḡ) must be
estimated using an HAC estimator.
An S-PLUS function to compute the moment conditions (21.44) for t =1, . . . , n is
sv.moments = function(parm, data = NULL){
omega = parm[1]836

21. Generalized Method of Moments

beta = parm[2]sigu = parm[3]mu = omega/(1-beta)
sig2 = (sigu*sigu)/(1-beta*beta)#
E.sigma = c(sqrt(2/pi) * exp(mu/2 + sig2/8),
exp(mu + sig2/2),
2 * sqrt(2/pi) * exp(3*mu/2 + 9*sig2/8),
3 * exp(2*mu + 2*sig2))
E.sigma.c = c(2/pi * exp(2*(mu/2 + sig2/8)+ beta^(1:10) * sig2/4),
exp(2*(mu + sig2/2)+ 4 * beta^(1:10) * sig2/4))#t(t(data) - c(E.sigma, E.sigma.c))}
The transformed simulated data to be passed to the function sv.moments
for GMM estimation are created using> sv.pow = cbind(abs(sv.as.sim), sv.as.sim^2,+
abs(sv.as.sim)^3, sv.as.sim^4)> sv.pow = sv.pow[-(1:10),]> sv.cm = tslag(sv.as.sim, 1:10, trim=T) *+as.vector(sv.as.sim[-(1:10)])> sv.data = cbind(sv.pow, abs(sv.cm), sv.cm^2)
The iterated eﬃcient GMM estimator based on (21.44) may be computedusing> start.vals = c(0,0.5,0.5)> names(start.vals) = c("omega","beta","sigu")> sv.fit.1 = GMM(start.vals, sv.moments, method = "iterative",+ts = T, data = sv.data)> summary(sv.fit.1)
Call:
GMM(start = start.vals, moments = sv.moments, method ="iterative", ts = T, data = sv.data)
Coefficients:
Value Std.Error
omega -0.4628
0.3731beta
0.93780.0502
sigu
0.22330.0978t value Pr(>|t|)-1.2405
0.2151
18.69590.00002.2841
0.0226

Test of Overidentification:

21.7 Examples of Nonlinear Models

837

J-stat Df P.value
18.8761 21 0.5931
Optimization Info:
Number of Iterative Steps: 7
The high p-value for the J-statistic indicates that the 24 moment conditions
are not rejected by the data. Consistent with the findings of Andersen andSorensen, ω 0 is not estimated very precisely, whereas β 0 is estimated fairly
precisely.
Now, consider estimation of the SV model (21.43) using the daily returns
on the S&P 500 index over the period March 14, 1986 through June 30,
2003:> SP500.ts = sp500.ts> SP500.ts@data = as.matrix(seriesData(sp500.ts))>> SP500.pow = seriesMerge(abs(SP500.ts), SP500.ts^2,+
abs(SP500.ts)^3, SP500.ts^4)> SP500.pow = SP500.pow[-(1:10),]@data> SP500.cm = tslag(SP500.ts, 1:10, trim=T)@data *+as.vector(SP500.ts[-(1:10)]@data)> SP500.data = cbind(SP500.pow, abs(SP500.cm), SP500.cm^2)> colIds(SP500.data) = NULL>
> # iterative GMM estimation of the SV model> start.vals = c(0,0.5,0.5)> names(start.vals) = c("omega","beta","sigu")> sv.fit.SP500 = GMM(start.vals, sv.moments,+
method = "iterative", ts = T,+data = SP500.data)
1-step objective = 2.10165e-7...
9-step objective = 39.9342> summary(sv.fit.SP500,print.moments=F)
Call:
GMM(start = start.vals, moments = sv.moments, method ="iterative", ts = T, data = SP500.data)
Coefficients:
Value Std.Error
omega -0.1541
0.1758beta
0.98380.0184sigu
0.15480.0908t value Pr(>|t|)-0.8767
0.380753.36920.00001.70410.0884

838

21. Generalized Method of Moments

Test of Overidentification:
J-stat Df P.value
39.9342 21 0.0076

Optimization Info:
Number of Iterative Steps: 9
The low p-value on the J-statistic indicates that the SV model (21.43) does
not fit S&P 500 daily returns.

21.7.5 Interest Rate Diﬀusion Model
Chan, Karolyi, Longstaﬀ, and Sanders (1992), hereafter CKLS, consideredestimating the parameters of the continuous-time interest rate diﬀusion
model
drt
θ0

γ

= (α0 + β 0 rt ) dt + σ 0 rt 0 dWt= (α0 , β 0 , σ 0 , γ 0 )0(21.45)using GMM. In (23.26), the drift function (α0 +β 0 rt )dt may be reparameterized as κ0 (μ0 − rt )dt, where α0 = κ0 μ0 and β 0 = −κ0 . The parameterμ0 is the long-run mean, and the parameter κ0 determines the speed ofmean reversion. The model (23.26) encompasses a variety of models thathave been proposed for the short-term interest rate. Simulated solutions
from (23.26) based on Euler’s method for various models using parameters
estimated by CKLS are presented in Chapter 19.
CKLS derived moment conditions for GMM estimation of θ 0 from theEuler discretization√γ
rt+∆t − rt = (α0 + β 0 rt )∆t + σ 0 rt 0 ∆tzt+∆t2
E[zt+∆t ] = 0, E[zt+∆t]=1
They defined the true model error asεt+∆t= a(rt+∆t − rt , rt ; α0 , β 0 , σ 0 , γ 0 ) = (rt+∆t − rt ) − (α0 + β 0 rt )∆t= a(zt+∆t , θ 0 )where zt+∆t = (rt+∆t − rt , rt )0 . Letting It represent information available
at time t, the true error satisfies E[εt+∆t |It ] = 0. Since {εt+∆t , It+∆ } isa MDS, potential instruments xt include current and lagged values of theelements of zt as well as a constant. As the basis for the GMM estimation,
CKLS used xt = (1, rt )0 as the instrument vector and deduced the following
four conditional moments:2γE[εt+∆t |It ] = 0, E[ε2t+∆t |It ] = σ 20 rt 0 ∆t2γE[εt+∆t rt |It ] = 0, E[ε2t+∆t rt |It ] = σ 20 rt 0 ∆t · rt

21.7 Examples of Nonlinear Models

839

For given values of α and β, define the nonlinear residualet+∆t = (rt+∆t − rt ) − (α + βrt )∆tand for wt+∆t = (rt+∆t − rt , rt , rt2 )0 , define the 4 × 1 vector of moments
⎞
⎛
et+∆t
µ
¶
⎟
⎜
et+∆t rt
et+∆t
⎟
⎜
2
2 2γ
g(wt+∆t , θ) =⊗ xt = ⎜
⎟
2
e
−
σ
r
∆t
t
et+∆t
t+∆t
⎝ ³
´ ⎠
2γ
e2t+∆t − σ 2 rt ∆t rt

Then, E[g(wt+∆t , θ0 )] = 0 gives the GMM estimating equation. Even
though {εt , It } is a MDS, the moment vector g(wt , θ0 ) is likely to be autocorrelated since it contains ε2t . For the most general specification (23.26),
K = L and the model is just identified. For all submodels of (23.26) listed
in Table 20.3 from Chapter 19, K > L, so that they are overidentified.
An S-PLUS function to compute the CKLS moments is
ckls.moments <- function(parm, data = NULL,dt = 1/12) {# parm = (alpha,beta,sigma,gamma)’# data = [r(t+dt)-r(t),r(t)]# dt = discretization stepe.hat = as.vector(data[,1] (parm[1] + parm[2]*data[,2])*dt)
m2 = e.hat*as.vector(data[,2])m3 = e.hat^2 - dt*parm[3]*parm[3]*(as.vector(data[,2])^(2*parm[4]))m4 = m3*data[,2]cbind(e.hat,m2,m3,m4)}
The function ckls.moments has arguments parm, specifying the model parameters θ = (α, β, σ, γ)0 , data, specifying an n × 2 matrix with tth row(rt+∆t − rt , rt ), and dt, specifying the discretization increment.
The GMM estimation is performed on (23.26) using monthly observations on the U.S. 30-day T-bill rate over the period June 1964 throughNovember 1989. The same data are analyzed in Cliﬀ (2003), and are illustrated in Figure 21.6.
The data to be passed to the function ckls.moments are constructedusing> data.ckls.ts = seriesMerge(diff(ckls.ts), tslag(ckls.ts))> colIds(data.ckls.ts)[1] = "RF.diff"> data.ckls = as.matrix(seriesData(data.ckls.ts))
The summary statistics for ∆rt and rt−1

21. Generalized Method of Moments

0.04

0.08

0.12

0.16

840

1970

1975

1980

1985

1990

1965

1970

1975

1980

1985

1990

-0.05

-0.01

0.03

1965

FIGURE 21.6. Monthly observations on the U.S. 30-day T-bill rate and its firstdiﬀerence.

> summaryStats(data.ckls)
Sample Quantiles:min
1Q median
3Q
max
RF.diff -0.05813 -0.00267 0.0002 0.00324 0.03285
RF.lag1 0.03127 0.04756 0.0603 0.07883 0.16150
Sample Moments:
mean
std skewness kurtosis
RF.diff 0.00012 0.008187-1.34713.354RF.lag1 0.06659 0.026375
1.206
4.332
Number of Observations:305

are very close to those presented in Table II of CKLS (1992).
First, consider GMM estimation of the most general model (23.26). Sensible starting values for α and σ may be determined using the following
approximations:
μ =
σ2

≈

α
≈ E[rt ]−βvar(rt+∆t − rt )var(rt )∆t21.7 Examples of Nonlinear Models

841

The GMM estimation of θ may be performed using> start.vals = c(0.06,-0.5,1,1)> names(start.vals) = c("alpha","beta","sigma","gamma")> gmm.ckls = GMM(start.vals,ckls.moments,ts = T,+data = data.ckls,dt = 1/12)> summary(gmm.ckls)
Call:
GMM(start = start.vals, moments = ckls.moments, ts = T, data
= data.ckls, dt = 1/12)
Coefficients:
Value Std.Error t value Pr(>|t|)alpha 0.0419 0.0193
2.1710 0.0307
beta -0.6077 0.3454-1.7592 0.0796
sigma 1.3337 1.0004
1.3331 0.1835
gamma 1.5081 0.2900
5.2010 0.0000
Test of Overidentification:model is just-identified
Optimization Info:
Number of Iterations: 14
Convergence: absolute function convergence
In the call to GMM, the parameter dt is set to 1/12 because the data areannual rates sampled monthly. Since the model is just identified, the Jstatistic is zero and the estimates do not depend on a weight matrix. The
results are similar to those obtained by CKLS. In particular, the estimateof γ is roughly 1.5 and is highly significant. The estimates of the long-run
mean and speed of adjustment are> theta.hat = coef(gmm.ckls)> theta.hat["alpha"]/-theta.hat["beta"]
alpha0.06895576> -theta.hat["beta"]
beta
0.6076583
The GMM estimates for the restricted models in Table 3 of Chapter 20
may be computed in a similar fashion by modifying the function ckls.
moments. For example, to estimate the CIR SR model, use
cir.moments <- function(parm, data = NULL, dt = 1/12) {# parm = (alpha,beta,sigma)’

842

21. Generalized Method of Moments# data = [r(t+dt)-r(t),r(t)]# dt = discretization stepe.hat = as.vector(data[,1] (parm[1] + parm[2]*data[,2])*dt)
m2 = e.hat*as.vector(data[,2])m3 = e.hat^2 - dt*parm[3]*parm[3]*(as.vector(data[,2]))m4 = m3*data[,2]cbind(e.hat,m2,m3,m4)}> start.vals = c(0.06,-0.5,1)> names(start.vals) = c("alpha","beta","sigma")> gmm.cir = GMM(start.vals, cir.moments, ts = T,+data = data.ckls, dt = 1/12)> summary(gmm.cir,print.moments = F)
Call:
GMM(start = start.vals, moments = cir.moments, ts = T, data =data.ckls, dt = 1/12)
Coefficients:
Value Std.Error
alpha0.02040.0166beta -0.2407
0.3023sigma
0.08410.0066t value Pr(>|t|)1.22580.2212-0.7963
0.426512.76610.0000

Test of Overidentification:
J-stat Df P.value
3.7977 1 0.0513
Optimization Info:
Number of Iterative Steps: 13
The low p-value on the J-statistic indicates a potentially misspecified model.

21.8 References
Andersen, T.G. and B.E. Sorensen (1996). “GMM Estimation of a
Stochastic Volatility Model: A Monte Carlo Study,” Journal of Businessand Economic Statistics, 14, 328-352.
Andrews, D.W. (1991). “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrica, 59, 817-858.

21.8 References

843

Andrews, D.W. and C.J. Monahan (1992). “An Improved Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator,” Econometrica, 60, 953-966.
Campbell, J.Y., A.W. Lo and A.C. MacKinlay (1997). The Econometrics of Financial Markets. Princeton University Press, Princeton, NJ.
Campbell, J. and G. Mankiw (1990). “Permanent Income, Current
Income and Consumption,” Journal of Business and Economic Statistics,
8, 265-279.
Chan, K.C., G.A. Karolyi, F.A. Longstaff and A.B. Sanders(1992). “An Empirical Comparison of Alternative Models of the Term
Structure of Interest Rates”, Journal of Finance, 47, 1209-1227.
Cliff, M.T. (2003). “GMM and MINZ Program Libraries for MATLAB,”
unpublished manuscript, Krannert Graduate School of Management, Purdue University.
Cochrane, J.H. (1996). “A Cross-Sectional Test of an Investment-Based
Asset Pricing Model,” Journal of Political Economy, 104, 572-561.
Cochrane, J.H. (2001). Asset Pricing. Princeton University Press, Princeton, NJ.
Davidson, R. and J. MacKinnon (2004). Econometric Theory and Methods. Oxford University Press, Oxford.
den Haan, W.J. and A. Levin (1997). “A Practioner’s Guide to Robust Covariance Matrix Estimation,” in G. Maddala and C. Rao (eds.),
Handbook of Statistics, Volume 15, Elsevier, Amsterdam, pp. 309-327.
Ferson, W.E. (1995). “Theory and Empirical Testing of Asset Pricing
Models,” in R.A. Jarrow, V. Maksimovic and W.T. Ziemba (eds.), Handbooks in OR & MS, Volume 9, Finance. Elsevier Science B.V., Amsterdam.
Greene, W.H. (1994). Econometric Analysis. Prentice Hall, Englewood
Cliﬀs, NJ.
Hall, R.E. (1978). “Stochastic Implications of the Life CyclePermanent Income Hypothesis: Theory and Evidence,” Journal of Political Economy, 86(6), 971-987.
Hall, A. (2005). Generalized Method of Moments. Oxford University Press,
Oxford.

844

21. Generalized Method of Moments

Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press,
Princeton, NJ.
Hansen, L.P. (1982). “Large Sample Properties of Generalized
Method of Moments Estimators,” Econometrica, 50, 1029-1054.
Hansen, L.P. and K.J. Singleton (1982). “Generalized Instrumental
Variables Estimation of Nonlinear Rational Expectations Models,” Econometrica, 50, 1269-1286.
Hansen, L.P., J. Heaton and A. Yaron (1996). “Finite-Sample Properties of Some Alternative GMM Estimators,” Journal of Business andEconomic Statistics, 14, 262-280.
Harris, D. (1999). “GMM Estimation of Time Series Models,” in L.
Mátyás (ed.), Generalized Method of Moments, Cambridge University Press,
Cambridge.
Hayashi, F. (2000). Econometrics. Princeton University Press, Princeton,
NJ.
Imbens, G.W. (2002). “Generalized Method of Moments and Empirical
Likelihood,” Journal of Business and Economic Statistics, 20, 493-506.
Jacquier, E., N.G. Polson and P.E. Rossi (1994). “Bayesian Analysis of Stochastic Volatility Models,” Journal of Business and Economic
Statistics, 12, 371-389.
Jagannathan, R. and G. Skoulakis(2002). “Generalized Method ofMoments: Applications in Finance,” Journal of Business and Economic
Statistics, 20, 470-482.
James, J. and N. Webber (2000). Interest Rate Models. John Wiley &Sons, Chichester, England.
Newey, W. (1985). “Generalized Method of Moments Specification Testing,” Journal of Econometrics, 29, 229-256.
Newey, W. and R. Smith (2004). “Higher Order Properties of GMM
and Generalized Empirical Likelihood,” Econometrica, 72, 219-255.
Newey, W. and K. West (1987). “Hypothesis Testing with Eﬃcient
Method of Moments,” International Economic Review, 28, 777-787.
Newey, W. and K. West (1994). “Automatic Lag Selection for Covariance Matrix Estimation,” Review of Economic Studies, 61, 631-653.

21.8 References

845

Ogaki, M. (1992). “Generalized Method of Moments: Econometric Applications”, in G. Maddala, C. Rao and H. Vinod (eds.), Handbook of Statistics, Volume 11: Econometrics, North-Holland, Amsterdam.
Ruud, P. A. (2000). An Introduction to Classical Econometric Theory.
Oxford University Press, Oxford.
Sargan, D. (1958). “The Estimation of Economic Relationships Using
Instrumental Variables,” Econometrica, 26, 393-415.
Shephard, N. G. (1996). “Statistical Aspects of ARCH and Stochastic
Volatility,” in D.R. Cox, D.V. Hinkley and O.E. Barndorﬀ-Nielsen (eds.),
Time Series Models in Econometrics, Finance and Other
Fields. Chapman & Hall, London.
Stock, J.H. and J.H. Wright (2000). “GMM with Weak Identification,”
Econometrica, 68, 1055-96.
Stock, J.H., J.H. Wright and M. Yogo (2002). “A Survey of Weak
Instruments and Weak Identification in Generalized Method of Moments,”
Journal of Business and Economic Statistics, 20, 518-529.
Taylor, S.J. (1986). Modelling Financial Time Series. John Wiley &Sons, Chichester, England.
Verbeek, M. (2000). A Guide to Modern Econometrics. John Wiley &Sons, Chichester, England.
White, H. (1980). “A Heteroskedasticity-Consistent Covariance Matrix
Estimator and a Direct Test for Heteroskedasticity,” Econometrica, 48,
817-838.
Wooldrige, J. (2002). Introduction to Econometrics, 2nd edition. SouthWestern, Cincinnati, OH.
Wright, J. (2003). “Detecting Lack of Identification in GMM,” Econometric Theory, 19, 322-330.

This is page 846
Printer: Opaque thisThis is page 847
Printer: Opaque this22
Seminonparametric Conditional
Density Models

22.1 Introduction
For a stationary multiple time series, the one-step-ahead conditional density represents the underlying stochastic process. The conditional density
incorporates all information about various characteristics of the time series,
including conditional heteroskedasticity, non-normality, time irreversibility,
and other forms of nonlinearity. These properties are now widely considered to be important features of many time series processes. Since theconditional density completely characterizes a stochastic process, it is thus
naturally viewed as the fundamental statistical object of interest.
SNP is a seminonparametric method, based on an expansion in Hermite
functions, for estimation of the one-step-ahead conditional density. The
method was first proposed by Gallant and Tauchen (1989) in connection
with an asset pricing application. It consists of applying classical parametric estimation and inference procedures to models derived from nonparametric truncated series expansions. Particularly, estimation of SNP models entails using a standard maximum likelihood procedure together witha model selection strategy that determines the appropriate degree of thenonparametric expansion. The general form of the SNP model is a nonlinear, non-Gaussian multivariate generalized autoregressive conditional heteroskedasticity (GARCH) model. Under reasonable regularity conditions,the estimator is consistent and eﬃcient for the unknown one-step-ahead
conditional density.

848

22. Seminonparametric Conditional Density Models

As a pure time series model, the SNP model extends the standard vector autoregressive (VAR) model described in Chapter 11 to allow for nonGaussian multivariate GARCH-type error structures. As a result, they provide a semiparametric alternative to the multivariate GARCH models described in Chapter 13. The SNP models can also be used as an alternative
to the copula methodology discussed in Chapter 19 for the general modeling of multivariate distributions. An important use of SNP models is toact as a score generator, or auxiliary model, for eﬃcient method of moments (EMM) estimation. The EMM estimation of general discrete-timeand continuous-time dynamic time series models based on SNP auxiliarymodels is described in the next chapter.
This chapter provides a general overview of the SNP methodology, andintroduces a comprehensive set of S-PLUS functions for the estimation andanalysis of SNP models that are based on the FORTRAN code of Gallant and Tauchen (2001). The exposition of the SNP methodology borrows
heavily from Gallant and Tauchen (2001, 2001b, and 2002). The outline of
the chapter is as follows. Section 22.2 gives a general overview of the SNP
methodology and the S-PLUS functions for fitting SNP models. Section 22.3
illustrates the use of the S+FinMetrics function SNP for estimating a variety of univariate and bivariate SNP models. Section 22.4 describes theestimation of SNP models using model selection criteria. Section 22.5 discusses various types of diagnostic analysis of SNP models, including residual diagnostics and simulation diagnostics. Prediction from SNP models
is covered in Section 22.6, and useful data transformations are covered in
Section 22.7. Finally, Section 22.8 presents several in-depth examples offitting SNP models to financial time series.

22.2 Overview of SNP Methodology
Consider a stationary and ergodic multivariate time series {yt }Tt=−L+1 ,where each yt is a vector of length M . Assume that yt is Markovian (i.e.,the conditional distribution of yt given the entire past depends only on a
finite number L of lagged values of yt )1 . Denote these lagged values by thestate vector00xt−1 = (yt−1, . . . , yt−L)0which is a vector of length M ·L. For simplicity, the time subscripts are often
suppressed and y and x are used to denote the contemporaneous value andlagged state vector, respectively. With these conventions, the stationary
density of the dynamic system under consideration can be written p(x, y|ρ)
1 This section is taken from Gallant and Tauchen (2001b) by permission of the authors.

22.2 Overview of SNP Methodology

849

and its transition density asp(y|x, ρ) = R

p(x, y|ρ)p(x, y|ρ) dx(22.1)p
If one expands p(x, y | ρo ) in a Hermite series and derives the transition
density of the truncated expansion, then one obtains a transition densityfK (yt | xt−1 , θ) that has the form of a location-scale transform
yt = Rzt + μxt−1
of an innovation zt (Gallant, Hsieh, and Tauchen, 1991). The density function of this innovation ishK (z|x) = R[P(z, x)]2 φ(z)[P(u, x)]2 φ(u) du(22.2)

where P(z, x) is a polynomial in (z, x) of degree K and φ(z) denotes themultivariate normal density function with dimension M , zero mean vector
zero, and identity variance-covariance matrix.
It proves convenient to express the polynomial P(z, x) in a rectangularexpansion
⎛
⎞
Kz
Kx
X
X
⎝
P(z, x) =aβ α xβ ⎠ zα(22.3)
α =0

β =0where α and β are multi-indexes of maximalR degrees Kz and Kx , respectively, and K = Kz +Kx . Because [P(z, x)]2 / [P(u, x)]2 φ(u)du is a homogeneous function of the coeﬃcients of the polynomial P(z, x), P(z, x) canonly be determined to within a scalar multiple. To achieve a unique representation, the constant term a00 of the polynomial P(z, x) is put to one.
With this normalization, hK (z|x) has the interpretation of a series expansion whose leading term is the normal density φ(z) and whose higher-orderterms induce departures from normality.
The location function is linear
μx = b0 + Bxt−1(22.4)

where b0 is a vector and B is a matrix.
It proves advantageous in applications to allow the scale R of the locationscale transformation y = Rz + μx to depend on x because it reduces thedegree Kx required to achieve an adequate approximation to the transition
density p(y|x, ρo ). With this, the location-scale transformation becomesy = Rx z + μx

(22.5)

where Rx is an upper triangular matrix that depends on x. The two choicesof Rx that have given good results in applications are an autoregressive

850

22. Seminonparametric Conditional Density Models

conditional heteroskedasticity (ARCH)-like moving average specification
and a GARCH-like autoregressive-moving average (ARMA) specification.
For an ARCH specification, let Rxt−1 be a linear function of the absolutevalues of the elements of the vectors yt−Lr −μxt−1−Lr through yt−1 −μxt−2 ;namely,vech(Rxt−1 ) = ρ0 +

Lr
X
i=1

P(i) |yt−1−Lr +i − μxt−2−Lr+i |where vech(R) denotes a vector of length M (M + 1)/2 containing the elements of the upper triangle of R, ρ0 is a vector of length M (M + 1)/2, P(1)through P(Lr) are M (M + 1)/2 × M matrices, and |y − μ| denotes a vector
containing the absolute values of y − μ. The classical ARCH (Engle, 1982)hasΣxt−1 = Rxt−1 R0xt−1
depending on a linear function of squared lagged residuals. The SNP versionof ARCH is more akin to the suggestions of Taylor (1986) and Davidian
and Carroll (1987).
Since the absolute value function is not diﬀerentiable, |u| is approximated in the above formula for Rx by the twice continuously diﬀerentiable
function½(|100u| − π/2 + 1) /100,|100u| ≥ π/2a(u) =(1 − cos(100u)) /100,|100u| < π/2
The above scale factor 100 represents a compromise. Small values, such
as 3, improve the stability of the computations but then a(·) does not
approximate | · | well.
For a GARCH specification, letvech(Rxt−1 ) = ρ0 +

Lr
X
i=1

+

Lg
X

P(i) |yt−1−Lr +i − μxt−2−Lr+i |(22.6)diag(G(i) )Rxt−2−Lg +i

i=1

where G(1) through G(Lg ) are vectors of length M (M + 1)/2.
The classical GARCH (Bollerslev, 1986) has Σxt−1 expressed in termsof squared lagged residuals and lagged values of Σxt−1 . As with the SNPvariant of ARCH, the SNP version of GARCH is expressed in terms of theabsolute value of lagged residuals and standard deviations.
Note that when Lg > 0, the SNP model is not Markovian and that one
must know both xt−1 and Rxt−2−Lg through Rxt−2 to move forward to
the value for yt . Thus, xt−1 and Rxt−2−Lg through Rxt−2 represent thestate of the system at time t − 1 and must be retained in order to evaluate

22.3 Estimating SNP Models in S+FinMetrics

851

the SNP conditional density of yt or to iterate the SNP model forward bysimulation. If one wants to compute the derivatives of the SNP density withrespect to model parameters, one must retain the derivatives of Rxt−2−Lgthrough Rxt−2 with respect to model parameters as well.
The change of variable formula applied to the location-scale transform(22.5) and innovation density (22.2) yields the SNP density¤£
hK R−1x (y − μx ) | x
fK (y | x, θ) =(22.7)
det(Rx )Hereafter, the lag lengths appearing in the various components of the expansion will be distinguished. The number of lags in μx is denoted Lu ,the number of lags in Rx is Lu + Lr , and the number of lags in the x
part of the polynomial, P(z, x), is Lp . The maximum lag is defined as
L = max(Lu , Lu + Lr , Lp ).
Large values of M can generate a large number of interactions (crossproduct terms) for even modest settings of degree Kz , and similarly for
M · Lp and Kx . Accordingly, two additional tuning parameters, Iz and Ix ,are introduced to represent filtering out of these high-order interactions.
Iz = 0 means that no interactions are suppressed, Iz = 1 means thatthe highest-order interactions are suppressed, namely those of degree Kz .
In general, a positive Iz means that all interactions of order larger thanKz − Iz are suppressed; similarly for Kx − Ix .
In summary, Lu , Lg , and Lr determine the location-scale transformationy = Rx zt + μx and, hence, determine the nature of the leading term of the
expansion. The number of lags in the location function μx is Lu and thenumber of lags in the scale function Rx is Lu + Lr . The number of lagsthat go into the x part of the polynomial P(z, x) is Lp . The parameters
Kz , Kx , Iz , and Ix determine the degree of P(z, x) and, hence, the natureof the innovation process {zt }.

22.3 Estimating SNP Models in S+FinMetrics
S+FinMetrics contains a variety of functions for estimation and analysisof SNP models. Table 22.1 gives a brief summary of these functions. The
main functions are based on the public domain FORTRAN code for theestimation of SNP models described in Gallant and Tauchen (2001).
The function SNP is used to estimate univariate and multivariate SNPmodels. The arguments expected by SNP are> args(SNP)
function(data, model = SNP.model(), n.drop = NULL, control =SNP.control(), coef = NULL, est = NULL, iAr = 0, iArch= 0, iGarch = 0, iLagP = 0, iZPoly = 0, iXPoly = 0,852

22. Seminonparametric Conditional Density Models

Function
SNP
SNP.model
SNP.control
expandSNP.auto
SNP.density

Description
Fit SNP model using quasi-maximum likelihood
estimation
Create SNP model list
Create list of control parameters for SNP model andestimation
Expand fitted SNP model to larger model
Automatically find best fitting SNP model byminimizing Bayesian information criterion (BIC)
Plot SNP conditional transition density

TABLE 22.1. S+FinMetrics functions for analysis of SNP models

iTrimZ = 0, iTrimX = 0, LStransform = NULL, save.data= F, trace = F)
The main arguments are data, which may be a univariate or multivariate
rectangular data object, model, which is a list describing the SNP model
to be fit, and control, which is a list containing various parameters that
control aspects of the SNP model and optimization algorithm. The otherarguments will be explained in the examples below.
The function SNP.model creates a list containing components that specify
the type of SNP model to be fit. The arguments of SNP.model are> args(SNP.model)
function(ar = 0, arch = 0, garch = 0, lagP = 1, zPoly = 0,
xPoly = 0, trimZ = 0, trimX = 0)
The form of a fitted SNP model is determined by a number of tuning parameters. These parameters and the corresponding arguments to the function
SNP.model are summarized in Table 22.2. When describing a particular
SNP model, it is convenient to refer to the model as a string of tuning
parameters: Lu Lg Lr Lp Kz Iz Kx Ix . For example, a semiparametric VAR(1)with Kz = 4 is denoted 10014000 and a nonlinear nonparametric VAR(3)GARCH(1,1) model with Kz = 4, Kx = 2 excluding all interactions andallowing two lags of xt−1 in P(zt , xt−1 ) is denoted 31124321.
Table 22.3 provides a useful taxonomy of SNP models, defined by putting
certain restrictions on the tuning parameters. The print and summary
methods of "SNP" objects reflect this taxonomy.
The function SNP.control creates a list with parameters that control restrictions on the SNP model and set options for the optimization algorithm.
The default arguments to SNP.control are> args(SNP.control)
function(initial.itmax = 15, final.itmax = 385, tol = 1e-005,
xTransform = "none", inflection = NULL, diagS = T, darch = T,
rgarch = F, dvec = T, n.start = 0, seed = 72674,

22.3 Estimating SNP Models in S+FinMetrics

Parameter
Lu
Lg
Lr
Lp
Kz
Iz
Kx
Ix

SNP.model
argument
ar
garch
arch
lagP
zPoly
trimZ
xPoly
trimX

853

Description
VAR lag length
GARCH lag length
ARCH lag length
Lags of xt−1 in P(zt , xt−1 )Order of zt in P(zt , xt−1 )Index of zt interactions in P(zt , xt−1 )Order of xt−1 in P(zt , xt−1 )Index of xt−1 interactions in P(zt , xt−1 )TABLE 22.2. SNP tunning parameters

Parameter Setting
Lu = 0, Lg = 0, Lr = 0, Lp ≥ 0, Kz = 0, Kx = 0
Lu > 0, Lg = 0, Lr = 0, Lp ≥ 0, Kz = 0, Kx = 0
Lu > 0, Lg = 0, Lr = 0, Lp ≥ 0, Kz > 0, Kx = 0
Lu ≥ 0, Lg = 0, Lr > 0, Lp ≥ 0, Kz = 0, Kx = 0
Lu ≥ 0, Lg = 0, Lr > 0, Lp ≥ 0, Kz > 0, Kx = 0
Lu ≥ 0, Lg > 0, Lr > 0, Lp ≥ 0, Kz = 0, Kx = 0
Lu ≥ 0, Lg > 0, Lr > 0, Lp ≥ 0, Kz > 0, Kx = 0
Lu ≥ 0, Lg ≥ 0, Lr ≥ 0, Lp ≥ 0, Kz > 0, Kx > 0

Characterization of{yt }iid Gaussian
Gaussian VAR
Semiparametric VAR
Gaussian ARCH
Semiparametric ARCH
Gaussian GARCH
Semiparametric
GARCH
Nonlinear
nonparametric

TABLE 22.3. Taxonomy of SNP Models

fOld = 0, fNew = 0, eps0 = 1e-005, version = 8.9)
These control parameters are explained in the examples in the following
subsections.
The following subsections describe in detail the SNP methodology for describing the conditional transition density of a stationary and ergodic timeseries, and the use of the S+FinMetrics function SNP for estimating the
SNP density. The analysis starts with a simple Gaussian VAR model. Thismodel is then generalized to a semiparametric VAR model. Finally, conditional heterogeneity is introduced into the SNP model. In each subsection,
examples of using the SNP function to estimate univariate and bivariate
SNP models are given.

22.3.1 Example Data
The data for the following examples are in the "timeSeries" object
dmRet.dat, which contains weekly observations on three variables: the percentage changes in the German mark and U.S. dollar (DM/US) exchange

854

22. Seminonparametric Conditional Density Models

-6

-4

-2

0

2

4

6

8

US/DM spot exchange rate

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1985

1986

1987

1988

1989

1990

1991

0.0

0.2

0.4

0.6

0.8

1.0

US/DM forward discount

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

FIGURE 22.1. Weekly observations on US/DM spot exchange rate and 30-day
forward discount.

rate; the diﬀerence between the 30-day forward rate and the spot exchange
rate; and the U.S. 30-day Treasury bill rate. The data cover the period
January 10, 1975 to December 28, 1990. This data set has been analyzed
by Bansal, Gallant, Hussey, and Tauchen (1994), and Gallant and Tauchen(2001) used it to illustrate their FORTRAN code for fitting SNP models.2
The three variables in dmRet.dat are> colIds(dmRet.dat)[1] "exch""forward" "tbill"which give the spot returns, the forward discounts, and the U.S. Treasury
bill rates, respectively. Figure 22.1 shows the log returns on the spot exchange rate and the forward discount, which will be used in the examples.
The sample autocorrelation and cross-autocorrelation functions for the twoseries are given in Figure 22.2. The spot returns are nearly uncorrelated,
whereas the forward discounts are highly persistent. Some sample descriptive statistics, from the S+FinMetrics function summaryStats, are> summaryStats(dmRet.dat[,c("exch","forward")])

2 The
data
are
taken
from
the
ftp://ftp.econ.duke.edu/pub/get/data/file

dm fri.dat

availablefrom22.3 Estimating SNP Models in S+FinMetrics

855

Multivariate Series : dmRet.dat[, c("exch", "forward")]exch and forward

0.0

-0.10

0.2

-0.05

ACF
0.4

0.6

0.0

0.8

0.05

1.0

exch

0

5

10

15

20

25

0

5

10

20

25

20

25

forward

-0.15

0.0

-0.10

0.2

0.4

ACF
-0.05

0.6

0.0

0.8

0.05

1.0

forward and exch

15

-25

-20

-15

Lag

-10

-5

0

0

5

10

Lag

15

FIGURE 22.2. Sample autocorrelation function (SACF) for weekly spot returnand forward discount data.

Sample Quantiles:
min
1Q
median3Q
maxexch -7.4622 -0.6977 0.006332 0.8175 8.225
forward -0.1291 0.1619 0.279385 0.3955 1.187
Sample Moments:
mean
std skewness kurtosis
exch 0.05516 1.4964
0.3367
6.172
forward 0.29111 0.1911
0.8574
4.672
Number of Observations:834

The distribution of spot returns is nearly symmetric but has fatter tails
than the normal distribution. The distribution of the forward discount isslightly positively skewed and has fatter tails than the normal.

22.3.2 Markovian Time Series and the Gaussian Vector
Autoregression Model
As in Section 22.2, consider an M -dimensional stationary and ergodic
Markovian multivariate time series {yt }Tt=−L+1 with state vector xt−1 =
0
0
(yt−1
, . . . , yt−L
)0 . The likelihood function conditional on the first L obser-

856

22. Seminonparametric Conditional Density Models

vations can be written as
L=

T
Y

t=1

f (yt |xt−1 , θ)

(22.8)

where f (·) is the density function of yt conditional on xt−1 and θ consistsof the parameters of the density function.
A traditional approach to modeling the Markovian multiple time series yt
uses the Gaussian VAR model as described in Chapter 11. For a Gaussian
VAR model, yt is assumed to be of the form
yt = μ0 + Bxt−1 + Rzt ,= μxt−1 + Rzt(22.9)

where μ0 is an M × 1 vector, B is an M × M L matrix, μxt−1 = μ0 +Bxt−1 , R is an upper triangular matrix, and zt follows an M -dimensional
Gaussian distribution with zero mean and identity covariance matrix. Notethat since there are L lagged values in xt−1 , this represents an Lth order
VAR (VAR(L) model). The term μxt−1 is referred to as the conditionalmean equation.
By using the location-scale transformationzt = R−1 (yt − μ0 − Bxt−1 ) = R−1 (yt − μxt−1 )(22.10)

it is easy to show that for the Gaussian VAR model, the one-step conditional
density of yt is given by1
N (R−1 (yt − μ0 − Bxt−1 ); 0, IM )| det(R)|= N (yt ; μxt−1 , R · R0 )f (yt |xt−1 , θ) =(22.11)

where N (z; μ, Ω) denotes the density function of a multivariate Gaussian
random vector z with mean μ and covariance matrix Ω. Note that for
the Gaussian VAR model, the model parameter θ consists of μ0 , B, andR (since Ω = R · R0 ). Lütkepohl (1990) showed that maximum likelihood
estimates of the Gaussian VAR parameters may be obtained using multivariate regression.
Example 147 Gaussian VAR for univariate exchange rate data
Consider fitting a Gaussian VAR(1) model to the log returns on the
US/DM spot exchange rate, ∆st . For a univariate time series yt , a VAR(1)model reduces to a simple first-order autoregressive, AR(1), model. This isin the form (22.9) with xt−1 = yt−1 and
μxt−1R= μ0 + byt−1p= σ = var(yt |xt−1 )22.3 Estimating SNP Models in S+FinMetrics

857

To fit a Gaussian AR(1) model to the spot rate return, call the SNP function
using the "exch" column of dmRet.dat:> fit.10010000 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1),+
n.drop=14)
The model specification is accomplished by calling the function SNP.modeland passing the result as a model argument to SNP. The ar=1 argument
to SNP.model determines the order of the VAR model, and the optional
argument n.drop=14 specifies setting aside 14 observations as presample
observations and starting the estimation from the 15th observation.
The returned object fit.10010000 is of class "SNP":> class(fit.10010000)[1] "SNP"
Typing the name of an object at the command line automatically invokes
the print method for the corresponding class and prints out the function
call that generated the object, the estimated model parameters, along withsome commonly used information criteria3 :> fit.10010000
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1),n.drop = 14)
Model: Gaussian VAR
Conditional Mean Coefficients:
mu ar(1)0.0005 0.0222
Conditional Variance Coefficients:
sigma
1.0025
Information Criteria:
BIC
HQ
AIC
logL
1.4337 1.4284 1.4251 -1165.6
Convergence Type:both x and relative function convergence
3 If the default settings are used to invoke the SNP function, the SNP estimates ofa Gaussian VAR model are not the same as traditional VAR estimates because of alocation-scale transform employed on the data.

858

22. Seminonparametric Conditional Density Models

number of iterations: 2
The conditional mean coeﬃcients are the estimates of the coeﬃcients in
μxt−1 , which include μ̂0 = 0.005 and b̂ = 0.0222. The conditional variancecoeﬃcients are the estimates of the coeﬃcients in R, which in this case isjust σ̂ = 1.0025.
To obtain standard errors of the estimated parameters, use the genericsummary function, which automatically invokes the summary method for thecorresponding class:> summary(fit.10010000)
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1),n.drop = 14)
Model: Gaussian VAR
Conditional Mean Coefficients:
mu ar(1)coef 0.0005 0.0222(std.err) 0.0356 0.0270(t.stat) 0.0143 0.8245

Conditional Variance Coefficients:
sigma
coef 1.0025(std.err) 0.0158(t.stat) 63.2994

Information Criteria:
BIC
HQ
AIC
logL
1.4337 1.4284 1.4251 -1165.6
Convergence Type:both x and relative function convergence
number of iterations: 2
Given the sample ACF of the spot rate return, it is not surprising that the
autoregressive coeﬃcient b̂ is not significantly diﬀerent from zero.
Example 148 Gaussian VAR for bivariate exchange rate data
Now consider fitting a bivariate Gaussian VAR(1) model to the spot rate
return and the forward discount. Here, yt = (∆st , f dt )0 and the VAR model22.3 Estimating SNP Models in S+FinMetrics

859(22.9) has xt−1 = yt−1 = (∆st−1 , f dt−1 )0 with= μ0 + Byt−1
µ
¶ µ
¶µ
¶
μ0,1
b11 b12
∆st−1
=
+
μ0,2
b21 b22f dt−1¶µr11 r21, RR0 = Ω = var(yt |xt−1 )
R =
0 r22

μxt−1

To fit the bivariate VAR(1) to the exchange rate data, use4
> bfit.10010000 = SNP(dmRet.dat[,1:2],model=SNP.model(ar=1),+
n.drop=14)> bfit.10010000
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1),n.drop = 14)
Model: Gaussian VAR
Conditional Mean Coefficients:
exch forward
mu 0.0025 -0.0016
var(1;1) 0.0087 -0.1065
var(1;2) -0.0201 0.9671
Conditional Variance Coefficients:
sigma11 sigma12 sigma22
0.992 -0.0994 0.2541
Information Criteria:
BIC
HQ
AIC
logL
1.4965 1.4806 1.4706 -1196.927
Convergence Type:both x and relative function convergence
number of iterations: 2
The first row of the conditional mean coeﬃcients are the intercept estimates
μ̂0 . The second row, labeled var(1;1), gives the estimates on the lag4 To match the output of the S+FinMetrics function VAR, the location-scale transformation must be turned oﬀ. To do this, in the call to SNP set the optional argument
LStransform=list(mu=c(0,0),cstat=diag(2),pstat=diag(2)). Note: the arrangementof the VAR coeﬃcients printed from an "SNP" object is diﬀerent than the arrangementof the coeﬃcients printed from a "VAR" object.

860

22. Seminonparametric Conditional Density Models

1 variables for the spot return equation (first row of B̂). The third row,labeled var(1;2), gives the estimates on the lag-1 variables for the forward
discount equation (second row of B̂). The estimated equations are then
∆st
f dt

= 0.0025 + 0.0087∆st−1 − 0.1065f dt−1
= −0.0016 − 0.0201∆st−1 + 0.9671f dt−1

The conditional variance coeﬃcients give the estimates of the elements of
the upper triangular matrix R:µ¶
0.992 −0.0994
R̂ =0
0.254122.3.3 Hermite Expansion and the Semiparametric VAR
In practice, especially for economic and financial time series modeling, thestandardized residuals zt are rarely Gaussian. To model diﬀerent characteristics usually observed in empirical data, like fat tails or asymmetry,
one needs a good general-purpose density f (yt |xt−1 , θ) that can accommodate any shape, especially the Gaussian as it is most likely a priori.
The SNP model proposed by Gallant and Tauchen (1989) provides such ageneral-purpose approximation.
The SNP model is based on an expansion in Hermite functions. Usinga Hermite expansion as a general approximation to a density function has
a long established tradition in statistics; examples are Gram-Charlier and
Edgeworth expansions. The form of a Hermite expansion is a multivariate
polynomial in the standardized residuals zt multiplied by the standard
Gaussian density; that is,f (yt |xt−1 , θ) ∝ [P(zt )]2 N (yt ; μxt−1 , Ω)

(22.12)where Ω = R · R0 and P(zt ) denotes a multivariate polynomial of degreeKz .5 The polynomial P(zt ) has the form
P(zt ) =

Kz
X

aα zα
t

|α|=0

with α denoting the non-negative multi-index (α1 , . . . , αM ), and|α| = α1 + · · · + αM ,α1zα
· · · · · (zt,M )αM
t = (zt,1 )5 The constant of proportionality in (22.12) is 1/

R[P(s)]2 φ(s)ds, where φ(s) denotes
the density of a multivariate normal random variable with zero mean and identity covariance matrix.

22.3 Estimating SNP Models in S+FinMetrics

861

For example, let Kz = 2 and M = 2. Then,6
z
|α|
|α|
|α|

=
=
=
=

(z1 , z2 )0 , α = (α1 , α2 )0 , aα = a(α1 ,α2 )0 ⇒ α = (0, 0)0
1 ⇒ α = (1, 0)0 , (0, 1)02 ⇒ α = (1, 1)0 , (2, 0)0 , (0, 2)0

andP(z) = 1 + a(1,0) z1 + a(0,1) z2+a(1,1) z1 z2 + a(2,0) z12 + a(0,2) z22

The order Kz of the Hermite polynomial expansion acts like a smoothness parameter in nonparametric analysis. When Kz is set to zero, the
SNP density f (yt |xt−1 , θ) reduces to the Gaussian VAR density functionin (22.11). As a result, the SNP density in (22.12) nests the Gaussian VAR
as a special case. In general, the SNP density (22.12) is referred to as
the semiparametric VAR. When Kz is positive, the semiparametric VAR
density can accommodate arbitrary shape departures from the Gaussian
VAR as long as Kz is large enough. The shape modifications achieved by
the semiparametric VAR are rich enough to accurately approximate densities from a large class that includes densities with fat, t-like tails, densities
with tails that are thinner than Gaussian, skewed densities, and even multimodal densities. See Gallant and Tauchen (1999) for details.
The parameter vector θ of the semiparametric VAR density consists ofthe coeﬃcients aα of the polynomial P(z), plus μ0 , B, and R. They canbe estimated using a quasi-maximum likelihood procedure, by minimizingTsn (θ) = −1X
log f (yt |xt−1 , θ)n t=1

(22.13)using nonlinear regression techniques. Under regularity conditions, Gallant
and Nychka (1987) showed that if the number of parameters in θ grows
with the sample size, the true density and various features of it, such asderivatives and moments, are estimated consistently. Fenton and Gallant(1996a, 1996b) proved the eﬃciency of SNP estimates and other properties.
Example 149 Semi-parametric VAR for univariate exchange rate data
Fitting a semiparametric VAR model using the SNP function requires
specifying the order of the Hermite polynomial Kz in the specification of
6 However, since the density function in (22.12) can be shown to be a homogeneousfunction of the coeﬃcients of the polynomial P(zt ), the coeﬃcients aα can only be determined to within a scalar multiple. Thus, to achieve model identification, the constantterm of the polynomial part is always set to 1.

862

22. Seminonparametric Conditional Density Models

the SNP model. The optional argument zPoly of the function SNP.model
corresponds to Kz , and setting this to a non-negative integer specifies asemiparametric VAR model. For example, to estimate a semiparametricAR(1) model with Kz = 4 to the spot return data, use7
> fit.10014000 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1,
zPoly=4), n.drop=14)> summary(fit.10014000)
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1,
zPoly = 4), n.drop = 14)
Model: Semiparametric VAR
Hermite Polynomial Coefficients:
z^0
z^1
z^2
coef 1.0000 -0.0214 -0.1812(std.err)
0.0795 0.0226(t.stat)-0.2699 -8.0043

z^3
0.01100.0102
1.0703z^40.02350.0032
7.3975

Conditional Mean Coefficients:muar(1)coef -0.0287 0.0460(std.err) 0.1151 0.0283(t.stat) -0.2493 1.6245

Conditional Variance Coefficients:
sigma
coef 1.0458(std.err) 0.0376(t.stat) 27.8252

Information Criteria:
BIC
HQ
AIC
logL
1.399 1.3866 1.3789 -1123.668
Convergence Type:relative function convergence
7 Gallant and Tauchen (2001, p. 15) state “Experience has taught us never to consider
a value of Kz < 4.”

22.3 Estimating SNP Models in S+FinMetrics
SNP

863

Normal

y

0.0

0.05

0.10

0.15

0.20

0.25

0.30

dmRet.dat[, "exch"]

-4

-2

0

2

4

x

FIGURE 22.3. Estimated conditional density from semiparametric AR(1) forweekly spot returns.

number of iterations: 12
The estimated coeﬃcient values for aα of P(z) are given under the tablelabeled Hermite Polynomial Coefficients. The estimate of P(z) has theformP̂(z) = 1 − 0.0214 · z − 0.1812 · z 2 + 0.0110 · z 3 + 0.0235 · z 4
The estimated coeﬃcients on the even powers of z are highly significant andindicate a conditional density with tails fatter than the Gaussian distribution. The coeﬃcients on the odd powers of z are not significantly diﬀerently
from zero and suggest that the conditional density is symmetric. To be sure,the function SNP.density can be used to compute and plot the estimated
SNP conditional density (22.12):> SNP.density(fit.10014000)
The estimated density, shown in Figure 22.3, is computed with xt−1 set toits sample mean value. Also shown is a normal density with the same mean
and variance as (22.12).
Given that the Hermite coeﬃcients on z1 and z3 are insignificant, it
may be of interest to re-estimate the semiparametric AR(1) with these
coeﬃcients restricted to zero. The optional argument est determines which
coeﬃcients of the SNP model are fixed at their initial values and which are864

22. Seminonparametric Conditional Density Models

estimated. The argument est is a vector of logical values with length equal
to coef, the vector of starting values. If an element of est is TRUE, thenthe corresponding element of coef is estimated freely; if an element of
est is FALSE, then the corresponding element of coef is fixed during theestimation. To re-estimate the semiparametric AR(1) with the coeﬃcientson z1 and z3 equal to zero, the starting values for these coeﬃcients must
be set to zero and the corresponding elements of the vector est must be
set to FALSE8 :> coef.start = fit.10014000$coef> est.start = fit.10014000$est> coef.start[c(2,4)] = 0> est.start[c(2,4)] = F> fitr.10014000 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1,+
zPoly=4), n.drop=14, coef=coef.start,+
est=est.start)> fitr.10014000
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1,
zPoly = 4), n.drop = 14, coef = coef.start,
est = est.start)
Model: Semiparametric VAR
Hermite Polynomial Coefficients:z^0 z^1z^2 z^3z^4
1
0 -0.1819
0 0.0236
Conditional Mean Coefficients:
mu ar(1)-0.0244 0.051
Conditional Variance Coefficients:
sigma
1.0481
Information Criteria:
BIC
HQ
AIC
logL
1.3927 1.3839 1.3784 -1125.25
Convergence Type:
8 Currently, the names of the SNP model coeﬃcients are not attached to the elementsof coef. However, these names can be determined using the coef() extractor function.

22.3 Estimating SNP Models in S+FinMetrics

865

relative function convergence
number of iterations: 4
Example 150 Semiparametric VAR for bivariate exchange rate data
With multivariate time series data, the semiparametric VAR can becomecomplicated and include many terms if Kz is large. To illustrate, consider
fitting a semiparametric VAR(1) model to the bivariate exchange rate datawith Kz = 4:
> bfit.10014000 = SNP(dmRet.dat[,1:2],model=SNP.model(ar=1,+
zPoly=4), n.drop=14)> summary(bfit.10014000)
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
zPoly = 4), n.drop = 14)
Model: Semiparametric VAR
Hermite Polynomial Coefficients:
z^00
z^01
z^10
z^02
coef
1.00000.09670.0908 -0.3057(std.err)
0.05330.07080.0196(t.stat)1.81401.2821 -15.6251

z^11
0.04890.03551.3766coef(std.err)(t.stat)z^20-0.1703
0.0210-8.1056z^03-0.0025
0.0096-0.2604z^12
0.02280.01181.9325z^21-0.00540.0130-0.4126z^30
0.00020.00990.0221coef(std.err)(t.stat)z^04
0.03930.0034
11.5137z^13-0.0271
0.0070-3.8679z^22
0.00660.00820.8103z^31
0.00010.00890.0084z^40
0.02110.0031
6.7813

Conditional Mean Coefficients:
exch
forwardmu-0.2372-0.0334(std.err)
0.10410.0139(t.stat)-2.2772-2.4017var(1;1)

0.0086-0.1501866

22. Seminonparametric Conditional Density Models(std.err)(t.stat)0.02830.30480.0331-4.5398

var(1;2)(std.err)(t.stat)-0.00220.0053-0.4124

0.97860.0056173.8433

Conditional Variance Coefficients:
sigma11 sigma12 sigma22
coef 1.0376 0.0068 0.2389(std.err) 0.0374 0.0410 0.0033(t.stat) 27.7294 0.1659 72.3597

Information Criteria:
BIC
HQ
AIC
logL
1.3014 1.2607 1.2354 -990.0173
Convergence Type:relative function convergence
number of iterations: 63
In the printout of the Hermite coeﬃcients, the indices in the superscriptsof the variable z correspond to the indices α = (α1 , α2 ) of the coeﬃcientsin the polynomial P(z). For example, the indices 01 and 10 correspond toa(0,1) and a(1,0) , which are the coeﬃcients on z2 and z1 in P(z), respectively.
With Kz = 4, the estimated value of P(z) has the formP̂(z) = 1 + 0.0908 · z1 + 0.0967 · z2
+0.0489 · z1 z2 − 0.1703 · z12 − 0.3057 · z22−0.0054 · z12 z2 + 0.0228 · z1 z22 + 0.0002 · z13 − 0.0025 · z23+0.0001 · z13 z2 + 0.0066 · z12 z22 − 0.0271 · z1 z23 + 0.0211 · z14+0.0393 · z24
Notice that only the even powers of z1 and z2 are highly significant in thebivariate semiparametric VAR(1).
The marginal conditional densities for the spot return and the forward
discount may be computed and plotted using> SNP.density(bfit.10014000)
These densities are illustrated in Figure 22.4.
For multivariate models, one may want to eliminate interaction terms(e.g., z1 z2 , z12 z2 , z2 z12 , etc.) in the polynomial P (z) to reduce the numberof estimated parameters and to improve the stability of the fit. By default,

22.3 Estimating SNP Models in S+FinMetrics
SNP

Normalforward

0

0.0

2

0.05

4

0.10

0.15

6

0.20

8

0.25

10

0.30

12

exch

y

867

-4

-2

0

2

4

-0.2

0.0

0.2

0.4

0.6

0.8

x

FIGURE 22.4. Estimated marginal conditional densities from bivariate semiparametric VAR(1) model for weekly spot returns and forward discounts.

all possible interactions are allowed. To suppress the highest-order interactions, set the optional SNP.model argument trimZ=1. This will suppress
all interaction terms of order Kz . For example,> bfit.10014100 = SNP(dmRet.dat[,1:2],model=SNP.model(ar=1,+
zPoly=4, trimZ=1), n.drop=14)> bfit.10014100
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
zPoly= 4, trimZ = 1), n.drop = 14)
Model: Semiparametric VAR
Hermite Polynomial Coefficients:
z^00
z^01
z^10
z^02
z^11
z^20
z^03
z^12
1 0.0671 0.0843 -0.2938 0.0398 -0.1417 -0.0026 0.0046
z^21
z^30
z^04
z^40
-0.0166 0.0019 0.0414 0.0202
...

868

22. Seminonparametric Conditional Density Models

With zPoly=4 and trimZ=1, the interaction terms z1 z23 , z12 z22 , and z13 z2 aresuppressed. The component idxZ of the fitted "SNP" object indicates whichterms are included in P(z):> bfit.10014100$idxZ[,1] [,2]
[1,]
0
0
[2,]
0
1
[3,]
1
0
[4,]
0
2
[5,]
1
1
[6,]
2
0
[7,]
0
3
[8,]
1
2
[9,]
2
1
[10,]
3
0
[11,]
0
4
[12,]
4
0
In general, all interaction terms of degree higher than zPoly-trimZ will besupressed. To filter out all interactions, set trimZ to zPoly-1:> bfit.10014300 = SNP(dmRet.dat[,1:2],model=SNP.model(ar=1,+
zPoly=4, trimZ=3), n.drop=14)> bfit.10014300
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1, zPoly= 4, trimZ = 3), n.drop = 14)
Model: Semiparametric VAR
Hermite Polynomial Coefficients:
z^00
z^01
z^10
z^02
z^20
z^03
z^30
z^04
1 0.0344 0.0322 -0.2884 -0.1289 -0.0008 0.0064 0.0407
z^40
0.0194...

22.3.4 Conditional Heterogeneity
Although a semiparametric VAR density can accurately approximate densities with arbitrary shape departures from the Gaussian VAR, the approximated shape is constant with respect to variations in xt−1 ; that is, thesemiparametric VAR density is conditionally homogeneous, because only

22.3 Estimating SNP Models in S+FinMetrics

869

the first moment of the density depends on xt−1 . However, for economic andfinancial time series data, it is usually found that the underlying stochastic
processes are conditionally heterogeneous, or conditionally heteroskedastic.
The SNP methodology allows one to model the conditional heterogeneity
in multiple time series using two approaches, which can be used either
separately or jointly. The first approach is to let the Hermite polynomial
coeﬃcients aα be a multivariate polynomial of order Kx in xt−1 :aα (xt−1 ) =Kx
Xaαβ xβt−1|β |=0with β denoting the non-negative multi-index (β 1 , . . . , β ML ), and|β| = β 1 + · · · + β M ,xβt = (xt,1 )β 1 × · · · × (xt,M L )β ML

The Hermite polynomial becomes
P (zt , xt−1 ) =

Kx
Kz X
X

aαβ xβt−1 zαt|α|=0 |β |=0

For example, let M = 1, Kz = 2, Kx = 1, and xt−1 = yt−1 . Then,
P(zt , xt−1 ) = (1 + a0,1 yt−1 ) + (a1,0 + a1,1 yt−1 )zt + (a2,0 + a2,1 yt−1 )zt2
which shows that the coeﬃcients on the powers of z are linear functions of
the conditioning variable yt−1 . If Kx = 2, then the coeﬃcients will containlinear and quadratic terms in yt−1 .
The SNP density function becomesf (yt |xt−1 , θ) ∝ [P(zt , xt−1 )]2 N (yt ; μxt−1 , Ω)

(22.14)

where P(zt , xt−1 ) is now a multivariate polynomial of degree Kz + Kx .
It is referred to as the nonlinear nonparametric VAR model . When Kx is
set to zero, the above density reduces to the semiparametric VAR densityin (22.12) and thus nests semiparametric VAR as a special case. WhenKx is positive, the above density can approximate any form of conditional
heterogeneity in principle, because the shape and thus all moments of thedensity will depend on xt−1 .
In the polynomial P(zt , xt−1 ), the number of terms associated with eachpower of zt is a function of the number of elements in xt−1 . If xt−1 contains
many lags of yt then the number of terms in P(zt , xt−1 ) can become verylarge. In applications, it may be advantageous to restrict the number oflags, Lp , of xt−1 to a small number. In addition, if Kz is large, there will
be many interaction terms between the elements of zt . Similarly, if Kx islarge there will be many interaction terms between the elements of xt−1 . In
applications, it may be desirable to limit the number of these interactions.
The following example illustrates how to do this.

870

22. Seminonparametric Conditional Density Models

Example 151 Nonlinear nonparametric VAR for univariate exchange ratedata

To fit a nonlinear nonparametric AR(1) model to the spot return datawith Kz = 4 and Kx = 1, use> fit.10014010 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1,+
zPoly=4, xPoly=1), n.drop=14)> fit.10014010
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1,
zPoly = 4, xPoly = 1), n.drop = 14)
Model: Nonlinear Nonparametric
Hermite Polynomial Coefficients:x^0x^1
z^0 1.0000 -0.2118
z^1 0.0054 -0.0508
z^2 -0.1939 0.0443
z^3 0.0081 -0.0031
z^4 0.0230 -0.0039
Conditional Mean Coefficients:
mu ar(1)-0.037 0.2047
Conditional Variance Coefficients:
sigma
1.0899
Information Criteria:
BIC
HQ
AIC
logL
1.4175 1.3963 1.3831 -1122.127
Convergence Type:relative function convergence
number of iterations: 36
With Kz = 4 and Kx = 1, the printout of the Hermite coeﬃcients is a
5 × 2 table giving the polynomial coeﬃcients aα β . The first row gives thecoeﬃcients on z 0 , the second row gives the coeﬃcients on z 1 , and so on.

22.3 Estimating SNP Models in S+FinMetrics

871

Since xt−1 = yt−1 , the estimated polynomial P (zt , xt−1 ) has the form9
P (zt , yt−1 ) = (1 − 0.2118yt−1 ) + (0.0054 − 0.0508yt−1 ) zt
+ (−0.1939 + 0.0443yt−1 ) zt2 + (0.0081 − 0.0031yt−1 ) zt3+ (0.0230 − 0.0039yt−1 ) zt4
If xPoly is not zero, the optional argument lagP to the SNP.model function can be used to set the number of lagged values in xt−1 to go into
the polynomial P (zt , xt−1 ). By default, lagP is set to 1 so that xt−1 only
contains yt−1 . To re-estimate the nonlinear nonparametric AR(1) with twolags of yt in xt−1 , use> fit.10024010 = SNP(dmRet.dat[,"exch"],+model=SNP.model(ar=1,zPoly=4,xPoly=1,lagP=2),+n.drop=14)> fit.10024010
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1,
zPoly = 4,xPoly = 1, lagP = 2), n.drop = 14)
Model: Nonlinear Nonparametric
Hermite Polynomial Coefficients:
x^00
x^01
x^10
z^0 1.0000 -0.1381 0.1434
z^1 0.0094 0.0030 0.0480z^2 -0.1956 0.0389 -0.0369
z^3 0.0073 -0.0049 -0.0074
z^4 0.0231 -0.0034 0.0021...
Example 152 Nonlinear nonparametric VAR for bivariate exchange ratedata
To fit a bivariate nonlinear nonparametric VAR(1) to the exchange rate
data with Kz = 4, Kx = 1, and Lp = 1, use> bfit.10014010 = SNP(dmRet.dat[,1:2],model=SNP.model(ar=1,+
zPoly=4,xPoly=1), n.drop=14)> bfit.10014010
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
9 Keep in mind that the constant term of the polynomial is always set to one.

872

22. Seminonparametric Conditional Density Models

zPoly = 4, xPoly = 1), n.drop = 14)
Model: Nonlinear Nonparametric
Hermite Polynomial Coefficients:
x^00
x^01
x^10
z^00 1.0000 -0.1985 -0.1009
z^01 0.0605 0.0041 0.1063
z^10 0.0279 -0.0566 0.0374
z^02 -0.2736 0.0598 0.0269
z^11 0.0656 -0.0198 -0.0149
z^20 -0.1472 0.0331 0.0258
z^03 -0.0035 -0.0004 -0.0317
z^12 0.0138 0.0061 -0.0123
z^21 -0.0103 -0.0002 -0.0002
z^30 0.0072 0.0064 -0.0066
z^04 0.0338 -0.0001 -0.0042
z^13 -0.0130 -0.0013 0.0156
z^22 -0.0027 0.0132 0.0002
z^31 -0.0002 0.0021 -0.0005z^40 0.0196 -0.0026 0.0004
Conditional Mean Coefficients:
exch forward
mu -0.1195 -0.0222
var(1;1) 0.0015 -0.0799
var(1;2) -0.0224 0.9721
Conditional Variance Coefficients:
sigma11 sigma12 sigma22
0.9789 -0.0948 0.2373
Information Criteria:
BIC
HQ
AIC
logL
1.3763 1.2826 1.2242 -950.8071
Convergence Type:relative function convergence
number of iterations: 113

The fitted SNP model has many parameters: 6 conditional mean parameters, 3 conditional variance parameters, and 44 Hermite polynomial coefficients. The form of the estimated multivariate polynomial P̂(zt , xt−1 ) is

22.3 Estimating SNP Models in S+FinMetrics

873

rather complicated:P̂(zt , xt−1 ) = (1 − 0.1985∆st−1 − 0.1009f dt−1 )+ (0.0279 − 0.0566∆st−1 + 0.0374f dt−1 ) z1t+ (0.0605 + 0.0041∆st−1 + 0.1063f dt−1 ) z2t+··· +
4+ (0.0338 − 0.0001∆st−1 − 0.0042f dt−1 ) z2t
In general, many of the coeﬃcients in P̂(zt , xt−1 ) may be insignificant.
In particular, many of the interaction terms between the elements of zt
and xt−1 may be insignificant. When zPoly>0 and xPoly>0, the numberof interaction terms allowed in the Hermite polynomial is controlled by
the SNP.model optional arguments trimZ and trimX. By default, trimZ=0
and trimX=0, which allows all interactions. Setting trimZ=1 will exclude
interactions between the elements of zt at the highest order (Kz ); setting
trimX=1 will exclude interactions between the elements of xt−1 at the highest order (Kx ). To exclude all interaction terms, set trimZ to zPoly-1 andtrimX to xPoly-1. For example, to fit a bivariate nonlinear nonparametric
VAR(1) model with Kz = 4, Kx = 2 and no interaction terms, use> bfit.10014321 = SNP(dmRet.dat[,1:2],+ model=SNP.model(ar=1,zPoly=4,xPoly=2,trimZ=3,trimX=1),+ n.drop=14)> bfit.10014321
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
zPoly = 4, xPoly = 2, trimZ = 3, trimX = 1),n.drop = 14)
Model: Nonlinear Nonparametric
Hermite Polynomial Coefficients:
x^00
x^01
x^10
x^02
x^20
z^00 1.0000 -0.2433 -0.4283 0.0797 0.1209
z^01 0.0359 0.0868 0.0554 -0.0240 -0.0494
z^10 0.0530 0.0260 0.0245 0.0030 -0.0163
z^02 -0.2807 0.0860 0.1495 -0.0213 -0.0360
z^20 -0.1627 0.0690 0.0364 -0.0157 0.0180
z^03 -0.0001 -0.0138 -0.0317 0.0033 0.0130
z^30 0.0090 -0.0012 -0.0003 -0.0029 -0.0007
z^04 0.0340 -0.0040 -0.0205 0.0090 0.0021
z^40 0.0173 -0.0036 -0.0017 -0.0023 0.0015
...

874

22. Seminonparametric Conditional Density Models

22.3.5 ARCH/GARCH Leading Term
In practice, in order to model conditional heteroskedasticity often observedin financial time series data, the order Kx of the polynomial in xt−1 canget quite large. To keep Kx small when the data are markedly conditionally heteroskedastic, the SNP methodology allows for a second approachbased on using a Gaussian ARCH model or Gaussian GARCH model as
the leading term in the Hermite expansion, rather than a Gaussian VAR
model.
The class of ARCH/GARCH models, discussed in Chapters 7 and 13,
are very successful at modeling conditional heteroskedasticity in macroeconomic and financial time series data; for example, see Engle (1995). To
use the Gaussian ARCH as the leading term for the SNP method, Gallant,
Hsieh, and Tauchen (1991) suggested modeling the upper triangular matrix R as a linear function of the absolute values of the lagged elements ofyt − μxt−1 :vech(Rxt−1 ) = ρ0 +

Lr
X
i=1

Pi |yt−1−Lr +i − μxt−2−Lr +i |(22.15)where vech(R) denotes a vector of length M (M + 1)/2 containing the elements of the upper triangle of R, ρ0 is a vector of length M (M + 1)/2, andPi is an M (M + 1)/2 × M matrix for i = 1, . . . , Lr . The expression (22.15),
and its extensions, is referred to as the conditional variance equation.
Equation (22.15) represents the SNP version of the Lr -th order ARCH
model, or ARCH(Lr ) model. However, the classical ARCH model has Rxt−1
depending on a linear function of squared lagged residuals instead of absolute values. The SNP version of ARCH is more akin to the suggestions of
Taylor (1986) and Davidian and Carroll (1987). In practice, this version has
been shown to perform better than the classical ARCH model, for example,
see Schwert (1990) and Ding, Granger, and Engle (1993).
The Gaussian ARCH leading term can usually be further improved byadding a Gaussian GARCH leading term:vech(Rxt−1 ) = ρ0 +

Lr
X
i=1

+

Lg
X

Pi |yt−1−Lr +i − μxt−2−Lr +i |(22.16)diag(Gi )vech(Rxt−2−Lg +i )

i=1where Gi is a vector of length M (M + 1)/2 for i = 1, . . . , Lg . This equation
represents a GARCH model with order (Lr , Lg ), where Lr gives the ARCH
order and Lg gives the GARCH order.
In summary, a Hermite polynomial with coeﬃcients modeled as a polynomial in terms of xt−1 , which is referred to as “state dependence,” and22.3 Estimating SNP Models in S+FinMetrics

875

a Gaussian ARCH/GARCH leading term can be used either separately or
jointly to model conditional heterogeneity.10 The resulting SNP density has
the following form:f (yt |xt−1 , θ) ∝ P(zt , xt−1 )2 N (yt |μxt−1 , Ωxt−1 )where Ωxt−1 = Rxt−1 · R0xt−1 . When Kz (and thus Kx ) is set to zero, the
SNP density becomes a Gaussian GARCH model ; when Kz is positive andKx is zero, the SNP density becomes a semiparametric GARCH model
similar to Engle and González-Rivera (1991); when both Kz and Kx are
positive, the resulting nonparametric density also allows state dependence.
In general, the model parameter θ contains the GARCH coeﬃcients inaddition to the coeﬃcients aα β of the Hermite polynomial, plus μ0 and B.
In practice GARCH processes have been very successful in modeling theconditional heteroskedasticity in univariate time series. However, for a multivariate time series, the GARCH structure requires many parameters in
the conditional variance equation. In addition, it may be diﬃcult to obtain convergence of the maximum likelihood estimate, and if convergenceoccurs, the resulting estimates may be sensitive to starting values. Experience has shown that the GARCH(1,1) process is a useful starting point
for analysis. To improve the performance of SNP models with a GARCH
structure and to ease the computational burden for multiple time seriesmodeling, Gallant and Tauchen (2001) recommended imposing two simplifying restrictions: (1) restrict the ARCH coeﬃcient matrix so that the
diagonal elements of Rxt−1 only depend on the corresponding elements of|yt−1−Lr +i − μxt−2−Lr +i | and (2) restrict the elements of each GARCH
coeﬃcient matrix, Gi , to be the same.
In the S+FinMetrics implementation of the SNP methodology, a diagonal VEC multivariate GARCH model is also available. This model for theconditional variance, discussed in Section 3 of Chapter 13, has the form
vech(Ωxt−1 ) = ρ0 +

Lr
X
i=1

+

Lg
X

¡
¢
diag(Pi )vech εt−i ε0t−i(22.17)diag(Gi )vech(Ωxt−2−Lg +i )

i=1where εt = yt − μxt−1 .
Example 153 Gaussian ARCH/GARCH model for univariate exchange
rate data
10 However, when the Gaussian GARCH leading term is used (i.e., when L > 0) theg
SNP model is no longer Markovian, because one must know lagged values of Rxt−1 aswell as xt−1 to iterate forward for yt .

876

22. Seminonparametric Conditional Density Models

To model conditional heteroskedasticity in spot returns with a
GARCH(1,1) leading term, use>+

fit.11110000 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1,
arch=1,garch=1), n.drop=14)

The returned object fit.11110000 represents the fit of an AR(1)GARCH(1,1) model, where the optional arguments arch and garch to the
SNP.model function specify the ARCH order Lr and GARCH order Lg , respectively. Since the model is univariate, the conditional variance equation(22.16) has the form
σ t = ρ0 + p1 |yt−1 − μxt−2 | + g1 σ t−1
where μxt−2 = μ0 − byt−2 . A summary of the estimated model is> summary(fit.11110000)
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1,
arch = 1, garch = 1), n.drop = 14)
Model: Gaussian GARCH
Conditional Mean Coefficients:
mu ar(1)coef 0.0180 0.0911(std.err) 0.0259 0.0333(t.stat) 0.6929 2.7351

Conditional Variance Coefficients:
s0 arch(1) garch(1)coef 0.0783 0.1881 0.7822(std.err) 0.0162 0.0229 0.0289(t.stat) 4.8174 8.2272 27.0207

Information Criteria:
BIC
HQ
AIC
logL
1.3635 1.3546 1.3491 -1101.283
Convergence Type:relative function convergence
number of iterations: 25

22.3 Estimating SNP Models in S+FinMetrics877

The estimated ARCH and GARCH coeﬃcients are p1 = 0.1881 and g1 =0.7822, respectively, and are highly significant.11
A semiparametric GARCH model, or state dependence model, can also
be fitted by setting zPoly or xPoly to a non-negative integer as shown inearlier examples.
Example 154 Gaussian ARCH/GARCH model for bivariate exchange ratedata
For the bivariate exchange rate data, consider fitting a VAR(1)GARCH(1,1) using the commands> bfit.11110000.r2 = SNP(dmRet.dat[,1:2],+
model=SNP.model(ar=1,arch=1,garch=1),+
control=SNP.control(dvec=F),+n.drop=14)
The SNP.control optional argument dvec=F turns oﬀ the default diagonal
VEC structure. The fitted model coeﬃcients are> bfit.11110000
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
arch = 1, garch = 1), n.drop = 14, control =SNP.control(darch = F, rgarch = F, dvec = F))
Model: Gaussian GARCH
Conditional Mean Coefficients:
exch forward
mu -0.0139 -0.0104
var(1;1) 0.0795 -0.0836
var(1;2) -0.0058 0.9946
Conditional Variance Coefficients:
R11
R12
R22
s0 0.0967 0.0312 0.0152
arch(1;1) 0.2066 -0.0504 -0.0065
arch(1;2) -0.0350 -0.2094 0.2656
garch(1) 0.7546 0.0002 0.7662
Information Criteria:
BIC
HQ
AIC

logL

11 Since the absolute value is used in the ARCH specification, the usual condition
p1 + g1 < 1 does not determine stationarity of the model.

878

22. Seminonparametric Conditional Density Models

1.1378 1.1059 1.0861 -872.6075
Convergence Type:relative function convergence
number of iterations: 47
The ARCH and GARCH coeﬃcients are given in the Conditional VarianceCoeﬃcients table. The form of the estimated conditional variance equation(22.16) has coeﬃcient matrices
⎛
⎞
⎛
⎞
⎛
⎞
0.09670.2066 −0.0350
0.7546
ρ̂0 = ⎝ 0.0312 ⎠ , P̂1 = ⎝ −0.0504 −0.2094 ⎠ , Ĝ1 = ⎝ 0.0002 ⎠0.0152−0.0065 0.2656
0.7662

To estimate the model imposing a diagonal ARCH structure on P1 , set
the SNP.control optional argument darch=T, and to estimate the model
imposing common GARCH parameters in G1 , set the optional argument
rgarch=T:> bfit.11110000.r2 = SNP(dmRet.dat[,1:2],+
model=SNP.model(ar=1,arch=1,garch=1),+
control=SNP.control(darch=T,rgarch=T,dvec=F),+n.drop=14)> bfit.11110000.r2
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
arch = 1, garch = 1),n.drop = 14, control =SNP.control(darch = T, rgarch = T, dvec = F))
Model: Gaussian GARCH
Conditional Mean Coefficients:
exch forward
mu -0.0079 -0.0086
var(1;1) 0.0712 -0.0853
var(1;2) -0.0084 0.9967
Conditional Variance Coefficients:
R11
R12
R22
s0 0.0856 -0.0076 0.0109
arch(1;1) 0.2034 0.0000 0.0000
arch(1;2) 0.0000 0.0000 0.2689
garch(1) 0.7637 0.7637 0.7637
Information Criteria:

22.3 Estimating SNP Models in S+FinMetrics

879

BIC
HQ
AIC
logL
1.1195 1.0982 1.085 -877.7125
Convergence Type:relative function convergence
number of iterations: 37
In the diagonal ARCH with restricted GARCH coeﬃcients, the estimatedcoeﬃcient matrices for the conditional variance equation are
⎛
⎞
⎛
⎞
⎛
⎞
0.08560.203400.7637⎠ , Ĝ1 = ⎝ 0.7637 ⎠00ρ̂0 = ⎝ −0.0076 ⎠ , P̂1 = ⎝0.010900.26890.7637
To estimate a VAR-DVEC(1,1) model, which is the default multivariate
GARCH model, use> bfit.11110000.dvec = SNP(dmRet.dat[,1:2],+
model=SNP.model(ar=1,arch=1,garch=1),+
control=SNP.control(dvec=T),+n.drop=14)> bfit.11110000.dvec
Call:
SNP(data = dmRet.dat[, 1:2], model = SNP.model(ar = 1,
arch = 1, garch = 1), n.drop = 14,
control = SNP.control(dvec = T))
Model: Gaussian GARCH
Conditional Mean Coefficients:
exch forward
mu -0.0055 -0.0082
var(1;1) 0.0352 -0.0828
var(1;2) -0.0052 0.9946
Conditional Variance Coefficients:
R11
R12
R22
s0 0.2894 0.0339 0.0218
arch(1) 0.3222 0.3364 0.3653
garch(1) 0.8440 0.0470 0.9292
Information Criteria:
BIC
HQ
AIC
logL
1.0902 1.0636 1.0471 -843.6081

880

22. Seminonparametric Conditional Density Models

Convergence Type:relative function convergence
number of iterations: 112
Common GARCH coeﬃcients may be imposed by setting rgarch=T.
For fitting SNP models with a multivariate GARCH structure, Gallant
and Tauchen (2001) recommended starting with a highly restricted specification to obtain a stable fit and then moving on to more general processes.

22.4 SNP Model Selection
The examples in the previous sections showed how to fit diﬀerent SNP
models using the S+FinMetrics function SNP. It was seen that the SNPmodel can become increasingly more complex by setting diﬀerent model
parameters, such as ar, arch, garch, zPoly, and xPoly, to large values.
Due to the nested structure of SNP models, model selection can be conducted using conventional information criteria, such as the Akaike Information Criterion (AIC), the Bayesian Information Criterion (BIC), or the
Hannan-Quinn (HQ) criterion. Model selection strategies for determining
the best fitting SNP models are discussed in Fenton and Gallant (1996a,
1996b), Coppejans and Gallant (2002), and Gallant and Tauchen (2001).
The general consensus from these articles is that minimizing the BIC
BIC(pθ ) = sn (θ) +

pθ
ln(n)
2n

where sn (θ) is the average log-likelihood (22.13) and pθ denotes the numberof parameters in the SNP model, performs best.
The model selection strategy entails moving upward along an expansion
path. For univariate models, Gallant and Tauchen (2001) suggest the searchorder:[1] Determine best VAR order Lu .
[2] Determine best ARCH and GARCH orders Lr and Lg .
[3] Determine best z polynomial order Kz (start at Kz = 4).
[4] Determine the best x polynomial order Kx .
The fitted SNP models become more richly parameterized at each level
along the path. At each step of the expansion, Gallant and Tauchen recommended using the previously fit (smaller) model coeﬃcients as starting
values for the currently fit model. In addition, for each SNP model fit,
they recommended that a random restart procedure of the optimizer be22.4 SNP Model Selection

881

implemented to avoid getting stuck at potential local minima. The function SNP implements the random restart procedure advocated by Gallant
and Tauchen, and is discussed in detail below.
The suggested search procedure is not an exhaustive search of all possible models and may not produce the model that globally minimizes BIC.
Furthermore, Fenton and Gallant (1996a) found that the strategy of minimizing BIC often produced an overly conservative model. In particular,
BIC had a tendency to select models with Kx = 0 in situations in which
one would expect Kx > 0. As a result, Gallant and Tauchen recommended
that a battery of residual and graphical diagnostics be run on any tentativemodel. Section 22.5 discusses SNP diagnostics and provides some examples.
Model selection issues in multivariate SNP models have not been studied as closely as model selection in univariate models. The main complications are with the form of the multivariate ARCH/GARCH model andthe form of the multivariate Hermite polynomial P(zt , xt−1 ). Multivariate
ARCH/GARCH models are notoriously unstable. As a result, Gallant andTauchen (2001) advocated first fitting a highly restricted specification (e.g.,
GARCH(1,1) with common GARCH coeﬃcients or DVEC-GARCH(1,1))
to obtain a stable fit and then moving on to more general processes. With
respect to the specification of P(zt , xt−1 ), Gallant and Tauchen recommended starting with models that do not allow any interactions (i.e., setting trimZ to zPoly-1 and trimX to xPoly-1) and then using BIC todetermine if any interactions need to be included.
In S+FinMetrics, the functions expand and SNP.auto are provided to facilitate model selection strategies to determine the best fitting SNP model.
The function expand allows one to expand an existing SNP model toward alarger model, and the function SNP.auto implements Gallant and Tauchen’ssuggested model selection strategy for determining the best fitting univariate SNP model. These functions are described in the following subsections.

22.4.1 Random Restarts
The function SNP implements the random restart procedure described in
Gallant and Tauchen (2001, Sec. 3). The procedure is best described throughan example. Consider fitting an AR(1)-ARCH(3) SNP model to the spot
return data using SNP. First, fit a simple AR(1) model using> fit.10010000 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1),+ n.drop=14)> fit.10010000
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1),n.drop = 14)

882

22. Seminonparametric Conditional Density Models

Model: Gaussian VAR
Conditional Mean Coefficients:
mu ar(1)0.0005 0.0222
Conditional Variance Coefficients:
sigma
1.0025
Information Criteria:
BIC
HQ
AIC
logL
1.4337 1.4284 1.4251 -1165.6
Convergence Type:both x and relative function convergence
number of iterations: 2
Since the quasilikelihood for an AR(1) model is a quadratic function of θ,
random restarts are not needed to check for local minima.
Next, consider fitting an AR(1)-ARCH(3) without random restarts using
the coeﬃcients from the AR(1) as starting values:> fit.10310000 = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1),+ iArch=3,coef=fit.10010000$coef,n.drop=14)> fit.10310000
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1),n.drop = 14, coef = fit.10010000$coef, iArch = 3)
Model: Gaussian ARCH
Conditional Mean Coefficients:
mu ar(1)0.008 0.1362
Conditional Variance Coefficients:
s0 arch(1) arch(2) arch(3)0.5059 0.2655 0.1764 0.2152
Information Criteria:
BIC
HQ
AIC
logL
1.3735 1.3629 1.3563 -1106.182
Convergence Type:22.4 SNP Model Selection

883

relative function convergence
number of iterations: 9
The optional argument iArch=3 tells the SNP function to increment the
ARCH component of the initial model specified in the call to SNP.modelby 3. When one or more of the optional increment arguments (iAr, iArch,iGarch, iZpoly, iXpoly, itrimZ, itrimX) to SNP are utilized, the model
specified in the call to SNP.model is called the initial model, and the model
fit is called the expanded model. By setting coef=fit.10010000$coef, the
SNP function uses the AR(1) model coeﬃcients as starting values for thecoeﬃcients in the initial model and sets the starting values for the additional ARCH(3) coeﬃcients to zero. The coeﬃcients for the initial model
are referred to as the “old” coeﬃcients, and additional coeﬃcients specified by increment arguments (e.g., iArch=3) are referred to as the “new”
coeﬃcients.12
The same fit to the AR(1)-ARCH(3) may be obtained using> fit.10310000b = SNP(dmRet.dat[,"exch"],model=SNP.model(ar=1,+
arch=3), n.drop=14)
The reason for this is that, by default, the SNP function uses a VAR(p)model (where p is specified by the ar component of SNP.model) to generatestarting values for any specified SNP model. As a result, if the starting value
and model increment arguments are not specified in the call to SNP, thenthe “old” coeﬃcients are the coeﬃcients to the VAR(p) and the “new”
coeﬃcients are the remaining parameters of the specified SNP model.
Now, consider refitting the AR(1)-ARCH(3) using Gallant and Tauchen’s
random restart procedure. The restart procedure involves refitting theAR(1)-ARCH(3) model using randomly perturbed starting values for boththe old (initial model) coeﬃcients and the new (expanded model) coefficients. The user has control over how the old and new coeﬃcients are
perturbed, as well as how many random perturbations are performed. The
random perturbations of the start values are generated using the scheme
ρold
i
new
ρi

old
→ ρold)i · (1 + u · tweaknewnew→ ρi · (1 + u · tweak )(22.18)where u is a random variable distributed as U [−1, 1], and tweakold andtweaknew are so-called tweak constants. For each random restart, the starting values are perturbed as in (22.18), and the SNP model optimization
is run for a specified number of iterations to explore the surface of the
SNP objective function (22.13). At convergence (or last iteration, whichevercomes first), the SNP objective function value is recorded, and the restart
12 This example is given to illustrate how the SNP function distinguishes between old
and new parameters. In practice, the user would not call SNP with the increment arguments. Instead, the user would call the expand function discussed in the next subsection.

884

22. Seminonparametric Conditional Density Models

procedure is repeated. Of all the randomly restarted fits, the best fit isdetermined as the one with the lowest SNP objective function value. If
necessary, this fit is then iterated to convergence.
In the function SNP, the random restart procedure is controlled throughthe SNP.control optional arguments fOld, fNew, n.start, initial.itmax,
final.itmax, and seed. The arguments fOld and fNew determine thetweak constants tweakold and tweaknew , respectively, and n.start determines the number of random restarts. The argument initial.itmax sets
the maximum number of iterations for each random-restart, andfinal.itmax sets the maximum number of iterations for the best of therandomly restarted models. The seed argument sets the random number
seed used to generate u.
For example, to refit the AR(1)-ARCH(3) using 25 random restarts with
tweakold = tweaknew = 0.1, use> fit.10310000.restart = SNP(dmRet.dat[,"exch"],+model=SNP.model(ar=1,arch=3), n.drop=14,+
control=SNP.control(fOld=0.1,fNew=0.1,n.start=25),+
trace=T)
random start 1, objective = 1.349
random start 2, objective = 1.349...
random start 25, objective = 1.349
Iteration No. 1, objective = 1.349
The optional argument trace=T allows the progress of the restart procedure
to be monitored on screen.13 For each random restart the SNP objectivefunction value (22.13) is printed. After the best fit is determined, it is
iterated to convergence. In the above output, it appears that all of therandomly restarted fits are very close.
The above example only utilized one tweak constant value for the oldand new parameters. In general, Gallant and Tauchen recommended thata variety of tweak constants of various magnitudes be used to thoroughly
explore the surface of the SNP objective function. To accommodate thisrecommendation, the arguments fOld, fNew, and n.start can be vectors.
For example, to implement a wave of random restarts using the collectionof tweak constants specified by Gallant and Tauchen (2001, p. 27), use> fOld = c(0, 1e-5, 0, -1e-5, 1e-5, 1e-5, 1e-4, 0, -1e-4,+
1e-4, 1e-4, 1e-3, 0, -1e-3, 1e-3, 1e-3, 1e-5, 0,+-1e-2, 1e-2, 1e-2, 1e-1, 0, -1e-1, 1e-1, 1e-1,+
1e0, 0, -1e0, 1e0, 1e0)
13 On Windows systems, the trace information may be printed to the screen all at
once at the end of the restart procedure. This occurs because of the way screen output
is buﬀered in the Windows environment.

22.4 SNP Model Selection

885> fNew = c(0, 0, 1e-5, 1e-5, -1e-5, 1e-5, 0, 1e-4, 1e-4,+-1e-4, 1e-4, 0, 1e-3, 1e-3, -1e-3, 1e-3, 0,+
1e-2, 1e-2, -1e-2, 1e-2, 0, 1e-1, 1e-1, -1e-1,+
1e-1, 0, 1e0, 1e0, -1e0, 1e0)
> n.start = c(0,rep(25,30))
The vectors fOld and fNew specify 31 diﬀerent tweak constant combination.
The vector n.start, which is the same length as fOld and fNew, specifies
how many random restarts are to be performed for each combination of
tweak constants. Notice that the first set of tweak constants are zero andthere are no random restarts. This corresponds to the SNP fit without arandom restart. To estimate the SNP model using the above list of restart
parameters use> fit.10310000.mruns = SNP(dmRet.dat[,"exch"],+model=SNP.model(ar=1, arch=3),+
control = SNP.control(n.start = n.start,+
seed = 011667, fOld = fOld, fNew = fNew),+
trace = T)
Run 1 , 0 starts, fOld = 0 fNew = 0
Iteration No. 1, objective = 1.52188
Iteration No. 2, objective = 1.38215...
Iteration No. 9, objective = 1.34811
Run 2 , 25 starts, fOld = 1e-005 fNew = 0
random start 1, objective = 1.34811
random start 2, objective = 1.34811...
Run 31 , 25 starts, fOld = 1 fNew = 1
random start 1, objective = 1.34811
random start 2, objective = 1.34811...
random start 25, objective = 1.48451
Iteration No. 1, objective = 1.34811
With the above restart specification, the SNP model is fit 1 + 30 × 25 = 751
times.14 The best fitting model is given by> fit.10310000.mruns
Call:
SNP(data = dmRet.dat[, "exch"], model = SNP.model(ar = 1,
14 Estimation of these 751 models takes about 20 s on a 1.6-GHz Pentium M underWindows XP Pro.

886

22. Seminonparametric Conditional Density Models

arch = 3), control = SNP.control(n.start = n.start,
seed = 11667, fOld = fOld, fNew = fNew), trace = F)
Minimum BIC (out of 31 runs) occurred for control parameters
fOld fNew n.start1
025
Model: Gaussian ARCH
Conditional Mean Coefficients:
mu ar(1)0.0059 0.1336
Conditional Variance Coefficients:
s0 arch(1) arch(2) arch(3)0.5078 0.2631 0.1707
0.218
Information Criteria:
BIC
HQ
AIC
logL
1.3724 1.3619 1.3553 -1118.931
Convergence Type:relative function convergence
number of iterations: 1
With multiple runs of random restarts, the print method displays the
optimal tweak constants (fOld=1, fNew=0). The fit information from eachrun is contained in the component runs, which is a list with components> names(fit.10310000.mruns$runs)[1] "nruns" "best""detail" "coef"
The best fit occurs for run 27,> fit.10310000.mruns$runs$best[1] 27and the information for this run is> fit.10310000.mruns$runs$detail[27,]fOld fNew n.start
BIC
HQ
AIC
logL
1
0
25 1.372405 1.361883 1.355339 -1118.931

22.4.2 The expand Function
The function expand is designed to allow a user to easily expand an existing
SNP model toward a larger model. The arguments expected by expand are22.4 SNP Model Selection887> args(expand)function(x, ar = 0, arch = 0, garch = 0, lagP = 0, zPoly = 0,
xPoly = 0, trimZ = 0, trimX = 0, control = NULL, trace= F)
The argument x denotes a fitted "SNP" object from which the model expansion is to take place. The arguments ar, arch, garch, lagP, zPoly, xPoly,
trimZ, and trimX give the desired increments to the SNP tuning parameters of the initial SNP model that define the new, expanded SNP model.
For example, to expand from a Gaussian AR(3)-ARCH(1) to a semiparametric AR(4)-ARCH(4) with Kz = 4 the increment specification would be
ar=1, arch=3, and zPoly=4. The expand function calls the SNP function
using the coeﬃcients of the supplied "SNP" object as starting values, and it
expands the SNP model according to the specified increments in a manner
similar to that described in the previous subsection. By default, the expand
function uses the control information from the supplied "SNP" object. The
result of expand is an object of class "SNP".
To illustrate, consider expanding the initial Gaussian AR(1) SNP model
fit to the spot return data to a Gaussian AR(1)-ARCH(3) model. The
Gaussian AR(1) model is represented by the "SNP" object fit.10010000.
To expand this model to an AR(1)-ARCH(3), use> fit.10310000 = expand(fit.10010000,arch=3)> class(fit.10310000)[1] "SNP"> fit.10310000
Call:
SNP(data = dmRet.dat[, "exch"], model = list(ar = 1, arch = 0,
garch = 0, lagP = 1, zPoly = 0, xPoly = 0, trimZ = 0,
trimX = 0), n.drop = 14,coef = c(1., 0.000508918280817768,
0.0222361334180125, 1.00252754954042), est = c(0, 1, 1, 1),
iArch = 3, trace = F)
Model: Gaussian ARCH
Conditional Mean Coefficients:
mu ar(1)0.008 0.1362
Conditional Variance Coefficients:
s0 arch(1) arch(2) arch(3)0.5059 0.2655 0.1764 0.2152
Information Criteria:
BIC
HQ
AIC

logL

888

22. Seminonparametric Conditional Density Models

1.3735 1.3629 1.3563 -1106.182
Convergence Type:relative function convergence
number of iterations: 9
The result of expand is an object of class "SNP" representing the expanded
SNP fit. In the Call information, notice that the initial model is specified as an AR(1) and that the starting values for the initial model (old)parameters are set to the estimated coeﬃcients from the AR(1) model.
The expansion to the AR(1)-ARCH(3) is indicated through the increment
argument iArch=3.
When expanding a SNP fit, Gallant and Tauchen (2001) stressed that
random restarts be used to avoid getting stuck at local minima. When using
the expand function, a random restart procedure may be specified in two
ways. The first way is to specify a random restart procedure for the initial
SNP model. This restart information, contained in the control componentof the "SNP" object, will then be passed on to the expand function andused. The second way is to directly specify the control information passed
to the expand function. For example, in the "SNP" object fit.10010000
representing the AR(1) fit, there is no restart information in the control
component:> fit.10010000$control$fold[1] 0> fit.10010000$control$fnew[1] 0> fit.10010000$control$nstart[1] 0
The vectors of restart information from the previous section may be inserted into the control component as follows:> control.new = fit.10010000$control> control.new$fold = fOld
> control.new$fnew = fNew
> control.new$nstart = n.start
The expanded model fit with random restarts may be obtained using> fit.10310000.mruns = expand(fit.10010000,arch=3,+ control=control.new,trace=T)
The expand function may be used to conduct a specific-to-general model
selection strategy for determining the best fitting SNP model. For eachexpanded model, the model selection criteria AIC, BIC, and HQ may be
used to select the best fitting model. This information is available in thecomponent obj of an "SNP" object. For example, a comparison of the model22.4 SNP Model Selection889

selection criteria for the AR(1) and AR(1)-ARCH(3) models may be obtained using> fit.10010000$obj - fit.10310000.mruns$obj
BIC
HQ
AIC
logL
0.0613315 0.06654345 0.06978232 -46.66811
The AR(1)-ARCH(3) is preferred to the AR(1) using all of the model selection criteria.

22.4.3 The SNP.auto Function
The function SNP.auto implements Gallant and Tauchen’s (2001) suggested model selection strategy for determining the best fitting univariate
SNP model by minimizing the BIC. The arguments expected by SNP.auto
are> args(SNP.auto)function(data, n.drop = NULL, control = SNP.control(), trace= T, arMax = 4, zPolyMax = 8, xPolyMax = 4, lagPMax= 4)
The first four arguments are the same as those expected by the SNP function. The remaining arguments, arMax, zPolyMax, xPolyMax, and lagPMax,
correspond to the maximum orders of Lu , Kz , Kx , and Lp , respectively,
for the prospective SNP models. The search for the best fitting SNP model
follows a restricted specific-to-general model expansion path based on thespecified maximum orders of a subset of SNP model parameters. Eachmodel is fit to the same span of data and the BIC is computed.15 The
initial SNP model is a simple Gaussian location model. The SNP model
is first expanded (using the expand function with increments of 1) towarda Gaussian VAR model with maximum lag determined by arMax. Then,the Gaussian VAR model is expanded to a GARCH(1,1) model. Next, thezt part of the polynomial P(zt , xt−1 ) is sequentially expanded up to the
maximum order set by zPolyMax and then the xt−1 part of the polynomial P(zt , xt−1 ) is sequentially expanded up to the maximum order set by
xPolyMax. Finally, the number of lags that go into the xt−1 part of thepolynomial P(zt , xt−1 ) is sequentially expanded up to the maximum order
set by lagPMax. The SNP model with the lowest value of BIC is returned
as an object of class "SNP".
To illustrate, consider using SNP.auto with the default parameters tofind the best SNP model preferred by BIC for the spot return data:> fit.auto = SNP.auto(dmRet.dat[,"exch"], n.drop=14)
15 The user must set the argument n.drop so that suﬃcient presample observations
are available to accommodate the maximum lags specified by arMax and lagPMax.

890

22. Seminonparametric Conditional Density Models

Initializing using a Gaussian model ...
Expanding the order of VAR: 1234
Expanding toward GARCH model ...
Expanding the order of z-polynomial: 12345678
Expanding the order of x-polynomial: 1234
> class(fit.auto)[1] "SNP"
By default, SNP.auto displays information about the order of the expansion
path on screen. The best fitting model is a semiparametric GARCH(1,1):> fit.auto
Call:
SNP(data = dmRet.dat[, "exch"], model = list(ar = 0, arch = 1,
garch = 1,lagP = 1, zPoly = 3, xPoly = 0, trimZ = 0,
trimX = 0), n.drop = 14, coef = c(1., -0.13265782318379,
0.00224280476347416, 0.0189674305327692, 0.122336259918009,
0.0624215245663479, 0.171626674286156, 0.808804423426396),
est = c(0, 1, 1, 1, 1, 1,1, 1), iZPoly = 1, trace = F)
Model: Semiparametric GARCH
Hermite Polynomial Coefficients:
z^0
z^1
z^2
z^3
z^4
1 -0.097 -0.1068 0.018 0.0227
Conditional Mean Coefficients:mu
0.0618
Conditional Variance Coefficients:
s0 arch(1) garch(1)0.037 0.1515
0.8343
Information Criteria:
BIC
HQ
AIC
logL
1.3437 1.3295 1.3207 -1074.972
Convergence Type:relative function convergence
number of iterations: 16
By default, SNP.auto does not utilize random restarts. To incorporate
random restarts at each stage of the expansion path, the user needs to supply restart information in the control argument to SNP.auto. For exam-

22.5 SNP Model Diagnostics

891

ple, to repeat the above estimation using Gallant and Tauchen’s suggested
restart information, use16> fOld = c(0, 1e-5, 0, -1e-5, 1e-5, 1e-5, 1e-4, 0, -1e-4,+
1e-4, 1e-4, 1e-3, 0, -1e-3, 1e-3, 1e-3, 1e-5, 0,+-1e-2, 1e-2, 1e-2, 1e-1, 0, -1e-1, 1e-1, 1e-1,+
1e0, 0, -1e0, 1e0, 1e0)
> fNew = c(0, 0, 1e-5, 1e-5, -1e-5, 1e-5, 0, 1e-4, 1e-4,+-1e-4, 1e-4, 0, 1e-3, 1e-3, -1e-3, 1e-3, 0,+
1e-2, 1e-2, -1e-2, 1e-2, 0, 1e-1, 1e-1, -1e-1,+
1e-1, 0, 1e0, 1e0, -1e0, 1e0)
> n.start = c(0,rep(25,30))> fit.auto.restart = SNP.auto(dmRet.dat[,"exch"],n.drop=14,+
control = SNP.control(n.start = n.start,+
seed = 011667, fOld = fOld, fNew = fNew))
The search procedure employed by SNP.auto is not an exhaustive searchof all possible models and may not produce the model that globally minimizes BIC. As mentioned earlier, the BIC has a tendency to select models
with Kx = 0 in situations in which one would expect Kx > 0. As a result,
it is recommend that a battery of residual and graphical diagnostics be run
on any tentative model to confirm the adequacy of the model. The next
section describes how to do this.
The function SNP.auto can be used on multivariate data, but it is notrecommended to do so. The default search procedure may not produce satisfactory results for a variety of reasons. In particular, the default DVECGARCH(1,1) model imposes no restrictions on the ARCH/GARCH parameters. Moreover, the default values for trimZ and trimX allow all interactions among the elements of zt and xt−1 , respectively, in the Hermite
polynomial P(zt , xt−1 ).

22.5 SNP Model Diagnostics
Although model selection criteria may be used to determine the apparent
best fitting SNP model, it is still important to perform diagnostic checks
on the fitted model to verify that the chosen model is adequate and appropriate. This is particularly important when the fitted SNP model is to
be used as a score generator (auxiliary model) for EMM estimation. In
this context, for the EMM estimation to work well, it is imperative that
the SNP density capture all of the salient features (correlation structure,
conditional heteroskedasticity, asymmetry, fat tails, etc.) of the observed
data. Two types of diagnostic check should be routinely conducted: residual
16 Since each model in the expansion is restarted 751 times, this may take a while!

892

22. Seminonparametric Conditional Density Models

analysis, and SNP model simulation. The S+FinMetrics implementationof SNP provides convenient method functions to facilitate this diagnostic
analysis.

22.5.1 Residual Analysis
Table 22.4 shows the generic extractor functions for "SNP" objects available in S+FinMetrics. To illustrate these functions, consider the Gaussian
AR(1)-GARCH(1,1) SNP object fit.1111000 computed from the spot return data. To extract the estimated coeﬃcients, use> coef(fit.11110000)
Conditional Mean Coefficients:
mu ar(1)0.018 0.0911
Conditional Variance Coefficients:
s0 arch(1) garch(1)0.0783 0.1881
0.7822
Note that S-PLUS allows “lazy evaluation” so that coef() may be usedinstead of coefficients().17 To return the residuals, use> resid.11110000 = residuals(fit.11110000)> resid.11110000[1:3]
Positionsexch
04/18/1975 0.09765122
04/25/1975 -0.10085499
05/02/1975 0.04326546
Since underlying data, dmRet.dat, is a "timeSeries" object, the residuals
are returned as a "timeSeries" object. The standardized residuals (residuals divided by an estimate of the conditional standard deviation) may also
be extracted by passing the optional argument standardized:> resid.11110000.std = residuals(fit.11110000,stardard=T)
The estimated conditional standard deviations may be extracted directly
using the extractor function sigma.t.SNP:> sig.11110000 = sigma.t.SNP(fit.1111000)
A comparison of these estimated conditional standard deviations with asimple nine-period moving average of absolute log returns, computed using
17 Similarly, fitted() may be used instead of fitted.values() and resid() may be
used instead of residuals().

22.5 SNP Model Diagnostics

893

0.5

1.0

1.5

2.0

2.5

3.0

Moving average of absolute log returns

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1987

1988

1989

1990

1991

0.6

0.8

1.0

1.2

1.4

1.6

1.8

conditional standard deviations from SNP fit

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

FIGURE 22.5. Nine-period moving average of absolute log returns and estimated conditional standard deviations from the Gaussian AR(1)-GARCH(1,1)
SNP model.

Functioncoefficients
residuals
fitted.values
sigma.t.SNP

Description
Extract the estimated coeﬃcients
Extract the residuals
Extract the fitted values
Extract the estimated conditional standard deviations
TABLE 22.4. SNP extractor functions> par(mfrow=c(2,1))> plot(SMA(abs(dmRet.dat[15:834,"exch"])),+
main="Moving average of absolute log returns",+
reference.grid=F)> plot(sqrt(sig.11110000),+
main="conditional standard deviations from SNP fit",+reference.grid=F)> par(mfrow=c(1,1))
is shown in Figure 22.5.
For multivariate models, the sigma.t.SNP extracts the estimated conditional standard deviations associated with each series. To extract conditional covariances, set the optional argument cov=T.

894

22. Seminonparametric Conditional Density Models

If the fitted SNP model adequately describes the conditional densityof the observed data, then the standardized residuals should resemble a
Gaussian white noise process. A variety of diagnostic functions for testing
if a series behaves like Gaussian white noise are available in S-PLUS and
S+FinMetrics. Chapter 3 reviews some of these functions.
Graphical diagnostics (correlograms, QQ-plots, histograms, etc.) may be
conducted on the extracted residuals from an "SNP" object. For example,
to plot the standardized residuals from the AR(1) SNP model for the spot
return, use> plot(residuals(fit.11110000,stardard=T),reference.grid=F)> abline(h=0)
Alternatively, the plot method for "SNP" objects may be used to automatically generate a number of useful diagnostic plots for residual analysis. To
display a menu of available plots, type> plot(fit.11110000)
Make a plot selection (or 0 to exit):
1: plot: All
2: plot: Response and Fitted Values
3: plot: SNP Marginal Density
4: plot: Std. Residuals
5: plot: Normal QQplot of Std. Residuals
6: plot: ACF of Std. Residuals
7: plot: PACF of Std. Residuals
8: plot: ACF of Squared Std. Residuals
9: plot: PACF of Squared Std. Residuals
Selection:
For example, selecting 4 gives a trellis plot of the standardized residuals
shown in Figure 22.6.
To create plots directly without using the menu, set the optional argument ask=F. For example, to create an ACF plot of the squared residuals,
use18> plot(fit.11110000,ask=F,which.plots=7)
This plot, displayed in Figure 22.7, shows that the Gaussian AR(1)GARCH(1,1) has captured the autocorrelation in the conditional heteroskedasticity present in the spot return data.
For multivariate data, the diagnostic plots produced by the plot method
adjust accordingly. To illustrate, consider the bivariate Gaussian VAR(1)18 Note that the number associated with the plot is one less than the number displayedin the menu of plot choices.

22.5 SNP Model Diagnostics

895

Std. Residuals versus Time

-3

-2

-1

0

1

2

3

4

dmRet.dat[, "exch"]

1976

1977

1978

1979 1980

1981

1982

1983

1984

1985 1986

1987 1988

1989

1990 1991

FIGURE 22.6. Standardized residuals from a Gaussian AR(1) SNP model forweekly spot returns.
Std. Residual^2 ACF

dmRet.dat[, "exch"]
1.0

0.8

ACF

0.6

0.4

0.2

0.0

0

5

10

15

20

25

30

Lag

FIGURE 22.7. SACF of standardized squared residuals from a Gaussian AR(1)SNP model fit to weekly spot returns.

896

22. Seminonparametric Conditional Density Models

Std. Residuals versus Time

-5

0

5

forward

-4

-2

0

2

4

exch

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991

FIGURE 22.8. Standardized residuals from a bivariate Gaussian VAR(1) SNP
model fit to weekly spot returns and forward discounts.

SNP model fit to the exchange rate data. Figures 22.8 and 22.9 show theplots of the standardized residuals and correlograms for the squared standardized residuals produced using> plot(bfit.10010000,ask=F,which.plots=c(3,7))

22.5.2 Simulation
In addition to examining the residuals from a fitted SNP model, one should
also examine the properties of simulated data from a fitted SNP model. If
the SNP density truly captures all of the features of the observed data,then simulated data from the model should also reflect these features. Further, as emphasized in Andersen and Lund (1997), if the fitted SNP model
is to be used in conjunction with EMM, it is important to check the dynamic stability of the SNP model. For complicated SNP models, it may benearly impossible to determine if the model is dynamically stable by examining the estimated coeﬃcients. A simple way to check dynamic stability
is to generate long simulations from the fitted model and observe if thesesimulations become explosive.
Once an SNP model has been fit, simulated data of length equal to theobserved data from the fitted model can be generated using the generic22.6 Prediction from an SNP Model

897

Std. Residual^2 ACF
0

5

10

exch

15

20

25

30

forward

1.0

0.8

ACF

0.6

0.4

0.2

0.0

0

5

10

15

20

25

30

Lag

FIGURE 22.9. SACF of squared standardized residuals from a bivariate Gaussian
VAR(1) SNP model fit to weekly spot returns and forward discounts.

simulate function. For example, to simulate from the Gaussian AR(1)SNP model fit to the spot return data, use> sim.10010000 = simulate(fit.10010000)
The simulated return data along with the actual return data are illustrated
in the top two panels of Figure 22.10. Clearly, the data simulated from theGaussian AR(1) does not reflect the marked conditional heteroskedasticity
in the actual return data. This can be further verified by comparing thecorrelograms of the squared returns for the two series. In contrast, thebottom panel of Figure 22.10 shows that simulations from a semiparametricAR(1)-GARCH(1,1) mimic the actual data quite well.

22.6 Prediction from an SNP Model
Once an SNP model has been fit, h-step-ahead predictions, along withestimated confidence bands, may be computed using the generic predict
function. For example, to compute h-step-ahead predictions (h = 1, . . . , 10)
from the Gaussian AR(1)-GARCH(1,1) model for the spot return, use> predict.11110000 = predict(fit.11110000,n.predict=10)> class(predict.11110000)

898

22. Seminonparametric Conditional Density Models

-6

2

8

US/DM spot returns

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1987

1988

1989

1990

1991

1988

1989

1990

1991

-4

0 2 4

Simulated returns from Gaussian AR(1) SNP model

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

-10

-2

6

Simulated returns from semi-parametric AR(1)-GARCH(1,1) SNP model

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

FIGURE 22.10. Actual returns and simulated returns from a Gaussian AR(1)SNP model and semiparametric AR(1)-GARCH(1,1) SNP model.

[1] "forecast"
The object returned by predict is of class "forecast", which has print,
summary, and plot methods.19 The summary method shows the forecasts
along with estimated standard errors:> summary(predict.11110000)
Predicted Values with Standard Errors:
prediction std.err
1-step-ahead 0.0870
0.12602-step-ahead 0.0863
0.1338...
10-step-ahead 0.0882
0.1495The plot method allows the forecasts and user specified confidence bands
to be plotted along with the original data. For example, to plot the spot
return forecasts appended to the actual data along with 95% confidence
bands use
19 Objects of class "forecast" are defined and used in S+FinMetrics. See the online
help for forecast.object.

22.7 Data Transformations

899

-2

-1

values

0

1

exch

825

830

835

840

index

FIGURE 22.11. h-step-ahead forecasts from a Gaussian AR(1)-GARCH(1,1) SNP
model.

> plot(predict.11110000,dmRet.dat[,"exch"],n.old=10,width=2)
The resulting plot is illustrated in Figure 22.11. The optional arguments
n.old=10 and width=2 specifies that the forecasts are appended to the 10
most recent observations and that the confidence band has width equal totwo times the estimated forecast standard error.

22.7 Data Transformations
A number of data transformations may be employed to improve the stability of the fitted SNP models. These transformations are described in thefollowing subsections. The S-PLUS code is structured so that the transformations are transparent to the user and all input and output is in the unitsof the untransformed data.

22.7.1 Centering and Scaling Transformation
To improve the stability of computation, the original data for SNP estimation are centered and scaled to have mean zero and identity covariancematrix. This location-scale transform is accomplished by computing esti-

900

22. Seminonparametric Conditional Density Models

mates of the unconditional mean and variance of the original data:ȳ = 1/T

T
X

yt

t=1

S = 1/TTX(yt − ȳ)(yt − ȳ)0
t=1

The SNP model is then estimated using the standardized data:ỹt = S−1/2 (yt − ȳ)(22.19)where S−1/2 denotes the Cholesky factor of S−1 . To aid interpretation of
results with multivariate data, it is often useful to make S diagonal before the standardization (22.19) is employed. Because of the location-scaletransform (22.10) employed in SNP estimation itself, the statistical properties of the SNP estimates are not aﬀected by the standardization (22.19).
When predictions, simulations, or other information about yt is produced
from the SNP estimates, the location-scale transformation is reversed sothat the desired quantities relate to the actual data.
Sometimes it may be necessary to use external estimates of ȳ and S
instead of the defaults to standardize the data or it may not be necessary to standardize the data. Within the SNP function, the standardization (22.19) is controlled using the optional list argument LStransform.
The components of LStransform are mu, cstat, and pstat, which correspond to ȳ, S1/2 , and S−1/2 , respectively. By default, LStransform=NULL
and the standardization (22.19) is performed with S diagonal. To perform the standardization with S nondiagonal, set the SNP.control optional argument diagS=F. To turn oﬀ the standardization, call SNP withLStransform=list(mu=0, cstat=diag(M), pstat=diag(M)), where M is
the number of columns in the data argument.
For example, to estimate an AR(1) model for the spot return data withno standardization, use> fit.ar1 = SNP(dmRet.dat[,"exch"], model=SNP.model(ar=1),+
n.drop=14,LStransform=list(mu=0,cstat=1,pstat=1))> coef(fit.ar1)
Conditional Mean Coefficients:
mu ar(1)0.0547 0.0222
The above result agrees with an AR(1) model estimated using the
S+FinMetrics function OLS:
> fit.OLS = OLS(exch~tslag(exch), data=dmRet.dat[14:834,])> coef(fit.OLS)

22.7 Data Transformations

901(Intercept) tslag(exch)
0.0546925 0.02223613
To estimate a bivariate VAR(1) for the exchange rate data with no standardization, use> fit.var1 = SNP(dmRet.dat[,1:2],model=SNP.model(ar=1),+
n.drop=14,+
LStransform=list(mu=c(0,0),cstat=diag(2),+pstat=diag(2)))> coef(fit.var1)
Conditional Mean Coefficients:
exch forward
mu 0.3012 0.0094
var(1;1) 0.0087 -0.8338
var(1;2) -0.0026 0.9671
The VAR(1) fit using the S+FinMetrics function VAR is> fit.VAR = VAR(cbind(exch,forward)~ar(1),+data=dmRet.dat[14:834,])> print(coef(fit.VAR), digits=2)
exch forward(Intercept) 0.3012 0.0094
exch.lag1 0.0087 -0.0026
forward.lag1 -0.8338 0.9671
Notice that the coeﬃcients are arranged diﬀerently in the VAR output.

22.7.2 Transformations to Deal with Heavy Tailed Data
Time series data, particularly financial market data, often contain extremeor outlying observations. This is not necessarily a problem when the extreme value is considered as a value for yt , because it just fattens thetails of the estimated conditional density. However, once it becomes a lag
and passes into xt−1 , the optimization algorithm can use an extreme value
in xt−1 to fit an element of yt nearly exactly, thereby reducing the corresponding conditional variance to near zero and inflating the likelihood.
This problem is endemic to procedures that adjust variance on the basis ofobserved explanatory variables.
The SNP function allows two methods to alleviate the problem of extreme
values in yt . First, for heavy-tailed, mean-reverting data such as daily stock
market returns, the following logistic transformation of the variables in xt−1
may be employed:x̂i = 4σ tr

exp{xi /σ tr }− 2σ tr i = 1, . . . , M · L
1 + exp{xi /σ tr }(22.20)22. Seminonparametric Conditional Density Models

4

902

-4

-2

0

2

spline
logit

-6

-4

-2

0

2

4

6

x

FIGURE 22.12. Logistic and spline transformations with σtr = ±2.

where xi denotes an element of xt−1 . This transformation has a negligible
eﬀect on values of xi between −σ tr and σ tr , but progressively compresses
values that exceed ±σ tr . Second, for highly persistent data (such as interest
rates) that have a strong ARCH/GARCH component, it can happen thatan estimated SNP density will generate explosive simulations. This createsproblems if a fitted SNP model object is to be used as an auxiliary model(score) for the EMM estimation. To deal with this problem, the SNP function allows the following spline transformation of xt−1 as an alternative tothe logistic transformation:⎧ 1
xi < −σ tr⎨ 2 [xi − σ tr − ln(1 − xi − σ tr )] forxi
for −σtr < xi < σ tr (22.21)
x̂i =
⎩ 1
for
σ tr < xi
2 [xi + σ tr + ln(1 − σ tr + xi )]Figure 22.12 illustrates the logistic and spline transformations. Because
the logistic and spline transformations only aﬀect x, not y, the asymptotic
properties of SNP estimator is not aﬀected.
To use the logistic transformation in SNP estimation, set the optional
argument xTransform of SNP.control to "logistic". For example, toestimate a semiparametric AR(1) model for the spot return with a logistictransformation, use> fit.10014000.logit = SNP(dmRet.dat[,"exch"],+model=SNP.model(ar=1,zPoly=4), n.drop=14,

22.7 Data Transformations+

903

control=SNP.control(xTransform="logistic"))

Since the original data are usually standardized before applying the SNP
estimation, the inflection point σtr of the logistic transformation is set
to two by default. To change the default value of σ tr , pass the optional
argument inflection to SNP.control:> fit.10014000.logit2 = SNP(dmRet.dat[,"exch"],+model=SNP.model(ar=1,zPoly=4), n.drop=14,+
control=SNP.control(xTransform = "logistic",+
inflection=2.5))
To use the spline transformation, set the optional argument xTransformof SNP.control to "spline". By default, the inflection point σ tr for the
spline transformation is set to 4. To change the default value of the inflection point, simply set the optional argument inflection to the desired
value. To illustrate the importance of the spline transformation for highly
persistent data with a strong ARCH/GARCH component, consider fitting
an AR(1)-ARCH(4) model to the weekly 3-month U.S T-bill data in the"timeSeries" object tbill.dat with and without a spline transformation:> tb3mo = tbill.dat[,"tb3mo"]> fit.204100 = SNP(tbill.dat[,"tb3mo"],model=SNP.model(ar=2,+
arch=4), n.drop=6)> fit.204100.s = SNP(tbill.dat[,"tb3mo"],model=SNP.model(ar=2,+
arch=4), n.drop=6,+
control=SNP.control(xTransform="spline",+
inflection=4))
Simulations from the two fits, along with the T-bill data, are given in Figure22.13. Notice that the untransformed fit generates explosive simulations,whereas the spline transformed fit does not.

22.7.3 Transformation to Deal with Small SNP Density
ValuesIf the fitted SNP model is used in conjunction with the EMM estimation,
it may happen that the estimated SNP density is smaller than the smallest
value that the FORTRAN DLOG function can evaluate at simulated values
generated by the EMM function. To avoid potential numerical problems, the
SNP density may be replaced byª© 2 −1
P [Rx (y − μx ), x] + ε0 nM (y|μx , Ωx )∗R
f (y|x, θ) =[P (s, x)]2 φ(s)ds + ε0
where the user sets the value of ε0 . The default value suggested by Gallant
and Tauchen (2001) is ε0 = 0.001. This value is set by the SNP.control
optional argument eps0.

904

22. Seminonparametric Conditional Density Models

4

8

12 16

T-bill data

1965

1970

1975

1980

1985

1990

1995

1990

1995

1990

1995

-250

-50

simulated data from fitted SNP 204100 model

1965

1970

1975

1980

1985

-20

0

20

simulated data from SNP 204100 model with spline transform

1965

1970

1975

1980

1985

FIGURE 22.13. Weekly 3-month U.S. T-bill rates and simulations from a SNP
AR(1)-ARCH(4) model without and with spline transformation.

22.8 Examples
The following subsections illustrate the process of fitting an SNP model tosome common univariate financial time series. The fitting process makes
use of the model selection strategy and residual diagnostics detailed in
the previous sections. The final models will be used as score generators(auxiliary models) for the EMM examples in the next chapter.20

22.8.1 SNP Models for Daily Returns on Microsoft Stock
Data
The data are daily continuously compounded returns, based on closing
prices, on Microsoft stock over the period March 14, 1986 through June 30,
2003. There are 4365 observations. Similar data, ending in February 2001,
was analyzed in Gallant and Tauchen (2002). Figure 22.14 shows the data
along with the sample ACF of the squared returns, and Figure 22.15 shows
four distribution summary plots. Some summary statistics, computed using
the S+FinMetrics function summaryStats, are
20 The "SNP" objects for these models are included with S+FinMetrics 2.0.

22.8 Examples

905

-0.30

-0.10

0.10

Daily returns on Microsoft stock

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

0.6
0.4
0.0

0.2

ACF

0.8

1.0

Series : msft.ts^2

0

10

20

30

Lag

FIGURE 22.14. Daily log returns on Microsoft and SACF of squared returns.

> summaryStats(msft.ts)
Sample Quantiles:min
1Q median
3Q
max-0.3012 -0.01274
0 0.01567 0.1957
Sample Moments:
mean
std skewness kurtosis
0.0016 0.02526 -0.2457
11.66
Number of Observations:

4365

From Figures 22.14 and 22.15, it is clear that the returns exhibit considerably conditional heteroskedasticity as well as a few large outliers. The
conditional transition density is likely to be non-Gaussian but symmetric.
SNP Model Selection
For heavy-tailed mean-reverting financial return data, it is recommended
to use the logistic transformation (22.20) when fitting the SNP models. In
the model selection strategy with financial return data, it is recommended
to initially increment Kz from 0 to 4. Financial return data have thicktailed densities relative to the Gaussian, and only a quartic term in Kz
can capture this. The Hermite polynomial P(z) has to “move” the mass

22. Seminonparametric Conditional Density Models

0.0

returns

4
0

-0.3

2

-0.1

6

8

0.1

0.2

10

906

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.2
0.0
-0.1

returns

0.1

10 12 14
8
6
4
0

-0.3

2

density estimate

returns

-0.3

-0.2

-0.1

0.0

returns

0.1

0.2

-2

0

2

Quantiles of Standard Normal

FIGURE 22.15. Distribution diagnostics for daily log returns on Microsoft.

of the Gaussian density around. It has to increase the mass around zero,depress the mass somewhat on either side of zero, and then increase themass in the tails by going to ±∞. Linear, quadratics, and cubics in P(z)cannot do this, but a quartic can. In fitting SNP models, it is possible that
incrementing Kz by 1 starting from Kz = 0 and moving up to Kz = 4
produces inferior fits. The reason is that a model with Kz = 1 is not agood model for the density of the return data, and the fit may involve an
area of the parameter space that is far away from SNP models with larger
values of Kz . It is not recommended to fit SNP models with Kz > 8. For
these models, P(z) generally only captures little wiggles and the nonlinear
fits can become somewhat unstable. If the density is roughly symmetric, itis preferred to increase Kz from 4 in increments of 2, as the even powersof P (z) are more important.
Based on the above discussion, the following model selection strategy isentertained to find the best fitting SNP model for the MSFT daily returns:
10010000 →→→→→

11110000
11114000
11116000 → 11116010 → 11116020
11118000
1121600022.8 Examples

907

The initial Gaussian AR(1) SNP model is fit using the random restart
parameters> fOld = c(0, 1e-5, 0, -1e-5, 1e-5, 1e-5, 1e-4, 0, -1e-4,+
1e-4, 1e-4, 1e-3, 0, -1e-3, 1e-3, 1e-3, 1e-5, 0,+-1e-2, 1e-2, 1e-2, 1e-1, 0, -1e-1, 1e-1, 1e-1,+
1e0, 0, -1e0, 1e0, 1e0)
> fNew = c(0, 0, 1e-5, 1e-5, -1e-5, 1e-5, 0, 1e-4, 1e-4,+-1e-4, 1e-4, 0, 1e-3, 1e-3, -1e-3, 1e-3, 0,+
1e-2, 1e-2, -1e-2, 1e-2, 0, 1e-1, 1e-1, -1e-1,+
1e-1, 0, 1e0, 1e0, -1e0, 1e0)
> n.start = c(0,rep(25,30))> fit.msft.10010000 = SNP(msft.ts, model=SNP.model(ar=1),+
control = SNP.control(xTransform="logistic",+n.start=n.start,fOld=fOld,+
fNew=fNew),+n.drop=14, trace=T)
Although it is not necessary to use random restarts with the initial model,
it will become essential to use them for more complicated models. The
remaining models in the model selection strategy are fit using the expand
function as follows:> fit.msft.11110000 = expand(fit.msft.10010000, arch=1,+
garch=1, trace=T)> fit.msft.11114000 = expand(fit.msft.11110000, zPoly=4)> fit.msft.11116000 = expand(fit.msft.11114000, zPoly=2)> fit.msft.11116010 = expand(fit.msft.11116000, xPoly=1)> fit.msft.11116020 = expand(fit.msft.11116010, xPoly=1)> fit.msft.11216000 = expand(fit.msft.11116000, arch=1)> fit.msft.12116000 = expand(fit.msft.11116000, garch=1)> fit.msft.11118000 = expand(fit.msft.11116000, zPoly=2)
The BIC values for the fitted SNP models are summarized in Table 22.5.
According to the BIC, the best fitting SNP models are the semiparametric
GARCH(1,1) with Kz = 6. The same specification was chosen by Gallant
and Tauchen (2002). A summary of this SNP fit is> summary(fit.msft.11116000)
Model: Semiparametric GARCH
Hermite Polynomial Coefficients:
z^0
z^1
z^2
coef
1.0000 -0.0189 -0.3336(std.err)
0.03020.0172(t.stat)-0.6254 -19.4181

z^3
0.01380.00971.4262z^40.05580.004312.9964

908

22. Seminonparametric Conditional Density Models

SNP Model
10010000
11110000
11114000
11116000
11116s00
11118000
11116010
11116020
11216000
12116000

Lu
1
1
1
1
1
1
1
1
1
1

Lg
0
1
1
1
1
1
1
1
1
2

Lr
0
1
1
1
1
1
1
1
2
1

Lp
1
1
1
1
1
1
1
1
1
1

Kz
0
0
4
6
6
8
6
6
6
6

Iz
0
0
0
0
0
0
0
0
0
0

Kx
0
0
0
0
0
0
1
2
0
0

Ix
0
0
0
0
0
0
0
0
0
0

BIC
1.42161.36511.3394
1.33771.3354
1.33901.3434
1.3487
1.3387
1.3383

TABLE 22.5. BIC values for SNP models fit to Microsoft returns

coef(std.err)(t.stat)z^5-0.00120.0008-1.5228z^6-0.00250.0003-8.7763

Conditional Mean Coefficients:muar(1)coef -0.0020 0.0186(std.err) 0.0318 0.0159(t.stat) -0.0612 1.1660
Conditional Variance Coefficients:
s0 arch(1) garch(1)coef 0.0679 0.1630 0.8615(std.err) 0.0073 0.0135 0.0099(t.stat) 9.2548 12.1146 86.9439
Notice that only the coeﬃcients on the even powers of z in the Hermite
polynomial P(z) are statistically diﬀerent from zero. This suggests refitting
the 11116000 model with the added restriction that the coeﬃcients on theodd powers of z are equal to zero
> fit.msft.11116s00 = SNP(data = msft.ts,+
model = SNP.model(ar=1, arch=1, garch=1, zPoly=6),+
control = SNP.control(xTransform="logistic",+n.start=n.start,+
fOld=fOld, fNew=fNew),+n.drop = 14, trace=T,+
coef = c(1,0,0,0,0,0,0,0,0,1,0,0),+
est = c(0,0,1,0,1,0,1,1,1,1,1,1))This restricted model, denoted 11116s00, gives the overall lowest BIC value.

22.8 Examples

909

The function SNP.auto may also be used to find the best fitting SNP
model:> fit.msft.auto = SNP.auto(msft.ts, n.drop=14,+
control = SNP.control(xTransform="logistic",+n.start = n.start, seed = 011667,+
fOld = fOld, fNew = fNew))
Here, the best fitting model is a semiparametric GARCH(1,1) with Kz = 7,01117000. It has a BIC value equal to 1.3373, which is slightly lower than
the 11116000 model but slightly higher than the restricted 11116000 model.
SNP Model Diagnostics
From the analysis in the previous section, the SNP 11116000 and 11116s00
models are determined to be the best fitting SNP models for the daily
returns on Microsoft stock. The estimated SNP density for the 11116000
model, compared to a normal density, is shown in Figure 22.16. As expected, the SNP density is roughly symmetric and has fatter tails than
the normal. The sample ACF and PACF plots of the standardized residuals and squared standardized residuals (not shown) indicate no remaining
serial correlation.
Simulations from the SNP 11116000 model are computed using> sim.msft.11116000 = simulate(fit.msft.11116000)
The simulated data, shown in Figure 22.17, mimics the actual data quite
well.

22.8.2 SNP Models for Daily Returns on the S&P 500 Index
Data
The data consist of daily continuously compounded returns, based on closing prices, on the S&P 500 index over the period March 14, 1986 throughJune 30, 2003 (same time span as the data for the Microsoft returns). Similar data were analyzed in Gallant and Tauchen (2002). Figure 22.18 shows
the data along with the sample ACF of the squared returns, and Figure22.19 shows four distribution summary plots. Sample summary statistics
are> summaryStats(sp500.ts)
Sample Quantiles:
min
1Q
median3Q
max-0.2047 -0.004686 0.0004678 0.005827 0.09099
Sample Moments:
mean
std skewness kurtosis

910

22. Seminonparametric Conditional Density Models

SNP

Normal

0

5

10

y

15

20

25

msft.ts

-0.05

0.0

0.05

x

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

FIGURE 22.16. Estimated conditional density from the SNP 11116000 model fit.

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

FIGURE 22.17. Simulated returns from the SNP 11116000 model fit.

22.8 Examples

911

-0.20

-0.10

0.00

Daily returns on S&P 500 index

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

0.6
0.4
0.0

0.2

ACF

0.8

1.0

Series : sp500.ts^2

0

10

20

30

Lag

FIGURE 22.18. Daily log returns on the S&P 500 index and SACF of squared
returns.

0.0003916 0.01124-1.486
Number of Observations: 4365

32.59

The S&P 500 returns appear to be highly non-normal, with negative skewness and very large kurtosis, and display considerable conditional heteroskedasticity.
SNP Model Selection
A model selection strategy similar to that used for the Microsoft returns
is followed. However, the S&P 500 index is expected to display richer dynamics than any single stock. The manual search strategy employed is10010000 → 11110000→ 11114000→ 11116000→ 11118000 → 11118010 → 11118020
Table 22.6 reports the BIC values for the above models using the
S+FinMetrics functions SNP and expand with random restarts.
The BIC preferred model is the semiparametric GARCH(1,1) model with
Kz = 8, 11118000. A summary of the fit is> summary(fit.sp500.11118000)

912

22. Seminonparametric Conditional Density Models

Model: Semiparametric GARCH
Hermite Polynomial Coefficients:
z^0
z^1
z^2
coef
1.0000 -0.0938 -0.4103(std.err)
0.03170.0239(t.stat)-2.9543 -17.1575

coef(std.err)(t.stat)z^5-0.0049
0.0036-1.3710z^6-0.00820.0014-5.7050z^70.00010.00020.5784z^30.03560.01871.8996z^40.09250.00989.4081z^80.00020.0001
3.8843

Conditional Mean Coefficients:
mu ar(1)coef 0.1032 0.0202(std.err) 0.0232 0.0162(t.stat) 4.4547 1.2451
Conditional Variance Coefficients:
s0 arch(1) garch(1)coef
0.01220.16200.9105(std.err)
0.00320.00720.0049(t.stat)3.8641 22.6131 184.9564
Information Criteria:
BIC
HQ
AIC
logL
1.2426 1.2365 1.2331 -5352.208
The best fitting model using SNP.auto> fit.sp500.auto = SNP.auto(sp500.ts,n.drop=14,+
control = SNP.control(xTransform="logistic",+n.start = n.start, seed = 011667,+
fOld = fOld, fNew = fNew))is an SNP 01118000 model, which is very similar to the 11118000 model
but has a slightly higher BIC value:> fit.sp500.auto$obj[,"BIC"]
1.243061
Gallant and Tauchen (2002) reported the results from a specification
search and suggested the SNP models 20b14000 and 20b14010, where b
represents Lr = 11. When these models are fit to the updated S&P 500
data, using an expansion path starting from a Gaussian AR(2) with ran-913

returns

10
0

-0.20

5

-0.10

15

0.0

20

25

0.10

22.8 Examples

-0.20

-0.15

-0.10

-0.05

0.0

0.05

0.10

returns

-0.10

0.0

20
15
10
0

-0.20

5

density estimate

25

0.10

returns

-0.2

-0.1

0.0

0.1

-2

returns02

Quantiles of Standard Normal

FIGURE 22.19. Distribution summary from daily log returns on the S&P 500
index.

SNP Model
10010000
11110000
11114000
11116000
11118000
11118010
11118020

Lu
1
1
1
1
1
1
1

Lg
0
1
1
1
1
1
1

Lr
1
1
1
1
1
1
1

Lp
1
1
1
1
1
1
1

Kz
0
0
4
6
8
8
8

Iz
0
0
0
0
0
0
0

Kx
0
0
0
0
0
1
2

Ix
0
0
0
0
0
0
0

BIC
1.42181.27981.2492
1.24791.2426
1.2465
1.2481TABLE 22.6. BIC values for SNP models fit to the SP 500 returns.

914

22. Seminonparametric Conditional Density Models
SNP

Normal

0

10

20

y

30

40

50

data2.SP

-0.02

0.0

0.02

x

FIGURE 22.20. Estimated conditional density from the SNP 11118000 model fit.

dom restarts, the BIC values are 1.2597 and 1.2598, respectively. The SNP
11118000 model still dominates in terms of BIC.

SNP Model Diagnostics
From the analysis in the last subsection, the SNP 11118000 model is determined to be the best fitting SNP model for the daily returns on the S&P
500 index. The estimated SNP density, shown in Figure 22.20, is roughly
symmetric and has fatter tails than the normal. Simulated data from thefitted 11118000 model, along with the SACF of the squared data, are depicted in Figure 22.21. These plots confirm that the fitted 11118000 model
adequately captures the salient features of the S&P 500 daily returns.

22.8.3 SNP Models for Weekly 3-Month U.S. T-Bill Rates
Data
The data are weekly (Friday) observations on the 3-month U.S. Treasury
Bill rate (annualized rates times 100) over the period January 5, 1962
through March 31, 1995. This data has been analyzed by Gallant andTauchen (2002) to illustrate the EMM estimation of a two factor interest

22.8 Examples

915

-0.12

-0.04

0.04

Simulated returns from SNP model

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

0.6
0.4
0.0

0.2

ACF

0.8

1.0

Series : sim.sp.11118000^2

0

10

20

30

Lag

FIGURE 22.21. Simulated returns and SACF of squared returns from the SNP
11118000 model fit.

rate diﬀusion model21 . The data, obtained from ftp://ftp.duke.econ, arein the first column of the S+FinMetrics "timeSeries" object tbill.dat> colIds(tbill.dat)[1] "tb3mo" "tb12mo" "tb10yr"
and are labeled "tb3mo". The data and sample ACF are displayed in Figure22.22. These plots show that interest rates are highly persistent and that thevolatility of rates increases with the level of rates. A simple AR(1) model
is often used to characterize the conditional mean. Summary statistics ofthe residuals from a fitted AR(1) to the interest rate data are> resid.ar1 = residuals(OLS(tb3mo~tslag(tb3mo),
data=tbill.dat, na.rm=T))> summaryStats(resid.ar1)
Sample Quantiles:
min
1Q
median3Q
max-2.156 -0.07646 -0.006411 0.08537 1.852
21 These data are similar to the data used by Andersen and Lund (1997). They used
weekly (Wednesday) observations on the 3-month U.S. T-bill over the period 1954 to1995.

22. Seminonparametric Conditional Density Models

4

6

8

10

12

14

16

916

1965

1970

1975

1980

1985

1990

1995

0.6
0.4
0.0

0.2

ACF

0.8

1.0

Series : tb3mo

0

5

10

15

20

25

30

Lag

FIGURE 22.22. Weekly observations on U.S. 3-month T-bill rate and SACF.

Sample Moments:
mean
std skewness kurtosis
1.465e-015 0.2667-0.925
16.89
Number of Observations:

1734

These statistics and the distribution summary plots in Figure 22.23, showthat the innovation to the one-step-ahead conditional density based on an
AR(1) model of interest rates is highly non-normal.
SNP Model Selection
Andersen and Lund (1997), Tauchen (1997), McManus and Watt (1999),
Jensen (2001), and Gallant and Tauchen (2002) fitted SNP models toweekly interest rates. For highly persistent data like interest rates, it isrecommended to use the spline transformation (22.20) when fitting the
SNP models. In addition, as stressed by Andersen and Lund (1997), Gallant and Long (1997), and Tauchen (1997, 1998), for interest rate data it isimportant to check the dynamic stability of any candidate SNP model thatis to be used as a score generator for the EMM estimation. Dynamic stability may be checked visually by plotting long simulations from the fitted
SNP model.

22.8 Examples

SNP Model

Lu

Lg

11110000
11114000
11116000
11118000
11118010
11118020

1
1
1
1
1
1

1
1
1
1
1
1

10110000
10210000
10310000
10410000
10510000
10610000
10710000
10810000
10814000
10815000
10816000
10817000
10818000
10818010
10818020

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

Lr Lp Kz
GARCH Models
1
1
0
1
1
4
1
1
6
1
1
8
1
1
8
1
1
8
ARCH Models
1
1
0
2
1
0
3
1
0
4
1
0
5
1
0
6
1
0
7
1
0
8
1
0
8
1
4
8
1
5
8
1
6
8
1
7
8
1
8
8
1
8
8
1
8

Iz

Kx

Ix

BIC

0
0
0
0
0
0

0
0
0
0
1
2

0
0
0
0
0
0

-1.4071-1.4524-1.4482-1.4603-1.4474-1.4320

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
1
2
0
0
0
0
0
0
0
1
2

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

-1.1700-1.2261-1.3104-1.3406-1.3554-1.3622-1.3631-1.3678-1.4217-1.4196-1.4174-1.4234-1.4289-1.4197-1.4044TABLE 22.7. BIC values for SNP models fit to weekly U.S. T-bill rates

917

0.0

0
-2

-1

percent times 100

0.8
0.6
0.4
0.2

1

2

22. Seminonparametric Conditional Density Models

1.0

918

-2

-1

0

1

2

-2

-1

0

tb3mo.mat

1.0
0.5
0.0

density estimate

1

1.5

2

percent times 100

-2

-1

0

1

2

percent times 100

-2

0

2

Quantiles of Standard Normal

FIGURE 22.23. Distribution summary of weekly returns on U.S. 3-month T-billrate.

Two types of manual model selection strategy are followed. In the firststrategy, an expansion path based on a GARCH(1,1) leading term is used.
This strategy produces the overall BIC minimizing SNP model. In thesecond strategy, following Gallant and Tauchen (2002), only pure ARCH
models with Lr ≤ 4 are considered. In both cases, the best fitting initial
Gaussian AR model has Lu = 5, which seems rather high. In fact, for both
strategies, SNP models with Lu = 1 lead to better overall fits as measured
by BIC.
The BIC values for a subset of the models fit with the S+FinMetrics
functions SNP and expand using the above strategies are given in Table 22.7. For the first strategy, the best fitting model is a semiparametric AR(1)-GARCH(1,1) with Kz = 8, 11118000. For the second strategy,
the best fitting model is a semiparametric AR(1)-ARCH(8) with Kz = 8,10818000. Gallant and Tauchen (2002) reported their preferred dynamically stable SNP model as a nonlinear nonparametric AR(1)-ARCH(4),10414010. The BIC values for these models are -1.4031 and −1.4113, respectively. Finally, the BIC minimizing model found using SNP.auto is asemiparametric AR(5)-GARCH(1,1) with Kz = 8. The BIC for this modelis −1.4594, which is slightly higher than the BIC for the 11118000 model.
A summary of the SNP 11118000 model fit is> summary(fit.tb3mo.11118000)

22.8 Examples

919

Model: Semiparametric GARCH
Hermite Polynomial Coefficients:
z^0
z^1
z^2
coef
1.00000.0440 -0.4300(std.err)
0.03690.0355(t.stat)1.1933 -12.1030

coef(std.err)(t.stat)z^50.00140.00330.4359z^6-0.01170.0016-7.1523z^7-0.0001
0.0002-0.3864z^3-0.00910.0204-0.4445z^40.12000.01398.6598z^80.00040.0001
6.9470Conditional Mean Coefficients:
mu
ar(1)
coef
-0.0034
0.9960(std.err)
0.00170.0013(t.stat)-1.9517 753.3579
Conditional Variance Coefficients:
s0 arch(1) garch(1)coef 0.0010 0.2095 0.8842(std.err) 0.0001 0.0201 0.0093(t.stat) 7.0564 10.3975 94.7220
Information Criteria:
BIC
HQ
AIC
logL
-1.4603 -1.4732 -1.4808 2561.52
Convergence Type:relative function convergence
number of iterations: 59

SNP Model Diagnostics
Tauchen (1997) and Gallant and Tauchen (2002) stressed that the BIC
minimizing SNP model may not be the best choice for weekly interest
rate data, especially if the SNP model is to be used as an auxiliary modelfor the EMM estimation. The above analysis identifies four potential SNP
models: 11118000, 10818000, 51118000, and 10414010. The standard residual diagnostics are similar for these models. All of the models except the5111800 model show slight autocorrelation in the standardized residuals.
Figure 22.24 shows simulated values and estimated conditional volatilities

920

22. Seminonparametric Conditional Density Models

11118000 model

-10

0.2

30

1.0

11118000 model

1965

1970

1975

1980

1985

1990

1995

1965

1970

1980

1985

1990

1995

1985

1990

1995

1985

1990

1995

1985

1990

1995

10818000 model

0.2

-20

1.0

20

10818000 model

1975

1965

1970

1975

1980

1985

1990

1995

1965

1970

1980

51118000 model

-10

0.2

1.0

35

51118000 model

1975

1965

1970

1975

1980

1985

1990

1995

1965

1970

1980

10414010 model

0

0.2

1.0

10 20

10414010 model

1975

1965

1970

1975

1980

1985

1990

1995

1965

1970

1975

1980

FIGURE 22.24. Simulations (left column) and estimated conditional volatility(right column) from fitted SNP models for weekly U.S. T-bill rates.

from the four models. All models appear to be dynamically stable and have
similar fitted conditional volatilities.

22.9 References
Andersen, T.G. and J. Lund (1997). “Estimating Continuous-Time
Stochastic Volatility Models of the Short-Term Interest Rate,” Journal of
Econometrics, 77, 343-377.
Bansal, R., A.R. Gallant, R. Hussey and G. Tauchen (1994). “Nonparametric Estimation of Structural Models for High-Frequency Currency
Market Data,” Journal of Econometrics, 66, 251-287.
Bollerslev, T. (1986). “Generalized Autoregressive Conditional Heteroskedasticity,” Journal of Econometrics, 31, 307-327.
Coppejans, M. and A.R. Gallant (2002). “Cross-Validated SNP Density Estimates,” Journal of Econometrics, 110, 27-65.
Davidian, M. and R.J. Carroll (1987). “Variance Function Estimation,” Journal of the American Statistical Association, 82, 1079-1091.

22.9 References

921

Ding, Z., C.W.J. Granger and R.F. Engle (1993). “A Long Memory
Property of Stock Market Returns and a New Model,” Journal of Empirical
Finance, 1, 83-106.
Engle, R.F. (1982). “Autoregressive Conditional Heteroskedasticity With
Estimates of the Variance of U.K. Inflation,” Econometrica, 50, 987-1008.
Engle, R.F. (1995). ARCH: Selected Readings. Oxford University Press,
Oxford.
Engle, R.F. and G. González-Rivera (1991). “Semiparametric ARCH
Models,” Journal of Business and Economic Statistics, 9(4), 345-359.
Fenton, V.M. and A.R. Gallant (1996a). “Convergence Rates of SNP
Density Estimators,” Econometrica, 64, 719-727.
Fenton, V.M. and A.R. Gallant (1996b). “Qualitative andAsymptotic Performance of SNP Density Estimators,” Journal of Econometrics, 74, 77-118.
Gallant, A.R. and J.R. Long (1997). “Estimating Stochastic Diﬀerential Equations Eﬃciently by Minimum Chi-Square,” Biometrika, 84, 125141.
Gallant, A.R. and D.W. Nychka (1987). “Seminonparametric Maximum Likelihood Estimation,” Econometrica, 55, 363-390.
Gallant, A.R., D.A. Hsieh and G. Tauchen (1991). “On Fitting a
Recalcitrant Series: The Pound/Dollar Exchange Rate, 1974-83,” in W.A.
Barnett, J. Powell and G. Tauchen (eds.), Nonparametric and Semiparametric Methods in Econometrics and Statistics. Cambridge University Press,
Cambridge.
Gallant, A.R. and G. Tauchen (1989). “Seminonparametric Estimation of Conditionally Constrained Heterogeneous Processes: Asset Pricing
Applications,” Econometrica, 57, 1091-1120.
Gallant, A.R. and G. Tauchen (1999). “The Relative Eﬃciency of
Method of Moments Estimators,” Journal of Econometrics, 92, 149-172.
Gallant, A.R. and G. Tauchen (2001). “SNP: A Program for Nonparametric Time Series Analysis, Version 8.8, User’s Guide,” Working Paper,
University of North Carolina at Chapel Hill.
Gallant, A.R. and G. Tauchen (2001b). “Eﬃcient Method of Moments,” unpublished manuscript, Department of Economics, University ofNorth Carolina.

922

22. Seminonparametric Conditional Density Models

Gallant, A.R. and G. Tauchen (2002). “Simulated Score Methods
and Indirect Inference for Continuous-time Models,” forthcoming in Y.
Ait-Sahalia, L.P. Hansen and J. Schienkman (eds.), Handbook ofFinancial Econometrics. North-Holland, Amsterdam.
Jensen, M.B. (2001). “Eﬃcient Method of Moments Estimation of theLongstaﬀ and Schwartz Interest Rate Model,” unpublished manuscript,
Department of Business Studies, Aalborg University, Denmark.
Lütkepohl, H. (1990). Introduction to Multiple Time Series Analysis.
Springer-Verlag, New York.
McManus, D. and D. Watt (1999). “Estimating One Factor Modelsof Short-Term Interest Rates,” Working Paper 99-18, Bank of Canada,
Ottawa, Canada.
Schwert, W. (1990). “Stock Volatility and the Crash of ’87,” Review of
Financial Studies, 3(1), 77-102.
Tauchen, G. (1997). “New Minimum Chi-Square Methods in Empirical
Finance,” in Advances in Econometrics, Seventh World Congress, D. Kreps
and K. Wallis (eds.), Cambridge University Press, 279-317.
Tauchen, G. (1998). “The Objective Function of Simulation Estimators
Near the Boundry of the Unstable Region of the Parameter Space,” Review
of Economics and Statistics, 80, 389-398.
Taylor, S. (1986). Modelling Financial Time Series. John Wiley & Sons,
New York.

This is page 923
Printer: Opaque this23
Eﬃcient Method of Moments

23.1 Introduction
Dynamic nonlinear models that have unobserved variables pervade science.1 Most often they arise from dynamic systems described by a systemof deterministic or stochastic diﬀerential equations in which the state vector is partially observed. For example, in epidemiology, the SEIR model
determines those susceptible, exposed, infected, and recovered from a disease, whereas usually data are from case reports that report only those
infected (Olsen and Schaﬀer, 1990). Other examples are continuous- anddiscrete-time stochastic volatility models of speculative markets from finance (Ghysels, Harvey, and Renault, 1996), general equilibrium models
from economics (Gennotte and Marsh, 1993), and compartment models
from pharmacokinetics (Mallet, Mentré, Steimer, and Lokiec, 1988).
Standard statistical methods, both classical and Bayesian, are usually
not applicable in these situations either because it is not practicable toobtain the likelihood for the entire state vector or because the integration
required to eliminate unobservables from the likelihood is infeasible. On
a case-by-case basis, statistical methods are often available. However, thepurpose here is to describe methods that are generally applicable.
Although determining the likelihood of a nonlinear dynamic system thathas unobserved variables is often infeasible, simulating the evolution of the
1 Sections 1 and 2 are based on Gallant and Tauchen (2001) by permission of theauthors.

924

23. Eﬃcient Method of Moments

state vector is often quite practicable. The eﬃcient method of moments(EMM) methodology described in this chapter relies on this.
Briefly, the steps involved in EMM are as follows: Summarize the data
by using quasi-maximum-likelihood to project the observed data onto atransition density that is a close approximation to the true data generatingprocess. This transition density is called the auxiliary model and its scoreis called the score generator for EMM. The SNP model described in theprevious chapter provides a convenient general-purpose auxiliary modelin this connection. Once a score generator is in hand, given a parametersetting for the system, one may use simulation to evaluate the expectedvalue of the score under the stationary density of the system and computea chi-squared, or generalized method of moments (GMM)-type, criterion
function. A nonlinear optimizer is used to find the parameter setting thatminimizes the criterion.
If the auxiliary model encompasses the true data generating process, thenquasi maximum likelihood estimates become suﬃcient statistics and EMM
is fully eﬃcient (Gallant and Tauchen, 1996). If the auxiliary model is a
close approximation to the data generating process, then one can expect
the eﬃciency of EMM to be close to that of maximum likelihood (Gallant
and Long, 1997; Tauchen, 1997). Because EMM is a GMM-type estimator, diagnostic tests are available to assess system adequacy as well as aregraphics that suggest reasons for failure.
Due to the fundamental importance of diﬀusion models for stock pricedynamics, there has been significant progress recently using alternative simulation strategies; see Brandt and Santa-Clara (1999), Durham and Gallant(2002), Elerian, Chib, and Shephard (2001), and the references therein.
Despite this recent progress, the alternatives to the simulation methods
discussed here are not as general purpose and are limited in their ability
to deal with latent variables.
This chapter is organized as follows. Section 23.2 gives an overview of
the EMM methodology based on Gallant and Tauchen (2001). Section 23.3
describes the implementation of EMM in S+FinMetrics. Section 23.4 gives
detailed examples of EMM estimation of moving average models for time series, discrete-time stochastic volatility models for asset returns, and continuous-time models for interest rates.
Eﬃcient method of moments has found several recent applications, andthe following is a selected list mainly from finance. Andersen and Lund
(1997), Dai and Singleton (2000), and Ahn, Dittmar, and Gallant (2002)
use the method for interest rate applications. Liu (2000), Andersen, Benzoni, and Lund (2002), and Chernov, Gallant, Ghysels, and Tauchen (2003)
use it to estimate stochastic volatility models for stock prices with suchcomplications as long memory and jumps. Chung and Tauchen (2001) use
it to estimate and test target zone models of exchange rates. Jiang andvan der Sluis (2000) use it to price options. Valderrama (2001) employs
it for a macroeconomic analysis and Nagypal (2001) employs it in a labor23.2 An Overview of the EMM Methodology

925

economics application. An excellent survey of EMM and related methods is
given in Gallant and Tauchen (2005). The S+FinMetrics implementation of
EMM described in this chapter is based on the public-domain FORTRAN
code developed by Ronald Gallant and George Tauchen and summarized
in Gallant and Tauchen (2002).2

23.2 An Overview of the EMM Methodology
The following subsections, based on Gallant and Tauchen (2001), give ageneral overview of the EMM methodology, and draw connections with the
GMM and maximum likelihood (ML) methodologies. The EMM methodology is first motivated by the problem of estimating a continuous-time
stochastic volatility model for short-term interest rates. Minimum chisquare estimators are then defined, and their eﬃciency properties are compared. The SNP model is introduced next as a general-purpose auxiliarymodel, and then the steps of the EMM methodology are described.

23.2.1 Continuous-Time Stochastic Volatility Model forInterest Rates
The data used to illustrate ideas are weekly observations on the 3-month
U.S. T-bill rate, from January 1962 through March 1995, yielding 1,735
raw observations.3 Figure 23.1 is a plot of the data. The series {yt } is theannualized return on the 3-month T-bill.
The finance literature normally treats interest rates as a diﬀusion, usually
expressed as a system of stochastic diﬀerential equations. The data shownin Figure 23.1 would thus be regarded as having resulted from discretely
sampling a diﬀusion.
An example is the following two-factor mean-reverting continuous-time
stochastic volatility model:
dX1t
dX2t

γ
= κ1 (μ − X1t )dt + exp(X2t /2)X1t
dW1t , γ > 0= κ2 (α − X2t )dt + ξdW2t(23.1)

where X1t is observable at time t and represents the instantaneous shortterm interest rate. The second component X2t is not observable and represents a factor that aﬀects the volatility of the process. The variables
dW1t and dW2t are increments to standard Brownian motion, as discussed
in Chapter 20. Versions of this model without an unobservable volatility
factordXt = κ(μ − Xt )dt + σXtγ dWt , γ > 0
2 The code is available at ftp.econ.duke.edu in directory pub/get/emm.
3 These data were analyzed in the previous chapter.

23. Eﬃcient Method of Moments

4

6

8

10

12

14

16

926

1965

1970

1975

1980

1985

1990

1995

FIGURE 23.1. Weekly observations on 3-month U.S. T-bill rate.

have a long history in financial economics.
In matrix notation, the system (23.1) is
dXt = A(Xt ) dt + B(Xt ) dWt

0≤t<∞(23.2)

where
Xt
Wt

µ

¶
X1t
,
X2t
µ
¶
W1t
=
,
W2t
=

A(X) =
B(X) =µµ¶κ1 (μ − X1t ),κ2 (α − X2t )γexp(X2t /2)X1t
0

0
ξ

¶

The process is discretely sampled so that the data available for analysis are
yt = 100 ∗ (X1,t − X1,t−1 ),t = 1, 2, ...

(23.3)

which corresponds directly to the interest rate series plotted in Figure 23.1.
The parameters of the system areρ = (κ1 , μ, γ, κ2 , α, ξ)(23.4)

As is well known since Lo (1988), the likelihood of the observed process
yt given ρ under the system dynamics
dXt = A(Xt ) dt + B(Xt ) dWt(23.5)

23.2 An Overview of the EMM Methodology

927

is not readily available in closed form. This aspect of the problem motivatesthe method of moments estimation and, in particular, simulated method of
moments as in Ingram and Lee (1991) and Duﬃe and Singleton (1993) andthe essentially equivalent indirect inference method proposed by Gourieroux, Monfort, and Renault (1993) and Smith (1993).
The system dynamics (23.5) suggest that, given a value for X 0 , one could
simulate an increment X ∆ −X 0 from the process {X t : 0 ≤ t < ∞} for asmall time value ∆ as follows: Generate two independent normal (0, ∆2 )variates x1 and x2 , simulate the Brownian motion increment W∆ − W0 by
puttingµ¶x1W∆ − W0 =x2
and simulate the increment X ∆ −X 0 by puttingX∆ − X0 = A(X0 )∆ + B(X0 ) (W∆ − W0 )To simulate a value for X t , sum over the increments:
Xt − X0

=

t/∆
X
¡
¢
Xi∆ − X(i−1)∆(23.6)

i=1

=

t/∆
X

A(X(i−1)∆ )∆ +

i=1

t/∆
X
i=1

¡
¢
B(X(i−1)∆ ) Wi∆ − W(i−1)∆

In passing, it is noted that under regularity conditions (Karatzas andPt/∆
Shreve, 1991), as ∆ decreases, the random variable i=1 B(X (i−1)∆ )(Wi∆ −W(i−1)∆ ) converges in mean square to a random variable that has many
Rtproperties of an integral and is therefore usually denoted as 0 B(X s ) dWs .
Rt
PSimilarly, the Reimann sum t/∆
i=1 A(X (i−1)∆ )∆ converges to 0 A(X s ) ds.
The process {X t : 0 ≤ t < ∞} is interpreted as the solution to the integral
equationZ
Zt

Xt − X0 =t

A(Xs ) ds +0

B(Xs ) dWs

0which exists under smoothness and growth conditions on the functions
A(X) and B(X) (Karatzas and Shreve, 1991).
The important feature of this example, and of all the applications thatare considered in this chapter, is that it can be simulated; that is, given a
value for the parameters of the model, it is straightforward to generate asimulation {ŷt }N
t=1 of arbitrary length N .
The simulation scheme (23.6) is known as an Euler scheme. This scheme
and more sophisticated simulation schemes are discussed in Chapter 20,
and in Kloeden and Platen (1992).
If the model (23.1) is stationary at a given value for ρ, then a timeinvariant stationary densityp(y−L , . . . , y−1 , y0 |ρ)(23.7)

928

23. Eﬃcient Method of Moments

exists for any stretch (yt−L , . . . , yt−1 , yt ) of obsevables. If, in addition, (23.1)
is ergodic at ρ, then the expectation of a time-invariant (nonlinear) functiong(y−L , . . . , y−1 , y0 ) with respect to (23.7)
ZZEρ [g] = · · · g(y−L , . . . , y0 )p(y−L , . . . , y0 |ρ) dy−L · · · dy0
can be approximated as accurately as desired by averaging over a simulation, namelyN. 1 X
g(ŷt−L , . . . , ŷt−1 , ŷt )Eρ [g] =N t=1

Throughout, it is presumed that the requisite initial lags for this computation are primed via draws from the stationary distribution, which is usuallyaccomplished by letting the system run long enough for transients to die
out. For examples such as that above, one typically uses values of ∆ on theorder of 1/7 per week or 1/24 per day, a burn-in period of 1000 to 5000,
and values of N on the order of 50,000 to 100,000.
The ability to compute Eρ [g] for given ρ and arbitrary g(y−L , . . . , y−1 , y0 )means that model parameters can be computed by the method of momentsor minimum chi-squared, as discussed in the next subsection.

23.2.2 Minimum Chi-Squared Estimators
In general, nonlinear systems are considered that have the features of thestochastic volatility model (23.1) just described. Specifically, (i) for a parameter vector ρ in a parameter space R, the random variables determinedby the system have a stationary densityp(y−L , . . . , y−1 , y0 |ρ)(23.8)

for every stretch (yt−L , . . .yt ) and (ii) for ρ ∈ R, the system is easily simulated so that expectations
Z
Z(23.9)
Eρ [g] = · · · g(y−L , . . . , y0 )p(y−L , . . . , y0 | ρ) dy−L · · · dy0
can be approximated as accurately as desired by averaging over a longsimulation
N. 1 X
Eρ [g] =g(ŷt−L , . . . , ŷt−1 , ŷt )(23.10)N t=1Henceforth, {yt } will be used to denote the stochastic process determinednby the system, {ŷt }N
t=1 to denote a simulation from the system, {ỹt }t=1 todenote data presumed to have been generated by the system, and (y−L , . . .,y−1 , y0 ) to denote function arguments and dummy variables of integration.

23.2 An Overview of the EMM Methodology

929

The true value of the parameter vector of the system (23.8) is denoted byρo .
A classical method of moments estimator ρ̂n of ρo is implemented by (i)setting forth a moment function, such as
⎞
⎛
y0
− μ̃1
⎜ y02
− μ̃2 ⎟
⎟
⎜
⎟
⎜
..
⎟
⎜
.
⎟
⎜
⎜ y0k
− μ̃k ⎟⎟
(23.11)ψ̃ c (y−L , . . . , y−1 , y0 ) = ⎜⎜ y−1 y0 − γ̃(1) ⎟⎟⎜⎜ y−2 y0 − γ̃(2) ⎟
⎟
⎜
⎟
⎜
..
⎠
⎝
.
y−L y0 − γ̃(L)P
P
where μ̃j = n1 nt=1 ỹtj , γ̃(h) = n1 nt=1+h ỹt ỹt−h , (ii) computing the moment equationsZZ
mn (ρ) = Eρ [ψ c ] = · · · ψ̃ c (y−L , . . . , y0 )p(y−L , . . . , y0 | ρ) dy−L · · · dy0and (iii) attempting to solve the estimating equations
mn (ρ) = 0
for the system parameters ρ. If a solution ρ̂n can be found, then thatsolution is the method of moments estimate of the system parameters. As
indicated earlier, the moment equations will usually have to be computedby generating a long simulation {ŷt }N
t=−L from the system at parametersetting ρ and then averaging over the simulation:N. 1 X
mn (ρ) =ψ̃ (ŷt−L , . . . , ŷt−1 , ŷt )N t=1 cIf there are multiple roots of the estimating equations mn (ρ) = 0, a particular solution can be selected as the estimate using methods discussed in
Heyde and Morton (1998).
In the event that there is no solution to the estimating equations because,
for instance, the dimension of ψ c is larger than the dimension of ρ (sothat there are more equations than unknowns), then one must resort tominimum chi-squared estimation (Neyman and Pearson, 1928) as adapted
to dynamic models (GMM estimation) by Hansen (1982). The minimum
chi-squared, or GMM, estimator is obtained by using a nonlinear optimizer
to minimize a quadratic form in the moment equations. Specifically,ρ̂n = arg min mn (ρ)0 S̃−1 mn (ρ)ρ

930

23. Eﬃcient Method of Moments

where the matrix S̃ appearing in the quadratic form is a heteroskedasticity
and autocorrelation consistent (HAC) estimate of the variance of√nmn (ρ) and may be computed using the methods described in Section
21.3 of Chapter 21.
If yt is multivariate, that is,
yt = (y1,t , . . . , yM,t )0
then, instead of (23.11), the vector ψ c is comprised of the elements
MY(yi,0 )αi − μ̃αi=1(yi,0 )(yj,−h ) − γ ij (h)where
α = (α1 , . . . , αM )0 ,
n

μ̃α

=

γ ij (h) =αi ≥ 0,M

1 XY(ỹi,t )αi
n t=1 i=1n
1 Xỹi,t ỹj,t−h ,
n
t=1+h

0<

M
X
i=1

αi ≤ K0 ≤ i ≤ j ≤ M,

0≤h≤L

The use of method of moments together with simulation to estimate theparameters of dynamic models with unobserved variables has been proposed by Ingram and Lee (1991), Duﬃe and Singleton (1993), Gourieroux,
Monfort, and Renault (1993), Smith (1993), and others. The particularmethods discussed next are due to Gallant and Tauchen (1996).

23.2.3 Eﬃciency Considerations
The previous subsection described a minimum chi-squared (GMM) estimation strategy based on the moment function ψ̃ c that can be used toestimate system parameters ρ. There are two open questions with regard
to this estimator: What is the best choice of the moment function ψ̃? How
many moments should be included in ψ̃?
These questions will be considered in the simplest case where the randomvariables defined by the system (23.8) generate univariate independently
and identically distributed random variables {yt } with density p(y|ρ). The
ideas for the general case of a multivariate, non-Markovian, stationary system are the same, but the algebra is far more complicated (Gallant andLong, 1997). Nothing essential is lost by considering the simplest case.

23.2 An Overview of the EMM Methodology

931

Consider three moment functions ψ̃ c,n , ψ̃ p,n , and ψ̃ f,n that correspondto the Classical Method of Moments (CMM), Maximum Likelihood (ML),and Eﬃcient Method of Moments (EMM), respectively, defined as follows:⎞⎛Pny − n1P i=1 ỹi⎜ y 2 − 1 n (ỹi )2 ⎟
i=1
n
⎟
⎜
ψ̃ c,n (y) = ⎜
⎟
..
⎠
⎝
.
P
n
1
K
K
y − n i=1 (ỹi )∂log p(y|ρ̃n )∂ρ∂ψ̃ f,n (y) =log f (y|θ̃n )∂θψ̃ p,n (y) =where the exponent K that appears in ψ̃ c,n (y) is the degree of the largest
moment used in a method of moments application, the function f (y|θ)
that appears in ψ̃ f,n (y) is a density that closely approximates the true
data generating process in a sense made precise later, and the statistics ρ̃nand θ̃ n that appear in ψ̃ p,n (y) and ψ̃ f,n (y) aren1Xρ̃n = arg max
log p(ỹi |ρ),
ρ
n i=1
n

θ̃ n = arg max
θ1X
log f (ỹi |θ);n i=1

where ρ is of length pρ and θ is of length pθ ≥ pρ .
Note that each of the moment functions ψ̃ p,n , ψ̃ c,n , and ψ̃ f,n is in thenull space of the expectation operator corresponding
dish to ithe empiricalhitribution of the data, denoted as EF̃n ; that is, EF̃n ψ̃ p,n = EF̃n ψ̃ c,n =hiEF̃n ψ̃ f,n = 0. The method of moments is basically an attempt to do
the same for the model p(y|ρ); that is, the method of moments attempts
to find a ρ that puts one of these moment functions, denoted genericallyas ψ̃ n , in the null space of the expectation operator Eρ corresponding top(y|ρ).
In addition to computing ψ̃ n , one computeshi0S̃ = EF̃n ψ̃ n ψ̃ nOnce ψ̃ n and S̃ have been computed, the data are summarized and what
is referred to as “the projection step” is finished.
For the estimation, defineh imn (ρ) = Eρ ψ̃ n

932

23. Eﬃcient Method of Moments

If the dimensions of ρ and ψ̃ n (y) are the same, then usually the equations
mn (ρ) = 0 can be solved to obtain an estimator ρ̂n . For ψ̃ p,n , the solution
is the maximum likelihood estimator (Gauss, 1816; Fisher, 1912). For ψ̃ c,nwith K = pρ , it is the classical method of moments estimator (Pearson,
1894). For ψ̃ c,n with K > pρ , no solution exists and the moment functions
ψ̃ c,n are those of minimum chi-squared or generalized method of moments(Neyman and Pearson, 1928; Hansen, 1982) as customarily implemented.
As just noted, when K > pρ , then ψ̃ n cannot be placed in the null spaceof the operator Eρ for any ρ, because the equations mn (ρ) = 0 have nosolution. In this case, the minimum chi-squared estimator relies on the factthat under standard regularity conditions (Gallant and Tauchen, 1996) andchoices of ψ̃ n similar to the above, there is a function ψ o such thatlim ψ̃ n (y)n→∞=ψ o (y) a.s.

Eρo [ψ o ψ o0 ] a.s.
¡
£
¤¢
√
d
n mn (ρo ) → N 0, Eρo ψ o ψ o0
lim S̃

n→∞=where Eρo denotes expectation taken with respect to p(y|ρo ). For the threechoices ψ̃ p,n , ψ̃ c,n , and ψ̃ f,n of ψ n (y) above, the functions ψ op , ψ oc , and ψ ofgiven by this result are⎞⎛y − Eρo [y]⎜ y 2 − Eρo [y 2 ] ⎟⎟⎜ψ oc (y) = ⎜
⎟
..
⎠
⎝
.
y K − Eρo [y K ]ψ op (y) =∂log p(y|ρo )∂ρψ of (y) =∂log f (y|θ o )
∂θ

and

where
θ o = arg max : Eρo [log f (·|θ)]θWith these results in hand, ρ may be estimated by minimum chi-squared(GMM), viz.,ρ̂n = arg min : mn (ρ)0 S̃−1 mn (ρ)
ρ

and
where

´
³
√
d
−1
n(ρ̂n − ρo ) → N 0, (Co )£¤£¤Co = Eρo ψ op ψ o0 Eρo [ψ o ψ o0 ] −1 Eρo ψ o ψ o0p

23.2 An Overview of the EMM Methodology

Note that for any nonzero a ∈ Rpρ ,
£
¡
¢2
¤2
min Eρo a0 ψ op − ψ o0 b = Eρo a0 ψ op − a0 Co a ≥ 0b933(23.12)£¤2=
Expression (23.12) implies that a0 Co a cannot exceed Eρo a0 ψ op£¤
a0 Eρo ψ op ψ op 0 a and therefore the best achievable asymptotic variance of£¤−1
the estimator ρ̂n is Ipo = Eρo ψ op ψ op 0
, which is the variance of the maximum likelihood estimator of ρ. It is also apparent from £(23.12)
that if¤2o g{ψ oi }∞
spans
the
L
probability
space
L
=
{g
:
E
<
∞}
and
2,p
2,p
ρ
i=1
ψ o = (ψ o1 , . . . , ψ oK ), then ρ̂n has good eﬃciency relative to the maximum
likelihood estimator for large K. The polynomials span L2,p if p(y|ρ) has amoment generating function (Gallant, 1980). Therefore, one might expect
good asymptotic eﬃciency from ψ̃ c,n for large K.
Rather than just spanning L2,p , EMM requires, in addition, that the
moment functions actually be the score vector ψ f,n (y) of some density
f (y|θ̃n ) that closely approximates p(y|ρo ). Possible choices of f (y|θ̃ n ) are
discussed in Gallant and Tauchen (1996). Of them, one commonly used inapplications is the SNP density, which was proposed by Gallant and Nychka(1987) in a form suited to cross-sectional applications and by Gallant andTauchen (1989) in a form suited to time series applications.
The SNP density, discussed in detail in the previous chapter, is obtainedby expanding the square root of an innovation density h(z) in a Hermite
expansion∞Xpph(z) =
θi z i φ(z)i=0where φ(z) denotes the standard normal density function.
Because the Herp
mite functions are dense in L2 (Lebesque) and h(z) is an L2 function,
this expansion must exist. The truncated density is

where

hK (z) = R

2PK(z)φ(z)2
PK (u)φ(u) duPK (z) =KX

θi z ii=0

and the renormalization is necessary so that the density hK (z) integrates
to 1. The location-scale transformation y = σz + μ completes the definitionof the SNP density
µ
¶
y−μ
1
fK (y|θ) = hK
(23.13)σσwith θ = (μ, σ, θ0 , . . . , θK ). Gallant and Long (1997) have shown that
ψ of (y) =∂log fK (y|θ o )
∂θ

934

23. Eﬃcient Method of Momentswithθ o = arg max : Eρo [log fK (·|θ)]θspans L2,p .
Although a spanning argument can be used to show that high eﬃciency
obtains for large K, it gives no indication as to what might be the best
choice of moment functions with which to span L2,p . Moreover, if ψ p isin the span of ψ o for some finite K, then full eﬃciency obtains at once
(Gallant and Tauchen, 1996). For instance, the score of the normal density
is in the span of both ψ̃ c,n and ψ̃ f,n for K ≥ 2. These considerations seemto rule out any hope of general results showing that one moment function
should be better than another.
With general results unattainable, the best one can do is compare eﬃciencies over a class of densities designed to stress test an estimator andover some densities thought to be representative of situations likely to be
encountered in practice to see if any conclusions seem to be indicated. Comparisons using Monte Carlo methods are reported by Andersen, Chung, andSorensen (1999), Chumacero (1997), Ng and Michaelides (2000), and vander Sluis (1999). Overall, their work supports the conjecture that EMM ismore eﬃcient than GMM in representative applications at typical sample
sizes.
Analytical comparisons are possible for the independently and identically
distributed case and are reported in Gallant and Tauchen (1999). Theirmeasure of eﬃciency is the volume of a confidence region on the parametersof the density p(y|ρ) computed using the asymptotic distribution of ρ̂n .
This region has the form {ρ : (ρ − ρo )0 (Co )−1 (ρ − ρo ) ≤ Xd2 /n} withvolume2πd/2 (Xd2 /n)ddΓ(d/2) det(Co )where Xd2 denotes a critical value of the chi-squared distribution on d degrees of freedom. As small volumes are to be preferred and the region{ρ : (ρ − ρo )0 Ipo (ρ − ρo ) ≤ Xd2 /n} has the smallest achievable volume,RE =det(Co )det(Ipo )is a measure of relative eﬃciency. Over a large collection of densities thought
to represent typical applications, their computations support the conclusionthat EMM dominates GMM. Moreover, their computations indicate thatonce fK (·|θo ) begins to approximate p(·|ρo ) accurately, the eﬃciency of
the EMM estimator begins to increase rapidly.
The second question to address is how many moments to include in the
moment function ψ f . As the computations in Gallant and Tauchen (1999)
suggest, the answer is as many as is required for f to well approximate p.
The natural conclusion is that one should use standard statistical model

23.2 An Overview of the EMM Methodology

935

selection criteria to determine f . This approach has a distinct advantage
over the use of ψ c in that there seems to be no objective statistical criterion
for determining the number of moments to include in ψ c .

23.2.4 A General Purpose Auxiliary Model
As indicated in the previous subsection, the best choice of a moment function ψ to implement a simulated method of moments is the score of an
auxiliary model that closely approximates the density of the data. The
SNP methodology of Gallant and Tauchen (1989) extended to the timeseries context serves as a general-purpose auxiliary model or score generator for use with the EMM estimation of general dynamic systems. In
this context, the observed data are assumed to be a stationary and ergodic
Markovian multivariate time series {yt }Tt=−L+1 , where each yt is a vectorof length M . Since yt is Markovian, the conditional distribution of yt given
the entire past depends only on a finite number L of lagged values of yt
00
denoted xt−1 = (yt−1, . . . , yt−L
)0 . The SNP model is then used to approximate the transition density p(yt |xt−1 , ρo ). The SNP methodology and its
implementation in S+FinMetrics is described in the previous chapter.

23.2.5 The Projection Step
The best choice of a moment function to implement simulated method of
moments estimation of general dynamic systems is the score of an auxiliarymodel f (y|x, θ) that closely approximates the transition density impliedby the system, where the parameter vector θ of the auxiliary model isevaluated at its quasi-maximum-likelihood estimate θ̃ n ; that is, the best
choice has the formψ̃ f (x, y) =∂log f (y|x, θ̃n )∂θ

(23.14)

where
θ̃ n

= arg min sn (θ)

sn (θ) = −θnX(23.15)

1
log f (ỹt |x̃t−1 , θ)n t=1

A considerable advantage of closely approximating the transition densityof the system is that the computational formula for the weighting matrix
S̃ for the chi-squared (GMM) estimator simplifies to
n

0
1X
S̃ =
ψ̃ (x̃t−1 , ỹt )ψ̃ f (x̃t−1 , ỹt )n t=1 f(23.16)

936

23. Eﬃcient Method of Moments

because the covariance terms of the HAC estimator can be neglected when
the auxiliary model closely approximates the transition density of the system in a sense made precise by Gallant and Long (1997).

23.2.6 The Estimation Step
The objectives are (i) to estimate ρ, (ii) test the hypothesis that the dynamic system under consideration generated the observed data {ỹt }nt=1 ,and (iii) provide diagnostics that indicate how a rejected system should be
modified to better describe the distribution of the observable process {yt }.
The EMM Estimator
It is presumed that the data have been summarized in the projection step,
as described in the previous subsection, and that a moment function of theform∂ψ̃ f (x, y) =log f (y | x, θ̃ n )∂θ
and a weighting matrix
¸∙
¸0
n ∙
∂
1X ∂
S̃ =
log f (ỹt | x̃t−1 , θ̃ n )log f (ỹt | x̃t−1 , θ̃n )n t=1 ∂θ∂θare available from the projection step. Here, it is assumed that f (y | x, θ̃n )closely approximates p(y | x, ρo ). If not, the weighting matrix given by a
HAC estimator must be used. If the SNP density fK (y | x, θ) is used as theauxiliary model with tuning parameters selected by the Bayesian Information Criterion (BIC), S̃ as computed above will be adequate (Gallant andLong, 1997; Gallant and Tauchen, 1999; Coppejans and Gallant, 2002).
Here, it is explicitly indicated that the dependence on n of the moment
equations mn (ρ) enters through the quasi-maximum-likelihood estimateθ̃ n by writing m(ρ, θ̃n ) for mn (ρ), where∙¸∂m(ρ, θ) = Eρlog f (y | x, θ)∂θRecall that the moment equations of the minimum chi-squared procedure
are computed by averaging over a long simulation:N. 1 X ∂m(ρ, θ̃n ) =log f (ŷt | x̂t−1 , θ̃ n )N t=1 ∂θThe EMM estimator isρ̂n = arg min m(ρ, θ̃ n )0 S̃−1 m(ρ, θ̃ n )ρ23.2 An Overview of the EMM Methodology

937

Asymptotic Properties
The asymptotics of the estimator are as follows. If ρo denotes the true valueof ρ and θo is an isolated solution of the moment equations m(ρo , θ) = 0,then under regularity conditions (Gallant and Tauchen, 1996; Gallant andLong, 1997),
lim ρ̂n = ρo a.s.
n→∞

©
ª
√
d
n(ρ̂n − ρo ) → N 0, [Mo0 I o−1 Mo ]−1
lim M̂n = Mon→∞lim S̃ = I on→∞(23.17)a.s

a.s.

where M̂n = M(ρ̂n , θ̃ n ), Mo = M(ρo , θo ), M(ρ, θ) = (∂/∂ρ0 )m(ρ, θ),
and
¸∙
¸0
∙
∂
∂
o
o
o
I = Eρolog f (y0 | x−1 , θ )log f (y0 | x−1 , θ )∂θ∂θUnder the null hypothesis that p(y−L , . . . , y0 | ρ) is the correct model,
the EMM J-statistic
J0 = n m(ρ̂n , θ̃ n )0 S̃−1 m(ρ̂n , θ̃ n )(23.18)is asymptotically chi-squared on pθ − pρ degrees of freedom.
Hypothesis Tests
Under the null hypothesis that h(ρo ) = 0, where h maps R into <q , theEMM likelihood ratio (LR)-type statistichi
LREMM = n m(ρ̆n , θ̃ n )0 S̃−1 m(ρ̆n , θ̃ n ) − m(ρ̂n , θ̃ n )0 S̃−1 m(ρ̂n , θ̃n )(23.19)is asymptotically chi-squared on q degrees of freedom whereρ̆n = arg min m(ρ, θ̃n )0 S̃−1 m(ρ, θ̃ n ) s.t. h(ρ) = 0
ρ

A Wald confidence interval on an element ρi of ρ√can by constructed in
the usual way from an asymptotic standard error σ̂ ii . A standard error
may be obtained by computing the Jacobian Mn (ρ, θ) numerically and taking the estimated asymptotic variance σ̂ ii to be the ith diagonal element ofΣ̂ = (1/n)[M̂0n S̃−1 M̂n ]−1 . These intervals, which are symmetric, are somewhat misleading because they do not reflect the rapid increase in the EMM
objective function sn (ρ) = m(ρ, θ̃ n )0 S̃−1 m(ρ, θ̃n ) when ρi approaches avalue for which the system under consideration is explosive. Confidence intervals obtained by inverting the criterion diﬀerence test LREMM do reflect

938

23. Eﬃcient Method of Moments

this phenomenon and are therefore more useful. To invert the test one putsin the interval those ρ∗i for which LREMM for the hypothesis ρoi = ρ∗i is lessthan the critical point of a chi-squared on one degree of freedom. To avoid
re-optimization one may use the approximationρ̆n = ρ̂n +ρ∗i − ρ̂in
Σ̂(i)
σ̂ ii

in the formula for LREMM where Σ̂(i) is the ith column of Σ̂. The above
remarks should only be taken to imply that confidence intervals obtainedby inverting the criterion diﬀerence test have more desirable structuralcharacteristics than those obtained by inverting the Wald test and notthat they have more accurate coverage probabilities.
Diagnostics
When J0 exceeds the chi-squared critical point, diagnostics that suggest
improvements to the system are desirable. Because©
ª√dn m(ρ̂n , θ̃n ) → N 0, I o − Mo [Mo0 (I o )−1 Mo ]−1 Mo0(23.20)inspection of the t-ratios√
n m(ρ̂n , θ̃n )Tn = S−1
n(23.21)³´1/2
where Sn = diag{S̃ − M̂n [M̂0n S̃−1 M̂n ]−1 M̂0n }, can suggest reasonsfor failure. Diﬀerent elements of the score correspond to diﬀerent characteristics of the data and large t-ratios reveal those characteristics that are
not well approximated.

23.3 EMM Estimation in S+FinMetrics
A general class of discrete-time and continuous-time dynamic time seriesmodels may be estimated by EMM using the S+FinMetrics function EMM.
The arguments expected by EMM are> args(EMM)
function(score, coef = NULL, appcode = NULL, ui, ur, control= EMM.control(), est = NULL, save.sim = F, trace = T,gensim.fn = NULL, gensim.language = "C", gensim.aux =NULL, save.intermediate = F)
These arguments are summarized in Table 23.1. The main arguments arescore, which is an object of class “SNP” representing the fit of the auxiliarymodel to the observed data, coef, which is a vector containing starting23.3 EMM Estimation in S+FinMetrics

Argumentscore
coef
appcodeui
urcontrol
estsave.sim
trace
gensim.fn
gensim.language
gensim.aux
save.intermediate939

Description
Object of class “SNP” representing the auxiliarymodel fit.
Starting values for the model coeﬃcients.
Integer identifying the simulation function built
into EMM.
Vector of integer parameters to the simulation
function.
Vector of real parameters to the simulation
function.
List of EMM control parameters. Values set
using EMM.control.
Vector of logical values indicating which
coeﬃcients are held fixed.
If TRUE, simulated values at final iteration are
saved.
If TRUE, iteration count and objective areprinted to screen.
The simulation function. If NULL, then value isspecified by appcode.
The language of the simulation function: C,
FORTRAN or SPLUS.
An S-PLUS object that will be passed to thesimulation function.
If TRUE, intermediate results are saved.

TABLE 23.1. Arguments to function EMM

values for the model coeﬃcients to be estimated, and gensim.fn, which isa function to create simulations from the structural model given a set ofparameters. The score object is usually created by a call to the S-PLUS
function SNP. See the previous chapter for details on the creation of “SNP”
objects. The function specified by gensim.fn is called repeatedly by the
EMM function to generate simulated data as a function of a set of model
parameters. For EMM to work correctly, the same random numbers must
be used to generate simulations for diﬀerent values of the model parameters.
The simulation function may be defined as an S-PLUS function or a C or
FORTRAN function.4 If an S-PLUS function is supplied, then the user must
set gensim.language="SPLUS". Details on creating an S-PLUS simulation
function for use with the function EMM are given in the next section. The
function EMM creates an object of class “EMM” for which there are print,
summary, and plot methods, and extractor functions coef and vcov.
4 See the online help for EMM for details on specifying a C or FORTRAN function.

940

23. Eﬃcient Method of Moments

The control component of EMM is a list containing parameters that control aspects of the EMM estimation. Defaults for these parameters are set
by the function EMM.control:> args(EMM.control)
function(initial.itmax = 10, final.itmax = 200, tol = 1e-005,
n.burn = 100, n.sim = 10000, n.start = 0, tweak = 0,
siglev = 0.95, sfac1 = 1, sfac2 = 2, sfdrho = 1e-006,
seed = 61883, optimize = T, cf.interval = T, std.err= T, scale = 1)
These components are summarized in Table 23.2. The most importantcomponents are n.burn and n.sim, which specify the number of burnin and simulation values, respectively, for the simulation function specified by the argument gensim.fn. In complicated nonlinear models forwhich good starting values are hard to obtain, it is recommended thatthe random restart option for the optimizer be utilized. This feature allows the user to restart the EMM optimization at random perturbationsof the initial values in order to explore the surface of the EMM objectivefunction and avoid getting stuck at possible local minima. The component
initial.itmax sets the maximum number of iterations for each random
restart, and final.itmax determines the maximum number of iterationsin the final estimation. The number of random restarts is determined byn.start, which may be a scalar or a vector and must have the same length
as tweak. If n.start is a vector, then the ith element of n.start determines how many random restarts are performed for the corresponding
value in the ith element of tweak. If n.start=0, then tweak is ignored.
The component tweak contains tweak constants for random restarts of the
optimizer. The starting value for each coeﬃcient, ρi , is perturbed such that
ρi gets transformed to (1 + u× tweak) × ρi , where u is a random draw fromthe uniform distribution on (−1, 1).

23.3.1 Simulator Functions
The user is required to supply a function to generate simulations from themodel under study as a function of the model parameters. This function
may be an S-PLUS, C, or FORTRAN function. The following subsections
describe how to create these functions.
S-PLUS Simulator Functions
The easiest way to use EMM is with a user-written S-PLUS simulator function. The simulator function supplied to the gensim.fn argument of EMM
must have the form
my.gensim = function(rho, n.sim, n.var, n.burn, aux) {... }23.3 EMM Estimation in S+FinMetrics

Component
initial.itmax
final.itmax
toln.burnn.sim
n.starttweak
siglevel
sfac1
sfac2
sfdrhoseed
optimize
cf.interval
std.err
scale

941

Description
Maximum number of iterations used for each random
restart.
Maximum number of iterations used for final model
convergence tolerance for optimization.
Number of burn-in values to discard from beginningof simulation.
Number of simulated values for each series to be
simulated.
Number of random restarts of the optimizer.
Tweak constants for random restarts of the optimizer.
Significance level for confidence intervals.
Scale factor for confidence intervals.
Scale factor for confidence intervals.
Perturbation to rho for numerical derivatives.
Random number seed used for random restarts.
If TRUE, perform optimization.
If TRUE, compute confidence intervals by inverting
LREMM -statistic.
If TRUE, compute Wald standard errors.
scale factor used by the S-PLUS optimizer nlminb.
TABLE 23.2. Arguments to EMM.control

where rho gives the model parameters, n.sim specifies the number of simulated observations, n.var sets the number of simulated variables, n.burngives the number of burn-in observations, aux is an S-PLUS structure containing any auxiliary components required by the simulation function, and... represents the body of the function. The function my.gensim must
return a numeric vector of length n.sim*n.var, containing the simulated
values.5
In most applications, the structural model to be estimated is a discretetime or continuous-time dynamic time series model. For computational eﬃciency, it is extremely important that the simulation function for the structural model be fast. For discrete-time models, it is recommended that theuser utilize existing S-PLUS or S+FinMetrics simulation functions within
the body of the function specified by the gensim.fn argument since thesefunctions rely on underlying C or FORTRAN code to speed computations.
A listing of the simulation functions available in S-PLUS and S+FinMetrics
is given in Table 23.3. The examples in the following sections illustrate the
5 If simulations for a multivariate time series are required, then the function still
returns a numeric vector. The multivariate simulations must be ordered such that thefirst n.var elements correspond to the first simulated values of all the series, the second
n.var elements correspond to the second simulated values of all the series, an so on.

942

23. Eﬃcient Method of Moments

Simulation Function
arima.fracdiff.sim

SourceS-PLUS

arima.sim

S-PLUSfilterS-PLUS

simulate.FARIMA

S+FinMetrics

simulate.garch

S+FinMetrics

simulate.mgarch

S+FinMetrics

simulate.SNP

S+FinMetrics

simulate.VAR

S+FinMetrics

SsfSimS+FinMetrics

atsm.ssfsim

S+FinMetrics

simulate.SVOLS+FinMetrics

ssfSimMS

S+FinMetrics

Description
Simulate from fractional
ARIMA model.
Simulate from ARIMA
model.
Compute convolution orrecursive filter.
Simulate from fractional
ARIMA model.
Simulate from univariate
garch model.
Simulate from multivariate
garch model.
Simulate from SNP
models.
Simulate from vector
autoregressive model.
Simulate from linear state
space model.
Simulate from aﬃne term
structure state space model.
Simulate from discrete-time
stochastic volatility model.
Simulate from Markov
switching state space model.

TABLE 23.3. S-PLUS and S+FinMetrics simulation functions

creation of user-specified simulation functions for discrete-time dynamic
models utilizing the functions from Table 23.3.
For continuous-time dynamic time series models, simulations are often
generated as the solution of a system of stochastic diﬀerential equations(SDEs) of the form
dXt = a(Xt , t)dt + b(Xt , t)dWt(23.22)where Xt is a univariate or multivariate stochastic process, a(Xt , t) is thedrift function, b(Xt , t) is the diﬀusion function, and Wt is a standard
Brownian motion (Wiener) process. This SDE may be univariate or multivariate. For most applications, the SDE solution must be approximatedusing numerical methods such as Euler’s method. S+FinMetrics contains
several functions for numerically simulating the solution to univariate andmultivariate SDEs, and the main functions are listed in Table 23.4. Chapter 20 describes the solution methods underlying these functions in detailand provides many examples describing how to use the functions. The fourp-code functions are particularly handy since they allow S-PLUS functions

23.4 Examples

Simulation function
euler1d.pcode.gensim
euler.pcode.gensim
strong1.pcode.gensim
weak2.pcode.gensim
CIR.gensim
IRD.gensim
OU.gensim

943

Description
Solve univariate SDE using Euler’s method
Solve SDE using Euler’s method
Solve SDE using Strong order 1 scheme
Solve SDE using Weak order 2 scheme
Euler’s method for Cox-Ingersoll-Rosssquare root diﬀusion
Euler’s method for two-factor interest rate
diﬀusion
Exact solution for Ornstein-Uhlenbeck(or Vasicek) SDE

TABLE 23.4. Simulation functions for continuous-time SDEs

to be used for specifying arbitrary drift and diﬀusion functions a(Xt , t)and b(Xt , t), respectively. These p-code functions are heavily utilized in
the examples in the following sections.
C or FORTRAN Simulator Functions
The p-code functions listed in Table 23.4 for simulating the solutions to
the SDE (23.22) are flexible and easy to use, but they generally run two to
five times slower than compiled C code. These functions, however, utilize
underlying C code. If computational speed is important, the user may wish
to write their own simulator functions in C or FORTRAN that call the C
code functions for implementing the SDE solution methods.

23.3.2 SNP Auxiliary Model Estimation
The auxiliary model, or score generator, used to summarize the observed
data is created using the S+FinMetrics function SNP. The previous chapter
describes the SNP and associated functions and gives several examples offitting SNP models to common financial time series. The fitted SNP models
from these examples are used as score generators for some of the examples
of EMM estimation in the following sections.

23.4 Examples
The following subsections describe the estimation of some common univariate and multivariate time series models for financial data using the function EMM. The examples cover the estimation of moving average models,
discrete-time and continuous-time stochastic volatility models for equity
returns, and continuous-time stochastic volatility models for interest rates.
Some of the examples utilize the financial time series described and analyzed in the preceding chapter, and the reader is referred to that chapter

944

23. Eﬃcient Method of Moments

for the details of fitting the SNP auxiliary models. The examples illustrate
the creation of S-PLUS simulation functions for use with EMM, and describe
in detail the many options for controlling the EMM estimation. The examples also illustrate aspects of statistical inference and model diagnostics
associated with EMM estimation.

23.4.1 MA(1) Model
Following Chumacero (1997), consider the first-order moving average, MA(1),
model
yt
ρ0

= μ0 + εt + ψ 0 εt−1 , t = 1, . . . , nεt ∼ iid N (0, σ20 )= (μ0 , ψ 0 , σ 20 )0 = true value(23.23)The MA(1) model is a convenient example for illustrating the mechanics
of EMM because it is easy to simulate, there is a simple auxiliary model
based on a pth order autoregressive, AR(p), model, and the method of
moments and maximum likelihood estimators are easy to compute. The
GMM estimation of (23.23) was discussed in Chapter 21.
Given the model parameters ρ, it is straightforward to simulate values foryt . In S-PLUS, the function arima.sim may be used to eﬃciently generate
simulated values. For ρ0 = (0, 0.5, 1)0 and n = 250, simulated values may
be created using> set.seed(123)> ma1.sim = arima.sim(model=list(ma=-0.5),n=250)
The simulated yt are illustrated in Chapter 21, Figure 21.2, along with thesample autocorrelation function (SACF) and sample partial autocorrelation
function (SPACF).
Auxiliary ModelIf |ψ| < 1, then the MA(1) is invertible and has the AR(∞) representation
yt

= c+

∞
X

φj yt−j + εt

j=1

φj

= (−1) · (−ψ)j , c = μ/(1 + ψ)A feasible auxiliary model is an AR(p) modelytθ= c + φ1 yt−1 + · · · + φp yt−p + ut

ut ∼ iid N (0, σ2u )= (c, φ1 , . . . , φp , σ 2u )0

23.4 Examples

945

with p suﬃciently large to adequately capture the dynamics of the MA(1)model. The log-density for the auxiliary model is1
1
ln f (yt |xt−1 , θ) = − ln(2π) − ln(σ 2u )22
1
− 2 (yt − c − φ1 yt−1 − · · · − φp yt−p )22σ u
where xt−1 = (yt−1 , . . . , yt−p )0 . If p → ∞ at rate n1/3 then AR(p) willencompass the MA(1) data generating process as n → ∞.
Quasi-Maximum-Likelihood Estimation
Given a sample of observed data {ỹt }nt=1 based on ρ0 , the quasi-maximumlikelihood estimation of the auxiliary model parameters θ based on theconditional log-likelihood solvesn1Xθ̃ n = arg min sn (θ) = −ln f (ỹt |x̃t−1 , θ)θn t=1
The sample score vector sn (θ̃ n ) has p + 2 elementsn−1 X(ỹt − c̃ − φ̃1 ỹt−1 − · · · − φ̃p ỹt−p )nσ̃ 2u t=1n−1 X(ỹt − c̃ − φ̃1 ỹt−1 − · · · − φ̃p ỹt−p )ỹt−1
nσ̃ 2u t=1

..
.

n

−1 X
(ỹt − c̃ − φ̃1 ỹt−1 − · · · − φ̃p ỹt−p )ỹt−pnσ̃ 2u t=1

n
X
1
−2
[n
+
σ̃
]
(ỹt − c̃ − φ̃1 ỹt−1 − · · · − φ̃p ỹt−p )
2σ̃ 2
t=1

and by construction, sn (θ̃ n ) = 0.
Estimating the Auxiliary Model Using SNP
For the EMM estimation in S-PLUS, the auxiliary model must be estimatedusing the function SNP. The SNP function estimates Gallant and Tauchen’s
general-purpose seminonparametric auxiliary model for stationary multivariate time series, as described in the previous chapter, and can be used
to estimate a wide class of auxiliary models, including the AR(p) model.
The object returned by SNP contains the information necessary to computethe analytic score required for computing the EMM objective function.

946

23. Eﬃcient Method of Moments

For illustrative purposes, let the auxiliary model for the MA(1) data
be a Gaussian AR(3) model. This model has a 5 × 1 parameter vectorθ = (μ, φ1 , φ2 , φ3 , σ2u )0 and may be estimated using SNP as follows:> ar3.fit = SNP(data=ma1.sim, model=SNP.model(ar=3))> class(ar3.fit)[1] "SNP"> summary(ar3.fit)
Call:
SNP(data = ma1.sim, model = SNP.model(ar = 3))
Model: Gaussian VAR
Conditional Mean Coefficients:
mu
ar(1)
ar(2)
coef -0.0045 0.5312 -0.2328(std.err) 0.0565 0.0637 0.0690(t.stat) -0.0788 8.3444 -3.3763

ar(3)0.10070.06591.5280Conditional Variance Coefficients:
sigma
coef 0.8828(std.err) 0.0462(t.stat) 19.1253

Information Criteria:
BIC
HQ
AIC
logL
1.35 1.3288 1.3145 -319.6804
Convergence Type:both x and relative function convergence
number of iterations: 1
For successful estimation using EMM, it is vitally important that theauxiliary model provide a good fit to the observed data and be able toreplicate the important dynamic features of the data. For the AR(3) auxiliary model, the estimated values for φ1 , φ2 , and φ3 are all statisticallydiﬀerent from zero and have values that are consistent with φj = −(−0.5)j
for j = 1, 2, 3. In addition, the estimated value of σ u is close to σε = 1.
Graphical residual diagnostics, produced using> plot(ar3.fit, which.plots=3:6)

947

-4

-2

0

2

23.4 Examples

0

50

100

150

200

250

-0.2

0.2

0.6

1.0

Series : ar3.sim

0

5

10

15

20

15

20

Lag

-0.2

0.2

Series : ar3.sim

0

5

10
Lag

FIGURE 23.2. Simulated data, SACF and PACF from the AR(3) auxiliary model
fit to simulated data from the MA(1) model.

indicate that the AR(3) has adequately captured the dynamics in the data.
To be sure, simulated data from the AR(3) auxiliary model should qualitatively match the characteristics of the MA(1) data. Simulations fromthe fitted SNP model of length equal to the actual data may be computed
using the generic simulate method function> ar3.sim = simulate(ar3.fit)
Figure 23.2 shows the simulated data along with the SACF and SPACF.
These plots verify that the auxiliary model adequately mimics the MA(1)model.
EMM Algorithm
For a fixed ρ, let {ŷt (ρ)}N
t=1 denote a simulated sample of size N >> nfrom the MA(1) model. The length, N , of the simulation should, in general,be much larger than the sample size, n, of the observed data. The sample
score of the auxiliary model evaluated using the simulated data isN

1 X ∂m(ρ, θ̃n ) =ln f (ŷt (ρ)|x̂t−1 , θ̃ n )N t=1 ∂θ

948

23. Eﬃcient Method of Moments

which, for the AR(p) auxiliary model, has elements
N

1 X(ŷt (ρ) − c̃ − φ̃1 ŷt−1 (ρ) − · · · − φ̃p ŷt−p (ρ))
N t=1
N

1 X
(ŷt (ρ) − c̃ − φ̃1 ŷt−1 (ρ) − · · · − φ̃p ŷt−p (ρ))ŷt−1 (ρ)
N t=1

..
.

N

1 X
(ŷt (ρ) − c̃ − φ̃1 ŷt−1 (ρ) − · · · − φ̃p ŷt−p (ρ))ŷt−p (ρ)
N t=1
−

N
X
1
−2
[N
+
σ̃
]
(ŷt (ρ) − c̃ − φ̃1 ŷt−1 (ρ) − · · · − φ̃p ŷt−p (ρ))
2σ̃ 2
t=1

If ρ = ρ0 , thenm(ρ0 , θ̃ n ) ≈ 0
whereas if ρ 6= ρ0 , thenm(ρ, θ̃n ) 6= 0

The EMM algorithm attempts to find the value ρ̂n such that m(ρ̂n , θ̃n ) is
as close to zero as possible. Since θ is (p + 2) × 1 and ρ is (3 × 1), it is
not possible to find ρ̂n such that m(ρ̂n , θ̃n ) = 0. Instead, ρ is estimatedby minimizing the GMM-type objective function:ρ̂n = arg min m(ρ, θ̃ n )0 S̃−1 m(ρ, θ̃ n )ρ

√where S̃ is a consistent estimate of avar( nm(ρ0 , θ̃n )).
The Simulator Function
An example of a simulation function for the MA(1) model in a form for use
with the function EMM is
MA1.gensim = function(rho, n.sim, n.var=1, n.burn,aux=MA1.aux()){# rho = (mu,psi,sigma2)’# aux is a list with components# z = vector of N(0,1) values of length (n.sim + n.burn)nz = n.burn + n.sim
if(is.null(z <- aux$z))
stop("aux$z must be supplied")if(length(z) != nz)
stop("aux$z must be of length",nz," if provided")23.4 Examples

949

ans = rho[1]+arima.sim(model = list(ma=-rho[2]),innov = z[(n.burn+1):nz]*sqrt(rho[3]),start.innov = z[1:n.burn]*sqrt(rho[3]))as.numeric(ans)}In the body of MA1.gensim, the S-PLUS function arima.sim is called toeﬃciently simulate from the MA(1) model. The function is specified suchthat the random numbers used to generate the simulation are computed inadvance and passed to the simulator through the optional aux list component. This allows MA1.gensim to use the same random numbers to generate
simulations for diﬀerent values of rho.
EMM Estimation
Once the auxiliary model has been fit using the SNP function and the user
has written the simulator function, the structural model (MA(1) model)
may be estimated by EMM using the EMM function. For example, commands
to estimate the MA(1) model using EMM with N = 10, 000 simulations and
10 burn-in values are> set.seed(345)> nsim = 10000> nburn = 10
> MA1.aux = list(z = rnorm(nsim+nburn))> start.vals = c(0,0.5,1)> names(start.vals) = c("mu","theta","sigma2")> EMM.MA1.fit = EMM(ar3.fit,coef = start.vals,+
control = EMM.control(n.sim = nsim, n.burn = nburn),+
gensim.fn = "MA1.gensim", gensim.language = "SPLUS",+
gensim.aux = MA1.aux, save.sim = T)
Iteration No. 1, objective = 114.452...
Iteration No. 12, objective = 0.0963164> class(EMM.MA1.fit)[1] "EMM"
In the call to EMM, the simulator function is specified by the arguments
gensim.fn="MA1.gensim" and gensim.language="SPLUS". The control
argument sets the number of burn-in and simulation values. The random
numbers, which are fixed across iterations in the optimization, are contained in the user-created list variable MA1.aux and are passed to the simulator within EMM using gensim.aux="MA1.gensim". The dimension of thevectors in MA1.aux must match the dimension n.burn+n.sim determinedby the corresponding arguments passed to the function EMM.control. By
setting save.sim=T, the simulated observations evaluated at the estimated950

23. Eﬃcient Method of Moments

value of ρ, ŷt (ρ̂), are added as the component smdat of the returned “EMM”
object.
The function EMM returns an object of class “EMM” for which there areprint, summary, and plot methods. Typing the name of an “EMM” object
invokes the print method> EMM.MA1.fit
Call:
EMM(score = ar3.fit, coef = start.vals, control = EMM.control(n.sim = nsim, n.burn = nburn), save.sim = T, gensim.fn= "MA1.gensim", gensim.language = "SPLUS", gensim.aux= MA1.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
mu 0.0985
0.09960 -0.09606
0.2955psi 0.5267
0.06473 0.41302
0.6689sigma2 0.9024
0.07060 0.76378
1.0403
Final optimization:
Convergence: relative function convergence
Iterations: 12
Score at final iteration: 0.09632
The estimated coeﬃcients are close to their true values, but have somewhat large standard errors. The 95% confidence intervals are based on
inverting the LREMM statistic for testing ρi = ρoi . To compute the usualc i ), set the optional arWald-type 95% confidence intervals, ρ̂i ± 1.96 · SE(ρ̂gument cf.interval=F in EMM.control. The final value of the score is thenormalized EMM objective function, and its small value indicates that theoveridentifying conditions are not rejected by the data.
The summary method provides some useful diagnostics for assessing the
EMM fit:> summary(EMM.MA1.fit)
Call:
EMM(score = ar3.fit, coef = start.vals, control = EMM.control(n.sim = nsim, n.burn = nburn), save.sim = T, gensim.fn= "MA1.gensim", gensim.language = "SPLUS", gensim.aux= MA1.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
mu 0.0985
0.09960 -0.09606
0.2955psi 0.5267
0.06473 0.41302
0.6689sigma2 0.9024
0.07060 0.76378
1.0403

23.4 Examples

951

Final optimization:
Convergence: relative function convergence
Iterations: 12
EMM objective at final iteration: 0.09632
P-Value: 0.953 on 2 degrees of freedom
Score Information:
Mean Score Std. Error t-ratio Adj. Std. Error t-ratiomu
0.005372
1.133 0.004743
0.09656 0.05564
ar(3)0.235363
1.174 0.200504
1.16387 0.20222
ar(2)0.354856
1.209 0.293554
1.14360 0.31030
ar(1)0.219727
1.105 0.198858
0.84555 0.25986
sigma
0.093842
1.427 0.065739
0.36389 0.25789
First, the p-value for the EMM objective function based on the limiting
chi-square distribution is given. The degrees of freedom for the chi-square
distribution is equal to the number of parameters in the auxiliary SNP
model minus the number of parameters in the structural model: 5 − 3 = 2.
Here, the very large p-value of 0.9558 supports the validity of the overidentifying conditions. Second, the individual normalized score values (23.20)
along with their (unadjusted and adjusted) asymptotic standard errors andt-statistics (23.21) are given.6 In a correctly specified model, the EMM objective function should not be significantly large and the absolute valueof the score t-ratios should be less than 2. If the model is misspecified insome way and the auxiliary model adequately describes the observed data,then the EMM objective should be significantly large, and one or more of
the score t-ratios should be large reflecting a characteristic of the auxiliarymodel that is not being captured by the structural model. For the MA(1)model, the EMM objective function is not significantly large and all of thescore t-ratios are less than 2 indicating a well specified-model.
The plot method gives a barchart of the unadjusted or adjusted score
t-ratios. For example, to display the adjusted score t-ratios, use> plot(EMM.MA1.fit)
The resulting plot is shown in Figure 23.3.
The simulated observations at the estimated value of ρ, ŷt (ρ̂), are in
the component smdat. These values should mimic the properties of theobserved data if the estimated structural model is correctly specified.
6 The t-ratios (23.21) are called the adjusted t-ratios. The unadjusted t-ratios use S̃
instead of Sn in (23.21). The unadjusted t-ratios are easier to compute but are biased
downward relative to 2.0.

952

23. Eﬃcient Method of Moments

Adjusted t-ratios of mean score

sigma

ar(1)

ar(2)

ar(3)

mu

0.0

0.05

0.10

0.15

0.20

0.25

0.30

FIGURE 23.3. Adjusted score t-ratios from the EMM estimation of an MA(1)model using a Gaussian AR(3) SNP model.

GMM and ML Estimation
The GMM estimation of the MA(1) model (23.23) using the moments
E[yt ]
E[yt2 ]
E[yt yt−1 ]E[yt yt−2 ]

=
=
=
=

μ0
μ20 + σ 20 (1 + ψ 20 )μ20 + σ 20 ψ 0
μ20

is discussed in Chapter 21. The GMM estimates of ρ for the simulated data
in ma1.sim, based on an HAC estimate of the asymptotic variance of the
moments with a truncated kernel, are repeated here:> summary(ma1.gmm.trunc)
Call:
GMM(start = start.vals, moments = ma1.moments, ts = T,
var.hac.control = var.hac.control(bandwidth = 1,
window = "truncated"), data = ma1.data)
Coefficients:
Value Std.Error t value Pr(>|t|)mu 0.1018 0.0927
1.0980 0.2733

23.4 Examples

psi
sig2

0.54710.86710.08070.07636.778811.3581953

0.00000.0000

Test of Overidentification:
J-stat Df P.value
0.3549 1 0.5513
Optimization Info:
Number of Iterative Steps: 3
The GMM estimates are close to the EMM estimates. The standard errors
for the EMM estimates of ψ and σ 2 are slightly smaller than the standard
errors of the corresponding GMM estimates. This eﬃciency gain reflects
the fact that EMM uses the eﬃcient moments based on the score of theauxiliary model, whereas GMM uses a set of arbitrary moments. EMM
also benefits from not having to use an HAC estimator for the asymptotic
variance of the moments.
The conditional maximum likelihood estimation of the MA(1) model(23.23) may be performed using the S-PLUS function arima.mle as follows:> mle.fit = arima.mle(ma1.sim,model=list(ma=-0.5),+
xreg=rep(1, length(ma1.sim)))> mle.fit
Call: arima.mle(x = ma1.sim, model = list(ma = -0.5),
xreg = rep(1, length(ma1.sim)))
Method: Maximum Likelihood
Model : 0 0 1
Coefficients:
MA : -0.52262
Variance-Covariance Matrix:ma(1)ma(1) 0.002907486
Coeffficients for regressor(s): rep(1, length(ma1.sim))[1] 0.11416
Optimizer has converged
Convergence Type: relative function convergence
AIC: 683.51018> mle.fit$sigma2[1] 0.8859344
The estimates of μ, ψ, and σ 2 are 0.5226, 0.1142, and 0.8859, respectively.
These estimates are close to the GMM and EMM estimates. The S-PLUS
function arima.mle only computes an estimate of the coeﬃcient covariance954

23. Eﬃcient Method of Moments

matrix for the autoregressive moving average (ARMA) parameters. The
estimated standard error for ψ is
> sqrt(mle.fit$var.coef)ma(1)ma(1) 0.05392111
which is slightly smaller than the EMM standard error. Here we see that
the EMM estimates are comparable to the ML estimates. As the sample
size increases and the number of AR terms in the SNP model increases,
the diﬀerence between the EMM and ML estimates will approach zero.

23.4.2 Discrete-Time Stochastic Volatility Models
This subsection describes the EMM estimation of some discrete-time stochastic volatility (SV) models used for modeling the daily continuously
compounded returns on the S&P 500 index. The first model is an elementary and empirically unrealistic SV model used by Gallant and Tauchen(2002), hereafter GT, to illustrate some of the workings and properties of
EMM estimation and inference. The second model is a more realistic SV
model with AR(1) dynamics considered by Andersen and Sorensen (1996)
and Andersen, Chung, and Sorensen (1999). The third model is a general
SV model with AR(p) dynamics that allows for leverage eﬀects considered
by Gallant, Hsieh, and Tauchen (1997). The analysis of the SV models
in this section is in the spirit of the analysis done in Gallant, Hsieh, andTauchen (1997) and GT.
Data
The data consist of daily continuously compounded returns, based on closing prices, on the S&P 500 index over the period March 14, 1986 throughJune 30, 2003. These data were extensively analyzed in the previous chapter. Similar data were analyzed in GT. Figure 22.18 of Chapter 22 shows thedata along with the sample ACF of the squared returns. Sample summary
statistics are> summaryStats(sp500.ts)
Sample Quantiles:
min
1Q
median3Q
max-0.2047 -0.004686 0.0004678 0.005827 0.09099
Sample Moments:
mean
std skewness kurtosis
0.0003916 0.01124-1.486
32.59
Number of Observations:4365

23.4 Examples

955

The S&P 500 returns appear to be highly non-normal, with negative skewness and very large kurtosis, and display considerable conditional heteroskedasticity.
SNP Auxiliary Model Fit
The details of finding the best fitting SNP model for the S&P 500 returns are described in the previous chapter. There it was found that a
non-Gaussian AR(1)-GARCH(1,1) model, 11118000, adequately describes
the data. The auxiliary model information is contained in the “SNP” object
fit.sp500.11118000, which is provided with the S+FinMetrics module.
For illustrative purposes, the EMM estimation using the simplistic nonGaussian AR(1) with Kz = 4 will also be considered. This fit is available
in the “SNP” object fit.sp500.10014000.
Gallant and Tauchen (2002)
To illustrate the workings of EMM, GT considered the very simple discretetime SV modelyt − μ = c(yt−1 − μ) + σ exp(wt )zt
wt − μw = σ w zwt(23.24)

where μ, c, σ, μw , and σ w are parameters and {zt } and {zwt } are mutuallyindependent iid Gaussian processes with mean zero and unit variance. GT
impose the normalization μw = −σ 2w , which implies that var(yt |yt−1 ) = σ 2 .
The model allows for serially correlated returns, but no serial correlationin the unobserved log volatility wt . This model is too simplistic for actual
data. Nonetheless, its estimation by EMM provides important insights tothe proper use of the EMM methodology.
Simulator Function
An S-PLUS function to eﬃciently simulate from the SV model (23.24) thatcan be used in conjunction with the function EMM is
sv.gt.gensim = function(rho,n.sim,n.var=1,
n.burn,aux=sv.gt.aux){# rho = (mu,c,sigma,sigma.w)# aux is a list with components# z = vector of N(0,1) values of length (n.sim + n.burn)# zw = vector N(0,1) values of length (n.sim + n.burn)nz = n.burn + n.sim
sv = rho[3]*exp(-rho[4]*rho[4] + rho[4]*aux$zw)*aux$z
y = arima.sim(model = list(ar=rho[2]),innov = sv[(n.burn+1):nz],

23. Eﬃcient Method of Moments

-0.05

0.0

0.05

956

0

1000

2000

3000

4000

FIGURE 23.4. Simulated data from a discrete-time SV model.

start.innov = sv[1:n.burn])y = y + rho[1]as.numeric(y)}Notice that the S-PLUS function arima.sim is used to eﬃciently simulate
from an AR(1) process. Also notice that the function sv.gt.gensim is set
up so that the user must pass two vectors of random numbers as components in the list sv.gt.aux.
To generate a simulated sample with N = 4000, with 100 burn-in values,
calibrated to match the S&P 500 returns, use> n.sim = 4000
> n.burn = 100> nz = n.sim + n.burn> set.seed(123)> sv.gt.aux = list(z=rnorm(nz), zw=rnorm(nz))> rho.gt = c(0.0004, 0.01, 0.01, 0.5)
> sv.gt.sim = sv.gt.gensim(rho = rho.gt, n.sim = n.sim,+
n.burn = n.burn, aux = sv.gt.aux)
The simulated sample is illustrated in Figure 23.4. The simple SV model(23.24) cannot produce the large outliers or volatility clustering observedin the actual data.

23.4 Examples

957

EMM Estimation
As in GT, first consider the EMM estimation of the simple SV model (23.24)
for the S&P 500 returns using the poorly fitting auxiliary model represented
by the “SNP” object fit.sp500.10014000. The EMM estimation with N =10, 000 simulations and 100 burn-in values may be performed using> n.sim = 10000> n.burn = 100> nz = n.sim + n.burn> set.seed(456)> sv.gt.aux = list(z=rnorm(nz), zw=rnorm(nz))> rho.gt = c(0.0004, 0.01, 0.01, 0.5)
> names(rho.gt) = c("mu","c","sigma","sigma.w")> emm.svgt.fit1 = EMM(fit.sp500.10014000, coef = rho.gt,+
EMM.control(n.burn=n.burn, n.sim=n.sim),+
gensim.fn = "sv.gt.gensim",+
gensim.language = "SPLUS",+
gensim.aux = sv.gt.aux)
Iteration No. 1, objective = 2.0701e+010...
Iteration No. 18, objective = 4.66991
The EMM estimation converges, and the small objective value indicatesthat the data, as summarized through the 10014000 auxiliary model, areconsistent with the structural model (23.24). A summary of the EMM fit
is> summary(emm.svgt.fit1)
Call:
EMM(score = fit.sp500.10014000, coef = rho.gt, appcode =EMM.control(n.burn = n.burn, n.sim = n.sim), gensim.fn= "sv.gt.gensim", gensim.language = "SPLUS",
gensim.aux = sv.gt.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
mu 0.0006102 0.0001488 0.000318 0.0009016
c 0.0111869 0.0157008 -0.019764 0.0417771
sigma 0.0104773 0.0002044 0.010078 0.0108867
sigma.w 0.4411473 0.0261296 0.388516 0.4929767
Final optimization:
Convergence: relative function convergence
Iterations: 18
EMM objective at final iteration: 4.67
P-Value: 0.1976 on 3 degrees of freedom

958

23. Eﬃcient Method of Moments

Score Information:
Mean Score Std. Error t-ratio Adj. Std. Error t-ratioz^1-0.3076
2.436 -0.12631
1.4234 -0.2161
z^2-0.2058
3.953 -0.05206
0.5299 -0.3884
z^3
3.07589.393 0.32745
8.5520 0.3597
z^4
3.697722.354 0.16541
3.1628 1.1691
mu-0.39311.118 -0.35160
0.2239 -1.7557
ar(1)0.9893
1.262 0.78408
0.6526 1.5160
sigma
4.40933.315 1.33009
3.0670 1.4377
The autoregressive coeﬃcient is not statistically diﬀerent from zero, but thevariance of the stochastic volatility term is highly significant. The p-valueof 0.1976 on the EMM objective function indicates that the overidentifying moment conditions from the 10014000 scores are not rejected by thedata. As a result, the absolute value of the t-ratios on the individual scoreelements are all less than 2. The results indicate that the simple SV model
fits the data, as summarized through the 10014000 auxiliary model.
Next, consider fitting the simple SV model (23.24) using the best fitting
SNP 11118000 auxiliary model with N = 10, 000 simulations and 100 burnin values:> emm.svgt.fit = EMM(fit.sp500.11118000, coef=rho.gt,+
EMM.control(n.burn=n.burn, n.sim=n.sim),+
gensim.fn = "sv.gt.gensim",+
gensim.language = "SPLUS",+
gensim.aux = sv.gt.aux)
Iteration No. 1, objective = 3.5508e+008...
Iteration No. 44, objective = 156.49
The EMM estimation converges with a high objective value, indicatingthat the data now reject the structural model (23.24). This is confirmed byexamining the EMM fit:
> summary(emm.svgt.fit)
Call:
EMM(score = fit.sp500.11118000, coef = rho.gt,
appcode = EMM.control(n.burn = n.burn, n.sim = n.sim),
gensim.fn = "sv.gt.gensim",
gensim.language = "SPLUS", gensim.aux = sv.gt.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
mu 0.0005499 0.00009724 0.0003601 0.0006583
c 0.0267012 0.01025450 0.0052093 0.0480410
sigma 0.0128051 0.00059129 0.0120651 0.0135576

23.4 Examples

sigma.w 0.4470297 0.00786893 0.4392415

959

0.4575820

Final optimization:
Convergence: relative function convergence
Iterations: 44
EMM objective at final iteration: 156.5
P-Value: 0 on 9 degrees of freedom
Score Information:
Mean Score Std. Error t-ratio Adj. Std. Error t-ratioz^1-4.5158
2.4735 -1.8257
2.1519 -2.0986
z^2-27.4201
3.7849 -7.2446
3.3053 -8.2958
z^3-4.71508.6945 -0.5423
8.1304 -0.5799
z^4 -157.5280
21.7408 -7.2457
20.0389 -7.8611
z^5
124.1809
65.0210 1.9099
60.8025 2.0424
z^6 -934.3530
213.5628 -4.3751
200.8732 -4.6515
z^7 2537.7699
760.7266 3.3360
726.0131 3.4955
z^8 -6559.4648 2856.7967 -2.2961
2758.2166 -2.3782
mu-2.4675
1.4805 -1.6666
0.8574 -2.8779
ar(1)0.53420.9803 0.5449
0.3660 1.4595
s0-42.1830
14.6437 -2.8806
13.9578 -3.0222
arch(1)-13.6875
8.0812 -1.6938
6.6419 -2.0608
garch(1)-42.1351
15.6225 -2.6971
12.7673 -3.3003
The zero p-value on the EMM objective function, based on the chi-square
distribution with 9 degrees of freedom, leads to a rejection of the overidentifying moment conditions. The large t-ratios on the individual score elements associated with the Hermite polynomial elements and the GARCH
elements shows that the simple SV model cannot capture the fat tails andvolatility clustering that exists in the data, as summarized by the 11118000
auxiliary model. This example highlights the importance of using an auxiliary model that captures all of the important features of the data whenestimating and evaluating a structural model by EMM.
Andersen, Chung, and Sorensen (1999)
The SV model considered by Andersen and Sorensen (1997), Chumacero(1997), and Andersen, Chung, and Sorensen (1999) is of the formyt
ln σ 2t= σ t zt
= ω + β ln σ 2t−1 + σu ut(23.25)where (zt , ut ) is iid N (0, I2 ). This model was estimated by GMM using 24
moments in Chapter 21. The return series yt is assumed to be demeaned.
For −1 < β < 1 and σ u > 0, the return series, yt , is strictly stationary andergodic, and unconditional moments of any order exist. Let wt = ln σ 2t so

960

23. Eﬃcient Method of Momentsthat σ t = exp(wt /2). Then, (23.25) may be rewritten as
yt
wt= exp(wt /2) · zt
= ω + βwt−1 + σ u ut

which is in a form similar to the model (23.24). The SV model (23.25)
allows log-volatility to be serially correlated, which produces autoregressive conditional heteroskedasticity (ARCH)-like features in the data. See
Shephard (1996) and Ghysels, Harvey, and Renault (1996) for a detailed
comparison of ARCH and SV models.
Simulator Function
An S-PLUS function to generate simulations from the SV model (23.25) is
sv.as.gensim = function(rho, n.sim, n.var=1, n.burn,aux = sv.as.aux){# rho = (alpha, beta.t, sigma.u)# aux is a list with components# z = vector of N(0,1) values of length (n.sim + n.burn)# u = vector N(0,1) values of length (n.sim + n.burn)nz = n.burn + n.sim
beta = exp(rho[2])/(1 + exp(rho[2]))mu = rho[1]/(1-beta)
w = mu + arima.sim(model = list(ar=beta),
innov = aux$u[(n.burn + 1):nz]*rho[3],start.innov = aux$u[1:n.burn]*rho[3])y = exp(w/2)*aux$z[(n.burn + 1):nz]
as.numeric(y)}To impose stationarity on the log-volatility process, the logistic transformation is used to define the autoregressive parameter β from the unrestrictedparameter β ∗ :exp(β ∗ )β=
, − ∞ < β∗ < ∞1 + exp(β ∗ )The logistic transformation restricts β to the interval (0, 1). This restriction
is reasonable since negative values of β are not empirically relevant for
asset returns. The unconditional mean of the log-volatility process is μ =ω/(1 − β).
Figure 23.5 shows a simulated sample of size N = 4000 from (23.25) using the parameterization α = −0.147, β = 0.98 and σ u = 0.166 taken from
Andersen, Chung, and Sorensen (1999) (Monte Carlo design II). The parameters are calibrated to match typical daily return data. The simulationis generated using

961

-0.1

0.0

0.1

23.4 Examples

0

1000

2000

3000

4000

FIGURE 23.5. Simulated data from the Andersen-Sorensen SV model with
α = 0.736, β = 0.90, and σu = 0.363.

> n.sim = 4000
> n.burn = 100> nz = n.sim + n.burn> set.seed(456)> sv.as2.aux = list(z=rnorm(nz), u=rnorm(nz))> rho.as2 = c(-0.147, 3.89182, 0.166)> sv.as2.sim = sv.as.gensim(rho = rho.as, n.sim = n.sim,+
n.burn = n.burn, aux = sv.as.aux)
The autocorrelated log-volatility process generates ARCH-like volatility
clustering in the simulated data.
EMM Estimation on Simulated Data
Andersen, Chung, and Sorensen (1999) studied the EMM estimation of(23.25) using an extensive Monte Carlo study. They found that EMM performs substantially better than GMM and comparably to direct likelihoodbased inference procedures. For samples of size 1000 or less they found thata simple Gaussian GARCH(1,1) SNP model was a good choice for a score
generator. Only for much larger samples did they find that adding Hermite polynomial terms to the SNP model improved eﬃciency. Regarding
inference, they found that the EMM objective function test for overidenti-

962

23. Eﬃcient Method of Moments

fying restriction was remarkably reliable in contrast to the analogous GMM
objective function test (Hansen’s J-test).
The estimation of (23.25) using the simulated data in sv.as2.sim isbased on the following Gaussian GARCH(1,1) SNP model:> fOld = c(0,1e-5, 0, -1e-5, 1e-5, 1e-5, 1e-4, 0, -1e-4,
1e-4, 1e-4, 1e-3, 0, -1e-3, 1e-3, 1e-3, 1e-2,
0, -1e-2, 1e-2, 1e-2, 1e-1, 0, -1e-1, 1e-1,
1e-1, 1e0, 0, -1e0, 1e0, 1e0)
> fNew = c(0,0, 1e-5, 1e-5, -1e-5, 1e-5, 0, 1e-4, 1e-4,-1e-4, 1e-4, 0, 1e-3, 1e-3, -1e-3, 1e-3, 0,
1e-2, 1e-2, -1e-2, 1e-2, 0, 1e-1, 1e-1, -1e-1,
1e-1, 0, 1e0, 1e0, -1e0, 1e0)
> n.start = c(0,rep(25,30))> fit.svas2sim.01110000 = SNP(as.matrix(sv.as2.sim),+model = SNP.model(ar=0, arch=1, garch=1),+
control = SNP.control(xTransform = "logistic",+n.start=n.start, fOld=fOld,+
fNew=fNew),+n.drop=4, trace=T)
The quasi-maximum-likelihood estimation of the SNP model utilizes random restarts of the optimizer to avoid getting stuck at potential local minima, as recommended by Gallant and Tauchen (2001).
The EMM estimation with N = 40, 000 simulations and 100 burn-in
values is performed using7> n.sim = 40000> n.burn = 100> nz = n.sim + n.burn> set.seed(123)> sv.as2.aux = list(z=rnorm(nz), u=rnorm(nz))> emm.svas2sim.fit = EMM(fit.svas2sim.01110000, coef=rho.as2,+
EMM.control(n.burn=n.burn, n.sim=n.sim),+
gensim.fn="sv.as.gensim",+
gensim.language="SPLUS",+
gensim.aux=sv.as2.aux)
Iteration No. 1, objective = 85170...
Iteration No. 30, objective = 0.168816> summary(emm.svas2sim.fit)
7 These are values similar to those used by Andersen and Sorensen (1996). They
utilized N = 20, 000 simulations with 20, 000 antithetic pairs. The S-PLUS function EMM
currently does not support antithetic variates.

23.4 Examples

963

Call:
EMM(score = fit.svas2sim.01110000, coef = rho.as2,
appcode = EMM.control(n.burn = n.burn, n.sim = n.sim),
gensim.fn = "sv.as.gensim",
gensim.language = "SPLUS", gensim.aux = sv.as2.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
alpha -0.2631
0.13018 -0.3839-0.1330
beta.t 3.3228
0.50730 2.8523
3.8301sigma.u 0.1950
0.04636 0.1486
0.2378
Final optimization:
Convergence: relative function convergence
Iterations: 30
EMM objective at final iteration: 0.1688
P-Value: 0.6812 on 1 degrees of freedom
Score Information:
Mean Score Std. Error t-ratio Adj. Std. Error t-ratio
mu
0.82302.004 0.410711
2.0029 0.4109
s02.490916.076 0.154946
6.0634 0.4108
arch(1)0.3551
8.196 0.043326
0.8647 0.4107
garch(1)0.113911.928 0.009551
0.2777 0.4102
The EMM estimation converges with a high objective function p-value indicating that the data do not reject the single overidentifying restriction
implied by the GARCH(1,1) score generator. The estimates of ω and σu
are reasonably close to their true values, and the sizes of their estimatedstandard errors are similar to the Monte Carlo root mean square errors
reported by Anderson, Chung, and Sorensen. The estimate of β ∗ is 3.3228,
which implies an estimate for β equal to 0.9652028. The standard error forβ, obtained using the delta method approximation, is> bt.hat = emm.svas2sim.fit$coef[2]> b.hat = exp(bt.hat)/(1 + exp(bt.hat))> dg = b.hat*(1 - b.hat)> se.b.hat = sqrt(dg%*%emm.svas2sim.fit$var[2,2]%*%dg)> se.b.hat[,1][1,] 0.01703867
Notice that all of the adjusted score t-ratios are the same. This occurs
because the asymptotic variance of the normalized moments has rank 1(degree of overidentification).

964

23. Eﬃcient Method of Moments

The GMM estimation of (23.25) based on 24 arbitrary moment conditions
is described in Chapter 21. The GMM estimates based on the simulated
data in sv.as2.sim are computed using> sv.pow = cbind(abs(sv.as2.sim), sv.as2.sim^2,+
abs(sv.as2.sim)^3, sv.as2.sim^4)> sv2.pow = sv.pow[-(1:10),]> sv2.cm = tslag(sv.as2.sim, 1:10, trim=T) *+as.vector(sv.as2.sim[-(1:10)])> sv2.data = cbind(sv2.pow, abs(sv2.cm), sv2.cm^2)> start.vals = c(0, 0.5, 0.5)
> names(start.vals) = c("omega", "beta", "sigu")> sv2.fit.gmm = GMM(start.vals, sv.moments, method="iterative",+
ts=T, data=sv2.data)> summary(sv2.fit.gmm,print.moments=F)
Call:
GMM(start = start.vals, moments = sv.moments, method ="iterative", ts = T, data = sv2.data)
Coefficients:
Value Std.Error
omega -0.2650
0.1348beta
0.96490.0179sigu
0.1926
0.0532t value Pr(>|t|)-1.9655
0.049454.02240.00003.62140.0003

Test of Overidentification:
J-stat Df P.value
15.297 21 0.8078
The GMM and EMM estimates are remarkably similar. However, based on
extensive Monte Carlo exercises, Chumacero (1997) and Anderson, Chung,and Sorensen (1999) recommend EMM over GMM for the following reasons:(1) EMM estimates are numerically more stable; (2) EMM estimates havesmaller root mean squared errors; (3) the problems associated with the
choice of weighting matrices in the case of GMM are absent in EMM; (4)the EMM test for overidentifying restrictions is more reliable; (5) inference
regarding the parameters based on EMM t-statistics is more reliable.
EMM Estimation on S&P 500 Returns
Consider EMM estimation of (23.25) for the S&P 500 daily returns using
the best fitting score generator summarized by the “SNP” object fit.sp500.
11118000. The EMM estimation with N = 40, 000 simulated values and100 burn-in values is computed using23.4 Examples

965> n.sim = 40000> n.burn = 100> nz = n.sim + n.burn> set.seed(456)> sv.as.aux = list(z=rnorm(nz), u=rnorm(nz))> rho.as = c(-0.147, 3.89182, 0.166)> names(rho.as) = c("alpha", "beta.t", "sigma.u")> emm.svas.sp500 = EMM(fit.sp500.11118000,coef=rho.as,+
EMM.control(n.burn=n.burn, n.sim=n.sim),+
gensim.fn = "sv.as.gensim",+
gensim.language = "SPLUS",+
gensim.aux = sv.as2.aux)
Iteration No. 1, objective = 255798...
Iteration No. 24, objective = 71.8841> emm.svas.sp500
Call:
EMM(score = fit.sp500.11118000, coef = rho.as, appcode =EMM.control(n.burn = n.burn, n.sim = n.sim), gensim.fn= "sv.as.gensim", gensim.language = "SPLUS", gensim.aux= sv.as2.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
alpha -0.2945
0.13010 -0.4059-0.1644
beta.t 3.4229
0.45495 3.0318
3.8778sigma.u 0.1759
0.03137 0.1445
0.1921
Final optimization:
Convergence: relative function convergence
Iterations: 24
EMM objective at final iteration: 71.88
P-Value: 1.916e-011 on 10 degrees of freedom
The small p-value on the final EMM objective value indicates that the SVmodel (23.25) is rejected by the S&P 500 returns as summarized throughthe SNP 11118000 model. A bar plot of the adjusted t-ratios associatedwith the EMM fit, produced using> plot(emm.svas.sp500)
is shown in Figure 23.6.
The large t-ratios on the z^4, z^5, and z^7 moments indicate the featuresof the returns that the SV model (23.25) cannot adequately describe.

966

23. Eﬃcient Method of Moments

Adjusted t-ratios of mean scoregarch(1)arch(1)
s0
ar(1)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1

-2

0

2

4

FIGURE 23.6. Adjusted t-ratios from the SNP 11118000 mean score associatedwith the EMM fit to the SV model (23.25) for the S&P 500 returns.

Gallant, Hsieh, and Tauchen (1997)
Gallant, Hsieh, and Tauchen (1997) considered the general univariate discretetime SV model
yt

= μ+

p
X

φj yt−j + exp(wt /2)σ z zt

j=1

wt

=

q
X

β j wt−j + σ u ut

j=1

where {zt } and {ut } are iid Gaussian random variables with mean zero, unit
variance, and correlation coeﬃcient ρ. The model allows for autoregressive
eﬀects in the mean and log-volatility. A negative correlation between theinnovations to the level and log-volatility allow for the so-called leverage
eﬀect.

23.4.3 Interest Rate Diﬀusion Models
In this subsection, the one-factor and two-factor interest diﬀusion models
for short-term interest rates introduced in Section 23.2 are fit by EMM.
The first example is based on Chan, Karolyi, Longstaﬀ, and Sanders (1992)
who fitted a one-factor model to monthly observations on the 3-month U.S.

23.4 Examples

967

T-bill. The second and third examples follow Andersen and Lund (1997)
who fittd one-factor and two-factor models to weekly observations on the
3-month U.S. T-bill.
CKLS (1992) One-Factor Model
Chan, Karolyi, Longstaﬀ, and Sanders (1992), hereafter CKLS, estimatedthe generalized Cox-Ingersoll-Ross (GCIR) one-factor continuous-time interest rate diﬀusion modelγ

drt = (α0 + β 0 rt )dt + σ 0 rt 0 dWt(23.26)

using GMM with four moment conditions derived from an Euler discretization. For their estimation, they used n = 307 monthly observations on the
3-month U.S. T-bill rate over the period June 1964 through December
1989.8The model (23.26) is of the form (23.22) where Xt = rt andγa(Xt , t) = (α0 + β 0 rt ), b(Xt , t) = σ 0 rt 0

(23.27)are the drift and diﬀusion functions, respectively. In (23.26), the drift
function a(Xt , t) = (α0 + β 0 rt )dt may be reparameterized as a(Xt , t) =
κ0 (μ0 − rt )dt where α0 = κ0 μ0 and β 0 = −κ0 . The parameter μ0 is thelong-run mean and the parameter κ0 determines the speed of mean reversion.
Simulator Function
Simulations of rt from the GCIR model (23.26) based on Euler’s method
using the S-PLUS function euler1d.pcode.gensim9 was discussed in Chapter 20. The auxiliary information to set the drift and diﬀusion functions(23.27) and to calibrate the simulation to monthly data on 3-month T-billsis> rho.names = c("alpha", "beta", "sigma", "gamma")> ckls.aux = euler.pcode.aux(+
drift.expr = expression(alpha + beta*X),+
diffuse.expr = expression(sigma*X^gamma),+
rho.names = rho.names,+t.per.sim = 1/12, ndt = 25,+
lbound=0, ubound=10)
Since the yields are annualized and the data are observed monthly, theargument t.per.sim is set to 1/12. Setting ndt=25 indicates that each
8 Chapter 21 also gives an example of estimating the GCIR model with monthly datausing GMM.
9 The function euler1d.pcode.gensim is more eﬃcient for simulating the solutions tosingle- factor SDEs than the more general function euler.pcode.gensim.

23. Eﬃcient Method of Moments

0.04

0.08

968

0

50

100

150

200

250

300

0.0

0.4

0.8

Series : ckls.sim2

0

5

10

15

20

Lag

0.0

0.4

0.8

Series : ckls.resid^2

0

5

10

15

20

Lag

FIGURE 23.7. Simulated annualized short-term interest rates, sampled monthly,from GCIR model.

monthly simulated value is based on 25 discretization points. The arguments lbound=0 and ubound=10 truncate the simulation to lie in the interval [0,10]. This prevents nonstationary parameterizations from generating
explosive simulations.
CKLS fitted the GCIR model (23.26) to n = 307 monthly observations
on the 3-month U.S. T-bill rate using GMM, and reported the estimatesα̂0 = 0.0408, β̂ 0 = −0.5921, σ̂ 20 = 1.6704, γ̂ = 1.4999(23.28)The estimates imply a long-run mean interest rate of 0.0408/0.5921 =0.069. Figure 23.7 shows a simulation of n = 300 annualized interest rates
sampled monthly, based on the above GMM estimates, computed using> rho.ckls = c(0.0408, -0.5921, sqrt(1.6704), 1.4999)> ckls.aux$seed = 123> ckls.aux$X0 = 0.0408/0.5921> ckls.sim = euler1d.pcode.gensim(rho.ckls,n.sim=300,+
n.burn=100, aux=ckls.aux)
The simulated rates have a long-run mean of about 0.069 and exhibit increased volatility as the level of rates rises. The SACF of the rates showshigh persistence in the mean. The SACF of the squared residuals from an
AR(1) model fit to the rates shows some evidence of stochastic volatility.

23.4 Examples

969

EMM Estimation on Simulated Data
The SNP auxiliary model for the simulated data is determined using thefunction SNP.auto:> ckls.sim = as.matrix(ckls.sim)> fit.snp.ckls = SNP.auto(ckls.sim,n.drop=8,+
control = SNP.control(xTransform="spline",+n.start = n.start,+
seed = 011667,+
fOld = fOld,+
fNew = fNew),+
xPolyMax=1,lagPMax=1)
where the parameters n.start, fOld and fNew are the same as those usedin fitting the SNP model for the simulations from the discrete-time SV
model discussed earlier. The SNP model that minimizes the BIC is a semiparameteric AR(1) - GARCH(1,1), 11111000, model:> coef(fit.snp.ckls)
Hermite Polynomial Coefficients:z^0z^1
1 -0.6622
Conditional Mean Coefficients:
mu ar(1)-0.3814 0.8125
Conditional Variance Coefficients:
s0 arch(1) garch(1)-0.0798 -0.2568
0.5421
A similar model was used by Jensen (2001) in his analysis of monthly
interest rate data using EMM. The above SNP model passes the standard
residual diagnostics. Figure 23.8 shows simulated values from the fitted
SNP model, along with the SACF of the simulated values and the SACF
of squared residuals from the SNP fit. The SNP model appears to capture
the relevant features of the simulated GCIR data.
Now, consider EMM estimation of the GCIR parameters from the simulated data based on the SNP 11111000 auxiliar model. EMM estimation
with N = 40, 000 simulated values and 100 burn-in values is computedusing> n.burn = 100> n.sim = 40000> ndt = 25
> set.seed(456)23. Eﬃcient Method of Moments

0.04

0.08

970

0

50

100

150

200

250

300

-0.2

0.2

0.6

1.0

Series : snp.ckls.sim

0

5

10

15

20

Lag

0.0

0.4

0.8

Series : snp.ckls.resid^2

0

5

10

15

20

Lag

FIGURE 23.8. Simulated data from SNP model fit to simulated values from
GCIR model.

> z.ckls = rnorm(ndt*(n.burn + n.sim))> rho.ckls.names = c("alpha", "beta", "sigma", "gamma")> ckls.aux = euler.pcode.aux(+
drift.expr = expression(alpha + beta*X),+
diffuse.expr = expression(sigma*X^gamma),+
rho.names = rho.ckls.names,+t.per.sim = 1/12, ndt = 25,+z = z.ckls, X0 = 0.06,+
lbound=0, ubound=10)> rho.ckls = c(0.04, -0.6, 1.3 , 1.5)
> emm.ckls.fit = EMM(fit.snp.ckls, coef=rho.ckls,+
control = EMM.control(n.sim=n.sim,+
n.burn=n.burn),+
gensim.fn = euler1d.pcode.gensim,+
gensim.language = "SPLUS",+
gensim.aux = ckls.aux, save.sim=T)
Iteration No. 1, objective = 248417...
Iteration No. 37, objective = 19.9188> emm.ckls.fit
Call:23.4 Examples

971

EMM(score = fit.snp.ckls, coef = rho.ckls, control =EMM.control(n.sim = n.sim, n.burn = n.burn), save.sim= T, gensim.fn = euler1d.pcode.gensim,
gensim.language = "SPLUS", gensim.aux = ckls.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
alpha 0.03545
0.004832 0.03416
0.03699
beta -0.57791
0.083459 -0.61014-0.55178
sigma 1.25784
0.151522 1.23507
1.29620
gamma 1.49369
0.036179 1.48007
1.50649
Final optimization:
Convergence: relative function convergence
Iterations: 37
EMM objective at final iteration: 19.92
P-Value: 4.728e-005 on 2 degrees of freedom
Interestingly, even though the starting values are set close to the true values,
EMM converges with a large objective value. It is possible that the EMM
estimates converged to a local minimum. To check this, the GCIR modelis re-estimated utilizing the random restart feature of EMM:
> rho.ckls = c(0.04, -0.6, 1.3 , 1)
> emm.ckls.fit.restart = EMM(fit.snp.ckls,coef=rho.ckls,+
control = EMM.control(n.sim = n.sim,+
n.burn = n.burn,+init.itmax = 10,+
final.itmax = 300,+
n.start = rep(5,3),+
tweak = c(0.25, 0.5, 1.0)),+
gensim.fn = euler1d.pcode.gensim,+
gensim.language = "SPLUS",+
gensim.aux = ckls.aux, save.sim=T)
Run 1 , 5 starts, tweak = 0.25
Iteration No. 1, objective = 248417...
Iteration No. 30, objective = 3.38473
The random restarts are controled by the arguments n.start=rep(5,3)
and tweak=c(0.25, 0.05, 1.0) to EMM.control. The starting values foreach coeﬃcient ρi are perturbed according to
ρi → ρi × (1 + u × tweak)(23.29)where u is a uniformly distributed random variable in (−1, 1). The optimization is restarted five times using starting values set by (23.29), with972

23. Eﬃcient Method of Moments

each of the three tweak constants specified in tweak and iterated 10 times.
The restart that generates the lowest objective function value after 10 iterations is then iterated to convergence. After allowing for random restarts,
EMM converges with a much smaller objective function value. The final
estimates are> emm.ckls.fit.restartCall:
EMM(score = fit.snp.ckls, coef = rho.ckls, control =EMM.control(n.sim = n.sim, n.burn = n.burn, n.start =rep(5, 3), tweak = c(0.25, 0.5, 1.)), save.sim = T,gensim.fn = euler1d.pcode.gensim, gensim.language ="SPLUS", gensim.aux = ckls.aux)
Minimum objective (out of 3 runs) occurred for control
parameters
tweak n.start1
5
Coefficients:
Value Std. Error
95% Conf. Int.
alpha 0.03246
0.006698 0.02425
0.05684
beta -0.51219
0.108073 -0.89322-0.37401
sigma 0.63430
0.170606 0.45965
0.82233
gamma 1.30524
0.099467 1.20165
1.42463
Final optimization:
Convergence: relative function convergence
Iterations: 30
EMM objective at final iteration: 3.385
P-Value: 0.1841 on 2 degrees of freedom
With random restarts of the optimizer, the EMM estimates converge with amuch smaller value of the objective function that does not reject the overidentifying restrictions. Interestingly, the EMM estimate of σ is considerably
smaller than the true value of 1.292 and it has the largest estimated standard error. Also, the EMM estimate of γ is smaller than its true value of
1.4999. As noted by Tauchen (1997), it is diﬃcult to estimate both σ andγ precisely when their values are large. The small sample sizes aﬀorded bymonthly data are likely to exercerbate this problem.
The GMM estimation based on moments derived from the Euler discretization used by CKLS was discussed in Chapter 21. Using the S+FinMetrics function GMM, the GMM estimates of the GCIR model parametersfrom the simulated data are> start.vals = c(0.06, -0.5, 1, 1)

23.4 Examples

973> names(start.vals) = c("alpha","beta","sigma","gamma")> gmm.ckls.sim = GMM(start.vals, ckls.moments, ts=T,+
data=data.ckls.sim, dt=1/12)> summary(gmm.ckls.sim)
Call:
GMM(start = start.vals, moments = ckls.moments, ts = T, data
= data.ckls.sim, dt = 1/12)
Coefficients:
Value Std.Error t value Pr(>|t|)alpha 0.0464 0.0157
2.9589 0.0033
beta -0.7309 0.2701-2.7064 0.0072
sigma 0.3981 0.2499
1.5932 0.1122
gamma 1.1082 0.2246
4.9336 0.0000
Test of Overidentification:model is just-identified
Optimization Info:
Number of Iterations: 5
Convergence: absolute function convergence
Notice that the GMM estimates based on four moments derived from theEuler discretization are more biased and less precise than the EMM estimates based on the six moments derived from the SNP 11111000 auxiliarymodel.
EMM Estimation on Monthly T-bill Data
Monthly observations on the U.S. T-bill rate over the period June 1964
through November 1989, similiar to the data analyzed by CKLS, are in the
S+FinMetrics “timeSeries” object ckls.ts. The SNP auxiliary modelfor the monthly T-bill data is determined using the function SNP.auto:> fit.snp.ckls.ts = SNP.auto(ckls.ts, n.drop=8,+
control = SNP.control(xTransform = "spline",+n.start = n.start,+
fOld = fOld, fNew = fNew),+
xPolyMax=1,lagPMax=1)
The BIC minimizing model is a semiparametric AR(1)-GARCH(1,1) model(11111000):> coef(fit.snp.ckls.ts)
Hermite Polynomial Coefficients:
z^0
z^1

974

23. Eﬃcient Method of Moments

1 -0.5984
Conditional Mean Coefficients:
mu ar(1)-0.2254 0.8532
Conditional Variance Coefficients:
s0 arch(1) garch(1)-0.0083 -0.1131
0.86
The above SNP model passes the standard residual diagnostics, and simulations from the model are dynamically stable.
The EMM estimation with N = 40, 000 simulated values and 100 burn-in
values is computed using> emm.ckls.ts.fit = EMM(fit.snp.ckls.ts, coef=rho.ckls,+
control = EMM.control(n.sim=n.sim,+
n.burn=n.burn),+
gensim.fn = euler1d.pcode.gensim,+
gensim.language = "SPLUS",+
gensim.aux=ckls.aux,save.sim=T)
Iteration No. 1, objective = 15897...
Iteration No. 42, objective = 3.70258> emm.ckls.ts.fit
Call:
EMM(score = fit.snp.ckls.ts, coef = rho.ckls, control =EMM.control(n.sim = n.sim, n.burn = n.burn), save.sim= T, gensim.fn = euler1d.pcode.gensim,
gensim.language = "SPLUS", gensim.aux = ckls.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
alpha 0.05821
0.05369 0.01217
0.1286beta -0.85281
0.82692 -1.88260-0.7343
sigma 1.11085
0.97791 0.16814
2.1487
gamma 1.44809
0.29755 1.16062
1.7611
Final optimization:
Convergence: relative function convergence
Iterations: 42
EMM objective at final iteration: 3.703
P-Value: 0.157 on 2 degrees of freedom

23.4 Examples

975

The EMM estimates converge with a low value of the objective function
such that the two overidentifying restrictions implied by the SNP 11111000
model are not rejected. The EMM estimates are similar to the GMM estimates (23.28) for the GCIR model obtained by CKLS, and the GMM
estimates using ckls.ts reported in Chapter 21. The estimated standard
error for the EMM estimate of σ, however, is very large and indicates considerable uncertainty about the value of σ. Although the GCIR model is
not rejected using a short span of monthly data, the analysis in the next
subsection shows that it may be rejected using a much longer span of weekly
data.
Andersen and Lund (1997) One Factor Model
Andersen and Lund (1997) considered fitting the one-factor GCIR model(23.26) to n = 2155 weekly observations on the annualized U.S. 3-monthT-bill rate (multiplied by 100) over the period January 1954 through April
1995. SNP models were fit to similar weekly interest rate data in the previous chapter. They fit the following reparameterized model
drt = κ(μ − rt )dt + σrtγ dWt , γ > 0(23.30)where μ represents the long-run mean and κ is a speed-of-adjustment parameter. They estimated (23.30) by EMM using a semiparametric AR(5)EGARCH(1,1) auxiliary model, and reported the following parameter estimates10 :κ̂ = 0.082, μ̂ = 6.279, σ̂ = 0.670, γ̂ = 0.676(23.31)
Notice that the estimates of both σ and γ are much smaller than thecorresponding estimates (23.28) based on monthly data obtained by CKLS.
Based on a large value of the EMM objective function, they rejected theone-factor model. Contrary to the results from EMM estimation of theone-factor model on a short span of monthly data, the one-factor model isstrongly rejected using a long span of weekly data.
Simulator Function
The auxiliary information to set the drift and diﬀusion functions of the onefactor model (23.30) for Euler’s method and to calibrate the simulation to
weekly data on the 3-month T-bill is> rho.gcir.names = c("k","mu","sigma","gamma")> gcir.aux = euler.pcode.aux(+
drift.expr = expression(k*(mu-X)),
10 These estimates are reported in Andersen and Lund (1997, Table 7, p. 370). Andersen and Lund did not use Gallant and Tauchen’s FORTRAN code to estimate SNP and
EMM models. Their EGARCH specification for the SNP model cannot be implemented
using Gallant and Tauchen’s SNP code.

23. Eﬃcient Method of Moments

5

10

15

20

25

30

976

0

500

1000

1500

2000

FIGURE 23.9. Simulated values from GCIR process calibrated to weekly observations on the annualized U.S. 3-month T-bill rate.

+
+
+
+

diffuse.expr = expression(sigma*X^gamma),
rho.names = rho.gcir.names,t.per.sim = 1/52, ndt = 25,
lbound=0, ubound=100)

For annualized data observed weekly, t.per.sim is set to 1/52. To computea solution path from Euler’s method with n = 2000 values and 1000 burn-in
values, based on the estimates (23.31), use> rho.gcir = c(0.082, 6.279, 0.670, 0.676)> gcir.aux$seed = 123> gcir.aux$X0 = 6> gcir.sim = euler1d.pcode.gensim(rho.gcir, n.sim=2000,+
n.var=1, n.burn=1000,+
aux=gcir.aux)
These values are shown in Figure 23.9. Notice that the simulated weekly
rates look more like actual rates than the simulated monthly rates shownin Figure 23.7.
EMM Estimation on Simulated Data
The SNP auxiliary model for the simulated weekly data is determined using
the function SNP.auto:

23.4 Examples

977> gcir.sim = as.matrix(gcir.sim)> fit.snp.gcir = SNP.auto(gcir.sim,n.drop=8,+
control = SNP.control(xTransform="spline",+n.start = n.start,+
fOld = fOld, fNew = fNew),+
xPolyMax=1,lagPMax=1)> coef(fit.snp.ckls.ts)
Hermite Polynomial Coefficients:z^0z^1
1 -0.5984
Conditional Mean Coefficients:
mu ar(1)-0.2254 0.8532
Conditional Variance Coefficients:
s0 arch(1) garch(1)-0.0083 -0.1131
0.86> coef(fit.snp.gcir)
Hermite Polynomial Coefficients:x^0x^1
z^0 1.0000 0.2037
z^1 -0.3793 -0.3124
Conditional Mean Coefficients:
mu ar(1)-0.0487 0.9537
Conditional Variance Coefficients:
s0 arch(1) garch(1)-0.0017 -0.0815
0.9057The BIC minimizing SNP model is a nonlinear nonparametric AR(1)GARCH(1,1) with Kz = 1 and Kx = 1, 11111010. The model passes theusual residual diagnostics, and simulations from the model appear dynamically stable.
The EMM estimation with N = 50, 000 simulated values and 1000 burnin values may be computed using> n.burn = 1000
> n.sim = 50000> ndt = 25
> set.seed(456)> z.gcir = rnorm(ndt*(n.burn+n.sim))

97823. Eﬃcient Method of Moments> gcir.aux$z = z.gcir> emm.gcir.fit = EMM(fit.snp.gcir,coef=rho.gcir,+
control=EMM.control(n.sim=n.sim,+
n.burn=n.burn),+
gensim.fn = euler.pcode.gensim,+
gensim.language = "SPLUS",+
gensim.aux = gcir.aux, save.sim=T)
Iteration No. 1, objective = 5099.04...
Iteration No. 31, objective = 1.73639> emm.gcir.fit
Call:
EMM(score = fit.snp.gcir, coef = rho.gcir, control =EMM.control(n.sim = n.sim, n.burn = n.burn), save.sim= T, gensim.fn = euler.pcode.gensim, gensim.language= "SPLUS", gensim.aux = gcir.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
k 0.2768
0.04432 0.2047
NAmu 6.8552
0.51202 6.1242
7.6481sigma 0.8490
0.11229 0.7006
1.0450
gamma 0.5554
0.05951 0.4608
0.6368
Final optimization:
Convergence: relative function convergence
Iterations: 31
EMM objective at final iteration: 1.736
P-Value: 0.7841 on 4 degrees of freedom
The EMM estimates converge with a low value of the objective functionthat does not reject the four overidentifying restrictions implied by the SNP
11111010 model. All of the estimates, except κ̂, are reasonably precise andclose to their true values. The asymmetric 95% confidence interval for κ
indicates that the EMM objective function is nonsmooth in κ near κ̂. As
Gallant and Tauchen (2002) pointed out, estimation can often be improvedby utilizing a very long simulation length (e.g., N = 100, 000) together withrandom restarts of the optimizer.
EMM Estimation on Monthly T-bill Data
Now, consider EMM estimation of the one-factor model using weekly observations on the U.S. 3-month T-bill contained in the first column of the
S+FinMetrics “timeSeries” object tbill.dat. The specification and estimation of SNP models for these data were discussed in the previous chapter.

23.4 Examples

SNP
11118000
10818000
51118000
10414010

κb
1.467(1.095)0.227(0.315)0.342(0.446)0.445(0.078)μb
5.559(1.419)4.080(3.023)8.219(1.731)
7.137(0.401)σb
0.685(1.073)0.1373(0.093)0.074(0.068)2.981(0.506)γb1.105(0.593)2.275(0.158)1.723(0.327)−0.4336(0.1172)

979

JEMM
35.7
(0.000)47.3(0.000)41.8(0.000)54.4(0.000)TABLE 23.5. EMM estimates of one-factor GCIR model for weekly 3-monthT-bill rates. Starndard errors are in parentheses for estimates; p-vaule in parenthesis for J statistic.

The following four models were suggested: 11118000, 10818000, 51118000
and 10414010. Table 23.5 shows the EMM estimates of the one-factor GCIR
model (23.30) based on the above four SNP models. Each model was estimated using N = 50, 000 simulations and 1000 burn-in values. For example,
the EMM estimates based on the SNP 11118000 model are computed using> n.burn = 1000
> n.sim = 50000> ndt = 25
> set.seed(456)> z.gcir= rnorm(ndt*(n.burn + n.sim))> gcir.aux$z = z.gcir> rho.gcir = c(0.082,6.279,0.670,0.676)> emm.gcir.11118000.fit = EMM(fit.tb3mo.11118000,+
coef=rho.gcir,+
control = EMM.control(n.sim=n.sim,+
n.burn=n.burn),+
gensim.fn = euler.pcode.gensim,+
gensim.language = "SPLUS",+
gensim.aux = gcir.aux, save.sim=T)
The one-factor model is rejected using all of the SNP models.11 Figure 23.10
shows the adjusted score t-ratios (23.21) for the four models. Generally, theone-factor model does not fit the Hermite polynomial terms of the SNPmodels.
Andersen and Lund (1997) Two-Factor Model
Given the rejection of the one-factor model using weekly data, Andersenand Lund (1997) considered the following two-factor extension of the GCIR
11 Similar results were obtained using N = 75, 000 simulations and random restarts.

980

23. Eﬃcient Method of Moments
Adjusted t-ratios of mean score

Adjusted t-ratios of mean scorearch(1)arch(2)arch(3)arch(4)arch(5)arch(6)arch(7)arch(8)
s0
ar(1)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1

garch(1)arch(1)
s0
ar(1)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1
-1.5

-1.0

-0.5

0.0

0.5

1.0

-6

-4

-2

0

2

SNP 11118000 Model

SNP 10818000 Model

Adjusted t-ratios of mean score

Adjusted t-ratios of mean scoregarch(1)arch(1)
s0
ar(1)
ar(2)
ar(3)
ar(4)
ar(5)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1

arch(1)arch(2)arch(3)arch(4)s0
ar(1)muz^4 x^1z^4 x^0z^3 x^1z^3 x^0z^2 x^1z^2 x^0z^1 x^1z^1 x^0z^0 x^1
-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

-2

-1

SNP 51118000 Model

0

1

2

3

SNP 10414010 Model

FIGURE 23.10. Adjusted t-ratios for SNP score elements from the EMM estimation of one-factor GCIR model (23.30) for weekly U.S. T-bill rates.

model:
drt
d ln σ 2t= κ1 (μ − rt )dt + σ t rtγ dW1t , γ > 0= κ2 (α − ln σ 2t )dt + ξdW2t(23.32)where W1t and W2t are independent Brownian motion processes. The specification implies mean reversion of the interest rate level as well as the (log-)
volatility. The parameter α is the long-run mean of log-volatility and μ is
the long-run mean for interest rates. Define vt = ln σ 2t = 2 ln σ t . Then,exp(vt /2) = σ t and (23.32) may be re-expressed as
drt
dvt= κ1 (μ − rt )dt + exp(vt /2)rtγ dW2t , γ > 0= κ2 (α − vt )dt + ξdW2t

Let Xt = (vt , rt )0 and Wt = (W1t , W2t )0 . Then (23.32) may be expressedin matrix form
dXt = a(Xt , t)dt + b(Xt , t)dWt
wherea(Xt , t) =µ

κ1 (μ − rt )
κ2 (α − vt )¶, b(X, t) =µexp(vt /2)rtγ
0

are the drift and diﬀusion functions, respectively.

0ξ¶(23.33)

23.4 Examples

981

Andersen and Lund estimated (23.32) by EMM using a semiparametricAR(5)-EGARCH(1,1) model and reported the following estimates12κ̂1 = 0.163, μ̂ = 5.95, γ̂ = 0.544, κ̂2 = 1.04, α̂ = −0.282, ξ̂ = 1.27
(23.34)
They reportedd an EMM objective function value of 24.25 with a p-valueof 0.019, and concluded that there is mild evidence against the two-factor
model for weekly U.S. 3-month T-bill rates.
Simulator Function
Simulated solutions for Xt from (23.32) based on Euler’s method may becomputed using the S-PLUS function euler.pcode.gensim. The auxiliary
information to set the drift and diﬀusion function (23.33) and to calibrate
the simulation to the weekly data on 3-month T-bills is> rho.tf.names = c("k1", "mu", "gamma", "k2", "alpha", "xi")> tf.aux=euler.pcode.aux(+
rho.names = rho.tf.names,+
drift.expr = expression(+
k1*(mu - X[1]),+k2*(alpha-X[2])),+
diffuse.expr = expression(+(exp((X[2]-5)*0.5))*X[1]^gamma,
+
0.0,
+
0.0,
+
xi),
+
N = 2, M = 2, t.per.sim = 1/52, ndt = 25,+
lbound = 0, ubound = 100,+
returnc = c(T,T))
Since there are two factors to simulate and the diﬀusion matrix is square,
N=2 and M=2. Setting returnc=c(T,T) specifies that simulated values ofboth rt and vt are returned by the simulator. To ensure that rt stays
positive, lbound=0. However, this lower bound applies to both rt andvt . Since vt = ln σt , vt can be both positive and negative. To allow vt tovary freely, the diﬀusion is reparameterized so that the level of vt is shifted
up by 5 units. As a result, the expression for the drift function for rt isexp((vt − 5)/2)rtγ instead of exp(vt /2)rtγ .
A solution path of length n = 2000, with 1000 burn-in values, using
Euler’s method calibrated to the estimates (23.34), may be computed using> tf.aux$seed = 678> tf.aux$X0 = c(6,5-0.28)
12 These estimates are reported in Andersen and Lund (1997, Table 6, p. 368). They
are based on a simulation length of N = 75, 000 with ndt = 25.

982

23. Eﬃcient Method of Moments

0

5

10

15

20

interest rates: r(t)

0

500

1000

1500

2000

1500

2000

-2

-1

0

1

2

3

log volatility: v(t) - 5

0

500

1000

FIGURE 23.11. Simulated values from two-factor GCIR process calibrated toweekly observations on annualized U.S. 3-month T-bill rate.

> tf.sim = euler.pcode.gensim(rho=rho.tf,+
n.sim=2000, n.var=2,+
n.burn=1000, aux=tf.aux)> tmp = matrix(tf.sim, 2000, 2, byrow=T)> tfird.sim = as.matrix(tmp[,1, drop=F])> logvol.sim = tmp[,2] - 5
To create the simulation, the initial value for vt is shifted up by 5 units.
Figure 23.11 shows the simulated values for rt and vt − 5.
EMM Estimation on Simulated Data
The SNP auxiliary model for the simulated data from the two-factor model
is determined using the function SNP.auto:> fit.snp.tfird = SNP.auto(tfird.sim,n.drop=8,+
control = SNP.control(xTransform="spline",+n.start = n.start,+
fOld = fOld,+
fNew = fNew),
lagPMax = 1)
The BIC minimizing SNP model is a nonlinear non-parametric AR(1)GARCH(1,1) with Kz = 4 and Kx = 1, 11114010:

23.4 Examples

983> coef(fit.snp.tfird)
Hermite Polynomial Coefficients:x^0x^1
z^0 1.0000 0.6246
z^1 -0.0765 -0.0517
z^2 -0.0515 -0.0085
z^3 -0.0045 -0.0022
z^4 0.0170 0.0099
Conditional Mean Coefficients:
mu ar(1)-0.0151 0.9885
Conditional Variance Coefficients:
s0 arch(1) garch(1)-0.0032 -0.1338
0.8569This model is similar to Gallant and Tauchen’s preferred SNP model forweekly interest rates. The model passes the usual residual diagnostics, andsimulations from the model appear dynamically stable.
The EMM estimation with N = 50, 000 simulated values and 1000 burnin values may be computed using> n.burn = 1000
> n.sim = 50000> ndt = 25
> set.seed(456)> z.tfird = rnorm(2*ndt*(n.burn + n.sim))> tfird.aux$z = z.tfird> tfird.aux$X0 = c(6,5-0.28)> tfird.aux$returnc = c(T,F))> rho.tfird = c(0.163, 5.95, 0.544, 1.04, 5-0.282, 1.27)> emm.tfird.fit = EMM(fit.snp.tfird, coef=rho.tfird,+
control = EMM.control(n.sim=n.sim,+
n.burn=n.burn),+
gensim.fn = euler.pcode.gensim,+
gensim.language = "SPLUS",+
gensim.aux = tfird.aux, save.sim=T)
Notice that for the two-factor model 2*ndt*(n.burn + n.sim), random
normals are required for Euler’s method. Also, since EMM only requires
simulations from rt , returnc=c(T,F) in tfird.aux. The results from theestimation are> emm.tfird.fit
Call:

984

23. Eﬃcient Method of Moments

EMM(score = fit.snp.tfird, coef = rho.tfird, control =EMM.control(n.sim = n.sim, n.burn = n.burn), save.sim= T, gensim.fn = euler.pcode.gensim, gensim.language= "SPLUS", gensim.aux = tfird.aux)
Coefficients:
Value Std. Error
95% Conf. Int.
k1 0.2631
0.06132 0.2019
0.4521mu 5.4689
0.73000 4.8023
6.4598gamma 0.4876
0.04728 0.4368
0.5648k2 2.0939
0.70286 1.2181
2.9536alpha 5.2284
0.14385 4.9863
5.3744xi 1.6774
0.29380 1.3268
2.1570
Final optimization:
Convergence: relative function convergence
Iterations: 29
EMM objective at final iteration: 5.546
P-Value: 0.698 on 8 degrees of freedom
The high p-value on the EMM objective function indicates that the twofactor model is not rejected by the SNP 11114010 model. All of the estimates are reasonable precise and close to their true values. The estimateof α is 5.2284 - 5 = 0.2284.
EMM estimation on monthly T-bill data
Now consider EMM estimation of the two-factor model using weekly observations on the U.S. 3-month T-bill rate. Table 23.6 shows the EMM estimates of the two-factor GCIR model (23.32) based on the same four SNP
models considered for the one-factor model. Each model was estimated using N = 50, 000 simulations and 1, 000 burn-in values13 . For example, theEMM estimates based on the SNP 11118000 model are computed using> n.burn = 1000
> n.sim = 50000> ndt = 25
> set.seed(456)> z.tfird = rnorm(2*ndt*(n.burn + n.sim))> tfird.aux$z = z.tfird> rho.tfird = c(0.163, 5.95, 0.544, 1.04, 5-0.282, 1.27)> emm.11118000.fit = EMM(fit.tb3mo.11118000, coef=rho.tfird,+
control = EMM.control(n.sim=n.sim,+
n.burn=n.burn),
13 Similar results were obtained using N = 75, 000 simulations.

23.4 Examples

κ̂1
μ̂
γ̂
κ̂2
α̂ + 5
ξ̂
JEMM
df11118000
1.292(1.623)6.848(3.371)0.705(0.725)0.879(1.650)
4.988(4.365)1.785(0.427)22.61(0.002)
710818000
1.215(3.825)7.031(0.893)0.014(1.535)2.623(3.079)3.501(6.227)3.251(0.484)32.02(0.002)1351118000
0.990(0.823)
10.532(2.869)0.813(0.507)1.457(0.983)3.576(2.923)1.945(0.305)29.03(0.002)11985

10414010
1.119(0.171)5.977(0.314)0.133(0.063)0.775(0.151)4.656(0.275)1.566(0.144)40.77(0.000)10

TABLE 23.6. EMM estimates of two factor GCIR model for weekly 3-month
T-Bill rates. Starndard errors are in parentheses for estimates; p-vaule in parenthesis for J statistic

Adjusted t-ratios of mean score

Adjusted t-ratios of mean scorearch(1)arch(2)arch(3)arch(4)arch(5)arch(6)arch(7)arch(8)
s0
ar(1)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1

garch(1)arch(1)
s0
ar(1)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1
-1.5

-1.0

-0.5

0.0

0.5

1.0

-2

-1

0

1

2

3

SNP 11118000 Model

SNP 10818000 Model

Adjusted t-ratios of mean score

Adjusted t-ratios of mean scoregarch(1)arch(1)
s0
ar(1)
ar(2)
ar(3)
ar(4)
ar(5)
mu
z^8
z^7
z^6
z^5
z^4
z^3
z^2
z^1

4

arch(1)arch(2)arch(3)arch(4)s0
ar(1)muz^4 x^1z^4 x^0z^3 x^1z^3 x^0z^2 x^1z^2 x^0z^1 x^1z^1 x^0z^0 x^1-1

0

1

SNP 51118000 Model

2

-2

-1

0

1

2

SNP 10414010 Model

FIGURE 23.12. Adjusted t-ratios for SNP score elements from EMM estimationof two-factor GCIR model (23.32) for weekly U.S. T-bill rates.

986

23. Eﬃcient Method of Moments+++

gensim.fn = euler.pcode.gensim,
gensim.language = "SPLUS",
gensim.aux = tfird.aux, save.sim=T)

The one-factor model is rejected using all of the SNP models. Figure 23.12
shows the adjusted score t-ratios (23.21) for the four models. As with the
one-factor model, the two-factor model generally does not fit the Hermite
polynomial terms of the SNP models.

23.5 References
Ahn, D-H., R.F. Dittmar and A.R. Gallant (2002). “Quadratic Term
Structure Models: Theory and Evidence,” Review of Financial Studies,
15(1), 243-288.
Andersen, T.G., L. Benzoni and J. Lund (2002). “An Empirical Foundation of Continuous-Time Equity Return Models,” Journal of Finance, 57,
1239-1284.
Andersen, T.G., H.-J. Chung and B.E. Sorensen (1999). “Eﬃcient
Method of Moments Estimation of a Stochastic Volatility Model: A Monte
Carlo Study,” Journal of Econometrics, 91, 61-87.
Andersen, T.G. and J. Lund (1997). “Estimating Continuous-Time
Stochastic Volatility Models of the Short-Term Interest Rate,” Journal of
Econometrics, 77, 343-377.
Andersen, T.G. and B.E. Sorensen (1996). “GMM Estimation of a
Stochastic Volatility Model: A Monte Carlo Study,” Journal of Businessand Economic Statistics, 14, 328-352.
Brandt, M. and P. Santa-Clara (2002). “Simulated Likelihood Estimation of Diﬀusions with an Application to Exchange Rate Dynamics inIncomplete Markets,” Journal of Financial Economics, 63, 161-210.
Chan, K.C., G.A. Karolyi, F.A. Longstaff and A.B. Sanders(1992). “An Empirical Comparison of Alternative Models of the Term
Structure of Interest Rates,” Journal of Finance, 47, 1209-1227.
Chernov, M., A.R. Gallant, E. Ghysels and G. Tauchen (2003).
“Alternative Models for Stock Price Dynamics,” Journal of Econometrics,
116, 225-257.
Chumacero, R. A. (1997). “Finite Sample Properties of the Eﬃcient
Method of Moments,” Studies in Nonlinear Dynamics & Econometrics, 2,35-51.

23.5 References

987

Chung, C-C. and G. Tauchen (2001). “Testing Target Zone Models
Using Eﬃcient Method of Moments,” Journal of Business and Economic
Statistics, 19(3), 255-277.
Coppejans, M. and A.R. Gallant (2002). “Cross Validated SNP Density Estimates,” Journal of Econometrics, 110(1), 27-65.
Dai, Q. and K.J. Singleton (2000). “Specification Analysis of Aﬃne
Term Structure Models,” Journal of Finance, 55(5), 1943-1978.
Duffie, D. and K.J. Singleton (1993). “Simulated Moments Estimation of Markov Models of Asset Prices,” Econometrica, 61, 929-952.
Durham, G.B. and A.R. Gallant (2002). “Numerical Techniques for
Maximum Likelihood Estimation of Continuous Time Diﬀusion Precesses,”
Journal of Business and Economic Statistics, 20(3), 297-338.
Elerian, O., S. Chib and N. Shephard (2001). “Likelihood Inference
for Discretely Observed Nonlinear Diﬀusions,” Econometrica, 69, 959-994.
Fisher, R.A. (1912). “On an Absolute Criterion for Fitting Frequency
Curves,” Messages of Mathematics, 41, 155-157.
Gallant, A.R. (1980). “Explicit Estimators of Parametric Functions in
Nonlinear Regression,” Journal of the American Statistical Association 75,
182-193.
Gallant, A.R., D.A. Hsieh and G.E. Tauchen (1991). “On Fitting
a Recalcitrant Series: The Pound/Dollar Exchange Rate, 1974-83,” in W.
A. Barnett, J. Powell, G. E. Tauchen (eds.), Nonparametric and Semiparametric Methods in Econometrics and Statistics, Proceedings of the Fifth International Symposium in Economic Theory and Econometrics. Cambridge
University Press, Cambridge, pp. 199-240.
Gallant, A.R., D.A. Hsieh and G.E. Tauchen (1997). “Estimation of
Stochastic Volatility Models with Diagnostics,” Journal of Econometrics,
81(1), 159-192.
Gallant, A.R. and J. Long (1997). “Estimating Stochastic Diﬀerential
Equations Eﬃciently by Minimum Chi-squared,” Biometrika, 84, 125-141.
Gallant, A.R. and D.W. Nychka (1987). “Seminonparametric Maximum Likelihood Estimation,” Econometrica, 55, 363-390.
Gallant, A.R. and G. Tauchen (1989). “Seminonparametric Estimation of Conditionally Constrained Heterogeneous Processes: Asset Pricing
Applications,” Econometrica, 57, 1091-1120.

988

23. Eﬃcient Method of Moments

Gallant, A.R. and G. Tauchen (1996). “Which Moments to Match,”
Econometric Theory, 12, 657-681.
Gallant, A.R. and G. Tauchen (1999). “The Relative Eﬃciency of
Method of Moments Estimators,” Journal of Econometrics, 92, 149-172.
Gallant, A.R. and G. Tauchen (2001). “Eﬃcient Method of Moments,”
unpublished manuscript, Department of Economics, University of North
Carolina.
Gallant, A.R. and G. Tauchen (2002). “EMM: A Program for Eﬃcient
Method of Moments, Version 1.6, User’s Guide,” available at ftp.econ.econ.duke.edu.
Gallant, A.R. and G. Tauchen (2005). “Simulated Score Methods andIndirect Inference for Continuous-time Models,” in Y. Ait-Sahalia and L.P.
Hansen (eds.), Handbook of Financial Econometrics. Elsevier Science B.V.,
Amsterdam.
Gauss, C.F. (1816). “Bestimmung der Genauigkeit der Beobachtungen,”Zeitschrift für Astronomie und verwandte Wissenschaften, 1, 185-196.
Gennotte, G. and T.A. Marsh (1993). “Variations in Economic Uncertainty and Risk Premiums on Capital Assets,” European Economic Review,37, 1021-1041.
Ghysels, E., A.C. Harvey and E. Renault (1996). “Stochastic Volatility,” in G.S. Maddala and C.R. Rao (eds.), Handbook of Statistics, Vol. 14.
Elsevier Science B.V., Amsterdam.
Gourieroux, C., A. Monfort and E. Renault (1993). “Indirect Inference,” Journal of Applied Econometrics, 8, S85-S118.
Hansen, L.P. (1982). “Large Sample Properties of Generalized Methodof Moments Estimators,” Econometrica, 50, 1029-1054.
Heyde, C.C. and R. Morton (1998). “Multiple Roots in General Estimating Equations,” Biometrika, 85, 949—953.
Ingram, B.F. and B.S. Lee (1991). “Simulation Estimation of Time
Series Models,” Journal of Econometrics 47, 197-250.
Jensen, M. B. (2001). “Eﬃcient Method of Moments Estimation of theLongstaﬀ and Schwartz Interest Rate Model,” unpublished manuscript,
Department of Business Studies, Aalborg University, Denmark.

23.5 References

989

Jiang, G.J. and P.J. van der Sluis (2000). “Option Pricing with theEﬃcient Method of Moments,” in Y. S. Abu-Mostafa, B. LeBaron, A. W. Lo
and A. S. Weigend (eds.), Computational Finance. MIT Press, Cambridge,
MA.
Karatzas, I. and S.E. Shreve (1991). Brownian Motion and Stochastic
Calculus, 2nd ed. Springer-Verlag, Berlin.
Kloeden, P.E. and E. Platen (1992). Numerical Solution of Stochastic
Diﬀerential Equations. Springer-Verlag, Berlin.
Liu, M. (2000). “Modeling Long Memory in Stock Market Volatility,” Journal of Econometrics, 99, 139-171.
Lo, A.W. (1988). “Maximum Likelihood Estimation of Generalized Ito
Process with Discretely Sampled Data,” Econometric Theory, 4, 231-247.
Mallet, A., F. Mentré, J-L. Steimer and F. Lokiec (1988). “Nonparametric Maximum Likelihood Estimation for Population Pharmacokinetics, with Application to Cyclosporine,” Journal of Pharmacokinetics andBiopharmaceutics, 16, 311-327.
Nagypal, E. (2001). “Learning-by-Doing versus Selection: Can We Tell
Them Apart?” unpublished manuscript, Department of Economics, Stanford University.
Neyman, J. and E.S. Pearson (1928). “On the Use and Interpretationof Certain Test Criteria for Purposes of Statistical Inference,” Biometrika,
20A, 175-240.
Ng, S. and A. Michaelides (2000). “Estimating the Rational Expectations model of Speculative Storage: A Monte Carlo Comparison of Three
Simulation Estimators, Journal of Econometrics, 96, 231-266.
Olsen, L.F. and W.M. Schaffer (1990). “Chaos Versus Noisy Periodicity: Alternative Hypotheses for Childhood Epidemics,” Science, 249,
499-504.
Pearson, K. (1894). “Contributions to the Mathematical Theory of Evolution,” Philosophical Transactions of the Royal Society of London, SeriesA, 185, 71-78.
Shephard, N. (1996). “Statistical Aspects of ARCH and Stochastic Volatility”, in D.R. Cox, D.V. Hinkley and O.E. Barndorﬀ-Nielsen (eds.), Time
Series Models: In Econometrics, Finance and Other Fields. Chapman &Hall, London.

990

23. Eﬃcient Method of Moments

Smith, A.A. (1993). “Estimating Nonlinear Time Series Models Using
Simulated Vector Autoregressions,” Journal of Applied Econometrics, 8,
S63-S84.
Tauchen, G. (1997). “New Minimum Chi-square Methods in Empirical
Finance,” in D. Kreps and K. Wallis (eds.), Advances in Econometrics,
Seventh World Congress. Cambridge University Press, Cambridge, pp. 279317.
Valderrama, D. (2001). “Can a Standard Real Business Cycle Model
Explain the Nonlinearities in U.S. National Accounts Data?” Ph.D. thesis,
Department of Economics, Duke University.
van der Sluis, P.J. (1999). “Estimation and Inference with the Eﬃcient
Method of Moments: With Application to Stochastic Volatility Models andOption Pricing,” Research Series Paper No. 204, Tinbergen Institute, Amsterdam.

This is page 991
Printer: Opaque thisIndex

acf function, 67, 104
acf.FARIMA function, 272
acf.plot function, 105
additive outlier, 636
aggregateSeries function, 32, 47,
315
Akaike information criterion, seemodel selection criteriaalign function, 35, 328
ar function, 272
ARCH model, 84, 226
ARCH test, 228, 237
Archimax copulas, 735
Archimedean copulas, 732
Archimedean generator, 732
archTest function, 228, 492
arima.diag function, 78
arima.forecast function, 78, 80
arima.fracdiff function, 285
arima.mle function, 78
arima.rob function, 637
arima.sim function, 66
asymp.var function, 87, 108
asymptotic principal component
analysis (APCA), 606

autocorrelation function (ACF),
58
autocorTest function, 63, 194, 237,
492
autoregressive distributed lag(ADL) model, 202
autoregressive fractionally integrated
moving average (ARFIMA),
99
autoregressive integrated moving
average (ARIMA) model,
76
autoregressive moving average (ARMA)
model
estimation, 76
representation, 65
state space form, 528
backtesting, 349
Bayesian vector autoregression (BVAR)
model, 424
BB1 copula, 733
BB2 copula, 734
BB3 copula, 734
BB4 copula, 735

992

Index

BB5 copula, 732
BB6 copula, 734
BB7 copula, 734
BDS statistic, 655
BDSTest function, 655
BEKK model, 498
bivd function, 743
bond.discount function, 620
bond.forward function, 620
bond.spot function, 620
BVAR function, 424
CCC model, 500
chain rule of forecasting, 399
characteristic equation
AR(1), 66
AR(p), 69
CheckSsf function, 523
CIR.aux function, 767
CIR.gensim function, 767
Clayton copula, 733
coint function, 463
cointegration
cointegrating residual, 435
common trends, 437
definition, 435
estimation by least squares,
450
Johansen’s maximum likelihood estimation, 467
normalization, 435
specification of deterministic
terms, 459
triangular representation, 437
cointegration tests
residual-based tests
using a pre-specified cointegrating vector, 445
using an estimated cointegrating vector, 447
colStdevs function, 103
colVars function, 103
common stochastic trends, 437
communality, 591
comonotonic, 725

compare.mgarch function, 260
concordance, 727
concordant, 727
condition number, 206
conditional forecasting
BVAR, 427
VAR, 405
contour.pcopula function, 738
contour.plot function, 738
countermonotonic, 725
covariance matrix
EWMA, 483
QMLE, 234
robust, 235
covRob function, 502
cpredict function, 405, 427, 474
cpredict.VAR function, 405
cross lag correlation, 103
cubic spline, 37, 621
CUSUM test, 218
CUSUMSQ test, 218d.pgram function, 282
d.ros function, 281
d.whittle function, 284
dcopula function, 741
dgev function, 145
dgpd function, 160
diagonal VEC model, 486
Diebold-Mariano (DM) statistic,354
diff.timeSeries function, 42
diﬀerence stationary process, 94
disaggregate function, 37
discordant, 727
discount rate, 618
dos.time function, 320
Durbin-Watson statistic, 185
dynamic OLS estimator, 451
EGARCH model, 241
EMM estimator, 936
EMM function, 938
EMM.control function, 940
empirical copula, 747

Indexempirical copula frequency, 748
empirical.copula function, 748
end function, 17
endogenous variable, 786
Engle-Granger two-step procedure,
447
ergodicity, 58
error correction model (ECM), 441
ESTAR model, 679
Euler’s method, 761
euler.pcode.aux function, 771
euler.pcode.gensim function, 771
euler.pcode.test function, 772
euler1d.pcode.gensim function,771
ev.copula function, 737
EVANESCE library, 142, 714
EVIS library, 142
EWMA covariance matrix, 483
EWMA function, 325
EWMA.cov function, 484
expand function, 886
expected shortfall (ES), 157, 171
exponentially weighted moving average (EWMA), 324
extreme value (EV) copula, 730
extreme value theory, 144
factanal function, 592
factor analysis, 590
factor loading, 570
factor mimicking portfolio, 582
FARIMA function, 286
fevDec function, 415
fgarch function, 298
fgarch.control function, 304
FIEGARCH model, 297
FIGARCH model, 297
Fisher-Tippet theorem, 144
forecast error variance decomposition (FEVD), 414
forecast evaluation statistics, 350
forecasting
ARIMA, 77
BVAR, 425

993

GARCH, 262
long memory, 304
multivariate GARCH, 510
state space models, 544
time series regression, 200
VECM, 474
forward rate, 618
fplot function, 603
fractional ARIMA model, see autoregressive fractionallyintegrated moving average (ARFIMA)
fractional diﬀerence, 274
fractional white noise process, 98
fractionally integrated process, 98
Frank copula, 733
fundamental factor model, 580
Galambos copula, 731
GARCH
prediction, 263
simulation, 265
unconditional variance, 230,
245
garch function, 232
GARCH model, 229
GARCH-M model, 251
Garman-Klass estimator, 286
generalized error distribution, 257
generalized extreme value (GEV)
distribution, 144
Frechet type, 144
Gumbel type, 144
maximum likelihood estimation, 148
Weibull type, 145
generalized least squares, 208
generalized method of moments,
785
linear GMM, 786
continuous updating eﬃcient
GMM estimator, 790
eﬃcient GMM estimator,
789
GMM estimator, 788

994

Index

iterated eﬃcient GMM estimator, 790
J-statistic, 791
order condition, 787
two-stage-least-squares, 793
two-step eﬃcient GMM estimator, 790
generalized Pareto distribution (GPD),
159
maximum likelihood estimation, 164
getReturns function, 46
GetSsfArma function, 529
GetSsfReg function, 533
GetSsfRegArma function, 536
GetSsfSpline function, 538
GetSsfStsm function, 532
gev function, 147
GJR model, 242
GMM function, 797
gpd function, 165
gpd.2p function, 748
gpd.q function, 173
gpd.sfall function, 173
gpd.tail function, 167
GPH test, 278
gphTest function, 279
Granger causality, 407
Gumbel copula , 731
gumbel function, 147
gumbel.copula function, 737
Hüsler and Reiss copula, 731
half life, 67, 203
GARCH, 248
heteroskedasticity, 209
heteroskedasticity and autocorrelation consistent standard
error (HACSE), 214
heteroskedasticity consistent standard error (HCSE), 210
heteroskedasticity test, 211
Breusch-Pagan test, 211
Koenker-Basset test, 212
White test, 212

heteroTest function, 212
high quantile, 141
Hill estimator, 174
hill function, 175
histPlot function, 53, 63
Hurst coeﬃcient, 274
I(0) process, 95
I(1) process, 94
IC function, 193, 205
iEMA.kernel function, 332
IGARCH model, 296, 486
iMA function, 333
iMA.kernel function, 332
image.plot function, 610
immediate impact multiplier, 202
impRes function, 411
impulse response function (IRF)
multivariate, 409
univariate, 65
inference from margins estimator,
751
inhomogeneous time series functions, 330
inhomogenous time series, 327
innovation outlier, 637
interpNA function, 40
invertibility, 73
IRD.gensim function, 769
Joe copula, 733
Johansen’s maximum eigenvalue
statistic, 462
Johansen’s trace statistic, 461
Kalman filter, 543
KalmanFil function, 545
KalmanSmo function, 546
Kendall’s tau statistic, 727
Kendalls.tau function, 741
Kimeldorf and Sampson copula,733
lag operator, 65
lapply function, 352

Indexlevel shift, 637
leverage eﬀects, 241, 504
linear interpolation, 328
linear process, 64
linear time series regression model,
182
asymptotic distribution, 183
ordinary least squares (OLS)
estimation, 183lines.render function, 51
Ljung-Box, see serial correlation
test
loadings function, 595
long memory, 98, 273
long range dependence, 273
long run eﬀect, 203long run variance
autoregressive estimate, 87, 108
multivariate, 107
Newey-West estimate, 87, 108
univariate, 86
long-run equilibrium, 435
LSTAR model, 679
macroeconomic factor model, 571
Markov switching AR(p) process,690
Markov switching state space models, 692
martingale, 83
martingale diﬀerence sequence (MDS),
83
matrix-diagonal model, 496
mchain.p0 function, 689
mean excess function, 162
mean excess plot, 163
meplot function, 163
mfactor function, 599
mgarch function, 485
mimic function, 604
model selection criteria
ARMA models, 77
VAR model, 390
momentum indicator functions, 338
multifactor model, 570

995

multivariate portmanteau test, 493
Nelson-Siegel function, 628
Newey-West estimate, 87, 108, 214
news impact curve, 245
nlregb function, 376
NLSUR function, 375
nonlinear nonparametric VAR model,
869
nonlinearTest function, 665
nonparametric cubic spline model,
see cubic spline
nonstationary time series, 93
normal copula, 729
normal mixture copula, 730
normal.copula function, 736
normality test
Jarque-Bera test, 61
Shapiro-Wilk test, 61
normalTest function, 61, 194
OLS function, 185
orthogonal impulse response function, 410
OU.aux function, 766
OU.gensim function, 766
outliers function, 646
par function, 50
partial autocorrelation function(PACF), 69
pbivd function, 744
pcoint function, 448
pcopula function, 741
pdl function, 208
peaks over thresholds (POT), 157
periodogram method, 282
perpMat function, 465
persp.pcopula function, 738
PGARCH model, 243
pgev function, 145
pgpd function, 160
plot function, 48
plot.OLS function, 194
plot.timeSeries function, 48

996

Index

plot.VAR function, 395
polynomial distributed lag (PDL)model, 207
positions function, 16
predict.BVAR function, 425
predict.FARIMA function, 305
predict.SETAR function, 670
predict.STAR function, 685
predict.VAR function, 400
prediction error decomposition, 552
previous tick interpolation, 328
principal component analysis (PCA),
597
principal component GARCH, 502
princomp function, 599
punitroot function, 116
qcoint function, 448
qgev function, 145
qgpd function, 160
qnorm function, 60
qplot function, 161
qq-plot, 60
qqPlot function, 63
qqPlot function, 53, 194
quadratic spline, 621
quant function, 174
qunitroot function, 116
R/S statistic, 276, 280
random walk, 95
random walk model, 88
rbivd function, 744
rcoupla function, 741
records function, 149
recursive causal ordering, 409
recursive least squares (RLS), 217
REGARIMA model, 636
return level, 153
rgev function, 145
rgpd function, 160
riskmeasure function, 172
rlevel.gev function, 154
RLS function, 219
rnorm function, 58

roll function, 358
rolling correlations, 321
rolling linear regression model, 342
rollMax function, 318
rollMin function, 318
rollOLS function, 343
rollVar function, 318
rosTest function, 277
rotate function, 595
rvfPlot function, 253
S language, 1
S+FinMetrics, vii
S+FinMetrics time and date functions, 28
sample autocorrelation, 58
sapply function, 353
Schwarz information criterion, seemodel selection criteria
screeplot.mfactor function, 602
seemingly unrelated regression (SUR)
model, 364
feasible generalized least squares(FGLS) estimator, 365
maximum likelihood estimator, 366
nonlinear model, 374
semi-parametric GARCH model,
875
semi-parametric VAR, 861
SEMIFAR function, 293
SEMIFAR model, 293
serial correlation test
modified Q-statistic, 62
multivariate, 493
Q-statistic, 62
seriesData function, 17
seriesMerge function, 39
seriesPlot function, 53
SETAR function, 668
SETAR model, 663
shape function, 166
shape.plot function, 166
Sharpe’s single index model, 572
sigma.t function, 235, 490

Indexsigma.t.SNP function, 892
simple moving average, 315
SimSmoDraw function, 565
simulate function, 897
simulate.FARIMA function, 99
simulate.garch function, 84, 226,
265
simulate.mgarch function, 515
simulate.VAR function, 387
simulation smoothing, 565
SMA function, 318
smooth transition autoregressive(STAR) models, 678
smooth.spline function, 626
smoothing spline, 624
SNP function, 851
SNP.auto function, 889
SNP.control function, 852
SNP.density function, 863
SNP.model function, 852
Spearman’s rho statistic, 728
Spearmans.rho function, 741
spot interest rate, 618
spurious regression, 432, 434
SsfCondDens function, 550
SsfFit function, 554
SsfFitMS function, 695
SsfLoglikeMS function, 695
SsfMomentEst function, 548
SsfPack library, 519
SsfSim function, 540
STAR function, 683
start function, 17
state space model, 520
stationarity, 58
stationarity tests, 112
KPSS test, 129
stationaryTest function, 131
stochastic diﬀerential equations,
759
Ornstein-Uhlenbeck process,765
stochastic trend, 95
stress period, 154
strong order 1 scheme, 763

997

strong1.pcode.aux function, 771
strong1.pcode.gensim function,
771
structural time series model (STSM),
531
summaryStats function, 191
SUR function, 367
Svensson function, 631
system of linear regression equations, 362
system of nonlinear regression equations, 362
t distribution, 257
multivariate, 508
t-ratio, 184
TA.adi function, 341
TA.chaikin function, 340
TA.garmanKlass function, 286
TA.macd function, 339
TA.typicalPrice function, 338
tail dependence, 728
tail index, 144
tail probability, 141
tail.index function, 741
tailplot function, 166, 173
TAR function, 674
technical analysis, 337
technical analysis functions, 337
term.struct function, 621
term.struct.nsx function, 629
TGARCH model, 242
threshold autoregressive (TAR) models, 662
timeCalendar function, 24
timeDate function, 19
timeDate object, 19
timeSeq function, 25
timeSequence function, 25
timeSeries object, 16
creating, 28
timeSpan object, 23
timeZoneConvert function, 22
trend stationary process, 94

998

Index

triangular structural VAR model,409
tslag function, 41, 204
Twan copula, 732
two components model, 248
uniqueness, 591
unit root test, 112
augmented Dickey-Fuller (ADF)
test, 120
DF-GLS test, 134
Dickey-Fuller test, 114eﬃcient tests, 133
Phillips-Perron (PP) test, 127
point optimal test, 133unit root tests
eﬃcient modified PP tests, 134
modified information criteria,
135
unitroot function, 122
Value-at-Risk (VaR), 157, 171
VAR function, 390
VaR.exp.sim function, 756
var.hac function, 797
variance inflation statistics, 206
variance ratios, 88mean averting, 89
mean reverting, 89
test statistic, 90
vector autoregression (VAR) model,
106, 386
conditional forecasts, 405
estimation by least squares,389
simulation-based forecasts, 402
traditional forecasting, 398
vector error correction(VECM), 456
volatility indicator functions, 340
volume indicator functions, 341
volume weighted average price, 34
Wald statistic, 184
waldTest function, 192

weak order 2 scheme, 764
weak2.pcode.aux function, 771
weak2.pcode.gensim function, 771
weighted least squares, 209
white noise, 58
Whittle’s method, 284
Wold decomposition, 64
Yule-Walker equations, 69