Estimating the Single Index Model

Eric Zivot

August 15, 2013

Estimating the Single Index Model
Sharpe’s Single (SI) model: =  +  +   = 1     2 ) 2
 ∼ iid (0  ∼ iid (   )cov( ) = 0 for  2 + 2
[] =  =  +   var() = 2 =  − cov( ) == 2var()2Main parameters to estimate:   and 

Plug-in Principle Estimators
Plug-in principle: Estimate model parameters using sample statistics̂̂ = 2̂
1 X̂ =( − ̂)( − ̂ ) − 1 =12 =̂
1 X( − ̂ )2
 − 1 =1

1 X
̂ =
 =1

1 X
̂ = =1

Plug-in principle estimator for  =  −  :̂ = ̂ − ̂̂
Plug-in principle estimator of  :̂ =  − ̂ − ̂
2 = var( ) :Plug-in principle estimator for 
X
12 =
̂
̂2
 − 2 =1

 ³
´2
1 X=
 − ̂ − ̂
 − 2 =1

Least Squares Estimation of SI Model Parameters
Idea: SI model postulates a linear relationship between  and  withintercept  and slope  : =  +  + Estimate  and  by finding the “best fitting line” to the scatterplot of data

• Problem: How to define the “best fitting line”?
• Least Squares solution: minimize the sum of squared residuals (errors)Least Squares Algorithm
̂ = initial guess for ̂ = initial guess for ̂ = ̂ + ̂ = fitted line̂ =  − ̂=  − (̂ + ̂) = residual

Determine the best fitting line by minimizing the Sum of Squared Residuals(SSR)
SSR(̂ ̂) ==X

̂2

=1
 ³X
=1

 − ̂ − ̂´2

That is, the least squares estimates solvemin SSR(̂ ̂) =̂̂ ³X

=1

 − ̂ − ̂´2

Note: Because SSR(̂ ̂) is a quadratic function in ̂ ̂ the first orderconditions for a minimum give two linear equations in two unknowns and so
there is an analytic solution to the minimization problem that we can find using
calculus.

Calculus Solution
The first order conditions for a minimum areXX
SSR(̂ ̂)0== −2( − ̂ − ̂) = −2̂ ̂
=1
=10=

SSR(̂ ̂) ̂= −2

X( − ̂ − ̂) = −2

=1X̂

=1

These are two linear equations in two unknowns. Solving for ̂ and ̂ gives̂ = ̂ − ̂̂̂
̂ = 2̂
which are exactly the plug-in principle estimators!

2 and  − Estimators for Utilize plug-in principle̂ =  − ̂ − ̂2 =̂

̂ =1 X
̂2 − 2 =1r

2 = SER̂= standard error of regression

Remarks• ̂ typical magnitude of residual = standard error of regression (SER)
2• Divide by  − 2 to get unbiased estimate of •  − 2 = degrees of freedom = sample size - number of estimated parameters ( and )Recall2 22 =  22

=1− 2

= % of variability due to market

Estimate using plug-in principle2̂2̂
2
̂ =
̂22
̂
=1− 2
̂

Least Squares Estimation Using RR commandlm - linear model estimation
Syntaxlm.fit = lm(y~x,data=my.data.df)
my.data.df = data frame with columns named y and x
Note: y~x is formula notation in R. It translates as the linear model =  +  + 
For multiple regression, the notation y~x1+x2 implies =  + 11 + 22 + Important method functions for lm objects
summary(): summarize model fit
plot(): plot results
residuals(): extract residuals
fitted(): extract fitted values
coef(): extract estimated coeﬃcients
confint(): extract confidence intervals

Least Squares Estimates are Maximum Likelihood Estimates Under Normal Distribution Assumption
 =  +  +   = 1     2 ) 2 ∼ iid (0  ∼ iid (   )Then2 )
| ∼ ( +  ⎛2 )−12 exp ⎝ (|) = (2−12⎠(−+)22−2 )ln(2) − ln(221 X( −  + )2− 22 =1ln (|R R ) =⎞2 )0 gives the least
Maximizing ln (|R R ) with respect to  = (  squares estimates!

Statistical Properties of Least Squares Estimates
Assuming the SI model generates the observed data, the estimators2̂ ̂ and ̂

are random variables.
Properties
2 are unbiased estimators• ̂ ̂ and ̂

[̂] = [̂] = 2 ] = 2[̂c ̂ ) and SE(c ̂ )• Analytic standard errors are available for SE(vuu X
̂
1u

2c ̂ ) = q
SE(·t2 =1
 · ̂̂
cSE(̂) = q

2
 · ̂

These are routinely reported in standard regression ouput (e.g. by R
summary command)c ̂ ) and SE(c ̂ ) are smaller the smaller is ̂— SE(c ̂ ) is smaller the larger is ̂ 2— SE(c ̂ ) and SE(c ̂ ) → 0 as  gets large ⇒ ̂ and ̂ are consistent— SE(estimators

2  ̂
• Standard errors for ̂
 or −square can be computed using thebootstrap• For  large enough, the central limit theorem (CLT) tells us thatc ̂ )2)
̂ ∼ ( SE(c ̂ )2)
̂ ∼ ( SE(• Approximate 95% confidence intervalsc ̂ )̂ ± 2 · SE(c ̂ )̂ ± 2 · SE(SI Model Using Matrix Algebra =  +  +   = 1     
Stack over observations  = 1     ⎛or⎞⎛⎞⎛⎞⎛⎞

11
1
1
⎜ .. ⎟⎜ . ⎟⎜⎜ . ⎟.. ⎟⎝⎠ =  ⎝ . ⎠ +  ⎝⎠+⎝ . ⎠


1

R =  · 1 +  · R +  == X  + 

X=

³1 R

´  =Ã³!

1 R´Ã


!

+ 

Recall the least squares normal equationsX
SSR(̂ ̂)0== −2( − ̂ − ̂)
 ̂
=10=

SSR(̂ ̂) ̂= −2
X( − ̂ − ̂)

=1

Using matrix algebra these equations areÃ!
Ã!Ã
!
PP̂
P=1P =1 = P 2̂
=1 
=1 
=1 

Equivalently,
Ã

10R
R0 R

!

=

Ã

101
10R

10R
R0 R

orX0R = X0X̂ Solving for ̂ gives the least squares estimates³´−10̂ = X X
X0R

!Ã

̂
̂

!

Estimating SI Model Covariance Matrix
Recall, in the SI model
2  0 + DΣ = ⎛⎞⎛2
11⎜⎜ .. ⎟=⎝⎠ D = ⎜⎝ 0
0Estimate Σ using plug-in principle⎞

00... 0 ⎟⎟⎠20 02 ̂ ̂ + D̂Σ̂ = ̂where⎛⎞⎛2̂1̂1⎜⎜ .. ⎟̂ = ⎝⎠  D̂ = ⎜⎝ 00̂

⎞

00... 0 ⎟⎟⎠20 ̂Single Index Model and Portfolio TheoryIdea: Use estimated SI model covariance matrix instead of sample covariancematrix in forming minimum variance portfolios:min x0Σ̂x s.t. x0̂ = 0 and x01 = 1


02 ̂ ̂ + D̂Σ̂ = ̂
̂=sample meansHypothesis Testing in SI ModelSingle Index Model and Assumptions
 =  +  + cov( ) = 0 cov( ) = 0 cov( − ) = 02 ) ∼  (  2 ) ∼  (0 2   2 are constant over time    

Hypothesis Tests of Interest• Tests on Coeﬃcients ( and )• Tests on Model Assumptions and Residuals— Normality of returns and residuals— No autocorrelation in returns and residualsHypotheses of Interest: Coeﬃcients• Basic significance test
0 :  = 0 vs. 1 :  6= 0
• Test for specific value0 :  = 0 vs. 1 :  = 0• Test of constant parameters0 :  is constant over entire sample
1 :  changes in some sub-sample

Basic significance test
0 :  = 0 vs. 1 :  6= 0
Test statistics: t-statistics
=0 =Intuition:̂ − 0d ̂ )(=̂d ̂ )(• If |=0| ≈ 0 then ̂ ≈ 0 and 0 :  = 0 should not be rejectedd ̂ ) away from 0• If |=0|  2, say, then ̂ more than 2 values of (This is very unlikely if  = 0 so 0 :  = 0 should be rejected.

Distribution of test statistics under 0Under the assumptions of the SI model, and 0 :  = 0
=0 =where̂d ̂ )(∼  −2

 −2 = Student t distribution with
 − 2 degrees of freedom (d.f.)Remarks:•  −2 is bell-shaped and symmetric about zero (like normal)• d.f. = sample size - number of estimated parameters. In SI model there
are two estimated parameters ( and )• Degrees of freedom determines kurtosis (tail thickness)
d.f. =  − 2  10, ( −2)  3
d.f. =  − 2  60 ( −2) ≈ 3

• For  ≥ 60  −2 ∼ (0 1). Therefore, for  ≥ 60
=0 =̂d ̂ )(∼ (0 1)

Test for specific value0 :  = 0 vs. 1 :  6= 0Test statistics: t-statistics

Intuition:̂ − 0
=0 = d ̂ )(• If |=0 | ≈ 0 then ̂ ≈ 0 and 0 :  = 0 should not be rejectedd ̂ ) away from• If |=0 |  2, say, then ̂ more than 2 values of (0 This is very unlikely if  = 0 so 0 :  = 0 should be rejected.

Residual Diagnostics• Time plots of actual values, fitted values and residuals• Histogram of residuals ̂ =  −  − 
• SACF of residualsDiagnostic for constant parameters: rolling Regression
Idea: Compute estimates of  and  from SI model over rolling windows oflength   () = () + ()() + ()If ̂() ̂() are roughly constant over the rolling windows then the hypothesis that  and  are constant is supported by the data.