L ' Actualité économique Wild Cluster Bootstrap Confidence Intervals

Confidence intervals based on cluster-robust covariance matrices can be constructed in many ways. In addition to conventional intervals obtained by inverting Wald (t) tests, the paper studies intervals obtained by inverting LM tests, studentized bootstrap intervals based on the wild cluster bootstrap, and restricted bootstrap intervals obtained by inverting bootstrap Wald and LM tests. It also studies the choice of an auxiliary distribution for the wild bootstrap, a modified covariance matrix based on transforming the residuals that was proposed some years ago, and new wild bootstrap procedures based on the same idea. Some procedures perform extraordinarily well even with the number of clusters is small. IntroductIon It is now routine to employ cluster-robust standard errors whenever observations at the individual level are associated with a number of geographical areas and/or with a number of time periods. Each geographical area, or each time period, or perhaps each area-period pair, can be thought of as a cluster. When key regressors are measured at the cluster level, as is often the case when assessing the effects of policy changes, fixed effects cannot be used to account for intra-cluster correlation, because the fixed-effect dummy variables would explain all the variation in the regressor(s) of interest. Instead, it is common to use cluster-robust standard errors, because they allow for heteroskedasticity within and across clusters and also for intra-cluster correlation. In large datasets, even very small levels of intra-cluster correlation can cause severe errors of inference if standard errors are not robust to clustering. For example, L’Actualité économique, Revue d’analyse économique, vol. 91, n 1-2, mars-juin 2015 ___________ * Research for this paper was supported, in part, by a grant from the Social Sciences and Humanities Research Council of Canada. I am grateful to Russell Davidson for valuable insights and to Matthew Webb and a referee for useful comments. 12 L’ACTUALITÉ ÉCONOMIQUE in a “placebo laws’’ experiment with over 500,000 observations on employment income data, where the average intra-cluster correlation coefficient is roughly 0.032, Bertrand, Duflo and Mullainathan (2004) find that using standard errors which are robust to heteroskedasticity but not to clustering yields rejection frequencies for interventions that did not actually take place which exceed 0.67 at the . 05 level. There has been a good deal of recent work on cluster-robust inference; see Cameron and Miller (2015) for a comprehensive survey. Much of this work has focused on testing, including bootstrap testing; see Cameron, Gelbach, and Miller (2008), and MacKinnon and Webb (2014). This paper focuses instead on confidence intervals. The next section discusses the conventional cluster-robust confidence interval, which is implicitly based on inverting a Wald test, and proposes a new interval based on inverting a Lagrange Multiplier, or LM, test. The latter is more computationally intensive than the former, but it should be quite feasible in most cases. Section 2 then reviews the procedure for constructing a studentized bootstrap interval based on the wild cluster bootstrap. Section 3 proposes two new “restricted bootstrap’’ intervals, which are based on inverting bootstrap P values for Wald and LM tests, respectively. Unfortunately, these procedures are very computationally intensive. Section 4 describes the design of simulation experiments to compare the performance of the five intervals considered in the previous two sections, and Section 5 presents the experimental results. The remainder of the paper deals with two different issues. Section 6 discusses the choice of an auxiliary distribution for the wild cluster bootstrap and presents some further experimental results. Section 7 discusses several ways in which cluster-robust confidence intervals can be improved by using transformed residuals in covariance matrices and/or wild bootstrap DGPs and presents additional simulation results. Finally, Section 8 concludes. 1. cluster-robust confIdence Intervals Consider the linear regression model y ≡ y1 y2 yG = Xβ + u ≡ X1 X2 XG β + u1 u2 uG , (1) where the data are divided into G clusters, indexed by g. The g cluster has N g observations, and the entire sample has N = g=1 G ∑ Ng observations. The matrix X and the vectors y and u have N rows, X has K columns, and the parameter vector β has K elements. Least squares estimation of equation (1) yields OLS estimates 13 WILD CLUSTER BOOTSTRAP CONFIDENCE INTERVALS and residuals û. The disturbances are assumed to be uncorrelated across clusters but potentially correlated and heteroskedastic within clusters, so that E(u g u g ') = Ωg E(ug g' , g = 1,.,G, where the Ng x Ng covariance matrices Ωg are unknown. Thus the covariance matrix of u is assumed to be block diagonal. Following, Liang and Zeger (1986), the covariance matrix of can be estimated by using a cluster-robust [co]variance matrix, or CRVE. The most widely-used CRVE is G(N −1) (G −1)(N −K) (X'X) g=1 G ∑Xg'ûgûg'Xg ⎛