Table 6: Observed and Predicted GPA Using Different Production Function Specifications

||(1)<br>Quartile 4|(2)<br>Quartile 1|
|---|---|---|
|Panel A: Mean GPA|||
|Observed|0.761|-0.486|
|Predicted, without dormitory fxed efects|0.889|-0.345|
|Predicted, least squares with dormitory dummies|0.698|-0.433|
|Predicted, within-group transformation|0.689|-0.503|
|Panel B: Mean treatment efect of tracking|||
|Estimated from diference-in-diferences design|0.032|-0.225|
|Predicted, without dormitory fxed efects|0.223|-0.050|
|Predicted, least squares with dormitory dummies|0.041|-0.139|
|Predicted, within-group transformation|0.032|-0.195|

_Notes:_ Table 6 panel A reports observed GPA (row 1) and predicted GPA from three different models. All predictions use observed regressor values from tracked dormitory students and estimated coefficients from randomly assigned dormitory students. The first prediction uses coefficients generated by estimating equation 4 without dormitory fixed effects (shown in column 5 of table 4). The second prediction uses coefficients generated by estimating equation 4 with dormitory indicator variables (shown in column 6 of table 4). The third prediction uses coefficients generated by estimating equation 4 with data from a withindormitory transformation (shown in column 6 of table 4). The second and third predictions differ because the values of the dormitory fixed effects respectively are and are not used in the prediction.

4. Excluding dormitory fixed effects (row 2) yields very inaccurate predictions, with GPA and treatment effects too high for students in the top and bottom quartiles. This reflects the estimated convexity of the GPA production function without dormitory fixed effects (ˆ _γ_ 22 = 0 _._ 31 but insignificant). With dormitory fixed effects included, the production function is not convex (ˆ _γ_ 22 = _−_ 0 _._ 16 but insignificant) and own and peer test scores are substitutes (ˆ _γ_ 12 = 0 _._ 13). The fixed effects estimates thus predict negative and zero treatment effects on the first and fourth quartiles respectively, matching the difference-in-differences estimates. However, the first quartile estimates are quite sensitive to specifying the fixed effects with dormitory dummies (row 3) or using a within-group data transformation (row 4).

This exercise illustrates that a simple reduced form GPA production function can come close to predicting the treatment effects of tracking. However, the predictions are very sensitive to specification choices regarding covariates and group fixed effects, which in turn influence the support of the data. These are precisely the choices for which economic theory is likely to provide little guidance. Statistical model selection criteria are also inconclusive in this setting.<sup>37</sup> This sensitivity may be due to out-of-sample extrapolation, dependence of GPA on group-level statistics other than the mean, or behavioral responses by students that

> 37For example, the Akaike and Bayesian information criteria are lower for the models respectively with and without dormitory fixed effects, while a likelihood ratio test for equality of the models has _p_ -value 0.083. Hurder (2012) finds that model selection criteria are also inconclusive in a similar application.

27