correct number of controls selected by PDS Lasso.<sup>15</sup> We explore the difference in the total number of controls selected using the _partial()_ option in comparison with the _aset()_ option as shown in table A3 in the appendix. We notice that the overall number of variables selected is almost the same using either options. The only main distinction is in the maximum number of controls selected, which is 68 variables using _partial()_ and 78 variables using _aset()_ . We also check the difference in the number of variables selected between both options and we find that while in rare cases there can be a large difference in the number of control variables selected between the two approaches, the mean of the difference is only 1 variable, and the median is no difference.

It is not clear _ex ante_ which method is most appropriate. On the one hand, since the plug-in penalty parameter can underselect in small samples, it seems appropriate not to further penalize the variables, especially in smaller samples. On the other hand, the increase in standard error due to degrees of freedom adjustment is particularly costly in small samples with many control variables. It could also depend on whether the strata dummies explain any variation in _yi_ , or how correlated it is with other potential control variables.

To answer this question we ran simulations for different numbers of strata (10 or 100), different number of observations (200 or 600), and varied whether the strata are correlated with the dependent variable or not as shown in table A4 in the appendix. Overall, we find almost no difference in standard errors. The decision whether to penalize the other variables or not therefore does not seem to matter much, at least for the DGP used in our simulations.

# **5.4 Is the double step even necessary?**

We have seen that in over half the empirical cases, no variables at all are selected in the second step (predicting treatment). When variables are selected in this second step, Table 2 shows that they are almost entirely variables that are not selected in step one, so are not strongly correlated with the outcome. This raises the question of whether it is worth having this second step at all in a randomized experiment.

We examine this through simulations, reported in Figure 7. We hold the sample size constant, and iid introduce selective imbalance, by setting _x_ 1 _,i_ = _α_ 1 _Ti_ + _ei_ before constructing equation 10, where _ei ∼ N_ (0 _,_ 1) and _α ∈_ (0 _._ 1 _,_ 0 _._ 2 _,_ 0 _._ 3). Such imbalance would occur if the attrition rates were different between the treatment and control groups and the probability of attrition is correlated with _xi,_ 1.<sup>16</sup> These are the conditions under which the double selection would potentially be valuable.

The main take-away from Figure 7 is that including Step 2 reduces the Mean Square Error (MSE) when

> 15The _dsregress_ program also allows one to specify a set of variables that must be included in the final regression, but there is no function to partial these variables in advance of performing variable selection.

> 16For example _α_ ≊ 0 _._ 2 if 11 percent of observations in the treatment group, but those with with highest values of _xi,_ 1, attrite.

21