of the time with more than 5 percent differential attrition.

Panels (c) and (d) of Figure 3 examine how much the estimated treatment coefficient changes using PDS Lasso compared to Ancova as the attrition and differential attrition rates vary. We see a significantly higher mean percentage change in the treatment coefficient when differential attrition is greater than 5 percent, but even so, the mean percent change is only 0.08 standard deviations.

We would also expect PDS Lasso to make more of a difference to standard errors over Ancova when there is no lagged dependent variable available, and PDS Lasso is then potentially able to predict the outcome with other controls. We compared the standard error ratio (PDS Lasso/Ancova) for cases where no lagged dependent variable was available, to those where it was available and also included in PDS Lasso. There is not much difference at the median (0.996 when there is a lagged variable compared to 0.99 with no lagged variable available). However, at the 25th percentile the ratio is smaller (meaning more improvement) at 0.938 with no lag, compared to 0.974 with a lagged dependent variable available.

Finally, using the replication data, we also looked in more detail at the outliers, where PDS Lasso had resulted in the biggest change in standard errors compared to Ancova. Qualitatively we see these arise from cases where there was no lagged dependent variable available, and where just a few<sup>9</sup> controls are chosen by PDS Lasso that help predict the outcome well. For example, the largest reductions in standard errors compared to PDS Lasso occurs in Garbiras-Diaz and Montenegro (2022). They conduct an experiment to look at the effects of encouraging citizen oversight in Colombia. No lagged outcome is available, and the standard error after using PDS Lasso is 50-60% of that using just strata fixed effects. However, almost all of this improvement is coming from the inclusion of a single control: the number of candidates in the election strongly predicts the vote share (a regression of the outcome on this single regressor in the control group has a _R_<sup>2</sup> of 0.17). Hence in many cases, the gains from PDS Lasso are coming from adding a couple of controls that researchers might otherwise have used contextual knowledge to add.

# **4.4 Why is Ancova sometimes more precise than PDS Lasso?**

We see that in one quarter of the empirical cases, PDS Lasso resulted in larger standard errors than Ancova. There are two potential reasons for this. The first is that by including many potential control variables for PDS Lasso to select among, it may fail to select the lagged dependent variable, or indeed any variables at all. In 23% of the replications in which a lagged dependent variable was available, but was not included by researchers in the amelioration set, the lagged variable was not selected by PDS Lasso.<sup>10</sup> Second, it could end up adding noise and reducing precision by selecting variables that are irrelevant.

> 9Of the 25 largest reductions in standard error, PDS Lasso selected a median of 3 controls to add.

> 10The lagged dependent variable was not selected in 21 out of 93 such cases.

15