by adding covariates that help reduce the variance in the outcome of interest. But smaller samples limit the ability to benefit from non-linear machine learning approaches, and also raises questions about how well asymptotic results used to justify the choice of regularization parameters in Lasso work with the sample sizes common in field experiment applications. W ̈uthrich and Zhu (2023) recently showed that PDS Lasso can underselect variables in finite samples when the number of variables is reasonably large, potentially leading to omitted variable bias when these variables have moderate relationships with treatment. Second, the majority of field experiments in developing countries rely on survey data for many outcomes, which can be subject to attrition. In a survey of 96 field experiments, Ghanem et al. (2023) find an average attrition rate of 15 percent. This raises the concern that if the determinants of attrition differ with treatment status, then the sample for which data are available may no longer be comparable across treatment groups. This offers a further potential rationale for double selection methods, which can select covariates that predict treatment status in the attrited data. But the question remains as to whether this makes much difference in practice.

We replicate and re-analyze field experiment papers published in three economic journals between 2017 and 2022 which used PDS Lasso, resulting in 780 treatment estimates. We use this analysis to identify the key practical issues and performance of PDS Lasso, and compare the estimates and their standard errors to those that would be obtained using simple Ancova. We find that while authors typically include a long list of variables to be inputted as potential controls in PDS Lasso (a median of 182 controls), PDS Lasso typically ends up selecting very few control variables. The median is three controls, and in over half the cases, no variables at all are selected in the treatment regression step. When variables are selected for treatment, they are almost never those that are also selected for predicting the outcome of interest. As a result, PDS Lasso leads to minimal changes to treatment estimates and standard errors on average, with a median change in the coefficient of 0.01 standard deviations, and a median standard error that is 99.2% of that with Ancova. In over a quarter of the cases, standard errors are actually slightly larger than they would be just using Ancova. Researchers should therefore not anticipate significant power gains on average from using this method.

We combine this re-analysis with simulations to look at when PDS Lasso performs better or worse, and to help answer practical questions facing applied researchers. In our re-analysis, we do find the treatment regression step to be more likely to select control variables when there is attrition, but even then, typical changes in coefficients are small, reflecting that attrition in field experiments is often due to reasons uncorrelated with the outcome. We show that PDS Lasso sometimes ends up being less precise than Ancova (and having a higher mean-squared error in some simulations), since by inputting very many control variables, there is a risk that the Lasso penalization results in key variables such as the lagged dependent variable not being selected. We recommend inserting such a variable in the amelioration set that must be included.

3