This paper serves to demonstrate through an application to a real world policy experiment how Bayesian impact evaluation can be carried out in practice with informative priors. Our hope is that it serves a similar role to Casey et al. (2012), who demonstrated the practical use of pre-analysis plans through a field experiment. Just as with pre-analysis plans, we believe the value of this Bayesian approach is likely to be highest for long-term field experiments that are expensive and time-consuming to replicate. In contrast, if replications are cheap and quick, as is the case in many lab experiments, then just as Coffman and Niederle (2015) argue against pre-analysis plans, one may be able to cheaply increase sample size to the point where posterior credible intervals coincide with traditional confidence intervals, even with informative priors, reducing the value of our approach. The same logic may apply to large sample A/B testing carried out on tech platforms. But when important policy decisions rest on the results of field experiments with limited sample sizes, investing in collecting informative priors and bringing this knowledge into evaluations through our approach seems beneficial.

This paper contributes to two main areas of the literature. The first is a growing literature on collecting and using priors and predictions about interventions. To date this has largely involved getting academics, policymakers, or program participants to make predictions about the likely effects of a treatment, and then measuring the accuracy of these predictions (e.g. Groh et al. (2016); Hirschleifer et al. (2016); Dellavigna and Pope (2018); McKenzie (2018); Dellavigna et al. (2019)). But in economics these predictions have typically not been elicited as full nonparametric distributions, and have not been formally incorporated into the analysis. Andrews and Shapiro (2021) study the question of how to best communicate results if audiences come with different priors, and suggest that there may be cases where analysts should censor the estimates they report as a result. Abadie (2020) notes that the information content in non-significant results will depend on the priors decision-makers had about the null hypothesis being rejected. Our approach explicitly elicits these priors, and so provides a way to communicate how much these priors should be updated given the data. Finally, while Bayesian analysis has not yet been used for analyzing individual impact evaluations, Bayesian meta-analysis approaches have been used to aggregate evidence from different studies (e.g. Meager (2019); Vivalt (2020)).

Second, the paper contributes to a literature on improving management and export performance in firms. Several recent experiments have found management consulting can improve management practices and firm performance (Bloom et al. (2013); Bruhn et al. (2018); Iacovone et al. (2022)). However, these existing studies have not focused on export-oriented firms with exporting as a main outcome. Bloom et al. (2021) use data on American and Chinese firms, and find that better managed firms are more likely to export, sell more products to more destinations, and have higher export revenues. Despite this relationship, our results suggest that purely improving management practices may 4