Title: Assumption-lean inference on treatment effect distributions
Primary area: causal reasoning
Keywords: causal inference, treatment effects, partial identifiability, efficient estimation

Abstract: In many fields, including healthcare, marketing, and online platform design, A/B tests are used to evaluate new treatments and make launch decisions based on average treatment effect (ATE) estimates. But this workflow can overlook distributional risks, such as a large fraction of individuals affected negatively by the treatment. In this paper, we propose a novel method for inference on the treatment effect distribution. Prior work in this setting has estimated partial identification bounds, known as Makarov bounds, on the cumulative distribution function of the treatment effect by making restrictive assumptions on the outcome distribution. In contrast, our method guarantees accurate estimation and valid asymptotic inference of the Makarov bounds for any outcome distribution. Our main technical contributions are to develop smoothed surrogates for the Makarov bounds, derive semiparametrically efficient estimators of these surrogates, and propose a procedure for optimal selection of the smoothing parameters. We show empirically on synthetic and semi-synthetic datasets that, by not relying on the assumptions made by other methods, our estimators achieve a better bias-variance trade-off and lower mean-squared error. Finally, we deploy our method on real A/B test data from a large social media platform, and show how estimates of the treatment effect distribution can inform decision-making.

TL;DR: We propose an assumption-lean method for estimating Makarov bounds on the distribution of the treatment effect.