  Double blind randomized controlled trials are traditionally seen as the gold
standard for causal inferences as the difference-in-means estimator is an
unbiased estimator of the average treatment effect in the experiment. The fact
that this estimator is unbiased over all possible randomizations does not,
however, mean that any given estimate is close to the true treatment effect.
Similarly, while pre-determined covariates will be balanced between treatment
and control groups on average, large imbalances may be observed in a given
experiment and the researcher may therefore want to condition on such
covariates using linear regression. This paper studies the theoretical
properties of both the difference-in-means and OLS estimators conditional on
observed differences in covariates. By deriving the statistical properties of
the conditional estimators, we can establish guidance for how to deal with
covariate imbalances. We study both inference with OLS, as well as with a new
version of Fisher's exact test, where the randomization distribution comes from
a small subset of all possible assignment vectors.
