Motivated by customer loyalty plans  , we study tie-breaker designs which are hybrids of randomized controlled trials (RCTs) and regression discontinuity designs (RDDs). We quantify the statistical efficiency of a tie-breaker design in which a proportion \Delta of observed customers are in the RCT. In a two line regression, statistical efficiency increases monotonically with \Delta, so efficiency is maximized by an RCT. That same regression model quantifies the short term value of the treatment allocation and this comparison favors smaller \Delta with the RDD being best. We solve for the optimal tradeoff between these exploration and exploitation goals. The usual tie-breaker design experiments on the middle \Delta subjects as ranked by the running variable. We quantify the efficiency of other designs such as experimenting only in the second decile from the top. We also consider more general models such as quadratic regressions .
Motivated by customer loyalty plans and scholarship programs , we study tie-breaker designs which are hybrids of randomized controlled trials (RCTs) and regression discontinuity designs (RDDs). We quantify the statistical efficiency of a tie-breaker design in which a proportion \Delta of observed subjects are in the RCT. In a two line regression, statistical efficiency increases monotonically with \Delta, so efficiency is maximized by an RCT. We point to additional advantages of tie-breakers versus RDD: for a nonparametric regression the boundary bias is much less severe and for quadratic regression, the variance is greatly reduced. For a two line model we can quantify the short term value of the treatment allocation and this comparison favors smaller \Delta with the RDD being best. We solve for the optimal tradeoff between these exploration and exploitation goals. The usual tie-breaker design applies an RCT on the middle \Delta subjects as ranked by the assignment variable. We quantify the efficiency of other designs such as experimenting only in the second decile from the top. We also show that in some general parametric models a Monte Carlo evaluation can be replaced by matrix algebra .
A
and scholarship programs
R
customers
subjects
R
That same regression model quantifies
We point to additional advantages of tie-breakers versus RDD: for a nonparametric regression the boundary bias is much less severe and for quadratic regression, the variance is greatly reduced. For a two line model we can quantify
R
experiments
applies an RCT
R
running
assignment
R
consider more general models such as quadratic regressions
show that in some general parametric models a Monte Carlo evaluation can be replaced by matrix algebra