  Two-sided marketplace platforms often run experiments to test the effect of
an intervention before launching it platform-wide. A typical approach is to
randomize individuals into the treatment group, which receives the
intervention, and the control group, which does not. The platform then compares
the performance in the two groups to estimate the effect if the intervention
were launched to everyone. We focus on two common experiment types, where the
platform randomizes individuals either on the supply side or on the demand
side. The resulting estimates of the treatment effect in these experiments are
typically biased: because individuals in the market compete with each other,
individuals in the treatment group affect those in the control group and vice
versa, creating interference. We develop a simple tractable market model to
study bias and variance in these experiments with interference. We focus on two
choices available to the platform: (1) Which side of the platform should it
randomize on (supply or demand)? (2) What proportion of individuals should be
allocated to treatment? We find that both choices affect the bias and variance
of the resulting estimators but in different ways. The bias-optimal choice of
experiment type depends on the relative amounts of supply and demand in the
market, and we discuss how a platform can use market data to select the
experiment type. Importantly, we find in many circumstances, choosing the
bias-optimal experiment type has little effect on variance. On the other hand,
the choice of treatment proportion can induce a bias-variance tradeoff, where
the bias-minimizing proportion increases variance. We discuss how a platform
can navigate this tradeoff and best choose the treatment proportion, using a
combination of modeling as well as contextual knowledge about the market, the
risk of the intervention, and reasonable effect sizes of the intervention.
