One of the reasons why RCT is a preferred design for such experiments is that randomization does away with the need for complex econometric techniques to control for selection in nonexperimental settings. However, RCT is no silver bullet in small sample environments, as it relies on the law of large numbers to ensure that expected untreated outcomes are equal in treatment and control groups. In low-income countries, interventions sometimes target very small numbers of firms (McKenzie, 2011b).<sup>27</sup> For instance, the Pesticides Initiative Program (PIP), an E.U. technical-assistance program designed to help fruit and vegetable producers cope with E.U. standards, covers less than a few dozen firms in some African countries (Jaud and Cadot, 2011). Randomization is not an option in such environments. Quasi-experimental methods may not do very well either; but if a cross-country sample is available with enough observations, econometrics may offer some scope to control for cross-country heterogeneity.<sup>28</sup> We will return to the small sample issue in the context of non-targeted interventions in section 4.2 when referring to the Cameroon customs project described in Cantens, Raballand, Bilangna, and Djeuwo (2011).

An intrinsic limitation of RCTs in the trade and other economic areas is that the study subjects are active economic agents who consciously choose their responses as opposed to the medical sciences where passive entities (e.g., cancer cells) respond endogenously following the laws of nature (Barrett and Carter, 2010). Unobservable perceptions about the benefits of a new traderelated intervention will vary among potential beneficiaries in ways that are likely to be correlated with other attributes and with the actual effects of the treatment. In section 6, we will discuss how randomization may fail to produce unbiased treatment effects in the presence of ―essential‖ heterogeneity or in the presence of spillovers.

# **4.1.2 Quasi-experimental methods**

When evaluation is not built into program design, RCT is not an option and quasi-experimental (QEM) methods must be used, all relying on econometric techniques to overcome selection bias.<sup>29</sup> The first is the difference-in-differences (DID) method briefly described above. By comparing differences in outcomes instead of comparing levels, DID controls for unequal performance levels of treatment and control groups not related to the program. However, DID

> 27 McKenzie (2011b) discusses the issue of small samples in World Bank private sector support programs in Africa. None of those programs has been subject to rigorous impact evaluations so far, but if such evaluations were to be conducted researchers would be faced with a serious problem of power given the small number of enterprises assisted by the projects and their large degree of heterogeneity.

> 28 Randomization across countries would be more difficult to implement than within a country and would not necessarily increase the test’s power.

> 29 How well quasi-experimental methods perform compared to randomization has been a subject of intense scrutiny since the seminal paper of Lalonde (1986), with largely inconclusive results. Glazerman, Levy and Myers (2003) found that quasiexperimental methods produced substantially biased results compared to experimental ones in twelve replication studies of welfare and employment programs in the U.S. Cook, Shadish, and Wong (2006) found less clear-cut results for education programs. See Ravallion (2008) on the evaluation of poverty programs in non-experimental settings.

20