substantially less improvement than the before-versus-after estimates for 6 out of 7 cases in Kenya, 8 out of 9 cases in Ghana, and 4 out of 9 cases in Nigeria.<sup>14</sup> We highlight this contrast to show how important it is to provide credible comparison groups. We do not claim that either set of estimates captures the true effects of the intervention, and the differences-in-differences estimates cannot be seen as rigorous measurements of the effects of the MV intervention. There are many reasons for this. First, it could be that the selection process of the MV sites made them more likely to experience different trends than the surrounding regions (we return to this issue in the next section). Second, the trends in the surrounding regions are not based on measurements in exactly the same years as the measurements at the MV sites, and we compare the two trends assuming they are both linear. Departures from linearity would change these estimates—though not greatly, because the measurements are taken in years that are not far apart, and because many of these indicators are unlikely to exhibit large departures from linearity such as volatile up-and-down jumps from year to year. (In many cases, departures from linearity such as exponential growth would further reduce the magnitude of the increase at the intervention sites relative to the surrounding region.) Third, the MV regions as shown do include the Millennium Villages themselves, and it is possible that there is some spillover effect from a Millennium Village to nearby areas.<sup>15</sup> This could potentially “contaminate” rural areas of the site’s region/province as a control group. But it is highly unlikely that the MV intervention substantially affected the measurements of change for rural areas of the entire region in which they are located, because the population of the MVs is in all cases a tiny fraction of the rural population of each region.<sup>16</sup> Furthermore, the MV region trends in most cases are similar to rural trends that span each nation. Despite these weaknesses, the differences-in-differences estimates presented here are better measures of the true effects than simple differences.

> 14  We define “substantially less improvement” as an absolute difference of more than 5 percent points between the estimates by the two methods.  Because MVP (2010c) does not include standard errors for the indicator estimates, we are unable to estimates confidence intervals for either the before-versus-after or the differences-in-differences estimates. As a result, we cannot determine which estimates are statistically significant. Also note that “improvement” in the indicator means a positive change, except for stunting, for which a decline is improvement.

> 15 For example, Sanchez et al. 2007 note that a health clinic in Sauri receives some patients who are non-residents.

> 16 Sauri comprises 1.32% of the rural population of Nyanza Province, Kenya. Bonsaaso comprises 1.59% of the rural population of Ashanti Region, Ghana. Pampaida comprises 0.02% of the rural population of the Northwest Region, Nigeria. This calculation assumes: 1) 65,000 residents of Sauri, 35,000 in Bonsaaso, and 6,000 in Pampaida, as reported by the MVP (2010c), 2) each national government’s census-based regional population estimates: 5,442,711 for Nyanza Province, Kenya in 2009; 4,589,377 for Ashanti Region, Ghana in 2008; and  35,786,944 for Northwest Region, Nigeria in 2006, 3) the percentage of people defined as living in “rural” areas weighted by sampling weight in the most recent DHS survey data: 90.2% for Nyanza Province, Kenya in 2008; 48.0% for Ashanti Region, Ghana in 2008; and 79.5% for Northwest Region, Nigeria in 2008.

17