- Country-level data on agricultural production and prices, available over a number of years, provided a broader picture and also to correct for seasonal variations in temperature and rainfall (both between different regions and over time). This was important in order to avoid the error of measuring trends from only two points in time.

All of the data sources were combined to develop relatively robust estimates of a set of social and economic changes in the project areas over the life of the project, and to compare these changes with a counterfactual (what would have been the condition of the project areas absent the project) constructed through combining data from a number of secondary sources. The credibility of the estimates of changes that could be (at least partially) attributed to the project intervention was then tested through focus groups with project participants, discussions with key informants, and direct observation of the changes that occurred during project implementation.

This evaluation, conducted with a relatively modest budget, and drawing on the kinds of secondary data and recall information that are often available, illustrates how mixed method designs can offer a promising approach to developing an alternative to the conventional statistical counterfactual, thus strengthening our understanding of the potential impacts of the majority of projects where the conventional counterfactual cannot be applied.

# **VII. Conclusion: Continuing Challenges and Opportunities**

# _Challenges_

While dramatic progress has been made in the application of mixed methods in the US and some other industrial nations, and despite a slow but steady increase in published studies in developing countries, a number of challenges continue to face evaluators wishing to apply mixed methods in the monitoring and evaluation of development projects and policies in these contexts.

A first challenge is the fact that mixed methods have been the evaluation design of choice for many development agencies for many years. However, many of these evaluations used somewhat ad hoc approaches and most do not apply the kinds of methodological and conceptual rigor that is required by academic journals such as the Journal of Mixed Method Research. So the mixed method approach is not new per se, but the professional, financial and other resources have usually not been available to increase methodological rigor.

While it is claimed (although not everyone would agree) that the “paradigm wars” are long ended in the US and Europe and that there is a general acceptance of mixed method approaches, in many, but certainly not all, developing countries the divisions between quantitative and qualitative researchers are still quite pronounced. In some countries qualitative researchers tend to be (or are perceived as being) more politically radical than their quantitative colleagues, so this can provide a further complication. Indeed, for precisely this reason. autocratic regimes are often amenable to training engineers and statisticians to help ‘manage’ the state but are much less sympathetic towards disciplines such as anthropology, sociology and journalism that might unearth material questioning the claims and legitimacy of the regime. Even in more open societies, this often means that, in practical terms, considerable time and effort is usually required to identify and build a team of local researchers who can work well together on a mixed 23