The results of the analysis can be used for three complementary purposes. The first is to provide feedback to project management and policymakers to strengthen project implementation and to identify any groups or sectors of the target group that are not fully benefiting (and perhaps to identify non-eligible groups that are benefiting). The second purpose is to contribute to the evaluation by helping interpret the extent to which failure to achieve intended outcomes can be attributed to weaknesses in the project design or to problems during implementation. When combined with contextual analysis, this can also help explain differences in outcomes in different project settings.

The third application of the combined process and contextual analysis is to provide guidance on potential project replicability. Conventional statistical evaluation designs match project and comparison groups, which in turn provide a robust analysis of differences between _typical_ project and _typical_ comparison group members, but by definition makes it impossible to determine how effectively the positive project outcomes could be generalized to different situations. In other words, the policy recommendation from a conventional statistical impact design would be to say: “ _If the project were replicated to a very similar population group these are the results that could be expected_ ”. However, the analysis does not permit recommendations on how the project would be likely to perform with different population groups or in different contexts. The combination of contextual and process analysis has the capacity to provide at least some general guidelines on how the project is likely to perform in different settings and with different population groups.

# **V. Using Mixed Methods to Strengthen Qualitative Evaluations**

A frequent criticism of qualitative methods is that they use small and often non-representative samples. Mixed methods approaches offer some strategies for responding to these concerns. For example, even if it is desirable (for whatever reason) to use small samples, researchers can sometimes make careful (and highly strategic) choices about those samples if quantitative information on the larger ‘universe’ of cases, and thus the nature of the distribution of key variables, is available. For example, selecting a small number of cases that are significantly different, as measured by variables collated in (say) a census or a large household survey, can give qualitative researchers a stronger basis on which to say something about the larger population. Mixed method sampling has great flexibility to select small samples with a range of different characteristics that can be used to strengthen the understanding and interpretation of quantitative data. Some of the options include selecting extreme cases, selecting cases that are typical of each major group covered by the sample, identifying cases that challenge the research hypotheses or that do not conform to the main patterns, or using reputational sampling to select people that are outstanding or particularly interesting. A well designed mixed method data collection and analysis strategy permits a series of iterations whereby statistical findings can be fed back to the qualitative researchers who then generate additional hypotheses or indicators that can be explored statistically and that then generate additional questions to be explored in the field (see Jha et al 2007).

Dynamic mixed methods designs use and analyze qualitative data quite differently than conventional approaches that only use qualitative methods for initial diagnostic studies to help develop the data collection instruments for the quantitative studies. One practical but little used 16