# _How can mixed methods address these real-world challenges?_

Mixed method approaches combine quantitative approaches that permit estimates of magnitude and distribution of effects, generalization and tests of statistical differences with qualitative approaches that permit in-depth description, analysis of processes and patterns of social interaction. These integrated approaches provide the flexibility to fill in gaps in the available information, to use triangulation to strengthen the validity of estimates, and to provide different perspectives on complex, multi-dimensional phenomena. When working under real-world constraints, a well designed mixed-methods approach can use the available time and resources to maximize the range and validity of information. The following are some of the specific ways that mixed methods can strengthen the impact evaluation design when working with real-world constraints.

<u>Reconstructing baseline data. There are a number of qualitative techniques that can be used to</u> reconstruct baseline data. These include: recall, key informants, participatory group interview techniques (such as PRA), and the analysis of administrative records (such as school attendance, patient records from health clinics, sales records from agricultural markets) (see Bamberger 2009a). Ideally a triangulation strategy should be used to increase validity by comparing estimates from different sources. Most of the above sources use purposive, often opportunistic, sampling methods relying on information that is easily accessible, and consequently there are questions of bias or non-representativity. A mixed method strategy can include statistical methods either to use random sampling techniques (for example to select the schools or clinics) or to assess the potential direction and magnitude of bias.

A well-designed monitoring system can often provide at least some of the baseline data required for impact evaluation (see following section). However, experience has shown that the generation of data in the format required for impact analysis will usually require the active participation of the evaluator in the design of the monitoring system, both to ensure that the required information is collected and also to ensure that it is organized and filed in a way that will be suitable for the evaluation. Often potentially valuable monitoring data has proved impossible to use for evaluation because certain critical classification data (such as code numbers identifying beneficiaries, types of services received, reasons for dropping out of the project etc.) have not been included or recorded systematically. In other cases the information is incomplete or some has been mislaid because staff did not realize that this information was intended for use in the evaluation. Sometimes the quality of the information could have been greatly enhanced if the evaluation budget could have funded additional administrative staff (usually at a very small cost) to collect and organize the additional information. Very often these simple organizational tasks are overlooked, or the evaluator is not involved until a later stage of the project – by which time it is often too late to put the system in place.

<u>Observing unobservables. An important application of the baseline reconstruction techniques is</u> to identify important missing variables from the secondary data sets and to provide estimates of these variables. In the case of the previous micro-credit project, techniques such as focus groups or key informant interviews could be used to identify differences between women who did and did not apply for loans that might affect project outcomes (such as successful launch of a small business). If possible, a small sample of beneficiaries and non-beneficiaries would then be 11