# **Figure 4  An iterative process for analyzing how contextual factors affect project outcomes**

|QUAL<br>assessments<br>of<br>contextual<br>factors|Ratings<br>“quantitized”|Regression analysis<br>identifies variance in<br>implementation or outputs<br>among project sites|In-depth QUAL<br>analysis to explain<br>why and how these<br>contextual factors<br>operate|
|---|---|---|---|

While contextual analysis examines the influence of external factors, process analysis looks at the internal organizational processes through which the project is implemented. However, this distinction is not rigid as external factors can affect implementation processes (for example, political pressures to provide benefits to non-eligible groups or problems within partner agencies that can affect the provision of certain services). Process analysis mainly focuses on steps P-2 (inputs), P-3 (processes) and P-4 (outputs). Some of the main sources of information for process analysis are the project’s internal monitoring and administrative systems. While in theory a wellfunctioning monitoring system could provide most of the information required for process analysis, in practice monitoring systems often are seen as having only a narrow accountability function and often only generate a limited set of quantitative indicators. Often (though not always), the quality of the information is poor and relies on self-reporting, where the staff completing the information have an incentive to avoid problems by checking the boxes saying that progress is at least satisfactory. However, even when the quality of data is quite satisfactory, most systems rely on quantitative indicators with little attention given to the quality of the service delivery, how effectively beneficiaries are involved in planning and management, which sectors of the target population do and do not benefit, or the opinions of beneficiaries. A common example of the lack of information on beneficiary participation is the fact that many monitoring systems provide little or no information on differences in male and female participation rates or access to services, or similar information for different ethnic groups. As monitoring systems are managed by the implementing agencies, issues such as corruption, major absenteeism (by teachers, medical staff or office staff) are rarely addressed.

The mixed method approach to process analysis will build on, and try to strengthen, monitoring and other administrative reporting systems while complementing these with a variety of quantitative and qualitative techniques. These may include: participant observation (visiting schools, clinics, small business programs etc to observe how they operate); key informant interviews (including a wide range of different types of informant); focus groups; PRA and other participatory group interview techniques; selecting a panel of informants who are visited regularly throughout the life of the project; citizen report card surveys (in which a large and randomly selected sample of the target population are interviewed about their experiences with, and opinions about, public service agencies). It is also possible to make more creative use of administrative records and communications through the use of content analysis, review of e- mails and other internal communications, and analysis of the information and messages provided to the public through posters, notice-boards, newsletters and radio announcements. The information from all of these sources is used to compare how the project was actually implemented with the plan as stated in the operational manner and other planning documents.

15