<u>Survey design: A pre-intervention baseline survey was carried out in 2007</u><sup>3</sup> . The baseline involved conducting a census of all households in the selected 200 villages, and the administration of a multi-module detailed household questionnaire to randomly sampled 15 households from each village (total of 3,000 households). The survey questionnaire was adapted from the Nepal Living Standards Survey<sup>4</sup> (NLSS) and included detailed information on consumption and income, socio-economic and demographic issues, including education, health and nutrition, physical assets, migration and remittances, employment, social environment, community relationship, voice and participation. For comparability with the national household survey based welfare measures, the PAF survey included a similar consumption module and followed the same consumption aggregation method. The follow-up survey, carried out more than two years after the baseline, included the same questionnaires from the baseline survey and also gathered basic information on the actual treatment status (PAF intervention) and non-treatment (control) at both household level and at the village/PSU level. Randomized assignment to early phase-in was not followed perfectly, but sufficiently strongly to allow us to identify a suitable empirical strategy to report causal impact estimates.

<u>Compliance with Randomization: Table 1 below describes planned treatment status and</u> actual treatment status of the sampled villages at the time of the follow-up survey. We note two patterns: actual status does not follow random assignment perfectly; however, random assignment strongly predicts actual treatment status (72 of the 100 randomly treatment-assigned villages are treated at the follow-up survey compared to only 39 from the control-assigned). Imperfect compliance with randomized assignment in this evaluation was to be expected given the nature of the PAF interventions. As mentioned earlier, PAF makes use of NGOs to initiate communitybased interventions in the rural villages in Nepal. The decision to select a village over another one for early phase-in cannot all be enforced through lottery alone. Implementation readiness, geography, socio-economic conditions and other factors may contribute towards inability to comply with random selection. Allowing the NGOs to implement their work may have resulted in imperfect compliance, but it may have helped maintain a certain degree of external validity (the extent to which the process and results of the interventions are generalized and replicable elsewhere). For the purpose of evaluation, we consider a village in the treatment status if there is a CO in that village supported by the PAF. We also note that not all 15 households (which were

> 3 Piloting for the impact evaluation was initiated in 2006 in a small sample. The design and implementation of the main impact evaluation benefited from lessons learnt from the pilot in term of sampling procedure, survey questionnaire, and accessibility of sampled villages for reasons related to weather seasonality and remoteness.

> 4 For details on national household surveys, see Nepal Living Standards Surveys (NLSS 1995/96, 2003/04, 2010/11). The reports can be found at www.cbs.gov.np

5