household latrine and spend approximately 18 minutes daily in fetching water. Average village population is close to 1,200 individuals and 257 households. On an average, there is a larger share of Scheduled Tribe (ST) communities residing in the sampled village than the overall state average. Although there is some diversity in economic activity, the average number of enterprises (off-farm) in the village is 14. More detailed descriptive statistics are available in the Annex in Table A 4.

# **3.3 Empirical Strategy**

Khandker, Barnes, and Samad (2009) mention that two methods are primarily used to address the counterfactual issue in observational studies. These include Propensity Score Matching (PSM) and Instrumental Variable (IV) Method. PSM has become increasingly popular in medical trials and in the evaluation of policy interventions (Becker and Ichino, 2002). In observational studies assignment of subjects to treatment and control areas is not random. Therefore, the estimation of the program effect is expected to be inconsistent due to existence of confounding factors. PSM corrects the estimation of treatment effects to the extent one can identify confounding factors. The bias is reduced when outcome variables are compared for treated and comparison group units that are as similar as possible, in the absence of treatment (Becker and Ichino 2002). However, the PSM approach has serious limitations in fixing the biased estimates of the program effect because unobserved factors that may influence program participation may also be related with the program outcomes. So, we have used the instrumental variable method to check the robustness of the PSM results.

## **3.3.1 Propensity Score Matching**

We estimate the emerging impacts (Intention-to-treat, ITT) of NRLM on household welfare outcomes such as income, assets, expenditure etc. (Godtland et al. 2004). Owing to nonrandom allocation of the program to villages and household self-selection, simple comparison of outcome variables will give us inconsistent estimates of program impact. Based on program design and rollout strategy, there are multiple sources of self-selection and program placement bias. For instance, NRLM is a demand driven program and hence households exposed to the treatment can be systematically different from those who did not choose to participate in the program. In that case, it is quite likely that the differences in outcomes are due to pre-program differences. In addition to selection bias, there could also be program placement bias.

We use propensity score based matching methods for estimating NRLM impacts. The propensity score matching (PSM) method is used for estimating the average program effects, while Inverse Probability Treatment Weights (IPTW) for heterogeneous outcomes<sup>9</sup> and Quantile regressions for distributional welfare effects have been used. Instrumental variable specification is used as a check for average program impacts.

We use PSM to construct a comparison group as an estimate for the counterfactual outcome (Heinrich, Maffioli, and Vazquez 2010). Following Rosenbaum and Rubin (1983) we match households using propensity scores estimated by the selection model. The treatment status is regressed on a set of pre-determined household and exogenous village characteristics. The selection model includes variables that are drawn from the literature as well as the strategic and implementation guidelines of NRLM such as proportion of SC and ST

> 9 As heterogenous results were weak, they have not been reported in the paper

7