# **C. Measurement Issues and Possible Threats to Identification**

The absence of baseline data across the treatment and control communities is not considered a serious limitation on causal identification. Power calculations suggest that a randomized cluster design that is clustered by communities (CfW subprojects), with 276 communities, is sufficient to ensure a balance between the treatment and control households, thereby guaranteeing the comparability of the two groups for the purpose of causal attribution. Nonetheless, the variables collected during the endline surveys that were not expected to change over the short duration of the program provide evidence— albeit limited—of pre‐intervention balance between the treatment and control groups. As shown in Table 1, the overall treatment group is statistically comparable with the control group in most of the variables tested (see Annex 2 for all tables).

Another measurement issue relates to the practice of rotation (described in section II) that was applied in an informal and largely undocumented manner among the treatment communities. This has implications for the measurement of program impacts because program impacts were measured through registered beneficiaries who had been identified on the first day of subproject implementation. If this informal rotation occurs, the impacts captured by the impact evaluation (IE) may be under‐ or overestimated. For instance, registered beneficiaries may be more well‐informed or have stronger networks than nonregistered beneficiaries. This may lead them to exhibit better outcomes even in the absence of the program, and the IE would then overestimate the program’s true effects. Conversely, nonregistered beneficiaries may be poorer and may enjoy higher returns to receiving the transfer (for instance, through lower substitution effects). In this case, the IE may underestimate the true impact of the program. Another cause of potential underestimation is the fact that, owing to informal rotation, some beneficiary households worked fewer days than intended in program design, leading to an underestimation of the welfare impacts of the program. Because of informal rotation, actual program impacts may have been diluted given that the same total transfer amount was distributed across a larger number of beneficiaries.

The data collected during the unannounced site visits indicate that, on average, 13.1 percent of beneficiaries on site at the time of the visits were not registered beneficiaries. Similarly, time sheet records show that, on average, 13.6 percent of the workers listed on the time sheets were not registered beneficiaries. On average, there were 65 registered beneficiaries per subproject and 10 nonregistered beneficiaries listed on time sheets. This is not definitive evidence of informal rotation because these nonregistered beneficiaries may have simply been alternates, that is, replacements from the same household who had been nominated to work in the event of absence by the beneficiaries. Nonetheless, at least a portion of these additional workers were likely to be from households not captured by the IE. Discussions with administrative staff confirm that this was a practice at least in some sites despite attempts to prevent it.

Program impacts have been measured through the endline survey administered simultaneously to treatment and control households shortly before the CfW was launched in control communities. At the time of the endline survey, control communities were aware that they had been approved for a CfW subproject in the coming months (see above). It is plausible that this knowledge could have influenced the survey responses among control households. Specifically, control households might have strategically underreported their incomes and assets to influence the perception that they are poor and, hence, eligible for program participation. If this is the case, the IE may have overestimated program impacts. However, 8