## _Power Limitations_

Our analysis of forest cover loss is unfortunately limited by low statistical power. The minimum detectable effect sizes (shown in the row above the number of observations in **Table 5** ) are very large relative to the variation in the data. For example, the MDE in the Hansen data for the percent deforested (column 1) is 0.304. The mean value of percent forest cover loss for the non-beneficiaries is 0.619, with a standard deviation of 3.38. The MDE of 0.30 suggests that the impact of the program would have to be larger than 49% in order to be detected (0.304/0.619). In order to detect impacts in the 25-point window sample with the 2011-2012 cohort, the impact would have had to have exceeded 67% (0.644/0.956), and the corresponding minimum detectable effectiveness for the 2013-2014 cohort would have had to exceed 50% (0.151/0.319).

Results from previous studies suggest that power limitations, rather than null effects, could explain the lack of statistically significant results. The estimated changes as a percentage of expected loss are similar to those from previous research, which suggest that the 2003-2010 cohorts lowered loss of forest cover by 25-50 percent depending on methodology and unit of analysis (Alix-Garcia et al. 2013, Alix-Garcia et al. 2015, Sims and Alix-Garcia 2017). This previous work, which finds statistically significant effects, used longer time frames and/or measures that included the Normalized Difference Vegetation Index (NDVI), the latter making the environmental analysis more sensitive to degradation. These differences helped to improve power of prior studies.

# **5.2 LAND-COVER MANAGEMENT ACTIVITIES**

In this section, we describe results for the land-cover management impacts, which provide an alternate measure of the environmental impacts of the program.<sup>15</sup> **Table 7** presents results for landcover management activities using our preferred specification, which is described in equation 2. We find evidence that the PES program substantially increased an index of forest management activities at the community level (Panel A). Compared to the control means, enrolled communities have an aggregate index that is 72.0% higher for the earlier 2011-12 cohorts (equivalent to 0.80 standard deviation) and 40.0% higher for the later cohorts (0.40 standard deviation). These effect sizes correspond to large and medium effects, respectively (Cohen 1988; Sawilowsky 2009). Although the magnitude of the difference between these two effects is large, there is no statistical difference between the impacts for the two cohorts (z statistic = 0.867). When combining both cohorts, the effect was an increase of 47.6% or 0.60 standard deviation. These effects are statistically significant at the 1 percent level.

**Figure 8** shows impacts for specific community-level land management activities. While we find no statistically significant effects for specific indicators in the 2013-14 cohorts, we find that PES payments had statistically significant effects for many indicators for the 2011-12 cohorts - the construction or support for fire breaks, post control, nurseries, reforestation, patrolling and soil conservation. This may suggest having more time to organize this type of community work is important. The apparent larger magnitude for the earlier cohort could also be partly explained by the fact that earlier controls started with lower overall land cover management activities when compared to more recent controls.

> 15 These results are also published in Alix-Garcia et al. 2018.

18