Learning lessons from over-crediting to ensure additionality in forest carbon credits

Domain: neuroscience
Published: 2026-04-11
Editorial outcome: major_revision

Abstract
Independent evaluations have shown substantial over-issuance of REDD+ (Reducing Emissions from Deforestation and Degradation) credits traded on the voluntary carbon market. We synthesise these evaluations to estimate the additional forest conservation achieved by first-generation REDD+ projects and to identify mechanisms underlying over-crediting. We combine six independent ex post evaluations of avoided deforestation covering 44 REDD+ projects. These evaluations show that most projects reduced deforestation, but that they claimed an aggregate of 10.7 times more avoided deforestation than is justified by independent estimates.

This discrepancy is not driven by the choice of forest cover data, but by selection bias in projects’ control areas and modelling approaches. Although recent initiatives that transfer assessment to unconflicted parties and restrict methodological flexibility are critical, they are insufficient. Ex post certification against credible counterfactuals is also necessary if carbon markets are to contribute to slowing deforestation.

Introduction
Halting tropical deforestation is essential to limit global temperature rise to below 2°C1 and prevent mass extinction2,3. Yet tropical forests continue to be lost4,5 and forest conservation is severely underfunded, with an estimated annual finance gap of US $216 billion6. There is substantial global interest in the potential for both compliance and voluntary markets to fund forest conservation despite widespread evidence that many first-generation REDD+ (Reducing Emissions from Deforestation and forest Degradation) projects have issued more credits than were justified by their impact on deforestation7–11. The resulting controversy12 contributed to more than US $1.1Bn being wiped from the value of the voluntary carbon market in 2023 and a further contraction in 202413. Understanding what these early projects did and did not achieve, and the mechanisms that resulted in over-crediting, is key to informing efforts to improve the integrity of forest carbon credits, which is an essential step if they are to contribute to closing the forest finance gap.

A major challenge in measuring the impacts of forest conservation projects is estimating counterfactual outcomes (what would have happened to deforestation in the absence of a project)14–18. The first-generation of REDD+ projects could choose from several certification methodologies (Table 1), each of which used measurements of historic deforestation in expert-selected ‘reference’ areas in conjunction with ex ante modelling approaches to predict ‘baseline’ deforestation (see supplementary note 1 for further details)11. Credits were issued by comparing the ex ante baseline with observed deforestation in project areas. In parallel, a rich scientific literature has developed methods for estimating counterfactual outcomes from observational data19–22 and for applying them to conservation14,15,17,18. These quasi-experimental methods account for confounders in selecting control areas and measure outcomes ex post in both project and control areas (Table 2).

The prominence of REDD+ has drawn scrutiny from several independent research groups that have applied these quasi-experimental methods to evaluate the impact of first-generation REDD+ projects on deforestation7–9,23–25. Consistently, these studies have found evidence of over-crediting. Proposed explanations include the use of inappropriate reference areas, unrealistic ex ante modelling that exaggerated expected deforestation, and selective use of certification methodologies that inflated credit issuance7,9,11,23. However, these hypothesised mechanisms through which over-crediting occurred have not been systematically tested10. Some industry insiders have rejected the evidence for over-crediting altogether26,27, arguing that (1) quasi-experimental estimates of avoided deforestation are inconsistent across studies due to methodological differences28, and (2) over-crediting is an artefact of using of publicly available global forest cover datasets that detect less deforestation than the bespoke remote sensing classifications used by projects29.

Carbon markets are, of course, not the only way tropical forest conservation can be funded30,31. There are well-known problems associated with carbon offsetting32 and with the integrity of forest carbon credits beyond the issues of over-crediting and additionality33,34. Nevertheless, carbon markets are likely to play a role in bridging the forest funding gap, at least in the short to medium term6. It is therefore critical to learn as much as possible from the first-generation of REDD+ projects, both their successes and failures, to inform the future development of methods for certifying credits. Given the adverse implications of over-crediting for the core objective of carbon markets (reducing carbon in the atmosphere), understanding the mechanisms through which over-crediting occurred is vital if future approaches are to avoid repeating the failures of the past.

In this study, we synthesise results from six quasi-experimental evaluations to provide a comprehensive assessment of the impact of 44 first-generation REDD+ projects (representing 45% of the projects producing credits by 2020). We compare these results with certified assessments from the projects themselves. We address critiques of independent quasi-experimental evaluations (questions 1 and 2 in figure 1) and investigate two hypothesised mechanisms by which over-crediting could have occurred (questions 3 and 4). Our results show that most projects slowed deforestation consistently across independent evaluations, yet over-crediting remained substantial. The discrepancy between quasi-experimental and certified estimates cannot be explained by differences in the remote sensing layers used, rather, it stems from bias in reference-area selection and unrealistic modelling of future deforestation. These findings point to changes needed in crediting methodologies to ensure forest carbon credits deliver their intended climate benefits.

Results
Q1. How consistent are estimates of project impacts assessed using independent quasi-experimental methods?

The majority of projects (36 of 44) reduced deforestation according to assessments by the quasi-experimental methods (Figure 2). Among projects with multiple quasi-experimental estimates, 12 (29% of the 42 with calculable confidence intervals) showed statistically significant reductions in deforestation at the 95% confidence level. Eight projects (shown in red in Figure 2) experienced more deforestation than their quasi-experimental controls, although only in one case was this significant.

Despite most projects reducing deforestation, over-crediting was widespread with certified estimates of avoided deforestation significantly exceeding mean quasi-experimental estimates (one-tailed paired Wilcoxon signed-rank test; V = 973, p < 0.001). Over-crediting was statistically significant at the 95% confidence level for 32 projects (76% of the 42 with calculable confidence intervals; Fig. 2a). Consequently, the mean over-crediting ratio (the certified estimate divided by the mean quasi-experimental estimate) was 4.1 (95% bootstrapped CI: 2.7 to 7.0), indicating the typical project achieved about a quarter of the avoided deforestation reported by certified estimates. Across all the credits assessed, the global over-crediting ratio was 10.7 (95% bootstrapped CI: 5.4 to 26.5), implying that, on average, certified credits represented approximately one-eleventh of the avoided deforestation claimed (Fig. 2b). Over-crediting was not universal: two projects from Madagascar and one from Peru had a mean over-crediting ratio less than 1 meaning these credits achieved more avoided deforestation than claimed (Supplementary Figs. 1-2).

Q2. Is the use of global deforestation layers the reason for the discrepancy between quasi-experimental and certified estimates?

A high-profile critique of independent studies which demonstrated over-crediting by first-generation REDD+ projects has suggested that global layers used in quasi-experimental analyses detect less deforestation than bespoke layers used by projects and that this, rather than over-crediting, explains the observed discrepancies29. We tested this critique by comparing estimates of deforestation (loss of undisturbed forest) in project areas derived from global deforestation layers with those reported by projects using their bespoke forest cover classifications. We were unable to perform the comparison for reference areas because deforestation rates there were not reported in project certification documents. For project areas, we found that, rather than being lower, deforestation rates measured independently using the European Union’s tropical moist forest annual change collection (ACC5) were actually greater than certified measurements based on bespoke forest cover layers. Among the 36 projects that reported certified measurements, the median rate measured using the ACC was 0.26%/year, compared with a median certified deforestation rate of 0.17%/year (Fig. 3a). The median pairwise discrepancy between ACC and certified rates was 0.13%/year (Fig. 3b), which was significantly different from zero (Wilcoxon paired signed-ranks test, V = 520, d.f. = 35, p = 0.001). When we ran the analysis to include pixels classified as degraded forest in the ACC-measured forest area at project start, and measured deforestation only as long-term (>2.5 years) transitions to non-forest, deforestation rates were lower and not significantly different from certified values (supplementary figs. 3-6). Together these results indicate that deforestation products (such as the ACC) used in quasi-experimental analyses have not detected less deforestation than the bespoke layers used in certification. Under-estimation of deforestation in control areas therefore cannot explain the observed over-crediting.

Q3. Were reference areas similar to projects in their exposure to deforestation?

Having established that reports of lower additionality were consistent across quasi-experimental assessments and were not driven by forest-change layers that underestimate deforestation, we next examine why certification methodologies may have overestimated project additionality. Specifically, we ask whether the reference areas used were representative of the deforestation pressures to which projects were exposed.

Reference areas were systematically different from project areas, exhibiting lower exposure to deforestation before project implementation. To illustrate this, we focus on project 958 in Peru (equivalent figures are shown for all projects in Supplementary Figs. 7-19). Project 958’s reference area was located immediately outside the project (Fig 4a). Despite this proximity, the reference area was less inaccessible (a mean of 13.8 hours compared with 25.5 hours), and less forested at the start of the relevant evaluation period (78% for the reference area in 2001 compared with 95% for the project in 2010 (Fig. 4b). Standardised mean differences of -1.02 and -0.58 respectively indicate that the reference area were significantly different from the project. By contrast, the control units selected by the quasi-experimental matching approach were distributed much more widely across Peru (Fig. 4a) and were nearly identical to the project in terms of observable confounders (Fig. 4b).

The same pattern was observed across all 17 projects that provided mapped reference areas (Fig. 4c, Supplementary Figs. 7-19). Compared with project areas, reference areas were more accessible (median standardised mean difference = -0.30, t = -2.41, d.f. = 16, p = 0.028). 82% of projects used reference which had locations more accessible than the most accessible locations within projects. Similarly, reference areas were less forested than projects at their start (median standardised mean difference = -0.42, t = -3.10, d.f. = 16, p = 0.007) as well as five and ten years beforehand (median standardised mean difference = -0.48, t = -4.04, d.f. = 16, p = 0.003; and -0.48, t = -4.28, d.f. = 5, p = 0.008, respectively). Historic deforestation rates were not significantly different. In contrast, none of the quasi-experimental control areas showed meaningful differences from the project areas in any of the observed confounders (Fig. 4c).

These differences in exposure to observable confounders between reference and quasi-experimental control areas were associated with a directional difference in their deforestation rates during the project period when assessed using the ACC layer. Across the 17 projects, median deforestation rates in reference areas were 0.69%/year compared with 0.41%/year in the quasi-experimental control areas (Fig. 4d). Although this was 1.69 times more deforestation the increase was not statistically significant (one-tailed Wilcoxon paired signed-ranks test, V = 103, p = 0.1123). The same pattern was found when considering the loss of either undisturbed or degraded forest in the ACC deforestation measurements (Supplementary Figs. 20-22).

Q4. After isolating the effects of forest cover layer and reference area selection, how much over-crediting can be explained by ex ante modelling?

To dissect the relative contribution of the mechanisms identified in preceding analyses, we examined their effects in the subset of projects (n = 17) for which all required data were available. The smaller sample size meant that most effects within this subset were not statistically significant, although they were of a similar magnitude to those observed in the larger analyses of each mechanism. Certified assessments reported substantially more avoided deforestation than quasi-experimental assessments (0.95 %/year shown in pink compared to 0.18%/year shown in green in Fig. 5; Wilcoxon paired signed-ranks test, V = 134, p = 0.002). We then sequentially re-estimated avoided deforestation using different combinations of the deforestation rates drawn from the certified and quasi-experimental assessments to isolate the effect of each mechanism, with any remaining over-crediting assumed to be explained by ex ante modelling.

First, to account for differences between the remote sensing layers in project areas, we substituted the ACC-measured deforestation rates with the certified rates. Because certified rates were lower, this substation increased median avoided deforestation slightly to 0.20 %/year (orange in Fig. 5), indicating that the choice of deforestation rate for the project area contributed only about 2% of the observed over-crediting.

Second, to isolate the effect of reference area selection, we substituted the quasi-experimental control areas with the certified reference areas and measured deforestation in both using the ACC. Median avoided deforestation then increased to 0.64 %/year (blue in Fig. 5), intermediate between quasi-experimental and certified assessments, suggesting that reference area choice probably accounted for 57% of the over-crediting.

Next, we estimated the effect of remote sensing differences on reference area deforestation rates. Because project documents did not report deforestation rates in reference areas, we inferred them by multiplying the ACC-derived control area deforestation rates by 0.58, which was the median ratio of bespoke to ACC rates across the 17 projects. Substituting these inferred values reduced avoided deforestation to 0.35 %/year (grey in Fig 5), suggesting that lower detection of deforestation by bespoke remote sensing (rather than the ACC) was not a cause of over-crediting.

Finally, having adjusted for reference area selection and differences in deforestation rate estimation, we attributed the remaining ~78% of over-crediting to the various ex ante modelling approaches used in certified assessments, recognising the considerable uncertainty surrounding this allocation.

Methods
Certified estimates of avoided deforestation

Certified rates of avoided deforestation, used to generate carbon credits, were sourced from the design and monitoring documents of 44 REDD+ projects accessed through the Verra registry between April and June 2023. These include the loss of all areas that had been forest for at least 10 years prior to the establishment of the project. Reference areas were selected so that they had similar characteristics to projects in terms of forest cover and expected drivers of deforestation (see table 1). Forest cover was classified using cloud-free satellite images and bespoke remote sensing approaches. Ex ante estimates of counterfactual deforestation were estimated using a variety of different modelling approaches according to the methodology used. Further detail on certified project monitoring and counterfactual estimation is presented in Supplementary Note 1. Projects included in the study were those that had sufficient publicly available data, including geospatial polygons of project areas, and had reported the area of avoided deforestation in their most recent monitoring reports (Supplementary Table 1).

Certified project documents did not report deforestation consistently; however, the ex post observed deforestation within the project and the ex ante modelled deforestation under the counterfactual scenario were available. Verra projects refer to the counterfactual scenario as the ‘baseline’. Importantly, deforestation was provided as a total amount in hectares over the whole evaluation period or broken down into annual amounts. Where avoided deforestation was not reported, it was calculated by subtracting project deforestation from counterfactual deforestation.

For Q1, the mean certified annual avoided deforestation in hectares was used. For Q2-Q4, for each project, the total amount of avoided deforestation claimed was disaggregated across the evaluation period (which had a mean length of 6.4 years) to produce annualised compound avoided deforestation percentage rates (see below). Taking this approach ensured that the avoided deforestation used was the same as that used to issue credits.

Projects issued credits using four different Verra methodologies (VM0006, VM0007, VM0009, VM0015) to estimate counterfactual outcomes. The different methodologies are summarised briefly in Table 1 and the process common across these methodologies is described in Supplementary Note 1. The extracted project and counterfactual deforestation rates as well as the certified avoided deforestation are presented in Supplementary Tables 3-4.

Quasi-experimental estimates of avoided deforestation

We compiled data from six quasi-experimental analyses which estimated ex post counterfactual and project outcomes for REDD+ projects (Table 2). These included work published by West et al.7,9 (two analyses), Guizar-Coutiño et al.8 and Tang et al.23.

We also included results from a study under review by Guizar-Coutiño et al.25, which implemented a novel framework to reanalyse the set of projects examined in Guizar-Coutiño et al.8 study. In their study, 7.1-ha circular sample regions were used instead of 0.09 ha pixels to characterise forest loss observations. Guizar-Coutiño et al.25 assess the causal impact of REDD+ projects on deforestation by adjusting for observed confounders and pre-treatment deforestation trends, using a two-stage matching approach, commencing with propensity score matching via random forests, followed by propensity score subclassification on the matched datasets69. They assess the average treatment effect of the treated using multiple model specifications with different algorithms, parameters and covariates. The avoided deforestation estimates used in our study correspond to the doubly robust model specification reported in this study.

Finally, we used the Permanent Additional Carbon Tonne (PACT) v2 method24 (described below) to produce additional estimates and to explore the mechanisms that may have been responsible for over-crediting. In each case, we included projects for which certified estimates were also available.

Avoided deforestation estimates from quasi-experimental methods were received from the study authors as project level estimates of cumulative avoided deforestation over a defined study period. Annual avoided deforestation in hectares was calculated by dividing cumulative avoided deforestation by the total number of years of the evaluation, to compare estimates produced over different time periods. Evaluation periods were 5 years for both Guizar-Coutiño et al. studies8,25, and a mean of 6.0, 7.6, 9.0, and 8.1 years for the West et al. (2020)7, West et al. (2023)9, Tang et al. (2025)23, and PACT24 studies respectively.

Deforestation was assessed using publicly available remote sensing products for tracking changes in forest cover, specifically the corrected MapBiomas70 dataset used by West et al. (2020)7, the Global Forest Change (GFC)71 dataset used by West et al. (2023)9 and Tang et al, (2025)23, and the Tropical Moist Forest Annual Change Collection (ACC)5 used by Guizar-Coutiño et al.8,25 and PACT24. GFC reports deforestation as the year of gross forest loss.

The ACC dataset5 classifies each 30 m Landsat pixel within the humid tropics as either undisturbed forest, water or non-forest in 1990. Then disturbance events were classified according to their duration, as either degradation (<2.5 years) or deforestation (>2.5 years). If forests subsequently recovered, after a period of at least 3 years, this was classified as regrowth. Some disturbed forests and plantations were erroneously classified as undisturbed forest due to a lack of availability of a longer time series of observations and because their spectral characteristics were similar to those of natural forests. This is particularly problematic in Africa due to limited Landsat coverage before 2000.

The Guizar-Coutiño and PACT studies considered deforestation to be degradation or deforestation of the undisturbed class, even if regrowth subsequently occurred. Degradation was included because it has been shown to result in biomass reduction of >50% within 12 months. Our analysis of above ground biomass densities for different ACC cover classes indicates that both degraded and regrowth classes had <50% of the above ground biomass of the undisturbed class (see Supplementary Table 5 and Supplementary Fig. 24). Therefore, for the mechanistic component of our study, we followed the PACT methods by defining forest cover solely as the undisturbed class within the ACC dataset. We explore the differences that arise using different definitions of forest cover in supplementary figures 3-6 and 20-23. We chose to focus our assessment on avoided deforestation because it lies at the centre of the REDD+ controversy, while recognising that estimating carbon fluxes is an additional, important and complex topic.

The PACT method

The PACT method used a bootstrapped one-to-one pixel (0.09 ha) matching approach to produce paired project and control units from which to estimate mean project and counterfactual outcomes72,73. Project areas, start dates, and the date of the most recent monitoring period were obtained from the VCS registry (accessed between April and June 2023). Projects were then assessed following four main steps: (1) compile layers for characterising project units; (2) determine the set of suitable untreated units; (3) form the control set by matching untreated and project units; and (4) measure and average outcomes across project and control sets (Supplementary Note 1).

First, geospatial raster layers were compiled to track changes in forest cover. The ACC dataset5 was used for this purpose. We used the 2021 version, a 30 m resolution (0.09 ha) time series spanning December 1990 to December 2021, classifying pixels as one of six classes: undisturbed, degraded, deforested, restored, water and other.

Secondly, layers were compiled to select control units analogous to project units using a method informed by a causal model of land use change. This model incorporated covariates that captured the effects of local policies and regulations, economic pressures, historical changes in land cover, and environmental conditions on deforestation rates and conservation efforts. We used the following layers: International country borders (OpenStreetMap74) to encompass national regulatory limits; ecoregions (Resolve75) to ensure biotic equivalence; and elevation (NASA’s Shuttle Radar Topography Mission76) as a proxy for abiotic conditions. From the elevation layer, we calculated slope using GDALDEM v3.8.4 (default settings), an important proxy for accessibility and suitability for economic activities. This produced a slope variable consisting of integer values representing the mean slope (0-90°) in each raster cell.

Motorised accessibility to healthcare in 2019 (Malaria Atlas Project54) was used as a proxy for market exposure/economic pressure. This accessibility layer integrates all known travel routes, including rivers, along with terrain, land cover, and road quality to assess the travel time to population centres large enough to have a healthcare facility. Finally, we computed the proportional cover of undisturbed and deforested classes from each year of the ACC time-series (1990-2021) to assess land-use change trajectories in the surrounding area through time. Proportional cover was calculated by counting the number of pixels in each class and dividing by the total number of pixels within the 1 km radius neighbourhood around each pixel.

It is important to note that accurate data regarding land value and accessibility to markets is not universally available, particular in frontier landscapes. As such we use spatial proxies, specifically proportional cover and motorised accessibility to healthcare, recognising that these leave space for unobserved confounders.

Finally, to minimise interference arising from local leakage, we excluded control units within the vicinity of any REDD+ project. This is important because, under the assumption that leakage arises through the reorganisation of local supply chains or markets, there would be concentrated interference at short distances from projects. To do this we produced a binary raster indicating the presence or absence of a REDD+ project within 5km. REDD+ polygons were accessed from the VERRA registry and supplemented by any REDD+ project polygons shared directly with the Cambridge Centre for Carbon Credits. All layers were reprojected to the same coordinate reference system as the ACC.

Projects were sampled using a spatial grid with a density of 0.25 points per ha for projects smaller than 250,000 ha and 0.05 points per ha for larger projects (to reduce processing time). This resulted in a mean of 17,397 thousand units across the projects (min = 687, max = 44,697). The characteristics of these sample points were extracted from the layers described above. Time-varying characteristics, including the ACC pixel class and proportional cover were extracted at the start of the project (t0) and at five (t-5) and ten (t-10) years prior.

The domain of untreated units was defined as all pixels within the same countries and ecoregions as each project, located at least 5 km from the project boundary, but no further than 2,000 km away. Leakage effects caused by the displacement of production could be present but were assumed to be small, given the large domain of the untreated units and the relatively limited effects of individual projects. To reduce computation time, untreated units were filtered to include only those within the range of values observed in the project area for each known driver of deforestation with an added tolerance of ±200 m for elevation, ±2.5° for slope, ±10 minutes accessibility, and ±10% points for proportional cover.

Matching proceeded by taking a random sample of 10% of the project units, which were then processed sequentially to identify control units with the smallest Mahalanobis distance across the continuous characteristics, and identical values for country, ecoregion and ACC land cover class at t0, t-5 and t-10.77. This matching approach is referred to as ‘greedy’ because the algorithm sequentially finds the best pairs, which are then removed from the pool for subsequent matching.

The standardised mean difference (SMD) between the control and project sets for each of the continuous characteristics was calculated as follows:

SMD = (μc - μp) / σ̅ Eq. 1

Where μc is the mean of the control set, μp is the mean of the project set and σ̅ is the square root of the mean of the variances. SMD values outside the range [-0.25, 0.25] for any continuous characteristic were considered statistically different and these samples were excluded from further analyses14,77. This process was repeated 100 times to produce matched project-control sets, each comprising 10% of the total points sampled from within the projects.

Ex post deforestation was measured for the project and control units by calculating changes in the undisturbed class since the start of the project. To calculate the area of undisturbed forest at each time point, we computed the proportion of all units in the undisturbed class and multiplied it by the project area. This process was repeated across all 100 matched sets to produce estimates of project and counterfactual forest cover.

Averaging across these sets, we produced annualised mean forest cover (in hectares). By subtracting counterfactual forest cover from the project forest cover, we derived annual cumulative avoided deforestation values. Total avoided deforestation during the project period was measured at the final year of the project’s most recent monitoring period, as additionality figures are cumulative.

Q1. How consistent are estimates of project impacts assessed using independent quasi-experimental methods?

For each of the 44 projects, we compiled the certified estimates of annualised avoided deforestation (in hectares) along with all available quasi-experimental estimates. For the 42 projects with more than one quasi-experimental estimate, we calculated the mean, standard deviation, coefficient of variation, standard error, and 95% confidence intervals. For the remaining two projects, which had only a single quasi-experimental estimate, this value was taken to be the mean.

To test for over-crediting, we determined how many projects had mean quasi-experimental estimates below their certified estimate, which was deemed significant at the project-level if their upper 95% confidence intervals were less than their certified estimates. To test whether certified estimates were systematically greater than mean quasi-experimental estimates, we applied a one-tailed Wilcoxon signed-ranks test, due to the non-normality of the data. We also evaluated how many projects had avoided a significant quantity of deforestation by testing if the lower bound of the 95% confidence intervals were greater than zero across quasi-experimental studies.

We next assessed over-crediting as the ratio of the certified to the mean quasi-experimental estimate of avoided deforestation. We refer to this as the over-crediting ratio, which expresses how many times more credits were issued through certification than suggested by quasi-experimental estimates. The mean over-crediting ratio was calculated as the mean of the over-crediting ratios of all projects that had both issued credits and had positive mean quasi-experimental estimates; projects with negative mean quasi-experimental estimates were excluded. The global over-crediting ratio was calculated by dividing the summed certified avoided deforestation by the summed mean quasi-experimental estimates (including negative values). The mean and global over-crediting ratios respectively represent how many hectares of certified avoided deforestation correspond to one quasi-experimentally assessed hectare for a project selected at random, and across the REDD+ portfolio. Because the data were non-normal, non-parametric 95% confidence intervals were produced for the mean and global over-crediting ratios by randomly sampling projects with replacement 10,000 times.

Determining comparable deforestation rates between Certified and ACC sources

We calculated percentage annual deforestation rates to make the quantities of deforestation measured by the certified assessments and the ACC layer (e.g. those produced by PACT) directly comparable. The forest cover (Ft) resulting from a constant annual deforestation rate (r) after an interval (t), given the starting forest cover (F0), was calculated as follows:

Ft = F0(1-r)t Eq. 2

Because we had extracted the forest cover at the beginning and end of the evaluation period, we calculated the rate by rearranging the formula to:

r = 1-(Ft/F0)1/t Eq. 3

To determine F0 we took the proportion of the ACC pixels classified as the undisturbed class in the yearly layer closest to the project start date and multiplied this by the total project area derived from the project polygon. Thus F0 was the total area of undisturbed forest (in hectares) at the start of the project. Ft was calculated by subtracting the area of deforestation in the certified or ACC measurements from F0. The evaluation interval t was the number of years between the start and end assessments. For the certified assessments, this was calculated from the number of total days between the start and end dates, whereas for the ACC data, this was the number of whole years between the ACC layers used. The calculations for each project are presented in section Supplementary Tables 6-7.

We produced certified and ACC mean annual percentage deforestation rates for 36 project areas and 17 counterfactual estimates. Between these two sets there was an intersection of 17 projects, for which we could produce mean annual percentage avoided deforestation rates using all combinations of certified and quasi-experimental estimates for project, reference and control areas, sufficient for inclusion in Q3 and Q4.

Q2. Is the use of global deforestation layers the reason for the discrepancy between quasi-experimental and certified estimates?

We used the annual deforestation rates to test if there was a difference in the amount of ex post deforestation measured in project areas. Because the data were non-normal, we tested whether the paired differences were significantly greater than zero using a one-tailed Wilcoxon signed-ranks test. In the main analysis, we focussed exclusively on the deforestation or degradation of undisturbed forest in the ACC measurements. In Supplementary Figures 3-6 we broaden the forest cover definition to include undisturbed and degraded forest and measure the deforestation of either.

Q3. Were reference areas similar to projects in their exposure to deforestation?

We compared the exposure to the drivers of deforestation across project, reference and control areas to assess the extent to which the selection of reference areas explained differences between certified and quasi-experimental assessments. For reference areas, we focused on the Reference Regions for Deforestation (RDD) used by certified methods as these were used to measure the rate of deforestation, rather than the Reference Regions for Location, which were used to model how much of the deforestation was expected to occur within project areas.

Reference area polygons were not publicly available as shapefiles and were therefore digitised by georeferencing maps available in project design documents from the Verra registry. This was done either by tracing the polygons by hand in QGIS (v3.26.3) or through colour thresholding and automated polygonisation procedure in R, equivalent to the process described by the Environmental Systems Research Institute78. Colour thresholding was applied when reference areas presented in project design documents were complex shapes represented by colour coded systems within the georeferenced maps. The colours representing the reference areas were identified as the most frequent pixel colours in the maps. Binary maps were then produced from these pixels using a Boolean comparison to the reference area colours, and their fidelity was checked against the original maps. Finally, the binary maps were converted to geospatial polygons using the polygonise function in the Terra package in R.

We successfully digitised reference areas for 17 projects (see Supplementary Table 1 for the projects included). We then sampled the characteristics of reference areas using the same PACT method for sampling project units (described above). Time-varying characteristics were sampled from the beginning of the historical reference period (usually 5-10 years before the start of the project), to ensure comparability with project characteristics.

To test the similarity of control and project areas in terms of exposure to deforestation risks, we examined the distributions of key characteristics prior to any measurement of forest loss. For the project areas, these covered the pre-project period up to a maximum of ten years prior. This was the same for the quasi-experimental controls. For reference areas, the period extended up to ten years prior to the start of the documented reference period.

We tested for differences in the univariate distributions of pre-project characteristics using the SMDs between the reference or control areas and the project areas. Values outside the range [-0.25, 0.25] indicated a significant project-level difference14,77. Across the set of projects, we tested whether the distribution of the SMDs for each characteristic differed significantly from zero using t-tests.

We also compared the observed annual deforestation rates between reference and quasi-experimental control areas using the ACC dataset. For reference areas, this was measured across the reference period; for quasi-experimental controls, it was measured across the project evaluation period. Because the data were non-normal, we applied a one-tailed Wilcoxon paired signed-ranks test to assess whether reference areas were exposed to significantly more deforestation than the quasi-experimental control areas.

Q4: What is the residual effect of ex ante modelling after isolating the effects of forest cover layer and reference area selection?

To assess the effect of ex ante modelling, we used a process of isolating all possible explanations for over-crediting for the same 17 projects used in Q3. This required determining the annual avoided deforestation rate produced by five different combinations of reference, control and project area estimates.

First, we quantified over-crediting as the overall difference between avoided deforestation from quasi-experimental estimates (PACT in combination with ACC) and certified estimates. To isolate any effect of project area remote sensing within this overall difference, we examined the change in avoided deforestation resulting from substituting quasi-experimental project area deforestation estimates with their certified equivalents.

From this new combination, we then isolated the effect of reference area selection by substituting quasi-experimental control area estimates with ACC-derived estimates for the reference areas (covering the reference period) made in Q3. This third combination therefore represents the combined effect of the two mechanisms explored in Q2 and Q3. The residual difference between this combination and the avoided deforestation rates produced by purely certified estimates captures the influence of two remaining factors: (1) the bespoke remote sensing measurements made in reference areas; and (2) the ex ante modelling of reference area deforestation to predict counterfactual outcomes for project areas.

Although we could not disentangle these explanations, we examined whether differences between remotely sensed forest cover layers affected the magnitude of the reference area effect. To do this, we inferred the deforestation rates that would have been observed if bespoke measurements in control areas occurred at the same relative proportion of ACC-measured deforestation rates observed in project areas. We divided certified estimates by quasi-experimental estimates in project areas and took the median of these ratios, to produce a correction coefficient. We then multiplied the ACC measurements in reference areas by the coefficient to generate corresponding “bespoke” estimates, from which we produced our final possible avoided deforestation rate. This yielded two possible residuals covering a range of impact attributable to ex ante modelling.

Statistical differences between ranges of avoided deforestation were tested using a one-tailed Wilcoxon paired signed-ranks tests because paired differences were not normally distributed.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "This paper presents a compelling assessment of the level of additionality calculated in multiple REDD+ analyses by certification agencies compared to an assessment of the same projects using statistical methods. The purpose of this is to help assess the level of overcertification and point the way towards a more constructive process in the future. I appreciate the effort and approach, but I have a few comments that I think may help clarify the analysis and policy recommendations and also temper some of the conclusions.\nMain comments:\n1. Measurement error in deforestation data: I really like the general approach of trying to separate out matching quality from remote sensing error. However, I am concerned about two things related to the quality of the remote sensing data. First, the measurement error in MapBiomas and the GFC product is significant. A number of papers document serious underestimation of deforestation in GFC (Burilavova et al. 2015, Mitchard et al., 2015). I have not seen such studies for MapBiomas, but I do know that the estimations of change are sometimes revised downwards in subsequent versions of the series. This is not to criticize these products, but rather to note that it is quite possible that this error creates significant problems for statistical assessment of impacts. If the measurement error is classical, this may attenuate your point estimates. If it is non-classical, the direction of the bias is unpredictable.\nFurther, when estimated rates of deforestation are low, this puts a ceiling on potential impacts. You might note this point somewhere in your description of your statistical assessments.\nYou provide little information on the “bespoke” remote sensing products used by certifiers. It is likely that these also have significant error, but if they are created at a lower resolution than 30m, they probably more accurately assess deforestation in low income, tropical settings where clearing takes place in small patches. You resolve part of this by noting that you find certified deforestation relative to your own assessment to be higher in reference areas than in program areas. However, it feels like there need to be error bars on this assertion. The error bars that you have accommodate the error in the data used for statistical analysis. It seems like there should also be an accommodation for the error in the certification data. If standard deviations for their estimations are available, then perhaps you could use these to adjust the error bars in your figure 1b, which has error in both the numerator and the denominator.\n2. Clarity on methodology:\na. It would be nice to have more clarity on the “statistical approaches” used in the paper. I am particularly interested in the covariates. The paper uses different “confounders” and approaches in every assessment. This makes it hard to figure out what is actually happening. To my mind, if you are trying to propose an alternative methodology, then you should propose something that is replicable and consistent. For example, you might want an approach that uses similar covariates across multiple projects. One can always, with refined data and endless iterations of increasingly complex statistical approaches, come up with much better estimates than are done with more coarse, globally consistent data. Further, hindsight, as they say, is 20/20. Ex post estimates will always show something different than ex ante predictions. In addition, given what you have demonstrated about the cherry picking of reference areas towards those that have higher levels of deforestation, one wouldn’t want to leave a huge number of degrees of freedom in the way that reference areas are selected. It seems unreasonable to suggest that certifying institutions should be conducting analysis with the same number of robustness checks that academic researchers do. We need to provide guidance on what might be both effective and feasible.\nb. It would be useful to describe in the appendix the ways in which VERRA is estimating their counterfactual areas. Creating a deforestation risk map based upon a number of covariates and then selecting areas of similar baseline risk is quite similar matching. Your analysis of covariate similarity suggests that they may not be doing a great job on this, but it is hard to tell because we don’t really know what they are doing.\nFurther, I have the impression from Table 1 that the VERRA estimates of deforestation are based upon predictions rather than on observed deforestation. Is that correct? In this case, it seems like a major source of the problem in the assessments could be that the prediction is not very good, not that the area of comparison is poorly selected. Perhaps you can clarify what is actually happening here.\n3. Clarity of policy recommendations:\nI think that a considerable amount of work could be done to make the policy recommendations of this paper more clear and helpful. Below are some suggestions.\na. Some constellation of the authors in this paper have already established that there is an issue with over certification on forest carbon markets. It would be useful to understand what the first result of this paper adds there that is new.\nb. The main constructive recommendation of the paper is that ex post statistical approaches be used in carbon crediting. As I was thinking about this, I got confused about what is actually being done versus what is proposed. Are the authors suggesting withholding payments until assessment of avoided deforestation can occur ex post? This seems like an impossible proposition, both at the broader institutional level and at the level of producer participation in the event that projects involve individual land users foregoing deforestation, which they must do for there to be additionality. Recent research suggests that successfully implementing PES in low-income settings may require ex ante payments to actually induce wide-scale participation (Izquierdo-Tort et al. 2024). It therefore seems more reasonable to suggest a way to consistently apply matching or the construction of synthetic controls ex ante using data that is generally available and covariates that are known to be correlated with both selection into treatment and deforestation outcomes. I think that is what is being suggested here. Counterfactual deforestation could then be predicted for reference areas, and then once time has passed actual deforestation rates would be used in assessment. I think what I would like to see is clarification of the policy proposition.\nc. Two implementation issues should be recognized in the discussion.\ni. The first of these is that perfect prediction of the location of future deforestation is impossible. Deforestation is a rare event statistically, and so targeting programs such that only future deforested land is enrolled does not make sense, regardless of how much data you have with which to make the prediction. Even trying to approach this kind of targeting in a REDD+ program would require taking money away from actual incentives and funneling it into bureaucratic processes intended to support targeting, which would be wasteful. This means that you have to pay for enrollment of larger areas of land than are likely to be deforested.\nii. This does not in itself undermine the principle of this paper. However, it is worth considering that the global community often says that avoided deforestation and, more recently, forest restoration, is “cheap.” The perceived cheapness of the intervention may be at least partly driven by the fact that when we calculate REDD+ transfers per hectare using current estimates of deforestation, they are very low. However, we might find that if we use current per hectare rates spread over a much lower, precisely estimated amount of avoided deforestation, we may not be able to induce any participation in these types of programs, because they will not sufficiently compensate producers for their opportunity cost. Environmental non-profit organizations and rich country governments often engage in wishful thinking regarding the true costs of nature-based solutions. It is important that the research community does not fall prey to this same mistake.\nSmaller issues:\nI believe that the acronym JRC TMF is not defined anywhere.\nWorks cited:\nBurivalova, Zuzana, Martin R. Bauert, Sonja Hassold, Nandinanjakana T. Fatroandrianjafinonjasolomiovazo, and Lian Pin Koh. 2015. Relevance of global forest change data set to local conservation: Case study of forest degradation in Masoala National Park, Madagascar. Biotropica 47 (2): 267–74.\nIzquierdo-Tort, S., Jayachandran, S. and Saavedra, S., 2024. Redesigning payments for ecosystem services to increase cost-effectiveness (No. 21022). https://seemajayachandran.com/pes_redesign_mexico.pdf\nMitchard, Edward, Karin Viergever, Veronique Morel, and Richard Tipper. 2015. Assessment of the accuracy of University of Maryland (Hansen et al.) Forest Loss Data in 2 ICF project areas: Component of a project that tested an ICF indicator methodology. Technical report, University of Edinburgh.", "reviewer_2": "Reviewer #1 (Remarks to the Author):\nThis paper presents a compelling assessment of the level of additionality calculated in\nmultiple REDD+ analyses by certification agencies compared to an assessment of the same\nprojects using statistical methods. The purpose of this is to help assess the level of\novercertification and point the way towards a more constructive process in the future. I\nappreciate the effort and approach, but I have a few comments that I think may help clarify\nthe analysis and policy recommendations and also temper some of the conclusions.\nMain comments:\n1. Measurement error in deforestation data: I really like the general approach of trying to\nseparate out matching quality from remote sensing error. However, I am concerned about\ntwo things related to the quality of the remote sensing data. First, the measurement error in\nMapBiomas and the GFC product is significant. A number of papers document serious\nunderestimation of deforestation in GFC (Burilavova et al. 2015, Mitchard et al., 2015). I\nhave not seen such studies for MapBiomas, but I do know that the estimations of change are\nsometimes revised downwards in subsequent versions of the series. This is not to criticize\nthese products, but rather to note that it is quite possible that this error creates significant\nproblems for statistical assessment of impacts. If the measurement error is classical, this\nmay attenuate your point estimates. If it is non-classical, the direction of the bias is\nunpredictable.\nreviewer that it is possible that certified classifications were more accurate than the ACC.\nHowever, our comparison of ex post measurements of deforestation in project areas, which\nshould be the same for both the ACC and the bespoke layers, demonstrates there was no\nsignificant difference and the ACC actually detected more deforestation. This is now\npresented clearly in the new figure 3. We also now make clear throughout the manuscript\nthat remote sensing constitutes only one source of error in the certified estimates. Other\nerrors come from the selection of the control areas and the choice of modelling approach.\nThese effects are isolated in new figure 5. Unfortunately, certified estimates do not typically,\nif ever, present statistical errors that properly account for all sources of error. This precludes\nthe possibility of us adjusting the error bars to what was figure 1b (now figure 2b).\n2. Clarity on methodology:\na. It would be nice to have more clarity on the “statistical approaches” used in the paper. I\nam particularly interested in the covariates. The paper uses different “confounders” and\napproaches in every assessment. This makes it hard to figure out what is actually\nhappening. To my mind, if you are trying to propose an alternative methodology, then you\nshould propose something that is replicable and consistent. For example, you might want an\napproach that uses similar covariates across multiple projects. One can always, with refined\ndata and endless iterations of increasingly complex statistical approaches, come up with\nmuch better estimates than are done with more coarse, globally consistent data. Further,\nhindsight, as they say, is 20/20. Ex post estimates will always show something different than\nex ante predictions. In addition, given what you have demonstrated about the cherry picking\nof reference areas towards those that have higher levels of deforestation, one wouldn’t want\nto leave a huge number of degrees of freedom in the way that reference areas are selected.\nIt seems unreasonable to suggest that certifying institutions should be conducting analysis\nwith the same number of robustness checks that academic researchers do. We need to\nprovide guidance on what might be both effective and feasible.\nReviewer #2 (Remarks to the Author):\nSummary\nThank you for the opportunity to read this paper. I very much enjoyed reading it.\nThis work investigates the Carbon Credit certification methods for avoiding deforestation\nfrom REDD+ (Reducing Emissions from Deforestation and Degradation) projects. Carbon\ncredits are an essential pillar of the agenda to reduce the impacts of deforestation on carbon\nemissions and climate change. The author’s work highlights the flaws of current\nmethodologies in creating reliable estimates of avoided deforestation from REDD+ projects.\nCrediting institutions mostly estimated high levels of avoided deforestation. In contrast, using\nreliable assessment methods, the paper shows that REDD+ projects had much lower levels\nof avoided deforestation. As a result, REDD+ projects have been the base for issuing overly\nhigh carbon credits. The authors highlight that over-crediting stems (a) from a failure to\ncreate credible counterfactual scenarios and (b) from systematically choosing methods that\ngenerate higher levels of avoided deforestation. This research is especially important as\nupcoming reforms aim to expand REDD+ projects and the monetary compensation to\njurisdictional entities. Maintaining the credibility of the certification system will be essential to\nsecure the future funding of these market-based forest conservation instruments.\nI appreciate the analysis of the overestimation of avoided deforestation in REDD+ projects.\nThis is an excellent depiction of the potential flaws in estimating overly high levels of avoided\ndeforestation. My primary concerns are about the novelty of the results, the author’s\nempirical reasoning, and the use of the empirical terminology. If the authors pursue these\nimprovements and others they receive elsewhere, I believe the paper will make a very\nvaluable contribution to the literature.\nMajor comments:\n1. Novelty: The database for this paper is collected from different sources. The authors\ncombine data from the crediting institution (i.e., VERRA) and from previous publications\nevaluating REDD+ projects using quasi-experimental methods published by\nWest et al. (2020), West et al. (2024), Guizar-Couti˜no et al. (2022), Guizar-Couti˜no\net al. (2024), and Balmford et al. (2023) (henceforth WGGB). The latter, Balmford\net al. (2023), is an unpublished document describing the PACT method. Based on the\nmanuscript, it is very difficult to understand if PACT or Balmford et al. (2023) are a\nsource of data or if the authors are referring to a methodology. Is the paper conducting\nnew impact evaluations or just using the results from Balmford et al. (2023) or even\nfrom (cf. Swinfield et al., 2024) – a recent publication of the lead authors? If so, what\nis the genuine contribution of the paper? Has the paper “only” pooled estimates and\nre-used the information to display the results in a more comprehensive way?1.\n1 1 Figure 1 is based on all 47 REDD+ projects as it collects data from VERRA, West et al.\n(2020), West et al. (2024), Guizar-Couti˜no et al. (2022), Guizar-Couti˜no et al. (2024), and\nBalmford et al. (2023) (henceforth WWGGB). Figure 2 is limited to 4 projects because West\net al. (2024) as data on it. Figure 3 is restricted to 16 projects where boundaries of VERRA’s\nThe authors should highlight their essential contribution better. It seems the essence\nlies in the two messages i) There is over-crediting, ii) Over-crediting can stem from\na) selecting dissimilar control, b) mismeasuring deforestation in control areas, and c)\nchoosing evaluation methods that rely on strong assumptions (see comment XXX). To\navoid confusion, I suggest sticking with the PACT method. First, show Figure 3, then\nFigure 1, and finally, Figure 4. Figure 2 should be an Appendix table. It lacks power\nand is merely based on 4 REDD+ projects analyzed by West et al. (2024).2\nReviewer #3 (Remarks to the Author)\nI co-reviewed this manuscript with one of the reviewers who provided the listed reports. This\nis part of the Nature Communications initiative to facilitate training in peer review and to\nprovide appropriate recognition for Early Career Researchers who co-review manuscripts.\nReviewer #3 (Remarks on code availability):\nThe link for the data availability did not work.\nReviewer #4 (Remarks to the Author):\nThe manuscript entitled “Independent statistical approaches address overcrediting problems\nin REDD+” compares the avoided deforestation rates from ex ante assessments using the\nmethods applied by a major carbon crediting agency with the ex post estimates of statistical\napproaches for 47 REDD+ projects in Latin America and Africa. The manuscript argues that\nthe ex ante methodology tends to overstate the avoided deforestation by the REDD+\nprojects due to the selection of often incomparable reference groups.\nWhile the topic of revising REDD+ and quantifying additionality from conservation and\nclimate change mitigation interventions is very timely and important, significant concerns\nremain about the manuscript regarding the statistical methods employed.\nStatistical and model considerations\n1. Without perusing the prior studies on which the manuscript is based, it is not clear what\nthe ex post statistical methods employed are: For example, the manuscript mentions\nsynthetic controls and one-to-one matching without replacement (Table 1), but does not\nprovide key details. For example, why not synthetic difference-in-difference models that\ncombine features of synthetic and heterogeneity-robust difference-in-difference estimators1?\nThese are applicable to small as well as large samples, and can dominate synthetic\ncontrols1. Matching without replacement is used (lines 400-410) but there is no information if\nthe resulting standard errors are clustered at the matched pair level (as they should)2. It is\nalso unclear if matching is combined with difference-in-difference techniques when the\nsample size permits. Further, one of the studies, from which ex post statistical estimates for\nthe current manuscript are used, is still unpublished (line 402); without evaluating the\nstatistical analysis, it is unclear if the estimates therein are credible. Further: Do the studies\nemploying pixels as the unit of analysis (Table 1) use sampling approach that results in the\nsame probability of selection for each pixel? If not, how are the different sampling\nprobabilities accounted for in the analysis? Since the point of the manuscript is to highlight\nthe advantages of statistical methods, details about the statistical methods are crucial,\nespecially for studies not peer-reviewed/published yet.", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_4": "The manuscript entitled “Independent statistical approaches address overcrediting problems in REDD+” compares the avoided deforestation rates from ex ante assessments using the methods applied by a major carbon crediting agency with the ex post estimates of statistical approaches for 47 REDD+ projects in Latin America and Africa. The manuscript argues that the ex ante methodology tends to overstate the avoided deforestation by the REDD+ projects due to the selection of often incomparable reference groups.\nWhile the topic of revising REDD+ and quantifying additionality from conservation and climate change mitigation interventions is very timely and important, significant concerns remain about the manuscript regarding the statistical methods employed.\nStatistical and model considerations\n1. Without perusing the prior studies on which the manuscript is based, it is not clear what the ex post statistical methods employed are: For example, the manuscript mentions synthetic controls and one-to-one matching without replacement (Table 1), but does not provide key details. For example, why not synthetic difference-in-difference models that combine features of synthetic and heterogeneity-robust difference-in-difference estimators1? These are applicable to small as well as large samples, and can dominate synthetic controls1. Matching without replacement is used (lines 400-410) but there is no information if the resulting standard errors are clustered at the matched pair level (as they should)2. It is also unclear if matching is combined with difference-in-difference techniques when the sample size permits. Further, one of the studies, from which ex post statistical estimates for the current manuscript are used, is still unpublished (line 402); without evaluating the statistical analysis, it is unclear if the estimates therein are credible. Further: Do the studies employing pixels as the unit of analysis (Table 1) use sampling approach that results in the same probability of selection for each pixel? If not, how are the different sampling probabilities accounted for in the analysis? Since the point of the manuscript is to highlight the advantages of statistical methods, details about the statistical methods are crucial, especially for studies not peer-reviewed/published yet.\n2. Are the ex ante and ex post estimates statistically different? Fig. 2 and 3 suggest that there may be substantial overlap between the point estimates and the confidence intervals. That is, are the differences driven by a few outliers (e.g., Fig. 2)?\n2. The annual average rates of avoided deforestation are not the total deforestation/#years (line 417-419). Instead, some compounding is necessary. Consider switching to effect sizes instead3,4\n3. What is the formula for the standardized mean difference used in this study (line 497-499)? There are two versions in the literature (for example, see5,6).\n4. The principal components analysis is not traditionally used to assess degree of covariate overlap (e.g., lines 502-510); instead, studies report the degree of overlap for each covariate used in the analysis. Why was the principal components analysis used? Can you reference prior studies that advocate for this method over comparisons of the distributions of each covariate? The approach should be backed up with citations.\n5. What are the “likely confounders for the project” (line 500). The only covariates listed are those in Fig. 3d—elevation, slope, inaccessibility, baseline and lagged versions of forest cover and deforestation. Were any other covariates (e.g., ones that capture local economic processes used?) For example, the proximity to rivers is of huge importance in some locations as this is how households transport timber illegally (e.g., 7,8)—was that captured by the inaccessibility variable? No other (e.g., socio-economic) variables seem to have been included. Have the original studies assessed the sensitivity of the results to unobserved confounders especially when no difference-in-difference models are used?\n6. What is the “forest scarcity factor” (line 134). Is it possible it is driving a wedge between the ex ante and ex post estimates?\n7. What are the reference sites used in the ex post statistical analysis? For example, the manuscript mentions leakage as a concern (line 53) but it is not clear how that leakage is accounted for in any of the estimations without a proper economic model of change.\n9. The calculations of the overcrediting (lines 150-155) are very hard to follow. Please elaborate how you obtained the %.\n10. It is unclear what the test on lines 465-471 is trying to achieve. Is there a prior study that proposes or uses this method? Why is this method better than alternatives (e.g., something that imposes an underlying distribution)?\n11. It seems that most of the projects assessed are from a few countries (Table S1). Is it possible to look at heterogeneity between countries? That is, it is possible the overestimation of avoided deforestation is country-specific due to country-specific factors.\nData\n1. Table 1 and lines 514-534 summarize the data sources used to quantify deforestation. It appears that the ex post and ex ante analyses use different data sources: The methodology employed by Verra uses the Bespoke data, whereas the ex post statistical analyses use publicly available remote sensing products. What remains unclear is to what extent are these datasets comparable? For example, what is the resolution of the data? Do they handle missing values in the same way? What are the error/uncertainty estimates for each—for example, the TMS dataset spans all moist forests; the Hansen data (ref 40)-the whole globe. For this reason, it is likely the two datasets have different errors than a dataset that is produced and groundtruthed locally: https://www.globalforestwatch.org/blog/data-and-tools/how-accurate-is-accurate-enough-examining-the-glad-global-tree-cover-change-data-part-2/.\nDo the different datasets classify deforestation in the same way? The manuscript alludes that that the ex post statistical studies may not be consistent with each other. First, the Hansen and TMS datasets use different methods and classifications. The TMS dataset defines forest degradation as the temporary (less than 2.5 years) loss of forest cover9—is this a consistent definition employed in all of the land cover datasets used to assess avoided deforestation? The Hansen data used in the manuscript (line 129, reference 40) does not define deforestation, but rather “forest cover loss” events. The Hansen data does not define what “forest” is—prior studies suggest employing a 25% tree cover cutoff to define “forests”10 but the cut-off may be too low for the Amazon but ok for parts of Africa. Was any pre-processing of the Hansen data done in any of the statistical analyses? It also does not allow for easy comparisons before and after 2015 because of different remote sensing classification methodologies used: https://www.globalforestwatch.org/blog/data-and-tools/tree-cover-loss-satellite-data-trend-analysis/. Finally, some of the statistical estimates include the switch for undisturbed forest to degradation as “deforestation” (line 433, lines 425-435).\nIs a consistent definition of “deforestation” used in both the ex ante and ex post analyses? For example, is the creation of roads considered deforestation? For example, clearing forests for logging roads in logging concessions may not be considered as “deforestation” in the policy world. Forest cover loss associated with traditional slash-and-burn agriculture is also not considered “deforestation” in the policy world. Both temporary clearing and road building will be picked up by the Hansen data as “forest loss events” as well as any event that results in a forest cell having less than 50% tree cover in a given year.\nDetails on the data used are crucial and should be inserted in the manuscript and the appendix. The credibility of the main claim of the manuscript needs to be beefed up by comparing the data used for ex ante analyses with the data used for ex post analyses.\nWriting:\n1. Given the heavy reliance on other, mostly peer-reviewed and published studies, that also question the credibility of REDD+ avoided deforestation estimates, it is not immediately clear what the contributions of this manuscript are. Is it a novel comparison with the methods employed by Verra? Is it a meta-analysis? Further, the manuscript mentions that VERRA has introduced new consolidated methods to replace earlier ones (Line 96). Do these new methods still rely on ex-ante forecasts or include ex-post methods like those used in the statistical approaches? Incorporating new approaches could suggest that VERRA acknowledges the weaknesses in their earlier methodology and is working on improving their assessment tools.\n2. It is not clear why the additionality ratios can be smaller than 0 (lines 141-149)? Out of the 47 REDD+ projects, did any certified estimates indicate increased deforestation in the project areas? Since the additionality ratio is the statistical estimate divided by the certified estimate (444-446), a negative ratio would imply that either one of the estimates had the opposite sign. If so, why and where was that the case?\n3. What is a “binary process” (line 434)?\n4. While the Introduction is generally well-written and easy to follow, the rest of the text has frequent grammar and punctuation errors. There are also very long sentences with interrupting parentheses with additional text (e.g., lines 104-107).\n5. The first 3 paragraphs of the Discussion section summarize the main findings of the manuscript. It appears unnecessary: please consider shortening them and avoiding redundancies.\nWork cited:\n1. Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W. & Wager, S. Synthetic Difference-in-Differences. Am. Econ. Rev. 111, 4088–4118 (2021).\n2. Abadie, A. & Spiess, J. Robust Post-Matching Inference. J. Am. Stat. Assoc. 117, 983–995 (2022).\n3. Börner, J. et al. Emerging Evidence on the Effectiveness of Tropical Forest Conservation. PLOS ONE 11, e0159152 (2016).\n4. Samii, C., Lisiecki, M., Kulkarni, P., Paler, L. & Chavis, L. Effects of decentralized forest management (DFM) on deforestation and poverty in low‐ and middle‐income countries: a systematic review. Campbell Syst. Rev. 10, 1–88 (2014).\n5. Imbens, G. W. & Wooldridge, J. M. Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47, 5–86 (2009).\n6. Abadie, A. & Imbens, G. W. Bias-Corrected Matching Estimators for Average Treatment Effects. J. Bus. Econ. Stat. 29, 1–11 (2011).\n7. Ferretti-Gallon, K. & Busch, J. What Drives Deforestation and What Stops It? A Meta-Analysis of Spatially Explicit Econometric Studies. 44–44 (2014).\n8. Miteva, D. A., Loucks, C. & Pattanayak, S. K. Social and Environmental Impacts of Forest Management Certification in Indonesia. PLoS ONE 10, e0129675–e0129675 (2015).\n9. Vancutsem, C. et al. Long-term (1990–2019) monitoring of forest cover changes in the humid tropics. Sci. Adv. 7, eabe1603 (2021).\n10. Sexton, J. O. et al. Conservation policy and the measurement of forests. Nat. Clim. Change 6, 1–6 (2015).", "reviewer_5": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts."}, "author_response": "Response to reviewers\nNCOMMS-24-45991-T: Learning lessons from over-crediting to\nensure additionality in forest carbon credits\nWe would like to thank the reviewers for their extremely helpful and valued comments. Their\nsuggestions helped us make substantial changes to our manuscript which we believe have\ngreatly strengthened the analysis and improved our exposition of its necessity and\nrelevance.\nWe have made the following edits to the manuscript:\nReviewer 1 (Remarks to the Author):\nThank you for your thorough responses to my comments. I feel quite satisfied with the main\nmethodological responses and I think the paper has been improved by restructuring. I am\nnot quite at ease with the changes made in response to my comments on policy framing. I\nwill present these one by one. My new comments are italicized.\nComment 1\nOriginal Comment:\nSome constellation of the authors in this paper have already established that there is an\nissue with over certification on forest carbon markets. It would be useful to understand\nwhat the first result of this paper adds there that is new.\nOriginal Response:\nAs the reviewer says, a series of papers indicated over-crediting but there has been\nsubstantial push-back arguing that quasi-experimental methods are unreliable because\nthey generate too varied estimates and use remote sensing layers that detect too little\ndeforestation. For the first time, this paper examines these claims in detail by applying\nseveral different quasi-experimental methods to a large sample of projects and finds that:\n1. While quasi-experimental estimates do indeed vary, they do so far less than certified\nestimates\n2. The global remote sensing layers used by quasi-experimental methods do not\ndetect less deforestation than bespoke layers used in certification\n3. There is evidence that certified control areas were exposed to greater threat of\ndeforestation than projects\n4. There is also evidence to suggest flexibility in ex ante modelling caused elevated\nclaims of avoided deforestation\n5. These issues together resulted in over-crediting, underscoring the fundamental\nunsuitability of current certification methods compared with quasi-experimental methods.\nWe have made this much clearer in our revision through the introduction of figure 1,\nwhich summarises these questions. The manuscript is then structured so that the\nquestions are posed one-by-one and answered in the results.\nNew Comment:\nIn some sense, many of these points appear to be refinements of work that you have\nalready published. It seems to me that a dissection of these mechanisms is a more field-\nrelevant focus, rather than an analysis that brings us information that should be widely\ndistributed to the readership of Nature Communications.\nNew Response:\nWe disagree with this perspective. We believe our work is vital for two reasons. Firstly, the\nfact that there was substantial over-crediting in REDD+ credits is still not widely accepted by\nthe carbon crediting industry. This was clear throughout London Climate Action Week (June\n2025) in panel presentations and by the questions co-authors of this paper got when on\npanels. It is difficult for the necessary reform of the voluntary carbon market to happen until\nthis is accepted. Our paper goes way beyond existing published papers in the level of\nsynthesis offered to confirm, beyond doubt, that over-crediting is a real problem. However,\neven more significantly our paper lifts the lid to explore why over-crediting occurred and what\ncan be done to prevent its recurrence. In case we were not clear enough about why our\nstudy continues to be important despite the studies already published. We have now added\nthe following to the start of paragraph 2:\n“Without recognition of the scale of over-crediting, and an understanding of the underlying\nmechanisms, there is a risk that the same mistakes will be repeated in new generations of\ncredits, seriously undermining the impact of nature-finance.” (lines 57 - 59)\nComment 2\nOriginal comment:\nThe main constructive recommendation of the paper is that ex post statistical approaches be\nused in carbon crediting. As I was thinking about this, I got confused about what is actually\nbeing done versus what is proposed. Are the authors suggesting withholding payments until\nassessment of avoided deforestation can occur ex post? This seems like an impossible\nproposition, both at the broader institutional level and at the level of producer participation in\nthe event that projects involve individual land users foregoing deforestation, which they must\ndo for there to be additionality. Recent research suggests that successfully implementing\nPES in low-income settings may require ex ante payments to actually induce wide-scale\nparticipation (Izquierdo-Tort et al. 2024). It therefore seems more reasonable to suggest a\nway to consistently apply matching or the construction of synthetic controls ex ante using\ndata that is generally available and covariates that are known to be correlated with both\nselection into treatment and deforestation outcomes. I think that is what is being suggested\nhere. Counterfactual deforestation could then be predicted for reference areas, and then\nonce time has passed actual deforestation rates would be used in assessment. I think what I\nwould like to see is clarification of the policy proposition.\nOriginal response:\nWe appreciate the reviewer’s confusion here. Current practice is that the ex ante\ncounterfactual scenario is produced before the project is implemented but credits are issued\nonce an ex post assessment of project stock has been completed. The problem that we are\nraising is that the ex post assessment is limited to the project deforestation, while the\ncounterfactual is not revised to accommodate the most up-to-date information. The reviewer\nis of course right that these projects have to raise investment in order to finance the project,\nprior to the sale of credits, but we have argued in Swinfield et al. (2024) that the credits\nshould not be issued without also revising the counterfactual in the ex post assessment. We\nnow make this clear at several points.\nIn the introduction:\nThese ex ante counterfactual estimates were compared with measurements of deforestation\nin project areas to determine a certified metric of avoided deforestation with which to issue\ncredits.\nIn the discussion:\nA potential solution is to move to only issuing credits ex post based on assessments which\nintegrate new developments in quasi-experimental methods.\nAND\nFor example, nature-based restoration projects typically assume that all increases in carbon\nor biodiversity above a pre-project level are additional44. However, this is a strong\nassumption which should be validated by comparing project stocks with the average stocks\nin control areas selected using quasi-experimental methods45.\nNew comment:\nI think that you might add a sentence after your first discussion sentence that says\nsomething like, “This does not change the timing of credit issuance, but rather mandates\nthat the counterfactual be taken into account.”\nNew response:\nWe thank the reviewers for this suggestion, and have added the following sentence in the\ndiscussion:\nQuasi-experimental crediting would not alter the timing of credit issuance, as even current ex\nante counterfactual methods rely on ex post project measurements for quantifying\nadditionality, but it does introduce uncertainty in credit yields via the unpredictable nature of\nthe counterfactual. (lines 437 - 438)\nComment 3\nOriginal comment:\nTwo implementation issues should be recognized in the discussion. i. The first of these is\nthat perfect prediction of the location of future deforestation is impossible.\nDeforestation is a rare event statistically, and so targeting programs such that only future\ndeforested land is enrolled does not make sense, regardless of how much data you have\nwith which to make the prediction. Even trying to approach this kind of targeting in a REDD+\nprogram would require taking money away from actual incentives and funneling it into\nbureaucratic processes intended to support targeting, which would be wasteful. This means\nthat you have to pay for enrollment of larger areas of land than are likely to be deforested.\nOriginal response:\nWe agree with the reviewer that predicting the exact location of deforestation is very difficult.\nThere are inevitable trade-offs between costs of targeting and additionality. For that reason\nwe think that moves towards jurisdictional programmes will be positive but the inclusion of\nquasi-experimental approaches would be a further improvement. We now say this in the\ndiscussion:\nUnder JREDD+ methods such as VM0048 and ART TREES, counterfactual deforestation\nrates are set as the mean annual deforestation rate over a recent historic period (usually 5-6\nyears)12. Our results suggest that this more straightforward and inflexible approach to\nestimating a counterfactual could significantly reduce over-crediting risks (Fig. 5). Further\nJREDD+ is designed to enable integration into Article 6 of the Paris agreement allowing\njurisdictions to raise finance through the trade of International Transfers of Mitigation\nOutcomes37. Under these methodologies, independent data providers, with no direct\nfinancial stake in the number of credits issued produce the assessment of deforestation over\nthe historic period. This is a positive step which should reduce the perverse incentives which\nresulted in over-crediting in earlier project-level REDD+ methodologies. ]\nUnfortunately, the movement to JREDD+ does not entirely remove the risk of perverse\nincentives leading to over-crediting. The methods for JREDD+ assume that the drivers of\ndeforestation remain constant between the historic period and the project period. This\nassumption will often not hold as forest conservation or trade policies, major infrastructure\nprojects38 or extreme climatic events39 will all vary over time and can impact deforestation. In\nfact, ex post measures of deforestation in untreated jurisdictions have been shown to differ\nby as much as 100% from ex ante predictions based on historic averages33,40. If projects\nwere to enrol in JREDD+ schemes randomly without regard for the likely change in future\ndeforestation trends, this may not result in systematic bias.\nHowever, evidence tells us that proponents will opt into crediting schemes when it is to their\nadvantage41,42, with projects being preferentially developed in jurisdictions with declining\ndeforestation pressures. A potential solution is to move to only issuing credits ex post based\non assessments which integrate new developments in quasi-experimental methods. Ideally,\ndifferent quasi-experimental results should be considered, from methods such as the\nPermanent Additional Carbon Tonne (PACT) method used in our study43 or advances in\nsynthetic controls that require fewer assumptions26. Although quasi-experimental methods are still\nsusceptible to unobservable, time-varying confounders, the scale of these effects can be\ntested and reported31. Therefore, until there is clear consensus on how best to measure\nimpacts, a precautionary approach should be taken, with conservative estimates of avoided\ndeforestation produced to safeguard the claims being made19.\nNew comment:\nI am happy with this discussion, but this suggests that a very policy-relevant analysis would\ntry to evaluate the improvement in assessment by comparing the JREDD counterfactual to\nyour more precise approach.\nNew response:\nWe agree with the reviewer that this is a very policy relevant analysis and have therefore\nadded the following statement:\n“This can be tested by comparing quasi-experimental estimates of counterfactual outcomes\nwith those from JREDD+ methods.” (lines 427 - 428)\nComment 4\nOriginal comment:\nThis does not in itself undermine the principle of this paper. However, it is worth considering\nthat the global community often says that avoided deforestation and, more recently, forest\nrestoration, is “cheap.” The perceived cheapness of the intervention may be at least partly\ndriven by the fact that when we calculate REDD+ transfers per hectare using current\nestimates of deforestation, they are very low. However, we might find that if we use current\nper hectare rates spread over a much lower, precisely estimated amount of avoided\ndeforestation, we may not be able to induce any participation in these types of programs,\nbecause they will not sufficiently compensate producers for their opportunity cost. Environmental\nnon-profit organizations and rich country governments often engage in wishful thinking regarding\nthe true costs of nature-based solutions. It is important that the research community does not\nfall prey to this same mistake.\nOriginal response:\nWe agree with the reviewer that there is a widespread belief that REDD+ credits are the\ncheap option. We have now made clear in the discussion that our finding reveals that\nconserving nature is likely to cost more than expected:\nIndeed, despite inflated supply obscuring their real value, nature-based credits remain a\ncritical tool in the fight against climate collapse and mass biodiversity loss.\nNew comment:\nI think you could say this more clearly. There is a real risk that if someone decided to issue\ncredits based upon your methodology, there would not be sufficient funds to actually get\npeople to reduce deforestation. In many PES programs that I know of, the payment levels\nare set to accommodate the fact that policymakers know that they are not paying only for the\nadditional hectares. Rather, they are spreading the cost of compensating for the additionality\nacross a much larger forest area. Therefore, the cost per hectare of forest looks low, while\nthe cost per hectare of avoided deforestation is high. If credits are issued based upon more\nprecise avoided deforestation measures, this will lower payments to project participants, and\nwill risk undermining the entire process unless the price of credits is increased.\nNew response:\nWe agree that our previous changes did not go far enough in explicitly explaining the need\nfor commensurate increases in credit prices to account for a decrease in credit production.\nWe have now made this clear as follows:\n“Across REDD+, our analysis shows that although claims of avoided deforestation have\nbeen overestimated, the scheme has still delivered real reductions in deforestation. The\nchallenge now is to adjust expectations about the cost of protecting tropical forests and the\nprice of the resulting carbon credits54. Reducing project-level supply of carbon credits by\nusing more robust counterfactual estimates will severely restrict forest conservation finance\nunless there is a commensurate increase in the price of credits57.” (lines 453 - 457)\nReviewer 2 (Remarks to the Author):\nThank you for the opportunity to read the revised paper. I very much appreciate that\nmost of my comments have been addressed. Previously, my main concerns focused on the\nmanuscript’s contribution and the use of empirical language/concepts. The authors have\naddressed both aspects adequately.\nI only have a few minor editing comments and a small suggestion about the title.\nComment 1\nThe new title is slightly misleading: “Understanding the reasons” implies that the au-\nthors will analyze the political and economic reasons of over-crediting. E.g., certification\ninstitutions have an economic incentive to certify higher avoided deforestation figures in\norder to please the client. This manuscript analyses the mechanisms of over-crediting.\nI suggest reformulating the title to “Understanding the mechanisms for over-crediting\nin REDD+”.\nResponse:\nWe agree with the reviewer that their suggestion is better and have updated the title\naccordingly.\nComment 2\nOriginal comment:\n𝐴𝑣𝑜𝑖𝑑𝑒𝑑 𝑑𝑒𝑓𝑜𝑟𝑒𝑠𝑡𝑎𝑡𝑖𝑜𝑛 𝑐𝑟𝑒𝑑𝑖𝑡𝑒𝑑 𝑏𝑦 𝑉𝐸𝑅𝑅𝐴\nFigure 1b shows there ratio . For overestimation,\n𝐴𝑣𝑜𝑖𝑑𝑒𝑑 𝑑𝑒𝑓𝑜𝑟𝑒𝑠𝑡𝑎𝑡𝑖𝑜𝑛 𝑢𝑠𝑖𝑛𝑔 𝑞𝑢𝑎𝑠𝑖−𝑒𝑥𝑝𝑒𝑟𝑖𝑚𝑒𝑛𝑡𝑎𝑙 𝑚𝑒𝑡ℎ𝑜𝑑𝑠\nthe ratio is < 1, and for underestimation, the ratio is > 1. I suggest flipping the definition of the\nratio so that values > 1 reflect overestimation.\nOriginal response:\nWe agree with the reviewer that this is confusing however because quasi-experimental\nmethods tend to estimate values close to, at or even below zero, whereas certifying methods\nare all positive, the proposed approach would make it impossible to present all the projects\nas they would have very large or infinite value. Logging the data is also problematic due to\nthe negative values. Therefore we have opted to retain the same presentation. However, we\nnow presented over-crediting as suggested in the results:\n“The mean additionality ratio (calculated as the mean quasi-experimental estimate of\navoided deforestation / certified estimate, averaged across the 44 projects claiming to have\navoided deforestation) of 0.27 (Fig. 2b) was significantly < 0 (one-tailed Mann-\nWhitney U against a group of 0s, U = 1628, p < 0.001). This suggests that on average\nprojects were additional but had over-credited by a factor of 3.8.”\nNew comment:\nYou could use an inverse asymptotic sine transformation. It has similar properties to the\nlog-transformation, and it is defined for negative values. Imagine I use this figure in a lecture:\nIt is complicated to explain to students that a point with a value of 0.16 on the y-axis means\novercrediting, and they should calculate 1/0.16=6.25, to understand that VERA calculated a\n625% higher level of avoided deforestation than the quasi-experimental method.\nNew response:\nWe thank the reviewer for their comment and have substantially reworked Figure 2. We\nexplored incorporating the suggested inverse sine transformation in the graph. This process\nprompted a re-examination of the underlying data, leading to the following adjustments in\ninterpretation.\nFirst, we reassessed the additionality ratio used in our initial analysis. Previously, we only\ncalculated this ratio by dividing the mean quasi-experimental estimate by the certified\nestimate, a statistic similar to the “OAR” used by Probst et al. (2024). However, as the\nreviewer noted, this resulted in a confusing presentation in Figure 2, where smaller values\nindicated greater over-crediting. We have now inverted the ratio for the figure and in the\nsupporting text as suggested. This adjustment, however, introduced an unintended\ncomplication: projects with negative quasi-experimental estimates remained below the zero\nline, with values closer to zero suggesting less over-crediting. Consequently, the\ninterpretation of the scales became inconsistent depending on the sign of the value.\nMoreover, confidence intervals crossing the y = 0 line ceased to convey meaningful\ninformation.\nTo address this, we made the following revisions to the ratios plot in Figure 2b. We removed\nconfidence intervals and highlighted negative values in red, which we positioned at the top of\nthe plot. We also updated our statistical analysis to use bootstrapping to derive confidence\nintervals for the means. To this end we introduced an additional test for over-crediting,\nevaluating whether projects would remain over-credited even when using the most generous\nquasi-experimental estimate within their respective 95% confidence interval ranges.\nIn the process of exploring the negative values we also noticed that these had been handled\ninconsistency across the quasi-experimental methods. Projects assessed to have negative\nadditionality or no significant additionality were assigned a value of zero by Guizar et al.\n(2024) and West et al. (2023). In our revision we now include these values as negatives,\nexcept in the calculation of the OAR. We also identified that we had been working with a\nversion of the West et al. (2023) data from their pro-print but that was updated on\npublication. Finally, Guizar et al. (2024) has now been updated (it was an early pre-print but\nis now submitted to a journal and includes more sophisticated analysis). We have therefore\nintegrated the most up to date data into our analysis. Together, these changes have\ninfluenced the global mean additionality ratio and our presentation of it. These changes are\ndescribed in full below.\nFrom Results:\n“The majority of projects (35 of 43) did reduce deforestation (Figure 2). For the projects with\nmultiple quasi-experimental estimates, 15 (40% of the 37 projects with calculable confidence\nintervals) showed statistically significant additionality at the 95% confidence level. Just eight\nprojects (shown in red in Figure 2) experienced more deforestation than their\nquasi-experimental controls.\nDespite most projects reducing deforestation, over-crediting was widespread with certified\nestimates of avoided deforestation significantly greater than mean quasi-experimental\nestimates (one-tailed paired Wilcoxon signed-rank test; V = 925, p < 0.001). Over-crediting\nwas statistically significant at the 95% confidence level for 29 projects (79% of the 37 with\ncalculable confidence intervals; Fig. 2a). Consequently, the mean additionality ratio was 4.3\n(95% bootstrapped CI: 2.9 to 7.3), indicating it offered about a fifth of the avoided\ndeforestation reported by certified estimates. Across all the credits assessed, the global\nadditionality ratio (the certified estimate divided by the mean quasi-experimental estimate)\nwas 9.3 (95% bootstrapped CI: 4.3 to 23.6), indicating that a credit bought at random would\noffer only a tenth of the avoided deforestation claimed (Fig. 2b). However, it is important to\nnote that over-crediting was not universal: the three projects from Madagascar, for example,\nhave a mean additionality ratio of 0.9 meaning the certified and quasi-experimental\nestimates were similar (S8).” (lines 451 - 469)\nFigure. 2: Estimates of avoided deforestation from quasi-experimental approaches\nwere frequently lower than certified estimates, indicating widespread over-crediting.\n(a) REDD+ projects’ certified estimates of avoided deforestation against mean estimates\nfrom the five quasi-experimental methods, error bars show 95% confidence intervals. (b) The\nadditionality ratio (the certified estimate divided by the mean quasi-experimental estimate)\nagainst certified estimates of avoided deforestation, the dashed grey line shows the global\nadditionality ratio. In both panels y-axes are on a log scale. Dashed red lines show where\ncertified and quasi-experimental estimates of avoided deforestation were equal. Points\nabove and below the line were assessed to have over-credited and under-credited\nrespectively. Darker colours indicate projects that were assessed by a greater number of\nquasi-experimental approaches (to a maximum of four because there was no overlap in the\nprojects assessed by the two West et al. studies). The additionality ratio cannot be\ncalculated for the eight red points as they experienced more deforestation than predicted by\ntheir quasi-experimental controls.\nAnd from Methods:\n“For each of the 43 projects, we compiled the certified estimates of annualised avoided\ndeforestation in hectares, along with all available quasi-experimental estimates. For the 37\nprojects with more than one quasi-experimental estimate, we calculated the mean, standard\ndeviation, coefficient of variation, standard error, and 95% confidence intervals. For the\nremaining six projects, which had only a single quasi-experimental estimate, this value was\ntaken to be the mean.\nWe determined how many projects had mean quasi-experimental estimates below the\ncertified estimate, as well as how many had upper 95% confidence intervals below the\ncertified estimate. To test whether certified estimates were significantly greater than mean\nquasi-experimental estimates, we applied a one-tailed Wilcoxon signed-ranks test, due to\nthe non-normality of the data. We also evaluated how many projects exhibited\nquasi-experimental avoided deforestation estimates that were significantly greater than zero,\nusing the lower bound of the 95% confidence intervals.\nWe also assessed the over-crediting as the ratio of the certified to the mean\nquasi-experimental estimate of avoided deforestation. We refer to this as the additionality\nratio, which expresses the number of times more credits produced through certification than\nsuggested by quasi-experimental estimates. The mean additionality ratio was calculated for\nall projects that issued credits that also had positive mean quasi-experimental estimates.\nProjects with negative mean quasi-experimental estimates were excluded. The global\nadditionality ratio, reflecting how many certified avoided deforestation units likely correspond\nto one quasi-experimentally validated unit across the REDD+ portfolio, we divided the total\ncertified avoided deforestation by the total of the mean quasi-experimental estimates\n(including the negative values). Both the mean additionality ratio and the global additionality\nratio were bootstrapped using 10,000 resamples with replacement to generate 95%\nconfidence intervals.” (lines 632 - 658)\nComment 3\nLine 51. ”Unfortunately, over-crediting by REDD+ Projects is still not widely accepted.” It is\nunclear who is not accepting and why everybody should accept. I recommend using a\nmilder, scientifically impartial language.\nResponse:\nWe are grateful for this comment regarding more impartial language and have updated the\nstart of the specific paragraph to the following:\n“Without recognition of the scale of over-crediting, and an understanding of the underlying\nmechanisms, there is a risk that the same mistakes will be repeated in new generations of\ncredits, seriously undermining the impact of nature-finance.” (lines 57 - 59)\nHopefully this new text removes a sense of blame while simultaneously conveying the\nimportance of our study as requested by reviewer 1.\nComment 4\nThe introduction should have a brief paragraph summarizing the results.\nResponse:\nWe appreciate the request for a paragraph highlighting the scientific contribution. In order to\nmaintain the flow of the introduction as a whole, we have attempted to achieve this in a\nsingle paragraph that summarises both our scientific contribution and our results. As follows:\n“Our study contributes to the literature examining how to infer causal effects from\nobservational data27,28, especially in conservation where randomised control trials are rare or\nimpossible19,20. Our results show clearly that despite variation in estimates from\nquasi-experimental methods, there is consistent evidence of over-crediting. This result\ncannot be explained by differences between the remote sensing layers used by\nquasi-experimental and certified assessments. Instead, we find that over-crediting was\nprimarily driven by the selection of control areas with greater exposure to deforestation than\nprojects and through unrealistic modelling of future deforestation rates.” (lines 125 to 133)\nComment 5\nThe introduction should have a paragraph highlighting the scientific contribution: ”We\ncontribute to the literature on ... ”\nResponse:\nSee response to comment 4.\nComment 6\nAbbreviations are used but not explained in the main text (as far as I see). E.g. ACC,\nQEM.\nResponse:\nWe thank the reviewer for catching this! These have now been defined the first time they are\nused in both the main text and the supplementary materials.\nComment 7\nLine 430. The concept of ”dynamic baselines” is recommended, but not explained at any\nPoint.\nResponse:\nWe are grateful to the reviewer for highlighting this. We have now removed the reference to\ndynamic baselines and use the following text instead:\n“Formalising the use of quasi-experimental methods with ex post counterfactuals to account\nfor dynamic changes in deforestation risk, would considerably reduce the risk of\nover-crediting in future53,54.” (lines 463 - 464)\nReviewer #3 (Remarks to the Author):\nThere is no reviewer 3\nReviewer #4 (Remarks to the Author):\nThe paper has the potential to make a significant contribution. However, while the\nmanuscript has improved significantly in terms of the analysis and writing, it is still not ready\nfor publication, in my opinion. There are 2 main remaining challenges: The quality of the\nwriting and the lack of proper description and justification of statistical techniques.\nComment 1\nWriting:\nWhile the contributions of the manuscript and the description of the methods are much\nclearer, grammatical errors and overly complicated sentences still abound in the text.\nResponse:\nWe are very sorry this reviewer felt that this is the case. We have spotted and corrected a\nfew grammatical errors and do not believe there are others. It is always challenging to get\nthe balance right in highly-technical writing: what one person considers an overly\ncomplicated sentence may be considered by others as including crucial detail to allow a\njudgement to be made on the methods. We have worked very hard in this revision to make\nthe language as clear as possible. Specifically we have ensured each paragraph starts with\na clear topic sentence and have shortened many of the longer sentences. We also asked\nindependent colleagues to read the paper and flag parts which they find difficult and have\nrevised iteratively.\nComment 2\nDoes deforestation in the bespoke layer capture forest loss due to road construction? Do the\nVerra approaches consider reforestation as a way to offset forest loss? It is important to\nclarify what the original certification mentions consider “deforestation” as some forest loss is\nnot deforestation.\nResponse:\nAll VERRA methods only assess deforestation using area-based forest vs non-forest\nclassifications commonly between two points in time (although more time points can be\nused). All forms of forest loss within project areas are meant to be recorded in monitoring\ndocuments, including from road construction. However, short-term disturbance events may\nbe missed, including from regrowth. We explain this in the discussion as follows:\n… we found that the ACC detected similar or higher deforestation rates in project areas than\ncertified estimates using bespoke remote sensing layers (Fig. 3 and S4). This may well have\noccurred because the ACC is a multi-temporal approach that considers all disturbance\nevents occurring since the start of the time series5, whereas bespoke approaches typically\nonly detect non-forest classes at two points in time, which may miss short-term\ndisturbances36. (lines 357 - 358)\nComment 3\nWhat is the default setting for GDALDEM for slope calculations? Is it degrees? (line 514)\nResponse:\nWe thank the reviewer for highlighting the lack of clarity on our part regarding the slope\nvariable. We have updated the section to now state the following:\n“From the elevation layer we calculated slope using GDALDEM (v3.8.4) with default settings.\nThis produced a slope variable consisting of integer values representing the mean slope in\neach raster cell ranging from 0-90°.” (lines 563 - 566)\nComment 4\nI suggest including citations of the actual datasets used in the analyses (e.g. lines 503-506)\nand not referencing just the secondary studies that use these datasets\nResponse:\nWe thank the reviewer for spotting this omission. We have now included references to\nMapbiomas and Global Forest Change. With the text updated as follows:\n“Deforestation was assessed using publicly available remote sensing products for tracking\nchanges in forest cover, specifically the corrected Mapbiomas dataset used by West et al.\n(2020)6, the Global Forest Change (GFC) dataset used by West et al. (2023)8, and the\nTropical Moist Forest Annual Change Collection (ACC)5 used by the Guizar-Coutiño et al.7,39\nand PACT analyses. GFC reports deforestation as the year of gross forest loss.” (lines 520 -\n524)\nComment 5\nI don’t know what the text on lines 625, 279-280, and 59-60 mean. For example, what are\n“areas used for spatial modeling”?\nResponse:\nWe have now edited the text to make our meaning clear.\nInstead of:\nWe chose to focus on control areas used to measure deforestation rates and not areas used\nfor spatial modelling. (line 625)\nWe now say:\nFor certified control areas, we focused on the ‘Reference Regions for Deforestation’ (RDD)\nused by certified methods as these were used to measure the rate of deforestation, rather\nthan the ‘Reference Regions for Location’, which were used to measure how much of the\ndeforestation was expected to happen in the project. (lines 703 - 705)\nInstead of:\nYet the deforestation rates used in certified ex-ante modelling were produced from bespoke\nmeasurements of deforestation in certified control areas\nWe now say:\n“This was difficult because we did not have access to the certified deforestation rates in\ncontrol areas, so instead inferred them by multiplying the ACC control area deforestation\nrates by 0.58 (the median ratio of the bespoke to ACC rates in the 17 projects in this\nanalysis).” (lines 291 - 294)\nInstead of:\n(1) their results lack consistency due to variations in model specification17, and (2) the\nglobally available, remotely sensed forest cover datasets they rely on detect less\ndeforestation than bespoke layers18.\nWe now say:\n“The criticisms levelled at the independent studies are based on two key arguments: (1) their\nestimates of avoided deforestation are inconsistent due to variation in how analyses were\nconducted17, and (2) the publicly available forest cover datasets used by these studies detect\nless deforestation than the bespoke remote sensing classifications used by projects18.” (lines\n61 - 65)\nComment 6\nWhy introduce the synthetic control method in lines 90-93? It seems out of place. Also, as\noutlined in my previous report, newer methods like synthetic difference-in-difference may\noutperform synthetic controls (Arkhangelskiy et al. 2021).\nResponse:\nWe agree with the reviewer that the statement on synthetic controls was a little out of place\nin isolation. We have now changed the text to improve our point that there are various\nquasi-experimental approaches available to, including adding a reference to Arkhangelskiy\net al. 2021:\n“A related approach is the synthetic control method that can replicate pre-project trends\nthrough the selection and weighting of untreated units8,23–25. This has been further refined in\nthe form of synthetic difference-in-differences to more effectively account for time-varying\nand unobserved confounders26.” (lines 94 - 98)\nComment 7\nWhat is bootstrapped in the PACT method (Table 1 and line 494)? How is the bootstrap\ndone?\nResponse:\nWe have substantially reworked our explanation of the PACT methodology to add clarity. The\nreference to bootstrapping was due to PACT using 100 unique iterations of matching sets\nfrom which to calculate the mean additionality across. This is now explained with the\nfollowing text:\n“This process was repeated to produce 100 matched project-control sets, each set is thus a\nlength equivalent to 10% of the project units set, and thus the total sample across all sets is\nten times larger than the project units set.”\nEx post deforestation was measured for the project and control units by calculating changes\nin the undisturbed class since the start of the project. To calculate the area of undisturbed\nforest at each time point, we calculated the proportion of all units in the undisturbed class\nbefore multiplying by the project area. This was done across all 100 matched sets to\nproduce estimates of project forest cover and counterfactual forest cover.\nAveraging across these sets, we produced annualised mean forest cover in hectares. By\nsubtracting the counterfactual forest cover by the project forest cover, annual cumulative\nadditionality/avoided deforestation figures were computed. Total avoided deforestation\nduring the project period was measured at the final year of the project’s most recent\nmonitoring period (as the additionality figures are cumulative).” (lines 613 - 627)\nComment 8\nI would clarify that Fig 1a is similar to a quantile-quantile plot for log-transformed data.\nCurrently, that figure and its legend are very difficult to read. I would clarify in the axes labels\nthat the outcome is log-transformed.\nResponse:\nIn response to Reviewer 2 Comment 2, we have changed this figure and updated its\nanalysis substantially. We believe that these changes have considerably improved the clarity\nof the figure. With regards to the scales on the y-axes whilst the units are not log\ntransformed, the breaks on the axes are, we have made the following clarification in the\ndescription:\n“In both panels the y-axis is on a log scale” (line 184)\nComment 9\nReword that you are showing 95% Confidence Intervals instead of “confidence intervals\nshow 1.96 standard errors” (line 174-175)\nResponse:\nWe have made the suggested alteration.\nComment 10\nThe presence and location of spillovers depends on the extent of markets (e.g., see Pfaff &\nRobalino 2017 for a review and Miteva et al. 2017 for an example of a specific mechanism of\nspillovers as a function of markets). The text mentions that some of the projects are large\n(lines 526-527). Given the lack of underlying theory of change, the manuscript cannot model\nwhere spillovers are likely to occur. However, this point should be acknowledged as a\nlimitation in the discussion.\nResponse:\nWe thank the reviewer for highlighting this and added the following to our discussion section\non limitations:\n“Finally, REDD+ projects could cause spillover effects within their vicinity, which would affect\nthe deforestation observed within control units41–43. While some studies have attempted to\novercome this issue by only matching to areas greater than some distance from project\nboundaries44, this strategy may be insufficient because of the complex and hard to predict\nway in which spillovers can occur.” (lines 406 - 411)\nComment 11\nReword Q4 (line 256)\nResponse:\nWe agree with the reviewer that the original wording of the question title was unclear, and\nhave reworded it as follows:\n“Q4. After isolating the effects of forest cover layer and control area selection, how much of\nover-crediting can be explained by ex ante modelling?” (lines 270 - 271)\nComment 12\nI am not sure what lines 285-286 and lines 343-344 mean—a critique of a specific ex ante\nmethod? A general critique of ex ante methods?\nResponse:\nWe agree that this wasn’t completely clear. We have completely rewritten the results section\nfor Q4 to explain more clearly our finding that a large proportion of over-crediting is likely due\nto the various ex ante modelling approaches taken:\nFinally, having adjusted for control area selection and the use of different deforestation rate\nestimates, we attributed the remaining 78% of over-crediting to the various ex ante modelling\napproaches used in certified assessments.\nIn the discussion, instead of:\nPredicting an ex ante counterfactual scenario is difficult. However, flexibility in the modelling\napproaches used to estimate this counterfactual likely played a major role in over-crediting.\nWe now say:\n“Projecting a counterfactual scenario of deforestation, likely to occur in the absence of a\nREDD+ project ex ante, is inevitably fraught with uncertainties and appears to be a key\ndriver of over-crediting. We suggest this is the predominant mechanism resulting in\nover-crediting (Fig. 5), after accounting for the effect of control area selection and different\ndeforestation rate estimates.” (lines 371 - 375)\nComment 13\nFig 5 is very hard to read: Why are there bars that don’t correspond to X axis labels?\nResponse:\nWe thank the reviewer for pushing us to look at this figure again. We have simplified the plot\nand labelled it more clearly. This involved removing the stacked bars, which could be seen\nas a source of confusion.\nFigure 5. Isolating the contribution of different possible mechanisms by which\nover-crediting occurred. The avoided deforestation rates from the subset of 17 projects for\nwhich this analysis is possible projects produced using different combinations of\nquasi-experimental and certified estimates in project and control areas. The percentage of\nover-crediting attributable to each possible mechanism is revealed through sequential\ncomparisons of the avoided deforestation estimated by the different combinations. ACC\nrefers to the tropical moist forest annual change collection used by quasi-experimental (QE)\nPACT method, and bespoke* indicates the values inferred by multiplying the ACC values by\n0.58 (the median ratio of the bespoke to ACC rates in the projects). Error bars show\nmedians and interquartile ranges with significance indicated by ** p < 0.01.\nComment 14\nI am not following the point on lines 334-335. If something is not statistically significant, the\ncoefficient is 0. Unless the point the text is trying to make is that there is a lot of\nheterogeneity (hence the wide confidence intervals)?\nResponse:\nWe have made the changed the text, to clarify what we mean, so that instead of:\n“Though this relationship lacked statistical significance (Fig. 4), when isolated from other\nexplanations, elevated ACC deforestation within certified control areas contributed around\n57% to total over-crediting”\nWe now state:\nWhen measured using the ACC, certified control areas had higher rates of deforestation\nthan their quasi-experimental counterparts (Fig.4), which likely contributed substantially to\nover-crediting (Fig. 5). (lines 367 - 369)\nComment 15\nI would emphasize that the credibility of quasi-experimental designs depends on, first and\nforemost, the theory of change that helps us formulate hypotheses, select relevant to a\nspecific context covariates, select a proper estimation approach, and explain observed\npatterns (lines 361-365). Also, the robustness of quasi-experimental designs can be\nassessed via sensitivity analyses (lines 363-364).\nResponse:\nThis is an important point, we have updated the manuscript to incorporate it. First, we now\nexplicitly mention a theory of change in the introduction:\n“All methods required control areas to be similar to project areas in terms of drivers of\ndeforestation in their landscapes, which were selected according to their reported ability to\npredict deforestation alongside a landscape-level theory of change.” (lines 76 - 78)\nWe have also explicitly described our causal model of land use change in our methods\nsection:\n“Secondly, layers were compiled to select control units analogous to project units using a\nmethod informed by a causal model of land use change. This was developed from empirical\nevidence of the effect of local policies and regulations, economic pressures, historical\nchanges in land cover, and environmental conditions on deforestation rates and\nconservation efforts.” (lines 556 - 559)\nRegarding the sensitivity analysis, we have updated the limitations paragraph in the\ndiscussion section to both reflect how new advancements in the field of QEMs have\nproduced results relating to their performance under sensitivity analysis and in a placebo\nlandscape setting:\n“However, simulations of landscapes where the rates of avoided deforestation are known or\nmanipulated can be used to demonstrate the reliability of these approaches37. For example,\nnew evidence demonstrates that the PACT method can predict 83% of the variance in\ntropical forest deforestation rates across such a set of untreated ‘placebo’ landscapes38.\nSecondly, a core assumption of some of the quasi-experimental methods used is that all\nsignificant confounders (covariates associated both with where REDD+ projects are\nimplemented and with deforestation) are controlled for20. Given that quasi-experimental\nstudies apply the same methods across multiple projects, they are not accounting for local\ndrivers of deforestation which may also be associated with the placement of REDD+\nprojects. However, Guizar-Coutiño et. al. (2025) explore the potential influence of such\nhidden confounders and reveal they would need to have an effect size multiple times larger\nthan the most impactful observed covariates to undermine this finding39.” (lines 388 - 400)\nComment 16\nWhat is a “modified method” (line 461)?\nResponse:\nWe have now addressed this lack of clarity by reformatting the section as follows:\n“We also included results from a study under review by Guizar-Coutiño et al.39, which\nimplemented a novel framework to reanalyse the set of projects examined in Guizar-Coutiño\net al.7 study. In this newer study 7.1 ha circular sample regions were used instead of 0.09 ha\npixels to characterise forest loss observations. The causal impact of REDD+ projects on\ndeforestation was examined by adjusting for observed confounders and pre-treatment\ndeforestation trends, using a two-stage matching approach, commencing with propensity\nscore matching via random forests, followed by propensity score subclassification on the\nmatched datasets. The average treatment effect of the treated was examined using multiple\nmodel specifications and sensitivity to unobserved confounders was assessed using the\nsensemakr framework29. The avoided deforestation estimates used in our study correspond\nto the doubly robust model specification reported in their study.” (lines 496 - 508)\nComment 17\nLines 494-496 & 627-635 need a citation to back up the approaches\nResponse:\nWe agree that references are necessary to justify the use of the bootstrapping approach\nused by PACT and the georeferencing approach used for digitising the certified control\nareas. For the former we reference the original Efron and Tibshirani (1985) paper as well as\nAustin and Small (2014), which talks about the use bootstrapping in propensity score\nmatching:\n“The PACT method used a bootstrapped one-to-one pixel (0.09 ha) matching approach to\nproduce paired project and control units from which to estimate mean project and\ncounterfactual outcomes64,65.” (lines 541 - 543)\nFor the approach used in digitising the certified control areas we reference the best practice\narticle on digitising maps produced by ESRI:\n“Certified control area polygons were not publicly available as shapefiles so were digitised by\ngeoreferencing maps available in project design documents (from the Verra Registry) either\nby tracing the polygons by hand in QGIS (v3.26.3) or by a colour thresholding and\nautomated polygonisation procedure in R, equivalent to the process described by the\nEnvironmental Systems Research Institute70.” (lines 707 - 711)\nComment 18\nWhich correlation was used --Spearman vs. Pearson (line 566)?\nResponse:\nWe thank the reviewer for highlighting this discrepancy. In our initial analysis of Q1, we used\na Pearson correlation to test if the ACC rate of deforestation within project areas was aligned\nwith the Certified rate. However, this result has now been omitted from the manuscript in the\nformatted Q1 section.\nComment 19\nSince F0 is a share, lines 595-596 do not make sense as worded.\nResponse:\nWe have modified the text to provide more clarity.\n“To determine F we took the proportion of the ACC pixels that were in the undisturbed class\n0\nfor the yearly layer closest to the project start date and multiplied this by the area of the\nproject, measured from the project polygon. Thus F was the total area of undisturbed forest\n0\nin hectares at the start of the project.” (lines 672 - 675)\nComment 20\nThe manuscript supports the range for, what is refered to as “standardized mean difference”\nby citing a recent paper in Conservation Biology (Ref. 22). Given the long history of the\nstatistic in other disciplines, I would reference studies by statisticians or econometricians\nwho propose that cut-off as a rule of thumb (e.g. Abadie & Imbens, 2011; Imbens et al. 2009\nbut also earlier studies). Also, line 551 references an article in the Guardian and not a\nstatistics study.\nResponse:\nThanks. We agree that there are more appropriate references. We now refer to Rosenbaum\nand Rubins 1985 paper, as well as a Stuart’s 2010 review of matching methods for causal\ninference. We have done this on each mention, which corrects the erroneous reference to\nthe Guardian article.\nComment 21\nIt is still not clear to me why the main text of the manuscript uses only “undisturbed” forests\nin the Vancutsem et al (2021) data and not all forest including forest regrowth (e.g.,\nparagraph starting on line 593). Only a note in passing is made in the text that some analysis\nusing all 4 forest categories can be found in the appendix (although I could not find\nthat--Section S4 uses just deforestation and degradation). The main text does not clarify\nwhat the comparison indicates (lines 616-617).\nResponse:\nDeciding how best to treat the data layers is, to some extent, a judgement call. We chose not\nto include degraded land in the definition of forest cover (instead focusing solely on\nundisturbed areas), as we were influenced by research from Holcomb et al 2024., which\nfinds that degradation within the ACC corresponds with a significant loss of biomass which is\nrelevant to REDD+. Expanding this analysis to regrowth we find that it has mean biomass\nlower than the degraded class. We have now added this analysis to the supplementary\nmaterial in the form of S11 and further clarify our decision in the text.\nWe have now updated the text to state the following:\n“PPACT considered degradation of the undisturbed class alongside deforestation, because\nthis has been shown to result in biomass reduction of >50% within 12 months61. Our own\nanalysis of the above ground biomass densities for different ACC forest classes for\nnumerous project areas also showed that regrowth landscapes were also far below the\nbiomass of their undisturbed counterparts (see S11).” (lines 528 - 532)\nFurthermore we have expanded the analysis presented in S4, which now also shows how\nincluding the regrowth class as forest cover lowers the ACC measured deforestation rates.\nThus, because we classify both degradation and regrowth as forms of forest loss, our\nPACT-derived estimates of avoided deforestation are likely higher. This is due to the fact that\nour approach detects more deforestation in the counterfactuals, which has a greater impact\non determining the magnitude of additionality (i.e., avoided deforestation) than\nmeasurements within the project area itself.\nComment 22\nLines 447-449 mention that whenever deforestation rates were not available, they were\n“totaled over the evaluation period”. However, later the methods present a compound\nformula for calculating the average deforestation over time (Eq. 2.2)—why were 2 different\nmethods used?\nResponse:\nWe thank the reviewer for catching this mistake. In a prior version of the manuscript, the\napproach we used to extract annual deforestation rates was the mean of the year on year\nchange. However, for some projects this information was absent, meaning that instead a\ncompounded rate over the evaluation period, was used. Now we use a compounded rate for\nall projects and use cases, as detailed in Eq. 2.2. We have now modified the prior reference\nto the following so that it is less misleading.\n“Where avoided deforestation was not reported, it was calculated by subtracting project\ndeforestation from counterfactual deforestation. These figures were then combined with the\nduring of the evaluation period (which had a mean length of 6.4 years) to produce\nannualised compound avoided deforestation rates (Eq. 2.2).” (lines 480 - 484)\nComment 23\nStatistics:\nThe credibility of counterfactual analyses depends on the underlying theory of change.\nOmitting key covariates (e.g., proximity to rivers, markets which may not be necessarily\ncaptured by the travel time to health centers etc) may drastically change the control units\nand, thus, results. For this reason, it is necessary to justify the baseline covariates used in\nthe quasi-experimental design with some theory of change. The current version of the\nmanuscript mentions only a handful, some of which may not refer to the baseline (period\nprior to the intervention) (lines 503-524).\nResponse:\nWe strongly agree with the reviewer’s comment. It is, of course, essential that the most\nappropriate covariates are included in order to select the most appropriate controls.\nNevertheless, we have selected the covariates carefully to ensure they do properly reflect\nthe underlying causal mechanisms that result in deforestation. We think it is disputable\nwhether proximity to rivers is a better covariate that time to healthcare layer. This is because\nthe travel to healthcare layer integrates over all known travel routes, including rivers, as well\nas terrain, land cover, and road quality to assess the travel time. We completely accept that\nhealthcare centres may well not be the most appropriate proxy for economic pressures and\nthat it would be better to use destinations that are economically important, such as\nprocessing facilities, distribution centres and markets, but, as far as we are aware, these are\nnot generally available as a single accessibility layer. Economic pressure is also somewhat\ncaptured using our proportional cover layers which describe the trajectory of land use\nchange in the surrounding area through time.\nWe now better explain our choice of covariates in the methods and acknowledge their\nlimitations:\nSecondly, layers were compiled to select control units analogous to project units using a\nmethod informed by a causal model of land use change. This was developed from\ncovariates that encode the effect of local policies and regulations, economic pressures,\nhistorical changes in land cover, and environmental conditions on deforestation rates and\nconservation efforts. We used the following layers: International country borders produced\nby OpenStreetMap64 to encompass national regulatory limits, Ecoregions provided by\nResolve65 to ensure biotic equivalence, and elevation provided by NASA’s Shuttle Radar\nTopography Mission66 as a proxy for abiotic conditions. From the elevation layer we\ncalculated slope using GDALDEM (v3.8.4) with default settings, which is an important proxy\nfor accessibility and suitability for economic activities (this produced a slope variable\nconsisting of integer values representing the mean slope in each raster cell ranging from\n0-90°). Motorised accessibility to healthcare in 2019 provided by the Malaria Atlas Project67\nwas used as a proxy for market exposure/economic pressure. This accessibility layer\nintegrates all known travel routes, including rivers, as well as terrain, land cover, and road\nquality to assess the travel time to population centres large enough to have a healthcare\nfacility. Finally, we also computed the proportional cover of undisturbed and deforested\nclasses from each year of the ACC time-series (1990-2021) to assess land use change\ntrajectories through time in the surrounding area. Proportional cover was calculated by\ncounting the number of pixels in each class and dividing by the total number of pixels within\nthe 1 km radius neighbourhood around each pixel. It is important to note that accurate data\nregarding land value and accessibility to markets is not universally available, particular in\nfrontier landscapes. As such we use spatial proxies such as the proportional cover and\nmotorised accessibility to healthcare, recognising that these leave space for unobserved\nconfounders. (lines 556 - 588)\nWe also agree that missing confounders could affect the results of our study but new work\nwe have conducted, testing our models using placebo projects, finds they predict 83% of the\nvariation in counterfactual outcomes over a 10 year period. And, we found that omitted\nconfounders would have to have effects much larger than the most important covariates\nincluded in order to remove the signal of over-crediting. We have added this information in\nthe discussion:\n…new evidence demonstrates that the PACT method can predict 83% of the variance in\ntropical forest deforestation rates across such a set of untreated ‘placebo’ landscapes38.\nSecondly, a core assumption of some of the quasi-experimental methods used is that all\nsignificant confounders (covariates associated both with where REDD+ projects are\nimplemented and with deforestation) are controlled for20. Given that quasi-experimental\nstudies apply the same methods across multiple projects, they are not accounting for local\ndrivers of deforestation which may also be associated with the placement of REDD+\nprojects. However, Guizar-Coutiño et. al. (2025) explore the potential influence of such\nhidden confounders and reveal they would need to have an effect size multiple times larger\nthan the most impactful observed covariates to undermine this finding39.” (lines 388 - 400)\nComment 24\nWhy was it necessary to limit the analysis to only 100 matched pairs based on 10% of the\ndata (lines 540-545)?\nResponse:\nOur apologies for the lack of clarity in explaining the matching process. We did not limit the\nanalysis to 100 matched pairs. We have updated the text below to make the method clear:\n“This process was repeated to produce 100 matched project-control sets, where the number\nof points in each set was 10% of the total number of points sampled from within the projects.\nEx post deforestation was measured for the project and control units by calculating changes\nin the undisturbed class since the start of the project. To calculate the area of undisturbed\nforest at each time point, we calculated the proportion of all units in the undisturbed class\nbefore multiplying by the project area. This was done across all 100 matched sets to\nproduce estimates of project forest cover and counterfactual forest cover.\nAveraging across these sets, annualised mean figures are produced. Subtracting the control\nmetric by the project metric, annual additionality/avoided deforestation figures are computed.\nTotal avoided deforestation during the project period was measured as the difference\nbetween the figures for project start year and the final year of the project’s most recent\nmonitoring period.” (lines 613 - 627)\nComment 25\nWhy was a t-test used to assess differences in the distributions for the certified areas (line\n654), whereas other assessments were based on the standardized mean difference? The\nissue is that the t-statistic is a function of the sample size: Just increasing the sample size\ncan make a t-statistic statistically significant, ceteris paribus (Imbens & Wooldridge 2009).\nConversely, a t-test based on a small # observations is likely to have insufficient power.\nResponse:\nWe thank the reviewer for this comment. We have updated the text to clarify that the t-tests\nwere only used across the distribution of SMDs at the project level. This was to test if across\nthe project units the SMDs were significantly different from zero. This means that the n for\nthe t-tests was only 17.\nAcross the set of projects, we tested whether the sample population of standardised mean\ndifferences were different from zero for each of the characteristics using t-tests. (lines 734 -\n735)"}]

Decision letter
Decision Letter

Dear Dr Swinfield,

Your manuscript, "Learning lessons from over-crediting to ensure additionality in forest carbon credits" (NCOMMS-24-45991-T), has been reviewed by our referees and the comments are included below. 

As you will see, the reviewers find the work to be of interest and the manuscript has improved substantially. However, there are a few remaining issues that need to be addressed before we can consider the manuscript for publication. 

In particular, please ensure that you address the points raised by Reviewer 4 regarding the comparability of the bespoke and ACC layers, and the need for appropriate citations. 

We look forward to receiving your revised manuscript. 

Best regards,

Nature Communications Editorial Office