# On track to achieve no net loss of forest at Madagascar's biggest mine

Katie Devenish, Sébastien Desbureaux, Simon Willcock, Julia Jones

► To cite this version:

Katie Devenish, Sébastien Desbureaux, Simon Willcock, Julia Jones. On track to achieve no net loss of forest at Madagascar's biggest mine. Nature Sustainability, 2022, 5, pp.498-508. 10.1038/s41893-022-00850-7. hal-03599731

HAL Id: hal-03599731

https://hal.inrae.fr/hal-03599731

Submitted on 7 Mar 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

OPEN

# On track to achieve no net loss of forest at Madagascar's biggest mine

Katie Devenish<fnref n="1" />, Sébastien Desbureaux², Simon Willcock<fnref n="1" />,³ and Julia P. G. Jones<fnref n="1" />

Meeting the UN Sustainable Development Goals requires reconciling development with biodiversity conservation. Governments and lenders increasingly call for major industrial developments to offset unavoidable biodiversity loss but there are few robust evaluations of whether offset interventions ensure no net loss of biodiversity. We focus on the biodiversity offsets associated with the high-profile Ambatovy mine in Madagascar and evaluate their effectiveness at delivering no net loss of forest. As part of their efforts to mitigate biodiversity loss, Ambatovy compensate for forest clearance at the mine site by slowing deforestation driven by small-scale agriculture elsewhere. Using a range of methods, including extensive robustness checks exploring 116 alternative model specifications, we show that the offsets are on track to avert as much deforestation as was caused by the mine. This encouraging result shows that biodiversity offsetting can contribute towards mitigating environmental damage from a major industrial development, even within a weak state, but there remain important caveats with broad application. Our approach could serve as a template to facilitate other evaluations and so build a stronger evidence-base of the effectiveness of no net loss interventions.

The UN Sustainable Development Goals underline the importance of economic growth and infrastructure development in alleviating poverty, while at the same time emphasizing that halting biodiversity loss is vital for global prosperity<fnref n="1" />,². Policies aimed at delivering no net loss (NNL) of biodiversity, in theory, allow development to proceed while avoiding environmental damage³,⁴. NNL depends on implementation of the mitigation hierarchy: damage to biodiversity resulting from development must first be avoided, minimized and restored⁵, and any residual biodiversity loss offset through equivalent gains elsewhere⁶. At present, 101 countries either mandate some form of biodiversity compensation or support voluntary measures⁷. In countries with less-established environmental governance, lender requirements, such as the International Finance Corporation performance standards, are important drivers of NNL commitments⁸,⁹. Over 12,000 biodiversity offsets exist worldwide¹⁰, yet evaluations of their effectiveness are rare and most do not use robust methods¹¹.

Offsets generate gains in biodiversity by creating or restoring habitat or protecting existing habitat which would have otherwise been lost (so called 'averted loss' offsets¹²). Offsets are controversial due to questions of permanence³, equivalence⁶ and equity¹³,¹⁴ and for generating gain against a background rate of biodiversity decline⁴,¹⁵. However, where high-quality habitat remains but is threatened by unregulated sectors, averted loss offsets may result in the best-possible biodiversity outcomes¹⁶. Biodiversity is an inherently complex concept so proxy measures are used to calculate losses and gains⁶. In forested ecosystems where most species are forest-dependent, forest loss can be a useful measure.

Quantifying the biodiversity gains from averted loss offsets requires estimation of the counterfactual scenario—the loss that would have occurred without protection¹⁵. While the counterfactual is inherently unknowable, statistical approaches exist to approximate it and consequently evaluate the impact of interventions on outcomes such as deforestation¹⁷⁻¹⁹. Statistical matching is commonly used to estimate the counterfactual on the basis of

outcomes in matched control units, yet can be contingent on arbitrary modelling choices²⁰. Recent advances, which test the robustness of estimates to a range of valid, alternative matching model specifications²⁰ and different regression models¹⁸,²¹, can improve the quality of inference.

The Ambatovy nickel and cobalt mine (Fig. 1) is one of the largest lateritic nickel mines in the world. It is located within the biodiverse eastern rainforests of Madagascar which are highly threatened by deforestation, driven principally by shifting agriculture²²,²³. From the outset, Ambatovy promoted itself as a world-leader in sustainable mining and committed to ensure NNL, and preferably net gain, of biodiversity²⁴,²⁵. Its offset strategy was a pilot for the influential Business and Biodiversity Offset Programme²⁴ which shaped guidelines widely used in mitigating biodiversity loss from development¹⁶,²⁵. We use statistical matching and regression models to estimate the avoided deforestation achieved by Ambatovy's four biodiversity offsets and check the robustness of our results to 116 alternative matching model specifications (Fig. 2). We provide encouraging evidence that this high-profile project, in one of the world's hottest biodiversity hotspots, is on track to achieve NNL of forest and critically reflect on this finding in the broader context of NNL.

## Results

Ambatovy's offset strategy is based on averted loss. It aims to generate biodiversity gains to offset the losses incurred at the mine site by preventing an equivalent amount of biodiversity loss within four biodiversity offset sites (which face a high rate of deforestation from shifting agriculture)²⁴. To this end, the company and its non-governmental organization (NGO) partners implemented conservation activities aimed at slowing forest clearance within the four offsets. These included ecological monitoring, establishing community forest management associations and supporting them with the monitoring and enforcement of resource-use restrictions, environmental education programmes and promoting

<footnote n="1">¹School of Natural Sciences, Bangor University, Bangor, UK. ²Center for Environmental Economics, University of Montpellier, CNRS, INRAE, Montpellier SupAgro, Montpellier, France. ³Rothamsted Research, Harpenden, UK. ⁶e-mail: ktd19ycv@bangor.ac.uk</footnote>
<figure><img src="image_3.png" /><figcaption>**Fig. 1** | Study area in eastern Madagascar showing the location of Ambatovy's biodiversity offsets and our study design. **a** The study area is the former province of Toamasina. Control pixels were sampled from pixels that were forested at baseline in 2000 (grey), excluding those within 10 km of a biodiversity offset or within established protected areas (grey hatched). The Corridor Ankeniheny-Zahamena (CAZ) new protected area was included in sampling (Methods). Inset area expanded in **b**. **b** Ambatovy's four biodiversity offsets: the Conservation Zone (yellow) which is within the mine concession area, the Corridor Forestier Analamay-Mantadia (CFAM; green), Torotorofotsy (blue) and Ankerana (orange). The 10 km buffer zone (which excludes established protected areas) around each offset is shown in lighter shades and was used to explore deforestation leakage. **c** Our grid-based sampling strategy (Methods). The top layer illustrates the selection of our subsample of pixels. Data layers labelled X represent the outcome variable and covariates; all data used in this study are publicly available (Supplementary Table 4).</figcaption></figure>

alternative income-generating activities in surrounding communities<sup>26,27</sup>. Occasionally the local police are brought in to assist with enforcement<sup>27</sup>.

According to our site-based difference-in-differences regressions (Methods), of the four biodiversity offsets associated with the Ambatovy mine, two significantly reduced deforestation relative to the counterfactual (Ankerana and the Conservation Zone; *P*<0.01). Protection reduced deforestation by an average of 96% (95% confidence interval (CI) 89–98%; *P*<0.001, *n*=38) per year in Ankerana and 66% (27–84%; *P*<0.01, *n*=38) per year in the Conservation Zone (Fig. 3 and Supplementary Table 9). One offset showed no significant effect (Torotorofotsy; −41 to +510%; *P*=0.28, *n*=38), while the remaining offset (Corridor Forestier Analamay-Mantadia (CFAM)) could not be assessed due to the lack of parallel trends in outcomes between the offset and matched

control sample in the pre-intervention period—a critical assumption in difference-in-differences analyses. In CFAM, there was a significant declining trend in deforestation before protection while the matched control sample showed a significant increasing trend (Supplementary Fig. 5). Therefore, CFAM could not be used in the difference-in-differences analysis.

Including all four offsets in a single analysis using a fixed-effects panel regression (Methods), we estimate that protection reduced deforestation by an average of 58% per year (95% CI 37–73%, *n* = 152) across all four biodiversity offsets, relative to the estimated counterfactual (Fig. 3). We also tested the effect of excluding CFAM and estimate a greater reduction in deforestation of 72% per year (54–83%, *n* = 114; Supplementary Table 12 and Supplementary Fig. 8). Given that the two estimates are not significantly different (Z-test, *P* > 0.2), we present the more conservative estimate, which
<figure><img src="image_4.png" /><figcaption>**Fig. 2** | Flowchart of methods. Statistical matching was used to match sampled pixels from each offset to control pixels sampled from the wider forested landscape with similar exposure to drivers of deforestation (Supplementary Table 4). Difference-in-differences regressions were run for each matched offset-control sample to estimate the effect of protection within each offset (termed site-based difference-in-differences). Pooled data were used in a fixed-effects panel regression to estimate the impact of protection across the whole offset portfolio. Resulting estimates were converted into hectares of avoided deforestation. **a,b,** To test the robustness of results to arbitrary modelling choices, the matching and outcome regressions were repeated using 116 alternative matching model specifications (**a**) to produce a range of estimates (**b**). The statistical distance measure used in matching (for example, Mahalanobis), caliper size, ratio of matched control to treated units and matching with or without replacement (shades of blue/purple) were varied in all 54 possible combinations. Holding these choices constant, we constructed 31 models comprising all possible combinations of five additional covariates (shown in shades of red/orange) with a core set of five essential covariates (green). Finally, we explore the robustness of the results to 31 randomly selected combinations of distance measure, model parameters and additional covariates.</figcaption></figure>

incorporates the effect of all four offsets, as our main result. Our results are also robust to the alternative specification of site and year as random effects (−53%, −27% to −69%; Supplementary Table 12).

**Results robust to alternative model specifications.** Arbitrary modelling choices, particularly associated with the decisions made in a matching analysis, are inevitable yet can exert a strong influence on estimated impacts²⁸. Following Desbureaux²⁹ we show that our results are robust to 116 alternative matching model specifications, all of which are a priori valid (Fig. 4). The vast majority of models for both Ankerana and the Conservation Zone confirm the results from the main model specification (Methods), presented in Fig. 3, of significant avoided deforestation. Where some models show an insignificant result (for example, for the Conservation Zone), in most cases these models are not a posteriori valid. By this we mean that >90% of treated units were unmatched (that is, a match within the caliper of the statistical distance measure could not be found), mean covariate balance exceeded the accepted threshold or parallel trends were not achieved. Exploring alternative model specifications also did not substantially change our results for Torotorofotsy; 78 of the 79 a posteriori valid models showed no significant impact of protection on deforestation, one suggested protection was associ-

ated with an increase in deforestation. For CFAM, the vast majority of alternative specifications, like our main model, were not a posteriori valid as they failed the parallel trends test. Of the seven a posteriori valid models, six showed no significant effect while one showed that protection was associated with a significant increase in deforestation relative to the counterfactual. Our result of a significant overall reduction in deforestation across all four offsets from the fixed-effects panel regression was robust for 106/116 alternative model specifications and none showed a significant increase in deforestation. Therefore, the evidence of avoided deforestation presented in Fig. 3 is robust.

We explored which modelling choices had the greatest influence on estimated impacts and found that the choice of statistical distance measure and model parameters had the most consistent, significant effect while the effect of including additional covariates is mixed (Supplementary Table 13).

**NNL of forest nearly achieved by the offsets.** The mine has destroyed or substantially degraded 2,064 ha of natural forest at the footprint and upper reaches of the slurry pipeline (henceforth mine site)²⁴. The offsets have been in operation for between 7 and 12 years. Using site-based difference-in-differences regressions
<figure><img src="image_5.png" /></figure>

**Fig. 3** | The estimated percentage reduction in annual deforestation within each offset (from the site-based difference-in-differences regressions) and overall, across the entire offset portfolio (from the fixed-effects panel regression). The treatment effect is expressed as the average percentage difference in annual deforestation between the offset(s) and the estimated counterfactual following protection. Error bars represent 95% CIs (the upper bound for TTF extends to +510%). The width of the bar is proportional to the area of forest within each offset at the year of protection (Supplementary Table 2). ANK, Ankerana (orange); CZ, the Conservation Zone (yellow); TTF, Torotorofotsy (blue). CFAM (green) could not be included in the site-based difference-in-differences analysis due to lack of parallel trends in the pre-intervention period (Supplementary Fig. 5). *n* = 38 for Ankerana, the Conservation Zone and Torotorofotsy, and *n* = 152 for the overall result.

we estimate that between the year of protection and January 2020, 1,922 ha (95% CI 669–5,260 ha) of deforestation has been avoided within Ankerana and 26 ha (5–71 ha) has been avoided within the Conservation Zone (Fig. 5; Supplementary Methods). This equates to 1,948 ha of total avoided deforestation (over 94% of the forest loss caused by the mine), with most achieved in Ankerana. Using the fixed-effects panel regression incorporating all four offsets, we estimate an overall reduction in deforestation of 1,644 ha (674–3,122 ha) between 2009, when the first offset was protected, and January 2020 (Fig. 5). This represents over 79% (33–151%) of the forest loss caused by the mine. From 2014, when all the offsets became protected, an average of 265 ha of deforestation was avoided each year until 2020. If this rate continued, by the end of 2021, 2,174 ha of deforestation will have been avoided, fully offsetting forest loss at the mine site. Using the upper and lower bounds of estimated avoided deforestation (674 and 3,122 ha) suggests that NNL will be achieved between 2018 and 2033. In 2014, the company estimated that they would achieve NNL between 2022 and 2035²⁴. Our data therefore suggest that Ambatovy is on track to achieve NNL of forest earlier than anticipated.

Our estimate of the reduction in deforestation achieved within the Conservation Zone (26 ha, 1.6% of the total reduction in deforestation achieved within the offsets) is probably attributable to a

combination of conservation management and the site's location within the mining concession. The company and its predecessor (Phelps Dodge Madagascar) have been present in the concession area since the early 1990s, albeit with a hiatus from 1998 to 2003 (Supplementary Fig. 1). Therefore, for most of the 19-yr study period, access to the concession area, including the Conservation Zone, has been restricted²⁷. This de facto protection reduced deforestation within the Conservation Zone to low levels before it was officially designated as an offset (Fig. 6).

A number of studies have documented leakage effects from conservation interventions whereby impacts within the project area are simply displaced outside the boundaries, negating the effect of the intervention at the landscape-scale²⁹. These leakage effects are not observed in our analysis of Ambatovy's offsets (Supplementary Results) as we found that protection of the biodiversity offsets had no significant effect on deforestation within a 10 km radius (Supplementary Table 16; *P* = 0.15).

**Putting these results in a broader context.** Despite two-thirds of the 12,000+ biodiversity offsets that have been implemented worldwide occurring within forested ecosystems¹⁰, by 2019, <0.05% of these had been evaluated to assess the effectiveness of forest offsets at achieving NNL, and none of these evaluations used robust methods¹¹ (although there have been several robust evaluations of wider offset policies¹²,³⁰). This makes our estimation of the effectiveness of Ambatovy's biodiversity offsets at avoiding deforestation valuable. Our results suggest that, by January 2020, the mine had offset 79% (33–151%) of the forest loss incurred at the mine site and is on track to achieve NNL by the end of 2021.

In recent years, there has been an explosion of studies using robust counterfactual methods to evaluate the effectiveness of other conservation interventions aimed at slowing tropical deforestation. Börner et al.¹⁹ synthesize these findings, using Cohen's *d* normalized effect sizes to compare the effectiveness of 136 conservation interventions at reducing deforestation. Converting our estimate of the total avoided deforestation achieved by Ambatovy's biodiversity offset policy (1,644 ha according to the fixed-effects model) to a Cohen's *d* effect size yielded an estimate of −0.51 (classed as a 'medium effect'³¹; Supplementary Results). This increases to −1.03 for the individual effect of Ankerana and −0.63 for the Conservation Zone (classed as 'large effects'³¹). Comparison to the normalized effect sizes of the 136 other conservation interventions compiled by Börner et al.¹⁹ shows that overall Ambatovy's biodiversity offsets were more effective at reducing deforestation than 97% of the other interventions and all bar one of the protected area interventions (Supplementary Fig. 10).

## Discussion

We lack the empirical evidence to explain why Ambatovy's offsets, as a whole, were so successful at reducing deforestation compared to other forest conservation interventions. We speculate that this may stem from the fact that offsetting is inherently centred on achieving measurable impact (NNL). All activities are designed specifically to meet this goal and progress can be regularly evaluated. Furthermore, large companies may possess sufficient funds to ensure, when they are committed, that they deliver this outcome. In contrast, public protected areas tend to be more focussed on measures such as coverage and investment and less explicitly impact-oriented³². Another important question is why conservation efforts were so successful in Ankerana but not in Torotorofotsy. It may be that enforcement of conservation restrictions was particularly effective within Ankerana, supported by evidence that local communities lost access to resources after the site was protected²⁷ (discussed in more detail below).

**Methodological caveats.** An important caveat to our positive central result relates to the uncertainty inherent in impact evaluation
<figure><img src="image_6.png" /><figcaption>**Fig. 4** | Raw estimates of treatment effect (points) and corresponding 95% CIs (bars) derived from 116 alternative matching model specifications. The alternative specifications included 54 possible combinations of matching distance measure and model parameters, 31 possible combinations of the 5 additional covariates with the core set of essential covariates, and 31 randomly selected combinations of distance measure, model parameters and additional covariates (Methods). Results from our main model specification, presented in Fig. 3, are shown in black. The asterisk indicates that the main model was not a posteriori valid. All alternative specifications are a priori valid but models that are not a posteriori valid (that is, >90% of treated units were unmatched, acceptable covariate balance or parallel trends were not achieved) are shown in lighter shades. See Supplementary Figs. 11 and 12 for full details of parameters and covariates associated with each result. Values are reported untransformed and represent the effect of treatment on the log(y+1)-transformed count of annual deforestation.</figcaption></figure>

using observational data³³. The validity of causal inference rests on our ability to accurately model the counterfactual deforestation in the offset sites (what would have happened in the absence of the intervention) using data from matched pixels in the wider landscape which were not protected as offsets. In difference-in-differences analyses this assumes that all important factors influencing selection to treatment and the outcome of interest have been controlled for (or proxied) in the matching, so that the matched offset and

<figure><img src="image_7.png" /><figcaption>**Fig. 5** | The total observed, counterfactual and the resulting estimate of avoided deforestation within each offset (estimated using site-based difference-in-differences regressions) and overall (using the fixed-effects panel regression) between the year of protection and January 2020.
The counterfactual is an estimate of the deforestation that would have occurred in the absence of protection and was calculated using the estimated treatment effect (n=38; Supplementary Methods). Avoided deforestation is the difference between the observed and counterfactual deforestation; negative values indicate that the offset resulted in a reduction in deforestation. The error bars show the 95% CI of the estimates of counterfactual deforestation (derived from the upper and lower CIs of the treatment effect) and the resulting estimates of avoided deforestation. The green dashed line indicates the 2,064 ha of forest loss caused by the mine itself. The number of years following protection is 9 yr for Ankerana, 11 yr for the Conservation Zone, 6 yr for Torotorofotsy and 11 yr overall (deforestation within later protected offsets is only counted from the year of protection).</figcaption></figure>

control samples have similar trends in deforestation before the intervention³³. Omitted variables may leave outstanding differences between the two samples, which can bias results³³. Our choice of matching covariates is based on a good understanding of the local drivers of deforestation and selection to the treatment²²,²³ (Supplementary Methods) and our robustness checks demonstrate that our results are robust to alternative specifications (Fig. 4).

Our small sample size (n=38 for the difference-in-differences regressions), limited by the length of the time series of the deforestation data³⁴, reduces the precision of our estimates. In addition, methods for impact evaluation using observational data are constantly evolving with recent research highlighting the challenges of evaluating projects with staggered implementation dates³⁵. Despite these caveats, which are the result of inherent challenges from such a real-world evaluation, our methodology represents a substantial advance in impact evaluation applied to biodiversity offsets. While our results seem relatively robust to alternative modelling specifications, this is only one case study. We hope that this work will stimulate further impact evaluations of biodiversity offsetting and emphasize the importance for future researchers to take considerable care over data selection and modelling choices (particularly the matching covariates, distance measure and model parameters) to ensure analyses are context-specific, appropriate and robust.
<figure><img src="image_8.png" /><figcaption>**Fig. 6** | Comparison of the annual deforestation rate within the sample of pixels from each offset and the matched controls over the whole study period.
The offset sample is shown in colour while the matched control sample is shown in grey. The dashed line indicates the year of protection. The offset and matched control samples contain an equal number of pixels (2,862 for Ankerana, 2,626 for CFAM, 1,340 for the Conservation Zone and 1,170 for Torotorofotsy) as the ratio of treated to control units in the matching was set to 1:1. For each offset, *n* = 38.</figcaption></figure>

**Wider concerns with offsetting.** Biodiversity offsets in general, and averted loss offsets in particular, are controversial¹⁵,¹⁶,³⁶. General criticisms include whether a concept as complex as biodiversity can be meaningfully reduced to proxies, questions of permanence³,³⁷ and the potential social and equity issues of trading biodiversity (including access to ecosystem services) in one place for that in another¹³,¹⁴. Specific criticisms of averted loss offsets focus on the accuracy of counterfactual scenarios of loss against which gains are measured⁴,¹⁵ and the mismatch between stakeholder expectations and how much averted loss offsets can actually deliver¹⁶,³⁶. We explore each of these criticisms in turn. In all cases they present clear and important caveats to our positive central result.

The aim of Ambatovy's offset policy is to achieve NNL of biodiversity, whereas our study uses forest cover as an imperfect proxy.

Rarely are the appropriate biodiversity data at the required spatial and temporal scale available to facilitate independent evaluation of NNL commitments. In forested ecosystems where most species are forest-dependent³⁸, forest loss is a transparent and crucially measurable³⁴ proxy for biodiversity loss. Furthermore, offsetting development-induced deforestation to achieve NNL of forest is a desirable outcome in itself, given its implications for biodiversity, ecosystem services and carbon storage. However, our measure of deforestation³⁴ does not capture damage to forest biodiversity occurring at smaller scales, from activities such as selective logging, artisanal mining and harvesting of forest products for food, fuel and building materials³⁹. Moreover, our method does not capture outcomes for species. In a context of high microendemism with many threatened species there is a real risk that large developments such
as Ambatovy could lead to species extinction. To mitigate this risk, the company surveyed areas scheduled for clearance to identify, catch and relocate priority species to conservation areas outside the mine footprint (Supplementary Methods gives other mitigation measures) and conducted follow-up monitoring of certain species²⁴. Whether the impacts of the mine on biodiversity are truly offset will depend on species responses to the changing pressures as well as the presence and efficacy of protection of these species within the offsets, which we were unable to capture in our analysis.

While we present strong evidence that Ambatovy has effectively conserved forest within its biodiversity offsets, questions remain regarding the permanence of this achievement. Although Ankerana and Torotorofotsy have been incorporated into the national protected area network and CFAM has been proposed as a new protected area²⁶, continued effective management after the mine's involvement ceases remains in doubt, given chronic underinvestment in Madagascar's protected areas⁴⁰. If the offsets become de facto unprotected after the company pulls out (expected between 2040 and 2050²⁴), deforestation is likely to resume and forest within the previously protected offsets may be lost. Offsets are intended to persist for as long as the impacts of the development remain³. Although Ambatovy have committed to restoring the impact site and have taken steps to prepare, tropical forest restoration is notionously difficult⁴¹. If restoration fails and the offsets are no longer protected, a future acceleration in biodiversity loss will jeopardize Ambatovy's claims to NNL.

Communities around Madagascar's forests depend on forests for land to practice shifting agriculture and to provide wild products for food, fuel and building materials²²,²⁷. The mine and its associated biodiversity offsets have removed or reduced access to these provisioning ecosystem services. To compensate for this loss of access, Ambatovy invested in promoting alternative income-generating activities (including training and the provision of materials) in communities around the mine site and offsets²⁶,²⁷. However, research conducted within four affected communities (two near the Conservation Zone and two near Ankerana) found that local people did not consider these benefits to outweigh the substantial opportunity costs of the conservation restrictions²⁷. The compensatory activities failed to reach those most affected by the restrictions and there was a temporal mismatch between the immediate loss of access to resources following establishment of the offsets and the time required for the alternatives to yield benefits²⁷. This indicates that poor, rural communities living around the biodiversity offsets are bearing the cost of achieving NNL. For infrastructure developments such as Ambatovy to truly contribute towards sustainable development, SDG 1 (No Poverty) cannot be traded-off for SDG 15 (Life on Land). Instead, project proponents should strive to achieve NNL for both people and planet¹⁴.

An important criticism of averted loss offsets focuses on the accuracy of estimation of the counterfactual scenario; the baseline against which biodiversity losses and gains are measured¹. Many offset policies use historical background rates of deforestation to define the counterfactual but previous studies have shown that this can overestimate the deforestation that would have occurred and consequently overstate the impact of the intervention¹⁷,³⁷. We found that the baseline deforestation rates used by Ambatovy in their loss-gain calculations (based on the highest and lowest background deforestation rates at the district level between 1990 and 2010²⁴) are actually lower than the counterfactual rates we estimate here using robust methods for impact evaluation, meaning their estimates were conservative (Supplementary Table 1). However, there is an important caveat to this: the mine resulted in in-migration to the region²⁶,²⁷ which may have indirectly increased pressures on forest resources within the wider landscape, as observed with Rio Tinto's QMM ilmenite mine in Southern Madagascar³⁷. If any mine-related pressures were captured within

the period used to define the 'background' rate of deforestation this would no longer represent baseline conditions in the absence of the mine and inflate the counterfactual (and the resulting estimates of biodiversity gains). Ambatovy employs ~9,000 people²⁶, many of whom moved to the area from other regions of Madagascar²⁶,²⁷. The influx of migrant workers probably increased local demand for food, charcoal and fuelwood, which may have increased forest clearance and bushmeat hunting²⁶,⁴². Such indirect impacts associated with industrial development are notoriously difficult to quantify and therefore offset⁴³. Neither our approach, nor Ambatovy's loss-gain calculations, could account for the indirect impacts of the mine on regional deforestation.

Another criticism of averted loss offsets is that they are premised on a background rate of biodiversity decline which can be slowed to generate the required biodiversity gains⁴,¹⁶. Therefore, even if NNL as defined by best-practice guidelines⁸ is achieved, loss of biodiversity has still occurred¹⁵,³⁶. This is not what many stakeholders would understand as NNL of biodiversity⁴⁴. However, given Madagascar's high rates of deforestation⁴⁵ and poor outcomes from tropical forest restoration⁴¹, averted loss is likely to be the better offsetting option¹⁶. Yet Madagascar has little remaining forest left to lose. Given the importance of the country's biodiversity and the multitude of threats facing it⁴⁰, future developments could aim to go beyond NNL and contribute towards the overall conservation of Madagascar's remaining biodiversity¹⁶.

**Hope for mitigating the environmental impacts of mines.** There are over 6,000 industrial mines operating worldwide, covering an estimated 57,000 km² (ref. ⁴⁶) and impacting around 10% of global forested lands⁴⁷. Low-income countries, like Madagascar, desperately need economic development. Mining, if well-regulated, can be part of the solution. From the start, Ambatovy promoted itself as a world-leader in sustainable mining and has some of the strongest commitments to conservation among 29 large-scale mines operating within forests⁴⁷. Given this, and the resulting substantial investment the company made in NNL, failure would have been worrying for the concept of mitigating biodiversity loss from development. However, the achievements are notable, especially considering the challenging institutional and political context¹⁸ in which Ambatovy operates. Our results provide encouraging evidence that Ambatovy's economic contributions to Madagascar¹⁹ (tens of millions of US dollars a year), were made while minimizing trade-offs with the island's precious remaining forest habitat. There are many important caveats to this finding, as to any claim of NNL achieved through offsetting, however, the result certainly demonstrates the value of high aspirations combined with substantial investment in mitigating the biodiversity impacts of mining.

## Methods

**Study site and context.** Ambatovy is a very large nickel, cobalt and ammonium sulphate mine in central-eastern Madagascar owned by a consortium of international mining companies⁵⁰. It represents the largest ever foreign investment in the country²⁴ (US$8 billion by 2016²⁰) and a substantial source of fiscal income⁹. In 2018, the company contributed ~US$50 million in taxes, tariffs, royalties and other payments¹⁹ and employed >9,000 people (93% of whom were Malagasy)²¹. Commercial production began in January 2014²⁴ (Supplementary Fig. 1). As key components in batteries, supplies of nickel and cobalt are critical to the green energy transition and demand for these metals is predicted to increase notably in future²².

The mining concession covers an area of 7,700 ha located in the eastern rainforests of Madagascar (Fig. 1) which have very high levels of biodiversity and endemism³³,⁵⁴. After avoidance and minimization measures were applied (Supplementary Methods) the mine was predicted to clear or substantially degrade 2,064 ha of high-quality natural forest at the mine footprint and upper pipeline³¹. Any impacts on plantations or secondary habitat are not included in this estimate. Losses at the impact site were not discounted in relation to a background rate of decline, meaning that the company took responsibility for the full area of forest lost³². Independent verification by our team (by measuring the size of the mine footprint on Google Earth) confirms the extent of forest loss at the mine footprint
(Supplementary Fig. 2). Clearance of the footprint accounts for most of the forest loss associated with the mine as losses associated with the pipeline are small<sup>24</sup>.

Ambatovy aims to generate biodiversity gains to offset the mine-induced losses by slowing deforestation driven by shifting agriculture elsewhere<sup>25</sup>. To this end the company designated four sites, totalling 28,740 ha, to be protected as biodiversity offsets; Ankerana, Corridor Forestier Analamay-Mantadia (CFAM), the Conservation Zone and Torotorofotsy<sup>24</sup> (Fig. 1). The offsets are considered like-for-like<sup>10</sup> and were selected on the basis of similarity to the impact site in terms of forest structure and type, geology, climate and altitude<sup>24</sup>. The large combined area of the offsets relative to the impacted area was designed to allow flexibility, account for uncertainty and incorporate as many of the affected biodiversity components as possible<sup>24</sup>. Ankerana is the flagship offset, selected on the basis of its size, connectivity to the Corridor Ankeniheny-Zahamena (CAZ) forest corridor and the presence of ultramafic outcrops thought to support the same rare type of azonal forest lost at the mine site<sup>24</sup>. Extensive surveys conducted within Ankerana to establish biological similarity concluded the offset to be of higher conservation significance than the forests of the mine site due to the presence of rare lowland tropical forest<sup>24</sup>.

The Conservation Zone is directly managed by the company, given its location within the concession area, while the other offsets are managed in partnership with local and international NGOs<sup>4,25</sup>. Ambatovy funds the management of Ankerana by Conservation International and local NGO partners (although before 2015 Ankerana was directly managed by Ambatovy via a Memorandum of Understanding with Conservation International<sup>24</sup>), supports BirdLife partner Asity with the management of Torotorofotsy and a number of local NGOs including Voary Voakajy<sup>25</sup> are involved in CFAM<sup>26</sup>. The company is also working to secure formal, legal protection for CFAM<sup>26</sup> as part of a proposed Torotorofotsy-CFAM complex new protected area (although progress on this has stalled).

**Overview of methods.** To estimate the impact of the offsets on deforestation and determine whether this has prevented enough deforestation to offset forest loss at the mine site, we combined several complementary methods for robust impact evaluation. First, we used statistical matching to match a sample of pixels from each biodiversity offset to pixels from the wider forested landscape with similar exposure to drivers of deforestation. Then we used a site-based difference-in-differences regression for each matched offset-control sample and a fixed-effects panel regression on the pooled data, to estimate the effect of protection. We systematically explored how arbitrary modelling choices (including the statistical distance measure used in matching, caliper size, ratio of control to treated units, matching with or without replacement and which, if any, additional covariates were included) affected our inference, exploring the robustness of our results to 116 alternative model specifications.

**Matching.** The former province of Toamasina was selected as the geographic area as the concession area with varying degrees of intactness and accessibility. The four biodiversity offsets are located within this province (Fig. 1).

The unit of analysis is a 30×30 m² pixel that was forested in the baseline year 2000<sup>15,55</sup>. It is important that the scale of analysis aligns with the scale at which the drivers of deforestation (in this case, small-scale shifting agriculture) operate<sup>56</sup>. The median agricultural plot size (from 564 measured plots) in the study region is ~36×36 m² (ref. 57). We took a subsample of pixels to reduce computational effort while maintaining the capacity for robust statistical inference<sup>58,59</sup>. We used a grid-based sampling strategy ensuring a minimum distance between sample units to reduce spatial autocorrelation<sup>60</sup> and equal coverage of the study area<sup>58</sup>. A 150×150 m² resolution grid, aligned to the other 30-m resolution data layers (Fig. 1c), was overlaid on the province and the 30×30 m² pixel at the centre of each grid square was extracted to produce a subsample of pixels that are 120 m away from their nearest neighbour. The 120 m is larger than the minimum distance between units used in another matching study in Madagascar (68 m; ref. 59) but smaller than that used in other studies (200 m; ref. 61) and so strikes an appropriate balance between the avoidance of spatial autocorrelation and maximizing the possible sample cells.

Protected areas in the study area managed by Madagascar National Parks were excluded from our control sample as they are actively managed and therefore do not represent counterfactual outcomes for the biodiversity offsets in the absence of protection (Fig. 1). However, control pixels were sampled from within the CAZ new protected area as legal protection was only granted in 2015 and resources for management are limited and thinly spread<sup>62</sup>. Additionally, Ankerana and parts of CFAM overlap with the CAZ and would have experienced the same management, and likely trajectory, as the rest of the CAZ, had they not been designated biodiversity offsets. Areas within 10 km of an offset boundary were excluded from the control sample to reduce the chance of leakage (where pressures are displaced rather than avoided) biasing results<sup>17,29</sup>. The 10 km was selected as it is a commonly used buffer zone within the literature<sup>7,58</sup>.

To test for leakage effects, we used Veronoi polygons to partition the buffer area for CFAM, the Conservation Zone and Torotorofotsy (which overlap) into three individual buffer areas according to the nearest offset centroid and took a subsample of pixels from each (Fig. 1). Areas that overlapped with the established

protected areas of Mantadia National Park and Analamazotra Special Reserve were excluded from the buffer zones.

The outcome variable is the annual deforestation rate sourced from the Global Forest Change (GFC) dataset<sup>34</sup>. Following Vieilledent et al.<sup>16</sup> these data were restricted to only include pixels classed as forest in a forest cover map of Madagascar for the year 2000<sup>45,55</sup>, reducing the probability of false positives (whereby tree loss is identified in pixels that were not forested). The resulting tree loss raster was snapped to the forest cover 2000 layer to align cells, resulting in a maximum spatial error of 15 m. The GFC product<sup>14</sup> has been shown to perform reasonably well at detecting deforestation in humid tropical forests<sup>63</sup>. In the north-eastern rainforests of Madagascar, Burivalova et al.<sup>39</sup> found that GFC data performed comparably to a local classification of very high resolution satellite imagery at detecting forest clearance for shifting agriculture (although it was not effective at detecting forest degradation from selective logging). As clearance for shifting agriculture is considered the principal agent of deforestation in the study area<sup>23</sup> and the forests of the study area are tropical humid (>75% canopy cover), the GFC data are an appropriate tool for quantifying forest loss. Although recent evidence suggests that GFC data may have temporal biases<sup>64</sup>, this phenomenon affects our control and treated samples equally and so is unlikely to impact our results.

The choice of covariates is extremely important in matching analyses. They must include, or proxy, all important factors influencing selection to treatment and the outcome of interest so that the matched control sample is sufficiently similar to the treated sample in these characteristics to constitute a plausible counterfactual, otherwise the resulting estimates may not be valid<sup>33</sup>. On the basis of the literature and a local theory of change we selected five covariates that we believe capture or proxy for the aspects of accessibility, demand and agricultural suitability that drive deforestation in the study area<sup>22,59,65,66</sup>. These are slope, elevation, distance to main road, distance to forest edge and distance to deforestation (Supplementary Methods). These five essential covariates comprise the main matching specification and form the core set used in all alternative specifications that we tested in the robustness checks. We also defined five additional variables (annual precipitation, distance to river, distance to cart track, distance to settlement and population density) and tested the effect of including these in the robustness checks. The additional covariates were so defined because they were of poorer data quality (population density and distance to settlement), correlated with an essential variable (annual precipitation and population density) or simply considered less influential (distance to river and distance to cart track; Supplementary Methods).

Statistical matching was conducted in R statistics using the MatchIt package v.4.1 (ref. 67). To improve efficiency and produce closer matches we cleaned the data before matching to remove control units with values outside the calipers of the treated sample in any of the essential covariates (see Supplementary Methods for caliper definition). Following the recommendations of Schleicher et al.<sup>68</sup> we tested several matching specifications and selected the one that maximized the trade-off between the number of treated units matched and the closeness of matches as the main specification (Supplementary Table 7). This was 1:1 nearest-neighbour matching without replacement, using Mahalanobis distance and a caliper of 1 s.d. This specification produced acceptable matches (within 1 s.d. of the Mahalanobis distance) for all treated units within all offsets. The maximum postmatching standardized difference in mean covariate values between treated and control samples was 0.05, well below the threshold of 0.25 considered to constitute an acceptable match<sup>69</sup>. This indicates that, on average, treated and control units were very well matched across all covariates.

Matching was run separately for each offset. The resulting matched datasets were aggregated by treated status (offset or control) and year to produce a matrix of the count of pixels that were deforested each year (2001–2019) in the offset and the matched control sample. Converting the outcome variable to a continuous measure of deforestation avoids the problem of attrition associated with binary measures of deforestation and is better suited to the framework of the subsequent regressions<sup>70</sup>.

**Robustness checks.** Statistical matching requires various choices to be made<sup>68</sup>, many of which are essentially arbitrary. There therefore exists a range of possible alternative specifications that are all a priori valid (although some may be better suited to the data and study objectives<sup>69</sup>) but which could influence the results<sup>20,28</sup>. We tested the robustness of our results to 116 different matching model specifications (Fig. 4). First, we tested the robustness of the estimates to the use of three alternative matching distance measures (Mahalanobis, standard propensity score matching using generalized linear model regressions with a logit distribution and propensity score matching using RandomForest), three different calipers (0.25, 0.5 and 1 s.d.), different ratios of control to treated units (one, five and ten nearest neighbours) and matching with/without replacement. Holding the choice of covariates constant (using only the essential covariates), the combination of these led to the estimation of 54 different models. Second, we tested the robustness of results to the inclusion of the five additional covariates. Holding the choice of distance measure and model parameters constant, we constructed 31 models comprising all possible combinations of additional covariates with the core set of essential covariates. Finally, we explore the robustness of results for 31 randomly selected combinations of distance measure, model parameters and additional covariates. All 116 specifications are a priori valid, assuming that the covariates
capture or proxy for all important factors influencing outcomes, but may fail to satisfy the parallel trends condition or produce matches for insufficient numbers of treated observations (<10%), rendering them a posteriori invalid. It remains important to test the assumptions of the alternative models as failure to do so may lead to erroneous conclusions about effect size and direction being drawn from invalid models. Results are presented through specification graphs based on codes developed in Ortiz-Bobea et al.⁷¹.

Additionally, we tested the robustness of our results from the site-based difference-in-differences regressions to alternative temporal specifications using an equal number of years before and after the intervention (8 yr for Ankerana and the Conservation Zone, 6 yr for CFAM and 5 yr for Torotorofotsy) and dropping individual years from the analysis. This did not change the significance or magnitude of our results (Supplementary Table 10 and Supplementary Figs. 6 and 7).

**Outcome regressions.** Deriving estimates of causal effect from statistical comparisons of outcomes between treated and control samples relies on the assumption that the latter is a robust counterfactual for the former. In a difference-in-differences analysis this assumes that, in the absence of the intervention, the treated sample would have experienced the same average change in outcomes over the before-after period as the control sample.⁷² Parallel trends in outcomes between treated and control before the intervention are an essential prerequisite for this assumption. We tested this for each matched offset-control dataset using the following formula:

$$ \log(\text{Count of deforestation } + 1)_{i,t} = \beta_0 + \beta_1 \text{Year}_t + \beta_2 \text{CI}_i + \beta_3 (\text{Year} \times \text{CI}_{it}) + \epsilon_{i,t} \quad (1) $$

where the outcome is the log(y+1)-transformed count of deforestation within sample *i* at year *t* and CI is a binary variable indicating whether the observation is from the offset (1) or control (0) sample.

Parallel trends in deforestation between offset and matched control samples in the years before the intervention were present for all offsets except for CFAM (Supplementary Fig. 5). Consequently, CFAM could not be used in the site-based difference-in-differences analysis. However, its effect is still captured in the results from the fixed-effects panel regression as this is not based on an identifying assumption of parallel trends between groups in the pretreatment period.⁷²

To estimate the impact of protection within each individual offset, we ran an ordinary-least-squares difference-in-differences regression for each matched offset-control dataset using the following formula:

$$ \log(\text{Count of deforestation } + 1)_{i,t} = \beta_0 + \beta_1 \text{BA}_t + \beta_2 \text{CI}_t + \beta_3 (\text{BA} \times \text{CI})_{it} + \epsilon_{i,t} \quad (2) $$

where BA and CI are binary variables indicating whether the observation occurred before (0) or after (1) the intervention, in the offset (1) or control sample (0). Given the non-normal properties of count data and the presence of zero values, a log(y+1) transformation was applied to the outcome variable.⁷⁰,⁷³ The coefficient of BA × CI and the corresponding confidence intervals were back-transformed (Supplementary Table 9) to obtain an estimate of the percentage difference in average annual deforestation between the offset and the matched control sample after protection, controlling for pre-intervention differences between samples (that is, the estimated counterfactual).

To estimate the overall impact of Ambatovy's biodiversity offset policy at reducing deforestation we pooled the data for all four offsets and their corresponding matched control samples and ran a fixed-effects panel regression. The pooled data (n=152) comprise an observation for each site (i=8, 4 offset and 4 control) for each year (t=19). The fixed-effects panel regression quantifies the effect of protection on the log-transformed count of deforestation controlling for site and year fixed effects, according to the following formula:

$$ \log(\text{Count of deforestation } + 1)_{i,t} = \beta_0 + \beta_1 \text{Tr}_{i,t} + \alpha_i + \gamma_t + \epsilon_{it} \quad (3) $$

where Tr is a binary measure indicating the treated status of sample *i* in year *t* (Tr=1 for observations from offset sites in the years following protection and 0 for all other observations), α, and γ, represent site and year fixed effects, respectively, and εᵢᵢ represents the composite error. The coefficient of interest (βᵢ) and the associated confidence intervals were back-transformed to obtain the percentage difference in average annual deforestation across all four biodiversity offsets following protection (the treatment effect).

**Evaluating deforestation leakage.** To determine whether protection of the four biodiversity offsets simply displaced deforestation into the surrounding forested landscape, we repeated the matching and outcome regressions with the subsample of units from each buffer zone assigned as the treated group¹⁷,⁵⁸ (Supplementary Results).

**Reporting Summary.** Further information on research design is available in the Nature Research Reporting Summary linked to this article.

## Data availability

All input data used in this study are available in the GitHub repository at https://github.com/katie-devs/Biodiversity_offset_effectiveness.

## Code availability

All computer code used in this study are available in the GitHub repository at https://github.com/katie-devs/Biodiversity_offset_effectiveness.

Received: 24 June 2021; Accepted: 13 January 2022; Published online: 03 March 2022

## References

1. Thacker, S. et al. Infrastructure for sustainable development. *Nat. Sustain.* **2**, 324–331 (2019).

2. Blicharska, M. et al. Biodiversity's contributions to sustainable development. *Nat. Sustain.* **2**, 1083–1093 (2019).

3. Bull, J. W., Suttle, K. B., Gordon, A., Singh, N. J. & Milner-Gulland, E. J. Biodiversity offsets in theory and practice. *Oryx* **47**, 369–380 (2013).

4. Maron, M. et al. The many meanings of no net loss in environmental policy. *Nat. Sustain.* **1**, 19–27 (2018).

5. McKenney, B. A. & Kiesecker, J. M. Policy development for biodiversity offsets: a review of offset frameworks. *Environ. Manage.* **45**, 165–176 (2010).

6. Quétier, F. & Lavorel, S. Assessing ecological equivalence in biodiversity offset schemes: key issues and solutions. *Biol. Conserv.* **144**, 2991–2999 (2011).

7. zu Ermgassen, S. O. S. E., Utamiputri, P., Bennun, L., Edwards, S. & Bull, J. W. The role of “no net loss” policies in conserving biodiversity threatened by the global infrastructure boom. *One Earth* **1**, 305–315 (2019).

8. Performance Standards on Environmental and Social Sustainability Vol. 2 (International Finance Corporation, 2012); https://www.ifc.org/wps/wcm/connect/Topics_Ext_Content/IFC_External_Corporate_Site/Sustainability-At-IFC/Policies-Standards/Performance-Standards

9. Bidaud, C., Schreckenberg, K. & Jones, J. P. G. The local costs of biodiversity offsets: comparing standards, policy and practice. *Land Use Policy* **77**, 43–50 (2018).

10. Bull, J. W. & Strange, N. The global extent of biodiversity offset implementation under no net loss policies. *Nat. Sustain.* **1**, 790–798 (2018).

11. zu Ermgassen, S. O. S. E. et al. The ecological outcomes of biodiversity offsets under “no net loss” policies: a global review. *Conserv. Lett.* **12**, e12664 (2019).

12. Sonter, L. J. et al. Local conditions and policy design determine whether ecological compensation can achieve no net loss goals. *Nat. Commun.* **11**, 2072 (2020).

13. Ives, C. D. & Bekessy, S. A. The ethics of offsetting nature. *Front. Ecol. Environ.* **13**, 568–573 (2015).

14. Jones, J. P. G. et al. Net gain: seeking better outcomes for local people when mitigating biodiversity loss from development. *One Earth* **1**, 195–201 (2019).

15. Maron, M., Bull, J. W., Evans, M. C. & Gordon, A. Locking in loss: baselines of decline in Australian biodiversity offset policies. *Biol. Conserv.* **192**, 504–512 (2015).

16. Simmonds, J. S. et al. Moving from biodiversity offsets to a target-based approach for ecological compensation. *Conserv. Lett.* **13**, e12695 (2019).

17. West, T. A. P., Börner, J., Sills, E. O. & Kontoleon, A. Oversted carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon. *Proc. Natl Acad. Sci. USA* **117**, 24188–24194 (2020).

18. Carlson, K. M. et al. Effect of oil palm sustainability certification on deforestation and fire in Indonesia. *Proc. Natl Acad. Sci. USA* **115**, 121–126 (2018).

19. Börner, J., Schulz, D., Wunder, S. & Pfaff, A. The effectiveness of forest conservation policies and programs. *Annu. Rev. Resour. Econ.* **12**, 45–64 (2020).

20. Desbureaux, S. Subjective modeling choices and the robustness of impact evaluations in conservation science. *Conserv. Biol.* **35**, 1615–1626 (2021).

21. Ferraro, P. J. & Simorangkir, R. Conditional cash transfers to alleviate poverty also reduced deforestation in Indonesia. *Sci. Adv.* **6**, eaaz1298 (2020).

22. Poudyal, M. et al. Who bears the cost of forest conservation? *PeerJ* **6**, e5106 (2018).

23. Tabor, K. et al. Evaluating the effectiveness of conservation and development investments in reducing deforestation and fires in Ankeniheny-Zahemena Corridor, Madagascar. *PLoS ONE* **12**, e0190119 (2017).

24. von Hase, A. et al. Working towards NNL of Biodiversity and Beyond: Ambatovy, Madagascar—A Case Study (2014) (Ambatovy Forest Trends, 2014); https://www.forest-trends.org/wp-content/uploads/imported/ambatovy-bbop-nnl-2014-final-pdf.pdf

25. Bidaud, C., Hrabanski, M. & Meral, P. Voluntary biodiversity offset strategies in Madagascar. *Ecosyst. Serv.* **15**, 181–189 (2015).

26. Ambatovy Sustainability Report 2017 (Ambatovy, 2017); https://minedocs.com/17/Ambatovy_Sustainability_Report_2017.pdf

27. Bidaud, C. et al. The sweet and the bitter: intertwined positive and negative social impacts of a biodiversity offset. *Conserv. Soc.* **15**, 1–13 (2017).
28. Silberzahn, R. et al. Many analysts, one data set: making transparent how variations in analytic choices affect results. *Adv. Methods Pract. Psychol. Sci.* 1, 337–356 (2018).

29. Ford, S. A. et al. Deforestation leakage undermines conservation value of tropical and subtropical forest protected areas. *Glob. Ecol. Biogeogr.* 29, 2014–2024 (2020).

30. Sonter, L. J., Barrett, D. J. & Soares-Filho, B. S. Offsetting the impacts of mining to achieve no net loss of native vegetation. *Conserv. Biol.* 28, 1068–1076 (2014).

31. Cohen, J. *Statistical Power Analysis for the Behavioural Sciences* (Routledge, 1988).

32. Pressey, R. L. et al. The mismeasure of conservation. *Trends Ecol. Evol.* 36, 808–821 (2021).

33. Ferraro, P. J. & Hanauer, M. M. Advances in measuring the environmental and social impacts of environmental programs. *Annu. Rev. Environ. Resour.* 39, 495–517 (2014).

34. Hansen, M. C. et al. High-resolution global maps of 21st-century forest cover change. *Science* **342**, 850–853 (2013).

35. Callaway, B. & Sant'Anna, P. H. C. Difference-in-differences with multiple time periods. *J. Econ.* **225**, 200–230 (2021).

36. Gordon, A., Bull, J. W., Wilcox, C. & Maron, M. Perverse incentives risk undermining biodiversity offset policies. *J. Appl. Ecol.* **52**, 532–537 (2015).

37. Virah-Sawmy, M., Ebeling, J. & Taplin, R. Mining and biodiversity offsets: a transparent and science-based approach to measure 'no-net-loss'. *J. Environ. Manage.* **143**, 61–70 (2014).

38. Goodman, S. M. & Benstead, J. P. Updated estimates of biotic diversity and endemism for Madagascar. *Oryx* **39**, 73–77 (2005).

39. Burivalova, Z., Bauert, M. R., Hassold, S., Fatroandrianjafinonjasolomiovazo, N. T. & Koh, L. P. Relevance of global forest change data set to local conservation: case study of forest degradation in Masoala National Park, Madagascar. *Biotropica* **47**, 267–274 (2015).

40. Jones, J. P. G. et al. Last chance for Madagascar's biodiversity. *Nat. Sustain.* 2, 350–352 (2017).

41. Crouzeilles, R. et al. Ecological restoration success is higher for natural regeneration than for active restoration in tropical forests. *Sci. Adv.* 3, e1701345 (2017).

42. Razafimanahaka, J. H. et al. Novel approach for quantifying illegal bushmeat consumption reveals high consumption of protected species in Madagascar. *Oryx* **46**, 584–592 (2012).

43. Lechner, A. M. et al. Challenges of integrated modelling in mining regions to address social, environmental and economic impacts. *Environ. Model. Softw.* **93**, 268–281 (2017).

44. Bekessy, S. A. et al. The biodiversity bank cannot be a lending bank. *Conserv. Lett.* **3**, 151–158 (2010).

45. Vieilledent, G. et al. Combining global tree cover loss data with historical national forest cover maps to look at six decades of deforestation and forest fragmentation in Madagascar. *Biol. Conserv.* **222**, 189–197 (2018).

46. Maus, V. et al. A global-scale data set of mining areas. *Sci. Data* **7**, 289 (2020).

47. *Forest-Smart Mining: Large-Scale Mining on Forests (LSM)* (World Bank Group, 2019); https://doi.org/10.1596/32027

48. Jones, J. P. G. et al. Madagascar: crime threatens biodiversity. *Science* **363**, 825 (2019).

49. Young, E. *Rapport de Réconciliation 2018: EITI Madagascar* (EITI, 2019); https://eiti.org/document/madagascar-2018-ekti-report

50. *Our Business* https://ambatovy.com/en/who-we-are/about-us/ (Ambatovy, 2021).

51. *Ambatovy Sustainability Report 2018* (Ambatovy, 2018); https://ambatovy.com/ang/wp-content/uploads/2021/01/Ambatovy-Sustainability-Report-2018-EN.pdf

52. Hund, K., La Porta, D., Fabregas, T., Laing, T. & Drexhage, J. *Minerals for Climate Action: The Mineral Intensity of the Clean Energy Transition* (World Bank Publications, 2020); http://pubdocs.worldbank.org/en/961711588875536384/Minerals-for-Climate-Action-The-Mineral-Intensity-of-the-Clean-Energy-Transition.pdf

53. Phillipson, P. B., Lowry, P. P. II, Andriamahefarivo, L., Antilahimana, P. & Birkinshaw, C. Floristic inventory of the Ambatovy-Analamay mine site and comparison to other sites in Madagascar. *Malagasy Nat.* **3**, 44–76 (2010).

54. Berner, P. O., Dickinson, S. & Andrianarimisa, A. *BBOP Pilot Project Case Study: The Ambatovy Project* (Forest Trends, 2009); https://www.forest-trends.org/wp-content/uploads/imported/Ambatovy-case-study.pdf

55. Harper, G. J., Steininger, M. K., Tucker, C. J., Juhn, D. & Hawkins, F. Fifty years of deforestation and forest fragmentation in Madagascar. *Environ. Conserv.* **34**, 325–333 (2007).

56. Avelino, A. F. T., Baylis, K. & Honey-Rosés, J. Goldilocks and the raster grid: selecting scale when evaluating conservation programs. *PLoS ONE* **11**, e0167945 (2016).

57. Poudyal, M., Rakotonarivo, O. S., Razafimanahaka, J. H., Hockley, N. & Jones, J. P. G. Household economy, forest dependency & opportunity costs of conservation in eastern rainforests of Madagascar. *Sci. Data* https://doi.org/10.1038/sdata.2018.225 (2018).

58. Blackman, A. Evaluating forest conservation policies in developing countries using remote sensing data: an introduction and practical guide. *Policy Econ.* **34**, 1–16 (2013).

59. Rasolofoson, R. A., Ferraro, P. J., Jenkins, C. N. & Jones, J. P. G. Effectiveness of community forest management at reducing deforestation in Madagascar. *Biol. Conserv.* **184**, 271–277 (2015).

60. Robalino, J. A. & Pfaff, A. Contagious development: neighbor interactions in deforestation. *J. Dev. Econ.* **97**, 427–436 (2012).

61. Bruggeman, D., Meyfroidt, P. & Lambin, E. F. Production forests as a conservation tool: effectiveness of Cameroon's land use zoning policy. *Land Use Policy* **42**, 151–164 (2015).

62. Hewson, J. et al. Land change modelling to inform strategic decisions on forest cover and CO₂ emissions in eastern Madagascar. *Environ. Conserv.* **46**, 25–33 (2019).

63. Galiatsatos, N. et al. An assessment of global forest change datasets for national forest monitoring and reporting. *Remote Sens.* **12**, 1790 (2020).

64. Palahí, M. et al. Concerns about reported harvests in European forests. *Nature* **592**, E15–E17 (2021).

65. McConnell, W. J., Sweeney, S. P. & Mulley, B. Physical and social access to land: spatio-temporal patterns of agricultural expansion in Madagascar. *Agric. Ecosyst. Environ.* **101**, 171–184 (2004).

66. Eklund, J. et al. Contrasting spatial and temporal trends of protected area effectiveness in mitigating deforestation in Madagascar. *Biol. Conserv.* **203**, 290–297 (2016).

67. Ho, D. E., Imai, K., King, G. & Stuart, E. A. MatchIt: nonparametric preprocessing for parametric causal inference. *J. Stat. Softw.* **42**, 1–28 (2011).

68. Schleicher, J. et al. Statistical matching for conservation science. *Conserv. Biol.* **34**, 538–549 (2019).

69. Stuart, E. A. Matching methods for causal inference: a review and a look forward. *Stat. Sci.* **25**, 1–21 (2010).

70. Desbureaux, S. & Damania, R. Rain, forests and farmers: evidence of drought induced deforestation in Madagascar and its consequences for biodiversity conservation. *Biol. Conserv.* **221**, 357–364 (2018).

71. Ortiz-Bobea, A., Ault, T. R., Carrillo, C. M., Chambers, R. G. & Lobell, D. B. Anthropogenic climate change has slowed global agricultural productivity growth. *Nat. Clim. Change* **11**, 306–312 (2021).

72. Cunningham, S. *Causal Inference: The Mixtape* (Yale Univ. Press, 2021).

73. Ives, A. R. For testing the significance of regression coefficients, go ahead and log-transform count data. *Methods Ecol. Evol.* **6**, 828–835 (2015).

## Acknowledgements

We thank A. Guizar Coutiño and H. Wauchope for valuable advice on the methodology and K. Goodenough for her support, insightful discussions and comments; J. Borner and D. Schulz for sharing data allowing us to create Supplementary Fig. 10. This work was funded by the Natural Environment Research Council Envision DTP awarded to K.D., the British Geological Survey and Rothamsted Research's Transformation Fund. S.D. received financial support from the PARSEC project.

## Author contributions

K.D., S.W. and J.P.G.J. conceived and designed the study with contributions from S.D. on the statistical analysis. K.D. compiled the data. K.D. and S.D. performed the statistical analysis. K.D., S.D. and J.P.G.J. wrote the paper.

## Competing interests

J.P.G.J. co-authored a paper that was quite critical of the social impact of Ambatovy's biodiversity offsets in 2017. She was later approached by the new leadership of Ambatovy who asked if she could return to villages visited in that research to explore the current situation. She collected independent information from a number of villages around the Conservation Zone, Torotorofotsy and CFAM biodiversity offsets and fed this back to the mine. This work was funded by an Economic and Social Research Council Impact Accelerator Award to J.P.G.J. She has never received any funding from Ambatovy. The other authors declare no competing interests.

## Additional information

**Supplementary information** The online version contains supplementary material available at https://doi.org/10.1038/s41893-022-00850-7.

**Correspondence and requests for materials** should be addressed to Katie Devenish.

**Peer review information** *Nature Sustainability* thanks Rachel Neugarten, Laura Sonter, Patrick Waeber and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

**Reprints and permissions information** is available at www.nature.com/reprints.
**Publisher's note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_9.png" /></figure>

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the

you give appropriate credit to the original author(s) and the source, provide a link to the

Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. © The Author(s) 2022
# Reporting Summary

Nature Research wishes to improve the reproducibility of the work that we publish. This form provides structure for consistency and transparency in reporting. For further information on Nature Research policies, see our [Editorial Policies](#) and the [Editorial Policy Checklist](#).

## Statistics

For all statistical analyses, confirm that the following items are present in the figure legend, table legend, main text, or Methods section.

n/a | **Confirmed**

☐ ☑ The exact sample size (n) for each experimental group/condition, given as a discrete number and unit of measurement

☐ ☑ A statement on whether measurements were taken from distinct samples or whether the same sample was measured repeatedly

☐ ☑ The statistical test(s) used AND whether they are one- or two-sided
*Only common tests should be described solely by name; describe more complex techniques in the Methods section.*

☐ ☑ A description of all covariates tested

☐ ☑ A description of any assumptions or corrections, such as tests of normality and adjustment for multiple comparisons

☐ ☑ A full description of the statistical parameters including central tendency (e.g. means) or other basic estimates (e.g. regression coefficient) AND variation (e.g. standard deviation) or associated estimates of uncertainty (e.g. confidence intervals)

☐ ☑ For null hypothesis testing, the test statistic (e.g. *F*, *t*, *r*) with confidence intervals, effect sizes, degrees of freedom and *P* value noted
*Give *P* values as exact values whenever suitable.*

☒ ☐ For Bayesian analysis, information on the choice of priors and Markov chain Monte Carlo settings

☒ ☐ For hierarchical and complex designs, identification of the appropriate level for tests and full reporting of outcomes

☐ ☑ Estimates of effect sizes (e.g. Cohen's *d*, Pearson's *r*), indicating how they were calculated

*Our web collection on statistics for biologists contains articles on many of the points above.*

## Software and code

Policy information about [availability of computer code](#)

Data collection | ArcMap v. 10.7.1 was used to collate data.

Data analysis | Data analysis was conducted using R Studio version "Apricot Nasturtium". The following additional packages were used: MatchIt v. 4.1.0, plm v.2.2-5, lmertest v.3.1-3, dplyr v. 1.0.2, foreign v. 0.8-8, ggplot2 v.3.3.2, tidyr v. 1.1.2, xlm v. 0.6.4.2, foreach.

For manuscripts utilizing custom algorithms or software that are central to the research but not yet described in published literature, software must be made available to editors and reviewers. We strongly encourage code deposition in a community repository (e.g. GitHub). See the Nature Research [guidelines for submitting code & software](#) for further information.

## Data

Policy information about [availability of data](#)

All manuscripts must include a [data availability statement](#). This statement should provide the following information, where applicable:

- Accession codes, unique identifiers, or web links for publicly available datasets

- A list of figures that have associated raw data

- A description of any restrictions on data availability

We have both a data availability statement and a code availability statement:

Data availability statement

All input data used in this study are available in the GitHub repository accessible here: https://github.com/katie-devs/Biodiversity_offset_effectiveness.

Code availability statement

All computer code used in this study are available in the GitHub repository accessible here: https://github.com/katie-devs/Biodiversity_offset_effectiveness.
# Field-specific reporting

Please select the one below that is the best fit for your research. If you are not sure, read the appropriate sections before making your selection.

<figure><img src="image_10.png" /></figure>

Life sciences

<figure><img src="image_11.png" /></figure>

Behavioural & social sciences

<figure><img src="image_12.png" /></figure>

Ecological, evolutionary & environmental sciences

For a reference copy of the document with all sections, see [nature.com/documents/nr-reporting-summary-flat.pdf](https://nature.com/documents/nr-reporting-summary-flat.pdf)

# Ecological, evolutionary & environmental sciences study design

All studies must disclose on these points even when the disclosure is negative.

<table><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>Study description</td><td>To determine whether Ambatovy is on track to achieve No Net Loss of forest, we use statistical matching combined with difference-in-differences regressions and a fixed effects panel regression to estimate the amount of avoided deforestation achieved within the mine's four biodiversity offsets. Statistical matching was used to match a sub-sample of pixels from each biodiversity offset to an equal sample of pixels from the wider forested landscape with similar exposure to drivers of deforestation. Outcomes in this matched control sample represent counterfactual deforestation outcomes for the biodiversity offsets in the absence of protection. We used site-based difference-in-differences regressions to estimate the effect of protection on deforestation within each individual offset. This method compares the average change in outcomes before and after the intervention between treated and control samples to estimate the impact of an intervention. To estimate the effect of protection on deforestation across the entire offset portfolio, we pooled the data and ran a fixed effects panel regression with site and year fixed effects. We test the robustness of our results to a range of alternative, valid modelling choices in the process of statistical matching and show our results are robust.</td></tr><tr><td>Research sample</td><td>This consists of a sub-sample of pixels from each biodiversity offset defined as the treated sample and a sub-sample of pixels from the wider study area defined as the control sample. All sample pixels were forested at the baseline year 2000. Control pixels were sampled from outside the established protected area network (excluding one recently created protected area) and a 10km buffer zone around the biodiversity offsets. The total number of pixels sampled from the four offsets is 7,998 which represents approximately 4% of pixels which were forested in 2000. The full control sample comprised 634,465 pixels. Comparison of the total deforestation rate for the whole offset and the deforestation rate within the sub-sample of pixels indicates our sample pixels are representative of deforestation outcomes within each offset (results presented in SI).</td></tr><tr><td>Sampling strategy</td><td>We used a grid-based sampling strategy to obtain the sub-sample of pixels from the study area. This ensured a minimum distance between sample pixels to reduce spatial autocorrelation and equal coverage of the study area. First, a 150m x 150m resolution grid was overlaid on the study area (the former province of Toamasina) using the Create Fishnet tool in ArcMap v. 10.1. 150m was selected as it is 5x the resolution of the other data layers allowing each grid cell to be aligned with blocks of 5x5 cells in these layers. Then the point layer containing the centroid of each grid cell was converted to a raster of cell size 30m. This produced a new layer of sample pixels which were 120m away from their nearest neighbour and aligned with the other data layers. Extract by Mask was used to restrict the sample to only include pixels classed as forest in the Year 2000. The Clip tool was used to extract sample pixels within the boundaries of each biodiversity offset (to form the four treated samples) and the 10km buffer zone around each offset (for the leakage analysis). To produce the control sample, Mask in the Image Analysis toolbar was used to remove pixels from the sample layer that were within established protected areas or within 10km of a biodiversity offset.</td></tr><tr><td>Data collection</td><td>All raw data used in this study were secondary spatial data that are publicly available online. Data layers were compiled into a GIS using ArcMap version 10.7.1. All data were reprojected to WGS 1984 UTM Zone 38S. Some data were processed to produce a secondary output (eg. to calculate distance to the nearest feature). Covariate and outcome data values were sampled at sample pixel locations to produce the input data for the matching. Data was collated by K Devenish.</td></tr><tr><td>Timing and spatial scale</td><td>The study period is January 2001 to December 2019. This provides sufficient temporal coverage both before (at least 8 years) and after (at least 6 years) the offsets were protected to evaluate changes in outcomes. It also corresponds to the duration of annual deforestation data from the Global Forest Change dataset of Hansen et al (2013). The study is sub-national in scale with the study area comprising the former province of Toamasina (area = 71,744km2), but focussing specifically on Ambatovy and its offsets (area = 287.4km2).</td></tr><tr><td>Data exclusions</td><td>Control pixels that fall within 1) established protected areas or, 2) the 10km buffer zone around each offset were removed from the control sample prior to data analysis. Forests within established protected areas are under some form of conservation management and therefore do not represent counterfactual outcomes for the biodiversity offsets in the absence of any protection. Control pixels within 10km of a biodiversity offset were removed to minimise the potential for bias resulting from leakage, whereby deforestation displaced from the protected area elevates deforestation within the surrounding landscape. To determine whether there was any leakage we performed a secondary analysis using sample pixels from the buffer zones as the treated sample (this is reported in the SI). Finally, control pixels with covariate values that were well outside the range of the treated sample in any of the five essential covariates were removed prior to matching. Removing these control pixels, which would never have been matched, improved the speed and efficiency of matching which was particularly important when running the robustness checks.</td></tr><tr><td>Reproducibility</td><td>The code for the main matching specification was re-run several times by both KD and SD (using the same sample of pixels as the input data) and the same results were obtained each time. To test the robustness of our results to arbitrary modelling choices we repeated the analysis 116 times using 116 alternative matching model specifications, varying the combination of model parameters, matching distance measure and the inclusion of five additional covariates. We show that our results are robust to a range of alternative, valid model specifications. All input data and computer code used in this study are publicly available online on GitHub (link in the data availability statement) to enable reproduction.</td></tr><tr><td>Randomization</td><td>Allocation to treated and control groups was non-random. Statistical matching was used to control for differences in characteristics</td></tr></tbody></table>
Randomization

hypothesised to influence selection to treatment and the outcome of interest. Evaluation of the covariate balance post-matching showed matched treated and control samples to be, on average, well matched (standardised difference in mean covariate values < 0.25) across all covariates.

Blinding

Blinding was not relevant as our study did not involve human participants. We completed our matching before exploring outcome regressions following best practice.

Did the study involve field work?

<figure><img src="image_13.png" /></figure>

Yes

<figure><img src="image_14.png" /></figure>

No

Reporting for specific materials, systems and methods

We require information from authors about some types of materials, experimental systems and methods used in many studies. Here, indicate whether each material, system or method listed is relevant to your study. If you are not sure if a list item applies to your research, read the appropriate section before selecting a response.

Materials & experimental systems

<figure><img src="image_15.png" /></figure>

Methods

<figure><img src="image_16.png" /></figure>