# Ocean-related options for climate change mitigation and adaptation: A machine learning-based evidence map protocol

Devi Veytia, Laura Airoldi, Joachim Claudet, Sarah Cooley, Alexandre Magnan, Vicky Marti Barclay, Simon Neill, U. Rashid Sumaila, Olivier Thébaud, Christian R Voolstra, et al.

## ► To cite this version:

Devi Veytia, Laura Airoldi, Joachim Claudet, Sarah Cooley, Alexandre Magnan, et al. Ocean-related options for climate change mitigation and adaptation: A machine learning-based evidence map protocol. 2024. [hal-04760158](https://hal.science/hal-04760158v1)

HAL Id: hal-04760158

https://hal.science/hal-04760158v1

Preprint submitted on 30 Oct 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Ocean-related options for climate change mitigation and adaptation: A machine learning-based evidence map protocol

**Devi Veytia** (✉ deviveytia@fondationbiodiversite.fr)

FRB-CESAB, Fondation pour la Recherche sur la Biodiversité – Centre de Synthèse et d'Analyse sur la Biodiversité, 5 rue de l'école de médecine 34000 Montpellier, France https://orcid.org/0000-0001-6268-1508

**Laura Airoldi**

Chioggia Hydrobiological Station “Umberto D’Ancona”, Department of Biology, University of Padova, Uo CoNISMa, Chioggia, 30015, Italy; NBFC, National Biodiversity Future Center, Palermo 90133, Italy https://orcid.org/0000-0001-5046-0871

**Joachim Claudet**

National Center for Scientific Research; PSL Université Paris, CRIOBE, CNRS-EPHE-UPVD, Maison de l'Océan, 195 rue Saint-Jacques 75013 Paris, France https://orcid.org/0000-0001-6295-1061

**Sarah Cooley**

Ocean Conservancy, 1300 19th St. NW, Suite 800, Washington DC 20036 USA https://orcid.org/0000-0002-1034-0653

**Alexandre Magnan**

Institute for Sustainable Development and International Relations, Sciences Po, Paris, France https://orcid.org/0000-0001-7421-5184

**Vicky Marti Barclay**

Civil and Environmental Engineering, University of Strathclyde, Glasgow, UK https://orcid.org/0000-0001-9547-3377

**Simon Neill**

School of Ocean Sciences, Bangor University, Menai Bridge, UK https://orcid.org/0000-0002-1674-3445

**U. Rashid Sumaila**

Institute for the Oceans and Fisheries and School of Public Policy and Global Affairs, University of British Columbia, 2202 Main Mall, Vancouver, B.C., V6T 1Z4, Canada https://orcid.org/0000-0002-1851-1621

**Olivier Thébaud**

Ifremer, AMURE, Plouzané, 29280, France https://orcid.org/0000-0001-8665-3827

**Christian R. Voolstra**

Department of Biology, University of Konstanz, Konstanz, Germany https://orcid.org/0000-0003-4555-3795
**Phillip Williamson**

School of Environmental Sciences, University of East Anglia, Norwich NR4 7TJ, UK

https://orcid.org/0000-0003-4149-5110

**Marie Bonnin**

IRD, Univ Brest, CNRS, Ifremer, LEMAR, 29280 Plouzane, France https://orcid.org/0000-0003-4140-2439

**Joseph Langridge**

CESAB - FRB, 5 Rue de l'École de Médecine, 34000, Montpellier, France https://orcid.org/0000-0002-9427-439X

**Adrien Comte**

National Center for Scientific Research; PSL Université Paris, CRIOBE, CNRS-EPHE-UPVD, Maison de l'Océan, 195 rue Saint-Jacques 75013 Paris, France https://orcid.org/0000-0001-9895-0669

**Frédérique Viard**

ISEM, Univ Montpellier, CNRS, EPHE, IRD, Montpellier, France https://orcid.org/0000-0001-5603-9527

**Yunne Shin**

1 MARBEC, IRD, Univ Montpellier, IFREMER, CNRS, Montpellier, France

**Laurent Bopp**

Laboratoire de Météorologie Dynamique, Institut Pierre-Simon Laplace, CNRS, Ecole normale supérieure / PSL, - Sorbonne Université, Ecole Polytechnique, Paris, France https://orcid.org/0000-0003-4732-4953

**Jean-Pierre Gattuso**

Sorbonne University, CNRS, Laboratoire d'Océanographie de Villefranche, Villefranche-sur-Mer, France

**Method Article**

**Keywords:** global warming, greenhouse gas, mitigation, adaptation, ocean-based solution

**Posted Date:** February 1st, 2024

**DOI:** https://doi.org/10.21203/rs.3.pex-2324/v1

**License:** © This work is licensed under a Creative Commons Attribution 4.0 International License.

Read Full License
# Abstract

## Background

Ocean-related options (OROs) to mitigate and adapt to climate change are receiving increasing attention from practitioners, decision-makers, and researchers. In order to guide future ORO development and implementation, a catalogue of scientific evidence addressing outcomes related to different ORO types is critical. However, until now, such a synthesis has been hindered by the large size of the evidence base. Here, we detail a protocol using a machine learning-based approach to systematically map the extent and distribution of academic evidence relevant to the development, implementation, and outcomes of OROs.

## Method

To produce this systematic map, literature searches will be conducted in English across two bibliographic databases using a string of search terms relating to the ocean, climate change, and OROs. A sample of articles from the resulting de-duplicated corpus will be manually screened at the title and abstract level for inclusion or exclusion against a set of predefined eligibility criteria in order to select all relevant literature on marine and coastal socio-ecological systems, the type of ORO and its outcomes. Descriptive metadata on the type and location of intervention, study methodology, and outcomes will be coded from the included articles in the sample. This sample of screening and coding decisions will be used to train a machine learning model that will be used to estimate these labels for all the remaining unseen publications. The results will be reported in a narrative synthesis summarising key trends, knowledge gaps, and knowledge clusters.

# Introduction

## Background

As mean global warming above pre-industrial levels rapidly approaches the 1.5°C limit of warming set by the Paris Agreement¹,², the first global stocktake in 2023 assessed progress towards climate goals of parties to the UNFCCC. The stocktake determined that the rapid implementation and upscaling of climate actions using good practices that minimise tradeoffs and maximise co-benefits is needed³. Amidst this critical juncture, "ocean-based solutions" or "ocean solutions" are receiving increasing attention as attractive responses to climate change challenges. The marine environment already contributes to anthropogenic greenhouse gas reduction⁴,⁵, which motivates an interest to increase its potential for climate change mitigation using renewable energy and negative emissions technologies. Marine and coastal socio-ecological systems are also highly vulnerable to climate change impacts, and solutions that enhance the adaptive capacity of natural and human systems are needed to respond to these challenges⁶,⁷. Additionally, these ocean-related interventions seem less constrained by the same land-use limitations of terrestrial-based counterparts.
Yet this "blue rush" often overlooks a growing range of existing activities and uses in the marine space⁸,⁹, and is accompanied by a relatively poor understanding of the marine ecosystems' capacity to cope with these interventions¹⁰⁻¹³. We therefore elect to use the term "ocean-related options" (OROs; cf., Table 1, Figure 1) when referring to "ocean-based solutions" to acknowledge that not all interventions may be effective, and that side-effects, trade-offs and dis-benefits may exist or have yet to be assessed. While we may not advocate all interventions as "solutions", all must nonetheless be systematically assessed as they have been proposed as climate mitigation or adaptation actions.

For mitigating climate change, OROs encompass a diverse array of actions. Negative emissions technologies such as blue carbon-based approaches¹⁴ and geo-engineering interventions to increase the ocean's capacity as a carbon sink and actively draw down carbon dioxide from the atmosphere¹² are in varying stages of development, scientific evaluation, and social acceptance. Renewable energy sources derived from natural oceanic processes, such as waves, tides, and bioenergy from macroalgae, potentially provide alternatives to displace fossil fuel consumption and lower emissions¹⁰. Optimising the energetic efficiency of maritime industries like shipping and fishing contributes to the overall mitigation effort¹³. The development of these technologies is occurring against the backdrop of a "blue rush" driven by a desire to leverage the ocean space as a location for infrastructure¹⁵, such as offshore solar and wind farms and seabed storage reservoirs for captured carbon, or for carbon dioxide removal options (i.e. "blue acceleration" sensu ⁸).

OROs can also play a crucial role in promoting societal and natural adaptation to observed impacts and projected risks from climate change. Climate impact drivers such as rising sea levels, extreme weather, marine heatwaves, and ocean acidification pose significant, and even existential threats to coastal socio-ecological systems¹⁶. To encourage societal adaptation, OROs ranging from built infrastructure, nature-based solutions and socio-institutional changes, aim to address these challenges⁷,¹⁷,¹⁸. OROs can also strengthen the resilience of natural systems through conservation measures¹⁹ to more active restoration interventions such as eco-engineering, species translocations or assisted evolution²⁰⁻²². Often, OROs that promote natural resilience are implemented to achieve multiple co-benefits across the climate mitigation and/or societal adaptation outcomes (although whether they deliver on these outcomes must be rigorously assessed), and thus lie at the conceptual intersection between mitigation and societal adaptation OROs⁷,¹⁰,¹⁹,²³.

The unrelenting pace of climate change has given urgency to scientific research aiming to understand climate change impacts, vulnerabilities, and potential solutions, thus contributing to a rapid explosion of academic literature across the climate change sciences from 1,697 to 92,308 publications during the First
to Sixth Assessment Cycles of the Intergovernmental panel on Climate Change²⁴. Given the extensive volume of academic literature, machine learning-assisted evidence maps have emerged as indispensable tools for characterising the distribution of evidence relevant to topical research questions in climate change science²⁵,²⁶. Utilising these advancements in machine learning applications, this systematic map aims to assess the extent and distribution of academic evidence that reports mitigation and adaptation outcomes, as well as co-benefits, trade-offs and dis-benefits, relating to OROs. This analysis will help shape future research priorities, identify topics for future quantitative evidence synthesis, and ultimately inform evidence-based decision-making.

## Stakeholder engagement

This research project is funded within the framework of the French government Priority Research Program (PPR) "Ocean and Climate" (2021-2027), which is led by the National Centre for Scientific Research (CNRS) and Institut Français de Recherche pour l'Exploitation de la Mer (Ifremer). This study seeks to support decision makers in their assessment of options for climate change mitigation, adaptation and sustainable development.

A multidisciplinary international expert panel of researchers in relevant fields identified a primary research question and scope that was suitable to addressing this evidence need, developed a draft protocol of the evidence map, and will provide input on interpreting the insights from this map.

## Objective of the review

This evidence map aims to identify, map, and describe the evidence base relevant to mitigation and adaptation outcomes resulting from OROs, as well as secondary outcomes which include co-benefits and trade-offs/dis-benefits.

## Primary questions

In this study, we aim to address the following primary research question:

*What peer-reviewed scientific literature exists on the primary and secondary outcomes of mitigation and adaptation OROs?*
# Secondary questions

Using the resulting evidence base, we aim to answer the following set of secondary research questions:

* What are the temporal trends in the extent and distribution of articles across different types of OROs, ORO development stages, types of outcomes and scientific disciplines?

* To what extent do articles link blue carbon habitats with OROs?

* What are the extent and distribution of articles across ORO type, primary outcomes (i.e. mitigation or adaptation) and secondary outcomes?

* Which climate impact drivers are motivating the development of ocean-related adaptation options?

* How are primary and secondary outcomes linked to different study types and methodologies?

* How does the country of the lead author's affiliation compare with the country where research is conducted/where the ORO is implemented?

# Elements of the primary question

To delineate the scope of our literature search we define the key elements of our research question (Table 2).

## Reagents

## Equipment

## Procedure

# Procedure

## 0. Overview

The procedure is inspired by the Guidelines and Standards for Evidence Synthesis in Environmental Management27 and includes three main phases: (1) the development of an effective method for searching for literature through an iterative standardised process, (2) the development of eligibility criteria for an objective and transparent selection of relevant articles, and (3) the development of a metadata-coding framework (i.e. a pilot-tested spreadsheet) used to extract relevant metadata from retained articles to qualitatively characterise the evidence base. For phases 2 and 3, we adapt the guidelines to include
assistance from machine learning models following the method of 26 by manually screening and coding
a sample of articles to train machine learning classifiers in order to predict these classifications for the
entire corpus. Following 26, we will also use a geoparser to identify geographic coordinates and extents
from place names in the study titles and abstracts.

# 1. Searching for articles

## 1.1 Estimating comprehensiveness of search

To assess the search comprehensiveness, a "test list" of 94 benchmark articles was compiled through
contributions from the interdisciplinary synthesis team (Additional File 1).

## 1.2 Search terms and languages

In evidence synthesis protocols, search strings that are used to search for relevant literature are assessed and refined through an iterative process of scoping searches. These scoping searches first aim to identify search terms that maximise the retrieval of the test list articles (i.e. an assessment of search comprehensiveness), and second to minimise the number of total results retrieved (i.e. an assessment of search specificity)²⁷. In our synthesis, we carried out scoping searches in Web of Science Core Collection (WOS), using the language tag to limit the search results to English (LA = English). While we acknowledge the drawbacks of language bias²⁸, this methodological choice was necessary due to limitations in the expertise of our review team and availability of a pre-trained machine learning model in other languages.

Our research question encompasses OROs that cover a broad range of concepts. Given this diversity, we
determined that a single search string was ineffective at achieving full comprehensiveness with a
realistic level of sensitivity (Additional File 2, sheet "General"). To address this, we classified all OROs
considered in our research question into a typology modified from ¹¹, and based on the types of OROs
identified, we developed a search strategy composed of five search strings (termed "substrings") (Figure
1). Each substring would be run individually and the search results subsequently pooled and de-
duplicated to form a single database.
Each substring combined groups of search terms (i.e. "blocks") that matched critical components of our topic (Table 3):

* Population block: terms relevant to oceans and coastal human/natural systems

* Intervention block: This block varied highly across the substrings, and was often represented through a combination of the following blocks joined with an "AND" operator:

* Climate_change: terms relevant to climate change or ocean-related climate impact drivers

* Option: general terms to represent an option or intervention or solution

* Ocean_renewables: terms that specifically target marine renewable energies

* Mitigation_options: terms to represent types of mitigation options other than marine renewable energies

* Natural_resilience: terms relevant to natural resilience options

* Societal_adaptation: terms relevant to societal adaptation options

* No_terms: terms excluded because we do not consider them within the scope of our review (e.g. reducing pollution, articles which only focus on reporting climate change impacts/vulnerability).

Each substring was tuned using iterative scoping searches, with the end goal that when the search results from all the substrings are pooled, 100% comprehensiveness would be achieved. When all the test list articles were retrieved, the number of records across all five substrings (before deduplication) totalled 392,111. This was an acceptable level of specificity for our machine-learning assisted approach (Table 3, Additional File 2). We therefore determined that the substrings were suitable for conducting the synthesis.

## 1.3 Searching the literature

Our machine learning-assisted approach requires access to title and abstract metadata exported for all search results in order to fit its classifiers. At the scale of our research question, this is best achieved using citation-indexed databases. We will conduct our searches in Web of Science Core Collection and Scopus (Additional Files 2 and 3).

## 1.4 Reference management

References from the searches will be exported from the citation-indexed databases as BibTeX files. All citation files will be read into the R statistical computing environment²⁹ and deduplicated using the revtools package³⁰. Duplicated records will be identified based on title and year matches. Titles will be pre-processed by converting to lowercase and removing punctuation, and the resulting titles with an
optimal string alignment distance of <= 5 that are published in the same year will be tagged as duplicates. Since our machine learning model relies on titles and abstracts to screen and code articles, entries with an "NA" value in either field will be removed; the most complete record will be retained.

# 2. Article screening and study eligibility criteria

## 2.1 Screening process

The eligibility screening step of an evidence map involves the application of eligibility criteria that determine which of the primary research studies identified in searches are relevant for answering the map question<sup>27</sup>. Our search queries from section 1 returned 265,436 unique articles. We therefore will use a machine learning-assisted approach to automate the evidence selection process. We will train a machine learning classification model based on a subset of manually screened article text (title, abstract and keywords) by a team of reviewers to predict inclusion with quantified uncertainty<sup>26</sup>. Manual screening decisions will be based on transparent and replicable eligibility criteria (cf. Section 2.2).

### Manually-screened sample

First, a manually-screened sample of articles is needed to train and test the machine learning classification model. Since the number of publications retrieved from each search string was unbalanced (Table 4), sampling randomly from the pool of all search results would likely return a subset that was biased towards the ORO types which retrieved the most search results. Since the manually-screened sample would be used to train and validate the model, we did not want this to bias the model's performance. Therefore, to achieve a more balanced representation of ORO types, we will select 4,000 articles for manual screening from the deduplicated search results (excluding test list articles) using the following procedure: 1000 articles randomly sampled from the results of the "General" search string, 1000 random articles from the two mitigation search strings (333 from "Mitigation: renewable energy" and 666 from "Mitigation: other"), 1000 random articles from the "Natural resilience" string, and 1000 from the "Societal adaptation" string. These articles along with the test list articles (4,094 articles in total) will be uploaded into sysrev (www.sysrev.com): a web-based platform for publication screening.
Reviewer training aims to reduce bias in the manual screening process. During this process, each reviewer will independently screen a randomly-selected training set of titles and abstracts (n=30), and their inclusion decisions will be used to calculate a Fleiss' Kappa statistic³¹,³² to assess inter-reviewer reliability. If the kappa statistic is below 0.6, a group discussion will be conducted to resolve discrepancies and modify/clarify the eligibility criteria until a consensus is reached. A new training set (n=30) will then be screened, and this process repeated until an acceptable Fleiss' Kappa statistic of ≥ 0.6 is achieved. Any remaining disagreements will be discussed before commencing the screening.

After the reviewer training, reviewers will proceed to screen the sample articles. Articles will be screened by a single reviewer, except for 300 randomly-selected articles flagged for double-blind screening. Any conflicts in terms of screening decisions will be resolved by a third reviewer. We acknowledge that the use of a single reviewer to screen articles is a limitation of this study, but was necessary given the size of the evidence base. Reviewers will not screen articles that they have authored. We will also use an active learning strategy to screen articles. The titles and abstracts of articles will first be screened by the screening team in random order. Since our aim is to produce a sample of both inclusion and exclusion labels to train the machine learning classifier effectively, sufficient examples of each classification are needed. To mitigate the high proportion of irrelevant articles often found in systematic evidence syntheses, we draw from the active learning approach from ²⁶, which sorts articles for screening by predicted relevance from a periodically-updating machine learning model embedded in their screening platform. For our study, we will begin the active learning process after at least 1500 articles have been screened in random order, and will use the predicted relevance from the machine learning model embedded in sysrev to sort the articles for screening. All articles with a predicted relevance above > 60% will be screened until no articles above that relevance threshold remain.

## Machine learning predictions

This manually-screened sample will be used to train and test a binary machine learning classifier used to predict the inclusion or exclusion of all the articles retrieved by the search strategy²⁶. In this approach, a nested cross-validation (CV) procedure is used to compare the performance of a support vector machine (SVM)³³ and a pretrained DistILBERT model34 which has been fine-tuned with the manually-screened sample.

To explain the role of CV in machine learning, data is needed to train (i.e. fit) a model, as well as to evaluate a model's performance. If the same data is used for both steps, the model's performance metrics may be inflated due to overfitting. K-fold CV aims to address this issue by splitting the dataset into k non-overlapping folds so that the data used for training is not used in evaluation. Over k loops, each fold is
reserved in turn as a test set while the remaining data is pooled and used to train a model, producing *k* candidate models.

Two nested loops of *k*-fold CV can be implemented to perform two essential functions: (1) finding the best model configuration by optimising the selection of hyperparameters in the inner loop, and (2) model evaluation in the outer loop. In the outer loop, the dataset is first divided into *k* folds, where one fold is reserved as a test set and the remaining *k*-1 folds as the training set. In this study, only randomly-selected articles are included in the test sets (as opposed to those screened using predicted relevance in the active learning procedure) so as not to bias the model evaluation. The remaining randomly-selected articles that are not included in the outer test set are incorporated into the training set. In the inner loop, a further *k* testing sets are drawn from the remaining randomly-selected articles in the outer loop's training set, whilst all remaining articles are allocated to the inner training set. Grid search is used to initialise a model with each combination of hyperparameters and fit the model on each inner training and testing set. The combination of hyperparameters with the best mean F1 score (the harmonic mean of precision and recall) across the inner folds will be selected as the best model. The training and testing data from the outer CV is then used to evaluate the performance of the best model from the inner folds. The outer CV therefore returns *k* scores for each evaluation metric.

The binary inclusion/exclusion classifiers (SVM and DistilBERT), will be evaluated across *k*=5 inner and outer folds and the performance of the two types of classifiers will be compared using F1 and ROC AUC scores. When the most appropriate classifier is identified, the final model configuration will be chosen using outer CV, where each combination of hyperparameter settings will be tested on each outer fold. The model configuration that yields the highest F1 score will be selected and used for predictions. A confidence interval (mean +/- 1 standard deviation) for each prediction will be generated using five versions of the model trained on five folds of data. All articles with a predicted mean relevance greater or equal to 0.5 will be included.

## 2.2 Eligibility criteria

We developed eligibility criteria to screen for literature that includes the necessary elements of our primary research question (Additional File 4). The inclusion criteria are outlined below.

### Relevant population(s)

We focus on marine and coastal ecosystems across all biomes. These include the open ocean (including all water column layers from the surface to deep-ocean), seas and gulfs, and natural coastal marine
ecosystems (i.e. all intertidal habitats), and artificial habitats (e.g. urban waterfronts). We also focus on
associated coastal human systems (e.g. coastal communities, fisheries).

**Relevant intervention(s) or exposure(s)**

We include literature on OROs (see Table 1 and Figure 1) at any developmental stage from design to implementation, including exposure to natural ORO analogues as part of the "design" developmental stage. We also include natural inventory studies measuring natural carbon fluxes/storage in blue carbon ecosystems, as these studies provide an indirect measure of conservation OROs (i.e. the management or mitigation of anthropogenic impacts and stressors). We do not consider studies without a direct link to an ORO (e.g. documenting climate change impacts, or indirect actions aiming to ameliorate the distribution of wealth to reduce climate change vulnerability), nor do we consider actions aiming to reduce anthropogenic stressors (e.g. reducing pollution in river runoff, habitat loss through conversion), without a direct link to climate change. We recognize that these indirect factors will likely interact with OROs to play an important role in their current and continued outcomes, however due to resource constraints we do not consider them here.

**Relevant comparator(s)**

No comparator is required.

**Relevant outcome(s)**

We include literature where the outcome is directly relevant to primary (i.e. mitigation or adaptation) or
secondary (i.e. co-benefits, trade-offs/dis-benefits) outcomes attributed to an ORO. Relevant primary
outcomes should be directly related to the effectiveness of the ORO at achieving its primary aim of
mitigation and/or adaptation. These outcomes can assess effectiveness directly (e.g. amount of energy
produced by a renewable energy technology, changes in biodiversity/ecosystem recovery), or across
dimensions which modify effectiveness (e.g. lead time, cost, feasibility, governability, upscaling potential,
enabling conditions). Relevant secondary outcomes can include co-benefits (e.g. increases in biodiversity
across all its components, from infra-specific to ecosystem diversity, associated natural and wilderness
values, synergies with other OROs) or trade-offs/side effects/dis-benefits (e.g. displacing communities,
reducing population abundance, negative impacts on other industrial sectors, etc). Secondary outcomes
can be unintentional or intentional. We include all study methodologies, including primary research (e.g.
observational, experimental or modelled quantitative or qualitative outcomes, or expert opinion) or
evidence syntheses.
We exclude indirectly relevant outcomes such as technological advancements that are not explicitly implemented within the framework/technology of an ORO (e.g. electric battery advancements that are indirectly relevant for increasing the impact/usefulness of marine renewable energies).

## 2.3 Study validity assessment

No formal validity appraisal of included articles/studies will be performed. All studies that are deemed eligible at the title and abstract screening based on the eligibility criteria will be included in the evidence map.

## 3. Metadata coding

### Manually-coded sample

Once articles have been screened and determined eligible, metadata will be coded from each article in order to characterise the distribution of the evidence base. We anticipate given the large size of the search results, that even after screening is performed the volume of articles will be too large to code manually. We therefore will continue to implement the machine learning-assisted approach²⁶.

To generate the data of coding decisions used to train and test the machine learning classifiers, the articles that were included from the manually-screened sample will be manually coded. All reviewers will code metadata using a standardised data codebook (Additional Files 5 and 6). To reduce inter-reviewer bias, each reviewer will use the codebook instructions and template to code the same randomly selected set of training articles. Any discrepancies will be discussed and resolved between coders, and any modifications to the codebook made for improved clarity. This process will be repeated until members are able to code articles with consistent agreement. Hereafter, articles will be coded by a single reviewer, except for 2 randomly selected articles out of every 50 coded that will be double-checked by a second reviewer and any conflicts discussed and resolved amongst the team. While it is unknown *a priori* which metadata variables will be accurately predicted by the machine learning model, metadata will be extracted across the following information categories relevant to our primary and secondary research questions: bibliographic, ORO description, study design and outcome types. If, after coding, any essential
variable labels are under-represented, supplemental non-random screening and coding will be done by searching for relevant keyword matches within the corpus and screening and coding the returned articles until a more balanced representation is reached. These additional screening decisions would then be added to the initial manually-screened sample to update the screening model.

## Machine learning predictions

To generate predictions for the coded variables, either a single-label or multi-label (depending on the variable) machine learning classifier will be trained and tested on the manually coded subset. Since the DistILBERT model has already been shown to out-perform an SVM in a similar field²⁶, if the DistILBERT model performs similarly in this study's binary inclusion/exclusion classifier, we will focus on the model configuration CV procedure of only the DistILBERT model for the predictions of the coded variables. In our study, we aim to code a large number of variables, and we therefore take this necessary methodological choice due to computational demands. For selecting the final DistILBERT model for each variable, we will use a k=3 fold CV and evaluate models based on their macro F1 score (where each label is weighted with equal importance). Unlike the binary classifier, since all the coded articles represent inclusions, they are likely representative of the dataset and are eligible for inclusion in the test sets used in the CV procedure. All articles with a predicted mean relevance greater or equal to 0.5 for a given label will be determined relevant for that label. For the variable indicating the type of ORO, if an article is predicted as included but no type of ORO is identified using the classifier, a label of "Other" will be assigned.

## Geoparser for location extraction

To extract location data about each article's population element, we will follow the approach of ²⁶ to extract the geographical entity names from each article's text (title, abstract and keywords) using the geoparser Mordecai. By using these geographical entity names to query the Natural Earth database³⁵, we will be able to identify the geographical entity of the smallest resolution and its corresponding spatial extent.

# 4. Study mapping and presentation

The R statistical computing environment²⁹,³⁶ will be used to plot the distribution of the evidence base. The distribution of evidence across the scientific discipline, intervention, study type and outcome
categories will be explored through frequency plots, heatmaps visualising the intersection between
multiple factors, geographical maps, and barplots of the number of citations published in a category over
time.

These plots will be used to characterise trends in academic evidence, and inform future research priorities for OROs. Across the different types of OROs, we will explore publishing trends across the different developmental stages from design to implementation, and identify knowledge gaps such as under-represented scientific disciplines, unexplored outcomes or under-researched ecosystems and marine systems. We also aim to identify knowledge clusters with sufficient information for subsequent systematic reviews, which will be informed by the distribution of citations across the intervention and outcome variables, and the proportion of these citations that represent existing systematic evidence synthesis.

The findings from this map will be summarised through a narrative report accompanied by relevant tables and figures. All associated code and materials will be uploaded into a Github repository which will be made publicly available upon the article's publication.

# Troubleshooting

## Time Taken

## Anticipated Results

## References

1. Gillett, N. P. et al. Constraining human contributions to observed warming since the pre-industrial period. *Nat. Clim. Change* **11**, 207–212 (2021).

2. IPCC. Summary for policymakers. In V. Masson-Delmotte, P. Zhai, A. Pirani, S. L. Connors, C. Péan, S. Berger, N. Caud, Y. Chen, L. Goldfarb, M. I. Gomis, M. Huang, K. Leitzell, E. Lonnoy, J. B. R. Matthews, T. K. Maycock, T. Waterfield, O. Yelekçi, R. Yu, & B. Zhou (Eds.). (2021).

3. UNFCCC Secretariat. Technical dialogue of the first global stocktake. Synthesis report by the co-facilitators on the technical dialogue. https://unfccc.int/documents/631600 (2023).
4. Duarte, C. M., Middelburg, J. J. & Caraco, N. Major role of marine vegetation on the oceanic carbon cycle. *Biogeosciences* **2**, 1–8 (2005).

5. Gruber, N. et al. The oceanic sink for anthropogenic CO2 from 1994 to 2007. *Science* **363**, 1193–1199 (2019).

6. Cabana, D., Rölfer, L., Evadzi, P. & Celliers, L. Enabling Climate Change Adaptation in Coastal Systems: A Systematic Literature Review. *Earths Future* **11**, e2023EF003713 (2023).

7. Cooley, S. et al. Ocean and Coastal Ecosystems and their Services. In: Climate Change 2022: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change [H.-O. Pörtner, D.C. Roberts, M. Tignor, E.S. Poloczanska, K. Mintenbeck, A. Alegría, M. Craig, S. Langsdorf, S. Löschke, V. Möller, A. Okem, B. Rama (eds.)]. *Camb. Univ. Press Camb. UK N. Y. NY USA* 379–550 (2022) doi:10.1017/9781009325844.005.

8. Jouffray, J.-B., Blasiak, R., Norström, A. V., Österblom, H. & Nyström, M. The Blue Acceleration: The Trajectory of Human Expansion into the Ocean. *One Earth* **2**, 43–54 (2020).

9. Link, J. S. et al. Keeping Humans in the Ecosystem. *ICES J. Mar. Sci.* **74**, 1947–1956 (2017).

10. Gattuso, J.-P., Williamson, P., Duarte, C. M. & Magnan, A. K. The Potential for Ocean-Based Climate Action: Negative Emissions Technologies and Beyond. *Front. Clim.* **2**, 575716 (2021).

11. Gattuso, J.-P. et al. Ocean Solutions to Address Climate Change and Its Effects on Marine Ecosystems. *Front. Mar. Sci.* **5**, 337 (2018).

12. GESAMP. *High Level Review of a Wide Range of Proposed Marine Geoengineering Techniques* (Boyd, P.W. and Vivian, C.M.G., eds.). 144 p http://www.gesamp.org/publications/high-level-review-of-a-wide-range-of-proposed-marine-geoengineering-techniques (2019).
13. Hoegh-Guldberg, O. et al. *The Ocean as a Solution to Climate Change: Five Opportunities for Action*. https://live-oceanpanel-wp.pantheonsite.io/publication/the-ocean-as-a-solution-to-climate-change-five-opportunities-for-action/ (2019).

14. Macreadie, P. I. et al. Operationalizing marketable blue carbon. *One Earth* 5, 485–492 (2022).

15. Bugnot, A. B. et al. Current and projected global extent of marine built structures. *Nat. Sustain.* 4, 33–41 (2021).

16. IPCC. Technical Summary [H.-O. Pörtner, D.C. Roberts, V. Masson-Delmotte, P. Zhai, E. Poloczanska, K. Mintenbeck, M. Tignor, A. Alegría, M. Nicolai, A. Okem, J. Petzold, B. Rama, N.M. Weyer (eds.)]. in *IPCC Special Report on the Ocean and Cryosphere in a Changing Climate* [H.- O. Pörtner, D.C. Roberts, V. Masson-Delmotte, P. Zhai, M. Tignor, E. Poloczanska, K. Mintenbeck, A. Alegría, M. Nicolai, A. Okem, J. Petzold, B. Rama, N.M. Weyer (eds.)] 39–69 (Cambridge University Press, Cambridge, UK and New York, NY, USA, 2019). doi:10.1017/9781009157964.

17. Comte, A. & Pendleton, L. H. Management strategies for coral reefs and people under global environmental change: 25 years of scientific research. *J. Environ. Manage.* 209, 462–474 (2018).

18. Saleh, F. & Weinstein, M. P. The role of nature-based infrastructure (NBI) in coastal resiliency planning: A literature review. *J. Environ. Manage.* 183, 1088–1098 (2016).

19. Jacquemont, J., Blasiak, R., Le Cam, C., Le Gouellec, M. & Claudet, J. Ocean conservation boosts climate change mitigation and adaptation. *One Earth* 5, 1126–1138 (2022).

20. Airoldi, L. et al. Emerging Solutions to Return Nature to the Urban Ocean. *Annu. Rev. Mar. Sci.* 13, 445–477 (2021).
21. Bindoff, N. L. et al. Changing Ocean, Marine Ecosystems, and Dependent Communities. in *IPCC Special Report on the Ocean and Cryosphere in a Changing Climate* [H.-O. Pörtner, D.C. Roberts, V. Masson-Delmotte, P. Zhai, M. Tignor, E. Poloczanska, K. Mintenbeck, A. Alegría, M. Nicolai, A. Okem, J. Petzold, B. Rama, N.M. Weyer (eds.)](2019).

22. DeFilippo, L. B. et al. Assessing the potential for demographic restoration and assisted evolution to build climate resilience in coral reefs. *Ecol. Appl.* **32**, e2650 (2022).

23. Shin, Y. et al. Actions to halt biodiversity loss generally benefit the climate. *Glob. Change Biol.* **28**, 2846–2874 (2022).

24. Minx, J. C., Callaghan, M., Lamb, W. F., Garard, J. & Edenhofer, O. Learning about climate change solutions in the IPCC and beyond. *Environ. Sci. Policy* **77**, 252–259 (2017).

25. Berrang-Ford, L. et al. A systematic global stocktake of evidence on human adaptation to climate change. *Nat. Clim. Change* **11**, 989–1000 (2021).

26. Callaghan, M. et al. Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies. *Nat. Clim. Change* **11**, 966–972 (2021).

27. Collaboration for Environmental Evidence. *Guidelines and Standards for Evidence synthesis in Environmental Management*. www/environmentalevidence.org/information-for-authors (2022).

28. Amano, T. et al. Tapping into non-English-language science for the conservation of global biodiversity. *PLOS Biol.* **19**, e3001296 (2021).

29. R Core Team. R: A Language and Environment for Statistical Computing. (2022).

30. Westgate, M. revtools: An R package to support article screening for evidence synthesis. *Res. Synth. Methods* **10**, 606–614 (2019).
31. Conger, A. J. Integration and generalization of kappas for multiple raters. *Psychol. Bull.* **88**, 322–328 (1980).

32. Fleiss, J. L. Measuring nominal scale agreement among many raters. *Psychol. Bull.* **76**, 378–382 (1971).

33. Chang, C.-C. & Lin, C.-J. LIBSVM: a library for support vector machines. *ACM Trans. Intell. Syst. Technol. TIST* **2**, 1–27 (2011).

34. Sanh, V., Debut, L., Chaumond, J. & Wolf, T. DistILBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. Preprint at https://doi.org/10.48550/arXiv.1910.01108 (2020).

35. Natural Earth integrated vector map dataset (https://github.com/nvkelso/natural-earth-vector/tree/master).

36. RStudio Team. RStudio: Integrated Development Environment for R. (2022).

## Acknowledgements

## Declarations

## Availability of data and materials

All data generated or analysed during this study will be included in the published article [and its supplementary information files].
**Competing interests**

The authors declare that they have no competing interests.

**Funding**

This research project is funded within the framework of the French government Priority Research Program (PPR) "Ocean and Climate" (2021-2027), which is led by the National Center for Scientific Research (CNRS) and Institut Français de Recherche pour l'Exploitation de la Mer (Ifremer).

**Authors' contributions**

All authors contributed to the development of the primary and secondary research questions, search string, and ORO typology refinement. DV wrote the initial draft of the protocol and all authors contributed to its revision. The motivation and main objectives for the project were conceived by JPG, LB, YS, FV, and MB. Along with JL, these authors worked closely with DV in the development of the protocol.

**Acknowledgements**

We acknowledge the value of the training provided by the Centre for the Synthesis and Analysis of Biodiversity on the systematic evidence protocol and natural language processing methods provided by Nicolas Casajus. The postdoctoral project of D.V. and the expert panel meetings were funded by the PPR Oceans & Climate project.

**Figures**
<figure><img src="image_2.png" /><figcaption>**Figure 1. Ocean-related option (ORO) typology.** The typology is inspired by <sup>11</sup> and specifies three hierarchical levels or ORO types. The first level at the centre of the figure encompasses all OROs considered in this study. These are classified in the second level into three main types based on their mechanisms of mitigating/adapting to climate change (from left to right): to mitigate climate change by reducing atmospheric greenhouse gases; to improve the resilience of natural systems; and to improve societal adaptation. We add a third level specifying ORO types which are unique to different types of interventions included within each mechanism. The OROs represented in a given search string are encompassed in light-green dashed lines, which are indexed by numbers corresponding to the search string name in the figure legend.</figcaption></figure>

<caption>**Figure 1**</caption>

**Figure 1. Ocean-related option (ORO) typology.** The typology is inspired by <sup>11</sup> and specifies three hierarchical levels or ORO types. The first level at the centre of the figure encompasses all OROs considered in this study. These are classified in the second level into three main types based on their mechanisms of mitigating/adapting to climate change (from left to right): to mitigate climate change by reducing atmospheric greenhouse gases; to improve the resilience of natural systems; and to improve societal adaptation. We add a third level specifying ORO types which are unique to different types of interventions included within each mechanism. The OROs represented in a given search string are encompassed in light-green dashed lines, which are indexed by numbers corresponding to the search string name in the figure legend.
<table><caption>Table 1. Definitions of commonly-used terms.</caption><thead><tr><td>Term</td><td>Definition</td></tr></thead><tbody><tr><td>Mitigation</td><td>Mitigating climate change by reducing greenhouse gas emissions or enhancing sinks<sup>1</sup>.</td></tr><tr><td>Ocean-related climate impact driver</td><td>Trends or extremes in ocean-related climate variables (e.g. ocean temperature) or metrics (e.g. timing of sea ice advance) and/or the direct impacts of these changes on human (e.g. increased coastal hazards) or natural systems (e.g. species invasions or harmful algal blooms driven by increasing temperature).</td></tr><tr><td>Adaptation</td><td>We modify the definition of adaptation used in<sup>1</sup> as the process of adjustment of ecosystems or social systems to actual or expected ocean-related climate impact drivers – to increase ecosystems’ resilience and decrease social vulnerability. The potential for adaptation also includes adaptive capacity, i.e. the ability of a system to adapt. Importantly “adaptation” in this context does not refer to “adaptation” as used in evolutionary biology (i.e., the process by which populations evolve towards higher fitness in response to natural selection).</td></tr><tr><td>Natural systems</td><td>The “dynamic physical and biological components of the environment that would operate in the absence of human impacts”<sup>1</sup>.</td></tr><tr><td>Human systems</td><td>“Any system in which human organisations and institutions play a major role. Often, but not always, the term is synonymous with society or social system. [This includes] systems such as agricultural systems, urban systems, political systems, technological systems and economic systems”<sup>1</sup>.</td></tr><tr><td>Ocean-based option (ORO)</td><td>An ORO is defined as a human action (or control/removal thereof) that directly aims to:<br>1. use oceanic systems to mitigate climate change (this includes hybrid options partly based on land and in the ocean, as well as actions to protect, rehabilitate or enhance blue carbon habitats); or<br>2. enable the adaptation of natural and/or human systems to ocean-related climate impact drivers.</td></tr><tr><td>Primary outcome</td><td>An outcome resulting from an ORO that fulfils the ORO’s primary objective of mitigation and/or adaptation. This includes evaluations of dimensions that modify the ORO’s effectiveness/feasibility to achieve its primary outcome(s), including: scalability, robustness to uncertainty, duration, readiness, lead time, cost, enabling factors, and governance.</td></tr><tr><td>Secondary outcome</td><td>An outcome linked to an ORO that is not a part of the ORO’s primary objective. Secondary outcomes that increase the overall benefit of an option can be termed co-benefits (e.g. restoration of blue carbon ecosystems for mitigation can also conserve biodiversity), or trade-offs/dis-benefits (e.g. deterioration of natural habitats due to coastal protection infrastructure) if they are negative effects resulting from an option aimed at producing a different positive effect<sup>1</sup>. A secondary outcome can affect human or natural systems, or the functioning of other OROs.</td></tr><tr><td>Natural analogue</td><td>A naturally-occurring phenomenon that possesses key characteristics that resemble an ORO, ideally occurring at a scale that corresponds to the desired deployment scale of the ORO (e.g. sargassum blooms as natural analogues for ocean afforestation in<sup>2</sup>). Natural analogues may provide an important bridge between small-scale experiments and modelling simulations for assessing the effectiveness and impacts of ORO at relevant scales and in a real-world complex system<sup>2</sup>.</td></tr></tbody></table>

<figure><img src="image_3.png" /><figcaption>Table 1. Definitions of commonly-used terms.</figcaption></figure>
<table><caption>**Table 2. Definitions of primary research question elements.** For more in-depth definitions on specific terms which are underlined, cf. Table 1.</caption><thead><tr><td>Question Element</td><td>Element Definition</td></tr></thead><tbody><tr><td>Population</td><td>All marine/coastal ecosystems, including:<ul><li>coastal and intertidal habitats (e.g. estuaries, saline coastal wetlands, mangroves)</li><li>pelagic & benthic zones across all biomes</li><li>seas, gulfs, and oceans</li><li>artificial habitats such as urban waterfronts, ports, etc</li></ul>or coastal human systems that interact with the ocean and can be impacted by ocean-related climate impact drivers (e.g. coastal communities, fisheries, etc).</td></tr><tr><td>Intervention</td><td>Ocean-based options (OROs) across all design and development stages from observing relevant characteristics of natural/human systems to implemented/deployed, including natural analogues to OROs (cf. Table 1 for examples).</td></tr><tr><td>Outcome</td><td>We consider both primary and secondary outcomes linked to an ORO (cf. Table 1 for examples).</td></tr></tbody></table>

**Figure 3**

**Table 2. Definitions of primary research question elements.** For more in-depth definitions on specific terms which are underlined, cf. Table 1.
<table><caption>**Table 3. Summary of the comprehensiveness and specificity of each substring.** For the full data supporting this summary, cf. Additional File 2. The total number of test list articles is 94.</caption><thead><tr><td> Substring </td><td> Structure of string used for topic search* </td><td> N test list articles </td><td> N total records </td></tr></thead><tbody><tr><td> General </td><td> TS = Population AND Climate_change AND Option NOT No_terms </td><td> 29 </td><td> 14,069 </td></tr><tr><td> Mitigation: Renewable energy </td><td> TS = Population AND Ocean_renewable NOT No_terms </td><td> 18 </td><td> 42,677 </td></tr><tr><td> Mitigation: Other </td><td> TS = Population AND Climate_change AND Mitigation_options NOT Nos </td><td> 35 </td><td> 47,245 </td></tr><tr><td> Natural resilience </td><td> TS = Population AND Climate_change AND Natural_resilience NOT No_terms </td><td> 65 </td><td> 128,467 </td></tr><tr><td> Societal adaptation </td><td> TS = Population AND Climate_change AND Societal_adaptation AND Option NOT No_terms </td><td> 37 </td><td> 18,673 </td></tr><tr><td> Total (all substrings pooled) </td><td> NA </td><td> 94 </td><td> 392,111 </td></tr></tbody></table>

Figure 4

**Table 3. Summary of the comprehensiveness and specificity of each substring.** For the full data supporting this summary, cf. Additional File 2. The total number of test list articles is 94.

<footnote>*The term LA=English was joined with the topic search using an "AND" operator, indicating to only include records written in English</footnote>
<table><caption>**Table 4. Numbers of articles retrieved from each search string and bibliographic source database.** For a tabulation of the number of duplicates between strings, see sheet "Co-mentions" in Additional File 2.</caption><thead><tr><th rowspan="2">Search string</th><th colspan="2">Database</th></tr><tr><th>Web of Science</th><th>Scopus</th></tr></thead><tbody><tr><td>General options</td><td>14,069</td><td>221</td></tr><tr><td>Mitigation: Renewable energy</td><td>42,677</td><td>83,197</td></tr><tr><td>Mitigation: Other</td><td>47,245</td><td>55,607</td></tr><tr><td>Natural resilience</td><td>128,492</td><td>1,442</td></tr><tr><td>Societal adaptation</td><td>18,673</td><td>488</td></tr></tbody></table>

**Figure 5**

**Table 4. Numbers of articles retrieved from each search string and bibliographic source database.** For a tabulation of the number of duplicates between strings, see sheet "Co-mentions" in Additional File 2.

**Supplementary Files**
This is a list of supplementary files associated with this preprint. Click to download.

* AdditionalFile3Scopusequationblocks.docx

* AdditionalFile4Eligibilitycriteria.docx

* AdditionalFile5Codebookinstructions.xlsx

* AdditionalFile6Codebooktemplate.xlsm

* AdditionalFile7ROSESforSystematicMapProtocols.xlsx

* AdditionalFile0Descriptionofadditionalfiles.docx

* AdditionalFile1testlist.xlsx

* AdditionalFile2searchstringblocks.xlsx