# Ten years of ecosystem services matrix: Review of a (r)evolution

C. Sylvie Campagne, Philip K Roche, Felix Müller, Benjamin Burkhard

<figure><img src="image_1.png" /></figure>

To cite this version:

C. Sylvie Campagne, Philip K Roche, Felix Müller, Benjamin Burkhard. Ten years of ecosystem services matrix: Review of a (r)evolution. One Ecosystem, 2020, 5, 10.3897/oneeco.5.e51103. hal-04228140

HAL Id: hal-04228140

https://hal.inrae.fr/hal-04228140

Submitted on 4 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# Ten years of ecosystem services matrix: Review of a (r)evolution

C. Sylvie Campagne<sup>‡</sup>, Philip Roche<sup>§</sup>, Felix Müller<sup>l</sup>, Benjamin Burkhard<sup>¶,#</sup>

‡ Institute of Physical Geography and Landscape Ecology, Leibniz Universität Hannover, Hannover, Germany

§ INRAE, RECOVER unit, Aix-en-Provence, France

| Christian-Albrechts-University of Kiel, Institute for Natural Resource Conservation, Dept. Ecosystem Management, 24118 Kiel, Germany

¶ Leibniz Universität Hannover, Hannover, Germany

# Leibniz Centre for Agricultural Landscape Research ZALF, Müncheberg, Germany

Corresponding author: C. Sylvie Campagne (sylviecampagne@gmail.com)

Academic editor: Sander Jacobs

Received: 13 Feb 2020 | Accepted: 28 Apr 2020 | Published: 30 Apr 2020

Citation: Campagne CS, Roche P, Müller F. Burkhard B (2020) Ten years of ecosystem services matrix: Review of a (r)evolution. One Ecosystem 5: e51103. https://doi.org/10.3897/oneeco.5.e51103

## Abstract

With the Ecosystem Service (ES) concept's popularisation, the need for robust and practical methodologies for ES assessments has increased. The ES matrix approach, linking ecosystem types or other geospatial units with ES in easy-to-apply lookup tables, was first developed ten years ago and, since then, has been broadly used. Whereas detailed methodological guidelines can be found in literature, the ES matrix approach seems to be often used in a quick (and maybe even "quick and dirty") way. Based on a review of scientific publications, in which the ES matrix approach was used, we present the diversity of application contexts, highlight trends of uses and propose future recommendations for improved applications of the ES matrix.

A total of 109 studies applying the ES matrix approach and one methodological study without concrete applications were considered for the review. Amongst the main patterns observed, the ES matrix approach allows the assessment of a higher number of ES than other ES assessment methods. ES can be jointly assessed with indicators for ecosystem condition and biodiversity in the ES matrix. Although the ES matrix allows us to consider many data sources to achieve the assessment scores for the individual ES, in the reviewed studies, these were mainly used together with expert-based scoring (73%) and/or ES
scores that were based on an already-published ES matrix or deduced by information
found in related scientific publications (51%). We must acknowledge that 27% of the
studies did not clearly explain their methodology. This points out a lack of method
elucidation on how the data had been used and where the scores came from. Although
some studies addressed the need to consider variabilities and uncertainties in ES
assessments, only a minority of studies (15%) did so. Our review shows that, in 29% of the
studies, an already-existing matrix was used as an initial matrix for the assessment (mainly
the same matrix from one of the Burkhard et al. papers). In 16% of the reviewed studies,
no other data were used for the matrix scores or no adaptation of the existing matrix used
was made. However, the actual idea of the ES scores, included in the Burkhard et al.'s
matrices published 10 years ago, was to provide some examples and give inspiration for
one's own studies. Therefore, we recommend to use only scores assessed for a specific
study or, if one wishes to use pre-existing scores from another study, to revise them in
depth, taking into account the local context of the new assessment. We also recommend to
systematically report and consider variabilities and uncertainties in each ES assessment.
We emphasise the need for all scientific studies to describe clearly and extensively the
whole methodology used to score or evaluate ES in order to be able to rate the quality of
the scores obtained. In conclusion, the application of the ES matrix has to become more
transparent and integrate more variability analyses. The increasing number of studies that
use the ES matrix approach confirms its success, appropriability, flexibility and utility for
decision-making, as well as its ability to increase awareness of ES.

Keywords

capacity matrix; ecosystem services assessment; tiered approach; expert-based; look-up
table

Introduction

Since the Ecosystem Service (ES) concept was largely popularised by the Millennium
Ecosystem Assessment (2005), the demand for robust and applicable methodologies for
ES assessments has increased. A wide range of methods for assessing and mapping ES
has now been listed (e.g. in Martinez-Harms and Balvanera 2012, Crossman et al. 2012,
Egoh et al. 2012, Burkhard and Maes 2017), illustrating the need for diverse methods and
degrees of related expertise from people implementing them and to harness data of
varying quantity and quality (Harrison et al. 2017). The choice of the right method should
communicate the goals of the respective ES assessment and mapping exercise (Jacobs et
al. 2017), but also the applicability and appropriation of the methods and the results
expected by potential users of the assessment outcomes including, for instance, various
stakeholders, policy- and decision-makers and land managers.

The ES matrix approach was originally published by Burkhard et al. (2009), with following
updates published in 2012 and 2014 (Burkhard et al. 2012 and Burkhard et al. 2014). Of
course, other comparable matrix/look-up table-based approaches were also published
around the same time (Dechazal et al. 2008, Koschke et al. 2012). However, one of the
strengths of the ES matrix approach, presented by Burkhard et al. (2009), Burkhard et al.
(2012) and Burkhard et al. (2014), is that it is a highly flexible way to assess and map ES,
based on various data sources and methods and in all kinds of study area settings from
local to regional and national scales. Numerous ES matrix applications have been
developed since its initial publication in 2009 (Campagne and Roche 2018). A decade after
the proposition of the matrix approach, we would now like to analyse the diversity of ES
matrix applications and highlight the different trends of the various uses. Our purpose is to
identify the strengths and weaknesses of this ES assessment method, based on actual
studies. Additionally, despite the flexibility and apparent ease of the method, we
hypothesise that, in many cases, the ES matrix approach was applied in an oversimplified
way, leading to comparably weak ES assessments. Through a review of scientific studies
and related publications, in which the ES matrix approach was used, we present the
diversity of application contexts, highlight trends of uses and propose future
recommendations for improving ES matrix applications. We looked at 10 years of
publications after the first seminal paper was published.

The initial ES matrix approach

The ES matrix approach is based on the use of a lookup table consisting of geospatial units which, for instance, can be Ecosystem Types (ET), habitat types or other geospatial units, such as Land Use and Land Cover (LULC) types and sets of ES, which are to be assessed in a specific study area. Thus, the selection of the study area is the starting point of the ES matrix approach, followed by the selection of relevant ET or geospatial units and the selection of relevant ES to be in the lines and columns of the matrix (look-up) table. Then, suitable indicators for the ES quantification and appropriate ES quantification methods have to be defined. Based on that, a score for each of the ES considered is generated, referring to ES potential, ES supply, ES flow/use or demand for ES (see Syrbe and Grunewald 2017 or Burkhard et al. 2012 for detailed definitions). In their seminal publication, Burkhard et al. (2009) proposed to use semi-quantitative scores on a relative scale ranging from 0 to 5. These scores can be based on or integrate diverse sources of data from expert judgements, statistical data to quantitative data from process-based models or direct or indirect measurements. The resulting ES matrix table can easily be joined to geospatial units in order "to evaluate capacities to provide ecosystem services in a spatial manner" (Burkhard et al. 2009: P.4).

A systematic review of the matrix approach

We conducted a systematic review of published studies through Web of Science and Scopus (terms used for the review research in Suppl. material 1), considering all articles published between 2009 and 2019 that used the ES matrix approach (we also included two studies from 2008 that applied similar approaches: Dechazal et al. 2008 and Haines-Young and Potschin 2008). With regard to the term "ES matrix approach", we mean the use of a look-up table relating ES and geospatial units as described in Burkhard et al. (2009). The
initial queries, done in September 2019, returned 880 different studies. Only studies published in the English language were considered. The selection of the final studies was made through a manual verification of the titles, keywords and abstracts and of the full article in case of further doubts, reducing the pool of relevant studies to 110 altogether (all references in Suppl. material 1). The reduction in the number of papers is mostly due to the fact that the word "matrix" is used in a wide variety of methods and so many papers did not use the ES matrix approach as we mean it. A total of 109 studies applying the ES matrix approach and one methodological study without concrete application were selected during the review.

In the following, the results of the review are presented, referring to analysed attributes including case study location, the matrix elements, the scoring system and the methods used.

## Applications of the ES matrix approach

Over the last 10 years, the number of published studies increased progressively, especially during the last five years (Fig. 1).

<figure><img src="image_4.png" /><figcaption>Figure 1.</figcaption></figure>

Number of analysed studies using the ES matrix approach to assess ES supply, demand or flows/use (at the end of 2019).

The flow of ES from nature to society is not always as straightforward as one could perhaps expect. Instead, it includes several components, including the ecosystem-based supply of ES and the societal demand for ES. In literature, many different terms are used, depending on the different ES frameworks, the perception of the authors and the individual applications. In the reviewed studies, we also found a diversity of ES components that were assessed through the ES matrix approach: ES capacity (e.g. Vihervaara et al. 2010), ES potential (e.g. Depellegrin et al. 2016), potential ES supply (e.g. Kamlun and Arndt
2019), potential and actual ES supply (e.g. Hainz-Renetzeder et al. 2015), ES supply (e.g. Nedkov et al. 2014; Sohel et al. 2015), ES flow (e.g. Burkhard et al. 2014), ES use (e.g. Karstens et al. 2019), current ES use (e.g. Nahuelhual et al. 2013), demand for ES (e.g. Burkhard et al. 2012). As the concepts behind the terms were not very clear in a majority of cases and, for simplification reasons, ES “capacity”, “potential”, “potential supply” and “supply” were grouped into “ES supply”. With this regroupment, the ES matrix approach was mainly used for assessing and mapping the “supply” of ES (Fig. 1). Matrices of ES flow or use were regrouped and matrices of ES demand are shown here without regrouping. The number of assessments dealing with demand for ES has increased during the last years of the reviewed period (e.g. Tao et al. 2018; Nurokhmah et al. 2019, Bicking et al. 2019), as well as the number of studies of ES flow or use (e.g. Li et al. 2016, Egarter Vigl et al. 2017) (Fig. 1).

The ES matrix approach has been applied mainly in Europe in 73 analysed studies with a concentration of studies in Germany and neighbouring countries (Fig. 2). There is also a notable increasing number of studies outside Europe, especially with 18 applications in East Asia and the Pacific (e.g. Li et al. 2016; Cai et al. 2017; Sun et al. 2018; Tao et al. 2018; Liu et al. 2019). The ES matrix approach was used at different spatial scales, for example, continental scale (Stoll et al. 2015), national scale (Depellegrin et al. 2016) and local scale (Nedkov and Burkhard 2012). The ES matrix approach was, however, mainly applied at the local (54 studies) and the regional scale (33 studies). The continental scale was used in 12 studies (Fig. 2). The extent of the individual case study sites varies between less than 1 km² to the area of the whole of Europe.

<figure><img src="image_5.png" /><figcaption>Figure 2.</figcaption></figure>

Distribution of the analysed studies amongst world regions, publication dates and scale of studies.

The ES matrix approach has been applied for a large diversity of purposes. While each study presents its own context and objectives, we can observe a broad pattern of application types (using illustrative examples):
* ES assessments in data-scarce areas: for instance in Nepal by Tamang (2011), in Burkina Faso by Sinare et al. (2016) or in Kenya by Wangai et al. (2018);

* Assessment of a specific ES: flood regulating services by Nedkov and Burkhard (2012); nutrient regulating services by Bicking et al. (2019) or global climate regulation by Ma et al. (2019);

* ES assessments provided by a specific ecosystem type: namely grasslands in Villoslada et al. (2018) or seabed biotopes in Salomidi et al. (2012);

* Past and future ecosystem/geospatial units changes' impacts on ES: the ES matrix approach was very often related to change of specific types of geospatial units such as LULC data (e.g. Polce et al. 2016; Huq et al. 2019). We found 20 studies that conducted spatio-temporal analyses (e.g. Kokkoris et al. 2019; Karstens et al. 2019 and Mukul et al. 2017). Temporal ES evolution assessments were done based on LULC change in García-Llamas et al. (2018) and Sanchez-Porras et al. (2018). Spatial ES supply-demand dynamics were assessed in Tao et al. (2018).

* Impacts ES assessment-orientated studies: Kaiser et al. (2013) present an example of a natural hazard impact assessment related to the 2004 Pacific tsunami, based on land cover and ES changes. The effects of EU Life+ project management actions on ES was analysed by Campagne and Roche (2018). Mangi (2016) did an impact assessment on ecological integrity and ES supply before and after a shallow water area creation in the river Elbe's tide management.

## ES matrix elements

The ES matrix approach has been used to assess an average of 15.6 different ES per matrix, whereof 7.0 (on average) were regulating ES, 6.7 provisioning ES and 3.8 cultural ES (Fig. 3). Next to ES, other elements were also included in the ES matrix: we found supporting ES (e.g. Lundy and Wade 2011; Hermann et al. 2014; La Bianca et al. 2018 and Liu et al. 2019) and components of ecosystem functions (e.g. Weyland et al. 2017). Due to the similarity and, sometimes, confusion between components (and related indicators) of ecological condition, ecosystem integrity and supporting ES and in order to avoid double-counting, we grouped these elements under the term "ecological condition indicators" (Fig. 3). They were assessed in 35 studies with a mean of 4.8 components of ecological condition (e.g. Burkhard et al. 2012, Burkhard et al. 2014, Cai et al. 2017, Kamlun and Arndt 2019).

Ecosystem disservices (ES with a negative impact on human well-being; for further definition, see Lyytimäki 2014; von Döhren and Haase 2015 and Shackleton et al. 2016) have been added to the matrix by Campagne and Roche (2018) and Campagne et al. (2018) with 1 and 6 ecosystem disservices, respectively.

Biodiversity was added in addition to ES in 16 studies mainly through one indicator - called "Biodiversity" (e.g. Tamang 2011; Bhandari et al. 2016; Depellegrin et al. 2016 and Kamlun and Arndt 2019).

The geospatial units, that were mainly used in the different ES matrices, were related to LULC types and many studies used the European CORINE Land Cover typology or a
related typology (EEA 1995). Besides CORINE Land Cover, the EUNIS habitat classification (European Nature Information System - EEA 2017) was used, notably for marine and benthic habitats (e.g. Salomidi et al. 2012; Galparsoro et al. 2014; Depellegrin et al. 2017). The ES matrix was generally used across different ecosystem types, but in some cases, it had also been used for specific ecosystem types, such as agroecosystems in Augstburger et al. (2018), wetland ecosystems in Ricaurte et al. (2017) or forest ecosystems in Zarandian et al. (2016).

<figure><img src="image_6.png" /><figcaption>Figure 3.</figcaption></figure>

Mean number of assessed ES and Ecosystem Disservices (EDS) and mean number of ecological condition and biodiversity components used in the published matrix studies (bars) with 95% error bars. Number of analysed studies assessing ES, ecological condition, ecosystem disservices and biodiversity (orange diamond). *The error interval was not computed for the ecosystem disservices since only 2 studies evaluated them.*

Furthermore, the ES matrix approach has been adapted in a “species matrix”, which linked different species types to ES supply, as in Potts et al. (2014) and Burdon et al. (2017) with seabird species.

## Methodologies used in the studies

Precise elaborations of the stepwise ES matrix application can be found in literature (Burkhard 2017, Campagne and Roche 2018, Elliott et al. 2019, Gorn et al. 2018). Burkhard (2017) proposed a 10-step ES matrix methodology for ES mapping, based on ES indicators and a collection of suitable spatial data. Campagne and Roche (2018) and Elliott et al. (2019) proposed methodologies on how to collect and integrate expert knowledge to address some of the biases and limitations of the expert-based elicitation method. Gorn et
al. (2018) combine the ES matrix assessment with the Delphi approach, confidence ratings, standardised confidence levels and scenario assessment.

Several methodological steps are common in all applications of the ES matrix and we propose to look closer at the data and approaches used in and with the matrix, the scoring systems and the scoring process used, as well as the confidence and reliability analyses done in the analysed papers.

The ES matrix approach involves a scoring process to assess ES (supply, flow/use, demand) in ecosystem types or other explicit geospatial units. These scores can be based on or can integrate data from diverse sources of varying quantity and quality (Burkhard et al. 2012). Following the "tiered approach" (Grêt-Regamey et al. 2015), the data used in the ES matrix can be seen as a gradient of increasing complexity (Fig. 4). Spatial GIS data, such as LULC data, are the main data used in the reviewed studies to define the geospatial units in the matrix. At the same time, LULC types can be used as proxies for the supply and use of many ES (e.g. rather obvious LULC - ES relations such as timber provisioning ES in forest ET, water supply ES in water bodies). Thus, LULC can be a suitable base for ES mapping, which has been used intensively.

<figure><img src="image_7.png" /></figure>

Figure 4.

Data or approaches commonly used in or with the ES matrix approach.

For the ES scoring process, expert scoring was the dominant data source, as it was used in 82 of the reviewed studies. When the scoring was expert-based, the number of experts involved in the scoring exercises varied between 2 and 170 with a mean of 31 experts. Nevertheless, the number of involved experts was not specified in 32 studies out of 82. Expert consultation was undertaken through workshops in 34 studies, interviews were conducted in 15 studies and specific surveys were carried out in 9 studies.

The second dominant data source was literature data transfer, which is when ES scores are based on an already-published ES matrix or deduced by information found in related scientific publications. This was used in 57 studies, more than half of the studies. Other
data or approaches, such as statistical data (13 studies, e.g. national statistics of yield production), models (12 studies), remote sensing data (6 studies) and field data (6 studies) were used less in the analyses studies (Fig. 4).

Several types of data or approaches were used in 57 studies, of which 29 only combined two types of data or approaches: literature data transfer and expert scoring.

## Scoring systems

One main characteristic of the ES matrix approach is to express ES provision with an ordinal scale and so allows the comparison of different ES. Several ranking scales were used to fill in the matrix. However, a numeric score ranging from 0 (no [relevant] supply) to 5 (very high [maximum] supply), as originally proposed by Burkhard et al. (2009), was mainly used (64 studies out of the 109 studies).

Several other scoring scales were used with 2, 3, 4, 5 or 7 levels in 33 studies (e.g. "0 to 1" in Baró et al. 2017; "0 to 2" Vihervaara et al. 2010; "1 to 3" in Távora and Turetta 2016; Nowak and Grunewald 2018 and Karrasch et al. 2019; "0 to 3" in Cotillon 2013 and La Bianca et al. 2018; "1 to 4" in Saunders et al. 2015; "0 to 4" in Van Looy et al. 2017; "1 to 5" in Clius and Pătroescu 2014; "1 to 7" in Weyland et al. 2017 and Maebe et al. 2019). A somewhat stretched scale from 0 to 100 can be found in Koschke et al. (2012) and Müller et al. (2020). The choice of the applied type of scoring system in the reviewed studies generally referred to a published study which, in the case of the 0 to 5 scale, was usually Burkhard et al. (2009). A different scoring system had been used in a few studies with specific aims: positive and negative rankings such as "-1 to 1" (Dechazal et al. 2008); "-2 to 2" (Hornung et al. 2018) and "-3 to 3" (Elliott et al. 2019; Karstens et al. 2019; Kopperoinen et al. 2014). Kopperoinen et al. (2014) used a scoring system with positive and negative values to assess favourable and harmful effects of the land cover on ES provision potential. This type of scoring can be related to an ecosystem services or disservices assessment, but disservices assessment can also be separated for services in the matrix (e.g. Campagne et al. 2018).

In two studies, the scoring system used non-numeric values (+) to (+++) (Maltby et al. 2017) or colour codes with 4 levels (Geange et al. 2019).

## ES scoring process

A first step for implementing a matrix-based ES evaluation is to define the initial matrix that is to be used (Campagne and Roche 2018). It can be either based on an existing matrix from an already-published study or an empty ES matrix that is to be filled. Seventy of the analysed studies used an empty ES matrix, whereas in 32 studies, it was specified that the initial ES matrix (with or without modification afterwards) came from an already-existing ES matrix or several existing matrices that were found in literature (Fig. 5 and Table 1). Amongst those studies, 23 studies specified that the initial ES scores came from one of the Burkhard et al. publications (Burkhard et al. 2009, Burkhard et al. 2012, Burkhard et al. 2014). For 7 studies, unclear information on the initial ES matrix was provided. For
example, the number of experts involved are presented in Bhandari et al. (2016), but no information on the initial matrix, the fill-in process or whether there has been a compilation of scores can be found in the paper.

<table><caption>Table 1.
Number of times the published matrix was used as an initial matrix.</caption><tr><td>21</td><td>Burkhard et al. publications: Burkhard et al. (2009), Burkhard et al. (2012), Burkhard et al. (2014)</td></tr><tr><td>3</td><td>Salomidi et al. (2012)</td></tr><tr><td>2</td><td>Stoll et al. (2015), Hermann et al. (2014), Galparsoro et al. (2014)</td></tr><tr><td>1</td><td>Xie et al. (2008); Nedkov and Burkhard (2012); Scolozzi et al. (2012); Potts et al. (2014); Depellegrin et al. (2016); Li et al. (2016); Goldenberg et al. (2017); Wu et al. (2019);</td></tr></table>

<figure><img src="image_8.png" /><figcaption>Figure 5.
Number of analysed studies with explicit information on the initial matrix (in the centre) and on the scoring data and approaches used in the initial matrix (arrows).</figcaption></figure>

After the definition of the initial matrix, the scoring process can be carried out with diverse sources of data. In the 32 studies that used an existing ES matrix as an initial matrix, 18 studies used no additional data to define the matrix scores and, therefore, made no adaptation of the values provided in the existing matrix (Fig. 5), whereas 14 studies modified the existing initial matrix, based on literature or expert opinions or models for one study. For example, Cai et al. (2017) used Burkhard et al. (2012) supply matrix and local expert adjustments to determine the final scores. Tao et al. (2018) applied the original matrices presented by Burkhard et al. (2014) and Li et al. (2016) and updated their scores through interviews of eight experts and “several rounds of panel discussions amongst all the experts to obtain consistent final scores for the matrix” (Li et al. 2016: P. 252). For the studies with an empty initial matrix, the scoring was mainly done through expert scoring or by a combination of expert scoring and data extracted from literature (Fig. 5). In addition,
Maebe et al. (2019) and Ma et al. (2019) used a literature review and a mix of data from published matrices and published quantitative data. They are noted here with empty initial matrix and scoring data with literature.

Finally, the methodology to define the final scores, using all kind of data and approaches presented in Fig. 5, should be detailed in the studies (e.g. for experts' scoring, how the experts' scores are collected and merged; when several data types and sources are used, how are they combined in a final score). From the 109 studies that were reviewed, only 61 studies explained clearly how the scores were obtained, for 19 studies the scores came from an existing ES matrix without modification or review of literature. Altogether, 29 studies did not provide any information about the method used to determine the scores, i.e. how exactly the scores were determined with the different data used in the paper.

## Confidence and reliability

Hou et al. (2013) and Burkhard (2017) elaborated in detail the potential uncertainties in the overall ES matrix approach. In the scoring process, an uncertainty analysis should also be included (Campagne et al. 2017). Depending on the data source used, we can find diverse methods for confidence analysis in literature. With a score based on literature data and expert opinions, Potts et al. (2014), Saunders et al. (2015), Burdon et al. (2017) and Geange et al. (2019) used a confidence analysis, depending on the data source and the difference between “expert opinion”; “grey literature from overseas” and “grey and peer reviewed literature from the country of the study”. Maebe et al. (2019)P.13) put confidence scores “determined by agreement and evidence quality of the literature review” with “high agreement (i.e. data corroboration, no or minor disagreements) and robustness evidence (i.e. data are supported by scientific argumentation or analysis)” to “medium/yellow = medium evidence and medium agreement”. La Bianca et al. (2018) used a confidence score, based on the geographic similarity between the location of the case study site of the literature used and their study area. Whenever the ES scores result from the combination of several values, the variability of the scores needs to be considered and a confidence score or indices of reliability or agreement can be used to evaluate the degree of agreement or consistency amongst the individual experts' answers (Jacobs et al. 2014, Campagne et al. 2017). Variability of the expert scores was, for instance, analysed in Tao et al. (2018). Confidence scores, expressed by experts together with their scores for each ES, were used in Campagne et al. (2017) and Elliott et al. (2019). Gorn et al. (2018) used a confidence level that was “developed for the means and standard deviations of the assessments of the ES provision potential and assessments for the confidence rating according to IPCC”. In case the ES matrix scores were based on methods from different tiers (see above), the values can be cross-checked in order to find the most suitable, reliable and useful (for the specific purpose) ES quantification method (Burkhard 2017, Roche and Campagne 2019). Data from the different tiers should, of course, be valid for the same area, time and spatial scale and in comparable resolution. In the reviewed studies, only 13 studies did a confidence analysis, including eight newer studies from 2018 and 2019.
Discussion

We considered a total of 109 studies over a period of 10 years that have applied the ES matrix approach. Those studies were mainly carried out in Europe, but an increasing number of applications outside Europe can be noted, particularly in Asia. Applications mostly focused on ES supply assessments, whereas ES demand and ES flow/use assessments remain a minority.

Our review shows a mean of 15.6 ± 1.9 different ES were assessed through the matrix approach in 109 studies, whereby a mean of 7.9 ± 4.7 was found in the review by Hölting et al. (2019) in 101 studies using quantitative methods to assess landscape or ecosystem multifunctionality. The ES matrix approach allows the assessment of more ES than other approaches, notably by overcoming the limitations of data availability or the lack of proper proxies to quantitatively evaluate ES. We observed a lower number of cultural and provisioning ES than regulating ES assessed in the reviewed studies. The tendency to assess less cultural and provisioning ES than regulating ES was also observed in Egeh et al. (2012); Haase et al. (2014); Malinga et al. (2015) and Hölting et al. (2019).

The flexibility of the ES matrix approach was illustrated through the diversity of the applied scoring systems, data sources, matrix elements and the different purposes of applications. Nevertheless, the scoring system that was mostly used was the original "0 to 5" range, based on expert opinions, harnessing existing matrices or scores defined by authors, based on published results.

Our review highlights several major limitations or even mistakes in existing ES matrix approach applications.

The critical use of a pre-existing matrix

The review shows that 29% of the studies used an existing matrix as an initial matrix, 16% of the studies used no other data in the matrix scores and made no adaptation of the existing matrix values. As for other value-transfer methods, the lack of adaptation bears the risk that incorrect or, for the specific case study, unsuitable values are used. A critical evaluation of the validity of the scores in the matrix should therefore be mandatory. A total of 21 studies specified that the scores came from one of the Burkhard et al. published matrices (matrices in Burkhard et al. 2009, Burkhard et al. 2012 or Burkhard et al. 2014). The matrices, developed in these studies, were initially created for "normal landscapes" in northern Germany, based on an integration of expert knowledge, statistical data, model outcomes and literature data applications derived from several long-term case studies in this specific region (e.g. Fränzle et al. 2008). Hence, the matrix values are basically only valid for comparable landscapes and human-environmental system settings. Otherwise, they need to be adapted to each case study's specific conditions. For example, the process used to create the different matrices in the study of Stoll et al. (2015), based on the matrix in Burkhard et al. (2012), could be recommended as several experts across Europe were involved in the matrix adjustment process. The adaptation of the scores from
an existing ES matrix is also possible through a participatory approach with adjustments in
consensus, as in Cai et al. (2017) and Tao et al. (2018).

The need for rigorous presentation of methods

In 27% of the reviewed studies, it is not clear how the data has been used and where the final scores came from. This leads to a deficit in the scientific robustness and replicability of the studies, as well as a lack of proper consideration of the importance of the data acquisition protocol by the reviewers. It is important to be precise and explicitly transparent about the methods that were used in order to allow the end-user to be aware of uncertainties inherent in the assessment. A categorisation of the used data and approaches used according to the "tiered approach" (see "Methodologies used in the studies" Section above) can help to understand the type and complexity of the applied approaches.

Variability and uncertainty analysis of scores should be the norm

The limits and uncertainties of the ES matrix approach have been listed, for instance in Hou et al. (2013), Jacobs et al. (2014) and Campagne et al. (2017), as well as some of the issues regarding the integration of assessments of experts' uncertainties. Campagne and Roche (2018) elaborated more in detail on how to collect and integrate expert knowledge to address some of the biases and limits of the expert elicitation method. In the 84 studies using expert scoring (no matter the initial matrix, Fig. 3), only 13 studies did a confidence or a variability analysis. Many of the issues raised by Saunders et al. (2015), Hou et al. (2013), Jacobs et al. (2014), Burkhard (2017), Campagne et al. (2017), Campagne and Roche (2018) were, unfortunately, not always properly addressed by many users of the ES matrix approach. When a score is expert-based, variability and confidence should be considered in the analysis with the final scores. Some studies developed or used confidence analysis and provided good examples (e.g. Elliott et al. 2019, Geange et al. 2019, Gorn et al. 2018, La Bianca et al. 2018). The results show that the number of studies that consider confidence analysis has increased in the last two years, but concerns still a too low number of papers.

The use of expert scoring in the ES matrix

A regular critique of the ES matrix approach is that it is too subjective, particularly when based on expert scoring alone. One way to tackle such remarks is to benchmark the ES expert scores against "more quantitative" estimates. However, up to today, only a few studies have dealt with the topic of comparing ES matrix experts-based scores with quantitative estimates. Ma et al. (2019) compared five indicators: one qualitative (expert-based) and four quantitative (data-based) indicators of the global climate regulation ES in Germany and found significant correlations. Roche and Campagne (2019) observed high levels of correlation amongst seven ES scores provided by an expert panel with eight spatial quantitative biophysical indicators at the landscape scale for the French Hauts-de-France Region. It has been shown that more complex ES assessment approaches do not
necessarily deliver more robust results than those harnessing expert knowledge (Jacobs and Burkhard 2017). Nevertheless, more comparison analysis studies need to be conducted in different contexts to strengthen the applicability of the ES matrix approach. This is most important, since quantitative ES assessment methods have not necessarily to be more reliable than expert scoring. Moreover, it could be more important to pay attention to the selection and the number of experts in the panel and the elicitation methods used to produce the estimates than to compare them to quantitative data. This, however, requires a careful consideration of the transparency of the methods used to fill in the ES matrix and to describe in detail what exactly has been done in order to achieve the scores extensively, as stated in the second point of the discussion.

## Improve the characteristics of geospatial units to assess ES

Basically, the ES matrix approach is based on spatial units as LULC categories, although LULC categories can be considered as important proxies for many ES. Nevertheless, LULC alone lacks information regarding important components of ecosystem conditions that support ES capacities, such as soil type and quality, water availability, geomorphology or overall ecosystem integrity. These components also vary in space and time within and between LULC categories. One approach is to consider that the generality of LULC, especially when using broad categories, can be associated with high confidence of ES scores and is, in itself, a strength and the main interest of the ES approach - applicability and genericity. Another approach is to complement the LULC categories based ES scoring with other sources of informations that can be used to tune the matrix ES scores, based on local ecosystem condition and thus improve local validity of scores. As a consequence, this reduces the manageability of the comparably simple look-up-table approach. Jacobs et al. (2014) recommended to use broad land-use classes associated with high confidence scores that can be more easily transferable than locally specific LULC classes. In an alternative approach, the experts could also be asked to provide ranges of ES values depending on different ecosystem states, that would allow a more detailed consideration of relationships between ES scores adaptation, based on local ecosystem states.

Despite these limitations, the ES matrix approach has proven its usefulness. The advantages of the approach were listed, amongst others, in Jacobs et al. (2014) and Campagne and Roche (2018). The ES matrix approach offers a good compromise to deal with the 'urgency-uncertainty dilemma' (Jacobs and Burkhard (2017) by its comparably quick application related to varying levels of methodological complexity (depending on the methods applied for ES quantification, see 'tiered approach' above). In a discussion paper, van Oudenhoven et al. (2018) listed the essential criteria for defining relevant ES indicators to provide information for public decision-making. They defined 16 essential criteria related to the credibility, salience, legitimacy and feasibility of ES indicators. A carefully-designed ES matrix application possesses many of the van Oudenhoven et al. (2018) essential criteria that are particularly important for actual use and consideration of ES assessments by policy-makers and land managers:
* credibility of the results; it was shown in two studies (Ma et al. 2019 and Roche and Campagne 2019) that experts-based scores and independent quantitative data have a significant correlation;

* salience through its relevance and adaptability to local issues, as well as its easy comprehension;

* legitimacy in expert workshop processes that promote acceptance and understanding by stakeholders, policy- and decision-makers and land managers;

* feasibility through its flexibility, time and resource efficiency and the advantage of overcoming the limitations of available data by creating data, according to experts.

## Conclusions

The ES matrix approach is widely applied in a high diversity of contexts and with various data and quantification approaches. Based on our analysis on ES matrix applications and methodologies within a ten year period, our **key recommendations** for future improvements include:

* Proper communication and transparency of the quantification methods used, including uncertainty assessments;

* Avoiding value-transfer from existing matrices to non-comparable case studies or adapting the values with a local participatory approach and local data. This means that an existing ES matrix cannot be used directly to estimate LULC types' ES capacities in a different context/region without being re-evaluated or adjusted as with a dedicated expert panel session;

* Improving quantification of ES scores in the matrix, harnessing and integrating methods from different tier levels (besides expert-based quantifications, also use of other data originating, for instance, from statistics, monitoring, citizen science, social media, remote sensing or/and model outcomes).

We also take the opportunity to provide the recommendations for improved applications of the ES matrix beyond the results achieved from the review (based on Campagne and Roche 2018 and Jacobs et al. 2014):

* Appropriate stakeholder/end-user involvement during the assessment process;

* Specification of the geospatial units, including information on ecosystem condition and spatial heterogeneities;

* Improving the selection of ES that are relevant to be assessed in the specific case study;

* Integration of different valuation methods, including biophysical, social-cultural and economic methods where appropriate;

* Consideration of each individual study's purpose and flexible choice of respective methods and data to be used.

The simplicity of the method has been acknowledged, on the one hand, as the main strength of the method. On the other hand, this is also considered its key weakness. The success of the method is also linked to its feasibility and its easy comprehensibility that can
promote the use and ability to increase awareness of ES for decision-making (Science for Environment Policy 2015). With the integration of data resulting from ES quantification methods that are combined with participatory approaches and the co-production of results, the final outputs are readily appropriable by stakeholders.

Nevertheless, the application of the ES matrix approach has to become more transparent and integrate more confidence analysis. It remains an important task to elaborate which are the most appropriate ES assessment methods for each individual ES or group of ES in different human-environmental system settings and for the different assessment purposes.

## Acknowledgements

We would like to thank Candice Christin and Emily Bank for their contributions to the literature review and the database construction, Bastian Steinhoff-Knopp for an informal review and Angie Faust for a language check before submission. Additionally, we would like to thank the reviewers for their constructive comments and suggestions which helped to improved the text. The publication of this article was funded by the Open Access Fund of the Leibniz Universität Hannover.

## References

* Augstburger H, Jacobi J, Schwilch G, Rist S (2018) Agroecosystem Service Capacity Index – A methodological approach. Landscape Online 64: 1-48. https://doi.org/10.3097/lo.201864

* Baró F, Gómez-Baggethun E, Haase D (2017) Ecosystem service bundles along the urban-rural gradient: Insights for landscape planning and management. Ecosystem Services 24: 147-159. https://doi.org/10.1016/j.ecoser.2017.02.021

* Bhandari P, KC M, Shrestha S, Aryal A, Shrestha UB (2016) Assessments of ecosystem service indicators and stakeholder's willingness to pay for selected ecosystem services in the Chure region of Nepal. Applied Geography 69: 25-34. https://doi.org/10.1016/j.apgeog.2016.02.003

* Bicking S, Burkhard B, Kruse M, Müller F (2019) Bayesian Belief Network-based assessment of nutrient regulating ecosystem services in Northern Germany. PLOS ONE 14 (4). https://doi.org/10.1371/journal.pone.0216053

* Burdon D, Potts T, Barbone C, Mander L (2017) The matrix revisited: A bird's-eye view of marine ecosystem service provision. Marine Policy 77: 78-89. https://doi.org/10.1016/j.marpol.2016.12.015

* Burkhard B, Kroll F, Müller F, Windhorst W (2009) Landscapes' capacities to provide ecosystem services - A concept for land-cover based assessments. Landscape Online 15: 1-22. https://doi.org/10.3097/lo.200915

* Burkhard B, Kroll F, Nedkov S, Müller F (2012) Mapping ecosystem service supply, demand and budgets. Ecological Indicators 21: 17-29. https://doi.org/10.1016/j.ecolind.2011.06.019
* Burkhard B, Kandziora M, Hou Y, Müller F (2014) Ecosystem service potentials, flows and demands-concepts for spatial localisation, indication and quantification. Landscape Online 34: 1-32. https://doi.org/10.3097/lo.201434

* Burkhard B (2017) Ecosystem services matrix. In: Burkhard B, Maes J (Eds) Mapping Ecosystem Services. Pensoft Publishers, Sofia, 374 pp.

* Burkhard B, Maes J (2017) Mapping Ecosystem Services. Pensoft Publishers, Sofia, 374 pp. https://doi.org/10.3897/ab.e12837

* Cai W, Gibbs D, Zhang L, Ferrier G, Cai Y (2017) Identifying hotspots and management of critical ecosystem services in rapidly urbanizing Yangtze River Delta Region, China. Journal of Environmental Management 191: 258-267. https://doi.org/10.1016/j.jenvman.2017.01.003

* Campagne CS, Roche P, Gosselin F, Tschanz L, Tatoni T (2017) Expert-based ecosystem services capacity matrices: Dealing with scoring variability. Ecological Indicators 79: 63-72. https://doi.org/10.1016/j.ecolind.2017.03.043

* Campagne CS, Roche P (2018) May the matrix be with you! Guidelines for the application of expert-based matrix approach for ecosystem services assessment and mapping. One Ecosystem 3 https://doi.org/10.3897/oneeco.3.e24134

* Campagne CS, Roche P, Salles J (2018) Looking into Pandora's Box: Ecosystem disservices assessment and correlations with ecosystem services. Ecosystem Services 30: 126-136. https://doi.org/10.1016/j.ecoser.2018.02.005

* Clius M, Pătroescu M (2014) An evaluation matrix for ecotourism potential in certain categories of protected areas in Romania. Case studies: national parc, nature parc, geopark. 14th International Multidisciplinary Scientific GeoConference SGEM 2014 Through.

* Cotillon S (2013) Impacts of Land Cover Changes on Ecosystem Services Delivery in the Black Hills Ecoregion from 1950 to 2010. South Dakota State University

* Crossman ND, Burkhard B, Nedkov S (2012) Quantifying and mapping ecosystem services. International. J Biodivers Sci Ecosyst Serv Manag 8: 1-4. https://doi.org/10.1080/21513732.2012.695229

* Dechazal J, QUETIER F, LAVOREL S, VANDOORN A (2008) Including multiple differing stakeholder values into vulnerability assessments of socio-ecological systems. Global Environmental Change 18 (3): 508-520. https://doi.org/10.1016/j.gloenvcha.2008.04.005

* Depellegrin D, Pereira P, Misiunė I, Egarter-Vigl L (2016) Mapping ecosystem services potential in Lithuania. International Journal of Sustainable Development & World Ecology 23 (5): 441-455. https://doi.org/10.1080/13504509.2016.1146176

* Depellegrin D, Menegon S, Farella G, Ghezzo M, Gissi E, Sarretta A, Venier C, Barbanti A (2017) Multi-objective spatial tools to inform maritime spatial planning in the Adriatic Sea. Science of The Total Environment 609: 1627-1639. https://doi.org/10.1016/j.scitotenv.2017.07.264

* EEA (1995) CORINE Land Cover report. European Environmental Agency. https://www.eea.europa.eu/publications/COR0-landcover

* EEA (2017) EUNIS habitat classification report. European Environmental Agency. https://www.eea.europa.eu/data-and-maps/data/eunis-habitat-classification

* Egarter Vigl L, Depellegrin D, Pereira P, de Groot R, Tappeiner U (2017) Mapping the ecosystem service delivery chain: Capacity, flow, and demand pertaining to aesthetic
experiences in mountain landscapes. Science of The Total Environment 574: 422-436.
https://doi.org/10.1016/j.scitotenv.2016.08.209

* Egoh B, Drakou E, Dunbar M, Maes J, Willemen L (2012) Indicators for mapping ecosystem services: a review. European Union, 2012 Reproduction, Joint Research Centre, Via Enrico Fermi 2749, TP 460, 21027 Ispra (VA), Italy, 113 pp. https://doi.org/10.2788/41823

* Elliott R, Motzny A, Majd S, Chavez FV, Laimer D, Orlove B, Culligan P (2019) Identifying linkages between urban green infrastructure and ecosystem services using an expert opinion methodology. Ambio 49 (2): 569-583. https://doi.org/10.1007/s13280-019-01223-9

* Fränzle O, Kappen L, Blume H-, Dierssen K (Eds) (2008) Ecosystem Organization of a Complex Landscape — Long-Term Research in the Bornhöved Lake District. Ecological Studies. Springer, Berlin, Heidelberg, 391 pp.

* Galparsoro I, Borja A, Uyarra M (2014) Mapping ecosystem services provided by benthic habitats in the European North Atlantic Ocean. Frontiers in Marine Science 1 https://doi.org/10.3389/fmars.2014.00023

* García-Llamas P, Geijzendorffer I, García-Nieto A, Calvo L, Suárez-Seoane S, Cramer W (2018) Impact of land cover change on ecosystem service supply in mountain systems: a case study in the Cantabrian Mountains (NW of Spain). Regional Environmental Change 19 (2): 529-542. https://doi.org/10.1007/s10113-018-1419-2

* Geange S, Townsend M, Clark D, Ellis J, Lohrer A (2019) Communicating the value of marine conservation using an ecosystem service matrix approach. Ecosystem Services 35: 150-163. https://doi.org/10.1016/j.ecoser.2018.12.004

* Goldenberg R, Kalantari Z, Cvetkovic V, Mörtberg U, Deal B, Destouni G (2017) Distinction, quantification and mapping of potential and realized supply-demand of flow-dependent ecosystem services. Science of The Total Environment 599-609. https://doi.org/10.1016/j.scitotenv.2017.03.130

* Gorn L, Kleemann J, Fürst C (2018) Improving the matrix-assessment of ecosystem services provision—the case of regional land use planning under climate change in the Region of Halle, Germany. Land 7 (2). https://doi.org/10.3390/land7020076

* Grêt-Regamey A, Weibel B, Kienast F, Rabe S, Zulian G (2015) A tiered approach for mapping ecosystem services. Ecosystem Services 13: 16-27. https://doi.org/10.1016/j.ecoser.2014.10.008

* Haase D, Larondelle N, Andersson E, Artmann M, Borgström S, Breuste J, Gomez-Baggethun E, Gren Å, Hamstead Z, Hansen R, Kabisch N, Kremer P, Langemeyer J, Rall EL, McPhearson T, Pauleit S, Qureshi S, Schwarz N, Voigt A, Wurster D, Elmqvist T (2014) A quantitative review of urban ecosystem service assessments: concepts, models, and implementation. AMBIO 43 (4): 413-433. https://doi.org/10.1007/s13280-014-0504-0

* Haines-Young R, Potschin M (2008) England's terrestrial ecosystem services and the rationale for an ecosystem approach. Overview Report. (Defra Project Code NR0107), 30 pp.

* Hainz-Renetzeder C, Schneidergruber A, Kuttner M, Wrbka T (2015) Assessing the potential supply of landscape services to support ecological restoration of degraded landscapes: A case study in the Austrian-Hungarian trans-boundary region of Lake Neusiedl. Ecological Modelling 295: 196-206. https://doi.org/10.1016/j.ecolmodel.2014.07.001
* Harrison P, Dunford R, Barton D, Kelemen E, Martín-López B, Norton L, Termansen M, Saarikoski H, Hendriks K, Gómez-Baggethun E, Czúcz B, García-Llorente M, Howard D, Jacobs S, Karlsen M, Kopperoinen L, Madsen A, Rusch G, van Eupen M, Verweij P, Smith R, Tuomasjukka D, Zulian G (2017) Selecting methods for ecosystem service assessment: A decision tree approach. Ecosystem Services 29: 481-498. https://doi.org/10.1016/j.ecoser.2017.09.016

* Hermann A, Kuttner M, Hainz-Renetzeder C, Konkoly-Gyuró É, Tirászi Á, Brandenburg C, Allex B, Ziener K, Wrbka T (2014) Assessment framework for landscape services in European cultural landscapes: An Austrian Hungarian case study. Ecological Indicators 37: 229-240. https://doi.org/10.1016/j.ecolind.2013.01.019

* Hölting L, Beckmann M, Volk M, Cord A (2019) Multifunctionality assessments – More than assessing multiple ecosystem functions and services? A quantitative literature review. Ecological Indicators 103: 226-235. https://doi.org/10.1016/j.ecolind.2019.04.009

* Hou Y, Burkhard B, Müller F (2013) Uncertainties in landscape analysis and ecosystem service assessment. Journal of Environmental Management 127 https://doi.org/10.1016/j.jenvman.2012.12.002

* Huq N, Bruns A, Ribbe L (2019) Interactions between freshwater ecosystem services and land cover changes in southern Bangladesh: A perspective from short-term (seasonal) and long-term (1973–2014) scale. Science of The Total Environment 650: 132-143. https://doi.org/10.1016/j.scitotenv.2018.08.430

* Jacobs S, Burkhard B, Van Daele T, Staes J, Schneiders A (2014) ‘The Matrix Reloaded’: A review of expert knowledge use for mapping ecosystem services. Ecological Modelling 295: 21-30. https://doi.org/10.1016/j.ecolmodel.2014.08.024

* Jacobs S, Burkhard B (2017) Applying expert knowledge for ecosystem services quantification. In: Burkhard B, Maes J (Eds) Mapping Ecosystem Services. Pensoft Publishers, Sofia, 374 pp. [ISBN 9789546428295].

* Jacobs S, Verheyden W, Dendoncker N (2017) Why to map? In: Burkhard B, Maes J (Eds) Mapping Ecosystem Services. Pensoft Publishers, Sofia, 374 pp.

* Kaiser G, Burkhard B, Römer H, Sangkaew S, Graterol R, Haitook T, Sterr H, Sakuna-Schwartz D (2013) Mapping tsunami impacts on land cover and related ecosystem service supply in Phang Nga, Thailand. Natural Hazards and Earth System Sciences 13 (12): 3095-3111. https://doi.org/10.5194/nhess-13-3095-2013

* Kamlun KU, Arndt RB (2019) Expert-based approach on mapping ecosystem services potential supply incircling a protected areas by integrating matrix model assessment. Journal of Physics: Conference Series 1358 https://doi.org/10.1088/1742-6596/1358/1/012032

* Karrasch L, Klenke T, Kleyer M (2019) Land-use elements and attributed ecosystem services: an archetype approach to land-use evaluation at the German North Sea coast. Ecology and Society 24 (2). https://doi.org/10.5751/es-10744-240213

* Karstens S, Inácio M, Schernewski G (2019) Expert-based evaluation of ecosystem service provision in coastal reed wetlands under different management regimes. Frontiers in Environmental Science 7 https://doi.org/10.3389/fenvs.2019.00063

* Kokkoris I, Bekri E, Skuras D, Vlami V, Zogaris S, Maroulis G, Dimopoulos D, Dimopoulos P (2019) Integrating MAES implementation into protected area management under climate change: A fine-scale application in Greece. Science of The Total Environment 695 https://doi.org/10.1016/j.scitotenv.2019.07.336
* Kopperoinen L, Itkonen P, Niemelä J (2014) Using expert knowledge in combining green infrastructure and ecosystem services in land use planning: an insight into a new place-based methodology. Landscape Ecology 29 (8): 1361-1375. https://doi.org/10.1007/s10980-014-0014-2

* Koschke L, Fürst C, Frank S, Makeschin F (2012) A multi-criteria approach for an integrated land-cover-based assessment of ecosystem services provision to support landscape planning. Ecological Indicators 21: 54-66. https://doi.org/10.1016/j.ecolind.2011.12.010

* La Bianca G, Tillin H, Hodgson B, Erni-Cassola G, Howell K, Rees S (2018) Ascension Island- Natural Capital Assessment: Marine ecosystem services report. JNCC; Funded by UK Government, 83 pp.

* Li J, Jiang H, Bai Y, Alatalo J, Li X, Jiang H, Liu G, Xu J (2016) Indicators for spatial-temporal comparisons of ecosystem service status between regions: A case study of the Taihu River Basin, China. Ecological Indicators 60: 1008-1016. https://doi.org/10.1016/j.ecolind.2015.09.002

* Liu W, Zhan J, Zhao F, Yan H, Zhang F, Wei X (2019) Impacts of urbanization-induced land-use changes on ecosystem services: A case study of the Pearl River Delta Metropolitan Region, China. Ecological Indicators 98: 228-238. https://doi.org/10.1016/j.ecolind.2018.10.054

* Lundy L, Wade R (2011) Integrating sciences to sustain urban ecosystem services. Progress in Physical Geography: Earth and Environment 35 (5): 653-669. https://doi.org/10.1177/0309133311422464

* Lyytimäki J (2014) Bad nature: Newspaper representations of ecosystem disservices. Urban Forestry & Urban Greening 13 (3): 418-424. https://doi.org/10.1016/j.ufug.2014.04.005

* Maebe L, Claessens H, Dufrêne M (2019) The critical role of abiotic factors and human activities in the supply of ecosystem services in the ES matrix. One Ecosystem 4 https://doi.org/10.3897/oneeco.4.e34769

* Ma L, Bicking S, Müller F (2019) Mapping and comparing ecosystem service indicators of global climate regulation in Schleswig-Holstein, Northern Germany. Science of The Total Environment 648: 1582-1597. https://doi.org/10.1016/j.scitotenv.2018.08.274

* Malinga R, Gordon L, Jewitt G, Lindborg R (2015) Mapping ecosystem services across scales and continents – A review. Ecosystem Services 13: 57-63. https://doi.org/10.1016/j.ecoser.2015.01.006

* Maltby L, Jackson M, Whale G, Brown AR, Hamer M, Solga A, Kabouw P, Woods R, Marshall S (2017) Is an ecosystem services-based approach developed for setting specific protection goals for plant protection products applicable to other chemicals? Science of The Total Environment 580: 1222-1236. https://doi.org/10.1016/j.scitotenv.2016.12.083

* Mangi HO (2016) Tide management in the Elbe River and changes in ecosystem services. Advances in Ecology 2016: 1-13. https://doi.org/10.1155/2016/9519637

* Martinez-Harms MJ, Balvanera P (2012) Methods for mapping ecosystem service supply: a review. https://doi.org/10.1080/21513732.2012.663792; 8: 17-25. https://doi.org/10.1080/21513732.2012.663792

* Millennium Ecosystem Assessment (2005) Ecosystems and human well-being: synthesis. Island Press, Washington, DC., 155 pp.
* Mukul S, Sohel MSI, Herbohn J, Inostroza L, König H (2017) Integrating ecosystem services supply potential from future land-use scenarios in protected area management: A Bangladesh case study. Ecosystem Services 26: 355-364. https://doi.org/10.1016/j.ecoser.2017.04.001

* Müller F, Bicking S, Ahrendt K, Bac DK, Blindow I, Fürst C, Haase P, Kruse M, Kruse T, Ma L, Perennes M, Ruljevic I, Schernewski G, Schimming C, Schneiders A, Schubert H, Schumacher J, Ulrike T, Wangai P, Windhorst W, Zeleny J (2020) Assessing ecosystem service potentials to evaluate terrestrial, coastal and marine ecosystem types in Northern Germany – An expert-based matrix approach. Ecological Indicators 112: 106-11.

* Nahuelhual L, Carmona A, Lozada P, Jaramillo A, Aguayo M (2013) Mapping recreation and ecotourism as a cultural ecosystem service: An application at the local level in Southern Chile. Applied Geography 40: 71-82. https://doi.org/10.1016/j.apgeog.2012.12.004

* Nedkov S, Burkhard B (2012) Flood regulating ecosystem services—Mapping supply and demand, in the Etropole municipality, Bulgaria. Ecological Indicators 21: 67-79. https://doi.org/10.1016/j.ecolind.2011.06.022

* Nedkov S, Gikov A, Nikolova M, Dimitrov P, Gachev E (2014) Mapping ecosystem services supply in mountain areas: a case study of seven Rila lakes, Bulgaria. 5th International Conference on Cartography and GIS, June 15-21, 2014 Riviera, Bulgaria, (JUNE).

* Nowak A, Grunewald K (2018) Landscape sustainability in terms of landscape services in rural areas: Exemplified with a case study area in Poland. Ecological Indicators 94: 12-22. https://doi.org/10.1016/j.ecolind.2018.01.059

* Nurokhmah I, Adrianto L, Sjafri N (2019) The linkage of social-ecological system of Mangrove in Jor Bay, East Lombok Regency, West Nusa Tenggara. IOP Conference Series: Earth and Environmental Science 241 https://doi.org/10.1088/1755-1315/241/1/012001

* Polce C, Maes J, Brander L, Cescatti A, Baranzelli C, Lavalle C, Zulian G (2016) Global change impacts on ecosystem services: a spatially explicit assessment for Europe. One Ecosystem 1 https://doi.org/10.3897/oneeco.1.e9990

* Potts T, Burdon D, Jackson E, Atkins J, Saunders J, Hastings E, Langmead O (2014) Do marine protected areas deliver flows of ecosystem services to support human welfare? Marine Policy 44: 139-148. https://doi.org/10.1016/j.marpol.2013.08.011

* Ricaurte LF, Olaya-Rodríguez MH, Cepeda-Valencia J, Lara D, Arroyave-Suárez J, Max Finlayson C, Palomo I (2017) Future impacts of drivers of change on wetland ecosystem services in Colombia. Global Environmental Change 44: 158-169. https://doi.org/10.1016/j.gloenvcha.2017.04.001

* Roche P, Campagne CS (2019) Are expert-based ecosystem services scores related to biophysical quantitative estimates? Ecological Indicators 106 https://doi.org/10.1016/j.ecolind.2019.05.052

* Salomidi M, KATSANEVAKIS S, BORJA A, BRAECKMAN U, DAMALAS D, GALPARSORO I, MIFSUD R, MIRTO S, PASCUAL M, PIPITONE C, RABAUT M, TODOROVA V, VASSILOPOULOU V, VEGA FERNANDEZ T (2012) Assessment of goods and services, vulnerability, and conservation status of European seabed biotopes: a stepping stone towards ecosystem-based marine spatial management. Mediterranean Marine Science 13 (1). https://doi.org/10.12681/mms.23
* Sanchez-Porras A, Tenorio-Arvide M, Peña-Moreno R, Sampedro-Rosas M, Silva-Gómez S (2018) Evaluation of the potential change to the ecosystem service provision due to industrialization. Sustainability 10 (9). https://doi.org/10.3390/su10093355

* Saunders J, Potts T, Jackson E, Burdon D, Atkins JP, Hastings E, Langmead O (2015) Chapter 9. Linking ecosystem services of marine protected areas to benefits in human wellbeing? In: Turner RK, Schaafsma M (Eds) Coastal zones ecosystem services: from science to values and decision making. Studies in Ecological Economics 9, Springer, Switzerland,

* Science for Environment Policy (2015) Ecosystem Services and the Environment. In-depth Report produced for the European Commission, 11. DG Environment by the Science Communication Unit, UWE, Bristol, 32 pp. URL: http://ec.europa.eu/science-environment-policy

* Scolozzi R, Morri E, Santolini R (2012) Delphi-based change assessment in ecosystem service values to support strategic spatial planning in Italian landscapes. Ecological Indicators 21: 134-144. https://doi.org/10.1016/j.ecolind.2011.07.019

* Shackleton CM, Ruwanza S, Sinasson Sanni GK, Bennett S, De Lacy P, Modipa R, Mtati N, Sachikonye M, Thondhlana G (2016) Unpacking Pandora's Box: understanding and categorising ecosystem disservices for environmental management and human wellbeing. Ecosystems 19 (4): 587-600. https://doi.org/10.1007/s10021-015-9952-z

* Sinare H, Gordon L, Enfors Kautsky E (2016) Assessment of ecosystem services and benefits in village landscapes - A case study from Burkina Faso. Ecosystem Services 21: 141-152. https://doi.org/10.1016/j.ecoser.2016.08.004

* Sohel MSI, Ahmed Mukul S, Burkhard B (2015) Landscape's capacities to supply ecosystem services in Bangladesh: A mapping assessment for Lawachara National Park. Ecosystem Services 12 (128-135). https://doi.org/10.1016/j.ecoser.2014.11.015

* Stoll S, Frenzel M, Burkhard B, Adamescu M, Augustaitis A, Baeßler C, Bonet F, Carranza ML, Cazacu C, Cosor G, Díaz-Delgado R, Grandin U, Haase P, Hämäläinen H, Loke R, Müller J, Stanisci A, Staszewski T, Müller F (2015) Assessment of ecosystem integrity and service gradients across Europe using the LTER Europe network. Ecological Modelling 295: 75-87. https://doi.org/10.1016/j.ecolmodel.2014.06.019

* Sun J, Liu L, Müller K, Zander P, Ren G, Yin G, Hu Y (2018) Surplus or Deficit? Spatiotemporal variations of the supply, demand, and budget of landscape services and landscape multifunctionality in suburban Shanghai, China. Sustainability 10 (10). https://doi.org/10.3390/su10103752

* Syrbe R, Grunewald K (2017) Ecosystem service supply and demand – the challenge to balance spatial mismatches. International Journal of Biodiversity Science, Ecosystem Services & Management 13 (2): 148-161. https://doi.org/10.1080/21513732.2017.1407362

* Tamang B (2011) An Assessment of Ecosystem Services of the Everest Region, Nepal. Universität zu Kiel, 222 pp.

* Tao Y, Wang H, Ou W, Guo J (2018) A land-cover-based approach to assessing ecosystem services supply and demand dynamics in the rapidly urbanizing Yangtze River Delta region. Land Use Policy 72: 250-258. https://doi.org/10.1016/j.landusepol.2017.12.051
* Távora GG, Turetta A (2016) An approach to map landscape functions in Atlantic Forest —Brazil. *Ecological Indicators* 71: 557-566. https://doi.org/10.1016/j.ecolind.2016.07.005

* Van Looy K, Tormos T, Souchon Y, Gilvear D (2017) Analyzing riparian zone ecosystem services bundles to instruct river management. *International Journal of Biodiversity Science, Ecosystem Services & Management* 13 (1): 330-341. https://doi.org/10.1080/21513732.2017.1365773

* van Oudenhoven AE, Schröter M, Drakou E, Geijzendorffer I, Jacobs S, van Bodegom P, Chazee L, Czúcz B, Grunewald K, Lillebø A, Mononen L, Nogueira AA, Pacheco-Romero M, Perennou C, Remme R, Rova S, Syrbe R, Tratalos J, Vallejos M, Albert C (2018) Key criteria for developing ecosystem service indicators to inform decision making. *Ecological Indicators* 95: 417-426. https://doi.org/10.1016/j.ecolind.2018.06.020

* Vihervaara P, Kumpula T, Tanskanen A, Burkhard B (2010) Ecosystem services–A tool for sustainable management of human–environment systems. *Case study Finnish Forest Lapland. Ecological Complexity* 7 (3): 410-420. https://doi.org/10.1016/j.ecocom.2009.12.002

* Villoslada M, Vinogradovs I, Ruskule A, Veidemane K, Nikodemus O, Kasparinskis R, Sepp K, Gulbinas J (2018) A multitiered approach for grassland ecosystem services mapping and assessment: The Viva Grass tool. *One Ecosystem* 3 https://doi.org/10.3897/oneeco.3.e25380

* von Döhren P, Haase D (2015) Ecosystem disservices research: A review of the state of the art with a focus on cities. *Ecological Indicators* 52: 490-497. https://doi.org/10.1016/j.ecolind.2014.12.027

* Wangai PW, Burkhard B, Müller F (2018) Quantifying and mapping land use changes and regulating ecosystem service potentials in a data-scarce peri-urban region in Kenya. *Ecosystems and People* 15 (1): 11-32. https://doi.org/10.1080/21513732.2018.1529708

* Weyland F, Barral MP, Laterra P (2017) Assessing the relationship between ecosystem functions and services: Importance of local ecological conditions. *Ecological Indicators* 81: 201-213. https://doi.org/10.1016/j.ecolind.2017.05.062

* Wu X, Liu S, Zhao S, Hou X, Xu J, Dong S, Liu G (2019) Quantification and driving force analysis of ecosystem services supply, demand and balance in China. *Science of The Total Environment* 652: 1375-1386. https://doi.org/10.1016/j.scitotenv.2018.10.329

* Xie GD, Zhen L, Lu CX, Xiao Y, Chen C (2008) Expert knowledge based valuation method of ecosystem services in China. *J. Nat. Resour.* 23 (5): 911-919. [In Chinese].

* Zarandian A, Baral H, Yavari A, Jafari H, Stork N, Ling M, Amirnejad H (2016) Anthropogenic decline of ecosystem services threatens the integrity of the unique Hyrcanian (Caspian) Forests in Northern Iran. *Forests* 7 (3). https://doi.org/10.3390/f7030051

## Supplementary material

### Suppl. material 1: Review Methodology doi

**Authors:** Campagne

**Data type:** Methodology and list of papers in the review

Download file (287.57 kb)