# What is the best way to communicate the uncertainty of a digital soil mapping product? Some lessons from an end-users survey

Léa Courteille, Léa Tardieu, Nadia Boukhelifa, Evelyne Lutton, Philippe Lagacherie

► To cite this version:

Léa Courteille, Léa Tardieu, Nadia Boukhelifa, Evelyne Lutton, Philippe Lagacherie. What is the best way to communicate the uncertainty of a digital soil mapping product? Some lessons from an end-users survey. Geoderma, 2025, 459, pp.117302. 10.1016/j.geoderma.2025.117302. hal-05121068

HAL Id: hal-05121068

https://hal.inrae.fr/hal-05121068v1

Submitted on 19 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# What is the best way to communicate the uncertainty of a digital soil mapping product? Some lessons from an end-users survey

Léa Courteille<sup>a, ID</sup>, Léa Tardieu<sup>b, c ID</sup>, Nadia Boukhelifa<sup>d, e</sup>, Evelyne Lutton<sup>d</sup>, Philippe Lagacherie<sup>a, *</sup>

## ARTICLE INFO

Handling Editor: Budiman Minasny

**Keywords:**

Uncertainty visualization

Spatial aggregation

End-users

Web-based survey

Digital soil mapping

Soil quality

## ABSTRACT

Uncertainty in digital soil mapping products is generally quantified and presented alongside the predictions in the form of a second raster map. However, it remains a challenge for end-users to integrate this additional information into their decision-making, and as a result, they tend to ignore uncertainty. The Digital Soil Mapping (DSM) literature has identified two levers for better communicating uncertainty: uncertainty representations that allow users to integrate uncertainty better than with a separate raster map, and spatial aggregation, which consists of representing the spatial information at a coarser resolution to reduce its uncertainty. For the first time in DSM, we involved a large number of end-users (263 people) via a web-based survey to evaluate alternative cartographic representations of soil map uncertainty. Respondents were assigned a decision-making task and asked to select the map that offered the best support, choosing between two maps that displayed the same soil information but differed in how uncertainty was represented and/or in the level of spatial aggregation. The choices made by the respondents as well as the comments they left at the end of the study provided a large dataset from which we could identify factors driving their preferences. We demonstrate that end-users strongly prefer uncertainty to be shown on a separate map, and also favor moderately aggregated maps. Our results question the conventional representation of soil data using high-resolution and high-uncertainty maps. We hope that this work will be valuable to scientists in soil science, as well as in other environmental mapping communities, such as ecosystem services and biodiversity, and help them more effectively communicate uncertainty to their end-users.

## 1. Introduction

Soil maps, just like most environmental maps (Jansen et al., 2022; Grêt-Regamey et al., 2013), are subject to a number of sources of error (Wadoux and Heuvelink, 2023) that introduce uncertainty into all displayed values. As argued by Jansen et al. (2022), presenting uncertainty maps alongside the mapped predictions is important, since it can help highlight where data deficiencies exist and where decision makers need to be cautious when using these maps for decision-making. On the other hand, ignoring uncertainty can lead to misuse and harmful decisions (Heuvelink, 2014; Grêt-Regamey et al., 2013; Richer-de Forges et al., 2019). However, it remains a challenge for end-users to understand and integrate this additional information into their decision-making (Wadoux et al., 2021; Lark et al., 2022; Chagumaira et al., 2020), and as a result, it tends to be disregarded by end-users (Richer-de Forges et al., 2019; Boukhelifa and Duke, 2009; Boukhelifa et al.,

2017). It is therefore necessary to develop methodologies for enhancing the conceptual and visual clarity of uncertainty. Of all other environmental mapping communities, the Digital Soil Mapping community has been at the forefront of uncertainty representation (Heuvelink, 2014), notably since the GlobalSoilMap project (Arrouays et al., 2014) and its ambition to quantify and communicate the uncertainty of all maps produced.

In the DSM literature, two levers have been identified for better representing uncertainty. The first one is uncertainty visualization. In DSM, this information is typically presented as a separate map, showing uncertainty values at the pixel support, for example as a width of some confidence interval. Most studies that question this approach of representing uncertainty on a separate map have focused on combining predictions and uncertainty values in a visually inseparable

<footnote><sup>a</sup> LISAH, INRAE, Institut Agro Montpellier, IRD, Univ Montpellier, Montpellier, France</footnote>

<footnote><sup>b</sup> TETIS, INRAE, AgroParisTech, CIRAD, CNRS, Univ Montpellier, Montpellier, France</footnote>

<footnote><sup>c</sup> CIRED, École des Ponts ParisTech, AgroParisTech, université Paris-Saclay, Cirad, CNRS, EHESS, Nogent-sur-Marne, France</footnote>

<footnote><sup>d</sup> MIA-PS, INRAE, AgroParisTech, Université Paris-Saclay, 22 place de l'Agronomie, Palaiseau, France</footnote>

<footnote><sup>e</sup> UAR 3611 Institut des Systèmes Complexes Paris Île de France, CNRS, Paris, France</footnote>

<footnote>* Corresponding author.
E-mail addresses: lea.courteille@inrae.fr (L. Courteille), philippe.lagacherie@inrae.fr (P. Lagacherie).</footnote>
manner using bivariate maps (Loiseau et al., 2020; Hengl et al., 2004), to prevent users from considering only the predicted values. The second lever is spatial aggregation, which consists in representing the spatial information at a coarser resolution to decrease prediction uncertainty (Vaysse et al., 2017; Wadoux and Heuvelink, 2023; Padarian and McBratney, 2023; Courteille et al., 2024). While this loss of spatial detail was found to be useful to communicate uncertainty to users who tend to associate low spatial resolution with high uncertainty (Padarian and McBratney, 2023), it might be detrimental for decision-making if too much spatial detail is lost. This can be partly mitigated by using a spatial aggregation algorithm designed for reducing this loss as far as possible (Courteille et al., 2024). However, spatial aggregation always introduces a trade-off between the amount of uncertainty and the level of spatial detail (Courteille et al., 2024), and it is still unknown where users' preferences lie along this compromise.

A few contributions questioned whether the conventional “high-resolution, high-uncertainty” representation of soil information is the most appropriate for decision-making (Arrouays et al., 2020; Vaysse et al., 2017). This issue was also studied in other environmental mapping communities. Notably, Arponen et al. (2012) investigated the impact of spatial resolution on decision-making. Their findings suggest that it is best to use high-resolution data, but they did not take into account the high prediction uncertainties associated with it. They however pointed out that the benefits of using fine-resolution data decrease when management units are much larger than the cell size. Other studies investigated how to present environmental information to support decision-making (Willemen et al., 2015; Klein et al., 2015), but these studies only quickly evoke uncertainty.

Similarly, the usefulness of the “separate map” uncertainty visualization for decision-making has been widely questioned in environmental mapping. Numerous studies have aimed to empirically compare visualization methods and identify user preferences, but they usually involve college students instead of professional end-users (Scholz and Lu, 2014; Kubíček and Šašinka, 2011). In the DSM literature, some innovative representations of uncertainty information have been introduced (Loiseau et al., 2020; Bonfatti et al., 2016; Hengl et al., 2004). Loiseau et al. (2020) encourage the soil mapping community to evaluate uncertainty representations in consultation with end-users, to ensure that maps are better used and accepted. Assessing users' preferences in the context of a domain-specific task (here, land planning) enables the identification of user needs and expectations, as well as the assessment of the usefulness of alternative representations of uncertainty for decision-making (Carpendale and S, 2008; Chagumaira et al., 2020). Notably, Chagumaira et al. (2020, 2022) engaged end-users to analyze their perception and preferences for different methods of communicating uncertainty, such as prediction intervals versus kriging variances or probabilities, but these levers to better represent uncertainty are beyond the scope of our study. To our knowledge, no study has involved users of soil maps (and even environmental maps) to identify their preferences regarding spatial resolution and uncertainty visualization.

This study is an attempt to identify (1) users' preferences regarding uncertainty visualization and (2) where these preferences lie along the trade-off between uncertainty and spatial resolution. We conducted a web-based survey, in which respondents were assigned a decision-making scenario and shown two alternate representations having varying uncertainty representations and/or aggregation level of the same soil quality spatial information. The respondents were then asked which option they deemed more adapted for making a specified land-planning decision. The choices made by the 263 respondents provided a large dataset from which users' preferences and factors influencing them were analyzed.

# 2. Materials and methods

## 2.1. Survey design

### 2.1.1. Overall survey structure

We designed an online survey using the LimeSurvey GmbH (2024) software that consisted in four main parts. The first part collected information on the respondents in relation to their socio-demographic characteristics and their experience of using or creating soil maps. These questions notably enabled us to obtain descriptive statistics of the sampled population in view of evaluating its representativeness with regard to known populations of users.

The second part of the survey comprised a tutorial designed to familiarize the respondents with the reading of the maps and to ascertain their ability to retrieve the values they had read in the presented maps. We presented four different maps with varying characteristics in terms of soil quality, aggregation level and uncertainty. We then asked them to indicate the uncertainty visualization (in the first two questions) or aggregation level (in the two last questions) of the map, and to read a soil quality value and an uncertainty value using the color/grayscale palettes. The tutorial ends with two examples where the user is asked to make a decision for the outlined plot with instructions specifying a decision rule: either to convert plots with a soil quality below a given threshold to artificial land, or to preserve plots with a soil quality above a given threshold (in both cases, the decision options are Yes/No/I am unable to decide). All questions were followed by corrections to clarify any ambiguities the respondents might have had regarding the visual encodings of maps or the task's decision rule. The reading exercise enabled us to detect a *posteriori* users who had trouble understanding the exercise.

In the third part, respondents were asked to make a decision regarding a particular plot, using different map representations of soil quality data. We designed two survey variants. In the first one, respondents were asked to select whether the delineated plots should be converted into an artificial area (“convert” scenario). In the other survey variant, the remaining respondents were asked to select whether to designate the outlined plots as protected areas (“conserve” scenario). Participants were assigned randomly to one of these two variants and made ten land-planning decisions. In the context of the present study, this part of the survey allowed respondents to get some experience of decision-making under uncertainty before eliciting their preferences.

The fourth section of the survey consisted of a series of ten choices between two different cartographic representations of the same spatial context (see Fig. 1). Choices of representation and spatial contexts are explained in the following subsection.

### 2.1.2. Maps production and selection over the maps' varying attributes

The evaluation of end-users' preferences aims at assessing the potential of two levers that, according to the literature, may help to better communicate uncertainty: uncertainty visualization and spatial aggregation. We designed a series of maps with different modalities of these two variables to cover the range of possible representations.

We relied on a map of a soil quality index called the soil potential multifunctionality index (Agri-SPMI) developed for the coastal plain of the Occitanie region, located in Southern France (Rabot et al., 2022; Angelini et al., 2023). This soil quality index evaluates whether a soil fulfills the four following functions on five alternative natural/semi natural and agricultural land-uses<fnref n="1" />: (i) provision of a physical habitat for plant growth, (ii) provision of chemical habitat for plant growth, (iii) retention and transfer of water and pollutants, and (iv) carbon

<footnote n="1">¹ This approach ensures that only perennial soil functions are investigated. The five land-uses are arable lands (winter crops and summer crops), pastures (meadows and lawns), perennial crops (orchards, vineyards), forests (deciduous or coniferous forests) and scrublands (woody heaths).</footnote>
<figure><img src="image_3.png" /></figure>

Step 4: Choice of cartographic representation of soil quality data

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Example of choice submitted to respondents via our Limesurvey questionnaire. Note that in the actual survey, to optimize readability, “separate maps” took up the entire screen width, while “dot pattern” maps took up half of it.</figcaption></figure>

storage. By summing the number of functions fulfilled for the five land uses, we obtain a soil quality score ranging from 0 to 20 (and in practice ranging from 15 to 20 in our study area).

A section of this map, spanning an area approximately 2500 km² around the city of Nîmes, selected for its diversity in terms of Agri-SPMI and associated uncertainty, was derived into a series of 14 maps with different levels of aggregation using a simple spatially constrained algorithm (called “Agglomerative clustering” Ward, 1963). Following the procedure described in Courteille et al. (2024), these maps were compared with respect to their uncertainty levels and information contents. We selected four contrasting levels of aggregation corresponding to different positions along the uncertainty/spatial resolution compromise (see Fig. 2) in order to capture, in as few representation options as possible, the variability of uncertainty and information content. Depending on their aggregation level, we will refer to maps using different names: raster maps for aggregation level 0, and slightly/moderately/heavily aggregated maps for levels 1, 2, and 3 respectively. Note that for aggregated maps, the soil quality levels at zonal support were represented as the areal proportion of high-quality soils (i.e. with Agri-SPMI ≥ 18) within the zone (like in Courteille et al. (2024)). Soil quality values are therefore communicated either as a soil quality index ranging from 15 to 20 (in raster maps) or as a proportion of high-quality soils ranging from 0 to 100% (in aggregated maps), using a color palette divided into 11 intervals. The color palette encoding soil quality values was selected from (Harrower and Brewer, 2003): we chose the “BrBG” diverging color scheme for its accessibility to colorblind individuals.

The prediction errors, or uncertainty values, associated with every displayed value were quantified (following the method described in Courteille et al. (2024)), and represented via a 4-levels grayscale palette. The uncertainty information was displayed using two different static visualizations to test whether end-users prefer an adjacent view (separate map) or a coincident view referring to maps where uncertainty is integrated in the same display as the soil quality data (MacEachren et al., 1998). The adjacent view is a simple and commonly used visualization in digital soil mapping, featuring a separate uncertainty map shown alongside the soil quality map. When choosing a coincident representation of uncertainty, we needed

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Evolution of total prediction uncertainty during spatial aggregation, starting from a raster map. Points corresponding to the 4 selected maps are highlighted with square symbols — circles correspond to the 10 other aggregated maps.</figcaption></figure>

to use the same visual variable – color value – as in the adjacent representation to ensure that uncertainty was equally easy to interpret in both visualizations. We decided to use glyphs, which are graphical objects typically superimposed on a map, useful for representing uncertainty (MacEachren et al., 2005). We used glyphs in their most simple form, namely dot symbols that depict uncertainty through their color value: their gray level changes depending on the uncertainty value they indicate. We will from now refer to these two visualizations as *S* (separate map) and *D* (dot pattern) uncertainty visualizations.

To evaluate preferences, we presented users with sections of the four selected maps, at a scale of 1:12 000 and covering an area of approximately 1400 × 1800 m (see Figs. 3 and 4). These map sections are designed to reflect the types of visualizations commonly used by
<figure><img src="image_6.png" /><figcaption>Fig. 3. Four *map sections* with the S uncertainty visualization, representing the same spatial context at four different spatial resolutions. In the web-based survey, maps were displayed at the maximum possible size, filling the entire width of the screen.</figcaption></figure>

stakeholders – who frequently make decisions at the administrative level (Arrouays et al., 2020) – while maintaining clarity and readabil- ity. It should be noted that these map sections did not include any indication of their real location in order to avoid biasing respondent’s decision. We added small sets of randomly selected outlined plots on the maps, in order to help participants remember the scale at which land-planning decisions are expected.

Our initial intuition was that the preferences of users regarding both
aggregation level and uncertainty visualization might vary depending
on the spatial context. We therefore sampled 15 spatial extents across
the study area using Latin Hypercube Sampling based on 4 parameters
describing the spatial context of these extents:

- mean soil quality over the section area,

- heterogeneity of soil quality values over the section area,

- mean uncertainty over the section area,

- heterogeneity of uncertainty values over the section area

For each of these 15 spatial contexts, 8 alternative representations
(4 aggregation levels × 2 visualizations) of the soil quality information
were submitted to respondents. 120 map sections, generated from the
4 selected maps covering the study area, were therefore evaluated. For
simplicity, and since the 4 initial maps will not be evoked anymore, we
will refer to these 120 map sections as “maps”.

2.1.3. Survey dissemination

The survey was distributed between March and July 2024. Several
strategies were used to disseminate the survey to all relevant stake-
holders. We disseminated the survey to many professional networks
through mailing lists, by posting messages on relevant forums, and
by introducing our survey at relevant workshops and conferences. A
list of all the networks and organizations contacted can be found in
the Table A.1 in the appendix. In order to reach more professional
networks, we used the snowball strategy where we reached out to three
network coordinators and administrators for further dissemination:
French national network “Zero Net Land Take” and environmental au-
thorities, the National Federation of Urban Planning Agencies, and the

National Federation of Territorial Coherence Schemes. This dissemina-
tion strategy has been designed in order to cover the range of potential
study participants working in areas relevant to our research questions,
namely geomaticians, urban planners, consultants and researchers.

2.1.4. Thematic analysis

At the very end of the questionnaire, respondents were allowed to
leave comments in response to the following question: "Do you have any
remarks about this survey?". These responses were then coded following
a thematic analysis approach (Clarke and Braun, 2017). This method
allowed us to identify key themes and capture main views shared
by many respondents. The themes that emerged were pooled and
discussed among three authors. Participants' quotes, when necessary,
were translated from French into English.

## 2.2. Data cleaning and statistical analysis

### 2.2.1. Identification of outliers

The aim of this step was to remove entries from respondents who
had too much difficulty reading the maps presented, who did not under-
stand the exercise correctly, or who answered without paying attention
to the task. Four criteria were used to identify these respondents:

1. Reading errors: We excluded the 5% of respondents who made the most errors in the tutorial phase using the Isolation Forest algorithm (Liu et al., 2008). Using the answers to the 12 tutorial questions, this algorithm detects the anomalous responses thanks to an anomaly score derived from tree-based random partitions. In the case of the color-reading questions, we considered the answer to be correct if the respondent indicated the correct shade or one level away from it (As such, 16.5 and 17.5 are both acceptable answers for map value of 17, as are 35% and 45% for a value of 40%).

2. Time answering the survey: We discarded respondents who took less than 15 min to complete the survey, as pilot surveys showed that this was the minimal time to provide meaningful answers.
<figure><img src="image_7.png" /><figcaption>Fig. 4. Four *map sections* with the D uncertainty visualization, representing the same spatial context at four different spatial resolutions. In the web-based survey, maps were displayed at the maximum possible size, filling half of the screen width.</figcaption></figure>

3. The same systematic response to all questions: we considered respondents who systematically selected the same land-planning decision (e.g. decided to convert all plots that were submitted to them to artificial area) as “protest answers”.

4. Poor understanding: we discarded respondents who, when asked to what extent they had understood the exercise, selected “not at all”.

### 2.2.2. Ranking maps from users' choices

A ranking procedure was designed to assess respondents' preferences regarding the representation of soil quality information for decision-making purposes. Map options were presented in pairs. The respondents were asked to select the option they saw as “more adapted for making land planning decisions in the frame of the directive [they] had previously applied”. Such pairwise comparison schemes have been successfully applied in the literature to identify preferences for choosing bicycling locations (Evans-Cowley and Akar, 2014) or to assess the aesthetic value of fishes (Langlois et al., 2022). Ranking might however encourage respondents who do not have a real preference order to answer in a superficial manner (Klein et al., 2004) and give rank differences which do not reflect differences in preference (Russell and Gray, 1994), therefore adding noise to the data. We tried to limit this effect by allowing respondents to select one of the following answers: “I have no preference. Both maps are suitable to use to classify parcels of

land according to the directive applied up to now”, or “I have no preference. Neither maps are suitable to use to classify parcels of land according to the directive applied up to now”.

Out of the several methods available for ranking objects from a series of pairwise comparisons, we followed the recommendation of Goodspeed (2017) and chose the Elo rating system to rank the evaluated maps.

The Elo rating is a rating system based on pairwise comparison and relying on the Elo algorithm, which was originally invented by physicist and chess player Arpad Elo to rank chess players (Elo and Sloan, 1978/2008). The Elo algorithm determines a unique rating for each image by iteratively considering the result of each pairwise comparison — here regarded as a match between two players, where the map that was chosen wins.

According to the Elo algorithm, after a match between two players, points are exchanged based on both players' prior probability of winning the match. Each player starts with a score based on their previous wins and losses, and the difference between the scores is used to predict the probability of outcomes according to the following formulas, extracted from Clark et al. (2018):

$$E_1 = \frac{1}{1 + 10^{-\frac{S_2-S_1}{400}}}$$

$$E_2 = \frac{1}{1 + 10^{-\frac{S_1-S_2}{400}}}$$
“where E₁ and E₂ are the estimated probabilities of a win for players 1 and 2 respectively, and S₁ and S₂ their current Elo scores. Players with equal scores are expected to be equally likely to win. After each match, Elo scores are updated according to the outcome of the duel, according to the following formulas:”

$$S'_{1} = S_{1} + k(O_{1} - E_{1})$$

$$S'_{2} = S_{2} + k(O_{2} - E_{2})$$

“where O₁ and O₂ are the actual outcomes of the game for players 1 and 2 respectively” (win = 1, loss = 0, tie = 0.5), S'₁ and S'₂ are their new scores, and k is a constant setting how quickly scores change but, given a sufficient number of matches, it will have a limited effect on player rankings. We set the k-factor to 32 since it is the most commonly used value in chess (Goodspeed, 2017).

It is important to note that the same contest outcomes will produce different values of Elo scores depending on the order in which they are observed. This is relevant when the targets being scored are entities that may become weaker or stronger with time (such as players gaining or losing skills), but we consider that maps are static in this regard and hence the order of events is not capturing any useful information. As a result, any variation in final scores introduced by sequence variation is undesirable noise. To counter this problem, we followed the recommendation from Clark et al. (2018): we shuffled randomly the original sequence of matches multiple times (we did so 100 times) and computed the resulting Elo scores. We then averaged the 100 resulting Elo scores for each map, which gives us a robust Elo score that we consider free from sequence noise.

Elo scores are most of the time computed from a single round-robin tournament, where each contestant meets every other participant. In this study, the 120 evaluated maps are split into 15 groups according to the spatial extent they cover, each group containing 8 alternative representations of the same spatial information. Only maps from the same group can meet each other: Elo scores are therefore computed from 15 round-robin tournaments.

## 2.3. Linking elo scores to map characteristics

From there, we tried to identify trends and relationships between user preferences and map characteristics. We computed descriptors for all of the 120 maps, such as their mean spatial resolution, mean and variance of uncertainty values, mean and variance of soil quality values, and performed a basic descriptive data analysis (scatter plots, boxplots...) to identify factors which might be related to Elo scores of the maps.

## 2.4. Positioning maps along the uncertainty/spatial resolution trade-off

According to Whelan and McBratney (1999), the value of a map for decision-making depends on two factors: the displayed attribute values and the displayed uncertainty values. In Courteille et al. (2024), we introduced metrics to quantify these two factors from a map whose set of zones or pixels is noted Z, and which allow to position each map along the uncertainty/spatial resolution trade-off we highlighted. These are:

* the variance of predicted attribute values:

$$I = \frac{1}{\Lambda} \sum_{z \in \mathcal{Z}} \mathcal{A}_z(\mu_z - \bar{\mu})^2 \qquad (1)$$

where A_z is the area of zone z ∈ Z, A the total area of the map, μ_z the attribute value of zone z, and μ the mean attribute value over the whole map, computed as follows:

$$\bar{\mu} = \frac{1}{\Lambda} \sum_{z \in \mathcal{Z}} \mathcal{A}_z \mu_z \qquad (2)$$

* the area-weighted sum of prediction uncertainties:

$$U = \frac{1}{\Lambda} \sum_{z \in \mathcal{Z}} \mathcal{A}_z \mathrm{Var}(\mu_z) \qquad (3)$$

where Var(μz) is the prediction uncertainty of μz (see Courteille et al. (2024) for more details).

These two metrics were computed for each map and standardized within each spatial context by using the mean and standard deviation of I and U for the given spatial context. The compromise between total uncertainty and total information content was unveiled by subtracting U from I.

To visualize how respondents' preferences align along the trade-off between information content and uncertainty, we calculated the relative ranking of each aggregation level across the 15 spatial contexts. To isolate preferences related solely to aggregation level – excluding the influence of uncertainty visualization – we excluded D-maps from the analysis and computed Elo scores based only on choices between S-maps. Aggregation levels were then ranked accordingly for each spatial context.

# 3. Results

## 3.1. Outliers' identification

We excluded invalid answers based on the four criteria presented earlier: (1) reading errors, (2) time answering the survey (less than 15 min when the median response time was 31 min and 4 s), (3) protest answers and (4) self-reported failure to understand the exercise. The first and second criteria resulted in discarding 14 and 3 responses respectively while the last two criteria were not met.

The time taken by respondents to make a choice between two maps varied between around 1 s and 3 h 20 (in cases where the survey browser tab was left open). Out of the 2630 collected choices, 240 were made in less than a second: we considered that they could be random answers as well as answers revealing strong preferences (e.g. S-maps versus D-maps), and therefore decided to keep them. We however made sure that our results did not change with these choices removed.

## 3.2. Sample characteristics

Out of the 1738 persons who clicked on the link to our online survey, 280 people completed the decision-making stage. After removing the 17 outlier respondents as identified by the procedure described in Section 2.2.1, the final sample included 263 survey participants.

The sample slightly over-represents male and highly educated respondents. Nonetheless, it may still reflect the targeted population of environmental geodata users and producers involved in planning decisions. To assess sample representativeness, we reached out to the administrators of two major professional networks within this field, namely AFES (French Association for the Study of Soils) and GeoRezo (network for all French-speaking geomaticians) to obtain data describing the characteristics of their members. We observe the same gender and education bias (Fig. 5).

The 263 respondents work in a wide diversity of organizations (see Fig. 6). Researchers are overrepresented (25%). Among the 57 respondents who did not indicate where they work, we notice a high share of students (19, or 33%).

The vast majority of our respondents come from France (237), but we gathered 25 responses from other countries: Algeria (2), Australia (1), Brazil (8), Czech Republic (1), Germany (1), Hungary (1), Italy (1), Morocco (1), the Netherlands (7), Tunisia (1) and the United States (1). 60% of the non-French respondents work in a research institute or a university.

## 3.3. Global preferences

Elo scores for all maps were computed from the 263 × 10 choices made by respondents and were related to their uncertainty visual-
<figure><img src="image_8.png" /><figcaption>Fig. 5. To assess the representativeness of our sample compared to the target population, we compared its characteristics with those of larger groups that bring together potential end-users.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 6. Respondents' workplaces.</figcaption></figure>

ization and aggregation level. One-way ANOVAs were conducted to determine whether the observed differences between the average Elo score of maps for each aggregation level were statistically significant. Tukey's post-hoc tests were used to perform pairwise comparisons between groups. Fig. 7 shows that end-users significantly preferred S-maps over D-maps, and prefer moderately aggregated maps (level 2) over raster and heavily aggregated ones (levels 0 and 3). Slightly

aggregated maps (level 1) do not have significantly higher or lower Elo scores than maps with different aggregation levels.

### 3.4. Factors influencing preferences

We noted no difference in the preferences across the different end-user groups, and notably researchers versus practitioners. Moreover, we
<figure><img src="image_10.png" /><figcaption>**Fig. 7.** Boxplots of Elo scores according to the modalities of the two variables tested in the frame of our study: uncertainty visualization and aggregation level.</figcaption></figure>

also controlled for many other parameters such as expertise in production of soil maps and soil uncertainty or on use of soil maps and soil uncertainty, confidence, global easiness, which also have no significant impact in explaining any heterogeneity of preferences. Moreover, we observed that preference patterns remained consistent whatever the decision respondents were asked to make in the survey (convert or conserve land).

Our descriptive analysis also showed that none of computed map descriptors (mean spatial resolution, mean and variance of uncertainty values, mean and variance of soil quality values) had a clear link with the Elo scores of the maps. Clear patterns could not be identified even when focusing on some groups of respondents: researchers, practitioners, experts in soil data, or by splitting the dataset according to the decision-making scenario, gender, or education level.

### 3.5. Preferences across spatial contexts

The expected trade-off between uncertainty and information content, as introduced by Courteille et al. (2024), was observed in all but one of the 15 spatial contexts. Both uncertainty and information content monotonously decreased with spatial aggregation (see Figs. 8 and 9). However, for spatial context E, the quantity of information seemed to dramatically increase for the highest aggregation level. This was due to the presence of a large zone with an extreme soil quality value which maximized the variance of the predicted values (see Fig. 11).

Consequently, as Fig. 10 shows, the I-U quantity, highlighting how optimal the representation is with respect to the uncertainty/spatial resolution trade-off, is always maximized at intermediate aggregation levels with the noticeable exception of spatial extent E that was evoked above. Moreover, in 10 out of the 15 spatial contexts, the preferred representation is among the two optimal representations along the highlighted compromise (A, C, F, G, H, I, J, K, M, N). In three spatial contexts (B, L and O) the respondents significantly preferred a map with a larger level of aggregation than the one matching the best compromise. Among the three, it is worth noting that O exhibited an extremely high level of information content which may explain this preference. Besides, in two spatial contexts (D and E), respondents significantly preferred a higher spatial resolution map than the one matching the compromise. Interestingly, spatial context D was the only one for which respondents preferred the usual raster representation. This can be explained by the extremely low level of uncertainty observed for this spatial context. Preferences therefore remain unexplained in only 3 out of the 15 studied spatial contexts.

### 3.6. Results of the thematic analysis

98 comments were left by respondents, whose length ranged from 1 to 356 words. They were broken down into 178 pieces of text (snipets), each of them expressing a single idea, and were assigned descriptive labels. The 9 labels were grouped into 3 main themes: *General comments, Comments on the D uncertainty visualization, Comments on aggregated maps*. The main expressed ideas are listed in Table 1.

Opinions about the survey were overall positive. A high number of respondents expressed their interest in the approach, and enjoyed being confronted to the complexity of deciding from uncertain data. Some respondents emphasized that the presented maps would be very useful from an operational point of view. Finally, many comments indicated that the survey was well constructed, often mentioning the benefits of the tutorial stage.

The comments on the D uncertainty visualization are coherent with what the global analysis clearly demonstrated, and indicate that S-maps are preferred over D-maps due to readability issues.

Negative comments on aggregated maps focus for the most part on their insufficient spatial resolution. Surprisingly, the positive comments about aggregated maps only evoke their improved readability, but none mention their reduced uncertainty.

## 4. Discussion

### 4.1. Survey methodology

As advocated by many authors in the Digital Soil Mapping community (Wadoux et al., 2021) and beyond (Neuendorf et al., 2018; Grêt-Regamey et al., 2013), we engaged with users of environmental data to identify meaningful representations of spatial uncertainty that help them make decisions. To the best of our knowledge, our study was the first to do so: Loiseau et al. (2020) and Kunz et al. (2011) had already applied alternative uncertainty visualizations respectively to soil maps and natural hazard maps, with the aim of better conveying
<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Evolution of the total uncertainty *U* of the submitted maps with spatial aggregation, for each of the 15 selected spatial contexts. Each point represents a map, with a given aggregation level: 0, 1, 2 or 3.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>**Fig. 9.** Evolution of the total information content *I* of the submitted maps with spatial aggregation, for each of the 15 selected spatial contexts.</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>**Fig. 10.** Evolution of the *I*–*U* of the submitted maps with spatial aggregation, for each of the 15 selected spatial contexts. The color of each dot indicates if it was judged to be the first, second, third or fourth most preferable representation of the spatial context by end-users.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>(a) Raster map representation of the spatial context *D*, which displays low uncertainty compared to other raster representations.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>(b) Very aggregated map representation of the spatial context *O*, which is highly contrasted and informational compared to other very aggregated representations.</figcaption></figure>

<caption>**Fig. 11.** Examples of maps that were preferred by respondents despite their suboptimal position along the uncertainty/spatial resolution trade-off.</caption>

the notion of uncertainty to users, but these studies did not involve end-users to verify whether this objective was met.

We designed a survey structured in four stages. The second stage is a tutorial which includes a map-reading exercise and two practice questions to prepare for the third stage. This tutorial allowed us to make sure they were able to interpret and use the visualized uncertainty information. The third stage is a decision-making exercise. We claim that this stage made the respondents aware of the complexity of making land-planning decisions from uncertain data before eliciting their preferences in the fourth and last stage. As a consequence, we assume

that their preferences reflect the ability of maps to support decision-making, and not other irrelevant criteria such as the aesthetic aspect of maps.

We collected a representative sample of 263 responses. Most of the general feedback we received was positive, and confirmed decision makers’ current needs for new ways to create maps that are both user-friendly (easy to read) and informative, in order to better assist decision-making. Thanks to the tutorial stage, the respondents’ self-reported levels of understanding are high, and we did not spot any protest answers that might have indicated their disengagement.
<table><caption>**Table 1**
Results of the thematic analysis showing the main ideas expressed through respondents' comments.</caption><thead><tr><th>Theme</th><th>Code</th><th>n of respondents contributing</th><th>Sample quotes</th></tr></thead><tbody><tr><td rowspan="4">Relevance of the study</td><td>Positive (insightful)</td><td>32</td><td>"It was very interesting to put yourself in the shoes of an elected official who is faced with tools that can sometimes be complex"; "Very interesting study on the representation of data and the perception we can have of it"; "This is a very interesting study that addresses a weak point in digital soil maps: their poor usability for non-specialist users".</td></tr><tr><td>Positive (relevant for decision makers)</td><td>7</td><td>"In my opinion, this type of representation would be a great help in understanding/deciding whether or not a plot of land should be artificialised, particularly in the 'Zero net land take' context"; "A project that will be extremely useful for local councils!"</td></tr><tr><td>Positive (well designed)</td><td>12</td><td>"Well designed (tutorial and choice of examples to analyse)"; "The tutorial helped a lot to make the survey easy to understand".</td></tr><tr><td>Negative (too tedious for decision makers)</td><td>16</td><td>"Operationally speaking, I can't see elected representatives working with such maps. Too complicated, not intuitive enough, with margins of error that are sometimes unmanageable, and therefore devoid of operational meaning"; "I doubt that uninformed and unaccompanied decision makers will be able to make decisions with this type of map"; "It's a difficult exercise, which took me over 30 min. It is not suitable for decision makers".</td></tr><tr><td rowspan="2">Comments on the D uncertainty visualization</td><td>Positive</td><td>4</td><td>"I realised that separate maps were my preference at first. But later, as I began to see maps with superimposed dots, I found them more satisfactory"; "Switching from one card to another can be complicated, so points are preferable".</td></tr><tr><td>Negative</td><td>18</td><td>"Dot pattern maps are the most practical, but they give you a headache. I couldn't see myself using them on a daily basis in the medium to long term"; "The association of the two maps without being able to dissociate them was destabilising".</td></tr><tr><td rowspan="2">Comments on aggregated maps</td><td>Positive</td><td>6</td><td>"I have a clear preference for a high level of aggregation (level 2). It makes the map easier to read, especially if the exercise is aimed at a non-specialist decision maker (colours are easier to read, and the generalisation of polygons simplifies reading and decision-making)".</td></tr><tr><td>Negative</td><td>6</td><td>"With heavily aggregated maps, it seems to me that there may be a temptation to rush to a decision, because it is simpler to make, when you overlook the diversity of situations within a plot"; "The non-aggregated map gives me the feeling that I'm deciding on a more detailed basis"</td></tr></tbody></table>

Reaching the intended map audience was a challenge, since their workloads do not leave them much time to delve into a long and complex survey. We believe that the participation success of this survey is due to the care with which we designed it. The survey took over a year to complete, and went through several pilot phases with a smaller number of test participants. This process led to substantial improvements of the survey, such as the addition of the tutorial. There were also other modifications that decreased the mental strain of the task, including: changes in the color palettes and spatial scales to improve map readability; the addition of explicit thumbnails above the map legend, to clearly indicate to the user the uncertainty visualization/aggregation level of the presented maps; and the addition of outlined plots to the displayed maps in the last stage, to help participants choose the most suitable map for decision-making.

Finally, it is worth noticing that the tutorial takes a substantial amount of time in the survey: respondents took a median time of 16 min to complete it, split into on average 8 min and 40 s for the map-reading exercise and 7 min and 20 s for the decision-making exercise. This duration accounts for more than half of the time required to complete the full survey, but for all the reasons stated above, it was an essential part of this user study. We advise readers planning to conduct similar studies to design a tutorial stage to train respondents and ensure the quality of responses.

## 4.2. Exploration of user preferences regarding uncertainty visualization

Our analysis demonstrated that end-users clearly prefer the “separate maps” adjacent uncertainty visualization to the “dot pattern” coincident one. This preference can be attributed to the difficulties participants encountered in reading the “dot pattern”, mainly due to two factors: (a) the size of the displayed maps, which was too small to comfortably read the point values (note that the map size was constrained by the LimeSurvey software); and (b) the chromatic

interactions between the background and foreground colors of the map. Such a representation is also unfamiliar for end-users who are used to dissociating soil quality and uncertainty information (Loiseau et al., 2020). However, three respondents found this representation convenient to see soil quality and uncertainty on the same map, as it saved them having to move their eyes back and forth between two separate maps (like in Viard et al. (2011), where adjacent maps were shown to induce a perceptual and cognitive overload).

In light of these results, it would be relevant to evaluate end-users' preferences regarding other coincident and adjacent uncertainty representations: uncertainty information could be encoded using other visual variables like color saturation, fuzziness, transparency (MacEachren et al., 2012; Scholz and Lu, 2014) instead of a color hue (the 7-color palette for soil quality values) and luminance (grayscale palette for uncertainty). Integral visualization approaches, which make uncertainty and soil information difficult to separate perceptually (Kinkeldey et al., 2014) could also be tested, like value-suppressing palettes (Correll et al., 2018, and applied to soil maps in Loiseau et al. (2020)).

A notable limitation of our study is that only one alternative uncertainty visualization (“dot pattern”) was evaluated. Assessing more representation options requires more comparisons and more participants, resulting in a even longer survey duration (MacKenzie, 2024). It also means having a longer tutorial stage, since making the respondents familiar with each representation takes time. In our questionnaire, this time was limited due to the similarity between the D and default S visualizations: in both, uncertainty is depicted via the same gray scale palette.

## 4.3. Exploration of users' perception of the trade-off between uncertainty and spatial resolution

In Courteille et al. (2024), we presented spatial aggregation as a way to decrease the – usually intractable – prediction uncertainty of
the spatial information displayed via raster maps. On the other hand, aggregating map products leads to a loss of spatial details which might be needed for decision-making. This study was carried out to assess where users' preferences lie along this compromise.

Our results show that end-users usually prefer moderately aggregated maps to both raster and heavily aggregated ones. It is the first time that user preferences regarding the level of spatial aggregation of a map product have been assessed. Some studies compared maps with different spatial resolutions and tried to identify the best one in terms of information content (Whelan and McBratney, 1999) and suitability for making a spatial conservation prioritization decision (Arponen et al., 2012), but did so without involving end-users. These global results are in line with the hypothesis we presented in Courteille et al. (2024), stating that end-users may prefer intermediate aggregated map products that have a much lower overall uncertainty, but whose spatial information is sufficiently detailed.

The analysis of user preferences by spatial context (Fig. 10) supports this view: in ten out the fifteen contexts, the preferred representation is among the two optimal representations along the presented compromise. In two of the five contexts where user preferences do not follow the compromise, preferences can be explained by unusually low uncertainty, or high information content. Preferences could not be understood in light of the uncertainty/spatial resolution compromise in only three contexts. Overall, the alignment observed in most contexts between the theoretical compromise and user preferences suggests that the I-U metric – which locates each map along the compromise spectrum (see Fig. 10) – could serve as a valuable foundation for developing guidelines to design maps that better meet end-user needs.

The comments of respondents provided some valuable insight into users' perception of spatial aggregation. Many comments on aggregated maps notably show that end-users tend to be suspicious of aggregated maps, and their distrust grows as the aggregation level increases. Admittedly, aggregated maps may mask the locations of small landscape elements such as areas of high conservation priority (Lombard et al., 1999) or isolated ecosystem services (Grét-Regamey et al., 2014), resulting in a loss of valuable information which can then lead to different planning decisions (Neuendorf et al., 2021). This echoes the remark made in Arrouays et al. (2020) about the tendency of end-users to interpret spatial information as more accurate when presented at a fine resolution compared to the same information delivered at a coarser resolution. We believe that this bias persists because end-users are not sufficiently aware of the high uncertainty that usually comes with high-resolution data. The fact that respondents completed a decision-making exercise in which they were explicitly asked to consider uncertainty, seems to have prevented this bias to drive user preferences: the raster representation was the preferred choice in only one spatial context, where it happens to have an unusually low uncertainty. High resolution data is not the most preferable option for end-users, when it involves dealing with high uncertainties — instead, participants are willing to let go some spatial detail to facilitate their decision-making.

Respondents' comments also point out that making decisions from heavily aggregated maps is easier, because the uncertainty is generally very low, and there are only one or two soil quality values to read when making a decision at the plot level. Moreover, many comments emphasized that aggregated maps are much more ergonomic, since their colors are easier to read as opposed to raster maps whose pixelation makes it harder to retrieve information. In many ways, aggregated maps seem to fulfill the need of decision makers for information that is easily understood (Neuendorf et al., 2018), and for reduced uncertainty – (Borowski and Hare, 2007), a study involving water managers, showed that they consider conducting and communicating uncertainty analysis to be less important than improving models to reduce uncertainty.

On the whole, aggregated maps raised suspicion, but were sometimes chosen even though their resolutions were objectively too coarse to support decisions at plot-level, probably because they were easier

to read and their uncertainties considerably lower. This prompts us to caution against the misuse of spatial aggregation as a means to reduce cartographic uncertainty. When end-users cannot make a decision from the map whose resolution matches the decision scale, such as due to excessive uncertainty, they should face uncertainty instead of turning to more aggregated maps which would only make decision-making artificially easier. Options they may consider to face this uncertainty include visiting the soil location in real in practice/real life before deciding, applying the precautionary principle, or deciding to postpone making a decision. More generally, in order to appropriately cope with inherent uncertainties, a dialogue between soil scientists, planners and decision makers is needed (Lark et al., 2022). This collaboration is important, for example, for deriving criteria to judge which levels of uncertainty are acceptable with respect to the potential consequences of the land planning decisions (Neuendorf et al., 2018).

### 4.4. The potential of interactivity for visualizing uncertain spatial information

Despite a large number of respondents, we could not pinpoint the exact factors influencing user preferences for spatial resolution and map uncertainty. We however found that these preferences were consistent across different end-user groups. However, we could only compare preferences between broad groups (e.g., researchers vs. decision makers, soil data experts vs. novices) since computing Elo scores require a large number of choices. Gathering more responses could reveal finer details, such as differences in preferences between people working in regional vs. communal councils. However, we believe that end-user preferences are also shaped by personal factors that are not related to end-users' jobs or workplaces, such as perception of spatial resolution, background, where they sit in respect of any conflict over land use, visual acuity, fatigue, and mood. This leads us to doubt whether we could identify homogeneous preference patterns, even when focusing on very specific end-user groups.

Identifying the factors driving preferences could have helped us create more precise map production guidelines and tailor them to specific end-user groups. However, our study suggests that there is no single map representation that works for everyone, even within a given end-user group. This is in line with many other studies involving respondents which concluded that no spatial representation type can be used as a panacea (Klein et al., 2015): according to the context and individual characteristics, user expectations and preferences vary regarding spatial resolution (Hauck et al., 2013) and uncertainty visualization (Evans, 1997). These preferences might also vary depending on the type of environmental data that is represented: maps of species distributions, of ecosystem services provision...

It is therefore worth exploring the many possibilities offered by interactive visualizations of environmental data in the future. Interactive map visualization refers to the visualization of a map through a computerized system, which extends users' exploratory capabilities far beyond what is possible with static maps (Rist and Masoodian, 2021): it allows the user to change the display parameters, to formulate requests to retrieve information, or to modify the map itself through more complex intervention. Interactive maps have been developed and tested for decades (Pebesma et al., 2007; Kunz et al., 2011), but are far from being the norm. Most maps are now produced in digital raster and vector formats, and are usually explored by end-users using the most basic functions of GIS software: zooming, clicking to retrieve the exact values from the displayed pixels or areas... In the case of the presented digital soil maps, an interactive cartographic information system would open up many more possibilities: additional information windows or widgets could offer important details about soil quality assessment methods and uncertainty calculations, different customizable data visualization methods could be offered (Kunz et al., 2011), the user could visualize the map at different aggregation levels before selecting the most suitable one... Our study can be seen as an attempt to identify
reference levels of aggregation and uncertainty visualizations that suit a majority of users and can be used as default settings, especially for novices who may not know what parameters to select.

But it is still questionable whether the user should be given the option to select a display mode according to their preferences. A limitation of the results presented here is that they rely on end-users' stated preferences. However, it has been shown that user preferences are not a good indicator of user performance in a given task (Pappenberger et al., 2013; MacEachren et al., 2005), generally evaluated through metrics such as accuracy and speed (Kinkeldey et al., 2014). In the case of decisions based on environmental maps, there is rarely a right answer and therefore evaluating accuracy on a decision-making task is highly error-prone (Hullman, 2016). Future research in this field should therefore determine whether displaying soil information with different aggregation levels or uncertainty visualizations (Aerts et al., 2003) leads to users arriving at different conclusions about the data when performing real-world decision tasks, using revealed preferences methods.

## 5. Conclusion

Spatial aggregation and uncertainty visualization have been identified by the DSM community as the two main levers for better communicating uncertainty to end-users. For the first time, users were asked for their opinions on the best way to represent uncertainty on maps used for decision support. The survey was judged favorably by the respondents despite its length and complexity. The study identified overall trends (preference for visualizing uncertainty on a separate map, and preference for intermediate aggregation levels) but a large share of preference patterns remains unexplained. Even if the majority of user choices are close to the optimal representation with respect to the uncertainty-spatial resolution trade-off that we highlighted, other objective and subjective factors come into play. Given this result, it is not possible to give precise indications as to how a soil map suited for decision-making should be designed. An interactive representation would allow each user to choose their preferred representation, according to the level of spatial detail they need and their tolerance to uncertainty. But this needs to be accompanied by more education and training in cartographic uncertainty to avoid misuse of such interfaces. On a final note, we hope that this work and its results will help scientists in soil science, but also in other environmental mapping communities (e.g. biodiversity, ecosystem services) to design maps that better meet the needs of their end-users.

<table><caption>**Table A.1**
Networks in which the survey has been distributed.</caption><thead><tr><th>Category of stakeholders</th><th>Network/institution</th><th>Medium for dissemination</th></tr></thead><tbody><tr><td rowspan="12">Urban planners</td><td>National Federation of urban planning agencies</td><td>Interview (4 interviewees)</td></tr><tr><td>National Federation of territorial coherence schemes</td><td>Interview (1 interviewee)</td></tr><tr><td>"Zero net land take" network</td><td>Interview (1 interviewee)</td></tr><tr><td>ADEME network</td><td>Mailing list</td></tr><tr><td>French Biodiversity Office (OFB)</td><td>Mailing list</td></tr><tr><td>National Federation for Territorial Coherence programs</td><td>Mailing list</td></tr><tr><td>Idealco Platform (collaborative platform for the public sector)</td><td>Post</td></tr><tr><td>National direction for Planning, Housing and Nature</td><td>Personal contacts</td></tr><tr><td>Institut Paris Région</td><td>Personal contacts</td></tr><tr><td>Montpellier Métropole</td><td>Personal contacts</td></tr><tr><td>SAFER Occitanie</td><td>Personal contacts</td></tr><tr><td>DRAAF Occitanie</td><td>Personal contacts</td></tr><tr><td>DRAAF Normandie</td><td>Personal contacts</td></tr><tr><td>Île-de-France Region</td><td>Personal contacts</td></tr><tr><td>Communes forestières</td><td>Personal contacts</td></tr><tr><td>ADEME "Soil quality evaluation" network</td><td>Mailing list</td></tr><tr><td>"Soil and territory" network</td><td>Mailing list</td></tr><tr><td>General Commission for Sustainable Development</td><td>Mailing list</td></tr><tr><td rowspan="7">Geomaticians</td><td>GeoRezo</td><td>Internet forum</td></tr><tr><td>French National Forest Office (ONF)</td><td>Personal contacts</td></tr><tr><td>National Institute of Geographic and Forest Information</td><td>Mailing list</td></tr><tr><td>OpenIG (Geodata resource center in Occitanie)</td><td>Newsletter</td></tr><tr><td>KaruGéo (Geodata resource center in Guadeloupe) via newsletter</td><td>Newsletter</td></tr><tr><td>IDéO (Geodata resource center in Bourgogne-Franche-Comté)</td><td>Website</td></tr></tbody></table>

(continued on next page)

## CRediT authorship contribution statement

**Léa Courteille:** Writing – original draft, Visualization, Validation, Methodology, Investigation, Formal analysis, Conceptualization. **Léa Tardieu:** Writing – review & editing, Supervision, Methodology, Investigation, Conceptualization. **Nadia Boukhelifa:** Writing – review & editing, Visualization, Supervision, Methodology, Investigation. **Evelyne Lutton:** Writing – review & editing, Supervision, Methodology, Investigation, Conceptualization. **Philippe Lagacherie:** Writing – review & editing, Supervision, Project administration, Methodology, Investigation, Conceptualization.

## Ethics

Our survey was approved by the Research Ethics Committee (CER) of the Paris-Saclay University (reference CER-Paris-Saclay-2023-081B).

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgments

The authors gratefully acknowledge INRAE for funding Léa Courteille's PhD-project.

The authors would like to warmly thank all the respondents for their time and effort, and for leaving insightful comments. Most importantly, we could not have gathered so many responses without the support of the people who helped disseminate the survey through their professional, virtual and sometimes personal networks. We particularly wish to thank Laurent Alborino, Maud Bessaguet, Jean-Pierre Chéry, Céline Collin-Bellier, Bruno Depresle, Patrick Duchez, Anne Fromage-Mariette, Stella Gass, Frédéric Portet, Nicolas Py, Davide Rizzo, Anouk Robillard and Joëlle Sauter.

## Appendix

See Table A.1.
<table><caption>Table A.1 (continued).</caption><thead><tr><th>Category of stakeholders</th><th>Network/institution</th><th>Medium for dissemination</th></tr></thead><tbody><tr><td rowspan="2">Soil experts</td><td>RGD Savoie-Mont-Blanc (Geodata resource center in Savoie)</td><td>Newsletter and LinkedIn</td></tr><tr><td>Theia (national geodata resource center)</td><td>Personal contacts</td></tr><tr><td rowspan="14">Researchers</td><td>Sol'En Vie</td><td>Blog post</td></tr><tr><td>AFES (French Association for the study of soils)</td><td>Mailing list</td></tr><tr><td>LISAH Research Unit</td><td>Mailing list</td></tr><tr><td>TETIS Research Unit</td><td>Mailing list</td></tr><tr><td>Espace-Dev Research Unit</td><td>Mailing list</td></tr><tr><td>LSE Research Unit</td><td>Mailing list</td></tr><tr><td>PSAE Research Unit</td><td>Mailing list</td></tr><tr><td>LESSEM Research Unit</td><td>Mailing list</td></tr><tr><td>International research center for Development and Environment (CIRED)</td><td>Mailing list</td></tr><tr><td>MUSE Project participants</td><td>Mailing list</td></tr><tr><td>Idefese project partners</td><td>Personal contacts</td></tr><tr><td>French National Ecosystem Assessment (Efease)</td><td>Platform post</td></tr><tr><td>Observatory of the nocturnal environment</td><td>Personal contacts</td></tr><tr><td>Géotamamt</td><td>Mailing list</td></tr><tr><td>Pedometron</td><td>Newsletter</td></tr><tr><td>French Foundation for Biodiversity Research (FRB)</td><td>Twitter</td></tr></tbody></table>

## Data availability

Data will be made available on request.

## References

Aerts, J., Clarke, K., Keuper, A., 2003. Testing popular visualization techniques for representing model uncertainty cartography and geographic information. Sci. 30, 249–261. http://dx.doi.org/10.1559/152304003100011180.

Angelini, M., Huvelink, G., Lagacherie, P., 2023. A multivariate approach for mapping a soil quality index and its uncertainty in southern france, European. J. Soil Sci. 74, e13345. http://dx.doi.org/10.1111/ejss.13345.

Arponen, A., Lehtomäki, J., Leppänen, J., Tomppo, E., Moilanen, A., 2012. Effects of connectivity and spatial resolution of analyses on conservation prioritization across large extents. Conserv. Biol. 26, 294–304. http://dx.doi.org/10.1111/j.1523-1739.2011.01814.x.

Arrouays, D., Grundy, M., Hartemink, A., Hempel, J., Huvelink, G., Hong, S., Lagacherie, P., Lelyk, G., McBratney, A., McKenzie, N., et al., 2014. Globalsoilmap: Toward a fine-resolution global grid of soil properties. Adv. Agron. 125, 93–134. http://dx.doi.org/10.1016/B978-0-12-800137-0.00003-0.

Arrouays, D., McBratney, A., Bouma, J., Libohova, Z., Forges, A.Richer-de., Morgan, C.L., Roudier, P., Poggio, L., Mulder, V., 2020. Impressions of digital soil maps: The good, the not so good, and making them ever better. Geoderma Reg. 20, e00255. http://dx.doi.org/10.1016/j.geodrs.2020.e00255.

Bonfatti, B., Hartemink, A., Giasson, E., Tornquist, C., Adhikari, K., 2016. Digital mapping of soil carbon in a viticultural region of southern brazil. Geoderma 261, 204–221. http://dx.doi.org/10.1016/j.geoderma.2015.07.016.

Borowski, I., Hare, M., 2007. Exploring the gap between water managers and researchers: difficulties of model-based tools to support practical water management. Water Resour. Manag. 21, 1049–1074.

Boukhelifa, N., Duke, D., 2009. Uncertainty visualization: why might it fail? In: CHI'09 Extended Abstracts on Human Factors in Computing Systems. pp. 4051–4056. http://dx.doi.org/10.1145/1520340.1520616.

Boukhelifa, N., Perrin, M.-E., Huron, S., Eagan, J., 2017. How data workers cope with uncertainty: A task characterisation study. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 3645–3656. http://dx.doi.org/10.1145/3025453.3025738.

Carpendale, S., 2008. Evaluating information visualizations. In: Information Visualization: Human-Centered Issues and Perspectives. Springer, pp. 19–45. http://dx.doi.org/10.1007/978-3-540-70956-5_2.

Chagumaira, C., Chimungu, J.G., Gashu, D., Nalivata, P.C., Broadley, M.R., Milne, A.E., Lark, R.M., 2020. Communicating uncertainties in spatial predictions of grain micronutrient concentration. Geosci. Commun. Discuss. 2020, 1–28.

Chagumaira, C., Nalivata, P.C., Chimungu, J.G., Gashu, D., Broadley, M.R., Milne, A.E., Lark, R.M., 2022. Stakeholder interpretation of probabilistic representations of uncertainty in spatial information: an example on the nutritional quality of staple crops. Int. J. Geogr. Inf. Sci. 36, 2446–2472.

Clark, A., Howard, K., Woods, A., Penton-Voak, I., Neumann, C., 2018. Why rate when you could compare? using the elochoice package to assess pairwise comparisons of perceived physical strength. PloS One 13, e0190393. doi:doi.org/10.1371/journal.pone.0190393.

Clarke, V., Braun, V., 2017. Thematic analysis. J. Posit. Psychol. 12, 297–298. http://dx.doi.org/10.1080/17439760.2016.1262613.

Correll, M., Moritz, D., Heer, J., 2018. Value-suppressing uncertainty palettes. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 1–11. http://dx.doi.org/10.1145/3173574.3174216.

Courteille, L., Lagacherie, P., Boukhelifa, N., Lutton, E., Tardieu, L., 2024. Using spatial aggregation of soil multifunctionality maps to support uncertainty-aware planning decisions. Eur. J. Soil Sci. http://dx.doi.org/10.1111/ejss.13523.

Elo, A., Sloan, S., 1978/2008. The Rating of Chessplayers: Past and Present. Ishi Press International.

Evans, B., 1997. Dynamic display of spatial data-reliability: Does it benefit the map user. Comput. Geosci. 23, 409–422.

Evans-Cowley, J., Akar, G., 2014. Streetseen visual survey tool for determining factors that make a street attractive for bicycling. Transp. Res. Rec. 2468, 19–27. http://dx.doi.org/10.3141/2468-03.

Richer-de Forges, A., Arrouays, D., Bardy, M., Bispo, A., Lagacherie, P., Laroche, B., Lemercier, B., Sauter, J., Voltz, M., 2019. Mapping of soils and land-related environmental attributes in france: analysis of end-users' needs. Sustain. 11, 2940. http://dx.doi.org/10.3390/su11102940.

Goodspeed, R., 2017. Research note: An evaluation of the elo algorithm for pairwise visual assessment surveys. Landsc. Urban Plan. 157, 131–137. http://dx.doi.org/10.1016/j.landurbplan.2016.06.009.

Grét-Regamey, A., Brunner, S., Altwegg, J., Bebi, P., 2013. Facing uncertainty in ecosystem services-based resource management. J. Environ. Manag. 127, S145–S154. http://dx.doi.org/10.1016/j.jenvman.2012.07.028.

Grét-Regamey, A., Weibel, B., Bagstad, K., Ferrari, M., Geneletti, D., Klug, H., Schirpke, U., Tappeiner, U., 2014. On the effects of scale for ecosystem services mapping. PloS One 9, e112601. http://dx.doi.org/10.1371/journal.pone.0112601.

Harrower, M., Brewer, C.A., 2003. Colorbrewer. org: an online tool for selecting colour schemes for maps. Cartogr. J. 40, 27–37.

Hauck, J., Görg, C., Varjopuro, R., Ratamäki, O., Maes, J., Wittmer, H., Jax, K., 2013. Maps have an air of authority: potential benefits and challenges of ecosystem service maps at different levels of decision making. Ecosyst. Serv. 4, 25–32. http://dx.doi.org/10.1016/j.ecoser.2012.11.003.

Hengl, T., Walvoort, D., Brown, A., Rossiter, D., 2004. A double continuous approach to visualization and analysis of categorical maps international. J. Geogr. Inf. Sci. 18, 183–202.

Heuvelink, G., 2014. Uncertainty Quantification of Globalsoilmap Products GlobalSoilMap. CRC Press, Boca Raton, FL, USA, pp. 335–340.

Hullman, J., 2016. Why evaluating uncertainty visualization is error prone. In: Proceedings of the Sixth Workshop on beyond Time and Errors on Novel Evaluation Methods for Visualization. pp. 143–151. http://dx.doi.org/10.1145/2993901.2993919.

Jansen, J., Woolley, S., Dunstan, P., Foster, S., Hill, N., Haward, M., Johnson, C., 2022. Stop ignoring map uncertainty in biodiversity science and conservation policy. Nat. Ecol. Evol. 6, 828–829. http://dx.doi.org/10.1038/s41559-022-01778-z.

Kinkeldey, C., MacEachren, A., Schiewe, J., 2014. How to assess visual communication of uncertainty? a systematic review of geospatial uncertainty visualisation user studies. Cartogr. J. 51, 372–386. http://dx.doi.org/10.1179/1743277414Y.00000000099.

Klein, T., Celio, E., Grét-Regamey, A., 2015. Ecosystem services visualization and communication: A demand analysis approach for designing information and conceptualizing decision support systems. Ecosyst. Serv. 13, 173–183. http://dx.doi.org/10.1016/j.ecoser.2015.02.006.

Klein, M., Dülmer, H., Ohr, D., Quandt, M., Rosar, U., 2004. Response sets in the measurement of values: a comparison of rating and ranking procedures. Int. J. Public Opin. Res. 16, http://dx.doi.org/10.1093/ijpor/edh041.

Kubiček, P., Sašinka, Č., 2011. Thematic uncertainty visualization usability-comparison of basic methods. Ann. GIS 17, 253–263. http://dx.doi.org/10.1080/19475683.2011.625978.

Kunz, M., Grét-Regamey, A., Hurni, L., 2011. Visualization of uncertainty in natural hazards assessments using an interactive cartographic information system. Nat. Hazards 59, 1735–1751. http://dx.doi.org/10.1007/s11069-011-9864-y.
Langlois, J., Guilhaumon, F., Baletaud, F., Casajus, N., Braga, C., Fleuré, V., Kulbicki, M., Loiseau, N., Mouillot, D., Renoult, J., et al., 2022. The aesthetic value of reef fishes is globally mismatched to their conservation priorities. PLoS Biol. 20, e3001640. http://dx.doi.org/10.1371/journal.pbio.3001640.

Lark, R., Chagumaira, C., Milne, A., 2022. Decisions, uncertainty and spatial information. Spat. Stat. 50, 100619. http://dx.doi.org/10.1016/j.spasta.2022.100619.

LimeSurvey GmbH, 2024. Limesurvey. URL https://www.limesurvey.org/.

Liu, F., Ting, K., Zhou, Z.-H., 2008. Isolation forest. In: 2008 Eighth Ieee International Conference on Data Mining. IEEE, pp. 413–422. http://dx.doi.org/10.1109/ICDM. 2008.17.

Loiseau, Richer-De-Forges, A., Roudier, P., Ducommun, C., Chen, S., Lagacherie, P., Arrouays, D., 2020. Essais de représentation cartographique de l'incertitude pour les utilisateurs de cartographies des sols par modélisation statistique. Etude et Gestion Des Sols 27, 257–275.

Lombard, A., Hilton-Taylor, C., Rebelo, A., Pressey, R., Cowling, R., 1999. Reserve selection in the succulent karoo, south Africa: coping with high compositional turnover. Plant Ecol. 142, 35–55. http://dx.doi.org/10.1023/A:1009866126729.

MacEachren, A., Brewer, C., Pickle, L., 1998. Visualizing georeferenced data: representing reliability of health statistics. Environ. Plan. A 30, 1547–1561. http://dx.doi.org/10.1068/a301547.

MacEachren, A., Robinson, A., Hopper, S., Gardner, S., Murray, R., Gahegan, M., Hetzler, E., 2005. Visualizing geospatial information uncertainty: What we know and what we need to know. Cartogr. Geogr. Inf. Sci. 32, 139–160. http://dx.doi.org/10.1559/1523040054738936.

MacEachren, A., Roth, R., O'Brien, J., Li, B., Swingley, D., Gahegan, M., 2012. Visual semiotics & uncertainty visualization: An empirical study. IEEE Trans. Vis. Comput. Graphics 18, 2496–2505. http://dx.doi.org/10.1109/TVCG.2012.279.

MacKenzie, I., 2024. Human–computer interaction: An empirical research perspective.

Neuendorf, F., Haaren, C.von., Albert, C., 2018. Assessing and coping with uncertainties in landscape planning: an overview landscape. Ecology 33, 861–878. http://dx.doi.org/10.1007/s10980-018-0643-y.

Neuendorf, F., Thiele, J., Albert, C., Haaren, C.von., 2021. Uncertainties in land use data may have substantial effects on environmental planning recommendations: A plea for careful consideration. Plos One 16, e0260302. http://dx.doi.org/10.1371/journal.pone.0260302.

Padarian, J., McBratney, A., 2023. Quadmap: Variable resolution maps to better represent spatial uncertainty. Comput. Geosci. 181, 105480. http://dx.doi.org/10.1016/j.cageo.2023.105480.

Pappenberger, F., Stephens, E., Thielen, J., Salamon, P., Demeritt, D., Van Andel, S., Wetterhall, F., Alfieri, L., 2013. Visualizing probabilistic flood forecast information: expert preferences and perceptions of best practice in uncertainty communication. Hydrol. Process. 27, 132–146. http://dx.doi.org/10.1002/hyp.9253.

Pebesma, E., Jong, K.de., Briggs, D., 2007. Interactive visualization of uncertain spatial and spatio-temporal data under different scenarios: an air quality example. Int. J. Geogr. Inf. Sci. 21, 515–527. http://dx.doi.org/10.1080/13658810601064009.

Rabot, E., Guiresse, M., Pittatore, Y., Angelini, M., Keller, C., Lagacherie, P., 2022. Development and spatialization of a soil potential multifunctionality index for agriculture (agri-spmi) at the regional scale. case study in the occitanie region (france). Soil Secur. 6, 100034. http://dx.doi.org/10.1016/j.soisec.2022.100034.

Rist, T., Masoodian, M., 2021. Interactive map visualizations for supporting environmental sustainable development goals. In: IFIP Conference on Human-Computer Interaction. Springer, pp. 36–46.

Russell, P., Gray, C., 1994. Ranking or rating? some data and their implications for the measurement of evaluative response. Br. J. Psychol. 85, 79–92. http://dx.doi.org/10.1111/j.2044-8295.1994.tb02509.x.

Scholz, R., Lu, Y., 2014. Uncertainty in geographic data on bivariate maps: an examination of visualization preference and decision making ISPRS. Int. J. Geo-Inf. 3, 1180–1197. http://dx.doi.org/10.3390/ijgi3041180.

Vaysse, K., Heuvelink, G., Lagacherie, P., 2017. Spatial aggregation of soil property predictions in support of local land management. Soil Use Manag. 33, 299–310. http://dx.doi.org/10.1111/sum.12350.

Viard, T., Caumon, G., Lévy, B., 2011. Adjacent versus coincident representations of geospatial uncertainty: Which promote better decisions. Comput. Geosci. 37, 511–520. http://dx.doi.org/10.1016/j.cageo.2010.08.004.

Wadoux, A.-C., Heuvelink, G., 2023. Uncertainty of spatial averages and totals of natural resource maps. Methods Ecol. Evol. 14, 1320–1332. http://dx.doi.org/10.1111/2041-210X.14106.

Wadoux, A.-C., Heuvelink, G., Lark, R., Lagacherie, P., Bouma, J., Mulder, V., Libohova, Z., Yang, L., McBratney, A., 2021. Ten challenges for the future of pedometrics. Geoderma 401, 115155. http://dx.doi.org/10.1016/j.geoderma.2021.115155.

Ward, J.J., 1963. Hierarchical grouping to optimize an objective function. J. Amer. Statist. Assoc. 58, 236–244. http://dx.doi.org/10.1080/01621459.1963.10500845.

Whelan, B., McBratney, A., 1999. Prediction uncertainty and implications for digital map resolution. In: Proceedings of the Fourth International Conference on Precision Agriculture. John Wiley & Sons, Ltd, pp. 1185–1196. http://dx.doi.org/10.2134/1999.precisionagproc4.c21b.

Willemen, L., Burkhard, B., Crossman, N., Drakou, E., Palomo, I., 2015. Est practices for mapping ecosystem services. Ecosyst. Serv. 13, 1–5. http://dx.doi.org/10.1016/j.ecoser.2015.05.008.