# Uncertainty in Science is Malleable. Advocating for User-Agency in Defining Uncertainty in Visualizations: a Case Study in Geology

Vanessa Peña-Araya, Consuelo Martínez Fontaine, Xiang Wei, Guillaume Delpech, Anastasia Bezerianos

► To cite this version:

Vanessa Peña-Araya, Consuelo Martínez Fontaine, Xiang Wei, Guillaume Delpech, Anastasia Bezerianos. Uncertainty in Science is Malleable. Advocating for User-Agency in Defining Uncertainty in Visualizations: a Case Study in Geology. CHI 2025 - 43rd SIGCHI conference on Human Factors in computing systems, ACM, Apr 2025, Yokohama, Japan. 10.1145/3706598.3713972. hal-05004600

HAL Id: hal-05004600

https://inria.hal.science/hal-05004600v1

Submitted on 26 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Uncertainty in Science is Malleable. Advocating for User-Agency in Defining Uncertainty in Visualizations: a Case Study in Geology

Vanessa Peña-Araya

LISN, Université Paris-Saclay, CNRS,
INRIA

Orsay, France

vanessa.pena-araya@inria.fr

Consuelo Martínez Fontaine

Crono Austral

Punta Arenas, Chile

consuelola.m.f@gmail.com

Xiang Wei

LISN, Université Paris-Saclay, CNRS,
INRIA

Orsay, France

xiang.wei@telecom-paris.fr

Guillaume Delpech

Geosciences Paris Saclay (GEOPS)

Paris Saclay University

Orsay, France

guillaume.delpech@universite-paris-saclay.fr

Anastasia Bezerianos

LISN, Université Paris-Saclay, CNRS,
INRIA

Orsay, France

anastasia.bezerianos@universite-paris-saclay.fr

## Abstract

Uncertainty is inherent in science built on previous results. In geoscience, for instance, researchers analyzing volcanic deposits assess the uncertainty around past deposit classifications. To aid this assessment, we followed a design by immersion approach to co-design uncertainty visualizations. We observed that besides visualizing it, it is challenging even to define what constitutes uncertainty, as how researchers understand and process uncertainty evolves. This motivated us to reach other members of the community to better understand how they integrate uncertainty in their work. Informed by a series of interviews, we first redesigned our visualization system and then introduced it as a technology probe to a broader community of geoscientists. Our results highlight that uncertainty in science is malleable and that visualization systems should be designed with this malleability in mind. Through a set of design implications, we advocate for visualizations that promote user agency and flexibility in defining and processing uncertainty.

## CCS Concepts

* Human-centered computing → Empirical studies in HCI; Empirical studies in visualization; Information visualization.

## Keywords

Visualization, Uncertainty, Qualitative Methods, Working with Domain Experts, Design Study, Geology

## ACM Reference Format:

Vanessa Peña-Araya, Consuelo Martínez Fontaine, Xiang Wei, Guillaume Delpech, and Anastasia Bezerianos. 2025. Uncertainty in Science is Malleable. Advocating for User-Agency in Defining Uncertainty in Visualizations: a Case Study in Geology. In *CHI Conference on Human Factors in Computing Systems (CHI '25)*, April 26-May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3706598.3713972

<figure><img src="image_2.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License. CHI '25, Yokohama, Japan

© 2025 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-1394-1/25/04

https://doi.org/10.1145/3706598.3713972

## 1 Introduction

Uncertainty is inherent to science [39] no matter how precise or careful the methods it employs. In some domains, like geosciences [32], this uncertainty exists due to the complexity of the natural phenomena it studies and the fact that subjective human interpretation is needed. For example, geologists studying volcanoes often collect soil samples with volcanic deposits (called tephras) and try to determine from which volcano or specific eruption they originate from [3, 49]. A core aspect of this process is the analysis of past research to compare new samples to ones that have been assigned to volcanoes in the past. This assessment requires scientists to identify possible sources of uncertainty that were introduced in past assignments.

To understand how to help geologists visualize the inherent uncertainty of past scientific results, we followed a design by immersion approach [31] within a team of visualization and geoscience researchers conducting tephra studies (co-authors in this work). Our first design iteration and resulting prototype highlighted that, even if our experts could identify the most important sources of uncertainty for assessing past research for their work, the interplay between these sources, but also how the rest of the scientific community would use them, was not clear. It also highlighted that visualizing these uncertainty factors is only one part of the challenge; when reaching a broad scientific community it is also extremely important to provide flexible ways to characterize them.

These insights motivated us to reach out to a broader group of domain experts. In a series of 6 interviews, we set out to better understand these sources of uncertainty. We used the results of these interviews to inform a redesign of our prototype. We then introduced it as a technology probe [36] to 10 additional experts to help us gain a better understanding of how these uncertainty factors are combined in practice and how a visualization system can support this integration.

Our analysis revealed that when designing visualizations of uncertainty linked to scientific knowledge built and improved progressively over time, the notion of uncertainty is also constructed and evolves with knowledge. Our process highlighted the limitations of
visualization design with a positivist lens where we assume a list of predefined, unambiguous and objective sources of doubts.

Instead, our work underlines the benefits of considering how knowledge is situated, constructed, and continuously improved [1] when designing visualizations to represent it, and that we need to design with the *malleability* of uncertainty in mind:

* **User agency in defining uncertainty:** Uncertainty in geoscience is multi-faceted, it is made up of many complex factors that are impossible to list a priori, and cannot always be reduced to a predefined set of measures (either quantitative or qualitative) to be mapped into a visualization. Our systems should allow users to define and surface their own meanings of uncertainty.

* **Malleability in representation:** Uncertainty in geoscience is malleable and context-dependent as multiple sources of uncertainty are combined differently depending on the available data, the experts' goals, but also the larger ecosystem of the analysis (authors, technical standards, etc.). This malleability needs to be supported in the visual analytics systems, not just by providing users ways to express their own uncertainty definitions, but also when choosing visual representation and direction of visual mapping.

* **Opportunities for visualization:** Uncertainty visualization aids in scientific discovery, but can go further. The definitions of uncertainty evolve as knowledge is built from a community of researchers. Visualization systems can become more than just a rigid representation of "one uncertainty", they could act as platforms of dialogue and archiving where researchers can share their own definitions of uncertainty and the contextual rationale behind it.

Our main contributions are: (i) a design study with experts in tephra studies, identifying what sources of uncertainty they are faced with; (ii) a deeper understanding of the malleable nature of uncertainty in a domain where human interpretation plays a predominant role, and the challenges in designing visualization systems around it; (iii) a set of design guidelines stressing the need to provide user agency in defining and visualizing uncertainty.

# 2 Related Work

We describe next the most relevant work on uncertainty definitions and visualization.

## 2.1 Defining and Dealing with Uncertainty

Uncertainty can have different meanings [12]. In the visualization community, for example, it has been used to refer to lack of information or knowledge [10, 79], moments of doubt [12], or implicit error [54], among others (e.g., [51, 59, 66]). Brodlie et al. [13] argue that this diversity of meanings in visualization makes uncertainty complex.

Further work on uncertainty classifies it by type (e.g., [48, 51, 52, 61, 73, 75, 79]) or sources (e.g.,[70, 79]). For example, Padilla et al. [61] and van der Bles et al. [79] make the distinction between two types of uncertainty: *direct* uncertainty, that can be communicated in quantitative terms, and *indirect* uncertainty, that refers to the quality of the underlying knowledge. Regarding sources of

uncertainty, van Der Bles et al. [79] mention computational inadequacies of measurement and expert disagreement among some of the sources of scientific uncertainty. In the context of the visualization pipeline, Sacha et al. [70] consider two sources of uncertainty: one from the data and the other introduced by the visualization. They state that the combination of uncertainties from multiple sources are bigger than the sum and propose a set of guidelines for handling these uncertainties.

Other work has focused on the strategies used by people with different backgrounds, including decision-making students [48], data workers [12], visualization creators [34] and researchers in general [38]. The strategies mentioned in these studies include *reducing, acknowledging, and suppressing* uncertainty. In the context of creating visualizations, the study by Hullman [34] found that the most commonly used strategy by visualization creators is not to include uncertainty. While other work has highlighted that in instances where several interpretations are plausible [59], ambiguity serves a critical role in sensemaking and should be designed for. Other factors that influence how practitioners deal with uncertainty include the level of expertise [9, 65], the practitioner's background [64], trust in known authors of data [65], internal biases [77], personal knowledge [47], and geographical distance to the uncertain event [7, 44, 62].

Regarding scientific communities building knowledge based on past results (as is our case), uncertainty may also relate to data sharing and interpretation. Feger et al. [25] found two main sources of uncertainty related to data sharing: *accessibility*, which is not knowing how to gather the data; and *volatility*, which is related to the reasons that do not allow the preservation of the data forever. Furthermore, merely creating repositories and uploading data to them does not provide enough context to enable others to understand and reuse it [58].

In this paper, we focus on the definition and elicitation of uncertainty within a scientific community, the challenges in assessing it in previously published data, and how the diversity of uncertainty meanings is inevitable and a driver for knowledge evolution. Our work extends prior studies by exploring the entanglements of the contextual factors that influence the creation of knowledge and how to consider them when designing visualization systems to explore them [1], the "fuzzy stuff that lies around the edges" [14].

## 2.2 Visualization of Uncertainty

For *quantitative* uncertainty, the survey by Padilla et al. [60] divides techniques to visualize uncertainty into two main categories: graphical representation of distributional properties (e.g., [33, 35, 40]) and visual encoding (e.g., [11, 16, 19, 46, 50, 74]). Graphical representations of distributions that only use intervals tend to be misinterpreted [18, 44], and studies show that more expressive representations of distributions, like quantile dots [26, 40] or icon arrays [24, 28, 29], perform better. Visual encodings to represent uncertainty as a variable include color properties [19, 74], transparency [16, 46], sketchiness [11], and redundancy of more than one visual variable [50]. Surveys [41, 42, 51, 52] summarize a set of studies on the use of visual encodings for uncertainty. In particular, the study by MacEachren et al. [52] shows that some visual variables, like fuzziness and location, are more intuitively understood as uncertainty.
<figure><img src="image_3.png" /><figcaption>**Figure 1:** The existing BOOM! [53] visualization interface acted as the scaffold around which we introduced uncertainty visualizations. Each volcano is assigned a unique color across the multiple views. On the left, we see a geographical representation of volcanoes as triangles on the map. Samples assigned to them are represented as circles with the same color. On hover, a white pop-up gives information on a given sample. In the middle, we see temporal information about volcanic eruptions. On the right, we see the results of the chemical analysis of volcanoes (biplots), where each dimension represents the presence of a chemical element. The online demo can be found at https://boom-vis.lisn.upsaclay.fr/.</figcaption></figure>

*Non-quantitative* forms of uncertainty have less standardized representations. For example, some work has used icons to represent predefined labels of categorical values of uncertainty [22, 69]. However, a recent review of communication of uncertainty in the digital humanities found that creating such categories may be artificial [63]. This survey also describes that besides *established visualization* techniques for quantifiable uncertainties, authors in the digital humanities have also used *improvisational visualizations* (isolated "tricks" to represent uncertainty), *experiential* (that aim to give the impression of uncertainty instead of quantifying it), and *dedicated* (custom visualizations tailored to the needs of the researcher).

Some work considers more than one source of uncertainty in visualization systems, either to directly visualize a priori known forms of uncertainty (e.g., [23, 65]) or to provide means for eliciting them [54]. Nevertheless, combining and interpreting multiple sources of uncertainty can challenging [30]. Studies such as the one by Padilla et al. [61] describe how people deal with “multiple uncertainties” (two types in their case). Their study considered how people dealt with indirect and direct uncertainty in decision-making and how they influenced each other. Their results show that people tend to make a more conservative interpretation of direct uncertainty if the indirect uncertainty distribution that defines its quality has high variances.

The case study on archaeology by Panagiotidou et al. [65] is the most similar to ours. Their work visualized three sources of uncertainty, called implicit error, but revealed the existence of six.

They show that some representations of implicit error can be more suitable than others and highlight that they will be used differently depending on the user's level of expertise. Apart from the difference in the domain, we focus on uncertainty when assessing past scientific results, on the interplay between different sources, and their contextual interpretation. We revisit this work in our discussions.

Our paper extends previous work by exploring the challenges of designing a visualization system that considers simultaneously multiple sources of uncertainty when assessing past scientific work and how what constitutes uncertainty can be different for the members of a single scientific community.

## 3 Context, Motivation and Roadmap

Tephrochronology and other tephra studies are geoscience disciplines that analyze volcanic eruption deposits, which are called *tephras*. This analysis helps reconstruct the volcanic history of a place [3, 49], used for the creation of volcanic hazard maps [8, 20], for understanding climate evolution over time [27, 56], or to reconstruct timelines by linking events across widely separated regions [76]. An important task in tephra studies is to identify the origin of a sample, i.e., assigning a tephra sample to a specific volcano eruption. To do so, geologists need to compare the new sample with other samples assigned in past research to the volcano in question. This comparison takes into consideration several dimensions, such as the sample's geochemical composition, its age, its geographic position and even physical characteristics such as its color.
<figure><img src="image_4.png" /><figcaption>Figure 2: The design process followed in our study of uncertainty, the two iterations and resulting prototypes.</figcaption></figure>

The starting point of our work is the existing visual analytics (VA) system Boom! [53], used to create a centralized dataset of the last 30 years of tephra studies in the Andes volcanic zones (see Figure 1). Such datasets allow geologists to find past analyses to compare their samples against, making their published work more FAIR, i.e., Findable, Accessible, Interoperable, or Reusable [84]. The original interface of BOOM!, seen in Figure 1, is a dashboard combining visualizations that were commonly used by geologists working on tephra studies: a map with the volcanoes and samples, a timeline with the eruptions, and biplots with the geochemistry of the samples (i.e., scatterplots of two measured chemical elements at a time). Our geologist collaborators had used this interface to compile a dataset of 16,768 sample observations collected across 79 published studies, visual analysis of the tephra samples in the visual analytics tool (geographical location and composition), as well as follow-up interviews with authors of past studies for clarifications. The dataset has 80 attributes, including geographical information, chemical analysis (the presence and percentage of 38 chemical elements), data on the machines or standards used, author and publication information. Martinez Fontaine et al. [53] recently published dataset and the initial VA system.

The geologist (co-authors of this work, and co-designers of solu-
tions) explained that sample assignment requires the assessment of
the inherent uncertainty and subjectivity of past assignments. This
is because tephrochronology, like other sciences, relies on the accu-
mulation of knowledge among several researchers over time. As
such, any source of uncertainty from previous analysis may affect
the results of future tephra assignments and the knowledge evo-
lution in the domain. For example, while analyzing their samples,
with or without the BOOM! system [53], it was common for them to
find previously analyzed samples whose assignment raised doubts
as they were chemically different from other samples assigned to
the same volcano or collected far away from the assigned volcano.
These doubts could be due to suspecting an incorrect transfer of the
data from a publication to the final dataset, due to an incomplete or
poorly reported analysis in past research, or just due to the inher-
ent variability of earth data/real-world data: there can be a mix of
ashes from different volcanoes in the same deposit, winds may have
moved ashes across large distances, and even the tephra chemistry
itself can become altered over time [21]. In addition, tephra analysis
is sparse and not necessarily standardized: researchers use different
soil analysis machines with varying accuracies and follow different
analysis protocols [45]. For all these reasons, comparing new tephra
samples against ones previously analyzed in past research can be
challenging.

We thus set out to explore the notion of uncertainty to under-
stand if and how visualization could aid geologists in their work.
Figure 2 shows the process we followed in our work that consists
of two iterations: (i) a co-design of a first prototype following a

design-by-immersion collaboration [31] approach with geologists
(co-authors of this paper), and (ii) a set of interviews to inform
a redesign used as a technology probe [37] with a broader set of
geologists.

4 1st Iteration: Design by Immersion for Characterizing Uncertainty

<figure><img src="image_5.png" /></figure>

We followed a
design by im-
mersion col-
laboration ap-

proach [31] to characterize the multiple aspects or sources of un-
certainty and co-design visualizations around it. We use the broad
definition of uncertainty by Boukhelifa et al. [12]: “Uncertainty is
when you are not sure about one or more aspects of your data.” Our
process included biweekly meetings of one or two hours over 6
months among this paper’s authors (between Mar 2022-Sept 2022).
We had two main goals: (i) for the geologists co-authors to reify
their knowledge about uncertainty into concrete measures that
could be used to assess the quality of samples’ assignments, and
(ii) to collectively find effective ways to visually include them in
the prototype implemented by the visualization co-authors. Our
initial aim was to produce a prototype that other members of the
Tephrochronology community could then evaluate.

In the rest of this section, we describe the iterative design pro-
cess and results, as well as the work’s value for the tephra studies
domain. We also detail lessons learned about the malleable nature
of uncertainty and why our initial simplistic approach of defining
and showing uncertainty was not enough.

## 4.1 Characterizing uncertainty when assessing collective scientific knowledge

When assigning a new tephra sample to a volcano, researchers
can obtain quantitative uncertainty measures of certain aspects of
the process. For example, machines that extract the geochemical
composition of a sample give confidence intervals of their results
according to standardized values. However, such information or
other standardized uncertainty metrics (either quantitative or qual-
itative) are not systematically published along tephra studies. More
importantly, our experts explained that they had to consider many
other sources of uncertainty in their work that were more abstract
and less easy to pinpoint or measure.

We used the existing BOOM! [53] visual analytics system as a
brainstorming tool to think about what uncertainty information is
missing, how we could use already available information to create
uncertainty metrics, and when these should be visualized. The
geologists in our team used the tool while attempting to assess
the quality of existing sample assignments in the dataset about the Southern Andes area. They declared they used several factors to find samples whose assignment may cause doubts or be uncertain. After several iterations, they selected what seemed to be the three most relevant sources of uncertainty:

*Geographical proximity to volcanoes and other samples.* A sample's assignment to a volcano can be doubted if it is too far away from its assigned volcano or from all other samples assigned to that volcano. However, proximity is more complex than just distance: It must consider factors like wind or explosiveness that can blow volcanic products far from their source. For instance, material from the Hudson volcano has drifted up to 900km from it. Is the sample far from its assigned volcano because of natural phenomena (e.g., big eruption or wind), or is it an incorrect assignment?

*Geochemical similarity to a volcano.* A sample's assignment to a volcano can be doubted if its geochemistry is not similar to that of previously assigned samples. This judgment is complex as experts consider a variable number of geochemical elements depending on the volcano. For example, the Chaitén volcano has a unique composition for SiO₂ and K₂O, whereas the Hudson volcano does not, so experts need to consider up to 38 geochemical elements when assigning new samples to it. Furthermore, there is no exact "threshold" to delimit when a sample belongs to one volcano and when it belongs to another. Is a sample's geochemistry different from the other samples assigned to a volcano because of the normal dispersion or variation, or because of an incorrect assignment?

*Number of samples assigned to a volcano.* The more publications assign samples to a volcano, the more we know about it. In other words, a volcano with several samples assigned to it has been analyzed by several authors and can be considered as more "certain". Similarly to the geochemical analysis, there is no "threshold" to the number of samples that are enough to characterize a volcano. Are there few samples assigned to a volcano because it has low activity and thus is more challenging to find samples from it or because previous research about it is not easily accessible?

We are dealing with at least three interconnected sources of uncertainty. For example, if a new sample has a geochemical composition similar to that of a very faraway volcano, then the assignment to that volcano could be questioned. However, if the volcano has been studied well and shows a defined geochemical pattern, then the assignment can be seen as more "certain". Assessing these uncertainties is crucial as past assignments have no ground truth. The composition of what we know about volcanoes comes from groups of assignments that require human interpretation over decades of research.

## 4.2 Visualization designs and their evolution

Our goal is to integrate the identified sources of uncertainty in a pre-existing visualization system to aid geologists in identifying issues with past sample assignments. We note that the aim is to help experts identify samples whose previous assignment may be doubted for further reflection, but it is then up to them to interpret the subtleties of these factors.

Over our weekly meetings, we would discuss the uncertainty sources described previously, and the visualization researchers would implement visualization designs as examples, often inspired by past literature. We started by considering simplified ways to express the sources of uncertainty and explored how to communicate them visually. The geologists would then test them, and we would all discuss the benefits, drawbacks, and possible design variations together.

Next, we summarize our designed iterations and lessons learned during the design process. More details on the iteration designs can be found in the supplementary material. We conclude with a section on the domain value of our work and design insights we got from this iterative co-design process that led us to a re-design.

*Showing Geographical coverage of samples.* To help experts understand how geographically "close/far" away a sample is from its assigned group (the volcano itself and other samples of that volcano), we considered how to visualize the coverage of a volcano, that is, the geographical spread of samples assigned to it. Following previous visualization of uncertainty in geo-data [46], we designed area-based representations to show coverage and uncertainty, trying variations such as semi-transparent and gradient circles, boxes, contours, and blob-based visualizations. Examples of our initial design sketches and later implementations are in Figure 3.

*Lessons learned.* After using the implemented designs, our domain experts found that these representations added unnecessary clutter, in particular combined with geochemical distance indicators (discussed next). In the end, they found the old-fashioned way better: they preferred just showing the samples' location individually and letting the eye and experience of the expert decide what is close or far to judge the volcano coverage. Dedicated coverage visualizations could be useful, but only if the geologist focuses on one specific volcano or a very small number of them.

*Showing the geochemical similarity of samples to a volcano.* The geochemical proximity of a sample (or group of samples) to a volcano is accessed by comparing its chemical composition to all samples of that volcano, plotted in a set of scatterplots. These show the amount of geochemical elements (two at a time) in these samples (e.g., right side panel of Figure 1). For our prototype, we considered whether it would be possible to compute a quantitative metric to aid experts in this process.

We decided this metric would be a regression line (RL) for each volcano using samples that could be considered as ground truth (e.g., samples taken from lava sources) and two geochemical elements (SiO₂ or K₂O). These elements were the ones our geologists felt were the most interesting as a starting point because they are representative of the geochemical composition of the majority of the volcanoes of the Southern Andes area. We then calculated the distance to the RL for all the samples assigned to a volcano. If a sample was "far" from the RL of its assigned volcano or "close" to the RL of another volcano, then the sample may have been misassigned. It is important to note that our experts knew some volcanoes are inherently chemically dispersed (e.g., volcanoes Carrán-Los Venados and Caburga-Huelemolle), so that this metric would only be used as a guide more than an exact measure. The details of the measure calculation can be found in our sup. material.

We tested several visualization designs (shown in Figure 4) to integrate this metric as a proxy for geochemical uncertainty in our
<figure><img src="image_6.png" /><figcaption>**Figure 3:** Different iterations to represent uncertainty by geographical proximity to volcanoes and other samples focused on variations of a volcano's sample coverage.</figcaption></figure>

prototype. We first started with visuals that highlighted suspicious assignments of samples, for example, by increasing the sample mark size on the map depending on uncertainty: the bigger the distance to the RL, the bigger the size (see implementation 1 Figure 4). We also considered glyphs that showed that samples may be candidates for other volcanoes, for example, attaching colored "tails" to samples to indicate they were pulled towards other volcanoes if their geochemical consistency was similar to these other volcanoes (design sketches 2 and implementations 2 and 3 in Figure 4). We also allowed users to control thresholds for such distances interactively.

*Lessons learned.* Once implemented and visualized with real-world data, our geologists had trouble grasping the measure's meaning and interpreting the tails or size of samples. Moreover they felt it added unnecessary clutter. In fact, they used the visualization was to find extremes rather than interpret them. For instance, they first identified samples whose assignment could be doubted (by filtering out samples below a threshold) without worrying about why (i.e. ignoring the visual indication of similarity to other volcanoes). They would then look at the full geochemical scatterplot of the volcanoes in question. They mentioned the *context* of the chemical scatterplot and RL was indispensable: they preferred to see the full calculation behind it, i.e., the scatterplots of other candidate volcanoes in the right panel, rather than its abstract representation on the map. Thus, we decided to remove the attraction tails and other visual marks that highlighted possible wrong assignments in favor of the simpler visualization where the sample size on the map showed potential chemical uncertainty. We then included interaction so that when users click on samples of interest, they could see the scatterplot on the right of the interface with (i) the introduced RL of the assigned volcano, (ii) the sample highlighted in context with (iii) the samples of all the other similar volcanoes (bottom of Figure 4).

**Showing the volcano's uncertainty.** As a first step to aid experts in understanding how well-formed their groups (volcanoes) are, we simply showed the number of samples the volcanoes have. This visual cue highlighted volcanoes with few samples assigned to them, and that therefore are not characterized or studied enough. However, this ended up being too simplistic as it did not provide information about the quality of the volcanoes' characterization. We thus changed this measure to show the geochemical distribution

of the samples assigned to a volcano as a means to represent how well-defined or disperse a volcano's geochemical pattern is.

We started by representing volcanoes as distribution glyphs on the map. Figure 5 shows several variations we tried, including showing the distribution as a colored cluster or just a border, showing the regression line within these clusters, and replacing the clusters with heatmaps to show how samples were chemically distributed. We additionally adapted the visual rendering based on zoom level: the more zoomed-out, the more abstract and cluster-like the distribution, and the more zoomed-in, the closer to the complete scatterplot.

*Lessons learned.* When our experts tried the implemented versions of these variations, they commented very positively on the aesthetics. Nevertheless, they found that the various glyphs did not aid their analysis. They mentioned it was not possible measure's context (the regression line) in high resolution. It was also hard to compare RLs across volcanoes since the glyphs appeared in the geographical location of volcanoes and were not aligned for easy comparison. And it was hard for them to identify patterns in a visualization they were unfamiliar with. We thus decided as the default visualization to simply highlight "uncertain" volcanoes, through their rendered size, and leave the information on the detailed geochemical composition on the dedicated scatterplot view on the right.

## 4.3 Domain value & visualization design insights

We underline the value of this work for our domain expert colleagues. Through the design iterations and testing with their data, they were able to identify samples whose assignment can be doubted and volcanoes that were not well defined (e.g., not enough samples to compute a regression line) and that require further study. This further validates the value of such multi-disciplinary design teams [31].

The period of design iteration led to the designing and implementation of several designs, which started with ambitious visualizations and progressively led to fairly minimal interventions, such as the default designs provided in our interface. An important
<figure><img src="image_7.png" /><figcaption>Figure 4: Designs to show a sample's geochemical similarity to a volcano included size (left) and tails pointing to the most similar volcano in terms of geochemistry (right). As tails were hard to interpret, we finally represented geochemical distance as the size of the sample on the map and its location in a geochemical scatterplot (bottom image) that includes the samples of the assigned volcano (purple) and those of similar volcanoes (other colors).</figcaption></figure>

driver for our changes was the weakness of the original sketches to capture the complexity and number of the underlying data, that we could only see once we implemented them. Our sketches and early prototypes, which initially made sense to our experts, seemed overwhelming once seen with real-world data. As previous researchers in visualization have observed, designs may fail when confronted with real-world data [67]. Although some of our visualizations were pleasing to the experts, applying all of them in a single overview visualization was overwhelming. It was also the case that some of our designs were too abstract or unfamiliar to our experts. Further training could alleviate this concern and even prove some of the introduced visualizations to be superior, but this requires further study of the individual point designs we introduced (4 for the geographical coverage, 3 for sample assignment, 3 for volcanoes with

several variations each). This remains an open question. It became nevertheless clear that it was inadvisable to introduce unfamiliar visualization to the broader scientific community (our original goal), in particular since the notion of uncertainty was already hard to define. We opted instead to focus on what we consider a fundamental yet orthogonal challenge related to uncertainty: what constitutes uncertainty in tephra studies so we can visualize it?

This question arises from the most surprising observations when our co-author experts used our resulting prototype for long real analysis sessions: the measures we co-created were hard to interpret regardless of how they were visualized and often ignored in practice. Over several discussions and design iterations, we reached the following conclusion: our treatment of uncertainty was too limiting and too limited at the same time.
<figure><img src="image_8.png" /><figcaption>Figure 5: Selection of design iterations to represent the distribution of a volcano's sample. On the top, some images of the initial paper designs. On the bottom, screenshots of the different designs we implemented and tested with our geologists colleagues. Details about implementation 4 can be found in supp. material.</figcaption></figure>

On the one hand, our quantitative measure of geochemical uncertainty was too limiting. It required making several choices that introduced approximation, like a small number of representative chemical elements, which added more uncertainty and might not be the choice of other researchers. In addition, it required the geologists to provide a mathematical definition of their groups (volcanoes) for the concept of "similarity," which is complex and not exact by definition. Our experts needed to remind themselves of the constraints of the quantitative measure we co-created and the context in which it was defined. Our experts also needed to consider several other sources of uncertainty, discussed in detail next, before including the quantitative measure in their analysis, such as a sample's geographical location, the nearby volcanoes, among many other factors.

On the other hand, our treatment of volcano and geographical uncertainty was too limited. One aspect that emerged from the iterations with our experts was that the notion of uncertainty was very malleable. For example, while some volcanoes were deemed uncertain with few samples, for others, only a handful were enough as they had very similar compositions or geographical locations reachable only by material from this volcano. In summary, we found that:

* Defining uncertainty upfront was not comfortable for the geologists. Instead, their notion of uncertainty was malleable and changed depending on other contextual factors, some we had already identified, but others emerged during our work together.

* Our experts often ended up using different uncertainty criteria, e.g., they judged samples from the same volcano differently depending on whether they were analyzed with recent machines vs. older machines.

* It was important to allow experts to define their own criteria, and that there was no clear-cut definition of what thresholds were deemed as uncertain: they would manipulate and explore different levels of values (geographical proximity, chemical composition) and the surrounding context before deciding what to focus on.

With these insights, we realized that (i) uncertainty in tephra studies cannot be defined a priori but is rather a collection of several factors, and that (ii) experts need to be able to progressively explore these factors and construct their definition of what is uncertain or doubtful under their context. Ultimately, the visual representations we co-designed were less crucial for our experts than the definition of uncertainty. It is possible that with more training, our visualizations can be effectively integrated into their analysis flow. However,
for now, a simple size mapping was enough (and preferred) to act
as a starting point for their exploration. We collectively concluded
that it would be challenging to introduce unfamiliar visualizations
to more members of the community, and that the definition of what
is uncertain is important to understand and pin down before more
design work can be conducted.

We thus reached out to the broader community to understand
these factors better and the different needs of analyzing and repre-
senting uncertainty.

## 5 2nd Iteration: Reaching out to the broader community, the malleable nature of uncertainty and the need for user agency

<figure><img src="image_9.png" /></figure>

Our findings
in the previ-
ous study high-
lighted that the

sources of uncertainty are more numerous, their interpretation is
less straightforward and their interconnection is more complex
than what we expressed in the first prototype. We thus set out
to investigate a research question orthogonal to our 1st iteration:
*when considering a broad set of scientists doing tephra studies, what*
*factors contribute to the uncertainty they need to visualize?*

To do so, we first conducted semi-structured interviews with 6
different scientists doing tephra studies. Based on their feedback
and lessons learned from the previous iteration, we refined our def-
inition of uncertainty and our tool accordingly. This modified tool
version was then used as a technology probe [36] with a broader
audience (10 domain experts). Through their use of the technology
probe in an exploration session, we studied how a broader set of
experts could use uncertainty metrics in a visual analytics system.
As the topics raised across both the interviews and during the explo-
ration sessions relate to the definition of uncertainty to help design
visualization systems, we report results together, explicitly high-
lighting issues raised in the interview and the use of the technology
probe.

### 5.1 Procedure and Participants

We describe the procedure and participants of the interviews with 6
experts and the sessions with the technology probe with 10 experts.
Sessions for both were conducted in English or Spanish via video
conference, lasted between 1-1.5hrs and required participants to
sign a consent form. As in previous research (e.g., [25]), we only
provide summarized information about our participants, as disag-
gregated descriptions could allow others to identify them. For the
same reasons, our supplementary material includes a summary of
common topics raised, not the full transcripts.

**Interviews with Experts.** Six experts participated in the series of semi-structured interviews. They had between 15 and 35 years of experience in tephra studies, working in Belgium, Chile, France, Switzerland, the United Kingdom, and the United States. The participants had diverse backgrounds and expertise, including geology, geochemistry, volcanology, tephrochronology, quaternary geology, and paleoclimate. Their geographical volcanic zones of

study include Africa, Antartica, Asia, New Zealand, South and North
America, and the Caribbean.

In the interviews, we first asked participants to describe their pro-
cess when assigning a new tephra sample to a volcano or eruption.
We then asked them how they dealt with the doubts of assignments
before and during the analysis of tephra samples. This part included
questions such as how they collected the data, how they would con-
duct a quality assessment and identify doubts, and how they would
communicate these doubts to others. A more detailed protocol
and questions of the interviews are available in the supplementary
material.

**Expert exploration using the Technology Probe.** We recruited 10 participants based in Argentina, Chile, the United States and Switzerland. Only one had participated in the first series of interviews. They had expertise in geology, volcanology, geography, geochemistry, and petrology, with experience ranging from 3 to 40 years. Their geographical volcanic zones of study include Asia, Antarctica, Europe, and South, Central and North America.

We started the sessions by conducting a short semi-structured interview to ask them about the sources of doubts they usually identify in their analysis, how they deal with them, and how they communicate them to other members of the scientific community. We then shared a link to the probe and provided an overview of its functionalities. Once participants declared that they had no further questions about how the prototype works, we asked them to search for samples they would consider dubious and then volcanoes that seemed more uncertain than others. We instructed participants to think out loud and provided simple interaction guidance if they needed help with the functionalities. In the last part of the session, we posed a set of questions about the functionalities of the tool (as a proxy to understand uncertainty), about the uncertainty sources we included, and the quantitative metric we calculated. In addition, we asked them if they could think of other uncertainty sources or metrics to use, and whether the way they treated uncertainty in the session was consistent across all their analysis work.

### 5.2 Technology Probe (2nd Prototype)

We refined our 1st prototype by incorporating the lessons learned
in section 4 and insights from the series of 6 interviews (discussed
in sec. 5.4). Our aim with this refined prototype was to use it as a
technology probe [36], which is a prototype system that is not a
full-featured tool. Its purpose is to help gain an understanding of
users and how well the proposed solution addresses their needs. In
our case, our goal was to encourage participants to reflect on the
notion of uncertainty. We made the following changes with respect
to the 1st prototype (section 4) as shown in Figure 6:

**Additional sources of doubts integrated in the prototype.**

Our experts, co-authors in the 1st iteration (section 4), and the 6 participants in the interview, explained that several additional factors are important in determining the quality and comparability of samples (and volcanoes) analyzed in the past. These are often related to the provenance of the data (i.e., publication year, author, and type of publication), as well as whether the published work followed secondary standards in the geochemical analysis. As these factors were numerous, qualitative and context-dependent (and
<figure><img src="image_10.png" /><figcaption>Figure 6: Technology probe and the three main features added to represent multiple sources of uncertainty. (A) Iterative access to predefined uncertainty measure: the dedicated doubts/uncertainty tab is open at the top. It shows the options to activate the mapping of uncertainty to samples on the map, the possibility to show and hide elements in the interface, and the quantitative metric of a sample in context once it is selected. (B) Additional sources of doubts: here we see the tab dedicated to filtering by provenance (scientific publication). Other sources include chemical analysis standards in the Analysis tab (not visible in this figure).</figcaption></figure>

thus not always useful in the analysis), we opted for using *filters*. We added a dedicated tab for users to filter by provenance ((B) in Figure 6) and secondary standards in the existing analysis tab. While these filters are not “uncertainty” measures per se (nor are labeled as such in our tool), they do represent some of the contextual factors that researchers consider when assessing if sample assignments are dubious or not.

**Iterative access to predefined uncertainty measures.** Given feedback on the 1st prototype, for the predefined measures of geochemical uncertainty (regression lines for volcanoes, distance from these RL for samples) we created a dedicated tab ((A) in Figure 6).

As with filters, users can activate or deactivate these uncertainty measures based on their preference. Our follow-up interviews reinforced this choice of allowing participants to choose when to activate this geochemical filter, as it highlighted that sources of doubts, such as the interpretation of geochemical uncertainty, is also subject to contextual factors. In addition, as our interviews indicated that identifying dubious assignments can also be a driving force for research, we also allow analysts to make uncertain samples or volcanoes bigger in the map, or in contrast making them smaller to ignore them.

For the next step, we chose the simplified visual representations of uncertain samples and volcanoes, only using size to communicate
uncertainty. We made this decision to ensure discussions would focus more on the potential of the tool and the notion of uncertainty rather than specific (often unfamiliar) visual representations (like the glyphs described in section 4).

**Dataset used in the Expert exploration sessions.** As our experts worked studying different volcanic regions, we contemplated asking them to collect their own datasets to use the technology probe. However, collecting such regional datasets is a complex and time-consuming task (it is a research endeavor in their domain). We thus used a reduced version of the BOOM! dataset of the Southern Andes region [53]. From this original dataset (35 volcanoes and 16,768 samples), we selected 10 volcanoes with 9,544 samples to reduce the interview time for participants unfamiliar with the region's volcanic activity. We stress that our goal was not to capture individual exploration insights (such as ones in insight evaluation). These are hard to obtain since the session is not using their own dataset or research questions.

## 5.3 Analysis of Transcripts

For both the expert interviews and the exploration sessions with the technology probe, we started by automatically transcribing the sessions with Adobe Premiere Pro. Then, we corrected them by one of the authors. We then analyzed them using reflexive Thematic Analysis (TA) [15] in three steps: We first did collaborative coding between one author expert in visualization and one author expert in tephra studies for two interviews. Our goal was to unify the vocabulary between the two domains and resolve any misunderstanding by the visualization experts (who lack domain expertise). In the second step, the visualization experts coded the rest of the interviews. Finally, a third pass was done to all transcribed interviews to homogenize the codes. We finished our analysis with a bottom-up approach to merge the initial code into broader themes. One of the authors with visualization expertise conducted the main process of merging the initial themes. Then, the two visualization experts iterated them in collaborative sessions. Resulting code books and analysis are available in the supp. material.

## 5.4 Results

Our analysis of the 6 interviews with experts and the 10 expert exploration sessions using the technology probe led us to high-level themes and insights around the complex and context-dependent nature of uncertainty, and a visualization system's role in analyzing it. For conciseness, we present together the results of both the interview and the experts' exploration sessions. We note nonetheless that the results represent an iterative process: the 6 interviews informed the design of the technology probe, that was used afterwards in the exploration sessions with 10 additional experts. Participants with IDs P1-P6 are experts who participated in the interviews and P7-P16 are those who participated in the exploration sessions using the technology probe.

**Uncertainty is very multifaceted.** There was consensus among all 6 participants of the expert interviews that assessing if sample assignments are suspicious requires considering several factors. We identified 39 distinct sources of doubts in the interviews. These included (and thus confirmed) the factors we had identified in

the first design iteration, such as the geolocation of samples and proximity to other volcanoes (P2,P3), geochemical composition, and (lack of) similarity to other samples in the volcano or similarity to other volcanoes (P2-P4). New categories also emerged, like the fact that analysis methods evolve (all Ps), the age of the volcano can affect sample quality (P2, P4), samples may not be well collected or preserved (P2-P4), or publications of previous results may lack information (P2-P6).

Beyond the numerous factors, the problem is compounded by the fact that many sources of doubts are qualitative and often subjective. The interviews revealed several new such factors that relate to the provenance of the sample assignment. Examples include whether a researcher agrees with the analysis process followed by the researchers who did the assignment or how they reported results (all Ps), and the trust in specific authors or likelihood of human error (P2,P4). As P4 mentioned, "*A lot of past data is not standardized, so there is a lot of uncertainty in the way people report it.*" And more directly, P2 said, "*I trust some authors more than others, because I know how they work.*"

In the exploration sessions using the technology probe, there was a consensus again among all 10 participants that assessing if a sample's assignment is dubious requires considering several factors, confirming our results from the 1st iteration and our interviews. Nevertheless, we also identified 22 new distinct sources of doubts, often related to provenance, bringing our total to 61 sources! For example, some new factors related to doubts were about the lab analysis and preservation of samples, doubts about collection procedures (consistency when samples are in country borders, disturbed sites), sample age and deterioration (because of age or because it has been analyzed multiple times), eruption period, etc. As P12 mentioned, "*Tephra that are older than the last glacial maximum are mostly eroded or mostly removed.*" And in the case of analysis doubts, P12 explained that "*Sometimes the different laboratories have instruments that have not the same calibration.*"

Identifying all sources of doubts a-priory is likely impossible given this variety, but we see three larger categories of sources emerging (first in the interviews, informing the design of the probe, and confirmed in the exploration session with the technology probe). We used these categories to group corresponding filters in the probe. They are: 27 (23 from interviews) data-related sources (e.g., missing information, geochemical inconsistency), that are grouped in our technology probe under the tab "Doubts/Uncertainty"; 12 (10 from interviews) sources of doubts related to subjective-interpretation and trust of others' assignments and reporting - these may require an assessment of authors, publication year, etc. For the sources we have available data, we have grouped them as filters in the probe under the tab "Provenance"; and 22 (6 from interviews) other non-data sources (e.g., old versus new practices or doubts about sample preservation) - when possible to express as a filter, for instance when it comes to analysis standards followed, we added these under the tab "Analysis".

**Uncertainty analysis requires customization:** In the exploration sessions where experts used the technology probe to visually access sample quality in practice, they often explicitly requested ways to add their own notions of uncertainty. For example, 6/10 participants asked explicitly for a way to add their own filters for
other features they considered important for analyzing uncertainty (P7, P9, P10, P12, P14, P16).

When we asked our participants about the new quantitative uncertainty metric we introduced in the first iteration (section 4), all participants for whom the metric seemed to resonate (P7, P9, P10, P12-P15) had suggestions about other aspects to incorporate or change about it. For example, they mentioned including other chemical elements, using non-linear models, updating it as new samples arrive, applying different uncertainty metrics to subsets of their data, etc. For example, P9 wanted to apply different metrics to different samples and "change the regression [type] for chemical samples associated with a certain age." And these suggestions were different for each participant. Here again some participants directly asked for a way to customize this metric or add their own (P12, P14, P15).

**Uncertainty is context-dependent, interconnected, and mal-leable.** In the *interviews* with experts we observed that whether the above factors are considered or not as a source of doubts, and how a researcher weighs it in, depends on the context of their analysis. By context, we refer to all factors that alter the interpretation of tephra assignment but are not considered as sources of doubt. We identified 13 instances where participants indicated that interpretation depends on contextual factors (mentioned at least once by all six participants).

Some of these contextual factors are related to the researcher's situation (e.g., their background, expertise, place of work, and machines they use). Others have to do with the area they study (e.g., the climate conditions). For example, P4 explained, "In Chile, it's a little bit complicated because tephra is not well preserved. It's just you know, a lot of aridity, a lot of rain, too warm. [...] New Zealand has a lot of very old volcanic eruptions that were [well] preserved, but there is not a lot of [...] recent ones."

The *exploration sessions* using the technology probe also confirm that whether a factor is a source of doubt or not depends on context (38 mentions, at least once per participant). As P9 mentioned while exploring, "It varies from volcano to volcano; it also varies longitudinally along the arc, because the further north, say, the closer to Santiago, the less the preservation of the tephras". Furthermore, all participants mentioned that the sources of doubts they considered are interconnected and need to be considered together. As P10 mentioned when using our tool, "It is important to see [the sample] in time, in space, and [...] in chemistry."

More surprisingly, our evaluation with the technology probe revealed that how factors are weighted and interpreted may change even for the same researcher, e.g., depending on the research question they are working on (P7, P8, P13, P15). For example, when asked to elaborate on this, P15 commented: "Yeah, I think definitely they can vary based on the type of problem that I'm trying to solve. [...] In this kind of projects the sources of the doubts in terms of "who is this from, this volcano or this volcano" are less important than the doubts of "is this tephra the same as this other tephra". [...] Whereas in this other project that I'm working on with [...] we've got these unknown tephra, what volcano is this likely from?"

**Uncertainty evolves with knowledge.** In the *interviews* with experts we observed that the definition of uncertainty is linked

with the evolution of methods inside a community, which are in constant revision. Some of these revisions are due to technological improvements and others can be related to new conventions within the community. As a consequence, it is common that samples from older publications tend to generate more doubts than recent ones (P2-P6). For example, P5 commented: "[*...] one of the problems with geochemical data is that you know, older geochemical data doesn't actually have probably the same analytical resolution or precision or standards, rigorous with standards, as the more recent geochemical data."

Our experts recognized that domain knowledge evolves and so does their interpretation of uncertainty. They mentioned that doubts about samples can be resolved over time by the re-analysis of older samples (P1, P3-P6), while others are left to be resolved in the future (P5-P6).

Furthermore, other participants (P3, P5), mentioned that dubious assignments can even be a source of new research questions. P3 commented about a dubious-looking result: "[*...] With the construction of simple models, are the chemical trends consistent with that, or is everything sufficiently different that could be something new."

Our *exploration sessions* with the technology probe confirms these results, as it is common that sample assignments from older publications tend to be seen as more dubious than recent ones (P10-P12, P14), and even their own results may be questioned. For example, P14 mentioned that: "So if I'm not sure of a correlation [between two volcanoes shown on the tool], I will make that known and say why I'm not sure, or why this is my conclusion based on the current evidence and acknowledge that further work could be done in those areas. [...] So that people might see the gaps in the knowledge and they might be in a better position to take these forward than I am."

**Uncertainty needs to be communicated.** Similar to previous work [12], the analysis of our *interviews* with experts shows that, at individual level, our experts apply several strategies to reduce or acknowledge uncertainty. For example, they mentioned crossing data to reduce uncertainty in their work (P2, P4-P5). In order to completely resolve their doubts, they sometimes also need to contact other authors to discuss or start new collaborations (P4-P6), and even request the physical samples for re-analysis (P1, P3-P4, P6).

This was also confirmed in the *exploration sessions* with the technology probe, where participants mentioned that to resolve uncertainty they require communicating with others (P8, P10, P14, P16). Participant P14 commented that: "I think if you have a doubt between a distal and a proximal correlation, and you're looking at somebody else's data, you could then use it to drive for a collaboration, to actually analyze things at the same time on the same probe, which should remove an aspect of analytical doubt."

Our evaluation revealed that other actions are of a bigger scale. Participant P16 mentioned work where a researcher collected and analyzed samples of several laboratories in order to evaluate the quality of the machines [45]: "[*...] he sent out a disk with four different tephra samples to electron microprobe labs all over the world to analyze. [...] So he assigned every lab numbers. And then, when I was working with someone, I could say 'Hey, I'm Lab [X], you know, which lab were you?' so we could look at the for the 4 tephra layers that
> we had analyzed. [...] And so we were able to between our two labs without having to exchange samples, we could compare our results directly.”

Interestingly, most participants stated that they openly communicate uncertain aspects of their own work or doubts about the works of others in their publications (P7-P9, P12-P14, P16). This indicates an openness to discuss uncertainty and encourages future work and replication within the community.

**Visualization helps to understand uncertainty, cross data, but also motivate research.** All participants in the exploration sessions with the technology probe were very excited about the prospect of seeing the collection of assigned tephras in one visual analytics system, commenting on how it helps provide an overview of past research (P8, P10-P14, P16), of tephra data (P12-P15), and find new data/publications (P7, P10).

When it comes to uncertainty, participants commented positively on the use of filters (P7, P10-P13, P15, P16) to help them identify dubious sample assignments. As P15 mentions “*That alone is a really strong part of this tool [...]*, very quickly you can kind of assess that *doubt and uncertainty and something that would have taken forever without this tool.*”

Our experts apply several strategies to reduce or acknowledge uncertainty. For example, they mentioned crossing data as a way to reduce uncertainty (P7, P9-P12, P14, P15). Visualizations can play a key role in reducing uncertainty and helping experts cross data. Even though their focus varied, all but one participant were excited about being able to see together different data: locations of samples and volcanoes, geochemistry, but also our introduced filters that express sources of doubts, such as the provenance of samples (literature), and the analytical filters (sources of doubts around analysis / secondary standards). As P13 mentions: “*I see which ones are out of the norm, which ones aren’t and which ones I would like to see. And on top of that, I’m looking at its geographic location, so it’s super good.*”

As in the interviews, several participants (P9, P13-P15) mentioned that dubious assignments can generate new research questions. Our visualization can play a key role in identifying such samples. As P9 says: “*The amount of information [in the tool] generates a series of questions regarding the terrain process that we do and the discovery of these eruptions, because there is a lot of information here that we may be ignoring at times.*”

# 6 Discussion and Implications for design

Next, we present what we feel are the most important implications for visualization design that stem from our work, and we continue with a general discussion and limitations.

## 6.1 Implications for design and opportunities for future research

**In science, one uncertainty does not fit all: flexibility when defining and visualizing what is uncertain.** Our work revealed that uncertainty can be a set of many complex factors when considering knowledge built and improved by a diverse scientific community. Some may be quantitative, like our geochemistry measures, but may not fit all researcher needs. Others may be qualitative,

like trust in prior research or doubts about sample preservation or analysis, and thus may not have pre-defined values. Our results prove that a list of these sources likely cannot be complete a priori. Prior work [65] has hinted that sources of uncertainty may be more than we design for (3 vs 6 in their case). We started with three most relevant ones and ended up with 61 (!) sources of uncertainty with only a subset of the community. Moreover, we found that whether factors are considered sources of doubt depends on the context and the research question, and their importance can vary even for the same researcher. Researchers bring their domain expertise to bear on assessing past tephra assignments; while investigating sources of doubts they actively construct their own understanding of uncertainty. We observed that any reification of such sources into a concrete metric would need to be malleable to adapt to the researcher’s context and needs. This is a point highlighted by the numerous requests for customization of filters and measures. These findings highlight that uncertainty in science cannot always be reduced to a pre-defined set of measures, either quantitative or qualitative, to be mapped into a visualization (as we first attempted in our work).

→ **Implications for Visualization:** Our work highlights an opportunity to design visualization systems that not only visualize uncertainty but allow experts to construct and express what uncertainty means to them. We advocate for flexible visualizations that give users the agency to define what factors they consider as sources of uncertainty, instead of imposing our definitions (e.g., a metric) or require our users to commit to a rigid definition of what is uncertain. Our systems should give researchers agency to build their own forms of uncertainty, not just because all sources are hard to define a priori, but also because this flexibility allows them to experiment, progressively build, and surface their own forms of uncertainty.

We envision visual analysis systems that combine the flexibility to define new uncertainty sources or measures with the liberty to choose the factors to visualize and the parts of the data to consider. The ability to define and combine filters (e.g., in the form of a visual language), to introduce mathematical formulas for computing metrics, and to simply add personal tags could go a long way in supporting such customization. Depending on the type of uncertainty experts are working with, they could additionally externalize characteristics of this uncertainty: for example, if it is implicit or explicit [79], if it is an assessment, structural or a value hunch [47], or if it is related to inadequate or incomplete information [48]. We observed all these types in our sources. This would require considering work on how to best elicit expert knowledge, for example through capturing implicit interactions or explicit annotations [47, 79]. Visualization systems could then make recommendations of visual encoding or interactions based on these characteristics and the long tradition of uncertainty visualization research (see subsection 2.2).

In terms of the creation of visualization systems, previous work (e.g., [47, 54]) has highlighted that supporting the externalization of users’ knowledge can give them a better view of the implicit errors present in their data. We extend these works by proposing that this externalization of knowledge can go beyond annotating data that cause doubts, giving users the power to introduce sources and construct definitions to express these doubts iteratively. This can better support the externalization of the sensemaking process of
framing and reframing, i.e., the process of updating the explanatory structures to interpret, understand, and organize data into patterns [43].

**Uncertainty in science can be good: flexibility in visualizing the direction of mapping.** In most cases for our experts, uncertain data were seen as a problem and ideally as information to be weighted less in the analysis, giving preference to more certain and trustworthy data. However, there were also cases where our experts expressed that uncertainty can also be a source of inspiration for future research. Panagiotidou et al. [65] also suggest treating implicit error as an inspirational signal rather than noise. This indicates that uncertainty visualizations need to be flexible and allow researchers to either tone down or, on the contrary, highlight sources of doubt and uncertain data.

→ **Implications for Visualization:** Besides allowing users to define the factors they consider as sources of doubts, we advocate for visualization tools that allow users to highlight or fade out uncertain data. It remains future work to understand if and how results from prior work on expressing and interpreting uncertainty using visual channels that attempt to communicate "fuzziness", like blurriness, transparency or sketchiness (e.g., [11, 85]), apply to both situations where uncertainty needs to be visually salient or subtle without users misinterpreting its meaning.

**Uncertainty in science evolves: tracing uncertainty definitions.** McCurdy et al. [54], describe "retrospective adjustment" as the evolution of data over time and the temporal discrepancies it can cause, which are sources of implicit error. Similarly, our experts acknowledged that the evolution of data is a source of uncertainty or doubts. However, in our case, this evolution of knowledge also affects how experts *characterize* uncertainty and what *approaches* they adopt to address it: a process, practice, or method that was considered reliable in the past might not be in the future, and our systems need to account for it. An example is the analysis of the volcanic source of MIC2 and COR1 tephra deposits that were initially assigned to Michinmahuida and Corcovado volcanic centers respectively [57], that were questioned in later publications (e.g., [4, 6, 78, 81, 82]). Apart from updating knowledge in the domain, one of these research works ([4]) initiated a discussion in the community about changing fundamental practices, like naming tephras, in a way that anticipates future reassignments. Visualization systems could not only adapt to new definitions but also act as repositories and archives of past or alternative definitions of uncertainty, as well as what past knowledge (e.g., sample assignments) has been revisited over time.

→ **Implications for Visualization:** Allowing for modularity and ways to introduce or customize new emerging uncertainty sources (definition) and basic visual marks to highlight them (encoding) can support the efforts of scientific communities in understanding past work and identifying open problems. As research methods and knowledge evolve inside the community, so does the definition of uncertainty. **Knowledge evolves: our tools should evolve with them.** We believe that our visualization systems can to adapt to these changes and can act as an archive of community knowledge. Considering the evolution of tools and knowledge over time introduces interesting challenges for visualization research.

For example, evaluating such evolving systems could consider levels of agreement or points of divergence between members of a scientific community, and visualization research could study which visualizations emerge within the community or how to visually represent divergent versions of the same tool. We can be inspired by work on exposing visualization design alternatives [86] or on provenance visualization of explorations (e.g., [68, 88]), but adapted not just for tracing the visual design or exploration choices, but also choices about uncertainty definitions and uncertainty visualization customizations, and studying how these evolve over time.

**Uncertainty visualization as a force for dialogue in a scientific community.** Similar to the interviews by Boukhelifa et al. [12], our results stress that scientists rarely work in isolation. Some of our experts explicitly commented that uncertainty was why they reached out and started collaborations. Uncertainty can thus be seen as a valuable driver not just for research but also for communication among members of the community.

→ **Implications for Visualization:** Visualizations have the potential to become a medium for dialogue in the community around uncertainty. Scientists could propose and share new definitions of uncertainty (for example, in the form of new filters), discuss their validity, and re-use definitions of others for their replications. This could help scientists compare alternative definitions of uncertainty and better contextualize their current sources of doubts with those that other members of the scientific community had in the past.

Visualizations have previously been seen as platforms for a community to exchange information and insights [80]. In addition, previous work around uncertainty (e.g., [47, 54]) has advocated that externalizing doubts can support collaboration. We can envision a combination of the two in the context of uncertainty in science: collaborative visualization platforms can act not only as a means to share information about doubts but also as platforms to collaboratively create uncertainty definitions that support the evolution of research methods and practices.

## 6.2 Design studies: challenges, transferability and contributions

Conducting design studies is costly and requires a substantial time investment from colleagues across all disciplines (in our case this work lasted two years). As it was our original goal, we opened it up to other domain experts outside our design team once we finished the first design. This highlighted that even in a small community like tephra studies research, the methods, processes, and opinions of what constitutes uncertainty are extremely varied. While our co-design work indicated the complexity of combining different sources of uncertainty, it was the open dialogue with the rest of the community that helped us see how surprisingly multi-faceted and contextual it was. Questions about the transferability of results in design studies (e.g., [55, 72]) and domain application papers (e.g., [83, 87]) are not new. However, our results also make us question how applicable our designs are, not just outside the domain of study (which is expected) but also outside our design group. Should our co-design and design study methodologies include an additional validation step outside the core design team? Or is the variety we identified in our work an artifact of the domain we studied?
We believe the variety we identified results from studying a branch of natural sciences where the community improves knowledge over time. As such, we believe our findings can also be transferred to the design process of visualization tools for uncertainty in other domains. For example, in domains such as geology, biology, and astronomy, where observation and human classification are molded by the currently used methods, equipment, and goals, uncertainty will inevitably change as the scientific community changes.

This leads to the question of the utility of our efforts. In the end, is the process of assessing uncertainty equivalent to the scientific process? Is it even worth trying to elicit and visualize sources of doubts? We believe that our results allowed us to observe the limitations of our tools when trying to integrate the non-objective nature of scientific data and the hidden uncertainty it contains [17] in our visualizations. The co-design process to externalize expert knowledge in a tool helped us to *name* factors that impact the scientific process to understand it better. Both the process of co-designing the tool with experts and later the use of the technology probe by other members of the scientific community helped the experts *surface, build, and articulate* their definition of uncertainty, which can uncover open questions and opportunities for research in their domain.

When reflecting on our contributions as a design study (e.g., [55, 71]), we have contributed: (i) a deeper understanding of the tephra science domain, (ii) the importance of adapting our visualizations to a community's commonly used visuals and practices (Sections 4.2 and 4.3), and (iii) design recommendations for visualizations that consider data/knowledge built by a community (Section 5.4). We feel our process prioritized matters of care [2], focusing on the needs of our collaborators and challenges faced by the tephra community over promoting newly introduced visualization designs. We believe this approach helped us arrive at a visualization design the tephra studies community is interested in integrating into their work. For example, we are currently collaborating with a different group of geologists to construct a new tephra dataset for another region of the world using our visualization tool.

Nevertheless, we struggled with how to communicate our results to the visualization community. After numerous design and implementation iterations for visualization glyphs, our visualization system shows, by default, the uncertainty measures using simply a size encoding. After two years of work, did we simply create a visualization with a set of filters and a representation of uncertainty using size? Our main contribution lies in how the malleable nature of uncertainty in science can influence its definition and, therefore, the designs of visualization systems. Our results highlight that uncovering the hidden uncertainty for visualization [17] is a complex process: design spaces on how to visualize uncertainty are extremely valuable in our community, but we also need to understand how the systems in which we integrate these visualizations need to evolve.

We also asked ourselves if understanding multi-source uncertainty is the same or similar to multi-dimensional exploration. We found the notions of outliers, focus, and interplay of dimensions (sources) that remind visual analytical task taxonomies [5]. Ultimately, our findings suggest that anything that is a data dimension can also be a source of uncertainty depending on context and research question.

## 6.3 Limitations

The complexity of collecting regional datasets reduced our ecological validity in searching for suspicious samples in the exploration sessions with the technology probe (subsection 5.2), as geologists usually know the region they are studying. Our simplification of the dataset also meant that participants familiar with the volcanoes in the Southern Andes zone felt that there was missing information. Some participants additionally mentioned that the time to explore the technology probe was too short to understand it in depth. Longer interventions with the full dataset could have provided us with more insights on the long-term use of the tool to define and visualize uncertainty.

The choice to focus on map-based visualizations of uncertainty can also be considered a limitation of our work. This was a decision we took early on the project, as our geologists expressed the importance of the geographic position of a tephra sample: being near a lake or a forest can be important factors to consider when evaluating a previous assignment that are harder to encode in other visualizations (e.g., a scatterplot). Furthermore, as we expected to open our resulting visualization tool to a larger community, we opted for more traditional views to make them more accessible and follow their domain idioms. Although we believe that alternative visualizations would not have altered our main results (i.e. the need for user agency to account for the evolving practices and context-dependent uncertainty), it remains future work to answer this question.

## 7 Conclusions and future work

We presented a design study of a two-year collaboration whose goal was to incorporate uncertainty into a visualization system to help geologists assess past work on analyzing volcanic deposits. We describe our procedure and lessons learned from two iterations and prototype development: a design by immersion approach with a team of geologists and visualization experts and a series of interviews followed by an evaluation using a design probe with a broader set of members of the community studying volcanic deposits. The analysis of our work showed us that designing visualizations to explore the uncertainty related to the evolving and contextual creation of scientific knowledge around natural phenomena needs to consider that uncertainty is malleable, multi-faceted, context-dependent, and evolves with the knowledge itself. We contributed a set of visualization implications highlighting the importance of user agency and system flexibility when visualizing uncertainty sources so that users could construct and surface their own uncertainty definitions and support their revision over time. In our future work, we aim to explore how to include interactive methods to define complex uncertainty measures and how to introduce communication features in our visualizations to promote community discussion around them.

## References

[1] Derya Akbaba, Lauren Klein, and Miriah Meyer. 2025. Entanglements for Visualization: Changing Research Outcomes through Feminist Theory. *IEEE Transactions on Visualization and Computer Graphics* 31, 1 (2025), 1279–1289. https://doi.org/10.1109/TVCG.2024.3456171

[2] Derya Akbaba, Devin Lange, Michael Correll, Alexander Lex, and Miriah Meyer. 2023. Troubling Collaboration: Matters of Care for Visualization Design Study. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems*
(Hamburg, Germany) (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 812, 15 pages. https://doi.org/10.1145/3544548.3581168

[3] Brent Alloway, G. Larsen, David Lowe, Phil Shane, and John Westgate. 2007. *Tephrochronology (Encyclopaedia of Quaternary Science, 1st edition)*. 2869–2898.

[4] Brent V. Alloway, Nick J.G. Pearce, Patricio I. Moreno, Gustavo Villarosa, Ignacio Jara, Ricardo De Pol-Holz, and Valeria Outes. 2017. An 18,000 year-long eruptive record from Volcán Chaitén, northwestern Patagonia: Paleoenvironmental and hazard-assessment implications. *Quaternary Science Reviews* 168 (2017), 151–181. https://doi.org/10.1016/j.quascirev.2017.05.011

[5] R. Amar, J. Eagan, and J. Stasko. 2005. Low-level components of analytic activity in information visualization. In *IEEE Symposium on Information Visualization, 2005. INFOVIS 2005*. 111–117. https://doi.org/10.1109/INFOVIS.2005.1532136

[6] A Amigo, LE Lara, and VC Smith. 2013. Holocene record of large explosive eruptions from Chaitén and Michinmahuida Volcanoes, Chile. Andean Geol. 40, 227e248.

[7] Kevin D. Ash, Ronald L. Schumann, and Gregg C. Bowser. 2014. Tornado Warning Trade-Offs: Evaluating Choices for Visually Communicating Risk. *Weather, Climate, and Society* 6, 1 (2014), 104–118. https://doi.org/10.1175/WCAS-D-13-00021.1

[8] L Bertin, H Moreno, and L Becerril. 2018. Peligros del Campo Volcánico Carrán-Los Venados. *Región de los Ríos, Servicio Nacional de Geología y Minería, Carta Geológica de Chile, Serie Geología Ambiental* 33 (2018), 52.

[9] Steve Blenkinsop, Pete Fisher, Lucy Bastin, and Jo Wood. 2000. Evaluating the Perception of Uncertainty in Alternative Visualization Strategies. *Cartographica: The International Journal for Geographic Information and Geovisualization* 37, 1 (2000), 1–14. https://doi.org/10.3138/3645-4V22-0M23-3T52 arXiv:https://doi.org/10.3138/3645-4V22-0M23-3T52

[10] Georges-Pierre Bonneau, Hans-Christian Hege, Chris R. Johnson, Manuel M. Oliveira, Kristin Potter, Penny Rheingans, and Thomas Schultz. 2014. *Overview and State-of-the-Art of Uncertainty Visualization*. Springer London, London, 3–27. https://doi.org/10.1007/978-1-4471-6497-5_1

[11] Nadia Boukhelifa, Anastasia Bezerianos, Tobias Isenberg, and Jean-Daniel Fekete. 2012. Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty. *IEEE Transactions on Visualization and Computer Graphics* 18, 12 (2012), 2769–2778. https://doi.org/10.1109/TVCG.2012.220

[12] Nadia Boukhelifa, Marc-Emmanuel Perrin, Samuel Huron, and James Eagan. 2017. How Data Workers Cope with Uncertainty: A Task Characterisation Study. In *Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17)*. Association for Computing Machinery, New York, NY, USA, 3645–3656. https://doi.org/10.1145/3025453.3025738

[13] Ken Brodlie, Rodolfo Allendes Osorio, and Adriano Lopes. 2012. *A Review of Uncertainty in Data Visualization*. Springer London, London, 81–109. https://doi.org/10.1007/978-1-4471-2804-5_6

[14] John Seely Brown and Paul Duguid. 2017. *The Social Life of Information: Updated, with a New Preface*. Harvard Business Review Press, Boston, MA, USA.

[15] Victoria Clarke and Virginia Braun. 2021. Thematic analysis: a practical guide. (2021).

[16] Christopher Collins, Sheelagh Carpendale, and Gerald Penn. 2007. *Visualization of Uncertainty in Lattices to Support Decision-Making*. In *Proceedings of Eurographics/IEEE VGTC Symposium on Visualization (EuroVis 2007)*. 51–58.

[17] Michael Correll. 2019. *Ethical Dimensions of Visualization Research*. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300418

[18] Michael Correll and Michael Gleicher. 2014. Error Bars Considered Harmful: Exploring Alternate Encodings for Mean and Error. *IEEE Transactions on Visualization and Computer Graphics* 20, 12 (2014), 2142–2151. https://doi.org/10.1109/TVCG.2014.2346298

[19] Michael Correll, Dominik Moritz, and Jeffrey Heer. 2018. Value-Suppressing Uncertainty Palettes. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18)*. Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3173574.3174216

[20] Dwight Raymond Crandell and Donal Ray Mullineaux. 1978. Potential hazards from future eruptions of Mount St. Helens Volcano, Washington. *US Geological Survey Bulletin, Report, 1383-C* (1978), 36 pages. http://pubs.er.usgs.gov/publication/b1383C

[21] Siwan M. Davies, Barbara Wohlfarth, Stefan Wastegård, Mamite Andersson, Simon Blockley, and Göran Possnert. 2004. Were there two Borobol Tephras during the early Lateglacial period: implications for tephrochronology? *Quaternary Science Reviews* 23, 5 (2004), 581–589. https://doi.org/10.1016/j.quascirev.2003.11.006

[22] N de Lange, A Panayotov, and G Rees. [n. d.]. Mapping the Jewish communities of the Byzantine Empire. http://byzantinejewry.net/

[23] Stephanie Deitrick and Elizabeth A. Wentz. 2015. Developing Implicit Uncertainty Visualization Methods Motivated by Theories in Decision Science. *Annals of the Association of American Geographers* 105, 3 (2015), 531–551. https://doi.org/10.1080/00045608.2015.1012635 arXiv:https://doi.org/10.1080/00045608.2015.1012635

[24] Angela Fagerlin, Brian J. Zikmund-Fisher, and Peter A. Ubel. 2011. Helping Patients Decide: Ten Steps to Better Risk Communication. *JNCI: Journal of the National Cancer Institute* 103, 19 (2011), 1436–1443.

https://doi.org/10.1093/jnci/djr318 arXiv:https://academic.oup.com/jnci/article-pdf/103/19/1436/7673005/djr318.pdf

[25] Sebastian S. Feger, Sünje Dallmeier-Tiessen, Albrecht Schmidt, and Pawel W. Woźniak. 2019. Designing for Reproducibility: A Qualitative Study of Challenges and Opportunities in High Energy Physics. In *Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19)*. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300685

[26] Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, and Matthew Kay. 2018. Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18)*. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173718

[27] Karen Fontijn, Stefan M. Lachowycz, Harriet Rawson, David M. Pyle, Tamsin A. Mather, José A. Naranjo, and Hugo Moreno-Roa. 2014. Late Quaternary tephrostratigraphy of southern Chile and Argentina. *Quaternary Science Reviews* 89 (2014), 70–84. https://doi.org/10.1016/j.quascirev.2014.02.007

[28] Mirta Galesic, Rocio Garcia-Retamero, and Gerd Gigerenzer. 2009. Using icon arrays to communicate medical risks: overcoming low numeracy. *Health psychology* 28, 2 (2009), 210.

[29] Rocio Garcia-Retamero and Ulrich Hoffrage. 2013. Visual representation of statistical information improves diagnostic inferences in doctors and their patients. *Social Science & Medicine* 83 (2013), 27–33. https://doi.org/10.1016/j.socscimed.2013.01.034

[30] Henning Griethe, Heidrun Schumann, et al. 2006. The visualization of uncertain data: Methods and problems.. In *Sim Vis*. 143–156.

[31] Kyle Wm. Hall, Adam J. Bradley, Uta Hinrichs, Samuel Huron, Jo Wood, Christopher Collins, and Sheelagh Carpendale. 2020. Design by Immersion: A Transdisciplinary Approach to Problem-Driven Visualizations. *IEEE Transactions on Visualization and Computer Graphics* 26, 1 (2020), 109–118. https://doi.org/10.1109/TVCG.2019.2934790

[32] György Hetényi, László Balázs, Zoltán Barcza, Eszter Békési, Erzsebet Gyori, Anikó Kern, Norbert Szabó, and Gábor Timár. 2022. The inherent uncertainty in geosciences. *Acta Geodaetica et Geophysica* 57 (09 2022), 1–8. https://doi.org/10.1007/s40328-022-00392-6

[33] Jerry L. Hintze and Ray D. Nelson. 1998. Violin Plots: A Box Plot-Density Trace Synergism. *The American Statistician* 52, 2 (1998), 181–184. https://doi.org/10.1080/00031305.1998.10480559 arXiv:https://www.tandfonline.com/doi/pdf/10.1080/00031305.1998.10480559

[34] Jessica Hullman. 2020. Why Authors Don't Visualize Uncertainty. *IEEE Transactions on Visualization and Computer Graphics* 26, 1 (2020), 130–139. https://doi.org/10.1109/TVCG.2019.2934287

[35] Jessica Hullman, Paul Resnick, and Eytan Adar. 2015. Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering. *PLOS ONE* 10, 11 (11 2015), 1–25. https://doi.org/10.1371/journal.pone.0142444

[36] Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Alison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, Nicolas Roussel, and Björn Eiderbäck. 2003. Technology Probes: Inspiring Design for and with Families. In *Proceedings of the Annual ACM Conference on Human Factors in Computing Systems, CHI '03*. Association for Computing Machinery, 17–24. https://doi.org/10.1145/642611.642616

[37] Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Alison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, Nicolas Roussel, and Björn Eiderbäck. 2003. Technology Probes: Inspiring Design for and with Families. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (Ft. Lauderdale, Florida, USA) (CHI '03). Association for Computing Machinery, New York, NY, USA, 17–24. https://doi.org/10.1145/642611.642616

[38] Alex Kale, Matthew Kay, and Jessica Hullman. 2019. Decision-Making Under Uncertainty in Research Synthesis: Designing for the Garden of Forking Paths. In *Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19)*. Association for Computing Machinery, New York, NY, USA, 17–24. https://doi.org/10.1145/3290605.3300432

[39] Kostas Kampourakis and Kevin McCain. 2019. 149C11Uncertainty Is Inherent in Science. In *Uncertainty: How It Makes Science Advance*. Oxford University Press. https://doi.org/10.1093/oso/9780190871666.003.0011 arXiv:https://academic.oup.com/book/0/chapter/321190139/chapter-pdf/49195034/oso-9780190871666-chapter-11.pdf

[40] Matthew Kay, Tara Kola, Jessica R. Hullman, and Sean A. Munson. 2016. When (Ish) is My Bus? User-Centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems. In *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems* (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 5092–5103. https://doi.org/10.1145/2858036.2858558

[41] Christoph Kinkeldey, Alan M. MacEachren, Maria Riveiro, and Jochen Schiewe. 2017. Evaluating the effect of visually represented geodata uncertainty on
decision-making: systematic review, lessons learned, and recommendations. *Cartography and Geographic Information Science* 44, 1 (2017), 1-21. https://doi.org/10.1080/15230406.2015.1089792 arXiv:https://doi.org/10.1080/15230406.2015.1089792

[42] Christoph Kinkeldey, Alan M. MacEachren, and Jochen Schiewe. 2014. How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies. *The Cartographic Journal* 51, 4 (2014), 372–386. https://doi.org/10.1179/1743277414Y.0000000099 arXiv:https://doi.org/10.1179/1743277414Y.0000000099

[43] Gary Klein, J.K. Phillips, E.L. Rall, and Deborah Peluso. 2007. A data-frame theory of sensemaking. *Expertise out of Context: Proceedings of the Sixth International Conference on Naturalistic Decision Making* (01 2007), 113–155.

[44] Kimberly E. Klockow-McClain, Renee A. McPherson, and Rick P. Thomas. 2020. Cartographic Design for Improved Decision Making: Trade-Offs in Uncertainty Visualization for Tornado Threats. *Annals of the American Association of Geographers* 110, 1 (2020), 314–333. https://doi.org/10.1080/24694452.2019.1602467 arXiv:https://doi.org/10.1080/24694452.2019.1602467

[45] S.C. Kuehn, D.G. Froese, and P.A.R. Shane. 2011. The INTAV intercomparison of electron-beam microanalysis of glass by tephrochronology laboratories: Results and recommendations. *Quaternary International* 246, 1 (2011), 19–47. https://doi.org/10.1016/j.quaint.2011.08.022 Enhancing tephrochronology and its application (INTREPID Project): Hiroki Machida commemorative volume.

[46] Johannes Liem, Eirini Goudarouli, Steven Hirschorn, Jo Wood, and Charles Perin. 2018. Conveying uncertainty in archived war diaries with GeoBlobs. In *IEEE VIS 2018 Electronic Conference*.

[47] Haihan Lin, Derya Akbaba, Miriah Meyer, and Alexander Lex. 2022. Data Hunches: Incorporating Personal Knowledge Into Visualizations. *IEEE Transactions on Visualization and Computer Graphics* (2022), 1–11. https://doi.org/10.1109/TVCG.2022.3209451

[48] Raanan Lipshitz and Orna Strauss. 1997. Coping with Uncertainty: A Naturalistic Decision-Making Analysis. *Organizational Behavior and Human Decision Processes* 69, 2 (1997), 149–163. https://doi.org/10.1006/obhd.1997.2679

[49] David J. Lowe. 2011. Tephrochronology and its application: A review. *Quaternary Geochronology* 6, 2 (2011), 107–153. https://doi.org/10.1016/j.quageo.2010.08.003

[50] Min Lu, Shuaiqi Wang, Joel Lanir, Noa Fish, Yang Yue, Daniel Cohen-Or, and Hui Huang. 2020. Winglets: Visualizing Association with Uncertainty in Multi-class Scatterplots. *IEEE Transactions on Visualization and Computer Graphics* 26, 1 (2020), 770–779. https://doi.org/10.1109/TVCG.2019.2934811

[51] Alan M. MacEachren, Anthony Robinson, Susan Hopper, Steven Gardner, Robert Murray, Mark Gahegan, and Elisabeth Hetzler. 2005. Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know. *Cartography and Geographic Information Science* 32, 3 (2005), 139–160. https://doi.org/10.1559/1523040054738936 arXiv:https://doi.org/10.1559/1523040054738936

[52] Alan M. MacEachren, Robert E. Roth, James O'Brien, Bonan Li, Derek Swingley, and Mark Gahegan. 2012. Visual Semiotics & Uncertainty Visualization: An Empirical Study. *IEEE Transactions on Visualization and Computer Graphics* 18, 12 (2012), 2496–2505. https://doi.org/10.1109/TVCG.2012.279

[53] Consuelo Martinez Fontaine, Vanessa Peña-Araya, Chiara Marmo, Marine Le Morvan, Guillaume Delpech, Karen Fontijn, Giuseppe Siani, and Lucile Cosyn-Wexsteen. 2023. BOOM! Tephrochronological dataset and exploration tool of the Southern (33–46° S) and Austral (49–55° S) volcanic zones of the Andes. *Quaternary Science Reviews* 316 (2023), 108254. https://doi.org/10.1016/j.quascirev.2023.108254

[54] Nina Mccurdy, Julie Gerdes, and Miriah Meyer. 2019. A framework for Externalizing Implicit Error Using Visualization. *IEEE Transactions on Visualization and Computer Graphics* 25, 1 (2019), 925–935. https://doi.org/10.1109/TVCG.2018.2864913

[55] Miriah Meyer and Jason Dykes. 2020. Criteria for Rigor in Visualization Design Study. *IEEE Transactions on Visualization and Computer Graphics* 26, 1 (2020), 87–97. https://doi.org/10.1109/TVCG.2019.2934539

[56] PI Moreno, Isabel Vilanova, R Villa-Martinez, RB Dunbar, DA Mucciarone, MR Kaplan, RD Garreaud, M Rojas, CM Moy, De Pol-Holz, et al. 2018. Onset and evolution of southern annular mode-like changes at centennial timescale. *Scientific Reports* 8, 1 (2018), 1–9.

[57] José A. Naranjo and Charles R. Stern. 2004. Holocene tephrochronology of the southernmost part (42'30'-45'S) of the Andean Southern Volcanic Zone. *Revista geologica de Chile* 31 (12 2004), 224 - 240. http://www.scielo.cl/scielo.php?script=sci_artext&pid=S0716-028200400200003&nrm=iso

[58] Gina Neff, Anissa Tanweer, Brittany Fiore-Gartland, and Laura Osburn. 2017. Critique and contribute: A practice-based framework for improving critical data studies and data science. *Big data* 5, 2 (2017), 85–97.

[59] Stan Nowak, Lyn Bartram, and Pascal Haegeli. 2020. Designing for Ambiguity: Visual Analytics in Avalanche Forecasting. In *2020 IEEE Visualization Conference (VIS)*. 81–85. https://doi.org/10.1109/VIS47514.2020.00023

[60] Lace Padilla, Matthew Kay, and Jessica Hullman. 2020. Uncertainty visualization. (2020).

[61] Lace MK Padilla, Maia Powell, Matthew Kay, and Jessica Hullman. 2021. Uncertain about uncertainty: How qualitative expressions of forecaster confidence impact decision-making with uncertainty visualizations. *Frontiers in Psychology* 11 (2021), 579267.

[62] Lace M Padilla, Ian T Ruginski, and Sarah H Creem-Regehr. 2017. Effects of ensemble and summary displays on interpretations of geospatial uncertainty data. *Cognitive research: principles and implications* 2, 1 (2017), 1–16.

[63] Georgia Panagiotidou, Houda Lamqaddam, Jeroen Poblome, Koenraad Brosens, Katrien Verbert, and Andrew Vande Moere. 2023. Communicating Uncertainty in Digital Humanities Visualization Research. *IEEE Transactions on Visualization and Computer Graphics* 29, 1 (2023), 635–645. https://doi.org/10.1109/TVCG.2022.3209436

[64] Georgia Panagiotidou, Jeroen Poblome, Jan Aerts, and Andrew Vande Moere. 2022. Designing a Data Visualisation for Interdisciplinary Scientists. How to Transparently Convey Data Frictions? *Computer Supported Cooperative Work (CSCW)* 31, 4 (2022), 633–667.

[65] Georgia Panagiotidou, Ralf Vandam, Jeroen Poblome, and Andrew Vande Moere. 2021. Implicit Error, Uncertainty and Confidence in Visualization: an Archaeological Case Study. *IEEE Transactions on Visualization and Computer Graphics* (2021), 1–1. https://doi.org/10.1109/TVCG.2021.3088339

[66] Alex T Pang, Craig M Wittenbrink, Suresh K Lodha, et al. 1997. Approaches to uncertainty visualization. *The Visual Computer* 13, 8 (1997), 370–390.

[67] A. Johannes Pretorius and Jarke J. Van Wijk. 2009. What Does the User Want to See? What do the Data Want to Be? *Information Visualization* 8, 3 (2009), 153–166. https://doi.org/10.1057/ivs.2009.13

[68] Eric D. Ragan, Alex Endert, Jibonananda Sanyal, and Jian Chen. 2016. Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes. *IEEE Transactions on Visualization and Computer Graphics* 22, 1 (2016), 31–40. https://doi.org/10.1109/TVCG.2015.2467551

[69] Gethin Rees, Nicholas de Lange, and Alexander Panayotov. 2018. Mapping the Jewish communities of the Byzantine Empire using GIS. *Migration and Migrant Identities in the Near East from Antiquity to the Middle Ages* (2018), 104–121.

[70] Dominik Sacha, Hansi Senaratne, Bum Chul Kwon, Geoffrey Ellis, and Daniel A. Keim. 2016. The Role of Uncertainty, Awareness, and Trust in Visual Analytics. *IEEE Transactions on Visualization and Computer Graphics* 22, 1 (2016), 240–249. https://doi.org/10.1109/TVCG.2015.2467591

[71] Michael Sedlmair. 2016. Design Study Contributions Come in Different Guises: Seven Guiding Scenarios. In *Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization* (Baltimore, MD, USA) (BELIV '16). Association for Computing Machinery, New York, NY, USA, 152–161. https://doi.org/10.1145/2993901.2993913

[72] Michael Sedlmair, Miriah Meyer, and Tamara Munzner. 2012. Design Study Methodology: Reflections from the Trenches and the Stacks. *IEEE Transactions on Visualization and Computer Graphics* 18, 12 (2012), 2431–2440. https://doi.org/10.1109/TVCG.2012.213

[73] Meredith Skeels, Bongshin Lee, Greg Smith, and George Robertson. 2008. Revealing Uncertainty for Information Visualization. In *Proceedings of the Working Conference on Advanced Visual Interfaces* (Napoli, Italy) (AVI '08). Association for Computing Machinery, New York, NY, USA, 376–379. https://doi.org/10.1145/1385569.1385637

[74] Aidan Slingsby, Jason Dykes, and Jo Wood. 2011. Exploring Uncertainty in Geodemographics with Interactive Graphics. *IEEE Transactions on Visualization and Computer Graphics* 17, 12 (2011), 2545–2554. https://doi.org/10.1109/TVCG.2011.197

[75] Judi Thomson, Elizabeth Hetzler, Alan MacEachren, Mark Gahegan, and Misha Pavel. 2005. A typology for visualizing uncertainty. In *Visualization and Data Analysis 2005*, Robert F. Erbacher, Jonathan C. Roberts, Matti T. Grohn, and Katy Borner (Eds.), Vol. 5669. International Society for Optics and Photonics, SPIE, 146 - 157. https://doi.org/10.1117/12.587254

[76] Chris S. M. Turney, Douglas D. Harkness, and J. John Lowe. 1997. The use of microtephra horizons to correlate Late-glacial lake sediment successions in Scotland. *Journal of Quaternary Science* 12, 6 (1997), 525–531. https://doi.org/10.1002/(SICI)1099-1417(199711/12)6<525::AID-JQS347>3.0.CO;2-M

[77] Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty. *science* 185, 4157 (1974), 1124–1131.

[78] Brent V. v. Patricio I. Moreno, Nick J. G. Pearce, Ricardo De Pol-Holz, William I. Henriquez, Oscar H. Pesce, Esteban Sagredo, Gustavo Villarosa, and Valeria Outes. 2017. Stratigraphy, age and correlation of Lepué Tephra: a widespread c. 11 000 cal a BP marker horizon sourced from the Chaitén Sector of southern Chile. *Journal of Quaternary Science* 32, 6 (2017), 795–829. https://doi.org/10.1002/jqs.2976 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/jqs.2976

[79] Anne Marthe Van der Bles, Sander Van Der Linden, Alexandra LJ Freeman, James Mitchell, Ana B Galvao, Lisa Zaval, and David J Spiegelhalter. 2019. Communicating uncertainty about facts, numbers and science. *Royal Society Open science* 6, 5 (2019), 181870.

[80] Fernanda B. Viegas, Martin Wattenberg, Frank van Ham, Jesse Kriss, and Matt McKeon. 2007. ManyEyes: a Site for Visualization at Internet Scale. *IEEE Transactions on Visualization and Computer Graphics* 13, 6 (2007), 1121–1128. https://doi.org/10.1109/TVCG.2007.70577
[81] SFL Watt, DM Pyle, and TA Mather. 2013. Evidence of mid-to late-Holocene explosive rhyolitic eruptions from Chaitén volcano, Chile. Andean Geol. 40, 216e226.

[82] Sebastian F. L. Watt, David M. Pyle, Tamsin A. Mather, Robert S. Martin, and Naomi E. Matthews. 2009. Fallout and distribution of volcanic ash over Argentina following the May 2008 explosive eruption of Chaitén, Chile. *Journal of Geophysical Research: Solid Earth* 114, B4 (2009). https://doi.org/10.1029/2008JB006219 arXiv:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2008JB006219

[83] Gunther H. Weber, Sheelagh Carpendale, David Ebert, Brian Fisher, Hans Hagen, Ben Shneiderman, and Anders Ynnerman. 2017. Apply or Die: On the Role and Assessment of Application Papers in Visualization. *IEEE Computer Graphics and Applications* 37, 3 (2017), 96–104. https://doi.org/10.1109/MCG.2017.51

[84] Mark D Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E Bourne, et al. 2016. The FAIR Guiding Principles for scientific data management and stewardship. *Scientific data* 3, 1 (2016), 1–9.

[85] Jo Wood, Petra Isenberg, Tobias Isenberg, Jason Dykes, Nadia Boukhelifa, and Aidan Slingsby. 2012. Sketchy Rendering for Information Visualization. *IEEE Transactions on Visualization and Computer Graphics* 18, 12 (2012), 2749–2758. https://doi.org/10.1109/TVCG.2012.262

[86] Jo Wood, Alexander Kachkaev, and Jason Dykes. 2019. Design Exposition with Literate Visualization. *IEEE Transactions on Visualization and Computer Graphics* 25, 1 (2019), 759–768. https://doi.org/10.1109/TVCG.2018.2864836

[87] Aoyu Wu, Dazhen Deng, Furui Cheng, Yingcai Wu, Shixia Liu, and Huamin Qu. 2023. In Defence of Visual Analytics Systems: Replies to Critics. *IEEE Transactions on Visualization and Computer Graphics* 29, 1 (2023), 1026–1036. https://doi.org/10.1109/TVCG.2022.3209360

[88] Kai Xu, Alvitta Ottley, Conny Walchshofer, Marc Streit, Remco Chang, and John Wenskovitch. 2020. Survey on the Analysis of User Interactions and Visualization Provenance. *Computer Graphics Forum* 39, 3 (2020), 757–783. https://doi.org/10.1111/cgf.14035 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14035