Vistrust: a Multidimensional Framework andEmpirical Study of Trust in Data Visualizations, Adam Stefkovics , Johanna Beyer , Eric Moerth
Cindy Xiong Bearfield , Carolina Nobre

Trust in Vis

Vis: Clarity

Vis: Aesthetic Cues

The comprehensibility andinterpretability of the visualization

Vis: Accuracy

Vis: Usability

How well the visualization
precisely displays theunderlying data

How easy the visualization
is to use and interact with.

Characteristics of the visualization that can
trigger an emotional response. E.g., Use of color

Vis: Benevolence & Ethics

Data: Accuracy

Data: Currency

The extent to which thedata is precise

The capability of the data to
reﬂect up-to-date information

Data: Coverage
Comprehensiveness andcompleteness of the data,

Affective TrustThe extent to which the visualization is anappropriate presentation of the data, with nointention to mislead the trustor.

Trust Antecedents

Trust in Data

Individual Characteristics

Cognitive Trust, Hanspeter Pfister

Data: Clarity
The comprehensibility andinterpretability of the data

Data: Affective Cues
Characteristics of the data that can inﬂuenceemotions. E.g., Source credibility

Behavioral Outcome

Hamza Elhamdadi

Data: Benevolence & Ethics
The responsible practices andsoundness of data collection and use.

Fig. 1: An integrated framework, which outlines the development of trust in visualizations. The framework defines the different
trust antecedents of the two basic components of trust (cognitive and affective trust). Both cognitive and affective trust can relate to
the visualization and the underlying data. Individual characteristics can play a role in shaping one’s level of trust in visualizations,and behavioral outcomes can emerge as a results of trust judgements.
Abstract—Trust is an essential aspect of data visualization, as it plays a crucial role in the interpretation and decision-making processesof users. While research in social sciences outlines the multi-dimensional factors that can play a role in trust formation, most datavisualization trust researchers employ a single-item scale to measure trust. We address this gap by proposing a comprehensive,
multidimensional conceptualization and operationalization of trust in visualization. We do this by applying general theories of trustfrom social sciences, as well as synthesizing and extending earlier work and factors identified by studies in the visualization field. We
apply a two-dimensional approach to trust in visualization, to distinguish between cognitive and affective elements, as well as between
visualization and data-specific trust antecedents. We use our framework to design and run a large crowd-sourced study to quantify therole of visual complexity in establishing trust in science visualizations. Our study provides empirical evidence for several aspects of our
proposed theoretical framework, most notably the impact of cognition, affective responses, and individual differences when establishingtrust in visualizations.
Index Terms—Trust, visualization, science, framework

1

I NTRODUCTION

As the field of data visualization matures and is more widely adopted
in public settings, understanding the role of trust in visualizations• Hamza Elhamdadi and Cindy Xiong Bearfield are with UMass Amherst.
E-mails: helhamdadi@umass.edu, cindy.xiong@cs.umass.edu• Adam Stefkovics, Johanna Beyer and Hanspeter Pfister are with Harvard
University E-mails: adamstefkovics@fas.harvard.edu,
jbeyer@g.harvard.edu, hpfister@g.harvard.edu• Eric Moerth is with Harvard Medical School E-mail:ericmoerth@g.harvard.edu• Carolina Nobre is with the University of Toronto E-mail:cnobre@cs.toronto.edu
Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication
xx xxx. 201x; date of current version xx xxx. 201x. For information on
obtaining reprints of this article, please send e-mail to: reprints@ieee.org.
Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx

becomes increasingly important, particularly when the data presented
is urgent (e.g., climate change, Covid-19, etc.) For example, recent
research exploring trust in Covid forecast visualizations showed how
differences in visual encodings can significantly affect viewers’ trust in
the information and willingness to incorporate the information in theirdecision-making process [41].
The concept of trust and the factors that can play a role in its formation has long been explored in the field of social sciences. Fromthat research, two parallel paradigms of trust emerge: trust definedby Mayer et al. as “the willingness of a party to be vulnerable to theactions of another party" [36] and trust defined by McAllister et al.
as composed from cognition-based “rational" and affect-based “emotional" factors [38]. More specifically, cognitive trust is defined as trust
based on the knowledge and evidence of someone’s ability and achievements, while affective trust is defined as trust based on the emotionalbond with someone [38]. Research in this field also defines the factorsthat precede trust formation as antecedents of trust. Examples of trust
antecedents include benevolence and behavioral integrity [36]. In thiswork, we build on the two-dimensional definition of trust as definedby McAllister and outline specific antecedents of both cognitive andaffective trust in the context of visualizations (Figure 1).
Researching trust in visualization has a fundamental difference fromits application in the social sciences in that it does not concern itself
with human-to-human relationships. Translating trust research fromthe social sciences to data visualizations can prove difficult because
the trustee is a non-human subject [37]. Nevertheless, several authors
have argued that people may relate to non-human subjects socially[26, 40] and are capable of trusting digital technology, information, or
media [25]. Furthermore, as shown by Kelton et al. [25], trust in media
can be interpreted on the same level as interpersonal trust (i.e., via atrustor-trustee relationship).
Recently, a handful of studies have attempted to measure trust in thecontext of data visualization [28, 31, 41, 50, 53]. Yet, these approaches
typically focused on one or a few specific elements of trust, failing tocapture the multi-faceted nature of trust in visualizations [37]. Manyof these approaches use a single-item scale that asks the user somevariation of “how much do you trust this visualization?" [18,28,41,61].
This question is also asked without defining trust, instead requiring theparticipant to define trust for themselves before answering the question, which introduces error [17]. To improve how we measure andstudy trust in visualization, we propose a comprehensive, multidimensional conceptualization and operationalization of trust in visualizationthat builds on the trust literature and synthesizes existing work in thevisualization field.
We operationalize this framework in a series of explicit measurements that cover cognitive and affective trust, as well as individual
characteristics such as visual literacy, need for cognition, and demographic factors. We propose this framework to measure how a reader
forms trust in a visualization. This can be different from the trust formation of a creator, who has the ability to manipulate visualizations andtheir underlying data. We run a large crowd-sourced study that serves
as a case study for the framework and investigates the effect of visualcomplexity on establishing trust in science visualizations using theexamples of Covid and crop disease visualizations. Our study provides
empirical evidence for several aspects of our proposed trust framework.
Among our most notable findings is the strong role that visualization
topic has on trust mechanisms, and how complex visualizations candecrease trust and trigger more affective-based than cognitive-based
trust judgments.
In this work, we make two main contributions:• a multidimensional conceptualization and operationalization oftrust in visualization.
• an empirical study using this framework that investigates the roleof visual complexity on cognitive and affective trust in sciencevisualizations.
2 R ELATED W ORK
Methods for measuring trust include uni-dimensional and multidimensional Likert scales, trust games, belief updating metrics, andprocessing fluency. Much research regarding these methods stems fromthe social sciences. Though, computer scientists and data visualization
researchers have recently started implementing some of these methods
to measure trust in our fields.
2.1 Trust Measurement in Computer Science
Prior approaches to measuring trust in computer science research haveused several different Likert scales that vary widely on the number andcontent of items included [1,24,28,29,34,41,53] as well as the definitionof trust they use [1]. Some approaches simply use uni-dimensional
scales that vary on the number of discrete values the user can choose(e.g., 0 to 100 [29, 41], 1 to 5 [13, 28]). Some approaches [13, 29]
additionally ask participants to explain their reasoning via text response.
Other computer science researchers have used multi-dimensional Likert
scales that vary along the number of dimensions and the number of
discrete values allowed for the response [20, 24, 34, 59]. A widelyused 12-item scale, developed by Jian et al. [24], captures user trust in
specific automated systems, as opposed to previous scales from social

sciences that aimed to measure general trust in automated systems. The
user is asked to rate each of the items, which are ordered from negative
to positive valence (e.g., The system is deceptive, ..., The system isdependable), from 1 to 7. However, the ordering of the items in thisscale was shown to significantly bias users towards positive ratings
when compared to both a randomized sequence of the twelve items anda flipped version where the items were ranked from positive to negative
valence [20].
A more complex method for measuring trust seen in computer science research involves measuring the likelihood and frequency of auser to update their initial beliefs based on the recommendation and accuracy of a machine learning model [54–56]. The use of this approach
to measure trust is validated by findings including users of machinelearning systems can determine the accuracy over successive uses ofa system [55], users’ trust in a system is more heavily impacted by
its failures than successes [56], and users are impacted by a system’s
response as well as its stated accuracy [54].
Few computer science trust researchers have employed the use oftrust games. As far as we are aware, from the non-visualization computer science trust research corpus, only Zheng et al. [60] have measured the impact of technology-mediated communication on trust using
an investment game. The goal of this research was to determine how
meeting a person via an online social chat room affected trust whencompared with meeting face-to-face, or not meeting at all. This study,
therefore, shares more in common with existing trust game research insocial science than with the hypothetical trust game approach proposed
by Elhamdadi et al. [17] in which one of the participants would be avisualization or an automated system.
2.2

Trust Measurement in Data Visualization

Measuring trust in data visualization research has been historicallyinconsistent and highly variable [27, 29, 31, 41, 53, 57, 61]. Many datavisualization trust researchers use a single-item scale to measure trust[13,28,29,41,62]. These scales ask users to rate their trust (or agreementwith statements that measure trust) on a scale with varying discretevalue ranges (e.g. “how trustworthy you think the graph is as a whole"on a scale from 0 to 100 [41], “how likely [you] thought it was thatthe data was manipulated" on a scale from 1 to 5 [28], “rate thetrust level of predictions on which decisions were made" on a scalefrom 1 to 9 [61], etc.), However, single-item scales are insufficient
to capture trust [19] due to the multi-faceted nature of trust [25].
Furthermore, these uni-dimensional scales do not use the same numberof discrete values, nor do they provide an explicit definition of trust,which can cast doubt on the validity of these approaches. Conversely,
our framework measures trust with multi-item scales that capture trustand its antecedents, as described in established social science research[38]. We also consider an existing multi-item trust measurement scale[53] from data visualization research that uses three components oftransparency (i.e., accuracy, clarity, and disclosure [45]) as a proxy tomeasure trust. Participants were asked to rate each of the componentsof trust on a scale from 1 to 5; the results of this work validated the
known positive correlation between transparency and interpersonal
trust [23] from existing social sciences research in the context of datavisualizations [53].
Recent trust research in visualization also indicates that processing
fluency, the ease with which one interprets a stimulus, may function as asurrogate measure of trust in data visualizations [17, 18], which follows
from prior social and computer science research that demonstrates alink between fluency and trust [46, 48, 52]. High-intensity affective
responses have also been linked to positive experiences of ease viewing
a stimulus (fluency) [32]. Hence, the affective trust items from ourframework indirectly capture participants’ experience of fluency and,by proxy, trust.
While Elhamdadi et al. [17] argue that trust games, particularly
investment games, can be used to measure trust in visualizations, visualizations present a difficult confound to overcome when using investment games. In a standard investment game [4], the reasoning for
one participant’s decision to invest a certain amount is not known tothe other participant. However, when using a visualization, the underly-ing data becomes a confound. The visualization investment game, as
discussed by Elhamdadi et al. [18], does not account for the assessmentof the underlying data in their decision to invest, thus confusing thecorrelation between investment and trust in the visualization. Future
research into trust games in visualizations may ameliorate this issue.
3

A N I NTEGRATED F RAMEWORK FOR T RUST IN V ISUALIZA TION

We propose a framework to comprehensively interpret and measuretrust in visualizations by applying general theories of trust [36, 38], andby synthesizing and extending earlier work and factors identified byprevious studies in the visualization field [25, 31, 37, 50, 51, 53].
Uni- and Two-dimensional Theories of Trust Some approaches totrust measurement suggest that trust is uni-dimensional, many scholars argue that trust is multi-dimensional. The most prominent unidimensional understanding of trust was proposed by Mayer et al. [36].
According to this approach, (1) trust is uni-dimensional around the key
concept of “being vulnerable” and (2) trust should be distinguishedfrom its antecedents. Mayer et al. [36] identified ability, behavioral
integrity, benevolence, and values congruence as the four antecedentsof trust.
McAllister’s two-dimensional conceptualization [38] has been by
far the most widespread theoretical basis for understanding trust. He
defined interpersonal trust as “the extent to which a person is confident
in and willing to act on the basis of, the words, actions, and decisions ofanother” [38, pg. 25]. Cognitive trust is grounded in judgments about
the trustee’s perceived reliability and dependability. These quality
judgments may serve as a rational basis for trust. Affect-based trust, on
the other hand, consists of emotional bonds between individuals basedon expectations of interpersonal care and concern. As Chua et al. [10]put it, cognitive and affective trust are “trust from the head” and “trustfrom the heart” respectively.
The Overlap in Cognition and Affect It is worth noting that thedistinction between cognitive and affective trust antecedents is made at
the experiential level [15]. That is, a person assesses trust in a stimulus
differently depending on whether they react affectively or cognitively to
aspects of the stimulus. Hence, in the context of visualizations, a visualcue is not objectively cognitive or affective but can be experienced byviewers in different ways. For example, someone can experience theclarity of a visualization as an aesthetic quality, leading to affective
trust. On the other hand, they may experience clarity as a facilitator
for interpreting the visualization, leading to cognitive trust. In thispaper, we consider cognitive and affective trust antecedents as thosethat capture the viewer’s cognitive and affective experiences with thevisualization respectively.
3.1

Our Trust Framework

Our framework relies on theoretical work by Tomlinson et al. [49],
who argues that the two paradigms (McAllister and Mayer) are not that
distinct and aims to build bridges between them. First, we can consider
the similarity of McAllister’s assessment that trust “enables people
to take risks” and the vulnerability concept from Mayer et al. [36].
Second, like McAllister et al., Mayer et al. [36] admit that trust ismultifaceted and its antecedents are multi-dimensional. Tomlinson etal. [49] theorize that Mayer’s antecedents can be related to McAllister’s
cognitive-affective framework; for instance, ability and behavioral
integrity predict cognition-based trust, and benevolence and values
congruence predict affect-based trust. Their findings indicate thatcognitive and affective trust may have different antecedents.
Cognitive vs Affective As depicted in Figure 1, we apply the twodimensional approach to trust in visualizations and characterize antecedents as cognitive or affective. Trust in visualization may bestrongly determined by cognitive factors such as cognitive understanding; nevertheless, when cognitive trust is high, individuals may still
distrust a visualization because of their negative emotions toward thevisualization or its topic (or vice versa). This phenomenon can be
attributed to the fact that feelings often arise with little or no cognition [21]. For example, people may have negative feelings towards
the data that is being visualized because it covers emotionally-negative

material (e.g., Covid-19 deaths) or towards the visualization itself because some elements of the visualization (e.g., aesthetic elements) may
induce negative feelings. Hence, these negative emotions may influence
trust and decision-making regardless of cognition-based assessments.
Data vs Visualization We identified trust antecedents separately forthe visualization and the underlying data as suggested by Mayr et
al. [37]. Our framework includes 7 cognition-based and 4 affect-based
antecedents of trust (see Fig 1)
Individual Characteristics Additionally, we argue that trust in visualizations may be subject to certain individual characteristics of thetrustor. On the one hand, individual characteristics can alter the pathsof trust development. For example, individuals with a higher need forcognition may weigh the cognitive aspects of a visualization higher,
whereas others may be more likely to rely on affective responses to thevisualization [8]. On the other hand, the likelihood to trust may differ
between individuals regardless of the visualization’s characteristics.
For instance, the Big Five personality traits have been linked to one’s
general propensity to trust [19]. Trust is positively associated withagreeableness, extraversion, and negative neuroticism, although these
relationships are not deterministic and the relevance of personality traits
in determining trust in visualizations remains unclear. Other known
associations between human-to-human trust, cognitive skills [22, 47],
the level of education [7, 22], and political attitudes [9] have remainedlargely unexplored in data visualization research.
Behavior Lastly, our model addresses the consequences of trust invisualizations. Trust is considered a key criterion for engagement in
information-related behavior [25]. When trust is established, the trustor
is willing to "take risks" [36], i.e., integrate the information provided
by the visualization into their actions. In turn, low levels of trust, or
mistrust, likely result in ignorance.
3.2 Antecedents of Cognitive Trust
Cognition-based trust may originate from different rational evaluations.
These assessments explain perception about the ability of the data or
visualization to successfully describe their subject matter. We collected
cognition-based antecedents by synthesizing previous studies in the visualization field [25,31,37,50,51,53,58]. For each category, we provide
data-driven examples from our survey that asked respondents an open
question about why they did or did not trust a specific visualization.
Data Antecedents
Accuracy Accuracy refers to the extent to which the data is precise
or free from error [25, 43]. Such errors may originate, for example,
from measurement error “The data is not entirely accurate, there werepeople who did not report having Covid, which made it difficult to keep
an accurate count." or lack thereof "The data falls right where I expect
it to be." Nevertheless, perceptions about the accuracy of the data canbe a function of affective responses to the data source. Thus, accuracy
may have both cognitive and affective antecedents. "One of the frames
stated that the data came from the CDC. They are probably as close toaccurate that we are going to get."Currency Currency refers to the capability of the data to reflect upto-date information. In some cases (e.g., rapidly-changing subjects or
topics), this aspect can be crucial in determining the personal relevanceof the information. “It is relevant, because it provides up-to-date
Covid-data."
Coverage Coverage refers to the comprehensiveness and completeness of the data. Individuals may trust or distrust the visualization
to the extent that it captures important aspects of the subject: “It has
numbers that span over a period of time which shows a pretty big
picture."Clarity Clarity refers to the comprehensibility and interpretabilityof the data [50, 51]. The extent to which individuals understand themeaning of the data can influence trust regardless of how the data isactually presented. “The data makes sense to me."Visualization Antecedents
Accuracy The accuracy of the visualization refers to how well
the visualization transparently and precisely displays all relevant elements of the underlying data. “The data seems to be presented in acomplete manner, not leaving out any facts that could obscure the point
presented." Accuracy is also related to the extent to which the visualization captures all possible states and alternatives considering theunderlying data [53]. The accuracy dimension is particularly relevantin relation to uncertainty communication [44].
Clarity Clarity of the visualization refers to the comprehensibility
and interpretability of the visualization [53]. Clear visualizations areeasy to understand and do not contain obscure elements. “Everythingis laid out in a way that is easy to see and understand."Usability Usability refers to how easy the visualization is to use
and interact with [2, 11, 37]. For instance, Beauxis-Aussalet et al. [3]argued that interactive visualizations are an “utmost important role infostering trust in AI”. "Considering the actual data shows up over themouse over, it provides more clarity".
3.3

Antecedents of Affective TrustPrevious research does not provide clear insights into the developmentof affective trust in human-non-human relationships. Affective trust is
constituted of general emotional attachments between the two parties
and the extent to which a trustee is believed to be good (and not causeharm) to the trustor [38]. To apply the first part of the definition tovisualizations, one can interpret affective trust as the immediate emotional responses to the data or visualization, as well as the affective
perceptions of the data or visualization [58]. Affective cues may include the objectivity or credibility of the data and/or the aesthetics ofthe visualization. Some research suggests that aesthetic judgments are
driven mainly by affective responses [6]. Notably, emotions toward
the data may also originate from the emotions toward the source ofthe data [25, 37]. The second part of the definition resonates well with
Mayer et al.’s notion of benevolence [36]. In the case of visualizations,benevolence can be understood as the ethical use of the data and perceptions about the benevolent handling of the data and its visualization
as free of any intention to cause harm to the trustor (e.g. by being
intentionally misleading). Accordingly, we define affective cues andbenevolence as separate antecedents of affective trust.
Data Antecedents
Affective Cues of the Data Affective cues are specific characteristics of the data that can influence emotions [58]. For instance, these
may include the perceived objectivity or believability of the data or theextent to which the data seems unbiased and credible. Negative affective trust in the data may also originate from a negative characteristicof the data source. “The visualization was presented with informationfrom the CDC. I trust the CDC." "I just do not believe vaccinated vs.
unvaccinated has any difference in cases."
Benevolence and Ethics regarding the Data Benevolence andethics regarding the data refer to the responsible practices and soundness of the data collection, the responsible use of data, and the appropriate citation of the data sources. The lack of information regardingthese details may evoke a negative affect toward the data. “I have noidea how the data was gathered, if this is a ’dishonest’ research study,
etc. Where did the visualization come from? Who provided the data?
How was it collected?"
Visualization Antecedents
Aesthetic Cues of the Visualization Aesthetic cues of the visualization are specific characteristics of the visualization that can trigger
an emotional response [58]. These may involve specific use of colors,
sizes, images, etc. "It looks scientific and seems professional." “ I don’t
really believe it, but I like that it’s colorful."
Benevolence and Ethics regarding the Visualization Benevolence and ethics regarding the visualization refer to the responsible
practices and accuracy of the visualization, the extent to which thevisualization provides an appropriate presentation of the data, free of
any intention to mislead the trustor. “I believe that charts like thismisrepresent the truth and present things in an oversimplified way inorder to mislead the viewer."

4

E MPIRICAL C ASE S TUDY

We apply the trust framework in a case study investigating the factorsthat play a role in establishing trust in science-based data visualizations.
More specifically, we are interested in determining the role of visualcomplexity in promoting trust. To capture the multi-dimensional aspects of trust in the study, we follow the two dichotomies described in
our framework: the first between cognitive and affective trust elements
and the second between trust in data and trust in visualization. A detailed discussion on how we measure the different components of trustin the study can be found in Sections 4.4, 4.8 and 4.9.
4.1

Study Design

We designed the study with three independent variables: visual complexity (simple, moderate, and complex), chart type (bar chart and line
chart), and data topic (Covid Vaccines and Crop Diseases in Croatia).
Visual Complexity Our main research question for this study was
whether varying levels of visual complexity affect trust in science-based
visualizations. We designed three levels of complexity for each of thetwo chart types used in this study: simple, moderate, and complex.
Simple visualizations show the least amount of data in the mostaccessible format. For bar charts, the simple visualization showed two
separate bar charts, each representing one category of data (Figure2a). The simple line chart contained a single chart with two lines, one
for each category (Figure 2d). Moderate visualizations provide added
visual complexity, either through stacked bars - for bar charts - (Fig 2b)
or by adding a confidence interval around the mean line - for line charts(Fig 2e). The complex condition shows the most information with 6
data categories instead of 2 for both bars (Fig 2c) and lines (Fig 2f).
Chart Types To assess whether our findings on visual complexity
generalize to more than one chart type, we chose two of the mostcommon visualizations used when communicating data to the generalpublic: bar charts and line charts. We designed both visualizations toreflect standard Covid charts that have been used in the media over the
past couple of years. All charts were equipped with tooltips, whichshowed detailed information on demand. Depending on the level ofcomplexity, interaction was also provided via a brush filter.
Data Topic To control for prior opinions on Covid data, we createda second condition where participants were shown visualizations of
Crop Diseases in Croatia. However, using two different datasets togenerate the visualizations would introduce an additional confound. To
avoid this, we used the same dataset for both data topics and simply
changed the labels, title, and legend accordingly. Participants who were
given the Croatian crops visualization saw a disclaimer at the end of
the study that clarified that the data was actually for Covid cases.
4.2

Pilots and Experiment Planning

We conducted three pilots to evaluate tasks, interactive visualization
features, measures, and our procedure. Initial pilots revealed thatasking participants for their "initial impressions" of a visualization was
not effective at capturing affective trust. Instead, most participants
cited clearly cognitive aspects, such as trends in the data, or theirtakeaways. To address this, we ran a separate pre-study to establish
the type of strictly affective responses participants could have to thedifferent visualizations. We recruited 200 participants on Prolific andshowed each a randomly assigned visualization from the final study for
15 seconds. This approach was based on the work done by Betella etal [5], who measured human emotions toward visual stimuli by showing
viewers an image for 15 seconds and then prompting them for their
responses. After viewing the visualization, our participants were asked
to describe the visualizations in 5 to 10 adjectives. The most common
adjectives informed the dimensions of affective trust in the final study.
We estimated the number of participants required to uncover effects
based on a final pilot run on Prolific with 60 participants. We used a
power analysis between the different conditions to estimate the variancein our quantitative measures, which we combined with our observedmeans to estimate the number of trials required.

(a) Bar chart of simple complexity.

(b) Bar chart of moderate complexity.

(c) Bar chart of high complexity.

(d) Line chart of simple complexity.

(e) Line chart of moderate complexity.

(f) Line chart of high complexity.

Fig. 2: Study stimulus for bar charts showing Covid visualizations, across all three complexities. Figures show interactive features available toparticipants, namely tooltips for all three levels of visual complexity, and a time filter for the complex condition
4.3

Participants and Procedure

We recruited 600 participants on Prolific, a crowdsourcing platform
with a research focus. Twenty-four participants timed out and were
excluded from the study. Of the remaining 576 participants, an additional 33 were excluded for either failing the attention checks or due
to a glitch in the survey that did not record their response to one of
the questions. This left us with 543 valid responses. Our study used
a between-subjects design in which each participant was randomly
assigned to one of the 12 combinations of independent variables (threevisual complexities, two chart types, and two data topics). We deployed
the study on the Qualtrics survey platform, which ensures random distribution amongst conditions. Using the javascript integration featurein Qualtrics allowed us to render interactive visualizations directly
within the survey (as opposed to static visualizations) as well as collectinteraction data from participants such as when they hovered on data elements or used the interactive filter. It also allowed us to log when theybrowsed away from the study, and for how long. This data was criticalfor quality control and to obtain more accurate time-on-task metrics foreach participant. The participants were recruited using Prolific’s [42]representative sample feature, which ensures that participants are representative of the United States across multiple demographics, including
age, sex, and ethnicity.
Based on completion times of pilot experiments, each participant
was paid $4 USD, for an estimated duration of 15 minutes, resulting inan hourly rate of about $16 USD. The median time of completion after
the survey was completed was 13 minutes. All participants viewed andagreed to an IRB-approved consent form. To be eligible for the study,
participants had to use a laptop or desktop device with a resolution ofat least 1400x850 pixels available screen space in the browser.
The study consisted of seven sections: Affective Trust,Visualization
Tour, VLAT, Explore the Visualization, Trust in Data, Trust in
Visualization, Individual Characteristics. The full study can beviewed at https://rotman.az1.qualtrics.com/jfe/form/SV_
03B33TJH9p1dtTU.
4.4 Affective Trust
The first section of the study following the consent form was geared
towards collecting quantitative measures of participants’ affective reac-

tions to the visualization. More specifically, we focused on quantifying
affective cues towards the visualization. Based on the results of the
pre-study on affective reactions to the visualization, we asked users tomove three sliders, each towards the adjective that best described thevisualization (Table 1). We capture other affective visualization trust
components, such as benevolence in an open text question at the end of
the study.
Type

Dimension

Study Item

Affective

Aesthetic Cues (Scientific)

Unscientific ———- ScientificAffective

Aesthetic Cues (Clarity)

Unclear ————— Clear

Affective

Aesthetic Cues (Pretty)

Ugly ——————- Pretty

Table 1: Affective Cues: List of 3 sliders given to participants to capture
affective trust in the visualization through different aesthetic cues.
4.5

Visualization TourAn interactive tour for each condition guided the user through the main
components of the visualization and prompted users to interact withthe visualization, including hovering or filtering. Because the brush
filter in the complex visualization may be unfamiliar to novice users,
we required participants to interact with the brush according to the
instructions before proceeding with the tour. We also logged detailed
provenance data that captured how participants interacted with thetour steps. This allowed us to perform post-hoc analysis and answer
questions such as: Did they progress linearly through the tour? Did
they go back to a previous step, or struggle to perform the operations
with the brush filter required to progress in the study?
4.6

VLAT

To capture participants’ ability to understand the visualizations, we
adapted the visual literacy assessment test (VLAT) [33] by creating equivalent questions for the visualizations used in the study. Our
intention was to create questions that required the participants to engage with the complexity of that visualization. For example, for the
stacked bar charts shown in the moderate complexity condition (see Fig.
2b), we asked participants questions that required understanding thedifferent baselines for the two data categories. In the complex visualization, we asked a question that required participants to compare trends
among different categories. We provided multiple-choice solutions
for these questions and normalized the final scores by using the formula VLAT_Score=C-(W/N) described in [33], where C is the numberof questions the participant answered correctly, W is the number ofquestions the participant answered incorrectly, and N is the number ofsolutions in a single question. The incorrect solutions we created wereintended to be intuitive or nearly correct.

is always shown last, as we similarly hypothesize that general feelingsof trust in the data are a combination of the beliefs expressed by theprevious five items.

4.7

Explore the Visualization

Participant interaction data logged during one of the pilot studies revealed that participants were not spending any time engaging with oranalyzing the visualization before reporting on their trust levels. Since
cognitive trust antecedents rely on processing the visualization at a
non-superficial level, we added this “explore section", which asked
participants to look at/interact with the visualization for 15 seconds
before allowing them to proceed to the sections on trust assessments.
During this time, we continued to collect interaction provenance data
including the data of visual elements that they hovered over and for
how long, the start and end points of the brush filters they created, andwhether they browse away from the study. Each of these provenancedata was stored with a timestamp for when they occurred.
4.8 Trust in Visualization
In the trust in visualization section participants were asked to assume
they trust the data, and rate their trust in the visualization. Trust in thevisualization was captured via a 6-item set of Likert scales (Table 2).
The first three items capture cognitive and affective trust antecedents
for trust in the visualization. We describe the fourth and fifth items in
Section 4.11.
Type

Dimension

Study Item

Cognitive Accuracy

The visualization transparently includes all importantelements of the data

Cognitive Clarity

I find it easy to understand this visualization

Affective

Aesthetic Cues

I like this visualization

Behavior

Actionable

I would likely use this visualization and its information
in my daily life

Behavior

Shareable

I would likely share this visualization with my family,
friends or on social media

Trust

Visualization

I trust this visualization

Table 2: Trust in Visualization: List of 6 Likert scale items given toparticipants to capture trust in visualization. Items 1-3 capture cognitive
and affective antecedents, and Items 4-5 capture behavioral outcomes.
Item 6 captures the high-level trust in the visualization.
The first five items of this section were displayed in random orderto mitigate the effect of the order on response, and the final item, “Itrust this visualization," is shown last regardless of the participant, as
we hypothesize that general feelings of trust in the visualization are a
combination of the beliefs expressed by the previous five items.
At the end of the trust in visualization section, participants wereasked to provide a text explanation of their trust rating. This qualitative
data allowed us to capture both cognitive and affective trust dimensions
beyond those in the 6-item Likert scales. This includes elements suchas usability, benevolence, ethics, and affective cues beyond the ones
captured in Section 4.4.
Participants saw this and the next section (Trust in Data) in random
order to mitigate any priming that may come from the ordering.
4.9

Trust in Data

In this section, the participants were asked to disregard the visualization
and rate their trust in the underlying data. As with the trust in visualization section, trust in data was captured via a 6-item set of Likert scales(Table 3). We did not explicitly elicit ratings of data currency, since thedata time frame was fixed from Dec 2020 to Dec 2021 for this study.
The first five items were again displayed in random order to mitigate
the effect of the order on response. The final item, “I trust this data,"

Type

Dimension

Study Item

Cognitive Accuracy

The data is accurate

Cognitive Coverage

The data is complete and does not leave out important
information

Cognitive Clarity

I understand the meaning of this data well

Affective

Benevolence

The data is unbiased and trustworthy

Affective

Affective Cues

The data source was clearly displayed

Trust

Data

I trust this data

Table 3: Trust in Data: List of 6 Likert scale items given to participants
to capture trust in the data. Items 1-5 capture cognitive and affective
antecedents to trust in data. Item 6 captures the high-level trust in thedata.
4.10 Individual Characteristics
As discussed in our trust framework (Section 3), individual differences
can greatly impact trust formation. In this study, we operationalize
these differences by capturing a broad set of participant traits. When
analyzing the study results, we include individual characteristics as
co-variates in our regression models. This allows us to better isolate themagnitude of trust variations that are due to changes in our controlled
variables, and not due to individual characteristics of the participants.
Interpersonal Trust A person’s generalized baseline trust in people can be used to compare their ratings of trust in the specific context
of data and visualizations. Hence, we use the ITS interpersonal trustscale [19] and asked participants to answer the question “Generally
speaking, would you say that most people can be trusted or that you
can’t be too careful in dealing with people?" on a scale from 1=“Mostpeople cannot be trusted" to 7=“Most people can be trusted."
Trust in Institutions Beliefs regarding Covid-19 are often impacted by an individual’s trust in related institutions (e.g., government,science, the new media, etc). Hence, we asked the participants to rate
their trust in the following institutions on a scale from 0 to 10: (1)political parties, (2) the government, (3) the police, (4) the legal system,(5) the new media, (6) business and industry, (7) scientists/science, and(8) doctors.
Need for Cognition Participant responses to the cognitive trust
items of our framework may be influenced by their motivation to engage
in cognitive activity in general (e.g., someone who is less motivated toengage in cognitive activity may rate the cognitive trust items lower).
Thus, we employ the 6-item version of the Need for Cognition test, as
proposed by Coelho et. al [35]. For this participants rated how strongly
they agreed/disagreed with statements such as “I would prefer complex
to simple problems."
Politics Because one of the datasets we use for the visualizationin this study is related to Covid-19, a heavily politicized topic, wecollect information regarding the political affiliation (7-point scalefrom “extremely liberal" to “extremely conservative") and Covid-19
media consumption of the participants, i.e., how often they sought
out information on Covid during the pandemic). For the participantsthat did not see the Covid-19 visualization, we did not ask about their
Covid-19 media habits.
Demographics To account for any effect of participant demographics on their trust behavior, we collect information regarding participants’ age, gender, state residency, education level, parents’ educationlevel, spoken language, ethnicity, income, and religious affiliation.
4.11 Behavioral Outcomes
We captured participants’ self-reported behavioral outcomes (Figure 1)
by asking them to rate the items “I would likely use this visualizationand its information in my daily life" and “I would likely share thisvisualization with my family, friends or on social media" (Table 2).
While self-reported measures are not always reliable indicators ofactual behavioral change [12], capturing intent provided us with initialinsight into ways in which trust might impact behavior. In Section 5.1,
we analyze how trust in the visualization and underlying data correlatedwith participant responses to these items.
5

S TUDY R ESULTS

The data collected in this study includes both individual characteristics
of each participant – such as visual literacy, need for cognition, anddemographic information – as well as the perceived cognitive and affective trust of the visualizations. We analyzed this data by constructing
linear regression models between properties in our framework. The
independent variables in our models are chart complexity, data topic,
and chart type. We also include demographic information (e.g., gender,
education, ethnicity, etc), need for cognition, interpersonal trust, andtrust in science as covariates to account for individual differences. Inmodels where we are interested in the predictive power of the trust
antecedents on overall trust, we include the trust antecedents as independent variables. Below, we report on our major findings, whichreflect the linear models for all 12 conditions (3 visual complexities, 2chart types, and 2 data topics). The complete dataset collected in the
study, the analysis scripts, and the code used to generate the interactive
visualizations are open-source and available at "https://osf.io/
frbaj/?view_only=de44feae51374a71ac2a9b8798559d24".
5.1

Predictive Power of Trust Antecedents

As discussed in Sections 4.8 and 4.9, the study was designed to capture
high-level trust in vis and data as well as the different antecedents
and behavioral outcomes of trust outlined in the trust framework. We
measured these aspects via the scale items shown in Tables 1, 2, and 3.
Which Antecedents Predict Trust?

bipolar scale from “Unscientific" to “Scientific" were significantly correlated with overall trust in the visualization. Generally, participants
were likely to trust the visualization if they had found it scientific,
accurate, and easy to understand, as well as if they had a positive
affective experience.
Trust in Data We also analyzed the predictive power of cognitive
and affective trust antecedents on trust in the underlying data and foundthat cognitive accuracy, cognitive clarity, cognitive coverage and data
source were significantly correlated with overall trust in the data. Hence,
participants were more likely to trust the underlying data from avisualization if they found it easy to understand, and complete, andif the source of the data was clearly displayed.
Which Trust Measurements Predict Behavioral Outcomes?
Though capturing behavioral outcomes as a result of trust in visualization would be best achieved with a longitudinal study, our study
includes two items that measure projected behavioral outcomes throughself-reported measures: (1) How likely it is the participant would use
the visualization in their daily life, and (2) How likely it is the participant would share the visualization with friends and family.
We investigate which of the trust measurements, whether antecedents
or high-level trust, serve as predictors for these behavioral metrics. For
example, are participants more likely to use or share the visualization
if they find the visualization clear? Or if they find the data unbiased?
We found that participants who highly rated the cognitive accuracy
and aesthetic cues of the visualization were significantly more likely toboth use the visualization in their daily life and share it with family andfriends. Additionally, participants who rated the visualization as moreclear on the bipolar slider from “Unclear" to “Clear" (affective clarity)
were significantly more likely to use the visualization in their daily life.
5.2

To examine the predictive power of the antecedents on overall trust in
the visualizations and their underlying data, we constructed a linear
regression model predicting visualization trust with all vis antecedents,
and another model predicting data trust with all data antecedents. Demographic and individual characteristic measures were included toaccount for their covariances. The results of these models are shown in
Table 4.
Visualization Trust Antecedent

Predictive Power on Vis TrustType

Dimension

Est

SE

P

Cognitive

Accuracy

2.361e-01

3.758e-02

8.02e-10

Cognitive

Clarity

1.419e-01

4.806e-02

0.00333

Affective

Aesthetic Cues (Like)

2.292e-01

4.631e-02

1.07e-06

Affective

Aesthetic Cues (Scientific)

6.597e-03

2.573e-03

0.01067

Affective

Aesthetic Cues (Clarity)-3.865e-04

1.966e-03

0.84425

Affective

Aesthetic Cues (Pretty)-1.326e-03

2.047e-03

0.51744

Data Trust Antecedent

Predictive Power on Data TrustType

Dimension

Est

SE

P

Cognitive

Accuracy

5.598e-01

4.243e-02

2e-16

Cognitive

Coverage

1.124e-01

3.145e-02

0.000392

Cognitive

Clarity

1.875e-01

3.387e-02

5.33e-08

Affective

Benevolence

1.912e-02

3.519e-02

0.587117

Affective

Affective Cues (Source)

6.567e-02

3.099e-02

0.034662

Table 4: Results of linear regressions modeling the predictive powerof trust antecedents in predicting trust in the visualization and trust in
data. The columns refer to the following: Est is the estimated slopeof the linear regression, SE is the standard error, and P is the p-value.
Significant p-values are highlighted in red.
Trust in Visualization From our analysis of the antecedents, we findthat two cognitive trust antecedents and two affective trust antecedents
are significant predictors of trust in visualization. Cognitive accuracy
and cognitive clarity were significantly correlated with the overall trustin the visualization. Similarly, responses to the aesthetic cues and theCorrelation and VIF between Trust Items

We used the variance inflation factor to analyze the multicollinearity
between the eleven antecedents to trust, the two measures of overall
trust, interpersonal trust, need for cognition and trust in science. The
complete matrix of VIF scores between all variable pairs is includedin the supplementary material. Although the metrics are not fully
independent, all VIFs, except for one, are below 3.4, which indicates a
low level of colinearity [30]. Trust in data had a slightly higher VIF of
4.3, suggesting that participants’ trust in the data can overlap with trustin the visualization, which is expected given the fact the visualizationis meant to represent the underlying data. Trust in visualization has
a VIF of 3.16, which suggests that participants’ overall trust in thevisualization overlaps with both visualization and data trust antecedents,
as well as the three personality traits. This potentially suggests that
overall trust in the visualization is impacted by trust in data, and isrelatively well-captured by the eleven antecedents. Importantly, thepersonality traits (interpersonal trust, need for cognition, and trust inscience) have very low VIFs (close to 1.0), indicating a low overlap
with the other metrics considered.
5.3

Trust in Visualization and Trust in Data

Does Visual Complexity Influence Trust in Visualization?
As described in Section 4.1, we designed two types of visualizations(bar charts and line charts) in three levels of complexity: simple, moderate, or complex. In this section, we examine the effect that different
levels of visual complexity have on perceived trust in the visualization.
To test the generalizability of our effect, we investigate whether theeffect of chart complexity varies across the two chart types (bar, line)
and two data topics (Covid data, crop disease data).
We constructed a linear regression model predicting trust in thevisualization (as measured by the 6 items shown in Table 2) with chartcomplexity, data topic, chart type, and their interactions, including
the three-way interaction (Table 5). We also added demographic andcharacteristic predictors (participants’ age, gender, state of residence,education level, their parent’s education level, fluent language, ethnicity,
income, religion, level of trust in science, general need for cognition,tendency for interpersonal trust) as co-variates to the model to accountBar Charta. trust in the visualizationb. trust in the data

F Value

Pr(>F)F Value

Pr(>F)
0.003029

F(2,437)=4.0798

2.919e-08

F(2,437)=2.5088

F(1,437)=8.3574

1.621e-11

F(1,437)=10.9253

6.446e-11

chart typeF(1,437)=1.5884

0.116163

F(1,437)=1.1763

0.317789

complexity*chartType F(1,437)=1.7620

0.025638

F(1,437)=1.2231

0.261815

F(1,437)=1.9079

0.049143

F(1,437)=2.6696

0.014868

GenderF(3,437)=0.9113

0.596349

F(3,437)=1.6250

0.047127

State

F(46,437)=1.1513

0.023586

F(46,437)=1.2542

0.004228

Education

F(9,437)=1.2956

0.039962

F(9,437)=1.2516

0.104194

Trust in Science

F(1,437)=13.1355< 2.2e-16

F(1,437)=19.2062< 2.2e-16

Need for Cognition

F(1,437)=2.8732

0.002644

F(1,437)=3.2527

0.003876Interpersonal TrustF(1,437)=2.1778

0.022599

F(1,437)=2.3003

0.033816

5.21

5.47

5.47

5.53
simpleAge

5.16

moderate

chart complexity
data topic

Line Chart

5.46

complex

Predictor

2

4

6

Trust in Visualization

2

4

6

Table 5: Results of linear regression models predicting trust in thevisualization/trust in the data, with chart complexity, data topic, chart
type, and demographic/individual characteristics. The column names
refer to the following: F Value refers to the effect size, Pr(>F) refers to
the p-value. Significant p-values are highlighted in red.

Fig. 3: Trust in the visualization as a function of chart type and visualcomplexity. The visualization shows a box and whisker plot. The
central line shows the mean of the distribution with a corresponding
text label. The whiskers show the quartiles and extrema. The grey dots
visualize all of the data points.

for their impact. We report the effect size and p-value for all model
predictors in Table 5.
Our results show that “Trust in science" has the largest effect sizein the model for both trust in visualization and trust in data. This is
somewhat expected since both visualizations were designed to communicate science to the general public. Including the effect of trustin science in our models allowed us to determine the smaller, yet still
significant, roles of our independent variables (e.g., visual complexityand data topic). The VIF analysis described in Section 5.2 also shows
low collinearity between trust in science and the other variables in themodel, further indicating that while trust in science and the underlying institutions play a significant role in establishing trust, the visualcomponents studied can still modulate trust formation in a significant
way.
Our results show a main effect of visual complexity and data topic
on participants’ trust ratings for the visualization. as shown in Table
5 (column a). We ran independent linear models of each of the 6 trust
measurements (see complete tables in the supplementary materials)to
identify which trust antecedents drive the main effects observed in theoverall trust model. The results show that the complexity effect isdriven by participants’ response to the antecedents related to clarity,
both cognitive and affective. Specifically, participants found the simple
visualizations significantly easier to understand than the more complex
visualizations. As discussed in Section 5.1, clarity of the visualization
is a significant predictor of overall trust. That is, participants weremore likely to trust visualizations that they found clear and easy tounderstand. The effect of the data topic on trust is driven by participants’ responses to self-reported behavioral intentions. Specifically,
participants were more likely to share and use visualizations related toa topic that mattered to them (COVID) than one that did not (Croatian
Crops). We also observed a significant interaction between chartcomplexity and chart type. This interaction was driven by participants’ responses to the affective antecedent on how much they liked
each chart type. Specifically, participants liked line charts morethan bar charts and reported decreasing trust ratings as the line
charts became more complex (Figure 3). The complexity effect was
not observed for participants who saw the bar charts.

data is unbiased and trustworthy") and source (“The data source wasclearly displayed"). The effect of complexity on the source antecedent
was surprising since the data source was shown equally for all visualizations. We speculate that participants were less likely to spot thesource of the data in complex visualizations, therefore rating the source
transparency lower in those cases. In regards to the data benevolence,
participants gave simple visualizations the highest ratings, and as thevisualizations became more complex, ratings for data trustworthinessdecreased. (Figure 4). That is, participants were more likely totrust the data for simple visualizations than complex ones.
The effect of data topic on the trust in data was also driven by participants’ response to the affective metrics on benevolence. Specifically,
participants who saw the COVID data topic rated the data significantly
more trustworthy than those who saw the data on Croatian Crops. Analysis of participants’ qualitative responses to their trust ratings suggeststhat participants were more likely to trust the data for a topic they
were familiar with than an unfamiliar one.

5.43

moderate

5.72
simpleWe constructed a similar linear regression model predicting trust in thedata. We used the same demographic and characteristic predictors as
co-variate factors. The effect sizes and p values for all predictors arereported in Table 5 (column b). Similar to the trust in visualization
analysis, we found a main effect of chart complexity and data topic
on trust in the data . Independent linear models of each of the 6data trust measurements show that the complexity effect is driven byparticipants’ response to the affective metrics on benevolence (“Thecomplex

Does Visual Complexity Influence Trust in the Underlying Data?

5.33

2

4

6

Trust in Data

Fig. 4: Trust in the data as a function of visual complexity. The
visualization shows a box and whisker plot. The central line shows themean of the distribution with a corresponding text label. The whiskers
show the quartiles and extrema. The grey dots visualize all of the data
points.

6 D ISCUSSION OF C ASE S TUDY R ESULTS
The study data provided rich insight into how trust is modulated byvisual complexity. We emphasize three main takeaways from our quantitative analysis. The first two relate to how trust in the visualization
can be modulated by visual complexity and different chart types. Thethird takeaway relates to the role of complexity in establishing trustin the data. As described in Section 5, in our results for both trust invisualization and trust in data, we also observed a main effect of thedata topic (COVID-19 and the Croatian crop data). We designed thestudy to control for differences in the relevance of data topics. However, differences in participant trust between these two conditions could
also be a result of trust in the underlying data source or institution
creating the visualization. Future studies could explore the effect ofdifferent data sources/visualization creators compared to data topics
when forming trust judgments. Along these lines, existing work on trust
in COVID-19 data has shown that different data sources can impact
trust formation depending on the demographics and region [59].
The first takeaway is that participants were more likely to trust
visualizations that they found clear and easy to understand. This aspect of trust was captured in two of the trust in visualization antecedents,
one cognitive (“I find it easy to understand this visualization") and one
affective (Participants were asked to rank how clear they found thevisualization after seeing it for the first time and for only 15 seconds).
This finding is further supported by qualitative responses such as “Thevisualization is easy to understand and gather info from, and I trust thatthe information provided is accurate. " and “I think it’s really easy tofollow and it appears to be realistic information that could potentiallyhappen."The second takeaway is that participants liked line charts morethan bar charts and reported decreasing trust ratings as the line
charts became more complex. Measuring participants affective responses to the visualizations in addition to their overall trust ratingallowed us to detect that when participants did not like a visualization(bar charts) varying the complexity had little to no effect on their trust
ratings. Conversely, the line chart, which received higher affective
ratings, captured a clear decrease in trust as the visualization became
more complex. Qualitative responses that support this finding include“The different colored lines made it easy for me to figure out the data between the different age groups of vaccinated and unvaccinated people.
It didn’t see like there was anything deliberately misleading about itor anything lie that, so I trust it." and “It looks like a graph that has
reliable information on it, a line graph is used often."
The third takeaway is that increased complexity in a visualization
leads to decreased trust in the underlying data. Given the results of
our linear regression models for each of the data antecedents, we canassign this decreased trust as a result of decreased affective benevolence
for increasingly complex visualizations. In other words, for simpler
visualizations, participants find the data to be less biased and moretrustworthy, leading to more overall trust in the data. Our qualitative
responses suggest that simpler visualizations allowed the participants
to validate their expectations of the data and infer that the data has a
low bias: “Everything is displayed cleanly. There is no evident bias".
7

D ISCUSSION OF T RUST F RAMEWORKMuch existing trust research in data visualizations has measured trust
using a single item that does not capture the underlying nuances anddrivers of trust. By explicitly outlining two general dichotomies present
in the formation of trust in visualizations, we provide guidance formeasuring trust more comprehensively. In practice, this gives visualization researchers a theoretical framework on how to elicit trust in
the visualization and underlying data that covers several antecedents
of both cognitive and affective trust (e.g., clarity, accuracy, etc.). This
also serves to lighten the burden of defining trust on the side of theparticipants.
By applying our framework to the case study on trust in visualizations, we were able to determine the specific trust antecedent whichcaused participants to trust simple visualizations more than complex
ones. Our framework also gave us more granular insight into what
aspects of our visual design most impacted trust. For example, changes
in the chart type and complexity drove participants’ assessment ofvisual clarity and their overall trust rating. These insights would nothave been achievable with a single-item trust scale. Additionally, by
separating the antecedents into two categories for trust in visualizationand trust in data, we found an interesting distinction. For visualization

trust, the effect of complexity can be modulated by the chart type. Thisphenomenon did not occur for data trust where complex visualization
inhibited trust in the data regardless of the chart type. This distinction
would not have been found via traditional methods of measuring trustin data visualizations.
Aside from the trust antecedents outlined above, our framework also
considers the individual differences that influence trust, as well as the
behavioral outcomes that follow a trust judgment. By capturing andintegrating individual differences into our regression models, we wereable to account for them and uncover the driving role that visual design
had on establishing trust. The self-reported behavioral outcomes werealso captured and, in combination with the trust antecedents, revealed
the importance of the accuracy, aesthetics, and clarity of a visualization
when using or sharing it with others.
We ultimately find that the development of trust is a multifaceted
process wherein neither contextual nor individual factors independently
determine the outcome. Rather, these factors jointly contribute to thedevelopment of trust. For example, trust in the organization responsible
for the data or the visualization can significantly influence an individual’s level of trust in the visualization, particularly when it comes tocontroversial and salient topics. Nevertheless, the impact of this trust
can still be mitigated by the overall quality of the visualization. Moreover, the relationship between the different domains of trust (trust in
the visualization, trust in the data, trust in science, interpersonal data)is shaped by the visual characteristics and topic of the visualization.
8

C ONCLUSIONS AND F UTURE W ORK

Computer science trust research is still young, and most prior approaches to measuring trust have not been comprehensive. In thiswork, we provide a multidimensional conceptualization and operationalization of trust in visualization in a framework that builds on thetrust literature and synthesizes existing work in the visualization field.
The framework clearly outlines the antecedents to both cognitive andaffective trust in visualizations and guides future research on measuring
trust more comprehensively.
We apply this framework to a large crowdsourced study that investigates the role of visual complexity on trust. Our findings provide initialevidence that elements such as visual complexity, viewers’ vested interest in the topic, and their assessment of benevolence strongly influence
their trust judgment.
In future work, we intend to refine the case study scale items withthe knowledge we gained. Some of these refinements include adding
positive and negative items for each antecedent (e.g., we can measurecognitive clarity via both “I find it easy to understand this visualization"and “this visualization was confusing") and including more behavioral
outcome items that target broader behavioral patterns (e.g., “I am more
likely to read similar visualizations in the future") as well as items
specific to the dataset (e.g., “I am more likely to get vaccinated").
We further intend to examine trust development in visualizations
that intentionally elicit negative affective responses. Trust is morechallenging to establish when dealing with trustees in a negatively
valenced affective state [16], as areas of the brain linked to negative
emotions (e.g., fear of loss) are associated with a lack of trust or distrust [14]. However, it is possible to cultivate trust in negative contexts.
For instance, anxiety was found to decrease trust, whilst high-certainty
emotions such as anger and guilt have no clear effect on trust [39]. Identifying the contextual factors that foster trust in negative visualizations
presents a compelling direction for future research.
Other possible future research includes the implementation of trust
games as a way of operationalizing the concepts in our trust framework. Additionally, work done in belief updating [29], can inform trust
measurements that compare a person’s initial beliefs to their beliefs
immediately after viewing the visualization. For capturing behavioral
outcomes, a longitudinal study would support investigating the longerterm effects of trust. Ultimately, our work moves us towards a richer
understanding of how visualizations can support or hinder trust formation, guiding the more intentional study and design of public-facing
visualizations.

ACKNOWLEDGMENTS
The authors wish to thank Barbara Kulaga for her role in developing thevisualizations, and Bo Yun Park for her role in the original conceptionof the “trust in science" experiment. This work was supported by
the Harvard Data Science Initiative Trust in Science Fund, supported
by Bayer, and partly funded by NSF awards IIS-1901030 and IIS2237585.
R EFERENCES[1] D. Artz and Y. Gil. A survey of trust in computer science and the Semantic
Web. Journal of Web Semantics, 5(2):58–71, June 2007. doi: 10.1016/j.
websem.2007.03.002 2[2] H. Atoyan, J.-R. Duquet, and J.-M. Robert. Trust in new decision aid
systems. In Proceedings of the 18th Conference on l’Interaction HommeMachine, IHM ’06, pp. 115–122. ACM, New York, Apr. 2006. doi: 10.
1145/1132736.1132751 4[3] E. Beauxis-Aussalet, M. Behrisch, R. Borgo, D. H. Chau, C. Collins,
D. Ebert, M. El-Assady, A. Endert, D. A. Keim, J. Kohlhammer, D. Oelke,
J. Peltonen, M. Riveiro, T. Schreck, H. Strobelt, and J. J. v. Wijk. The
Role of Interactive Visualization in Fostering Trust in AI. IEEE Computer
Graphics and Applications, 41(6):7–12, 2021. doi: 10.1109/MCG.2021.
3107875 4[4] J. Berg, J. Dickhaut, and K. McCabe. Trust, Reciprocity, and Social
History. Games and Economic Behavior, 10(1):122–142, July 1995. doi:
10.1006/game.1995.1027 2[5] A. Betella and P. F. M. J. Verschure. The Affective Slider: A Digital
Self-Assessment Scale for the Measurement of Human Emotions. PLOS
ONE, 11(2):e0148037, Feb. 2016. Publisher: Public Library of Science.
doi: 10.1371/journal.pone.0148037 4[6] U. Bhandari, K. Chang, and T. Neben. Understanding the impact ofperceived visual aesthetics on user evaluations: An emotional perspective.
Information & Management, 56(1):85–93, Jan. 2019. doi: 10.1016/j.im.
2018.07.003 4[7] F. Borgonovi. The relationship between education and levels of trust andtolerance in Europe. The British Journal of Sociology, 63(1):146–167,
2012. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.14684446.2011.01397.x. doi: 10.1111/j.1468-4446.2011.01397.x 3[8] J. T. Cacioppo and R. E. Petty. The need for cognition. Journal of
Personality and Social Psychology, 42(1):116–131, 1982. Place: US
Publisher: American Psychological Association. doi: 10.1037/0022-3514.
42.1.116 3[9] R. E. Carlin and G. J. Love. Political Competition, Partisanship andInterpersonal Trust in Electoral Democracies. British Journal of Political
Science, 48(1):115–139, 2018. Publisher: Cambridge University Press.
doi: 10.1017/S0007123415000526 3[10] R. Y. J. Chua, P. Ingram, and M. W. Morris. From the Head and the Heart:
Locating Cognition- and Affect-Based Trust in Managers’ Professional
Networks. Academy of Management Journal, 51(3):436–452, June 2008.
Publisher: Academy of Management. doi: 10.5465/amj.2008.32625956 3[11] E. Costante, J. den Hartog, and M. Petkovic. On-line Trust Perception: What Really Matters. Proceedings of the First Workshop on SocioTechnical Aspects in Security and Trust (STAST’11, Milan, Italy, September 8, 2011), pp. 52–59, 2011. Publisher: Institute of Electrical andElectronics Engineers. doi: 10.1109/STAST.2011.6059256 4[12] J. Dang, K. M. King, and M. Inzlicht. Why Are Self-Report and Behavioral
Measures Weakly Correlated? Trends in Cognitive Sciences, 24(4):267–
269, Apr. 2020. Publisher: Elsevier. doi: 10.1016/j.tics.2020.01.007
7[13] A. Dasgupta, J.-Y. Lee, R. Wilson, R. Lafrance, N. Cramer, K. Cook,and S. Payne. Familiarity Vs Trust: A Comparative Study of Domain
Scientists’ Trust in Visual Analytics and Conventional Analysis Methods.
IEEE Transactions on Visualization and Computer Graphics, 23(1):271–
280, Jan. 2017. doi: 10.1109/TVCG.2016.2598544 2[14] A. Dimoka. What Does the Brain Tell Us About Trust and Distrust? Evidence from a Functional Neuroimaging Study. MIS Quarterly, 34(2):373–
396, 2010. Publisher: Management Information Systems Research Center,
University of Minnesota. doi: 10.2307/20721433 9[15] S. Duncan and L. F. Barrett. Affect is a form of cognition: A neurobiological analysis. Cognition and Emotion, 21(6):1184–1211, Sept. 2007. Publisher: Routledge _eprint: https://doi.org/10.1080/02699930701437931.
doi: 10.1080/02699930701437931 3[16] J. R. Dunn and M. E. Schweitzer. Feeling and Believing: The Influenceof Emotion on Trust. Journal of Personality and Social Psychology,
88(5):736–748, 2005. Place: US Publisher: American Psychological
Association. doi: 10.1037/0022-3514.88.5.736 9[17] H. Elhamdadi, A. Gaba, Y.-S. Kim, and C. Xiong. How Do We Measure
Trust in Visual Data Communication? In 2022 IEEE Evaluation and
Beyond - Methodological Approaches for Visualization (BELIV), pp. 85–
92. IEEE, Oklahoma City, Oct. 2022. doi: 10.1109/BELIV57783.2022.
00014 2[18] H. Elhamdadi, L. Padilla, and C. Xiong. Using Processing Fluency as a Metric of Trust in Scatterplot Visualizations, Sept. 2022.
arXiv:2209.14340 [cs]. doi: 10.48550/arXiv.2209.14340 2, 3
[19] A. M. Evans and W. Revelle. Survey and behavioral measurements ofinterpersonal trust. Journal of Research in Personality, 42(6):1585–1593,
Dec. 2008. doi: 10.1016/j.jrp.2008.07.011 2, 3, 6
[20] R. Gutzwiller, E. Chiou, S. Craig, C. Lewis, G. Lematta, and C.-P. Hsiung.
Positive bias in the ‘Trust in Automated Systems Survey’? An examinationof the Jian et al. (2000) scale, vol. 63. SAGE Publications, Nov. 2019.
Journal Abbreviation: Proceedings of the Human Factors and Ergonomics
Society Annual Meeting Pages: 221 Publication Title: Proceedings ofthe Human Factors and Ergonomics Society Annual Meeting. doi: 10.
1177/1071181319631201 2[21] S. J. Hoch and G. F. Loewenstein. Time-inconsistent Preferences andConsumer Self-Control. Journal of Consumer Research, 17(4):492–507,
Mar. 1991. doi: 10.1086/208573 3[22] M. Hooghe, S. Marien, and T. de Vroome. The cognitive basis of trust.
The relation between education, cognitive ability, and generalized andpolitical trust. Intelligence, 40(6):604–613, Nov. 2012. doi: 10.1016/j.
intell.2012.08.006 3[23] J. Jahansoozi. Organization-stakeholder relationships: Exploring trust
and transparency. Journal of Management Development, 25(10):942–955,
2006. Accepted: 2012-09-06T09:05:32Z Publisher: Emerald. doi: 10.
1108/02621710610708577 2[24] J.-Y. Jian, A. M. Bisantz, and C. G. Drury. Foundations for an Empirically
Determined Scale of Trust in Automated Systems. International Journalof Cognitive Ergonomics, 4(1):53–71, Mar. 2000. Publisher: Routledge_eprint: https://doi.org/10.1207/S15327566IJCE0401_04. doi: 10.1207/
S15327566IJCE0401_04 2[25] K. Kelton, K. R. Fleischmann, and W. A. Wallace.
Trust indigital information.
Journal of the American Society for Information Science and Technology, 59(3):363–374, 2007.
_eprint:
https://onlinelibrary.wiley.com/doi/pdf/10.1002/asi.20722. doi: 10.1002/
asi.20722 2, 3, 4[26] S. Kiesler and L. Sproull. “Social” human-computer interaction. In Humanvalues and the design of computer technology, pp. 191–199. Center for
the Study of Language and Information, USA, Dec. 1997. 2[27] Y. S. Kim. Designing Belief-driven Interactions with Data. Thesis, University of Washington, 2020. Accepted: 2020-08-14T03:31:13Z. 2[28] Y.-S. Kim, P. Kayongo, M. Grunde-McLaughlin, and J. Hullman.
Bayesian-Assisted Inference from Visualized Data. IEEE Transactions on
Visualization and Computer Graphics, 27(2):989–999, Feb. 2021. doi: 10.
1109/TVCG.2020.3028984 2[29] Y.-S. Kim, K. Reinecke, and J. Hullman. Data Through Others’ Eyes: The
Impact of Visualizing Others’ Expectations on Visualization Interpretation.
IEEE Transactions on Visualization and Computer Graphics, 24(1):760–
769, Jan. 2018. doi: 10.1109/TVCG.2017.2745240 2, 9[30] N. Kock and G. Lynn. Lateral Collinearity and Misleading Results in
Variance-Based SEM: An Illustration and Recommendations. Journal of
the Association for Information Systems, 13(7):546–580, 2012. doi: 10.
17705/1jais.00302 7[31] H.-K. Kong, Z. Liu, and K. Karahalios. Trust and Recall of Information
across Varying Degrees of Title-Visualization Misalignment. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems,
pp. 1–13. ACM, Glasgow, May 2019. doi: 10.1145/3290605.3300576 2,
3[32] J. R. Landwehr and L. Eckmann. The nature of processing fluency: Amplification versus hedonic marking. Journal of Experimental Social Psychology, 90:103997, Sept. 2020. doi: 10.1016/j.jesp.2020.103997 2[33] S. Lee, S.-H. Kim, and B. C. Kwon. VLAT: Development of a Visualization Literacy Assessment Test. IEEE Transactions on Visualization
and Computer Graphics, 23(1):551–560, Jan. 2017. Conference Name:IEEE Transactions on Visualization and Computer Graphics. doi: 10.
1109/TVCG.2016.2598920 5, 6

[34] M. Liao, S. S. Sundar, and J. B. Walther. User Trust in Recommendation
Systems: 2022 CHI Conference on Human Factors in Computing Systems,
CHI 2022. In CHI 2022 - Proceedings of the 2022 CHI Conference on
Human Factors in Computing Systems, Conference on Human Factors
in Computing Systems - Proceedings, pp. 1–14. ACM, New Orleans,
Apr. 2022. Publisher: Association for Computing Machinery. doi: 10.
1145/3491102.3501936 2[35] G. Lins de Holanda Coelho, P. H. P. Hanel, and L. J. Wolf. The Very Efficient Assessment of Need for Cognition: Developing a Six-Item Version.
Assessment, 27(8):1870–1885, Aug. 2018. Publisher: SAGE Publications
Inc. doi: 10.1177/1073191118793208 6[36] R. C. Mayer, J. H. Davis, and F. D. Schoorman. An Integrative Model Of
Organizational Trust. Academy of Management Review, 20(3):709–734,
July 1995. Publisher: Academy of Management. doi: 10.5465/amr.1995.
9508080335 1, 3, 4[37] E. Mayr, N. Hynek, S. Salisu, and F. Windhager. Trust in Information
Visualization. EuroVis Workshop on Trustworthy Visualization (TrustVis)(2019), pp. 1–5, 2019. doi: 10.2312/trvis.20191187 2, 3, 4[38] D. J. McAllister. Affect- and Cognition-Based Trust as Foundations forInterpersonal Cooperation in Organizations. Academy of Management
Journal, 38(1):24–59, Feb. 1995. Publisher: Academy of Management.
doi: 10.5465/256727 1, 2, 3, 4[39] C. D. Myers and D. Tingley. The Influence of Emotion on Trust. Political
Analysis, 24(4):492–500, 2016. doi: 10.1093/pan/mpw026 9[40] C. Nass, J. Steuer, and E. R. Tauber. Computers are social actors. In
Proceedings of the SIGCHI conference on Human factors in computing
systems celebrating interdependence - CHI ’94, pp. 72–78. ACM, Boston,
1994. doi: 10.1145/191666.191703 2[41] L. Padilla, R. Fygenson, S. C. Castro, and E. Bertini. Multiple Forecast
Visualizations (MFVs): Trade-offs in Trust and Performance in Multiple
COVID-19 Forecast Visualizations. IEEE transactions on visualizationand computer graphics, 29(1):12–22, Jan. 2023. doi: 10.1109/TVCG.
2022.3209457 1, 2[42] S. Palan and C. Schitter. Prolific.ac—A subject pool for online experiments.
Journal of Behavioral and Experimental Finance, 17:22–27, Mar. 2018.
doi: 10.1016/j.jbef.2017.12.004 5[43] S. Y. Rieh and N. Belkin. Understanding Judgment of Information Quality
and Cognitive Authority in the WWW. Journal of The American Society
for Information Science and Technology - JASIS, 35:279–289, Jan. 1998.
3[44] D. Sacha, H. Senaratne, B. C. Kwon, G. Ellis, and D. A. Keim. The
Role of Uncertainty, Awareness, and Trust in Visual Analytics. IEEE
Transactions on Visualization and Computer Graphics, 22(1):240–249,
Jan. 2016. Conference Name: IEEE Transactions on Visualization andComputer Graphics. doi: 10.1109/TVCG.2015.2467591 4[45] A. K. Schnackenberg and E. C. Tomlinson. Organizational transparency:
A new perspective on managing trust in organization-stakeholder relationships. Journal of Management, 42(7):1784–1810, 2016. Publisher: Sage Publications Section: Journal of Management. doi: 10.1177/
0149206314525202 2[46] S. Sohn. Consumer processing of mobile online stores: Sources andeffects of processing fluency. Journal of Retailing and Consumer Services,36:137–147, May 2017. doi: 10.1016/j.jretconser.2017.01.008 2[47] P. Sturgis, S. Read, and N. Allum. Does intelligence foster generalized
trust? An empirical test using the UK birth cohort studies. Intelligence,38(1):45–54, Jan. 2010. doi: 10.1016/j.intell.2009.11.006 3[48] E. Thorndike. A constant error in psychological ratings. Journal ofApplied Psychology, 4(1):25–29, 1920. Place: US Publisher: American
Psychological Association. doi: 10.1037/h0071663 2[49] E. C. Tomlinson, A. K. Schnackenberg, D. Dawley, and S. R. Ash. Revisiting the trustworthiness–trust relationship: Exploring the differential
predictors of cognition- and affect-based trust. Journal of Organizational
Behavior, 41(6):535–550, 2020. Place: US Publisher: John Wiley & Sons.
doi: 10.1002/job.2448 3[50] A. M. van der Bles, S. van der Linden, A. L. J. Freeman, J. Mitchell,
A. B. Galvao, L. Zaval, and D. J. Spiegelhalter. Communicating uncertainty about facts, numbers and science. Royal Society Open Science,
6(5):181870, May 2019. doi: 10.1098/rsos.181870 2, 3[51] A. M. van der Bles, S. van der Linden, A. L. J. Freeman, and D. J.
Spiegelhalter. The effects of communicating uncertainty on public trustin facts and numbers. Proceedings of the National Academy of Sciences,
117(14):7672–7683, Apr. 2020. Publisher: Proceedings of the National
Academy of Sciences. doi: 10.1073/pnas.1913678117 3[52] C. G. Wetzel, T. D. Wilson, and J. Kort. The halo effect revisited: Forewarned is not forearmed. Journal of Experimental Social Psychology,
17(4):427–439, July 1981. doi: 10.1016/0022-1031(81)90049-4 2[53] C. Xiong, L. Padilla, K. Grayson, and S. Franconeri. Examining the Components of Trust in Map-Based Visualizations. TrustVis 2019 - EuroVis
Workshop on Trustworthy Visualization, pp. 19–23, 2019. Publisher: The
Eurographics Association. doi: 10.2312/trvis.20191186 2, 3, 4[54] M. Yin, J. Wortman Vaughan, and H. Wallach. Understanding the Effectof Accuracy on Trust in Machine Learning Models. In Proceedings of the2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12.
ACM, Glasgow, May 2019. doi: 10.1145/3290605.3300509 2[55] K. Yu, S. Berkovsky, D. Conway, R. Taib, J. Zhou, and F. Chen. Trustand reliance based on system accuracy. UMAP 2016 - Proceedings of the2016 Conference on User Modeling Adaptation and Personalization, pp.
223–227, 2016. Place: New York Publisher: Association for Computing
Machinery (ACM). doi: 10.1145/2930238.2930290 2[56] K. Yu, S. Berkovsky, R. Taib, D. Conway, J. Zhou, and F. Chen. User Trust
Dynamics: An Investigation Driven by Differences in System Performance.
Association for Computing Machinery, Mar. 2017. Pages: 317. doi: 10.
1145/3025171.3025219 2[57] R. Zehrung, A. Singhal, M. Correll, and L. Battle. Vis Ex Machina: An
Analysis of Trust in Human versus Algorithmically Generated Visualization Recommendations. In Proceedings of the 2021 CHI Conference on
Human Factors in Computing Systems, CHI ’21, pp. 1–12. ACM, New
York, May 2021. doi: 10.1145/3411764.3445195 2[58] P. Zhang. The Affective Response Model: A Theoretical Framework of
Affective Concepts and Their Relationships in the ICT Context. MIS
Quarterly: Management Information Systems, 37(1):247–274, Mar. 2013.
doi: 10.25300/MISQ/2013/37.1.11 3, 4[59] Y. Zhang, N. Suhaimi, N. Yongsatianchot, J. Gaggiano, M. Kim, S. Patel,
Y. Sun, S. Marsella, J. Grifn, and A. Parker. Shifting Trust: Examining
How Trust and Distrust Emerge, Transform, and Collapse in COVID-19
Information Seeking. Association for Computing Machinery, Apr. 2022.
doi: 10.1145/3491102.3501889 2, 9[60] J. Zheng, E. Veinott, N. Bos, J. S. Olson, and G. M. Olson. Trust without
Touch: Jumpstarting long-distance trust with initial social activities. In
Proceedings of the SIGCHI conference on human factors in computing
systems, pp. 141–146. ACM, Minneapolis, 2002. doi: 10.1145/503376.
503402 2[61] J. Zhou, Z. Li, H. Hu, K. Yu, F. Chen, Z. Li, and Y. Wang. Effects ofInfluence on User Trust in Predictive Decision Making. In Extended
Abstracts of the 2019 CHI Conference on Human Factors in Computing
Systems, CHI EA ’19, pp. 1–6. ACM, New York, May 2019. doi: 10.
1145/3290607.3312962 2[62] M. X. Zhou and S. K. Feiner. Visual task characterization for automatedvisual discourse synthesis. In Proceedings of the SIGCHI conference on
Human factors in computing systems - CHI ’98, pp. 392–399. ACM, Los
Angeles, 1998. doi: 10.1145/274644.274698 2