# An Image-based Typology for Visualization

Jian Chen, Petra Isenberg, Robert S Laramee, Tobias Isenberg, Michael

Sedlmair, Torsten Möller, Rui Li

<figure><img src="image_1.png" /></figure>

## To cite this version:

Jian Chen, Petra Isenberg, Robert S Laramee, Tobias Isenberg, Michael Sedlmair, et al.. An Image-based Typology for Visualization. 2024. hal-05079549

HAL Id: hal-05079549

https://inria.hal.science/hal-05079549v1

Preprint submitted on 22 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An Image-based Typology for Visualization

Jian Chen<sup>D</sup>, Petra Isenberg<sup>D</sup>, Robert S. Laramee<sup>D</sup>, Tobias Isenberg<sup>D</sup>, Michael Sedlmair<sup>D</sup>, Torsten Möller<sup>D</sup>, Rui Li<sup>D</sup>

**Abstract**—We present and discuss the results of a qualitative analysis of visualization images to derive an image-based typology of visualizations. For each image, we seek to identify its main focus or the essential stimuli. As a result, we derived 10 image-based visualization types. We describe coding decisions we made in the derivation process. The resulting image typology can serve a number of purposes: enabling researchers and practitioners to identify visual design styles, facilitating the categorization of visualization images for the purpose of research and teaching, enabling researchers to study the evolution of the community and its research output over time, and facilitating a discussion of standardization in visualization. In addition, the tool and dataset enable scholars to closely examine the images and how they are published and communicated in our community. osf.io/dxjwt presents a pre-registration and all supplemental materials.

**Index Terms**—Typology, image-based, visual representation, evaluation, image types.

## 1. INTRODUCTION

WHILE characterizations of visualization representations have been proposed in the literature, they focus on what designers construct rather than what viewers see from images. We attempt to fill this gap by describing an image-based typology. Our goal is to provide a framework for categorizing visualization images with a focus on the *visual appearance* of the *essential stimuli*—the main visual focus of an image.

**Why is it important to categorize visualization images?** Categorization is “an abstraction of things and ideas into groups and most if not all categories have verbal labels” [137, p. 318]. Categorization allows us to reason about a phenomenon, discover order, and simplify discussion and experience [63]. Visualization research has taken advantage of these benefits throughout its history. Existing categorizations enable us to understand the field, its practices, and progress better and inspire future work; for example, to study representations for specific data types [2], [111], research topics and keywords [61], evaluation goals and practices [62], [76], interaction techniques [147], or tasks and activities [3], [38]. Finally, in the era of artificial intelligence, one can also use our categories to index knowledge and to benchmark human or artificial information processors, to eventually assist humans [104], [119].

**Why a categorization based on visual appearance?** We consider visualization images to be standalone entities, and we posit that categorizing them enables us to derive new insights. For example, when developing a new research method for various applications, testing the method on a representative set of images is important; for an example see the development of the BeauVis scale [51]. In addition, a categorization focused

on visual appearance also allows us to conduct a dedicated historical analysis. We can observe and compare the evolution of rendering styles and contextual representations within a category [73], compare design styles across categories, and attempt to reason about possible influences such as individual papers, designers, or rendering hardware. Finally, for writing overview articles, textbooks, or lecture series, categorizations based on the main visual focus of attention can highlight the variety of approaches to specific techniques or identify aspects that need further investigation.

**Why a system of main focus or essential stimuli by visual similarity?** Approaches for categorizing data visualizations often rely on evaluating them with respect to the amount of *data* they represent (e. g., a few or millions) [69], with respect to the difference in *coordinate systems* (a part of the visual implantation by Bertin [11], also see Claessen and Van Wijk’s work [29]), with respect to what aspect of the *represented data is being visually encoded* (e. g., multivariate & multifaceted [68]), or with respect to *continuity* [33]. These considerations belong to the fundamental aspects of a representational system [99]. Here, we take a different approach that is neither a *top-down* categorization (focusing on techniques and/or rendering methods) nor a categorization based on designer’s basic drawing elements (grouping “marks and channels” that may not be able to distinguish image similarity). Instead, we use essential stimuli as *prototype representations*, where a *representation* is “something static, something we can see and contemplate from images”; and *prototype* [106] means “essential stimuli that take a salient position in the formation of the category” [127]. This categorization is human-centric, based on the observations and discussions of six senior visualization researchers. In later parts of the paper we show, however, that the categorizations can also be successfully applied by others.

**Our method:** The artifacts we categorized are images from papers published in IEEE VIS (or known as VisWeek between 2008–2012). Specifically, we coded 6,833 figures from 695 papers published in 1990, 1995, 2000, 2005, 2010, 2015, and 2020, spanning a 30-year history of the conference [27]. To prioritize the main focus or essential stimuli in these images, we did not consult the captions or original papers. We follow an interpretation of categorization [63] that supports graded structures and allows categories to vary depending on contextual use. Thus, images can be members of a category to varying degrees. That is, some members of a category may be seen as more or less representative of a category than others, a view that a taxonomy might prohibit. Also, fuzzy boundaries between categories mean that images can belong to more than one category. Finally, we also used our background knowledge to influence the discussion of images that were not clearly in or out of certain categories. As such, our work is considered a

This paper was produced by The Ohio State University, USA; Université Paris-Saclay, CNRS, Inria, LISN, France; University of Nottingham, UK; University of Stuttgart, Germany; University of Vienna, Austria
Manuscript received March –, 2024; revised –, –.
<figure><img src="image_3.png" /><figcaption>Fig. 1: **Our 10 visualization types (VisTypes).** They characterize the associations of what we see from images. Here we show the overall image counts and relative frequency trends in % for 1990–2020. Surface-based techniques & volumes are most common, followed by line-based techniques and points. Together, these three main types make up about half of the images we coded (See our online tool visimagenavigator.github.io for additional examples). Each type represents a family of visual appearances of essential stimuli, the main focus of a visualization image.</figcaption></figure>

categorization rather than a classification [63].

**Results and validation:** Our final typology, VisTypes, describes 10 visualization types (Fig. 1), two purposes (schematic and GUI), and their perceived spatial dimensionality (2D and 3D). We also report on the results of a user study to validate the usability of our typology (Sect. 6). Finally, we discuss how to apply VisTypes in the form of three scenarios: to explore new empirical study methods, to analyze visualization evolutions, and to promote standardization (Sect. 7).

In summary, we make three contributions:

* **Method.** We have, for the first time, considered visualizations from the perspective of the essential stimuli in images.

* **Types and the VisTypes Dataset.** We develop the currently largest carefully annotated dataset about visualization types.

* **User study.** We validate the usability of VisTypes.

## 2. TERMS AND RELATED WORK

Related work includes visualization categorization studies of construction guidelines, data types, tasks, and work that analyzes research figures. We will review these areas next.

### A. Visualization Typology: Definitions

The visualization images we discuss refer to figures in papers, which we visually assessed. We differentiate *visual element* from the term *visual stimulus*. By *visual elements*, we refer to the individual components or building blocks used by visualization designers or practitioners to create an effective composition in an image. Visual elements often include marks and channels such as lines, shapes, colors, textures, icons, and typography (see Munzner's textbook [92]). Furthermore, we take a psychophysics perspective and use the term *visual stimulus* to mean any visual input or signal that stimulates the visual system [136, pp. 394–395]. It is a broad term that can include objects, patterns, colors, and any other low-level visual input (e. g., lines) [143]. As a result, while both *visual element* and *visual stimulus* are related to visual representations, we use

*visual element* as input to create visual design compositions and *visual stimulus* to mean visual input perceived by visual information processors. Given this distinction, we use the primary visual focus to represent a stimulus that takes a salient position in a visualization image.

### B. Categorization as an analogy of “What is it like?”

There is a long history of categorization methods in philosophy and psychology [47]. Wittgenstein [142] argues that users are fluid about categories: different people may provide different answers, and the same person may respond with different answers at different times. As a result, a top-down categorization that draws clear boundaries of shared properties to those of other categories is often too strict to represent how people understand categories. Instead, a “this-looks-like-that” bottom-up association can be used. Rosch [106] later proposed that natural categories exist (of colors or forms) which are abstracted to *natural prototypes* which take a salient position in the formation of a category from a bottom-up cluster of similar instances. Psychologists (e. g., Medin and Schaffer [89], Nosofsky [97], and Krushki [74]) have gone further and argued for what they call the *sample-based theory* of categorization, where humans store *instances* and *examples*; similarities and relatedness between entities allow us to learn *associations* between these examples. For *objects* that are closely associated, they *look like* a cluster of learned examples and belong to the same category. Inspired by these works, we summarize and provide examples of visualization images and what we see in them. We make these associations the basis of our typology.

### C. Visualization categorizations

Textbooks, in particular, often rely on categorizations to structure content [103]. While some (older) books [18] are collections of graphical representations, many of today's textbooks regularly use one of a few structures:
<table><caption>TABLE I: **Definitions of our 10 labels.** They define the visual appearance of the essential stimuli independent of data, tasks, or rendering techniques. We color-coded the features in our codes in **orange**. The object informs a viewer what to see to process the information in an image properly in order to build meaning from data represented in an image. Our code types are independent of data, tasks, or rendering techniques.</caption><thead><tr><th>Visualization Type Codes</th><th>Description</th><th>Examples</th></tr></thead><tbody><tr><td>(1) Generalized Bar Representations (Bar)</td><td>Graphs that represent data with straight bars that can be arranged on a straight or curved baseline and whose heights or lengths are proportional to the values they represent.</td><td>bar charts, stacked bar charts, box plots, or sunburst diagrams.</td></tr><tr><td>(2) Point-based Representations (Point)</td><td>Representations that use point locations. These locations are often shown using dots or circles, but also other shapes such as 3D spheres, triangles, stars, etc.</td><td>scatterplots, point clouds, dot plots, or bubble charts.</td></tr><tr><td>(3) Line-based Representations (Line)</td><td>Representations where information is emphasized through (straight or curved) lines.</td><td>line charts, parallel coordinates, contour lines, radar/spider charts, streamlines, or tensor field lines.</td></tr><tr><td>(4) Node-link Trees/Graphs, Networks, Meshes (Node-link)</td><td>Representations using points for and explicit connections between these points to convey relationships between data values.</td><td>node-link diagrams, node-link trees, node-link graphs, meshes, arc diagrams, or Sankey diagrams.</td></tr><tr><td>(5) Area-based Representations (Area)</td><td>Representations with a focus on areas of 2D space or 2D surfaces including sub-sets of these surfaces. Areas can be geographical regions or polygons whose size or shape represents abstract data.</td><td>(stacked) area chart, streamgraph, ThemeRiver, violin plot, cartograms, histograms, ridgeline chart, Voronoi diagram, treemaps, pie chart.</td></tr><tr><td>(6) Surface-based and Volume Representations (Surface/Volume)</td><td>Representations of the inner and/or outer features and/or boundaries of a continuous spatial phenomenon or object in 3D physical space or 4D space-time, or slices thereof.</td><td>terrains, isosurfaces, stream surfaces, volume rendering using transfer functions, slices through a volume (e.g., X-ray, CT slice).</td></tr><tr><td>(7) Generalized Matrix / Grid (Grid)</td><td>Representations that separate data into a discrete spatial grid structure. The grid often has rectangular cells but may also use other shapes such as hexagons or cubes. Elements such as glyphs or a color encoding can appear in the grid cells.</td><td>network matrices, discrete heatmaps, scarf or strip plots, space-time cubes, or matrix-based network visualizations.</td></tr><tr><td>(8) Continuous Color and Grey-scale, and Textures (Cont.-ColorPattern)</td><td>Representations of structured patterns across an image or atop a geometric 3D object. These patterns can be evoked by changes in intensity, changes in hue, brightness, and/or saturation. The changes are typically smooth (continuous) but could show sharp transitions as well.</td><td>Directional patterns such as Line Integral Convolution (LIC), Spot Noise, and Image-Space Advection (ISA) to show flow fields, continuous heatmaps, intensity fields, or even a binary image.</td></tr><tr><td>(9) Glyph-based Representations (Glyph)</td><td>Multiple small independent visual representations (often encoded by position and additional dimensions using color, shape, or other geometric primitives) that depict multiple attributes (dimensions) of a data record. Placement is usually meaningful and typically multiple glyphs are displayed for comparison.</td><td>Star glyphs, 3D glyphs, Chernoff faces, vector field glyphs</td></tr><tr><td>(10) Text-based Representations (Text)</td><td>Representations of data (often text itself) that use varying properties of letters/words such as font size, color, width, style, or type to encode data.</td><td>Tag clouds, word trees, parallel tag clouds, typomaps.</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>(a) Map vs. Text.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b) Lines vs. Schematic.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(c) Bar vs. Bar & Point.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(d) Donut chart vs. Bar & Node-link.</figcaption></figure>

<caption>Fig. 2: **Code type comparisons.** We compare codes between Borkin et al. [15]'s types color-coded in cyan and ours (color-coded in orange) in (a)-(c) and between Deng et al. [37]'s types (color-coded in green) and ours in (d). We label image (a) [1] text—because the essential stimuli that represent the geospatial data and metadata here are text. Our typology does not include “map” because maps are a representation technique that may have very different visual appearances (e.g., consist of only points, areas, or text as seen here). We label image (b) [21] as a schematic representation, elucidating its conceptual illustration; actual data are not the main focus. Image (c) has two labels Bar and Point. We have more complete essential stimuli to signify the importance of seeing both Bar and Point. Image (d) [149] emphasizes data types (networks), and we use Bar & Node-link, beyond the technique label of Donut-chart.</caption>

**Focus on construction rules in design or techniques.** A seminal approach to characterizing visual designs is Bertin's visual semiotics [11]. He discussed the fundamental building blocks of a visualization that are modified by visual elements (channels) that encode data. Similarly, researchers have proposed to describe visual designs through the lens of a visual language with a set of syntactic rules. Examples include Wilkinson's Grammar of Graphics (GoG) [141], Engelhardt's Language of Graphics [131], and Mackinlay's automatic design [85]. Applying rules formulated in a visual language can yield a broad range of visual designs [92] and tools and libraries are based on them, e.g., Tableau [86], D3 [17], Vega-Lite [108], and Draco [91]. Others, e.g., Tufte's Envisioning Information [126], differentiate techniques by higher-level

construction rules, e.g., small multiples, or principles. e.g., layering and separation. Again others use concrete technique names such as Lohse et al.'s [82] 11 categories: graphs, tables, time charts, network charts, diagrams (process and structure diagrams), maps, cartograms, icons, and photorealistic images. This type of categorization sometimes mixes data-type specific representations (maps, network charts, time charts) and data-type agnostic representation (icons, diagrams, tables). In the case of Lohse et al. [82], there are 60 images inspired by Bertin [11] and Tufte [125], viewers were 16 students and staff at the University of Michigan with no special subject training. In our case, we selected out input images from the VIS proceedings, and our coders were visualization experts with extensive experience in visualization research. Experts are
generally more suitable to define categories [121].

**Focus on data types.** Other researchers have categorized prototypical visual designs based on the data type. This approach makes sense as, in a typical iterative design process, data are systematically mapped, winnowed, and refined to visual encoding [112]. Ward et al. [135], e. g., classify visualization techniques for spatial data, geospatial data, time-oriented data, multivariate data, trees, graphs, and networks, and text and document visualizations. Heer et al.'s visualization zoo [52] classifies time series, statistical data, maps, hierarchies, and networks. Brodlie [19, p. 40] classifies techniques into those for point, scalar, vector, and tensor data. Franke and Haehn [44] present implementations on the web. Compared to these characterizations, where data are input and visual images are output, we attempt to characterize images, where an image is input, and the primary foci or essential stimuli are the output, without necessarily knowing the characteristics of the data that led to the final image. For example, we make no distinction between a line chart that shows temporal data and one that shows, e. g., point relationships in parallel coordinates. Finally, our categorization only concerns the visual stimuli that represent data, and the visual representations taking a salient position in the image; we also excluded the non-essential stimuli that do not directly map to data (e. g., axis and labels).

**Focus on task and analysis question types.** Another set of textbooks introduces visualizations by linking representation and analysis tasks/questions. Fisher and Meyer [43], for instance, group techniques such as histograms and boxplots under the analysis question of “showing how data is distributed.” Maciejewski [84] also takes this approach in his grouping of techniques. Again, a focus on analysis questions considers a-priori criteria to choose and categorize visualization techniques in the same vein as data and construction rules do. Visually similar techniques are thus considered in separate categories; for example, Fisher and Meyer [43] categorize bar charts under “visualizations that show how groups differ” and histograms under “visualizations that show how data is distributed.”

**A cross-disciplinary perspective for categorization.** We are certainly not the first to realize the differences between *what we design* and *what we see*. For example, the vision science community has realized that composition solutions from low-level features (e. g., Canny edge detector [23], orientation map [87], and HOG algorithms [41]) to composite high-level objects do not align with *how* people see [101], [143]. Recent deep convolutional neural networks (DNNs) can assign categories to items because the categories are treated as a continuous space of related concepts [24], [88], [146]. For example, BioCLIP [116] combined a large- language model and images to produce biological concepts. However, the authors worked on natural scenes rather than on abstract images. As we show in this paper, however, the diversity of representations from images is large: a single category can be broad and contain various spatial arrangements, compositions, and viewpoints.

## D. The role of graphs in scientific communication

Even though our approach to categorizing images differs from that of many textbooks, our method of studying them to evaluate

scientific advances has been used before. Latour [77] presented graph features that make them essentially a pervasive form of visualization and a specialized vocabulary for transforming and analyzing data to represent scientific findings. The pervasiveness and centrality of scientific figures led Latour to conclude that scientists exhibit a “graphical obsession” and suggest that graphs distinguish scientific domains.

Other image datasets [6], [109] are available, but they focus on what designers draw rather than what viewers should see.

Instead, our work is closely related to Borkin et al. [15] and Li and Chen [81] who studied high-level visualization categories from large image datasets. Borkin et al. [15], e. g., suggested a taxonomy of techniques. Theirs is a mix of encoding (area, bar, ...), data (text), layout (circle), and analysis-focused (distribution) categories. Similarly, Deng et al. [37] and Lu et al. [83] added data and technique labels.

In our categorization we instead attempted to separate data, techniques, and essential stimuli. For example, we considered whether essential stimuli are more similar to each other within a category than they are compared to other categories. For example, lines in Fig. 2(a) are a schematic view as they do not encode real data and bars and points are both essential stimuli in Fig. 2(b). These examples show that our VisTypes typology provides an abstract, and a general way to organize and store visual designs, independent of the underlying techniques and data (as in Fig. 2(b) and (c)).

## 3. THE IMAGE CODING PROCESS

We begin with an overview of our process that led to VisTypes; graphically represented in Fig. 3. We did not originally start out with a focus on essential stimuli. Instead, we applied other categorizations that used specific techniques (e. g., Isenberg et al. [60]), design features (e. g., “marks and channels” [11]), or a mix of them (e. g., Borkin et al. [15]) in Phases 1-2. A consequence of these failed attempts was that we rejected the idea that “marks and channels” or “techniques” could describe what we perceived in these images. Visual designs are often composed of many marks, channels, and techniques combined together. It became very difficult to label each element and describe which ones formed the visual focus of each image. In Phases 3–6 we then developed our focus on essential stimuli to support categorize what we see. In this section we describe many difficult decisions that were part of this development process. Readers who are not interested in the decision process can skip this section and continue with Sect. 4 (Results) directly.

### A. Goals of our image coding

These primary goals guided our coding process in Phases 3–6:

**Provide a small set of broad categories.** We purposefully wanted to create only a few categories that would remain manageable despite the detailed and often complex images produced in the community. The type abstraction should enable analyses between and within categories to support researchers while analyzing the diversity of representations and inferring the prevalence of rendering methods, algorithms, or dimensions.

**Focus on visual similarity.** As described earlier, our typology focuses on describing visual similarity via essential
<figure><img src="image_8.png" /><figcaption>Fig. 3: **The image coding process.** We developed the method, identified the type codes, and performed the image labeling and curation in a multi-year team effort. This lengthy process includes three failed attempts (Phases 1 & 2). We finally used, as shown in Phases 3–6, a similarity-based appearance categorization to derive the essential stimuli. **Observation:** visualization techniques and design elements are distinct from what we see for making conscious judgments of representations.</figcaption></figure>

stimuli. Our types will appear visually similar regardless of the usage of data or tasks. Low-level marks and channels are not always enough although some of our types share similarities with some marks and channels. Yet, the layout and their morphology are irrelevant for our types.

**Collect experiences on the difficulties of categorizing visualization images.** We documented our multi-stage process to derive a relatively high-level categorization of images and describe inherent uncertainty, failed attempts, and current limitations. We also recorded how difficult it was to understand images taken out of the context of the text and captions since many examples did not necessarily contain a descriptive title and clear attribute annotations.

## B. Visualization image data source

*Image Sampling.* We developed our typology using the VIS30K [27] collection of images and its associated VisPubData [60] meta data. This dataset largely represents visualization as a field because it contains every visualization image published at IEEE VIS (including Visual Analytics, Information Visualization, and Scientific Visualization) since 1990. Our domain's research progress and the scientific solutions reveal themselves visually in the images in these papers and thus provide a focused perspective without managing the diverse publishing goals of journals and conferences. Also, based on our code development process (described below), it became clear that we would not be able to code all 30,000 images. We thus prioritized covering images across time to understand historic changes. We labeled images in five-year intervals for our full coding process, starting with 1990 and up until 2020 (inclusive), yielding 6,833 images from 695 IEEE VIS full papers (incl. case studies).

*Removal of GUI and Schematic Representations.* We also suppressed two “image purpose” categories that we initially used: “GUI (screenshots)/user interface depiction” and “schematic representations and concept illustrations”. GUIs are screenshots of visualization tools showing multiple categories

and schematics do not encode any data. This choice allowed us to focus on data representations instead of compositions.

*Exclusion of Symbolic Representations.* Algorithms, mathematical equations, and tables are symbolic representations often used to structure and interpret unstructured text data. These symbolic representations do not visually represent the texts, and we have thus excluded them. Our *text-based representation* in our categorization (Table I) is only applicable when the text itself encodes data. Thus, it excludes cases where text is used solely for labeling and annotation (e. g., Stokes et al. [117]) or cases in which text serves as the underlying data source but its properties are not encoded as text (e. g., word or document embedding coded as “generalized point representations” [66] or “generalized area representations” [34]).

## C. Image classification process

Included in our process description are failed attempts, key decisions, and iterative coding refinement (Fig. 3) that precede the final codes of primary foci or essential stimuli (Table I).

**Phase 1—Initial image classification based on keywords** (from circa Aug. 2020, approx. 2 months): We began our work focusing on visualization techniques, where we considered that technique names (such as treemaps, parallel coordinates, etc.) could describe the content of the images we analyzed well. To improve objectivity and reduce bias, we tagged images with the most common technique names extracted from author keywords used for VIS papers. We ranked the author keywords extracted in prior work [61] to derive the initial top-21 keywords for specific techniques (Appx. III-A). In addition to the encoding techniques used in each image, we added two code categories that described additional image characteristics: the rendering dimensionality (i. e., 2D or 3D) and the functional purposes of creating the image (i. e., the reason why the authors created each image, for example, illustration of a visualization technique, experiment results, or screenshot of GUI. (Table II in Appx. IV). The initial label set thus included 25 codes.
**Phase 2—Initial coding** (from circa Oct. 2020, approx. 1 month): To test our initial code set, each coder categorized visualization images from 2006. We used these images only in this phase to refine our code set. We subsequently introduced new technique codes by merging techniques that had been tagged “other,” and added “schematic diagram” to the list of image purposes, resulting in 22 technique codes. We discussed the definition of these terms and gave all coders (we call ourselves “ExpertResearchers”) written instructions and example images from each category (Fig. 19b in Appx. IV). One coder initially labeled each image in this stage, and a second coder validated the image. We based the validation assignment of the second coder on their respective expertise to verify all images included or excluded in a specific category (e. g., volume depictions were coded and verified by someone with a background in volume graphics). We removed false positives, avoided false negatives, and ensured image categorization consistency with these steps.

**Phase 3—Consolidation, seeing by association and analogies** (from circa Nov. 2020, approx. 1 month): We then discussed what worked well, what did not, and why. The codes that focused on visualization techniques quickly became difficult to apply as the number of techniques grew increasingly large. We had difficulties defining when a technique should receive its own label or be covered under “other.” Also, some seemingly similar images where coded under different techniques. Point clouds and 3D scatterplots, e. g., both render points according to underlying coordinates in 3D space, with the main difference that scatterplots typically include reference structures such as axes and gridlines. These conflicts led us to re-frame our code set using higher-level, more general visualization type codes.

We decided to focus on describing the *main focus of a given image* or *essential stimuli* and to create codes that enable the viewer to distinguish graphical similarities and differences. We thus re-grouped and merged the codes, which shared similar visual characteristics, into a more general code category. For example, we put *isosurface* into a more general *surface-based techniques* category and grouped *point clouds* and *scatterplots* into a more abstract *point-based techniques* category (in Appx. III-C we list more decisions).

This consolidation process resulted in 10 high-level visualization-type codes that became part of our final set shown in Table I. We reduced our “*image purpose*” category to just two codes: GUI (screenshots) and Schematics, as these were visually identifiable without requiring knowledge of the underlying data. Both categories represent the codes from “other representations” from Phase 1, which did not appear on our technique-focused labels. Given the challenges of labeling the visualization images we encountered, we also decided to collect subjective ratings of difficulty (easy, neutral, and hard). see Fig. 19c in Appx. IV.

**Phase 4—Code calibration** (from circa Dec. 2020, approx. 3 months): In the first two months after having arrived at the new labels, we discussed, debated, and coded two sets of 50 (i. e., 100 in total) randomly chosen images from our seven-year target image data set to calibrate our collective understanding. During this exercise, we clarified code definitions and discussed

ambiguities. We assigned these 100 images to each expert researcher for quality control, aligning our decisions, and discussing potential pitfalls with our new code set. We implemented a dedicated web-based labeling tool (Fig. 19d in Appx. IV) for the coding in this phase. At the end of this phase, our project was one year old. We had almost weekly meetings and discussions throughout this time and reached a consensus on the typology.

**Phase 5—Result coding and validation** (from circa Apr. 2021, approx. 6 months): In this phase, we coded all 6,833 images in our chosen dataset based on the refined definitions and characterizations, using largely the same coding tool as before (Appx. IV). We used the 10 visualization type labels, two “*image purpose*” labels (GUI, Schematics), two labels for the dimensionality (2D vs. 3D), and three difficulty labels (easy, neutral, hard) that capture how difficult the categorization was for the coder. We first looked at the function of an image. We assigned the respective label if the image showed either a schematic or a GUI. If not, the “*image purpose*” was implicitly considered a “visualization example,” and we proceeded to assign a visualization type label. Both categories shared an “I cannot tell” code assigned when neither explicit purpose nor a visualization type could be assigned. In addition, coders could freely add new labels for visualization or image types when they found something new that was not covered by existing categories. For dimensionality, we also added a code called “I cannot tell,” which could be checked when coders were unsure whether the visualization was a 2D or 3D rendering. Coders could assign multiple types and dimensionalities to one image because many images show more than one visual design.

In the coding process, we also recoded the 100 images we had previously labeled during Phase 4. We randomly assigned two coders to each image to ensure high-quality results and capture potential difficulties in applying the labels. This phase was laborious due to the number of images we coded. We met regularly to resolve further questions that arose during coding and to discuss or clarify code descriptions further.

**Phase 6—Verification** (from circa Oct. 2021, approx. 6 months): In this final phase, the two coders assigned to each image worked to reach an agreement when their labels did not match. For this purpose, we developed more web-based visual interfaces (see Fig. 20 in Appx. IV) that focused on conflict resolution and on providing an overview of applied labels. We then filtered the results so that only the inconsistently coded images were shown, and we used this process to resolve all disagreements. This verification was also a lengthy and laborious process that required constant discussion. We analyzed difficult and ambiguous cases as a team every week until we could agree on a solution (we describe some of the most difficult decisions in more detail in Sect. 5). As part of these discussions, we added our last visualization type “*generalized continuous color/grey-scale/texture representations*”, listed as point (8) in Table I. We also further refined our definitions. Consequently, we went through all previously labeled images again to check if they had to be re-coded for consistency and resolved potential resulting disagreements as part of our discussion process.
<figure><img src="image_9.png" /><figcaption>Fig. 4: The proportion of applied image codes for each category, relative to the total 4,070 visualization images (after excluding the pure schematic and GUI images). We can see that the most common visualization types were “generalized surface and volume representations” and “line-based representations.”</figcaption></figure>

## 4. RESULTS

Our coding process resulted in 6,252 image type labels, 2,754 function labels (GUI and Schematics), 7,174 2D/3D dimensionality labels, and 13,647 difficulty labels for the 6,833 images. Fig. 1 and Fig. 4 show the type distribution. We now describe each type of essential stimuli in more detail and show canonical or typical examples for each. Fig. 21, Fig. 22, and Fig. 23 in Appx. V show additional results.

### A. Visualization types

#### Surface-based Representations and Volumes

represent the inner and/or outer features and/or boundaries of a continuous spatial phenomenon or object in 3D physical space (e. g., isosurfaces [48]), 4D space-time, or slices thereof. Frequent characteristics in this category include surface or volume-based rendering with lighting, shading, camera perspectives, shadows, and/or transparency; or 2D medical imaging slices. Common applications include medical images, terrain surfaces, and images of physical simulations (e. g., [39]).

Volume rendering has been one of the most important areas of visualization in the early years [61]. As such, it is perhaps not surprising that surface and volume representations were the most common techniques in our images. They are also the easiest to identify. This category has the highest coder consistency, at 74%, among all types. It is also the only representation technique with primarily 3D renderings. The few 2D renderings we found included volume slices (e. g., MRIs or CTs) or X-rays. Similar to what was shown in prior work on keywords [61]. There has been a significant decrease in the proportion of surface and volume images, dropping from approximately 47% in 2005 to around 20% recently. Canonical examples, from [57], [122], [93], are:

<figure><img src="image_10.png" /></figure>

Line-based Representations are graphs where information is emphasized through straight or curved lines. Canonical visualization techniques are line charts, parallel coordinates, radar/spider charts, contour lines, streamlines, or tensor field lines.

Lines, edges, and curves are the second most common representations indicated by our image labels. We did not code lines that simply delineate screen-space areas as edges. About a sixth (17.7%) of line-based representations are rendered in 3D. For a sizable proportion of 12% of the line chart images, the coders also could not tell whether a line chart is 3D or 2D due to a lack of clear depth cues. Coder consistency ranks among the top three, on par with “generalized surface and volume representations” and “generalized bar representations,” at a consistency level of 65%. Most line charts are the typical 2D line charts that most often represent temporal data. Canonical line-based representations, taken from [4], [118], [130], are:

<figure><img src="image_11.png" /></figure>

Point-based Representations encode point locations. These locations are often shown using dots or circles, but also other shapes such as 3D spheres, triangles, stars, etc.

Similar to Bertin [11], we considered point-based representation to encode point locations in a 2D or 3D space. Point marks could be small circles but also 3D spheres and sometimes other shapes like triangles, stars, etc. Canonical examples of visualization techniques of this type are scatterplots, (volumetric) point clouds, or dot plots. Point-based representations were the third most common visualization type according to our coding. We found that identifying them was slightly more difficult than “generalized bar representations” we discuss next, and the overall consistency of the coders was 57%. Surprisingly, we saw only a small percentage of 3D point-based representations, perhaps due to a large amount of work on scatterplots or using scatterplot-like representations of, e. g., dimensionality reduction or clustering results. Canonical examples, taken from [102], [144], [128], are:

<figure><img src="image_12.png" /></figure>

Generalized Bar Representations are images that represent data with straight visual encodings (bars) that can be arranged on a straight or curved baseline and whose heights or lengths are proportional to the values that they represent. Bars can be stacked, have error bars, or appear as histograms.

Generalized bar charts are the third most common visualization type and have increased in number in recent years. We find that they are among the easiest to identify, and the consistency between coders is at 67%. 3D generalized bar charts are extremely rare, we found them either in the early years or, more recently, to depict data on 3D surfaces such as on a globe. In contrast to pie charts, we include doughnut
charts here as they use the segment length to encode data [114]. Canonical examples, taken from [140], [145], [71], are:

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

**Continuous Color and Grey-scale, and Texture Representations** show structured color patterns across an image or atop a geometric 3D object. These patterns can be evoked by changes in intensity, hue, brightness, and/or saturation. The changes are typically smooth (continuous) but can also show sharp transitions. Typical examples include directional patterns (e. g., flowfields), continuous heatmaps, intensity fields, or, in extreme cases, even a binary image. Coders' consistency was among the lowest across all categories, marked at 42%. Canonical examples, taken from [42], [46], [129], are:

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

**Node-link Trees/Graphs, Networks, Meshes** depict points and explicit connections between these points to convey relationships between data items. Node-link relationships can be found in trees, graphs, networks, and meshes. Both nodes (in a network graph) or vertices (in a wireframe mesh) represent topological points in the structure: nodes represent abstract entities in network graphs, while in geometry, they correspond to vertices), and connections represent the relationships between these nodes. Thus, they are visually similar for the creation of a network-based structure. Examples include node-link networks or trees, topological graphs, or wireframe meshes.

Node positions can be given, e. g., as geospatial locations or be derived from the data (e. g., projections). Connections can be continuous (e. g., in a Reeb graph, as the topological structure is given by showing continuous functions in space) or discrete (e. g., edges in a tree). Representations of this type were the 6th most common representation type and their representation has stayed relatively stable at ≈10–20% of images per year. Most images were 2D in nature, but 27% were 3D images. Canonical representations, taken from [100], [148], [12], are:

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

**Generalized Area Representations** are representations with a focus on areas of 2D space or 2D surfaces, including sub-sets of these surfaces. Areas can be geographical regions or polygons whose size or shape represents data. Areas often feature explicit boundaries and, within, are filled with categorical colors or use contrast in luminance and shading to encode attributes of the areas. Common examples of generalized area charts are pie charts, streamgraphs, (stacked) area charts, treemaps, cartograms, choropleth maps, or violin plots.

Generalized area charts were the 7th most common type of representation type in the images we coded. Most areas were part of surfaces rendered in 2D. Over the years, the proportion of area charts increased by 1–2% every 5 years to just under 10% by 2005 and reached about 15% in 2020. Canonical examples, taken from [20], [113], [105], include:

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

**Generalized Matrix or Grid** separate data into a discrete spatial grid structure. The grid often has rectangular cells but may also use shapes like hexagons or cubes. Elements such as glyphs or a color encoding can appear in the grid cells. Common visualization techniques in this representation type are discrete heatmaps, scarf/strip-plots, space-time cubes, or matrix-based network visualizations.

The grid can vary in resolution. This type includes figures whose underlying grid is part of the data structure but does not include figures where the underlying grid is merely used as a convenient arrangement of sub-sets of the data (as in small multiples and scatterplot matrices). It also does not include visualizations such as a treemap [113], a representation of hierarchical data using nested rectangles. While the treemap has rectangular cells, each cell area corresponds to a data value, and the visualization belongs to the area category.

For all generalized matrix or grid visualizations, other elements such as color or glyphs can appear at their discrete grid positions (e. g., grid-based vector field visualization). Under 10% of all images contained generalized matrices/grids, consistently over the years by 2015 and then increased to 15% in 2020. Of these, 89% were 2D images and 4.2% were 3D (the remaining are unclear). Canonical examples, taken from [58], [94], [46], [90], include:

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

**Glyph-based Representations** commonly include multiple small independent visual representations arranged in space. These representations depict multiple attributes (dimensions) of a data record and often use multiple geometric primitives to encode data. For example, glyphs may consist of a small 3D cuboid where height, width, and depth encode different data dimensions. Other common examples of glyphs are star glyphs, Chernoff faces, whisker glyphs, and so on.

Glyph-based encodings were not particularly frequent (≈5% of all images contained glyphs), and we saw only slightly more glyphs rendered in 2D than in 3D. Glyphs, however, were difficult to identify and our consistency in coding this visualization type was initially only 35%. Canonical examples, taken from [40], [54], [55], [46], are:
<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>

**Text-based Representations** show data (usually text itself) using varying properties of letters or words such as font size, color, width, style, or type. Common visualization techniques for this type are tag clouds, word trees, or typomaps.

Text-based representations were the rarest in our coding. All text-based representations were rendered in 2D. The initial coding consistency was low at 39%, primarily because some coders initially also coded representation where text was used as a data source. Example images, taken from [80], [133], [1], include:

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

## B. Visualization functions

To all images showing one of the visualization types above, we implicitly assigned the function to “showcase a visualization technique.” This function was by far the most common. In addition to this implicit function, we coded screenshots or images of graphical user interfaces (GUIs) and schematic representations. We did not assign additional individual visualization types for images with both functions.

**GUI (screenshots)/User Interface Depiction.** These images are generally screenshots or photos of a system interface. GUIs require the presence of window components or other UI widgets such as buttons, sliders, boxes, scroll bars, pointers (e. g., the hand cursor showing interaction), etc.

Non-WIMP interfaces (e. g., for VR or touch-based applications) are indicated by, e. g., a hand/finger touching a surface or clearly visible interface hardware such as a tablet, a tabletop display, or other types. We found 825 GUI images in total, representing about 12% of all images. The proportion of GUI images over time is relatively stable. Only about 13% are 3D images. Coders find that identifying GUI images is relatively easy and the overall consistency of coders was 70%. Canonical examples, taken from [10], [36], [148], are:

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>

**Schematic Representation and Concept Illustrations.**
These were often simplified depictions that show the appearance, structure, or logic of a process or concept. Typical examples include flowcharts to illustrate algorithms, process diagrams, or sketches. Schematics and illustrations are common in research papers, not just in visualization papers. We coded 1,929 images in this category, 79% of which were 2D. Canonical examples taken from [78], [65], [59] are:

## C. Dimensionality: 2D and 3D

We also coded the spatial dimensionality of the 10 visualization types (Bar, Point, etc.). We labeled a flat representation on a 2D plane without perceived depth as 2D, while we classified images that appeared to be in 3D (or volumetric) space as 3D. To code an image as 3D we looked for depth cues such as occlusion, lighting and shading, parallel and perspective projection, rotation, or any other depth cues.

We also required continuous depth with smooth transitions between depth values. In other words, we did not generally code images as 3D when the content resided in just one or two 2D planes or layers, e. g., an artificial discrete depth. Even this simple exercise turned out to be non-trivial for many images due to the presence or absence of a mixture of depth cues. The use of 2D representations became more common than 3D after 2005 (Fig. 5) and

had been steadily increasing. It is also not surprising that 3D representations were used more broadly for “surface-based representations and volumes.”

<figure><img src="image_42.png" /><figcaption>Fig. 5: Percentage of 2D/3D by total images. The sum in each year is larger than 100% because some images contain both types.</figcaption></figure>

## 5. DISCUSSION OF EXPERT RESEARCH GROUP FINDINGS

Reflecting on our results, we (Expert Researchers) examine the challenges and limitations of our categorization process.

### A. Choosing the level of abstraction for categories

One of our biggest challenges was to find a categorization we could apply without requiring the details of the data each image represents or what construction rules were used.

1) **Bertin's Marks and Channels as Inspiration:** One of our first attempts was to use Bertin's semiology of graphics [11] and, particularly, his marks and visual variables for describing visual designs. This approach resulted in numerous (low-level) codes per image that did not enable us to derive meaningful categories. A bar chart, for example, could be coded as line marks with a length encoding for quantity and a position encoding for category (Fig. 16 in Appx. III-B). Instead, we needed higher-level categories that could apply to entire visualization images Still, Bertin's definition of visual variables and marks inspired the identification of several of our categories: Point primarily reference a position, Bar a length, and Area a two-dimensional size. This result indicates that those design elements (line or position in bars) are different from what we see (length in bars). Therefore, we chose not to label according
to the individual design elements, but more collective visual foci such as *Surface/Volume*, *Text*, and *Cont.-ColorPattern*.

2) **Visualization Techniques as a Typology:** Attempting to abstract from visualization techniques failed as well. Different techniques could result in similar visuals. In addition, identifying techniques sometimes requires knowledge of the data represented. Timeline visualizations, for instance, could only be identified when temporal attribute was indicated on the axis labels (Fig. 17 in Appx. III-B). These additional (often insurmountable) difficulties made the visualization techniques approach impractical. What we retained from this failed attempt, however, was a focus on the essential stimuli. We intentionally chose not to code legends, coordinates, time, labels, or embellishments, and instead focused on the most visually important part of an image that show actual data (as opposed to simpler illustrations).

3) **Classification vs. Categorization:** We began our work with the goal of deriving a classification of images, where each image would be assigned to a single category. In a classification system, each image represents the class as any other. By the time we had experimented (and failed) with the first two approaches (Bertin and visualization techniques), we noticed several problems with trying to come up with a classification: we saw many visual representations that would form a distinct category (e. g., all that used bars to encode data) but also many that would not. While we show representative images in this paper, our VisImageNavigator tool shows many more that are less clearly members of their assigned category. Therefore, we shifted our focus and turned to a modern view of categorization [63], which enables us to assign images to multiple categories, but also identify images as more or less representative of their category.

4) **Is "generalized continuous color/grey-scale/texture representations" a Separate Type?:** Despite the observation that properties of color, such as hue, saturation, and luminance, are visually dominant in data representations, our first two attempts did not include a dedicated visualization type related to color. Instead, we initially coded most continuous and discrete heatmap type encodings under "generalized matrix/grid" based on the logic that these encodings are applied on pixel-level grids. In our iterative coding refinement phase, many discussions centered on when a continuous color-based encoding should be coded as a grid, especially when it did not look like a grid. When continuous color scales were applied to 3D geometries, such as streamlines, e. g., the matrix/grid encoding no longer seemed appropriate. Therefore, after many discussions, we created a distinct category to recognize continuous colormaps. To distinguish it from "generalized matrix and grid", we required the color mapping to be systematic and not a result of illumination or an author-chosen categorical representation. At this point, we already had a separate category for texture-based representations that we eventually merged with the new color category as both focus on revealing continuous patterns.

5) **The Influence of Domain Knowledge:** Throughout the creation of our categories, we struggled with identifying and suppressing the influence of our individual knowledge. The team members (ExpertResearchers) with a background in volume/surface-based techniques, e. g., often mentioned

<figure><img src="image_43.png" /><figcaption>Fig. 6: Challenging cases. (a) We choose to code geospatial data as *Area* and route connections *Node-link*. (b) We choose to code the terrain *Surface/Volume* and the arrows on top *Glyph*. Images from (a) Overbye et al. [98] and (b) Jenny et al. [64].</figcaption></figure>

<figure><img src="image_44.png" /><figcaption>Fig. 7: Challenging cases of coding *Schematic*. (a) Is this a glyph-based or schematic representation? (b) We choose not to code the visualization type inside *Schematic*. Images from (a) Hlawitschka & Scheuermann [55] and (b) Daniels et al. [35].</figcaption></figure>

specific techniques, tasks, or goals for images while creating our categories, to which the other team members had little or no relation. Similarly, our definitions of glyphs were not aligned between experts with a background in flow visualization and those in abstract visualization. In practice, we constantly reminded ourselves to suppress our knowledge of non-visual aspects of the images to resolve ambiguities. We resolved many issues through discussions of individual images and revising our type codes iteratively with examples and counter-examples.

## B. General coding ambiguity

While we adopted a view of categorization with fuzzy boundaries, we engaged in discussions to resolve ambiguities for frequent cases. The most important challenges were:

1) **Surfaces and Volume Rendering:** Early on, we separated surface and volume renderings into two separate codes. Several of us, however, found it challenging to perceive differences between the two. Volumes, for example, can be rendered semi-transparently (looking cloud-like) or opaquely, the latter having the appearance of a surface. We decided that "recognizing" an image type at this level of detail (e. g., whether surfaces are rendered from voxel or mesh data) was unreliable and may not even be important for describing a figure's visual content as the underlying algorithmic techniques transparent to viewers. Eventually, high-level concepts can directly contribute to reasoning [24], [132]. Applying this principle and removing the specific details of a technique enabled us to focus on what all instances of a given type have in common, and we thus combined surface and volume techniques.

2) **Ambiguous Area-based Images:** We tried to avoid using data types in our categorization because we wanted to focus consistently on visual impressions. We originally had a "cartographic map" category but removed it because
it focused on a specific data type and technique. Instead, we decided that depicting areas and their relationships was the underlying principle for many cartographic maps and related techniques such as area charts, stream graphs, etc. (Fig. 6). There were, however, exceptions. Route maps, e. g., where lines indicate a direct route, were coded separately as networks because the routes encoded topological relationships (Fig. 6a). One difficulty we encountered was the distinction between a “map” (cartographical map) and a “terrain” (wireframe or surface) (Fig. 6b). Conceptually, these are very similar. Using visual appearance as our guide, however, map images generally appeared to delineate distinct areas, while terrains showed continuous surfaces with elevations. Another map-related difficulty arose when maps were used as a reference structure for data representations layered on top, akin to how gridlines are used on scatterplots. In these cases, we had to derive elaborate procedures for consistently treating reference structures, e. g., axis lines. If, in addition to maps, other visual encodings are present, we decided to code these if removing them changes the meaning or information conveyed in the image. These decisions, however, can be difficult and resulted in some coding inconsistencies.

### 3) Ambiguous Schematic Images:
A large number of figures are schematic representations or concept illustrations. It is often challenging to differentiate between schematics and a demonstration of a visual encoding technique. We had to abandon our initial goal of ignoring what data was encoded to be able to judge whether the representation showed a “toy” dataset.<fnref n="1" /> While toy datasets are common in schematic representations, the frequent absence of context, such as coordinate axes, labels, or scales, made their identification difficult. Many figures did not depict scales, which aligns with observations by Cleveland and McGill [32] in their review of the use of graphics in other scientific journals.

We also struggled with using annotations in figures as identification criteria for schematics. Fig. 7a, for example, can be coded either as glyph-based (it shows a mathematical tensor) or as a schematic (it illustrates the authors’ design idea or a mathematical function)—a majority of us chose to code “schematic representations and concept illustrations”. Schematic images are often meant to be particularly pedagogical and, thus, include several labels and arrows or other annotations. We experimented with specific coding guidelines in which we considered whether or not, after the removal of annotations, we could still see an example of a visualization type—in the former case, the image would have been a schematic. Later, we agreed that the appearance of labels and annotations to explain an image did not automatically mean the image was a schematic. Our general heuristic for schematics involved establishing if we saw (1) a well-known (or toy) dataset, (2) a pedagogical purpose, and (3) an illustration of a concept.

### 4) Ambiguous Glyph Cases:
Glyphs are notoriously difficult to define. Recent attempts have emphasized different aspects of delineating a glyph from other encodings. Fuchs et al. [45] defined data glyphs as “data-driven visual entities, which

<figure><img src="image_45.png" /><figcaption>Fig. 8: Challenging cases of coding *Glyph*. (a) The arrows show the 2D position as well as orientation. (b) StarGlyph-like dimensional comparisons and thus a type of *Glyph*. Images from (a) Bian et al. [13] and (b) Fanea et al. [40].</figcaption></figure>

<figure><img src="image_46.png" /><figcaption>Fig. 9: Challenging cases of multiple coding ambiguity: we did not code *Surface/Volume* and only chose the primary code, which differentiates these visualization techniques (images from (a) Weiskopf et al. [139] and (b, c) Garcke et al. [46]).</figcaption></figure>

use different visual channels to encode multiple attribute dimensions.” Borgo et al. [14] followed Ward [134] to define glyphs as “a visual representation of a piece of data where the attributes of a graphical entity are depicted by one or more attributes of a data record.” Munzner’s [92] definition is broad and requires a data encoding to be assembled out of multiple marks that encode data. Every bar in a stacked bar chart [135], e. g., would be a “microglyph” according to Munzner, because it is a composite object of multiple length-encoding marks. In our coding, we used a mix of the given definitions because, for all coders, it was difficult to dissociate their research background on glyphs from the coding.

While many existing encodings focus on describing a (single) glyph, glyph-based visualization is often used in practice when multivariate data needs to be presented simultaneously.

A challenge is thus to determine when a primitive becomes a glyph. There is no agreed-upon threshold of how many data dimensions are encoded in a glyph and when a glyph becomes a chart. There seems to be, however, a general consensus that a glyph requires a certain level of complexity to be categorized as such. We labeled an image as a glyph-based representation if multiple representations of data points represent both position and additional data dimensions using color, shape, or other geometric primitives. Fig. 8 illustrates two difficult cases.

Finally, we first had a separate “tensor glyph” category. This category, however, required knowledge of affine transforms applied to a basic shape, which could not be concluded from pure perceptual observations. So, we removed this category.

<footnote n="1">¹Here, a “toy” dataset is a synthetic, exemplar dataset to demonstrate a principle for ease of understanding.</footnote>
<figure><img src="image_47.png" /><figcaption>(a) Node-link+Surface.</figcaption></figure>

<figure><img src="image_48.png" /><figcaption>(b) Line, Point, and Surface.</figcaption></figure>

<caption>Fig. 10: Challenging cases of multiple coding ambiguity: two codes signify different aspects of the data that can be separated to stand independently. As a result, multiple codes apply (images from (a) Suits et al. [120] and (b) Bach et al. [5]).</caption>

### C. Multiple encoding ambiguity

Many images showed multiple visual encodings (e. g., Fig. 9 and Fig. 10), which is one of the primary reasons for the coding inconsistencies we encountered. We agreed to code multiple visualization types if these types were distinctive, could be perceived clearly, and were essential. If multiple visual designs, layered or nested, could be distinguished from one another, we tagged more than one encoding, e. g., Cont.color + Glyph for Fig. 9b and node-link + surface for Fig. 10a.

We also decided when (not) to check multiple labels (e. g., Fig. 11). For example, points appear in both confidence interval and line charts. For confidence intervals, we chose to code both “generalized bar representations” and “generalized point representations” as primary categories because the bar length encodes the interval, and because the confidence interval is not always symmetric (Fig. 11b). In contrast, we considered those points that sometimes are added to line charts (Fig. 11c) as an annotation and did not code them. These decisions were difficult, sometimes inconsistent, and error-prone (e. g., the dot for the average could be an annotation when the drawing of the error bars is symmetric). We identified conflicting understandings through many hours of conversations on individual and difficult images, which often required some coders to compromise their opinion.

## 6. VALIDATION EXPERIMENT WITH EXPERT USERS

Having finished coding the images, we next designed a controlled experiment to explore whether or not viewers with visualization experience can apply our VisTypes coding schema to categorize images (pre-registered at osf.io/dxjwt and with IRB approval from The Ohio State University, ID: 2022B0363). We had two high-level working hypotheses:

* First, we expected that participants could use our VisTypes category descriptions and identify images with better-than-chance agreement.

* Second, we expected certain image categories to be more difficult for participants and result in less agreement (matrix grid, glyphs, and continuous color & greyscale-based representations) based on our own experience.

### A. Methods

**Tasks.** We assigned each participant a single visualization type category, *c*. In each trial, we showed the participant one image from the assigned or another category (Fig. 12 shows the

<figure><img src="image_49.png" /><figcaption>(a) Bar.</figcaption></figure>

<figure><img src="image_50.png" /><figcaption>(b) Bar + Point.</figcaption></figure>

<figure><img src="image_51.png" /><figcaption>(c) Line.</figcaption></figure>

<caption>Fig. 11: Challenging cases of coding statistical charts: We coded Bar only in the Box-whisker plot in (a) because it shows a group of quartiles. In (b), we coded Point when points show the mean values. In (c), we excluded Point, because these points function as an annotation to the Line and do not represent the underlying data (images from (a) Chlan and Rheingans [28], (b) Ziemkiewicz and Kosara [150], and (c) Lakare et al. [75]).</caption>

interface we used during the experiment), and let them answer the yes/no question: “Does this image belong to category *c*?”

**Image Selection.** We pre-selected 18 images for each visualization type, for a total of 180 images for the 10 types. Each set contained images with the same difficulty levels based on the two coders who had initially labeled them: 2x easy (6 samples, hardness level L=1), easy-neutral (6 samples, L=2), 2x neutral (2 samples, L=3), and easy/neutral-hard or 2x hard (4 samples, L=4). All images had been previously identified as having a single visualization type only.

**Study Design and Order of Trials.** We used a between-subject design with the visualization type (10 categories) as the only independent variable. Dependent variables were *agreement* with our assigned labels for each image and task completion *time*. Each participant completed 36 trials, including 18 images from the assigned category (target) and 18 from the other (non-target) categories. For example, when their assigned category was “generalized bar representation,” they saw 18 Bar images and 18 images from 9 other categories, with 2 randomly selected in each of these 9 categories. We ordered these 36 images randomly per participant.

**Participants.** We recruited 110 participants with some form of visualization education (e. g., having taken or currently taking a visualization course) to experts (i. e., researchers or practitioners with a longer practice in visualization). We call these participants *ExpertViewers* to differentiate them from ourselves (*ExpertResearchers*). We conducted a power analysis (osf.io/dxjwt) to address possible biases of the yes/no test [72] and to determine the number of participants. We also recruited participants with diverse visualization backgrounds; i. e., people working in visual analysis, encoding methods, evaluation, simulation, and algorithmic foundations, experienced with different data types and stages of the analysis pipeline.

After the trials, we asked participants to describe their answers for two images they had seen: one for an answer that agreed with our coding and one that did not (if all answers agreed or did not agree, we randomly chose two images). We also collected optional free-text comments.

### B. Results

We collected 3,960 yes/no answers from 110 participants with self-reported expertise levels being ‘expert’ (49x), ‘advanced’ (23x), ‘intermediate’ (22x), and ‘novice’ (16x). Among them,
<figure><img src="image_52.png" /></figure>

<figure><img src="image_53.png" /><figcaption>Fig. 12: **Empirical evaluation workflow:** We invited 110 participants (ExpertViewers) to participate in the study. They first saw a consent form and completed a demographic questionnaire. Then we gave them the definition (corresponding to the text in italics in Sect. 4) and example images for their assigned category. Next, they were invited to familiarize themselves with the other categories, their definitions, and example images. The following screen introduced the study interface. Each participant then completed ten training trials before moving on to the 36 study trials. Readers can try the study at go.osu.edu/vistype.</figcaption></figure>

46 participants held a ‘Ph.D. degree’, 41 were enrolled in a ‘graduate school,’ 20 in a ‘college,’ and 3 in a ‘professional school.’ The participants’ self-reported gender distribution was 35 women, 73 men, and 2 others (roughly similar to the gender distribution of IEEE VIS authors now [123]). We also collected 215 image-specific and 32 general comments about the study.

1) **Agreement:** Our first hypothesis, i. e., that participants could apply our category definitions, is supported. Our second hypothesis on the difficulty levels is also reflected in the degree of agreement between participants. Participants could categorize images with an agreement above 80% for all categories (Fig. 13). *Text* (94.4%), *Node-link* (90.7%), *Surface/Volume* (89.9%), and *Point* (88.3%) had the highest percentage of agreement. Matching our expectation, *Cont.-ColorPattern* (85.1%), *Glyph* (85.9%), and *Grid* (85.9%) scored slightly lower, and *Area* (87.1%), *Line* (87.3%), and *Bar* (87.9%) were in the middle. The light-grey bars on the bottom of Fig. 13 represent the false-positive rates (Type I error, where a viewer incorrectly affirmed an image as the target type) and the false negative rates (instances where a viewer erroneously denied an image as the target type). Here, the categories of color/greyscale (9.6%), *Area* (9.6%), and *Bar* (9.1%) had the highest false-positive rates, in contrast to grid (8.3%), *Glyph* (7.8%), *Point* (6.6%), *Surface/Volume* (6.1%), and *Node-link* (5.3%). *Text* had the lowest incidences of both false-positives (3.5%) and false-negatives (2.0%). Overall, *Glyph* (6.3%), *Grid* (5.8%), and *Cont.-ColorPattern* (5.3%) had the highest false negatives, further confirming our hypothesis that these types were more difficult to recognize compared to other types, such as *Line* (5.0%), *Point* (5.0%), *Surface/Volume* (4.0%), *Area* (3.2%), *Bar* (3.0%), and *Node-link* (4.0%). Fig. 24 and Fig. 25 in Appx. VI show the image examples that ExpertViewers with a Ph.D. degree mostly agree with our labels.

2) **Participants’ Comments:** From the 247 comments made by the participants (ExpertViewers), we found their comments aligned well with our perceived difficulties in coding these images, as discussed in Sect. 5. All comments we quote in this section are from the ‘advanced experts’ holding a ‘Ph.D. degree,’ unless specified otherwise.

*The Influence of Domain Knowledge.* In the coding phase,

<figure><img src="image_54.png" /><figcaption>Fig. 13: **Empirical Results on Agreement Percentage for Each Visualization Type:** the dots indicate the mean accuracy of each category; the error bars show 95% confidence intervals.</figcaption></figure>

we constantly reminded ourselves to suppress our knowledge of nonvisual aspects of the images to resolve ambiguities. We can see that a similar difficulty arose for participants who left comments that indicated an inclination to consider specific techniques, data, or even preexisting definitions in their coding: “*The point-based classification I followed is based on one information visualization book. I am unfamiliar with the scientific visualization literature. If ‘point-based’ term is also used by the scientific visualization community, my choices may be random for such a type of visualization.*”

Some of the images would be inherently ambiguous. Participants commented on the background knowledge about techniques and disagreed with some example techniques appearing in the training phase. For example, we coded violin plots as *Area* because a shaded area often represents a kernel density estimate (KDE), where the area under the curve is meaningful in the plot. However, a participant mentioned: “*I would like to additionally comment violin-plots [...] If a probability distribution is shown using a histogram (which is often done to ease implementation), then I would call it a bar chart technique. If proper KDE is computed and shown, it becomes a line-based technique (in combination with the original boxplot, i. e., bar chart technique). That’s why I said*
*no for the violin plot example.*" So here, this participant took the graphical elements of lines, which only signify boundaries, instead of the essential stimuli of *area* bounded by these *lines*.

We also saw that sometimes participants used a *functional* view of an image rather than our interpretation based on essential stimuli. Several participants explained, for example, their encoding of a set of stacked circles as a generalized bar chart rather than a point-based representation, as we had done. A participant commented: "*I thought that the stacked circles represent the quantity of the x-axis, so I consider it is more close to bar-chart representation.*"

*The Influence of Data.* Several participants commented on the fact that some images in the study had a resolution too low to read text, indicating that they also wanted more information on some images to help them make correct decisions. In our own coding process, we resolved many of these issues through discussions on individual images and iterative revision of our image labels with examples and counter-examples for image coding. We also sometimes were drawn to text in legends and labels describing the data to help us interpret images, especially when categorizations became difficult, even though our goal was not to consider the represented data type.

*Coding Ambiguities.* Participants also struggled with the type of meshes and surfaces. We had originally put meshes in the *Surface/Volume* and later in the category of *Node-link* to emphasize the connectivity rather than the underlying geometry. The same challenge was reflected in participants' comments, e.g.: "*For my category of 'surface-based or volume,' I found the differentiation to the group that contains 'mesh' difficult for some of the shown examples, as surface extractions are frequently rendered as meshes, and meshes frequently depict a surface of some kind.*" Another participant stated: "*In CG [computer graphics] context, meshes describe a discretization of a 2D manifold. Displaying such a manifold is just rendering this discretization, since no data given on the manifold is visualized. Think of 1D: if you discretize a straight line into connected line segments, this is not a visualization of a linear graph. There is no underlying data given on that graph, except if you want to visualize the discretization.*" This point was reflected in our own conversation. Since the connecting line in a node-link diagram does not always map directly to underlying data, one may consider a mesh to be closer to node-links, even though it is an abstract representation of a surface.

*Glyph Coding.* Some participants also mentioned the challenge to determine when a primitive becomes a glyph as described in Sect. 5-B4. There is no agreed-upon threshold of how many data dimensions are encoded in a glyph and when a glyph becomes a chart. There seems to be, however, a general consensus that a glyph requires a certain level of complexity to be categorized as such. Yet reaching a consensus on the level of complexity is challenging, as echoed by some participants in our study. Commenting on Fig. 8a, one stated that "*each arrow is the same and it is only their orientation + position that shows a property. I would not categorize 2-3 dimensions as multiple (the definition given in the study).*" In a similar way, some of the tensor fields provided in the study examples are, I believe, somewhat pushing the definition. For the vector field, I consider the spiral as part of the glyph."

<figure><img src="image_55.png" /><figcaption>Fig. 14: **Challenging cases of coding implicit and explicit stimuli.** (a) could show a tree though the edges are absent or a bar graph. (b) is too abstract to code without the context. Images from (a) Weiskopf et al. [138] and (b) Suits et al. [120].</figcaption></figure>

One participant mentioned the challenge to label glyphs: "*In doing this study I realized that glyph may have a slightly different definition when thinking about glyphs for abstract data (which typically vary in their shape) as compared with glyphs for continuous spatial data (that might just be little icons, all the same, that vary only in position/orientation). I used more of the abstract data definition.*" Another participant wrote "*I thought that the stacked circles represent the quantity of the x-axis, so I consider it is more close to bar-chart representation.*"

*Implicit vs. explicit drawings, and continuity.* Our community uses implicit edges to characterize trees and graphs [95], [110], e. g., treemaps use containment where edges are explicit [8]. An expert called out the ambiguous case between "generalized node-link trees/graphs, networks, meshes" and "generalized bar representations", where a figure can be a "tree" with edges implicitly shown or a "bar" otherwise. The participant (who has a Ph.D. degree) stated: "*While this could be understood as a tree structure, this tree just shows the underlying space partitioning of spherical polar coordinates... e. g., one could argue that this is 3D version of a sunburst chart and thus a generalized bar chart.*" Part of the reason, as the participant explained "*it is unclear whether the data is visualized at all (why are some of the cutoff pyramids visible and not others).* So, without knowing what the domain is and what the data are, it is difficult to categorize."

### C. Summary

Our findings and participant results reinforce that experts can effectively utilize our categorization methods to distinguish between different categories. The presence of ambiguous cases should not be seen as a drawback or a factor that undermines the validity of our typology or the study's overall results. Instead, we interpret this ambiguity as evidence of the diverse uses of knowledge and context prevalent in visualization representations in our community. In addition, categorization specifically accounts for this ambiguity and the context-specific reinterpretation of category assignments.

## 7. TYPOLOGY USE CASE SCENARIOS

Our image-based VisTypes from the visual appearance of the essential stimuli can be useful for various evaluation purposes and downstream tasks. Here, we put our theory into practice and conjecture future research activities and relevant questions.
## A. Quantification and qualification of visualization: From point evaluation to large-scale evaluation

Measuring the effectiveness of visualization images is at the core of visualization research. Many controlled lab experiments are by themselves “point evaluation,” a comparison of often elegant, yet simplified representations. Here, in order to understand why visualization techniques do or do not work, and to suggest possibilities for new design, it is important to move beyond point evaluation and to introduce generalizable scientific measurement of human visual and behavior principles. In particular, the commonly conclusion that one technique is better than the other may not always be an effective way to think about visualization design because the techniques under investigation may have removed from context uses. Our typology offers an opportunity to reimagine the theory of evaluation. For example, we can broaden empirical data collection by employing a more diverse set of techniques when comparing visualization techniques. This broader approach ensures that evaluation recommendations are not constrained by a limited number of data points or techniques. Moreover, leveraging large data-driven evidence in large-scale empirical studies may also enables the formation of new hypotheses that may not emerge from smaller-scale empirical studies, which often isolate variables but lack the complexity of real-world, context-rich uses. Consequently, the richness of VisTypes can serve as a valuable benchmark, guiding our interpretations of design. An example would be to extend Cleveland and McGill’s 1984 paper [31] to study a broader set of *Bar*.

## B. Broadening cross-cutting science beyond human information-processing systems

Visualization transcends traditional disciplinary boundaries, drawing significantly on advances in cognitive and vision science. Our visualization images contain rich stimuli representations that may help gauge and compare information processors in settings beyond natural environments [104]. This trend has been reflected, for example, by understanding human-AI differences for chart interpretation in large-language models and supervised learning algorithms [9], [50], [67]. In these cases, AIs can be independent observers or collaborators side-by-side with human observers. Our typology can provide a benchmark image set to assist the tasks of quantifying observers.

## C. A supporting tool for a standard in visualization

Our work can also facilitate community discussions on standardizing visualization processes. For instance, for the coordinate system we mentioned in *Sect. 1*, we could study how bars appear in the *Bar* category, observing that a Sunburst chart is just the result of positioning bars through a polar transformation. Thus, we could develop a set of transformation functions to describe the morphological structures in visual layout, to inform design, e. g., Wilkson’s GoG [141].

We can use our data to understand VIS as a scientific discipline. *Essential stimuli* enable visualization scholars like us to engage in historical analysis of visualization evolution.

Our typology can facilitate studying how the field evolved over time and compared to other fields. For example, we can observe and compare the evolution of rendering styles and methods in a single category, compare design styles across categories, and attempt to reason about possible influences such as individual papers, designers, or rendering hardware. Finally, when writing an overview article, a textbook, or a lecture series on visualization, categorizations based on visual appearance can highlight the variety of approaches to specific techniques or identify aspects essential to a technique.

## D. Limitations and future work

Our work is certainly not without limitations. The most important one is that our categorization is influenced by the research team behind it and the dataset we use. It is also influenced by our background and experience in visualization, the prior categorization we have attempted, and the techniques we study daily. As such, we release our codes for the community to explore the data to develop alternative categorizations. Furthermore, it might be interesting, for instance, to see how D3.js [17] and SVG [7], [56], and their examples have influenced the types of images published in the community. Finally, it would be also useful to examine how type codes may vary considering data, tasks, and contexts.

## 8. CONCLUSION

Our image-based typology is a categorization of the visual appearance of essential stimuli in our research field and how viewers can use these stimuli to think and drive concepts. The premise is audacious: the message conveyed by visualization rooted in the perception of concepts and the knowledge they contain could be the foundation for all visual sensemaking. While scholars attempt to evaluate visual representations, bridge distinct information processors, and educate the general public and our students, VisTypes can bring representations into a space where ideas can be discussed, quantified, and compared. To this end, our VisTypes categorize visualizations, enables designers and theorists to cope with visual complexity such that structures and forms of visualizations can be compared, quantified, and studied within and between types. The vast amount of image data within our community has greatly advanced our understanding of how design shapes these types for downstream tasks, enabling a deeper understanding of the message conveyed by types.

Our original motivation was partially rooted in the fact that our community’s intellectual contributions were, for a long time, organized in three narrow silos (formerly known as “InfoVis,” “SciVis,” and “Visual Analytics”). Yet visualization is a much richer area, with many facets that go far beyond such a simplified consideration [61], [79]. Don Norman says that it is “things that make us smart” [96]. He argues that our ability to reason about complex phenomena is only possible when we externalize our thoughts, data, and concepts. Arguably, data visualization is one of the most compact and sophisticated ways to support ‘thinking,’ and the visualization community creates intriguing and intricate ways for such visual depictions of data. By attempting to categorize this rich body of work directly
from its source, the images, and the stimuli in publications, we
shed further light on the breadth of our field. Visualizations
are increasingly the currency for understanding intelligent
information-processing systems, and we hope that VisTypes
can expand our understanding of visual design rules for all
information processors, be they natural or artificial.

OPEN ACCESS STATEMENT

go.osu.edu/vistypes has the VisTypes dataset. osf.io/
dxjwt has the experimental details, the intermediate and
final codes of the 6,833 images, and participants' codes.
go.osu.edu/vistypesdatatypepaperfigures in the Suppl.Mat
subfolder has Google CoLab code to analyze our data
online and recoded MASSVIS [15] types with our typology.
Our VisImageNavigator (visimagenavigator.github.io)
provides a web-based interactive tool to explore and request
changes to the tagged VisTypes image data. and perform image
types with papers and their keywords (Fig. 15).

IMAGE COPYRIGHTS

We, as authors, state that the images in Fig. 3–5, 13, and the
bottom-row plots in Fig. 1 (“frequency/temporal evolution”) are
and remain under our own copyright, with the permission to be
used here. They are also available under the Creative Commons
Attribution 4.0 International (©① CC BY 4.0) license and
share them at osf.io/dxjwt. All remaining images in the
paper are © IEEE, with permission to be used here.

ACKNOWLEDGMENTS

We thank Drew Dimmery at the Hertie School for his help
with our power analysis. We also thank Han-Wei Shen for
being part of ExpertResearchers in the initial coding phase. We
appreciate the expertise and comments made by ExpertViewers
in our validation. We also appreciate the help of about 60
students from the Data Visualization course (CSE5544, Fall
2022/Spring 2023) at The Ohio State University, who piloted
our tools. The work is supported in part by NSF IIS-1302755,
NSF CNS-1531491, and NIST-70NANB13H181. However, any
opinions, findings, conclusions, or recommendations expressed
in this material are exclusively our own.

REFERENCES

[1] S. Afzal, R. Maciejewski, Y. Jang, N. Elmqvist, and D. S. Ebert, "Spatial text visualization using automatic typographic maps," *IEEE Trans Vis Comput Graph*, vol. 18, no. 12, pp. 2556–2564, 2012. doi: 10/f4ftpq

[2] W. Aigner, S. Miksch, H. Schumann, and C. Tominski, *Visualization of Time-oriented Data*. London: Springer, 2011. doi: 10/b834rc

[3] R. Amar, J. Eagan, and J. Stasko, "Low-level components of analytic activity in information visualization," in *Proc. InfoVis*. Los Alamitos: IEEE CS, 2005, pp. 111–117. doi: 10/bwrm27

[4] T. M. Athawale, B. Ma, E. Sakhaee, C. R. Johnson, and A. Entezari, "Direct volume rendering with nonparametric models of uncertainty," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1797–1807, 2021. doi: 10/ghgt7n

[5] B. Bach, C. Shi, N. Heulot, T. Madhyastha, T. Grabowski, and P. Dragicevic, "Time curves: Folding time to visualize patterns of temporal evolution in data," *IEEE Trans Vis Comput Graph*, vol. 22, no. 1, pp. 559–568, 2016. doi: 10/gg2c52

[6] L. Battle, R. Chang, J. Heer, and M. Stonebraker, "Position statement: The case for a visualization performance benchmark," in *Proc. DSIA*. Los Alamitos: IEEE CS, 2017, pp. 6:1–6:5. doi: 10/gdm499

[7] L. Battle, P. Duan, Z. Miranda, D. Mukusheva, R. Chang, and M. Stonebraker, “Beagle: Automated extraction and interpretation of visualizations from the web,” in *Proc. CHI*. New York: ACM, 2018, pp. 594:1–594:8. doi: 10/ggtmxf

[8] B. B. Bederson, B. Shneiderman, and M. Wattenberg, “Ordered and quantum treemaps: Making effective use of 2D space to display hierarchies,” *ACM Trans Graph*, vol. 21, no. 4, pp. 833–854, 2002. doi: 10/fw3shv

[9] A. Bendeck and J. Stasko, "An empirical evaluation of the GPT-4 multimodal language model on visualization literacy tasks," *IEEE Trans Vis Comput Graph*, 2025, to appear. doi: 10/np3q

[10] L. D. Bergman, B. E. Rogowitz, and L. A. Treinish, “A rule-based tool for assisting colormap selection,” in *Proc. Visualization*. Los Alamitos: IEEE CS, 1995, pp. 118–125. doi: 10/cqp28q

[11] J. Bertin, *Semiology of Graphics: Diagrams, Networks, Maps*. Esri, 1983.

[12] P. Bhaniramka, R. Wenger, and R. Crawfis, “Isosurfacing in higher dimensions,” in *Proc. Visualization*. Los Alamitos: IEEE CS, 2000, pp. 267–273. doi: 10/bv4sqc

[13] R. Bian, Y. Xue, L. Zhou, J. Zhang, B. Chen, D. Weiskopf, and Y. Wang, “Implicit multidimensional projection of local subspaces,” *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1558–1568, 2021. doi: 10/ghgt6z

[14] R. Borgo, J. Kehrer, D. H. Chung, E. Maguire, R. S. Laramee, H. Hauser, M. Ward, and M. Chen, “Glyph-based visualization: Foundations, design guidelines, techniques and applications,” in *EG State of the Art Reports*. Goslar: EG Assoc, 2013, pp. 39–63. doi: 10/f3stv

[15] M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister, “What makes a visualization memorable?” *IEEE Trans Vis Comput Graph*, vol. 19, no. 12, pp. 2306–2315, 2013. doi: 10/f5h3pd

[16] ——, “MASSVIS dataset,” Online dataset: http://massvis.mit.edu/, visited Sept. 2024.

[17] M. Bostock, V. Ogievetsky, and J. Heer, “$D^3$: Data-driven documents,” *IEEE Trans Vis Comput Graph*, vol. 17, no. 12, pp. 2301–2309, 2011. doi: 10/b7bhhf

[18] W. C. Brinton, *Graphic Presentation*. New York: Brinton Assoc., 1939, urn: urn:oclc:record:1045601113.

[19] K. W. Brodlie, “Visualization techniques,” in *Scientific Visualization: Techniques and Applications*, K. W. Brodlie, L. A. Carpenter, R. A. Earnshaw, J. R. Gallop, R. J. Hubbold, A. M. Mumford, C. D. Osland, and P. Quarendon, Eds. Berlin: Springer, 2012, ch. 3, pp. 37–85. doi: 10/bbrw7g

[20] C. Bu, Q. Zhang, Q. Wang, J. Zhang, M. Sedlmair, O. Deussen, and Y. Wang, “Sinestream: Improving the readability of streamgraphs by minimizing sine illusion effects,” *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1634–1643, 2021. doi: 10/ghv58n

[21] L. Byron and M. Wattenberg, “Stacked graphs–geometry & aesthetics,” *IEEE Trans Vis Comput Graph*, vol. 14, no. 6, pp. 1245–1252, 2008. doi: 10/dq8747

[22] A. Cairo, *The Functional Art: An Introduction to Information Graphics and Visualization*. Berkeley: New Riders, 2012, urn: urn:oclc:record:1036960105.

[23] J. Canny, “A computational approach to edge detection,” *IEEE Trans Pattern Anal Mach Intell*, vol. 8, no. 6, pp. 679–698, 1986. doi: 10/fn3fdk

[24] C. Chen, O. Li, D. Tao, A. Barnett, C. Rudin, and J. K. Su, “This looks like that: Deep learning for interpretable image recognition,” in *Proc. NeurIPS*. Red Hook, NY: Curran Associates, Inc., 2019, pp. 8930–8941. doi: 10/kg6p

[25] C. Chen, F. Ibekwe-SanJuan, E. SanJuan, and C. Weaver, “Visual analysis of conflicting opinions,” in *Proc. VAST*. Los Alamitos: IEEE CS, 2006, pp. 59–66. doi: 10/cf4bwf

[26] J. Chen, M. Ling, R. Li, P. Isenberg, T. Isenberg, M. Sedlmair, T. Möller, R. Laramee, H.-W. Shen, K. Wünsche, and Q. Wang, “IEEE VIS figures and tables image dataset,” Dataset and online search, visimagenavi gator.github.io, 2020. doi: 10/kdq

[27] J. Chen, M. Ling, R. Li, P. Isenberg, T. Isenberg, M. Sedlmair, T. Möller, R. S. Laramee, H.-W. Shen, K. Wünsche, and Q. Wang, “VIS30K: A collection of figures and tables from IEEE visualization conference publications,” *IEEE Trans Vis Comput Graph*, vol. 27, no. 9, pp. 3826–3833, 2021. doi: 10/gmsvxd

[28] E. B. Chlan and P. Rheingans, “Multivariate glyphs for multi-object clusters,” in *Proc. InfoVis*. Los Alamitos: IEEE CS, 2005, pp. 141–148. doi: 10/fv24bk

[29] J. H. Claessen and J. J. Van Wijk, “Flexible linked axes for multivariate data visualization,” *IEEE Trans Vis Comput Graph*, vol. 17, no. 12, pp. 2310–2316, 2011. doi: 10/b66s9m

[30] W. S. Cleveland, “Graphs in scientific publications,” *The American Statistician*, vol. 38, no. 4, pp. 261–269, 1984. doi: 10/c3j8hz
[31] W. S. Cleveland and R. McGill, "Graphical perception: Theory, experimentation, and application to the development of graphical methods," *J American Stat Asso*, vol. 79, no. 387, pp. 531-554, 1984. doi: **gdvmwd**

[32] ——, "Graphical perception and graphical methods for analyzing scientific data," *Science*, vol. 229, no. 4716, pp. 828-833, 1985. doi: **10/fkhq59**

[33] C. Collins, G. Penn, and S. Carpendale, "Bubble sets: Revealing set relations with isocontours over existing visualizations," *IEEE Trans Vis Comput Graph*, vol. 15, no. 6, pp. 1009-1016, 2009. doi: **10/c99shd**

[34] X. Dai and R. Prout, "Unlocking Super Bowl insights: Weighted word embeddings for Twitter sentiment classification," in *Proc. MISNC, SI, DS*. New York: ACM, 2016, pp. 12:1-12:6. doi: **10/mk6h**

[35] J. Daniels, II, E. W. Anderson, L. G. Nonato, and C. T. Silva, "Interactive vector field feature identification," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1560-1568, 2010. doi: **10/fq6d7n**

[36] K. Dasu, K.-L. Ma, J. Ma, and J. Frazier, "Sea of Genes: A reflection on visualising metagenomic data for museums," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 935-945, 2021. doi: **10/ghv58j**

[37] D. Deng, Y. Wu, X. Shu, M. Xu, J. Wu, and S. F. Y. Wu, "VisImages: A large-scale, high-quality image corpus in visualization publications," *IEEE Trans Vis Comput Graph*, vol. 29, no. 7, pp. 3298-3311, 2023. doi: **10/k8vt**

[38] E. Dimara and J. Stasko, "A critical reflection on visualization research: Where do decision making tasks hide?" *IEEE Trans Vis Comput Graph*, vol. 28, no. 1, pp. 1128-1138, 2022. doi: **10/gnb9b2**

[39] M. Edmunds, R. S. Laramee, G. Chen, N. Max, E. Zhang, and C. Ware, "Surface-based flow visualization," *Comput Graph*, vol. 36, no. 8, pp. 974-990, 2012. doi: **10/f4mhfp**

[40] E. Fanea, S. Carpendale, and T. Isenberg, "An interactive 3D integration of parallel coordinates and star glyphs," in *Proc. InfoVis*. Los Alamitos: IEEE CS, 2005, pp. 149-156. doi: **10/fvdg5f**

[41] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," *IEEE Trans Pattern Anal Mach Intell*, vol. 32, no. 9, pp. 1627-1645, 2010. doi: **10/fgv7fd**

[42] Z. Feng, H. Li, W. Zeng, S.-H. Yang, and H. Qu, "Topology density map for urban data visualization and analysis," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 828-838, 2021. doi: **10/k8sf**

[43] D. Fisher and M. Meyer, *Making Data Visual*. O' Reilly Media, 2018.

[44] L. Franke and D. Haehn, "Modern scientific visualizations on the web," *Inf*, vol. 7, no. 4, pp. 37:1-37:35, 2020. doi: **ghnnm2**

[45] J. Fuchs, F. Fischer, F. Mansmann, E. Bertini, and P. Isenberg, "Evaluation of alternative glyph designs for time series data in a small multiple setting," in *Proc. CHI*. New York: ACM, 2013, pp. 3237-3246. doi: **10/gh52rn**

[46] H. Garcke, T. Preußer, M. Rumpf, A. Telea, U. Weikard, and J. van Wijk, "A continuous clustering method for vector fields," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2000, pp. 351-358. doi: **10/bft6b9**

[47] T. Givón, "Prototypes: Between Plato and Wittgenstein," in *Noun Classes and Categorization*, C. Craig, Ed. Amsterdam: John Benjamins Publishing, 1986, pp. 77-102, urn: **urn:oclc:record:1245623243.**

[48] B. F. Gregorski, J. G. Senecal, M. A. Duchaineau, and K. I. Joy, "Adaptive extraction of time-varying isosurfaces," *IEEE Trans Vis Comput Graph*, vol. 10, no. 6, pp. 683-694, 2004. doi: **10/cn3st4**

[49] D. L. Gresh, B. E. Rogowitz, R. L. Winslow, D. F. Scollan, and C. K. Yung, "WEAVE: A system for visually linking 3-D and statistical visualizations applied to cardiac simulation and measurement data," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2000, pp. 489-492. doi: **10/djvdvk**

[50] D. Haehn, J. Tompkin, and H. Pfister, "Evaluating 'graphical perception' with CNNs," *IEEE Trans Vis Comput Graph*, vol. 25, no. 1, pp. 641-650, 2019. doi: **10/gd52dt**

[51] T. He, P. Isenberg, R. Dachselt, and T. Isenberg, "BeauVis: A validated scale for measuring the aesthetic pleasure of visual representations," *IEEE Trans Vis Comput Graph*, vol. 29, no. 1, pp. 363-373, 2023. doi: **10/kt3n**

[52] J. Heer, M. Bostock, and V. Ogievetsky, "A tour through the visualization zoo: A survey of powerful visualization techniques, from the obvious to the obscure," *Queue*, vol. 8, no. 5, pp. 20-30, 2010. doi: **10/k8sw**

[53] J. L. Helman and L. Hesselink, "Surface representations of two-and three-dimensional fluid flow topology," in *Proc. Visualization*. Los Alamitos: IEEE CS, 1990, pp. 6-13. doi: **10/cs4s66**

[54] M. Hlawatsch, P. Leube, W. Nowak, and D. Weiskopf, "Flow radar glyphs—Static visualization of unsteady flow with uncertainty," *IEEE Trans Vis Comput Graph*, vol. 17, no. 12, pp. 1949-1958, 2011. doi: **10/cvc442**

[55] M. Hlawitschka and G. Scheuermann, "HOT-lines: Tracking lines in higher order tensor fields," in *Proc. VIS*. Los Alamitos: IEEE CS, 2005, pp. 27-34. doi: **10/c77b3m**

[56] K. Hu, S. Gaikwad, M. Hulsebos, M. A. Bakker, E. Zgraggen, C. Hidalgo, T. Kraska, G. Li, A. Satyanarayan, and Ç. Demiralp, "VizNet: Towards a large-scale visualization learning and benchmarking repository," in *Proc. CHI*. New York: ACM, 2019, pp. 662:1-662:12. doi: **10/gf2b85**

[57] M. Hummel, C. Garth, B. Hamann, H. Hagen, and K. I. Joy, "Iris: Illustrative rendering for integral surfaces," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1319-1328, 2010. doi: **10/d5xx88**

[58] S. Ingram, T. Munzner, V. Irvine, M. Tory, S. Bergner, and T. Möller, "DimStiller: Workflows for dimensional analysis and reduction," in *Proc. VAST*. Los Alamitos: IEEE CS, 2010, pp. 3-10. doi: **10/bhcd6v**

[59] P. Isenberg, D. Fisher, M. R. Morris, K. Inkpen, and M. Czerwinski, "An exploratory study of co-located collaborative visual analytics around a tabletop display," in *Proc. VAST*. Los Alamitos: IEEE CS, 2010, pp. 179-186. doi: **10/cpc8vv**

[60] P. Isenberg, F. Heimerl, S. Koch, T. Isenberg, P. Xu, C. D. Stolper, M. Sedlmair, J. Chen, T. Möller, and J. Stasko, "Vispubdata.org: A metadata collection about IEEE visualization (VIS) publications," *IEEE Trans Vis Comput Graph*, vol. 23, no. 9, pp. 2199-2206, 2017. doi: **10/ggwwrw**

[61] P. Isenberg, T. Isenberg, M. Sedlmair, J. Chen, and T. Möller, "Visualization as seen through its research paper keywords," *IEEE Trans Vis Comput Graph*, vol. 23, no. 1, pp. 771-780, 2017. doi: **10/f92gps**

[62] T. Isenberg, P. Isenberg, J. Chen, M. Sedlmair, and T. Möller, "A systematic review on the practice of evaluating visualization," *IEEE Trans Vis Comput Graph*, vol. 19, no. 12, pp. 2818-2827, 2013. doi: **10/f5h29z**

[63] E. K. Jacob, "Classification and categorization: A difference that makes a difference," *Lib Trends*, vol. 52, no. 3, pp. 515-540, 2004. doi: **hl: 2142/1686.**

[64] B. Jenny, M. Heitzler, D. Singh, M. Farmakis-Serebryakova, J. C. Liu, and L. Hurni, "Cartographic relief shading with neural networks," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1225-1235, 2020. doi: **10/ghv58f**

[65] W.-K. Jeong, J. Schneider, S. Turney, B. E. Faulkner-Jones, D. Meyer, R. Westermann, R. C. Reid, J. Lichtman, and H. Pfister, "Interactive histology of large-scale biomedical image stacks," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1386-1395, 2010. doi: **10/fc7rd9**

[66] X. Ji, H.-W. Shen, A. Ritter, R. Machiraju, and P.-Y. Yen, "Visual exploration of neural document embedding in information retrieval: Semantics and feature selection," *IEEE Trans Vis Comput Graph*, vol. 25, no. 6, pp. 2181-2192, 2019. doi: **10/ggjbsg**

[67] S. Jiang, W.-L. Chao, J. Chen, D. Haehn, M. Ling, C. Shang, and H. Pfister, "Are machines more effective than humans for graphical perception tasks?" *J Vision*, vol. 22, no. 14, pp. 3784-3784, 2022. doi: **10/nkgk**

[68] J. Kehrer and H. Hauser, "Visualization and visual analysis of multifaceted scientific data: A survey," *IEEE Trans Vis Comput Graph*, vol. 19, no. 3, pp. 495-513, 2013. doi: **10/f4kwr6**

[69] D. A. Keim, "Designing pixel-oriented visualization techniques: Theory and applications," *IEEE Trans Vis Comput Graph*, vol. 6, no. 1, pp. 59-78, 2000. doi: **10/fnqrw9**

[70] S. Kim, I. Woo, R. Maciejewski, D. S. Ebert, T. D. Ropp, and K. Thomas, "Evaluating the effectiveness of visualization techniques for schematic diagrams in maintenance tasks," in *Proc. APGV*. New York: ACM, 2010, pp. 33-40. doi: **10/bs23ww**

[71] Y. Kim, J. Kim, H. Jeon, Y.-H. Kim, H. Song, B. Kim, and J. Seo, "Github: Visual analytics for understanding software development history through Git metadata analysis," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 656-666, 2021. doi: **10/gmghvf**

[72] F. A. A. Kingdom and N. Prins, *Psychophysics: A Practical Introduction*, 2nd ed. Amsterdam: Academic Press, 2016. doi: **10/k8r7**

[73] L. Koesten, D. Dimmery, M. Gleicher, and T. Möller, "Subjective visualization experiences: Impact of visual design and experimental design," U. Wien, arXiv preprint 2310.13713, 2023. doi: **10/k8sc**

[74] J. K. Kruschke, "ALCOVE: An exemplar-based connectionist model of category learning," *Psychol Rev*, vol. 99, no. 1, pp. 22-44, 1992. doi: **10/ftqx2m**

[75] S. Lakare, M. Wan, M. Sato, and A. Kaufman, "3d digital cleansing using segmentation rays," in *Proc. VIS*. Los Alamitos: IEEE CS, 2000, pp. 37-44. doi: **10/d7w9n3**

[76] H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale, "Empirical studies in information visualization: Seven scenarios," *IEEE*
Trans Vis Comput Graph, vol. 18, no. 9, pp. 1520–1536, 2012. doi: 10/drrh6j

[77] B. Latour, "Wizualizacja i poznanie: zrysowywanie rzeczy razem (Visualisation and cognition: Drawing things together)," Avant Trends Interdiscip Stud, vol. 3, no. T, pp. 207–257, 2012, online: philpapers.org/rec/LATVAC-3.

[78] B. Lee, D. Brown, B. Lee, C. Hurter, S. Drucker, and T. Dwyer, "Data visceralization: Enabling deeper understanding of data using virtual reality," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1095–1105, 2021. doi: 10/ghv58h

[79] B. Lee, K. Isaacs, D. A. Szafir, G. E. Marai, C. Turkay, M. Tory, S. Carpendale, and A. Endert, "Broadening intellectual diversity in visualization research papers," *IEEE Comput Graph Appl*, vol. 39, no. 4, pp. 78–85, 2019. doi: 10/gg2j2m

[80] B. Lee, N. H. Riche, A. K. Karlson, and S. Carpendale, "Sparkclouds: Visualizing trends in tag clouds," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1182–1189, 2010. doi: 10/fv5z96

[81] R. Li and J. Chen, "Toward a deep understanding of what makes a scientific visualization memorable," in *Short Papers of IEEE Visualization/SciVis*. Los Alamitos: IEEE CS, 2018, pp. 26–31. doi: 10/gr633t

[82] G. L. Lohse, K. Biolsi, N. Walker, and H. H. Rueter, "A Classification of Visual Representations," *Commun ACM*, vol. 37, no. 12, pp. 36–50, 1994. doi: 10/b92kbi

[83] Y. Lu, A. Wu, H. Li, Z. Deng, J. Lan, J. Wu, Y. Wang, H. Qu, D. Deng, and Y. Wu, "VAID: Indexing view designs in visual analytics system," in *Proc. CHI*. New York: ACM, 2024. doi: doi.org/mj5d

[84] R. Maciejewski, *Data Representations, Transformations, and Statistics for Visual Reasoning*. Cham: Springer, 2011. doi: 10/d4565q

[85] J. Mackinlay, "Automating the design of graphical presentations of relational information," *ACM Trans Graph*, vol. 5, no. 2, pp. 110–141, 1986. doi: 10/dxdkkp

[86] J. Mackinlay, P. Hanrahan, and C. Stolte, "Show me: Automatic presentation for visual analysis," *IEEE Trans Vis Comput Graph*, vol. 13, no. 6, pp. 1137–1144, 2007. doi: 10/fgwbh9

[87] J. Malik and R. Rosenholtz, "Computing local surface orientation and shape from texture for curved surfaces," *Int J Comput Vision*, vol. 23, no. 2, pp. 149–168, 1997. doi: 10/cpx8m4

[88] T. Malisiewicz and A. A. Efros, "Recognition by association via learning per-exemplar distances," in *Proc. CVPR*. Los Alamitos: IEEE CS, 2008, pp. 42:1–42:8. doi: 10/c73x5z

[89] D. L. Medin and M. M. Schaffer, "Context theory of classification learning." *Psychol Rev*, vol. 85, no. 3, pp. 207–238, 1978. doi: 10/b4wm44

[90] W. Meulemans, M. Sondag, and B. Speckmann, "A simple pipeline for coherent grid maps," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1236–1246, 2021. doi: 10/ghgt53

[91] D. Moritz, C. Wang, G. L. Nelson, H. Lin, A. M. Smith, B. Howe, and J. Heer, "Formalizing visualization design knowledge as constraints: Actionable and extensible models in draco," *IEEE Trans Vis Comput Graph*, vol. 25, no. 1, pp. 438–448, 2019. doi: 10/cs68

[92] T. Munzner, *Visualization Analysis and Design*. New York: CRC Press, 2014. doi: 10/gd3xgq

[93] N. Nguyen, O. Strnad, T. Klein, D. Luo, R. Alharbi, P. Wonka, M. Maritan, P. Mindek, L. Autin, D. S. Goodsell, and I. Viola, "Modeling in the time of COVID-19: Statistical and rule-based mesoscale models," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 722–732, 2021. doi: 10/k8sh

[94] G. M. Nielson and B. Hamann, "The asymptotic decider: Removing the ambiguity in marching cubes," in *Proc. Visualization*. Los Alamitos: IEEE CS, 1991, pp. 83–91. doi: 10/fg4d6v

[95] C. Nobre, M. Meyer, M. Streit, and A. Lex, "The state of the art in visualizing multivariate networks," *Comput Graph Forum*, vol. 38, no. 3, pp. 807–832, 2019. doi: 10/ghpp2g

[96] D. Norman, *Things That Make Us Smart: Defending Human Attributes in the Age of the Machine*. New York: Diversion Books, 2014. urn: urn:oclc:record:1036960105.

[97] R. M. Nosofsky, "Attention, similarity, and the identification-categorization relationship," *J Exp Psychol Anim Learn Cognit: Gener*, vol. 115, no. 1, pp. 39–57, 1986. doi: 10/bjgj3w

[98] T. J. Overbye and J. D. Weber, "New methods for the visualization of electric power system information," in *Proc. InfoVis*, Los Alamitos, 2000, pp. 131–136. doi: 10/fdk73d

[99] S. Palmer, "Fundamental aspects of cognitive representation," in *Cognition and Categorization*, E. Rosch and B. Lloyd, Eds., 1978, ch. 9, pp. 259–303, urn: urn:oclc:record:1409364022.

[100] D. Phan, L. Xiao, R. Yeh, and P. Hanrahan, "Flow map layout," in *Proc. InfoVis*. Los Alamitos: IEEE CS, 2005, pp. 219–224. doi: 10/fq44hs

[101] M. C. Potter, A. Staub, and D. H. O'Connor, "Pictorial and conceptual representation of glimpsed pictures," *J. Exp. Psychol Hum Percept Perform*, vol. 30, no. 3, p. 478, 2004. doi: 10/c26zf9

[102] P. Pu and D. Lalanne, "Interactive problem solving via algorithm visualization," in *Proc. InfoVis*. Los Alamitos: IEEE CS, 2000, pp. 145–153. doi: 10/bh73bn

[103] D. Rees and R. Laramee, "A survey of information visualization books," *Comput Graph Forum*, vol. 38, no. 1, pp. 610–646, 2019. doi: 10/gftgv6

[104] R. A. Rensink, "Visualization as a stimulus domain for vision science," *Journal of Vision*, vol. 21, no. 8, pp. 3–3, 2021. doi: 10/mg29

[105] P. Rheingans, M. Marietta, and J. Nichols, "Interactive 3D visualization of actual anatomy and simulated chemical time-course data for fish," in *Proc. Visualization*. Los Alamitos: IEEE CS, 1995, pp. 393–396. doi: 10/bst9vq

[106] E. H. Rosch, "Natural categories," *Cognit Psychol*, vol. 4, no. 3, pp. 328–350, 1973. doi: 10/cj6qvb

[107] D. Sacha, L. Zhang, M. Sedlmair, J. A. Lee, J. Peltonen, D. Weiskopf, S. C. North, and D. A. Keim, "Visual interaction with dimensionality reduction: A structured literature analysis," *IEEE Trans Vis Comput Graph*, vol. 23, no. 1, pp. 241–250, 2017. doi: 10/f9zvx5

[108] A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer, "Vega-Lite: A grammar of interactive graphics," *IEEE Trans Vis Comput Graph*, vol. 23, no. 1, pp. 341–350, 2017. doi: 10/f92f32

[109] M. Savva, N. Kong, A. Chhajta, L. Fei-Fei, M. Agrawala, and J. Heer, "ReVision: Automated classification, analysis and redesign of chart images," in *Proc. UIST*. New York: ACM, 2011, pp. 393–402. doi: 10/bcbnfw

[110] W. Scheibel, M. Trapp, D. Limberger, and J. Döllner, "A taxonomy of treemap visualization techniques." in *VISIGRAPP (3: IVAPP)*, 2020, pp. 273–280. doi: 10/gp46vp

[111] H.-J. Schulz, "Treevis.net: A tree visualization reference," *IEEE Comput Graph Appl*, vol. 31, no. 6, pp. 11–15, 2011. doi: 10/dmgxfs

[112] M. Sedlmair, M. Meyer, and T. Munzner, "Design study methodology: Reflections from the trenches and the stacks," *IEEE Trans Vis Comput Graph*, vol. 18, no. 12, pp. 2431–2440, 2012. doi: 10/f4fv7x

[113] B. Shneiderman and M. Wattenberg, "Ordered treemap layouts," in *Proc. InfoVis*. Los Alamitos: IEEE CS, 2001, pp. 73–78. doi: 10/dtq8rj

[114] D. Skau and R. Kosara, "Arcs, angles, or areas: Individual data encodings in pie and donut charts," *Comput Graph Forum*, vol. 35, no. 3, pp. 121–130, 2016. doi: 10/f8wcg8

[115] Y. Song, J. Ye, N. Svakhine, S. Lasher-Trapp, M. Baldwin, and D. Ebert, "An atmospheric visual analysis and exploration system," *IEEE Trans Vis Comput Graph*, vol. 12, no. 5, pp. 1157–1164, 2006. doi: 10/d7t3cb

[116] S. Stevens, J. Wu, M. J. Thompson, E. G. Campolongo, C. H. Song, D. E. Carlyn, L. Dong, W. M. Dahdul, C. Stewart, T. Berger-Wolf, W.-L. Chao, and Y. Su, "BioCLIP: A vision foundation model for the tree of life," in *Proc. CVPR*, 2024, pp. 19412–19424. doi: 10/mg3n

[117] C. Stokes, V. Setlur, B. Cogley, A. Satyanarayan, and M. A. Hearst, "Striking a balance: Reader takeaways and preferences when integrating text and charts," *IEEE Trans Vis Comput Graph*, vol. 29, no. 1, pp. 1233–1243, 2023. doi: 10/mcsk

[118] C. Stoll, S. Gumhold, and H.-P. Seidel, "Visualization with stylized line primitives," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2005, pp. 695–702. doi: 10/bh4jp9

[119] I. Sucholutsky, L. Muttenthaler, A. Weller, A. Peng, A. Bobu, B. Kim, B. C. Love, E. Grant, I. Groen, J. Achterberg, J. B. Tenenbaum, K. M. Collins, K. L. Hermann, K. Oktar, K. Greff, M. N. Hebart, N. Jacoby, Q. Zhang, R. Marjeh, R. Geirhos, S. Chen, S. Kornblith, S. Rane, T. Konkle, T. P. O'Connell, T. Unterthiner, A. K. Lampinen, K.-R. Müller, M. Toneva, and T. L. Griffiths, "Getting aligned on representational alignment," *arXiv preprint* 2310.13018, 2023. doi: doi:10/mh8m

[120] F. Suits, J. T. Klosowski, W. P. Horn, and G. Lecina, "Simplification of surface annotations," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2000, pp. 235–242. doi: 10/dw25dk

[121] J. W. Tanaka and M. Taylor, "Object categories and expertise: Is the basic level in the eye of the beholder?" *Cognit Psychol*, vol. 23, no. 3, pp. 457–482, 1991. doi: ft55zq

[122] A. Tikhonova, C. D. Correa, and K.-L. Ma, "Visualization by proxy: A novel framework for deferred interaction with volume data," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1551–1559, 2010. doi: 10/dvb76h

[123] N. Tovanich, P. Dragicevic, and P. Isenberg, "Gender in 30 years of IEEE visualization," *IEEE Trans Vis Comput Graph*, vol. 28, no. 1, pp. 497–507, 2022. doi: 10/nkkz
[124] A. Treisman, "The binding problem," *Curr Opin Neurobiol*, vol. 6, no. 2, pp. 171-178, 1996. doi: 10/b6jzkc

[125] E. R. Tufte, *Visual Explanations: Images and Quantities, Evidence and Narrative*. Cheshire: Graphics Press, 1998, urn: urn:oclc:record:1280715234.

[126] ——, *Envisioning Information*. Cheshire: Graphics Press, 2001, urn: urn:oclc:record:1034670525.

[127] B. Tversky, *Mind in Motion: How Action Shapes Thought*. UK: Hachette, 2019.

[128] D. Ushizima, D. Morozov, G. H. Weber, A. G. C. Bianchi, J. A. Sethian, and E. W. Bethel, "Augmented topological descriptors of pore networks for material science," *IEEE Trans Vis Comput Graph*, vol. 18, no. 12, pp. 2041–2050, 2012. doi: 10/f4fqtv

[129] J. J. van Wijk, "Image based flow visualization for curved surfaces," in *Proc. VIS*. Los Alamitos: IEEE CS, 2003, pp. 123–130. doi: 10/fws2sq

[130] V. Verma, D. Kao, and A. Pang, "A flow-guided streamline seeding strategy," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2000, pp. 163–170. doi: 10/drkmh9

[131] J. von Engelhardt, "The language of graphics," Ph.D. dissertation, University of Amsterdam, 2002, hdl: 11245/1.208097.

[132] X. Wang, Z. Bylinskii, A. Hertzmann, and R. Pepperell, "Toward quantifying ambiguities in artistic images," *ACM Trans Appl Percept*, vol. 17, no. 4, pp. 13:1–13:10, 2020. doi: 10/gm8gm8

[133] Y. Wang, X. Chu, C. Bao, L. Zhu, O. Deussen, B. Chen, and M. Sedlmair, "EdWordle: Consistency-preserving word cloud editing," *IEEE Trans Vis Comput Graph*, vol. 24, no. 1, pp. 647–656, 2018. doi: 10/gcp8bd

[134] M. O. Ward, "Multivariate data glyphs: Principles and practice," in *Handbook of Data Visualization*. Berlin: Springer, 2008, pp. 179–198. doi: 10/ddmw82

[135] M. O. Ward, G. Grinstein, and D. Keim, *Interactive Data Visualization: Foundations, Techniques, and Applications*, 2nd ed. New York: A K Peters/CRC Press, 2015. doi: 10/k8sk

[136] C. Ware, *Information Visualization: Perception for Design*, 2nd ed. San Francisco: Morgan Kaufmann, 2004, urn: urn:oclc:record:56344454.

[137] ——, *Information Visualization: Perception for Design*, 4th ed. San Francisco: Morgan Kaufmann, 2020. doi: 10/mj55

[138] D. Weiskopf, M. Borchers, T. Ertl, M. Falk, O. Fechtig, R. Frank, F. Grave, A. King, U. Kraus, T. Müller, H.-P. Nollert, I. R. Mendez, H. Ruder, T. Schafhitzel, S. Schär, C. Zahn, and M. Zatloukal, "Visualization in the Einstein year 2005: A case study on explanatory and illustrative visualization of relativity and astrophysics," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2005, pp. 583–590. doi: 10/bbsrhm

[139] D. Weiskopf and G. Erlebacher, "Overview of flow visualization," in *Visualization Handbook*, C. D. Hansen and C. R. Johnson, Eds. Oxford: Elsevier, 2005, ch. 12, pp. 261–278. doi: 10/d4hzdf

[140] M. Weiß, K. Angerbauer, A. Voit, M. Schwarzl, M. Sedlmair, and S. Mayer, "Revisited: Comparison of empirical methods to evaluate visualizations supporting crafting and assembly purposes," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1204–1213, 2021. doi: 10/ghgt5z

[141] L. Wilkinson, *The Grammar of Graphics*, 2nd ed. New York: Springer, 2005. doi: 10/bhsbp

[142] L. Wittgenstein, *Philosophical Investigations*. New York: MacMillan, 1953, urn: urn:oclc:record:1245626409.

[143] J. M. Wolfe, "Guided Search 6.0: An updated model of visual search," *Psychon Bull Rev*, vol. 28, no. 4, pp. 1060–1092, 2021. doi: 10/gh2s45

[144] J. Xia, T. Chen, L. Zhang, W. Chen, Y. Chen, X. Zhang, C. Xie, and T. Schreck, "SMAP: A joint dimensionality reduction scheme for secure multi-party visualization," in *Proc. VAST*. Los Alamitos: IEEE CS, 2020, pp. 107–118. doi: 10/k8st

[145] S. Ye, Z. Chen, X. Chu, Y. Wang, S. Fu, L. Shen, K. Zhou, and Y. Wu, "ShuttleSpace: Exploring and analyzing movement trajectory in immersive visualization," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 860–869, 2021. doi: 10/ghgt4x

[146] Y. Ye, R. Huang, and W. Zeng, "VISAtlas: An image-based exploration and query system for large visualization collections via neural image embedding," *IEEE Trans Vis Comput Graph*, vol. 30, no. 7, pp. 3224–3240, 2024. doi: 10/grq3m3

[147] J. S. Yi, Y. ah Kang, J. Stasko, and J. A. Jacko, "Toward a deeper understanding of the role of interaction in information visualization," *IEEE Trans Vis Comput Graph*, vol. 13, no. 6, pp. 1224–1231, 2007. doi: 10/fmrs6r

[148] V. Yoghourdjian, Y. Yang, T. Dwyer, L. Lawrence, M. Wybrow, and K. Marriott, "Scalability of network visualisation from a cognitive load perspective," *IEEE Trans Vis Comput Graph*, vol. 27, no. 2, pp. 1677–1687, 2021. doi: 10/ghs96j

[149] X. Yue, X. Shu, X. Zhu, X. Du, Z. Yu, D. Papadopoulos, and S. Liu, "BitExTract: Interactive visualization for extracting Bitcoin exchange intelligence," *IEEE Trans Vis Comput Graph*, vol. 25, no. 1, pp. 162–171, 2019. doi: 10/ggjbt3

[150] C. Ziemkiewicz and R. Kosara, "Laws of attraction: From perceptual forces to conceptual similarity," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1009–1016, 2010. doi: 10/cz74g2

<figure><img src="image_56.png" /></figure>

**Jian Chen** is an associate professor in the Department of Computer Science and Engineering at The Ohio State University. She did her postdoctoral work in Computer Science and BioMed at Brown University to study evaluation and biological motion. She received her PhD degree in Computer Science from Virginia Polytechnic Institute and State University on 3D interaction. Her current research interests include visualization theory, evaluation, and interaction.

<figure><img src="image_57.png" /></figure>

**Petra Isenberg** is a senior research scientist at Inria, Saclay, France in the Aviz team. She received her Diploma degree from the University of Magdeburg, Germany and her PhD degree from the University of Calgary, Canada in 2010. She is particularly interested in exploring how people can most effectively work together when analyzing large and complex data sets on novel display technology such as small touchscreens, wall displays, or tabletops.

<figure><img src="image_58.png" /></figure>

**Robert S. Laramee** is a professor at the University of Nottingham, UK in the School of Computer Science. He received the PhD degree from the Vienna University of Technology, Austria at the Institute of Computer Graphics and Algorithms. Between 2001 and 2006 he was a researcher at the VRVis Research Center and a software engineer at AVL in the department of Advanced Simulation Technologies. He was a Lecturer, then Associate Professor at Swansea University, Wales in the Department of Computer Science. His research interests are in the areas of scientific visualization, information visualization, and visual analytics.

<figure><img src="image_59.png" /></figure>

**Tobias Isenberg** is a senior research scientist at Inria, France. He received his PhD degree from the University of Magdeburg, Germany in 2004. Previously he held positions as postdoctoral fellow at the University of Calgary, Canada, and as assistant professor at the University of Groningen, the Netherlands. He is particularly interested in interactive visualization environments for 3D spatial data as well as illustrative visualization and abstraction techniques.

<figure><img src="image_60.png" /></figure>

**Michael Sedlmair** is a professor at the University of Stuttgart and is leading the research group for Visualization and Virtual/Augmented Reality (VR/AR) there. He received the PhD degree in Computer Science from University of Munich, Germany in 2010. His specific research interests focus on interactive machine learning, interactive visualization in VR/AR, as well as the methodological and theoretical frameworks underlying them.

<figure><img src="image_61.png" /></figure>

**Torsten Möller** is a professor at the University of Vienna, Austria. Between 1999 and 2012, he was a Computing Science faculty member at Simon Fraser University, Canada. He received his PhD in Computer and Information Science from Ohio State University in 1999 and a Vordiplom (BSc) in mathematical computer science from Humboldt University of Berlin, Germany. He is a senior member of IEEE and ACM, and a member of Eurographics. His research interests include algorithms and tools for analyzing and displaying data with principles

rooted in computer graphics, human-computer interaction, image processing, machine learning, and visualization.
<figure><img src="image_62.png" /></figure>

**Rui Li** is a Ph.D. student in Computer Science and Engineering at Ohio State University. He received the B.E. degree in Computer Science and Engineering from Central South University, China. His research interests include evaluation of data visualization, visual analytics, and data mining.
# APPENDIX I

## SOURCE IMAGE DATA AND OUR VIS TYPES DATA

The first six of the seven years of image data come from the collection of figure and table data from IEEE VIS publications from 1990–2019 in Chen et al.’s VIS30K [26]. We added the 2020 dataset to the VIS30K data collection. In doing this, we reused the authors’ (our) approach and the meta-data to remove all tables (unless part of a figure) and their open-source model and tools to extract and clean the new figure data of 2020 after scraping all papers from the IEEE Xplore. The first author collected and curated the image data for the full year. Other co-authors checked the final image set. The overall image dataset is accessible at go.osu.edu/vis30k. This web-based tool (visimagenavigator.github.io) also allows the viewers to send an email request to update our image codes (Fig. 15).

# APPENDIX II

## WHAT THIS PAPER IS NOT?

We compare our typology to others for the purpose of categorization. Our work neither judges the quality of visualizations nor studies the morphological relationships between visual designs. Cleveland’s critique of visual constructions [30] and Wilkinson’s morphology in GoG [141] did so. We also do not study how designs relate to meaning in images; Cairo’s Functional Art [22] did this. Our work provides a level of categorization of main visual foci or essential stimuli — so we can clearly describe the main visual focus of attention in these images. In addition, our codes do not cover non-essential elements, e. g., axis and labels, which are indispensable, but do not map to data directly.

Approaches for representing data visualizations fall into two broad categories: those that focus on designers’ drawings and those that prioritize viewers’ understanding. Ours belongs to the latter approach. Wilkinson’s GoG [141] is similar to ours in that it does not use techniques to describe graphics but contains a ‘generalizable’ list of categories. His categories are perhaps most typical—called prototypes [127], useful for communication. Wilkinson cautiously noted that his categorization taxonomy was incomplete, did not represent the visualization domain, and was designed solely for implementation purposes. In contrast, our categorization is meant to be complete covering a much broader set and formally coded by experts through iterations. Compared to Wilkinson, Heer [52] focused on atypical drawings of a set of what he called “unusual” and “exotic” techniques: “you don’t go to the zoo to see Chihuahuas and raccoons; you go to admire the majestic polar bear, the graceful zebra, and the terrifying Sumatran tiger.” Also, his focus was specific advances of specific techniques rather than categorizing representations. In our typology, we assigned the atypical images by comparing their similarity to the typical ones. Finally, in contrast to Cleveland’s vision to understand visualizations from images—“a graphical method is successful only if the decoding is effective” [32], he enables ‘elementary graphical perception’ work through ‘elementary level visual marks in statistical charts’. In this regard, our work broadens the purview of representations at the level of essential stimuli.

# APPENDIX III

## ADDITIONAL CODES: FUNCTION, DIMENSIONALITY, AND CODING TRANSITIONS

We further describe how difficult it can be to understand images that were taken out of the context of the text and captions. Thus, additional decisions were needed in the coding process. We have also coded two *function codes* and two *dimensionality codes*, as well as experiences concerning the *difficulties* of categorizing visualization images (see definitions in Table II.)

### A. Pilot Study: Failed VisKeywords-Based Method

The top-21 authors’ keywords for specific techniques are: *bar chart*, *cartographic map*, *circular node-link tree*, *flow chart*, *flowline*, *glyphs*, * heatmap*, *isosurface rendering*, *line chart*, *matrix*, *node-link diagram*, *parallel coordinate plot*, *pie chart*, *point cloud*, *scatter plot*, *tag cloud*, * timeline*, *treemap*, *volume rendering*, *Voronoi diagram*, and *wireframe rendering*.

### B. Pilot study: Failed Bertin’s Marks and Channels-based Method

One may consider Bertin’s marks and channels to describe how we see visualization images. This perspective decodes the complex perceptual process by describing low-level *features*, such as color, shape, size, orientation, curvature, and lines, in the images first, and then treat visual processing as a means for the visual system to “bond” together these features to become a “single representation of an object” [124]. This system uses five criteria of categorization: printable on white paper, visible at a glance, reading distance of book or atlas, normal and constant lighting, and readily available graphics means, which unfit our categorization of coding essential stimuli to represent what viewers should see from images.

We demonstrate that relying on low-level features alone is not enough to understand the images. For example, we may observe in Fig. 16 that edges (lines) in (b) (computed using the Canny Edge detector) can describe the design feature of bar edges in (a). However, the bar height in (a) should be the essential stimulus, and the line descriptor does not show the bar structures well. In the second example in Fig. 16c, we also show features defined by the Histograms of Oriented Gradients (HOGs) (d) for the surface in (c). HOGs let us see structures. However, it does not show the ‘surface’ in (c).

### C. Pilot Study: Consolidation

In **Phase 3 Consolidation** (Sect. 3-C) we made several decisions to group stimuli, (1) if two representations share specific stimuli, (e. g., donut charts use length and thus fall into the category of “*Generalized Bar Representations*”, see Fig. 2) and (2) if a stimulus is closer to one category than to others, e. g., the geometric wireframe mesh is closer to “*node-link*”. We further devise category-specific characteristics, e. g., the bar examples share common length feature regardless of the coordinate system; “*surface-based Representations and Volumes*” resembles the inside and outside boundaries, and “*Grid*” shows the family of layout appearances. Some additional examples are below. Fig. 17 shows our rationale for removing
timline from our typology—the judgment depends on the
viewer’s knowledge because drawing the temporal cue may
not always be necessary.

**We merged techniques if the visual representations of the essential stimuli are alike.** For example, flowlines, parallel coordinate plots, and line charts all contain line features. *We merged them into the Line category.*

The flow chart category was prevalent and represented the largest number of figures in the other category of our 2006 coding, thus adding and later further expanding to “schematic representations and concept illustrations”. The original flow chart has only captured a limited number of wireframe diagrams. Many charts and diagrams in visualization papers are visually rich and contain techniques to illustrate concepts. *We expanded flowchart to “General Schematic Representation, schematic images, schematic concept illustration”.*

**We ignored the drawing media used to visualize the data.** Sometimes, it is challenging to determine whether or not views are hand-drawn or computer-generated. For example, schematic ones can be drawn by algorithms [70]. “Illustrative visualization” develops algorithms that would render images that appear to be hand-drawn. The illustrations in the classical book of Bertin’s semiotics were also drawn by hand. Hence, we chose to ignore the media in the subsequent coding phase. Instead, we emphasize the elements in the figure rather than the drawing media. For example, a photograph of an environment (e. g., a virtual reality ) would be coded as “3D”.

**We managed annotation, legend, and context.** One of the challenges was treating context information, such as annotation marks or color legends. Here, we focus on the primary visual encoding and agree not to code such context information separately. The color legend is in the “other” category. Context, unless relevant to the data, is not coded. Some contextual data, such as geometric models or boundaries, play a crucial role in comprehending visualization techniques and, as such, were coded (e. g., “generalized surface and volume representations” is coded besides “generalized line representations”, see Fig. 18).

**We avoided including data types in the type names.** For example, scalar, vector, and tensor field visualization techniques can be defined using our typology without mentioning flow fields or tensors.

# APPENDIX IV
## WEB INTERFACES FOR ANNOTATING FIGURES

We annotated all images and discussed and compared our codings via our own web-based interfaces to support our collaborative work (Fig. 19(a)–(d)). Our web-based tool automatically loads images and authorizes the users. The users can tag the given image according to the keyword- or type-based terms. On the backend of our coding tool, we recorded every button click from each participant during the coding process for post-hoc analyses. To resolve code consistency, we also implemented a comparative interface (Fig. 20) to resolve all coding conflicts. Again, all coders’ choices were recorded.

# APPENDIX V
## ADDITIONAL RESULTS (EXPERTRESEARCHERS)

Fig. 21(a) shows the consistency of the image types captured in our empirical study. Echoing our own experiences and the nature of any categorization, we also found in our discussion that the inconsistent types could be related to personal differences in education, knowledge, and preferences. Fig. 21(b) shows the total number of image types by year. We can see a steady increase in almost all types. Fig. 22 shows “generalized surface and volume representations” and “generalized line representations” uses over time. Fig. 23 illustrates the co-occurrence of these types.

# APPENDIX VI
## ADDITIONAL RESULTS (EXPERTVIEWERS)

Fig. 24 shows a list of images where all participants chose the exact same labels as we did in our own coding. These examples show that, despite the rich visual features in each of our types, participants can use our category descriptions to identify image types. Fig. 25 shows some example images ExpertViewers participants had mostly disagreed with our answers. The cases that ExpertViewers found challenging matched those we ExpertResearchers encountered during the coding process. Labels were not made available to indicate the data and the image captions. Participants read the images alone. For example, Image 1 was coded by ExpertViewers as *Generalized line-based representation*. Here, it is very difficult to determine the types without knowing what the data represents.

# APPENDIX VII
## ADDITIONAL IMAGE EXAMPLES ABOUT VISUAL TYPES

While Sect. 4 shows the canonical examples, we present more image results to show the visual appearances of the images in each of the 10 types in Fig. 26.

# APPENDIX VIII
## REUSE OF VISTYPES FOR NON-SCHOLARLY IMAGES

We demonstrated the reuse of our Typology to code Borkin et al.’s MASSVIS image set [16], in massive media online images. MASSVIS contained 2,070 images of online media, infographics, and charts. The code comparison is online at go.osu.edu/vistypescodedmassvis. MASSVIS contained many *Bar*, followed by *Area* and *Line*. Not surprisingly, it has a limited number of images in the *Surface/Volume* category. We also observed that the visualizations in mass medium tended to support fact retrieval since many figures were annotated.

# IMAGE COPYRIGHT

We as authors state that the images in Fig. 15, Fig. 16b, Fig. 16d, Fig. 19— Fig. 26 in this appendix are and remain under our own copyright, with permission to be used here. We have also made them available under the [Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/) license and share them at [osf.io/dxjwt](https://osf.io/dxjwt). All remaining images are © IEEE, with the permission to be used here.
<figure><img src="image_63.png" /><figcaption>Fig. 15: **VisTypes Exploration Tool.** An example query showing the *Generalized node-link, mesh representation*, when both coders labelled *hard*. Readers can try the interface online at visimagenavigator.github.io.</figcaption></figure>

<table><caption>TABLE II: **The visualization function and dimensionality codes.** We coded images based on their function and dimensionality. Additional codes not listed here were “I cannot tell” to label images that had unclear functions or dimensionalities.</caption><thead><tr><th>Function Codes</th><th>Description</th><th>Examples</th></tr></thead><tbody><tr><td>A. GUI Screenshots or GUI Photos</td><td>Images that show a system or user interface.</td><td>a photograph of a person sitting in front of a given system, a figure containing GUI features such as windows, icons, cursor, and pointers (WIMP), or non-WIMP VR/AR interfaces.</td></tr><tr><td>B. Schematic Representations, Concept Illustrations</td><td>Often simplified representations showing the appearance, structure, or logic of a process or concept.</td><td>workflow diagrams, algorithm diagrams, sketches.</td></tr><tr><th>Dimensionality Codes</th><th>Description</th><th>Examples</th></tr><tr><td>2D</td><td>Flat representations, no specific depth codes added to renditions.</td><td>Most statistical charts, most maps, ...</td></tr><tr><td>3D</td><td>Representation with specific depth cues that achieve the perception of 3D (shading, perspective, lighting, ...).</td><td>Most volume renderings, ...</td></tr></tbody></table>

<figure><img src="image_64.png" /><figcaption>Fig. 16: **Low-level features (edges and histograms of oriented gradients (HOG)) extracted from images do not necessarily represent how humans see.** (a) The original image in Sacha et al. [107] and (b) its edges; (c) The original paper image in Gresh et al. [49] and (d) its HOGs.</figcaption></figure>
<figure><img src="image_65.png" /><figcaption>Fig. 17: **Rationale to Remove Timeline from VisTypes.** We ask the question: Is there a timeline in the figures? Here shows two examples. (a) yes because of the dates on the x-axis and (b) yes but not so obvious and the judgment counts on the coders' knowledge. Images from (a) Chen et al. [25] and (b) Song et al. [115].</figcaption></figure>

<figure><img src="image_66.png" /><figcaption>Fig. 18: **Coding context.** In addition to “generalized line representations”, we incorporated “generalized surface and volume representations” because the geometry of the model itself determines the shape of the lines. Image taken from [53].</figcaption></figure>
<figure><img src="image_67.png" /><figcaption>Fig. 19: Screenshots of the web interfaces used in Phases 1–3 and Phases 4–5. Phase 1 focused on techniques derived from authors’ keywords: The coders use the interfaces to code each image. The coding labels and categories were updated reflecting the results of weekly discussions. These label interfaces show that code revolution over time: from **Keyword-based** to Bertin-based to **Type-based** updates.</figcaption></figure>
<figure><img src="image_68.png" /><figcaption>Fig. 20: A screenshot of the web interface for Phase 6. In Phase 6 we resolved coders' inconsistency through pair-wise comparisons. Two coders results are shown together. For complex images, coders can also look up the images in the same paper coded by other coders through paper-based or image similarity-based search.</figcaption></figure>
<figure><img src="image_69.png" /><figcaption>Fig. 21: **Additional User Study Result on Overall Code Distributions.** (a) The initial consistency of visualization type codes applied to images. We can see that the expert coders (ExpertResearchers) had least consistency related to “generalized glyph representations”. (b) The total number of image types by year.</figcaption></figure>

<figure><img src="image_70.png" /><figcaption>Fig. 22: **Additional User Study Result on Dimensionality (color-coded 2D in teal, 3D in lime and 'I cannot tell' in gray).** Temporal overview of the proportions of 2D and 3D images for “generalized surface and volume representations” and “generalized line representations”. We observed a decrease in 3D use over time.</figcaption></figure>

<figure><img src="image_71.png" /><figcaption>Fig. 23: **VisTypes Co-occurrence Results.** Coding results from categorizing IEEE VIS paper images according to visualization types (baby-lue), their dimensionality (gray), and additional image functions (red). There are many 2D Schematics and 2D GUIs. Surface/Volume occurred the most with 3D representation. 2D bars are also frequently used.</figcaption></figure>
<figure><img src="image_72.png" /><figcaption>Fig. 24: **Example images for which the participants' coding completely matched our own coding results.** Among these, we see, from top left to bottom right, 3 area, 6 bar, 2 color/grey-scale, 2 glyph, 5 line-based, 5 node-link, 4 point, 4 surface/volume, 6 text representations, and 2 grid. All these images had difficulty ratings of easy to neural. These examples show that, despite the rich visual features in each of our types, participants can use our category descriptions to identify image types.</figcaption></figure>

<figure><img src="image_73.png" /><figcaption>Fig. 25: **The Top 24 Most Challenging Images.** Among the 180 target images chosen for evaluation, 24 images are ordered from least to most codes coded by ExpertViewers with a Ph.D. degree, from top left to bottom right. The ExpertResearchers-coded type was tagged on each image. **Intuition.** People tend to count on the presence or absence of the 'design element', rather than the 'essential stimuli'. e.g., people mostly coded 1, "generalized line representations"; Participants mostly responded no to our code 2. "generalized matrix and grid" and 18. "generalized area representations"; In our codes, we do not code color when *color* does not encode data. Thus, image 22 has only one code "generalized text representations". Participants also coded "generalized continuous color/grey-scale/texture representations" for 22. Image 17 is a difficult case. Lines could reveal surfaces rather than the geometry from alone. We coded "generalized line representations" because we were uncertain about the representation and thus took the visual appearance alone. Readers can find the participants' answers using the **green name tag** online at go.osu.edu/visTypeExpResult.</figcaption></figure>
<figure><img src="image_74.png" /><figcaption>Fig. 26: VisTypes Image Examples. More single-code image examples in our VisTypes data.</figcaption></figure>