<figure><img src="image_1.png" /></figure>

HAL
open science

# Pondering the Concept of Abstraction in (Illustrative) Visualization

Ivan Viola, Tobias Isenberg

<figure><img src="image_2.png" /></figure>

## To cite this version:

Ivan Viola, Tobias Isenberg. Pondering the Concept of Abstraction in (Illustrative) Visualization. IEEE Transactions on Visualization and Computer Graphics, 2018, 24 (9), pp.2573-2588. 10.1109/TVCG.2017.2747545 . hal-01581177

HAL Id: hal-01581177

https://inria.hal.science/hal-01581177

Submitted on 4 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Pondering the Concept of Abstraction in (Illustrative) Visualization

Ivan Viola and Tobias Isenberg

**Abstract**—We explore the concept of abstraction as it is used in visualization, with the ultimate goal of understanding and formally defining it. Researchers so far have used the concept of abstraction largely by intuition without a precise meaning. This lack of specificity left questions on the characteristics of abstraction, its variants, its control, or its ultimate potential for visualization and, in particular, illustrative visualization mostly unanswered. In this paper we thus provide a first formalization of the abstraction concept and discuss how this formalization affects the application of abstraction in a variety of visualization scenarios. Based on this discussion, we derive a number of open questions still waiting to be answered, thus formulating a research agenda for the use of abstraction for the visual representation and exploration of data. This paper, therefore, is intended to provide a contribution to the discussion of the theoretical foundations of our field, rather than attempting to provide a completed and final theory.

**Index Terms**—Abstraction, visual abstraction, axes of abstraction, abstraction space, spatial data, illustrative visualization, stylization.

<figure><img src="image_4.png" /></figure>

## 1 INTRODUCTION

To visualize essential aspects of the data we need dedicated mechanisms that abstract—from Latin *abstractus*, “drawn away”—the (unnecessary) detail to allow the viewer of a visualization to focus on the important elements. The crucial problem in this context is that it is impossible to know what is important and what is not in a general way—importance changes based on the research question, on the application domain, on the data size, on the user, on the specific situation, etc. Visualization technology, therefore, needs to support a dynamic change of data’s visual abstraction to reflect these contextual changes. A fundamental research challenge in visualization is thus to understand what (visual) abstraction really is, what it means, how it can be controlled, and how it is, can be, and should be used in visualization.

In this article we thus attempt a formalization of the concept of (visual) abstraction as it relates to visualization. For this discussion we are inspired by past work, in particular, in illustrative visualization—a sub-field of visualization in which abstraction is one of the main concepts. In doing so, we go far beyond the initial analysis of abstraction in illustrative visualization by Rautek et al. [88] in 2008 and use research that has been published since then to guide our discussion. In our work we specifically focus on the abstraction of spatial data, i. e., such data that has an inherent mapping to 3D or 2D space. Specifically, in addition to formally defining it, we present a number of propositions<fnref n="1" /> about abstraction that identify important properties, constraints, and usage scenarios for the concept. Based on this analysis, we postulate important research questions that arise from our formalization. In particular, we discuss use cases such as (semi-)automated visualization design, superimposed visualization for multi-attribute data, abstraction and temporal changes, as well as the role of abstraction in large-scale changes. Finally, we place our discussion into context, mention

those elements that are still missing from a complete theory, and discuss the limitations of our current formulation.

While in our following discussion we propose new interpretations of the foundational theory of visualization, we cannot yet provide a completed and final theory. Our work, however, is a significant step forward in forming our understanding of the concept of abstraction in visualization, along with a research agenda for going forward in formalizing this theoretical foundation of visualization in general. Our article is thus part of the scientific “conversation carried out through paper-sized units” [78], in which our goal is to move the discussion forward—not to provide a final answer. Nevertheless, we believe that our article is an important contribution to the discussion of the theoretical foundations of the field of visualization that will be extended in the future.

## 2 RELATED WORK

Most of the prior work, in particular in computer graphics and non-photorealistic rendering, has been using the term visual abstraction by intuition. In virtually all non-photorealistic rendering approaches, for example, researchers are trying to emulate artistic forms of expression which introduce abstraction to the depiction, both as part of the chosen medium (e. g., watercolor [17], painterly rendering [23], [119], etc.) as well as as part of the (human) input (e. g., the abstraction in drawing and sketching [9], [75], [80]). For example, Berger et al. [9] used a data-driven approach for synthesizing sketches of human portraits and forced the artists who created their dataset to “abstract” the shapes they drew by giving them less time to finish a particular drawing.

Such an implicit notion of abstraction, however, is not at all sufficient for visualization—here the abstraction serves the goal of facilitating the understanding of the subject matter. Hence we must understand the implications of abstraction and how can we control it effectively and efficiently. Such an abstraction concept needs to go beyond a straight-forward approach that either removes detail for a shape/model/object in general or that keeps detail where people look in an image or where a high saliency exists, and removing it elsewhere [29], [93]. Initial attempts to establish an

Manuscript received Jan. 9, 2017, revised June 21, 2017, finally accepted Aug. 27, 2017.

<footnote>• Ivan Viola is with TU Wien, Austria. E-mail: viola@cg.tuwien.ac.at.
• Tobias Isenberg is with Inria and Université Paris-Saclay, France. E-mail: tobias.isenberg@inria.fr.</footnote>

<footnote n="1">1. We briefly explain our motivation for how we arrived at a proposition, where needed, using footnotes to avoid interrupting our general argumentation.</footnote>
<figure><img src="image_5.png" /><figcaption>Fig. 1. Abstracted subway network maps: (a)–(e) creation of an abstracted map by the map researcher and psychologist Maxwell J. Roberts [90], [91] and (f)–(h) artistic abstraction by the visualization researcher and designer Till Nagel [79]. (a) topographically correct map of the London Underground tracks, (b) map distorted and re-oriented, (c) manually designed abstraction overlaid, (d) final abstract map design, (e) alternative design with curved tracks, (f) topographically correct map of the Shanghai Metro tracks (with cars), (g) some abstraction applied, and (h) full abstraction with a layout inspired by the official Metro map. Images (a)–(e) © Maxwell J. Roberts, used with permission. Images (f)–(h) © Till Nagel, used with permission.</figcaption></figure>

understanding of visual abstraction for spatial 3D data include the discussion of the field of illustrative visualization by several authors (e. g., [6], [51], [61], [62], [112]) and, in particular, by Rautek et al. [88]. Rautek et al. distinguish *low-level visual abstractions* which incorporate the use of stylistic rendering techniques that do not provide any dedicated control over the abstraction process. In contrast to such low-level methods, Rautek et al. also describe *high-level visual abstractions* that are inspired by “expressive techniques” used by illustrators such as “cutaways, breakaways, close-ups, or exploded views.” They also state, however, that even higher levels of abstraction are needed to support reasoning and insight communication—a fundamental challenge that still stands today.

Only few approaches in the subfield of illustrative visualization<fnref n="2" /> have investigated such data exploration strategies that purposefully control the abstraction in a visualization. For structural biology, Zwan et al. [110] established an abstraction space that allowed them to control structural abstraction of molecules (Fig. 5), abstraction by means of their depiction style (Fig. 10), and abstraction through the support of depth perception independently. This approach can be used to interactively explore a continuous three-dimensional abstraction space [110] or to adjust two of these abstraction dimensions depending on the location in a single illustration [69] (Fig. 11(a)). Also in molecular visualization, Cipriano et al. [24], [25] derive surface abstractions of complex molecules with varying levels of scale that abstract from the specific molecular structure and that support the higher-level analysis of molecular interactions. In the same domain, Parulek et al. [85], [86] demonstrate how to use different molecular surface representations at different scale levels and show how to seamlessly transition between them (Fig. 11(b)), to facilitate the interactive analysis of complex molecules. In brain connectivity analysis, Böttger et al. [15] and Everts et al. [32] use bundling/contraction strategies to visually expose nerve connections between different brain parts (Fig. 7), inspired by related work

in abstract data visualization [44]. Abstraction also plays a role in the interactive visualization of urban landscapes [35], [97] as well as in the creation of map-based visualizations [2], [50]. Within cartography, the dedicated control of abstraction plays a role, in particular, in the creation of maps of transportation networks [84] (e. g., Fig. 1) and in general navigation (e. g., Fig. 12).

In the visualization of non-spatial data, the term abstraction is used when original data is shown in another form than drawing every data item on the display. Such techniques are defined as multi-resolution visualizations and are realized through filtering, clustering, or sampling the original data. To represent the information effectively, quality metrics have been introduced that compare the original visualization with the abstracted multi-resolution representation. Cui et al. [27], for example, have proposed histogram-difference and nearest-neighbor measures, while Johannson and Cooper [55] used quality metrics based on similarity of distance fields derived from visualizations. Earlier, Tufte [108] proposed his *data-ink ratio* to encourage minimalist data representations<fnref n="3" />—in a way also a form of abstraction in visualization. Yet, while all these approaches present important contributions to the understanding of the role of visual abstraction for spatial or non-spatial data, they all are only isolated solutions that, individually, do not shed light on a more general model for the use of abstraction in visualization.

Instead we can thus look at other fields that can inform our discussion of abstraction. For example, *semiotics* is the study of symbols and their relation to physical artifacts or meaning they represent. Visual abstraction (e. g., from object to its corresponding symbol) is, therefore, very deeply rooted in this philosophical discipline. Bertin [10] has established an encyclopedic classification of visual symbols and their relation to data. Ware [115] discusses the symbols from the context of whether they have a direct relation to the object they represent and, therefore, can be immediately understood, or symbols that need to be learned. The first category of symbols which can be interpreted without learning comprises *sensory* ones. Here, perceptual processing creates the link between

<footnote n="2">2. So far, illustrative visualization techniques have largely been created for spatial data (like our focus in this article)—only few examples for the illustrative visualization of “abstract data” exist (e. g., [4], [41], [59], [68], [73], [83], [111], [114], [116]; also see Fig. 9). Many of these examples, however, also concentrate on stylization (rather than abstraction) aspects.</footnote>

<footnote n="3">3. The minimalist data representations encouraged by the data-ink ratio have since then been criticized (e. g., [5], [49]).</footnote>
the object and corresponding symbol. The second category are arbitrary symbols where there is no relation between the object and its symbol from the perceptual point of view.

Cox [26] relates the data-visualization mapping to metaphors, arguing that every visualization is a metaphor of the data and should always be seen like one. Moreover, different metaphors on the same data can be applied. Cox' use of the term (conceptual) metaphor in the context of data visualization suggests that the mapping is, in fact, an abstraction.

The term *abstraction* is also used in the context of visualizations related to a user's task or the acquired data. Munzner [77] describes a task taxonomy, classifying tasks at the highest level as *analyze*, *search*, and *query*. The data can be characterized by its *statistics*, *derivatives*, *data element relations*, its *attribute relations*, or *shape descriptors*. We see visual abstraction, while it is related to the task and data abstractions, as orthogonal to Munzner's classification.

In the field of computer graphics, abstraction is typically interpreted as simplification (similar to the generalization in cartography). For example, in geometric modeling researchers have derived techniques to remove detail from geometric models to maintain the overall shape (e. g., [74], [117]). Similarly in image processing, abstraction can remove detail from both shape and color values (e. g., [57], [96]). An alternative notion of abstraction in computer graphics was proposed by Gomes and Velho [37] in form of a *universes paradigm*: A physical object is on the lowest level of abstraction, its mathematical description is the first level of abstraction, which is further abstracted in a discrete representation, and the highest level of abstraction is the implementation universe. These abstraction levels interact with each other and a mathematical concepts of mapping can be utilized using this paradigm. While Gomes and Velho's work [37] thus allows them to relate different forms of objects and shapes or their (theoretical) description with each other, we are more interested in different forms of visual representation of data and the relationships between them.

Hibbard et al. [43] also borrow mathematical reasoning for establishing a relationship between the data and its visualization. They define a display function $D: U \to V$, where $U$ is a set of data objects and $V$ is a set of displays. Each set they define as *lattice* which means they create a partial order of models where each model is ranked by how well it corresponds to the underlying ideal mathematical objects. Relating the lattices $U$ and $V$, the function $D$ is a display function if and only if it is a lattice isomorphism. This concept by Hibbard et al. [43] relates to our own as the lattices can be understood as abstraction levels.

The work most related to our own conceptual understanding of abstraction in the context of illustrative visualization has been compiled in the book edited by Strothotte [101]. In this work, the definition of abstraction has been stated and various forms of abstraction have been suggested. The first two features of the process of abstraction Strothotte discusses in Section 1.3.2 ("gradually removing detail and adjusting the rendering style" and "adjusting the size, shape and orientation of parts of a model in combination with their level of detail and their style") are similar in nature to what we previously noted about the use of abstraction in computer graphics in general and somehow relate to our photometric and geometric abstraction axes, discussed in our Proposition 3.4 Strothotte's third feature of the process of abstraction ("bringing text and graphics into unison with one another") raises the interesting question of how an image can be abstracted (*verbalized*) into a text. While it is

not a focus of our paper, the authors also comment on abstraction in user interface design where a simplified interface is an abstraction of the full functionality and flexibility of the corresponding system. Abstraction has been also mentioned as the key for being able to deal with different scales, in particular in the geographical context. In the final chapters of Strothotte's book [101], the authors touch upon the topic of continuity and discontinuity in abstraction.

To facilitate a more in-depth discussion and to allow us to better understand the concept of abstraction with respect to (illustrative) visualization we thus propose, in the remainder of this article, a first formal definition of visual abstraction, we categorize its forms, we examine its advantages and constraints, and we investigate those forms of visual abstraction that have not been in the scope of visualization and computer graphics research to date. We demonstrate the various visual abstraction forms, in particular, on structural biology data due to our specific background and as this field offers a rich spectrum of aspects on which one can apply abstraction.

# 3 THEORETICAL FOUNDATIONS

Based on this understanding of existing work on abstraction in visualization/graphics, we now attempt a more formal discussion of the concept. We start by defining some basic terms and then formalize several aspects of abstraction.

## 3.1 Basic Terms

The term *abstract* is an antonym of *concrete* or *tangible*, resulting in an inherent difficulty to describe it. The act of *abstraction* is consequently hard to capture. We set the following definitions of the terms *abstraction*, *visual representation*, and *visualization* as a baseline for our reasoning.

**Definition 1.** An *abstraction* is a transformation which preserves one or more key concepts and removes detail that can be attributed to natural variation, noise, or other aspects that one intentionally wants to disregard from consideration.<fnref n="5" />

**Definition 2.** The term **visual representation** refers to any graphical form that can be perceived as a stimulus of the visual system and that is further processed by means of perceptual and cognitive machinery.

Visualization uses visual representations to encode data for humans to look at and gain insight from. It is typically realized as a concatenated pipeline of data *acquisition*, *filtering*, its *mapping* onto geometry, the assignment of *visual* representations, from which an image is synthesized in the *rendering* step. People can then view these images, perceptually and cognitively process them, and consequently gain insight [20].<fnref n="6" />

**Definition 3.** Visualization is a multi-stage transformation of digital data into visual representations which are cognitively consumable by humans.<fnref n="7" />

<footnote n="4">4. The geometric abstraction is here exemplified on Feiner's APEX [33].</footnote>

<footnote n="5">5. Before converging to a presented definition, we have investigated the usage of the term *abstraction* in various online dictionaries and related literature [101] so that our formulation fits the majority of them.</footnote>

<footnote n="6">6. While we consider interaction as a key integral part of data visualization, in this paper we put emphasis on the visual part of the pipeline.</footnote>

<footnote n="7">7. Many visualization pipelines that describe this multi-stage transformation have been proposed; a frequently cited pipeline is the one by Card et al. [20].</footnote>
<figure><img src="image_6.png" /><figcaption>Fig. 2. Seven visual abstractions of the heart muscle physiology (a detailed description of the figure is contained in the main text of the article).</figcaption></figure>

## 3.2 Visual Abstraction

As part of this visualization pipeline, data is thus abstracted and detail is removed. In illustration, infographics, data visualization, and the visual arts, however, the term visual abstraction is frequently mentioned when loosely referring to certain visual encodings. Based on Def. 1–3, we can state its meaning more precisely as:

**Definition 4.** Visual abstraction is a concept-preserving transformation used in visual arts and data visualization, which transforms (digital) information into visual representations by removing details attributed to natural variation, noise, etc.⁸

**Proposition 1.** Visual abstraction may serve a plethora of purposes. In general terms visual abstraction can promote **understanding**. It can also **simplify** a complex structure to reveal a membership of an individual to particular species or particular behavior to a corresponding behavior type. Given a dynamic phenomenon, visual abstraction can **summarize** the process into main stages. Visual abstraction can be further utilized in **guiding** attention both to structural or temporal aspects. Another example of visual abstraction utilization is interpretation of a particular phenomenon and extracting its elementary principles.⁹

Along the described visualization stages, visual abstraction is used predominantly when non-visual data is transformed into visual representations. In the visualization of certain 3D structures, visual abstraction can affect the geometric representation, shading technique, positional transformations, camera parameters, or a combination thereof. For dynamic phenomena or phenomena that are complex and that consist of an entire spectrum of important aspects, the term visual abstraction can be understood as a composite function that combines together several transformations resulting into a complex visual abstraction.

Let us explain this concept using a specific example of a visualization of the heart muscle function (refer to Fig. 2). The acquisition that captures the heart muscle function can be done, e. g., using a 4D ultrasound device coupled with an electrocardiography (ECG) sensor. A resulting visualization can employ visual abstraction to promote different aspects of the studied phenomenon. The electrocardiogram with a time-intensity plot is one possible visualization of the phenomenon. Such representation abstracts

8. This definition simply combines aspects from abstraction (Def. 1) and visualization (Def. 3).

9. Conversely, pure **stylization** is not necessarily a visual abstraction according to our definition, unless there is a *concept* associated to such a visual representation.

away every aspect of the heart function besides the electrical potential magnitude changes over the pump's cycle. The visualization of a 2D slice through a 3D structure from the echocardiogram shows an entirely different view, while it is still conveying the same underlying phenomenon. A 3D visual representation shows the overall shape of the organ. By extracting specific heart muscle landmarks from patient data, doctors can personalize a generic heart muscle model, represented as a wireframe model or as a shaded and textured model. Furthermore, Doppler imaging and strain-rate imaging provide physiological information about the muscle, for example using what is called a bulls-eye plot that shows the strain rate in a polar view from the left ventricle tissue. This example lists seven arguably different visual abstractions of one complex physiological phenomenon. If we take into account multiple spatio-temporal scales besides the tissue level, considering cell level down to the molecular level, the number of meaningful visual abstractions would grow substantially.

## 4 ASPECTS OF VISUAL ABSTRACTION

Based on these theoretical foundations we now discuss the aspects of visual abstraction that are important for our model. First, we argue for different axes of abstraction and abstraction spaces that we bring together into one overarching model. We then discuss how visual abstraction can be used to convey multiple data attributes. Next, we investigate how abstraction relates to temporal phenomena and, finally, examine aspects that need to be observed as abstraction covers multiple scale levels. In our discussion, we refer back to the related work and use examples to illustrate our points. In the following two sections, we then propose a path toward an overarching theory (Sect. 5) and discuss how our considerations raise several important questions for research on abstraction (Sect. 6).

### 4.1 Axes of Abstraction

We can see that abstraction affects very diverse aspects of the data and its visual representation. We can thus generally say that it happens along different axes of abstraction—each having its own meaning, purpose, and characteristics:

**Proposition 2.** Visual abstractions, depending on the meaning or intent of a particular visualization user, can lead to fundamentally different transformations from the acquired digital information to visual representations. Given a finite set of meaningful transformations, these would then be points in a visual abstraction **space** for a given phenomenon. A visual abstraction along a
<figure><img src="image_7.png" /><figcaption>Fig. 3. McCloud’s visual abstraction space. Image from [72], © Scott McCloud, used with permission.</figcaption></figure>

single data attribute of a studied phenomenon will be consid-
ered as a pure visual abstraction and will define an **axis** of the
multi-dimensional visual abstraction space.<fnref n="10" />

Several prior works from visual arts and visualization also es-
tablish a relation to a space spanned by different visual abstractions
or abstraction axes. For example, McCloud [72] describes an ab-
straction space as a triangle with vertices defining extremal points
(Fig. 3). One vertex is *reality*, which could be also considered as
the visual appearance of an object without any abstraction applied.
The horizontal abstraction direction towards *language* depicts sim-
plication, generalization, or conceptualization of the particular
object instance. The top vertex is what McCloud calls *the picture
plane*, which somehow represents a mixture of visual elements,
sketchiness, style, or medium. While the abstraction along the hori-
zontal edge relates to object or object space, the vertical direction
upwards is increasingly related to elements in the image space. In
the context of data visualization, this triangle could be redrawn
such that the *reality* vertex would be the origin of our imagined
abstraction space, the *language* vertex would define the direction
along the geometric visual abstraction, and the *picture plane* vertex
is defining the orientation along the photometric visual abstraction.
These two abstraction terms will be further discussed below.

If we look at the works of masters of visual arts such as
Mondrian or Picasso, their artwork documents experiments with
varying levels of visual abstraction. For instance, Picasso explored
the simplification of geometry of the presented object as well as
the shading style from detailed shading, via flat shading, to line
drawing in his lithography series “The Bull” (1945–46).<fnref n="11" /> His
visual abstraction gradually removes all detail until only a kind of
geometric skeleton represented by lines is preserved. A hypothesis
about two abstraction axes can thus be drawn from these art studies:

**Proposition 3.** Visual abstraction of (three-dimensional) structures
comprises **at least two axes of visual abstraction**: the geo-

<figure><img src="image_8.png" /><figcaption>Fig. 4. Flow data depicted with a *direct* (LIC) visualization that is overlaid with the corresponding topology, i. e., the abstraction of the flow [105]. Image © and courtesy of Tino Weinkauf, used with permission.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 5. Structural abstraction for molecules: space fill, balls-and-sticks, licorice, backbone, ribbon. From [110], images are in the public domain.</figcaption></figure>

metric and the **photometric** one.<fnref n="12" /> The geometric axis would
abstract from particular shape details of the 3D object, while
preserving distinct features that characterize a given 3D struc-
ture. Often some form of a semantic or topological skeleton
would drive the geometric simplification. The photometric axis
would simplify the light propagation to simpler shading tech-
niques such as flat shading for example, furthermore to a line
drawing and finally to a silhouette.

Topology, namely scalar and flow topology [60], [87] as well
as skeletonization [98] are useful tools for geometric abstraction.
Fig. 4 shows flow characteristics *directly* using the Line Integral
Convolution visualization method and a corresponding flow topol-
ogy which is a higher geometric abstraction depicting critical points
and separatrices, partitioning the domain into conceptually different
flow behavior. Geometric abstraction techniques also relate to the
level-of-detail schemes used in computer graphics applications
(e. g., [45], [74]) where, however, the primary intent might be com-
putation time reduction instead of visual abstraction *per se* [85].
Geometric abstraction can—besides performing a topology-preser-
ving simplification—be driven by high-level semantics rather than
by topological skeleton extraction techniques. If a structure can be
decomposed into chemical, physical, functional, or other semantic
subunits, the abstraction can change the geometry substantially
from one representation to another one, encoding semantics on a
higher level of detail, e. g., for molecules (Fig. 5, [110]). Even if
this decomposition is not necessarily inherent or many possibili-
ties exist, abstracting visualizations can show meaningful excerpts
from otherwise continuous datasets as it is frequently done, for
example, in flow visualization [14], [47], [100], [102] (Fig. 6).
Other examples for geometric abstractions are subway networks
(Fig. 1) and the contraction-based abstraction of brain connectiv-
ity [21], [22], [32] (Fig. 7). In geographic information systems,
maps are typically stored with the highest level of geometric detail.
For a larger perspective, structures such as borders, roads, and

<footnote n="10">10. Initial considerations of abstraction space and axes of abstraction are related to our earlier work on the abstraction of molecular structures [69], [110]. Based on this initial work, we investigated other visualizations and related work in the arts and illustration to arrive at the formulation of Proposition 2.</footnote>

<footnote n="11">11. See images from the series, e. g., at https://www.moma.org/interactives/exhibitions/2010/picassoprints/main.html#/states/lithography.</footnote>

<footnote n="12">12. While using the terms surface and shading abstraction resp. structural abstraction and ‘illustrativeness,’ these considerations have been stated in our earlier work [85], [110]. The same axes can also be found in abstraction approaches for other visual representations, such as the (map generalization) ones for traffic networks (see Fig. 1).</footnote>
<figure><img src="image_10.png" /><figcaption>Fig. 6. Examples for the use of meaningful excerpts to visually abstract continuous datasets (stream surfaces of a 3D flow simulation in this case). Images from [14], © IEEE, used with permission.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 7. Abstraction of brain connectivity through fiber tract contraction [32]. Images © Everts et al., used with permission.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 8. Jianu et al.'s [54] projecting abstraction of brain connectivity to support bundle selection. Image © Radu Jianu, used with permission.</figcaption></figure>

streets are automatically abstracted into simpler representations to communicate significance. While there is substantial amount of semantics involved in this process called *map generalization* [19], this simplification process would fall into the category of geometric abstraction. Either as part of the geometric abstraction axis or potentially as an independent one, geometric projections (e. g., from 3D to 2D) also introduce abstraction. For instance, Jianu et al.'s [54] interactive brain connectivity visualization system simplifies fiber tract bundles and projects them to 2D space to facilitate the interactive selection of fiber tract bundles (Fig. 8). Such bundled/clustered and thus simplified representations can also be found for representations of non-spatial data such as parallel coordinates [59], [73], [83], [114] (Fig. 9).

The photometric abstraction axis, instead, simplifies the illumination and/or shading—while preserving visibility of key structural features—and retains shape and depth understanding. We can see much research going into the direction of abstracting the light propagation, while still preserving the realistic appearance at much lower computational costs. One way how to approach the abstraction of the illumination model is to define how global the illumination algorithms are. More local illumination models would then be higher abstractions of realism. A nice example of the potential of

<figure><img src="image_13.png" /><figcaption>Fig. 9. Examples of illustrative abstractions of parallel coordinate representations. Images (a) © Matej Novotný (from [82]) and (b) © Kevin T. McDonnell (approach from [73]), both used with permission.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 10. Abstraction along 'illustrativeness': 'photorealistic,' cel shading, and black-and-white. From [110], images are in the public domain.</figcaption></figure>

this axis is the use of ambient occlusion (e. g., [104]), a form of shading between local and global illumination. In van der Zwan et al.'s work [110] this abstraction axis would roughly correspond to spatiality (i. e., presence or absence of depth cues).

From local illumination models, an abstraction can continue further by transforming large intensity gradients into sharper intensity transitions [30], up to abstractions into line drawings [65]. While this visual abstraction is of photometric nature, perhaps there are more abstraction axes that deal with photometric aspects of the visualization. Van der Zwan et al. [110] suggest that a separate but related axis denoted as the *illustrativeness* could be used (Fig. 10): a transition from “photorealism” to toon shading to line drawing.

It is an open question whether the aspect of texture is a part of geometric abstraction, of photometric abstraction, or if it forms a separate category. Considering texture as a surface detail it would relate it to the geometric abstraction axis, while seeing the texture as a material property would relate it rather to the photometric abstraction axis. Further below we discuss the abstractions across scales and, in this regard, we can understand a texture as a photometric abstraction of (a much more detailed) geometry.

Finally, while both photometric and geometric abstractions communicate key aspects of the data, they additionally convey a level of certainty about the data itself. When using photorealistic architectural visualizations for planned real estate, the expectations often do not match the built reality. Photorealism, in such a case, communicates a high level of certainty. A sketchy drawing often conveys that the idea is still being formed and might differ from the reality in certain level of detail (e. g., [95]).

**Proposition 4.** Visual abstraction can convey uncertainty associated to the displayed data/information. Photorealistic depictions have virtually no uncertainty, while sketchy drawings covey rather conceptual ideas rather than physical implementations.<fnref n="13" />

The existence of several abstraction axes raises the question of when axes can be considered to be independent from/orthogonal to each other. The independence of axes arguably requires that changes introduced by one are not affected by changes along another. The semantics of the introduced abstraction, however, may also affect their independence:

<footnote n="13">13. This argumentation is inspired by Schumann et al.'s [95] work and is supported by several architectural modeling/rendering systems; e. g., Piranesi [89].</footnote>
**Proposition 5.** Axes in an abstraction space can be considered **orthogonal** to each other if their respective changes to the visual representation are independent of each other and if the semantics of the introduced change is unique for each axis.¹⁴

## 4.2 Abstraction for Conveying Multiple Data Attributes

Bringing back the focus to the challenge of communicating a comprehensive insight about a complex phenomenon, a central question here is how to distribute the visual estate to provide essential information about each relevant data attribute of the underlying data. Visualization systems must necessarily contain, generally speaking, a compositor layout that integrates the data attributes into the final visualization.

Rautek et al. [88] had initially categorized the visual abstrac-
tions into low-level and high-level visual abstractions. The low-
level ones were stated to deal with shape representation and its
appearance. The visual abstractions we mentioned above would
all fall into the category of low-level visual abstractions as they
are describing how to show a particular object. The high-level
visual abstractions, on the other hand, determine what to show
and how dominant a particular structure should be in the result-
ing visualization. Revisiting this categorization with an integrative
multi-attribute or multi-variable visualization in mind, the low-level
visual abstractions are, in fact, visual encodings, while the high-
level visual abstractions are complex n-ary compositing operators
applied on the visual encodings of individual data attributes. The
work by van der Zwan et al. [110], however, suggests that there
may be a continuous transition (i. e., one where the visual repre-
sentation does not abruptly change) from low-level to high-level
visual abstractions such that we can state:

**Proposition 6.** Visual abstractions determine the visual appearance of structure, but also which structural component should be visually emphasized and which should be rather suppressed. Visual abstractions often gradually introduce changes to visualizations, and **many axes of abstraction are continuous**. Other abstractions, however, are introduced as **discrete** steps such as dimensionality reductions.

**Proposition 7.** Each axis of abstraction can affect the visualization at a different level of semantic conceptualization, at a **low level**, a **high level**, or a **range of levels** of abstraction.

Geometric and photometric abstractions would predominantly fall
into low-level abstractions end of the spectrum. Occlusion man-
agement techniques, focus+context techniques, semantic zooming
techniques, and visual guidance would fall into high-level abstrac-
tions. Depth of field, for example, could be considered to cover a
range of abstraction in-between the extremes.¹⁵

## 4.3 Abstraction Conveying Emergence and Process

So far we have been concerned with an abstraction that applies
to a static structure. The phenomena studied using visualizations,

however, often take place over time. Their own development or the
process in which they take part often forms a sequence or graph of
events that we generally denote as *story*. A story can be abstracted
into essential information—for a linear process this corresponds to
a story line, consisting of nodes defining a state and edges defining
transformations between these states. In more complex processes
in which several structures, i. e., *actors*, are involved that affect
each other's state a story can be abstracted into an interconnected
story-graph instead of a story-line.

Sometimes the process can be abstracted furthermore into a set of elementary rules for structures taking part in the story. These rules govern how the story develops over time. Such construction is often coupled with randomness, resulting into a *recipe* of a stochastic process. This approach is widely used in procedural modeling in computer graphics and is a frequent approach for modeling in general (context-free grammars, context-sensitive grammars, individual-based modeling, etc.).

**Proposition 8.** *Dynamic and procedural phenomena can be abstracted into a story* described by nodes of states and edges of transformations. For a single entity involved in the process, an *actor*, the abstraction would form a story-line, while the story of several interacting actors would result into a story graph. Such a representation would be an explicit form of a procedural description. An implicit form of describing a procedure would be by defining a rule set for each actor and by letting the actors interact with their environment will model a desired process.¹⁶

Based on this concept of a temporal process, we need to under-
stand how such a process affects abstraction and how the temporal
processes themselves can be abstracted.

**Proposition 9.** *Three trivial visual representations exist for visually abstracting a process.* One representation is a direct animation of the process emergence itself, another is the explicit story graph, and the third is the implicit rule-based state machine abstraction.¹⁷

A straightforward animation, however, presents several prob-
lems as we explore visual abstractions or even visualizations in
general. The cognitive sciences have a split opinion on the value
of animation [109]. Animation is, on the one hand, praised for its
value for various kinds of tutorials; on the other hand, it is not
considered to be a good method for conveying phenomenal emer-
gence. Visualization of data often serves as an *external memory
with fast random access* for the severely limited working memory
of attention. Animation thus counteracts this value, as a particular
state is conveyed to the viewer only for a short instance of time,
while adding the benefit of understanding the motion of the data
instances [92]. To date no satisfactory solution exists that would
couple animation with a fast random access to the data. In addition,
animation as a change of the displayed information is invoked by
user interaction (e. g., users often change viewpoint settings, visual
parameters), resulting in a change of the data appearance [3], [11],

14. In our previous work on an abstraction space for molecular visualizations [110], we described the three axes to be independent and thus orthogonal. The analytical work of Livingstone on famous visual arts [67] suggests that various abstraction aspects must act together in a harmony. So while the abstraction axes could be analyzed independently, they are also linked in the sense that a combination of particular individual visual abstractions must match together.

15. In our work on importance-driven feature enhancement [113] we introduce the terms *levels of sparseness* and *importance compositing*. The former relates to low-level, while the latter one to high-level abstractions.

16. These abstractions relate to categories in computer animation. The stonyline or -graph would relate to keyframing, while the implicit form relates to agent-based animation. Such relation suggests that perhaps there are other dynamic visual abstractions related to inverse kinematics or other categories of computer animation.

17. Direct animation means no abstraction unless there are artificial cuts created, the storyline or -graph relate to the craft of comic [72], and storyline visualizations [66], [103] and the state machine is defined as an abstract machine from mathematics and computer science which represents procedures and sequential logic circuits.
[31], [70]. The third scenario where animation is used in visual- ization are smooth animated transitions between visual encodings of the same data revealing different aspects [7], [42], [48], [69]. These three animation types in visualization have to be distinct so that a viewer can decode which type of animation was shown:

**Proposition 10.** For time-dependent data, an **inherent ambiguity** exists **between the animation of the data and the changes of a visual representation** resulting from an interactive exploration or animated transitions between visual encodings. Such ambiguity leads to a general guideline for visual abstraction stating that a **visual abstraction must not be misinterpreted for being part of the original data.**<fnref n="18" />

This aspect of abstraction has not been comprehensively explored in visualization to date and we thus have to investigate how we can use abstraction of the temporal aspects of a dynamic dataset in order to be able to distinguish them from the necessarily continuous changes that result from the interactive exploration. We envision a visually clearly separate representation of these two types of effects that result from the dynamic nature of the data and interaction—in particular through abstraction of the temporal changes. One inspiration for this ultimate goal could come, for example, from work in non-photorealistic rendering that has developed techniques for the portrayal of sequences, such as for furniture assembly instructions [1].

## 4.4 Abstraction Across Multiple Levels of Scale

In addition to the separation of abstraction effects into different axes, we also need to consider the spatial or temporal scale level of our data. We encounter dramatically different levels of scale, for instance, in biology: Researchers study whole organisms, how individual organs behave, how building blocks of organs work, how individual cells are operating—down to the mechanisms at the molecular level of proteins and genetic information. Here it is essential to convey how these distinct scales mutually interact. We thus need to study the interaction of abstractions between these levels and understand visual abstractions that operate across spatio-temporal scales.

**Proposition 11.** Visual abstraction is a key ingredient of a **multi-scale** data visualization. At each scale, a visualization should convey properties that characterize this given scale. To understand how adjacent scales relate to each other, however, a visual integration of these scales is necessary. A continuous visual abstraction transition between these scales can be an effective way to convey their mutual relationship. While the transition across different spatial levels of scale in itself constitutes a form of structural abstraction, it is likely that other forms and axes of abstraction need to work differently at different levels of scale—both spatially as well as temporally.<fnref n="19" />

For example, researchers have investigated the visual integration of different scale levels of 3D spatial phenomena using multi-scale zooming techniques [46], using changes in geometric representations for genetic molecules [69], using LOD representations [40], or using the blending of different rendering approaches (photometric abstractions) for the different scales at which proteins were studied [63], [86] (examples in Fig. 11). Maps are another

well-studied example, where different levels of scale need to be considered for spatial data. As noted before, cartography has developed its own rules in abstracting details, known as map/cartographic generalizations [19]. These include the following operators applied in combination: selection, simplification, combination, smoothing, enhancement, and displacement. Fig. 12 shows examples of different scales and the applied cartographic generalizations.

Within illustrative visualization, Agrawala et al. [2], e. g., have demonstrated that on route maps different scale levels need to be integrated into a single visualization due to the different information needs at various points along a route. A similar case can be made for tourist maps [39], where only some data attributes are shown in high detail, while others can be abstracted further. Moreover, abstraction across different levels of scale is often guided by a set of rules that depend on the application domain, such as Gestalt rules in architecture [80]. These rules control the effect of abstraction at a given scale level—essentially a form of semantic zooming.

**Proposition 12.** Multiscale visual abstractions offer varying levels of detail. Some generic visual abstractions are thus closely related to **level-of-detail techniques** from the real-time rendering literature, others can be found in **map generalizations** from map and cartography designs. These techniques need to consider the particular application domain, so that constraints are observed for given scale levels and that transitional inter-level

<figure><img src="image_15.png" /><figcaption>Fig. 11. Spatially continuous change of abstraction for (a) the DNA [69] & (b) a protein [85]. (a) © Lueks et al.; (b) © EG; both used with permission.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Fig. 12. Scale abstraction (cartographic generalization) in OpenStreetMap.</figcaption></figure>

<footnote n="18">18. The proposition can be clarified on an obvious example: the visualization user needs to know whether the camera has moved or the data point.</footnote>

<footnote n="19">19. This proposition arose primarily from our work on spatially integrated geometric abstraction for molecular data [69], [85], [86].</footnote>
<figure><img src="image_17.png" /><figcaption>Fig. 13. *Graph Trinity* from left to right: process descriptions, entity relationships, and activity flows are schematic SBGN representations of reaction pathways that can be seen as different hierarchical levels of the same dynamic phenomenon. Image from [81], © Nature Publishing Group, used with permission.</figcaption></figure>

representations remain meaningful to the domain user.<fnref n="20" />

Similar to the multi-scale visual abstractions related to the structure, the process (emergence or, generally speaking, a *story*) can be abstracted along multiple scales. In biology, for example, we can observe heart muscle physiology on a tissue level, where we observe the cycle responsible for the blood circulation. The next level on which the process can be observed is the study of a group of muscle fibers acting together to create a desired contraction pattern. Several scales down we can observe how groups of cells are contributing to the contraction of a single fiber and, ultimately, we can observe individual muscle proteins interacting with each other powered by the ATP molecules. Each scale of this process is carried out within a certain timescale: while a second is reasonable measure for a heart cycle, the observation of one second of processes on the molecular level such that the details of all molecular interactions are clearly conveyed would take more than a lifetime.

Le Muzic et al. [64], for instance, use visual abstraction to simultaneously view two temporal phenomena, whose occurrence frequency differs by three orders of magnitude. This work has touched upon the abstraction of a trajectory of individual molecular elements. The shape of the trajectory has been simplified analogously to geometric simplification approaches, which suggests a relation that motion path can be a subject of a geometric abstraction. In another work from cell biology [28], a model of cell content is used which consists of a geometric structure where molecules, modeled in detail, are acting as agents in their environment. The same model can be viewed on a low magnification level, where the activity is abstracted into a intensity of a particular color, mimicking molecular activity staining. So the same phenomenon has two different visual abstractions, depending on the inspection scale. Similarly, polymerization can be observed at different levels of spatial and temporal scales [58]. Such biological reaction networks data can also be represented in a schematic way as graphs, as it is common for visualization of reaction pathways. A schematic representation of pathways, denoted as Systems Biology Graphical Notation (SBGN; see https://sbgn.github.io/sbgn/), can be observed at different scales of detail (e. g., Fig. 13).

**Proposition 13.** A multiscale process can be observed on various scales and, on each scale, a fundamentally different visual ab-

<figure><img src="image_18.png" /><figcaption>Fig. 14. Multiscale visualization of DNA nanostructures with scale-adaptive modifications. Image from [76], © IEEE, used with permission.</figcaption></figure>

straction might be meaningful. A molecular metabolic process, for instance, can be conveyed in detailed structural view through an animation, on the cell level it can be depicted diagrammatically, and on the body level through hot spots describing the concentration changes over time. The interaction with the different scales of abstraction of a visualization, however, may or may not follow similar principles.

Miao et al. [76], for instance, described a multiscale visualization of DNA nanostructures and, in particular, *scale-adaptive modifications* of the datasets that adapt to the used scale level and act in a similar way across multiple scale levels (Fig. 14). In other examples, however, different interpretations of similar interaction techniques may be meaningful for different abstraction levels.

# 5 CONSTRUCTING AN OVERARCHING THEORY

Previous sections have discussed various forms of visual abstraction that could possibly span an abstraction space. Besides the abstractions related to the 3D structure and illumination, we showed that multiple attributes can compete for visual estate, we discussed the temporal aspect of visual abstraction conveying the process or structural emergence, and we argued that all these abstraction types might be characteristic only for one spatio-temporal scale and that abstractions designed for a single scale might possibly be integrated with visual abstractions characterizing adjacent scales.

Building theoretical foundations about visual abstraction will be especially beneficial for the science of visualization, as the field of visualization generally lacks on theoretical foundations [56]. Such theory could be seen as a “kernel” of visualization because abstraction is at the heart of each step of the visualization process. The developed

theory will have a fundamental impact—not only on future visualization research but also on the development of interactive visualization software and toolkits. These tools are expected to integrate means to provide and control visual abstraction for a variety of application problems and domains. Taking a step back, we can see that an initial structuring of visual abstraction emerges. The axes of abstraction seem to be associated with different embedding spaces, as depicted in Fig. 15.

<figure><img src="image_19.png" /><figcaption>Fig. 15. Schematic view of the visual abstraction space spanned by four axes where each is associated with a distinct space.</figcaption></figure>

<footnote n="20">20. Our work on structural abstraction of molecular data [69], [110] demonstrated this point. The abstraction from the space-filling representation via the balls-and-sticks, licorice, and backbone views to the secondary structure of the ribbon representation could easily be constructed in a seamless way. Transitions from such internal views to surface-based representations [24], [25], [85], [86] or higher-level structures [38], however, are not straight-forward.</footnote>
<figure><img src="image_20.png" /><figcaption>Fig. 16. Example for sub-axes of structural abstraction for 1ANK: a molecular structure can both be abstracted using a surface representation (top; using Cipriano and Gleicher's [24] technique, images © IEEE, used with permission) and using its "internal" structure (bottom; images generated with Zwan et al.'s [110] approach).</figcaption></figure>

**Proposition 14.** Our initial **visual abstraction space** for conveying complex spatio-temporal phenomena is spanned by four independent axes. **Geometric** abstraction is associated with a 3D object, techniques and algorithms that perform geometric abstraction operate in object space. **Photometric abstraction** is predominantly associated with the image space. We also identified a **temporal** axis of **abstraction** that is associated with changes over time. Across these three independent axes we cast additional **scale abstraction** axis associated to the scale space.

The concept of an abstraction axis, however, should not be understood in a strictly mathematical way. Within each of these abstraction directions there may be different sub-axes that can or may not form a clear linear progression of abstraction. For instance, the structural abstraction of molecules can use both an external (surface) representation and an internal (generally line/graph-based) representation (see, e. g., Fig. 16). In addition, it is yet unclear how the scale axis interacts with progressions along the other

three axes. While it is possible to combine different abstraction levels along different axes to some degree (e. g., Fig. 17), it may not be useful to use extremely distant degrees of abstraction along different axes in one image—this could send mixed messages. This first attempt is thus likely to be incomplete. One could also consider further spaces with associated abstraction axes, such

<figure><img src="image_21.png" /><figcaption>Fig. 17. Use of different degrees of abstraction along different axes: the background of the ion channel 2XQ3 combines a low structural with a high photometric abstraction, while the strand in the foreground uses high structural and low photometric abstraction. From [110], image is in the public domain.</figcaption></figure>

as frequency where a histogram would visually characterize the occurrence of structural or temporal elements. Relating our generic abstraction space to the earlier discussion of abstraction in illustrative visualization [88], however, we can conjecture that:

**Proposition 15.** The visual abstraction space embeds all abstraction techniques that relate to image, object, time, or scale. Relating to previous terminology, these are low-level abstraction techniques. High-level abstraction techniques, on the other hand, are abstraction **compositing operators** that combine abstraction techniques, given a certain logic associated with a particular task or purpose. So the techniques and compositors together form a meaningful visual abstraction.

Both, abstractions techniques as well as operators, are visually processed. They are, however, often targeting different visual processing stages so we conjecture that

**Proposition 16.** Visual abstraction techniques are more related to perceptual processing of vision, and their application can be used to produce **sensory symbols** (Sec. 2). Visual abstraction operators, on the other hand, strongly rely on cognition and need to be learned, similar to **arbitrary symbols**.<fnref n="21" />

In addition to this connection to cognition and perception, several additional links to other fields that discuss abstraction can be made. Refinement in computing, for instance, is a concept that uses a verifiable transformation from an abstract program specification to a specific form that can then be executed. Our forms of abstractions do typically not have this verifiable character, they are typically based on decades or centuries of experience in the illustration and depiction of objects or concepts.

# 6 ABSTRACTION OPPORTUNITIES IN VISUALIZATION

Due to the current preliminary status of our concept, several detailed questions on the inner workings of the abstraction space remain unclear. Next we point out a number of these open questions, embedded into a discussion of several exemplary opportunities that an understanding of visual abstractions offers and we link these opportunities with particular future research foci.

## 6.1 (Semi-)Automated Visualization Design

Automating the use of visual abstractions for the purpose of generating effective and expressive visualization designs has been the goal of visualization for already some decades [71]. For the use of abstraction in visualization, however, we first need to understand the characteristics associated with these abstractions. Certain abstractions can be parameterized from a low to a large abstraction value, such as in case of 3D visualization the suggested geometric or photometric abstractions. We need to understand their validity ranges, whether or not these abstractions preserve specific aspects such as topology. Imagine a representation of a contextual 3D structure. The context is important for orientation purposes so it needs to be present in the visualization (e. g., [106]), but should take only as much of visual estate as really needed, the rest should be dedicated to the object in focus. But how much is *as much as really needed*? We thus need to characterize the amount of information human observers can extract from specific visual encodings defined by the visual abstraction and how the *gained* insight scales with increasing or decreasing the abstraction level. With such a characterization we could predict its effectiveness in a particular use case and possibly find a good compromise between its *value* (i. e., insight gained by a human observer) and *cost* (i. e., its image estate coverage, assumed cognitive load, computational requirements, etc.).

Based on these thoughts one may now propose to include the viewer's expertise as a separate axis of abstraction. We would argue, however, the viewer's expertise does not characterize the introduced abstraction itself but instead specifies the needs of abstraction for the viewer. This decision on the needed amount of abstraction is made before or during the creation of the visualization and can

<footnote n="21">21. The photometric abstractions are related to early stages of visual processing until the signal reaches higher visual processing centers (LOC, V4, IT). The geometric abstraction can be associated with the theory of *geons* proposed by Biederman [12]. The visual abstraction operators are always associated with a certain purpose or task, which are associated a cognitive processing.</footnote>
typically not be changed as the visualization is viewed. Even in
interactive approaches where the abstraction's type, degree, and
locality can be controlled (e. g., [52], [94]), the abstraction present
at any given time is fixed (for a given interpretation of abstraction).

An open question is how to obtain a meaningful characteri-
zation of visual abstractions. One systematic approach can be to
establish a formal protocol for how a visual abstraction should be
characterized, based on how correctly it conveys information to
humans, how long it takes to consume the visual representation,
and how acceptable the visual encoding is for a given individual
and for his or her task at hand (e. g., the different visualizations
in Fig. 2) or personality profile (e. g., some may prefer straight
transportation maps as in Fig. 1(d), others curved ones such as
Fig. 1(e)). Employing well-designed experimental user research
methodologies will result in a coherent statistical characterization
for the considered visual abstractions.

Establishing the characterization protocol is a research chal-
lenge on its own and is closely related to understanding the concept
of visual abstraction. Axes of abstraction can serve as a scaffolding
in such a characterization process. Yet, we still need to understand
the characteristics of these axes. For instance, are the axes continu-
ous or discrete (Proposition 6)? At which level of abstraction do
individual axes operate (Proposition 7), and how do we combine
them to cover the entire, potentially large abstraction space. How
do we efficiently and effectively control the traversal of these axes?
Are the axes independent from each other (Proposition 5) or do
they form alternative directions of abstraction (branching/forking)?
What are the characteristics of a resulting abstraction space?

## 6.2 Superimposed Visualization for Multi-Attribute Data

The problem with understanding data often relates to the complexity
of the analyzed phenomenon and the fact that data simultaneously
captures multiple attributes that need to be understood at the same
time [118].<fnref n="22" /> In the resulting visualization, however, they compete
for the same screen estate. In the example of context visualization
from Sect. 6.1, only one structure per spatial location in object
space could be shown because all structures were competing for the
same visual estate in image space. In a multi-attribute data scenario,
the competition for resources starts already in object space.

Today's visualization designs, therefore, cope with multiple
attributes through so called multiple-coordinated views [18] and the
brushing and linking metaphor [8]. Each subset of data attributes is
shown in a separate view and, through interaction with these in one
view, the analyst can observe how the data *behaves* in other views
that display different data attributes. It is this principle upon which
most current visual analytics frameworks are based. While such
setup scales well as long as the visualizations *fit* into the visual
field of view, the fact that the attributes do not share explicitly
the same spatio-temporal reference makes it hard to relate one
attribute to another. Interaction, selection, and visual links help to
restore the relationship across the data attributes, but these are only
loosely integrated, the interaction estate is by a large extent taken
by establishing across-attribute relationships, and selection, visual
links, and additional visual elements might add to visual clutter.

Instead of such a *juxtaposed* design [36], [53], relationships
across attributes could be better understood if the spatio-temporal
reference frame is more tightly shared among the data attributes

through superimposition [36], [53]—no interaction is then needed
to establish the relationships across data attributes as they are
explicit in the visualization. Designing effective and efficient super-
imposed visualizations is, however, much more challenging than
juxtaposed multiple-coordinated views. Sharing the same reference
space implies that data attributes fight for the visual estate at the
same locations. Moreover, the display of several data attributes
quickly leads to visual clutter. There is thus no general methodology
on how to design effective *integrated* multi-attribute visualizations.

With a thorough understanding of visual abstraction and means
to control it, however, such superimposed visualization of multi-
attribute data promises to become possible. We would need ways to
control the amount of visual resources spent on each data attribute
at each spatio-temporal location. There is no general methodology
yet, however, that would serve as a design guideline on how to
distribute the visual budget among the competing attributes of the
data. It thus becomes apparent that, especially for multi-attribute
visualizations, we need a deeper understanding and theoretical
foundations of visual abstraction. The different characteristics of
the axes of abstraction as mentioned in Sect. 6.1 will be essential
to be able to know how to apply abstraction to effectively encode
multiple data attributes in the same space. Moreover, we specifically
need to understand how the different axes affect each other and
how they affect the reading of the visual variables that encode the
displayed data attributes. We thus need to understand which axes
are mutually exclusive or which can be used simultaneously as
well as whether we can transition/blend between them.

## 6.3 Temporal Changes and Interpretation Ambiguity

A computational model is used in many real-world situations to
simulate possible developments of a certain process. For instance,
CFD simulations can lead to complex changes of the data while
molecular dynamic simulations generate hundreds of thousands
of simulation frames that are no longer possible to be inspected
manually—advanced computational instruments are necessary to
aid the data analysis. Visualization is a promising approach to
analyze such data, but straightforward visual playback will not
uncover hidden causal relationships. Yet, only few examples of
abstraction for time-dependent data exist (e. g., [34], [47]). More-
over, the situation gets even more critical when the resulting data
is streamed to the computer without storing every frame to enable
revisiting procedural details on demand. An abstraction of the
simulated process itself will thus be a crucial analytical instrument.

Understanding visual abstractions for procedures and dynam-
ics, such as story-line, rules, and state machines will enable us
to convey temporal phenomena at different levels of complexity.
Given a procedural description to the last detail, such abstraction
mechanisms can automatically create a simplified story. Moreover,
together with visual abstractions related to the structure, it would be
possible to create a continuum that allows us to transition between
the structural and the procedural abstraction. While structural ab-
straction has been implicitly addressed in many scientific works in
computer graphics and visualization already, the visual abstraction
of processes lacks significantly behind—raising the question of
which axes of abstraction directly relate to dynamics. In addition,
we need to understand the implications of temporal changes to
the underlying data if parts of it are only shown at a higher scale
of abstraction—how does the higher-level abstraction need to be
adjusted and is this even possible?

The animation-interaction ambiguity of time-dependent data
(Proposition 10) is a particularly challenging aspect of future work.

<footnote n="22">22. Notice that Zabusky et al. [118] call their integrated visualizations “jux-
tapositions,” while in the context of this section and the general use of the
terminology in visualization research [36], [53] most are superimpositions.</footnote>
<figure><img src="image_22.png" /><figcaption>Fig. 18. Spatially integrated multi-scale abstraction of DNA. Image from [107], © Nature Publishing Group, used with permission.</figcaption></figure>

Ultimately, we have to also investigate the question of ambiguity between visual abstraction and actual data. In both cases we have to understand how differently abstracting visual representations can be interpreted by viewers in different ways, and have to study means to (strongly) suggest one interpretation over another.

## 6.4 Addressing Large Scale Changes

As we pointed out in Sect. 4.4, many phenomena in real life need to be studied on vastly different scales. We thus have to support visual representations that support the scale traversal through abstractions that can cover multiple orders of magnitudes. Current visual abstraction approaches, however, only cover relatively small scale ranges such as the examples shown in Fig. 5 and 11. For communicating how wider scales interact with each other, we will have to find visual representations that use a dramatically different form—up to topology changes. For example for molecular data in structural biology, one has to transition from an internal abstraction of the molecule to a surface-based representation, to ultimately different organizational primitives such as nucleosomes, chromosomes, cell organelles, and the cell itself (Fig. 18). Similarly in applications such as CFD, as one zooms into a dataset an increasing amount of details has to be shown—maybe even requiring a new simulation run at a lower scale, while only large-scale features need to be shown when looking at a zoomed-out view (e. g., Fig. 6). Several of these transitions may not be possible in a continuous fashion, so we have to identify means to blend between different scales—either temporally or spatially. An essential problem remains how to represent wide ranges in scales that are visually as continuous as possible while each scale shows at the same time the most relevant aspects for the scale in question. For integrations of multiple scale levels in a single image we have to examine existing focus+context mechanisms in visualization to determine if they permit such spatially integrated depictions. Here it is especially

challenging to include dynamics, as communicating several temporal scales will be constrained by temporal consistencies.

## 6.5 Encoding Uncertainty

Data uncertainty in visualization is often conveyed through techniques developed in descriptive statistics. The box plot, confidence intervals, bag plots, for example, are all techniques that describe how statistics vary within a certain group and how confident we can be about predicting behavior or a structure of a new member of this group. One challenge that visualization community attempts to address is how to convey the distribution or uncertainty visually for dimensions higher than two—even 3D data representation struggles with how to convey data and uncertainty simultaneously [13].

At the same time, abstraction is strongly related to uncertainty, as the details are removed an abstraction describes a larger span of concrete instances. In fact, visual arts have been using abstraction for conveying uncertainty on purpose. A challenge for visualization research is whether we can follow-up on this exemplary work and use abstraction for conveying uncertainty of the data in a more quantifiable way [16], [116]. How many levels of sketchiness, for example, would we be able to perceive? Which are the particularly useful visual abstractions that should convey the information about data uncertainty? Can separate attributes of data uncertainty be mapped on separate visual abstraction axes so that they can be visually separable? Is it even possible to use abstraction independent of uncertainty or depict uncertainty without abstracting? Can missing data be masked using visual abstraction?^23

## 7 LIMITATIONS

Visual abstraction is a field of interest of several scientific disciplines and also artistic genres. Throughout the article we relate visual abstraction concepts primarily to illustrative visualization, but often also outside this field. It is not our ambition to completely analyze the visual abstraction of all possible related fields. However, none of those fields where visual abstraction plays a significant role which we came across during our research contradicts our statements about visual abstraction and its proposed characteristics. It is thus possible to imagine to expand the domain of validity to broader regions of 3D visualization or even visualization in general.

## 8 CONCLUSION

Abstraction has been and continues to be one of the core concepts and building blocks in visualization, not only in illustrative visualization. In this article we initiate a formalization of the concept, trying to identify major properties of abstraction and relating it to past work in visual arts and many forms of visualization including illustrative visualization. We identified several important aspects, such as the concept of abstraction axes and abstraction spaces, the abstraction of temporal data, and the abstraction across multiple scales. Based on this discussion we identified foci of research that need to be addressed in the future to better understand the principles behind abstraction and its potential, and we sketched a number of promising research questions that we see. Of course, this collection is not final and there are many more important questions to be addressed. Our article, therefore, should also not be seen as a final theory or model of abstraction in visualization, but rather as

<footnote n="23">23. In our recent work we have proposed to relate data from distinct developmental stages of a virus using visual abstractions [99].</footnote>
a contribution to the scientific discourse that will, ultimately, lead us to a better understanding of the principles of visualization itself and, eventually, a theory of visualization science.

Tying again our discussion back into our initial motivation of studying the concept of illustrative visualization, however, we may be able to learn from the proposed model that there are at least two fundamental but related aspects to it. We saw that stylization often relates to photometric abstraction, but can also affect other axes, yet that there are also stylization approaches independent of abstraction (e. g., the emulation of a specific historical style [4]) that can also affect how a visualization is perceived/processed by a person viewing it. Moreover, certain forms of abstraction itself can sometimes be seen as a dedicated style (e. g., the different metro map styles shown in Fig. 1) that better illustrates the subject matter at hand. We thus close by proposing that:

**Proposition 17.** Illustrative visualization = abstraction + stylization, where abstraction can (but does not have to) imply stylization and stylization can (but does not have to) imply abstraction.

## ACKNOWLEDGMENTS

We would like to thank the anonymous reviewers of this article for their valuable comments which helped us to improve it from the first submission to its current form. We also thank all those authors and institutions who allowed us to use their images in our article. Part of this work was funded under the ILLUSTRARE grant by both Austrian Science Fund (FWF): I 2953-N31 and French National Research Agency (ANR): ANR-16-CE91-0011-01.

## REFERENCES

[1] M. Agrawala, D. Phan, J. Heiser, J. Haymaker, J. Klingner, P. Hanrahan, and B. Tversky. Designing effective step-by-step assembly instructions. *ACM Transactions on Graphics*, 22(3):828–837, July 2003. doi: 10.1145/882262.882352

[2] M. Agrawala and C. Stolte. Rendering effective route maps: Improving usability through generalization. In *Proc. SIGGRAPH*, pp. 241–249. ACM, New York, 2001. doi: 10.1145/383259.383286

[3] H. Akiba, C. Wang, and K.-L. Ma. AniViz: A template-based animation tool for volume visualization. *IEEE Computer Graphics and Applications*, 30(5):61–71, Sept./Oct. 2010. doi: 10.1109/MCG.2009.107

[4] B. Bach, P. Dragicevic, S. Huron, P. Isenberg, Y. Jansen, C. Perin, A. Spritzer, R. Vuillemot, W. Willett, and T. Isenberg. Illustrative data graphics in 18th–19th century style: A case study. In *Posters at IEEE Visualization*, 2013.

[5] L. Bartram and M. C. Stone. Whisper, don't scream: Grids and transparency. *IEEE Transactions on Visualization and Computer Graphics*, 17(10):1444–1458, Oct. 2011. doi: 10.1109/TVCG.2010.237

[6] D. Bartz, H. Hagen, V. Interrante, K.-L. Ma, and B. Preim. Illustrative rendering techniques for visualization – Future of visualization or just another technique? In *Proc. Visualization*, number 4 in Panels, pp. 715–718. IEEE Computer Society, Los Alamitos, 2005. doi: 10.1109/VISUAL.2005.1532863

[7] C. Basch. Animated transitions across multiple dimensions for volumetric data. Master's thesis, Institute of Computer Graphics and Algorithms, TU Wien, Austria, 2011.

[8] R. A. Becker and W. S. Cleveland. Brushing scatterplots. *Technometrics*, 29(2):127–142, 1987. doi: 10.1080/00401706.1987.10488204

[9] I. Berger, A. Shamir, M. Mahler, E. Carter, and J. Hodgins. Style and abstraction in portrait sketching. *ACM Transactions on Graphics*, 32(4):55:1–55:12, July 2013. doi: 10.1145/2461912.2461964

[10] J. Bertin. *Semiology of Graphics: Diagrams, Networks, Maps*. Esri Press, Redlands, CA, USA, 2010.

[11] A. Bezerianos, F. Chevalier, P. Dragicevic, N. Elmqvist, and J.-D. Fekete. GraphDice: A system for exploring multivariate social networks. *Computer Graphics Forum*, 29(3):863–872, June 2010. doi: 10.1111/j.1467-8659.2009.01687.x

[12] I. Biederman. Recognition-by-components: A theory of human image understanding. *Psychological Review*, 94(2):115–147, Apr. 1987. doi: 10.1037/0033-295X.94.2.115

[13] G.-P. Bonneau, H.-C. Hege, C. R. Johnson, M. M. Oliveira, K. Potter, P. Rheingans, and T. Schultz. Overview and state-of-the-art of uncertainty visualization. In *C. D. Hansen, M. Chen, C. R. Johnson, A. E. Kaufman, and H. Hagen, eds., Scientific Visualization: Uncertainty, Multifield, Biomedical, and Scalable Visualization*, pp. 3–27. Springer, London, 2014. doi: 10.1007/978-1-4471-6497-5 1

[14] S. Born, A. Wiebel, J. Friedrich, G. Scheuermann, and D. Bartz. Illustrative stream surfaces. *IEEE Transactions on Visualization and Computer Graphics*, 16(6):1329–1338, Nov./Dec. 2010. doi: 10.1109/TVCG.2010.166

[15] J. Böttger, A. Schäfer, G. Lohmann, A. Villringer, and D. S. Margulies. Three-dimensional mean-shift edge bundling for the visualization of functional connectivity in the brain. *IEEE Transactions on Visualization and Computer Graphics*, 20(3):471–480, Mar. 2014. doi: 10.1109/TVCG.2013.114

[16] N. Boukhelifa, A. Bezerianos, T. Isenberg, and J.-D. Fekete. Evaluating sketchiness as a visual variable for the depiction of qualitative uncertainty. *IEEE Transactions on Visualization and Computer Graphics*, 18(12):2769–2778, Dec. 2012. doi: 10.1109/TVCG.2012.220

[17] A. Bousseau, M. Kaplan, J. Thollot, and F. X. Sillion. Interactive watercolor rendering with temporal coherence and abstraction. In *Proc. NPAR*, pp. 141–149. ACM, New York, 2006. doi: 10.1145/1124728.1124751

[18] A. Buja, J. A. McDonald, J. Michalak, and W. Stuetzle. Interactive data visualization using focusing and linking. In *Proc. IEEE Visualization*, pp. 156–163. IEEE Computer Society, Los Alamitos, 1991. doi: 10.1109/VISUAL.1991.175794

[19] B. P. Buttenfield and R. B. McMaster, eds. *Map Generalization: Making Rules for Knowledge Representation*. Longman Scientific & Technical, Harlow, Essex, UK, 1991.

[20] S. K. Card, J. D. Mackinlay, and B. Shneiderman. *Readings in Information Visualization: Using Vision to Think*. Morgan Kaufmann, San Francisco, 1999.

[21] W. Chen, Z. Ding, S. Zhang, A. MacKay-Brandt, S. Correia, H. Qu, J. A. Crow, D. F. Tate, Z. Ya, and Q. Peng. A novel interface for interactive exploration of DTI fibers. *IEEE Transactions on Visualization and Computer Graphics*, 15(6):1433–1440, Nov./Dec. 2009. doi: 10.1109/TVCG.2009.112

[22] W. Chen, S. Zhang, S. Correia, and D. S. Ebert. Abstractive representation and exploration of hierarchically clustered diffusion tensor fiber tracts. *Computer Graphics Forum*, 27(3):1071–1078, May 2008. doi: 10.1111/j.1467-8659.2008.01244.x

[23] M.-T. Chi and T.-Y. Lee. Stylized and abstract painterly rendering system using a multiscale segmented sphere hierarchy. *IEEE Transactions on Visualization and Computer Graphics*, 12(1):61–72, Jan./Feb. 2006. doi: 10.1109/TVCG.2006.14

[24] G. Cipriano and M. Gleicher. Molecular surface abstraction. *IEEE Transactions on Visualization and Computer Graphics*, 13(6):1608–1615, Nov./Dec. 2007. doi: 10.1109/TVCG.2007.70578

[25] G. Cipriano, G. N. Phillips, Jr., and M. Gleicher. Multi-scale surface descriptors. *IEEE Transactions on Visualization and Computer Graphics*, 15(6):1201–1208, Nov./Dec. 2009. doi: 10.1109/TVCG.2009.168

[26] D. Cox. Metaphoric mappings: The art of visualization. In P. A. Fishwick, ed., *Aesthetic Computing*, chap. 6, pp. 89–114. MIT Press, Cambridge, MA/London, UK, 2006.

[27] Q. Cui, M. Ward, E. Rundensteiner, and J. Yang. Measuring data abstraction quality in multiresolution visualizations. *IEEE Transactions on Visualization and Computer Graphics*, 12(5):709–716, Sept./Oct. 2006. doi: 10.1109/TVCG.2006.161

[28] P. de Heras Ciechomski, R. Mange, H. Koepl, and M. Klann. From biochemical reaction networks to 3D dynamics in the cell: The ZigCell3D modeling, simulation and visualisation framework. In *Proc. BioVis*, pp. 41–48. IEEE Computer Society, Los Alamitos, 2013. doi: 10.1109/BioVis.2013.6664345

[29] D. DeCarlo and A. Santella. Stylization and abstraction of photographs. *ACM Transactions on Graphics*, 21(3):769–776, July 2002. doi: 10.1145/566654.566650

[30] P. Decaudin. Cartoon-Looking Rendering of 3D-Scenes. Technical Report 2919, Inria, 1996.

[31] N. Elmqvist, P. Dragicevic, and J.-D. Fekete. Rolling the dice: Multidimensional visual exploration using scatterplot matrix navigation. *IEEE Transactions on Visualization and Computer Graphics*, 14(6):1141–1148, Nov./Dec. 2008. doi: 10.1109/TVCG.2008.153

[32] M. H. Everts, E. Begue, H. Bekker, J. B. T. M. Roerdink, and T. Isenberg. Exploration of the brain's white matter structure through visual abstraction and multi-scale local fiber tract contraction. *IEEE Transactions on Visualization and Computer Graphics*, 21(7):808–821, July 2015. doi: 10.1109/TVCG.2015.2403323
[33] S. Feiner. APEX: An experiment in the automated creation of pictorial explanations. *IEEE Computer Graphics and Applications*, 5(11):29–37, Nov. 1985. doi: 10.1109/MCG.1985.276329

[34] S. Frey and T. Ertl. Flow-based temporal selection for interactive volume visualization. *Computer Graphics Forum*, 36(8), Dec. 2017. To appear. doi: 10.1111/cgf.13070

[35] T. Glander and J. Döllner. Abstract representations for interactive visualization of virtual 3D city models. *Computers, Environment and Urban Systems*, 33(5):375–387, Sept. 2009. doi: 10.1016/j.compenvurbsys. 2009.07.003

[36] M. Gleicher, D. Albers, R. Walker, I. Jusufi, C. D. Hansen, and J. C. Roberts. Visual comparison for information visualization. *Information Visualization*, 10(4):289–309, Oct. 2011. doi: 10.1177/1473871611416549

[37] J. Gomes and L. Velho. Abstraction paradigms for computer graphics. *The Visual Computer*, 11(5):227–239, May 1995. doi: 10.1007/BF01901401

[38] D. S. Goodsell. Visual Methods from Atoms to Cells. *Structure*, 13(3):347–354, Mar. 2005. doi: 10.1016/j.str.2005.01.012

[39] F. Grabler, M. Agrawala, R. W. Sumner, and M. Pauly. Automatic generation of tourist maps. *ACM Transactions on Graphics*, 27(3):100:1–100:11, Aug. 2008. doi: 10.1145/1360612.1360699

[40] D. Guo, J. Nie, M. Liang, Y. Wang, Y. Wang, and Z. Hu. View-dependent level-of-detail abstraction for interactive atomistic visualization of biological structures. *Computers & Graphics*, 52:62–71, 2015. doi: 10. 1016/j.cag.2015.06.008

[41] C. G. Healey and J. T. Enns. Perception and painting: A search for effective, engaging visualizations. *IEEE Computers Graphics and Applications*, 22(2):10–15, Mar./Apr. 2002. doi: 10.1109/38.988741

[42] J. Heer and G. Robertson. Animated transitions in statistical data graphics. *IEEE Transactions on Visualization and Computer Graphics*, 13(6):1240–1247, Nov./Dec. 2007. doi: 10.1109/TVCG.2007.70539

[43] W. L. Hibbard, C. R. Dyer, and B. E. Paul. A lattice model for data display. In *Proc. Visualization*, pp. 310–317. IEEE Computer Society, Los Alamitos, 1994. doi: 10.1109/VISUAL.1994.346304

[44] D. Holten. Hierarchical edge bundles: Visualization of adjacency relations in hierarchical data. *IEEE Transactions on Visualization and Computer Graphics*, 12(5):741–748, Sept. 2006. doi: 10.1109/TVCG.2006.147

[45] H. Hoppe. Progressive meshes. In *Proc. SIGGRAPH*, pp. 99–108. ACM, New York, 1996. doi: 10.1145/237170.237216

[46] W.-H. Hsu, K.-L. Ma, and C. Correa. A rendering framework for multi-scale views of 3D models. *ACM Transactions on Graphics*, 30(6):131:1–131:10, Dec. 2011. doi: 10.1145/2070781.2024165

[47] W.-H. Hsu, J. Mei, C. Correa, and K.-L. Ma. Depicting time evolving flow with illustrative visualization techniques. In *Proc. ArtsIT*, vol. 30, pp. 136–147. Springer, Berlin, Heidelberg, 2010. doi: 10.1007/978-3-642-11577-6_18

[48] C. Hurter, A. R. Taylor, S. Carpendale, and A. Telea. Color tunneling: Interactive exploration and selection in volumetric datasets. In *Proc. PacificVis*, pp. 225–232. IEEE Computer Society, Los Alamitos, 2014. doi: 10.1109/PacificVis.2014.61

[49] O. Inbar, N. Tractinsky, and J. Meyer. Minimalism in information visualization: Attitudes towards maximizing the data-ink ratio. In *Proc. ECCE*, pp. 185–188. ACM, New York, 2007. doi: 10.1145/1362550.1362587

[50] T. Isenberg. Visual abstraction and stylisation of maps. *The Cartographic Journal*, 50(1):8–18, Feb. 2013. doi: 10.1179/1743277412Y.00000000007

[51] T. Isenberg. Interactive NPAR: What type of tools should we create? In *Proc. NPAR*, pp. 89–96. The Eurographics Association, Goslar, Germany, 2016. doi: 10.2312/exp.20161067

[52] T. Isenberg, M. H. Everts, J. Grubert, and S. Carpendale. Interactive exploratory visualization of 2D vector fields. *Computer Graphics Forum*, 27(3):983–990, May 2008. doi: 10.1111/j.1467-8659.2008.01233.x

[53] W. Javed and N. Elmqvist. Exploring the design space of composite visualization. In *Proc. PacificVis*, pp. 1–8. IEEE Computer Society, Los Alamitos, 2012. doi: 10.1109/PacificVis.2012.6183556

[54] R. Jianu, Ç. Demiralp, and D. H. Laidlaw. Exploring brain connectivity with two-dimensional neural maps. *IEEE Transactions on Visualization and Computer Graphics*, 18(6):978–987, June 2012. doi: 10.1109/TVCG. 2011.82

[55] J. Johansson and M. Cooper. A screen space quality method for data abstraction. *Computer Graphics Forum*, 27(3):1039–1046, May 2008. doi: 10.1111/j.1467-8659.2008.01240.x

[56] C. Johnson. Top scientific visualization research problems. *IEEE Computer Graphics and Applications*, 24(4):13–17, July/Aug. 2004. doi: 10.1109/MCG.2004.20

[57] H. Kang and S. Lee. Shape-simplifying image abstraction. *Computer Graphics Forum*, 27(7):1773–1780, Oct. 2008. doi: 10.1111/j.1467-8659. 2008.01322.x

[58] I. Kolesar, J. Parulek, I. Viola, S. Bruckner, A.-K. Stavrum, and H. Hauser. Interactively illustrating polymerization using three-level model fusion. *BMC Bioinformatics*, 15(1):345:1–345:16, Oct. 2014. doi: 10.1186/1471-2105-15-345

[59] L. Kühne, J. Giesen, Z. Zhang, S. Ha, and K. Mueller. A data-driven approach to hue-preserving color-blending. *IEEE Transactions on Visualization and Computer Graphics*, 18(12):2122–2129, Dec. 2012. doi: 10.1109/TVCG.2012.186

[60] R. Laramee, H. Hauser, L. Zhao, and F. Post. Topology-based flow visualization, the state of the art. In *Topology-based Methods in Visualization*, pp. 1–19. Springer, Berlin/Heidelberg, 2007. doi: 10.1007/978-3-540-70823-0_1

[61] K. Lawonn, T. Mönch, and B. Preim. Streamlines for illustrative real-time rendering. *Computer Graphics Forum*, 32(3):321–330, June 2013. doi: 10.1111/cgf.12119

[62] K. Lawonn and B. Preim. Feature lines for illustrating medical surface models: Mathematical background and survey. In *Visualization in Medicine in Life Sciences III*, pp. 93–132. Springer, Berlin/Heidelberg, 2016. doi: 10.1007/978-3-319-24523-2_5

[63] M. Le Muzic, J. Parulek, A.-K. Stavrum, and I. Viola. Illustrative visualization of molecular reactions using omniscient intelligence and passive agents. *Computer Graphics Forum*, 33(3):141–150, June 2014. doi: 10.1111/cgf.12370

[64] M. Le Muzic, M. Waldner, J. Parulek, and I. Viola. Illustrative timelapse: A technique for illustrative visualization of particle-based simulations. In *Proc. PacificVis*, pp. 247–254. IEEE Computer Society, Los Alamitos, 2015. doi: 10.1109/PACIFICVIS.2015.7156384

[65] Y. Lee, L. Markosian, S. Lee, and J. F. Hughes. Line drawings via abstracted shading. *ACM Transactions on Graphics*, 26(3):18:1–18:5, July 2007. doi: 10.1145/1276377.1276400

[66] S. Liu, Y. Wu, E. Wei, M. Liu, and Y. Liu. StoryFlow: Tracking the evolution of stories. *IEEE Transactions on Visualization and Computer Graphics (Proceedings of IEEE InfoVis 2013)*, 19(12):2436–2445, 2013. doi: 10.1109/TVCG.2013.196

[67] M. S. Livingstone. *Vision and Art: The Biology of Seeing*. Abrams, New York, 2008.

[68] M. Luboschik and H. Schumann. Illustrative halos in information visualization. In *Proc. AVI*, pp. 384–387. ACM, New York, 2008. doi: 10.1145/1385569.1385639

[69] W. Lueks, I. Viola, M. van der Zwan, H. Bekker, and T. Isenberg. Spatially continuous change of abstraction in molecular visualization. In *IEEE BioVis Abstracts*, 2011. Extended abstract and poster.

[70] K.-L. Ma, I. Liao, J. Fraziers, H.-N. Kostis, and H. Hauser. Scientific storytelling using visualization. *IEEE Computer Graphics and Applications*, 32(1):12–19, Jan./Feb. 2012. doi: 10.1109/MCG.2012.24

[71] J. Mackinlay. Automating the design of graphical presentations of relational information. *ACM Transactions on Graphics*, 5(2):110–141, Apr. 1986. doi: 10.1145/22949.22950

[72] S. McCloud. *Understanding Comics: The Invisible Art*. HarperCollins Publishers, Inc., New York, 1993.

[73] K. T. McDonnell and K. Mueller. Illustrative parallel coordinates. *Computer Graphics Forum*, 27(3):1031–1038, May 2008. doi: 10.1111/j. 1467-8659.2008.01239.x

[74] R. Mehra, Q. Zhou, J. Long, A. Sheffer, A. Gooch, and N. J. Mitra. Abstraction of man-made shapes. *ACM Transactions on Graphics*, 28(5):137:1–137:10, Dec. 2009. doi: 10.1145/1618452.1618483

[75] X. Mi, D. DeCarlo, and M. Stone. Abstraction of 2D shapes in terms of parts. In *Proc. NPAR*, pp. 15–24. ACM, New York, 2009. doi: 10. 1145/1572614.1572617

[76] H. Miao, E. De Llano, J. Sorger, Y. Ahmadi, T. Kekic, T. Isenberg, M. E. Gröller, I. Barišić, and I. Viola. Multiscale visualization and scale-adaptive modification of DNA nanostructures. *IEEE Transactions on Visualization and Computer Graphics*, 24(1):1014–1024, Jan. 2018. doi: 10.1109/TVCG.2017.2743981

[77] T. Munzner. *Visualization Analysis and Design*. A K Peters Visualization Series. CRC Press, Boca Raton, FL, USA, 2014. doi: 10.1201/b17511

[78] T. Munzner. On conventions between fields in experimental design and analysis. Blog post, https://tamaramunzner.wordpress.com/2016/01/16/ on-conventions-between-fields-in-experimental-design-and-analysis/., Jan. 2016. page visited in Feb. 2016.

[79] T. Nagel and B. Groß. Shanghai metro flow – Multiple perspectives into a subway system. In *Proc. VISAP*, pp. 137–138, 2014.

[80] L. Nan, A. Sharf, K. Xie, T.-T. Wong, O. Deussen, D. Cohen-Or, and B. Chen. Conjoining Gestalt rules for abstraction of architectural drawings. *ACM Transactions on Graphics*, 30(6):185:1–185:10, Dec. 2011. doi: 10.1145/2070781.2024219
[81] N. L. Novère, M. Hucka, H. Mi, S. Moodie, A. S. Falk Schreiber and, E. Demir, K. Wegner, M. I. Aladjem, S. M. Wimalaratne, F. T. Bergman, R. Gauges, P. Ghazal, H. Kawaji, L. Li, Y. Matsuoka, A. Villéger, S. E. Boyd, L. Calzone, M. Courtot, U. Dogrusoz, T. C. Freeman, A. Funahashi, S. Ghosh, A. Jouraku, S. Kim, F. Kolpakov, A. Luna, S. Sahle, E. Schmidt, S. Watterson, G. Wu, I. Goryanin, D. B. Kell, C. Sander, H. Sauro, J. L. Snoep, K. Kohn, and H. Kitano. The systems biology graphical notation. *Nature Biotechnology*, 27(8):735–741, Aug. 2009. doi: 10.1038/nbt.1558

[82] M. Novotný. Visual abstraction for information visualization of large data. Diplom thesis, Comenius University Bratislava, Slovakia, 2004.

[83] M. Novotný. Visually effective information visualization of large data. In *Proc. CESCG*. TU Wien, Austria, 2004.

[84] M. Ovenden. *Transit Maps of the World*. Penguin Random House, New York, 2nd ed., 2007.

[85] J. Parulek, D. Jönsson, T. Ropinski, S. Bruckner, A. Ynnerman, and I. Viola. Continuous levels-of-detail and visual abstraction for seamless molecular visualization. *Computer Graphics Forum*, 33(6):276–287, May 2014. doi: 10.1111/cgf.12349

[86] J. Parulek, T. Ropinski, and I. Viola. Seamless abstraction of molecular surfaces. In *Proc. SCCG*, pp. 107–114. ACM, New York, 2013. doi: 10.1145/2508244.2508258

[87] A. Pobitzer, R. Peikert, R. Fuchs, B. Schindler, A. Kuhn, H. Theisel, K. Matković, and H. Hauser. The state of the art in topology-based visualization of unsteady flow. *Computer Graphics Forum*, 30(6):1789–1811, Sept. 2011. doi: 10.1111/j.1467-8659.2011.01901.x

[88] P. Rautek, S. Bruckner, E. Gröller, and I. Viola. Illustrative visualization: New technology or useless tautology? *ACM SIGGRAPH Computer Graphics*, 42(3):4:1–4:8, Aug. 2008. doi: 10.1145/1408626.1408633

[89] P. Richens. The Piranesi system for interactive rendering. In *Proc. CAADfutures*, pp. 381–398. Springer, Berlin/Heidelberg, 1999. doi: 10.1007/978-1-4615-5047-1_25

[90] M. J. Roberts. How to create a topographically reasonable Underground map. Technical report, 2011. Published online at http://www.tubemapcentral.com/articles/geoarticle.pdf.

[91] M. J. Roberts. *Underground Maps Unravelled: Explorations in Information Design*. Maxwell J. Roberts, 2012.

[92] H. Rosling. Gapminder world. Web site, https://www.gapminder.org/, visited Dec., 2015.

[93] A. Santella and D. DeCarlo. Visual interest and NPR: An evaluation and manifesto. In *Proc. NPAR*, pp. 71–78. ACM, New York, 2004. doi: 10.1145/987657.987669

[94] D. Schroeder and D. F. Keefe. Visualization-by-sketching: An artist's interface for creating multivariate time-varying data visualizations. *IEEE Transactions on Visualization and Computer Graphics*, 22(1):877–885, Jan. 2016. doi: 10.1109/TVCG.2015.2467153

[95] J. Schumann, T. Strothotte, A. Raab, and S. Laser. Assessing the effect of non-photorealistic rendered images in CAD. In *Proc. CHI*, pp. 35–42. ACM, New York, 1996. doi: 10.1145/238386.238398

[96] A. Semmo and J. Döllner. Interactive image filtering for level-of-abstraction texturing of virtual 3D scenes. *Computers & Graphics*, 52:181–198, Nov. 2015. doi: 10.1016/j.cag.2015.02.001

[97] A. Semmo, M. Trapp, J. E. Kyprianidis, and J. Döllner. Interactive visualization of generalized virtual 3D city models using level-of-abstraction transitions. *Computer Graphics Forum*, 31(3):885–894, June 2012. doi: 10.1111/j.1467-8659.2012.03081.x

[98] K. Siddiqi and S. Pizer, eds. *Medial Representations: Mathematics, Algorithms and Applications*. Springer, Berlin/Heidelberg, 2009. doi: 10.1007/978-1-4020-8658-8

[99] J. Sorger, P. Mindek, T. Klein, G. Johnson, and I. Viola. Illustrative transitions in molecular visualization via forward and inverse abstraction transform. In *Proc. VCBM*, pp. 21–30. Eurographics Association, Goslar, Germany, 2016. doi: 10.2312/vcbm.20161267

[100] A. Stompel, E. B. Lum, and K.-L. Ma. Visualization of multidimensional, multivariate volume data using hardware-accelerated non-photorealistic rendering techniques. In *Proc. PG*, pp. 394–402. IEEE Computer Society, Los Alamitos, 2002. doi: 10.1109/PCCG.2002.1167883

[101] T. Strothotte, ed. *Computational Visualization: Graphics, Abstraction and Interactivity*. Springer, Berlin/Heidelberg, 1998. doi: 10.1007/978-3-642-59847-0

[102] N. A. Svakhine, Y. Jang, D. S. Ebert, and K. Gaither. Illustration and photography inspired visualization of flows and volumes. In *Proc. VIS*, pp. 687–694. IEEE Computer Society, Los Alamitos, 2005. doi: 10.1109/VISUAL.2005.1532858

[103] Y. Tanahashi and K.-L. Ma. Design considerations for optimizing story-line visualizations. *IEEE Transactions on Visualization and Computer Graphics*, 18(12):2679–2688, Dec. 2012. doi: 10.1109/TVCG.2012.212

[104] M. Tarini, P. Cignoni, and C. Montani. Ambient occlusion and edge cueing to enhance real time molecular visualization. *IEEE Transactions on Visualization and Computer Graphics*, 12(5):1237–884, Sept./Oct. 2006. doi: 10.1109/TVCG.2006.115

[105] H. Theisel, T. Weinkauf, H.-C. Hege, and H.-P. Seidel. Topological methods for 2D time-dependent vector fields based on stream lines and path lines. *IEEE Transactions on Visualization and Computer Graphics*, 11(4):383–394, July/Aug. 2005. doi: 10.1109/TVCG.2005.68

[106] C. Tietjen, T. Isenberg, and B. Preim. Combining silhouettes, shading, and volume rendering for surgery education and planning. In *Proc. EuroVis*, pp. 303–310. Eurographics Association, Goslar, Germany, 2005. doi: 10.2312/VisSym/EuroVis05/303-310

[107] S. Tonna, A. El-Osta, M. E. Cooper, and C. Tikellis. Metabolic memory and diabetic nephropathy: Potential role for epigenetic mechanisms. *Nature Reviews Nephrology*, 6(6):332–341, June 2010. doi: 10.1038/nrneph.2010.55

[108] E. R. Tufte. *The Visual Display of Quantitative Information*. Graphics Press, Cheshire, CT, USA, 2nd ed., 2001.

[109] B. Tversky, J. B. Morrison, and M. Bertrancourt. Animation: Can it facilitate? *International Journal of Human-Computer Studies*, 57(4):247–262, Oct. 2002. doi: 10.1006/ijhc.2002.1017

[110] M. van der Zwan, W. Lueks, H. Bekker, and T. Isenberg. Illustrative molecular visualization with continuous abstraction. *Computer Graphics Forum*, 30(3):683–690, May 2011. doi: 10.1111/j.1467-8659.2011.01917.x

[111] A. Vande Moere, M. Tomitsch, C. Wimmer, B. Christoph, and T. Grechenig. Evaluating the effect of style in information visualization. *IEEE Transactions on Visualization and Computer Graphics*, 18(12):2739–2748, Dec. 2012. doi: 10.1109/TVCG.2012.221

[112] I. Viola, H. Hauser, and D. Ebert. Editorial note for special section on illustrative visualization. *Computers & Graphics*, 34(4):335–336, Aug. 2010. doi: 10.1016/j.cag.2010.05.011

[113] I. Viola, A. Kanitsar, and M. E. Gröller. Importance-driven feature enhancement in volume visualization. *IEEE Transactions on Visualization and Computer Graphics*, 11(4):408–418, July/Aug. 2005. doi: 10.1109/TVCG.2005.62

[114] L. Wang, J. Giesen, K. T. McDonnell, P. Zolliker, and K. Mueller. Color design for illustrative visualization. *IEEE Transactions on Visualization and Computer Graphics*, 14(6):1739–1754, Oct. 2008. doi: 10.1109/TVCG.2008.118

[115] C. Ware. *Information Visualization: Perception for Design*. Morgan Kaufmann, Waltham, MA, USA, 3rd ed., 2013. doi: 10.1016/B978-0-12-381464-7.00018-1

[116] J. Wood, P. Isenberg, T. Isenberg, J. Dykes, N. Boukhelifa, and A. Slingsby. Sketchy rendering for information visualization. *IEEE Transactions on Visualization and Computer Graphics*, 18(12):2749–2758, Dec. 2012. doi: 10.1109/TVCG.2012.262

[117] M. E. Yumer and L. B. Kara. Co-abstraction of shape collections. *ACM Transactions on Graphics*, 31(6):166:1–166:11, Nov. 2012. doi: 10.1145/2366145.2366185

[118] N. J. Zabusky, D. Silver, R. Pelz, and Vizgroup '93. Visiometrics, juxtaposition and modeling. *Physics Today*, 46(3):24–31, Mar. 1993. doi: 10.1063/1.881394

[119] M. Zhao and S.-C. Zhu. Sisley the abstract painter. In *Proc. NPAR*, pp. 99–107. ACM, New York, 2010. doi: 10.1145/1809939.1809951

<figure><img src="image_23.png" /></figure>

**Ivan Viola** is Assistant Professor at the TU Wien, Austria. He received M.Sc. in 2002 and Ph.D. in 2005 from TU Wien, Austria. Previously he has been affiliated with the Univ. of Bergen, Norway and Christian Michelsen Research, Norway. His research is focusing on illustrative visualization methods. Viola co-authored several scientific works published in international journals and conferences such as IEEE TVCG, IEEE Vis, EG CGF, and EG EuroVis and acted as a reviewer and IPC for conferences in the field of visualization.

<figure><img src="image_24.png" /></figure>

**Tobias Isenberg** is a senior research scientist at Inria, France. He received his doctoral degree from the Univ. of Magdeburg, Germany, in 2004. Previously he held positions as post-doctoral fellow at the Univ. of Calgary, Canada, and as assistant professor at the Univ. of Groningen, the Netherlands. His research interests comprise topics in scientific visualization, illustrative and non-photorealistic rendering, and interactive visualization techniques. He is particularly interested in the benefit of abstraction for illustrative visualization.