# Micro Visualizations

Petra Isenberg

► To cite this version:

Petra Isenberg. Micro Visualizations: Design and Analysis of Visualizations for Small Display Spaces. Human-Computer Interaction [cs.HC]. Université Paris-Saclay, 2021. tel-03584024

HAL Id: tel-03584024

https://inria.hal.science/tel-03584024

Submitted on 29 Mar 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Micro Visualizations
Design and Analysis of Visualizations for Small Display Spaces

Habilitation à diriger des recherches
de l'Université Paris-Saclay

présentée et soutenue à Gif-Sur-Yvette,
le 30 novembre 2021, par

Petra Isenberg

## Composition du jury

<table><tr><td><strong>Michèle Sebag</strong><br>Directrice de recherche, CNRS</td><td>Présidente</td></tr><tr><td><strong>David Auber</strong><br>Professeur, Université de Bordeaux</td><td>Rapporteur</td></tr><tr><td><strong>Jason Dykes</strong><br>Professeur, City, University of London</td><td>Rapporteur</td></tr><tr><td><strong>Silvia Miksch</strong><br>Professeure, Vienna University of Technology</td><td>Rapportrice</td></tr><tr><td><strong>Steven Drucker</strong><br>Principal Researcher, Microsoft Research</td><td>Examinateur</td></tr><tr><td><strong>Tamara Munzner</strong><br>Professeure, University of British Columbia</td><td>Examinatrice</td></tr></table>
**Titre:** Micro visualisations : Conception et analyse de visualisations pour les petits espaces d'affichage

**Mots clés :** Visualisation, glyphs de données, *word-scale visualizations*, micro visualisations

**Resumé :** Le sujet de cette habilitation est l'étude de très petites visualisations de données, les micro visualisations, dans des contextes d'affichage qui ne peuvent consacrer qu'un espace de rendu minimal aux représentations de données. Depuis plusieurs années, avec mes collaborateurs, j'étudie la perception humaine, l'interaction et l'analyse conduite avec des micro visualisations dans de multiples contextes. Dans ce document, je rassemble trois de mes axes de recherche liés aux micro visualisations : les glyphs de données, où ma recherche s'est concentrée sur l'étude de la perception de micro visualisations dans un context *small-multiple*, les *word-scale visualizations*, où ma recherche s'est concentrée sur les petites visualisations intégrées

dans les documents textuels, et les petites visualisations de données mobiles pour les montres connectées. Je considère ces types de petites visualisations sous le terme générique de "micro visualisations." Les micro visualisations sont utiles dans de multiples contextes de visualisation et j'ai travaillé à une meilleure compréhension de la complexité des conceptions et utilisations des micro visualisations. Je définirai ici le terme de micro visualisation, je résumerai mes propres recherches et celles d'autres chercheurs, ainsi que les directives de conception, et j'esquisserai plusieurs espaces de conception pour différents types de micro visualisations, sur la base de certains des travaux auxquels j'ai participé depuis mon doctorat.

**Title:** Micro Visualizations: Design and Analysis of Visualizations for Small Display Spaces

**Keywords:** Visualization, data glyphs, word-scale visualizations, micro visualizations

**Abstract:** The topic of this habilitation is the study of very small data visualizations, micro visualizations, in display contexts that can only dedicate minimal rendering space for data representations. For several years, together with my collaborators, I have been studying human perception, interaction, and analysis with micro visualizations in multiple contexts. In this document I bring together three of my research streams related to micro visualizations: data glyphs, where my joint research focused on studying the perception of small-multiple micro visualizations, word-scale visualizations, where my joint research focused on small visualizations embedded in text-

documents, and small mobile data visualizations for smartwatches or fitness trackers. I consider these types of small visualizations together under the umbrella term "micro visualizations." Micro visualizations are useful in multiple visualization contexts and I have been working towards a better understanding of the complexities involved in designing and using micro visualizations. Here, I define the term micro visualization, summarize my own and other past research and design guidelines and outline several design spaces for different types of micro visualizations based on some of the work I was involved in since my PhD.
This manuscript covers content from the following publications.

[1] Ranjini Aravind, Tanja Blascheck, and Petra Isenberg. "A Survey on Sleep Visualizations for Fitness Trackers." In: *Posters of the European Conference on Visualization (EuroVis)*. 2019.

[2] Tanja Blascheck, Frank Bentley, Eun Kyoung Choe, Tom Horak, and Petra Isenberg. "Characterizing Glanceable Visualizations: From Perception to Behavior Change." In: *Mobile Data Visualization*. To appear. CRC Press, 2021.

[3] Tanja Blascheck, Lonni Besançon, Anastasia Bezerianos, Bongshin Lee, and Petra Isenberg. "Perception des Visualisations sur Smartwatch." In: *Journée Visu*. 2018. URL: https://www.aviz.fr/smartwatchperception.

[4] Tanja Blascheck, Lonni Besançon, Anastasia Bezerianos, Bongshin Lee, and Petra Isenberg. "Glanceable Visualization: Studies of Data Comparison Performance on Smartwatches." In: *IEEE Transactions on Visualization and Computer Graphics* 25.1 (2019), pp. 616-629. DOI: 10.1109/TVCG.2018.2865142.

[5] Tanja Blascheck, Anastasia Bezerianos, Lonni Besançon, Bongshin Lee, and Petra Isenberg. "Preparing for Perceptual Studies: Position and Orientation of Wrist-worn Smartwatches for Reading Tasks." In: *Proceedings of the Workshop on Data Visualization on Mobile Devices held at ACM CHI*. 2018. URL: https://www.aviz.fr/smartwatchperception.

[6] Tanja Blascheck and Petra Isenberg. "A Replication Study on Glanceable Visualizations: Comparing Different Stimulus Sizes on a Laptop Computer." In: *Proceedings of the Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3 IVAPP: IVAPP. SciTePress*, Feb. 2021, pp. 133-143. DOI: 10.5220/0010328501330143.

[7] Sheelagh Carpendale, Petra Isenberg, Charles Perin, Tanja Blascheck, Foroozan Daneshzand, Alaul Islam, Katherine Currier, Peter Buk, Victor Cheung, Lien Quach, and Laton Vermette. "Mobile Visualization Design: An Ideation Method to Try." In: *Mobile Data Visualization*. To appear. CRC Press, 2021.

[8] Eun Kyoung Choe, Raimund Dachselt, Petra Isenberg, and Bongshin Lee. "Mobile Data Visualization (Dagstuhl Seminar 19292)." In: *Dagstuhl Reports* 9.7 (2019), pp. 78-93. DOI: 10.4230/DagRep.9.7.78.
[9] Johannes Fuchs, Fabian Fischer, Florian Mansmann, Enrico Bertini, and Petra Isenberg. "Evaluation of Alternative Glyph Designs for Time Series Data in a Small Multiple Setting." In: *Proceedings of the Conference on Human Factors in Computing Systems (CHI)*. Received a **best paper honorable mention**. ACM, 2013, pp. 3237-3246. DOI: 10.1145/2470654.2466443.

[10] Johannes Fuchs, Petra Isenberg, Anastasia Bezerianos, Fabian Fischer, and Enrico Bertini. "The Influence of Contour on Similarity Perception of Star Glyphs." In: *IEEE Transactions on Visualization and Computer Graphics* 20.12 (Dec. 2014), pp. 2251-2260. DOI: 10.1109/TVCG.2014.2346426.

[11] Johannes Fuchs, Petra Isenberg, Anastasia Bezerianos, and Daniel Keim. "A Systematic Review of Experimental Studies on Data Glyphs." In: *IEEE Transactions on Visualization and Computer Graphics* 23.7 (July 2017), pp. 1863-1879. ISSN: 1077-2626. DOI: 10.1109/TVCG.2016.2549018.

[12] Pascal Goffin, Tanja Blascheck, Petra Isenberg, and Wesley Willett. "Interaction Techniques for Visual Exploration Using Embedded Word-Scale Visualizations." In: *Proceedings of the Conference on Human Factors in Computing Systems (CHI)*. ACM, 2020. DOI: 10.1145/3313831.3376842.

[13] Pascal Goffin, Jeremy Boy, Wesley Willett, and Petra Isenberg. "An Exploratory Study of Word-Scale Graphics in Data-Rich Text Documents." In: *IEEE Transactions on Visualization and Computer Graphics* 23.10 (2017), pp. 2275-2287. DOI: 10.1109/TVCG.2016.2618797.

[14] Pascal Goffin, Wesley Willett, Anastasia Bezerianos, and Petra Isenberg. "Exploring the Effect of Word-Scale Visualizations on Reading Behavior." In: *Extended Abstracts of the Conference on Human Factors in Computing Systems (CHI)*. 2015. DOI: 10.1145/2702613.2732778.

[15] Pascal Goffin, Wesley Willett, Jean-Daniel Fekete, and Petra Isenberg. "Exploring the Placement and Design of Word-Scale Visualizations." In: *IEEE Transactions on Visualization and Computer Graphics* 20.12 (Dec. 2014), pp. 2291-2300. DOI: 10.1109/TVCG.2014.2346435.

[16] Pascal Goffin, Wesley Willett, Jean-Daniel Fekete, and Petra Isenberg. "Design Considerations for Enhancing Word-Scale Visualizations with Interaction." In: *Posters of the IEEE Conference on Information Visualization (InfoVis)*. 2015.

[17] Alaul Islam, Anastasia Bezerianos, Bongshin Lee, Tanja Blascheck, and Petra Isenberg. "Visualizing Information on Watch Faces: A Survey with Smartwatch Users." In: *Short Papers of the IEEE Con-*
ference on Visualization (VIS). Oct. 2020. doi: 10.1109/VIS47514.2020.00038.

[18] Ricardo Langner, Lonni Besançon, Christopher Collins, Tim Dwyer, Petra Isenberg, Tobias Isenberg, Bongshin Lee, Charles Perin, and Christian Tominski. "Introduction to Mobile Data Visualization." In: *Mobile Data Visualization*. To appear. CRC Press, 2021.

[19] Bongshin Lee, Matthew Brehmer, Petra Isenberg, Eun Kyoung Choe, Ricardo Langer, and Raimund Dachselt. "Data Visualization on Mobile Devices." In: *Extended Abstracts of the Conference on Human Factors in Computing System (CHI)*. ACM, 2018, W07:1-W07:8. doi: 10.1145/3170427.3170631. URL: https://mobilevis.github.io.

[20] Bongshin Lee, Raimund Dachselt, Petra Isenberg, and Eun Kyoung Choe, eds. *Mobile Data Visualization*. To appear. CRC Press, 2021. doi: 10.1201/9781003090823.

[21] Lijie Yao, Anastasia Bezerianos, and Petra Isenberg. *Situated Visualization in Motion*. Posters of the IEEE Conference on Visualization (VIS). Oct. 2020.

FUNDING

My students and postdocs were funded by:

* The French National Research Organization, project grant ANR-11-JS02-003 (Pascal Goffin, Wesley Willett)

* The French National Research Organization, project grant ANR-18-CE92-0059-01 (Alaul Islam)

* The French National Research Organization, project grant ANR-19-CE33-0012 (Lijie Yao)

* Inria (Tanja Blascheck, Ranjini Aravind)
I thank my students, postdocs, and collaborators for the many discussions and their joint work on the bigger topic of micro visualizations. In particular I co-authored and collaborated on the topic with:

Students:

* Ranjini Aravind (Université Paris-Saclay)
  Smartwatch visualization

* Alaul Islam (Inria, Université Paris-Saclay)
  Smartwatch face visualization

* Fabian Fischer (University of Konstanz)
  Data glyphs

* Johannes Fuchs (University of Konstanz)
  Data glyphs

* Pascal Goffin (Inria, Université Paris Sud)
  Word-scale visualization

* Tingying He (Université Paris-Saclay)
  Design space for micro visualization

* Tom Horak (Technische Universität Dresden)
  Mobile visualization

* Ricardo Langner (Technische Universität Dresden)
  Mobile visualization

* Lijie Yao (Université Paris-Saclay)
  Visualization in motion

* Foroozan Daneshzand, Katherine Currier, Peter Buk, Victor Cheung, Lien Quach, and Laton Vermette (Simon Fraser University)
  Mobile visualizations

Postdocs:

* Tanja Blascheck (Inria, University of Stuttgart)
  Smartwatch visualization, mobile visualization

* Enrico Bertini (University of Konstanz)
  Data glyphs

* Lonni Besançon (Lonköpping University)
  Smartwatch visualizations, mobile visualization

* Matthew Brehmer (Microsoft Research, Tableau Research)
  Mobile data visualization

* Wesley Willett (Inria, University of Calgary)
  Word-scale visualization

Collaborators:

* Frank Bentley (Yahoo/Verizon Media)
  Mobile visualization
* Anastasia Bezerianos (Université Paris Saclay, Inria, CNRS, LISN)
Smartwatch visualizations, data glyphs

* Jeremy Boy (UN Global Pulse)
Word-scale visualizations

* Sheelagh Carpendale (Simon Fraser University)
Mobile visualization

* Eun Kyoung Choe (University of Maryland)
Mobile visualization

* Christopher Collins (Ontario Tech University)
Mobile visualization

* Raimund Dachselt (Technische Universität Dresden)
Mobile visualization

* Tim Dwyer (Monash University)
Mobile visualization

* Jean-Daniel Fekete (Inria)
Word-scale visualization

* Tobias Isenberg (Inria)
Mobile visualization

* Daniel Keim (University of Konstanz)
Data glyphs

* Bongshin Lee (Microsoft Research)
Smartwatch visualization

* Florian Mansmann (University of Konstanz)
Data glyphs

* Charles Perin (University of Victoria)
Mobile visualization

* Christian Tominski (University of Rostock)
Mobile visualization
# CONTENTS

## 1 INTRODUCTION 1

1.1 Context and Definition 1

1.2 Available Display Space 2

1.3 Micro Visualization "Complexity" 5

1.4 Research Scope 8

1.5 Research Goals and Methodologies 9

1.6 Contributions 9

## 2 RELATED WORK 11

2.1 Past Work on Data Glyphs 11

2.1.1 What are data glyphs 11

2.1.2 Overview Articles on the Glyph Design Space 12

2.1.3 User Studies on Data Glyphs 14

2.1.4 Summary 17

2.2 Past Work on Word-Scale Visualizations 18

2.2.1 Micro Graphics in Text 18

2.2.2 Micro Visualizations in Text: Word-Scale Visualizations 18

2.2.3 Word-scale Visualizations in Non-Continuous Text 19

2.2.4 Interaction with Word-scale visualizations 20

2.3 Micro Visualizations on Mobile Devices 21

2.3.1 (Some) Mobile Visualization Characteristics 21

2.3.2 Mobile (Micro) Visualization Examples 24

2.4 Studies on Visualization Size 25

2.5 Summary 26

## 3 DATA GLYPHS 29

3.1 Low and High-Density Time Series Glyphs in a Small Multiple Setting 29

3.1.1 Motivation 29

3.1.2 Study 30

3.1.3 Result Summary 32

3.1.4 Design Considerations for Micro Visualization 34

3.2 The Influence of Contour on Similarity Perception of Star Glyphs 36

3.2.1 Motivation 37

3.2.2 Study 1: Contours For Novices vs. Experts 38

3.2.3 Study 2: Perception of Similarity 40

3.2.4 Study 3: Improvements for Star Glyphs 42

3.2.5 Design Considerations 44

3.3 What did we learn about micro visualizations? 45

## 4 WORD-SCALE VISUALIZATIONS 47

4.1 Word-Scale Visualization Placement and Design 47
<table>
  <tbody>
    <tr>
      <td>4.1.1</td>
      <td>Design Space</td>
      <td>48</td>
    </tr>
    <tr>
      <td>4.1.2</td>
      <td>Word-Scale Visualization Placement in Practice</td>
      <td>51</td>
    </tr>
    <tr>
      <td>4.1.3</td>
      <td>Evaluation and Design Considerations</td>
      <td>53</td>
    </tr>
    <tr>
      <td>4.1.4</td>
      <td>Summary</td>
      <td>54</td>
    </tr>
    <tr>
      <td>4.2</td>
      <td>An Exploratory Study of Word-Scale Graphics in Data-Rich Text Documents</td>
      <td>56</td>
    </tr>
    <tr>
      <td>4.2.1</td>
      <td>Study: Designing Word-Scale Graphics</td>
      <td>57</td>
    </tr>
    <tr>
      <td>4.2.2</td>
      <td>Results</td>
      <td>57</td>
    </tr>
    <tr>
      <td>4.2.3</td>
      <td>Discussion: Using Word-Scale Graphics</td>
      <td>66</td>
    </tr>
    <tr>
      <td>4.2.4</td>
      <td>Summary</td>
      <td>68</td>
    </tr>
    <tr>
      <td>4.3</td>
      <td>Interaction Techniques for Embedded Word-Scale Visualizations</td>
      <td>70</td>
    </tr>
    <tr>
      <td>4.3.1</td>
      <td>A Design Space of View Manipulation Interactions for Word-Scale Visualizations</td>
      <td>71</td>
    </tr>
    <tr>
      <td>4.3.2</td>
      <td>Prototypes</td>
      <td>77</td>
    </tr>
    <tr>
      <td>4.3.3</td>
      <td>User Studies</td>
      <td>79</td>
    </tr>
    <tr>
      <td>4.3.4</td>
      <td>Discussion</td>
      <td>79</td>
    </tr>
    <tr>
      <td>4.3.5</td>
      <td>Document-Centric to Visualization-Centric Analysis</td>
      <td>81</td>
    </tr>
    <tr>
      <td>4.4</td>
      <td>What did we learn about micro visualizations?</td>
      <td>82</td>
    </tr>
    <tr>
      <td>5</td>
      <td>MOBILE MICRO VISUALIZATIONS</td>
      <td>83</td>
    </tr>
    <tr>
      <td>5.1</td>
      <td>Designing Smartwatch Visualizations in Context</td>
      <td>83</td>
    </tr>
    <tr>
      <td>5.1.1</td>
      <td>Mobile Visualization Ideation Methodology</td>
      <td>84</td>
    </tr>
    <tr>
      <td>5.1.2</td>
      <td>Ideation Activity</td>
      <td>85</td>
    </tr>
    <tr>
      <td>5.1.3</td>
      <td>Discussion</td>
      <td>88</td>
    </tr>
    <tr>
      <td>5.1.4</td>
      <td>Summary</td>
      <td>90</td>
    </tr>
    <tr>
      <td>5.2</td>
      <td>Smartwatch Visualization</td>
      <td>91</td>
    </tr>
    <tr>
      <td>5.2.1</td>
      <td>Methodology</td>
      <td>92</td>
    </tr>
    <tr>
      <td>5.2.2</td>
      <td>Analysis &amp; Results</td>
      <td>92</td>
    </tr>
    <tr>
      <td>5.2.3</td>
      <td>Q1: Which data types do people show on their watch faces?</td>
      <td>93</td>
    </tr>
    <tr>
      <td>5.2.4</td>
      <td>Q2: In which form is the data currently represented?</td>
      <td>94</td>
    </tr>
    <tr>
      <td>5.2.5</td>
      <td>Q3: What more can we visualize?</td>
      <td>95</td>
    </tr>
    <tr>
      <td>5.2.6</td>
      <td>Discussion and Future Work</td>
      <td>97</td>
    </tr>
    <tr>
      <td>5.3</td>
      <td>Glanceable Visualization: Studies of Data Comparison Performance on Smartwatches</td>
      <td>99</td>
    </tr>
    <tr>
      <td>5.3.1</td>
      <td>Pre-Study: Position and Orientation of Wrist-worn Smartwatches for Reading Tasks</td>
      <td>100</td>
    </tr>
    <tr>
      <td>5.3.2</td>
      <td>Two Studies on Minimal Task Times</td>
      <td>101</td>
    </tr>
    <tr>
      <td>5.3.3</td>
      <td>Results—Condensed and Summarized</td>
      <td>103</td>
    </tr>
    <tr>
      <td>5.3.4</td>
      <td>Design Considerations</td>
      <td>105</td>
    </tr>
    <tr>
      <td>5.4</td>
      <td>What did we learn about micro visualizations?</td>
      <td>107</td>
    </tr>
    <tr>
      <td>6</td>
      <td>SUMMARY AND PERSPECTIVES</td>
      <td>109</td>
    </tr>
    <tr>
      <td>6.1</td>
      <td>The Role of Context</td>
      <td>109</td>
    </tr>
  </tbody>
</table>
6.2 Micro Visualization Application, Design and Scalability 110

6.3 Interaction 112

6.4 Closing Remarks 112

BIBLIOGRAPHY 113
My research has, for many years and in multiple ways, dealt with the problem of limited display space. During grad school, I worked on focus+context displays where the goal was to show detail in a focus region that would otherwise not be visible at the current zoom level. During my PhD, I combat the problem by looking into increasing the display space itself. I worked on creating visualizations and visualization tools for large displays such as tabletops or large vertical display walls and worked on collaboration as a socio-technical issue. After my PhD, I became interested in yet another way to think about limited display space: creating and using small-scale visualizations that fit into limited display space by design.

Making visualizations small allows to either show more of them, to integrate them into contexts that are already visually dense, or to broaden the communication of data to physically small display contexts. Data glyphs, such as these star, clock, and profile glyphs 🕒 🔧 🔧, for example, often appear in a small multiple, nested, or embedded context where many data points need to be compared. As popularized by Tufte [204], small in-line charts such as this stock sparkline 104.39, can display additional information in text documents and make it quickly accessible to a reader. Nowadays, portable and wearable personal devices are prominent examples of small displays with attached computing power. Fitness tracking armbands, hand-held GPS trackers, smart watches, or mobile phones are examples of such devices that frequently carry software that renders small visualizations of a variety of data. Given these use-cases for small-scale visualizations, I became interested in understanding more broadly how data can best be communicated in contexts where we have to “shrink” the visualizations themselves. It is this angle of my research on the problem of limited display space that I cover in this manuscript. A general lack of guidance on the topic and my personal curiosity drove much of the research presented here. The goal of this manuscript is to offer a description of the research space of micro visualizations, a discussion of my past work, and to offer a look into fruitful future research questions and avenues.

<figure><img src="image_2.png" /><figcaption>Micro visualizations on a smartwatch face.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Small multiple data glyphs.</figcaption></figure>

## 1.1 CONTEXT AND DEFINITION

Throughout this document I will use the term “micro visualization” to describe small-scale visualizations. I developed a first definition of the term together with my PostDoc Dr. Tanja Blascheck for an ANR-DFG
grant application (project name: MicroVis). The definition here only
slightly varies from our first joint work and, thus, I will use the term
"we" here to refer to both Tanja and myself.

The term micro visualization previously appeared twice in the
visualization literature. Parnow [165] defined micro visualizations in
a way that “micro” stands for “minimalistic” or “fragmentary”:

[the term micro visualization] aims to describe visualisations
that are not only small in physical space but can also be narrowed
in terms of data dimension or points. Visualisations that have
one or multiple of these characteristics are considered as micro.
Hence “micro visualisations” are basic graphical modifications
or additions, that enhance the comprehension of text. [165]

As such, Parnow’s definition is closely related to sparklines by
Tufte [204] and word-scale visualizations by my PhD student Pascal
Goffin [88] which both are word-sized graphics used to accompany
text. Parnow’s usage of the term relates to ours but only describes
a subset of visualizations we consider as “micro.” In our definition
we apply the word “micro” primarily to the perceived small display
space but not the number of dimensions or data points and expand
the application context beyond text. A year before Parnow’s thesis,
Brandes [24] used the term micro visualization to propose a new
research direction in visual analytics: very high-resolution visualiza-
tions for small to medium-sized displays. Brandes argues that micro
visualizations allow to display data in eye span and—when properly
designed—also allow reading on a micro (detail) and macro (overview)
level. He further describes challenges related to displaying complex
interactions across multiple data dimensions and finding appropriate
analytically-driven abstractions, augmentations, or recombinations of
micro visualizations.

Our use of the term “micro visualization” is closely related to
Brandes’ usage where we focus on small data representations that
can be read “in eye span.” Next, I describe two characteristics that
are important to discuss the space of micro visualizations: available
display space and complexity.

1.2 AVAILABLE DISPLAY SPACE

Micro visualizations use a display space that, at a typical viewing
distance, would be around the size of foveal vision, plus or minus a
few degrees of visual angle. These can be visualizations embedded in
small spaces (for example within text or small multiple contexts) or
seen on small devices such as smartwatches or fitness bands. Ware [213,
p. 145] in his chapter on glyphs recommends to make visualization
displays as compact as possible to minimize the cost of visual search.
Ware recommends to stay inside an average range of 5° of visual angle
<table><caption>Table 1.1: Size of foveal vision at different viewing distances.</caption><thead><tr><th>Viewing distance</th><th>typical context</th><th>viewing</th><th>1° visual angle</th><th>5° visual angle</th></tr></thead><tbody><tr><td>310 cm</td><td>75" TV [59]</td><td></td><td>5.4 cm</td><td>27.1 cm</td></tr><tr><td>57 cm</td><td>monitor [213]</td><td></td><td>1 cm</td><td>5 cm</td></tr><tr><td>28 cm</td><td>smartwatch [16]</td><td></td><td>0.49 cm</td><td>2.45 cm</td></tr><tr><td>20 cm</td><td>smartphone [226]</td><td></td><td>0.35 cm</td><td>1.75 cm</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Figure 1.1: Example display contexts for micro visualizations.</figcaption></figure>

due to the effect of saccadic eye movements – which are so fast that they possibly dwarf important differences in perception performance. Foveal vision itself covers about 1°–2° of visual angle, which at 57 cm viewing distance (a good approximation of monitor viewing distances) would hold an object of 1-2cm width/height [213]. Visual acuities are well below this limit. A viewer with perfect acuity of 1 cannot distinguish objects that are smaller in size than 1' (= 1/60 = 0.0167°) of visual angle. For computer displays, this means that they cannot distinguish objects smaller than 1.67 mm on monitors and 0.58 mm on mobile phones at the above assumed viewing distances. As such, if viewers can actually see differences this small in a visualization also depends on the resolution of the display. Most recent displays for smartphones, -watches, and newer desktop displays have a high-enough pixel density that resolution plays a negligible role in most situations.

Given an approximate visual-angle based definition for available display space still results in large possible ranges of physical display sizes that can hold micro visualizations. Table 1.1 shows sizes of objects at 1° and 5° of visual angle. Next, I describe several possible physical display types and sizes on which micro visualizations can be found in certain viewing scenarios.

**PIXEL SIZED INDICATORS:** Very small data displays, in the range of a few to several millimeters fit into this category. Examples include status indicators such as single LEDs that show battery charging levels or error states of a machine as illustrated in Figure 1.1a.

1.8 mm and 3 mm
LED-sized circle if
printed without
scaling.
<figure><img src="image_5.png" /></figure>

0.5 cm *square if*
*printed without*
*scaling.*

<figure><img src="image_6.png" /></figure>

0.35 cm *square if*
*printed without*
*scaling.*

<figure><img src="image_7.png" /></figure>

1 cm *square if*
*printed without*
*scaling.*

<figure><img src="image_8.png" /></figure>

5.4 cm *line if printed*
*without scaling.*

**SMARTWATCHES:** Smartwatch displays (Figure 1.1b) are typically around 3–4cm wide or high and are viewed from around 28 cm [16], making micro visualizations on these displays around 0.5 cm–2.5 cm wide. Micro visualizations are very common on smartwatch complications (data carrying elements on a watch other than time) and show information such as step counts, calories burned, or heart rate.

**MOBILE PHONES:** Smartphone screens as shown in Figure 1.1c are now commonly around 15 cm on the diagonal and have a high resolution of more than 150 px per cm. Compared to watch-sized displays, phone-sized displays can convey more information but are held closer than smartwatches (Table 1.1) and subsequently the size of micro visualizations is smaller at 0.35 cm–1.75 cm. Micro visualizations on phones can often be found in widgets or as part of fitness or health apps.

**MONITORS:** Monitor-sized displays can cover a fair portion of the viewer's field of view even when the viewer is positioned further away. Micro visualizations on monitors can be as large as 1 cm–5 cm and the space is often available to arrange many in small-multiple displays. Examples of micro visualizations designed for monitor viewing are data glyphs and word-scale visualizations. In Figure 1.1d very small line charts are embedded on a webpage to show price changes for graphics cards in a table layout.

**TVS:** TV-sized large displays are often viewed from distances of several meters. Consumer displays such as TVs are rarely used for data analysis but visualizations may still be seen, for example as part of video games to communicate information about an in-game character or progress in the game. At a recommended viewing distance of 310 cm for a 75" display, the TV will cover 30° of a viewer's field of view, leading to a good head-on viewing experience [59]. At this viewing distance micro visualizations could be cm5–27 cm large.

While the size definitions above seem relatively precise, they should be taken only as rough indications. We still do not know enough about the differences in reading visualizations of different small sizes and as such, it is premature to discuss a precise size range for micro visualizations and it is important to allow for more precise definitions further on. As such, I want the above discussion to be seen as a range and not as a restriction to a pre-set visual angle. During human vision there are certainly other factors that may influence how quickly or correctly a small visualization can be perceived and at which size we might see differences in performance. In addition to visual angle, Healey and Sawant [104] cite resolution, viewing parameters such as amount of data being displayed, the visualization technique, as well as
data properties and analysis tasks. For example, this Horizon Graph [182] or this unit-based micro visualization [90] were designed for monitor reading. They are both less than 0.5 cm high but extend beyond 1–2 cm of foveal vision width outlined above and I would still categorize them as micro-visualizations. As with most other types of categorizations, the border between micro- and larger visualizations might be fluid. Next, I highlight additional complexity-related characteristics that allow to discuss types of micro visualizations in more detail.

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

1.3 MICRO VISUALIZATION “COMPLEXITY”

The definitions of Parnow [165] and Brandes [24] point out another
important characteristic of micro visualizations and that is, their com-
plexity. Both definitions seem to contradict each other regarding how
complex micro visualizations are meant to be. While Parnow limits
micro visualizations to a few data dimensions and points, Brandes
discusses them as allowing for complex interactions of multiple data
dimensions. In practice, it is very difficult to describe visualizations
according to how “complex” they are, as complexity is a term that may
apply to a variety of aspects related to a visualization: the data, the
visualization design, the interaction, context of use, tasks, or intents
for example. Previous research on visualization complexity has made
attempts at this question and suggested several complexity measures.
Brath [26], for example suggests to use the number of data points and
different data dimensions encoded, as well as the mapping of visual
channels to data dimensions, how understood or known a used visual
representation is (e. g., bar charts are more known than treemaps),
the presence of occlusion, and how many data points are identifiable.
Zhu [232] proposed two complexity scores for visual mappings: a)
how difficult visual mappings are to interpret ranging from “common
knowledge: there is no need to memorize or refer to a legend” to “very difficult
to interpret. There is no legend. A typical reader has to memorize the mapping
between the value of the visual attribute and the value of the corresponding
data parameter;” and b) how efficient visual search is in terms of how a
target stands out from its background.

Janlert and Stolterman [117] discuss the complexity of digital ar-
tifacts more broadly in the context of human-computer interaction
attempting to derive a description of complexity using an artifact-
centric approach. Using an artifact-based approach allows to describe
the complexity of an artifact as a property of the artifact rather than
a subjective experience of how a viewer relates to the artifact. My
description is inspired deeply by Janlert and Stolterman [117]'s work.
Throughout this document I attempt to describe how complexity can
manifest in micro visualizations. Important in this categorization of
complexity is that it refers relative to the purpose a visualization has
<figure><img src="image_12.png" /><figcaption>A scatterplot of 150 points.</figcaption></figure>

been designed for. For example, a scatterplot may have been chosen to show clusters of 150 data points as in the image in the margin. An analyst attempting to get an overview of the 3 clusters present might find this scatterplot of low complexity; another analyst trying to derive an average of the x- and y-axis dimensions might find the scatterplot too complex. The latter analyst's purpose does not align with the visualization's purpose. Sometimes a visualization might also be poorly designed and might not fit the purpose it was designed for. In the following discussion, however, I assume effectively designed visualizations with a purpose that aligns with a viewer's purpose. I use the four of Janlert and Stolterman [117]'s loci where the complexity of a micro visualization might manifest itself:

**INTERNAL COMPLEXITY** relates to the data collection, processing, and wrangling that happens behind the surface of a micro visualization and that is typically invisible to a viewer. For example, for a fitness tracker display of calories burned 396 an algorithm might calculate a basal metabolic rate (e. g., from a person's weight, height, gender, age) and combine it with activity data from device sensors such as accelerometers or heart rate sensors and manually entered activity data [74]. Each of these calculations will include specific algorithms to turn sensor readings into activity information and as such are rather complex. On the other hand, a display of average star ratings ⭐☆☆☆☆☆ for a restaurant might be a simple average and less complex internally. This type of complexity is very related to the application of a micro visualization and the embedding in a potential device or device ecology. My past work is least concerned with this type of complexity but I will mention it occasionally.

**EXTERNAL COMPLEXITY** relates to how the micro visualization represents the underlying data. This external complexity can be apparent and subjective and will vary based on a viewer's knowledge of the data and representation type, analysis skill, context of use, etc. Apparent external complexity is often what user studies in visualization measure with Likert items relating to comprehensibility or understandability of the visualization. Janlert and Stolterman [117] differentiate apparent external complexity from *real* external complexity. Real external complexity relates to an objective and fixed way to determine the complexity of a visualization. This could be some of the countable measures proposed by Brath [26], such as the number of data dimensions or data points that a visualization exposes, the density of the encoding, and others. For example, a heart rate display could include a simple single number display of the current heart rate 68 or could include multiple data points showing data on the fluctuations of heart rate in the last 10 minutes 68. Micro visualizations
of sleep may include a single data dimension such as hours slept (7h 45min) or include multiple dimensions of sleep such as the difference between sleep phases (REM, Deep, Light Sleep). The design of a data encoding through choice of visual variables is also an objective property of a (micro) visualization.

While it seems intuitive to think of a visualization that shows more data points or more data dimensions as more complex, such a natural progression might not exist for data encoding types (e. g., length vs. angle vs. position). For example, should an encoding of a data glyph as a line chart 📉 be considered more or less complex than an encoding that uses a sequential color scale ▸▸▸? Some of the past studies on visual variables (e. g., [54, 106, 201]) allow us to say something about how effective people are on average at judging certain data encodings; but it is not (yet) clear whether a more effective encoding leads to a lower apparent complexity but some indications exist that this might not be the case. Throughout this document, I will not treat visual encoding as a potential factor of complexity but will discuss the encodings based on what we know about their effectiveness in representing data.

**INTERACTION COMPLEXITY** describes the complexity of what a person does in relationship to how the micro visualization reacts. Interaction complexity can but does not have to relate to internal or external complexity but is highly connected to the purpose of the visualization. For example, a visualization designed to communicate a single value such as the current humidity HUMIDITY could include very simple *passive interaction*. Passive interaction involves a person simply looking at and consuming information from a visualization; the visualization itself does not change based on a viewer's actions and will rest as externally or internally complex as it has been designed. If a visualization has been designed to show multiple personal health factors in the same physical space, interaction complexity will increase beyond passive interaction when an interaction has been specified to initiate a change in view to a new visual representation; ideally with a transition animation that also increases internal complexity. A person could swipe or tap a visualization, for example to trigger a fading out of the previously shown data dimension and the fading in of the new dimension. A simple tap could also be used to show and hide detail-on-demand. Higher interaction complexity might be required when the visualizations are meant to reveal how they relate to other representations in a multi-view setting. Micro visualizations may, for example, support brushing and linking or other comparison operators to relate the data in multiple word-scale visualizations shown on a single page. Generally, micro visualizations with low external complexity (simple

<figure><img src="image_13.png" /></figure>

Simple view change
interaction

<figure><img src="image_14.png" /></figure>

Steps: 9091
Distance: 4850m

Tap for
details-on-demand

<figure><img src="image_15.png" /></figure>

Brushing and
linking as part of a
complex interactive
micro visualization
setup.
designs) do not necessarily have low interaction complexity as
we will see later in Section 4.3.

MEDIATED COMPLEXITY describes external, contextual factors im-
pacting the complexity of micro visualizations. For example,
a micro visualization displayed on a smartwatch may require
certain design features that can adjust to changing environments
of the wearer. A smartwatch visualization will have to be able
to effectively show data to a person sitting in bright sunlight at
the beach, while riding a bicycle, or when watching a play in a
dark theatre. In the latter two scenarios, we can, for example not
assume long viewing timespans and micro visualizations need
to be readable in sub-second glances.

In summary, complexity as a property of a (micro) visualization
is difficult to define concretely and more research is necessary. In
particular to expand on past work towards real external complexity
measures for visualization would be very interesting (and difficult)
avenues for future work. In this document, I will mostly use intuitive
notions of complexity based on my observations of complexity from
the results of my user studies. As none of my work set out to study
complexity directly, I rely on my own post-hoc review of my past
work in this context.

1.4 RESEARCH SCOPE

Within the general definitions of micro visualizations outlined above,
the work I present in this document is narrowed in scope. Specifically,
I focus on scenarios that:

* include non-animated micro visualizations. While using animation to change content of a representation is one way to tackle problems of limited display space, we still do not know enough about how limited display size impacts even static representations and I started my work there.

* occur in static viewing environments. In particular smartwatches and fitness trackers are prime usage contexts for micro visualizations but here the visualizations often need to be read while the wearer is in motion—running, walking, or sitting in a moving vehicle. It is still unclear to which extent our knowledge of desktop-sized visualizations transfers to contexts that involve either a moving observer or a moving display and I am actively working with my PhD student Lijie Yao on tackling this challenge. The work is still ongoing and will not be covered here.

* are made for single viewers. While much of my past research focused on synchronous collaborative data analysis scenarios, this work is focused on single viewers. Given their size, micro visualizations are likely mostly useful for personal viewing
contexts but could of course be used in systems designed for
asynchronous collaborative work or on large screens seen by a
group of people from afar.

* include abstract 2D representations. While I did not actively exclude continuous spatial (3D) data representations such as volumes or surfaces, they do not fall into my main research expertise. Yet, I acknowledge that 3D glyphs are in active use in tensor visualizations, for example, where additional forms of mediated complexity arise due to the underlying spatial data.

1.5 RESEARCH GOALS AND METHODOLOGIES

Ultimately, much of my past research, as well as the work covered
here, has the goal to pave the way for a pervasive use of visualizations
in “beyond-the-desktop” contexts. Here I cover my past and current
specific focus on small data representations. In particular, I wanted to
understand more about specific application contexts (small multiple
displays, text, and smartwatches) but also more fundamentally about
how people perceive data when displayed in small display spaces.
When starting this work I still knew little about micro visualization
viewing contexts. My first goal was to conduct fundamental research
on the perception of micro visualizations answering questions that
compared different types of visual encodings. Gaining an understand-
ing of the fundamental components of visualization at this level is
essential for the creation of sophisticated micro visualizations that
can be used with different small displays and convey relevant infor-
mation in an appropriate manner. To tackle these questions I relied
mostly on mixed-methods user studies where a quantitative analy-
sis methodology is coupled with interviews or questionnaires. My
work on micro-visualizations, on the other hand, focused on more
application-centric questions regarding design spaces for placement,
visual encoding, and interaction. My work on mobile micro visualiza-
tions for wearable devices follows both an application-driven approach
with a focus on perception but also aims at understanding current
practices, as micro visualizations in this context are deployed and in
active use by millions of people.

1.6 CONTRIBUTIONS

Throughout this document I added content from my past work and acknowledged my co-authors in each associated chapter or section. I reframed and extensively shortened the past work in relationship to the discussion on micro visualization properties above and did not copy entire papers. In summary, I contributed:

* Perceptual user study results on micro visualizations in small multiple contexts and in smartwatch usage contexts
* Design spaces regarding representation and interaction design for micro visualization in the context of text documents

* Research on current practices and experiences with smartwatch micro visualizations

* A design methodology for smartwatch visualization
Throughout this document I will discuss micro visualizations of three types: data glyphs, word-scale visualizations, and mobile micro visualizations for wearable devices such as smartwatches. This related work section introduces each type and relevant related work. Given our definition from Chapter 1, several other existing visualization techniques have been designed as micro visualization outside one of these three types. Assuming rendering on desktop monitors, Horizon Graphs [182] are one such example. This time series visualization manages to use a small vertical display space through an approach called “two-tone pseudo-coloring,” by cutting filled line charts into bands, layering them on top of each other, and using different color hues. ScentedWidgets are an example of simple chart-type micro visualizations embedded in the context of GUI widgets to provide information scent [217]. Important for my work are papers related to the three types of micro visualizations covered here but also studies on micro visualizations that have assessed the impact of small physical display spaces on the readability of the data; I will review them in the last section of this chapter.

## 2.1 PAST WORK ON DATA GLYPHS

In 2017 we published a systematic review of experimental studies on data glyphs in IEEE TVCG [82]. This work was led by Johannes Fuchs in collaboration with myself, Anastasia Bezerianos, and Daniel Keim. In this section, I use the pronouns “we/our” to refer to this set of co-authors. Here, I present a condensed version of this work with a focus on characteristics related to the complexities and size characteristics of micro visualizations introduced in Chapter 1.

### 2.1.1 What are data glyphs

Data glyphs are a visualization technique for multivariate data where typically multiple data points (records) are being displayed at once. Data glyphs encode data points by assigning values for multiple dimensions to one or more marks and their corresponding visual variables. Data glyphs in the domain of visualization date back to at least the 1950s, with metroglyphs ♂♀♂ being one of the first designs published [4]. A somewhat infamous, and thus well researched, example of data glyphs are Chernoff faces (Figure 45) which encode data values
<table><caption>Table 2.1: Overview of defining glyph characteristics mentioned in the literature.</caption><thead><tr><th rowspan="2">References</th><th colspan="3">Data glyph characteristics</th></tr><tr><th>small/-<br>compact</th><th>encodes differ-<br>ent dimensions</th><th>uses different<br>visual channels</th></tr></thead><tbody><tr><td>Borgo et al. [20]</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Fuchs et al. [80]</td><td>X</td><td>X</td><td>-</td></tr><tr><td>Maguire et al. [147]</td><td>X</td><td>X</td><td>-</td></tr><tr><td>Brandes et al. [25]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Chung et al. [53]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Levkowitz [137]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Lie, Kehrer, and Hauser [140]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Munzner [153]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Ropinski, Oeltze, and Preim [176]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Ware [213]</td><td>-</td><td>X</td><td>X</td></tr><tr><td>Ward [212]</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Chuah and Eick [51]</td><td>-</td><td>-</td><td>X</td></tr><tr><td>Ribarsky et al. [175]</td><td>-</td><td>-</td><td>X</td></tr></tbody></table>

in 2D facial features such as the length of the nose or the orientation of eyebrows. Star Glyphs are another type of glyph-based encoding that has received research attention [188] and has been used in various applications [42, 167, 222].

Definitions of data glyphs vary in the literature. In our past work, we found that in early years the term “glyph” was often used as a synonym for the metroglyph [65, 114] but that with increasing numbers of different glyph designs published, more abstract definitions of data glyphs emerged. To compare how researchers have defined the term, we extracted descriptive keywords from published definitions, and summarized in which publications they appeared in Table 2.1. While Table 2.1 is certainly not exhaustive, it serves to show the wide variety of ways researchers think of, and define, data glyphs, an observation similarly stated by Munzner [153, p. 280]. It also shows that in their discussion of glyphs, many researchers did not restrict glyphs to a small display size. As we will see, however, they have often been studied in a small multiple setting and, thus, at a small size. In addition, glyphs often share another frequent characteristic of micro visualizations such as word-scale visualizations and smartwatch visualizations: a lack of labels and reference structures.

### 2.1.2 Overview Articles on the Glyph Design Space

Over the years, many different glyph variations were introduced to better fit certain data types, or to solve specific tasks more effectively.
Yet, while many designs have already been explored, the mapping possibilities of data dimensions to visual glyph encodings is nearly endless [153] and many more designs are certainly imaginable. Researchers have attempted to structure existing glyphs according to various criteria. Ward [211, 212] was one of the first to provide a structure of the glyph design space by classifying different layout possibilities into data-driven and structure-driven layouts. In a data-driven arrangement spatial position is determined by data: this can either be the raw data used as in a scatterplot, or a projection-based approach such as PCA. A structure-driven layout makes use of relations between the data points to calculate a layout. For example, hierarchical information can be used to lay out glyphs in a tree structure. Ward extended this work with a categorization of the visual characteristics of data glyphs. He structured data glyphs based on their mapping of data to visual attributes into three different classes: a *many-to-one mapping* where each data dimension is mapped to the same visual variable (e.g., profile glyph [65]); a *one-to-one mapping* showing each data attribute with a different visual variable (e. g., Chernoff faces [45]); and a *one-to-many mapping* representing the data dimensions redundantly with many different visual attributes (e. g., using both position and color to encode a category ).

In contrast, Chung et al. [52] proposed a categorization based on the visual channels used to represent the data and the spatial dimensionality of the glyph (2D, 2.5D, and 3D). The authors also discussed design criteria and guidelines for glyph visualizations, such as the normalization of data input for each dimension or the use of redundant mappings. Glyph size is only tangentially mentioned by the authors as a factor influenced by the number of glyphs that need to be drawn in a given space. The authors suggest that in the case of many small glyphs, simpler designs might be desirable.

An extensive survey on data glyphs was presented by Borgo et al. [20]. The authors cover different glyph representations and propose guidelines for designing data glyphs based on a collection of design principles in the literature. Relating to the small visual footprint of glyphs, the authors suggest that in particular the global features of a glyph play a role in how a glyph of interest is found amongst other glyphs. While Borgo et al. also include several empirical studies in their survey, their focus is on design study papers showing the applicability of data glyphs to different data sets and tasks.

A more data-specific survey on glyphs in the medical domain was presented by Ropinski, Oeltze, and Preim [176]. The authors classified glyph-based visualizations for medical data into two groups: pre-attentively and attentively identifiable glyph designs and discuss and extend some past work, e. g., the placement strategies by Ward [211, 212] to better fit this domain. From an analysis of the literature and best practices mentioned, the authors further derived design guidelines for

<figure><img src="image_16.png" /><figcaption>Geospatial
data-driven
arrangement of
glyphs.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Alphabetical
structure-driven
arrangement of
glyphs.</figcaption></figure>
<table><caption>Table 2.2: **Presentation Setting:** This table distinguishes between the number of data glyphs shown to the participants during the studies and the used layout. Color is used to better distinguish between the different categories.</caption><thead><tr><th></th><th>Layout</th><th>References</th></tr></thead><tbody><tr><td rowspan="2">Single</td><td></td><td>[2][22][40][119][199][146]</td></tr><tr><td>Text</td><td>[91]</td></tr><tr><td rowspan="5">Multiple Glyphs</td><td>Grid</td><td>[19][32][39][44][46][57][58][75][77][76][80][81][100][103][110][115][114][116][126][125]</td></tr><tr><td></td><td>[136][142][149][150][154][170][184][186][197][216]</td></tr><tr><td>Geo map</td><td>[139][143][144][157][148][171][215][227]</td></tr><tr><td>Scatterplot</td><td>[49][72][189]</td></tr><tr><td>Other</td><td>[1][35][60]</td></tr><tr><td></td><td>Node-link</td><td>[43][183]</td></tr><tr><td rowspan="3">Varying</td><td>Grid</td><td>[79][151][190][192][191][229]</td></tr><tr><td>Node-link</td><td>[41][66][221]</td></tr><tr><td>Geo map</td><td>[156][220]</td></tr></tbody></table>

developing glyphs for this domain. The authors do not mention the influence of size on the readability of data glyphs but mention that size would depend on dataset resolution and that overlap needs to be considered. The authors do mention the overall size of a glyph as a possible channel to map a data dimension to. Should a size-mapping be chosen, the authors recommend not to use perspective projection.

In contrast to the categorizations above, we focused on providing a summary of performance assessments of different glyph designs. From our work, we summarized some design guidelines and provided further open research questions.

### 2.1.3 User Studies on Data Glyphs

Other researchers have categorized subsets of the glyph study design space. Nelson [155], for example, discusses the history of Chernoff faces [45] with its many variations such as the Flury-Rydwiel [75] or Kabulov faces [120]. She also discusses studies investigating performance changes for different data types or visual variations. We took this work as inspiration, but provided a much more comprehensive view on the study design space.

For our systematic review we focused on sampling user studies from 64 papers in which participants performed controlled, quantitatively measured tasks with data glyphs. These quantitative measurements could (but did not have to) be accompanied by a subjective assessment of the tested glyphs (e. g., according to aesthetics, confidence, etc.). Next, I present some of the most important results from this survey related to micro visualization size and complexity:
STUDIES ON NUMBER OF GLYPHHS AND THEIR LAYOUT: Glyph complexity has been studied via several characteristics of complexity. Table 2.2 presents studies conducted using different types of presentation settings, which can be considered aspects of mediated complexity. For example, the presence of background information or the possibility of overlap may influence the design and perception of data glyphs.

We identified three types of studies that considered the number of glyphs presented as a factor of real external complexity: those that presented only individual glyphs to the viewers (10.94% ■), those that presented a fixed number of more than one glyph at a time (71.88% ■), and those in which the number of presented glyphs varied but was always higher than one (17.19% ■). Seven of 64 papers did not report the exact number of glyphs represented on the screen: [1, 35, 41, 43, 44, 66, 171].

The goal of most of the studies with multiple glyphs was to investigate changes in performance when increasing the number of visible data points in grid layouts [151, 190, 192], geographic maps [156, 220], and node-link diagrams [221]. The amount of glyphs visible to participants changed from 5–50 [151]; 5–15 [190, 192]; 6–18 [191]; 9–23 [156]; 4–300 [220]; and 30–48 [221]. The glyphs used in these experiments were either faces [151, 156, 190, 192], unique glyph designs (i. e., MIL-STD2525 [191], arrow glyphs [220]), or star glyphs [221]. In all studies the performance of participants dropped with an increasing number of data points.

For the 46 studies that tested a fixed number of multiple glyphs at a time, we found five types of layouts. The most frequent was a common small-multiples grid (65.22% ■), followed by geographic maps (17.39% ■), scatterplots (6.52% ■), node-link diagrams (4.35% ■), and other layouts (6.52% ■) like different 3D environments. Four studies investigated the influence of positioning or background information on the performance of data glyphs [79, 103, 148]. Frisson et al. used a visual search task to examine the benefits of a two dimensional projection compared to a grid layout used in small multiple settings [79]. Performance was lower for the two dimensional projection, since after projection, some data glyphs ended up overlapping each other, which caused a loss of information making it difficult to detect the stimulus. The influence of reading data glyphs with different geographic backgrounds was investigated in only one study conducted by Martin [148]. He measured the performance of participants working with weather vane glyphs ⏎ placed in a grid on a varying geographic map. Surprisingly, his results indicated the background had no influence on the performance of reading these data-glyphs. Healey and Enns conducted an experiment to compare the interaction of different visual features in the surroundings of the glyph stimulus for a visual search task [103]. Results indicated that color variations due to the presence of other glyphs in the neighbor-
<table><caption>Table 2.3: **Number of Dimensions:** This table illustrates the different data dimension densities used in the studies.</caption><thead><tr><td>No. of Dimensions</td><td>References</td></tr></thead><tbody><tr><td>2 & 3 Dimensions</td><td>[1][39][40][41][60][67][76][77][103][199][139][148][171][186][215][227][229]</td></tr><tr><td>4 & 5 Dimensions</td><td>[22][43][44][49][72][114][119][142][143][144][146][154][157][156][192][190][191][220]</td></tr><tr><td>6 & 7 Dimensions</td><td>[110][197]</td></tr><tr><td>8 & 9 Dimensions</td><td>[125][189][170][126][115][46][151]</td></tr><tr><td>10 - 15 Dimensions</td><td>[2][136][183][58][150][32]</td></tr><tr><td>17 - 20 Dimensions</td><td>[75][149][19][57][100][116][184]</td></tr><tr><td>Varying</td><td>[80][216][81][221]</td></tr></tbody></table>

hood of the stimulus glyph, caused a significant interference effect when participants had to judge heights of glyphs or density patterns. However, different densities in the surroundings of the stimulus or heights of neighboring glyphs had no effect on the detection of colored glyphs. In summary, the influence of background and layout on reading data glyphs had so far received little research attention. The limited evidence from this work suggests that the background and neighborhood of a glyph did not affect glyph readability. Nevertheless, more work is needed to determine the perceptual difficulties of reading glyphs on different backgrounds and using varying layouts.

**Studies related to the Number of Dimensions Visualized:** The number of data dimensions may have an influence on the external complexity of a glyph. Only four studies (6.25% ■■■■) used the number of dimensions itself as a study factor and thus varied between glyphs with different dimension counts [80, 81, 216, 221]. The remaining 60 studies tested glyphs with various fixed numbers of dimensions. Of these, 44 tested less than 10 dimensions. An overview of different dimensionality settings is provided in Table 2.3. Three papers did not report about the number of dimensions encoded by the glyph designs [66, 79, 91].

The results of studies varying the number of dimensions as a factor showed that different designs were impacted to different extents. In our own previous work [80] discussed later, for example, the performance of star glyphs dropped significantly in a lookup task when increasing the number of dimensions from 24 to 96, whereas the performance of line glyphs stayed stable. Wilkinson [216] also varied the number of dimensions to investigate changes in performance for different glyph representations. His results indicated that increasing the number of dimensions had no significant effect on the *ranking* of tested glyph designs, although there was a drop in performance overall. However, it is interesting to note that even slight variations of a glyph design can be affected differently by the number of dimensions. We later tested the effect of increasing the number of dimensions on whisker glyphs (star glyphs without a contour line), traditional star glyphs.
glyphs and polygon variations. Although the performance dropped for all variations, whisker glyphs were affected the least [81]. Thus, for all four studies increasing the number of dimensions negatively affected the performance of participants using data glyphs.

### 2.1.4 Summary

In summary, glyphs are one of the most relevant types of related work for micro visualizations, because researchers often aim at their design being holistically perceivable by a data shape (the macro reading Brandes [24] referred to). From our assessment of studies on data glyphs we extracted several open research directions including the following:

**TYPES OF USER STUDIES:** Even though we focused on user studies with quantitative components, we found only a few qualitative studies that considered how glyphs are used in practice within real applications. One such exception is the experiment conducted by Sreng et al. [194] where participants used a 3D automotive assembly tool and answered questions about the perceived usefulness of the embedded glyphs. Although this study provided qualitative observations in the form of questionnaires, more in-depth observational studies could inform our understanding of how glyph-based applications are adopted and used in practice and could, thus, provide new insights on which to base design choices.

**STUDY TASKS AND MEASURES:** In the majority of studies participants had to perform synoptic tasks (i. e., similarity search, visual search, trend detection). This is not surprising given that glyphs are often used to provide quick overviews over a large number of multi-dimensional data points. Exploration tasks or extracting insights from an unknown dataset are other realistic real-world analysis tasks. They should, therefore, be added to the repertoire of user study tasks in glyph evaluation to further reason about the practical applicability of data glyphs.

**GLYPH PRESENTATION SETTING:** A large number of studies presented glyphs as small multiples using a grid layout. This is interesting, as it is not clear that grid layouts present the most commonly assumed usage context for glyphs. Evaluating different arrangements of more complex layouts would help to better understand the influence of specific data glyph designs on the context and vice versa.

**GLYPH TYPES AND DATA ENCODINGS:** Quantitative user studies should be conducted to compare data glyph designs which have not yet received much research attention (i. e., pie chart glyphs). The
number of dimensions should be varied during the experiment and considered as a factor for analysis, to better understand glyph scalability.

## 2.2 PAST WORK ON WORD-SCALE VISUALIZATIONS

Text documents are often central to a person's work. In such contexts, integrating small pieces of contextual information within the text can be a promising approach to allow the reader to extract detailed information without moving his or her eyes from a specific text location. Here, I briefly review past work related to micro visualizations placed in the context of surrounding text as a factor of mediated complexity.

### 2.2.1 *Micro Graphics in Text*

Small pictogrammatic, iconic, and symbolic graphics have been used alongside text for centuries. Galileo [83]'s drawings of Jupiter and its moons *Jupiter's moons* are one famous example as well as Byrne [34]'s illustration of Euclid's elements ▲ = ▲. Emoticons 😊 😞 are a more recent example of iconic representations that are incredibly common in computer mediated communication. Dresner and Herring [64] describe emoticons' rich set of functions within written communication that goes well beyond the expression of emotion. Children's books are another place where iconic graphics are sometimes used to replace words in a sentence to help beginning readers as in "Benny's Big Bubble" by J. O'Connor [161]. Designers have also used symbols or icons as word-scale graphics to tag words in text to provide meta-information. ⬤ Verjat [207], for example, uses a human profile icon to make names stand out on his online CV.

### 2.2.2 *Micro Visualizations in Text: Word-Scale Visualizations*

Tufte [204] was one of the first to argue strongly for using in-line data graphics – sparklines – to enrich text documents with information that could otherwise not be succinctly conveyed by the text itself. He describes sparklines as *small, intense, simple, word-sized graphics with typographic resolution*. Examples include small stock charts embedded next to the name of a company such as Apple 104.39 or average rainfall charts next to a city name such as Berlin 26mm.

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

In our own previous work [92], we expanded on Tufte's definition and proposed the term *word-scale visualization* to define a wider variety of small, embedded data-driven graphics. We defined word-scale visualizations so they can take on a greater variation of display space than sparklines, ranging from as small as a single character to as large as several words. We also use the term "visualization" to depart
from the strong connection to line-based visualization that the term
"sparkline" suggests. The term word-scale visualization more clearly
allows the use of a variety of encodings, including geographical maps,
heat maps, pie charts, and more complex visualizations. As such,
word-scale visualizations are often micro visualizations. Word-scale
visualizations, however, are always placed in context with text, which
is closer to the definition of micro visualizations used by Parnow [165]
without his restriction to only a few data dimensions.

Brath, MacMurchy, and Banissi [27] introduced a design space for
SparkWords, a related concept which embeds categorical, ordered, or
quantitative data into letters or words by modifying their typogra-
phy, color, and other properties. More recently, Latif and Beck [131]
introduced a design space for word-sized graphics based on the data
visualized and show that even multivariate, spatial, and relational
data can be represented at a small scale.

Researchers have previously proposed and tested some specific
word-scale visualization designs. Beck and Weiskopf [12] presented
an overview of word-sized graphics for scientific texts citing many
examples and here, I only highlight some. *SportLines* [169] , for
example, show phases of player actions across a soccer field. Brandes
et al. [25] introduced the concept of *Gestaltlines* which are sparkline-
sized data-intensive graphics designed to convey a general *gestalt* or
overall form. The authors tested one Gestaltline design (||/||/||/||/) and
found that untrained readers could quickly detect holistic patterns,
outliers, and breaks in the data. Greenhill, Ward, and Sacks [98]'s
*Separation Plots* ( ) can help readers assess the predictive
power of models with binary outcomes. A perfect model would cor-
rectly predict all actual outcomes of an event and produce a plot with
all dark red stripes clustered to the right side (as in this example).
Greenhill et al. discuss several examples and potential problems with
the interpretation of their plots, but did not conduct a user study.

2.2.3 Word-scale Visualizations in Non-Continuous Text

While the word-scale visualizations above are meant to be used within
a text document, other designs have been proposed for non-continuous
documents such as tag clouds or tables. For example, *SparkClouds* [135]
(*medical data*) convey temporal trends for words in a tag cloud by attach-
ing small line charts to each word. In their original paper, Lee et al.
[135] compared SparkClouds to three other trend visualizations, and
found that participants could read SparkClouds comparatively quickly
across several tasks, and preferred them over other representations.
Nguyen et al. [160] followed a similar approach, and proposed several
tag designs that use the background of a word to convey temporal
information, while placing tags on a map to further convey geospatial
relationships. Beck et al. [10] propose *word-sized eye tracking visualiza-
tions ([(colorful diagram)](https://i.imgur.com/12345.jpg)) which are meant to be used alongside think-aloud transcripts in a tabular view. The authors re-analyzed data from a previous small user study using the new graphics and show how their observations could be reported using word-sized eye tracking visualizations inline.

### 2.2.4 Interaction with Word-scale visualizations

Work on interaction for word-scale visualizations relate to a variety of tools that help analysts interactively organize and compare elements within a text or a visualization. Victor [208]'s "explorable explanations" show how to use direct manipulation of text or visual elements to highlight relationships resulting in an improved active reading experience. In many of his examples, interacting with one element changes other elements based on a previously established relationship. Closely related are Dragicevic et al. [63]'s explorable multiverse analysis reports (EMAR) which introduce interactive graphics into the results sections of research articles.

Previous research has also introduced several kinds of interaction techniques for word-scale visualizations. Parnow and Dörk [166] describe techniques for placing micro visualizations into the text, brushing and linking between micro visualizations, and highlighting connections between entities and visualizations. Beck and Weiskopf [12] propose three levels of interaction (no interaction, local interaction, and global interaction) for word-sized graphics. Their global interaction level, which includes interactions between multiple charts comes closest to the spirit of our design space [89] introduced in Section 4.3.

Several concrete examples of interactive word-scale visualizations also exist in other domains. To assist in debugging electronic circuits Frishberg [78] proposed interactions for details-on-demand by clicking on a data item in the sparkline. Similarly, Watts [214] implemented a jQuery library for dynamic and interactive sparklines that allows hovering over data items to provide more information. Meanwhile, Hoffswell, Satyanarayan, and Heer [109] described and implemented a design space of word-scale visualizations for source code. Their designs demonstrate several interactions, including hovering to show details-on-demand as well as a variant of brushing and linking.

Finally, Latif, Liu, and Beck [133] describe a framework that integrates text, word-scale visualizations, and larger visualizations. In their framing, each element has triggers and filters to provide details-on-demand, highlight content, reset a visualization, or switch its content. Latif and Beck [132] extend this approach in VIS Author Profiles a visual analytics system for examining research profiles. They provide a tighter connection between word-scale visualizations and larger visualizations, including interactions that overlay word-scale visualizations onto larger charts.
## 2.3 MICRO VISUALIZATIONS ON MOBILE DEVICES

Research on micro visualizations for wearable devices belongs to a larger stream of research on mobile visualization I am interested in. With Bongshin Lee, Eun Kyoung Choe, and Raimund Dachselt we organized a Dagstuhl seminar in 2019 [50] on mobile data visualization. After the Dagstuhl seminar we co-edited a book on mobile data visualization [134]. In this section, I will present from the introduction chapter of the book I co-wrote with Ricardo Langner, Lonni Besançon, Christopher Collins, Tim Dwyer, Tobias Isenberg, Bongshin Lee, Charles Perin, and Christian Tominski [130]. Any mentions of 'we/our' in this section refer to this set of co-authors.

We may have an intuitive understanding of what is meant by mobile data visualization. Yet, in the context of data visualization, the term mobile can be interpreted in several ways. For example, it may describe visual representations shown on devices that are inherently mobile. It may also describe visualizations meant to react to viewers who are mobile relative to the display. Alternatively, it may describe visualizations that are themselves mobile across devices and screens, or in space. In this section, I use characteristics of mobile visualizations presented in our book chapter [130] to introduce and describe complexities inherent to micro visualizations for wearable devices such as smartwatches or fitness trackers that I have been mostly working on.

### 2.3.1 (Some) Mobile Visualization Characteristics

**PHYSICAL DISPLAY SIZE:** Looking at today's mobile devices, it is clear that the *Physical Data Display Size* is an essential aspect as size impacts mobility. As mentioned in Chapter 1, micro visualizations can appear on displays of any size and the distance of the viewer will determine whether they are perceived as micro visualizations or not. Most of my recent research has focused on smartwatch-sized displays that are typically around 3–4 cm wide or high. Some deviate from a standard rectangular form and use a circular geometry, which is an interesting design constraint for visualization.

**DATA DISPLAY MOBILITY:** Another key aspect of mobile data visualization is the *Data Display Mobility*, which captures the movement of the display(s) containing visual representations of data. Fixed, movable, carryable, wearable, and independently moving displays can be differentiated along this dimension (Figure 2.1).

Most of the existing mobile visualization research has been conducted in the category of carryable devices, including some of my own [28, 29] that I do not cover here because it did not focus on micro visualizations. My current work is focused on wearable displays: Data displays that can be worn on a person and thus do not need to be
<figure><img src="image_20.png" /><figcaption>Figure 2.1: *Data Display Mobility* types ordered roughly from least to most mobile.</figcaption></figure>

actively carried. The fact that visualizations can be seen while on-the-
go increases mediated complexity for micro visualization design as
visualizations need to take into account relative movement between
viewer and visualization.

DATA SOURCE: For mobile visualizations it often makes sense to characterize the source of the data being visualized. We divided this dimensions into four categories: pre-loaded, connected, and captured data, as well as in combination (Figure 2.2).

<figure><img src="image_21.png" /><figcaption>Figure 2.2: Categories in the *Data Source* dimension.</figcaption></figure>

The fact that smartwatches and fitness trackers capture much of the data they show increases the internal complexity of visualization tools that now need to preprocess the data. In addition, smartwatches often show dashboards that combine data captured from the device (battery life) and data from external servers (weather) as shown in Figure 2.2d.

REACTION OF VISUALIZATION TO DISPLAY MOVEMENT: Another
distinctive aspect of mobile data visualization is if and how a visualiza-
tion changes due to the movement of the display in the environment.

We identified four broad categories for this dimension: no change,
indirect change, direct change, and direct+indirect (Figure 2.3). Visual-
izations that react indirectly to viewer movement show and update
data that is *affected* by the movement, such as heart rate and EEG
<figure><img src="image_22.png" /><figcaption>Figure 2.3: Categories in the *Reaction of Visualization to Display Movement* dimension. Figure 2.3a) reprinted with permission from Ramik Sadana.</figcaption></figure>

signals. Figure 2.3b shows a fitness app's heart rate visualization, in which the heart rate increases because the owner is exercising. Visualizations that directly change due to viewer movement show movement-related data such as step counts, velocity, or position and location. The in-seat display in Figure 2.3c shows a plane's location relative to the Earth. Smartwatch faces as in Figure 2.3d are a common type of data display dashboard that includes both direct (e. g., step counts) and indirect (e. g., heart rate) visualizations reacting to movement. How a micro visualization needs to react to movement has an influence on the internal complexity of the visualization design.

**INTENDED VIEWING TIMESPAN:** Another perspective of mobile data visualization opens up when considering the time available for viewing a visual representation. The *Intended Viewing Timespan* is highly related to the context of use and affects mediated complexity. An intended viewing timespan of a few hundred milliseconds often arises in situations when people's attention is directed elsewhere and they can only briefly take their eyes away from their primary task. GPS devices in moving vehicles belong to this category. Contexts with relatively simple information needs such as seeing today's or tomorrow's weather require visualizations to be read within a few seconds. Mobile micro visualizations often have intended viewing timespans of (sub-) seconds, but in small-multiple settings or embedded in larger analysis environments could also be used in contexts with more complex information needs.

**INTERACTION COMPLEXITY:** Compared to traditional data visualization, mobile data visualization needs to support usage scenarios where people are on-the-go or are engaged in other activities. Analyzing a data visualization may not always be a person's primary task, but rather an auxiliary step to enhance or support other mobile tasks. We categorize four levels of visualization interaction complexity ranging from passive interaction to highly interactive as shown in
**Figure 2.4.** Brehmer et al. [30] give a broader overview of mobile data visualization interaction.

<figure><img src="image_23.png" /><figcaption>Figure 2.4: Four categories in the *Visualization Interaction Complexity* dimension.</figcaption></figure>

**SUMMARY** Overall, we presented several dimensions each with four to six categories according to which mobile data visualizations can be organized. Along the described dimensions, we can mark ranges that are typical for mobile wearable (smartwatch and fitness tracker) data visualization. On these devices we typically find small, mobile displays with visualizations that show data that are fetched from the cloud or captured via sensors. Visualizations often have interactions of low or moderate complexity and also react to the display movement. A mobile wearable visualization is typically in use only for shorter time spans.

### 2.3.2 Mobile (Micro) Visualization Examples

A crisp definition of mobile data visualization is difficult to articulate, as computing devices converge and propagate to more and more different types of activity. The academic literature and web collections such as MobileVis [177] and Mobile Infovis [181] showcase various mobile data visualizations developed by researchers, practitioners, and technology companies. In this section, I show only a few example that relate to my work on micro visualizations.

Research on mobile data visualization began in the nineties [71, 123]. The term mobile computing emerged to describe people's interactions with computing devices that are wirelessly connected, able to exchange information, and portable. With the wide availability of personal digital assistants (PDAs), the topic of mobile data visualization also began to gain momentum [48]. The key research challenges were primarily focused on technical issues due to limited computing power and memory, and for visualization also the limited display capabilities.
Display resolution was low (240 × 320 pixels) and only a few or no
colors were available on early PDAs. Therefore, a visualization had
to be designed carefully to avoid wasting precious pixels, a challenge
shared by micro visualizations more generally. At the same time, the
implementation had to be efficient: any overhead by run-time libraries
or interpreted languages had to be avoided to keep the visualization
reactive and to reduce battery drain. Similar challenges still exist for
small-screen devices such as smartwatches and fitnessbands that are
severely restricted in battery life and computing resources [158] which
is partially why they are usually coupled with a smartphone.

The plethora of data being available on smartwatches is a great
opportunity for visualization. Smartwatches offer various faces and
widgets for people to customize what data to show and how to show
it. Recently, we have seen some research efforts in understanding
how data and representations are currently displayed on smartwatch
faces [113] (see Section 5.2) and how people perceive small-scale visu-
alizations on a smartwatch [15, 95, 159]. Others’ visualization research
described novel visualization designs specifically for smartwatches.
Examples include research on representing health and fitness data on
smartwatches [3], for line charts [159], temporal data [198], activity
tracking more broadly [96], and even for integrating visualizations in
watch straps [124].

2.4 STUDIES ON VISUALIZATION SIZE

It is sensible to assume that making visualizations increasingly small
will impact their readability, similar to what has been observed for
decreasing font sizes [174].

Only few studies compared physical display sizes for small data
representations [36, 104, 107, 118, 169]. Healey and Sawant [104] con-
ducted perceptual experiments at very small visual angles (0.061 25°,
0.1225°, and 0.245°). The authors placed 40 small lines on a jittered
regular grid and varied hue, size, luminance, or orientation of some el-
ements. Participants had to judge if a region with a different encoding
was present or absent. The authors found a decrease in response time
and error with increasing visual angle. Response time and error were
lowest for size encodings and highest for orientation (response time)
and hue (error) encodings. At the lowest visual angle accuracy for
size encoding was only 40% but increased to 99% at the largest visual
angle. Lyons [145] investigated the size of a circle on a smartwatch as a
notification indicator and found that larger circles led to faster reaction
times. Some researchers assessed more complex representations. Heer,
Kong, and Agrawala [107], for example, provide a first indication
that studying micro visualizations under different size and viewing
conditions is important, because it may lead to unexpected results
compared to what we know about larger-sized data representations.
The authors compared small filled line charts against two types of Horizon Graphs and investigated the impact of chart height on both designs. The authors found that small chart heights negatively affected accuracy and speed of data comparison and that smaller size had a greater impact on the filled line charts than on the Horizon Graphs. This is surprising, because line charts are a familiar technique that has been previously shown to be quickly and accurately readable. The authors conclude with the recommendation to draw line charts at 24 pixels (6.8mm) height and Horizon Graphs at 6 or 12 pixels (1.7 / 2.4 mm) for desktop monitor viewing. Cai et al. [36] tested simple donut charts at visual angles of approximately 3.3°–9° (viewing distance estimated from picture of study setup) and found no evidence of an effect of size on the accuracy or speed of proportion estimation tasks. In their work on SportLines [169] tested several design alternatives under different display sizes (ranging from 80×60–20×15px). The authors did not control for the participants’ screens; therefore, the results can only be compared relative to one another, because different DPIs and pixel sizes result in different physical display sizes. The authors asked the participants to rank representations and the smallest designs were always the least preferred. The authors did no assessment of reading comprehension or effectiveness.

## 2.5 SUMMARY

Research on data glyphs has the longest history of the three types of micro visualizations I investigated here. Many studies on glyphs have been published but still rare are studies that investigate glyph size. More frequent have been studies related to external complexities where the results were so far mostly consistent: showing more data dimensions or more data points decreased participant performance, although encoding type has been shown to play a role in the extent of the negative impact. Less frequent were results related to mediated complexities such as the background on which glyphs are displayed. Yet, this is an interesting research space, for example for micro visualizations in video games or augmented sports analytics.

Research on word-scale visualizations has been relatively active in recent years but only from a small number of research groups. We have seen interesting applications and designs as well as some work on interaction but still lack more studies on how word-scale visualizations are actually read when embedded in text and how or if they affect the readability of the text.

Mobile visualization research has begun about 30 years ago. In this time and with the advent of new mobile devices some research challenges are still important: finding efficient space-saving data encodings that consider the often limited computing power, memory, and connectivity of mobile devices. Research on the smallest devices—
smartwatches, fitness trackers, or even smaller—is still sparse and we
still know little especially about issues of mediated complexity that
come with the wearable aspect of these devices and the associated
usage scenarios.

In the following chapters I will present condensed versions of the
past research on these three types of micro visualizations I have been
involved with, before discussing open challenges in more detail in the
final chapter.
Most of my work on data glyphs focused on studying perception of glyphs for tasks in small multiple contexts, and in particular using grid structure to control for distance between target and distractors. The two studies reported here also tested the effect of the number of dimensions, as a factor of external complexity, on the effectiveness of participants.

## 3.1 LOW AND HIGH-DENSITY TIME SERIES GLYPHS IN A SMALL MULTIPLE SETTING

One of my first co-authored publications on data glyphs as micro visualizations was presented at the ACM CHI 2013 conference and received an honorable mention award [80]. This work was led by Johannes Fuchs in collaboration with Fabian Fischer, Florian Mansmann, Enrico Bertini, and myself. In this section, I use the pronouns "we/our" to refer to this set of co-authors.

The goal of our work on temporal glyph designs was to ultimately understand how different types of small data encodings for time series affect people's performance in specific tasks. Through a controlled experiment, we compared the performance of four glyphs that used different visual channels to encode the data, a radial vs. linear arrangement, and data density while keeping display size constant at approximately 2.9° of visual angle. In addition, we varied the viewer's purpose by testing three different tasks. Our results showed that depending on tasks and data density, the chosen glyphs performed differently. Line Glyphs [9] were generally a good choice for peak and trend detection tasks but radial encodings [6] were more effective for reading values at specific temporal locations.

### 3.1.1 Motivation

Time series data is the basis for decision making in many different application domains—such as finance, network security, or traffic management—and, thus, constitutes an important area of research for visualization and data analysis. In these settings, temporal glyphs can represent data compactly in a small multiple setting and enable quick visual comparison. Different visual variables such as length, color, or position can be used to encode two aspects of a data point dimension in one glyph: a) the location of a data value in time, and
b) the quantity of the data value. The visual encoding type and the number of data values in time are objective features of a data glyph that a designer can vary and in particular the number of data values in time are determinants of the real external complexity of a micro visualization (see Chapter 1). While one could follow the ranking of Cleveland and McGill [54] and try to predict the performance of glyphs based on their ranking of elementary perceptual tasks, it is not clear whether their results will hold. Temporal glyphs include two encodings, are used in specific temporal analysis tasks, and their size and density might influence perception.

### 3.1.2 Study

In order to learn about the effectiveness of different encodings we designed a study in which we controlled the following experimental factors: *glyph*, *task*, and *data density*.

#### 3.1.2.1 Glyphs

We chose four different glyphs based on their characteristics. We chose the Line Glyph 📉 because of its effective position-based encoding and because it is commonly used, the Stripe Glyph 📃 for its similar temporal encoding (position) but different value encoding (color saturation). Glyphs are often designed to encode intuitive pairings of data to visual variables [212] and, thus, we chose two circular designs that take people's potentially intuitive notion of time encoded in a clock-like fashion into account. We chose to test the Star Glyph ☀️ for its similar value encoding to the Line Glyph 📉 and the Clock Glyph 📉 for its similar value encoding to the Stripe Glyph 📃. We chose a square aspect ratio for each glyph to achieve similar display sizes in the small multiple setting.

For the color encoded glyphs (the Clock Glyph ☀️ and the Stripe Glyph 📃) we chose a heatmap colorscale, which was motivated by the yellow to red colorscale from ColorBrewer [31]. This scale takes advantage of the fact that the human visual system has maximum sensitivity to luminance changes for the orange-yellow hue [138] and it is also suitable for color blind people.

For each trial, we drew the same type of glyph—but showing different data—on the screen in a small multiple layout of 8 × 6 = 48 glyphs in total (Figure 3.1). Each glyph was drawn at a resolution of 96 × 96 px on a 24 in screen set to a resolution of 1920 × 1200 px and viewed from a distance of approx. 50 cm. This put the glyphs at approx. 2.9° of visual angle, so slightly larger than the 1–2° of visual angle discussed for monitor viewing in Chapter 1 but still within the 5° discussed by Ware [213].

<figure><img src="image_24.png" /><figcaption>Tested data encodings for temporal glyphs at the physical size displayed in the experiment (25.8 mm) when printed without scaling.</figcaption></figure>
<figure><img src="image_25.png" /><figcaption>Figure 3.1: **Peak detection:** Illustration of the different glyphs with one high data value at a random point in time. For a better understanding the correct glyph is artificially highlighted here.</figcaption></figure>

### 3.1.2.2 Tasks

We chose our tasks taking two criteria into account: (1) their ecological validity, i. e. whether they are performed in environments where the quick comparison of multiple time series is needed. (2) their heterogeneity in terms of the elementary perceptual tasks, i. e. we picked tasks that involve the comparison of visual variables for encoding data values, investigating different layouts for time and the combination of the two. In terms of ecological validity our tasks were inspired by my collaborators' work with network security analysts from a large university computer center who had to monitor large amounts of network devices. The analysts had to be able to efficiently detect anomalous traffic patterns (e. g., peak values in none working hours) to be able to quickly react on a possible threat. Our three tasks were:

**Task 1—Peak Detection:** Amongst all small multiple glyphs, participants had to select the glyph that contained the highest data value (Figure 3.1). This task, thus, involved scanning each glyph for its highest value and comparing across glyphs using length (for the Line Glyph 📉 and Star Glyph ⚙️) or saturation (for the Stripe Glyph 📃 and Clock Glyph 🕒) judgements.

**Task 2—Temporal Location:** Among all glyphs, participants were asked to select the glyph with the highest value at a predefined time-point. This time-point was shown as text to the participant in advance (e. g., "3am"). This task, thus, involved first identifying the location of
<figure><img src="image_26.png" /><figcaption>Figure 3.2: Differences between the two data densities *low* (24 values, top row) and *high* (96 data values, bottom row) for each glyph design.</figcaption></figure>

a time-point by making positional (for the Line Glyph 🐘 and Stripe Glyph 🐘) or angular judgements (for the Star Glyph 🐘 and Clock Glyph 🐘) and then comparing the peaks as in Task 1.

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

**Task 3—Trend Detection:** Among all small multiples, participants had to select the glyph with the highest value decrease over the whole displayed time period. This task, thus, involved first detecting all decreasing trends and then comparing the first and the last value.

### 3.1.2.3 *Data Density*

In order to test the scalability of each glyph in terms of the number of data values it can encode, we tested two data densities. The smaller density consisted of 24 data values (1 for each hour), and the larger of 96 data values (1 for each 15 minutes). The rendered size of the glyphs holding these data points was not varied between each density (Figure 3.2).

### 3.1.3 *Result Summary*

For details on data generation, experiment procedure, and further analysis details I refer to the original publication [80]. Here, I focus on a discussion of the results. Figure 3.3 shows the mean and standard deviations for each task and density while Figure 3.4 highlights the statistically significant differences in time and accuracy based on a repeated-measures ANOVA for time and a Friedman's test for accuracy.

In the peak detection task, nearly no mistakes were made with the Line Glyph 🐘 and only few with the Stripe Glyph 🐘. Apparently, participants had most problems reading value with the circular layout that used a length encoding as the Star Glyph 🐘 was the least accurate glyph. In the post-session interview several participants argued that they had problems comparing lengths with different orientation. Comparing position/length in a radial design perhaps involves mental rotation to transfer the overall design to a comparable linear layout.
<figure><img src="image_29.png" /><figcaption>Figure 3.3: Bar charts with mean and standard deviation showing the results for each task and factor. The x-axis represents the different dependent variables. The y-axis illustrates the different tasks.</figcaption></figure>

This is not needed for color comparison, since color does not need an identical baseline and the Clock Glyph 🕒 performed better than the Star Glyph 👋 in both densities. We observed another interesting effect between the colored glyphs: while accuracy was not significantly different for low data density, the Clock Glyph 🕒 outperformed the Stripe Glyph 🕒 at high data density. Perhaps the additional pixels towards the outer rings of the circle led to better color discrimination, wheras the stripes got too small, making the comparison more difficult.

The trend detection task fundamentally required value judgements, similar to the peak detection task. Again, the Line Glyph 🕒 had highest accuracy independent of the data density. With an increased data density the accuracy of the Line Glyph 👋, the Clock Glyph 🕒 and the Stripe Glyph 🕒 dropped significantly. The completion time remained stable with no changes between the two density conditions.

Unlike for the other two tasks, the temporal location task required finding a location along the x-axis or the circumference and then reading and comparing the value at this location. In terms of accuracy both polar designs outperformed the linear designs when data density was low. To find an explanation for this result, we looked at the selections made by our participants and discovered an interesting side effect. The data sets corresponding to these wrongly answered questions were enriched with distractors very similar to the correct data instances by showing the same high value but at a different point in time. Participants seemed less likely to select such distractors when
<figure><img src="image_30.png" /><figcaption>Figure 3.4: Summary of the statistical significant differences in accuracy and answer time for each task and density condition. Thick arrows indicate a difference in both accuracy and time and thinner arrows with an “a” label indicate a statistically significant difference in accuracy only. Arrow direction stands for “significantly better than.”</figcaption></figure>

using the circular layouts for the time dimension. When data density was high we observed the same trend, even though only the Star Glyph showed significant differences with respect to the Stripe Glyph and the Line Glyph. The good performance of the Star Glyph can be explained with the combination of the encodings. The length encoding for the data values makes it possible to easily spot the highest value even with lots of datapoints. With the color encodings, participants had problems spotting the peak value. The circular layout performed better than the linear one and worked for estimating the correct point in time.

### 3.1.4 *Design Considerations for Micro Visualization*

With the results gained from the analysis and discussions we derive a number of design considerations relevant to consider for micro visualizations.

* For quantitative value encoding, position/length encodings should be preferred to a color encoding:

As can be seen in the results gained from the peak and trend detection tasks, where a value detection and comparison was necessary, the Line Glyph performed best. Even with an increased data density values could still be compared.
* For color encodings of quantitative values, circular designs may be better than rectangular shapes:

The slices used in the Clock Glyph 🕒 for encoding single data values form a triangular shape because of the circular layout. The additional space given to the slices might have aided value reading in several conditions.

* Color encodings for higher data densities should be used with caution: The results from the Peak and Trend Detection task illustrate that the performance of the color encoded designs (Clock Glyph 🕒 and Stripe Glyph 🕒) depended on the data density. Having a higher data density led to a decreased performance.

Polar designs were better for detecting specific points in time. This guideline results from the analysis of the Temporal Location task. Participants performed significantly better using the Clock Glyph 🕒 and Star Glyph 🕒 compared to the Line Glyph 🕒 and Stripe Glyph 🕒. The clock metaphor seemed to increase participants' chronological orientation.
<figure><img src="image_31.png" /><figcaption>Figure 3.5: Three categories of star glyph variations used in our three experiments. *Data lines only* (D): only the data lines encode the data; *Data lines+Contour* (D + C): data lines are connected at the endpoints to create a closed shape; *Contour only* (C): only the contour line is drawn. Additional variations are tickmarks (*T*) (in b, e)), gridlines (*G*) (in c, f)), and fill style (in g, h)).</figcaption></figure>

## 3.2 THE INFLUENCE OF CONTOUR ON SIMILARITY PERCEPTION OF STAR GLYPHS

Our follow-up work on the previous study was published at IEEE InfoVis 2014 [81]. This work was led by Johannes Fuchs in collaboration with myself, Anastasia Bezerianos, Fabian Fischer, and Enrico Bertini. In this section, I use the pronouns “we/our” to refer to this set of co-authors.

While the previous work tested the influence of different types of data encodings, our follow-up work concentrated on variations of a single glyph type, the star glyph as seen in Figure 3.5. We hypothesized that for data comparisons in small multiple settings, the overall shape of a star glyph—enhanced through contour lines—would aid the viewer in “macro readings” and specifically in making accurate similarity judgments. To test this hypothesis, we conducted three experiments. In our first experiment, we explored how the use of contours influenced how visualization experts and trained novices chose glyphs with similar data values. Our results showed that glyphs without contours made the detection of data similarity easier. Given these results, we conducted a second study to understand intuitive notions of similarity. Star glyphs without contours most intuitively supported the detection of glyphs that represented similar data. In a third experiment, we tested the effect of star glyph reference structures (i. e., tickmarks and gridlines) on the detection of similarity. Surprisingly, our results showed that adding reference structures did improve the correctness of similarity judgments for star glyphs with contours, but not for the standard star glyph. As a result of these experiments, we concluded that the simple star glyph without contours performed best under several criteria, reinforcing its practice and popularity in the literature. Contours seemed to enhance the detection of other types of similarity, e. g., shape similarity and are distracting when data similarity has to be judged. As in our previous experiment we included the number of dimensions as a study factor and the physical display size of each glyph was approximately the same at 2.9° of visual angle.
### 3.2.1 Motivation

There exists a great variety of alternative designs for star glyphs that differ in the amount of reference structures used, the use of additional visual variables on the “rays,” or whether or not the individual rays are connected to form a contour for the glyph [211]. The version of the star glyph with unconnected rays is also sometimes called *whisker or fan plot*, while the connected version also carries the name *star plot* [213]. Star glyphs are frequently used but very little advice exists on how to choose between different star glyph encodings. The question arises to what degree changes in the design of a star glyph influence its perception and, thus, the effectiveness of the glyph in certain tasks.

One important task for glyphs in small-multiple settings is the comparison of the encoded data points to one-another. Such a comparison task may be conducted to find data points that are very close over all dimensions, very different, or similar in just a subset of dimensions. We focus on the first task: finding data points encoded as star glyphs that are very similar to a target glyph. We hypothesized that the ability to perceive a star glyph as a coherent and closed shape would strongly influence the correctness of data similarity detection tasks—as it would potentially be easier to compare a single shape than having to compare individual rays. This hypothesis was motivated by prior research showing that a closed contour has an influence on the perception of a coherent shape [68]. As Palmer noted: *“Shape allows a perceiver to predict more facts about an object than any other property”* [164].

A wide variety of approaches have been taken to study glyph similarity and these vary in methodology and the factors of the glyph designs. A large body of literature also exists on the effect of contours on shape perception. A few [68, 85, 179] have particularly inspired our hypotheses that the presence of contours may be of particular importance to the effective use of glyphs in visual data analysis tasks. The study by Klippel, Hardisty, and Weaver [126] is probably the most related to our work as it also looked at the influence of shape on glyph perception based on similarity judgments. Yet, instead of the influence of contour, as in our case, they varied shape by reordering the dimensions in a star glyph with contour. The authors studied how shape changes influenced the interpretation of data points in a similarity-based grouping task. They found that differences in shape influenced cognitive processing of the data and that perceptually salient features (such as spikes) strongly influenced how people thought about a data point. We share the question of how design factors influence the perception of *data similarity* vs. shape similarity. Data similarity judgments are cognitive tasks, where the viewer has to judge the absolute difference in data values across all dimensions of two data points. This differs from other types of similarity judgments, such as detecting shape similarity e. g., under rotation or scale.
<figure><img src="image_32.png" /><figcaption>Figure 3.6: **Experiment 1 Setting:** The participant was seated in front of a 24" Screen with a resolution of 1920x1200. The only input device was a computer mouse.</figcaption></figure>

### 3.2.2 Study 1: Contours For Novices vs. Experts

In our first study we were interested in the fundamental question: does contour affect people's perception of data similarity with star glyphs? We used three different variations of the star glyph shown in the margin. To investigate the effect of contours on different data densities we varied the number of dimensions shown in the glyphs. The low dimension density consisted of three data dimensions with corresponding data values, while the high density consisted of ten data dimensions. We considered ten dimensions to be high, as glyphs used in the literature rarely visualize more than ten dimensions.

#### 3.2.2.1 Study Setup and Procedure

Participants were seated (Figure 3.6) and shown a highlighted stimulus glyph surrounded by 8 more glyphs in a 3 × 3 matrix configuration (Figure 3.7). The participant had to select the glyph closest to the stimulus in terms of data value. Each participant repeated 4 training and 4 real trials for each contour variation. Twelve novices (7 female) and twelve experts (2 female) participated in our study. For further details on the study design, participants, and analysis methods, I refer to the original paper [81].

#### 3.2.2.2 Result Summary and Discussion

Our experts were not significantly more correct than novices on average. This is especially true for the low dimensional condition where both user groups had a good performance (≈ 80% correct). However, for higher dimensionalities experts using variation *D* were significantly more accurate compared to novices. When comparing the two

<figure><img src="image_33.png" /><figcaption>Tested star glyph variations in Experiment 1.</figcaption></figure>
<figure><img src="image_34.png" /><figcaption>Figure 3.7: **Experiment 1 Setting:** For each trial glyphs were arranged in a 3×3 matrix. The stimulus is highlighted and positioned in the middle to assure an equal distance to the other glyphs. This setting is used in all three experiments.</figcaption></figure>

dimensionalities, similarity judgments were significantly more accurate for both user groups in the low dimensional condition compared to higher dimensionalities, providing further evidence that dimensionality is a good measure for external complexity when the number of marks increases in the same display space. Contrary to intuition from previous work that contour can improve similarity judgments [68, 85], we found that contour affected the accuracy of judgments negatively for experts. No significant effects were found for novice participants. Mean accuracy for C (50%) was lower compared to D + C (59.4%) and D (57.3%). Contrary to our expectations, the variation without a contour (D) led to significantly more correct answers for high-dimensional glyphs. The effect was not visible in the low dimensionality case.

Trying to explain the unexpected negative effect of contour on experts, especially in high dimensional cases, we noted that at least half of the erroneous answers in the contour variations (D + C, C) were in the form of scaled versions of the stimulus glyph, and to a lesser extent rotated versions, i. e. glyphs that have a geometric form similar to the stimulus glyph. In retrospect, this negative effect of contour could be explained by the fact that contour, and closure in general, is one of the factors promoting the notion of unity according to Gestalt psychology [128]. In our case, contours led our experts to erroneously consider glyphs as coherent shapes when judging similarity, rather than data values along specific dimensions. This resulted in judgments and comparison of geometrical shapes rather than data, with experts being led to consider data points as more similar that were either scaled or rotated versions of the stimulus, rather than the one closest in data space.

Given the overall poor performance of novices in the high dimensional case we conjecture that due to their lack of familiarity and experience they tended to fall back to judging shape rather than data
<figure><img src="image_35.png" /><figcaption>Figure 3.8: **Experiment 2 design space:** We enriched the design space from our previous study by adding a “fill” version of the star glyph. The design variations of the first study (i. e., *D*, *D* + *C*, *C*) are applied to both Fill and No-Fill.</figcaption></figure>

similarity for all star glyph variations. This is evidenced by the fact
that at least half of their errors were a combination of scaled and
rotated versions of the stimulus glyph.

3.2.3 *Study 2: Perception of Similarity*

Results from Study 1 indicated that in high dimensional cases contours
mislead even experts to perceive rotated or scaled versions of the
stimulus as more similar, rather than the one closest in data space.
Based on this finding, we conducted a second experiment to better
understand what type of similarity star glyphs naturally support. To
this end, participants were not given any training or explanation of
what similarity means, and we did not inform them that the glyphs
encoded multi-dimensional data. Their only instruction was to select
the most similar glyph. Our goal in this experiment was to examine
what viewers naturally perceive as similar in different star glyph
variations, without being instructed on how to judge similarity. Based
on our results we hoped to identify the star glyph variations, if any,
that naturally promote data similarity rather than shape similarity
and, therefore, are more suitable for data visualization.

3.2.3.1 *Design and Procedure*

The experiment tested the glyph variations from Experiment 1, as well
as a filled version of the C and D + C glyph. We wanted to examine
whether variations of glyphs that are filled reinforce more strongly the
notion of a closed shape, due to the strong foreground/background
contrast [128]. We conjectured that fill color may lead to more shape
rather than data similarity choices.
The experiment was a between-subjects design with fill type as the between-subjects factor. Thus, the *D* glyph was included in each group as the baseline. We had a total of 2 fill types (Fill, No-Fill) with 3 glyph variations each, as illustrated in Figure 3.8. To complete a trial, participants selected the most similar glyph compared to a stimulus glyph. Participants were shown a highlighted stimulus surrounded by another 8 glyphs in a 3×3 matrix configuration.

Our study was conducted on Amazon Mechanical Turk. After removing participants who failed control questions we ended up with 36 participants (18 per fill type). Each participant worked on 4 trials for each variation and dimensionality, and viewed either the fill or the no-fill types. The order of presenting the glyph variations was randomized. For further details on the study design, participants, and analysis methods, I refer to the original paper [81].

### 3.2.3.2 Result Summary and Discussion

Independent of the fill type, participants using the *D* ✠ glyph variation selected the data target as more similar significantly more often than any other type, giving strong evidence that glyphs without contours promote data similarity comparison rather than shape. On the other hand, the most selected targets in contour variations *D* + *C* 🟧, *C* 🟥 were indeed either rotated or scaled variations of the stimulus. This reinforces our findings from the first study, that factors enforcing perceptual unity of shape [128], such as contour containment led viewers to naturally make shape judgments of similarity rather than data, while open variations of the glyphs led to similarity choices closer to data comparisons, even without being told what similar means. The above effects are due mainly to the high dimensional condition. In the low dimensional condition, across all glyph designs, data targets were the ones more often select than all other target types.

When comparing filling types we could not prove that filled star glyphs promote shape judgments more strongly than no-fill star glyphs. There was also no difference in the selection of data targets across fill type. In our experiment the stronger figure and ground distinction, that in the past has been shown to promote unity of shape [128] did not have a noticeable effect in data selections.

We note again that in this study participants were never told that they were viewing data visualizations, they were just asked to find the most similar glyphs without further instructions. Thus, our results indicate the natural tendency of people to judge glyphs instinctively in a more “data-centric” manner in low dimensionalities, and in high ones when factors that enforce coherent shapes are absent. It is clear that with training we can further enforce data similarity judgments– but given that some glyphs and glyph variations seem to be naturally well suited for data judgments, we focus on those designs and try to further improve their performance with small design variations.
<figure><img src="image_36.png" /><figcaption>Figure 3.9: **Experiment 3 design space:** We have chosen the star glyph only with data whiskers (D) and with an additional contour line (D + C) and applied tickmarks (T) and gridlines (G) to these designs.</figcaption></figure>

### 3.2.4 Study 3: Improvements for Star Glyphs

The first experiment showed that people judge data similarity with non-contour designs more accurately while the second experiment showed that non-contour designs also lead to data similarity judgments to be made more naturally. Yet, accuracy in the high-dimensional case was quite low for all main design variations we tested previously. In this last experiment, we thus explore whether we can improve the accuracy of data similarity judgments by adding simple reference structures to the designs. Reference structures such as grids and tickmarks are frequently recommended for data charts to aid in relating content to axes [129]. In small scale visualizations, reference structures can quickly increase the visual complexity and designers have to consider a tradeoff between adding more “ink” [204] and potentially aiding the completion of certain tasks. In this experiment we focused on tickmarks and grids that use two different types of reference mechanisms. While tickmarks add information to each individual data line only, grids connect the overall glyph design. While there are many different ways to draw grids and tickmarks we settled on the designs shown in Figure 3.9. Of course, the readability of glyphs could further be improved by adding double encodings (e. g., additionally using color to distinguish dimensions or data values), dimension ordering [167], or sorting the glyphs on the display. Yet, all of these encodings have limitations: use of color is limited to glyphs with a small number of dimensions, dimension ordering may not improve legibility for a large number of variable glyphs in a small-multiple setting, and sorting glyphs may disrupt a pre-defined layout based on other meta-data such as time. We, thus, focused on reference structures in this study.
### 3.2.4.1 *Design and Procedure*

We tested the two star glyph variations that performed best in the first experiments: the data-only glyph ($D$) and the star glyph with data lines and a contour line ($D+C$). The reason for discarding the contour only design ($C$) is the bad performance for previous similarity judgments, the lack of ability to place tickmarks, and the minimal number of real-world examples of this glyph type in use. For baseline comparisons we kept the originally tested versions of the star glyph ($D, D+C$) and added two types of reference structures ($T, G$). The experiment, thus, compared the six different designs ($D, D+T, D+G, D+C, D+C+T, D+C+G$) in Figure 3.9.

We recruited 12 data visualization experts (3 female). Participants completed data similarity search trials with all 6 designs. Since participants were already ≈ 80% correct in the low dimensional condition in Experiment 1, we only used high-dimensional glyphs in Experiment 3. For more details on study results and analysis, I refer to the original paper [81].

### 3.2.4.2 *Result Summary and Discussion*

Adding reference structures to the star glyph did not have the effect on accuracy we were expecting for our data similarity search task. Additional anchor points on the data line (i. e., tickmarks) did not significantly improve the comparison of data points. Nevertheless, there was a statistical trend indicating that an overall reference in the background (i. e., gridlines) may increase accuracy, especially in the case of contour star glyphs. This lack of strong significant effects is surprising, especially given that most participants mentioned in a post-questionnaire that for the simple star glyph $D$, gridlines (81%), and to a lesser extent tickmarks (72%), helped them find the most similar data point. Although the mean accuracy for the $D+G$ variation (68.8%) was indeed higher, the effect was not significant, perhaps due to the already very good performance of the $D$ variation (60.4%). The value of gridlines and tickmarks in general may warrant further research. As Few notes [73], gridlines may be useful only in specific cases, e. g., when small differences have to be compared. Therefore, it is possible that for other tasks, such as direct lookup, these additional reference marks could help more strongly.

For the star glyph with contour ($D+C$), only 54% of our participants reported using tickmarks and 36% gridlines to complete the task. From their reports they felt (erroneously) that glyphs with contours are easier to compare and, thus, did not make conscious use of the additional improvements. Thus, in the contour case, participants were not only more error prone, but also misled to feeling confident in their choices, ignoring the marks that could help them improve their performance. Nevertheless, it is highly likely that the addition of read-
ing marks was taken into account, even if unintentionally, explaining
the trend we see for both the tickmark and grid variation to be more
accurate than simple contour glyphs.

Even though participants using variation (D) performed very well,
it is interesting that they did not like this design variation. On a 7-step
Likert scale 63% of the participants rated the design with either 6
(difficult to use) or 7 (very difficult to use). Many participants (46%)
preferred the star glyph with contour and gridlines, with only 1
participant rating it with a 5 (slightly difficult to use) and the others
with 3 or better.

Given the results of this experiment the benefit of using reference
structures for star glyphs is limited. Especially since in real world
scenarios when multi-dimensional glyphs are projected to two dimen-
sional surfaces, there is the possibility of over-plotting, and adding
marks or gridlines could worsen this effect due to the additional ink
introduced.

3.2.5 *Design Considerations*

With the results gained from the analysis and discussions we derive
the following design considerations.

*When judging data similarity avoid contours in glyph designs.* Viewers have a natural tendency to judge data similarity in star glyphs without contours. In all our experiments viewers were tricked into making shape-based, rather than data-based judgments when using contours. This is especially true if glyphs in the visualization are scaled or rotated versions of each other.

*For low number of dimensions (around 4) any glyph variation can safely be used for data similarity judgments.* In the first and second experiment viewers naturally leaned towards data similarity for each glyph variation in low dimensions, even without training.

*When clutter is an issue avoid reference structures in non-contour star glyphs for similarity search tasks.* Results of Experiment 3 illustrate that even though participants preferred using tickmarks or grids they did not perform significantly better with them, especially for glyphs without contours. Nevertheless, there is a statistical trend that shows that tickmarks and grids improve glyphs with contours.

*If references are required use grids rather than tickmarks.* Independent from the design (i. e., with or without contour) gridlines always increased mean accuracy, which is not true for tickmarks.
3.3 WHAT DID WE LEARN ABOUT MICRO VISUALIZATIONS?

The past research on data glyphs I collaborated in targeted specific
questions about the encoding of multi-dimensional data points as
glyphs. What unites both studies was the focus on tasks across a set
of multiple glyphs. The first study targeted a specific type of data and
related tasks: temporal quantitative data and peak detection, temporal
locations, and trend detection tasks. The second study, in contrast,
covered only one glyph type and a general similarity judgment tasks.
We saw in both studies that, perhaps not surprisingly, increasing the
number of dimensions decreased participants’ performance. Design
variations in both studies led to different performance outcomes and
we were able to make recommendations for the tested encodings,
data, and tasks. It would be interesting to test the effects of increasing
dimensionality further. Theoretically, we might predict that individual
data values in an encoding are readable up to a certain level of visual
acuity and limits of display resolution [104] but being able to predict
these limits would be useful further work, especially taking other
types of mediated complexity into account such as backgrounds and
other types of glyph layouts.

It would also be interesting to test the influence of shrinking glyphs
further. None of the multi-dimensional glyphs we tested, for example,
were tested in environments that require to shrink them even further
such as on smartwatches or in embedded text.
My joint work on word-scale visualization was led by my PhD student Pascal Goffin and supervised by myself, Jean-Daniel Fekete, and post-doc Wesley Willett. We worked primarily on issues related to text as a factor of mediated complexity where we looked into how to best place these visualizations inside text, how to establish connections between text and visualization, and how to interact with micro visualizations placed inside text. In our work on a design space for word-scale visualizations, we also touched on factors regarding external complexity. We also published a few smaller workshop papers regarding the influence of word-scale visualizations on text readability [91] and a word-scale visualization application for understanding personal notes [94] that I do not cover here in detail.

## 4.1 WORD-SCALE VISUALIZATION PLACEMENT AND DESIGN

Work reported in this section was published as "Exploring Sparkline Placement and Design for In-Text Visualization" co-authored by Pascal Goffin, Wesley Willett, Jean-Daniel Fekete and myself at VIS 2014 [92]. In this section, I use the pronouns "we/our" to refer to this set of co-authors.

In our first work on word-scale visualizations we presented an exploration and a design space that characterized the usage and placement of word-scale visualization within text documents. One particularly important aspect of our discussion on placement options was the notion of a *textual entity*. We defined an entity as a word or phrase to which the word-scale visualization is directly related. In most prior work, word-scale visualizations have typically simply been placed before or after the word they are related to. This is not always possible—for example, when annotating scanned documents and other immutable texts. Furthermore, placing word-scale visualization in-line with text may not be the most desirable solution as it requires text reflowing and binds the visualization's height roughly to font height. A more complete characterization of word-scale visualization placement options, was our goal to help designers choose among a range of design alternatives. Figure 4.1 introduces terminology used in our work on word-scale visualization placement.
<figure><img src="image_37.png" /><figcaption>Figure 4.1: Important terminology to describe placement of word-scale visualization in the context of entity and surrounding text.</figcaption></figure>

### 4.1.1 *Design Space*

The dimensions of our design space described both impacts on the word-scale visualization design and text layout. Our goal was to provide a design-centered view on word-scale visualization placement that includes considerations about how much one wants to or can disrupt the text and whether or not requirements for word-scale visualization size have to be met.

#### 4.1.1.1 *Design Space Focus and Definitions*

We described a design space that focuses on a broad definition of sparklines as outlined in Section 2.2 and a specific in-text usage context as described next.

**STATIC VS. DYNAMIC INTEGRATION:** To place word-scale visualizations into text documents we have two fundamental options: static or dynamic integration. *Dynamic integration* places word-scale visualization based on user-input and, thus, requires a screen viewing setting. The dynamic presentation of word-scale visualizations has the advantage that the text is not cluttered and can be read without additional meta-information present. However, it also has several disadvantages: comparison between word-scale visualizations becomes more difficult, incidental hovering over text might trigger unwanted pop-ups of word-scale visualizations, and—depending on placement options chosen—the text may have to reflow once word-scale visualization display has been triggered.

*Static integration* of word-scale visualizations, on the other hand, has word-scale visualizations always present within the text (as in many places in this document). Advantages of static integration are that the text could be printed with contextual meta-data and that word-scale visualizations can be more easily compared across the document (as noted by Zellweger et al. [228]). Yet, depending on the number of tagged entities, word-scale visualizations might overwhelm or disturb reading the text.

<figure><img src="image_38.png" /></figure>

Dynamic placement example.

Visualization shown on hover.
<figure><img src="image_39.png" /><figcaption>Figure 4.2: Visual description of the three placement positions: *baseline* position, *top* position, and *right* position.</figcaption></figure>

Our design space applies equally to static and dynamic placements but we did not discuss ways to activate word-scale visualizations placement for dynamic integration. We covered an interaction design space in a follow-up publication (Section 4.3).

**PLACEMENT CONTEXT:** According to our definition, context is related to an entity's bounding box (= the bounding box of the text measured by: font ascent+descent; width as the size of the word or word combination) and the word-scale visualization's bounding box. The important lengths can be seen in Figure 4.1. The three types of contextual placement are:

*strong context:* an entity's bounding box and a word-scale visualization's bounding box touch, e. g. a word-scale visualization is placed just adjacent to a word.

*weaker context:* bounding boxes do not touch but the position of word-scale visualizations is still defined by the entity's position in the text, e. g. a word-scale visualization is loosely connected to the position in the text in the same general region as the entity.

*out-of-context:* word-scale visualization position is not related to the entity position, e. g. the word-scale visualization is placed somewhere in the margin of the text.

Our design space concentrated on the strong context case where word-scale visualizations are placed in a sense “as close as possible” to the entity. We do not consider the out-of-context and the weaker context case as these introduce their own set of challenges related to linking the word-scale visualization back to the entities. This is interesting future work with potential solutions in previous research, e. g., context-preserving visual links by Steinberger et al. [196].

### 4.1.1.2 *Design Space Dimensions*

The dimensions for our design space can be roughly divided into two categories: dimensions that have an impact on the word-scale visualization (1–4) and dimensions that impact the text (5 and 6). The dimension are:
**CONTROL OVER THE MAXIMUM HEIGHT:** The height of a word-scale visualization is closely related to the amount of potentially distinguishable information in the vertical dimension of a visualization. A designer has limited control over the height of a word-scale visualization if it is intrinsically bound to a property of the text, such as the font height or the leading. Full control over word-scale visualization height means that the designer can freely choose how to size the word-scale visualization's height and, thus, make choices, for example based on the visual encoding.

**CONTROL OVER THE MAXIMUM WIDTH:** The width of a word-scale visualization is closely related to the amount of potentially distinguishable dimensions or data values on the x-axis of data graphics. A designer has limited control over the width of a word-scale visualization if it is bound to text properties such as the width of the bounding box of a word. With full control over word-scale visualization width the designer can choose the width freely.

**WORD-SCALE VISUALIZATION POSITION:** The strong-context position of a word-scale visualization is defined by a reference point on the word-scale visualization's bounding box—we use the left bottom corner. Generally, the word-scale visualization can be placed anywhere around the entity as long as the two bounding boxes intersect. For simplification purposes we discuss three main placement positions more closely: a) *baseline position*: on the left bottom corner of the entity's bounding box, b) *top position*: on the upper left corner of the entity's bounding box, c) *right position*: the bottom right corner of the entity's bounding box (see Figure 4.2).

**VISUAL ENCODING:** The choice of visual encoding (e. g., line chart, bar chart, etc.) for the word-scale visualization has an effect on how much information can potentially be read within a given aspect ratio. The choice of data and visual encoding can thus lead to the designer requiring more or less control over width and height in order to ensure effective data presentation.

**AMOUNT OF INTER-WORD SPACING INTRODUCED:** Given a specific word-scale visualization width (freely chosen or defined through text properties), a designer can decide to introduce additional inter-word space before or after the entity in order to control the amount of overlap between word-scale visualization and text.

**AMOUNT OF INTER-LINE SPACING INTRODUCED:** Given a specific word-scale visualization height (freely chosen or defined through text properties), a designer can decide to introduce additional inter-line space above or below the entity in order to control the amount of overlap between word-scale visualization and text.
<table><caption>Table 4.1: Several word-scale visualization placements options and the design space-related decisions that lead to them. All top placement options do not overlap with surrounding text. Case 7 is a variant of Case 2 in which overlap with surrounding text is allowed. All other cases can be similarly modified.</caption><thead><tr><th>case</th><th>control over max width</th><th>control over max height</th><th>introduce inter-word spacing</th><th>adjust inter-line spacing</th><th>overlap with entity</th><th>overlap with surr. text</th><th>possible wsv collision</th><th>interaction potentially necessary</th><th>example placement</th></tr></thead><tbody><tr><td>1</td><td>✓</td><td>—</td><td>✓</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet,<br>consectetur sit amet, dolor</td></tr><tr><td>2</td><td>—</td><td>—</td><td>—</td><td>—</td><td>✓</td><td>—</td><td>—</td><td>✓</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet, dolor<br>consectetur sit amet, dolor</td></tr><tr><td>3</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet, dolor<br>consectetur sit amet, dolor</td></tr><tr><td>4</td><td>✓</td><td>—</td><td>✓</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet<br>consectetur sit amet, dolor</td></tr><tr><td>5</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>—</td><td>—</td><td>—</td><td>—</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet<br>consectetur sit amet, dolor<br>consectetur sit amet, dolor</td></tr><tr><td>6</td><td>✓</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>✓</td><td>✓</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet, color</td></tr><tr><td>7</td><td>—</td><td>✓</td><td>—</td><td>—</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>uis autem vel pum iriure dolor<br>m <strong>consetetur</strong> sit amet, dolor</td></tr></tbody></table>

For example, additional inter-line space could be introduced to fit a word-scale visualization of given height between two lines of text without any overlap.

### 4.1.2 Word-Scale Visualization Placement in Practice

When considering how to place word-scale visualizations, a designer may face several practical problems based on the characteristics of the document, the sparkline data, and usage scenario. Important factors include:

* Is the text static, or can it be reflowed? If word-scale visualizations are added to a scanned document (see Figure 4.3a), for example, it may not be possible to adjust the positions of words or phrases to accommodate the word-scale visualization. However, in an electronic reading environment reflowing is easier and the *inter-word spacing* can be expanded to make space for word-scale visualizations.
<figure><img src="image_40.png" /></figure>

(a) Word-scale visualizations used in a scanned World War I archival document. Words are overlaid with color-coded rectangles to express certainty of the OCR applied to the word. Small bar charts appear in the inter-line space when the mouse hovers over an entity and show alternative transcriptions of the word according to the OCR algorithm.

<figure><img src="image_41.png" /></figure>

(b) Word-scale visualizations that provide information scent to help viewers decide whether or not they are interested in following an article linked to from a Wikipedia page. Small bar charts show how frequently the page linked to has been visited in the last 30 days. Detail-on-demand from each sparkline gives information about the exact visit count for each link.

<caption>Figure 4.3: Word-scale visualization placement examples: Scanned documents and online articles</caption>

* Can the inter-line spacing be modified? By increasing the space between lines, a designer can insert larger visualizations positioned above or below the text (see Figure 4.3b). However, adding inter-line spacing can also considerably increase the overall length and amount of whitespace in the document.

* Should the word-scale visualization be read along with the text? If word-scale visualization are intended to be read in the context of the sentences (as with the stock trends in Tufte's original example [203]), the designer may wish to place them in-line with the text. If the word-scale visualization provides supplemental information that could disrupt reading, positioning it in the inter-line space may be more appropriate.

* How important are the word-scale visualization's size and dimensions for readability? The designer may need to enforce a minimum size for visualizations like maps that can contain many small marks. Line charts and other slope-based visualizations may also need to be rendered using a particular aspect ratio in order to facilitate accurate reading and comparison (see Heer and Agrawala [105] or Talbot, Gerth, and Hanrahan [200]).

* Is there an appropriate visual encoding for the data? Visualizations that include text labels, axes, or visually complex marks may not be effective at small scale, or may need to be re-interpreted to work within the available space.

Answering these questions can be challenging, in part, because each individual choice may impact the size and layout of the document, as well as the readability of the text and word-scale visualizations. In addition, we see again that more knowledge on the impact of visual-
ization size on readability is needed to make informed design choices. Moreover, the relationships between these decisions can be complex and their severity may depend heavily on the characteristics of the source document. For example, assume two word-scale visualization designs:

(A) All word-scale visualizations are rendered in front (as an overlay) of the entities at 20% opacity and displayed at full opacity only when hovered with the mouse.

(B) All word-scale visualizations are always shown at full opacity and placed over or next to an entity.

In case A the visual encoding makes overlapping text and word-scale visualization less problematic, but also makes word-scale visualization comparison tasks more difficult. In case B, comparison is easier, but overlap between word-scale visualization and text *will* impede reading. However, avoiding overlap, by increasing the inter-line spacing and/or inter-word padding, can change the size and layout of the document considerably. To help illustrate the impact of and interactions between these design decisions, we show seven common placement options in Table 4.1. They are described in more detail in the original paper.

### 4.1.3 Evaluation and Design Considerations

In order to explore how different word-scale visualization placement options alter the size and layout of documents, we conducted a quantitative evaluation in which we measured the impact of a number of different layout parameters, including word-scale visualization size and position, line height, and padding. We ran the evaluation using real text documents and systematically varied the number of entities in the text so that we could characterize the trade-offs between placement options for both sparse and dense distributions of word-scale visualizations. The original paper describes the choice of documents, factors, procedure and measures we calculated. From our evaluation we derived the following design considerations.

#### 4.1.3.1 Placement Advice

One of the main problems when choosing a placement for word-scale visualizations is the wealth of inter-related factors. When the placement constraints are relatively well defined (as in our OCR example in Figure 4.3a), the space of options becomes relatively narrow and it is easier to make an informed decision. If there are no constraints, the options may become very overwhelming. In this case, we can contribute two general points of advice from our quantitative analysis:

**Use inter-line space where available.** If the inter-line space is sufficiently tall to accommodate the desired visual representation, placing
visualizations above the entity in the inter-line space is often the best
choice. This placement strategy reduces the need to reflow or expand
the text and—unless word-scale visualizations have very long aspect
ratios—it typically results in little overlap between them. For some
small visualizations like horizon graphs and minimalist line charts
Heer, Kong, and Agrawala [107] give experimentally-validated mini-
mum sizes but similar readability or minimum size guidelines do not
yet exist for most chart types, so the discrimination of whether the
space is sufficient rests in the hands of the designer.

**Placing sparklines to the right (usually) saves space.** If text spacing must be modified, adjusting the inter-word spacing and placing word-scale visualizations adjacent to their entities is almost always more space-efficient than inflating the inter-line spacing to accommodate them (provided word-scale visualization heights and aspect ratios are word-like). We found that adding additional inter-line space to place word-scale visualizations above their entity was more space-efficient only in a few cases—typically when the inter-line space was already very close to the desired height or when the number of entities in each line was very large. Right-side placement also has the added benefit that word-scale visualizations can typically be taller, including the height of the font plus whatever inter-line space is available.

4.1.3.2 *Considering Interaction*

Even though we did not investigate options for interaction in depth,
they should be considered for real-world applications that include
word-scale visualizations. We see four main reasons to include in-
teraction with word-scale visualizations: to disambiguate word-scale
visualizations in case of overlap, to see word-scale visualizations on
demand (hide and show), to increase saliency for specific word-scale
visualizations when the default rendition is set to high transparency,
and for details on demand. Of course, which interactions are most
useful and how exactly the interaction should be implement is highly
context- and word-scale visualization-dependent.

4.1.4 *Summary*

In this work, we described a space of options for in-text word-scale
visualization placements from a design-centered view. We especially
investigated the impact of word-scale visualization placements on
various factors that influence word-scale visualization design and text
layout. We illustrated the space of options by discussing seven common
placement options and mention how they can be varied and proposed
some design considerations from a small quantitative study on the
effect of placement option on the text itself. Given our exploration it is
clear that text as an embedding context creates mediated complexity to
deal with, especially when one is interested in maintaining readability.
More in-depth studies of readability of text with embedded word-
scale visualizations are needed. In a follow-up extended abstract we
report on our first investigations in this direction [91]. We eventually
abandoned this line of work ourselves as we did not have access to an
eye tracker with a high-enough resolution that we could—at regular
font sizes—see whether people dwelled on an embedded word-scale
visualizations or on a close-context entity. Instead, we focused on
expanding the design space options through an investigation of the
visual design of word-scale visualizations when embedded in data-rich
documents.
<figure><img src="image_42.png" /><figcaption>Figure 4.4: Examples of word-scale graphics and text-connection created by the graphics designers in our study.</figcaption></figure>

## 4.2 AN EXPLORATORY STUDY OF WORD-SCALE GRAPHICS IN DATA-RICH TEXT DOCUMENTS

The work reported on in this section is based on a collaboration led by my PhD student Pascal Goffin and co-authors Jeremy Boy, Wesley Willett, and myself. The work was published at IEEE TVCG in 2017 [90]. In the following mentions of “we” or “our” refer to this set of co-authors.

Previous work on word-scale visualizations, including our own, had focused mainly on the design of these graphics. However, there was little understanding related to *how, when, and why* such graphics can or should be used. Our main focus in this next work was thus the *aut horing* of word-scale graphics, and in a lesser state, their integration in text documents. We conducted an open ended exploratory study with 9 graphic designers who were tasked with annotating three Wikipedia articles with small visualizations. Interestingly, the designers created not only data-driven graphics but also non-data-driven graphics. We introduced the term *word-scale graphics* to refer to the broader category of graphics that includes both data-driven and non-data-driven graphics. From our study, we analyzed the graphics created and derived a rich collection of different types of graphics (examples shown in Figure 4.4), data provenance, and relationships between text, graphics, and data. Based on this corpus, we presented a systematic overview of word-scale graphic designs, and examined how designers used them. We also discussed the designers’ goals in
creating their graphics, and characterized how they used word-scale graphics to visualize data, add emphasis, and create alternative narratives. Building on these examples, we discussed implications for the design of authoring tools for word-scale graphics and visualizations, and explored how new authoring environments could make it easier for designers to integrate them into documents. Here, I present only an abbreviated version of the original paper.

### 4.2.1 *Study: Designing Word-Scale Graphics*

To explore the diversity of possible uses and designs of word-scale graphics, we conducted a qualitative study in which we asked nine professional graphic designers to graphically annotate existing documents. We chose three Wikipedia articles as the documents to annotate, because they provide a realistic scenario and environment for the integration of word-scale graphics. We designed the study expressly to elicit new and interesting possible designs for word-scale graphics, and to explore how designers might use them to enhance existing documents and engage readers.

Here, I abbreviate several of the study details and focus on the results. Additional details about participants, data, and analyses as well as results can be found in the original paper [90].

#### 4.2.1.1 *Participants, Tasks, and Data*

We recruited nine designers (4 male, 5 female) via word-of-mouth. All designers had formal training in visual communication and storytelling and were expert users of Adobe Illustrator. We provided participants with three Wikipedia articles Figure 4.5 in Adobe Illustrator format. Participants had to read the articles carefully, and envision how small visualizations embedded in the text could provide a richer reading experience. We asked them to create small word-scale graphics, and embed them into the document using Illustrator. We chose these articles for their high density of various data types and diversity of sub-topics. Taken together, the articles contained examples of the seven data types highlighted by Shneiderman [187], including 1D, 2D, 3D, multi-dimensional (MD), temporal, hierarchical (tree), and network data. In order to reduce the overall study time, we allowed participants to approximate the actual data to visualize, rather than trying to retrieve and reproduce exact values in their designs.

#### 4.2.2 *Results*

We collected exactly 200 different word-scale graphics—a mean of 22 (SD = 10) per participant. In order to better understand the broader space of possible word-scale designs or potential uses for them, we
<figure><img src="image_43.png" /><figcaption>Figure 4.5: Thumbnails of the three Wikipedia articles. From left to right: "Regions of France", "Europe", and "Human Body".</figcaption></figure>

coded all graphics according to a number of characteristics. Our main focus was to describe the diversity of designs, with an attention to the designers’ editorial and rhetorical choices. The results here are abbreviated. Additional information regarding data provenance and connections between text and graphics can be found in the original paper.

### 4.2.2.1 Data vs. Non-Data-Driven Encodings

We first examined whether the graphics in our collection represented quantitative data or more conceptual ideas. We coded graphics that used one or more visual variables (position, length, hue, value, ...) to encode data as *data-driven*. We refer to the **79.5% (159/200)** of word-scale graphics that were data-driven as *word-scale visualizations*. The remaining **20.5%** generally represented broad concepts, ideas, or messages not backed by abstract, quantitative data (see Figure 4.6).

### 4.2.2.2 Types of Data

We then coded the *types* of data used in our word-scale visualizations. While we designed the three Wikipedia articles to contain all of
<figure><img src="image_44.png" /><figcaption>Figure 4.6: Examples of data-driven and non-data-driven graphics.</figcaption></figure>

<figure><img src="image_45.png" /><figcaption>Figure 4.7: Levels of types of data, their counts and examples.</figcaption></figure>

Shneiderman's data types [187], our coding deviated from his set to make more fine-grained distinctions. From Shneiderman's set we kept *locations*, *temporal data* and *relations*, but added *proportions & cumulative quantities*, *absolute quantities*, *rankings*, and *change, variation, or trends* (see Figure 4.7).

**PROPORTIONS, CUMULATIVE QUANTITIES:** 🧎, 🧎, 🧎.

Graphics in this category—such as stacked bars or pie charts—put absolute quantities in relation to other quantities. We classified 57/159 (35.8% 🧎) graphics in this category.

**ABSOLUTE QUANTITIES:** 🧎, 🧎, 🧎, 🧎.

The 35 (22.0% 🧎) designs in this category represented single quantitative data values, often using graphics made up of countable units [101]—such as the black dots for countries, and blue bars for regions in the examples above.

**LOCATION DATA:** 🧎, 🧎, 🧎.

Location context or highlights were the basis of 22 (13.8% 🧎) graphics—many of which were small maps.

**CHANGE, VARIATION OR TREND:** 🧎, 🧎.

The 20 (12.6% 🧎) graphics in this category showed data values changing according to another data dimension such as time.
<table><caption>Figure 4.8: Levels of types of graphic, their counts, and examples</caption><thead><tr><th>Level</th><th>Count</th><th>Examples</th></tr></thead><tbody><tr><td>quantitative abstract visualizations</td><td>141</td><td>*, <img src="" alt="A bar chart with a line on top."></td></tr><tr><td>geographic and schematic maps</td><td>16</td><td>*, <img src="" alt="A world map with a region selected."></td></tr><tr><td>icons &amp; symbols</td><td>36</td><td>*, <img src="" alt="A flag with a cross on it."></td></tr><tr><td>illustrations</td><td>7</td><td>*, <img src="" alt="A drawing of a person with a hat."></td></tr></tbody></table>

<figure><img src="image_46.png" /></figure>

The designers included 19 (11.9% ■ ) graphics depicting tem- poral data; mostly in the form of timelines, and often highlighting a specific time point or period.

<figure><img src="image_47.png" /></figure>

Rankings were present in 15 (9.4% ■ ) designs. These showed ordered sets, in which typically a focus object was highlighted.

<figure><img src="image_48.png" /></figure>

A single design (0.1% ■ ) included relational data depicting connections between different systems in the human body.

### 4.2.2.3 Types of Visual Representation

We found examples of quantitative abstract visualizations, geographic or schematic maps, icons and symbols, and illustrations (see Figure 4.8).

<figure><img src="image_49.png" /></figure>

The 70.5% ■ (141/200) of word-scale graphics in this cat- egory used visual variable(s) like length, position, or color to encode quantities—which were typically numbers drawn from the text.

<figure><img src="image_50.png" /></figure>

8% ■ (16/200) of the word-scale graphics were geographic and/or schematic maps. We coded these separately from quantita- tive abstract visualizations, as it was often unclear whether they ac- tually encoded quantitative data (beyond the geo-coordinates of regional boundaries). For example, ▲○N● is a schematic representation of the topological surroundings of the city of Zurich—from left to right: a mountain, a lake, and a forest.

<figure><img src="image_51.png" /></figure>

Of the word-scale graphics, 18% ■ (36/200) did not encode any quantitative data at all. Instead, they replaced a single con-
France is divided into 18 administrative regions (French: région, [ke.ʒjɔ̃]), 13 of which are in Metropolitan France and five of which are overseas regions.[1] The mainland regions and Corsica are each further subdivided into departments, ranging in number from 2 to 13 per region for the

<caption>Figure 4.9: Example of an implementation of a reusable graphical system. The square stands for a region unit and the circle for a department. A bar chart and a trend visualization are assembled from these units.</caption>

cept or word in the text. Some *icons and symbols* directly resem-
bled the object they referred to (such as a 👂 standing for cranial
bones), while others described an entity—these needed to be
learned to be understood (for example, 🇮[!]: standing for France).

Icons and symbols were also sometimes used as a part of quantitative abstract visualizations. Here, they had a secondary function to help relate the shown quantities to known ones [47], or to give contextual cues. For example in 👤, the standard heights of men and woman are related to a “known” height of a cow (according to designer who created the graphic). In a second example, 🏠, the graphic shows a ranking with a human figure on the left signaling that the graphic shows population data.

ILLUSTRATIONS: 🤫, 👤, 🌀, porträt av mettaren

This group of 7 (3.5% 🏠) word-scale graphics included more
complex diagrams and sketches. Often, these illustrations tried
to convey a message. For example, the following small graphic
 👤, shows two human skeletons with one sitting in a wheelchair.
The designer wanted to convey the message that people with
disabilities are often not considered in articles on the human
body.

4.2.2.4 Reusable Graphical Systems

Several designers used a coherent visual theme for their word-scale graphics. They largely remained consistent in terms of coloring schemes and connection between entity and graphic. One designer (Po) went further, proposing a visual language based on the assembly of small data-driven units (see Figure 4.9). This system consisted in introducing tokens (token grammar), and assembling them (assembly model) to express various quantities—similarly to the construction of unit-based visualizations discussed by Huron et al. [111].
<figure><img src="image_52.png" /><figcaption>Figure 4.10: Several ways in which graphics and entities serve as legends or captions.</figcaption></figure>

### 4.2.2.5 Legends and Captions

The design of small-scale visualizations allows little space to add captions or labels to explain a visualization. Interestingly, the designers in our study addressed this challenge in a variety of ways, summarized in Figure 4.10.

The top graphic in Figure 4.10, for example, shows how *word-scale graphics* themselves can serve as part of a legend to a larger graphic. Here, small icons added to the individual element names (carbon, calcium, phosphorus) label the elements of the stacked bar chart above. In cases such as this one, the larger graphic was typically connected to multiple entities, which in turn had small associated iconic representations placed in close proximity to them. For example, the designer who introduced a token-based grammar for the design of her graphics (Section 4.2.2.4) used this mechanism to introduce the tokens of her grammar, placing individual instances of each token near the first occurrence of the corresponding term.

The second example in Figure 4.10 shows the same text illustrated by another designer. The designer colored the element names to serve as the legend to the associated pie chart. In general, graphics with associated *entities as legends* included multi-entity associations, in which the entities were colored according to the elements of the graphic they were associated with.

These first two techniques use entity text to label parts of word-scale graphics. The last example on the bottom, instead, uses an *entity as a caption* to describe the overall content of the graphic, without giving any details about the individual stacked bars.

While Figure 4.10 summarizes three common uses of legends and captions, designers also explored a range of other variations. In a few cases, designers even created compound captions and legends like the one shown in Figure 4.11a. Here, the designer introduced a multi-line entity with nested caption and legend elements that help readers interpret the larger bar chart.
<figure><img src="image_53.png" /></figure>

(a) Nested example of legends and a caption.

(b) Explicit legend in the margin.

**Figure 4.11:** A nested legend and caption explains the top bar chart in (a). The right image (b) shows an example of an explicit legend created and placed in the margin by a designer.

Finally, one designer also created an explicit legend in the margin explaining the types of graphics introduced in her article, including representations showing geographic, temporal, and population data.

### 4.2.2.6 Supporting Comparison and Providing Context

Designers also used a variety of strategies to support comparison between word-scale graphics in the document:

**SHARED REFERENCE STRUCTURE:** Multiple designers used shared reference structures, including reference lines and grids which spanned multiple graphics. In one example (Figure 4.12), P1 added a grid connecting two word-scale graphics on the same line such that the viewers could more easily compare them. In cases where graphics were further apart in the document, some designers created duplicate reference structures to preserve comparability. In Figure 4.13, for example, P3 repeated the same reference grid and icons in two word-scale graphics, highlighting small differences between heights more clearly.

**TRACES:** Another static method we saw (P1, P4, P7) for comparing word-scale graphics used a graphical overlay or trace of one graphic in another (Figure 4.14).

**SEQUENCING:** Several designers (P3, P5, P7) created sequences of word-scale graphics in which each graphic built upon the previous ones. For example, in Figure 4.15, P7 placed a small timeline over each date in the text. Each successive timeline also included desaturated versions of all of the previous dates, helping to place the current date in context with the earlier ones. Other examples of sequencing included sets of maps which built upon one another, each containing elements of the previous one.
<figure><img src="image_54.png" /></figure>

can vary from a high 75% in a newborn infant to a lower 45% in an

<figure><img src="image_55.png" /><figcaption>Figure 4.12: Connecting multiple graphics with a support grid.</figcaption></figure>

female is about 1.6—1.7 m (5'2" to 5'7"). Height is largely determined by genes and diet.

<caption>Figure 4.13: Repeated grid and icons support comparison.</caption>

4.2.2.7 Interaction

We did not ask the designers to consider interaction when creating their word-scale graphics. However, during the post-study debrief several participants proposed ways of augmenting their designs with interaction:

* *Connecting word-scale graphics:* such as with interactive brushing and linking for exploring relationships.

* *Comparing word-scale graphics:* through overlay or dragging word-scale visualizations next to each other.

* *Revealing word-scale graphics:* to show them on demand.

* *Data interactions:* to trigger filtering or highlighting on the graphics themselves.

<figure><img src="image_56.png" /><figcaption>Figure 4.14: Overlaying a trace of one graphic over another.</figcaption></figure>
<figure><img src="image_57.png" /><figcaption>Figure 4.15: Sequencing of timeline word-scale graphics.</figcaption></figure>

4.2.2.8 *Designer's Goals*

During their debrief sessions, designers described a diverse range of goals that guided the ways they designed and integrated word-scale graphics with the documents.

**FACILITATING UNDERSTANDING:** The main goal all nine designers cited was *facilitating understanding*. Concretely, they mentioned using graphics to illustrate concepts (3 out of 9 designers), make quantities explicit (6 out of 9 designers), provide scales (3 out of 9 designers), or put information into context (4 out of 9 designers). One designer also suggested using word-scale graphics to provide more detailed information about the entity as a way to facilitate understanding.

**SUMMARIZING THE CONTENT:** Six designers expressed the use of word-scale graphics to help *summarizing the content* of a sentence. Designers discussed using word-scale graphics to provide a preview of the content or to expose the most important information from the text, providing readers with a clearer take-home message.

**EMPHASIS:** Two designers mentioned using word-scale graphics to emphasize and draw attention to content they felt was particularly important. We were surprised that only two designers mentioned this explicitly, as the presence of word-scale graphics in text already provides a type of intrinsic emphasis effect [99] where the process of mapping content to a graphic increases its visual prominence. In addition, many of the text-to-graphic connections (font color, frames, underlining, background color) give further extrinsic emphasis [99] to the associated entities.

**PROVIDING ALTERNATIVE REPRESENTATIONS:** Two designers also explicitly stated that they used word-scale graphics to *provide alternative representations* as a supplement to the written text. They emphasized that introducing a redundant encoding of the information in the text could help make the text both more
understandable and more memorable [21]. For example one of
the designers suggested to add a visual representation of a cell—
as known from Biology text books—to the entity “cell” in
order to make it more memorable.

<figure><img src="image_58.png" /></figure>

**PROVIDING CONTRASTING VIEWPOINTS:** Interestingly, two designers also described creating graphics to *express a contrasting viewpoint* that differed from what was written in the text. Here, designers mentioned wanting to encourage readers to think more deeply about the text and consider alternative interpretations of it. For example, one designer proposed to add a map of the world showing where Europe was perpetrating slavery in the 16th century to provide a contrasting viewpoint on how Europe “controlled” the world at that time. Especially, pointing out that the word choice “controlled” seems too neutral.

<figure><img src="image_59.png" /></figure>

4.2.3 *Discussion: Using Word-Scale Graphics*

Having developed a fine grained understanding of the different design dimensions of word-scale graphics, we here propose a higher-level definition of use-cases for word-scale graphics. We intend these to serve as inspiration and starting points for authors and designers who wish to integrate word-scale graphics into text documents. This higher-level approach may generally be more accessible for generating *new* designs and applications.

4.2.3.1 *Applications of Word-Scale Graphics*

We characterize three different ways in which designers can apply
word-scale graphics to alter the rhetorical structure of a document.
Specifically, word-scale graphics can *support the content, extend the*
*content, or go beyond the content*.

**SUPPORTING THE CONTENT**
From an editorial perspective, one of the simplest ways of applying a word-scale graphic is using it to *support* its associated entities. In these cases, word-scale graphics display data (left image) that comes directly from the entity text itself (shown above using a *green* arrow), or use non-data-driven symbolic representations (right image) that visually emphasize or reiterate the content of the text.

<figure><img src="image_60.png" /></figure>

Thus, when creating word-scale graphics that support the content,
editorial choices are relatively restricted. The editor or designer needs
to choose which entities to visualize or illustrate, how to design them,
and how to place and connect them to the associated text. However,
they do not need to locate or prepare additional data. For example, an
editor could choose a number in the text as an entity and represent that value using a chart. Alternatively, an author could add a symbolic representation such as a flag to represent the name of a country already included in the text. Although they restate the existing content, these graphics can still provide useful emphasis—facilitating better understanding of numbers or concepts [21], providing summaries, or highlighting alternative versions of the entities (see Section 4.2.2.8).

### EXTENDING THE CONTENT
Word-scale graphics can also be used to provide contextual data that extends the ideas put forward by their associated

entities. These word-scale graphics *extend* the content with compatible information (shown above using *blue* arrows), and do so without purposefully changing the message of the sentence, or proposing novel interpretations. Yet, the creation of word-scale graphics that *extend* content is somewhat more complicated than for the previous group—it involves finding data from elsewhere in the text or from outside the document. It can also involve searching for or creating non-data-driven graphics that provide additional information. For example, an author could extend a timeline connected to the date of a WWII battle by adding outside data such as the start and end dates of the war. Graphics that *extend* the content can help provide context or comparisons for data already contained in the text, or provide additional material about specific entities.

<figure><img src="image_61.png" /></figure>

<figure><img src="image_62.png" /></figure>

### GOING BEYOND THE CONTENT
The previous two applications of word-scale graphics generally complement and reinforce the content of the original

document. However, authors and editors can also use word-scale graphics that *go beyond* the content, including contrasting ideas and messages (shown above using *red* arrows) which may significantly broaden, annotate, or even subvert the original meaning. As a simple example, an editor can choose to add illustrations or symbolic representations that express an opinion, make a statement, or show emotion. For example, by adding a “thumbs up” icon 👍 next to an entity expressing an action, the editor can express agreement. Similarly, by including a word-scale graphic showing external data that contradicts the existing narrative, an editor or designer can suggest alternative, contrasting viewpoint (see Section 4.2.2.8). While these contrasting viewpoints may originate with the original author, they can also be added in response to the article at a later point.

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>
### 4.2.3.2 Challenges when Creating Word-Scale Graphics

One of the main challenges our participants had was to find suitable pieces of text they could complement with word-scale graphics. The process of deciding what should and could be represented usually happened at the very beginning of the process of integrating word-scale graphics. The designers also expressed concerns that the associated entities in the text had to be below a certain length to be representable or connectable to a word-scale graphic. We, therefore, updated our definition of an entity. A word-scale graphic can have one or multiple entities, or none, and proximity can be a way to link graphic and text. We argue that this updated definition provides more freedom to support the linking of word-scale graphics to the text. Entities can also serve multiple purposes—including acting as a label or caption for the graphic—a fact that was not recognized in previous work. Moreover, this definition explicitly acknowledges that word-scale graphics do not necessarily need an entity if it is clear what the word-scale graphic represents, and the connection to the text is clear.

In addition, and linked to this thematic of defining associated text, our participants had difficulties finding suitable places to position their word-scale graphics. In our instructions, we restricted the positioning of word-scale graphics to the interline space. However, designers still hesitated at times, and sought alternative placement options—for example at the end of a title or along a paragraph. One participant also told us that she would have liked to rewrite the sentences to make it easier for her to integrate word-scale graphics. She found the sentences were too convoluted, which led her to find it difficult to find suitable positions for her word-scale graphics. Moreover, some of the information or data that the designers wanted to integrate into the word-scale graphics was spread over multiple lines. This made it more difficult to find good places to position graphics. Sometimes, designers also started to create a graphic first, and then had a difficult time finding suitable text to associate with the word-scale graphic.

### 4.2.4 Summary

We presented the results of an exploratory study on the use of word-scale graphics in text documents. The study resulted in a rich collection of different types of graphic designs and also helped us to identify specific design challenges unique to word-scale graphics and the mediated complexity of the surrounding text. Designers creating micro visualization for text embedding must consider the immediate connections between visualizations and text. Also common to other micro visualization contexts, they need to find alternatives to common visual elements like axes, legends, and captions that work given reduced space—but here found that the surrounding text actually helped in
providing reference. In addition to the interaction challenges that already showed in the previous section, we saw again that designing interactions would present a challenge, since graphics may be dispersed, small, and only loosely associated with one another. Based on the work presented in this section we developed a first library (http://inria.github.io/sparklificator/) to embed word-scale graphics in online texts. In follow-up work we explored in more detail how to integrate interaction with word-scale visualizations.
<figure><img src="image_65.png" /><figcaption>Figure 4.16: Examples of word-scale visualization use in the wild: (A) in a political news article from FiveThirtyEight [33], (B) embedded inline in the SurVis analysis tool [11], (C) a word trend ticker on dictionary.com [61], and (D) inline in Hoffswell et al.'s [109] source code editor.</figcaption></figure>

## 4.3 INTERACTION TECHNIQUES FOR EMBEDDED WORD-SCALE VISUALIZATIONS

In our previous work, interaction with word-scale visualizations emerged as a clear challenge and opportunity. Designers suggested interactions to connect, compare, filter, and highlight multiple word-scale visualizations. They also proposed interactions for incorporating word-scale visualizations on-demand and switching between alternative views of the same data. In our follow-up work entitled "Interaction Techniques for Visual Exploration Using Embedded Word-Scale Visualizations" and published at CHI 2020 [89], Pascal Goffin, Tanja Blascheck, myself, and Wesley Willett described a design space of view manipulation interactions for word-scale visualizations. Any mention of "we/our" in this section refers to this set of co-authors.

Interactive word-scale visualizations represent an opportunity to create rich reading experiences that transcend the limitations of traditional text + figure paradigms. While larger and more complex visualizations outside the text can provide other benefits including more detail and easier comparisons, we showed how view manipulation interactions can also add many of these benefits to word-scale visualizations. Most examples of word-scale visualizations in-the-wild (Figure 4.16) are *static* charts embedded at fixed locations in documents with no support for interactivity. This makes sense when text and visualizations are intended for print. Yet, today most text is consumed on personal computers such as phones, tablets, e-readers, desktops, or laptops. In these environments people can engage actively with the text and its embedded content to explore, make comparisons, and extract details.

We outlined a design space of view manipulation interactions to expand the active reading capabilities of embedded word-scale visualizations. This design space, in particular, allows interactive word-scale visualizations to serve as a bridge between document-centric and visualization-centric analyses. In document-centric analyses, the document is the main point of interest and figures provide supporting evidence or contextual information. In visualization-centric analysis
ses, source text is a secondary element (if it is present at all) and
large visualizations become the focus of the viewer's attention. Ex-
ample tools that use documents as a data source while providing
larger visualization environments include DocuBurst [55] or Jigsaw
[195]. However, no current environment supports fluidly transitioning
between document-centric analyses that include visualizations and
visualization-centric analyses that still have a close connection to the
text.

4.3.1 *A Design Space of View Manipulation Interactions for Word-Scale Visualizations*

*View Manipulation* [108] tasks are vital to visualization scenarios in which elements need to be selected, navigated, coordinated, and organized. Word-scale visualizations are elements that could benefit from interactive organization and comparison while keeping a close tie to the source text. Our work explores view manipulation interactions to support richer active reading and data exploration with word-scale visualizations. We organize them into a design space of view manipulation interactions for small data representations. In particular, we focus our design space around four types of analysis tasks:

T1: **COLLECTING WORD-SCALE VISUALIZATIONS.** Creating small multiple visualizations that bring together visualizations from across the document for higher-level analyses.

T2: ARRANGING WORD-SCALE VISUALIZATIONS. Organizing visualizations to expose trends and support reasoning.

T3: COMPARING WORD-SCALE VISUALIZATIONS. Providing tools to make direct comparisons between visualizations.

T4: NAVIGATING THE DOCUMENT. Using visualizations as entry points to the text to aid navigation to related locations.

We explored the design space through iterative rounds of ideation,
sketching, and prototyping. We began by examining each of the gen-
eral classes of interactions above, then worked to describe, categorize,
and test a diverse set of interactions by which each could be carried
out.

4.3.1.1 *Collecting Word-Scale Visualizations*

When reading long scrollable documents enriched with multiple
word-scale visualizations it may be difficult to obtain a clear enough
overview of the visualizations to analyze the underlying data. This is
especially true if the visualizations are *embedded* throughout the docu-
ment and impossible to view simultaneously on one screen. However,
seeing an overview is often an important first step before engaging
EMBEDDED

<figure><img src="image_66.png" /></figure>

GATHER

<figure><img src="image_67.png" /></figure>

DRAG

<figure><img src="image_68.png" /></figure>

in other data exploration tasks such as arranging, comparing, sorting, and navigating the data. To support full or partial overviews, we propose two interaction techniques for collecting word-scale visualizations from across a document: *gather* and *drag*.

The *gather* interaction, inspired by Moscovich et al. [152]'s "Bring & Go" and Ghani, Henry Riche, and Elmqvist [86]'s "Dynamic Insets" allows a reader to quickly collect a set of word-scale visualizations at one specified location in the document. Using this interaction, a reader summons some or all of the visualizations in the text, making it possible to examine them simultaneously. When gathered, visualizations animate from their original distributed positions in the document to a central location. While this point could fall anywhere on or off-screen, we suggest gathering visualizations around a specific entity of interest, which we call a *focus entity*. This permits readers who are focused on the text to gather multiple visualizations at their current reading position, allowing them to continue reading while still using the gathered visualizations for context and comparison. In cases in which a reader only wishes to compare a small number of visualizations, *dragging* individual charts together provides a simpler and more predictable collection method.

Both gathering and dragging require design decisions about several different aspects of these two interactions:

**MULTIPLE ENCODINGS/SEMANTICS IN DOCUMENT.** A document may contain word-scale visualizations showing different datasets which may or may not be comparable to one another. These initial conditions should be considered when gathering or dragging word-scale visualizations. In these cases, readers may wish to only gather those visualizations which share the same data types, encodings, or semantics. Alternatively, a tool could gather each type of word-scale visualization into its own distinct cluster.

**SCALING.** To support meaningful comparisons between multiple word-scale visualizations which display the same kind of data, all visualizations must use the same layout, size, and color scales. However, using a shared scale for all word-scale visualizations may also hide important details in individual charts. Dynamically aligning the scales for multiple charts when they are collected together can help ensure that they are comparable, but may also entail complex transitions.

**CONNECTING TO PREVIOUS LOCATIONS.** When gathering dragging a word-scale visualization to another place in a document, it is important to consider how and if to fill the original location of the dragged item. Completely removing the word-scale visualizations leaves gaps in the text. As a result, we find it helpful to leave a copy of the entity and visualization at the original position in the document. We call this copy a *ghosted word-scale*
visualization as it keeps a visually de-emphasized copy of the original element in the text, provides provenance information, and indicates that a copy of the word-scale visualization is currently being shown elsewhere. Leaving the ghosted visualizations in place also prevents the text from re-flowing and preserves the document layout.

**DRAGGING OUT OF THE VIEWPORT.** A word-scale visualization may need to be dragged to a location in the document outside of the current viewport not visible to a reader. Selecting and dragging a word-scale visualization should automatically scroll the view- port when the mouse reaches the viewport's edge. Word-scale visualizations can also have suggested interactivity cues [23] to let the reader know whether or not a word-scale visualization can be dragged to a given location.

**RETAINING CONNECTIONS TO ENTITIES.** To preserve the connection between the text and the visualizations when gathering or dragging, word-scale visualizations can be graphically coupled to the text entities to which the data refers (such as a name, a place, or a stock symbol SPX). These small snippets of entity text can then serve as labels that identify the charts and preserve a semantic link back to the document when word-scale visualizations are collected. In addition, rendering traces that connect gathered entities to their original positions (as in Figure 4.17-bottom right) further reinforces visualization-to-document relationships.

**DUPLICATE WORD-SCALE VISUALIZATIONS.** A document can include multiple occurrences of the same entity and duplicate word-scale visualizations. One option to avoid multiple copies in gathered overviews is to explicitly disallow duplicates and render only one word-scale visualization per entity. However, designers must then choose which instance of the entity to annotate with a word-scale visualization. In documents that include duplicate word-scale visualizations, designers can combat redundancy, for example, by visually stacking duplicates on top of each other when gathered and adding branching traces that link charts back to all original locations in the document.

**RELEASE BACK TO ORIGINAL LOCATIONS.** When a reader has finished their comparison, the collected word-scale visualizations need to be released back to their original positions. Using animated transitions to move charts back to their original positions can reinforce where the word-scale visualizations came from and help signal their distribution in a document.
<figure><img src="image_69.png" /><figcaption>COLUMN</figcaption></figure>

<figure><img src="image_70.png" /><figcaption>ROW</figcaption></figure>

<figure><img src="image_71.png" /><figcaption>CLUSTERED</figcaption></figure>

<figure><img src="image_72.png" /><figcaption>IN-FRONT</figcaption></figure>

<figure><img src="image_73.png" /><figcaption>NO-OVERLAP</figcaption></figure>

<figure><img src="image_74.png" /><figcaption>PERSISTENT</figcaption></figure>

<figure><img src="image_75.png" /></figure>

### 4.3.1.2 Organizing Word-Scale Visualizations

Collecting word-scale visualizations around a focus entity using a *gather* interaction helps to provide an overview of the visualizations but does not support more detailed comparisons. To see trends or differences more clearly, the gathered word-scale visualizations need to be arranged into layouts that convey order and support examination.

**LAYOUTS** As Beck and Weiskopf [12] highlight, existing uses of word-scale visualizations often tend to place them in tables, lists, or other small multiples layouts [204]. Once a reader collects visualizations, the choice of layout determines how many word-scale visualizations can be viewed simultaneously as well as how readers can examine and compare them.

*Grid layouts* make good use of available screen space by placing visualizations both vertically and horizontally. However, they make comparisons across rows or columns difficult. *Column layouts* and *row layouts* each support consistent comparisons along the charts' principle axes, but permit fewer visualizations on screen. *Column layouts* support comparison along the charts' x-axis, but may require considerable scrolling and can obscure large sections of the document. *Row layouts*, meanwhile, may obscure as little as one line of text, but are limited by the relatively narrow width of most documents.

While few current applications use non-rectilinear layouts for word-scale visualizations [12], they may be useful in some scenarios. For example *clustered layouts*, which organize entities based on common characteristics could support identification and comparison of groups of charts.

**LAYOUT PLACEMENT** Once a layout has been chosen, it needs to be placed relative to the document, entailing a number of further design decisions.

Being able to quickly glance at the text when analyzing gathered word-scale visualizations can provide clues that help contextualize the data. Placing a layout *in-front* of the text in a separate layer obscures some text, but keeps the layout of the document stable. Alternatively, embedding rectilinear layouts like *grids* and *rows* into the text can require re-flowing the document to create the necessary space—similar to Yoon et al.'s approach for placing ink annotations [225]. These *no-overlap* placements avoid collisions between the visualizations and text and guarantee that the layout is placed close to a focus entity. However, by introducing additional space, no-overlap placements also change the overall size of the document.

Applications may also use *persistent* placements that position the collected word-scale visualizations at a fixed location on the page and allow the document to be scrolled independently of the layout. This can make it easier to use the word-scale visualizations as an index to
the document and help readers skim the document without losing track of the collected visualizations. However, it may also increase the distance between the layout and visualizations' original positions in the document.

**FILTERING** In documents with large numbers of word-scale visualizations, readers may want to see only a subset at any given time. Interactive filtering controls allow readers to dynamically hide and show visualizations based on properties of the data, their associated entities or their semantics. For example, an article with many embedded charts could be tailored to show a limited number of visualizations per paragraph, to show only visualizations linked to specific entities, or with particular semantics. Alternatively, a reader could gather word-scale visualizations to obtain an overview of the document, then filter the overview to examine and compare the most relevant subset.

**ORDERING** Interactively reordering word-scale visualizations within a gathered layout can make it easier for readers to see trends and relationships within groups of visualizations.

Ordering visualizations based on their *document position* provides structural consistency between the layout and the text as word-scale visualizations appear in the same order in both places. Alternatively, *manually reordering* allows readers to impose their own structure by grouping related visualizations together. Meanwhile, *sorting* visualizations based on their data values (max, min, average), similarity, or properties of their entities (name, date, etc.), can reveal relationships between visualizations that exist far apart in the document.

### 4.3.1.3 Comparing Word-Scale Visualizations

Word-scale visualizations in an ordered or sorted layout are already much easier to compare through juxtaposition [87] than word-scale visualizations scattered across a document. However, fine-grained comparisons, such as comparing values between charts with different axes is still difficult. We describe two techniques which support more detailed comparisons between word-scale visualizations—interactions for *aligning* grouped charts and interactions for *directly comparing values*.

**ALIGNING** In row and column layouts, word-scale visualizations can be aligned in various ways. We present three alignment operators that are useful for different data comparisons. We focus on cases in which the document contains comparable word-scale visualizations. However, if a document contains multiple types of charts or data which cannot be compared directly (such as stock prices and employment figures) it may be necessary to group related visualizations before aligning them.

DOC POSITION

<figure><img src="image_76.png" /></figure>

REORDER

<figure><img src="image_77.png" /></figure>

SORT

<figure><img src="image_78.png" /></figure>
<figure><img src="image_79.png" /><figcaption>AXIS-ALIGN</figcaption></figure>

<figure><img src="image_80.png" /><figcaption>PAN-ALIGN</figcaption></figure>

<figure><img src="image_81.png" /></figure>

The first interaction left-aligns the word-scale visualizations in a column (*column-align*) and allows readers to vertically compare across the *x*-axis. However, if the *x*-axes of different visualizations are shifted relative to one another an *axis-alignment* may be more useful. For example, when comparing timelines that have different start and end dates, axis-alignment chooses a common date as the alignment's center. Interactive (*pan-align*) techniques can also be applied when the word-scale visualizations are arranged in a column or row adjacent to a focused chart. In this case, panning the column or row slides the word-scale visualizations of interest past the focused chart, enabling side-by-side comparisons.

**DIRECT COMPARISON** While side-by-side comparison are useful, more direct comparison mechanisms are often necessary [202].

<figure><img src="image_82.png" /></figure>

One simple technique is *brushing and linking* between word-scale visualizations. When a reader brushes over values in one word-scale visualization, corresponding values in other visualizations are highlighted [13]. *Drag* interactions can also be used to support comparison by either superimposing word-scale visualizations on top of one another, by triggering the creation of an explicit encoding such as difference computations between the data in multiple charts, or by juxtaposing word-scale visualizations [87].

#### 4.3.1.4 Navigating the Document

Word-scale visualizations in documents are useful, in part, because they support ad-hoc transitions between reading a document and exploring associated data. Interactions that allow a reader to jump from a collected word-scale visualization back to its original position in the document support fluid transitions between these modes. These kinds of navigation interactions exhibit two properties of Elmqvist et al.'s [69] characterization of fluid interfaces, promoting *flow* and enabling *seamless switching* between exploration and reading.

With these kinds of navigation interactions, sets of collected word-scale visualizations can serve as an index or entry point into the document, providing graphical overviews reminiscent of interfaces like Koch et al.'s "VarifocalReader" [127].
<figure><img src="image_83.png" /><figcaption>Figure 4.17: Our economic news prototype (left) showing stock charts in a *no-overlap grid*. Our European history prototype (top right) showing timelines in an *in-front grid*. (bottom right) Our eye-tracking report prototype with attention maps in a *row* layout. In all three, charts are grouped around a focus entity (COMP, Vladimir Lenin, and P09). Left and right steppers on the row layout jump back and forth through the full set of attention maps.</figcaption></figure>

### 4.3.2 Prototypes

To demonstrate how elements from this design space translate into concrete applications, we implemented an active reading environment that supports interactive word-scale visualizations and illustrate its functionality using documents from economic journalism, contemporary European history, and eye tracking research. For each context we designed or used custom word-scale visualizations.

Our economic journalism example allows readers to collect, arrange, compare, and navigate word-scale visualizations embedded in news articles extracted from MarketWatch [112]—one of many economic news outlets that already uses small stock charts in their articles (typically as tooltips over individual stock symbols or indexes). Our prototype (Figure 4.17—left) embeds small stock charts directly into text of articles and adds view manipulation interactions to make the charts easier to examine and compare. We use a stock chart design SPX, which shows the performance of a single stock or financial index over the 30 days leading up to the publication of the article. The dark line on the chart shows the stock's performance
relative to its value at the beginning of the 30-day period. The light grey horizontal baseline shows the price on the first day. If the price on a given day was lower than the price at the beginning of the 30-day period, the area between the line and the baseline is colored red. If the price was higher, the area is colored green. A red dot shows the day with the lowest price, while a green dot shows the day with the highest price. A grey horizontal line highlights distance between the lowest and highest days.

In our second example, we embed timeline charts into articles on European history to help readers better understand the temporal relationships and overlaps between historical figures, movements, and events (Figure 4.17-top right). We use a simple timeline design with fixed start and end dates that visualizes lifespans and periods using blue highlights. Viewers can use the timelines to quickly check for overlaps between events and the lifespans of historical figures, making them easier to contextualize and compare.

For the third example, we created an interactive version of an academic research report, which incorporates word-scale eye tracking visualizations by Beck et al. [9] (Figure 4.17-bottom right). Throughout the document we visualize eye movement data using Beck et al.'s gridded attention maps, each of which shows the spatial distribution of visual fixations for one participant during an experimental trial. The cells with the most fixations appear dark, while those with the least appear white. Viewers can use grouping and sorting operations to more easily compare the performance of multiple participants, and identify ones who used similar strategies.

To minimize distracting overlaps between word-scale visualizations and text and help preserve readability and context, all three examples used *in-front* and *no-overlap* layout placements when gathering charts. The in-front placement always keeps the line of text containing the focus entity visible, while the no-overlap placement keeps all text visible by gathering charts between paragraphs.

Our prototype extends our Sparklificator library [92], which allows developers to insert a variety of word-scale visualizations into HTML documents. We implemented visualizations using JavaScript and D3.js. The source code and the three working examples of our prototype are available at https://github.com/InteractionWSV. To use this library, the entities in an HTML text must be tagged with span tags and should use a distinctive class name. Developers can use existing word-scale visualization designs or implement their own word-scale visualization renderers.
4.3.3 User Studies

While many examples of static word-scale visualizations exist, interac-
tive word-scale visualizations are still a new concept. Moreover, we
still know little about the effect of introducing interactive versions of
these charts in online reading environments. Therefore, we ran two
experiments to investigate participants’ experience using our active
reading prototype. We focused on first studying a small set of primary
interaction techniques that make it possible to create and explore
small multiples overviews and compared these against using statically
embedded word-scale visualizations. In our second experiment, we
gave participants a broader set of interaction techniques and allowed
them to freely explore 3 articles and answer an open-ended question.
Here, I only report on the main results and refer the reader to the
original paper [89] for experiment details.

Participants of the first experiment experienced a static and an
interactive word-scale visualization condition. Participants completed
**search** tasks, during which they located and extracted information
from a *single* word-scale visualization, and more complex **comparison**
tasks, which required them to visually scan *all* of the word-scale
visualizations and identify a single outlier. Participants found both
the static and the interactive word-scale visualization helpful for both
tasks. They chose to use the interactive features when they were
available and appreciated the ease of comparison and reduced need
for scrolling. Participants' perceived reduction in task completion time
was further confirmed by the quantitative results for the comparison
task. We also collected suggestions from participants which included
requests for faster gathering animations, showing arrangements next
to the cursor, alternative grid arrangements and sorting options, as
well as the ability to annotate word-scale visualizations.

Our log analysis of Experiment 2 showed that participants made
good use of the available interactive options, using a variety of actions.
Yet, participants used a somewhat different mix of interactions than
in Experiment 1, which is not surprising given the more exploratory
prompt. Despite training, participants made little use of sorting and
navigation in particular. Interestingly, most participants used the grid
layout that was provided in Experiment 1 and only four participants
tried the other grid options at all. The improvements and feedback
we received for Experiment 2 were similar to those we collected for
Experiment 1 and were mainly concerned with the positioning of the
grid as well as the slow animation speed.

4.3.4 Discussion

Overall, our prototype implementation highlights one possible ap-
proach for integrating the key interactions from our design space.