# Data Physicalization

Pierre Dragicevic, Yvonne Jansen, Andrew Vande Moere

► To cite this version:

Pierre Dragicevic, Yvonne Jansen, Andrew Vande Moere. Data Physicalization. Jean Vanderdonckt; Philippe Palanque; Marco Winckler. Springer Handbook of Human Computer Interaction, Springer, Cham, 2021, Springer Reference, 978-3-319-27648-9. 10.1007/978-3-319-27648-9_94-1.hal-02113248v3

HAL Id: hal-02113248

https://inria.hal.science/hal-02113248v3

Submitted on 10 Oct 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Data Physicalization

Pierre Dragicevic, Yvonne Jansen, and Andrew Vande Moere

¹ Inria, France

pierre.dragicevic@inria.fr

² Sorbonne Université, CNRS, ISIR, France
yvonne.jansen@cnrs.fr

³ Research[x]Design, KU Leuven, Belgium
andrew.vandemoere@kuleuven.be

**Abstract.** Data physicalization is a rich and vast research area that studies the use of physical artifacts to convey data. It overlaps with a number of research areas including information visualization, scientific visualization, visual analytics, tangible user interfaces, shape-changing interfaces, personal fabrication interfaces, as well as graphic design, architecture, and art. This chapter surveys academic work on data physicalization up to 2018 and also provides a broad overview of nonacademic work. It discusses how data physicalization has been used for analytical purposes, communication and education, accessibility, self-reflection and self-expression, and finally for enjoyment and meaning. It also discusses enabling technologies, reviews empirical studies, and surveys models and theories of data physicalization.

**Keywords.** Data physicalization, physical visualizations, information visualization, tangible user interfaces, data sculptures, tangible thinking, making processes.
Table of Contents

<table><tr><td>1</td><td>Introduction</td><td>3</td></tr><tr><td>2</td><td>Physicalization for Analytics</td><td>5</td></tr><tr><td>2.1</td><td>Early Uses in Science and Engineering</td><td>5</td></tr><tr><td>2.2</td><td>Digitally-Fabricated Physicalizations for Analytics</td><td>6</td></tr><tr><td>2.3</td><td>Dynamic Physicalization for Analytics</td><td>7</td></tr><tr><td>3</td><td>Physicalization for Communication and Education</td><td>8</td></tr><tr><td>3.1</td><td>Early Uses</td><td>9</td></tr><tr><td>3.2</td><td>Data-Driven Presentations</td><td>10</td></tr><tr><td>3.3</td><td>Self-Directed Exploration and Public Displays</td><td>11</td></tr><tr><td>3.4</td><td>Participatory Data Collection and Physicalization</td><td>12</td></tr><tr><td>4</td><td>Physicalization for Accessibility</td><td>13</td></tr><tr><td>4.1</td><td>Early Systems</td><td>13</td></tr><tr><td>4.2</td><td>Recent Developments</td><td>14</td></tr><tr><td>5</td><td>Physicalization for Self-Reflection and Self-Expression</td><td>15</td></tr><tr><td>5.1</td><td>Self-Reflection</td><td>16</td></tr><tr><td>5.2</td><td>Self-Expression</td><td>17</td></tr><tr><td>5.3</td><td>Physicalization Making</td><td>18</td></tr><tr><td>6</td><td>Physicalization for Enjoyment and Meaning-Making</td><td>20</td></tr><tr><td>6.1</td><td>Ambient Visualization and Informative Art</td><td>21</td></tr><tr><td>6.2</td><td>Artistic Expression</td><td>22</td></tr><tr><td>6.3</td><td>Multisensory Expression</td><td>24</td></tr><tr><td>7</td><td>Enabling Technologies</td><td>24</td></tr><tr><td>7.1</td><td>Digital Fabrication</td><td>24</td></tr><tr><td>7.2</td><td>Actuation Technologies</td><td>25</td></tr><tr><td>7.3</td><td>Specialized Applications for Creating Data Physicalizations</td><td>28</td></tr><tr><td>8</td><td>Studies of Data Physicalization</td><td>29</td></tr><tr><td>8.1</td><td>Comparative Studies</td><td>29</td></tr><tr><td>8.2</td><td>Non-Comparative Studies</td><td>32</td></tr><tr><td>9</td><td>Models and Theories</td><td>34</td></tr><tr><td>9.1</td><td>Pipeline Models</td><td>35</td></tr><tr><td>9.2</td><td>Data Embodiment and Contextual Sensemaking</td><td>36</td></tr><tr><td>10</td><td>Conclusion</td><td>41</td></tr></table>
# 1 Introduction

The discipline and practice of visualization aims to augment the human understanding of data. The primary challenge of visualization design consists of developing techniques that turn abstract data into easily perceivable and interpretable representations. Computer displays have demonstrated several clear strengths for this purpose, including their ability to convey visual imagery that is extremely rich in details, and their ability to let users explore data by dynamically altering its visual representation [19, 20, 106]. Meanwhile, advances in tangible computing have illustrated how humans can interact with digital information by using their natural ability to perceive and manipulate physical objects and materials [135, 162]. As such, a research area has emerged that questions why the display of, and interaction with, data should remain limited by the constraints of pixel matrices, two-dimensional gestures and keyboards buttons, and whether moving data representations from flat displays into the physical world could create novel and useful ways of exploring, experiencing and communicating data [72, 163].

In this chapter we refer to a data physicalization (or simply a physicalization) as a *physical artifact whose geometry or material properties encode data* [72]. Thus the artifact must be physical, and must encode data. In particular, there is a conceptual distinction between a physical model and a data physicalization: a scale model such as a solid terrain model, an architecture model, or a figurative sculpture are not data physicalizations, unless part of their shape or appearance conveys abstract data as it is sometimes the case for scientific visualization [105]. Such pure physical models will not be covered in this chapter other than occasionally, in order to inform discussions on data physicalization.

Several terms exist that are closely related to data physicalization. For instance, *physical visualization* is a synonym for data physicalization [71]. *Data sculpture* was one of the first terms that have characterized this emerging domain, yet the term generally denotes data physicalizations that aim for more artistic expressions of data in physical form [72,178]. In this chapter, we discuss data physicalization irrespective of its function and its creator's intent.
Data physicalization has been further defined as a research area which “ex-
amines how computer-supported, physical representations of data (i.e., physical-
izations), can support cognition, communication, learning, problem solving, and
decision making” [72]. This chapter provides an overview of this research area.
Although the research area as we define it refers to the support of computers, this
chapter also discusses manually-crafted physicalizations, as they are often rele-
vant for informing computer-supported physicalization. Furthermore, computer
support can take various forms, such as the modelling and the digital fabrication
of physicalizations, or their dynamic actuation during use.

Data physicalization is closely related to several research areas within and
around HCI [72]. The most closely connected area is visualization, which we use
here as an umbrella term to capture data visualization, information visualization,
scientific visualization and visual analytics. The general goal of visualization is
to study “the use of computer supported, interactive, visual representations of
abstract data to amplify cognition” [19]. Data physicalization shares many research
questions with visualization, but with an explicit focus on the physical, tangible
and material character of physical data representations. Data physicalization is
also closely related to the area of tangible interaction, whose goal is to study
interactive systems that “give physical form to digital information, employing
physical artifacts both as representations and controls for computational me-
dia” [162]. Other areas within HCI that are connected to data physicalization
include ambient displays [174], shape-changing interfaces [4,127] and personal
fabrication [9]. Despite overlaps in terms of goals and methods, data physical-
ization differs from these aforementioned areas in that it exclusively focuses on
data-driven tasks, such as data exploration and data communication. Finally,
data physicalization and the production of data sculptures in particular have
tight connections with more artistically-driven disciplines, such as graphic design,
architecture, and installation art, among many others [75,164].

The structure of this chapter is loosely inspired by Munzner’s multi-level
typology of abstract visualization tasks [16], which distinguishes three main
motivations for creating and using visualizations: to discover, to present, and to
enjoy. Similarly, we start by discussing the use of physicalization for analytical
purposes, particularly in science and engineering (section 2). We then discuss how
data physicalizations have been used for communication and education (section
3), for supporting accessibility to blind and visually-impaired users (section 4),
for self-reflection and self-expression (section 5), and finally for enjoyment and
meaning-making (section 6). We complete this review by discussing recent and
emerging technologies enabling different forms of physicalizations (section 7),
the different empirical studies that have been conducted on data physicalization
subsection, and finally existing models and theories of data physicalization
subsection). In several places, we will mention historical data physicalizations
that have been in use before graphical computers existed, because they are useful
to provide context and can serve as a source of inspiration for novel designs.
## 2 Physicalization for Analytics

One of the main purposes of visualization is to help people solve problems by making it easier to reason with or about data: much of visualization research focuses on the pragmatic aspects of data visualization [84], and on helping people carry out useful tasks: “computer-based visualization systems provide visual representations of datasets designed to help people carry out tasks more effectively” [106]. This has also been a major goal of early data physicalizations. Here we discuss examples of uses of data physicalization for analytical purposes, before and after computers were available.

### 2.1 Early Uses in Science and Engineering

<figure><img src="image_2.png" /><figcaption>Fig. 1: (a) Maxwell’s “thermodynamic surface” showing the relationship between energy, entropy and volume (1871), and (b) John Kendrew constructing a physical representation of the molecule myoglobin called the “forest of rods” (1960).</figcaption></figure>

A number of scientists from the 19–20th centuries crafted physical, three-dimensional representations of data as part of their research. For example, in the 1870s, the physicist James Maxwell created plaster models of three-dimensional thermodynamic functions that helped him and Josiah Willard Gibbs understand and develop the thermodynamic theory of state [86] (see Figure 1-a). The construction of solid 3D molecular models also played an important role in many major discoveries in chemistry, including in Watson and Crick's discovery of the helix structure of DNA in 1953 [42] and John Kendrew's discovery of the structure of myoglobin in 1958 [99] (see Figure 1-b). All three chemists received a Nobel prize for their discoveries.

During this time period, physical 3D data representations were also in use in industry. In the 1910s, human factors pioneer Frank Gilbreth created a number of wire-based representations of factory worker hand movements in 3D space, in

<footnote>Credits: (a) Photo S. Trewin, 2007. YPM HST.290012. Courtesy of the Peabody Museum of Natural History, Yale University, New Haven, Connecticut, USA. (b) Copyright MRC Laboratory of Molecular Biology. Used with permission.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2: (a) Frank Gilbreth's 3D wire model of a factory worker's hand movement (c. 1915), and (b) a data physicalization by the Detroit Edison Company showing electricity consumption for the year 1935, with a slice per day [17].</figcaption></figure>

Credits: (a) Collection: Frank B. Gilbreth Motion Study Photographs (1913-1917). Repository: The Kheel Center for Labor-Management Documentation and Archives.

order to better understand how to optimize these movements [30] (see Figure 2-a). In the 1930s, several American electricity companies were building and maintaining solid 3D visualizations of electricity consumption, presumably to better anticipate power demands [71] (see Figure 2-B).

All these 3D data representations were very time-consuming to build and with the advent of graphical computers, many of them were eventually superseded by virtual 3D models and visualizations [38, 99]. However, physical data representations never stopped being used, and they are becoming more and more prevalent as they are easier and easier to build.

## 2.2 Digitally-Fabricated Physicalizations for Analytics

Many scientific activities currently involve the visualization of 3D data [132], but examining 3D representations on computer displays is subject to a number of issues such as perceptual distortion and occlusion, and the difficulty of navigating in 3D [136]. Meanwhile, with digital fabrication it became easy to create accurate and visually rich 3D data physicalizations.

Mike Bailey from the San Diego Supercomputer Center was a pioneer in the use of 3D printing for scientific visualization. In 1995, he created the SDSC TeleManufacturing Facility to help scientists visualize their data in physical form (Figure 3-a). The facility produced a number of physical scientific visualizations of molecular, physical, geological, anatomical, and mathematical data using laminated manufacturing [7]. Two decades later, other research labs started to investigate the use of modern fabrication technology such as full-color 3D printing to physicalize scientific visualizations [2] (Figure 3-bc).

These research labs saw benefits in complementing screen-based 3D renderings with solid models. For example, biochemists working with the SDSC TeleManufacturing Facility mentioned they got insights that they could not get from the screen-based 3D models alone, and concluded that "modern physical models are
<figure><img src="image_4.png" /><figcaption>Fig. 3: (a) Model of the black beetle virus from the SDSC TeleManufacturing Facility [7]; (b) A model of the HIV protease, a target for AIDS therapy from the Molecular Graphics Laboratory at the Scripps Research Institute [157]; (c) model of diffusion-tensor MRI brain data from the Visualization Research Lab at Brown University [2].</figcaption></figure>

Credits: (b) Courtesy Prof. Arthur J. Olson, copyright 2004 The Scripps Research Institute.

important tools that significantly extend the understanding of protein assembly
available from modern three-dimensional computer graphical analysis" [7]. Physi-
calizations are particularly easy to manipulate and rearrange, which is helpful in
chemistry for understanding molecule configurations. As we will further see in
section 8, studies have shown that 3D physicalizations can facilitate information
retrieval compared to 3D visualizations on computer displays [71].

3D-printed data physicalizations are also used outside chemistry. As a recent
example of data physicalization used for biomedical research, Thrun et al. [158]
generated physical landscapes of “pain phenotypes”, which are typical responses
patterns to different types of nociceptive stimuli. Their method involves a com-
bination of advanced data processing (dimensionality reduction, clustering and
classification) and color 3D-printing. According to the authors, “through its haptic
form, the 3D print makes high dimensional structures more understandable for
experts in the data’s field” [158].

## 2.3 Dynamic Physicalization for Analytics

A clear limitation of most 3D-printed data physicalizations is that they are static: they cannot be updated and cannot update themselves. There are however several ways this limitation can be overcome. One simple (albeit cumbersome) way is to use construction blocks that allow physicalizations to be updated at will. For example, business executives from General Motors and WellStar Health Systems use LEGOs to help them log, update, and explore data [173].

Meanwhile, there is ongoing research in trying to make data physicalizations that can update themselves. One simple approach to add dynamicity to physical artefacts, which has been commonly used in research on tangible user interfaces (TUI), is to combine physical objects with virtual displays [146]. For example, the Scripps Research Institute has extensively experimented with adding augmented reality overlays to solid molecule models for the purposes of chemistry research and teaching [43, 44]. With such systems, users can explore information about a
<figure><img src="image_5.png" /><figcaption>Fig. 4: (a) A proteine model (left) and the same model (right) with an augmented-reality overlay showing electrostatic field (small arrows) and the potential (red and blue clouds) [43] (b) The *Planwell* system for helping distributed teams collaborate during petroleum well planning and drilling: the left interface is the overseer's interface and consists of a solid terrain model with an augmented-reality overlay; The right interface is the explorer's interface, who sees the same information in-place [112].</figcaption></figure>

Credits: (a) Courtesy Prof. Arthur J. Olson, copyright 2004 The Scripps Research Institute.

molecule by holding a physical model and switching between different overlay representations (Figure 4-a). Such “hybrid” physical/virtual data representations have also been explored for industrial applications. For example, *Planwell* is an experimental system for petroleum well planning and drilling that uses a solid terrain model with overlaid visualizations in order to facilitate data exchange and collaboration between distributed teams [112] (Figure 4-b).

One common limitation of hybrid representations, however, is that most of the data is typically conveyed by virtual means (e.g., augmented-reality overlays), while the physical part (e.g., a molecule model or a terrain model) does not convey much data in the information visualization sense. Thus, these examples of data representations are more virtual than physical. In order to address these limitations, researchers have been recently investigating “pure” dynamic physicalizations, i.e., data physicalizations that can physically reconfigure themselves. But since dynamic data physicalization is a nascent stream of research with currently few real-world applications, we will cover this topic in section 7 on enabling technologies and in section 8 on user studies.

# 3 Physicalization for Communication and Education

Data physicalizations are often used in the context of data communication and education. While analytical use suggests that the represented data is as of yet unknown and meant to be analyzed for insights, use for communication implies that the represented data has already been analyzed and that the purpose is to communicate specific data insights. Similar to analytical applications, we are interested in clear encodings but here combined with storytelling elements guiding an audience towards the discovery of the insights that the designer wants to communicate [16,134]. Possible communication scenarios are diverse and include pedagogical contexts as well as collaborative decision making contexts,
for example when an analyst guides an audience of stakeholders through the findings of a data analysis.

## 3.1 Early Uses

It can be difficult to determine in hindsight whether a specific data physicalization was created for the purpose of analyzing data or whether it was created for the sole purpose of communicating or teaching data insights, independent of how that finding came to be. We therefore focus here on examples where the context strongly suggests that the main focus was on some form of communication. In particular, in the 18^th^ and 19^th^ century, the use of physical data representations and models was popular for science education purposes, to explain concepts to students or lay audiences.

<figure><img src="image_6.png" /><figcaption>Fig. 5: Examples for physical data representations and models used for communication in Mathematics and Chemistry. (a) George Adams’ Solid Geometry Models (1750) [129], (b) Dorothy Hodgkin’s electron density map and molecular model of penicillin (1945).</figcaption></figure>

Credits: (a) Boxed set of geometrical solids (models) by George Adams, c.1750. Whipple Museum of the History of Science, University of Cambridge. (b) Photo courtesy of Jon Agar.

For example, the instrument maker and science writer George Adams created and distributed solid models (Figure 5-a) to help explain solid geometry concepts. There exist some other examples of physical representations used to explain mathematical concepts [10], but they are much less numerous than in fields such as medicine or chemistry, where models are still in wide use today, both for analytical purposes (as we saw in section 2) and for communication and teaching purposes. Some of these models could be physically rearranged or taken apart [117] to explain, for example, spatial relationships between organs or reactions between molecules and atoms [52].

Besides such uses of physical representations for teaching purposes, some scientists also created physical representations of their findings for communication purposes. For example, Dorothy Crowfoot Hodgkin, who had worked out the three-dimensional structure of the penicillin molecule, created physical molecular models to demonstrate her findings to her colleagues (Figure 5-b).
<figure><img src="image_7.png" /><figcaption>Fig. 6: (a) A Moniac analogue computer with its interior visible. (b) A photograph by Davenport showing a histogram of men arranged by their height [29].</figcaption></figure>

Credits: (a) LSE Library, public domain.

Later examples, such as the Moniac, a hydromechanical analog computer (1949), were used to teach dynamic processes, here basic economic principles [12] (Figure 6-a). Its creator William Philips realized that flow metaphors were commonly used to teach economics, but had never been made physical [124]. The hydraulic computer embodies a set of equations to model an economic system. The variables of the model can be set by adjusting flow rates and the effect of such adjustments can be immediately observed in the waterflow. While Philips initially constructed the machine to better understand Keynesian economics himself, it turned out to be of great educational value and about 14 of these machines were constructed and used worldwide for teaching in the 1950s and 1960s [103].

Other data communication efforts were more directed at the general public. For example, the biologist Charles Davenport aimed to explain the concept of variation in evolution and basic statistical principles by sorting people and sea shells according to some property such as their size [29] (Figure 6-b).

## 3.2 Data-Driven Presentations

In more recent years, physical data representations have been a medium of choice for data-driven presentations. For example, Hans Rosling, a Swedish expert on global health and co-founder of the non-profit venture Gapminder, used physical blocks (see Figure 7) to explain data about world development issues. He often used everyday objects such as toilet paper rolls, jars of juice and milk, or LEGO blocks in his presentations [155], and once held a casual exposition by means of arranging and rearranging simple rocks, and humorously coined it “the shortest TED talk ever” [156]. Similar approaches have been taken by a YouTuber who used 10,000 pennies to explain the US federal budget [1], and by other educators who used LEGO blocks to explain a range of issues, including social mobility in America and the influence of federal taxes on income inequality [171], and the results of the 2015 federal election [109].
<figure><img src="image_8.png" /><figcaption>Fig. 7: Hans Rosling using physical boxes to explain changes in world population.</figcaption></figure>

Credits: (a) Still image from the video "Will saving poor children lead to overpopulation?" by Gapminder, CC BY 3.0. (b) Still image from the movie "Don't Panic: The Facts About Population" © Wingspan Productions Ltd. Used with permission.

A commonality in these examples is that the data representations are composed of unit elements, similar to Isotype visualizations [51] and to constructive visualization [64] (which will be discussed in subsection 5.3). A presenter manipulates these elements while explaining the data to underline the message they want to convey. The audience can observe in real time how the data changes following the actions of the presenter. While the audiences seem to appreciate these data-driven presentations, their effectiveness compared to the use of screen-based data representations in such a context has so far not been studied and presents a rich open field of research.

## 3.3 Self-Directed Exploration and Public Displays

Another form of data communication to large audiences takes place in science museums, galleries, and other public places showing data-driven installations. A difference to the aforementioned data physicalizations used in data-driven presentations is that the story to be told about the data needs to be integrated in the design of the data physicalization — generally no presenter is required to guide the audience through the data insights.

Many data-driven installations have artistic purposes and will be discussed in section 6. Among examples of data physicalizations mainly used for communicative purposes, a research project undertaken by Microsoft Research studied how the use of physical charts influenced citizen engagement in local politics [93,154]. They provided small electronic voting units to the inhabitants of a local road, and deployed physical bar and pie charts located in a public place to draw people in to observe and discuss the voting results (see Figure 8-a). No specific storytelling elements were used in this example, as the charts benefited from being situated in the area where the data was collected, and from the audience being already familiar with the type of data being shown, since they had voted on these questions beforehand. Another study investigated the differences of showing data via physicalization versus a common, touch-enabled display in a
<figure><img src="image_9.png" /><figcaption>Fig. 8: (a) The storefront where the physical charts of the Tenison Road Project where shown [93]. (b) The physical visualizations placed in public places for the study by Claes and Vande Moere [23].</figcaption></figure>

Credits: (a) Image of physical charts displayed as part of the Tenison Road project. Image courtesy of Microsoft Research and Alex Taylor.

public setting [23]. Passers-by of a busy street were able to select particular line charts depicting local statistics either via a touch screen or by manipulating Plexiglas plates that showed the same graphs. Such studies will be discussed in more detail in section 8, which covers user studies specifically.

## 3.4 Participatory Data Collection and Physicalization

Certain data physicalizations installed in public spaces further promote public engagement by going beyond mere consumption, and by allowing people to engage in the data collection and in the construction of the data physicalization itself. For example, artist Hans Haacke created in 1970 an exhibition at the Museum of Modern Art (MoMA) in New York where he asked a current socio-political question and provided two ballot boxes for visitors to vote on the issue [32]. The transparent boxes can be considered a physicalization: they functioned as physical bar charts giving an indication of the answer distribution.

Such physical surveys can take on a more complex form as illustrated by the project Cairn [46] (see Figure 9). This system was installed in a digital fabrication laboratory (fab lab) in a museum in Paris. The aim there was to replace questionnaires handed out to visitors to collect data about the activities in the fab lab. Each visitor was invited to construct a little stack of wooden tiles representing the different activities they engaged in, such as learning or teaching a skill. The placement on the table surface encoded how frequently they visited the space and how much time they spent there. A number of other examples of participatory data physicalizations exist, several of which have been created by the consultancy firm Domestic Data Streamers [31].

All such data collection methods result immediately in a data physicalization, which has advantages, for example, as they may improve the experience of answering the underlying survey [46] and may lead to higher participation rates. At the same time they may also result in limitations concerning the quality of
<figure><img src="image_10.png" /><figcaption>Fig. 9: Close-up of the Cairn table [46] showing individual data points (left), and overview of the Cairn table (right).</figcaption></figure>

the collected data: since all previous answers are visible to new participants, they
may influence answer behaviors.

4 Physicalization for Accessibility

Accessibility of data to the vision-impaired is a particularly promising application area for data physicalization. As before, we briefly overview early systems and then discuss recent developments in HCI and related areas.

4.1 Early Systems

<figure><img src="image_11.png" /><figcaption>Fig. 10: (a) An example of tactile graphics [37] and (b) a graph board [100].</figcaption></figure>

Credits: (a) Image from the Tactile Graphics Starter Kit, courtesy American Printing House for the Blind, Louisville, Kentucky.

Embossed paper has been used to convey pictures, maps and other types of graphics to vision-impaired people since the 1970s, particularly in schools [37] (Figure 10-a). Such haptic representations of 2D pictorial content are called *tactile graphics*. A comprehensive 1992 book by Polly Edman [37] explains how
to design tactile graphics, with extensive advice on how to reproduce classical statistical charts and thematic maps. Tactile graphics stand in a gray area between traditional data visualizations and physicalizations because they are constrained to a static 2D medium (paper), but they also extend slightly to the third dimension, allowing them to be sensed haptically. Tactile graphics is not limited to pure information consumption, as “drawing kits” can be used by blind students to create their own data representations [77,120].

Another method traditionally used in schools to teach mathematics and graphs to blind students is the *graph board*, which consists of a cork board with raised grid lines, on which students can place push pins and link them with rubber bands [100,120] (Figure 10-b). Like the tactile graphics drawing kits we mentioned before, graph boards allow students to construct tactile 2D graphs by themselves, with the difference that they are created by direct manipulation and can be freely rearranged once built.

## 4.2 Recent Developments

<figure><img src="image_12.png" /><figcaption>Fig. 11: (a) 3D-printed tactile graphics [18]; (b) Two extruded charts [61].</figcaption></figure>

While computer-supported tactile graphics require embosser machines that are not easily accessible to the general public, 3D printers now make it possible to rapidly reproduce existing tactile infographics at home and create original ones [18, 61] (see Figure 11-a). 3D printers are also able to generate more pronounced reliefs, which may facilitate graph reading [18]. More importantly, 3D printers make it possible to experiment with a range of form factors that go beyond tactile graphics. In particular, some research is underway suggesting that extruded 2D charts or 2D chart cutouts may convey data more effectively to blind users than tactile graphics because they tap into their ability to follow object contours [61,76] (see Figure 11-b).

Although many data physicalizations for the blind are passive objects, there has been recent developments in making systems that are more interactive. For example, the traditional graph board has inspired the design of an experimental system called the *Tangible Graph Builder*, which uses a combination of tangible object tracking and sonification to let blind students easily build 2D graphs such as line graphs and bar charts [100] (see Figure 12-a). Similarly, *Tangible Reels* is
<figure><img src="image_13.png" /><figcaption>Fig. 12: Two tangible charting systems for blind users: (a) Tangible Graph Builder [100]; and (b) Linespace [149].</figcaption></figure>

a system that helps blind users explore and construct networks of physical lines on a planar surface, and can be used for example to physicalize hierarchies or bar charts [35]. Finally, an experimental system called *Linespace* has been developed that can support advanced data exploration tasks through interactive tactile graphics: a custom 3D printing system equipped with a mechanical scraper allows blind users to produce or erase tactile graphics on demand using a combination of gesture and speech input [149] (Figure 12-b).

Despite some exciting developments in using data physicalization for the vision impaired, many opportunities for further research remain. For example, all the aforementioned approaches are based on 2D data representations that have been initially created and optimized for visual displays. Although giving access to standard charts is important, it is unlikely that 2D representations are fully tapping into human haptic perception abilities. Data physicalization makes it possible to convey data using richer physical representations that are fully three-dimensional and may have varying physical properties such as texture, softness, or temperature. However, these capabilities have not yet been used to empower vision-impaired users in their ability to explore data.

# 5 Physicalization for Self-Reflection and Self-Expression

Having covered the uses of data physicalization for the purposes of analysis, communication, education, and accessibility, we now discuss less pragmatic and more “casual” applications. In this section, we focus on the use of data physicalization for self-reflection and self-expression.

Our daily lives are becoming increasingly entangled in data, ranging from physical activity logs, over archives of social media posts, to any kind of digital resource pertaining to our hobbies. These data form a new potential to use in order to understand ourselves better and make positive changes in our lives [62]. Perhaps enabled by an increasing accessibility to crafting and fabrication tools, specific forms of physicalization have emerged that specifically focus on the expression of one's self through data, and related to that, to facilitating different types of personal reflection on one's behaviors, attitudes or opinions. While
such forms of “personal” visualization are not that uncommon in traditional
visualization [62], physicalization seems to encourage new kinds of data practices
in this realm, such as in terms of how people can be enabled to self-create data
representations for themselves, and in how the resulting physicalizations are then
appropriated to be carried or worn for one’s own benefit or for others to observe.

**5.1 Self-Reflection**

Data physicalization has been used to represent different forms of personal data,
such as to reveal and make sense of one's own actions, thoughts, or character.
Such acts of self-reflection often tend to be motivated by data practices like
the *quantified self* [151] or *personal informatics* [96]. By relying on a conscious
process of digitally capturing personal activities in order to examine or show
them in a wide range of private or social contexts, these data-driven forms of
self-reflection can lead to thoughts or actions towards self-improvement [22] and
can potentially lead to *digital epiphanies* like the acceptance or alteration of one's
own behavior [26].

<figure><img src="image_14.png" /><figcaption>Fig. 13: Two physicalization projects that were meant for self-reflection and self-expression, both depicting physical activity data: (a) the Activity Sculptures explore multiple and different object typologies, among which a piece of jewelry [142]; (b) SweatAtoms are simple objects that can be used as decorative objects for the home [80].</figcaption></figure>

While most research on personal visualization has focused on deploying
traditional data representations that are primarily designed to be looked at,
physicalization holds the promise to be more readily perceptible and might
therefore be more easily interpretable to their owners or the people in their
vicinity [97]. For instance, the Activity Sculptures project [142] mapped personal
physical activity data generated by running trackers unto the dimensions of
3D-printed symbolic artifacts like a human figure, a necklace, a lamp or a jar
(see Figure 13-b). In an attempt to control these resulting shapes, participants
deliberately considered aspects of playfulness and aesthetics to plan and then
perform their runs that ultimately generated those shapes. The SweatAtoms
project [80], shown in Figure 13-a, involved a more abstract translation of
personal physical activity data, heart-rate data and steps taken, which became
transformed into unique and sculpture-like objects generated by off-the-shelf 3D
printers. Personal physical activity data has also been visualized in terms of
the living conditions of a plant [15]. This project brought forward the sensitive
relationship between a physicalization and its data, as the owner is in the end
emotionally and morally responsible for the life of a plant through and by her
own activities.

## 5.2 Self-Expression

Data physicalization has also been used as a medium for self-expression. As any
kind of visualization of personal data only forms a limited representation of one's
body and self, the act of expressing one self to others via data reveals a particular
tension in the subjective perception of information privacy [139]. Receiving a
physicalization of one's self as a gift from others can thus reveal the potential
conflict between what people think of their self and what others are able to
find [78]. Proudly worn as a necklace, or put as a decorative piece of art on the
shelf at home, such kinds of physicalization can act as expressive or provocative
starting points of discussion with others. One of the earliest digital experiments
in data-driven self-expression is a *wearable visualization*, a device that slowly
generated dynamic folds in consecutive layers of fabric according to daily activity
sensor data [165]. Here, the main concern was to negotiate the level of privacy
between the wearer and onlookers: the display was meant not to be readily
understood by strangers, whereas friends and family were believed to learn how
the folds related to the data, and thus the wearer's state of mind. More recent
examples have explored the design space of *data jewelry* further, as Necklan [39]
conveyed particular facets of one's language knowledge and proficiency, with the
goal to encourage onlookers to get involved in conversations with the wearer.

Due to its relative abstractness yet physical and accessible nature, a data
physicalization can be appropriated in a variety of ways. Accordingly, some
physicalizations have been explicitly designed so to encourage varying ways of use
and ownership within people's life. For instance, facilitated by the possibilities of
modern digital fabrication technologies, the same set of physical activity data
has been represented by a diverse set of functional objects, such as jars, lamps,
necklaces or 3D figures [142] (see Figure 13-a). More abstract 3D-printed objects,
such as the SweatAtoms [80] that were shown in Figure 13-b, or the Fantibles
[79] that represent one's commentary about sports on Twitter along with the
uniqueness of each sports match, are more meant to be used as improvised
decorations within the home. Such practices however come at a cost, as for
example they create a lot of plastic objects which will likely be discarded sooner
or later. The users are unlikely to keep such activity representations for extended
amounts of time, as their home would become very cluttered with objects which
become meaningless as more and more of them appear.

Personal data physicalization clutter is less likely to occur if personal data
physicalizations are considered as unique and rare objects. Several projects have
investigated how data physicalizations meant as expressions of one self can be
worn as jewelry [49]. For instance, the company Meshu [102] renders a city or
places of personal value into a stylized physical mesh that can be worn as a
necklace, earrings, or cufflinks. More anonymous, yet still personally relevant, data
such as about air pollution has been turned into clothes [123] and necklaces [125].
People who wear such data physicalizations may wish to express the personal
relevancy of the data to immediate onlookers, but may also aim to experience
data in a more embodied way, such as in Posavec's pollution necklace [125], where
specific "dangerous" data values stimulate the bodily senses of the wearer.

5.3 Physicalization Making

<figure><img src="image_15.png" /><figcaption>Fig. 14: In the Data Things project, people were directly involved in the fabrication of physicalizations, hereby fostering different forms of social interaction and reflection [111]. In (a) participants visualized the sensor-logged movements of their crochet hooks as path-like patterns. In (b) participants were 'trying on' the resulting physicalizations.</figcaption></figure>

*Social Fabrication* [116], the sharing of digital fabrication activities with others, has been suggested to facilitate conversations and reflections about the created outcomes and their creation process. Moreover, people that partake in such activities tend to invest a certain meaning into the artifacts that they made themselves [113], which they might not do when confronted with something that was mass-produced. In the context of data physicalization, the process of making could equally provoke a certain level of curiosity, such as to find out what the fabricated shape will look like, and how it relates to the data [111]. Moreover, by lowering the accessibility for lay people to create visualizations, various collaborative behaviors could be supported, and as such lead to a shared, improved understanding of particular datasets or the abstract concepts that underlie them [74]. The Data-Things project, shown in Figure 14, gives an example of how the translation of data into a physical form can by itself be perceived as an opportunity for meaning-making and reflection, hereby suggesting a rich and largely unexplored potential for a much more active role of the user in how physicalizations are created, and in how their meanings emerge alongside [111]. By incorporating personal visualizations into everyday objects, mementos, keepsakes
and gifts have been created of which the owners feel they reflect upon their social relationships with others [159]. Moreover, the manual creation and subsequent habitual use of self-made physicalizations of personal data can augment their perceived value and authenticity, and can thus spark various forms of individual or shared recollection [159].

<figure><img src="image_16.png" /><figcaption>Fig. 15: Examples of constructive visualization, which allows lay people to construct graspable visualizations by arranging simple tokens [66].</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 16: Physicalizations can be authored by lay people through technological or manual means. (a) shows PhysiKit, a sensor-driven kit that makes data tangible in the home context [60]. Alternatively, (b) shows a rich collection of physicalizations that were manually made, yet also in the context of the home [160].</figcaption></figure>

Data physicalization can also be considered as a useful fabrication activity in and by itself, as its transformative qualities can become relatively easily integrated in existing fabrication and craft activities, such as crochet and its allied practices, among many others [111]. Several projects have demonstrated how physicalization can democratize the practice of information visualization by requiring less skills or expertise than its digital counterparts. For instance, the ideas behind *constructive visualization* focus on offering an easily and intuitively adaptable collage of small blocks akin to kindergarten play to allow non-experts to visualize their data using
easily manipulable tools [64, 66] (see Figure 15). Alternatively, physicalization authoring can be facilitated on a more technological level, as the Physikit [60], shown in Figure 16-a, combines various custom hardware and data configuration tools to allow users to capture and visualize environmental data through physical cubes that self-illuminate, vibrate, move or cause air flows.

The recent focus on making reveals how data physicalization can be understood as a manifestation of data within a “trajectory of use”, which starts from before the first acquaintance with the data, over its actual design and physical fabrication, and ending in the hands of others, in the future [111]. By considering data as a trajectory, one can recognize how people also make sense of data during the fabrication of visualization itself [159], as physicalization makers make conscious decisions about how the data should be represented in order to evoke associations with experiences or people, to support activities that allow for episodes of reminiscing, or to encourage potential social interactions. Moreover, during the physicalization construction itself, aspects of data collection, construction and self-reflection can be deeply intertwined [160] (see Figure 16-b for some concrete results). These qualities have been exploited during the process of hands-on physicalization making workshops in order to make principles of data representation more graspable [65], or to push the creativity of students in visualization [166]. Overall, research in personal physicalization fabrication has revealed a yet largely unexplored potential to represent more qualitative and subjective aspects that cannot yet be readily captured in data [160].

# 6 Physicalization for Enjoyment and Meaning-Making

When data physicalization is understood within the context of the information visualization discipline, the representation of data occurs mostly for pragmatic purposes. Yet broader conceptualizations like aesthetic [88], artistic [168], or casual visualization [126] have recognized how the use of visualization can also be exploited to *express* rather than purely communicate particular viewpoints that are related to data. As such forms of visualization tend to reach a much broader and larger, yet often less visually literate, audience, some researchers have focused on whether and how hedonic traits such as “fun” or “pretty” might actually become valuable and potentially complementary to the already existing pragmatic measures that benchmark visualization performance [48, 167]. For instance, the Promoter-Inhibitor Motivation Model (PIMM) revealed how a variety of promoting and inhibiting motivational factors tend to drive the “consumption” of visualization, how they change over time and hereby influence both the frequency and duration of visualization use [138]. Other research has explored the impact of visualization on recognition and recall [14] and persuasiveness [121], suggesting that visualizations that are memorable “at-a-glance” are also capable of effectively conveying the message of the visualization. Some consensus seems to be emerging that more abstract representations benefit task-performance and reflection, whereas more aesthetic, playful and pleasurable data representations are more able to “engage” people, an effect that can be further augmented by the
triggering of emotional commitments [15]. Therefore, both scientific and artistic research communities could greatly benefit from bridging the gap between them by opening a more multi-perspectival discursive space for future collaboration and exchange [75].

## 6.1 Ambient Visualization and Informative Art

<figure><img src="image_18.png" /><figcaption>Fig. 17: Examples of early ambient displays. (a) The Pinwheels installation presents information within a space through subtle changes in light, sound, and movement, which can be processed in the background of awareness [68,174]. (b) The BusMobile installation indicated how close each used bus was to the nearest bus stop by changing the height of the corresponding bus number [98].</figcaption></figure>

The research area of *ambient displays* has created perhaps some of the earliest examples of computer-enabled physicalizations. In a desire to move the access to information away from the computer display and into the everyday built environment, the most early prototypes of ambient displays experimented with translating data values into environmental aspects that were unobtrusive but still could be perceived via multiple human senses, including air displacement, ambient temperature, lighting effects; or more abstract animated shapes and patterns, air bubbles or actuated physical objects [68,174] (see Figure 17-a). A little later, early examples of *informative art* [58] mapped data as parametrically-controlled simulations of already well-known or newly invented art forms. It was believed that by embedding data into widely acceptable forms of art, a certain level of aesthetics and unobtrusiveness could be ensured that would allow data to be brought in and embedded in public contexts, such as entrance halls or living rooms [128]. The level of utility and in extension the usefulness of ambient display is challenging to ascertain [57, 58], insofar that such forms of visualization firstly
<figure><img src="image_19.png" /><figcaption>Fig. 18: Examples of plant-based physicalization. (a) Infotropism is a robotic plant that grows into the direction of one of two lamps, which turn on depending on the use of a recycling bin [59]. (b) PlantDisplay treats growth of plants as a way of information display, by connecting the watering and the lighting of the plant to data sources [87].</figcaption></figure>

need to convey “that something” is visualized, “what” is visualized and “how” it
is visualized. Once the “tyranny” of common digital displays is overcome, and
information can be visualized beyond the use of pixels, the creative design space
becomes immense [163]. Many ambient displays with obvious physicalization
qualities exist, of which the first ones include Breakaway [69], a subtly moving
sculpture that suggests people when it is time to take a break; BusMobile, a
installation in a university seminar room that moves physical tokens up and down
depending on the impending arrival of a bus nearby [98] (see Figure 17-b). Due to
their unobtrusive nature and common presence in public context, many ambient
display designers have experimented with the use of plants and their naturally
interpretable growth patterns to depict information [59, 87] (see Figure 18).

6.2 Artistic Expression

Visualization is also used well outside the scientific community, such as by individ-
uals and groups with a background in, graphic design, typography, architecture,
interaction design, media art, among many others [75]. Moreover, there exists a
wide designerly spectrum of visualization, ranging from pragmatic renditions of
data that can be recognized as being “visualizations” in that they are accessible
and readable, to more artistic renditions that are not meant for the user to be
readily understood, but rather aim to raise the awareness of some concerns that
underlie the data [88, 126, 168]. Naturally, similar practices can be recognized
within physicalization, among which those that revolve around more liberal and
abstract mappings that promote the – often artistically motivated – expression
of viewpoints related to data.

For instance, well before the term “data physicalization” was in use, the
term “data sculpture” was coined and defined as “*a data-based physical artefact,
possessing both artistic and functional qualities, that aims to augment a nearby’s
audience’s understanding of data insight or socially relevant issues that underlie*
<figure><img src="image_20.png" /><figcaption>Fig. 19: Physicalization as an artistic form of expression. (a) The different surface areas of the Data Table correspond to global wealth statistics. If the table would be flipped over, which is challenging due to its weight, the distribution of table surface is the same for each person [137]. (b) As an example of “data materialization”, a silver-plated 3D-print teapot that expresses the monthly finances of the artist was generated by an artistic workflow that focused on using an everyday object and a traditional material [140].</figcaption></figure>

Credits: (b) Photo © Courtney Starrett, used with permission. Work by Courtney Starrett, code by Susan Reiser.

it" [178]. Similarly, Loren Madsen, an artist and pioneer in the domain of data sculpture, defines data sculpture as a three-dimensional variety of "art whose form in large part is determined by data or information" [33]. Accordingly, many data physicalizations have been exhibited in musea as art installations, with the goal to expand the human experience of particular data-evidenced phenomena [72]. Meanwhile, the concept of "data object" frames the use of data as a "design material" that ultimately allows the development and fabrication of everyday objects that embody data [48,137]. For example, the physical dimensions and proportionality of a table (shown in Figure 19-a) can reflect certain statistical facts mirroring the inequalities in global health, while its important weight can represent the "difficulty to flip the economic system". The concept of "data materialization" [140] proposes a more liberal and computational interpretation of using data as a design material, as it explores how physicalization can be merged with generative art and physical craftsmanship to create functional objects that inherently represent concept-relevant data. In Figure 19-b, the shape of a silver-plated teapot was determined by autobiographical financial data to demonstrate the viability of a data-informed yet artistically-inspired workflow. Objektiviserung [85] framed a similar concept within the aspect of "neomaterility", i.e., the newly found ability of materials and their fabrications to incorporate networked digital technologies. Here, the resulting physicalizations are deliberately rough, abstract and "glitchy" in order to encourage the audience to reflect upon the crudeness of how data, algorithms and humans interact with each other. In contrast, some data physicalizations have deliberately used patina as a resource for aesthetic properties, in order to lend credibility and bestow feelings of "warmth" [91].
Many other examples exist that exemplify the creative power of physicalization for artistic expression, yet the vast majority did not appear in peer-reviewed articles within the HCI and visualization communities. For those, we recommend the list of physical visualizations and related artefacts [34].

## 6.3 Multisensory Expression

Through physicalization, data can be represented beyond the purely visual and tangible universe, to include our other human senses. While the actual feasibility of "touching, swiping, grasping, feeling, hearing, smelling, and even tasting data" [130] can only be imagined in the future, these visions are often based on the assumption that representing identical datasets through different modalities may support a different user experience and affective response [54].

Although most data physicalizations are meant to be explored primarily through sight, they can be designed to convey additional information or emotions through touch, and even audition [8] and taste [169]. In terms of mapping data unto taste, TastyBeats [81] creates a personalized drink both in color and taste, offering alternative ways of engagement with the data, while EdiPulse [82] creates 3D printed chocolates displaying cheerful messages. Both edible physicalizations make some kind of direct conceptual link with the data they attempt to convey, here to provide some kind of physiological reward after a physical activity. Yet, as the many examples from the Data Cuisine workshops compellingly demonstrate [28], the potential space for abstraction is vast, and potentially should consider both taste as its visual and olfactory presentation.

# 7 Enabling Technologies

Historically, data physicalizations were crafted manually (see examples in sub-sections 2.1 and 3.1), often requiring considerable time and effort to come into existence. Technology is now making them easier and easier to build, and makes it even possible to embed dynamicity and computation in the data physicalizations themselves. These trends parallel the history of classical (flat) visualizations, which were also crafted manually and used to take considerable expertise and time to produce. Computer software and printers facilitated the design and production of static visualizations and statistical charts, while graphical computers made it possible to create rich and powerful tools for interactive data exploration [20, 38]. Here we discuss similar enabling technologies for data physicalization, which are currently an active subject of research.

## 7.1 Digital Fabrication

The process of creating static data physicalizations has been dramatically sped up since the availability of digital fabrication technologies such as 3D printing and laser cutting. We already mentioned how these technologies have been quickly adopted by scientists for visualizing three-dimensional scientific data (see
section 2) and have been more recently used for a range of purposes, such as producing accessible graphs (see section 4) or giving physical form to personal data (see section 5). Presumably, the wide availability of digital fabrication technologies has also led to an explosion of data sculptures in the past ten years [34]

Digital fabrication machines facilitate the physical work and increase the accuracy of the resulting artifacts, but they require design files specifying what work a machine should be doing. These design files can be cumbersome to create manually and some work in HCI has focused on facilitating their creation by reading in data and proposing more or less parameterizable data physicalizations. Some systems are rather specialized and constrained in the types of physicalizations they allow to create, such as the physical activity physicalization tools from Khot et al. [80] and Stusak et al. [142] discussed in section 5 (see Figure 13). Lee et al. [91] modified existing artifacts, activity trackers themselves, to represent the data they had collected. Another rather specialized system is the one by Zhang et al. [177] for automating the creation of 3D-printed motion sculptures. MakerVis [150] is a bit more generalist in that it supports a small set of 3D charts inspired from classical statistical charts such as bar charts, line charts, and prism maps (maps where height of map areas encodes a data variable). MakerVis also supports different fabrication technologies, namely 3D printing, laser cutting, and machining. Meanwhile, Bader et al. [6] presented a fairly general method for physicalizing 3D scientific datasets using multimaterial 3D printers. Finally, as we already mentioned in section 4, Brown and Hurst [18] proposed tools to facilitate the 3D-printing of classical statistical charts for the visually impaired (see Figure 11-a). Despite all these tools, there is currently no general-purpose tool that can allow any user to easily physicalize any dataset, and thus, there is a vast open area for future research.

## 7.2 Actuation Technologies

While many of the examples mentioned in this chapter are not dynamic, that is, they are either immutable or rely on human manipulation to support interaction, the ultimate goal is for data physicalizations to be as interactive and responsive as screen-based visualizations. This vision was already presented by Ivan Sutherland in his 1965 essay "The ultimate display" [147]. At this time we are still far from achieving such an ultimate display but we are definitely getting closer. Within HCI, the sub-community working on shape-changing interfaces is mostly concerned with proposing technologies that enable the required computer-driven control of physical geometries or material properties [4, 127]. We discuss here a range of examples which each enable some form of actuation relevant for data physicalization, that is, which enables either positioning in space, control over an object's geometry or their material properties.

**Positioning in Space.** Physical positioning of multiple elements within 2D space has been a topic of interest for a while within the tangible interaction
community. The actuated workbench by Pangaro et al. [122] used magnetic forces to move objects on a tabletop surface such that their position corresponded to user actions. It required an array of electromagnets to generate the forces and could only move objects receptive to magnetic forces. Madgets by Weiss et al. [170] used a similar setup but enriched the controlled objects such that they themselves became interactive user interface widgets giving haptic feedback or indicating state changes. Major drawbacks of approaches relying on arrays of electromagnets is that they do not scale well to larger interaction spaces, and they require a considerable amount of energy released mainly as heat to operate the electromagnets. The Reactile system built by Suzuki et al. [148] improves considerably on these shortcomings but at the cost of speed, that is, objects move much slower and can only move in sequence, one after the other, which introduces a different scaling issue. For example, if the technology was used to display data as a 2D scatterplot, then a simple scaling action, which would require to update all data points, would take a long time to complete.

<figure><img src="image_21.png" /><figcaption>Fig. 20: (a) Constraint-based interaction with Zooids robots [89]; (b) Different data points floating freely in 3D through acoustic levitation [119].</figcaption></figure>

The use of small robots such as the Zooids by Le Goc et al. [89, 90] addresses these issues by relying on fast independent wheeled robots controlled through the use of structured light (see Figure 20-a). The system scales better than those relying on special surfaces, as each robot knows where it is and where it should be. The control through structured light enables their use on arbitrary surfaces and even their combination into more complex 3D objects [179]. The remaining bottleneck is that one robot is needed for each data point to represent.

Free positioning within 3D space is more challenging as the system has to constantly overcome gravity. The two main approaches that have been explored so far are the use of acoustic levitation (using standing waves) to enable floating microparticles [119] (see Figure 20-b), and the use of drones to enable the control of larger scale physical voxels [45]. Both approaches are subject to physical limitations due to interferences when particles or voxels are in close proximity to each other.

All such position-control technologies are useful for implementing composite data physicalizations, which Le Goc et al. [90] define as data physicalizations
"made of multiple elements whose topology can be reconfigured, or can reconfigure itself". Le Goc et al. [90] define three dimensions for characterizing composite data physicalizations: i) their *level of granularity* refers to the number of elements they are composed of; ii) their *degree of manipulability* refers to the extent to which their elements can be manually rearranged; and iii) their *level of actuation* refers to the extent to which their elements can rearrange themselves without human intervention. They note that an ideal composite physicalization should be able to support a high level of granularity, a high manipulability, and a high degree of actuation, but that no such system exists yet.

**Control over Shape and Geometry.** Position control of independent objects is not the only way to represent data physically. For example, with the use of 2.5-D pin array displays (as shown in Figure 21), interactive 3D bar charts and surfaces can be realized [152, 153, 152, 151]. These displays are able to react quite quickly to user or system commands but are fixed in their basic geometry, that is, the pin layout is fixed and cannot easily be extended or rearranged. Furthermore, pins cannot be manipulated sideways. Thus their degree of manipulability is low [90]. Single element approaches such as ShapeClips by Hardy et al. [50] enable flexible layouts although at the cost of miniaturization and speed. As with position control, achieving scalability remains an open challenge.

<figure><img src="image_22.png" /><figcaption>Fig. 21: (a) The EMERGE shape display featuring 10 by 10 bars [152]; (b) The inFORM shape display featuring 30 by 30 bars [41].</figcaption></figure>

While shape displays based on linear pin modules are best suited to realize data physicalizations resembling bar charts, many shape-changing systems have been proposed which transform their shape in a variety of ways. For example, Morphees [133] discuss 10 features of *shape resolution* such as curvature, amplitude, or zero-crossing which can be exploited to represent data in new ways. Most shape-changing interfaces focus on one specific shape manipulation, for example, LineForm controls the shape of a line [107] while TiltDisplays [3] use actuation to reorient display surfaces arranged in arrays. Tiltstacks [161] combine linear actuation and tilting to enable actuation with more degrees of freedom. The
combination of degrees of freedom is especially promising for data physicalization, but has so far not been explored. Shape-changing interfaces are still facing great challenges [4] and as research progresses, it will greatly enrich the diversity of possible actuated data physicalizations.

**Control over Material Properties.** Besides position and geometry, data can also be encoded in material properties such as roughness, weight, or reflectivity. Controlling material properties is in most cases non-trivial, especially if one wants to control multiple properties at the same time. Nonetheless, solutions have been proposed to computationally control certain material properties such as stiffness. For example, Jamming User Interfaces [40] relies on a hydraulic control to jam particles into different shapes and control their stiffness, and Materiable [108] simulates different levels of stiffness using a pin array. Niiyama and colleagues explored how to control weight and volume changes by pumping liquid metal into interior channels of an object [110]. Meanwhile, PneUI [176] uses pneumatic means to enable texture changes through actuated air bubbles of different sizes. Metamaterial textures [67] also enable the control of textures but through purely mechanical means, that is, a special design of 3D printed interior cells allow people to modify the texture of a 3D printed object.

## 7.3 Specialized Applications for Creating Data Physicalizations

Most of the systems discussed above enable some form of actuation potentially useful for the purposes of data physicalization, but do not explicit target this application area. Here we discuss a few systems which principally target a specific type of data physicalization and are meant to facilitate their realization.

<figure><img src="image_23.png" /><figcaption>Fig. 22: A Vol²velle using a transparent medium to show different layers of a scan [141].</figcaption></figure>

As we already discussed in subsection 7.1, several software tools have been proposed to facilitate the use of digital fabrication machines such as 3D printers and laser cutters to create static data physicalizations. Other tools exist that
help users create data physicalizations with other technologies. For example, Vol²velle [141] is a software tool intended for data communication purposes, and which transforms screenshots from an interactive volume visualization system into a printable document with cutting and assembly annotations for making a volvelle (a paper construction with rotating parts) (see Figure 22). Physikit, which we already mentioned in subsection 5.3 on Physicalization Making, lets users create ambient dynamic physicalizations. It consists of a set of blocks which each support one type of actuation such as producing air flow, light, movement, and vibrations (see Figure 16). The elements were used in combination with a citizen science project where data types such as temperature or air quality from an environmental sensor in people's home could be linked to each of these blocks [60].

# 8 Studies of Data Physicalization

A vast body of research in educational sciences, developmental psychology and tangible user interfaces have suggested that the examination and manipulation of physical objects can promote understanding and learning [118,135]. Here, we focus exclusively on studies involving physical *representations of data* as opposed to, e.g., physical models or tangible controls. Although some of the systems discussed here have been already mentioned before in this chapter, we focus now on research questions and findings. Studies can be roughly classified into *comparative*, where physicalizations are compared with non-physicalizations, and *non-comparative*, where physicalizations are studied in isolation.

## 8.1 Comparative Studies

In his unpublished doctoral dissertation, Dwyer [36] presented participants with time-evolving country indicator data conveyed either as a visualization printed on regular paper or as a space-time cube physicalization [5]. He asked participants questions about the data and found that the physicalization facilitated some tasks but not all. However, the purpose of the study was not to compare physicalizations with non-physicalizations but to compare 2D with 3D data representations. Since the two conditions used a different encoding, the study does not allow to conclude about the effect of physicalizing data.

The first study to compare data physicalizations with equivalent visualizations was probably by Jansen et al. [71] (see Figure 23-a). They had participants perform elementary information retrieval tasks on interactive 2D matrix visualizations, screen-based 3D bar charts (both monoscopic and stereoscopic) and equivalent physical 3D bar charts. They found that participants performed best with the 2D visualizations, but when a 3D representation was used, the physical condition outperformed even the stereoscopic display. A second experiment suggested that the likely reason was that physicalizations allowed participants to quickly and efficiently “bookmark” data points (bars) of interest with their fingers. Bookmarking data points reduced the cognitive effort required to accomplish
some of the tasks, and although the screen-based visualizations let users highlight data points by clicking on them, it was presumably not as efficient as using multiple fingers on physical objects. The authors also speculated that the higher visual realism of the physical objects (e.g., their perfect rendering of depth cues) might have made them easier to read. However, a 2017 study by Bérard and Louis [11], although not specifically on data physicalization, challenged the notion that visual realism is always desirable. The study asked participants to solve 3D puzzles on a 2D screen, on a high-performance handheld perspective corrected display (HPCD), and on a physical model. The physical model was used as a “gold standard” control condition but it happened to be inferior to all digital conditions, likely because of visual legibility issues: inhomogeneous shading of the object and, most importantly, reflections resulting from its acrylic casing. Such issues were easy to overcome or simply non-existent on the computer-rendered 3D model. Meanwhile, Stusak and Aslan [143] noted that data physicalization permits more complex designs than the basic 3D bar charts used in Jansen et al., and compared two physical star plot designs with two 2D interactive visualization designs in their ability to support information retrieval tasks. However, no advantage in performance was detected and participants reported finding the physicalizations harder to use, partly due to their lack of affordances and their fragility. The authors concluded that for physicalizations to be usable, it is important to get their physical design right.

<figure><img src="image_24.png" /><figcaption>Fig. 23: (a) Comparison of a screen-based 3D bar chart with the equivalent physicalization [71]; (b) Comparison of a 2D paper-based visualization with a 3D physicalization made of standing wooden blocks [144].</figcaption></figure>

Some comparative studies were also conducted on memorability. An early study by Cockburn and McKenzie [25] compared six different versions of the document management technique Data Mountain [131] in their ability to support spatial memory: a 2D, 2.5D and 3D version were implemented both on a computer and as a physical version using paper printouts and fishing lines. Although this was not strictly a data physicalization study (participants were asked to recall and relocate windows), it is noteworthy that the 3D condition performed worse than 2D overall, but its physical implementation outperformed the screen-based
implementation. More recently, Stusak et al. [145] showed participants either a 2D bar chart on a computer display, or an extruded 2D bar chart made of laser-cut acrylic. They then asked them to recall facts about the data right after the study and after two weeks. Participants who saw the extruded bar chart were able to better remember facts about extrema values after two weeks, although they also spent slightly more time looking at the physicalization on average. Similarly, Stusak et al. [144] had participants assemble a 2D paper-based visualization and a 3D physicalization made of standing wooden blocks (see Figure 23-b), and gave them data retrieval tasks. They then asked them to recall facts right after the study and after a week. Again participants recalled extrema values better when shown with the 3D physicalization, presumably because of the “vivid physical height of the bars” [144]. Both experiments tested implicit memory, i.e., participants were not told in advance that they would be tested for recall. Although these findings need replications to be definitely confirmed, they provide suggestive evidence that data physicalizations can help people memorize some facts better than flat visualizations shown on paper or on computer screens.

Moving beyond pure performance metrics, Hogan and Hornecker [54] compared people's affective responses when shown data with a regular bar chart vs. tangible data-driven artefacts. They used the Repertory Grid Technique (RGT) to elicit thirteen personal constructs (e.g., instinctual-cerebral, artificial-organic, fun-dull) that capture differences in how people experience the data on an affective or hedonic level. However, the two tangible data-driven artefacts were not strictly physicalizations in that they did not convey data using physical properties or geometry, but using computer-driven audio and haptics. Later, Hogan and Hornecker [55] conducted a similar study with three simplified designs and derived constructs to characterize the subjective impression elicited by data values conveyed through visual, audio and haptic modalities. Claes and Vande Moere [23] examined how physicalization affects engagement using a “controlled in-the-wild” method. They deployed three versions of a data exploration device in public streets: one virtual (digital display + touch interaction), one physical (physicalization + tangible interaction, see Figure 8-b) and one mixed (digital display + tangible interaction). The device allowed passersby to explore the evolution of different local dimensions such as birth rate, unemployment, CO2 emissions or traffic accidents. Observations and interviews suggested that the ability to grab, share and investigate the physical line graphs lead to more playful, collaborative and active forms of information discovery.

Finally, one comparative study examined the construction, rather than the consumption of data representations. Wun et al [175] ran a qualitative study comparing the use of spreadsheet software to the use of tangible tiles for authoring bar chart visualizations (see Figure 24). They found that while the spreadsheet software imposed a certain workflow in the authoring process, the wooden tiles allowed more freedom in exploring different encodings and layouts. People also spent time exploring the spreadsheet tool itself, while this tool familiarization process was not necessary with the wooden blocks. While these observations
<figure><img src="image_25.png" /><figcaption>Fig. 24: A bar chart made with a spreadsheet software and a bar chart made with tangible tiles [175].</figcaption></figure>

emphasize qualities unique to physical objects, the authors also discuss how they
can be used to inform the design of better software tools.

8.2 Non-Comparative Studies

Among the studies that investigated physicalizations alone, some involved showing
physicalizations to people and recording their impression or their opinion. For
example, Gwilt et al. [48] showed examples of data sculptures to people from
different communities (engineers/scientists, designers, and the general public),
and interviewed them about the potential of this medium to communicate data.
Despite some initial skepticism particularly among engineers and scientists,
participants commented on the usefulness of having a direct tactile experience
with the objects, and the fact that they were more likely to stimulate discussions
than regular visualizations. Engineers and scientists expressed a preference for
more analytical data representations, while designers preferred more open-ended,
experiential representations. More recently, Boem and Iwata [13] designed a shape-
changing display for conveying vital signs of a hospitalized person to remote
relatives, and exhibited it during two major Media Art festivals. They observed
peoples' spontaneous interactions with the object and conducted interviews
suggesting that the object is reliably experienced as a living organism, elicits
both positive and negative affects, but its movements can be hard to interpret.

Other studies, generally conducted in the lab, involved more extensive use of
the physicalizations. For example, McGookin et al. [100] evaluated the usability of
their Tangible Graph Builder (see Figure 12-a) by giving chart construction and
chart reading tasks to both sight-deprived and blind users. They observed several
advantages provided by the use of tangible objects such as “two-handed interaction,
quick overviews, spatial frame of reference and flexibility to employ fingers for
marking” [100]. More recently, Taher et al. [152] conducted an observational
study to gain insights into which types of interaction techniques participants
prefer when interacting with a tabletop actuated bar chart. They found that
people generally preferred to directly interact with the bars (pushing and pulling
them) although these gestures do not naturally map to all functions (for example
scrolling within a larger data set). In a follow-up study, Taher et al. [153] asked participants to carry out data exploration and data presentation tasks on the same actuated bar chart. They analyzed hand actions and body movements, and found much variation between participants in their data exploration behavior, with some exploring very little of the available data. They concluded that overview techniques are necessary when displaying large data sets on a much smaller shape display. Meanwhile, Daniel et al [27] evaluated the actuated ambient data physicalization CairnFORM (see Figure 31-a) by asking exhibition visitors to carry out elementary tasks similar to Jansen et al. [71], and found people to be accurate except with the sorting task. They then conducted a lab study to assess people's ability to perceive and memorize motion sequences shown in their peripheral vision. Of the three motion types tested, a slow then accelerating motion was found to be the best. While all of these studies investigate specific physicalization designs, a study from Jansen and Hornbæk [73] investigated in the tradition of graphical perception studies (cf. [24]) how accurately people can judge the size of physical marks, that is, building blocks of data physicalizations. They showed participants two differently sized bars or spheres and asked them to indicate the size of the smaller one relative to the larger one (see Figure 25-a). They compared their findings to those of previous studies on graphical perception and report that physical bars are judged with a just slightly higher error than 2D bars. For spheres, they found that the systematic error reported by previous studies can be considerably reduced by encoding data relative to the surface of spheres instead of their volume.

<figure><img src="image_26.png" /><figcaption>Fig. 25: (a) setup of a psychophysical experiment on the judgment of physical size [73]; (b) five data sculpture designs used in a study on personal activity data physicalization [80].</figcaption></figure>

Several studies were longitudinal studies requiring people to engage with
physicalizations over the course of several days. Khot et al. [80] conducted a field
study investigating the impact of 3D-printed sculptures of personal activity data.
They equipped several households with a heart rate monitor, a 3D printer, and an
application supporting five different types of data sculptures (see Figures 13-b and
25-b). The authors found that the sculptures made participants more conscious
of their physical activity and acted as personalized rewards and memorabilia. Participants' favorite design was a frog whose size is proportional to the level of physical activity. As a follow-up, Stusak et al. [142] investigated 3D-printed sculptures of personal running data (see Figure 13-a). Participants first chose one among four designs, all of which required assembling multiple pieces, one for each run. After each run, a piece was 3D-printed by the authors and handed to the participant after one to three days. Based on observations and interviews, the authors discuss the many trade-offs behind different designs, including aesthetics, degree of abstraction, legibility, and practicality. The preferred design was a necklace with beads of different shapes representing different runs. Later, Khot et al. [79] ran a field study with cricket fans, where they gave participants 3D-printed objects whose shape was determined by tweets they made while watching cricket games. Taylor et al. [154] report a year-long study where they had residents from a street engage with community data using various technologies, including a dynamic physical chart acting as a public display (see Figure 8-a). The study however does not discuss the specificities of the physicalization medium, but instead examines the relationships between people, places, and data on a general level.

As with comparative studies, some studies focused on the construction of physicalizations. In an early study by Vande Moere and Patel [166], design students were asked to convey a data-driven insight through a data sculpture. The authors analyzed the 20+ data sculptures produced, allowing them to articulate qualities unique to physical representations as well as the different design strategies available (see section 9 for a more extensive discussion). Huron et al. [66] conducted an observational study to investigate how participants spontaneously use colored wooden tiles to communicate data to peers (see Figure 15). They found that assembling physicalizations using physical tokens encourages experimentation and makes it easy to try out different ways of encoding data to optimize communication of insights, but that tokens are also cumbersome to update. In a field study, Thudt et al. [160] taught participants how to manually craft data physicalizations with materials such as beads or plasticine, and then asked them to record personal data of their choice on a daily basis (see Figure 16-b). Through a diary analysis and interviews, they observed that the exercise promoted self-awareness and sometimes behavior change, and found that the iterative construction of personal data physicalizations allowed the processes of data collection, visualization and self-reflection to be deeply intertwined.

# 9 Models and Theories

Although physicalization practice has existed over many decades, academic research has only recently started synthesizing knowledge into models and theories. We briefly review this stream of work in this section.
## 9.1 Pipeline Models

A commonly used model of information visualization is the “information visualiza-
tion reference model” or “infovis pipeline”, which sees information visualization
as a process of turning raw data into a picture through a series of successive
transformations [21,70]. Although useful as a descriptive model of computer
visualizations, this model is purely logical and does not account for the physical
world, thus it does not capture data physicalization.

<figure><img src="image_27.png" /><figcaption>Fig. 26: Extended infovis pipeline that captures data physicalizations [70].</figcaption></figure>

One proposed extension to the standard model consists of adding a “rendering” step at the end of the pipeline, which turns the visualization picture or model into a concrete physical entity called the “physical presentation” [70] (Figure 26). For example, there are many ways a particular 3D molecule visualization can be brought into existence in the physical world, including by projecting it on a flat computer display, by printing it on a sheet of paper, by 3D-printing it, by showing it with a hologram, etc. The 2D projection on the computer display, the 3D-printed object, etc., are all different types of physical presentations. The extended pipeline of Figure 26 also adds subsequent transformations that account for how the visualization is perceived and interpreted by the end user, starting from the “percept transformation” which turns the physical presentation into a percept. The notion of percept transformation makes it possible to capture interactions that take place in the physical world, such as a user inspecting a handheld physicalization by turning it around, or a user physically walking into a room-sized physicalization [70]. The extended infovis pipeline can also be used to capture how data physicalizations are created and updated. The transformation steps leading to the physical presentation can be either fully automated (e.g., all transformations are made by a computer program and propagated to a 3D printer or a shape display), fully manual (e.g., all calculations are done by hand and the physicalization is manually crafted), or anywhere in-between [70,72,150].

A further extension was proposed to capture situated and embedded data
representations [172], which can be either physicalizations or screen-based visual-
izations. The diagram in Figure 27-1 reproduces part of the pipeline of Figure 26,
with some differences. First, some stages (from processed data to visual presenta-
tion) are collapsed for simplicity. Then, a new element, the *physical referent*, has
<figure><img src="image_28.png" /><figcaption>Fig. 27: (1) pipeline for situated and embedded data representations (adapted from [172]); (2) a shower with heat-sensitive color-changing tiles is an example of an embedded physicalization [104].</figcaption></figure>

been added (b). The physical referent is “*the physical object or physical space to which the data refers*” [172]. For example, if the data is the power consumption of a consumer device over time, the physical referent is the device itself. When the distance between the physical referent (b) and the physical presentation (a) is small (c) – for example, device consumption is visualized on the device itself, the visualization can be considered as *situated*. Situated visualizations allow for a form of interaction that classical visualizations do not support, and that consists of manipulating the physical referent to change the data (d) – for example, tuning a consumer device to observe effects on consumption. When a visualization consists of a combination of many instances of the pipeline of Figure 27-1 – for example when a small visualization is placed on every device in the house, the entire visualization can be considered as *embedded* [172]. Figure 27-2 shows an example of a highly-embedded physicalization of temperature data, where each point on the wall can be conceptually seen as implementing the situated visualization pipeline of Figure 27-1.

## 9.2 Data Embodiment and Contextual Sensemaking

Data physicalization promises to facilitate more elaborate forms of sensemaking that go beyond those used in traditional forms of information visualization [56]. For instance, it has been repeatedly argued that physicalization potentially offers more freedom for encoding information in perceivable form, as it can exploit a wide variety of physical, material and tangible characteristics, and can allow humans to explore it using all their sensory and motor skills [53, 72, 95, 163, 174]. As most common information visualization techniques are purely visual and two-dimensional in nature, and are rendered on paper or digital display media, one can easily imagine that no real compelling reason exists as to why the physical mapping of data should necessarily mimic these techniques. Our everyday experience of physical reality seems to offer much promise to map data in ways
<figure><img src="image_29.png" /><figcaption>Fig. 28: A model of embodiment in data sculpture, mapped along two characteristics: the metaphorical distance from data and the distance from reality [178].</figcaption></figure>

that are more accessible and intuitive to our human mental image, preconceptions or expectations. In other words, there exists a design space that relates to the presence, materiality and situatedness of a physicalization in that it “embodies” data [178] rather than that it just communicates data by its physical dimensions. This design space potentially allows for the more hedonic senses and interaction modalities to become stimulated in purposeful ways [53].

Early theoretical models of physicalization, then tentatively called “data sculpture”, recognized the potential of combining functional with artistic qualities. For instance, the concept of “embodiment” was proposed to capture how a physicalization typically conveys data to its audience via the use of a metaphor that is linked to the meaning of the data it depicts, as well as the perceived reality surrounding the physicalization itself [178]. For instance, while a particular monetary value can be easily represented “abstractly”, such by the height of a water fountain [101] which may not necessarily be easily understood by onlookers, it can equally be represented in a more “literal” way, such by a heap of currency coins that together exactly correspond to that monetary value. Yet to emphasize a particular meaning, the same monetary value could even manifest itself by a
<figure><img src="image_30.png" /><figcaption>Fig. 29: Different forms of abstraction through physicalization [166]: (a) *symbolic* physicalization of the unemployment rates between males and females in Australia as an abstract helix; (b) *iconic* physicalization of the relative growth of the Australian economy, resulting in an implicit imbalance of a flower shape; (c) *indexical* physicalization showing the dangers associated with wearing thongs by a series of physical thongs that were pierced with a nail for every thousand thong-related injuries that required hospitalization.</figcaption></figure>

<figure><img src="image_31.png" /><figcaption>Fig. 30: Different combinations of physicalization strategies [166]: (a) the number of perforations in sequential paper filters impact the fabrication of the physicalization, so that fabrication process rather than the end result encodes the information; (b) the height of the keys depict their relative usage, which thus mapped or embedded the statistical data literally ‘unto’ the object it analyzes; (c) a real puzzle that depicts the relative use of Tetris puzzle piece in itself, thus depicting the meaningful aspects of data and play in one.</figcaption></figure>

seemingly minuscule nugget of real gold. Shown in Figure 28, a two-dimensional design space was proposed to capture how different physicalizations position themselves in terms of their metaphorical distance to data, and to reality.

Illustrated in Figure 29 and Figure 30, the design space of data embodiment was also articulated by analyzing a collection of physicalizations in the context of a visualization course, which then resulted in a semiotic model consisting of three design strategies – *symbolic*, *indexical*, and *iconic* and their pairwise combinations. For instance, a physicalization can be designed by an abstracted mapping that needs to be learned to be understood, or is only metaphorically linked in terms of shape or material to the meaning of the data [166]. Moreover, a physicalization can also convey data by its *fabrication process* itself. Then,
the data that is represented becomes encoded by *material operations*, such as the printing, cutting or milling of the physicalization, rather than its physical manifestation itself (see Figure 29-a for an example).

<figure><img src="image_32.png" /><figcaption>Fig. 31: Two examples of abstraction towards a similar goal: (a) CairnForm is an abstracted ring chart that conveys rich patterns regarding energy-related usage patterns [27]; (b) A power-illuminated energy cord offers less insights, but might be more intuitively understandable and actionable [47].</figcaption></figure>

While a traditional data visualization infers some form of data abstraction and dimension reduction, this operation is not necessarily easily replicable in the physical world. Consequently, a more accessible metaphor might negatively impact the resolution of the data shown, such as the number or dimensions of data values it is able to convey. For instance, whereas an actuated physical ring chart [27] allows for detailed insight creation regarding complex energy-related usage patterns (Figure 31-a), its symbolic abstraction requires some guidance to be interpreted. Similarly to ambient displays, such a symbolic visualization first needs to understood as being a visualization, and then as representing content that is related to energy [57]. In contrast, the small metaphorical distance [178] of a power cable that self-illuminates depending on its actual energy use [47] is probably more accessible and intuitive to understand, yet only carries a single quantitative value (Figure 31-b).

Based on a study of representative exemplars, the parameters that physicalizations tend to deploy to convey information has been identified and captured in a comprehensive framework, shown in Figure 32 [53]. Notably, the resulting complementary design space of physicalization includes the purposeful combination and crossing of human modalities that stimulate our human senses, the materials from which the physicalization is produced, as well as the tacit, human experience they are meant to generate [56]. In addition, the framework calls for articulating and benchmarking the interaction between people and physicalization, in terms of provoking passive or active forms of user engagement, as well as whether and how the data itself is static or dynamically changing.

The area of data physicalization can also learn from emerging discussions in computational design and fabrication [63]. For instance, in architectural design,
<figure><img src="image_33.png" /><figcaption>Fig. 32: A parallel coordinate plot of 18 different physicalizations with a focus on their multisensory capacity, organized across three design space dimensions [53].</figcaption></figure>

it has been argued that the designerly negotiation between structure and material production, between diagram and model, and between how something works and what it looks like, often determines the actual outcome. Likewise, the design rationale of a physicalization could be determined by reasoning more holistically about how the material choices, fabrication methods and structural aspects link its physical components to the intrinsic structures present within the data.

Lastly, the phenomenon of how some physicalizations are able to convey their meaning through their environmental context has been analyzed from various perspectives. As we saw in subsection 9.1, embedded data representations combine the visual and physical representation of data in ways that are deeply integrated with the physical spaces, objects, and entities to which the data refers [172]. When the corresponding physical data referent, i.e., the real-world entity or space to which data corresponds, is in close proximity, the visualizations are considered to be situated, whereas spatially coinciding referents determine embedded representations. A so-called ‘indexical’ visualization [115] makes use of a referent that can be causally linked to its fabrication, as it achieves meaning by emphasizing a particular causal link between its own embodiment in relation to its immediate surrounding environment, and as such can become practically ‘invisible’. Similarly, various ‘invisible’ data mapping metaphors and operations [114] in physical space have been identified, such as those that exploit particular qualitative characteristics of physical phenomena, and the relationships between them. The embedded physicalization of Figure 27-b is one such example. More recently, so-called qualitative displays [95] equally aim to exploit our natural
ability to interpret real-world contexts, in order to subtly yet naturally augment
digital interfaces with meaning [94]. Plant-based physicalizations, mentioned in
subsection 6.1, provide good examples. For instance, the natural size and relative
direction of a plant's growth can be interpreted as an alive physicalization of
its own environmental conditions [83]. This natural understanding has already
been exploited by the plant-like robotic contraption Infotropism [59] (Figure 18).
By deconstructing the processes and the level of involvement of a natural or
deliberate phenomenon in fabricating such indexical physicalizations, these kinds
of research highlight the yet unexplored potential width and breadth of data
abstraction in physicalization practice and research.

10 Conclusion

This chapter provided an overview of data physicalization, a vast research area that studies the use of “physical artifacts whose geometry or material properties encode data” [72]. This area overlaps with a number of disciplines and research areas including information visualization, scientific visualization, visual analytics, tangible user interfaces, shape-changing interfaces, personal fabrication interfaces, as well as graphic design, architecture, and art.

We went through different application areas of data physicalization by dis-
cussing how it has been used for analytical purposes, for communication and
education, for accessibility, for self-reflection and self-expression, and finally
for enjoyment and meaning. We also discussed enabling technologies, reviewed
empirical studies, and summarized models and theories of data physicalization.

Data representations that are optimized for flat media such as paper and computer screens are extremely useful and effective in many situations, and will undoubtedly remain in use. However, for too long, visualization research has exclusively focused on such representations. Even historical accounts trace back the first visualizations to hand-drawn visualizations on paper, ignoring the physical data representations that have been in use for many centuries, and for some of them, were in use well before paper was even invented [34]. The aim of this chapter was to restore the balance by offering an overview of the rich possibilities offered by physical representations of data, as well as the many pending questions and challenges.

Since the advent of graphical computers, interactive computer displays came
to dominate static media such as paper and physical models for representing
data. But now, with the advent of digital fabrication and physical actuation
technologies, we are entering an era where the production of data physicalizations
is becoming cheap and easy, and where computation and automation can be
embedded in data physicalizations in a way that will eventually make them as
dynamic, interactive and powerful as today's interactive visualization software.
However, much research is still necessary to better understand the design space of
data physicalizations, the processes involved in consuming and producing them,
and their benefits compared to flat representations or virtual renditions.
References

1. 10000Pennies: Obama budget cuts visualization (video). https://www.youtube.com/watch?v=cWt8hTayupE (2009), accessed: 2018-10-13

2. Acevedo, D., Zhang, S., Laidlaw, D.H., Bull, C.W.: Color rapid prototyping for diffusion-tensor mri visualization. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 1076–1078. Springer (2004)

3. Alexander, J., Lucero, A., Subramanian, S.: Tilt displays: designing display surfaces with multi-axis tilting and actuation. In: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services. pp. 161–170. ACM (2012)

4. Alexander, J., Roudaut, A., Steimle, J., Hornbæk, K., Bruns Alonso, M., Follmer, S., Merritt, T.: Grand challenges in shape-changing interface research. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 299. ACM (2018)

5. Bach, B., Dragicevic, P., Archambault, D., Hurter, C., Carpendale, S.: A review of temporal data visualizations based on space-time cube operations. In: Eurographics conference on visualization (2014)

6. Bader, C., Kolb, D., Weaver, J.C., Sharma, S., Hosny, A., Costa, J., Oxman, N.: Making data matter: Voxel printing for the digital fabrication of data across scales and domains. Science advances 4(5), eaas8652 (2018)

7. Bailey, M.J., Schulten, K., Johnson, J.E.: The use of solid physical models for the study of macromolecular assembly. Current opinion in structural biology 8(2), 202–208 (Apr 1998), https://www.ncbi.nlm.nih.gov/pubmed/9631294

8. Barrass, S.: Digital fabrication of acoustic sonifications. Journal of the Audio Engineering Society. Audio Engineering Society 60(9), 709–715 (2012), http://www.aes.org/e-lib/browse.cfm?elib=16375

9. Baudisch, P., Mueller, S.: Personal fabrication: State of the art and future research. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp. 936–939. ACM (2016)

10. Beddoes, T.: Observations on the nature of demonstrative evidence: with an explanation of certain difficulties occurring in the elements of geometry, and reflections on language. J. Johnson (1793)

11. Berard, F., Louis, T.: The object inside: Assessing 3d examination with a spherical handheld perspective-corrected display. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 4396–4404. ACM (2017)

12. Bissell, C.: Historical perspectives - The Moniac A Hydromechanical Analog Computer of the 1950s. IEEE Control Systems 27(1), 69–74 (Feb 2007), http://dx.doi.org/10.1109/MCS.2007.284511

13. Boem, A., Iwata, H.: “it’s like holding a human heart”: the design of vital+ morph, a shape-changing interface for remote monitoring. AI & SOCIETY pp. 1–21 (2017)

14. Borkin, M.A., Bylinskii, Z., Kim, N.W., Bainbridge, C.M., Yeh, C.S., Borkin, D., Pfister, H., Oliva, A.: Beyond memorability: Visualization recognition and recall. IEEE transactions on visualization and computer graphics 22(1), 519–528 (2016)

15. Botros, F., Perin, C., Aseniero, B.A., Carpendale, S.: Go and Grow: Mapping Personal Data to a Living Plant. In: Proceedings of the International Working Conference on Advanced Visual Interfaces. pp. 112–119. AVI ’16, ACM, New York, NY, USA (2016), http://doi.acm.org/10.1145/2909132.2909267

16. Brehmer, M., Munzner, T.: A multi-level typology of abstract visualization tasks. IEEE transactions on visualization and computer graphics 19(12), 2376–2385 (2013)
17. Brinton, W.C.: Graphic presentation. Graphic presentation. (1939)

18. Brown, C., Hurst, A.: VizTouch: automatically generated tactile visualizations of coordinate spaces. Proceedings of the Sixth International Conference on (2012), https://dl.acm.org/citation.cfm?id=2148160

19. Card, M.: Readings in information visualization: using vision to think. Morgan Kaufmann (1999)

20. Card, S., Mackinlay, J., Shneiderman, B.: Information visualization. Human-computer interaction: Design issues, solutions, and applications 181 (2009)

21. Chi, E.H.h., Riedl, J.T.: An operator interaction framework for visualization systems. In: Information Visualization, 1998. Proceedings. IEEE Symposium on. pp. 63–70. IEEE (1998)

22. Choe, E.K., Lee, N.B., Lee, B., Pratt, W., Kientz, J.A.: Understanding quantified-selfers’ practices in collecting and exploring personal data. In: Proceedings of the 32nd annual ACM conference on Human factors in computing systems. pp. 1143–1152. ACM (2014)

23. Claes, S., Vande Moere, A.: The role of tangible interaction in exploring information on public visualization displays. In: Proceedings of the 4th International Symposium on Pervasive Displays. pp. 201–207. ACM (2015)

24. Cleveland, W.S., McGill, R.: Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American statistical association 79(387), 531–554 (1984)

25. Cockburn, A., McKenzie, B.: Evaluating the effectiveness of spatial memory in 2d and 3d physical and virtual environments. In: Proceedings of the SIGCHI conference on Human factors in computing systems. pp. 203–210. ACM (2002)

26. Cox, A.L., Bird, J., Fleck, R.: Digital epiphanies: how self-knowledge can change habits and our attitudes towards them (2013)

27. Daniel, M., Rivière, G., Couture, N.: Cairnform: A shape-changing ring chart notifying renewable energy availability in peripheral locations. In: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 275–286. TEI ’19, ACM, New York, NY, USA (2019), http://doi.acm.org/10.1145/3294109.3295634

28. Data Cuisine: http://data-cuisine.net/ (2019), accessed: 2019-04-24

29. Davenport, C.B.: The statistical study of evolution. Popular Science Monthly pp. 447–460 (1901)

30. Debatty, R.: The chronocyclegraph. http://we-make-money-not-art.com/the_chronocyclegraph/ (2012), accessed: 2018-10-12

31. Domestic Data Streamers: http://domesticstreamers.com (2019), accessed: 2019-04-24

32. Dragicevic, P.: 1970 – moma poll: Participatory bar chart. list of physical visualizations. http://dataphys.org/list/moma-poll-haackes-participatory-bar-chart/ (2015), accessed: 2019-04-24

33. Dragicevic, P.: Interview with loren madsen: The birth of data sculpture. http://dataphys.org/list/loren-madsen-interview/ (2015), accessed: 2019-04-24

34. Dragicevic, P., Jansen, Y.: List of physical visualizations and related artefacts. https://www.dataphys.org/list (2012), accessed: 2018-10-14

35. Ducasse, J., Macé, M.J., Serrano, M., Jouffrais, C.: Tangible reels: construction and exploration of tangible maps by visually impaired users. In: Proceedings of the 2016 CHI conference on human factors in computing systems. pp. 2186–2197. ACM (2016)

36. Dwyer, T.: Two-and-a-half-dimensional Visualisation of Relational Networks. Ph.D. thesis (2004)
37. Edman, P.: Tactile graphics. American Foundation for the Blind (1992)

38. Fisherkeller, M.A., Friedman, J.H., Tukey, J.W.: Prim-9: An interactive multidimensional data display and analysis system. Dynamic Graphics for Statistics pp. 91–109 (1988)

39. Flemisch, T., Rajabiyazdi, F., Loorak, M.H., Carpendale, S.: Necklan: Language as jewellery (2015)

40. Follmer, S., Leithinger, D., Olwal, A., Cheng, N., Ishii, H.: Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In: Proceedings of the 25th annual ACM symposium on User interface software and technology. pp. 519–528. ACM (2012)

41. Follmer, S., Leithinger, D., Olwal, A., Hogge, A., Ishii, H.: inform: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of the 26th annual ACM symposium on User interface software and technology. pp. 417–426. ACM (2013)

42. Fredholm, L.: The discovery of the molecular structure of dna—the double helix. Science 9 (2003)

43. Gillet, A., Sanner, M., Stoffler, D., Goodsell, D., Olson, A.: Augmented reality with tangible auto-fabricated models for molecular biology applications. In: Visualization, 2004. IEEE. pp. 235–241. IEEE (2004)

44. Gillet, A., Sanner, M., Stoffler, D., Olson, A.: Tangible augmented interfaces for structural molecular biology. IEEE computer graphics and applications 25(2), 13–17 (2005)

45. Gomes, A., Rubens, C., Braley, S., Vertegaal, R.: Bitdrones: Towards using 3d nanocopter displays as interactive self-levitating programmable matter. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 770–780. ACM (2016)

46. Gourlet, P., Dassé, T.: Cairn: A tangible apparatus for situated data collection, visualization and analysis. In: Proceedings of the 2017 Conference on Designing Interactive Systems. pp. 247–258. ACM (2017)

47. Gustafsson, A., Gyllenswärd, M.: The power-aware cord: energy awareness through ambient information display. In: CHI’05 extended abstracts on Human factors in computing systems. pp. 1423–1426. ACM (2005)

48. Gwilt, I., Yoxall, A., Sano, K., et al.: Enhancing the understanding of statistical data through the creation of physical objects. In: DS 73-1 Proceedings of the 2nd International Conference on Design Creativity Volume 1. pp. 117–126 (2012)

49. Häkkilä, J., Virtanen, L.: Aesthetic physical items for visualizing personal sleep data. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. pp. 1173–1177. ACM (2016)

50. Hardy, J., Weichel, C., Taher, F., Vidler, J., Alexander, J.: Shapeclip: towards rapid prototyping with shape-changing displays for designers. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 19–28. ACM (2015)

51. Haroz, S., Kosara, R., Franconeri, S.L.: Isotype visualization: Working memory, performance, and engagement with pictographs. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. pp. 1191–1200. ACM (2015)

52. von Hofmann, A.W.: On the combining power of atoms. In: Proceedings of the Royal Institution. pp. 401–430. 4 (1865)

53. Hogan, T., Hornecker, E.: Towards a Design Space for Multisensory Data Representation. Interacting with computers 29(2), 147–167 (Mar 2017), http://dx.doi.org/10.1093/iwc/iww015
54. Hogan, T., Hornecker, E.: How does representation modality affect user-experience of data artifacts? In: International Conference on Haptic and Audio Interaction Design. pp. 141–151. Springer (2012)

55. Hogan, T., Hornecker, E.: Blending the repertory grid technique with focus groups to reveal rich design relevant insight. In: Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces. pp. 116–125. ACM (2013)

56. Hogan, T., Hornecker, E.: In touch with space: embodying live data for tangible interaction. In: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. pp. 275–278. ACM (2013)

57. Holmquist, L.E.: Evaluating the comprehension of ambient displays. In: CHI’04 extended abstracts on Human factors in computing systems. pp. 1545–1545. ACM (2004)

58. Holmquist, L.E., Skog, T.: Informative art: information visualization in everyday environments. In: Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia. pp. 229–235. ACM (2003)

59. Holstius, D., Kembel, J., Hurst, A., Wan, P.H., Forlizzi, J.: Infotropism: living and robotic plants as interactive displays. In: Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques. pp. 215–221. ACM (2004)

60. Houben, S., Golsteijn, C., Gallacher, S., Johnson, R., Bakker, S., Marquardt, N., Capra, L., Rogers, Y.: Physikit: Data Engagement Through Physical Ambient Visualizations in the Home. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 1608–1619. CHI ’16, ACM, New York, NY, USA (2016), http://doi.acm.org/10.1145/2858036.2858059

61. Hu, M.: Exploring new paradigms for accessible 3d printed graphs. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility. pp. 365–366. ACM (2015)

62. Huang, D., Tory, M., Adriel Aseniero, B., Bartram, L., Bateman, S., Carpendale, S., Tang, A., Woodbury, R.: Personal visualization and personal visual analytics. IEEE Transactions on Visualization and Computer Graphics 21(3), 420–433 (March 2015)

63. Hull, C.H., Willett, W.: Data tectonics: A framework for building physical and immersive data representations. IEEE computer graphics and applications 38(5), 11–17 (2018)

64. Huron, S., Carpendale, S., Thudt, A., Tang, A., Mauerer, M.: Constructive Visualization. In: ACM conference on Designing Interactive Systems in 2014. Vancouver, Canada (Jun 2014), http://hal.inria.fr/hal-00978437

65. Huron, S., Gourlet, P., Hinrichs, U., Hogan, T., Jansen, Y.: Let’s get physical: Promoting data physicalization in workshop formats. In: Proceedings of the 2017 Conference on Designing Interactive Systems. pp. 1409–1422. ACM (2017)

66. Huron, S., Jansen, Y., Carpendale, S.: Constructing Visual Representations: Investigating the Use of Tangible Tokens. IEEE transactions on visualization and computer graphics 20(12), 2102–2111 (2014), http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6875946

67. Ion, A., Kovacs, R., Schneider, O.S., Lopes, P., Baudisch, P.: Metamaterial textures. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 336. ACM (2018)
68. Ishii, H., Ren, S., Frei, P.: Pinwheels: Visualizing information flow in an architectural space. In: CHI '01 Extended Abstracts on Human Factors in Computing Systems. pp. 111–112. CHI EA '01, ACM, New York, NY, USA (2001), http://doi.acm.org/10.1145/634067.634135

69. Jafarinaimi, N., Forlizzi, J., Hurst, A., Zimmerman, J.: Breakaway: an ambient display designed to change human behavior. In: CHI'05 extended abstracts on Human factors in computing systems. pp. 1945–1948. ACM (2005)

70. Jansen, Y., Dragicevic, P.: An interaction model for visualizations beyond the desktop. IEEE transactions on visualization and computer graphics 19(12), 2396–2405 (Dec 2013), http://dx.doi.org/10.1109/TVCG.2013.134

71. Jansen, Y., Dragicevic, P., Fekete, J.D.: Evaluating the Efficiency of Physical Visualizations. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2593–2602. CHI '13, ACM, New York, NY, USA (2013), http://doi.acm.org/10.1145/2470654.2481359

72. Jansen, Y., Dragicevic, P., Isenberg, P., Alexander, J., Karnik, A., Kildal, J., Subramanian, S., Hornbæk, K.: Opportunities and Challenges for Data Physicalization. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 3227–3236. CHI '15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2702123.2702180

73. Jansen, Y., Hornbæk, K.: A Psychophysical Investigation of Size as a Physical Variable. IEEE transactions on visualization and computer graphics 22(1), 479–488 (Jan 2016), http://dx.doi.org/10.1109/TVCG.2015.2467951

74. Jofre, A., Szigeti, S., Diamond, S.: Citizen engagement through tangible data representation. Foro de Educación 14(20), 305–325 (2016)

75. Judelman, G.: Aesthetics and inspiration for visualization design: bridging the gap between art and science. In: Information Visualisation, 2004. IV 2004. Proceedings. Eighth International Conference on. pp. 245–250. IEEE (2004)

76. Kalia, A., Hopkins, R., Jin, D., Yazzolino, L., Verma, S., Merabet, L., Phillips, F., Sinha, P.: Perception of tactile graphics: Embossings versus cutouts. Multisensory research 27(2), 111–125 (2014)

77. Kamel, H.M., Landay, J.A.: A study of blind drawing practice: creating graphical information without the visual channel. In: Proceedings of the fourth international ACM conference on Assistive technologies. pp. 34–41. ACM (2000)

78. Karyda, M., Rodríguez, I., Lucero, A.: Gifting interpretations of personal data. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. p. LBW081. ACM (2018)

79. Khot, R.A., Andres, J., Lai, J., von Kaenel, J., Mueller, F.f.: Fantibles: Capturing Cricket Fan's Story in 3D. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems. pp. 883–894. DIS '16, ACM, New York, NY, USA (2016), http://doi.acm.org/10.1145/2901790.2901886

80. Khot, R.A., Hjorth, L., Mueller, F.f.: Understanding Physical Activity Through 3D Printed Material Artifacts. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 3835–3844. CHI '14, ACM, New York, NY, USA (2014), http://dx.doi.org/10.1145/2556288.2557144

81. Khot, R.A., Lee, J., Aggarwal, D., Hjorth, L., Mueller, F.f.: TastyBeats: Designing Palatable Representations of Physical Activity. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 2933–2942. CHI '15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2702123.2702197
82. Khot, R.A., Pennings, R., Mueller, F.: Edipulse: Turning physical activity into chocolates. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. pp. 331–334. ACM (2015)

83. Kiesl, T., Moser, H., Wilks, T.O.: Garden of eden. http://dataphys.org/list/garden-of-eden/ (2007), accessed: 2018-10-14

84. Kosara, R.: Visualization criticism-the missing link between information visualization and art. In: null. pp. 631–636. IEEE (2007)

85. Koutsomichalis, M.: Objektivisering: Text Physicalization and Self-introspective Post-digital Objecthood. In: Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 521–528. TEI ’18, ACM, New York, NY, USA (2018), http://doi.acm.org/10.1145/3173225.3173301

86. Kriz, R.D.: Thermodynamic case study: Gibbs’ thermodynamic graphical method. http://esm.rkriz.net/classes/ESM4714/methods/Gibbs.html (2007), accessed: 2018-10-12

87. Kuribayashi, S., Wakita, A.: Plantdisplay: turning houseplants into ambient display. In: Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology. p. 40. ACM (2006)

88. Lau, A., Vande Moere, A.: Towards a model of information aesthetics in information visualization. In: 2007 11th International Conference Information Visualization (IV’07). pp. 87–92. IEEE (2007)

89. Le Goc, M., Kim, L.H., Parsaei, A., Fekete, J.D., Dragicevic, P., Follmer, S.: Zooids: Building Blocks for Swarm User Interfaces. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology. pp. 97–109. ACM (Oct 2016), https://dl.acm.org/citation.cfm?doid=2984511.2984547

90. Le Goc, M., Perin, C., Follmer, S., Fekete, J.D., Dragicevic, P.: Dynamic Composite Data Physicalization Using Wheeled Micro-Robots. IEEE transactions on visualization and computer graphics (2018), https://hal.inria.fr/hal-01848436/

91. Lee, M.H., Cha, S., Nam, T.J.: Patina Engraver: Visualizing Activity Logs As Patina in Fashionable Trackers. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 1173–1182. CHI ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2702123.2702213

92. Leithinger, D., Follmer, S., Olwal, A., Ishii, H.: Shape displays: Spatial interaction with dynamic physical form. IEEE computer graphics and applications 35(5), 5–11 (2015)

93. Lindley, S.E., Thieme, A., Taylor, A.S., Vlachokyriakos, V., Regan, T., Sweeney, D.: Surfacing small worlds through data-in-place. Computer Supported Cooperative Work (CSCW) 26(1-2), 135–163 (2017)

94. Lo, D., Lockton, D., Rohrbach, S.: Experiential augmentation: Uncovering the meaning of qualitative visualizations when applied to augmented objects. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 490. ACM (2018)

95. Lockton, D., Ricketts, D., Aditya Chowdhury, S., Lee, C.H.: Exploring qualitative displays and interfaces. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp. 1844–1852. ACM (2017)

96. Lupton, D.: Self-tracking cultures: towards a sociology of personal informatics. In: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design. pp. 77–86. ACM (2014)

97. Lupton, D.: Feeling your data: Touch and making sense of personal digital data. New Media & Society 19(10), 1599–1614 (Oct 2017), https://doi.org/10.1177/1461444817717515
98. Mankoff, J., Dey, A.K., Hsieh, G., Kientz, J., Lederer, S., Ames, M.: Heuristic evaluation of ambient displays. In: Proceedings of the SIGCHI conference on Human factors in computing systems. pp. 169–176. ACM (2003)

99. Martz, E., Francoeur, E.: History of visualization of biological macromolecules. http://www.umass.edu/microbio/rasmol/history.htm (2004), accessed: 2018-10-12

100. McGookin, D., Robertson, E., Brewster, S.: Clutching at straws: using tangible interaction to provide non-visual access to graphs. In: Proceedings of the SIGCHI conference on human factors in computing systems. pp. 1715–1724. ACM (2010)

101. van Mensvoort, K.: Data fountain: Money translated to water. https://www.koert.com/work/datafountain/ (2004), accessed: 2018-10-14

102. meshu.io: http://meshu.io/ (2019), accessed: 2019-04-24

103. Morgan, M.S., Boumans, M.J.: The Secrets Hidden by Two-Dimensionality: Modelling the Economy as a Hydraulic System. In: Models: the third dimension of science. papers.ssrn.com (2004), http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1496354

104. Moving Color, I.: Moving color. https://www.movingcolor.net/ (2017), accessed: 2018-10-14

105. Munzner, T.: Process and pitfalls in writing information visualization research papers. In: Information visualization, pp. 134–153. Springer (2008)

106. Munzner, T.: Visualization analysis and design. AK Peters/CRC Press (2014)

107. Nakagaki, K., Follmer, S., Ishii, H.: Lineform: Actuated curve interfaces for display, interaction, and constraint. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. pp. 333–339. ACM (2015)

108. Nakagaki, K., Vink, L., Counts, J., Windham, D., Leithinger, D., Follmer, S., Ishii, H.: Materiable: Rendering dynamic material properties in response to direct physical touch with shape changing interfaces. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 2764–2772. ACM (2016)

109. News, C.: If canada had proportional representation: Dave meslin shows with lego (video). https://www.youtube.com/watch?v=laUPeXZlPEg (2015), accessed: 2018-10-13

110. Niiyama, R., Yao, L., Ishii, H.: Weight and volume changing device with liquid metal transfer. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. pp. 49–52. ACM (2014)

111. Nissen, B., Bowers, J.: Data-Things: Digital Fabrication Situated Within Participatory Data Translation Activities. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 2467–2476. CHI ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2702123.2702245

112. Nittala, A.S., Li, N., Cartwright, S., Takashima, K., Sharlin, E., Sousa, M.C.: PLANWELL: Spatial User Interface for Collaborative Petroleum Well-planning. In: SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications. pp. 19:1–19:8. SA ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2818427.2818443

113. Norton, M.I., Mochon, D., Ariely, D.: The ikia effect: When labor leads to love. Journal of consumer psychology 22(3), 453–460 (2012)

114. Offenhuber, D.: The invisible display-design strategies for ambient media in the urban context. In: International Workshop on Ambient Information Systems, Colocated with Ubicomp (2008)

115. Offenhuber, D., Telhan, O.: Indexical visualization-the data-less information display. Ubiquitous computing, complexity and culture 288 (2015)
116. Ogawa, H., Mara, M., Lindinger, C., Gardiner, M., Haring, R., Stolarsky, D., Ogawa, E., Hörtner, H.: Shadowgram: a case study for social fabrication through interactive fabrication in public spaces. In: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction. pp. 57–60. ACM (2012)

117. Olry, R.: Wax, wooden, ivory, cardboard, bronze, fabric, plaster, rubber and plastic anatomical models: Praiseworthy precursors of plastinated specimens. J Int Soc Plastination 15(1), 30–35 (2000), http://journal.plastination.org/archive/jp_vol.15/jp_vol.15_30-35.pdf

118. O'Malley, C., Fraser, D.S.: Literature review in learning with tangible technologies (2004)

119. Omirou, T., Perez, A.M., Subramanian, S., Roudaut, A.: Floating charts: Data plotting using free-floating acoustically levitated representations. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI). pp. 187–190. IEEE (2016)

120. Osterhaus, S.: Teaching math to students who are blind or visually impaired. http://www.perkinselearning.org/videos/webcast/teaching-math-students-who-are-blind-or-visually-impaired (2015), accessed: 2018-10-13

121. Pandey, A.V., Manivannan, A., Nov, O., Satterthwaite, M., Bertini, E.: The persuasive power of data visualization. IEEE transactions on visualization and computer graphics 20(12), 2211–2220 (2014)

122. Pangaro, G., Maynes-Aminzade, D., Ishii, H.: The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In: Proceedings of the 15th annual ACM symposium on User interface software and technology. pp. 181–190. ACM (2002)

123. Perovich, L.J.: Data Experiences: novel interfaces for data engagement using environmental health data. Ph.D. thesis, Massachusetts Institute of Technology (2014)

124. Phillips, A.W.: Mechanical models in economic dynamics. London School of Economics and Political Science (1950)

125. Posavec, S., Quick, M.: Air transformed: Better with data society commission. http://www.stefanieposavec.com/airtransformed (2015), accessed: 2019-04-24

126. Pousman, Z., Stasko, J., Mateas, M.: Casual information visualization: Depictions of data in everyday life. IEEE transactions on visualization and computer graphics 13(6), 1145–1152 (2007)

127. Rasmussen, M.K., Pedersen, E.W., Petersen, M.G., Hornbæk, K.: Shape-changing interfaces: a review of the design space and open research questions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 735–744. ACM (2012)

128. Redström, J., Skog, T., Hallnäs, L.: Informative art: using amplified artworks as information displays. In: Proceedings of DARE 2000 on Designing augmented reality environments. pp. 103–114. ACM (2000)

129. Rich, M.: Wooden geometric models made by George Adams. http://www.hps.cam.ac.uk/whipple/explore/models/geometricmodels/ (2006), accessed: 2016-4-11

130. Roberts, J.C., Ritsos, P.D., Badam, S.K., Brodbeck, D., Kennedy, J., Elmqvist, N.: Visualization beyond the desktop–the next big thing. IEEE Computer Graphics and Applications 34(6), 26–34 (2014)

131. Robertson, G., Czerwinski, M., Larson, K., Robbins, D.C., Thiel, D., Van Dantzich, M.: Data mountain: using spatial memory for document management. In: Proceedings of the 11th annual ACM symposium on User interface software and technology. pp. 153–162. ACM (1998)
132. Rosenblum, L.J., Earnshaw, R., Encarnacao, J., Hagen, H., Kaufman, A., Klimenko, S., Nielson, G., Post, F., Thalmann, D.: Scientific visualization: advances and challenges, vol. 21. Academic press London (1994)

133. Roudaut, A., Karnik, A., Löchtefeld, M., Subramanian, S.: Morphees: toward high shape resolution in self-actuated flexible mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 593–602. ACM (2013)

134. Segel, E., Heer, J.: Narrative visualization: Telling stories with data. IEEE transactions on visualization and computer graphics 16(6), 1139–1148 (2010)

135. Shaer, O., Hornecker, E., et al.: Tangible user interfaces: past, present, and future directions. Foundations and Trends® in Human-Computer Interaction 3(1–2), 4–137 (2010)

136. Shneiderman, B.: Why not make interfaces better than 3d reality? IEEE Computer Graphics and Applications 23(6), 12–15 (2003)

137. Sosa, R., Gerrard, V., Esparza, A., Torres, R., Napper, R., et al.: Data objects: Design principles for data physicalisation. In: DS92: Proceedings of the DESIGN 2018 15th International Design Conference. pp. 1685–1696 (2018)

138. Sprague, D., Tory, M.: Exploring how and why people use visualizations in casual contexts: Modeling user goals and regulated motivations. Information Visualization 11(2), 106–123 (2012)

139. Stark, L.: The emotional context of information privacy. The Information Society 32(1), 14–27 (2016)

140. Starrett, C., Reiser, S., Pacio, T.: Data materialization: A hybrid process of crafting a teapot. In: ACM SIGGRAPH 2018 Art Gallery. pp. 381–385. SIGGRAPH ’18, ACM, New York, NY, USA (2018), http://doi.acm.org/10.1145/3202918.3203087

141. Stoppel, S., Bruckner, S.: Vol2velle: Printable Interactive Volume Visualization. IEEE transactions on visualization and computer graphics 23(1), 861–870 (Jan 2017), http://dx.doi.org/10.1109/TVCG.2016.2599211

142. Stusak, S., Tabard, A., Sauka, F., Khot, R.A., Butz, A.: Activity Sculptures: Exploring the Impact of Physical Visualizations on Running Activity. IEEE transactions on visualization and computer graphics 20(12), 2201–2210 (Dec 2014), http://dx.doi.org/10.1109/TVCG.2014.2352953

143. Stusak, S., Aslan, A.: Beyond physical bar charts: An exploration of designing physical visualizations. In: CHI’14 Extended Abstracts on Human Factors in Computing Systems. pp. 1381–1386. ACM (2014)

144. Stusak, S., Hobe, M., Butz, A.: If your mind can grasp it, your hands will help. In: Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 92–99. ACM (2016)

145. Stusak, S., Schwarz, J., Butz, A.: Evaluating the memorability of physical visualizations. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 3247–3250. ACM (2015)

146. Stusak, S., Teufel, M.: Projection augmented physical visualizations. In: Proceedings of the 2nd ACM symposium on Spatial user interaction. pp. 145–145. ACM (2014)

147. Sutherland, I.E.: The ultimate display. Multimedia: From Wagner to virtual reality pp. 506–508 (1965)

148. Suzuki, R., Kato, J., Gross, M.D., Yeh, T.: Reactile: Programming swarm user interfaces through direct physical manipulation. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 199. ACM (2018)
149. Swaminathan, S., Roumen, T., Kovacs, R., Stangl, D., Mueller, S., Baudisch, P.: Linespace: A sensemaking platform for the blind. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 2175–2185. ACM (2016)

150. Swaminathan, S., Shi, C., Jansen, Y., Dragicevic, P., Oehlberg, L., Fekete, J.D.: Supporting the Design and Fabrication of Physical Visualizations. In: CHI2014 - Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Toronto, ON, Canada (Apr 2014), http://dx.doi.org/10.1145/2556288.2557310

151. Swan, M.: The quantified self: Fundamental disruption in big data science and biological discovery. Big Data 1(2), 85–99 (2013)

152. Taher, F., Hardy, J., Karnik, A., Weichel, C., Jansen, Y., Hornbæk, K., Alexander, J.: Exploring Interactions with Physically Dynamic Bar Charts. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 3237–3246. ACM (Apr 2015), http://dl.acm.org/citation.cfm?doid=2702123.2702604

153. Taher, F., Jansen, Y., Woodruff, J., Hardy, J., Hornbæk, K., Alexander, J.: Investigating the Use of a Dynamic Physical Bar Chart for Data Exploration and Presentation. IEEE transactions on visualization and computer graphics 23(1), 451–460 (Jan 2017), http://dx.doi.org/10.1109/TVCG.2016.2598498

154. Taylor, A.S., Lindley, S., Regan, T., Sweeney, D., Vlachokyriakos, V., Grainger, L., Lingel, J.: Data-in-Place: Thinking Through the Relations Between Data and Community. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 2863–2872. CHI ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2702123.2702558

155. TED: Hans rosling’s ted talks. https://www.ted.com/speakers/hans_rosling (2014), accessed: 2018-10-12

156. TED Blog: Watch hans rosling’s shortest tedtalk ever. https://blog.ted.com/watch-hans-roslings-shortest-tedtalk-ever/ (2012), accessed: 2019-04-24

157. The Molecular Graphics Laboratory: Tangible models. http://mgl.scripps.edu/projects/tangible概念/models (2005), accessed: 2018-10-15

158. Thrun, M.C., Lerch, F.: Visualization and 3D printing of multivariate data of biomarkers (2016), https://dspace5.zcu.cz/handle/11025/29682

159. Thudt, A., Hinrichs, U., Carpendale, S.: ddata craft: integrating data into daily practices and shared reflections (2017)

160. Thudt, A., Hinrichs, U., Huron, S., Carpendale, S.: Self-Reflection and Personal Physicalization Construction. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 154. ACM (Apr 2018), https://dl.acm.org/citation.cfm?doid=3173574.3173728

161. Tiab, J., Boring, S., Strohmeier, P., Markussen, A., Alexander, J., Hornbæk, K.: Tiltstacks: composing shape-changing interfaces using tilting and stacking of modules. In: Proceedings of the 2018 International Conference on Advanced Visual Interfaces. p. 44. ACM (2018)

162. Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM systems journal 39(3.4), 915–931 (2000)

163. Vande Moere, A.: Beyond the tyranny of the pixel: Exploring the physicality of information visualization. In: IV’08 (2008)

164. Vande Moere, A.: Form follows data: the symbiosis between design and information visualization pp. 31–40 (2005)

165. Vande Moere, A., Hoinkis, M.: A wearable folding display for self-expression. In: Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments. pp. 301–304. ACM (2006)
166. Vande Moere, A., Patel, S.: The physical visualization of information: designing data sculptures in an educational context. In: Visual information communication, pp. 1–23. Springer (2009)

167. Vande Moere, A., Purchase, H.: On the role of design in information visualization. Information Visualization 10(4), 356–371 (2011)

168. Viégas, F.B., Wattenberg, M.: Artistic data visualization: Beyond visual analytics. In: Schuler, D. (ed.) Online Communities and Social Computing. pp. 182–191. Springer Berlin Heidelberg, Berlin, Heidelberg (2007)

169. Wang, Y., Ma, X., Luo, Q., Qu, H.: Data edibilization: Representing data with food. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp. 409–422. ACM (2016)

170. Weiss, M., Schwarz, F., Jakubowski, S., Borchers, J.: Madgets: actuating widgets on interactive tabletops. In: Proceedings of the 23nd annual ACM symposium on User interface software and technology. pp. 293–302. ACM (2010)

171. Wessel, D.: How much does the tax code reduce inequality? https://www.brookings.edu/blog/up-front/2015/04/09/how-much-does-the-tax-code-reduce-inequality/ (2015), accessed: 2018-10-13

172. Willett, W., Jansen, Y., Dragicevic, P.: Embedded Data Representations. IEEE transactions on visualization and computer graphics 23(1), 461–470 (Jan 2017), http://dx.doi.org/10.1109/TVCG.2016.2598608

173. Wilson, M.: How gm is saving cash using legos as a data viz tool. https://www.fastcompany.com/1669468/how-gm-is-saving-cash-using-legos-as-a-data-viz-tool (2012), accessed: 2018-10-15

174. Wisneski, C., Ishii, H., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P.: Ambient displays: Turning architectural space into an interface between people and digital information. In: International Workshop on Cooperative Buildings. pp. 22–32. Springer (1998)

175. Wun, T., Payne, J., Huron, S., Carpendale, S.: Comparing bar chart authoring with microsoft excel and tangible tiles. In: Computer Graphics Forum. vol. 35, pp. 111–120. Wiley Online Library (2016)

176. Yao, L., Niiyama, R., Ou, J., Follmer, S., Della Silva, C., Ishii, H.: Pneui: pneumatically actuated soft composite materials for shape changing interfaces. In: Proceedings of the 26th annual ACM symposium on User interface software and technology. pp. 13–22. ACM (2013)

177. Zhang, X., Dekel, T., Xue, T., Owens, A., He, Q., Wu, J., Mueller, S., Freeman, W.T.: Mosculp: Interactive visualization of shape and time. In: The 31st Annual ACM Symposium on User Interface Software and Technology. pp. 275–285. ACM (2018)

178. Zhao, J., Vande Moere, A.: Embodiment in data sculpture: a model of the physical visualization of information. In: Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts. pp. 343–350 (2008)

179. Zhao, Y., Kim, L.H., Wang, Y., Le Goc, M., Follmer, S.: Robotic assembly of haptic proxy objects for tangible interaction and virtual reality. In: Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. pp. 82–91. ACM (2017)