Data Physicalization

Jason Alexander, Petra Isenberg, Yvonne Jansen, Bernice E Rogowitz,
Andrew Vande Moere

► To cite this version:

Jason Alexander, Petra Isenberg, Yvonne Jansen, Bernice E Rogowitz, Andrew Vande Moere. Data Physicalization. Dagstuhl Reports, 2019, Dagstuhl Reports, 8 (10), pp.127-147. 10.4230/DagRep.8.10.127. [https://doi.org/10.4230/DagRep.8.10.127](https://doi.org/10.4230/DagRep.8.10.127).

HAL Id: hal-02090877

https://hal.sorbonne-universite.fr/hal-02090877

Submitted on 5 Apr 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Data Physicalization

Edited by

Jason Alexander¹, Petra Isenberg², Yvonne Jansen³, Bernice E. Rogowitz⁴, and Andrew Vande Moere⁵

1 Lancaster University, GB j.alexander@lancaster.ac.uk

2 Inria, FR petra.isenberg@inria.fr

3 CNRS & Sorbonne University, FR jansen@isir.upmc.fr

4 Visual Perspectives & Columbia University, US bernice.e.rogowitz@gmail.com

5 KU Leuven, BE andrew.vandemoere@kuleuven.be

## Abstract

Data physicalization involves representing numbers and relationships using physical, tangible displays. These displays provide tactile, as well as visual metaphors for expressing and experiencing data, and can unlock new analytical insights and emotional responses. This Dagstuhl seminar brought together a diverse group of researchers and practitioners to explore the benefits and challenges of physicalization – computer scientists trained in visualization, virtual reality and human-computer interaction; architects of virtual and augmented systems; perceptual and cognitive scientists; and artists and designers. Through interactive discussions and demonstrations, we explored physicalization, as a set of methodologies for representing data, for engaging audiences, and for artistic expression.

**Seminar** October 28–November 2, 2018 – http://www.dagstuhl.de/18441

**2012 ACM Subject Classification** Human-centered computing → Collaborative and social computing, Human-centered computing → Interaction design, Human-centered computing → Ubiquitous and mobile computing, Human-centered computing → Visualization

**Keywords and phrases** Art, Data Physicalization, Design, Human-Computer Interaction, Information Design, Visualization

**Digital Object Identifier** 10.4230/DagRep.8.10.127

## 1 Executive Summary

Jason Alexander (Lancaster University, GB)

Petra Isenberg (INRIA, FR)

Yvonne Jansen (CNRS-Sorbonne University, FR)

Bernice E. Rogowitz (Visual Perspectives & Columbia University, US)

Andrew Vande Moere (KU Leuven, BE)

**License** © Creative Commons BY 3.0 Unported license
© Jason Alexander, Petra Isenberg, Yvonne Jansen, Bernice E. Rogowitz, Andrew Vande Moere

This executive summary gives a brief overview of the goals and agenda for our seminar.

## Introduction

Among the earliest man-made artifacts are physical representations of semantic concepts. These “physicalizations” externalize numerical and abstract concepts, providing physical metaphors that allow us to reason, remember, and communicate. Physicalizations are created

<figure><img src="image_2.png" /></figure>

Except where otherwise noted, content of this report is licensed under a Creative Commons BY 3.0 Unported license

Data Physicalization, *Dagstuhl Reports*, Vol. 8, Issue 10, pp. 127–147

Editors: Jason Alexander, Petra Isenberg, Yvonne Jansen, Bernice E. Rogowitz, and Andrew Vande Moere

<figure><img src="image_3.png" /></figure>

DAGSTUHL Dagstuhl Reports

REPORTS Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany
<figure><img src="image_4.png" /><figcaption>Figure 1 Examples of data physicalizations exhibited by our seminar participants. Left: Data objects for visualizing the experience, expression, and description of chronic pain. Right: Acoustic Sonifications whose sounds convey useful information about a dataset.</figcaption></figure>

for many different purposes, or “intents.” The sundial, for example, transforms shadows into a readable representation of time of day, the mercury thermometer transforms temperature into a displacement along a number line, and a scatterplot transforms the values of two variables into a form that allows the reader to interpret correlation. Our first data representations were based on natural objects, such as charcoal scraped onto walls or built from clay, or later, ink on paper. With the advent of computers, we’ve substituted physical representations with pixels on a computer screen. The resurgence of physicalization asks what we have lost in this transformation. Certainly, a computer-based visualization allows us to zoom an image, transform variables in real time, and to zoom through virtual computer-based world. However, these representations can sever the relationship to the natural world, depriving us of the touch, feel, and emotion that comes from interacting with real objects.

This Dagstuhl seminar brought together a diverse group of researchers and practitioners to explore the benefits and challenges of physicalization – computer scientists trained in visualization, virtual reality and human-computer interaction; architects of virtual and augmented systems; perceptual and cognitive scientists; and artists and designers. Through interactive discussions and demonstrations, we explored physicalization, as a set of methodologies for representing data, for engaging audiences, and for artistic expression. There were no formal paper presentations. Instead, the work was done through interactive discussions, hands-on workshops, and interactive demonstrations outside the lecture hall. Figure 1 shows two examples of data physicalizations exhibited by our participants.
<figure><img src="image_5.png" /><figcaption>Figure 2 A collaborative affinity diagramming session on Tuesday, to organize the large amount of thematic research questions into cross-disciplinary themes of research interest.</figcaption></figure>

## The Week at a Glance

**Monday.** After explaining various organization matters and formalities, the first day began with the physical construction of custom name badges. The hands-on session offered the first occasion to get to know each other, also by comparing the name badges themselves, then and during the whole duration of the seminar. After this session, each of the 40 participants gave a two-minute presentation of their achievements and interests in the field of physicalization. Some described their artistic creations based on data, some showed how mapping data onto physical dimensions enhanced data analysis. Others showed how scientists and artists collaborated to explain biological and physical principles through physicalization, or how principles of perception and cognition could be used to guide how data are effectively mapped onto visual dimensions. The electricity in the air was palpable; everyone realized that we were on the threshold of a new discipline. This energy drove the first deliverable for this seminar: Dagstuhl monograph, and perhaps a book, highlighting extended versions of these 40 contributions.

To allow a more focused approach towards yet unexplored research topics, the first activity focused on defining “pillars” of physicalization, crystallizing the learning and background of the different intellectual communities identified in our proposal. In break-out sessions, each group ideated around and then synthesized the most fundamental papers, examples, principles and challenges for their pillar, relative to physicalization. The Perception and Cognition Group focused on sensory processing, especially touch perception and embodied cognition, which deals with the way we learn about the world through our motor interactions
with it. The Evaluation group shared their experiences in measuring human responses, and
how methods might be extended to physicalization. The Design group explored artistic and
design approaches to data physicalization. The Applications group identified existing and
future application areas. The technology group surveyed the range of materials and devices
for physicalization. In the reports-back, the group was encouraged to explore challenges and
limitations for each of these core areas, to set the stage for the cross-disciplinary discussions
beginning on Tuesday.

**Tuesday.** At the start the second day, each participant generated three questions for each of the five pillar areas. A massive exercise was undertaken wherein all these questions were organized and grouped into emergent categories. Spontaneous discussions started around the meaning and validity of the themes that emerged, as these would align the next round of thematic discussions (Figure 2). The “Design Patterns” group focused on identifying general templates for characterizing data physicalization. The “Emotion” group explored unique affordances that touching data enable. The Vis vs. Phys Group delved into the ways vision and touch were different, and what unique advantages that might enable. The “Critical” group explored the range of ethical and critical matters that could be related to physicalization practice in particular.

**Wednesday.** Wednesday morning was devoted to hands-on workshops where practitioners engaged small groups in interactive activities. Samuel Huron's group explored a set of physicalization examples, to identify common principles; Daniel Keefe demonstrated a virtual reality exploration of the human heart; Till Nagel, Laura Perovich, and Dietmar Offenhuber led a group into the forest to collect natural objects which they used to create physicalizations; Robert Friska provided surprising problem solving examples that drove a discussion about physical reasoning; Barbara Tversky showed examples of how physical gestures we make contribute to problem solving; Daniel Schneider led a workshop on computational embroidery, where the data are represented in yarn color and texture. By the time we piled into the bus for Trier, the large and diverse group of 40 had transformed into a dynamic community of scholars. Listening into the conversations, you could hear artists explaining how sculpture is taught in Art School, perceptual psychologists describing touch perception; and engineers revealing expertise on control systems for autonomous micro-robots.

**Thursday.** Thursday was the major work day. Building on the explorations of Tuesday, we divided into groups and worked on a variety of topics. One group, for example, focused on how to teach visualization and how to use data physicalization to teach other areas. Another group worked on a white paper on emotion and physicalization. Another worked on ideas for using physicalization for environmentally-situated projects. Another group focused on categorizing the critical considerations that emerged on Tuesday.

**Friday.** The final day aimed into transforming the progress of the past days into concrete
contributions. For example, plans for a physicalization contest were discussed, ideas for the
book of 40 contributions were pushed further, as well as on a special issue on physicalization
for the Journal of Perceptual Imaging. Another group discussed the topic of scientific funding,
among which the experience of past H2020 research proposals, and the potential of future
research proposal initiatives that relate to physicalization.
<figure><img src="image_6.png" /><figcaption>Figure 3 Exhibition tables outside our seminar room.</figcaption></figure>

## Hands-on Demonstrations

The area outside the main conference room was populated with demonstrations provided by the participants. For example, Andrew Vande Moere brought a wireless, networked display that allowed participants to vote about particular topics that were shown; Volker Schweinsfurth, Daniel Schneider, and many others brought along and exhibited 3-D printed, hand-crafted, or even embroidered physicalizations. Bernice Rogowitz ran participants through several experiments around the topics of touch perception and touch/vision interactions. Section 4 details several of the hands-on experiences seminar participants and organizers could engage in; while Figure 3 shows the diversity of the physical artifacts that participants exhibited.
## 2 Table of Contents

### Executive Summary

<table><tr><td>Jason Alexander, Petra Isenberg, Yvonne Jansen, Bernice E. Rogowitz, Andrew Vande Moere</td><td>127</td></tr></table>

### Overview of Talks

<table><tr><td>Visual Communication: Gesture and Diagrams<br>Barbara Tversky</td><td>133</td></tr><tr><td>Strong Spatial Cognition<br>Christian Freksa</td><td>134</td></tr></table>

### Hands-On Workshops and Demos

<table><tr><td>Data Badges<br>Andrew Vande Moere, Sinem Görücü, Georgia Panagiotidou</td><td>135</td></tr><tr><td>Environmental Art as Physicalization Technique<br>Till Nagel, Laura Perovich, Dietmar Offenhuber</td><td>136</td></tr><tr><td>Data Visualization with Machine Embroidery<br>Daniel Schneider</td><td>137</td></tr><tr><td>Physicalization for Scientific Visualization and Art+Science Collaborations<br>Daniel Keefe</td><td>138</td></tr><tr><td>Screening “Visualizing Agriculture”<br>Leanne Elias, Denton Frederickson</td><td>138</td></tr><tr><td>The data physicalization card game<br>Samuel Huron, Anaelle Beignon</td><td>139</td></tr><tr><td>Haptic Perception Experiments<br>Bernice Rogowitz</td><td>141</td></tr><tr><td>Data Sculpture Exhibition<br>Volker Schweisfurth</td><td>142</td></tr><tr><td>Dagstuhl Data Physicalization<br>Pauline Gourlet, Till Nagel, Aurélien Tabard</td><td>142</td></tr></table>

### Working Groups

<table><tr><td>Data Physicalization Toward Sustainability and Peace<br>Yuri Engelhardt, Brygg Ullmer, Christian Freksa, Pauline Gourlet, Samuel Huron</td><td>143</td></tr><tr><td>Physicalization vs. Visualization<br>David Kirsh</td><td>144</td></tr><tr><td>The Role of Emotion and Engagement in Data Physicalization<br>Yun Wang, Tim Dwyer, Daniel Keefe, Petra Isenberg, Roberta Klatzky, Eva Hornecker, Jörn Hurtienne, Leanne, Adrien Segal, Steven Barras</td><td>144</td></tr><tr><td>Sound in Data Physicalisation<br>Stephen Barrass</td><td>145</td></tr></table>

Summary 145

Participants 147
<figure><img src="image_7.png" /><figcaption>Figure 4 The seminar included two very interesting lectures for participants. Left: Using Gesture to Communicate Semantics (Barbara Tversky); Right: Strong Spatial Cognition (Christian Freksa)</figcaption></figure>

## 3 Overview of Talks

The seminar featured two talks on topics related to perception and cognition. Figure 4 shows Barbara Tversky and Christian Freksa lecturing. Details of the talks follow.

### 3.1 Visual Communication: Gesture and Diagrams

Barbara Tversky (Stanford University & Columbia Teachers College)

License © Creative Commons BY 3.0 Unported license

© Barbara Tversky

Like diagrams, sketches, and other graphics, gestures communicate more directly than words [1, 2, 3, 4]. Gestures can convey an overall spatial structure for thought, for examples, a relationships between two ideas or juxtaposing two sets of ideas or indicating a continuum of ideas, a dimension, such as events ordered in time or sports teams or movies in order of value or the workings of a complex system. Our work has shown that people learning static or dynamic spatial or conceptual relations spontaneously gesture when they study, and that doing so improves memory and inference. Similarly, people learning from explanations learn better when teachers gesture, especially dynamic information. The same holds true for diagrams and other graphics, but gestures are especially effective for dynamic information.

## References

1. Barbara Tversky. Visualizing thought. *Topics in Cognitive Science*, 3(3):499–535, July 2011.

2. Barbara Tversky. The cognitive design of tools of thought. *Review of Philosophy and Psychology*, 6(1):99–116, March 2015.

3. Barbara Tversky. *Mind in Motion: How Action shapes thought*. Basic Books, 2019.

4. Barbara Tversky and Angela Kessell. Thinking in action. *Pragmatics & Cognition*, 22(2):206–223, 2014.
## 3.2 Strong Spatial Cognition

Christian Freksa (Universität Bremen, DE)

**License** © Creative Commons BY 3.0 Unported license
© Christian Freksa

This workshop introduced the Strong Spatial Cognition (SSC) paradigm for spatial problem solving. Strong Spatial Cognition (SSC) – Definition ‘SSC’ is a cognitive agent’s capability of processing spatial information through perception, representation, mental processing, and action. In comparison with spatial reasoning SSC includes the participation of spatial structures and processes in physical environments, the perception of configurations and spatial change in these environments, as well as the agent’s locomotion and manipulation of spatial configurations.

In other words, the spatio-temporal substrate of an agent’s environment is an integral part of his/her/its cognitive engine [2, 1, 3]. It allows to make direct use of factual, structural, and procedural knowledge in the world (Norman). The strong spatial cognition paradigm employs affordance-based object-level problem solving to complement knowledge-level computation.

According to SSC, cognitive agents equipped with suitable perception, mental capabilities, and effectors constitute not merely a tool in the study of the mind (as cognitive simulation programs on a computer do); rather, appropriately configured agents really are cognitive agents, in the sense that they can interact with spatial environments in a meaningful way and have cognitive states.

### References

1. Christian Freksa. Strong spatial cognition. In Sara Irina Fabrikant, Martin Raubal, Michela Bertolotto, Clare Davies, Scott Freundschuh, and Scott Bell, editors, *Spatial Information Theory*, pages 65–86, Cham, 2015. Springer International Publishing.

2. Christian Freksa, Thomas Barkowsky, Frank Dylla, Zoe Falomir, Ana-Maria Olteteanu, and Jasper van de Ven. Spatial problem solving and cognition. In Jeffrey M. Zacks and Holly A. Taylor, editors, *Representations in Mind and World*, pages 156–183. Routledge, New York, USA, 2018.

3. Christian Freksa, Thomas Barkowsky, Zoe Falomir, and Jasper van de Ven. Geometric problem solving with strings and pins. *Spatial Cognition & Computation*, 2018. To appear.

## 4 Hands-On Workshops and Demos

Many of the workshop participants had previously explored physicalizations and haptic methods, and offered to lead workshops, demonstrations, and discussions that allowed others to share their experiences. These workshops provided hands-on experience with physicalization from many different perspectives.
<figure><img src="image_8.png" /><figcaption>Figure 5 Participants fabricating their own bespoke data badge.</figcaption></figure>

## 4.1 Data Badges

Andrew Vande Moere (KU Leuven)

Sinem Görücü (KU Leuven) (via videoconference)

Georgia Panagiotidou (KU Leuven) (via videoconference)

**License** © Creative Commons BY 3.0 Unported license
© Andrew Vande Moere, Sinem Görücü, Georgia Panagiotidou

All participants were invited to construct a bespoke, individualized “data badge” for themselves to be worn during the seminar (Figure 5). Specifically for this seminar, two graduate students of Andrew Vande Moere designed and then constructed a modular physicalization system via rapid prototyping methods. The system was meant to represent the research domain-related aspects of a person as a lightweight and wearable physicalization. The two students presented the core underlying ideas via a short videoconference session, and explained the instructions for building them. Four different colorful shapes each conveyed a particular domain (e. g. interactive technologies and hardware platform; evaluation and methodologies; arts, design and applications; perception and cognition), whereas their relative size correlated to the weight or importance of that domain. On the top left, these shapes conveyed the academic background of the attendee, whereas the bottom right corner related to future research directions. The length of white and blue wire corresponded with the duration of travel to Dagstuhl and the number of physicalisation-related publications respectively.

All attendees actively participated in the data badge fabrication session. The session formed the ideal informal platform to get to know one another by helping out or commenting on each other’s badge. All resulting name badges were characterized by a sense of individual creativity and personal craftsmanship, while conveying shared types of information with which attendees could commence or refocus joint discussions and conversations (Figure 6). As a result, many data badges were worn for the whole length of the seminar.
<figure><img src="image_9.png" /><figcaption>Figure 6 Different examples of the hands-on fabricated data badges, reflecting the research interests of each attendee, as well as a certain sense of originality and craftsmanship.</figcaption></figure>

## 4.2 Environmental Art as Physicalization Technique

Till Nagel (Hochschule Mannheim)

Laura Perovich (MIT Media Lab)

Dietmar Offenhuber (Northeastern University)

**License** Ⓔ Creative Commons BY 3.0 Unported license

© Till Nagel, Laura Perovich, Dietmar Offenhuber

In this hands-on workshop (Figure 7), we explored environmental art as a visualization technique and investigated how natural materials can be employed in meaningful ways. After a brief introductory outline of this workshop's scope by the organizers, participants were asked to create physicalizations outdoors. We walked up the forest path to Burg Dagstuhl, observed the environment, collected materials, identified opportunities for intervention, and finally, created physicalizations. The three participating groups presented their physicalizations in-situ and reflected on the creation process.

One group focused on exploiting the biological shapes and textures to combine and connect them in ways that reflected quantitative data. Another group made a parcours of different zones, with each zone generating a different kind of sound when one stepped on it, and each sound representing a different numerical value. The last group constructed constellations of situated materials such as leaves and drops of water to resemble graphical data depictions. One more out-of-the-box example included the differences in length-of-flight of plant seeds that resemble a wing. This exploratory workshop was successful in bringing together different perspectives towards material and context from different fields around a new and unfamiliar task. This informed a conversation about the preliminary design space of a potential new research area. As a next step, we are going to fully document the process, results, and discussion and refine the workshop setup.
<figure><img src="image_10.png" /><figcaption>Figure 7 Participants engaging in the creation of data physicalizations using natural materials.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 8 Examples of digital embroidery shared by Daniel Schneider.</figcaption></figure>

## 4.3 Data Visualization with Machine Embroidery

Daniel K. Schneider (University of Geneva, CH)

**License** © Creative Commons BY 3.0 Unported license
© Daniel Schneider

This technical workshop introduced participants to principles of computerized embroidery (Figure 8), Participants discussed its potential for data physicalization. At the end of the workshop, each participant created a (very simple) data visualization that could be stitched with any low or high-end embroidery machine. The workshop agenda was as follows:

- Workflow(s) of computerized embroidery (10 min)

- Typology of embroidery stitch types (5min)

- Physical constraints (resolution, layers, tissue, size, etc.) (5 min)

- Hands-on: Using InkStitch (a free extension to the free Inscape program) (40 min)

- Hands-on: Generating, importing and adopting appropriate SVG graphics (30 min)
<figure><img src="image_12.png" /><figcaption>Figure 9 Multiple views of heart model parameters in Virtual Reality.</figcaption></figure>

## 4.4 Physicalization for Scientific Visualization and Art+Science Collaborations

Daniel Keefe (University of Minnesota, US)

License Ⓒ Creative Commons BY 3.0 Unported license

© Daniel Keefe

I presented a virtual reality demo of “Bento Box”, a visualization method for arranging virtual spaces to make visual comparisons of multiple related datasets. In this case, we looked at a parameter study of 10 blood flows in the heart simulated on supercomputers using 10 different initial conditions. I also handed out a physical “pick up and touch” demo of 3D printouts of the same hearts. In the group we discussed how these two techniques might be combined to create more informative visualizations and intuitive user interfaces.

## 4.5 Screening “Visualizing Agriculture”

Leanne Elias (University of Lethbridge, CA)

Denton Frederickson (University of Lethbridge, CA)

License Ⓒ Creative Commons BY 3.0 Unported license

© Leanne Elias, Denton Frederickson

The growing field of Data Visualization is situated at the fertile intersection of art and science. It explores the symbiotic coupling of these seemingly disparate disciplines, by finding connection among the distinct goals, methodologies, and contexts of artistic and scientific pursuit. Data, in its simplest sense, can be described as facts or statistics collected together for analysis. But things are rarely so simple, and in a moment of human development when data continues to escalate in quantity, complexity, and value, it may be timely to ask: What are the cultural, environmental, and political implications of data? How is data collected and used in contemporary society? Can art make sense of data? And how will data fundamentally change the character of art?
In September 2016, the Southern Alberta Art Gallery and the Data Physicalization Lab at the University of Lethbridge invited six artists to participate in a residency, documentary film, and exhibition that asked them to respond to agricultural data developed by Dr. Jamie Larson and Dr. Andre Laroche from the Lethbridge Research and Development Centre: Agriculture and Agri-food Canada. Two datasets were provided: The first is the result of an experiment on breeding cereal wheats and wheat grasses with the intention of developing a perennial wheat cultivar; the second relates to experimentation in genetic modification aimed at reducing wheat's susceptibility to the devastating pathogen known as stripe rust. The artists were invited to engage with the instruments, test subjects, contexts, methods, and people associated with the development of this data, which they would later consider while creating new work for this exhibition. The aim of this endeavour is to investigate the effect of intensive collaboration – how artists can use scientific process to guide their art, and how scientists can use artistic ways of knowing to approach their data in new ways.

The artwork that emerges from this investigation explores the potential that lies within scientific inquiry when strict standards for fact and method are allowed to be considered and probed through an expanded perspective. Through this inquiry, the work is allowed to affectively engage viewers by evoking feelings of wonder, curiosity, and consciousness about data and agricultural research, while creating a place for contemplation about the land and our inherent connection to it.

We are currently working on a documentary movie about our project. The Dagstuhl participant group received an early screening.

Summary Details on the project:

- Visualizing Agriculture: Southern Alberta Art Gallery

- February 17 to April 22, 2018

- Artists: Jackson 2Bears, Tori Foster, Mary-Anne McTrowe, Robyn Moody, Adrien Segal, Michelle Sylvestre

- Exhibition Design and Data Analysis: Christine Clark

- Curators / Organizers: Christina Cuthbertson, Leanne Elias and Denton Fredrickson (Data Physicalization Lab, University of Lethbridge)

## 4.6 The data physicalization card game

Samuel Huron (Telecom Paris Tech, FR), Anaelle Beignon (DSAA Villefontaine, FR)

**License** © Creative Commons BY 3.0 Unported license
© Samuel Huron, Anaelle Beignon

This workshop was a hands-on activity to identify, describe and discover design patterns among the history of data physicalization. A design pattern is the description of the core of a solution to a recurrent problem. Design patterns are useful to describe, explain, understand, and generate novel solutions of a kind. In this activity we provide a card deck and a process to identify, describe, discover and structure data physicalization design patterns.

The data physicalization card deck (Figure 10) is composed of 280 cards, each one representing an artifact from the dataphyslist [1]. In a way this card deck is a physicalization of the data phys list. The data physicalization card game process (Figure 11) allows 10 to 100 researchers to engage in a hands-on open coding activity of these cards with several templates forms. The result of the activity is the identification of various design patterns and
<figure><img src="image_13.png" /><figcaption>Figure 10 From left to right: the data physicalization card deck; the design pattern templates; a summary of the activity process.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure 11 From right to left: picture of the activity; close up on a pattern; Example of the use of the cards by an others groups, here Sarah Hayes presenting.</figcaption></figure>

meaningful categories that describe the data physicalization design space from the perspective of the participants.

The data physicalization card game is free, open source, and accessible online [2]. The data physicalization card game has already been used during the IEEE VIS 2018 workshop “Toward a Design Language for Data Physicalization” [3]. During the seminar participant were using it for all sort of activities. This workshop is part of a workshop series about data physicalization including let’s get physical [5], vizkit [4], and other [6]. As a next step, I am going to document the results of the workshop and the use of the cards.

References

1. Pierre Dragicevic and Yvonne Jansen. List of physical visualizations. www.dataphys.org/list, 2012. Last accessed Jan 2019.

2. Samuel Huron and Anaelle Beignon. Data physical visualizations card game. data-physicalisation.github.io/data-phys-cards.html, 2018. Last accessed Nov 2018.

3. Jason Alexander Samuel Huron, Sheelagh Carpendale, Eva Hornecker, Trevor Hogan, Uta Hinrichs, editor. *Toward a Design Language for Data Physicalization, Workshop at IEEE Visualization (VIS)*, 2018.

4. Samuel Huron, Sheelagh Carpendale, Jeremy Boy, and Jean-Daniel Fekete. Using viskit: A manual for running a constructive visualization workshop. In *Pedagogy of Data Visualization Workshop at IEEE Visualization (VIS)*, 2016.

5. Samuel Huron, Pauline Gourlet, Uta Hinrichs, Trevor Hogan, and Yvonne Jansen. Let's get physical: Promoting data physicalization in workshop formats. In *Proceedings of the Conference on Designing Interactive Systems (DIS)*, pages 1409–1422, New York, USA, 2017. ACM.

6. Trevor Hogan, Uta Hinrichs, Yvonne Jansen, Samuel Huron, Pauline Gourlet, Eva Hornecker, and Bettina Nissen. Pedagogy & physicalization: Designing learning activities around physical data representations. In *Proceedings of the Conference Companion on Designing Interactive Systems (DIS)*, pages 345–347, New York, NY, USA, 2017. ACM.
<figure><img src="image_15.png" /><figcaption>Figure 12 Volker Schweinsfurth presenting one of his data physicalizations.</figcaption></figure>

## 4.7 Haptic Perception Experiments

Bernice Rogowitz (Visual Perspectives & Columbia University – New York, US)

License © Creative Commons BY 3.0 Unported license

© Bernice Rogowitz

One application of data physicalization is to communicate meaning in data. In visualization, Jacques Bertin explored how different visual dimensions could be used to effectively convey insight into category membership and magnitude. I presented a series of hands-on experiments that explored touch as a vehicle for communicating qualitative and quantitative mapping. The two-touch threshold experiment showed the participants that we can discriminate very fine differences with the fingertips, but may need up to 10x the separation on our forearms. Another experiment highlighted differences in texture discrimination using vision (very good), tactile discrimination without moving the finger (not so good) and tactile discrimination with movement (excellent). A third experiment demonstrated how tactile information can conflict with visual information. Two bottles of the exact same weight were presented. If the two bottles were placed on the palm, and judged by haptics alone, they seemed equal. If they were grasped without looking, the larger bottle was judged to be lighter. That is, extra weight had to be added to the large bottle for them to feel equal. When the bottles were grasped with visual as well as haptic cues, the larger bottle needed 50% more weight in order to be judged equal. This striking effect demonstrates that even a simple judgment of magnitude can be processed differently by the two senses, which has deep implications for data physicalizations that are both seen and felt.
## 4.8 Data Sculpture Exhibition

Volker Schweisfurth (Melies Art – Düsseldorf, DE)

License © Creative Commons BY 3.0 Unported license

© Volker Schweisfurth

During the breaks I laid out a collection of my (hand- to A4- sized) 3D-printed color
data sculptures (based mostly on World Bank country data and Forbes, see Figure 12); they
had been photographed by different participants and mostly covered topics from global
and economic contexts; such as the size of 130 economies (GIP/ppp; $) vs. country risk
for investments, the evolution of age pyramids of various countries, country risk profiles
over the years, business figures of multinational companies, etc. As participants in the
discussion of these physical objects, we emphasized the fact that, according to recent
research (including my reference to Hutmacher/Kuhbandner; University of Regensburg), the
brain stores information about the tactile sense longer than expected (“Detailed, durable,
long-term memory representations are stored as a natural product of haptic perception”).
In addition, I have pointed out that in my experience, the CAD files created for 3D
printing can be used elegantly in two additional ways without media discontinuities: The
creation of animations (.mov) and as input for mixed media (AR/VR) environments. In this
manner, the virtualization of economic and global phenomena can effectively complement
the physicalization of data- for teaching and industrial use, especially also for planning
purposes. Regarding a patent, which I hold in the field of active data physicalization, I
am unfortunately currently behind schedule in prototyping, mainly because the component
market (small OLED-/AMOLED) displays is characterized by the strong requirements and
demand on the part of VR glasses manufacturers.

## 4.9 Dagstuhl Data Physicalization

Pauline Gourlet (University of Paris VIII, FR)

Till Nagel (Hochschule Mannheim, DE)

Aurélien Tabard (Université de Lyon, FR)

As one of our workshop activities we got participants engaged in creating a data physicalization
on their personal relationship to this Dagstuhl seminar. In Figure 13 you can see participants
organized by how many Dagstuhl seminars they have attended (including this one) as well
as their travel time to the seminar.
<figure><img src="image_16.png" /><figcaption>Figure 13 Participants joining in a data physicalization on their relationship to the seminar.</figcaption></figure>

# 5 Working Groups

## 5.1 Data Physicalization Toward Sustainability and Peace

Yuri Engelhardt (University of Twente – Enschede, NL)

Brygg Ullmer (Clemson University, US)

Christian Freksa (Universität Bremen, DE)

Pauline Gourlet (University of Paris VIII, FR)

Samuel Huron (Telecom ParisTech, FR)

**License** © Creative Commons BY 3.0 Unported license
© Yuri Engelhardt, Brygg Ullmer, Christian Freksa, Pauline Gourlet, Samuel Huron

In this working group we explored the potentials of using data physicalizations to promote sustainability and peace. We discussed related initiatives that the members of our working group were affiliated or familiar with such as the UN sustainable development goals, the PRAM knowledge sharing network, the SDG DataViz Camp on Visualizing Inequalities, the Geospatial Information Section of the United Nations' project on "Making the World a Better Place with Maps", or the UNGlobalPulse's initiative on how data science and analytics can contribute to sustainable development. We also discussed the possibility to create a data physicalization challenge on the topic of sustainability and peace.
## 5.2 Physicalization vs. Visualization

David Kirsh (University of California – San Diego, US)

**License** © Creative Commons BY 3.0 Unported license
© David Kirsh

Our goal was to appreciate the relative strengths of the two approaches – in part by understanding how they differ in a principled way – and in part by a collection of looks at physicalization examples. There were four topics we opened.

1. **Embodiment:** how does having a body and using it in space, or using our hands, eyes in head, etc – how do these things affect how we frame problems, shape cognition and bias or facilitate how we think about things. Sometimes the body helps, sometimes it hinders or makes difficult certain transformations. Gestures help many things. We have volumetric understanding, knowing where we are etc

2. **Expressivity:** every representational system can be assessed by its expressivity – its capacity to encode / represent a certain dimensionality and form of data. We talked about what this means and in particular how there are so many dimensions in which data can be encoded in physical things. There are haptic dimensions – squeeze, texture, etc. there is shape – how expressive is that!!!, there is size, color, position, time.

3. **Readability:** the body plays a role in how we read off information. Essential that we note the interactive strategies available with a phyz and human body. There is no haptic reading without a haptic strategy to get it. These interactive strategies are often more complex than in viz. Embodiment also means that our bodies play a role in what we detect and read off.

4. **Concept formation:** phyz are often advertised as great for engendering new concepts or enabling insight into new sorts of relationships. We want to explore this much more.

## 5.3 The Role of Emotion and Engagement in Data Physicalization

Yun Wang (Microsoft Research – Beijing, CN)

Tim Dwyer (Monash University – Caulfield, AU)

Daniel Keefe (University of Minnesota – Minneapolis, US)

Petra Isenberg (Inria, Saclay, FR)

Roberta Klatzky (Carnegie Mellon University, US)

Eva Hornecker (Bauhaus-Universität Weimar, DE)

Jörn Hurtienne (Universität Würzburg, DE)

Leanne Elias (University of Lethbridge, CA)

Adrien Segal (California College of the Arts, US)

Steven Barras (Sonification – Ainslie, AU)

**License** © Creative Commons BY 3.0 Unported license
© Yun Wang, Tim Dwyer, Daniel Keefe, Petra Isenberg, Roberta Klatzky, Eva Hornecker, Jörn Hurtienne, Leanne Elias, Adrien Segal, Steven Barras

Visualisation, as a field of academic research, has for a long time sought rigour in terms of metrics and methodologies for the evaluation of data visualisation techniques. Typically, effectiveness or value of a visualisation is measured in terms of the efficiency of the technique in terms of user speed and comprehension in completing (relatively) basic data comprehension tasks. However, this pragmatic focus on readability and usability is missing consideration
of some qualities of visualisation that are more difficult to define but, arguably, also very
important. For example, a visualisation may not necessarily be the most efficient representa-
tion of a given data set, for a given task, but it may achieve greater user engagement through
evoking an emotion or simply enjoyment in the user. By contrast, recent discussions about
the possibilities of and proposals for data physicalisations have focussed on the importance of
emotional engagement with the embodiment of the data. But what do we mean by emotional
engagement, and why is it valuable? In this group, we explored these questions with
consideration from findings in cognitive and perceptual psychology, art, design, marketing,
activity theory, and user experience.

5.4 Sound in Data Physicalisation

Stephen Barrass (Sonification – Ainslie, AU)

License © Creative Commons BY 3.0 Unported license
© Stephen Barrass

In this working group we discussed about the potential application of sound and sonification
in data physicalisation. Sound is a natural consequence of the manual manipulation of
physical objects that conveys material properties, forces, modes of interaction and events
over time. Film sound is used to convey emotions, cultural references, and to direct attention.
Product sound is used for branding, guidance, engagement and other functional and aesthetic
purposes. This leads to the idea that sound could be a way to augment physicalisations in
similar ways. As an example an acoustic sonification is a dataset that is intentionally 3d
printed in the form of a sounding object, such as a bell or a singing bowl. Since acoustic
vibrations are a consequence of the 3D shape, the sound is effected by the dataset. Sounds
can also be produced by embedding a digital synthesis器, such as the Mozzi sonification
synth, inside an object. Sensors can be used to synthesis sounds in response to interactions
such as shaking, squeezing or striking the object. For example, Zizi the Affectionate Couch
is a piece of furniture that produces a range of different characteristic and emotional sounds
such as whining, yipping and purring in response to human proximity, sitting and stroking
its fur. Through further discussions in the workshop we heard about other examples, and
thought about other ways that sounds could be applied in physicalisation. Many thanks to
those who took part in an enjoyable and enlightening workshop.

6 Summary

The week went by very quickly, with impromptu evening discussions complementing the
work during the day. New collaborations and new friendships emerged from the mixing
of colleagues from so many different academic and artistic fields. Data physicalization is
a new discipline; its practitioners are still defining its scope and limits. The Dagstuhl
seminar will be seen as a critical moment in its crystallization, a formative time in the
evolution of its research and aesthetic agenda. The seminar had several possible outcomes
that were discussed in various working groups: developing a data physicalization challenge
for the community, grant collaborations, the idea of a follow-on seminar, journal articles, the
<figure><img src="image_17.png" /><figcaption>Figure 14 After-hours collaborations.</figcaption></figure>

possibility of a dedicated conference or symposium, and a special issue in a journal with a dedicated call-for-participation. These outcomes continue to be worked on by seminar participants and organizers. We thank Dagstuhl for providing the necessary context and organization to make our seminar a very fruitful, engaging, and exciting event.
<figure><img src="image_18.png" /></figure>

Participants

- Jason Alexander
  Lancaster University, GB

- Stephen Barrass
  Sonification – Ainslie, AU

- Stephen Brewster
  University of Glasgow, GB

- Sheelagh Carpendale
  University of Calgary, CA

- Tim Dwyer
  Monash University –
  Caulfield, AU

- Leanne Elias
  University of Lethbridge, CA

- Yuri Engelhardt
  University of Twente –
  Enschede, NL

- Sean Follmer
  Stanford University, US

- Denton Fredrickson
  University of Lethbridge, CA

- Christian Freksa
  Universität Bremen, DE

- Pauline Gourlet
  University of Paris VIII, FR

- Ian Gwilt
  University of South
  Australia, AU

- Sarah Hayes
  Cork Institute of Technology, IE

- Uta Hinrichs
  University of St Andrews, GB

- Trevor Hogan
  Cork Institute of Technology, IE

- Eva Hornecker
  Bauhaus-Universität Weimar, DE

- Samuel Huron
  Telecom ParisTech, FR

- Jörn Hurtienne
  Universität Würzburg, DE

- Petra Isenberg
  Inria – Saclay, FR

- Daniel Keefe
  University of Minnesota –
  Minneapolis, US

- David Kirsh
  University of California –
  San Diego, US

- Roberta Klatzky
  Carnegie Mellon University, US

- Till Nagel
  Hochschule Mannheim, DE

- Bettina Nissen
  University of Edinburgh, GB

- Lora Oehlberg
  University of Calgary, CA

- Dietmar Offenhuber
  Northeastern University –
  Boston, US

- Laura Perovich
  MIT – Cambridge, US

- Bernice E. Rogowitz
  Visual Perspectives & Columbia
  University – New York, US

- Daniel K. Schneider
  University of Geneva, CH

- Volker Schweisfurth
  Melies Art – Düsseldorf, DE

- Adrien Segal
  California College of the Arts, US

- Aurélien Tabard
  Université de Lyon, FR

- Barbara Tversky
  Columbia Teachers College –
  New York, US & Stanford
  University, US

- Brygg Ullmer
  Clemson University, US

- Andrew Vande Moere
  KU Leuven, BE

- Karin von Ompteda
  The Ontario College of Art and
  Design University, CA

- Yun Wang
  Microsoft Research – Beijing, CN

- Wesley J. Willett
  University of Calgary, CA

<figure><img src="image_19.png" /></figure>