Workshop Summary

CHI 2013: Changing Perspectives, Paris, France

Displays Take New Shape: An Agenda
for Future Interactive Surfaces
Jürgen Steimle

Daniel Leithinger

Abstract

MIT Media Lab

MIT Media Lab

Cambridge, MA USA

Cambridge, MA USA

steimle@media.mit.edu

daniell@media.mit.edu

Hrvoje Benko

Pattie Maes

Microsoft Research

MIT Media Lab

Redmond, WA USA

Cambridge, MA USA

benko@microsoft.com

pattie@media.mit.edu

This workshop provides a forum for discussing emerging trends in interactive surfaces that leverage alternative display types and form factors to enable more expressive interaction with information. The goal of theworkshop is to push the current discussion forward towards a synthesis of emerging visualization and interaction concepts in the area of improvised, minimal,curved and malleable interactive surfaces. By doing so,
we aim to generate an agenda for future research anddevelopment in interactive surfaces.

Alvaro Cassinelli
University of TokyoTokyo, Japan
Alvaro_Cassinelli@ipc.i.utokyo.ac.jp

Ivan Poupyrev
Disney Research
Pittsburgh, PA USA
ivan.poupyrev@disneyresearch.com

Hiroshi Ishii
MIT Media Lab
Cambridge, MA USA
ishii@media.mit.edu

Copyright is held by the author/owner(s).
CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris, France.
ACM 978-1-4503-1952-2/13/04.

Keywords
Interactive surface; projected interfaces; improvised display; minimal display; flexible display; on-body display;everywhere display; input; tracking

ACM Classification Keywords
H.5.m Information interfaces and presentation (e.g., HCI).

Motivation
The physical objects we encounter every day come in atremendous variety of materials and shapes, which allows us to interact with them expressively and effectively. In contrast, today’s interactive surfaces (such as
mobile phones, tablets, or wall displays) support input
through touch on flat surfaces, usually guided by a GUIinterface. While we interact on the surface, we rarely

3283

Workshop Summary

CHI 2013: Changing Perspectives, Paris, Franceinteract with it, which is in sharp contrast with how weinteract with physical objects.

for input [4, 9, 8] and by actuating the display for output [10, 7].

To start with, any display, being itself a physical object,already provides a set of physical affordances that canbe exploited in a meaningful way to extend the range ofpossible interactions with the displayed content. Looking at the problem from the other side, we could saythat understanding and exploiting the affordances ofany object considered as a potential display may provide the basis for a true revolution in the field.

These research trends are accompanied by ongoingadvances in projection and display hardware, smart
materials and programmable matter that hold great
promise for this field. They allow for ever increasing
resolution and brightness, thin, flexible form factors,
and haptic output. In addition, novel sensor technologies are emerging that enable capturing the physicalenvironment and interaction with varied shapes [6, 5,12].

Everyday artifacts, parts of the body or entire architectural spaces can be recruited by a smart projection system or fitted with active displays; in the future deformable three-dimensional objects could materialize today’s virtual 3D representations; but also simple, minimalistic cues are promising means for changing theperception of our environment. It may be useful to bearin mind that future interfaces may even become indistinguishable from the represented data itself – as outlined by the “Radical Atoms” vision [7].

Figure 1. Examples of novel
forms of interactive surfaces[3,6,5,8,10]

This workshop explores emerging research on alternative display types and form factors, with the goal ofgenerating an agenda for future interactive surfaces. It
covers the following topics: Improvised displays investigate the transformation of everyday objects, surfaces
in the environment and body parts into displays
through sensing [6, 5, 12], projection [6, 5, 3] andhaptic augmentation [1]. Minimal displays [5] provide
more simple cues, e.g. a laser pointer automaticallyhighlighting an object in the environment that the user
is searching. Interaction with curved and malleable
displays investigates how to communicate through theshape itself [2, 13], by the user deforming the display

We argue that this not only promises to transform ourunderstanding of displays, but has the potential to fundamentally change how we perceive information andinteract with it. Improvised, minimal and on-body displays allow for in-situ representation and access to digital information that is seamlessly integrated with thereal world. Enabling the display shape to conform to thecontent rather than forcing content to fit to a predefined shape improves the mapping of representation toinformation. Malleable shapes allow people to leverage
their hands more expressively, while transformableshapes can conform to varying places and modes ofinteraction.

Workshop Topics and Goals
The workshop intends to provide a forum for brainstorming about future types and forms of displays,
based on currently emerging technologies. Moreover
there is a need for deeper and more systematic understanding of how we leverage novel displays and newshapes for output, how we interact with them andpromising fields of application. The main goal of ourworkshop is to address these points by bringing togeth-3284

Workshop Summary

CHI 2013: Changing Perspectives, Paris, France

er experts from academia and industry to discuss challenges, brainstorm, and identify an agenda for futureresearch. By advertising broadly we hope to engage
people from academia and industry with backgrounds
from various disciplines, including engineering, design,material sciences, cognitive science, arts, and business.
Discussion in this workshop will revolve around enabling technologies, applications and interaction techniques in the major areas of novel interactive surfaces,
as introduced above. We will specifically focus on thefollowing cross-cutting issues. Each of them addresses
one aspect that requires more systematic understanding and will help us define a research agenda:
Novel forms of interactive surfaces
Emerging technologies promise to alter our understanding of what displays and interactive surfaces are. As
outlined above, there are currently rapid advances inenabling technologies. This not only involves display
and tracking technologies (e.g. depth-based projection
mapping [6], combination of tangible objects with stereoscopic visualizations [3], integrated laser-based projection and capturing of the display surface [5], sprayable displays [11]), but also smart materials andstretchable electronics, as well as novel forms of actuation. We aim at generating an integrated high-levelpicture of these technologies, provide the basis forbrainstorming about how these technologies enableredefinition of interactive surfaces. This comprises thefollowing questions: What are future forms of interactive surfaces? Which objects and surfaces can beturned into displays? Which visualizations and interactions do they enable? In turn, this will allow us to de-fine an agenda for future research and development
into enabling technologies.
Emerging interactions
Emerging research has presented first interaction techniques in several applications, each providing a spotlight in the design space. However, the field is lacking amore systematic understanding of successful application domains and interaction techniques. Our goal is toshare lessons learnt from recent projects and to use
these as a basis for generating a first structuring of thefield. We aim at (a) identifying commonalities of interaction techniques across different forms of novel interactive surfaces, based on the affordances involved, (b)discussing whether first standards are emerging, (c)ultimately being able to provide recommendations onwhat applications and interactions are most promising
to be successful.
General purpose displays vs. specific-purpose
tangibles
Interactive surfaces that merge with physical objects
add haptic and/or tangible characteristics to visual information displays. As such, they can be characterized
as a hybrid that is situated in-between general purposegraphical user interfaces, and tangible user interfaces,
which usually are targeted to specific purposes. We aim
at characterizing these hybrid characteristics in moredetail, with the ultimate goal of giving recommendations of how to integrate the competing goals of generalness and specialization.
To foster stimulating discussions, the agenda reduces
plenary talks to a minimum, leaving ample time for3285

Workshop Summary

CHI 2013: Changing Perspectives, Paris, Franceinteractive work. A keynote and four short talks by participants of the workshop will set the stage for extended phases of work in focus groups on the above topics.
A final plenary session will allow us to integrate resultsfrom focus groups into a first taxonomy and a set ofrecommendations for future work.

Expected Outcomes
The goal of the workshop is to push the current discussion forward towards synthesis of emerging visualization and interaction concepts and identification of successful application domains. By doing so, we aim at
generating an agenda for future research and development, both into interaction concepts and into enabling
technologies. The workshop will also provide a venue
for establishing a community of researchers and practitioners interested in the topic. To support communication and collaboration after the workshop, the workshop
wiki will remain available for continued use and planning of next steps.
The results of this workshop will be shared on theworkshop webpage, including all position papers andmaterials created during the workshop. Moreover, we
will create a participatory poster to be displayed during
the CHI conference, on which attendees of CHI can post
their comments and suggestions. In addition, we plan
to invite workshop participants to expand their submission for a special issue of a journal.

References[1] O. Bau, I. Poupyrev, M. Le Goc, L. Galliot, and M.
Glisson. Revel: tactile feedback technology for augmented reality. SIGGRAPH’12 Emerging Technologies.

[2] H. Benko. Beyond flat surface computing: challenges of depth-aware and curved interfaces. Multimedia
2009.
[3] H. Benko, R. Jota, A. Wilson. Miragetable: freehandinteraction on a projected augmented reality tabletop.
CHI 2012.
[4] A. Cassinelli, M. Ishikawa. Khronos projector. SIGGRAPH 2005 Emerging technologies.
[5] A. Cassinelli, A. Zerroug, Y. Watanabe, M. Ishikawa,J. Angesleva. Camera-less smart laser projector. SIGGRAPH 2010 Emerging Technologies.
[6] C. Harrison, H. Benko, A. Wilson. Omnitouch: wearable multitouch interaction everywhere. UIST 2011.
[7] H. Ishii, D. Lakatos, L. Bonanni, J.B. Labrune. Radical atoms: beyond tangible bits, toward transformablematerials. interactions, 19(1), 2012.
[8] M. Khalilbeigi, R. Lissermann, M. Mühlhäuser,J.Steimle. Xpaaand: Interaction techniques for rollabledisplays. CHI 2011.
[9] B. Lahey, A. Girouard, W. Burleson, R. Vertegaal.
Paperphone: understanding the use of bend gestures inmobile devices with flexible electronic paper displays.
CHI 2011.
[10] D. Leithinger, D. Lakatos, A. DeVincenzi, M. Blackshaw, H. Ishii. Direct and gestural interaction with relief: a 2.5d shape display. UIST 2011.
[11] D. Saakes, M. Inami, T. Igarashi, N. Koizumi, R.
Raskar. Shader printer. SIGGRAPH 2012 Emerging
Technologies.
[12] M. Sato, I. Poupyrev, C. Harrison. Touché: enhancing touch interaction on humans, screens, liquids,
and everyday objects. CHI 2012.
[13] M. Weiss, S. Voelker, C. Sutter, J. Borchers.
Benddesk: dragging across the curve. ITS 2010.

3286