Workshop Summary

CHI 2012, May 5–10, 2012, Austin, Texas, USA

The 3rd Dimension of CHI (3DCHI):
Touching and Designing 3D User Interfaces
Hrvoje Benko

Frank Steinicke

Microsoft Research

University of Würzburg

Redmond WA, USA

Würzburg, Germany

benko@microsoft.com

frank.steinicke@uni-wuerzburg.de

Antonio Krüger

Ken Anderson

Innovative Retail Laboratory,Intel Labs

DFKI GmbH, Saarbrücken

Hillsboro, OR USA

Germany

ken.anderson@intel.com

antonio.krueger@dfki.de
Jonna Häkkilä
Daniel Keefe

Nokia Research Center

University of Minnesota,Oulu, Finland

Minneapolis, MN 55455, USA

Jonna.Hakkila@nokia.com

keefe@cs.umn.edu
Leena Arhippainen
Jean-Baptiste de la Rivière

Minna Pakanen

IMMERSION SAS,

Center for Internet Excellence

Bordeaux, FranceOulu, Finland

jb.delariviere@immersion.fr

firstname.lastname@cie.fi

Abstract

In recent years 3D has gained increasing amount ofattention - interactive visualization of 3D data has become
increasingly important and widespread due to therequirements of several application areas, andentertainment industry has brought 3D experience to thereach of wide audiences through games, 3D movies andstereoscopic displays. However, current user interfaces (UIs)
often lack adequate support for 3D interactions: 2D
metaphors still dominate in GUI design, 2D desktop systems
are often limited in cases where natural interaction with 3D
content is required, and sophisticated 3D user interfaces
consisting of stereoscopic projections and tracked input
devices are rarely adopted by ordinary users. In the future,novel interaction design solutions are needed to bettersupport the natural interaction and utilize the special
features of 3D technologies.
In this workshop we address the research and industrial
challenges involved in exploring the space where the flat
digital world of surface computing meets the physical,
spatially complex, 3D space in which we live. The workshop
will provide a common forum for researchers to share theirvisions of the future and recent results in the area ofimproving 3D interaction and UI design.

Keywords

3D User Interfaces; Virtual Reality; 3D Interaction; MultiTouch Technology; Stereoscopic Displays
Copyright is held by the author/owner(s).
CHI’12, May 5–10, 2012, Austin, Texas, USA.
ACM 978-1-4503-1016-1/12/05.

ACM Classification Keywords

H.5.2 [Information Interfaces and Presentation]: UserInterfaces - Interaction styles

2695

Workshop Summary

CHI 2012, May 5–10, 2012, Austin, Texas, USA

Introduction & Motivation
Within past few years, 3D technology has made its
appearance through several areas of entertainment andinformation technology – 3D movies are made inincreasing numbers, 3D cameras are opening content
creation possibilities, Nintendo has launched a game
console 3DS with a 3D display, LG a 3D phone model,and more. Current computing applications are still
heavily bound to the legacy of 2D UI design, and on theinput side, 3D interaction systems are often
cumbersome. Current 3D user interfaces, for example
provided by virtual reality (VR) systems, consist of
stereoscopic projection and tracked input devices which
are often systems giving priority to immersion andequipment rather than collaboration and simplicity. As
3D is intuitively connected to our experiences with thereal world, it provides means for immersive UIs andmore holistic interaction design. Touch technology hasreceived considerable attention in the last years, inparticular for 2D user interfaces, but it has also great
potential for exploring complex content, e.g. threedimensional data, in an easy and natural manner. Using
stereoscopic displays allow users to perceive 3D data ina more immersive way. Interacting with objects thatare displayed at different depths relative to the display
surface is a challenging task even in VR-based
environments. In this workshop we want to address theresearch as well industrial challenges that arise wheninteracting with 3D information. Especially, we wish toconsider the 3D UI design challenges, as well asinteraction which is performed on a 2D or even 3D
touch surface.

Related Work
Today mice and keyboards are still the primary meansof navigating, exploring, and interacting with 3D UIs

and complex 3D datasets, even though they are often
limited for these purposes, in particular when it comesto direct manipulation. In contrast, virtual reality
systems using tracking technologies and stereoscopic
projections of 3D synthetic worlds have shown great
potential to improve exploration of complex 3Ddatasets. However, the most effective methods ofinteracting with today’s complex 3D environments are
not yet known, for instance, in a stereoscopic
representation of a 3D scene it may be hard to accessdistant objects [5,9].
The prior art in 3D GUI design has mainly considered
use of desktops with 2D displays, including 3D effects
such as tilting and folding windows [1]. Most 3D
applications also include 2D user interface elements,
such as menus, texts and images, in combination with
3D content. While 3D content usually benefits fromstereoscopic visualization, 2D GUI items often do nothave associated depth information. Stereoscopic output
can also increase the cognitive load. Lo et al. reportthat with stereoscopic images, users took more time tomake a decision than with a 2D presentation in anexperiment, where the participants had to judge therealism of the images shown to them [7]. In [6], theuse of other visual cues is recommended together withdisparity effect with autostereoscopic UIs.
Today, several hardware solutions exist that allow
multi-touch input on surfaces of different sizes, such as[2, 3], Apple iPad or Microsoft Surface. Although somesurfaces sense pressure or depth information, the useof multi-touch in current applications for interactingwith three-dimensional data sets, e.g., with geospatial
data is challenging, since movements in depth are only
supported rudimentarily with low resolution. While bi-2696

Workshop Summary

CHI 2012, May 5–10, 2012, Austin, Texas, USA

Figure 1: Example screenshots of multi-touch interaction in the context of 3D stereoscopic display: (left) A 3D
medical visualization is displayed on a vertical stereoscopic screen so that it appears to the user to “float” in the air
above the table. Interaction is performed by touching its “shadow” on the multi-touch table. (right) A userinteracting by means of simple 2D gestures on a monoscopic touchscreen, while visualizing occlusion-free 3D
stereoscopic objects floating above the surface at an optically correct distance.
manual interaction with 2D data has been studied
extensively [5], it is still not clear how to use these
input concepts for complex 3D data.
For this reason, recently several researchers havebegun to address the problem of 3D interaction on 2D
multi-touch surfaces. Schöning et al. have consideredsome of the challenges of multi-touch interaction withstereoscopically rendered projections, and in 2009 and2011 IMMERSION launched commercially available
multi-touch enabled stereoscopic projection displays [5,8]. Figure 1 shows two applications on stereoscopic
multi-touch displays. In other related work, Grossman
et al. considered touch-based interaction, andpresented a suite of interaction techniques for use witha spherical 3D volumetric display; they also proposed ataxonomy for 3D interaction on interactive table-tops[4]. One limitation of these approaches is that theinteraction and visualization is constrained to almost

zero parallax because the plane of the interactive
surface limits the interaction space more or less to the2D surface. Relying on a cubic multi-touch volume forindirect tactile interaction has also been investigated[4]. Valkov et al. have presented techniques to shiftobjects onto the 2D surface as the user moves in frontof a stereoscopic projection system. Furthermore, they
analyzed touch behavior for objects displayed on astereoscopic projection screen [9].

Design and Interaction Challenges
When developing systems for 3D HCI, many design
challenges arise. The workshop addresses (but is notlimited to) e.g. the following issues:
3D Interaction with Touch-Sensitive Devices
Multi-touch interfaces can represent a good trade-off
between intuitive, constrained interaction on a touch
surface providing tangible feedback, and unrestricted

2697

Workshop Summary

CHI 2012, May 5–10, 2012, Austin, Texas, USA

natural interaction without cumbersome
instrumentation. In particular, stereoscopic display of
3D data provides an additional depth cue, but thechallenges and limitations for touch interaction in thiscontext have yet to be sufficiently investigated. We willaddress basic as well as advanced interaction
techniques that make it possible to interact withstereoscopically displayed objects using fluid, gestural
input from a 2D surface.
Perception and Action with 3D Interaction
Human factors is another crucial design challenge for
combining touch technology with stereoscopic displays.
We will discuss perceptual discrepancies that occur
when users interact with objects displayed at different
depths relative to the touch surface, and how toachieve good user experience design for such UIs. In
addition it should be considered what kind ofaffordances, metaphors, and navigation structures need
to be developed in order to design intuitive and usable
3D UIs.
Portable and Mobile Devices
The use of portable devices, especially multi-touchenabled ones, opens up new applications for 3D UIsand interaction. 3D GUIs for small size screens require
careful design. We will address mobile and portableconcepts for 3D interaction, especially with stereoscopic
display environments, which can make it possible forusers to interact with objects e.g. in front of thedisplay.

[2] Coffey, D., Korsakov, F., Keefe, D. F. (2010): Low
Cost VR Meets Low Cost Multi-Touch. Proceedings ofInternational Symposium on Visual Computing, 2010.
[3] Rekimoto, J. (2002): SmartSkin: an infrastructure
for freehand manipulation on interactive surfaces,
Proceedings of ACM CHI, pp. 113-120.
[4] Grossman, T. & Wigdor, D. (2007), Going Deeper: aTaxonomy of 3D on the Tabletop, IEEE International
Workshop on Horizontal Interactive Human-Computer
Systems, pp. 137-144.
[5] Hachet M., Bossavit B., Cohe A., de la Rivière, J.B.
(2012): Toucheo: Multitouch and Stereo Combined in a
Seamless Workspace, ACM UIST (accepted).
[6] Huhtala, J., Karukka, M., Salmimaa M., Häkkilä, J.
Evaluating Depth Illusion as Method of Adding
Emphasis in Autostereoscopic Mobile Displays. In Proc.
MobileHCI 2011.
[7] Lo, C. H. and Chalmers, A. 2003. Stereo vision forcomputer graphics: the effect that stereo vision has on
human judgments of visual realism. In Proc. SCCG '03.
ACM, 109-117.
[8] de la Rivière, J.B., Kervégant, C., Orvain, E., Dittlo,
N. : CubTile, a multi-touch cubic interface. Proceedingsof ACM VRST, pp 69 – 72.
[9] Valkov, D., Steinicke, F. Bruder, G. Hinrichs K.:(2010): 2D Touching of 3D Stereoscopic Objects, ACM
CHI 2011.

References[1] Chapuis, O., Roussel, N. Metisse is not a 3D
desktop! In Proc. UIST’05, ACM 2005.

2698