Understanding Touch Selection Accuracy on Flat and Hemispherical
Deformable Surfaces
Felipe Bacim1,2 *, Mike Sinclair1 †, Hrvoje Benko1 ‡1

2

Microsoft Research, Redmond, WA, USA and Center for Human-Computer Interaction, Virginia Tech

ABSTRACT
Touch technology is rapidly evolving, and soon deformable,
movable and malleable touch interfaces may be part of everyday
computing. While there has been a lot of work on understanding
touch interactions on flat surfaces, as well as recent work about
pointing on curved surfaces, little is known about how surface
deformation affects touch interactions. This paper presents thestudy of how different features of deformable surfaces affecttouch selection accuracy, both in terms of position and control ofthe deformation distance, which refers to the distance traveled by
the finger when deforming the surface. We conducted three
separate user studies, investigating how touch interactions on a
deformable surface are affected not only by the compliant force
feedback generated by the elastic surface, but also by the use ofvisual feedback, the use of a tactile delimiter to indicate themaximum deformation distance, and the use of hemisphericalsurface shape. The results indicate that, when provided withvisual feedback, users can achieve sub-millimeter precision for
deformation distance. In addition, without visual feedback, users
tend to overestimate deformation distance especially in conditions
that require less deformation and therefore provide less surface
tension. While the use of a tactile delimiter to indicate maximum
deformation improves the distance estimation accuracy, it does
not eliminate overestimation. Finally, the shape of the surface also
affects touch selection accuracy for both touch position anddeformation distance.
Keywords: Touch; flexible displays; pointing.
Index Terms: H.5.2. Information interfaces and presentation(e.g., HCI): User Interfaces: Input Devices and Strategies,
Interaction Styles.

knowledge of the impact that surface deformation has on touch
interaction is still limited.
In this paper, we investigate how touch interactions are affected
when the surface is non-rigid and when surface deformation isused as an additional dimension in touch input. While we are not
the first to propose adding surface deformations as an additional
input dimension to touch input [3, 13, 19, 22], to our knowledge,
this is a first systematic evaluation of the effects of surface
deformation on touch performance.
The main contribution of this paper is in gathering data about
touch selection tasks on deformable touch surfaces and inproviding insights that can be used in the design of such interfaces
in the future. There are many aspects about a deformable touch
surface that merit deeper investigation. For example, what kind of
feedback is necessary for users to achieve high levels of accuracy?
Is compliant force feedback enough? Can user’s estimation of
deformation distance be improved by using a rigid barrier at the
pre-set maximum surface deformation (we refer to this feedback
as a tactile delimiter)? When is it necessary to have a visualindication of how far your finger is pushed into the deformablesurface (visual feedback)? How does the surface shape affectprecision in pointing? How does multi-touch function on a
deformable surface? Given such a rich design space, a complete
analysis of this space is beyond the scope of this paper.
Instead, in order to answer some of these questions, we built
several prototype devices and applications, and performed threeuser studies to evaluate touch selection accuracy on deformabletouch surfaces focusing on three characteristics: (1) the impact ofvisual feedback, (2) the impact of tactile delimiters, and (3) the
impact of a deformable surface shape (hemisphere vs. flat shape).
Figure 1 illustrates the different measures we investigated.

1 INTRODUCTION
With the dissemination of tablets and smartphones, touch-based
interaction is already an integral part of everyday computing.
Given such wide adoption of touch and multi-touch technology,
much research has gone into extending the touch interaction
vocabulary by adding additional input dimensions to touchsensing. For example, numerous projects demonstrated thebenefits of non-flat touch surfaces [2], touch pressure [15],tracking of fingers and gestures above the surface [9], and the useof deformable surfaces [19]. However, while there is a deep
understanding of touch interactions on typical flat rigid surfaces[10], and some understanding of the effects of curvature [16],* e-mail: fbacim@vt.edu†
e-mail: sinclair@microsoft.com‡
e-mail: benko@microsoft.com

Figure 1: Different measures investigated in our experiments.

The main findings presented in this paper can be summarizedas:• Users can perform touch selection tasks on deformable surfaces
with sub-millimeter accuracy when provided with some form ofvisual feedback;• Users overestimate deformation distance when no visualfeedback is provided, meaning they deform the surface morethan they think they do;• The use of the rigid tactile delimiter improves deformation
distance estimates, but does not eliminate overestimation;

• The shape of a deformable surface affects both position andtravel path in touch selection accuracy.
2 RELATED WORKAs mentioned before, much research has gone into extending
touch interaction capabilities. For example, non-flat shaped
objects such as a sphere [2] allow users to interact withapplications that can be better mapped to the surface
approximating the object. In the case of a sphere, for example, thesurface of the Earth can be directly mapped to the display toprovide more intuitive interactions. BendDesk [23] combines
vertical and horizontal touch screens with a curved, touch-enabledarea. Another example is the OmniTouch [6], which allows users
to transform any surface within their reach into a touch-screen.
While such examples extend touch interactions onto non-flat
surfaces, the interactions themselves remain completely in two
dimensions.
Another solution for this problem is provided by pressuresensitive touch devices, such as the UnMousePad [15]. With it,
users can use force on the touch surface as an extra dimension,and new types of gestures and interactions can be used. This
concept is presented by Ramos et al. [14] using pressure with astylus to define new gestures. Another alternative is to use
tangible devices above the surface [8]. The problem with thepressure sensing approach, however, is that while the depth
dimension can be provided, it flattens out 3D interaction, andimportant cues such as Z-position in space are only provided byforce. This could make it difficult, for example, for users toestimate the amount of force necessary to travel a determinedamount along the depth axis [1, 18].
One approach to address these problems is to provide different
interactions above the touch surface. For example, Spindler et al.
[17] provide a position tracked surface that is used as a lens tovisualize different parts of a 3D dataset. This approach is similar
to the one implemented in our volume visualization demo. The
continuous interaction space [12] integrates touch and gestures inthe space above the surface. LightSpace [24] extends this idea
further by allowing users to interact between displays. Although
this approach provides distance as a new dimension, it forces theuser to perform interactions mid-air, without force-feedback. With
this, precision issues may arise for both position and distancefrom surface.
Shape changing devices can also be used to augment normal
touch interfaces. For example, Harrison and Hudson [7] used
deformability to provide tactile feedback for buttons on a touch
screen. More recently, Follmer et al. [4] presented the concept ofJamming User Interfaces, in which a malleable device can bejammed into different shapes and be used as input devices.
Leithinger et al. [11] showed the use of gestures to control 2.5D
display.
Finally, surfaces that deform as the user applies force to themcan be used to overcome these limitations. For example, Peschke
et al. [13] showed how a Microsoft Kinect can be used to track
deformation of large surface areas and demonstrated an
application that uses deformation to interact with virtual spheres.
Another example was presented by Watanabe et al. [22], which
tracked deformation of an elastic vertical surface using fiducial
markers projected onto the surface with an IR projector and acamera with an IR-pass filter. A few applications were
demonstrated, such as 3D surface sculpting and volume slicing.
We implemented these two applications using our prototypes. The
Khronos Projector [3] is perhaps the most similar in terms of
implementation of hardware. A camera with an IR illumination

ring is used to detect variations in light reflected from a malleablesurface to track deformation. The setup is used in an applicationthat gives temporal control over video data based on the
deformation of the surface. Finally, Stevenson et al. [19]presented an inflatable hemispherical display that supported
multi-touch, and used infrared illumination to determine surface
deformation. The main application for this was Google Earth, inwhich the surface inflates/deflates to match the Earth’s curvature
at different levels of zoom. Our hemisphere prototype is different
in that it has constant air pressure and shape when not touched,and employs a hard internal wall to indicate the end of travel isachieved (the tactile delimiter). These design decisions wererelated to one of our goals: to evaluate how much shape affects
our perception of deformation distance. This is also related to themain difference between previous work in deformable surfaces
and our work. Our focus is not on the system or the applications
that could use devices like these, but understanding how
deformation affects touch selection tasks on touch surfaces.
While touch selection tasks on regular touch surfaces are fairly
well studied [10, 21], there is still little to no understanding of
how the extensions presented here affect these tasks. Roudaut etal. [16] presented an evaluation of touch on curved surfaces that
explored different types of curvature (convex and concave) anddifferent amounts of curvature, and evaluated how well users
could perform touch selection tasks on them. The results wereused to provide recommendations to interaction designers, such asa method for calculating the minimum size of a button when it ispresented in specific conditions of curvature and position. Our
focus, on the other hand, is on deformable surfaces and how
visual feedback, the use of a tactile delimiter, and the shape affecttouch performance.
3 DEFORMABLE SURFACE PROTOTYPES
To enable us to study how deformation of the surface affects thetouch pointing task, we designed a series of custom hardware
prototypes and custom touch tracking software. In particular, we
needed a platform which would let us experiment with both flatand curved deformable surfaces as well as offer an easy way toprovide or remove the tactile delimiter at varying depths to denotemaximum deformation to the user.
Our current test hardware (Figure 2) is large and not easily
adaptable for real-world products; however, we designed it
specifically as a test framework for our experiments to enable us
to understand human touch behaviour when interacting with
deformable surfaces.
3.1 Hardware
We designed and built two deformable surfaces: flat andhemispherical (Figure 2). Both prototypes consist of a compliant
top surface (latex + Spandex combination), a tactile delimiter, anda wide angle camera and projector unit underneath to track touch
locations and deformation amounts. Our flat prototype also has an
adjustable height mechanism for the tactile delimiter,
For the deformable surface, we chose latex because it provides
elasticity and compliant force feedback. However, we found that
latex alone causes specular reflections when stretched, which inturn generated problems for our IR camera-based tracking of
fingertips. Our solution was to glue a layer of Spandex stretchable
fabric on the inside of the deformable surface (facing the camera)
to create a diffuse reflecting surface for tracking.
For touch sensing, we tested a wide range of setups, including
depth sensing cameras; however, we settled on using a
configuration which combines an IR camera and a projector
which share the same optical path through a wide angle secondarylens (Figure 3). We mounted a custom IR light emitter mounted
around a wide angle lens, and we use the amount of light reflectedby the user’s fingers to compute the distance to the surface (forvalidation see Section 3.1.1). The reason for using a wide angle
lens was to minimize the distance between the surface and the IR
illumination source, which in turn gives us a wider dynamic range
and greatly improved accuracy for calculating deformation
distance. This setup is similar to the Sphere project by Benko etal. [2]. The projector allowed us to use our prototypes as output
devices as well.
Latex outer
covering

and hemispherical prototypes, with a confidence interval (at 95%)
of 0.22mm. For 22.5° inclination on the hemisphere, the final
measured deformation distance was 29.57mm, with a confidence
interval (at 95%) of 0.14mm. Finally, for 45° inclination on thehemisphere, the final measured deformation distance was
30.03mm, with a confidence interval (at 95%) of 0.22mm.

Spandex matte

Spandex matte Latex outerreflection screen
reflection screen covering

Adjustable end of
travel acrylic plate

Acrylic
hemispherefor end of
travelInflatingtube

Figure 4: The device used to measure precision for deformation
distance and position of our prototypes. Note: the device wasangled to test different inclinations on the hemisphere to movealong the surface normal.

Figure 2: The two prototypes used in the studies: (left) the flat
surface design, and (right) the inflatable hemisphere.
IR-LED
illuminator ring

Wide-angle
lensIR
camera50% beam splitter

3.1.2 Force Response Function
To further characterize our devices, we used the same validation
apparatus to measure the necessary force to deformation functionof the different prototypes. For both flat and hemispherical
prototypes, we measured the amount of force necessary to deformthe surface at the center of the device, starting from 0mm andmeasuring up to 30mm with a 5mm interval. We used a small
precision weight-scale underneath the prototypes, measured theforce and converted to Newtons (N). Figure 5 shows the different
measurements and the linear regression used to find the force todeformation function for each prototype. Since verticaldisplacements in a horizontally configured rubber membrane
adhere to Hooke’s Law (i.e., linear force vs. displacement
behavior) [5] we expected to find linear relationship between theforce and displacement.

Projector

Figure 3: Setup used for projection and tracking.

With both the setup and materials decided, we designed andbuilt the final set of prototypes (seen in Figure 2). On the other
hand, the new prototypes, both flat and hemispherical, provided atactile delimiter and required a similar amount of force to reachthe same amount of deformation. Finally, while the hemispherical
prototype provided a delimiter by using a fixed acrylic dome
beneath the deformable surface, the flat prototype featured aheight adjustable/removable acrylic plate. All prototypes could
also be used as an output device using the optically aligned
projector.
3.1.1 Prototype Validation
To ensure our prototypes provided us with enough accuracy, we
built a device to simulate touch with a constrained travel andmeasurable path (Figure 4). For every target on the flat andhemispherical surface we repeated the same task of moving thedevice away from the surface and then pushing it until it touched
the tactile delimiter located at 30mm a total of 50 times.
For the hemisphere, the device was also angled to test different
inclinations on the hemisphere and to move along the hemispheresurface normal. This test was done iteratively to adjust the
tracking algorithm values. For 0° inclination, our device averagemeasurement was 29.89mm for 30mm deformation on both flat

Figure 5: Measured force for different deformation distances.

While the measurements for the hemisphere prototype clearlyshow a linear response for deformation, the flat prototype seems
to present an exponential response up to 10mm, and linear
response after that. This is slightly different from what we
expected, but we believe the reason for that is the very small
amount of weight necessary to perform small deformations.
Overall, since our measured results show near-linear response forboth flat and hemisphere surface, we believe that our subsequent
experimental results (Sections 4) can generalize well for any otherlinear or near-linear response deformable surfaces.
3.2 Software
The experimental software was written using OpenGL for
rendering and a customized version of openFrameworks fortracking of both position and deformation. Tracking was
performed in two parts: first, tracking of touch position (without
considering deformation distance) was done with the blob
detection and tracking techniques available in the library; next,
illumination at the center of each detected blob was recorded andused to determine new thresholds to be used in the first part.
Using the difference in illumination (the closer the surface was tothe camera, the brighter it was) and the calibration processdescribed in the previous section, we are able to determine
deformation distance. Performing this process for each detected
touch point increases accuracy and allows us to track deformation
distance for multiple blobs precisely.

4.1 Experimental Setup
In addition to the prototypes described in the previous section, ourstudies had a few other components that together composed thefinal setup. Participants sat in a chair in front of the device, atapproximately the same height from the ground. This was done inorder to guarantee that participants would see the targets from thesame position. The enter key (painted red) of a numeric keyboard
was used to start and end each trial. Participants used their left
hand to press the key and their right hand to perform the task. A
30mm reference ruler was placed beside the prototype so thatparticipants would know the size of the maximum required travel
distance. The participant setup is illustrated in Figure 7.

3.2.1 DemosTo test how well our system would fare in real world applications,
we built two prototype applications: a terrain modeler, and a
volume visualization tool (Figure 6).

Figure 7: Participant point-of-view during the study: the circles
indicate the numeric keyboard with the red key and the reference
ruler (green); and the arrow shows the depth meter displayed on
the vertical screen with a triangle marker denoting the required
deformation distance for a given task.

Figure 6: Two prototype applications developed to test anddemonstrate how deformable surfaces can be used.

The terrain modeler allows users to manipulate a threedimensional mesh, using our flat prototype as an input device andmonitor as the output. The surface of the device is mapped to themesh, and deformations on the surface affect the terrain. The
volume visualization tool maps volumetric data, such as stacks of
medical imaging, to the surface of the hemispherical prototype.
Deforming the surface along the surface creates a lens-likevisualization tool allowing users to navigate the volume byviewing the 2D image slice of the 3D data at the selected depth.
4 EVALUATION OF TOUCH SELECTION PERFORMANCE
We conducted three experiments comparing the use of visualfeedback, a tactile delimiter, and different shapes for deformablesurfaces. Visual feedback allows us to evaluate how accurate
users can be if provided with information about deformation
distance. This can be the baseline for other studies, and shows the
maximum performance. The use of a tactile delimiter relates toreal world applications, in which users may have a limit in the
deformation of the surface. It also shows the benefits of not only
providing tactile feedback when the user touches the surface withno deformation, but providing information about what the
maximum deformation is. Finally, the motivation for studying a
different shape is to gather insights about how the initial shape ofthe surface affects the travel path, from the initial touch position
to final touch position. We evaluated the task of selection withdifferent positions and required deformation distance. Sections
4.1, 4.2, and 4.3 refer to the overall experimental design, and thefollowing sections discuss design and results of each experiment.

Two vertical displays were used: one for the participant and one
for the experimenter. In addition, the deformable surface was a
projected display as described above. The target was displayed
directly on the surface using the projection system and consistedof a white cross. The vertical display, directly in front of them,
contained a virtual meter to illustrate deformation distance. A
triangle marker was used to provide an indication of the required
travel distance, and visual feedback provided with an animatedblue bar. An additional display was used to provide information
about the system, task and performance to the experimenter.
4.2 Participants
A total of 12 participants were recruited to perform the all threestudies in one session, and they were compensated with a $10
coupon. Participants' ages ranged from 18 to 30 years old, with a
median age of 20. Five participants were female. All participants
were right-handed.
We used a factorial within-subject design with repeatedmeasures across all three studies, and participants performed a
total of 300 tasks, with 71 different conditions. The order ofpresentation of the different studies was counterbalanced.
4.3 Procedure
Upon arrival, participants were screened for right-handedness andproceeded to complete a background questionnaire. After that,they were shown the experimental setting and given initial
instructions. All participants completed a total of 32 practice trials
with different travel distances and visual feedback on the flat
surface before starting the experiment. This was provided for allparticipants allowing them to experience the travel distances using
visual feedback. After completing the initial tutorial, they were
given a practice session on the current condition in which they
had to practice all different travel distances in that condition.
Before starting the trials, participants were reminded that theyshould try to be as accurate as possible, perform the trials usingtheir fingertips and not rest their palms on the device. This wasdone to minimize possible confounds such as using the device tostabilize their hands for more position accuracy. Additionally,
before the start of each task, participants were asked to remove
fingers from the touch surface so that they would always have tostart touching and deforming the surface when starting a task. An
error message would appear until there were no fingers detected.
The task procedure was always the same: first press the red key
to start the task; then touch the target and match the travel
distance indicated by the marker; and finally press the red key
again to confirm.
At the end of each study session, participants rested for two
minutes while the next study was setup. Then they moved on to
the next study, following the same protocol until all three studies
were completed. Finally, the participant filled out a postexperiment questionnaire, comparing all studies and rating thedifficulty of performing the pointing task in each one of them.
4.4 Study 1 – Evaluation of Deformation DistanceAccuracy with Visual Feedback
The overall goal of this study was to determine how accurate canparticipants be in deforming the surface with constant direction(vertically) and a certain distance if given visual feedback. Thistask gives us an upper bound on user’s accuracy given precise
visual feedback and the results serve as a baseline for the otherstudies.
The main hypothesis for this study was that users would be able
to perform touch selection accurately with an interval of 2mm
between required deformation distances (H1). This number isdefined by Hong et al. [20] as an estimation of the humandetection threshold for fingertip movement, and we expectedvisual feedback to help users achieve this threshold. We also
expected distance to not affect the position precision for our task
(H2).
4.4.1 Experimental Design
For study 1, there was only one independent variable:
Deformation Distance (from 0mm to 30mm, every 2mm for a
total of 16 different deformation distances). This study was
performed using the flat prototype, and visual feedback asprovided for all conditions. Targets were always in the center ofthe surface to ensure the same material resistance across allconditions (requiring ~3.8N to achieve maximum deformation).
The order of presentation of the deformation distances was
randomized. Each condition was repeated five times, giving a
total of 80 trials.
4.4.2 Results
We first analyzed the measured deformation distance. A repeatedmeasures ANOVA found a significant effect of measured
deformation distance for the different required Deformation
Distances (F15,165=6481.5, p<0.001). Further analysis showed ahigh fit coefficient (R²=0.99) of the mean measured deformation
distance to the task deformation distance. This confirms our
hypothesis H1 that visual feedback can help users surpass thehuman detection threshold for fingertip movement.
However, the measured deformation distance has a slight offset
when the task deformation distance is small. To analyze this effectfurther, we calculated the difference between task deformation
distance and measured deformation (deformation distance error).
Again, a repeated measures ANOVA showed significant effects(F15,165=4.518, p<0.001) for error in the measured deformation
distance. The graph in Figure 8 shows that the less deformation atask required, the more participants overestimated it.

Figure 8: Mean deformation distance error. Error bars represent
standard error.

Finally, we also analyzed data relative to the offset in position
from the initial touch position to the final touch position.
Surprisingly, the required Deformation Distance had a statisticallysignificant effect on the offset distance (refuting H2). A factorial
repeated measures ANOVA showed significant effects ofDeformation Distance for the overall offset distance(F15,165=13.549, p<0.001). Figure 9 shows the mean offset forevery user with different required Deformation Distances.
Participants tended to push the target to the left and back the more
deformation distance was required by the task.

Figure 9: Offset error for each one of the required Deformation
Distances in Study 1. Each point represents the mean offset error
for one user.

4.5 Study 2 – Evaluation of Accuracy with Tactile
Delimiter
The goal of this study was to find how effective is compliant
passive force feedback alone for users to estimate deformation
distance, and how much better would they get if a firm tactiledelimiter was used to provide a reference point at the maximum
distance. This evaluation was designed to give us insight into
whether users can perform selection tasks at different deformation
distances without visual feedback, and how many different
distances they can effectively estimate.
The main hypothesis for this study was that users would be
precise in estimating different deformation distances withoutvisual feedback, but would not perform tasks accurately without
the tactile delimiter (H3). For this reason, we chose to use fivedifferent distances, in a greater interval than in the previous study(7.5mm). Our secondary hypothesis for deformation distance wasthat the tactile delimiter at the maximum deformation distancewould improve precision (H4). Third, we expected deformation
distance and the use of a tactile delimiter to not affect the position
precision for our task (H5).

4.5.1 Experimental Design
For study 2, there were two independent variables: Deformation
Distance (0mm, 7.5mm, 15mm, 22.5mm, 30mm), and Tactile
Delimiter (either with or without). This study was also performed
using the flat prototype. Visual Feedback was not provided during
these tests. Again, targets were always placed in the center toensure the same lateral resistance. The order of presentation ofDeformation Distance was counterbalanced, and Travel Delimiter
randomized. Each condition was repeated four times, for a total of
40 trials.
4.5.2 Results
The first analysis for this study is about measured deformation
distance. A factorial repeated measures ANOVA found asignificant effect for the different required Deformation Distances(F4,44=251.248, p<0.001), which means that users can estimate
different distances based on surface deformation, and Tactile
Delimiter (F1,11=72.992, p<0.001), which means that the use of atactile delimiter affects the estimated deformation distance.
Further analysis shows that the interaction between Deformation
Distance and Tactile Delimiter was significant (F4,44=16.232,
p<0.001). Figure 10 shows this interaction, in which the mean
measured deformation distance is closer to that required by thetask when tactile delimiter is used. These results confirm ourhypotheses H3 and H4 that error is reduced when tactile delimiteris available.

back the more deformation distance was required by the task.

Figure 11 – Offset error for each one of the required Deformation
Distances in Study 2. Each point represents the mean offset error
for one user.

4.6 Study 3 – Evaluation of Accuracy with a
Hemispherical Touch Surface
The main goal of this last study was to shed light on how user’s
performance differs on a different deformable shape. While manydifferent shapes are possible, we chose to experiment on ahemispherical shape, since it allows us to control and evaluate
surface curvature more precisely. This shape, in addition tohaving been previously explored in other touch performanceexperiments, has continuous and uniform curvature. By analyzing
touch selection in different positions of the hemisphere, we areable to understand how the curvature affects travel and path
precision. The position of the different points on the hemisphere isrepresented in Figure 12.

Figure 12: Different target positions on the surface of the sphere,with 3 latitudes (0°, 22.5°, 45°) and 4 longitudes (0°, 90°, 180°,270°), for a total of 9 positions.

Figure 10: Mean deformation distance with and without tactiledelimiter, and required deformation distance represented by theline. Error bars represent standard error.

Figure 10 also shows that participants had the tendency tooverestimate the distance when they were required to deform
short distances, similarly to Study 1. In this case, however, thiseffect is accentuated by the lack of visual feedback. A factorial
repeated measures ANOVA found significant effects on
deformation distance error for Deformation Distance(F4,44=14.455, p<0.001), Tactile Delimiter (F1,11=72.994,
p<0.001), and their interaction (F4,44=16.232, p<0.001).
The analysis of the position offset between initial and final
touch positions showed that Tactile Delimiter did not affect theoffset distance. On the other hand, like in Study 1, Deformation
Distance showed a significant effect overall (both with andwithout the Tactile Delimiter) in the offset distance (F4,44=23.86,
p<0.001), which refutes H5. The interaction between the twovariables was not significant. Figure 11 shows the mean offset
distance for every user with each different Deformation Distance,with and without touch feedback. Following the same tendency
presented in Study 1, participants pushed the target to the left and

The hypotheses for this study were that: (H6) there would be asignificant effect of position of the target on the hemisphere on
path accuracy; and (H7) deformation distance accuracy would be
influenced by the position of the target. These hypotheses can be
related to the results presented in [16], which showed thatdifferent levels of curvature result in different amounts of error(e.g., convex surfaces result in more accuracy).
4.6.1 Experimental Design
For study 3, there were two independent variables: Deformation
Distance (0mm, 7.5mm, 15mm, 22.5mm, 30mm), and Target
Position (3 latitudes: 0°, 22.5°, 45°; 4 longitudes: 0°, 90°, 180°,
270°; total of 9 positions). We carefully chose the points on thehemisphere to keep the amount of force required to reachmaximum deformation constant. This study was performed using
the hemispherical prototype. The Touch Delimiter was provided
for all conditions, and Visual Feedback was not. The order ofpresentation of both Deformation Distance and Target Position
was randomized. Each condition was repeated four times, for a
total of 180 trials.
4.6.2 Results
The analysis of measured deformation distance for this study
takes into account both required Deformation Distance and Target
Position on the hemisphere. A factorial repeated measures

ANOVA found a significant effect of measured deformation
distance for the different required Deformation Distances(F4,44=123.573, p<0.001), which means that users were able toestimate different distances for the different required Deformation
Distances, and Target Position (F8,88=15.923, p<0.001), whichmeans that Target Position affected the estimation.
The analysis of the interaction between Deformation Distanceand Target Position also shows significant effects (F32,352=4.976,
p<0.001). This means that the measured deformation distances forthe different required Deformation Distances changed for eachone of the different target positions. This confirms our hypothesis
H7). This effect can be seen in Figure 13, which shows how themeasured deformation distance compares to the required
Deformation Distances for each one of different target positions.

Figure 14: Offset error for each one of the different Target Positionsin Study 3, with Deformation Distance constant at 30mm. Eachpoint represents the mean offset error for one user.

Figure 13: Mean deformation distance for different target positions
on the sphere. Different shapes indicate different required
deformation distances, and error bars represent standard error.

The analysis of the position offset between the initial and final
touch positions showed that, overall, Deformation Distance did
not have an effect on offset distance. On the other hand, as
expected, Target Position had a significant effect on the offset
distance (F8,88=5.037, p<0.001), confirming our hypothesis H6.
Figure 14 illustrates the mean offset distance for each user anddifferent target positions, when the requested Deformation
Distance was the maximum distance (30mm). As can be seen, thedevice shape affects the participant’s ability to select targets
differently than the flat shape.
4.7 User Preference
It is also interesting to look at subjective ratings the participants
gave for the different studies and conditions. Participants rated the
difficulty of the conditions in a scale from 1 (very hard) to 7 (veryeasy). We performed Wilcoxon Signed Rank tests on each of thequestions. The conditions in which participants were provided
with visual feedback (mdn=6) were rated significantly easier toperform than without visual feedback and without tactile delimiter(mdn=3, n=12, W=75, p<0.01) and with the hemispherical shape(mdn=3, n=12, W=76, p<0.01). The conditions that provided atactile delimiter (mdn=6) were also considered significantly easierthan without tactile delimiter (mdn=3, n=12, W=78, p<0.01) andwith the hemispherical shape (mdn=3, n=12, W=78, p<0.01). No
statistically significant differences were found in preference
between visual feedback and the use of a tactile delimiter, andbetween shape and not using a tactile delimiter.
5 DISCUSSION AND CONCLUSIONS
The results for the visual feedback experiment show thatparticipants were able to achieve sub-millimeter precision for
travel when using it. If the application being designed requires ahigh level of accuracy from the user, some form of visualfeedback is a clear requirement.

Without visual feedback, participants are able to identify fivedifferent deformation distances within 30mm, and even thoughaccuracy was low, meaning that they missed the target
deformation distance, precision was high, meaning that the error
was consistent. With this in mind, we think it is possible to use
deformation as a new dimension for touch even if there is novisual feedback or a tactile delimiter, given that the system iscalibrated to account for error.
On the other hand, the tactile delimiter proved to be an
important asset in improving on the accuracy issue even when therequired deformation distance is far from it, since there was asignificant difference in the amount of error that was seen whenusing the delimiter. Based on our results, we can say that without
a tactile delimiter, the number of levels that users will be able toidentify reduces if given the same maximum deformation.
However, even with the help of the tactile delimiter to indicate
maximum distance, participants overestimated the distance thatthey needed to travel for every task. Our experiments do not offer
any evidence on why such systematic overestimation is occurring.
We speculate that this could be an indication that the users are
simply poor at distance estimation given these target distances, orthat there was not enough compliant force feedback forparticipants to estimate the required distance. Nevertheless, more
research on this aspect of deformable surfaces is necessary inorder to determine the explanation of these phenomena.
Another interesting finding is that, even for flat surfaces,
requiring users to push into the surface to deform it causes them tochange the position of their finger. While the offset error is notlarge (within 2mm), it may be enough to cause a selection error if
the target requires too much deformation. Additionally, the shapeof a device not only affects the participant’s ability to point
precisely, but also to precisely deform the surface. Greater
inclines for travel resulted in less accuracy for positions, but
increased accuracy for deformation distance. This indicates thatcompensation or even some sort of visual feedback is necessary toachieve higher levels of accuracy overall. To avoid that, it ispossible to use the method in [16] to calculate a minimum button
size for a user by taking into account the surface deformation.
An alternative solution for the position problem is to move theprojection surface. For example, the projection surface might nothave to be on the deformable surface (latex), but instead on thehard surface underneath it. With head-tracking and a translucent
surface on top, it could even be possible to use motion parallax tocreate the illusion that the target is in between both surfaces. We
speculate that such approach could increase the final touch
position accuracy, but reduce the overall path accuracy. However,further investigation and comparison with the approach used inthis paper is necessary.
In addition, as mentioned earlier, our system was designed forexperimenting and investigating how touch is affected by surface
deformation. The surface material (latex) is not ideal for many
types of interactions (e.g., sliding your finger when the surface isdeformed). The form of the flat surface results in non-uniform
distribution of tension. In this case, it means that the center
produces less compliant force feedback than any other point of thesurface, and the closer the user is to the edges, the more tension
will be generated in the direction of that edge. The use of LEDs
and illumination for determining deformation distance requiresvery precise and careful calibration, and the use of a wide-angle
lens for the flat surface is less than ideal since resolution towards
the edges is reduced. Different solutions for these problems
should be found before moving efforts on to making newinteractions.
Finally, we believe that this paper provides initial evidence on
the feasibility of using deformable flexible surfaces for touch
interaction. While previous work focuses on possible applications
and basic interactions, we show what problems arise when
deformation is used as an additional dimension, and discuss how
to avoid or minimize them. Our approach was to investigate someof the core issues of performing touch interactions on a
deformable surface, particularly when trying to account for a
specific deformation. Because devices like these do not exist
beyond research labs and there are only a few applications forthem, we believe it is premature to discuss the canonical tasks for
deformable surfaces.
In the future, we plan to evaluate other features of deformablesurfaces, such as compliant force feedback, different distances forthe tactile delimiter, and also design and evaluate new interactions
with deformable surfaces.
REFERENCES[1]. Benko, H., T.S. Saponas, D. Morris, and D. Tan, Enhancing input on

tangible user interfaces, in Proc. of ACM ITS2011: Kobe, Japan. pp.
45-48.
[9]. Hilliges, O., S. Izadi, A.D. Wilson, S. Hodges, A. Garcia-Mendoza,
and A. Butz, Interactions in the air: adding further depth tointeractive tabletops, in Proc. of ACM UIST2009, ACM: Victoria,
BC, Canada. pp. 139-148.
[10]. Holz, C. and P. Baudisch, Understanding touch, in Proc. of ACM
CHI2011: Vancouver, BC, Canada. pp. 2501-2510.
[11]. Leithinger, D., D. Lakatos, A. DeVincenzi, M. Blackshaw, and H.
Ishii, Direct and gestural interaction with relief: a 2.5D shapedisplay, in Proc. of ACM UIST2011: Santa Barbara, California,
USA. pp. 541-548.
[12]. Marquardt, N., R. Jota, S. Greenberg, and J.A. Jorge, The continuousinteraction space: interaction techniques unifying touch and gesture
on and above a digital surface, in Proc. of INTERACT2011,
Springer-Verlag Berlin: Lisbon, Portugal. pp. 461-476.
[13]. Peschke, J., F. Göbel, T. Gründer, M. Keck, D. Kammer, and R.
Groh, DepthTouch: an elastic surface for tangible computing, in
Proc. of AVI2012: Capri Island, Italy. pp. 770-771.
[14]. Ramos, G., M. Boulos, and R. Balakrishnan, Pressure widgets, in
Proc. of ACM CHI2004: Vienna, Austria. pp. 487-494.
[15]. Rosenberg, I. and K. Perlin, The UnMousePad: an interpolating
multi-touch force-sensing input pad, in ACM SIGGRAPH 2009
papers2009: New Orleans, Louisiana. pp. 1-9.
[16]. Roudaut, A., H. Pohl, and P. Baudisch, Touch input on curved
surfaces, in Proc. of ACM CHI2011: Vancouver, BC, Canada. pp.
1011-1020.
[17]. Spindler, M. and R. Dachselt, PaperLens: advanced magic lensinteraction above the tabletop, in Proc. of ACM ITS2009: Banff,
Alberta, Canada. pp. 1-1.
[18]. Srinivasan, M.A. and J.-s. Chen, Human performance in controllingnormal forces of contact with rigid objects. Advances in Robotics,
Mechatronics, and Haptic Interfaces, 1993. 49: pp. 119-125.
[19]. Stevenson, A., C. Perez, and R. Vertegaal, An inflatable

and above the interactive surface with muscle sensing, in Proc. ofhemispherical multi-touch display, in Proc. of ACM TEI2011:

ACM ITS2009, ACM: Banff, Alberta, Canada. pp. 93-100.

Funchal, Portugal. pp. 289-292.

[2]. Benko, H., A.D. Wilson, and R. Balakrishnan, Sphere: multi-touch[20]. Tan, H.Z., M.A. Srinivasan, C.M. Reed, and N.I. Durlach,interactions on a spherical display, in Proc. of ACM UIST2008:

Discrimination and identification of finger joint-angle position using

Monterey, CA, USA. pp. 77-86.
[3]. Cassinelli, A. and M. Ishikawa, Khronos projector, in ACM

active motion. ACM Trans. Appl. Percept., 2007. 4(2): pp. 10.
[21]. Vogel, D. and P. Baudisch, Shift: a technique for operating pen-

SIGGRAPH 2005 Emerging technologies2005: Los Angeles,

based interfaces using touch, in Proc. of ACM CHI2007: San Jose,

California. pp. 10.

California, USA. pp. 657-666.

[4]. Follmer, S., D. Leithinger, A. Olwal, N. Cheng, and N. Cheng,[22]. Watanabe, Y., A. Cassinelli, T. Komuro, and M. Ishikawa. The

Jamming User Interfaces: Programmable Particle Stiffness anddeformable workspace: A membrane between real and virtual space.

Sensing for Malleable and Shape-Changing Devices. , in To appear
in Proc. of ACM UIST. 2012.
[5]. Fulop, W., The rubber membrane and the solution of Laplace's
equation. British Journal of Applied Physics, 1955. 6(1): pp. 21.

in Proc. of IEEE TABLETOP 2008. 2008.
[23]. Weiss, M., S. Voelker, C. Sutter, and J. Borchers, BendDesk:
dragging across the curve, in Proc. of ACM ITS2010: Saarbrücken,
Germany. pp. 1-10.

[6]. Harrison, C., H. Benko, and A.D. Wilson, OmniTouch: wearable[24]. Wilson, A.D. and H. Benko, Combining multiple depth cameras andmultitouch interaction everywhere, in Proc. of ACM UIST2011:

projectors for interactions on, above and between surfaces, in Proc.

Santa Barbara, California, USA. pp. 441-450.

of ACM UIST2010: New York, New York, USA. pp. 273-282.

[7]. Harrison, C. and S.E. Hudson, Providing dynamically changeable
physical buttons on a visual display, in Proc. of ACM CHI2009:
Boston, MA, USA. pp. 299-308.
[8]. Hennecke, F., F. Berwein, and A. Butz, Optical pressure sensing for