2013 International Conference on Cyberworlds

Clickable Virtual Button in Real Space

Atsushi Sugiura Masahiro Toyoura Xiaoyang Mao
Interdisciplinary Graduate School of Medicine and Engineering
University of Yamanashi
Kofu, Yamanashi, Japan{g12dhl02, mtoyoura, mao}@yamanashi.ac.jp
Abstract— Clicking a virtual object is the most fundamental
and important interaction in Augmented Reality (AR). This
paper presents a new natural click interface for AR systems.
Through a primary study, we found the acceleration offingertips provides cues for detecting click gesture andsucceeded in use it for recognizing natural click gestures with asingle camera. The proposed technique was evaluated througha virtual calculator application.

I.

depth, we detect the pointing by checking whether the 2D
position of the fingertip has been within the region of thebutton for a given time period.
C. Acceleration-based Click Detection
Due to the 3rd fact observed in the primary study, it isvery difficult to detect the click gesture with a single camera
or with existing finger/hand gesture recognition methods
using trajectory and speed analysis. However, through acareful analysis of the 2nd fact, we found that all click
gestures share a common feature that the fingertip stops
suddenly at the end of the gesture. By the complementary
use of both the 2D acceleration and the approximatedacceleration in depth direction, we succeeded in detecting
click gestures at a high success rate.

INTRODUCTION

We propose a natural input interface for AR systems.
Some systems force users to make unintuitive gestures suchas pinching with the thumb and forefinger for the ease of
detection [1]. We succeeded in detecting natural gesture ofclicking a virtual object with a single camera by tracking theacceleration of a user’s fingertip.
II.

III.

NATURAL CLICK GESTURE

A. Primary Study
Virtual buttons were displayed on a video-see-throughHMD with a single camera for 12 subjects. The subjects
were asked to “click” virtual buttons with their pointing
fingers without any training. During the operation, thefinger is always displayed in front of the virtual button. No
other visual, aural or haptic feedbacks were provided for theinteraction between the finger and the buttons. We observed
following facts in the study.
1.
2.

3.

The depth at which subjects tried to click the buttons
varies by subjects. They could not perceive the correct
position of the buttons in depth.
The click gesture is similar to but more exaggerated
than a tapping gesture. Since there is no haptic feedback
about the interaction with a virtual button, the subjects
try to represent the clicking by raise their finger up,
push down quickly and then stop on the button
suddenly.
The click gesture includes 3D movements which varyby subjects as well as the positions of the buttons.

(a) A snapshot of the(b) AR Calculator
experimentFigure 1 Clicking virtual buttons in AR environmentIV.

CONCLUSIONS

A natural clicking interface for augmented reality has
been presented. As a future work, we plan to extend thetechnique for supporting other gesture based intuitive
interactions in AR system.

B. Visual Feedback of Pointing
The observed facts suggest the importance of visualfeedback. To provide an interface consistent withconventional point-and-click metaphor, we change the colorand size of a button when it is pointed(Figure1(b)). Since auser cannot place his/her fingertip at an intended position in978-1-4799-2245-1/13 $31.00 © 2013 IEEE
DOI 10.1109/CW.2013.73

EXPERIMENT

Five subjects in their 20s or 30s joined the evaluation
experiment. Figure 1(a) is a snapshot of the experiment andFigure 1(b) is what a subject saw on HMD. They were asked
to perform a task of using the virtual calculator. Each subject
clicked 80 times and on average, 4.0% could not be detected
and 0.4% was falsely detected.

REFERENCES
[1]

384

A. Wilson, Robust Vision-Based Detection of Pinching for One andTwo-Handed Input, UIST, 2006.