UBICOMP/ISWC ’16 ADJUNCT, SEPTEMBER 12-16, 2016, HEIDELBERG, GERMANY

Solar System - Smooth Pursuit
Interactions using EOG glasses
Junichi Shimizu
Keio University
Yokohama, Kanagawa, Japan
jun.shimi@kmd.keio.ac.jp

Thad Starner
Georgia Institute of Technology
Atlanta, Georgia, United States
thad@cc.gatech.edu

Juyoung Lee
KAIST
Daejeon, Korea
ejuyoung@kaist.ac.kr

Woontack Woo
KAIST
Daejeon, Korea
wwoo@kaist.ac.kr

Murtaza Dhuliawala
Georgia Institute of Technology
Atlanta, Georgia, United Statesmurtzdhulz@gmail.com

Kai Kunze
Keio University
Yokohama, Kanagawa, Japan
kai.kunze@gmail.com

Andreas Bulling
Max Planck Institute forInformatics
Saarbrücken, Germany
bulling@mpi-inf.mpg.deAbstract
Solar System implements smooth pursuit eye movement
interactions on commercial smart glasses using electrooculography. The system requires no calibration and little to notraining. We present a prototype implementation, describe
initial user tests and show several application scenarios forhands-free eye gaze interactions.

Author Keywords
Eye tracking; gaze interaction; wearable computing.

ACM Classification Keywords
H.5.m [Information interfaces and presentation (e.g., HCI)]:
Miscellaneous

Introduction

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
UbiComp/ISWC’16 Adjunct, September 12-16, 2016, Heidelberg, Germany
ACM 978-1-4503-4462-3/16/09.
http://dx.doi.org/10.1145/2968219.2971376

369

Although the eye is one of the fastest and most accurate
muscles in the human body, eye-based interactions are still
far from being mainstream [1]. Many users find using eye
movements for direct interactions wearying [5]. Eye movements for controlling the interface must be sufficiently different from regular eye movements so as not to be confused,which can lead to an interface that feels unnatural.
In recent years, however, researchers have found smooth
pursuits can work well for direct eye interactions as following moving targets is a natural and intuitive eye behavior.

UBICOMP/ISWC ’16 ADJUNCT, SEPTEMBER 12-16, 2016, HEIDELBERG, GERMANY

Research so far has focused on using optical eye trackers
to create smooth pursuit interfaces. Although there havebeen advances to make these devices smaller, they areoften still quite obtrusive and have limitations in regard tobattery power.
Figure 1: Solar System requires
the user to track an orbiting cursor
with the eyes to select an option
whose icon is (optionally) renderedin the center of the orbit.

This paper contains the following contributions: (1) We assess the possibility of implementing a smooth pursuit interface similar to Orbits [2] using Electrooculography, (2) we
describe the algorithm we use to create a prototype system on commercially available EOG glasses (J!NS MEME)
and (3) we show the functionality of the system in an initialdemonstration with three users. We also describe potential
applications and use cases. Our system is calibration freeand works on commercial, unobtrusive smart eyewear.

Related WorkFigure 2: MEME: smart glasses
with 3 electrodes to measure eyemovements.

Several researchers are exploring smooth pursuit for eye
gaze interactions [3, 5, 6, 7]. In their 2013 paper, Vidal etal. [5] describe six usage scenarios for the technique. TheOrbits paper by Esteves et al. [2] is the most closely related effort to this work as it utilizes circular movements
to distinguish between prompts. However, Orbits focuseson interactions with a smart watch and uses a large andobtrusive optical eyetracking system. Špakov et al. [3, 7] introduce widgets for smooth pursuit similar to our cookbook
application described below. A good resource about thefundamental mechanics of smooth pursuit eye movements
is given by Robinson [4].
We are not aware of research that implements smooth pursuit interfaces using electrooculography on a consumer device. Other publications focus on optical eyetracking which
is more expensive, requires more energy, is sensitive tosunlight, and often requires camera alignment before use.

Figure 3: Orbits with different
phases370

Hardware
For our study, we use the J!NS MEME, an off-the-shelf pairof eyeglasses sold in Japan, which is able to measure eye
movement using three electrodes in the nosebridge thatare in contact with the skin (see Figure 2). When worn, theJ!NS MEME is difficult to distinguish from a normal, unaugmented pair of eyeglasses. The device also incorporates
an accelerometer and gyroscope. Data can be sent to aremote device over USB or wirelessly using Bluetooth. Calibration is not required, and the device fits most users withminimum adjustment to the nosepads.
We calculate the horizonal EOG value by subtracting thevalue returned from the left nosepad electrode (L) from theright (R). The vertical signal is calculated by subtracting thevalue from the bridge electrode (B) from the average of thetwo nosepad electrode. In practice, the horizontal signal
tends to have larger magnitude than the vertical signal forthe same amount of eye deflection; however, that difference
is not a difficulty when comparing the relative motion of twosystems.

Smooth Pursuit Detection Algorithm
Eye interfaces typically rely on the absolute position of theeye position relative to a given prompt. Recovering a static
eye gaze point using EOG has significantly lower accuracy than with optical gaze trackers. However, with smooth
pursuit interfaces, only relative movement is needed to select between prompts. Here, we focus on the phase of therepeated eye movements as the user tracks the cursors
around the prompts.
Figure 1 demonstrates the Solar System concept. Two options are displayed on a screen. A cursor orbits each of
the options. The cursors’ movements are rendered out ofphase with each other. We detect the user’s smooth pursuit

SESSION: DEMOS

eye movements in the EOG data and compare them to thecursors’ positions to determine which of the two the user istracking.
More specifically, we first establish a center of rotation byaveraging the vertical and horizontal EOG signals over asmall window of time. After the center is established, we
calculate the angle of motion relative to that center for each
incoming EOG sample. We compare those angles to theangles of the two prompts and select the one with the highest correlation.

Figure 5: Example user interface for cooking.

Signal Level Evaluation

Figure 4: Calculated phase anglevalue and Sensor value from J!NSMEME with reference Orbit values

Figure 4 demonstrates the signal calculated from the EOG
data. As can be seen, the phase of the data is easy to detect and correlate with the motions of the orbiting cursors
on the screen. Note that while we present the method using
only two choices with a 180 degree phase difference, morechoices could be distinguished by using smaller phase differences or varying the speed of the cursor movement. In
addition, different shapes of cursor movement can be used.
For example, one cursor could be moving in a square pattern while another moves in a circular pattern. The process of selecting which prompt is being tracked remains thesame; simply choose whichever prompt’s motion has thehighest correlation with the EOG data.

Architecture
For portability, we implemented the detection process towork in web browsers. As the data from J!NS MEME is sent
over Bluetooth by default, we modified the data transfer
method so that the receiving machine can forward the J!NSMEME data over a socket, and the data be compared to thecursor movement rendered in the web browser. The detection process occurs in real time which allows an interface
designer to apply it to many scenarios.

371

Application Scenario
We did a proof-of-concept with three users using Solar System. The initial test displayed two different cursors travelingclockwise and 180 degrees out of phase (see Figure 3). Allusers could select either cursor intentionally.
Figure 5 demonstrates one of the scenarios in which Solar
System might be used. While cooking from a recipe, thechef’s hands are often covered in food. Solar System allows
the chef to turn the pages of a cookbook without touchingit. Here, we have used the cursors with two different phases
previously described so that the cook can select a recipeand navigate between the next and previous steps for foodpreparation. The chef can prepare dishes and cut or cook
foods for the recipe while reading and interacting with the
cookbook hands-free.

Future Work and Conclusion
We are extending our experiments and preparing somerigorous performance tests. We are interested in how fast
selection can occur using the MEME as well as the minimum visual angle a rotating cursor can subtend and still be
distinguishable in the EOG data. Additionally, we wish toUBICOMP/ISWC ’16 ADJUNCT, SEPTEMBER 12-16, 2016, HEIDELBERG, GERMANY

investigate how many cursors can be distinguished simultaneously by varying phase, speed of rotation, and shape ofmovement.
We are also investigating different application scenarios.
Any situation requiring hands-free interaction, such as soldering, maintenance work, circuit testing, and assistive interfaces, is potentially interesting.
We have presented an implementation of smooth pursuit
eye gaze interaction on commercial electrooculography
glasses. Visualization of the data and testing in an initial
application scenario suggest that the method is viable.

REFERENCES
1. Richard A Abrams, David E Meyer, and Sylvan
Kornblum. 1989. Speed and accuracy of saccadic eye
movements: characteristics of impulse variability in theoculomotor system. Journal of Experimental
Psychology: Human Perception and Performance 15, 3(1989), 529.
2. Augusto Esteves, Eduardo Velloso, Andreas Bulling,and Hans Gellersen. 2015. Orbits: Gaze Interaction forSmart Watches using Smooth Pursuit Eye Movements.

372

In UIST. 457–466.
3. John Paulin Hansen, Haakon Lund, Florian Biermann,
Emillie Møllenbach, Sebastian Sztuk, and Javier SanAgustin. 2016. Wrist-worn pervasive gaze interaction.
In ETRA. ACM, 57–64.
4. D. A. Robinson. 1965. The mechanics of humansmooth pursuit eye movement. In The Journal ofPhysiology. 569–591.
5. Mélodie Vidal, Andreas Bulling, and Hans Gellersen.
2013. Pursuits: spontaneous interaction with displays
based on smooth pursuit eye movement and movingtargets. In Ubicomp. 439–448.
6. Mélodie Vidal, Andreas Bulling, and Hans Gellersen.
2015. Pursuits: Spontaneous Eye-Based Interaction for
Dynamic Interfaces. ACM SIGMOBILE Mobile
Computing and Communications Review 18, 4 (2015),
8–10.
7. Oleg Špakov, Poika Isokoski, Jari Kangas, Deepak
Akkil, and Päivi Majaranta. 2016. PursuitAdjuster: an
exploration into the design space of smooth pursuit–based widgets. In ETRA. ACM, 287–290.