UBICOMP/ISWC '15 ADJUNCT, SEPTEMBER 7–11, 2015, OSAKA, JAPANMEME – Eye Wear Computing to
Explore Human Behavior
Kai Kunze

Abstract

Yuji Uema

Graduate School of Media Design Graduate School of Media Design
Keio University
Keio University
Yokohama, 223-8526 Japan
Yokohama, 223-8526 Japan
uema@kmd.keio.ac.jp
kai@kmd.keio.ac.jp

Katsuma Tanaka

Koichi Kise

Graduate School of Engineering
Osaka Prefecture University
Sakai, Osaka, Japanishimaru@m.cs.osakafu-u.ac.jp

Graduate School of Engineering
Osaka Prefecture University
Sakai, Osaka, Japan
kise@cs.osakafu-u.ac.jp

In this demonstration, we focus on eye wear to assist people, sensing their physical, social and mental activities. Detecting and quantifying our behavior can raise awarenesstowards unhealthy practices. We use J!NS MEME prototypes, smart glasses with integrated electrodes to detect
eye movements, in application cases from reading detection
over ergonomics to talking recognition for social interaction
tracking.

Author Keywords
Smart Eye Wear; Eye Wear Computing; Glasses; Electrooculography

Shoya Ishimaru

Masahiko Inami

Graduate School of Engineering
Osaka Prefecture University
Sakai, Osaka, Japankatsuma@m.cs.osakafu-u.ac.jp

Graduate School of Media Design
Keio UniversityYokohama, 223-8526 Japaninami@kmd.keio.ac.jp

ACM Classification Keywords
H.5.2 [User Interfaces]: Input devices and strategies.

Introduction
Permission to make digital or hard copies of part or all of this work for personalor classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this noticeand the full citation on the first page. Copyrights for third-party componentsof this work must be honored. For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
Ubicomp/ISWC’15 Adjunct, September 7-11, 2015, Osaka, Japan.
ACM 978-1-4503-3575-1/15/09.
http://dx.doi.org/10.1145/2800835.2800900

361

Eye Wear Computing is a new paradigm that gains more
and more traction [1]. Most of our senses are situated on
the head near the eyes, making it a suitable location for
sensing devices. Still head-worn electronic devices (maybe
with the exception of hearing aids and mobile headsets) are
not common place. We believe that smart devices in theform factor of eye glasses will fill this gap.

UBICOMP/ISWC '15 ADJUNCT, SEPTEMBER 7–11, 2015, OSAKA, JAPAN3"Point(ElectrographyBa;eryMo4on(Sensors
Bluetooth(LE(etc.

Figure 1: The MEME Prototype

We show a couple of demonstrations on an early prototype of J!NS MEME, smart glasses with integrated electrodes to detect eye movements. First, we depict a simple
eye movement visualization, detecting left/right eye motion and blink. Users can play a couple of games in whicha they need navigate/interact using eye movements. We
detect and recognize reading and talking behavior of theuser using a combination of blink, eye movement and headmotion. People can get a long term view of their reading,talking, and also walking activity over the day. We can also
give now detailed word count for the reading behavior anddesigned a couple of interactions related to ergonomics.
This paper is an extension of our submission to SIGGRAPH
Emerging Technologies and shows a demonstration of technology from our UbiComp submission focusing on detecting
reading habits [4, 5].

MEME Prototype
Figure 2: User wearing MEME
while reading

The J!NS MEME prototype, sensing eye glasses able to detect head and eye movements we are using for our demonstrations is depicted in Figure 1 [3]. It is has 3 electrodes(electrooculography), 2 as nose pads and one on top of thenose, for eye movement analysis and motion sensors (accelerometer and gyroscope). It can connect over Bluetooth
LE module to a PC, laptop, smartphone or tablet. Theelectrodes are sampled with over 100 Hz, the motion sensor with over 50 Hz. The prototype already has a battery
runtime of over 8 hours while streaming sensor data (willbe improved). The glasses weigh 32 grams. The glasses
don’t do any advanced processing of the sensor data, theapplication logic is implemented on laptop, tablets or smartphones.

362

Demonstrations
Reading Detection and Word CountOne of the new functionalities we show a word count algorithm based on line-break detection 2. We recognize reading and afterwards detect how many lines a person read toestimate the word count. We apply the algorithm presented
in related work [5].
Interactions to Support Knowledge Work
Two of the biggest problems in relation to computer work
are dry eyes (also computer vision syndrome) and a too
steep head angle. Both can lead to chronic headaches andin severe cases to long term inquiries [6, 2].
To combat computer vision syndrome, we detect if a user
is not blinking for a given period of time. If this is the case,the screen slowly blurs forcing the person to blink.
Similar we detect the too steep head angle using the motion
sensors in the glasses. If the head angle falls under a given
threshold for a longer period of time, we flip the screen
contents away from the user. The user needs to adjust thehead position to continue reading.
These interactions are by no means unobtrusive so far.
They are to illustrate that detection of these states is possible and should spark discussions on how to improve posturewith less invasive techniques.
Talking Recognition
In addition to reading and ergonomics support we can also
recognize talking. Blinking frequency usually doubles andhead motion increases if people talk. Therefore, MEME isequipped with the right type of sensors to detect talking.

DEMOS

Gaze Interactions
The user can try out simple gaze interactions: extreme left
and right movements of the pupils as well as eye blinks
are detected and can be used to interact with a couple ofsimple games (e.g. Fruit Ninja or Flappy Bird).

Application Scenarios
Ergonomics seems to be the first and most straight forward use case for smart eye wear. Especially problems with(head) posture, gait and eye fatigue should be easy to detect from sensors on a glasses position.
Education is the second scenario that comes to mind. We
can already implement a word count algorithm for reading,giving quantified feedback to students/pupils about theirreading habits. They can also easily compare their performance. To support this application scenario more, tracking
concentration or attention on smart eye wear would be akiller feature.
Smart eye wear might also help in depression application
cases. Severe episodes of depression are usually accompanied with a decline in social interactions. Talking recognition is a first step towards quantifying them.

Conclusion
We introduced an early prototype of Eye Wear Computing
sensing glasses (MEME), showed a couple of demonstrations to underline the application cases we see for this new
application class.

Acknowledgements
This work is supported in part by the CREST project fromJST and JSPS Grant-in-Aid Young Researchers B No.
26730095.

363

References[1] Amft, O., Wahl, F., Ishimaru, S., and Kunze, K. Making regular eyeglasses smart. Pervasive Computing,IEEE 14, 3 (July 2015), 32–43.
[2] Bonney, R., and Corlett, E. Head posture and loadingof the cervical spine. Applied Ergonomics 33, 5 (2002),415–417.
[3] Ishimaru, S., Kunze, K., Tanaka, K., Uema, Y., Kise,K., and Inami, M. Smart eyewear for interaction andactivity recognition. In CHI Interacitity, ACM (2015),307–310.
[4] Kunze, K., and al et. Meme smart glasses to promotehealthy habits for knowledge workers. In to be publishedat ACM SIGGRAPH ETECH, ACM (2015), 11.
[5] Kunze, K., and al et. Quantifying reading habits counting how many words you read. In to be publishedat UbiComp, ACM (2015), 11.
[6] Yan, Z., Hu, L., Chen, H., and Lu, F. Computer visionsyndrome: A widely spreading but largely unknown epidemic among computer users. Computers in Human
Behavior 24, 5 (2008), 2026–2042.