GazeSync: Eye Movement Transfer Using an Optical Eye Tracker
and Monochrome Liquid Crystal Displays
Qing Zhang

qzkiyoshi@gmail.com
Keio University, Graduate School ofMedia Design
Yokohama, Kanagawa, Japan

Juling Li

lijuling@mail.ustc.edu.cn
Keio University, Graduate School ofMedia Design
Yokohama, Kanagawa, Japan

Yifei Huang

hyf@iis.u-tokyo.ac.jp
Institute of Industrial Science, The
University of TokyoJapan

Yun Suen Pai

pai@kmd.keio.ac.jp
Keio University, Graduate School ofMedia Design
Yokohama, Kanagawa, JapanGeorge Chernyshov

Keio University Graduate School ofMedia DesignJapan
chernyshov@kmd.keio.ac.jp

Kai Kunze

kai@kmd.keio.ac.jp
Keio University, Graduate School ofMedia DesignYokohama, Kanagawa, Japan

Figure 1: a. The Leader is the gaze input resource that we get two-dimensional x and y coordinates by having him wear a Pupil
CORE eye tracker. b. The Follower follows the transparent circle that represents the gaze movements of the leader.
the follower (person who follows the eye gaze of the leader). We
report the results of an initial user test and discuss future works.

ABSTRACTCan we see the world through the eyes of somebody else? We
present an early work to transfer eye gaze from one person toanother. Imagine you can follow the eye gaze of an instructor while
explaining a complex work step, or you can experience a painting
like an expert would: Your gaze is directed to the important partsand follows the appropriate steps. In this work, we explore thepossibility to transmit eye-gaze information in a subtle, unobtrusive
fashion between two individuals. We present an early prototype
consisting of an optical eye-tracker for the leader (person whoshares the eye gaze) and two monochrome see-through displays forCCS CONCEPTS• Human-centered computing → Empirical studies in ubiquitousand mobile computing; Collaborative and social computing systems
and tools; Ubiquitous and mobile devices.

KEYWORDS
gaze detection, gaze synchronization, eye tracking, gaze communication

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
IUI ’22 Companion, March 22–25, 2022, Helsinki, Finland© 2022 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-9145-0/22/03.
https://doi.org/10.1145/3490100.3516469

ACM Reference Format:
Qing Zhang, Yifei Huang, George Chernyshov, Juling Li, Yun Suen Pai,and Kai Kunze. 2022. GazeSync: Eye Movement Transfer Using an Optical
Eye Tracker and Monochrome Liquid Crystal Displays. In 27th International
Conference on Intelligent User Interfaces (IUI ’22 Companion), March 22–25,2022, Helsinki, Finland. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3490100.3516469

54

IUI ’22 Companion, March 22–25, 2022, Helsinki, Finland

1Zhang et al.

crystal displays to share eye gaze unidirectional in real-time. (3) wereport insights of an initial user test and present future directions
for gaze-communication-related research.

INTRODUCTION

The sclera, the white in our eyes, is more prominent compared toprimates or other animals, enabling us to communicate and recognize gaze more efficiently [13]. We often rely on gaze information
in social situations, from learning over coordination to guessing
intentions [1, 12, 18, 23]. Yet, estimating the gaze of a person fromjust their eye position is difficult, especially when we are engaging
in an unfamiliar task. Have you ever imagined seeing the worldfrom another perspective or the eyes of somebody else? In ourresearch, we aim at augmenting the social components of eye gazeand searching for ways of sharing, transferring, and extending gazeinformation.
Take an offline visual creative activity as an example, namely,
learning how to draw a sketch precisely and how to observe themotif systematically. In general, due to our limited visual memory,
professional painters switch their gaze between their canvas and themotif quickly and frequently to carry as much visual information
as they can observe [4]. That observational method enables professional painters to suppress perceptual (visual) constancy (color,size). However, beginners of drawing are less likely to switch theirgaze from the motif and their canvas as many as experts do [4]. In
this context, gaze imitation is a powerful mechanism for transferring knowledge from a well trained one to an unskilled one [11].
Thus, one of the original ideas is to design a method that can synchronize the professional instructor’s gaze moving patterns andthe beginners’. Those indicate the potential that real-time gazesynchronization can compensate or even enhance both online andoffline communication experiences.
There are a lot of works dealing with gaze visualization in virtual
reality, augmented reality, mixed reality, and real-world environments [17, 20, 22, 24]. Efficient visual guidance could contributeto reducing cognitive load while conducting visual search activities [15], as well as [3] used HoloLens for augmented collaboration
under co-design scenarios in shared space. Shared gaze was used
to improve robot performance as well [19]. It was also found thatgaze information strongly reflects the human interest or their attention [8]. Hata et al. [9] proposed a method to guide their users’attention to the intended location without being noticed by using
subtle blur effects. Shared attention between two parts is also used
to modulate gaze following [2].
Researches regarding ocular activities such as saccadic eye movements and smooth pursuit suggest the potentials that we can leverage those inherent properties to build better or reshape our visualexperience. The phenomenon that saccades can be triggered voluntarily or involuntarily [21] indicates the possibility to synchronize
gaze explicitly or implicitly. Regarding leveraging smooth pursuit
to build gaze interaction, [5] enables hands-free interaction while
using smartwatches.
We propose GazeSync, a gaze movement transferring device
that synchronizes gaze movements between the leader side andfollower side, which will be described in detail in the following
paragraphs. The contributions of this paper are as follows: (1) wepresent the concept of GazeSync which is using wearable smart
eyewear to synchronize gaze movements from one user (leader) toanother user (follower), (2) we present an initial, fully functional
prototype using an optical eye tracker and 2 monochrome liquid

2IMPLEMENTATIONAs the gaze synchronization material, we use two monochrome
Liquid Crystal Displays (LCDs), similar to the ones used by [10]for the generation of a semi-transparent layer in between the realworld and our eyes to suppress excessively bright illumination, see
figure 2B. We design a pattern that can work as blurry effects [9]to induce the user to move their gaze to the unobscured area or
the relatively high-resolution area, see figure 2C. To track the eye
movements as well as to generate the corresponding data of eye activities, we use Pupil Core Eye Tracker [14, 16]. It is widely used invarious studies such as [6, 7, 26] because of its robust eye-tracking
performance. To have the eye movement tracking cameras detecteyeball movements successfully and stably through the LCD panels, we replaced the original camera arms with our re-designed 3D
printed ones, which made sure that we can set the cameras at a relatively optimal angle to detect the pupils. To decide the appropriate
interpupillary distance and the radius of the transparent circle, wefollow [25] and set 8cm as the interpupillary distance and 30 pixels
as the radius of the transparent circle.

3INITIAL TESTAs an initial usability test, we recruit two male graduate studentsto test our gaze synchronization glasses, one of them assigned as
the leader (white-mask male), the other as a follower (black-maskmale). The leader wore a Pupil Core to obtain the data of his gazemovements. To generate a Cartesian coordinate system to get the xand y coordinates of gaze, we set four Apriltag Markers on a vertical
1.2m x 1.8m whiteboard in front of him, the distance between thefront camera of his Pupil Core and the vertical whiteboard is 50cm.
Then we conduct calibration of gaze movement, making sure wecan obtain stable and reliable x and y coordinates from the leader.
Meanwhile, the follower wears our customized Pupil Core with
two transparent monochrome LCD attached on the Pupil Core
frame as the lenses, in which we displayed a peripheral-black but
center-transparent pattern, see figure 1b, for both left and rightlenses.
We take two steps to test the synchronization. In the first step,
the experimenter gives directional signs in Japanese to the leader(native Japanese), e.g., slowly moving your gaze up, down, left, orright and back to the center. The follower is a non-Japanese student
having little Japanese experience, which prevents him from being
distracted by the language signal. Both the leader and the follower
could not see each other so that the follower does not have accessto the leader’s gaze directly. To further make sure that the follower
would not be affected by external visual stimuli, we require him to
face a whiteboard that covers his entire field of view. As the step
two, to test their synchronization performance over natural gaze
patterns, we ask the leader to move his eye freely and observe thecorresponding eye movements of the follower.

55

GazeSync: Eye Movement Transfer

IUI ’22 Companion, March 22–25, 2022, Helsinki, Finland

Figure 2: Demonstration of gaze Sync and its Configuration. A. indicates how we map the leader’s gaze to the follower, suchas the directional signs that the experimenter gave to the leader, slowly looking up, slowly looking down, slowly looking
right, and slowly looking right, meanwhile, the follower reacted as the corresponding directions. B. demonstrate the gazesynchronization working flow.

4smooth pursuit and involuntary subtle gaze guiding. Nevertheless, based on the feedback and visual observations, our device isperforming reasonably well and does allow a certain level of eye
movement synchronization. In the future, we will work on reducing the system latency and evaluate the synchronization levels in amore quantifiable way.

INSIGHTS AND DISCUSSION

According to the observational result and the oral feedback fromthe follower, if the movement of the transparent circle (corresponding to the leader’s eye movement) is too fast, the follower was notable to chase it. On the other hand, if the speed of the movement ismoderate and smooth, it is relatively easy for him to follow the transparent region. Besides, this current prototype is somewhat heavier
than ordinary glasses and its design did not distribute well the pressure on his nose. Also, we observed an obvious delay between theleader and the follower despite the devices being controlled overthe RS-232 link via USB cables. Besides, to generate a virtual surface
as the x and y coordinate plane, this current prototype requires afront camera which is somewhat sensitive regarding privacy whenusing it in public space.

5ACKNOWLEDGMENTS
We appreciate the anonymous reviewers for their practical insights
and valuable comments. As well as the generous help of the volunteers. This work was supported by JST SPRING, Grant Number
JPMJSP2123.

REFERENCES[1] Simon Baron-Cohen, Sally Wheelwright, Jacqueline Hill, Yogini Raste, and Ian
Plumb. 2001. The “Reading the Mind in the Eyes” Test revised version: a study
with normal adults, and adults with Asperger syndrome or high-functioningautism. The Journal of Child Psychology and Psychiatry and Allied Disciplines 42,2 (2001), 241–251.
[2] Anne Böckler, Günther Knoblich, and Natalie Sebanz. 2011. Observing sharedattention modulates gaze following. Cognition 120, 2 (2011), 292–298.
[3] Yoonjeong Cha, Sungu Nam, Mun Yong Yi, Jaeseung Jeong, and Woontack Woo.
2018. Augmented collaboration in shared space design with shared attention andmanipulation. In The 31st Annual ACM Symposium on User Interface Software andTechnology Adjunct Proceedings. 13–15.

CONCLUSION AND FUTURE WORKS

We expect that to leverage this device with the visual clues we canbetter react to the auditory information or vice versa in terms ofboth online and offline learning, teaching, and co-working scenarios. For future works, since this current initial test could not clarify
whether the gaze following movements of the follower is activeor passive, we will conduct a different study that covers voluntary56

IUI ’22 Companion, March 22–25, 2022, Helsinki, Finland

Zhang et al.

[4] Dale J Cohen. 2005. Look little, look often: The influence of gaze frequency on
drawing accuracy. Perception & Psychophysics 67, 6 (2005), 997–1009.
[5] Augusto Esteves, Eduardo Velloso, Andreas Bulling, and Hans Gellersen. 2015. Orbits: Gaze Interaction for Smart Watches Using Smooth Pursuit Eye Movements.
In Proceedings of the 28th Annual ACM Symposium on User Interface Software&amp; Technology (Charlotte, NC, USA) (UIST ’15). Association for Computing Machinery, New York, NY, USA, 457–466. https://doi.org/10.1145/2807442.2807499[6] Augusto Esteves, Eduardo Velloso, Andreas Bulling, and Hans Gellersen. 2015.
Orbits: Gaze interaction for smart watches using smooth pursuit eye movements.
In Proceedings of the 28th annual ACM symposium on user interface software &technology. 457–466.
[7] Kunal Gupta, Gun A Lee, and Mark Billinghurst. 2016. Do you see what I see?
The effect of gaze tracking on task space remote collaboration. IEEE transactions
on visualization and computer graphics 22, 11 (2016), 2413–2422.
[8] Aiko Hagiwara, Akihiro Sugimoto, and Kazuhiko Kawamoto. 2011. Saliencybased image editing for guiding visual attention. In Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction.
43–48.
[9] Hajime Hata, Hideki Koike, and Yoichi Sato. 2016. Visual guidance with unnoticed
blur effect. Proceedings of the Workshop on Advanced Visual Interfaces AVI 07-10June-2016, 28–35. https://doi.org/10.1145/2909132.2909254[10] Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, and Maki Sugimoto. 2017. Assisting
Eye Adaptation Via Occlusive Optical See-Through Head-Mounted Displays
AdaptiVisor.
[11] Matthew W Hoffman, David B Grimes, Aaron P Shon, and Rajesh PN Rao. 2006.
A probabilistic model of gaze imitation and shared attention. Neural Networks
19, 3 (2006), 299–310.
[12] Roxane J Itier and Magali Batty. 2009. Neural bases of eye and gaze processing:
the core of social cognition. Neuroscience & Biobehavioral Reviews 33, 6 (2009),843–863.
[13] Sarah Jessen and Tobias Grossmann. 2014. Unconscious discrimination of social cues from eye whites in infants. Proceedings of the National Academy ofSciences 111, 45 (2014), 16208–16213. https://doi.org/10.1073/pnas.1411333111
arXiv:https://www.pnas.org/content/111/45/16208.full.pdf[14] Moritz Kassner, William Patera, and Andreas Bulling. 2014. Pupil: an open
source platform for pervasive eye tracking and mobile gaze-based interaction.
In Proceedings of the 2014 ACM international joint conference on pervasive andubiquitous computing: Adjunct publication. 1151–1160.
[15] Tiffany CK Kwok, Peter Kiefer, Victor R Schinazi, Benjamin Adams, and Martin
Raubal. 2019. Gaze-guided narratives: Adapting audio guide content to gazein virtual and real environments. In Proceedings of the 2019 CHI Conference on
Human Factors in Computing Systems. 1–12.
[16] Pupil Labs. 2014. Pupil Core Eye Tracker. https://pupil-labs.com/products/core/[17] Yuan Li, Feiyu Lu, Wallace S Lages, and Doug Bowman. 2019. Gaze direction
visualization techniques for collaborative wide-area model-free augmented reality.
In Symposium on spatial user interaction. 1–11.
[18] Christine E Looser and Thalia Wheatley. 2010. The tipping point of animacy:
How, when, and where we perceive life in a face. Psychological science 21, 12(2010), 1854–1862.
[19] AJung Moon, Daniel M Troniak, Brian Gleeson, Matthew KXJ Pan, Minhua Zheng,
Benjamin A Blumer, Karon MacLean, and Elizabeth A Croft. 2014. Meet me wherei’m gazing: how shared attention gaze affects human-robot handover timing.
In Proceedings of the 2014 ACM/IEEE international conference on Human-robotinteraction. 334–341.
[20] Zahar Prasov and Joyce Y Chai. 2008. What’s in a gaze? The role of eye-gaze inreference resolution in multimodal conversational interfaces. In Proceedings ofthe 13th international conference on Intelligent user interfaces. 20–29.
[21] Dale Purves and Stephen Mark Williams. 2001. Neuroscience. 2nd edition. Sinauer
Associates 2001. http://lib.ugent.be/catalog/ebk01:3450000000002013[22] Ben Steichen, Giuseppe Carenini, and Cristina Conati. 2013. User-adaptiveinformation visualization: using eye gaze data to infer visualization tasks anduser cognitive abilities. In Proceedings of the 2013 international conference onIntelligent user interfaces. 317–328.
[23] Hiroki Tanaka, Sakriani Sakti, Graham Neubig, Tomoki Toda, Hideki Negoro,
Hidemi Iwasaka, and Satoshi Nakamura. 2015. Automated social skills trainer.
In Proceedings of the 20th International Conference on Intelligent User Interfaces.
17–27.
[24] Vincent Van Rheden, Bernhard Maurer, Dorothé Smit, Martin Murer, and Manfred
Tscheligi. 2017. LaserViz: Shared gaze in the Co-located physical world. In
Proceedings of the Eleventh International Conference on Tangible, Embedded, andEmbodied Interaction. 191–196.
[25] Qing Zhang, Hiroo Yamamura, Holger Baldauf, Dingding Zheng, Kanyu Chen,
Junichi Yamaoka, and Kai Kunze. 2021. Tunnel Vision–Dynamic Peripheral Vision
Blocking Glasses for Reducing Motion Sickness Symptoms. In 2021 International
Symposium on Wearable Computers. 48–52.
[26] Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2017. Mpiigaze:
Real-world dataset and deep appearance-based gaze estimation. IEEE transactions
on pattern analysis and machine intelligence 41, 1 (2017), 162–175.

57