Programmable Peripheral Vision: augment/reshape humanvisual perception
Qing Zhang

qzkiyoshi@gmail.com
Keio University
Yokohama, Kanagawa, Japan

Figure 1: a) Dynamic Periperal Vision Blocking Glasses. b) Saliency Reducing Glasses. 3) Gaze Guiding and Saliency Adjustable
Glasses.

ABSTRACTKEYWORDS

Over time our daily visual tasks become more complex continuously, however, the natural adaptation of our visual system does
not adapt as fast as the living environment changes. As the representations of this unbalanced trend, concentration difficulty andvisual overload are experienced and studied intensively. As wellas universal motion sickness occurs in both real-life and virtualenvironments. Besides, online learning and co-working experience
are far from satisfactory, which is partly due to lacking engagement and instantaneous visual interaction that we used to havewhen conducting those activities offline. Thus, my goal is to propose methods helping us better adapt to rapidly changing visualcontexts. In the formative research, I created dynamic peripheral
vision blocking glasses, and its experimental result indicates thatwearing such glasses helped its users suffer fewer motion sickness
symptoms while accessing fast-moving surrounding scenery in a
VR environment. For the following studies, I am creating dynamic
saliency adjusting glasses and gaze guiding glasses to augment andreshape daily-life visual perception.

smart eyewear, gaze guiding, peripheral vision, visual saliency adjusting

CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile devices; Interactive systems and tools.
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
CHI ’22 Extended Abstracts, April 29-May 5, 2022, New Orleans, LA, USA© 2022 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-9156-6/22/04.
https://doi.org/10.1145/3491101.3503821

ACM Reference Format:
Qing Zhang. 2022. Programmable Peripheral Vision: augment/reshape human visual perception. In CHI Conference on Human Factors in Computing
Systems Extended Abstracts (CHI ’22 Extended Abstracts), April 29-May 5,2022, New Orleans, LA, USA. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3491101.3503821

1

CONTEXT

Compared to the drastic changes in our living environment overthe past centuries, our visual system is almost the same as ourancestors. This unbalanced development of the environment andhuman vision system resulted in a series of vision-related negative consequences. For instance, increased myopia rate worldwide.
Unavoidable long screen use and other modern visual tasks partly
reflect such unbalance. Besides, vision-related illnesses like computer vision syndrome are becoming a widespread phenomenon [4].
As adaptational methods, if we are nearsighted we use the concave lenses. If we are farsighted we use convex lenses. If we want tosee something far away, we use the telescope. Some novel glasses
make those lenses adjustable [12]. Hence I wonder if we can makeother properties of our vision system programmable. For example, if we want to focus on something can we block distracting
objects in our peripheral vision area, saving part of our cognitive resource. Meanwhile, although augmented reality and virtual reality
have been created and developed for decades, severe and universal
motion sickness symptoms impede their further popularization.
Moreover, unavoidable online/ online-offline-hybrid teaching,learning, and co-working circumstances will at least last for a while

CHI ’22 Extended Abstracts, April 29-May 5, 2022, New Orleans, LA, USAor even be entrenched. At the same time, complaints and impaired
teaching and learning experience are being discussed intensively.
For instance, lacking gaze contact or gaze interaction that we used tohave while attending or teaching off-line classes is now unavailable.
To deal with those physical limitations such as the disparity between
the locations of the subject’s gaze and the camera while making
video calls have been well addressed and solved [14].

1.1

Research Situation

I am doing my Ph.D. studies in the Graduate school of Media Design
at Keio University, Kanagawa, Japan. My advisor is professor Kai
Kunze. I am in the second half of my candidature which I started
in April 2020. I had my first paper published in ISWC 2021, the25th annual International Symposium on Wearable Computers [28].
I received my B.A. and M.F.A. degrees from fine art universities,
which have a relatively less apparent correlation between HumanComputer Interactions, however, with those previous experiences ofcreating and studying visual art content, I am conducting my humanvision augmentation-related studies smoothly and enthusiastically.
I believe that I am on track to complete my Ph.D. in April 2023.

1.2

Motivation

My motivation, first, we can expect a world where we have more
control of our vision system, therefore we can adjust our vision
to best match corresponding visual tasks. Currently, to be less affected by audio noise, we can use noise-canceling earphones orheadphones. Yet, besides well-addressed audio noise, visual noise
also brings negative influences on both our mental state and cognitive performance. Thus, I wonder if we can also leverage external
devices that help us suffer less from visual noise or current work
irrelevant visual information.
As well as for myself, I am one of those people who experiencevisual overload frequently in daily life. In the case of my personal
experience, my past practice in art universities required me to behighly sensitive to subtle visual differences, which ultimately mademy visual information processing threshold keep at a relatively
low level. It also means I can access the visual details quickly andeffectively. On the other hand, I am not able to adjust my visual
processing threshold properly, such as when is the optimal timing
to be sensitive or when is the best timing to be visually numb to
avoid potential visual interruption and overload.
Hence, I am eager to contribute to those areas to propose potential methods and better vision-adaptive systems for the gradually
universal vision-related syndrome.

2

RELATED WORKS

Regarding vision augmentation, [17] presented many ways to modulate our visual experience, for instance, by using their device
people who suffering color blindness can now distinguish the hidpatterns [11]. [8] addressed the light adaptation and dark adaptation which usually take a long time when people get older orundergo substantial luminance changing. They presented a novel
method that assists their users to deal with significant illumination
change as well as enhance underexposure visual objects. In terms ofaugmenting spatial awareness even beyond the invisible area of oureyes, [3] presented a device that aims to allow its users to perceive

Qing

and respond simultaneously to multiple spatial information sourcesby leveraging haptic stimulus.
In terms of reshaping visual experience, [24] introduced a concept that programmable physical architectures, where we can decide
what part of the external world we see based on our changing needs.
It also mentioned another concept called programmable shadows,
in which we can adjust the amount of sunlight without blocking
outside views completely. Both of them demonstrate the possibilities that we can intentionally leverage agency devices to adjust
visual information based on our instant needs.
Besides, peripheral vision is one of the visual features that weare taking advantage of in order to augment or reshape our visual perception. Specifically, [25] pointed out that peripheral-vision
space around the user is a valuable resource for awareness andcommunication systems. [18] aims to offer intuitive off-screen information obtaining experience. As well as [21] used two LED
matrix displays to show information to the peripheral vision area
in order to influence its user’s behavior. [10] introduced a methodthat not only displays information to its user but can also change
the information type from simple to detailed accordingly basedon whether the user’s visual focus is on the display area or not.
Sparse Peripheral Display demonstrates methods to augment thefield-of-view of head-mounted displays as well as to reduce potential motion sickness symptoms [26]. To influence the perceptionof speed, [19] designed patterns and displayed them based on theconcept of Sparse Peripheral Displays.
On the contrary, rather than adding information to the peripheral vision area, removing or blocking part of visual information
is used to evaluate visual task performance [16]. [28] presented
eyewear that can dynamically block part of its user’s peripheral
vision area in order to reduce motion sickness symptoms while accessing fast-moving scenery. [13] applied the concept of diminishedreality to remove content-irrelevant visual objects while accessing
augmented reality contents.
To augment human vision by computational eyewear, [15] presented a device that can compensate for color vision deficiency. [6]indicates that gaze information strongly reflects the human interestor their attention; if we can smoothly guide the user’s visual attention towards a certain target without interrupting them from theircurrent visual attention, we are able to enhance the gaze-based
interface. [7] proposed a method to guide their users’ attention to
the intended location without being noticed by using a subtle blureffect. Besides, [1, 5] addressed the involuntary attention in visual
processing performance.
However, gaps remain in those topics aforementioned. For instance, those studies are insufficient regarding how to deal withdramatically changing visual processing demands. Sensory conflict
such as motion sickness occurs due to our current vision system
did not well-adapted to fast-moving-surrounding scenery when our
body is relatively static. Second, saliency adjusting methods above
did not cover daily visual processing activities and even failed when
dealing with some cases such as objects that have significantly superior saliencies, items having a face or faces, or face-like features
versus no face or no face features-having objects.

Programmable Peripheral Vision: augment/reshape human visual perception

CHI ’22 Extended Abstracts, April 29-May 5, 2022, New Orleans, LA, USA

Figure 2: Current state of my PhD progress.

Figure 3: Tunnel Vision glasses help the user deal with motion sickness. Left, the user can see through the glasses as usual;
middle, the glasses start to block visual stimuli in the peripheral vision; right, if the user starts to move their gaze away fromthe center, the glasses return to their transparent state.

2.1

Research Question

The main research question that how can we well adapt to dramatically changing daily visual activity is divided into threesubsets:• How can we suffer less motion sickness symptoms without
sacrificing immersion?
• How can we cancel visual noise or be less susceptible tocurrent-task-irrelevant visual stimuli?
• How can we enhance our learning and teaching experience
with gaze guiding?

3

RESEARCH APPROACH AND PROGRESSTO DATE
3.1 Reduced motion sickness symptoms inboth VR and real-life using scenario.
To examine the question that how can we suffer less motion sickness while accessing potential motion sickness triggers. I built
a device that can adjust its user’s field of view, especially to prevent its user from seeing visual information via peripheral vision,
I used a transparent-opaque switchable film instead of an opticallens, known as polymer dispersed liquid crystal (PDLC) switchable
smart film (see figure 1). I measured 16 college students (8 maleand 8 female) in advance and found their average interpupillary
distance was 63.3 mm (SD = 3.3, MAX = 70, MIN = 58). Based on thisdata and actual experience with the glasses, I cut out a horizontal
elliptical area with a major axis of 8 cm. The participants seem tobe comfortable in the setup if looking straight.
As the controlling system for changing between transparent
and opaque mode, I used an Arduino compatible Pro Mini development board. Concerning its interactive design, we use two photoreflective sensors to detect users’ eye movements. Regarding theinteractive system of the glasses, I assume that if our participants
feel sick, they can focus on any place without moving their eyes for3.5 seconds, then the dynamic glasses’ switchable PDLC film will become opaque to prevent its user from experiencing the fast-moving
surrounding scenery via their peripheral vision. In comparison, if
the users move their eyeballs, I assume that they feel well, thenthe film will be transparent to support them getting more visual
information.
The result of dynamic peripheral vision adjusting glasses showsthat using our method to reduce motion sickness symptoms simultaneously has less impact on immersion. In detail, there was astatistically significant difference in the symptoms, Nausea, Difficulty concentrating, and Fullness of the head, without impacting
immersion.

3.2

Saliency Reducing Adjustable Glasses

How can we cancel visual noise or be less susceptible to currenttask-irrelevant visual stimuli. I finished the prototypes of Saliency

CHI ’22 Extended Abstracts, April 29-May 5, 2022, New Orleans, LA, USA

Qing

Figure 4: Patterns designed for reducing saliency of the objects around the peripheral vision area.
Reducing Glasses, see figure 4, 5. It aims to address if we can dynamically adjust the saliency of current-work-irrelevant visual objects
around our peripheral vision area. It was indicated by the previous
peripheral vision blocking glasses that instead of blocking partof the visual information of the peripheral vision area, it is moreintuitive and acceptable to alter those current-task-unrelated visualobjects around the peripheral vision area less salient, or in otherwords, making them cognitively meaningless.
To examine what kind of patterns of design works well suchas what can make current-task-irrelevant visual information lesssalient or less cognitively attractive as well as how much space canwe adjust while without sacrificing the resolution of the central
area of view. I designed and tested three types of patterns basedon Wizard of Oz, see figure 4. As the result of a pilot test, thecentral radial pattern worked well as a semi-transparent filter toblur the peripheral vision without feeling significant difference
while gradient dots pattern and fully-block pattern reached uneasy
feelings.

3.3

Gaze Guiding Glasses

Besides, a gaze-guiding glasses that I am building recently is in its
70% completion and which is increasing gradually, see figure 1 (c).
It aims to address how can we enhance our learning and teaching experience with gaze guiding? For instance, offer methods
to guide its user’s gaze to the speaker’s intended place. As well as,
to help its users to better catch up with instructors’ or co-workers’
points while lots of people are complaining about dissatisfied online
education and online co-working experience nowadays. When the
prototype building is finished, then a pilot experiment will start. As
the gaze guiding material, I used two monochrome Liquid Crystal
Displays. Those kinds of displays were used by [8] that generate asemi-transparent layer in between the real-world visual information and our eyes to suppress extreme bright illumination change.
Now I am designing patterns that can work as blurry effects [7]to induce the user to move their gaze to the unobscured area or
the relatively high-resolution area with the consideration of how
to avoid significantly salient objects such as face-having objects
affect the gaze guiding performance. Therefore, it is more adaptive
and practical to blur the area beside the intended place based oncontext-aware design instead of making the blur effects uniformly.

3.4

Contributions

To the best of my knowledge, the peripheral vision blocking glasses
within this research scope is the first study that presents a wearable
device to reduce motion sickness symptoms in VR/AR and reallife situations without impacting immersive experience. This study
suggests a practical method that will, directly and indirectly, propel
the popularization of the VR/AR contents related industry. Sincewith our glasses’ help, users will feel less sick while enjoying visualcontent without sacrificing immersion.
The study of saliency-reducing glasses will improve our visualtask performance such as working in an environment where fulfilled with visual noise, or in other words, current-tasks-irrelevant
visual information.
Third, my gaze guiding glasses are expected to help its users
to better catch up with instructors’ or co-workers’ points whilelots of people complain about dissatisfied online education andonline co-working experiences. These studies may also contributeto increased concentration since [23] clarified that with or without
gaze, we have different sound processing reaction times from short
to long.

3.5

Challenges

According to [7] that how-to guide users attention without failingto deal with objects have significant and superior saliency comparedto others, such as face-having objects versus no-face objects, as well
as some certain designs have superior saliency than others among
the same kind of group is still a tough problem. Besides, various
visual preferences would also influence the robustness of the gaze
guiding system.
Myopia rate increases worldwide [9, 27]. This trend also appears
when I was recruiting participants to join my experiments, I foundthe majority of them are usually nearsighted or farsighted. As aresult, how to implement my works to cover poor-eyesight users’
diverse situations could be an inevitable issue.

4

EXPECTED NEXT STEPS AND LONG TERM
GOALSThinking human-computer integration, for instance, [2] introduceda system that provides real-time computer assistance to its user
based on EEG signals from the user’s visual cortex. It reveals thepotential that we can leverage pre-attentive signals to act beyond
the biological limitations of our bodies. Therefore, I am planningto implement such a method into my future work to gain deeper
insights into human vision augmentation scenarios.
I am also inspired by the studies regarding bodily ownership [2,20, 22] which indicates that there are crucial timings exist when wereach the illusion that computational assistance is part of our own
capability. Therefore, besides making computational intervention
subtle, I am going to dig into more natural or more intuitive humancomputer interactive scenarios especially in terms of auto-adaptive
human-vision-enhancing systems. I envision a future when we canhave more control of our cognitive resource distribution such asattention and concentration, which will not only increase productivity but also will contribute to our mental health.

Programmable Peripheral Vision: augment/reshape human visual perception

CHI ’22 Extended Abstracts, April 29-May 5, 2022, New Orleans, LA, USA

Figure 5: Saliency Reducing Glasses Woring Demonstration. R. indicates Saliency Reducing Mode. T. is Transparent Mode.

ACKNOWLEDGMENTS
This work was supported by JST SPRING, Grant Number JPMJSP2123.
I appreciate my advisors, collaborators, and volunteers for theirgenerous support and patience.

REFERENCES[1] Jared Abrams, Antoine Barbot, and Andmarisa Carrasco. 2010. Voluntary attention increases perceived spatial frequency. Attention, Perception, & Psychophysics
72, 6 (2010), 1510–1521.
[2] Josh Andres, MC Schraefel, Nathan Semertzidis, Brahmi Dwivedi, Yutika C Kulwe,
Juerg von Kaenel, and Florian Floyd Mueller. 2020. Introducing Peripheral Awareness as a Neurological State for Human-Computer Integration. In Proceedings ofthe 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[3] Alvaro Cassinelli, Carson Reynolds, and Masatoshi Ishikawa. 2006. Augmenting
spatial awareness with Haptic Radar. 61 – 64. https://doi.org/10.1109/ISWC.2006.
286344[4] Artem Dementyev and Christian Holz. 2017. DualBlink. Proceedings of the ACM
on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 (3 2017), 1–19. Issue
1. https://doi.org/10.1145/3053330[5] Joetta Gobell and Marisa Carrasco. 2005. Attention alters the appearance of
spatial frequency and gap size. Psychological science 16, 8 (2005), 644–651.
[6] Aiko Hagiwara, Akihiro Sugimoto, and Kazuhiko Kawamoto. 2011. Saliencybased image editing for guiding visual attention. In Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction.
43–48.
[7] Hajime Hata, Hideki Koike, and Yoichi Sato. 2016. Visual guidance with unnoticed
blur effect. Proceedings of the Workshop on Advanced Visual Interfaces AVI 07-10June-2016, 28–35. https://doi.org/10.1145/2909132.2909254[8] Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, and Maki Sugimoto. 2017. Assisting
Eye Adaptation Via Occlusive Optical See-Through Head-Mounted Displays
AdaptiVisor.
[9] Brien A Holden, Timothy R Fricke, David A Wilson, Monica Jong, Kovin S Naidoo,
Padmaja Sankaridurg, Tien Y Wong, Thomas J Naduvilath, and Serge Resnikoff.
2016. Global prevalence of myopia and high myopia and temporal trends from2000 through 2050. Ophthalmology 123, 5 (2016), 1036–1042.
[10] Yoshio Ishiguro and Jun Rekimoto. 2011. Peripheral vision annotation: noninterference information presentation method for mobile augmented reality. In
Proceedings of the 2nd Augmented Human International Conference. 1–5.
[11] Shinobu Ishihara. 1987. Test for colour-blindness. Kanehara Tokyo, Japan.
[12] Yuta Itoh, Tobias Langlotz, Stefanie Zollmann, Daisuke Iwai, Kiyoshi Kiyokawa,and Toshiyuki Amano. 2019. Computational phase-modulated eyeglasses. IEEE
transactions on visualization and computer graphics (2019).
[13] Hanseob Kim, TaeHyung Kim, Myungho Lee, Gerard Jounghyun Kim, and Jae-In
Hwang. 2020. Don’t bother me: how to handle content-irrelevant objects inhandheld augmented reality. In 26th ACM Symposium on Virtual Reality Software
and Technology. 1–5.
[14] Claudia Kuster, Tiberiu Popa, Jean-Charles Bazin, Craig Gotsman, and Markus
Gross. 2012. Gaze Correction for Home Video Conferencing. ACM Trans. Graph.
31, 6, Article 174 (Nov. 2012), 6 pages. https://doi.org/10.1145/2366145.2366193[15] Tobias Langlotz, Jonathan Sutton, Stefanie Zollmann, Yuta Itoh, and Holger
Regenbrecht. 2018. ChromaGlasses: Computational glasses for compensating
colour blindness. Conference on Human Factors in Computing Systems - Proceedings
2018-April. https://doi.org/10.1145/3173574.3173964[16] Hyelip Lee, Seungwoo Je, Rachel Kim, Himanshu Verma, Hamed Alavi, andAndrea Bianchi. 2019. Partitioning open-plan workspaces via augmented reality.
Personal and Ubiquitous Computing (2019). https://doi.org/10.1007/s00779-01901306-0[17] Andrew Maimone, Douglas Lanman, Kishore Rathinavel, Kurtis Keller, David
Luebke, and Henry Fuchs. 2014. Pinlight displays: Wide field of view augmented
reality eyeglasses using defocused point light sources. ACM Transactions on
Graphics 33. Issue 4. https://doi.org/10.1145/2601097.2601141[18] Shizuko Matsuzoe, Shan Jiang, Miwa Ueki, and Keiju Okabayashi. 2017. Intuitive
visualization method for locating off-screen objects inspired by motion perception
in peripheral vision. ACM International Conference Proceeding Series. https://doi.org/10.1145/3041164.3041198[19] Alexander Meschtscherjakov, Christine Döttlinger, Tim Kaiser, and Manfred
Tscheligi. 2020. Chase Lights in the Peripheral View: How the Design of Moving
Patterns on an LED Strip Influences the Perception of Speed in an Automotive
Context. In Proceedings of the 2020 CHI Conference on Human Factors in Computing
Systems. 1–9.
[20] Florian Floyd Mueller, Pedro Lopes, Paul Strohmeier, Wendy Ju, Caitlyn Seim,
Martin Weigel, Suranga Nanayakkara, Marianna Obrist, Zhuying Li, Joseph Delfa,
Jun Nishida, Elizabeth M. Gerber, Dag Svanaes, Jonathan Grudin, Stefan Greuter,
Kai Kunze, Thomas Erickson, Steven Greenspan, Masahiko Inami, Joe Marshall,Harald Reiterer, Katrin Wolf, Jochen Meyer, Thecla Schiphorst, Dakuo Wang, andPattie Maes. 2020. Next Steps for Human-Computer Integration. In Proceedingsof the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu,
HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA,
1–15. https://doi.org/10.1145/3313831.3376242[21] Takuro Nakao, Liwei Chan, Masashi Nakatani, and Kai Kunze. 2016. Smart glasses
with a peripheral vision display. ACM International Conference Proceeding Series.
https://doi.org/10.1145/2927929.2927938[22] Jun Nishida, Shunichi Kasahara, and Kenji Suzuki. 2017. Wired muscle: generating
faster kinesthetic reaction by inter-personally connecting muscles. In ACM
SIGGRAPH 2017 Emerging Technologies. 1–2.
[23] Ulrich Pomper and Maria Chait. 2017. The impact of visual gaze direction on
auditory object tracking. Scientific Reports 7 (12 2017). https://doi.org/10.1038/
s41598-017-04475-1[24] Jun Rekimoto. 2012. Squama: Modular Visibility Control of Walls and Windows for Programmable Physical Architectures. https://doi.org/10.1145/2254556.
2254587[25] Lindsay Reynolds, Jeremy Birnholtz, Eli Luxenberg, Carl Gutwin, and Maryam
Mustafa. 2010. Comparing awareness and distraction between desktop andperipheral-vision displays. In CHI’10 Extended Abstracts on Human Factors in
Computing Systems. 3571–3576.
[26] Robert Xiao and Hrvoje Benko. 2016. Augmenting the field-of-view of headmounted displays with sparse peripheral displays. In Proceedings of the 2016 CHI
Conference on Human Factors in Computing Systems. 1221–1232.
[27] Erisa Yotsukura, Hidemasa Torii, Mikako Inokuchi, Mitsuaki Tokumura, Miki
Uchino, Kyosei Nakamura, Mari Hyodo, Kiwako Mori, Xiaoyan Jiang, Shin-ichi
Ikeda, et al. 2019. Current prevalence of myopia and association of myopia withenvironmental factors among schoolchildren in Japan. JAMA ophthalmology 137,
11 (2019), 1233–1239.
[28] Qing Zhang, Hiroo Yamamura, Holger Baldauf, Dingding Zheng, Kanyu Chen,
Junichi Yamaoka, and Kai Kunze. 2021. Tunnel Vision–Dynamic Peripheral Vision
Blocking Glasses for Reducing Motion Sickness Symptoms. In 2021 International
Symposium on Wearable Computers. 48–52.