Peripheral Cognition Technology: Approach and Implementation
Seiji Yamada
National Institute of Informatics
SOKENDAITokyo Institute of Technology
2-1-2 Hitotsubashi, Chiyoda,
Tokyo 101-8430, Japan

Kazuki Kobayashi
Graduate School of Science andTechnology
Shinshu University
4-17-1 Wakasato, Nagano
380-8553, Japan

Naoki Mori
Department of Computational
Intelligence and Systems Science
Tokyo Institute of Technology
4259 Nagatsuta, Midori,
Yokohama, 226-8503 Japanseiji@nii.ac.jp

kby@shinshu-u.ac.jp

mori@ntt.dis.titech.ac.jp

Abstract— In this paper, we describe a novel information
notification method, Peripheral Cognition Technology (PCT)that utilizes human cognitive properties including visual field
narrowing and change blindness (or inattentional blindness).
Information notification on a display for e-mail, micro blog,
and application update, is becoming increasingly important invarious environments such as offices, and the home. To develop
a smart information notification, a user model-based approachand a peripheral display have been studied. However, building
a user model is difficult and a peripheral display catches user
attention. We propose a novel PCT for information notification.
The PCT utilizes the human cognitive properties that a visual
field narrows when he/she concentrates on a task, and that ahuman tends to be unable to recognize subtle and slow changes.
The core idea of PCT is that a human does not recognize
subtle changes in a peripheral area of cognition when he/sheconcentrates on a task, and he/she automatically recognizes thechanges when not concentrated. By setting a software agent or
a device of information notification with subtle changes in sucha peripheral area, a user automatically and easily accepts thenotification only when his/her concentration breaks. Also PCT
provides a way to determine various properties of HAI suchas agents appearances and positions. Two concrete implementations, “Peripheral Agent” and “Shape Shifting Notification”,
of PCT are introduced.

I. INTRODUCTION
In the current office and home environment connectedto the Internet, a user tends to frequently receive various
information notifications [1][2] for e-mails, tweets, instantmessages, and alerts of application updates on a display,cellar phones and smart phones (Fig.1). Although some
notifications are quite emergent, most of them are not so
emergent and a user should get the information only when
he/she is not busy. Such a user state in which he/she canaccept notified information and can access the information
is called interruptible, and the opposite user state is called
uninterruptible. A typical interruptible state is when a user
is not involved in completing a task and has time to read theinformation.
A problem with such information notifications is they
appear simultaneously as they are occurred, i.e., without thesystem being aware of whether the user is interruptible ornot. If notifications arrive when a user is uninterruptible,
they can cause significantly stress and reduce the user’s
productivity [3]. One way to deal with this would be tocontrol the information notification period in accordance with

Fig. 1.

Information notifications

the period the user is interruptible. In other words, this meansa system would need to estimate user’s interruptibility andsend a notification only when he/she can be interrupted.
In order to estimate user’s interruptibility, various studies
have been done [4][5][1][6]. Most of them applied machine learning techniques like classification learning, SVMs,
Bayesian networks to identify whether a user is interruptible
or not. The input to classification learning were implicit
feedback including keyboard typing patterns, trajectory of
the mouse operation, vision information of a user’s face andposture. Although this approach provides a general framework which can be applied to other user state estimation likeuser’s emotion, it is difficult to build a accurate model for
user’s interruptibility because a lot of training data is needed
for training classifiers and user state estimation might besignificantly dependent on individuals. Also, capturing theuser’s face and posture is problematic in terms of privacy.
Another approach to information notification does not
estimate whether the user can be interruptible. A peripheral
display [7][8][9][10] is such an approach. In this approach,
information itself, not the notification, is constantly displayedon a small sub-window (or a sub-display) in the side of amain-window (or a main-display) in which a user achieves amain task. A user is assumed to recognize the displayed information and understand the contents while he/she achieves
the main task. Since the user’s understanding of the subtask information is done in parallel to the main task andthe cognitive load is small, a user can easily understand thesub-task information. Although this peripheral display has
advantage that it does not need user state estimation, thegeneral design policy has not been developed and ad-hoc
implementations have been done in various fields. Also, sincea notification is basically not used in a peripheral display
and a user does not change the task mode from a maintask to a sub-task, more complicated information difficult
to understand the contents like e-mails, application updates,
can not be dealt with in a peripheral display.
We propose a novel information notification method, Peripheral Cognition Technology (PCT) that utilizes a humancognitive properties like visual field narrowing and change
blindness (or inattentional blindness). The PCT provides aninformation notification by utilizing a human cognitive property that a visual field narrows when he/she concentrates ona task intently. Also PCT utilizes another cognitive property
similar to change blindness, in which a human tends to beunable to recognize subtle and slow changes. The core ideaof PCT is that a human does not recognize subtle changes
in a peripheral area of cognition when he/she concentrates
on a task, and he/she automatically recognizes the changes
first when not concentrated. By setting a software agent or a
device of information notification with subtle changes in sucha peripheral area, a user automatically and easily accepts
the information notification only when his/her concentration
breaks. Two concrete implementations of PCT with different
subtle changes are introduced. We are developing PCT as
an elemental technology of HAI application to information
notification, and it provides a way to determine various
properties of HAI like agent’s appearances, positions, andso on.

slow and subtle. For example, an individual fails to notice aslight difference between two images that are almost identical. The reasons these changes usually remain unnoticed
by the observer include obstructions in the visual field, eyemovements, a change of location, or a lack of attention [15].
Change blindness has become a highly researched topic due
to newly discovered implications in practical applications
such as improving the accuracy of eyewitness testimony andreducing the number of distractions while driving.
Inattentional blindness [17] is also related to a humancognitive property which can be utilized in PCT. All of theinformation in the visual environment is potentially available
for attentive processing. However, without attention, notmuch of this information is retained. Inattentional blindness
means that without attention, observers do not recognize
visual features of the environment at all. In addition to
PCT implementation “Peripheral Agent”, “Shape Shifting
Notification” will be introduced in IV. We think this inattentional blindness might be more adequate for supporting
the effectiveness of the shape shifting notification.
We define these human cognition properties above that
occur in the abstract peripheral areas of “cognition” anddefine the interaction design utilizing such human cognition
properties as Peripheral Cognition Technology (PCT). We
think this interaction design policy can provide concrete
implementations for application of HAI in various ways. For
example, PCT-based life-like agents, determining positions
in which an agent should appear, and this enables us todesign the appearance of an agent such as the shape, thecolor, and the size. Two concrete implementations of PCT
will be introduced in the remainder of this paper.

II. PCT: P ERIPHERAL C OGNITION T ECHNOLOGY
Peripheral Cognition Technology (PCT) is cognitive interaction design for information notification. The “cognitive”
interaction design means a design method of interaction
utilizing human cognitive properties [11]; e.g. visual field
narrowing, change blindness, inattentional blindness. By
using such human properties, we can design an effective userinterface that does not force a high cognitive load without
learning ability such as an adaptive user interface [12].
VFN (visual field narrowing) [13][14] is a primary humancognitive property utilized in PCT. A visual field is an area
which an individual can see when looking straight ahead
without moving the eyes or gaze. It can be measured in degrees from the fixation point; the normal vision field is about
160-180 degrees horizontally and 120 degrees vertically. Thisvisual field is significantly affected by cognitive load orconcentration on a task [13]. When a human concentrates
on a hard task and the cognitive load becomes heavy,
his/her visual field significantly narrows. This phenomena
is call visual field narrowing and studies have been done toinvestigate it in various tasks [13][14].
Change blindness [15][16] is also a human cognitive property utilized in PCT. The change blindness is a psychologicalphenomenon that occurs when a change in a visual stimulus
goes unnoticed by the observer because the change is very

III. P ERIPHERAL AGENT
Peripheral agent is an implementation of PCT by using
Visual Field Narrowing (VFN). The peripheral agent has asimple human-like appearance and notifies information likee-mail arrival, twitter, application updates (Fig.1) to a user
by appearing on a display in an appropriate position. The
functions can be summarized in the following.

VFN region

Notification
Main Task
Peripheral
Agent
Concentrated
visual field
Fig. 2.

Peripheral agent

Innerboundary600-830 , 521-15 , 496
782 , 510

400

200

-829 , 45

786 , 45
0

-1000

-800

-600

-400

-200

0

200

400

600

800

1000

-200

-400
-15 , -401
-832 , -433

Fig. 3.

-600

Experimental environment
Fig. 4.

1) A peripheral agent appears in a VFN region simultaneously as information to be notified occurs.
2) If a user clicks the peripheral agent, it will display thecontent of the information by opening a sub-window.
We define the VFN region above as a peripheral visual
field which can be narrowed by VFN phenomena. Fig.2
shows a concentrated visual field when a user concentrated
a task and is uninterruptible, a VFN region outside of theconcentrated region with the inner boundary and a peripheral
agent. When we can adequately place a peripheral agent
in the VFN region, a user does not notice an information
notification when he/she is uninterruptible and he/she automatically notice the notification just when he/she becomesinterruptible.
We conducted preliminary experiments with participants
to investigating the VFN region. First, we define the VFN
region as a square region on a display and gather data indicating the positions at which a user notices the appearanceof a peripheral agent. We used a dark room-like editor withblack background and asked participants to just typing lettersscrolling horizontally in a white box in the center of thedisplay. This typing is a main task in this experiment andwe can control the difficulty by changing the speed of thescrolling. In this preliminary experiment, we set the scrolling
speed so that a user may be interruptible for determining theinner boundary of a VFN region in Fig.2. The shape andcolor of the peripheral agent was also tuned.
We made a peripheral agent to appear in various positions
from the far to near the center of the display. Since the darkroom-like editor is considered to be an extreme situation inwhich a user most easily notices a peripheral agent, the estimated VFN region corresponds to the smaller limit region.
Thus, in practical application in a display with a practical
desktop and a certain background image, the VFN region
becomes lager than this estimated one. An experimental
environment with a display, a keyboard and a foot switch
with a red circle is shown in Fig.3. A participant was asked toturn on the foot switch immediately just when he/she noticed
the peripheral agent.
Data of the inner boundary of the VFN region obtained
from 12 participants are plotted on the x-y coordinate of

795 , -428

VFN region dataa display like Fig.4. The broken line connects the average
positions in the eight directions, and we verified a VFN
region in a display from these data of this preliminary
experiment.
We are currently conducting a large experiment to investigate VFN region more precisely. After building a VFN, we
will determine the adequate position of a peripheral agent in
the VFN region, and conduct an experiment for evaluating
the effectiveness of the peripheral agent by comparing with aconventional notifications such as a balloon-like notificationin the corner of a display.
IV. S HAPE S HIFTING N OTIFICATION
We are developing another implementation of PCT, shape
shifting notification as a novel information notification
method. This shape shifting notification utilizes a humancognitive property, change blindness (or inattentional blindness) [15][16] (described in II). The basic idea on shape
shifting notification is to change the posture of a notification
device sufficiently slow so that a user may notice it only
when he/she is interruptible.
We experimentally developed a device for providing ashape sifting notification shown in Fig.5. A user can put
any information terminal like a smart phone or a PDA on
the device, and it covers the terminal at the initial state.
When the information terminal receives a call, e-mail or

Fig. 5.

Device for shape shifting notification

The core idea of PCT is that a human does not recognize
subtle changes in a peripheral area of cognition when he/sheconcentrates on a task, and he/she automatically recognizes
the changes when they are not concentrating. We think PCT
provides an effective design policy for various properties
of HAI like the appearance, the shape, and the positions
and so on. Two concrete ongoing implementations of PCT,
“Peripheral Agent” and “Shape Shifting Notification”, wereintroduced and preliminary experiments for them were conducted.
R EFERENCES

Fig. 6.

Slow shape shifting

tweet, the device begins to open the top and gets up with theterminal very slowly for three minutes. All of the motions are
executed by a servomotor and Fig.6 shows such behaviorsof the device. Since the motion of the device is sufficiently
slow, a user notices the information notification only when
he/she is interruptible.
We already conducted a preliminary experiment with asingle participant. In this experiment, the device was located45cm from the display and the participant could see itin the peripheral visual field. It is not important that thedevice is in the peripheral field because the sufficiently
slow motion of the device makes a user unaware of thenotification even though the device is nearby. The participant
actually used the device for two weeks in the office. The
device notified a user randomly from a minute to 30 minutes
and questionnaires on the notification timing and they wereasked about interruptibility just after every notification. The
questionnaires obtained promising experimental results; The
hight ratio of notification without bothering and the low ratioof bothering a participant were obtained experimentally.
We are currently developing the next version of this device
for shape shifting notification by improving its execution
noise of a servomotor and adjusting the position. After the
improvements, to evaluate the effectiveness of the device wewill conduct experiments by comparing it with a device thatgets up quickly.
V. C ONCLUSION
We proposed a novel information notification method:
Peripheral Cognition Technology (PCT), which utilizes ahuman cognitive properties including visual field narrowing
and change blindness (or inattentional blindness). Information notification on a display for e-mail arrival, micro blog,
and application update, is becoming increasingly important.
In contrast with conventional approaches to the information
notification like a user model-based approach and a peripheral display, PCT does not need to estimate a user state andit can deal with notification for complicated contents.

[1] S. T. Iqbal and B. P. Bailey, “Oasis: A framework for linking
notification delivery to the perceptual structure of goal-directed tasks,”
ACM Transactions on Computer-Human Interaction, vol. 17, pp. 15:1–
15:28, December 2010.
[2] ——, “Effects of intelligent notification management on users andtheir tasks,” in Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, ser. CHI ’08. New
York, NY, USA: ACM, 2008, pp. 93–102.
[3] B. P. Bailey, J. A. Konstan, and J. V. Carlis, “The effects of interruptions on task performance, annoyance, and anxiety in the userinterface,” in INTERACT 2001. IOS Press, 2001, pp. 593–601.
[4] E. Horvitz and J. Apacible, “Learning and reasoning about interruption,” in Proceedings of the 5th international conference on Multimodal interfaces. New York, NY, USA: ACM, 2003, pp. 20–27.
[5] J. Fogarty, S. E. Hudson, and J. Lai, “Examining the robustnessof sensor-based statistical models of human interruptibility,” inProceedings of the SIGCHI conference on Human factors in computing
systems, ser. CHI ’04. New York, NY, USA: ACM, 2004, pp. 207–
214. [Online]. Available: http://doi.acm.org/10.1145/985692.985719[6] J. Fogarty, S. E. Hudson, C. G. Atkeson, D. Avrahami, J. Forlizzi,
S. Kiesler, J. C. Lee, and J. Yang, “Predicting human interruptibility
with sensors,” ACM Transactions on Computer-Human Interaction,
vol. 12, no. 1, pp. 119–146, Mar. 2005.
[7] T. Matthews, A. K. Dey, J. Mankoff, S. Carter, and T. Rattenbury,“A toolkit for managing user attention in peripheral displays,” in
UIST ’04: Proceedings of the 17th annual ACM symposium on Userinterface software and technology, 2004, pp. 247–256.
[8] J. Mankoff, A. K. Dey, G. Hsieh, J. Kientz, S. Lederer, and M. Ames,“Heuristic evaluation of ambient displays,” in Proceedings of the
SIGCHI conference on Human factors in computing systems, ser. CHI’03. New York, NY, USA: ACM, 2003, pp. 169–176.
[9] D. S. McCrickard, R. Catrambone, and J. T. Stasko, “Evaluating
animation in the periphery as a mechanism for maintaining awareness,”
in Proceedings of the IFIP TC.13 Conference on Human- Computer
Interaction (INTERACT 2001), 2001, pp. 148–156.
[10] P. P. Maglio and C. S. Campbell, “Tradeoffs in displaying peripheral
information,” in Proceedings of the SIGCHI conference on Human
factors in computing systems (CHI-2000). New York, NY, USA:ACM, 2000, pp. 241–248.
[11] T. Stafford and M. Webb, Mind Hacks -Tips & Tricks for Using Your
Brain-. O’Reilly Media, 2004.
[12] B. Shneiderman and P. Maes, “Direct manipulation vs. interface
agents,” interactions, vol. 4, no. 6, pp. 42–61, Nov. 1997.
[13] L. Williams, “Peripheral target recognition and visual field narrowing
in aviators and nonaviators.” International journal of aviation psychology, 1995.
[14] D. Crundall, G. Underwood, and P. Chapman, “Driving experienceand the functional field of view,” Perception, vol. 28, pp. 1075–1087,
1999.
[15] J. Cavanaugh and R. H. Wurtz, “Subcortical modulation of attention
counters change blindness,” The Journal of Neuroscience, vol. 24,
no. 50, pp. 11 236–11 243, 2004.
[16] R. A. Rensink, J. K. O’Regan, and J. J. Clark, “To see or not to see:
The need for attention to perceive changes in scenes,” Psychological
Science September, vol. 8, no. 5, pp. 368–373, 1997.
[17] D. J. Simons and C. F. Chabris, “Gorillas in our midst: sustained
inattentional blindness for dynamic events,” Perception, vol. 28, pp.
1059–1074, 1999.