Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

Peripheral Agent: Implementationof Peripheral Cognition Technology
Abstract
Seiji Yamada
National Institute ofInformatics / SOKENDAI /Tokyo Institute ofTechnology
2-1-2 Hitotsubashi, Chiyoda
Tokyo, 101-8430 Japanseiji@nii.ac.jp

Kazuki Kobayash
Graduate School of Scienceand Technology
Shinshu University
4-17-1 Wakasato
Nagano, 380-8553 Japan
kby@shinshu-u.ac.jp

Naoki Mori
Department of Computational
Intelligence and Systems
ScienceTokyo Institute of Technology
4259 Nagatsuta, Midori
Yokohama, 226-8503 Japan
mori@ntt.dis.titech.ac.jp

Information notiﬁcation on a display for e-mail arrival,
micro-blog updates, and application updates is becoming
increasingly important. We propose a novel information
notiﬁcation method, the peripheral agent (PA) as animplementation of peripheral cognition technology (PCT)that uses the human cognitive properties that a humandoes not recognize subtle changes in a peripheral area of
cognition when he/she concentrates on a task and thathe/she automatically recognizes the changes when notconcentrating on the task. By only setting a PA in the
peripheral area, a user automatically and easily accepts
the notiﬁcation only when his/her concentration breaks.
We conducted two experiments to investigate a VFN areaand evaluate the eﬀectiveness of PAs.

Author Keywords
Information notiﬁcation; peripheral cognition technology;
visual ﬁeld narrowing

ACM Classification Keywords
H.5.2 [[Information Interfaces And Presentation]]: User
Interfaces - Theory and methods;
Copyright is held by the author/owner(s).
CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris,
France.
ACM 978-1-4503-1952-2/13/04.

General Terms
Experimentation; Design; Human Factors.

1701

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

Introduction
In the current oﬃce and home environment connected to
the Internet, a user tends to frequently receive various
notifications [4] for e-mails, tweets, instant messages, andalerts of application updates on displays, cellular phones,
and smart phones (Figure1). Although some notiﬁcations
are quite emergent, most of them are not, so a user should
get such information only when he/she is not busy. Such auser state in which he/she can accept notiﬁed informationand can access the information is called interruptible, andthe opposite user state is called uninterruptible. A typicalinterruptible state is when a user is not involved incompleting a task and has time to access the information.
Figure 1: Information
notifications.

A problem with such notiﬁcations is that they appear assoon as they occur, i.e., without the system being awareof whether the user is interruptible or not. If thenotiﬁcations arrive when a user is uninterruptible, they
can cause signiﬁcantly stress and reduce the user’s
productivity [1]. One way to deal with this would be tocontrol the notiﬁcation period in accordance with theperiod of time in which the user is interruptible.
To estimate a user’s interruptibility, various studies havebeen done [2][4]. Most of them applied machine learningtechniques to identify interruptibility. The input forclassiﬁcation learning was implicit feedback, which
included keyboard typing patterns, the trajectory of mouse
operation, visual information of a user’s face, and posture.
Although this approach provides a general framework, it isdiﬃcult to build an accurate model for estimating a user’s
interruptibility because much training data are necessary
and capturing a user’s face and posture is problematic interms of privacy.
A peripheral display [6][5] provides another approach to
information notiﬁcation, that does not estimate

interruptibility. In this approach, information itself, notthe notiﬁcation, is constantly displayed on a small
sub-window (or a sub-display) on the side of a main
window (or a main display) in which a user works on amain task. A user is assumed to recognize the displayed
information and understand the contents while he/sheworks on the main task. Although this peripheral display
has an advantage in that it does not need user state
estimation, ad-hoc implementations have been done invarious ﬁelds and more complicated information that isdiﬃcult to understand, such as the content of e-mails andapplication updates, can not be dealt with.
We propose a novel information notiﬁcation method, theperipheral agent, as an implementation of peripheral
cognition technology (PCT) that uses human cognitive
properties such as visual ﬁeld narrowing and inattentional
blindness[10]. PCT provides notiﬁcation by using thehuman cognitive property that the visual ﬁeld narrows
when he/she concentrates on a task intently. The coreidea of PCT is that humans do not recognize subtle
changes in a peripheral area of cognition when they
concentrate on a task, and they automatically recognize
the changes ﬁrst when not concentrating on the task.
With the peripheral agent, by only making a software
agent appear in a region where visual ﬁeld narrowing
occurs, a user automatically and easily accepts thenotiﬁcation only when his/her concentration breaks.

PCT: Peripheral Cognition Technology
Peripheral cognition technology (PCT) is cognitive
interaction design for information notiﬁcation.
“Cognitive” interaction design means a design method ofinteraction that uses human cognitive properties [8], e.g.,visual ﬁeld narrowing and inattentional blindness. By
using such human properties, we can design an eﬀective

1702

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

user interface that does not force a high cognitive load
without user state estimation.

Figure 2: Device for shape
shifting notification.

Visual ﬁeld narrowing (VFN) [9] is a primary humancognitive property used in PCT. A visual ﬁeld is the areathat an individual can see when looking straight ahead
without moving the eyes or gaze. It can be measured indegrees from a ﬁxation point; the normal vision ﬁeld isabout 160-180 degrees horizontally and 120 degrees
vertically. This visual ﬁeld is signiﬁcantly aﬀected bycognitive load or concentration on a task [9]. When ahuman concentrates on a hard task and the cognitive load
becomes heavy, his/her visual ﬁeld signiﬁcantly narrows.
This phenomena is called visual field narrowing andstudies have been done in various tasks [9].
Inattentional blindness [7] means that without attention,observers do not recognize visual features of theenvironment at all. On the basis of this property, we areexperimentally developing a shape shifting notification
device [10] shown in Figure 2.
We deﬁne these above human cognition properties that
occur in the peripheral areas of “cognition” and deﬁne theinteraction design that uses such human cognitive
properties as peripheral cognition technology (PCT).

Peripheral Agent
The peripheral agent (PA) is an implementation of PCT
by using visual ﬁeld narrowing (VFN). The peripheral
agent has a simple human-like appearance and notiﬁes auser of information such as e-mail arrival, micro-blogs,
and application updates (Figure 1) by appearing on adisplay in a VFN region. The functions are the following.
1. A PA appears in a VFN region as soon as
information to be notiﬁed occurs.

2. If a user clicks the peripheral agent, it will display
the content of the information by opening a
sub-window.
When we can adequately place a PA in the VFN region, auser does not notice a notiﬁcation when he/she isuninterruptible and he/she automatically notices thenotiﬁcation just when he/she becomes interruptible.
To realize a PA, we ﬁrst need to deﬁne the VFN region
mentioned above as a peripheral visual ﬁeld that can be
narrowed by VFN phenomena. Figure 3 shows a visualﬁeld when a user is concentrating on a task and isuninterruptible, a VFN region outside of the visual ﬁeld
with an inner boundary, and a PA. We assume the innerboundary of the VFN region is squared and the VFN
region can be deﬁned by the absolute positions of the foursides of the inner boundary. Although the boundaries of a
VFN region might be changed depending on the user’s
viewpoint on a display, the variance of the movement is
considered to be relatively small because the user’s
viewpoint does not move greatly outside of the centerarea of the display in oﬃce work. Even if the VFN region
signiﬁcantly moves because of the movement of the user’s
viewpoint, we can use various methods to estimate theviewpoint with ordinary sensors like a mouse and a
web-cam [3]. When the VFN region is deﬁned, we canrealize a PA only by placing it there as a notiﬁcation
occurs.
Investigating a VFN region
We conducted an experiment with participants toinvestigate the VFN region (the inner boundary of the
VFN region in Figure 3) on a PC display. We used a darkroom-like editor with black background and green
foreground, and we asked participants to simply type theletters scrolling horizontally in the squared area

1703

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France(40 mm×6 mm) in the center of the display and the typed
letters appeared under the scrolling area. This typing is
the main task in this experiment and we adjusted thescrolling speed so that each participant would beuninterruptible when concentrating on the main task.
Participants were also asked to step on a foot switch assoon as they noticed the PA as a sub-task. The display
size was 23 inches with 1920×1080 pixels and the display
was located about 50 cm from the head of eachparticipant.

24 mm

10 mm

VFN
regionFigure 4: PA’s appearance.
Notification

Main task

Inner
Peripheral

boundary

agent

three trials with short breaks. The numbers and thetiming from the start time of the PA’s appearances were
three, 40 sec, 30 sec, and 50 sec for the ﬁrst trial, two,
30 sec, and 40 sec for the second one, and one and 50 sec
for the third one. These diﬀerent times and timings werealso for avoiding participants’ prediction. All the typing
logs and the positions and the time when participants
noticed the PA were recorded.
Since the dark room-like editor is considered to be an
extreme situation in which a user most easily notices aperipheral agent, the estimated VFN region corresponds
to the smaller limit region. Thus, in practical application
on a display with a practical desktop and a certain
background image, the VFN region becomes larger than
this estimated one. An experimental environment with adisplay, a keyboard and a foot switch with a red circle isshown in Figure 5.

Visual field
concentrated on

600-830 , 521-15 , 496
782 , 510

400

200

Figure 3: Peripheral Agent.
-829 , 45

Figure 5: Experimental
environment.

The shape, size, and color of a PA was also tuned asshown Figure 4 so that a participant would not
immediately notice it when uninterruptible. We made a
PA to randomly fade in for 3 sec in eight positions on theedges of the screen (the eight dotted circles from far tonear the center of the display by 5 mm. This random
appearance prevented participants from predicting theappearances of the PAs.
Participants were asked to join a training in which they
performed the main task without the sub-task before theexperiment with a sub-task. The experiment consisted of

786 , 45
0

-1000

-800

-600

-400

-200

0

200

400

600

800

1000

-200

-400
-15 , -401
-832 , -433

795 , -428

-600

Figure 6: Experimental results of VFN region.

Data of the inner boundary of the VFN region obtained
from the 20 participants (9 females, 11 males, ages

1704

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

23∼51) are plotted on a x-y coordinate of a display in
Figure 6. The broken line connects the average positionsin the eight directions, and we veriﬁed the VFN region ina display from these data of this preliminary experiment.
As seen from this graph, since the vertical variances wererelatively large and the horizontal ones were small, we
used horizontal boundaries to evaluate a PA. Eventually,
we deﬁned two rectangle regions located about 60 mm
from both left and right edges of this display (shaded
regions in Figure 6), and a PA appeared within theseregions in the next evaluation experiment.

Figure 7: Snapshot of theevaluation experiment.

Evaluation experiment for peripheral agent
Next we conducted an experiment to evaluate theeﬀectiveness of a PA in a more realistic environment. We
prepared an environment similar to one in everyday oﬃce
work on a computer with a main task window and two
sub-windows on both sides, as shown in Figure 7. In thisenvironment, the main task and the sub-task were thesame as those in the previous experiment. Participants
were told to suppose that notiﬁcations were application
updates and e-mail arrival, and they were asked to accept
the notiﬁcation by using a mouse to click on a PA if
necessary.
We prepared a traditional balloon-like notiﬁcation (theleft-bottom in Figure 7) that appeared on theright-bottom corner of a display for comparison with our
PA notiﬁcation and conducted a within-participant
experiment because there was no learning eﬀect. Thus,
the independent variable was a notiﬁcation method with
two levels of a traditional notiﬁcation and a PA
notiﬁcation, and the dependent variables were various
timing mentioned later.
The time-line of this experiment is shown in Figure 8 andboth of the two levels were made on this time-line. The

task load (attention) was uninterruptible with the text
scrolling for 0∼80 sec and was then made interruptible bystopping the text scrolling. A notiﬁcations appeared at
40 sec when a user was still uninterruptible; hence, theuser’s optimal noticing, ON T of the notiﬁcation was justafter 80 sec. We recorded each user’s actual noticing
timing, N T , by monitoring the foot switch, and measured
the diﬀerence between ON T and N T , d(ON T , N T ) ={ON T − N T if ON T > N T , 0 if otherwise }, where d =0 as the N T was after ON T because the notiﬁcation was
not emergent and the delay of a user’s noticing from the
ON T was not problematic. When the d(ON T , N T ) was
smaller, the notiﬁcation was better. Also, we asked
participants to answer some questionnaires on usability
based on NASA-TLX with a seven-point scale that
included “Did the notiﬁcations bother you?,” “Did you
worry about the notiﬁcation?,” and “How good were thenotiﬁcation timings for you?”
Attention（Task load）

Task start
t=0

1st
notificationt = 40

Notification Timing
Load Change Timing
Optimal Notice Timing (ONT)

Task stopt = 80Time [s]Notice Timing (NT)Acceptance Timing

Figure 8: Time-line of the evaluation experiment.

Fifteen participants (6 females, 10 males, ages 21∼49)
joined this experiment. The average and SD of d for the
PA and traditional notiﬁcation is shown in Figure 9. We

1705

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

applied a t-test to the results, and there was signiﬁcantdiﬀerence between the two levels (p = 4.0×10−7 ). Thus,
we found that the results showed that our PA notiﬁcation
was better than the traditional notiﬁcation in terms of
notiﬁcation timing.
d [sec]60

Discussion**

The environment might be restricted and small even in
the evaluation experiment. However, the typing task is
one of the most general oﬃce and home tasks performedon PCs, and the two sub-windows are considered to be a
popular layout. Hence, we think the experimental results
and knowledge derived from them have much generality.

50
40
30
20
10
0

Traditional

PA

-10

Figure 9: Results of d in theevaluation experiment.

We are currently analyzing the results of the participants’questionnaire results. For the questionnaires on“notiﬁcation’s bothering,” the PA notiﬁcation was
signiﬁcantly better than the traditional one. However
there was not a signiﬁcant diﬀerence between them on“notiﬁcation timing’s suitability.” We need to discussthem more.

Conclusion
We proposed a novel information notiﬁcation method,
peripheral agent (PA), as an implementation of peripheral
cognition technology (PCT), which uses human cognitive
properties such as visual ﬁeld narrowing (VFN) andinattentional blindness. Notiﬁcation on a display of e-mailarrival, micro-blogs updates, and application updates, isbecoming increasingly important. In contrast withconventional approaches to notiﬁcation, such as a usermodel-based approach and a peripheral display, a PA does
not need to estimate a user state and can deal with
notiﬁcation of complicated contents. We conducted two
experiments to investigate a VFN region and evaluate theeﬀectiveness of PAs. We then deﬁned the VFN region

where a PA should be placed and obtained results showing
the advantages of using PAs.

References[1] B. P. Bailey, J. A. Konstan, and J. V. Carlis. The
eﬀects of interruptions on task performance,
annoyance, and anxiety in the user interface. In
INTERACT’01, pages 593–601, 2001.
[2] E. Horvitz and J. Apacible. Learning and reasoning
about interruption. In ICMI’03, pages 20–27, 2003.
[3] J. Huang, R. White, and G. Buscher. User see, user
point: gaze and cursor alignment in web search. In
CHI’12, pages 1341–1350, 2012.
[4] S. T. Iqbal and B. P. Bailey. Oasis: A framework forlinking notiﬁcation delivery to the perceptual
structure of goal-directed tasks. ACM Transactions
on Computer-Human Interaction, 17:15:1–15:28,
2010.
[5] J. Mankoﬀ, A. K. Dey, G. Hsieh, J. Kientz,
S. Lederer, and M. Ames. Heuristic evaluation ofambient displays. In CHI’03, pages 169–176, 2003.
[6] T. Matthews, A. K. Dey, J. Mankoﬀ, and S. C. T.
Rattenbury. A toolkit for managing user attention in
peripheral displays. In UIST’04, pages 247–256,
2004.
[7] D. J. Simons and C. F. Chabris. Gorillas in our
midst: sustained inattentional blindness for dynamic
events. Perception, 28:1059–1074, 1999.
[8] T. Staﬀord and M. Webb. Mind Hacks -Tips &Tricks for Using Your Brain-. O’Reilly Media, 2004.
[9] L. J. Williams. Peripheral target recognition andvisual ﬁeld narrowing in aviators and nonaviators.
International journal of aviation psychology, 1995.
[10] S. Yamada, K. Kobayashi, and N. Mori. Peripheral
cognition technology: Approach and implementation.
In iHAI’12, TW8 0001, 2012.

1706