Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

Estimating User Interruptibility
by Measuring Table-top Pressure
Takahisa Tani
The Graduate University for
Advanced Studies
2-1-2 Hitotsubashi, Chiyoda
Tokyo 101-8430, Japan
tani@nii.ac.jp

Seiji Yamada
National Institute ofInformatics
2-1-2 Hitotsubashi, Chiyoda
Tokyo 101-8430, Japanseiji@nii.ac.jp

Abstract
A user working with his/her desktop computer would
beneﬁt from notiﬁcations (e.g., the arrival of e-mail,micro-blogs, and application updates) being given at
adequate times when he/she is interruptible. To do so, a
notiﬁcation system needs to determine the user’s state ofactivity. In this paper, we propose a novel method for
estimating user states with a pressure sensor on a desk.
We use a lattice-like pressure sensor sheet and distinguishbetween two simple user states: interruptible or not. The
pressure can be measured without the user being aware ofit, and changes in the pressure reﬂect useful informationsuch as typing, an arm resting on the desk, mouse
operation, and so on. We carefully developed featuresthat can be extracted from the sensed raw data, and weused a machine learning technique to identify the user’s
interruptibility. We conducted experiments for two
diﬀerent tasks to evaluate the accuracy of our proposed
method and obtained promising results.

Author Keywords
Figure 1: Various information notifications
Copyright is held by the author/owner(s).
CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris,
France.
ACM 978-1-4503-1952-2/13/04.

User state estimation; Notiﬁcation; HAI(Human-AgentInteraction).

ACM Classification Keywords
H.5.2 [[Information Interfaces And Presentation]]: User
Interfaces - Theory and Methods;

1707

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

General Terms
Experimentation; Design; Human Factors.

Introduction
In the current oﬃce environment connected to theInternet, a user tends to get a lot of notifications[7] in theform of e-mails, micro-blogs, instant messages, application
update alerts, and so on, like in Figure 1.
A signiﬁcant problem with such notiﬁcations is that they
arrive as they are sent, i.e., without the system being
aware of whether the user has time to receive them ornot. If messages arrive at inopportune times, they cancause serious stress and reduce the user’s productivity [2].
One way of alleviating this problem would be to control
the notiﬁcation period in accordance with the user’s stateof activity. In other words, this means a system needs toestimate whether a user is interruptible or not and to send
information only when he/she is interruptible [8].
Figure 2: LL-Sensor.

Figure 3: Output of sensor.

Hence, a notiﬁcation system needs to monitor user
behaviors like typing, operating a mouse, and so on toestimate whether he/she is interruptible or not. There are
a number of studies on systems that use the frequency of
keyboard strokes and mouse operations [5]. However,these methods cannot be applied to cases in which thefrequency does not reﬂect the user’s interruptibility or
when the user does not use such input equipment. There
are also estimation methods that use additional sensors
like a web camera [6]. However, these methods need tomonitor user behavior by watching their faces and bodies,and thus, they could cause psychological stress on them.
In this study, we developed a novel method for estimating
a user’s interruptibility by measuring tabletop pressure. At
a desk with a PC, there are changes in pressure on thetabletop caused by the forces of various user behaviors

such as typing, resting one’s arm on the desk, and so on.
We considered that useful information not acquired byanother methods for estimating the user’s interruptibility
can be extracted from slight changes in tabletop pressure.
A paper says unconscious inﬂuences grip force[1]. Our
method may detect such diﬀerences in force that areundetectable with a method using a web camera.
However, there are only a few studies on estimatinginterruptibility by using tabletop pressure. First, we
carefully identiﬁed features adequate for the estimation.
Then, we compared three machine learning techniques toclassify a user’s interruptibility. Finally, we conducted
experiments to evaluate the accuracy of our method in
two diﬀerent tasks.

Estimating a User’s Interruptibility by Measuring Tabletop Pressure
For measuring the tabletop pressure, we spread a pressure
sensor sheet having measurement points in a reticularpattern on a tabletop. We assumed that a user does all
his/her work on the sheet and that all objects on thetabletop are placed on it. We investigated the forces
involved in typing, resting one’s arm, and placing objects
on the tabletop, and we found that we need a sensor sheet
about 1 meter square with pressure gradation ability of 10
grams.
Hence, we used the LL-sensor (Xiroku Co., Ltd ) shown in
Figure 2. This sensor leverages the feature of mutual
induction. It is 600 mm square, and its resolution is 10
mm square. It outputs not a physical quantity but aunique value. Figure 3 shows the output of the LL-sensor
in a heat-map fashion. The white unit means the lowest
value and the blue, green, red, and black mean higher
values. A keyboard is placed on the square area, and theuser’s arms are placed on the elliptical areas.

1708

Work-in-Progress: UI Design

Figure 4: Experimental
environment.

(a) Typing.

CHI 2013: Changing Perspectives, Paris, France

Features Extraction for Interruptibility Estimation
We extract features from the raw pressure data. In
particular, we use the weight of the key pressing force, thelocation, and changes in them as features. We assume
that the objects on a tabletop are only the keyboard of
the PC and the users’ arms. However, we can relax thisrestriction by developing procedures to detect otherobjects like a mug, a book, and so on. We considered
typical tasks on a tabletop to be typing andmouse-operation and that the features related to the twotasks should be extracted. Also, the information related toarm regions is useful because it might change due to auser’s busyness. Thus, we introduce 24 features: 4parameters (the pressure, the region size, thex/y-coordinates of the center of gravity) for 6 regions (akeyboard’s left/right/front feet regions, the mouse region
and the left/right arm regions).
We used a simple pattern matching algorithm toautomatically extract the features from the raw pressuredata. First, templates for the regions pressed on by the
keyboard’s feet and the mouse are manually obtained, andthey are normalized. Then, by scanning the whole areawith the templates, the regions corresponding to thetemplates are identiﬁed. Eventually four parameters forthe regions are calculated.
To extract left/right-arm regions, the whole data arescanned from the left/right-bottom to the right/left-top
and the regions having pressure values over a thresholdare extracted. Then four parameters for the regions arecalculated.

(b) Mouse operation.

The raw pressure data are obtained from a LL-sensor every
30ms, and the features can be quickly computed anytime
with an average of nine frames of data 1 sec before.

Classifying User Interruptibility
After extracting features from the raw data, we needed to
identify a user’s interruptibility from the data. We usedclassiﬁcation learning to classify the state of a user into
interruptible or uninterruptible.
In the experiments, we try to apply state-of-the-arts
classiﬁcation learning algorithms: random forests [3],SVM (Support Vector Machine) with a kernel [10], and a
traditional algorithm, C4.5 [9]. We can expect theclassiﬁcation accuracy of the random forests and SVM to
be higher than that of the C4.5. However, the C4.5 has
human readability, so it is easily interpreted. We apply
these classiﬁcation learning algorithms and compare theresults in the experiments.

Experiments
Experimental Environment and Tasks
We built a simpliﬁed desk work environment to eliminate
complicated factors. Figure 4 is an overview of it. A
participant sits down in front of a desk, and the monitor
shows a task window.
We use two typical tabletop tasks: a typing task and amouse operation task. In the typing task, an experimenter
required participants to type the scrolling display ofcharacters as correctly as possible (Figure 5(a)), and theirtyping was recorded. The scroll speed could be used tocontrol the task diﬃculty. When typing, the system asked
them whether they were interruptible or uninterruptible bydisplaying a notiﬁcation dialog in the center (Figure 6).
At that time, participants step on a foot switch on theirleft side to accept it (interruptible), and step on a footswitch on their right side to reject it (uninterruptible).
These judgments were used as class labels for training
data. The “accept” and “reject” corresponded to

Figure 5: Task windows for twotasks.

1709

Work-in-Progress: UI Design

Do you accept
this notification?
AcceptRejectFigure 6: Notification window inthe typing task.

CHI 2013: Changing Perspectives, Paris, France“interruptible” and “uninterruptible.” The notiﬁcation
window closed after the answer. The participants wereasked to suppose that the notiﬁcations provided themwith small amounts of information like weather reports
and news. These procedures were explained toparticipants in the instructions.

displayed every 20 sec as described before.

In the mouse operation task, the task window in
Figure 5(b) was displayed, and a small blue squareappeared and disappeared repeatedly at random positions.
This appearance cycle could be used to control the task
diﬃculty. Participants were required to repeatedly click
the small blue square when it appeared. The notiﬁcation
procedures were the same as the typing task.

Table 1 shows part of the data obtained for the left arm in
the typing task. The number of all the data obtained fromthe two tasks was 1199 (interruptible: 722,
uninterruptible: 477), and those of the typing and mouse
operation tasks were respectively 600 (interruptible: 333,
uninterruptible: 267) and 599 (interruptible: 389,
uninterruptible: 210). Since the data were notsigniﬁcantly imbalanced, we did not apply
under-sampling[4].

Participants and Experimental Procedures
The participants were 20 students and staﬀ members in
the information science department (ages ranging from 23
to 51, mean 35.4, S.D. = 11.7), and they consisted of 9
males and 11 females.
To obtain both accept and reject data, we needed to set
hard and easy phases in the two tasks. In the hard phase,participants have to perform the task, so they tend toreject the notiﬁcations (uninterruptible), and in the easy
phase, they tend to accept them (interruptible). Before
trials with each participant, an experimenter manually set
both phases by investigating his/her typing and mouse
operation ability.
For each participant, two trials of the typing task and themouse operation task were given, and the order of thefour trials was counterbalanced. In each trial, the hardand easy phases were alternately applied for one minute,and each trial was ﬁve minutes long, consisting of ﬁve
phases. Eventually, the total time for a participant was
about 20 minutes. In each trial, notiﬁcations were

We used random forests, SVM (a RBF kernel, γ = 0.0)
and C4.5 (conﬁdence factor = 0.25) implemented on
weka3.6.4 1 as classiﬁcation learning algorithms.

Experimental Results

After giving the whole data to the three classiﬁcation
learning algorithms as the training data, the data were
evaluated by using 10-fold cross validation. The
experimental results on the accuracy of the threeclassiﬁcation learning algorithms are shown in Table 2,
where the “TP rate” means the rate of correctly classiﬁed
data and the accuracy for the two tasks and the wholetask is described.
Table 1: Data examples (left arm).

Area
7.6
4.9
3.7
15.1

Pressure
20.9
24.6
8.8
22.4

COGx
2.8
1.9
3.0
12.1

COGy
20.1
20.8
39.0
16.5

Class
acceptrejectacceptrejectFrom the table, we ﬁnd that the SVM outperforms theother two CL algorithms in the typing task, the mouse
1 http://www.cs.waikato.ac.nz/ml/weka/

1710

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

operation task, and the whole task. Thus, SVM should beapplied to this work.
Table 2: Accuracy of estimation of the three classification
learning algorithms (The best in bold).

CL algorithm
SVM
Random forests
C4.5

TP Rate [%]Typing Mouse op. Whole
76.8
72.3
73.6
75.8
69.6
73.1
71.7
71.6
71.3

Discussion
Evaluation of Estimating Interruptibility with Tabletop Pressure
The experimental results show our method’s estimation ofuser interruptibility has about 77% accuracy for the typing
task and about 72% for the mouse operation. We
consider our main target to be the typing task because
editing documents is the main desk work at an oﬃce. We
consider this level of accuracy (about 77%) for the typing
task to be suﬃcient as a ﬁrst trial of using tabletop
pressure and that it shows our approach to be promising.
Furthermore, we can combine our method withconventional methods with other sensors.
As seen in Table 2, the accuracy of the mouse operation
task is less than that of the typing task. In contrast withthe keyboard’s feet pressure, the mouse pressure might be
hard to sense with the LL-sensor and might slightly change
during mouse operation. We have signiﬁcant problems to
overcome to improve accuracy in the mouse operation.
Estimation of the State of a User Not Typing
Our method can estimate the user’s state only when theuser is typing on a keyboard on a desktop because apressure sensor cannot sense any change in pressure when
the user is not typing. Thus, in the experimental
evaluation, we assumed that a user could be interrupted
when s/he was not typing.
However, this assumption is not always valid in realenvironments. For example, a user might be thinking,reading web pages, or watching a movie on the display
when they are not typing, and they would not want to beinterrupted in such situations. To cope with this problem,
we plan to extend the current features to cover no-typing
situations. We will introduce additional features such as
sensing the pressure of a mug and the shape and area
occupied by arms resting on a desktop, which the pressure
sensor can sense. We consider these additional features tobe promising because a user does not pick up a mug
frequently and does not change the position of his/herarm much when concentrating on something.
Feature Selection for Practical Applications
Although we veriﬁed that our approach of estimating auser’s interruptibility by measuring tabletop pressure is
promising, the LL-sensor with a lattice of small pressure
sensors is unique and expensive, and this makes our
method diﬃcult to apply to practical applications in a realenvironment. We consider a way to solve this problem is
to use small and inexpensive pressure sensors. There arevarious such pressure sensors that can be easily attachedto the three feet of a keyboard and the bottom of amouse. Thus, if a user’s interruptibility can be estimatedonly with inexpensive sensors as accurately as with all thepressure sensors of a LL-sensor, our approach will be quite
more practical.
Such pressure sensors can be simulated by detecting thedata corresponding with the keyboard feet regions and themouse region from all the data obtained from the1711

Work-in-Progress: UI Design

CHI 2013: Changing Perspectives, Paris, France

LL-sensor. The results on accuracy are shown in Figure 3.
As seen in the table, in the typing and the mouse
operation tasks, the accuracy of only the keyboard
pressure (+ a mouse pressure) is almost equal to that of
all the sensors (LL-sensor). Thus, our approach can bemade more practical by using small and inexpensive
sensors. Meanwhile, there are signiﬁcant diﬀerences inwhole tasks.
Table 3: Accuracy of feature selection.

Selected features
Keyboard pressure
Mouse pressure
Keyboard + mouse
All

Typing
74.7
–
75.8
76.8

TP Rate [%]Mouse op. Whole
–
–
64.3
–
70.0
69.8
72.3
73.6

CONCLUSION
We developed a novel method for estimating a user’s
interruptibility by using tabletop pressure. We conducted
experiments with participants in two typical tabletop tasks
to show that our method could estimate when a user was
too busy to receive typical messages. As a result, we
conﬁrmed our method could estimate the interruption.
Then we discussed the accuracy of our method and a way
to make our method more practical.
In the future, we will use richer features taken from realexperimental environments. This will help to increase
accuracy and make it possible to estimate activity statesof users when they are not using the keyboard. For that
purpose, we will try to determine the optimal number ofthe features. In addition, we will assess the utility of
cost-sensitive learning.

References[1] Aravena, P. et al. Grip force reveals the context
sensitivity of language-induced motor activity during
action words processing: Evidence from sentential
negation. PLoS ONE 7, 12 (12 2012), e50287.
[2] Bailey, B. P., Konstan, J. A., and Carlis, J. V. The
eﬀects of interruptions on task performance,
annoyance, and anxiety in the user interface. In
Proceedings INTERACT ’01, IOS Press (2001),
593–601.
[3] Breiman, L. Random forests. Machine Learning 45, 1(2001), 5–32.
[4] Drummond, C., Holte, R., et al. C4.5, classimbalance, and cost sensitivity: why under-samplingbeats over-sampling. In Proc. of Workshop on
Learning from Imbalanced Datasets II (2003).
[5] Epp, C., Lippold, M., and Mandryk, R. Identifying
emotional states using keystroke dynamics. In CHI’12(2011), 715–724.
[6] Fogarty, J. et al. Predicting human interruptibility
with sensors. ACM Transactions on
Computer-Human Interaction 12, 1 (2005), 119–146.
[7] Iqbal, S. T., and Bailey, B. P. Eﬀects of intelligent
notiﬁcation management on users and their tasks. In
CHI’08 (2008), 93–102.
[8] Iqbal, S. T., and Bailey, B. P. Oasis: A framework
for linking notiﬁcation delivery to the perceptual
structure of goal-directed tasks. ACM Transactions
on Computer-Human Interaction 17 (2010),
15:1–15:28.
[9] Quinlan, J. R. C4.5: programs for machine learning.
Morgan Kaufmann Publishers Inc., 1993.
[10] Vapnik, V. The nature of statistical learning theory.
Springer-Verlag New York Inc, 2000.

1712