Human-Robot Interaction Design for Low Cognitive Load in
Cooperative Work
Kazuki KOBAYASHIThe Graduate University for Advanced Studies
2-1-2 Hitotsubashi, Chiyoda
Tokyo, 101-8430, Japankazuki@grad.nii.ac.jp

Abstract
This paper describes effective interaction design fora cooperative task in which a mobile robot sweeps overa table and a human can help it by picking up an obstacle. For efficient cooperation, we introduce behaviors
of a robot to sweep a region under an obstacle whena human picks up it vertically, and design the robot
based on interaction embedded in a human’s operations
to achieve a task. This interaction enables a humanto easily cooperate with a robot without additional andexplicit commands. We made experiments to evaluate
the interaction embedded in a task by comparing withother explicit interaction, and the experimental results
show our cooperative sweeping is more efficient thanautonomous one and our interaction embedded in atask needs less cognitive load than other explicit interaction such as voice or leading the robot by hand.

1

Introduction

Home robots such sweeping robots, carrying robots
begin to spread among people through progression ofrobot technology [1]. Most of the home robots are designed to autonomously achieve every life task such as
seeping rooms, carrying baggage and so on. However,
it is not practical that such an autonomous robot completely achieves its task by itself. For instance, a homerobot “Roomba” can sweep a room autonomously and
it is very useful to sweep a simply structured room
without obstacles like a chair or a trash can. However,a human actually needs to help the robot by removing obstacles, since it cannot remove such obstacles
during sweeping. Thus designing a robot to cooperate
effectively and efficiently with a human is important.
Various researches have been studied in a field
human-robot interaction [2] [3] [4] [5] [6] . Most of
the researches have tried to develop a robot that communicates explicitly with a human by gesture [7] [8][9], speech [10] and other ways [11][12] . Explicit interaction has been applied to communication between

Seiji YAMADA
National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda
Tokyo, 101-8430, Japanseiji@nii.ac.jp

a human and a robot, it is however inappropriate forhuman-robot cooperative work in terms of user’s cognitive load because it forces a user to become familiar
with explicit communication protocol.
Meanwhile, implicit interaction has been utilized
for human-robot cooperative work. These are studies for carrying a long object [13] [14] , handling an
object [15] , outdoor cooperative tasks [16]. Unfortunately these studies lack generality as a frameworkfor human-robot cooperation. First, their implicit interaction is restricted in force and physical connection
between a human and a robot. Second, their cooperation has low applicability because it limited between
actions. Cooperation between tasks including actions
has high applicability.
There are no study providing generality and utility
for human-robot cooperation. We therefore propose
an interaction design of IET (Interaction Embedded
in at Task) and apply it to human-robot cooperative
sweeping over a table. This IET leads low cognitive
load of human because it does not need to understandhuman-robot communication protocol, and it needs
no additional actions for a human task. IET thus has
both generality and utility.
After the introduction, in Section 2, we define IET
by embedding implicit commands in a human’s operations to achieve his/her task. In Section 3, weapply IET to design a cooperative sweeping robot
and its interaction with a human. A developing of
a behavior-based robot is described in Section 4. Section 5 describes experiments to investigate efficiency
of cooperative sweeping and compare IET with otherexplicit interaction such as voice or hand manipulationin terms of human’s cognitive load. The experimental
results are indicated in Section 6. Finally, Section 7
concludes this paper.

2Interaction Embedded in a Task

44 cm

The interaction embedded in a task is “the minimum additional interaction (including robot commands) in terms of time and space cost to typical human’s actions for a task.” We call this interaction as
IET, and it has advantages described as follow:

Projected Image
Wall• No additional cognitive load in execution:
Since IET has minimum additional interaction
to typical human’s actions to achieve a task, ahuman does not need additional cognitive loadby explicit communication with a robot andsmoothly does cooperation with a robot by executing only typical actions.
• No understanding a way to communication:
Since explicit communication is not necessary in
IET, a human does not need to understand communication protocol like gesture commands, special speech commands. IET thus releases a humanfrom learning protocol and training to communicate with a robot.

KheperaII

Hand

Figure 1: Experimental environment

encoders. The program written by C-language runson the RAM. An extra infrared sensor which measuresthe distance in vertical direction is added on its general
I/O turret.

3.2

3Interaction Design on Cooperative
Sweeping

We deal with a human-robot cooperative sweeping
between a human and a small mobile robot as a cooperative work. A goal of the cooperative task is to
sweep out a desk including the region of under an object, human’s task is to move the object, and robot’s
task is to sweep out a desk. The hardware resource ofa robot is equal to that of a consumer sweeping robot.
Since a robot cannot move an object by itself, cooperative task of cleaning is achieved when a human anda robot work out their task cooperatively.
In this section we first describe an experimental environment and a specification of small robot.
And then we consider an effective cooperation which
shortens the time of whole environment sweeping on
human-robot cooperative sweeping. The effective cooperative work is achieved through IET.

3.1

Environment and robot

Fig.1 shows an experimental environment where ahuman and a robot work cooperatively. This environment simulates a place where a human uses routinely
such as a desktop. A desk where a robot sweeps out
has flat surface and a wall which encloses the region
for keeping a robot not to fall.
We use a small mobile robot KheperaII. The robot
has eight infrared proximity and ambient light sensors
with up to 100mm range, a processor Motorola 68331(25MHz), 512 Kbytes RAM, 512 Kbytes Flash ROM,and two DC brushed servo motors with incremental

Object33 cm

Cooperative sweeping by IET

Fig.2 shows cooperative sweeping through IET bya human and a robot. We consider this cooperative
work shortens the time of whole environment sweeping, and leads low cognitive load. The procedure is
described as follow:(1) A human picks up an object when a robot approaches an object. Then a robot comes into theregion under the object picked up.
(2) While a robot is in under an object, it keeps turning at the object’s edge.
(3) A robot goes out of the region when a humanapproaches an object to it.
The human’s action has no changes in the trajectory from the typical one, and it has little additionaltime to keep picking up. A human does not need tomove an object to another place. Since the minimumchange of the action trajectory and the minimum costof time, this interaction forms IET. When no commands are embedded in human’s task, a human has
to achieve two tasks at least. He/she needs to controla robot explicitly to work well and to replace an object
as his/her assigned task. On the other, IET can reduce
his/her loads without sending explicit commands.

4

Behavior Design of Mobile Robot

Our robot performs obstacle avoidance, going forward (when no obstacles are on its front) and turning
for random direction. We use sweeping with selecting
directions at random because it has lower cost than aFigure 2: Cooperative sweeping by IET

calculated sweeping using a map. Many methods forregion covering have been developed[17], and most ofthem need precise position of a robot. However, ourrobot can not use reliable methods like dead reckoning. We hence consider that the random sweeping isadequate to our study because the most of consumer
sweeping robots adopt this kind of method.
A robot is implemented by behavior-based approach, and we adopt subsumption architecture[18] tomanage following behaviors.
• Obstacle avoidance.
• Autonomous sweeping while no object is sensed.
• Interactive sweeping while an object is above a
robot.
Fig.3 shows the robot’s behaviors into the threelayers in subsumption architecture. Each layer asynchronously checks the applicability of behaviors andexecutes applicable ones. Higher layers suppress lower
layer’s behaviors, and lower layers have more reactive behaviors. The behaviors of each layer consist ofmultiple actions. When the system obtains multiple
outputs, it generally selects the highest layer’s action.
Each layer has output frequency of action to control
the robot smoothly. We set the frequency as obstacle
avoidance: an action by 5msec, sweeping: 10msec, interaction: 5msec, obstacle avoidance and interaction
occur most frequently.

5

Experiments

We conduct two experiments. In a first experiment, we confirm whether a robot improves cleaning
time by using a behavior of a cooperative sweeping
through IET. In a second experiment, we examine human cognitive load when a human interacts with aLayer 3 : Interaction(Cooperative Behavior)

Layer 2 : Sweeping(Autonomous Behavior)

Sensor Value

Layer 1 : Obstacle AvoidanceAction(Autonomous Behavior)

Figure 3: Subsumption architecture

robot through IET in a cooperative work of cleaning
on a desk. Both experiments are performed on our
experimental system described in the next subsection.

5.1

Experimental system

Fig.4 shows the experimental system which can indicate a robot’s trajectory. This system consists ofa sweeping area and a projection system. In experiments, a human interacts with a robot on the sweeping area in Fig.1 indicating a swept location by the
projection system including a personal computer, aprojector, and a USB camera.
The projection system detects a robot’s location
using a picture of two beams of infrared LEDs equipping on the robot. The robot’s location is calculated
by image processing in the picture from the camera,
and then an image indicating the trajectory is createdwith the location. This image is ultimately projectedon the sweeping area.
The projected image also includes small squarecells. A cell is lit when a robot enters the cell in realtime. These small cells therefore express the trajectory of a robot. The sweeping area having a width of
44 cm and a height of 33 cm is divided into 16 × 12
cells. Cells of 3 × 3 approximately correspond to theBox

Cooperative sweeping
Autonomous sweeping

KheperaII

Average Sweeping Time [sec]Significant difference (alpha=0.05)
p=0.0173 (s=31)

Sweeping Area

PC

Projector

350
300

p=0.1782 (s=48)

p=0.097(s=1.719, df=22)

250
200

150
100
50
0

USB Camera

3x3

4x4

5x5

Figure 5: Result of average time in each size of boxes

Figure 4: Experimental systemarea of a robot.

5.2

Ex.1: Sweeping efficiency

The purpose of Ex.1 is to confirm that the timeof cooperative sweeping is shorter than that of autonomous sweeping. We measure the time until therobot covers the 98% of all cells in three cell sizes ofboxes : 3 × 3, 4 × 4, 5 × 5. A sweeping time ismeasured 3 times in each size of boxes. The initial location of a robot and a box are determined at random
in each measuring. Larger sizes of boxes (such as 6 ×6) are not employed, because these conditions make afree space too small and narrow for a mobile robot tofully sweep there.
In cooperative sweeping, a subject continues to pickup an object until all the cells of under the object areswept, and he puts down it. After that he performs noactions. In autonomous sweeping, on the other hand,a subject relocates the box to the place decided beforehand when the experiment begins. He puts down thebox to the initial location when those cells are swept,
after that he performs no actions.

5.3

Result of Ex.1

Fig.5 shows average times of sweeping in each sizeof boxes, error bars indicating their standard deviation, and differences of each size of boxes between
cooperative sweeping and autonomous one. This experiment was performed by a subject of a late 20s
man. Each pare of average times are tested by ttest or Wilcoxon’s rank sum test. As a result, theTI (Task)

SI (Sound)

HI (Hand)Figure 6: Three types of interactions
4 × 4 box has a significant difference of sweeping time(p = 0.0173, α = 0.05, statistic = 31).

5.4

Ex.2: Human cognitive load

We measure the cognitive load of a human interacting with a robot through IET, and compare theload to that of explicit interaction. These loads areinvestigated by using robots interacting with a humanthrough three types of interaction. These interactions
are shown in Fig.6, and the detail is described as follow:
TI : A robot performs sweeping through IET.
SI : A robot sweeps with receiving a sound commandby hand clapping.
HI : A robot sweeps with receiving a command byhand as blocking the robot’s line.
Each robot performs cooperative sweeping in different interactions. The last two robots receive each explicit command which a human sends besides his/hertask of raising a box. We choose these two methods
for sending command because they need no extra spe-

5.5

Result of Ex.2

Fig.7 shows scores for 12 subjects. They are 8 men
and 4 women between the age of 22 to 32. In the
figure, each subject has three scores: TI, SI, and HI.
A score is the average of normalized number of correct counting answers per second for a subject. The
normalization is to divide the correct answers per second by correct answers per second without operationsof a robot. Therefore, 1.0 means counting ability ofeach subject without operations of a robot. The figure
also includes averages of TI , SI, and HI. TI has thehighest average. Fig.8 shows averaged scores, standard deviations and differences tested by Dunnett’s
test. The difference between TI and HI has a significant difference (p << 0.01, α = 0.01, t = 3.938). The
difference between TI and SI also has a significant difference (p = 0.033, α = 0.05, t = 2.414).

6

Discussion

The result of Ex.1 shows that the cooperative
sweeping made the whole area sweeping efficient, andit depends on proportion of the size of box and thesize of the whole sweeping area. This effect however isavailable only for the random sweeping strategy, and1.2
average and standard deviation

1.0

score

0.8
0.6
0.4
0.2

0

HI

SI

TI

Figure 7: Result of score in each subject
Signiticant Difference (alpha=0.01)
p << 0.01 (t=3.938)
Signiticant Difference (alpha=0.05)
p = 0.033 (t=2.414)
1.0

averaged scorecial devices such as a remote controller, and a humancan create a command by only his/her body.
The experimental system is the same as Ex.1, andthe 4 × 4 box is selected because it has a significant
difference in the result of Ex.1. One of the experiments
runs until the all of 16 cells are swept. In the TI,
subjects pick up an object until all the cells of underthe box are swept. In the SI and HI, first, they relocate
a box to a corner of the sweeping area, and then send
a command for a robot to be turn by making sounds
or approaching their hand to it.
We use a dual task method to measure human cognitive load. Subjects have to do mental arithmetic asa second task while controlling the robot as a maintask[19][20]. They count backwards by three from arandomly selected three-digit number vocally. We obtain the average number of correct answers per second,and evaluate the human’s cognitive load for controlling each robot. Subjects are required to calculate as
quickly as possible, and to prioritize the controllinga robot over the counting. They practice controlling
robots and the counting well before experiments begin.
The order of TI, SI and HI for each subject is determined at random, and these experiments are recorded
three times respectively for a subject. Subjects arealso measured counting ability without operations ofa robot before a measuring of TI, SI, or HI respectively. The counting ability is the number of correct
answers of the counting for 30 seconds.

0.8
0.6
0.4
0.2
0

TI

SI

HI

Figure 8: Result of scores and differences

can be applicable not only in the case of using the IET,
but in the case of using the explicit commanding interactions. We conform that such cooperation shorten
the sweeping time significantly and how to design aninteraction for drawing human’s operations toward cooperative sweeping. Our IET plays an important roleon this issue.
The result of Ex.2 shows that the IET minimizes
human cognitive load in comparison with other explicit commanding interactions. An additional task ofsending command has high cognitive load in explicit
interaction and we consider that physical loads affectthe counting task as well. On the other hand, IET
has low cognitive load because of minimizing cost ofsending a command and also shortening the trajectoryof moving a box. Therefore, IET plays the significant
role in human-robot cooperative work.

7

Conclusion

We design an interaction for reducing human cognitive load, which is an IET (Interaction Embedded
in a Task) for human-robot cooperative work, and theinteraction includes an embedded command within ahuman’s action for achieving his/her task. This interaction enables a human to easily cooperate with a mobile robot without additional and explicit commands.
A real small robot’s behaviors are implemented to
sweep cooperatively with a human through IET. We
conduct two experiments. In a first experiment, we
confirm whether a robot improves cleaning time by using a behavior of a cooperative sweeping through IET.
In a second experiment, we examine human cognitive
load when a human interacts with a robot throughIET in a cooperative work of cleaning on a desk. The
first experimental result showed that the cooperative
sweeping made the whole area sweeping efficient, andthe second experimental result showed that IET minimized human cognitive load in comparison with otherexplicit commanding interactions. These results showthat a robot with poor hardware resource can work
cooperatively with a human, and IET reduce humancognitive load in a human-robot cooperative work.

References[1] G. S. Hornby, S. Takamura, J. Yokono, O. Hanagata, T. Yamamoto, and M. Fujita, “Evolving robust
gaits with aibo,” in IEEE International Conference on
Robotics and Automation(ICRA‘00), 2000, pp. 3040–
3045.
[2] T. Ono and M. Imai, “Reading a robot’s mind: Amodel of utterance understanding based on the theoryof mind mechanism,” Proceedings of Seventeenth National Conference on Artificial Intelligence, pp. 142–
148, 2000.
[3] J. T. Butler and A. Agah, “Psychological effects of
behavior patterns of a mobile personal robot,” Autonomous Robots, vol. 10, no. 2, pp. 185–202, 2001.
[4] K.
Severinson-Eklundh,A.
Green,
andH. Hüttenrauch, “Social and collaborative aspects of interaction with a service robot,” Robotics
and Autonomous Systems, vol. 42, pp. 223–234, 2003.
[5] T. W. Fong, I. Nourbakhsh, and K. Dautenhahn, “A
survey of socially interactive robots,” Robotics andAutonomous Systems, vol. 42, no. 3–4, pp. 143–166,
2003.
[6] D. Katagami and S. Yamada, “Active teachingfor an interactive learning robot,” in Proc. IEEE
Workshop Robot and Human Interactive Communication(ROMAN‘03), 2003.
[7] J. Triesch and C. von der Malsburg, “Robotic gesturerecognition,” in Proceedings of the Bielefeld GestureWorkshop, 1997, pp. 233–244.

[8] S. Waldherr, R. Romero, and S. Thrun, “A gesture
based interface for human-robot interaction,” Autonomous Robots, vol. 9, no. 2, pp. 151–173, 2000.
[9] F. Marrone and M. Strobel, “Cleaningassistant - aservice robot designed for cleaning tasks,” in Proc.
Advanced Mechatronic Systems (AIM‘01), 2001.
[10] S. Kajikawa, S. Hiratsuka, T. Ishihara, and H. Inooka,“Robot position control via voice instruction including ambiguous expressions of degree,” in IEEE Int.
Workshop on Robot and Human Interactive Communication(ROMAN‘03), 2003.
[11] F. Kaplan, P. Oudeyer, E. Kubinyi, and A. Miklosi,“Robotic clicker training,” Robotics and Autonomous
Systems, vol. 38, no. 3-4, pp. 197–206, 2002.
[12] A. M. Khamis, F. J. Rodrı́guez, and M. A.
Salichs, “Remote interaction with mobile robots,” Autonomous Robot, vol. 15, no. 3, 2003.
[13] Y. Hayashibara, Y. Sonoda, T. Takubo, H. Arai,
and K. Tanie, “Assist system for carrying a long object with a human – analysis of a human cooperative behavior in the vertical direction –,” in Proc.
1999 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS‘99), 1999.
[14] H. Arai, T. Takubo, Y. Hayashibara, and K. Tanie,“Human-robot cooperative manipulation using a virtual nonholonomic constraint,” in Proc. 2000 IEEE
International Conference on Robotics and Automation (ICRA‘00), 2000.
[15] K. Nakai, K. Kosuge, and Y. Hirata, “Control of robot
in singular configurations for human-robot coordination,” in IEEE Int. Workshop on Robot and Human
Interactive Communication(ROMAN‘02), 2002, pp.
356–361.
[16] K. Yokoyama, J. Maeda, T. Isozumi, and K. Kaneko,“Application of humanoid robots for cooperative
tasks in the outdoors,,” in IEEE/RSJ IROS Workshop on Explorations towards Humanoid Robot Applications, 2001.
[17] H. Choset, “Coverage for robotics - a survey of recent
results,” Annals of Mathematics and Artificial Intelligence, vol. 31, pp. 113–126, 2001.
[18] R. A. Brooks, “A robust layered control system fora mobile robot,” IEEE Journal of Robotics and Automation, vol. 2, no. 1, pp. 14–23, 1986.
[19] J. W. Crandall and M. A. Goodrich, “Characterizing
efficiency of human-robot interaction: A case study ofshared-control teleoperation,” in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS‘02), 2002.
[20] J. E. Condron and K. D. Hill, “Reliability and validity of a dual-task force platform assessment of balance
performance : Effect of age, balance impairment, andcognitive task,” Journal of American Geriatrics Society, vol. 50, pp. 157–162, 2002.