Generated Image Feature based Selective Attention Mechanism
by Visuo-Motor Learning
Takashi Minato

Minoru Asada

Dept. of Adaptive Machine Systems, Graduate School of Engineering, Osaka University
2-1 Yamada-oka Suita Osaka 565-0871 Japan
Email: {minato,asada}@ams.eng.osaka-u.ac.jp
TEL&FAX: +81-6-6879-4180
Visual attention is an essential mechanism of an intelligent robot to avoid processing an enormous
amount of data. Existing research typically specifies in advance the attention control scheme required fora given robot to perform a specific task. However, a robot should be able to adapt its own attention control
for varied tasks and environments. In our previous work, we proposed a method for a mobile robot togenerate a filter to extract an image feature by visuo-motor learning. The generated image feature extractor
is considered to be generalized knowledge from which a kind of image feature should be extracted forthe robot to accomplish a task of a certain class. In this paper, we propose an attention mechanism,
by which the robot selects the generated feature extractors based on its task-oriented criterion. A subset
of supervised data which gives the local information of the task makes the selective mechanism more
effective. We discuss the results of applying the method to indoor navigation and soccer shooting tasks.
Key Words: Automation, Mobile Robot, Selective attention, Image feature generation, Image feature
selection, Task-oriented
1. Introduction
Attention control is an essential mechanism for an
intelligent robot to avoid processing an enormous amountof data. It is a data reduction process to facilitate decision
making.With regard to visual attention control, it involves
selection of viewpoint, focus, image features, and so on.
Existing research typically specifies in advance the attention control scheme required for a given robot to perform a
specific task. However, a robot should be able to adapt its
own attention control for varied tasks and environments.
Human beings have very highly developed mechanisms of attention. Much research focused on early visual
processing and proposed a computational model in which
a bottom-up system computes low-level image featuresand saliency maps and a top-down system selects thesalient parts (e.g., [1]). Some computer vision researchers
proposed a viewpoint selection method to facilitate object
recognition based on information gain (e.g., [2]). The
mechanisms are intended to obtain a better observationfor object recognition, but are not directly related to thephysical actions needed to accomplish a given task.
Some robot researcher focused on the attention problem of robot vision. Vlassis et al. [3] extracted image
features correlated with a mobile robot’s self-localizationfrom the observed images based on a probabilistic method.
Kröse and Bunschoten [4] proposed a method to decide therobot’s camera direction by maximizing information gain.
Winters and Santos-Victor [5] proposed a method to extract
pixels correlated with a robot’s localization. These methods
are considered to be task-relevant visual attention but are
not related to any physical actions.

We have focused on visual attention control related
to a robot actions to accomplish a given task and proposeda method in which a robot generates an image feature
extractor (i.e., image filter) which is needed for the selection of actions through visuo-motor map learning [6].
The robot’s learning depends on the experience gathered
while performing a task. The robot’s state is calculated
in two stages. First the image feature is extracted fromthe local area of the observed image, and then the stateis calculated from the entire area of the feature image.
Consequently, a generalized feature extractor is generated
because it works much like a bottleneck layer of neuralnetwork in the state calculation process. In this model, therobot uses only one feature extractor for a given task. It
is, however, obvious that the robot needs to select and use
multiple feature extractors properly to accomplish various
tasks.
A number of connectionist models have been proposed that constitute systems that selectively respond tovisual stimuli. Scheier and Egner [7] proposed a systemthat selectively connects image feature maps with robot’s
actions according to saliency. The connection is given apriori and has less adaptability. A simulated nervous system
was proposed that learns the connection based on the cooccurrence between sensor activities of NOMAD [8]. In
this system, the robot follows its innate preference and,
therefore, cannot learn any given task.
Some research has addressed a method of feature
selection based on task-relevant criteria. McCallum [9]proposed a method in which a robot learns not only its
action but feature selection using reinforcement learning.

Observed image Filtered image
Io
If
averagingfiltering

Reduced image
Ic

State

s

Mapping matrix

W

reduction

Actions1
s2

filter F.
.
.

nx x ny

sigmoid functiong(x)nx x ny

image feature extraction

Fig. 1.

nx > ncx
ny > ncy

ncx x ncyasm

Observed
image
Io

Filtered
imagesReducedimagesIf1

Ic1

Mitsunaga and Asada [10] proposed a method to selecta landmark according to the information gain on action
selection. In these methods, however, the image features
to detect the landmarks from the observed image is givena priori. It is desirable that the image feature adapts toenvironmental changes.
This paper proposes a method in which a robot
learns to select image feature extractors generated byitself according to a task-relevant criterion. The generated
feature extractors are not always suitable for new tasks,
though they are generalized. The robot must learn to selectthem to accomplish the task. The criterion of selection is
the information gain calculated from given task instances(supervised data). Furthermore, a part of supervised data
which gives the local information of the task makes theselective mechanism more effective. The method is applied
to indoor navigation and soccer shooting tasks.
2. The basic idea
Image feature extraction is necessary in a robot’svisual attention and as well as in human visual processing,consists of a low-level feature extraction process and ahigh-level recognition process. In the proposed method, arobot generates an image feature extractor that is necessary
for the action selection through visuo-motor map learning[6] as shown in Fig. 1. The state calculation process isdecomposed into feature extraction and state extraction.
A robot learns the effective feature extractor and statemapping matrix for a given task through a mapping fromobserved images to supervised actions. During feature
extraction, the interactions between raw data are limited
to local areas, while the connections between the filteredimage and the state spread over the entire space to represent
non-local interactions. We, therefore, expect that the feature
extractors are more general.
The robot calculates the filtered image If from theobserved image Io using the feature extractor F . To avoid
the curse of dimensionality, the size of If is reduced to Ic .
The state s is calculated from Ic by the sum of weighted
pixel values (the weight matrix is W ). The robot decides
the appropriate action for the current state s. The function
model of the feature extractor is given, and the robot learns
its parameters and the mapping matrix by maximizing theinformation gain of s with respect to action a.

W1

F1

s1
State

F2

state vector extraction

Image feature generation model

Substates

Fn

Fig. 2.

If2

Ic2

..
.

..
.

Ifn

Icn

W2

s2

..
.
Wn

[]
s1
s2

sE = ..

.
sn

Actionasn

Image feature selection model

The robot, which generates one feature extractor fora given task, obviously needs multiple feature extractors
for more complex tasks. It is unnecessary to learn afeature extractor for every given task. The generated feature
extractor must be generalized to make the robot more
adaptable.
In the proposed method, the robot reuses a numberof generated feature extractors from past experiences andselects effective ones for action decision. The system isshown in Fig. 2. The robot is given a number of different feature extractors, but must select which extractors
are effective for given task. The robot, therefore, learns
the state mapping matrix using the supervised data andevaluates which feature extractor is appropriate from the
distribution of supervised data on the learned state space.
If the robot uses all of the supervised data in the evaluation,
optimality in a local part of the task is lost. To evaluate theeffectiveness in the local task, the robot estimates which
local task it is performing from the history of observations
and selects the feature extractor using a portion of thesupervised data corresponding to the local task.
3. Selective attention mechanism based on generated image feature extractors
3.1 The system overview
The robot is given n different feature extractors(Fi , i = 1 . . . n) and calculates the substate si using the
mapping matrix Wi corresponding to Fi . Each mapping
matrix is learned by maximizing the information gain of sE(direct product of s1 . . . sn ) with respect to the supervised
action a.
The robot selects the feature extractor which has
a maximum expected information gain and decides theappropriate action for the substate calculated using theselected feature extractor. It cannot always decide theappropriate action using one feature extractor. It, therefore,
estimates the reliability of selected feature extractors andselects repeatedly until the reliability exceeds a given
threshold.
For evaluation in the local task, the supervised data
is segmented in temporal order. The robot selects a subsupervised data according to the history of observationand selects feature extractors to decide an action using theselected one.

In this paper, three feature extractor models are used.
9• 3 × 3 spatial filter Fs ，the parameter f s ∈ < :•(2)Color filter Fc ，the parameter f c ∈ <3 :I¯x,y = fc1 Ir x,y + fc2 Ig x,y + fc3 Ib x,y ,¡¢If x,y = g I¯x,y ,

•

(1)

5
X

fmi It−i+1 x,y ,

(4)

(5)

i=1

¡
¢
If x,y = g I¯x,y ,(6)

where x and y denote the position of the pixel, I, Ir , Ig , Ib ;the gray, red, green and blue components of the observed
image, respectively, It , the gray component of the observed
image at time t, and g(·), a sigmoid function.
3.3 State learning
First the robot collects supervised successful instances of the given task for T episodes. An episode ends
when the robot accomplishes the task. An instance consistsof the observed image Io and a given action a ∈ <l . An
instance of ith episode at time t is shown as following:iuit =< Iot, ait > .

(7)

Next the robot learns the mapping matrices. The
substate si ∈ <m is calculated as following:sj = g (Wj icj ) ,g(x) = (g(x1 ), . . . , g(xm ))T ,(8)

where icj ∈ <mcx ncy denotes the vector of the jth Icj , andWj ∈ <m×ncx ncy is the jth mapping matrix.
The evaluation function used to learn Wj is tomaximize the information gain of sE with respect to a.
It is equivalent to minimizing the following risk function
R (e.g., [3]).
N

R=−

1 X
log p(ai |sEi ),N iepisode T

Ur

U1

sE

Fig. 3.

Segmentation of supervised data

Using the gradient method, the mapping matrices Wj ,which minimize R, are obtained.
∂R
Wj ← Wj − α
,
(10)
∂Wj
where α is a step size parameter.

(3)

Temporal filter Fm ，the parameter f m ∈ <5 :I¯x,y =

episode 1

end

start

3.2 The model of feature extractor

I¯x,y = fs1 Ix−1,y−1 +fs2 Ix,y−1 +fs3 Ix+1,y−1+fs4 Ix−1,y +fs5 Ix,y +fs6 Ix+1,y+fs7 Ix−1,y+1 +fs8 Ix,y+1 +fs9 Ix+1,y+1 ,¡¢If x,y = g I¯x,y .

U2

(9)

where N denotes the number of instances. The probability
density functions are represented by kernel smoothing.

3.4 Feature extractor selection
All instances of U are divided into r subsets Uj , j =1 . . . r before performing the task (Fig. 3). The subsets arearranged in time order. The choice of r includes a tradeoff between a locality of the evaluation and a reliabilityof the action decision. To evaluate it, U is divided sothat instances of similar state and action are included in a
subset. The following vector cit is defined from the instanceuit , and U is divided by applying the ISODATA algorithm
for the set {cit }.
µ
¶
i
i
i t
ct = sEt , at ,
,
(11)
Li
where Li is the time taken to accomplish the task. The
value of each component is normalized to [0,1]. To avoid
an aliasing problem, the robot always uses the two neighboring subsets to evaluate the effectiveness of a feature
extractor.
The robot executes the following process at every
interval.
1) Selecting subsets of instance;
Select subsets of instance U according to a procedureshown in the next section. k = 0.
2) Calculating a reliability of action decision;
Calculate substates so1 , . . . , sok corresponding to theselected k(≥ 0) feature extractors and the entropy of
the action HU (a|So ) using the instances in U.
X
HU (a|So ) = −PU (a|So ) log PU (a|So ), (12)where So = {so1 , . . . , sok }. HU (a|So ) means anuncertainty of the action decision. Evaluate the uncertainty using a threshold Hth .
• If HU (a|So ) ≤ Hth , then go to 4.
• Otherwise, if k = n and U = U , then go to 4.
• Otherwise, if k = n and U 6= U , then go to 1
with U = U .
• Otherwise, go to 3.
3) Selecting a feature extractor;

Observed image

Calculate an expected entropy of the action foreach unselected feature extractor Fu . The expected
entropy is:X
PU (su )HU (a|So , su ),
(13)

red

blue

Spatial filter Fs

Color filter Fc1
white

U

white red Color filter Fc2

Temporal filter Fm

Fig. 4.

Feature extractors
goal

(14)
sign

The robot selects subsets of instance U which areused to calculate a probability and an entropy according to
the states Sot−1 , . . . , Sot−h observed in the past h steps.
Each Sot consists of a number of substates. The size of
Sot is differs depending on t, because the number of theselected feature extractors is different.
For each subset Ui the robot calculates a matchingratio, Pbi , that Sot satisfies Eq. 15 in h substates. If the ratio
is greater than a threshold Pbth , Ui and Ui+1 are added to
U . Ui+1 is the successor of Ui . If there is no Pbi which isgreater than Pbth , the robot uses all instances (U = U ).
PUi (Sot ) > 0.

yellow

green

where su is a substate corresponding to Fu . Selectthe feature extractor which has the minimum entropy,
that is, has the maximum information gain. k ← k +1. go to 2.
4) Deciding an action;
Execute the following action a:
P (a0 |So ).
a = arg max
a0 U
3.5 Selecting subsets of instance

gray

white

Filtered image(15)

obstacle

robot
robot

Task A

Task B
goal

sign
robot

ball

robot

Task CTask D

Fig. 5. Tasks in which the robot learned feature extractors

4. Experiment
4.1 Experimental setting
We use a small mobile robot which is about 40 cm
high and has a camera with a fixed orientation to lookahead at the floor. The task is to move along a given path
to a destination. The size of Io and If in pixels is 64 × 54
and that of Ic is 8 × 6. The robot is controlled at the rateof 15 Hz. Each pixel value of Ic is the average value ofthe corresponding region in If . We defined the dimensionof substate as m = 1.
The robot can move at a translational speed v anda steering speed ω independently, so the action vector isrepresented as follows:Ta = (v, ω) .

(16)

To reduce the computation cost, we discretized thestate and action space and calculated the probabilities. The
thresholds are set as Hth = 0.4 and Pbth = 0.8.
4.2 Feature extractors
We prepared four feature extractors shown in Fig.
4. Fs , Fc1 , Fc2 , Fm are generated in tasks A, B, C, and D(Fig. 5), respectively. The environment of task A, B, andC is a corridor in a laboratory and that of task D is a fieldof robot soccer.
The feature extractors have the following characteristics.
• Fs : Emphasizing and inhibiting horizontal edge
• Fc1 : Inhibiting red••

Fc2 : Emphasizing red and yellow, and inhibiting white
Fm : Emphasizing current image and inhibiting past
image

4.3 Feature extractor selection
The robot was given Fs , Fc1 , Fm and learned task 1
shown in Fig. 6. The robot moves to the front of the door
and waits for it to open. It moves to the destination after
the door opens. The environment is same as that of tasks A,
B, and C. We gave three episodes of successful instances(L = 234, 254, 233). After learning, the robot divided all
instances into 13 subsets. We set the history length h = 10.
Fig. 7 shows the learned behavior and Fig. 8 shows
the selected feature extractors at each time step. The
selected feature extractors, their number, and their orderchange according to the situation. The average number per
step is 1.57.
Fig. 9 shows the selected subsets of instances at
each step. When the robot cannot choose an action from

goal

robot

wait for open
signFig. 6.

Task 1

step0

30

60

90

120
white

red

goal
gray

keeper robot

150

180

Fig. 7.

210

240

244

ball

white
&
black

Resultant behavior (Task 1)
robot

Fs
Fc1
Fm

1st
2nd
3rd

blue

(a) Task 2(b) Resultant behavior

Fig. 11.
0

50

100

step

150

200

250

Fig. 8. Selected feature extractors with subsets of instance(Task 1)

Fs
Fc2
Fm

1st
2nd
3rd
0

the selected subsets because of low reliability, it uses all
instances to decide again. U in the figure shows the stepwhen the robot can choose an action from the selected
subsets. If the robot chooses an action with few feature
extractors in the past, many subsets of instance are usedbecause the number of subset satisfies Eq.15 increases.
We verify that the robot accomplishes the task selecting
effective feature extractors.
4.4 Verification of subset of instanceTo verify the subset of instances, we performed anexperiment which was the same as Sec. 4.3 except theprocedure to select the subsets. In this experiment, the robot

using

8
4
0

50

Fig. 9.

100

step

150

200

250

Selected subsets of instance (Task 1)
Fs
Fc1
Fm

1st
2nd
3rd
0

Fig. 10.
(Task 1)20

Fig. 12.

40

60

80

step

100

120

140

160

Selected feature extractors (Task 2)

always selects all instances. Fig. 10 shows the selectedfeature extractors at each step. The feature extractor thatthe robot selects first does not change. In this result, Fc1 isalways selected at first. The average number of the selectedfeature extractors per step is 1.97, which is larger than the
result of Sec.4.3. This means that using all instances brings
about inefficient selection for the action decision. Hence,
the robot effectively decides the action using part of theinstances.
4.5 Reuse of feature extractors

u
12

Ui

Task 2 and resultant behavior

50

100

step

150

200

250

Selected feature extractors with all instances

In the above, it has been shown that the robot canaccomplish new tasks with feature extractors generated in
the past tasks. This section shows that a generated feature
extractor can be reused in new environment.
The robot was given Fs , Fc2 , Fm , and learned task
2 shown in Fig. 11 (a). The environment was same asthat of task A, B, and C, however, Fc2 was generated
in the different environment. We gave three episodes ofsuccessful instances (L = 132, 131, 134). The number ofthe subset was 7 and the history length, h, was 15.
Fig.11 (b) shows the learned behavior and Fig. 12shows the selected feature extractors at each time step. The
robot selected Fc2 at first in some situations. This indicatesthat Fc2 is effective for this task. Hence, the generated
feature extractor can be used in different environment andtask.
4.6 Irrelevant feature extractor
Contrary to the previous section, this section showsthat the robot can neglect the irrelevant feature extractor
to the task. The robot was given Fs , Fc1 , Fc2 , and learned

graykeeper robot

1

1

s3

white

s2

ball

robot

blue

1

(a) Task 3

Fig. 13.

(b) Resultant behavior

0

s1

1

(a) Perspective view

Fs
Fc2
Fc1

1st
2nd
3rd
20

Fig. 14.

0s11(b) Projection ontos1 –s2 plane

Task 3 and resultant behavior
Fig. 15.

0

s2

40

60

80

step

100

120

140

Selected feature extractors (Task 3)

task 3 shown in Fig. 13 (a). The environment was sameas that of task 3 except the goal. We gave two episodes ofsuccessful instances (L = 143, 153). The number of subset
was 6 and history length, h, was 15.
Fig. 13 (b) shows the learned behavior and Fig. 14
shows the selected feature extractors at each time step.
The importance of Fc2 is low because the robot rarely
selected it. It can also be seen from the distribution of
the instances. Fig. 15 (a) shows the distribution of all
instances on the learned state space. s1 , s2 , s3 are thesubstates of Fs , Fc2 , Fc1 , respectively. Fig.15 (b) shows
the s1 –s2 plane. The value of state s2 is irrelevant to
identifying the state because the instances do not distributealong the axis of s2 . This means that Fc2 is irrelevant tothe action decision. Fc2 emphasizes red and yellow, andinhibits white. It is, however, useless to identify the state in
the environment. Hence, the robot can neglect an irrelevant
feature extractor.
5. Conclusion
This paper has proposed a method in which a robot
learns to select image feature extractors generated by itself
according to task-relevant criterion. A portion of supervised
data which gives the local information of the task makes
the selection of feature extractors more effective.
In the proposed method, a robot can accomplish
more complicated tasks using multiple feature extractors.
This paper, however, does not mention a method to generalize a feature extractor. This must be considered in order
to increase the robot’s adaptability.
References[1] J. M. Wolfe, K. R. Cave, and S. L. Franzel. Guided
search: An alternative to the feature integration

Distribution of supervised data

model. Journal of Experimental Psychology: Human
Perception and Performance, 15(3):419–433, 1989.
[2] T. Arbel and F. P. Ferrie. Viewpoint selection by
navigation through entropy maps. In Proceedings of
the Seventh International Conference on Computer
Vision, pages 248–254, 1999.
[3] N. Vlassis, R. Bunschoten, and B. Kröse. Learningtask-relevant features from robot data. In Proceedings of the 2001 IEEE International Conference on
Robotics and Automation, pages 499–504, 2001.
[4] B. J. A. Kröse and R. Bunschoten. Probabilistic
localization by appearance models and active vision.
In Proceedings of the 1999 IEEE International Conference on Robotics and Automation, pages 2255–
2260, 1999.
[5] N. Winters and J. Santos-Victor. Visual attentionbased robot navigation using information sampling.
In Proceedings of IEEE/RSJ International Conferenceon Intelligent Robots and Systems, pages 1670–1675,
2001.
[6] T. Minato and M. Asada. Towards selective attention:
Generating image features by learning a visuo-motor
map. Robotics and Autonomous Systems, 45:211–221,
2003.
[7] C. Scheier and S. Egner. visual attention in a mobile
robot. In Proceedings of the International Symposiumon Industrial Electronics, pages 48–53, 1997.
[8] J. L. Krichmar and J. A. Snook. A neural approachto adaptive behavior and multi-sensor action selection
in a mobile device. In Proceedings of the IEEE International Conference on Robotics and Automation,
pages 3864–3869, 2002.
[9] A. K. McCallum. Learning to use selective attention
and short-term memory in sequential tasks. In Proceedings of the foruth International Conference on
Simulation of Adaptive Behaivior: From Animals to
Animats 4, pages 315–324, 1996.
[10] N. Mitsunaga and M. Asada. Observation strategy
for decision making based on information criterion.
In Proceedings of the 2000 IEEE/RSJ International
Conference on Intelligent Robots and Systems, pages
1038–1043, 2000.