Object tracking system by integrating
multi-sensored data
Kouji Murakami

Tokuo Tsuji

Tsutomu Hasegawa

Faculty of Engineering,
Institute of Science and Engineering, National Institute of
Kyushu Sangyo University,
Kanazawa University,
Technology,
Fukuoka, Japan
Ishikawa, Japan
Kumamoto College,
Kumamoto, Japan
Abstract—We propose an object tracking system which recognizes everyday objects and estimates their positions by using
distributed sensors in a room and mobile robots. The placementof objects is frequently changed according to human activities.
Although a passive RFID tag is attached to each object for theobject’s recognition, the placement is often not uniquely determined due to the deficiency of measured data. We have alreadyproposed a method for estimating the placement of objects byusing the moving trajectories of objects. This estimation result is
expressed as the probability distribution of the object placement.
However intersections of trajectories cause the decrease of theestimation accuracy. So we propose a new method based on
Bayesian inference to improve the estimation accuracy by using
the size and the shape of an object measured by laser range finder.
Then a mobile robot settles the placement with small workload
by using the mounted sensor. The system successfully recognized
and localized 10 objects in the experiment.

I.

type, and is larger than an usual thin RFID tag. The positionsof residents walking in a room have been measured by usingpressure sensors distributed on the floor [5]. However, theweights of everyday objects are sometimes too light to detect
it. Multiple vision sensors are used for the object tracking [6].
However, a resident will feel invasion of privacy under constantsurveillance by vision sensors installed in a personal space.
We have already proposed a object tracking system thatrecognizes everyday objects and estimates their positions in aroom by using distributed sensors and mobile robots [7]. The
system integrates information measured by distributed sensors
for the object’s recognition and position estimation. However,the deficiency of measured data causes the decrease of theestimation accuracy. In this paper, we improve the system
by using the size and the shape of an object as additional
information in the data integration process.

I NTRODUCTION

II. O BJECT T RACKING
We propose an object tracking system which recognizes
everyday objects and estimates their positions by using distributed sensors. Assumptions for this system are as follows:

It is expected that a service robot supports the daily needsof elderly people in a home. An object fetching task is one ofprospective applications for a service robot. If a service robot
can bring a requested everyday object, it will make people’s
life comfortable. A service robot needs the placement ofobjects in a room to perform this task. Many position tracking
or estimation methods have been presented. However, most ofthem use vision sensors and focus on the human tracking. The
sizes of most everyday objects are small. Everyday objects are
frequently held, carried and housed by a resident. In addition,
everyday objects may be put into hidden spaces such as a
pocket or a box. So, a conventional vision system for the
position tracking will not be straightly applicable to this object
tracking.

1)
2)

A passive RFID tag is attached to each object. An
object is uniquely identified by its tag ID.
Sensor systems installed in an environment measure
either both the position of an object and its tag ID or
only the position of an object.

Each measured position by a sensor system is labeled to
track it continuously. We call this label tracking-label. The
result of the object’s recognition and position estimation is
obtained as a combination of the tag ID of an object and its
position. We call a set of this combination ”object placement”.
Fig.1 shows the object placement of five object. This object
placement has five combinations of a tag ID and a ”trackinglabel” as the position of an object. If a sensor system canmeasure both the position of an object and its tag ID, a
combination of the tag ID and the position of an object
is obtained as a measurement result. On the other hand, asensor system can measure only the position of an object, acombination of the tag ID and the position of an object is
not obtained as a measurement result. The proposed systemestimates the tag ID related to the measured position byusing the SIR particle filter [8]. The SIR(sampling importanceresampling) particle filter integrates sensored information forthis estimation.

Some object tracking systems have been reported. A
RFID(Radio Frequency IDentifier) tag [1] is often used for the
position estimation of objects. A container with a RFID reader
can detect whether or not an object with a RFID tag is in it[2]. However, the position of an object with a RFID tag is notmeasurable when the object is out of the container. The signal
strength of an active tag is used for the position estimation[3]. The presented method estimates 3-dimensional positionof an object with an UHF(Ultra High Frequency) RFID tag.
However, an active tag is box-type and needs a battery. An
ultrasonic tag is used for the position estimation [4]. The
position of an ultrasonic tag is estimated by using ultrasonicreceivers installed on the ceiling. An ultrasonic tag is attached
to a target object. However, an usual ultrasonic tag is box978-1-5090-3474-1/16/$31.00 ©2016 IEEE

Ryo Kurazume
Faculty of Information Scienceand Electrical Engineering,
Kyushu University,
Fukuoka, Japan

0The object placement changes according to human activities. In the estimation process of the object placement, thesystem needs the transition model of the object placement toreflect the change of the object placement caused by humanactivities in the estimation process.
Our previous system [7] uses only the moving trajectories
of objects for this transition model. However intersections oftrajectories cause the decrease of the estimation accuracy. In
this paper, we improve the transition model by adding the sizeand shape of an object in order to raise the estimation accuracy.

Fig. 1.

(a) three bottles

Fig. 2. Position estimation and recognition of three bottles in a sensoredcabinet.

The contour of an object on the floor is measured as apoint cloud (Fig.4). A strip of mirror placed on a wall reflects
the scanning beam of a LRF. These reflected scanning beams
increase the number of scanable points on the surface of an
object. The measurement result for four objects on the floor isshown in Fig.5.

Object placement: a combination of tag ID and tracking-label.

The estimation accuracy of the object placement is correlated with the number of objects whose tag IDs are unmeasurable by sensors distributed in an environment. A mobile
robot with a RFID reader can move to the position of eachunmeasurable object and measure its tag ID. The result of thisactive sensing for one unmeasurable tag ID is integrated in
the estimation process and improves the estimation accuracyof the object placement. We propose a method for choosing
a prospective position of unmeasurable object to improve theestimation accuracy.
III.

(b) top view

LRF

mirrorLRF

Fig. 3.

The sensored floor.

Fig. 4.

A point cloud of an object measured by the sensored floor.

O BJECT T RACKING S YSTEM

A. System components
Several types of sensors are installed to a cabinet, thefloor, and a mobile robot. The proposed system collects three
types of measured information from each sensored component.
These information is integrated to estimate the object placement.
1) Measured information from a sensored cabinet: We
installed a RFID reader and load cells to a cabinet. Thissensored cabinet performs the object’s recognition and position
estimation. Fig.2 shows the measurement result for threebottles. A sensored cabinet outputs combinations of the tag
IDs and the positions of the objects stored in it. Each position
is related to tracking-label. Each tracking-label is uniquely
generated when an object get put into a sensored cabinet. A
sensored cabinet i outputs information Z i as a measurement
result.
(1)
Z i = {obj1 , obj2 , ... , objS }obj = {tracking − label, tagID}

LRF

Fig. 5.

Position measurement of four objects by using the sensored floor.

A sensored floor outputs combinations of the positions andarticles of the objects on the floor. Each position is related to
tracking-label. Each tracking-label is uniquely generated whenan object get placed onto the floor. The sensored floor outputs
information Y as a measurement result.

(2)

The set of output information of all sensored cabinets atthe time t is expressed as Zt = {Zti }.
2) Measured information from a sensored floor: We installed a laser range finder (LRF) and a mirror to the floor(Fig.3). This sensored floor performs the position estimationof objects and walkers on the floor [9]. A LRF can measure
the distances between the LRF and objects on the floor. The
positions of objects and walkers are continuously measured byusing the background subtraction.

Y = {y1 , y2 , ... , yM }(3)

y = {tracking − label, article}(4)

article = {person or object}(5)

Output information of the sensored floor at the time t isexpressed as Yt = {Yt }.
13) Measured information from a mobile robot with RFID
readers: A robot with a RFID reader performs the object’s
recognition and position estimation. This robot outputs combinations of the tag IDs and the positions of objects within thesensing are of the robot. Each position is related to trackinglabel. A robot i outputs information Ai as a measurement
result.
Ai = {obj1 , obj2 , ... , objN }(6)
obj = {tracking − label, tagID}

Transition operation(I): When the total number oftracking-labels increases, the operator replaces the
tracking-label of a person with the newly appeared(r)tracking-label in vector xt−1 of each particle. This
operation indicates the action (i) mentioned above.
Transition operation(II): When the total number oftracking-labels decreases, the operator replaces thedisappered tracking-label with the tracking-label of a(r)
person in vector xt−1 of each particle. This operation
indicates the action (ii) mentioned above.
Apply these transition operations to R(1 − K) parti(r)cles rt−1 chosen from the R particles. The constantvalue K (0 < K < 1) depends on the reliability ofeach sensor system installed in an environment.
In the transition operations (I) and (II), if several
persons are in a room, an operator needs to determine
who has caused the action (i) or (ii). An operator
chooses one of them by considering the probability
which depends on the distance between the positionof each person and the position of the tracking-label
newly generated or removed.
In the transition operation (I), if a person carriessome objects, their tag IDs are related to the trackinglabel of the person in the object placement. The
operator needs to determine which object has been
put on the floor. However, the sensored floor can
not measure its tag ID of the object. In the previous system, the operator chooses one of them witheven probability. On the other hand, in the proposed
method, the operator selects one of them with the selection probability of each object in order to improvethe estimation accuracy. The selection probability isderived by using the contour of an object on thefloor. The contour of an object is measured as a point
cloud by the sensored floor (Fig.4). The detail of thistransition operation is described in the next section.

(7)

The set of output information of all robots at the time t isexpressed as At = {Ait }.
B. Estimation of object placement using SIR particle filter
We use the SIR particle filter to estimate the object
placement by integrating sensored information. Each particle
has the hypothesis of the object placement and its likelihood.
The event Xt means the object placement is xt at time t. The
posterior probability p(Xt |Z, Y, A) after the observation Zt ,Yt , and At is recursively estimated using particles. Here, Z,
Y , and A means events in which information Zt , Yt and At
are obtained from sensored cabinets, the sensored floor, andmobile robots, respectively.
The procedure for using the SIR particle filter is:
1)

Generation of initial particles:
(r)
(r)
(r)
R particles r0 = {x0 , w0 } are generated at time(r)(r)
0. Here, wt is the weight at time t. Vector xt ∈N represents the hypothesis of the object placement(r)
at time t (Fig. 6). In this initial step, vector x0 is
randomly generated.

3)

Fig. 6.

2)

Likelihood calculation:(r)
The likelihood p(Z, Y, A | Xt|t−1 ) of each particle
is calculated as(r)p(Z, Y, A | Xt|t−1 ) =
(r)

(r)

(r)

p(Z|Xt|t−1 ) p(Y |Xt|t−1 ) p(A|Xt|t−1 )Object placements of particles

State transition:
The object placement changes according to humanactivities. We modelize the change of the object
placement caused by human activities. By applying(r)
this transition model to particles rt−1 , they shift to
(r)
(r)
(r)
the next state rt|t−1 = {xt|t−1 , wt }.

(8)

where
(r)

p(Z|Xt|t−1 ) = C

S


(r)

gs (xt ).

(9)

s=1
(r)

gs (xt ) is a function that returns 1.0 if the tracking(r)label related to the object s in xt corresponds to thetracking-label related to the object s in Zt , and returns the numerical constant α(0 < α < 1) if it doesnot. The numeric constant C (0 < C < 1) depends(r)on the reliability of the sensored cabinet. p(A|Xt|t−1 )Transition model of object placement:
We assume the change of the object placement iscaused only by the following two actions:(i) a person( or a mobile robot) puts a held object
onto furniture or the floor,(ii) a person( or a mobile robot) takes up an
object.
The occurrence of the transition can be observed as
the change in the total number of tracking-labels
measured by the sensor systems.

(r)is also calculated from Eq.(9). p(Y |Xt|t−1 ) is theconstant value.
(r)
We define the weight wt of a particle as
(r)

wt
2

(r)

= p(Z, Y, A| Xt|t−1 ).

(10)The total weight of all particles is calculated as
(all)

wt

=

R


IV.

S ELECTION PROBABILITY OF AN OBJECT PLACED ON
THE FLOOR

(r)

wt .

(11)

In the transition operation (I) of the proposed method, if aperson holds several objects, the operator needs to determine
which object of the held objects is placed on the floor. In theprevious method, the operator chooses one of them with evenprobability. In the proposed method, the operator chooses oneof them with the selection probability of each object in orderto improve the estimation accuracy.

r=1

4)

Resampling:(r)
RD (0 ≤ D ≤ 1) particles rt are chosen from(r)
R particles. New RD particles rt+1 are generated
(r)
(all)
depending to the probability wt /wt . New R(1 −(r)D) particles rt+1 are randomly generated.

5)

Estimation of object placement:
The object placement is estimated based on the
distribution of R particles. The probability pij thatthe object i is related to the tracking − label j is
obtained as
R
1 
(r)
pij =
bij (rt ).
(12)
R r=1

The selection probability is obtained by using the contourof an object on the floor. The contour of an object is measured
as a point cloud by the sensored floor (Fig.7). We define theobject size as the maximum distance between two points in apoint cloud (Fig.7(b)). We also define the contour image as
the elliptical approximation of this point cloud (Fig.7(c)). The
object size and the contour image indicate a scale factor andan approximate shape of an object, respectively.

(r)bij (rt ) is a function that returns the numerical constant 1.0 if the tracking-label related to the object i in(r)(r)
xt of a particle rt corresponds to the tracking −label j, and returns 0 if it is not.

size

C. Active sensing for the tag ID of an object
The estimation accuracy of the object placement is correlated with the number of objects whose tag ID is unmeasurable
by sensors installed in an environment. In order to improve theestimation accuracy, a mobile robot with a RFID reader moves
to the position of this unmeasurable object and measures its
tag ID. When there are several unmeasurable objects, a robot
chooses one of them. For this active sensing, we propose amethod for choosing the most prospective tracking-label toimprove the estimation accuracy. This method is based on
the entropy H(E) of the probability distribution of the object
placement. The entropy H(E) is the index of insufficientlyof sensored information. The entropy increases when sensored
information is not enough to estimate the object placement.
The proposed method chooses the most prospective trackinglabel to reduce the entropy. The entropy H(E) is calculated
as
M
M 
N


H(E) =
H(Li ) = −P (Li = lj ) · log P (Li = lj ).
i

i

(a)

(b)

(c)

Fig. 7. A point cloud measured by the sensored floor ( (a) appearance ofthe object, (b) the object size, (c) the contour image ).

The object size and the contour image of each object
include measurement errors. We determine the probability density function of measurement errors through the preliminary
measurements. We derive the selection probability by using theprobability density function, the object size, and the contour
image. This procedure is divided into two partsA)B)

calculation of the probability density function of
measurement errors, andcalculation of the selection probability of each object.

The contour of an object on the floor depends on its
orientation (Fig.8). At the part A), we derive 1) the probability
density function with respect to the object size and 2) the
probability density function with respect to the contour imageat each orientation of each object.

j(13)Here, M indicates the total number of tracking-labels. The
probability P (Li = lj ) that the object lj is related tothe tracking − label Li is calculated in the same as Eq.(12).
We defines the evaluation function F (Li ) as the expectation value of mutual information after the tag ID of the objectof the tracking-label Li is measured. The tracking-label Lmax
which maximizes the evaluation function is chosen for theactive sensing.
F (Li ) =N


Fig. 8.

P (Li = lj ) · I(E; Li = lj )(14)

I(E; Li = lj ) = H(E) − H(E|Li = lj )

(15)

j

The contour images of an object with different orientations.

At the part B), we derive the selection probability pselectiby using Bayesian inference. pselectindicates the selectioni
probability that the object i is selected from the objects heldby a person. The detail of the procedure is as follows.

Here, H(E|Li = lj ) is the entropy under the assumption thatthe object lj is related to the tracking-label Li .
3

A. Probability density function for the measurement error
The contour of an object on the floor depends on its
orientation. At the orientation j of the object i, we derive(1) the probability density function fijsize for the measurement
error of the object size, and (2) the probability density function
fijcontour for the measurement error of the contour image. We
prepare fijsize and fijcontour at each orientation j of the object
i through the preliminary measurements. At each orientation
j of the object i, a point cloud of the object is obtained fifty
times through the measurements of the sensored floor. Eachdata set Dij has fifty point clouds for the object i with its
orientation j.

Fig. 9. The calculation results of the image similarity between the left image
and the middle or the right image.

image similarity is shown in Fig.9. The numerical values on
the center and the middle image indicate the image similarity
between the left image and each image.

1) Probability density function for the measurement errorof the object size: The object size is calculated from each point
cloud of the data set Dij . The average μij and the standard
2
deviation size σijof the object size are calculated at each Dij .
The probability density function fijsize for the measurement
error of the object size is expressed as the Gaussian distribution2
N (0, size σij).

contour
The contour image Iijk
(1 ≤ k ≤ 50) is obtained fromeach point cloud k of the data set Dij . The standard devia2
idealcontourtion contour σij
of the image similarity Gijk (Iij
, Iijk)are calculated at each Dij . The probability density function
fijcontour for the measurement error of the contour image is2
expressed as the Gaussian distribution N (0, contour σij).

2) Probability density function for the measurement error ofideal
the contour image: We generate the ideal contour image Iijbased on the actual shape of the object i with its orientation
j. This ideal contour image is compared with the contour
image measured by the sensored floor. The image similarity is
calculated through this comparison. We use seven Hu invariant
moments for this comparison [10]. These seven Hu invariant
moments are well known and independent parameters for thetranslation, the rotation, and the scale change. Hu invariant
moments are expressed as following equations.
hu1 = η20 + η02(16)2
hu2 = (η20 − η02 )2 + 4η11(17)

hu3 = (η30 − 3η)212 + (3η21 − η03 )2

(18)

2

2

hu4 = (η30 + η12 ) + (η21 + η03 )B. The selection probabilityof the object
We derive the selection probability pselectii from a point cloud dt of an object through the following
procedure.
1) Likelihood with respect to the object size: The object
size St is calculated from a point cloud dt measured by thesensored floor at the time t. At each orientation j of the object
i, the measurement error esizeof the object size St is expressed
ij
as
esize
= St − μij .
ij

(19)

The probability psize
is obtained by substituting esize
into
ij
ij
size
as
fij (Fig.10). We determine psizeihu5 = (η30 − 3η12 )(η30 + η12 )[(η30 + η12 )2 − 3(η21 + η03 )2 ]+(3η21 − η03 )(η21 + η03 )[3(η30 + η12 )2 − (η21 + η03 )2 ] (20)= max(psize
psize
i
ij ).

hu6 = (η20 − η02 )[(η30 + η12 )2 − (η21 + η03 )2 ]+4η11 (η30 + η12 )(η21 + η03 )

j

(21)

2

−(η30 − 3η12 )(η21 + η03 )[3(η30 + η12 )2 − (η21 + η03 )2 ] (22)probability

The image similarity G(A, B) between the image A andthe image B is expressed as
7

i=1

|

1
1
− B |
A
mi
mi

(25)

psize
indicates the likelihood that the object i is selectedifrom the objects held by a person, which is based on themeasurement error of the object size.

2

hu7 = (3η21 − η03 )(η30 + η12 )[(η30 + η12 ) − 3(η21 + η03 ) ]G(A, B) =

(24)

(23)

where
A
A
B
B
B
mA
i = sign(hi ) · log(hi ) , mi = sign(hi ) · log(hi ).

eijsize [mm]BhA
i and hi indicates Hu invariant moments of the image A
and the image B respectively.

Fig. 10. Probability density function for the measurement error of the object
size.

The image similarity G(A, B) is zero if the image A andthe image B are same image. The calculation result of the4P (Hi |E size ) is obtained by applying Eq.(29) andEq.(30) to Eq.(28).

2) Likelihood with respect to the contour image: The
contour image Ct is calculated from a point cloud dt measured
by the sensored floor at the time t. At each orientation j of theof the contour image
object i, the measurement error econtourij
Ct is expressed asideal= G( Iij, Ct ).
econtour
ij

(ii)

(26)

The probability pcontour
is obtained by substituting econtour
ij
ij
contour
into fij(Fig.11). We determine pcontourasi= max(pcontour).
pcontouriijjBayesian inference by using the contour image:E contour indicates observing the measurement error
with respect to the contour image. We determine
P (E contour |Hi ) as.
P (E contour |Hi ) = pcontouri(31)

We determine P (Hi ) as

(27)

P (Hi ) = P (Hi |E size ).

pcontourindicates the likelihood that the object i is selectedi(32)

P (Hi |E contour ) is obtained by applying Eq.(31) andEq.(32) to Eq.(28).

from the objects held by a person, which is based on themeasurement error of the contour image.

We determine the selection probability pselectasiprobability

pselect= P (Hi |E contour ) , (1 ≤ i ≤ n).
i(33)

to the
The estimation accuracy is improved by applying pselectitransition operation (I) in the proposed method.
V.

The proposed system recognized ten objects and estimatedtheir positions in the experiment. The experimental setup is
shown in Fig.12. We installed two sensored cabinets and thesensored floor as mentioned in section III. In this experiment,
we used ten everyday objects with RFID tags as shownin Fig.13. One person walked around in a room, and hemoved several objects from sensored cabinets to the floor. The
proposed system estimated the object placement which was
frequently changed with human activity.

eijcontour
Fig. 11. Probability density function for the measurement error of the contour
image.

3) Selection probability of each object: The selection probindicates the probability that the object i isability pselectiselected from the objects held by a person. pselectis obtainedicontour
by applying Bayes’ theorem to psizeandp. Bayes’iitheorem is expressed as
P (Hi |E) =E XPERIMENT

bed

table

P (E|Hi )P (Hi ).
P (E|H1 )P (H1 ) + . . . + P (E|Hn )P (Hn )
(28)

4m
chair

Hi is an event that the object i is selected. P (Hi ) is
the probability of observing Hi . E is an event that themeasurement error e is measured. P (Hi |E) is the probabilityof observing Hi given that E has occurred. P (Hi ) is the priorprobability. P (Hi |E) is the posterior probability. P (E|Hi ) is
the likelihood.

50 cm

Fig. 12.

Experimental setup.

Fig. 13.

Tracking target objects.

We determine P (Hi ) with even probability under the
principle of insufficient reason. P (Hi ) is obtained as
1
.
n

LRF
6m

Bayesian inference by using the object size:
E size indicates observing the measurement error withrespect to the object size. We determine P (E size |Hi )asP (E size |Hi ) = psize
.
(29)
i

P (Hi ) =chair
sensored cabinet B

The number of the objects held by a person is n. pselectiis the probability that the object i(1 ≤ i ≤ n) is selected. We
through the following procedures (i) and (ii).
derive pselecti(i)sensored cabinet A

(30)
5

desk

The scenario of the experiment is explained as following
three steps. At the step 1, all objects are in two sensoredcabinets. The estimation result of the object placement at thestep 1 is shown in Fig.14. The measured positions by sensoredcabinets are shown as colored circles. The numbers from 1 to
10 mean the tag IDs of objects. The position of each object isestimated as the probability of the relationship between eachobject and each measured position. This probability is obtained
from Eq.(12). The length of each color in a bar graph indicatesthe value of its probability. Each color indicates a measured
position. The sum of the probabilities in each bar graph is 1.0.
Each bar graph is composed of mono color in Fig.14. Thismeans that the position of each object is uniquely estimated. In
addition, the object placement is correctly estimated. Because
sensored cabinets can measure both the tag IDs and thepositions of the objects in them.

held by person tracking-label B

object on the floor

Fig. 16. Tracking result at the step 2 (Three objects are on the floor. Threeobjects are held by the subject. Four objects are in the cabinet B).

chosentracking-label

tag IDs
objects in cab.

Fig. 14. Tracking result after the step 1 (Six objects are in the cabinet A,
and four objects are in the cabinet B).

Fig. 17. Tracking result after the step 2 (Six objects are on the floor. Fourobjects are in the cabinet B.).

The proposed system tracks ten objects by using both themoving trajectories of objects and the size and the shapeof an object placed on the floor. On the other hand, theprevious system uses only the moving trajectories of objects.
The estimation result of the previous system after the step 2 isshown in Fig.18. The accuracy rate of the position estimationof each object on the floor is under 20 %. This means thatthe moving trajectory information is not enough to determine
which object is placed on the floor. So the previous systemchooses one of them with even probability. This simple choice
causes the decrease of the estimation accuracy.

Fig. 15. Experimental process at the step 2 (Six objects are on the floor andfour objects are stored in the cabinet B).

At the step 2, a person takes six objects from the sensoredcabinet A. The tag IDs of these objects are 1, 2, 6, 8, 9,
and 10. The person puts them one-by-one onto the floor as
shown in Fig.15. At the step 2, the estimation results of theobject placement are shown in Figs.16 and 17. The measured
positions by the sensored floor are shown as colored squares
with alphabets which indicate tracking-labels. The white color
in a bar graph indicates the position of the person. So, thewhite color means that object is held by a person. When abar graph is composed of several colors, the position of its
object is not uniquely estimated. Because the sensored floor
cannot measure tag IDs of objects on the floor. A bar graph ineach lower image in Fig.17 also indicates the probability of therelationship between each object and each tracking-label. The
vertical axis indicates the probability in each lower image. The
horizontal axis indicates the tag IDs. From Fig.17, the accuracy
rate of the position estimation of each object is ranging from70 % to 100 % after the step 2.

At the step 3, the active sensing is performed to improve
the accuracy rate of the position estimation. The entropy of theprobability distribution of the object placement is calculated
from Eq.(13). The calculation result of the entropy in theexperiment is shown in Fig.19. The entropy increases whensensored information is not enough to estimate the object
placement. When the entropy increases, the accuracy rateof the position estimation decreases. The entropy increases
due to the handling of objects by the person in the step 2.
For this active sensing, the most prospective tracking-label
D is chosen by applying Eq.(14). The tag ID of an object
6

Fig. 20.

as the probability distribution of the object placement. The
object placement is frequently changed according to humanactivities. Intersections of trajectories cause the decrease of
the estimation accuracy. The object placement is often notuniquely determined due to the deficiency of measured data.
So we proposed a new method that improves the estimation
accuracy by using the size and the shape of an object asadditional information in the data integration process based on
Bayesian inference. The size and the shape of an object areapproximately calculated from a point cloud measured by laser
range finder. Although the tag IDs of the objects on the floor
are not measurable by sensors distributed in the environment,
the system successfully estimated the object placement in theexperiment.

Fig. 18. Tracking result by using the previous system after the step 2 (Sixobjects are on the floor. Four objects are in the cabinet B.).

at the position with tracking-label D was measured throughthe active sensing. The estimation result after the step 3 isshown in Fig.20. The accuracy rate of the position estimationof each object is nearly 100 %. The accuracy rate is clearly
improved through the active sensing. The entropy in the caseusing additional information such as the size and the shapeof an object is smaller than the entropy in the case not using
it. The proposed system conducted the active sensing once atthe step 3. On the other hand, the previous system required to
conduct the active sensing four times in order to reduce theentropy enoughly.

R EFERENCES[1] K. Finkenzeller, RFID handbook: fundamentals and applications incontactless smart cards and identification, Wiley, 2003.
[2] R. Fukui, et al., “Development of a Home-use Automated Container
Storage/Retrieval System”, IEEE/RSJ Int. Conf. on Intelligent Robots
and Systems, pp.2875-2882, 2008.
[3] T. Deyle, et al., “RFID-Guided Robots for Pervasive Automation”, IEEE
Pervasive Computing, Vol.9, No.2, pp.37-45, 2010.
[4] Y. Nishida, et al., “3D Ultrasonic Tagging System for Observing Human
Activity”, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems,
pp.785-791, 2003.
[5] R. Fukui, et al., “Expression Method of Human Locomotion Records
for Path Planning and Control of Human-symbiotic Robot System basedon Spacial Existence Probability Model of Humans,” IEEE Int. Conf.
Robotics and Automation, pp.4178-4184, 2003.
[6] S. Odashima, et al., “Household Object Management via Integration of
Object Movement Detection from Multiple Cameras”, IEEE/RSJ Int.
Conf. on Intelligent Robots and Systems, pp.3187-3194, 2010[7] Murakami et al. , “Position Tracking and Recognition of Everyday
Objects by using Sensors Embedded in an Environment and Mounted
on Mobile Robots ”, Int. Conf. on Robotics and Automation, pp.22102216, 2012.
[8] S. Arulampalam, et al., “A tutorial on particle filters for on-line nonlinear/non-gaussian bayesian tracking”, IEEE Trans. on Signal Processing, vol.50, no.2, pp.174-188, 2002.
[9] Y. Nohara, et al., “Floor Sensing System Using Laser Range Finder andMirror for Localizing Daily Life Commodities,” IEEE/RSJ Int. Conf. onIntelligent Robots and Systems, pp.1030-1035, 2010[10] H. Ming-Kuei, “Visual pattern recognition by momentinvariants”, Information Theory, IRE Transactions, vol.8, pp.179-187, 1962.

Entropy
the previous
methodthe proposed
methodstep 1

Fig. 19.

step 2step 3experimental
process

The calculation result of the entropy.

VI.

Tracking result after the step 3.

C ONCLUSION

We propose an object tracking system which recognizes
everyday objects and estimates their positions by using distributed sensors in a room and mobile robots. Our previous
system estimates the object placement by using only the moving trajectories of objects. This estimation result is expressed7