3D Ultrasonic Tagging System for Observing Human Activity
Yoshifumi Nishida
Digital Human Research Center
AIST, & CREST, JST
Aomi Koto-ku, Tokyo
135-0064 Japan

Hiroshi Aizawa
Dept. of Electronic Engneering
Tokyo Denki Univ.
Nishiki-cho Chiyoda-ku, Tokyo
101-8457 Japan

Takeo Kanade
Nell H. Hoffman
Digital Human Research Center
School of Computer Science
AIST, & CREST, JST
Carnegie Mellon University
Aomi Koto-ku, Tokyo
5000 Forbes Avenue Pittsburgh, PA
135-0064 Japan
15213 USA

Abstract— This paper describes an ultrasonic tagging
system developed for robustly observing human activity
in a living area. Using ultrasonic transmitter tags withunique identifiers, the system is shown through experimental
application to be able to track the three-dimensional motionof tagged objects in real time with high accuracy, resolution
and robustness to occlusion. The use of an ultrasonic system
is desirable because of its low cost and use of commercial
components, and the proposed system achieves high accuracy
and robustness through the use of many redundant sensors.
The system employs multilateration to locate tagged objects
using one of two estimation algorithms, a least-squares
optimization method or a random sample consensus method.

I. I NTRODUCTION
Information processing services centered around humanactivity in the real world has attracted increased attention
recently [1]. Human-centered applications require the facility to observe and recognize activities as a basis, andthe present paper describes a method for observing andrecognizing behaviors robustly and in real time based on
sensorizing objects in the real world.
Generally, the problem of human behavior recognition
can be formulated as a kind of pattern recognition problem
as follows.
PY Wi PWi 
(1)
PŴ Y   max
WiPY 
where PWi Y  denotes the posterior probability that themeaning of an observed behavior pattern Y is Wi , PY 
denotes the probability that a behavior pattern Y will beobserved, PWi  denotes the probability that the behavior
meaning Wi will occur, and PY Wi  denotes the conditional probability. Thus, the problem of human behavior
recognition becomes that of searching for the maximum
posterior probability PŴ Y .
Two problems complicate the recognition of humanbehavior: the ability to observe a behavior pattern Y

Toshio Hori
Digital Human Research Center
AIST, & CREST, JST
Aomi Koto-ku, Tokyo
135-0064 Japan
Masayoshi Kakikura
Dept. of Electronic Engneering
Tokyo Denki Univ.
Nishiki-cho Chiyoda-ku, Tokyo
101-8457 Japanrobustly, and the efficient recognition of meaning W fromthe observed pattern. Without solving the first problem,
equation (1) cannot be formed. Without tackling thesecond problem, guaranteeing a solution to the equation
within the time frame demanded by the application is
impossible.
As a method for efficient recognition of behaviors,
the idea of object-based behavior recognition has been
proposed [2]. In theory, the behavior of handling objects
in an environment such as an office or home can be recognized based on the motion of the objects. However, when
applying the method to real environments, it is difficult
to even achieve an adequate level of object recognition,
which is the basis of the method.
Separating the problems of object recognition and behavior recognition is becoming increasingly realistic withthe progress in microcomputers, sensor, and wireless networks technology. It has now become possible to resolve
object recognition into the problems of sensorizing objects
and tagging the objects with identification codes (IDs),and to address behavior recognition separately through thedevelopment of applied technology.
The present authors have developed a three-dimensional
ultrasonic location and tagging system for the fundamental
function of robustly tracking objects. This system enables a new approach of tag-based behavior recognition.
In terms of cost and robustness against environmental
noise, the ultrasonic system is superior to other location
techniques such as visual, tactile, and magnetic systems.
A number of ultrasonic location systems have alreadybeen proposed or commercialized [3], [4], [5]. However,
the work [3] does not describe a method for improving
the robustness, accuracy, and resolution of position, andalthough Shih [4] proposed a robust estimation method by“direct substitution”, the system had difficulty in maintaining accuracy of position and calculation in real time.

The system presented in the present paper is developed
specifically to address the issue of robustness and accuracy
in real time.
The ultrasonic location system calculates the threedimensional (3D) position of an object by trilateration
based on three distance measurements. Like other location sensing systems such as motion capture, the system
requires more than a certain minimum number of receivers
to eliminate the effect of occlusion and outliers. The
system is comparably inexpensive due to the availabilityof cheap ultrasonic receivers, which also makes it possible
to increase the number of ultrasonic receivers to mitigate
undesirable effects. An ultrasonic location system therefore provides significant advantages in terms of robust
positioning, high accuracy, and high resolution throughthe collection of redundant distance data.
This research focuses on the development of a function
for estimating the 3D position of objects with high accuracy, high resolution and robustness to occlusion throughthe use of redundant distance data. This paper describes
the 3D position estimation function and the results ofexperiments conducted in a regular room area. In the next
secton, the developed 3D ultrasonic tagging system is first
introduced briefly. Section III describes the algorithms for
estimating the 3D position of objects in detail. Trilateration or multilateration algorithms have been proposedin the field of aerospace[6], [7]. This paper presents themultilateration algorithms applicable to a more generalcase that multiple ultrasonic receivers are put on arbitrary
positions. The results of experimental application of thesystem are then presented and discussed.
II. U LTRASONIC TAGGING SYSTEM䎳䎦䎵䎶䎕䎖䎕䎦䎃䎋䎔䎔䎘䏎䏅䏓䏖䎌䎶䎼䎱䎦䎃䏖䏌䏊䏑䏄䏏
䏇䏄䏗䏄

䎫䏘䏅
䎳䎬䎦

䎰䏈䏄䏖䏘䏕䏈
䏗䏌䏐䏈䎃䏒䏉䎃䏉䏏䏌䏊䏋䏗䎵䏄䏇䏌䏒䏗䏕䏄䏑䏖䏐䏌䏗䏗䏈䏕䎹䎫䎩䎖䎔䎗䎑䎜䎰䎫䏝䎃䏚䏌䏕䏈䏏䏈䏖䏖䎃䎵䎶䎕䎖䎕

䎳䎬䎦

䎳䎬䎦

䎶䏈䏑䏇䎃䎶䎼䎱䎦䎃䎉䎃䎬䎧䎸䏏䏗䏕䏄䏖䏒䏑䏌䏆䎵䏄䏇䏌䏒䏕䏈䏆䏈䏌䏙䏈䏕 䏗䏕䏄䏑䏖䏐䏌䏗䏗䏈䏕䎗䎓䏎䎫䏝䎸䏏䏗䏕䏄䏖䏒䏑䏌䏆䎃䏕䏈䏆䏈䏌䏙䏈䏕䎃䏄䏑䏇䎃䏄䏐䏓䏏䏌䏉䏌䏈䏕䎳䎬䎦䎸䏏䏗䏕䏄䏖䏒䏑䏌䏆䎃䎖䎧䎃䏗䏄䏊

Fig. 1.

Configuration of 3D ultrasonic tagging system

microcomputer (FLASH PIC) and power (two alkaline AAbatteries).

䎧䏈䏙䏌䏆䏈䎃䏉䏒䏕䎃䏑䏈䏗䏚䏒䏕䏎䏌䏑䏊 䎧䏈䏙䏌䏆䏈䎃䏉䏒䏕䎃䏐䏈䏄䏖䏘䏕䏌䏑䏊䏄䏑䏇䎃䏖䏜䏑䏆䏋䏕䏒䏑䏌䏝䏌䏑䏊䏗䏌䏐䏈䎃䏒䏉䎃䏉䏏䏌䏊䏋䏗

A. System configuration
Figure 1 shows the system configuration for the 3D
ultrasonic tagging system. The system consists of an ultrasonic receiving section, an ultrasonic transmitting section,a time-of-flight measuring section, a network section,
and a personal computer. The ultrasonic receiving section
receives ultrasonic pulses emitted from the ultrasonictransmitter and amplifies the received signal. The timeof-flight measuring section records the travel time of thesignal from transmission to reception. The network section
synchronizes the system and collects time-of-flight datafrom the ultrasonic receiving section. The positions ofobjects are calculated based on more than three timeof-flight results. The sampling frequency of the proposedsystem is 50 Hz.
Figure 2 shows a photograph of the prototype networkand time-of-flight measurement components, which are
to be attached to a wall. Figure 3 is a photograph ofthe tagging unit (transmission unit), which consists of an
ultrasonic transmitter, a wireless communication unit, aFig. 2.

Network and time-of-flight measurement components
䎘䎓䏐䏐
䎕䎛䏐䏐

䎚䎘䏐䏐

Fig. 3.

Ultrasonic tag

The room used to conduct the experiments is shownin Fig. 4. The room was 35  35  27 m in size, andwas fitted with 307 ultrasonic receivers embedded in thewall and ceiling. Tags were attached to various objects,
including a cup and a stapler as shown in Fig. 5. Some
objects were fitted with two transmitters. The purpose of
the experimental room is to clarify the effect of the use of
redundant sensors. More than 300 receivers do not mean

P=(x,y,z): intersection pointthat the algorithms described in the next section needsuch a large number of sensors. In actual usage, a smaller
number of receivers can be used.

l2
l1
l3
Fig. 6.

εi  li 
Fig. 4. Room with embedded ultrasonic sensors for prototype development

Intersection pointxi  x2  yi  y2  zi  z2  (3)

By solving the minimization problemnx̂ ŷ ẑ

Fig. 5.

Example of attaching tags to objects

III. U SE OF REDUNDANT ULTRASONIC RECEIVERS

min ∑ εi xyz(4)i

we can estimate the optimal value x̂ ŷ ẑ.
The minimization problem of Eq. (4) involves thesolution of a non-linear equation and therefore requires
repetitive numerical computation. Shih [4] proposed a
direct substitution method to solve Eq. (4) that involved
substituting random and arbitrary x y z into Eq. (4) andadopting the coordinate giving the minimum error as the
optimal value. This method is a robust estimation (Mestimator), but involves large calculation cost to guarantee the accuracy of the estimated position. For ultimate
accuracy, it would therefore be necessary to evaluate all
possible coordinates x y z. However, such calculation is
not suitable for real-time application.

A. Trilateration
The ultrasonic tagging system calculates the 3D positionof an object by trilateration using three distance measurements. Two methods of trilateration are investigated foruse with the proposed system: multilateration based on aleast-squares method using redundant distance data, andmultilateration based on robust estimation.
The basic principle of triangulation can be described by2222
xi  x  yi  y  zi  z  li i  1 2 3

(2)

where li denotes the distance measured by the ith ultrasonic receiver at position xi  yi  zi  from the ultrasonictransmitter at x y z, as shown in Fig. 2. Thus, the position x y z of an ultrasonic transmitter can be calculated
given three distance measurements li i  1 2 3 obtainedby three receivers that do not lie on the same line.
B. Multilateration: Basics
The estimation error εi can be defined by

C. Multilateration method 1: linearization of the minimization problem
To obtain an algorithm suitable for accurate estimationin real time, Eq. (4) may be linearized to allow ananalytical solution.
Using distance data li  l j and the receiver positions
xi  yi  zi  x j  y j  z j , we obtain the following sphericalequations for the possible position of the target.
222
xi  x  yi  y  zi  z222
x j  x  y j  y  z j  z



li2 

(5)



l 2j 

(6)

By subtracting Eq. (6) from Eq. (5), we obtain an equation
for intersecting planes between the spheres, as shown in
Fig. 7.
2x j  xi x  2y j  yi y  2z j  zi y 
li2  l 2j  x2i  y2i  z2i  x2j  y2j  z2j(7)

P=(x,y,z): intersection point

Simultaneous equations of planeon which an intersection line between the two spheres2( x j − xi ) x + 2( y j − yi ) y + 2( z j − zi ) z= li2 + l 2j − xi2 − yi2 − zi2 + x 2j + y 2j + z 2j n1  n 
A= 2

 
n 
 M

l2
l1
l3

n1

n2

 x
 
x =  yz 

M-by-3 Matrix

Ax = bα: intersection planeniRank(A) ?

Fig. 7. Planes of intersection between spheres used to give the estimatedposition

Rank(A)=1

Rank(A)=2

Rank(A)=3(Collinear)(Coplanar)(Non-coplanar)n

By inputting pairs of i j into the above equation, we
obtain simultaneous linear equations, as expressed by
AP
where

P




B
 
x
y 
z



A





Solution is indeterminate.
At most two solutions exist.
Candidateof solution

Candidateof solution

(9)

x0

n

Candidateof solutionl12  l02  x21  y21  z21  x20  y20  z20
l 2  l 2  x2  y2  z2  x2  y2  z2 
0
2
2
2
0
0
0
2
l 2  l 2  x2  y2  z2  x2  y2  z2   (11)
0
3
3
3
0
0
0
3
..
.

The position x̂ ŷ ẑ can then be calculated by a leastsquares method as follows.
AT A 1 AT B

A single position can be fixed.

x = ( At A) −1 At b

Candidateof solutionP

Solution is determinate.
A single solution exists.

Pi = ( xi , yi , zi )2x0  x1  2y0  y1  2z0  z1 2x  x  2y  y  2z  z   (10)
0
2
0
2
0
2
2x0  x3  2y0  y3  2z0  z3 


B

(8)

Center of sphere
Solution is indeterminate.
Infinite solutions exist.

(12)

If there are conditions to selectone solution from the two,a single position can be fixed.
1) Solve the minimum norm solution

x 0 = A+ b

A+ is Moore-Penrose inverse matrix.

A position cannot be fixed.

x 0 is the minimum norm solution.
2) Solve two positions using
the equations below.

x = tn + x 0( x − x i ) 2 + ( y − y i ) 2 + ( z − z i ) 2 = li2

n is a base vector of nullspace of A.
3) Select a single solution using
conditions such as(x − p co n d) ⋅ n co n d > 0

Fig. 8. Algorithm for estimating 3D position by a least-squares method
considering the rank of A

This method minimizes the square of the distance between
the planes expressed by Eq. (7) and the estimated position.
The algorithm is described in detail in Fig. 8. In actual
usage, the rank of matrix A must be considered.
D. Multilateration method 2: Robust estimation by
RANSAC
Data sampled by the ultrasonic tagging system is easily
contaminated by outliers due to reflections. Method 1
above is unable to estimate the 3D position with highaccuracy if sampled data includes outliers deviating from anormal distribution. In the field of computer vision, robust
estimation methods that are effective for sampled data
including outliers have already been developed. In thiswork, the random sample consensus (RANSAC) [8], [9]estimator is adopted to eliminate the undesirable effectsof outliers. The procedure is as follows.

1) Randomly select three distances measured by threereceivers ( jth trial).
2) Calculate the position xc j  yc j  zc j  by trilateration
using Eq. (2).
3) Calculate the error εc ji for all receivers (i 0 1  n) by Eq. (13), and find the median εm j ofεc ji .
4) Repeat steps 1 to 3 as necessary to find the combination of measurements giving the minimum error,
and adopt the corresponding 3D position.

x̂ ŷ ẑ ¬¬
zm j 2 ¬Õ xi  xm j  yi  ym j  zi  
2

2

(13)
(14)

med j εc ji εm jmin εm j

 

200
150
100
50

40

150
100
50

50

60

0

10

20

30

40

50

60

The number of sensors

The number of sensors

DensityFig. 9. Relationship between resolution and the number of sensors forthe least-squares method (left) and RANSAC (right)

0.04
0.02
0
30

0.6

y[m

Density

0.5

30

m]

30

0.4
0.3
0.2
0.1

-30

-30

x

]
[mm

100
50

250
200
150
100
50

0

0
0

10

20

30

40

50

60

The number of sensors

0

10

20

30

40

50

60

The number of sensors

B. Positioning accuracy

0
30

150

200

0
20

200

Fig. 11. Relationship between positioning accuracy and the number ofreceivers for the least-squares method (left) and RANSAC (right)

250

Deviation [mm]Deviation [mm]

250

10

300

250

(15)

IV. E XPERIMENTAL APPLICATION

0

350

300

Error [mm]



εc ji

Error [mm]

¬
¬
¬li

350

0
-30

-20

-10

0

10

20

30

Z [mm]

Fig. 10. Resolution in the x and y directions (left) and z direction (right)(grid size: 2  2 mm)

A. Resolution
Figure 9 shows the relationship between the number ofreceivers and the deviation of the estimated position for
4, 6, 9, 24, and 48 receivers in the ceiling. To compare
the effect of the RANSAC method and that of the leastsquares method, one receiver is selected randomly and500[mm] is added to the distance data of the selected
receiver as outlier. Each point was derived from 30 estimations of the position. The 5 lines in the figures represent
estimation for 5 different locations of the transmitter. The
resolution increases with the number of receivers, and the
RANSAC method provides a more stable estimation withhigher resolution compared to the least-squares method.
The resolution in the x, y, and z directions is illustrated
in Fig. 10, which shows the probability density distribution
for 1000 estimations using RANSAC. The resolution inx and y directions is about 15 mm, while that in the z
direction is about 5 mm.

Figure 11 shows the relationship between the number ofreceivers and the error of the estimated position for 4, 6,
9, 24, and 48 receivers. The error is taken as the distancefrom the position measured by a visual motion capturesystem. One receiver is selected randomly and 500[mm]is added to the distance data of the selected receiver as
outlier. Each point was derived from 30 estimations of theposition. The 5 lines in the figures represent estimation for
5 different locations of the transmitter. The error decreases
as the number of receivers is increased, and the RANSAC
method is appreciably more accurate with fewer receivers.
It is considered that the least-squares method is easily
affected by outliers, whereas the RANSAC method is not.
Figure 12 shows the 3D distribution of error for 1400
measured positions in the room. The figures show thatthe error is lowest (20–80 mm) immediately below the 48
receivers in the ceiling, increasing toward the edges of theroom.
The results of experiments for evaluating accuracy andresolution demonstrate that it is possible to improve accuracy and resolution by increasing the number of receivers,
and that the undesirable effect of outliers can be mitigatedthrough the use of RANSAC estimation.
C. Robustness to occlusion
As in other measuring techniques such as vision-based
methods, it is necessary to increase the number of sensors
to solve the problem of sensor occlusion, where the line ofsight to the target object is obstructed by other objects suchas walls or room occupants. In the present tagging system,
the problem of occlusion occurs often when a person
moves or operates an object. These situations give rise totwo separate problems; a decrease in the number of usable
sensors for the target, and an increase in reflections due
to obstruction and movement. As one of the most typicalsituations where occlusion occurs, this section focuses on
occlusion due to a hand.
Figure 13 shows how the error increases and the numberof usable sensor decreases as a hand approaches an object
fitted with an ultrasonic transmitter for the least-squares
and RANSAC methods. Although the error increases
significantly by both methods when the hand approaches
the object, the RANSAC method is much less affected

D. Real-time position measurement䎓䎖䎘䎓䎳䏒䏖䏌䏗䏌䏒䏑䎃䏒䏉䎃䏖䏈䏑䏖䏒䏕䎕䎚䎓䎓䏝䎠䎕䎓䎓䎓
䏝䎠䎔䎘䎓䎓
䏝䎠䎔䎓䎓䎓䏝䎠䎘䎓䎓䎗䎓䎓䎓䎋䏅䎌䎃䎳䏒䏖䏌䏗䏌䏒䏑䏖䎃䏒䏉䎃䎗䎛䎃䏖䏈䏑䏖䏒䏕䏖䏝䎃䎾䏐䏐䏀

䎖䎖䎖䎾䏐䏐䏀

䎋䏄䎌䎃䎰䏈䏄䏖䏘䏕䏈䏇䎃䎔䎗䎓䎓䎃䏓䏒䏌䏑䏗䏖

䎕䎓䎓䎓
䎔䎘䎓䎓
䎔䎓䎓䎓
䎖䎓䎓䎓

䎘䎓䎓
䎓

䏀
䎕䎓䎓䎓

䏛䎃䎾䏐
䎖䎘䎓䎓

䏐䏀

䎖䎓䎓䎓

䏐䏐

䎔䎓䎓䎓

䏜䎃䎾

䎔䎓䎓䎓

䎓
䎗䎓䎓䎓

䎖䎘䎓䎓

䎖䎓䎓䎓

䎖䎓䎓䎓

䎕䎘䎓䎓

䎕䎘䎓䎓

䎕䎓䎓䎓

䎕䎓䎓䎓

䎼䎃䎾䏐䏐䏀

䎼䎃䎾䏐䏐䏀

䎖䎖䎖䎾䏐䏐䏀

䎕䎓䎓䎓
䎓

䎔䎘䎓䎓
䎔䎓䎓䎓

䎔䎓䎓䎓

䎘䎓䎓
䎓

䎘䎓䎓

䎓

䎘䎓䎓

䎔䎓䎓䎓

䎔䎘䎓䎓

䎕䎓䎓䎓

䎕䎘䎓䎓

䎖䎓䎓䎓

䎖䎘䎓䎓

䎗䎓䎓䎓

䎓

䎗䎘䎓䎓

䎘䎓䎓

䎔䎓䎓䎓

䎔䎘䎓䎓

䎖䎓䎓䎓

䎕䎘䎓䎓

䎕䎘䎓䎓

䎕䎓䎓䎓

䎕䎓䎓䎓

䎼䎃䎾䏐䏐䏀

䎖䎘䎓䎓

䎖䎓䎓䎓

䎔䎘䎓䎓

䎕䎓䎓䎓

䎕䎘䎓䎓

䎖䎓䎓䎓

䎻䎃䎾䏐䏐䏀

䎖䎘䎓䎓

䎗䎓䎓䎓

䎖䎘䎓䎓

䎗䎘䎓䎓

䎋䎽䎠䎔䎓䎓䎓䎾䏐䏐䏀䎌

䎔䎘䎓䎓
䎔䎓䎓䎓

䎔䎓䎓䎓

䎘䎓䎓

䎘䎓䎓
䎓

䎓

䎋䎽䎠䎘䎓䎓䎾䏐䏐䏀䎌䎻䎃䎾䏐䏐䏀

䎼䎃䎾䏐䏐䏀

䎔䎘䎓䎓

Figure 14 shows the measured trajectory for a person
moving a cup to a chair, the floor, and a desk. The
figure demonstrates that the system can robustly measure
the positions of the objects in most places of the room
regardless of occlusion by a hand or body.
In the current system, the sampling frequency is about
50 Hz. This frequency decreases to 50n Hz when nobjects are being monitored. However, it is possible tomaintain a high sampling frequency by selecting whichtransmitters to track dynamically. For example, a transmitter can be attached to a person’s wrist, and the system
can select transmitters in the vicinity of the wrist to
be tracked, thereby reducing the number of transmittersthat need to be tracked at one time and maintaining thehighest sampling frequency possible. Figure 15 shows themeasured trajectory in a dynamic selection mode. The red
sphere in the figure shows the position of the hand.

䎓

䎘䎓䎓

䎔䎓䎓䎓

䎔䎘䎓䎓

䎕䎓䎓䎓

䎕䎘䎓䎓

䎖䎓䎓䎓

䎻䎃䎾䏐䏐䏀

䎕䎓

䎗䎓

䎖䎘䎓䎓

䎗䎓䎓䎓

䎗䎘䎓䎓

䎓

䎓

䎘䎓䎓

䎔䎓䎓䎓

䎔䎘䎓䎓

䎙䎓

䎛䎓

䎔䎓䎓

䎕䎓䎓䎓

䎕䎘䎓䎓

䎖䎓䎓䎓

䎻䎃䎾䏐䏐䏀

䎋䎽䎠䎔䎘䎓䎓䎾䏐䏐䏀䎌

䎔䎕䎓

䎔䎗䎓

䎔䎙䎓

䎗䎓䎓䎓

䎖䎘䎓䎓

䎗䎘䎓䎓

䎋䎽䎠䎕䎓䎓䎓䎾䏐䏐䏀䎌䎔䎛䎓

Fig. 14.

䎕䎓䎓Measured trajectory for moving a cup around the room䎨䏕䏕䏒䏕䎃䎾䏐䏐䏀

Fig. 12.

3D distribution of error in the experimental room

than the least-squares method. This demonstrates that the
proportion of outliers increases when occlusion occurs,
and that RANSAC is more robust in this situation because
it can mitigate the effect of such outliers.

Fig. 15.

Dynamic selection of transmitters

180

Least Square
RANSAC160

The number of valid sensors

40

140
30

120
100

20

80
60

10

40
20

0

0
10

15

20

25

30

35

40

45

The number of valid sensors

Error [mm]

200

50

Distance from object [cm]Fig. 13. Accuracy of the ultrasonic tagging system when occlusion due
to a hand occurs

E. Recognition of human behavior
Figure 16 shows the measured trajectories when several
objects are moved one after another (see video). Behavior
recognition is performed by interpreting the change ofstate using the ultrasonic tag.
Output example .....,
04:03:55 place yellow cup on desk
04:04:05 hold mobile phone
04:04:12 place mobile phone on floor
04:04:19 hold chair
04:04:31 place chair on floor
04:04:34 hold trash
04:04:40 place trash on floor
04:04:46 hold stapler

cell phone

cup

trash

chair

development of new applications based on human activity
data.
VI. REFERENCES

documents
staplerFig. 16. Trajectories for movement of several objects one after another

04:04:52 place stapler on desk
04:04:59 hold documents
04:05:13 staple documents with stapler .....

V. C ONCLUSION
A 3D ultrasonic tagging system that provides robust
observation of human activity was presented. The ultrasonic tagging system consists of an ultrasonic transmitter/receiver, a wireless communication unit and a hostcomputer, and can implemented at low cost. The systemmeasures the 3D position of any object fitted with anultrasonic transmitter with a unique ID.
In order to estimate the 3D position with high accuracy,high resolution, and robustness to occlusion, the authors
propose two estimation methods, one based on a leastsquares approach and one based on RANSAC.
The system was tested in an experimental room fittedwith 307 ultrasonic receivers; 209 in the walls and 98 in
the ceiling. The results of experiments conducted using
48 receivers in the ceiling for a room with dimensionsof 35  35  27 m show that it is possible to improve
the accuracy, resolution, and robustness to occlusion byincreasing the number of ultrasonic receivers and adopting
a robust estimator such as RANSAC to estimate the 3D
position based on redundant distance data. The resolutionof the system is 15 mm horizontally and 5 mm verticallyusing sensors in the ceiling, and the total spatially varying
position error is 20–80 mm. It was also confirmed that thesystem can track moving objects in real time, regardlessof obstructions.
Further development of the system will include refinement of the method for measuring the 3D position
with higher accuracy and resolution, miniaturization of
the ultrasonic transmitters, development of a systematic
method for defining and recognizing human activities
based on the tagging data and data from other systems, and[1] T. Hori, “Overview of Digital Human Modeling,” inProceedings of 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2000),
Workshop Tutorial Note, pp. 1–14
[2] H. Mizoguchi, T. Sato, T. Ishikawa, “Robotic Office
Room to Support Office Work by Human Behavior
Understanding Function with Networked Machines,”
IEEE/ASME Transactions on Mechatronics, Vol. 1,
No. 3, September 1996, pp. 237–244[3] A. Hopper, P. Steggles, A. Ward, P. Webster, ”The
Anatomy of a Context-Aware Application,” in Proceedings of 5th Annual International Conference
Mobile Computing and Networking (Mobicom99),
pp. 59-68[4] S. Shih, M. Minami, H. Morikawa, T. Aoyama, “AnImplementation and Evaluation of Indoor UltrasonicTracking System,” in Proceedings of the 2001 IEICE
Domestic General Conference, B-7-20[5] http://www.isense.com[6] K.C. Ho “Solution and Performance Analysis of
Geolocation by TDOA, ” IEEE Transaction on
Aerospace and Electronic Systems, Vol. 29, No. 4,
October 1993, pp. 1311-1322[7] D.E. Manolakis, “Efficient Solution and Performance
Analysis of 3-D Position Estimation by Trilateration,” IEEE Trans. on Aerospace and Electronic Systems,
Vol. 32, No. 4, October 1996, pp. 1239-1248[8] P. J. Rousseeuw, A. M. Leroy, “Robust Regression
and Outlier Detection,” Wiley, New York, 1987
[9] M.A. Fishler, R.C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Application
to Image Analysis and Automated Cartography,”Communication of the ACM, Vol. 24, June 1981, pp.
381–395