Letters

Spot Information Navigator
Hiromitsu Tanabe†, Tamio Kihara, and Takashi Honishi
Abstract
To provide advertising information effectively, one must consider when it is provided, as well as whereand how many times the customer encounters it. In the future ubiquitous society, the systems using various kinds of sensors connected to the network will be needed. Spot Information Navigator gathers data
about shopping conditions in real time through radio frequency tags, camera images, and other types ofsensors, and provides appropriate advertising information to customers accordingly.

1. Effective advertising
In the field of marketing, one growing area is recency effects (recency is the property of having happened or appeared not long ago). For example, purchasing decisions often depend not on the number oftimes the customer encounters an advertisement, buton the timing of the encounter. Average spending per
customer can be increased by expanding customer
traffic in a store using “cross-merchandising”, inwhich information is provided across several sales
locations and in relation to multiple products. One
example of this would be attempting to increase salesof both beer and snacks by placing advertisements forthe snacks where beer is being sold. (In Japan, people
often drink beer when eating yakiniku (grilled meat),so placing posters advertising yakiniku as well as beerin beer shops might be effective.)
Putting this type of co-marketing into practicerequires an information providing technology that
focuses on the sales location (referred to here as aspot), and the system should adapt to the circumstances of the individuals or groups in that spot, as
well as the products and other objects present.
2. Outline of Spot Information Navigator
Spot Information Navigator, which is undergoing† NTT Cyber Space Laboratories
Yokosuka-shi, 239-0847 Japan
E-mail: tanabe.hiromitsu@lab.ntt.co.jp

84

research and development at NTT Cyber Space Laboratories, is a system for providing information to theindividuals or groups in that spot. It gathers situation
data about the conditions in a given spot in real timeand performs appropriate actions. As a result, thepeople in the spot notice the information and their
behavior is affected by it. Service providers or theparties providing the spot can expect to enjoy a rangeof benefits including increased purchase rates and arise in the popularity of these sales spots.
Here, we use the term “spot” to refer to a region ofspace where the situation is understood and information control is conducted. Examples of such spots
might be sales locations within a shop, or the shop
itself within a shopping mall. Stations, concourses,and train platforms are other examples.
3. Approach
Figure 1 shows the concept of a service in which
the Spot Information Navigator is applied in a shop.
When a customer approaches Spot A, the system
begins to play back an audio/visual advertisement,
presenting attractive sound and images that catch thecustomer’s attention. In this way, the customer can bemade aware of the advertisement, and recency effects
can be expected. Besides simply playing standard
advertisements when the customer arrives at the spot,
if the system can access the customer’s profile via theshop’s frequent customer programs, it can present
customized advertisements directed at that individual, thus enabling even more effective marketing.
NTT Technical Review

Letters

Advertisements inSpot A for productsin Spot B (crossmerchandising)SensorSensor
Display

Display
Information provided topersons or communities
gathered in a spot (spotactivation).

ProductProductSpot B
Spot A

When a customer
approaches the spot,
images start and gain
the customer’s attention(recency effect).

When the customer
remains in the same place,the advertisement
promotes purchase ofproducts in Spot A(increased purchase rate).

Different advertisementsfor persons coming from
Spot A and Spot C(cross merchandising)

Spot C

Fig. 1. Service image for Spot Information Navigatior.

Furthermore, if the customer stays for longer than acertain period of time, the system interprets this tomean that the customer is interested in that spot andprovides relevant information.
Advertisements for a product placed in a given spot
can be shown in the same spot or in a different spot
(Fig. 1). In this way, the customer can be encouraged
to move from Spot A to Spot B, thus enabling crossmerchandising. Moreover, the system can change theadvertisements displayed in Spot B depending onwhether the customer comes from Spot A or C.
When there is more than one customer in a given
spot, information control is required to maximize theeffects for both individual customers and all the customers in that spot. There are a few different methodsof providing this information navigation. For example, individual advertisements with a short playback
time can be played one after another, advertisements
can be directed at “excellent customers” who havemade large purchases before coming to the spot, oradvertisements can be directed at the decision-makeramong the group, such as the mother rather than herchildren.
4. System configuration
Figure 2 shows the system configuration of Spot
Information Navigator. Each spot is equipped withsensors geared toward the type of data to be gathered
and output devices (such as displays and speakers) for
providing information to the customers. A Spot InforVol. 1 No. 7 Oct. 2003

mation Navigator server is installed on the center
server side. It consists of an “actual space metadata
allocation mechanism” for creating the metadata
about the actual spot space, a “metadata matchingmechanism” for monitoring the spot situations, andan “action execution mechanism” for executing theactions when a specified situation occurs in the spot.
The sensor server must be connected to the sensors
and output devices in each spot via a network. Thissensor-based configuration can provide informationtied to multiple spots and can provide information
based on past history data. Below, we explain the various elements of the system.
4.1 Gathering data about conditions at a spot
Figure 3 shows an example of the space configuration when the sales location in a shop is considered asa spot. The display and speakers required to play backinformation are installed in the product shelves wherecustomers can see them easily. Sensors are installedaccording to the situations to be monitored. In thisexample, radio frequency (RF) tags and video cameras are used as sensors.
Each RF tag contains a unique ID. These IDs are
picked up by a receiving antenna and individuallyidentified. There are two types of RF tags: active andpassive. We used active tags, which have internal batteries and transmit their unique IDs at regular intervals.
Antennas are installed at the spot to pick up theradio waves transmitted by the RF tags, and the sen85

Letters

Rule authoring tool

Rule database

Content database

Metadata matchingmechanism

Action execution
mechanism

Spot Information Navigation server
Actual space metadata
allocation mechanism

Sensor data

Sensor data

ActionActionSensor data

Spot AAction
Spot B

Fig. 2. System configuration.

CameraReceived ID Received datexxxxx
2003/4/22 14:20:21::::

Product shelf

Display & speakers

RF antenna

Shopping basket
with RF tag

Fig. 3. Configuration of spot spaces.

86

NTT Technical Review

Letters

sitivity of the antennas is adjusted to roughly matchthe area of the spot. An RF tag is installed on each
shopping basket and cart in advance. In this way, ID
reception begins when a person with an RF tag enters
the reception range of the antenna. It is possible toobtain information such as the movement history ofeach RF tag between spots or how long each RF tag
remained in each spot. By establishing a connectionin advance between the tag’s ID and a frequency program or other customer information, we can identify
the individual based on a customer profile.
A video camera installed above the spot monitors
the user behavior. It captures images of people in thespot, distinguishes people from the background, andextracts the human forms. The central coordinates foreach human form are calculated for each frame of thevideo. Then, by calculating the changes in central
coordinates for each video frame, the system can calculate, in real time, the speed and acceleration of thehuman figures etc. In this way, we can obtain a wide
range of information related to customer behavior,including how many users are in which position within the spot or whether the users are facing the shelves
or facing away from the spot. Two sensors can also becombined to gain a complete understanding of the situation: for example, we could recognize that “a person who approached the shelves in Spot A has
reached Spot B”.
4.2 Creation of actual space metadata
The sensor data obtained in each spot is gathered in
the sensor server. In order to convert this into information that expresses the spot situations, meaning
must be assigned to the gathered sensor data. Here,
we refer to this sensor data with attached meaning as“actual space metadata”. When a video camera isused as the sensor, the actual space metadata is created from the information obtained from the video
images, such as the number of people and direction ofmovement. Converting the situations into metadata
simplifies the process of rule description, as
explained below, and also facilitates referencing after
this information has been stored in a database.
4.3 Metadata matching
According to the situation in the spot, an action isperformed. For example, “the advertisement is playedback on the display when the customer comes to thespot”. The relationships between these situations andactions are described in rules. We have provided a“rule authoring tool” for describing these rules.
The rules describe the conditions that must be fulVol. 1 No. 7 Oct. 2003

filled, including actual space metadata, history information, and sensor information. They can also
describe multiple actions, such as creating a playlist
that contains the optimum contents and playbackorder for a given spot.
Sensor information from the spot is continuallysent to the server, where it is converted into actual
space metadata. The metadata matching function
monitors whether the actual space metadata createdfulfills the conditions described by the rules—that is,whether a specified situation has occurred in the spot.
This does not necessarily mean that only one situation will occur in a given spot at one time; two or
more situations may occur simultaneously. The question of which situation takes precedence can beresolved by establishing a “rule priority” when therules are described.
4.4 Action execution
When it is determined through metadata matchingthat a specified situation has occurred, an action isexecuted according to the relevant rule. The action
execution function controls the actions for a given
spot. When an action is to be executed in that spot,optimum timing must be used to ensure that theeffects obtained are those intended by the spot or service provider. Examples of timing are immediate execution, execution after completing the currently playing contents, and execution after a specified period oftime. The timing can be specified when the action for
the rule is described.
When the timing for action execution is controlled,there may be cases in which several actions could beexecuted at the same time, so it is necessary to control
the order in which actions are executed. This order iscontrolled by the action execution function.
5. Action execution status and usual status
It is important to ensure that people in the spot get
the impression that there has been a dramatic change
in status before and after the execution of a particularaction. For example, contents must be adjusted toimpress upon a person in a given spot that playback
has begun: quiet contents are played back under usual
conditions and more upbeat contents are played backwhen the action is executed. We have prepared a toolfor scheduling usual conditions in the spots to define
actions in the context of usual status.

87

Letters

6. Future developments
Spot Information Navigator executes actions inspots, but the contents determine whether the action
will affect people there or not. Furthermore, because
the intentions of the spot or service providers must be
expressed in the form of rules, rule creation is a burden for them. We will continue to promote researchand development on rule creation mechanisms bydeveloping authoring tools for creating contents thatwill maximize the effects of Spot Information Navigator and by describing services that can be provided
in spots.
References
[1]
[2]

[3]

[4]

88

J. Novo, “Drilling Down: Turning Customer Data into Profits With aSpreadsheet,” Booklocker. com., 2002.
T. Kihara, “Multi-user Navigation System Manipulating Real VideoImages,” DICOMO2002 Symposium, IPSJ, Nishi-Izu, Shizuoka,
Japan, July, 2002 (in Japanese).
T. Kihara, M. Kusahara, and H. Yasuda, “Spatial Artworks and Networking Artworks,” IPSJ Journal, Vol. 44, No. 2, pp. 212-222, 2003
(in Japanese).
H. Tanabe and T. Kihara, “Information Navigator making Real World
Meta-data,” DICOMO2003 Symposium, IPSJ, Akan, Hokkaido,Japan, June, 2003 (in Japanese).

Hiromitsu Tanabe
Information-Base Project, NTT Cyber Space
Laboratories.
He received the B.E. and M.E. degrees inindustrial systems engineering from Tokyo Institute of Technology, Tokyo in 1996 and 1998,respectively. In 1998, he joined NTT Information and Communication Systems Laboratories.
He has been researching geographical information systems and database management systems
and is currently researching metadata management systems. He is a member of the Information
Processing Society of Japan (IPSJ).

Tamio KiharaSenior Research Engineer, Information-BaseProject, NTT Cyber Space Laboratories.
He received the B.E. and M.E. degrees inindustrial and systems engineering from Aoyama
Gakuin University, Tokyo in 1989 and 1991,
respectively. He joined NTT in 1991. He has
been engaged in the research and development ofinteractive media systems. He is also an Associate Researcher in the Research Center for
Advanced Science and Technology of the University of Tokyo.

Takashi Honishi
Senior Research Engineer, Supervisor, Information-Base Project, NTT Cyber Space Laboratories.
He received the B.E. degree in electronic engineering from Kyushu Institute of Technology,Fukuoka in 1981 and the M.E. degree from theInterdisciplinary Graduate School of Science andEngineering of Tokyo Institute of Technology,
Tokyo in 1983. In 1983, he joined the Communication and Information Laboratories, Nippon
Telegraph and Telephone Public Corporation(now NTT), Kanagawa, Japan. He has been
engaged in the research and development of multimedia database management systems and iscurrently researching metadata management systems. He is a member of IPSJ and IEEE.

NTT Technical Review