Study on real world relations among humans and objects
using cellular phone and barcode
Akifumi Kimura
Department of Systems Innovation
School of Engineering,
The University of Tokyo

Tomohiro Fukuhara
RACE (Research into Artifacts, Center
for Engineering)
The University of Tokyo

Ikki Ohmukai
National Institute of Informatics

Hideaki Takeda
National Institute of Informatics

Abstract
The purpose of this research is to find the methods for understanding our lifestyles by analyzing therelational network among humans and objects. We created a cellular phone application for use as our
experimental system that can read barcodes. Users can describe the relations between humans and objects orobjects and objects using this application. We experimented using this application and conducted a quantitative
analysis of the relational network among humans and objects. Relational networks among humans and objects
extracted from this experiment are described, and benefits and values of these networks for users are discussed.

Keywords: Real World, Cellular Phone, Barcode, Relation, Network, Degree Centrality

1

INTRODUCTION

There has recently been a lot of research on the achievements of the ubiquitous society [1]. We now have theability to store data on objects that we use in our lives, and we can conveniently use this data whenever we want.
For example, KDDI1 carried out an experiment in which people can read barcodes attached to commodities byusing a cellular phone and find related information from November, 2005 to March, 2006. In KDDI’s
experiment, users can describe their impression of the commodity when they read the barcode and post it on a
blog using their cellular phone [2]. Then, the information on the commodity is automatically retrieved anddisplayed by the Amazon2 web services.
There was also an experiment on registering all the things in a person’s house in a database [3]. However,these experiments do not take the relations between humans and objects or objects and objects into
consideration.
The image on the left in Figure 1.1 shows a situation where people are talking and eating. We assume that a
network exists in this scene [4]. The figure on the right in Figure 1.1 shows only the network map among
humans and objects. In this network, the nodes consist of the humans and objects, and links consist of therelations between them. This network is very complicated and makes a small world [5] in which the distance
between two nodes is very short. The social network analysis method is available for any complex world and isvery effective for understanding the property of such complexity [4]. For example, we can evaluate centrality ofnodes based on the degree centrality which is the simplest method [6]. We formed a hypothesis that importantnodes or links which is effective as the method of understanding our lifestyles exist in the relational network
among humans and objects.
The purpose of our research is to find these important nodes or links, and confirm the methods ofunderstanding our lifestyles by analyzing the relational network among humans and objects. Therefore, we
conducted an experiment to extract the relational network from our surrounding living environment, like in thehouse, by using a cellular phone application that can read barcodes. Users can describe the relations between
humans and objects or objects and objects using this application. There are three advantages of using a cellular
phone and barcodes. First, we can easily make3 QR codes to compare with RFID tags. Second, cellular phones,
1

Japanese IT and Telecommunications Company
http://www.amazon.co.jp/Web
3
We made QR codes using a label printer.
2

basically reading tools, are widespread in Japan. Third, we can collect data that are closely related to the realworld, because we carry them anytime and anywhere we go.
This paper consists of following sections. In Section 2, we describe our experimental system. In Section 3,
we describe the experimental results and consideration by using a quantitative analysis on the relational networkand network visualization map. In Section 4, we make discussions on the experiment. We conclude in Section 5.

2

EXPERIMENTAL SYSTEM

Figure 2.1 is an overview of the experimental system that we constructed for this research. First, we gave
objects ID numbers by attaching QR and JAN codes to them. Figure 2.2 shows examples of the QR code and
JAN codes. A QR code is a two-dimensional barcode developed by Denso Wave Co. Ltd. in 1994. The QR code
can store a number of characters [7]. A JAN code is a one-dimensional barcode that can be attached to almost
any commodity in Japan. Second, we read the barcodes which store the ID numbers using the i-Appli 4
application available on Japanese cellular phones (NTT DoCoMo). Then we described the relations between
humans and objects or objects and objects using this application. Third, we can send the data about relations
among humans and objects to a server where we could see the relations on a web browser.
.

Figure 1.1 Relational network extracted from one situation where people are talking and eating.

Figure 2.1 Flowchart of entire experimental system(Giving objects ID numbers by barcodes, reading barcodes using a cellular phone application, describing therelations between humans and objects or objects and objects, and sending and receiving the relational data
between the application and the server).

4

Java application of NTT DoCoMo’s cellular phone.

2.1 METHOD OF GIVING ID NUMBER TO OBJECTS AND HUMANS

Figure 2.2 QR code (left) and JAN code (right).
We gave objects and humans ID numbers. There are two ways of giving ID numbers to objects. The first is toattach a QR code that we printed onto the objects. QR codes can store information on the ID number and theversion of the experiment, such as “54&ver1.0” (Figure 2.2 (left)). The second is to read the JAN codes (Figure
2.2 (right)), which can be appended to almost all commodities, such as plastic bottles, books, and so on. The
number of JAN codes parallels the ID numbers. We gave ID numbers to humans when they downloaded i-Applifor the first time.

2.2 CLIENT SYSTEM (CELLULAR PHONE APPLICATION)
Figure 2.3 shows the experimental procedure. We can read QR codes or JAN codes by starting i-Appli and thebarcode reader. Then, we describe the relations or attributes and send them to the server. In particular, we
describe what we do as a relation and describe the name of the objects as an attribute. We can describe relations
between objects and objects by reading two barcodes in series when we use two objects concurrently. Figure 2.4
shows screen images of the i-Appli for this experiment. We can send relations or attributes simply and easily
using this i-Appli.

Figure 2.3 One type of experimental situation. A user can read a barcode and describe relation or attribute.

Figure 2.4 Screen image of i-Appli display.
Left side of the Figure 2.4(1) We can input a description of a relation in this blank space. (e.g. ate, used together)(2) Display the content of barcode by pushing the “1” key.

(3)
(4)
(5)
(6)

We can get specific information about (2) by pushing the “4” key.
Display the content of barcode by pushing the “1” key.
We can get specific information about (1) by pushing the “4” key.
Our ID number and time you last sent information.

Right side of the Figure 2.5(1) Result of reading QR code.
(2) We can input an attribute’s name here. (e.g. color, height)(3) We can input an attribute value here. (e.g. red, 3 inches)

2.4 SERVER SYSTEM
Figure 2.4 shows the server system for this experiment. Users can enter data in the database from their cellular
phone through CGI. Users can also see the attribute data of the objects on the cellular phone, which are stored
on the database. In addition, users can see the registered data on the database using the browser. There are three
types of views. First, we can see the database table directly through easy viewing. Second, by choosing one ID
number, we can see all the ID numbers and subsequent information that is related to the ID number chosen. We
call this page the ID page, and Figure 2.5 shows a screen image of this ID page. Third, when we choose one,
two, or three human ID numbers, we can see all the relations they have registered. In Figure 2.4, NETWORKCGI 1 makes the second type possible, and NETWORK CGI 2 makes the third type possible.

Figure 2.4 Server system.

Figure 2.5 Screen image of ID page (ID154).

3

EXPERIMENT

We conducted an experiment to investigate the relations between humans and objects by using our system.
Table 3.1 lists the specifics of the experiment. In addition, we twice sent out questionnaires. The first one wasused to ask questions before the experiment. In this questionnaire, we asked subjects which objects in theirhouse were important to them. The second one was used to ask questions after the experiment, and thisquestionnaire asked the same questions as the first.
We analyzed the relations between humans and objects quantitatively based on the social network analysis[8]. We used the Simple Network Analysis Tool5, which is a tool for analyzing networks. For example, we
analyzed the number of nodes, the number of links, the diameter of the graph, the average distance, the densityof the graph, the transitivity, the degree centrality, and the central tendency [6, 9, 10]. In addition, we visualized
the network by using Pajek6, which is a network visualization tool. In Section 3.1, we analyze and take into
account the nodes that are important in one’s life by looking at the relational network that is formed by one
person. Then, in Section 3.2, we analyze and take into account the relational network formed by two people. In
Section 3.3, we analyze and consider the network formed by three people.

Table 3.1 Outline of experimentNo. of subjects
10
Days
5
No. of objects one subject registered
About 30~50
No. of relations one subject described
About 80~100

56

http://www.geocities.jp/snatool/index.html
http://vlado.fmf.uni-lj.si/pub/networks/pajek/

Figure 3.1 Degree centrality on binary and multiple value graphs.

3.1 ONE-PERSON NETWORK
We analyzed person A’s network. Figure 3.1 shows the degree centrality of each object person A registeredduring the experiment. The left side of the figure shows the degree centrality of a binary graph and right side of
the figure shows the degree centrality of a multiple value graph. The binary graph is a non-weighted graph ofwhich the elements of adjacency matrix consists of 0 and 1. On the other hand, the multiple value graph is aweighted graph. If a subject describes and registers relations towards the same combination of objects, thenumber of registrations is reflected towards the elements of adjacency matrix. For example, if a subject
registered a relation named “I watched TV” three times, we count it as three degrees. As shown in Figure 3.1,
different objects rank higher on each side. Blue-circled objects are high ranking nodes on both the binary andmultiple value graphs. These nodes cover the highest part of the ranking. Figure 3.2 shows the network
visualization map of the blue-circled nodes. Red-circled objects rank higher only on the binary graph. We
consider these objects to have relations with a wide range of other objects. Figure 3.3 shows the network of redcircled nodes. These nodes have many links. Green-circled objects, on the other hand, rank higher only on themultiple value graph. We consider these objects repeatedly used. Figure 3.4 shows the network of green-circled
nodes. These nodes have thick links.

Figure 3.2 High ranking nodes on both binary and multiple value graphs.

Figure 3.3 High ranking nodes on binary graph.
Then there were the questionnaire results. Before the experiment, Person A answered that their personal
computer, dining chair, dining table, television, and washing machine are valuable to them. These objects arehigh ranking nodes on the binary and multiple value graphs, as shown in Figure 3.1. This means that we can findimportant objects in our daily lives by looking for nodes whose degree centrality is high. Moreover, we can findother important objects we could not anticipate before the experiment. For example, after the experiment,
Person A says in the questionnaire that he recognized the importance of the TV remote control. The TV remotecontrol is a node whose degree centrality is high only on the multiple value graph. We found that Person A
gradually found the importance of the TV remote control through repetitive usage.
We assumed that there are several important nodes whose degree centrality is not high. We can find them byvery carefully looking at the network visualization map. For example, as shown in Figure 3.5, we can see thatthe table pot is important for Person A’s life because Person A could not water down the spirit “Kyogetsu”
without table pot. The Kyogetsu and the cup are high-degree centrality nodes and these nodes that include table
pot formed a triangular structure on the network. Similarly, beer, linked to dining table, dining chair, and Person
A, is also important for Person A’s life. These items are unexpectedly important. The pot would not be
important in itself if Kyogetsu and the cup did not exist. Only when Person A drinks Kyogetsu out of the cup is
the pot used. We assume, as a result, there are important objects in our daily lives that are only important when
they are used with other things that have relation to them.

Figure 3.4 High ranking nodes on multiple value graph.

Figure 3.5 Important nodes whose degree centrality is not high.

3.2 TWO-PERSON NETWORK
We analyzed the networks of the Persons B and C concurrently. Persons B and C are a husband and wife. Figure
3.5 shows the change when we combine the graphs of these two people. When we define N as the overlapping
nodes and L as the overlapping links, N and L are as follows.
N = 40 + 29 - 60 = 9 (3.1)
L = 39 + 75 – 111 = 3 (3.2)
The results of the N and L values indicate that there are few parallel nodes and links between Persons B andperson C.
However, as shown in Figure 3.7, there are shifts in the nodes whose degree centrality is high, when looking
at only Person B or C and when looking at both B and C. In the rightmost table in Figure 3.7, the red nodes are
the high degree centrality nodes that can be seen only in Person B’s network, the blue nodes are the high degree
centrality nodes for only Person C’s network, and the green nodes are the high degree centrality nodes for both
Person B and Cs’ networks. It is natural that the green nodes are high degree nodes in the rightmost table,
because the degree of these nodes is counted by both people. The most significant thing, however, is the factthat important objects are different when we look from one and two person’s viewpoint. We can find important
shared objects by looking at the networks of B and C concurrently. Figure 3.8 shows the network visualization
map of Persons B and C. We plotted the red, blue, and green nodes on the map. The red nodes often exist near
the nodes of B and the blue nodes often exist near the nodes C. On the other hand, green nodes often exist
between the node of person B and C. We can easily find some of the shared nodes by looking at the nodes thatlink to both people on the network visualization map.

Figure 3.6 Combination of Persons B and C (Characteristic value of graph).

Figure 3.7 Combination of Persons B and C (Degree centrality of graph).

3.3 MULTI-PERSON NETWORK
We analyzed a network that consists of several people by looking at the visualization map. Figure 3.9 shows a
network obtained from the relations made by a family (Persons D, E, and F). Similar to the situation describedin Section 3.2, we can find the objects that the three people share. In this graph, there are several nodes that arelinked to both the nodes of D and E or the nodes of D and F. However, there are few nodes linked to both thenodes of E and F. This means that Persons E and F have less relation with each other than D and E or D and F inthis family. Actually, D is the father, E is the mother, and F is the son. Therefore, in this family, we consider thatthe son and his mother have very little relation with each other. Thus, we can guess the human relationship bylooking at a network of several people.

Figure 3.8 Network visualization map of people B and C.

Figure 3.9 Network visualization map of people D, E, and F.

4

DISCUSSION

4.1 MERITS AND DEMERITS OF THE EXPERIMENTThe biggest merit of the experiment is the fact that we can get significant data actually from real homes. It is
hard to set up RFID system which is efficient identification tool of objects [12] in the general household quickly.
Furthermore, while RFID tags get cheaper, it takes a lot of money if we need to append many tags. In contrast,
our experimental system depended on a cellular phone which is very familiar with the public. Therefore, we canget data not from testing site constructed only for the experiment but from general households.
However, there are some demerits and limitations of the experiment. First, it is difficult to collect all the
relations from the subject’s life. In our daily life, there are infinites numbers of actions and subjects. So we can’t
read barcode every time they act. Besides, it is impossible for subjects to experiment when they engaged insomething. Therefore, there is a loss of precision when we analyzing the relation network. On the other hand,
we consider that the experimental data is effective because subjects unconsciously enter relations which isparticularly important in their lives despite they can’t enter all relations. Second, it is troublesome to readbarcodes because it takes time to launch both an application and a barcode reader. Most subjects said that it was
troublesome to enter relations one by one manually in the questionnaire.
To solve this problem, for example, we can construct the much easier experimental system by using RFID
tags in the future. RFID is much effective because we can enter relations automatically [12] and get every datawhen subjects are active. Moreover, RFID have advantage in giving ID numbers to objects. The future global
use of RFID makes it pivotal to provide globally unique identification numbers such as EPC [13]. Besides, we
can deal with not only sensor data but also user-specific data [14] using RFID systems. For these reasons, RFID
is expected for extracting relations among humans and objects.

4.3 COLLECTING DATA VS. USER BENEFITS
We had an experiment for investigating relations among humans and objects by using the cellular phone system.
As the number of entered relations grows, users receive some benefits. There are four considerable benefits asfollows. The more relations users enter, the more accurate the outputs become.
(1) Users can recognize important objects in their daily lives by looking at the high degree centrality nodes.
In this case, the important objects means the objects used repeatedly.
(2) Users can recognize something important in their daily lives when it is used with other closely-related
things.
(3) Users can recognize the important shared objects by seeing the network of two persons concurrently.

(4) Users can guess human relationship by looking at a network visualization map of several people.

4.4 FUTURE WORK
Benefits we described in Section 4.3 are not everything. For example, there are two further researches we want
to try to find the methods of understanding our lifestyles. First, we extrapolate our lifestyles by looking at
attribute’s names of the objects. For example, a banana, an orange and an apple have the same attribute’s name“fruit”. Therefore, if a subject eats them every day, we can suggest that the subject likes to eat fruits. We have toorder attribute’s names to make such a prediction. In this case, the attribute’s names and the attribute value arebase on the Ontology researches [15]. It is also possible to combine the attribute’s names and web information
through Semantic Web [16] technology. Second, we extrapolate our lifestyles by looking at time-line of entering
the relations. It was difficult for us to looking at time-line because experimental data is not available when asubject is too busy to experiment. However, RFID system makes it possible. We can extrapolate life pattern [17].

5

CONCLUSION

We created and used a cellular phone application and succeeded in extracting the actual relations among humans
and objects. Then we found some ways of understanding our lifestyles by using network analysis. This
experiment is positioned as preliminary experiment and showed the fact that it is worthwhile to pay attention tothe objects we uses in our lives. It is important to see how objects link to the other objects or humans. We canextrapolate much accurately when we construct automated system and get more data in the future research.

6

ACKNOWLEDGEMENT

The authors appreciate for reviewer's comments. This work was supported by Japanese Grant-in-Aid for
Scientific Research (KAKENHI 17200010 & 19024066).

REFERENCES[1] H. Nakajima (2004). Technique of assisting human activity. Shinyo-sha.
[2] A. Yoshida (2006). Life log that can record everything in our life. IT media.
http://www.itmedia.co.jp/bizid/articles/0610/04/news067.html[3] K. Sato (2002). Seoul Style. National Museum of Ethnology.
http://htq.minpaku.ac.jp/databases/seoulstyle-db/house.jsp[4] Barabási Albert-László (2002). New Network Thinking (Shin Network Shiko). NHK-published.
[5] Travers, Jeffrey & Stanley Milgram (1969). “An Experimental Study of the Small World Problem”.
[6] Nieminen, Juhani (1973). “On the Centrality in a Directed Graph”. Social Science Research 2.
[7] Standardization Research Academy. QR code Story. Japanese Standards Association.
[8] Stanley Wasserman and Katherine Faust (1994). Social Network Analysis: Methods and Applications, New
York: Cambridge University Press.
[9] Y. Yasuda (2001). Practical Network Analysis (Jissen Network Bunseki). Shinyo-sha.
[10] Boissevain, J. (1974). Friends of friends: Networks, manipulators, and coalitions. Oxford:Blackwell.
[11] Freeman, Linton C. (1979) “Centrality in Social Networks : I A Conceptual Clarification”. Social
Networks 1.
[12] Want, R (2004) Rfid – a key to automating everything. Scientific American.
[13] Sergei Evdokimov, Benjamin Fabian, Oliver Gunther (2008) Multipolarity for the Object Naming Service.
First International Conference, IOT 2008.
[14] Kosuke Osaka, Jin Mitsugi, Osamu Nakamura, Jun Murai (2008) Generalized Handling of User-SpecificData in Networked RFID. First International Conference, IOT 2008.

[15] R. Mizoguchi (2006) Primer of Ontology Construction (Ontology Kochiku Nyumon). Ohmsha.
[16] Grigoris Antoniou, Frank van Harmelen (2005) A Semantic Web Primer.
[17] Takeshi Tanaka, Shunzo Yamashita, Kiyoshi Aiki, Hiroyuki Kuriyama, Kazuo Yano. (2008) Life
Microscope: Continuous daily-activity recording system with tiny wireless sensor. Networked Sensing
Systems, 2008. INSS 2008. 5th International Conference.