Feasibility Study of IoRT Platform “Big Sensor Box”
Ryo Kurazume1 , Yoonseok Pyo2 , Kazuto Nakashima3 , Akihiro Kawamura1 , and Tokuo Tsuji4

Abstract— This paper proposes new software and hardware
platforms named ROS-TMS and Big Sensor Box, respectively,
for an informationally structured environment. We started thedevelopment of a management system for an informationallystructured environment named Town Management System(TMS) in the Robot Town Project in 2005. Since then we havebeen continuing our efforts to improve performance and toenhance TMS functions. Recently, we launched a new version of
TMS named ROS-TMS, which resolves some critical problems
in TMS by adopting the Robot Operating System (ROS) andutilizing the high scalability and numerous resources of ROS.
In this paper, we first discuss the structure of a software
platform for the informationally structured environment anddescribe in detail our latest system, ROS-TMS version 4.0.
Next, we introduce a hardware platform for the informationallystructured environment named Big Sensor Box, in which avariety of sensors are embedded and service robots are operated
according to the structured information under the managementof ROS-TMS. Robot service experiments including a fetch-andgive task and autonomous control of a wheelchair robot are also
conducted in Big Sensor Box.

I. INTRODUCTION
An informationally structured environment (or an intelligent space) is a key technology for realizing a service robot
in a daily-life environment in the near future. Instead ofdeveloping an intelligent robot equipped with a number of
sensors, the environment itself is designed to be intelligent bystructuring the information of the robot and its surroundings
and providing the required information to the service robot
on time and on demand. To do so, various sensors areembedded in the environment, including walls, ceilings, furniture, or objects, to acquire a variety of information, such as
positions of humans, objects, and robots, or the current statusof environments. Acquired information is stored in a cloud
database, analyzed in cyber space, and provided to robots
and users. A number of studies related to informationallystructured environments have been published so far. These
include “Robotic Room” [1] and “Intelligent Space” [2] at
Tokyo University, “Smart Room” at MIT MediaLab [3],“Intelligent Room” at AILab [4], “Aware Home” at Georgia
Tech. [5], and “Wabot House” at Waseda University [6].
1 Ryo Kurazume and Akihiro Kawamura are with Faculty of Information
Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395,
Japan {kurazume, kawamura}@ait.kyushu-u.ac.jp
2 Yoonseok Pyo is with ROBOTIS CO., LTD., Seoul 153-787, Korea
passionvirus@gmail.com
3 Kazuto Nakashima is with Graduate School of Information Scienceand Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japank nakashima@irvs.ait.kyushu-u.ac.jp
4 Tokuo
Tsuji
is
with
FacultyofMechanicalEngineering,
Kanazawa
University,
Ishikawa
920-1192,Japan
tokuo-tsuji@se.kanazawa-u.ac.jp

Many of these are still studied actively in laboratories [7][8] [9] [10] [11].
The current authors also started to develop a town-scale
informationally structured environment named “Robot Town
Project” in 2005 and have been developing a software
platform named Town Management System (TMS). In 2015
we launched the latest version named ROS-TMS 4.0, which
adopted the Robot Operating System (ROS) [12] as the
middleware of TMS.
The idea of using a variety of sensors embedded in theenvironment and adapting the behavior of the system tothe current situation is also called “ambient intelligence”[13]. The synergy effect of ambient intelligence and robots
was studied in “CompanionAble project” [14] in the FP7
project. Additionally, in Horizon 2020, some projects related
to “personalising health and care (PHC)” have been planned
to utilize service robots for health-care purposes.
Although our research is closely related to ambient intelligence or smart homes [15], [16], and more recently, Internetof Things (IoT) [17] or a cyber-physical system (CPS), oneof the characteristics of our approach is that we focus on thedevelopment of not only several sensor systems but also a
total framework of the informationally structured system. We
are also developing a software platform with high flexibility
and expandability named “ROS-TMS”.
II. S OFTWARE P LATFORM ROS-TMS
The CPS is a fundamental social system for the next IT
generation. In a CPS, actual situations, events, and behaviors
in the real world are measured by distributed sensors or
obtained as open/social data. Optimum solutions are derived
in the cyber world based on huge computer resources. Then,the real world is managed and controlled according to the
optimum solutions given in the cyber world to realize an
ideal real world. A “smart grid”, which optimizes generation,transmission, and storage of electricity, is a representative
example of a CPS. After the National Science Foundation(NSF) identified the CPS as a key research area in 2006[18], this concept has been attracting much attention and anumber of studies have been presented so far. To realize a
CPS, IoT (Internet of Things) is one of the key technologies
for gathering real-world information. In addition, to realize
designed plans in the real world, robot technology (RT) isalso quite important. Therefore, IoT and RT are indispensablefor driving the CPS, and we refer to these two key technologies as Internet of Robot and Things (IoRT), as shown in
Fig. 1.
ROS-TMS [19] is a core software platform for IoRT. We
released the latest version, ROS-TMS 4.0, in September 2015

Cyber world
01001011110101101010010100
10111101110101001010010111
10101101010010100101111010
11010100101001011110101101
01010100101111010110101001
01001011110101101010010100
10111101011010100110010111

Real worldPlan
RT (Robot
Technology)

IoRT

Optimize

Control

IoT (Internetof Things)
MeasureFig. 1.

Cyber Physical System (CPS) and IoRT

and proposed ROS-TMS in [20]. We briefly summarize ROSTMS 4.0 in this section.
ROS-TMS 4.0 consists of more than 150 informationallystructured nodes classified into several sensor processing
modules, such as laser range finders or IC card readers, robot
planning modules, task planners and schedulers, human interface modules, and database modules. The functions required
for supporting life services by service robots are designed
and implemented as nodes. These nodes are connected andreconfigured flexibly according to the desired service tasks.
Fig. 2 shows the total structure of ROS-TMS. The summaryof each module is as follows:• TMS DB: nodes for communicating with a database(DB Server and File Server).
• TMS UR: nodes for communicating with users. Requests received from users are sent to TMS TS.
• TMS TS: nodes for planning service tasks performedby robots.
• TMS SD: nodes for controlling sensors such as laser
range finders, cameras, or RFID tag readers.
• TMS SS: nodes for integrating sensor data.
• TMS SA: nodes for estimating the status of the environment.
• TMS RP: nodes for planning robot motions.
• TMS RC: nodes for executing robot motions.
User

TabletTMS_UR: User Request
TMS_TS: Task Scheduler

ServerBoard PC

TMS_SA
State Analyzer

TMS_RP
TMS_SS
Sensor System Robot Planning

TMS_SD
Sensor Driver

TMS_RC
Robot Controller

SensorRobot

TMS_DB: Database

Service Target

execution of each subtask and sends execution commands to
the Robot Controller (TMS RC) module with proper timing.
By modularizing various sensor systems and processing
algorithms based on ROS, reusability and flexibility of modules are improved and the efficiency of system development
is enhanced. For example, we replaced the database fromMySQL in ROS-TMS ver. 3.4.2 to MongoDB in ver. 4.0.
However, most of the modules other than the TMS DB
module (interface for the database) were not influenced at all.
In addition, by dividing total processes into small nodes, we
can choose the proper computer resources for each process.
These resources can range from single-board computers suchas Raspberry Pi or the Next Unit of Computing (NUC)
for low-level sensor drivers, to workstations for high-level
recognition or planning processes.
Furthermore, it is quite easy to install newly developed
processing nodes to the system stably to carry out thefunctions of the ROS, and a number of open resources andpackages available worldwide can be easily utilized in ROSTMS.
Fig. 3 shows an example of the node structure in a room.
Pressure sensors and RFID tag readers are installed in acabinet and a refrigerator, and the positions and names ofobjects in the cabinet and refrigerator are recorded in the
database (TMS SD) though the sensor nodes (TMS SD).
Laser range finders are installed on the floor, and range data
are sent from the sensor nodes (TMS SD) to the processing
nodes (TMS SA) to recognize the objects and track their
movements. The motions of robots, humans, and objects
are estimated according to the tracking results. A 9-axis
acceleration sensor, an angular acceleration sensor, and amagnetic field sensor are installed in the furniture, such as awagon or a wheelchair that can move freely. These sensors
are used to track each position by combining the tracking
information obtained by the laser range finders. RGB-D
cameras (Kinect) are attached to the robots and the walls
to recognize objects or human behavior.
6WDWHDQDO\]HQRGH

DBServer

8VHU

706'DWDEDVH

8VHU

5RERWVWDWH
2YHUORDG
FKHFNHU:ULWHQRGH

File
Server

5RRP

2EMHFWLQFDELQHW

Sensor & Robot

3ODQQLQJQRGH

8VHU

5RERW

)XUQLWXUH

3HUVRQ:DJRQ:KHHOFKDLU

7UDFNLQJQRGH

Fig. 2.

ROS-TMS architecture

Among them, User Request (TMS UR), Sensor Driver(TMS SD), and Robot Controller (TMS RC) modules areinterface modules between users, sensors, and robots. The
Task Scheduler (TMS TS) module receives a sequence ofsubtasks from the Database (TMS DB) module according to
the user’s requests. Task Scheduler is designed to achieve
the desired task and to schedule the execution timings ofsubtasks. The Robot Planner (TMS RP) module manages the,GHQWLI\QRGH

2EMHFW,PDJHQRGH'DWD
PHVVDJH&ODVVLI\QRGH

6HQVRUGULYH

6HQVRUGULYH

6HQVRUGULYH

6HQVRUGULYH

6HQVRUGULYH/RDGFHOO

5)7DJVHQVRU/5)VHQVRU

$[LVVHQVRU

5*%'FDPHUD&RQWUROPHVVDJH'DWD
3URFHVV
6HQVRU

Fig. 3.

Processing nodes of TMS

Hierarchical modular structures similar to ROS-TMS canbe seen in other robot control architectures and smart housecontrollers [21] [22] [23] [24] [25] [26] [27] [28] [29]. For
example, Fong et al. [26] proposed a middleware called“Human-Robot Interaction Operating System” (HRI/OS) for
collaboration of humans and robots, and adopted a similar
hierarchical structure consisting of distributed processing
modules. Techniques that divide the total process into several
small processes as modules, which are connected freely
according to the required tasks, are popular ways to improve
the flexibility and the efficiency in the development of largescale systems. The characteristics of the proposed ROS-TMS
are as follows: the proposed system is based on ROS as a
middleware, the flexibility and the expandability are quite
high, and a variety of open resources in the world can beutilized by adopting ROS as a middleware.
III. H ARDWARE PLATFORM B IG S ENSOR B OX
We developed a hardware platform for an informationallystructured environment named Big Sensor Box for demonstrating the performance of the software platform ROS-TMS
in the daily life environment. The environment, shown Fig.
4, consists of a bedroom, a dining room, and a kitchen.

Fig. 4.

Vicon Bonita

MarkerFig. 5. Optical tracking system (Vicon Bonita) and pose measurement ofa wagon2) Distributed RGB-D camera system: We install nine
RGB-D cameras (Kinect v2) in Big Sensor Box and capture
the behavior of a human in the room. The position of eachcamera is calibrated by using a large checker pattern andthe optical tracking system mentioned above. Depth images
taken by multiple RGB-D cameras are processed by Kinect
for Windows SDK 2.0, and the positions of the body andjoints of the human are estimated individually in each depth
image. Finally, redundant information is fused according to
the reliability, which is proportional to the relative angle
between the camera’s optical axis and body orientation. Only
the joint information with the highest reliability is selected as
the joint position and stored in the TMS DB module. Fig. 6
shows the estimated behavior of a human in the room using
multiple RGB-D cameras.

Big Sensor Box
Kinect v2

In this platform, several sensors, robots, intelligent furniture, and electric appliances are installed. These include thefollowing.
Sensors•

Optical tracking system (Vicon Bonita)
Distributed RGB-D camera system (Kinect, Xtion)• Intelligent electric appliances•Service robots•

Humanoid-type service robot (SmartPal V)
Mobile robot (KXP, Kobuki)• Wheelchair robot (Mimamoru-kun)•

We introduce some sensors, intelligent furniture, and service
robots in Big Sensor Box in the following sections.
A. Sensors
1) Optical tracking system: In Big Sensor Box, 18 infrared cameras (Bonita, Vicon Motion Systems) are installed
and the three-dimensional positions of infrared markers aretracked. By attaching markers on robots or furniture, as
shown in Fig. 5, not only the human motion but also
the motion of a robot and the position of furniture arecontinuously measured with a high level of accuracy thatis under one millimeter.

Fig. 6.

Motion estimation using RGB-D cameras (Kinect v2)

3) Intelligent furniture: In the cabinet and the refrigerator
in Big Sensor Box, several load cells and RFID tag readers
are installed, and the positions and names of objects can bedetected as shown in Fig. 7.
The bottom of each object in Big Sensor Box has an
attached RFID tag. The position of the object placed on thecabinet or the refrigerator is calculated by using the weight
distribution measured by the load cells, and the name ofthe object is read by the RFID tag reader. Measured object
information is automatically stored in the database throughthe ROS DB module. Thus, by accessing the database, we
can check objects in the refrigerator from outside the room
(for example, while shopping in a market). In addition, AR
markers are attached on each item of furniture, such as therefrigerator. By capturing images by using smart phones
or smart glasses, we can know the objects inside without
opening the door. Fig. 7 shows how we check the registeredobjects in the intelligent refrigerator through a smart phone.

1

Vicon markers

2
RGB-D camera Retroreflective markers

Joystick

Smartphone3

4

Object information

Fig. 7. Registration of objects in intelligent refrigerator. 1. Select refrigerator, 2. Plastic bottle is detected, 3. Ceramic bottle is detected, 4. Informationof ceramic bottle is displayed.

B. Service robots
1) Humanoid-type service robot: Fig. 8 shows ahumanoid-type service robot (SmartPal V, Yaskawa Electric). On top of the robot, an omnidirectional laser scanner(Velodyne HDL-32E) and an RGB-D camera (Xtion, ASUS)
are attached, and so the robot can avoid collisions with anenvironment, human, and object in real time, as shown in
Fig. 8.
1

2

(a)

(b)

Fig. 9.

(c)Service robot (KXP) and wheelchair robot

Furthermore, the position of the wheelchair robot in Big
Sensor Box is tracked by the optical tracking system described above and stored in the database. If the user tapsa tablet to select the destination on a displayed map, thewheelchair robot can move to the destination automatically.
C. Simulators
Several simulators (Choreonoid (AIST), RViz, andGazebo) are currently available for Big Sensor Box. Fig.
10 shows an example of the Choreonoid simulator. Fig. 11
shows an example of Big Sensor Box reproduced by RViz
and Gazebo.

Omni-directional LRF
Velodyne HDL-32E
RGB-D Camera
Xion, ASUS

3

4

Fig. 8. 1. Humanoid-type service robot (SmartPal V, Yaskawa Electric
Corporation). 2-4. The robot avoids to hit a human hand using Velodyne
HDL-32e, thus tries to keep holding an object by avoiding a human hand.

2) Mobile robots: Fig. 9(a) shows a small service robot
with a manipulator (Katana, Neuronics), an RGB-D camera(Xtion, ASUS), and a mobile platform (Pioneer3-AT, MobileRobots) named KXP. Katana is a 5-DoF manipulator witha maximum capacity of 0.5 Kg and the ability to grasp an
object on a floor by using a 1-DoF gripper hand. Fig. 9(b)is the “kobuki”, which is a standard platform in ROS. An
RGB-D camera is mounted on the top of the kobuki.
3) Wheelchair robot: We developed a wheelchair robot
for transportation assistance and vital data acquisition, as
shown in Fig. 9(c). The user wears vital sensors such as anelectroencephalography (EEG) sensor (MindWave Mobile,
NeuroSky) and a heart rate sensor (T31C transmitter, POLAR). The acquired vital data are transmitted to ROS-TMS
via Wireless LAN/Bluetooth and stored in the database. A
doctor or a nurse can access these data remotely by usinga smartphone or a tablet to monitor the health status of theuser. With this system, ROS-TMS can manage not only the
information of robots or objects but also vital data of thehuman.

Fig. 10.

3D model of ”Big Sensor Box” (Choreonoid)

RVizFig. 11.

Gazebo3D model of ”Big Sensor Box” (RViz and Gazebo)

IV. ROBOT S ERVICE E XPERIMENTS U SING ROS-TMS
AND B IG S ENSOR B OX
We carried out robot service experiments in Big Sensor
Box. The aim of these experiments is to confirm that thedeveloped software platform ROS-TMS 4.0 and the hardware
platform Big Sensor Box have competent ability and performance capable of providing some typical robot services.
Especially in this paper, we focus on two service tasks indaily life, fetch and give task and automatic transportation
task by wheelchair robot. Both of them are frequent activities

of daily life in care homes. More concretely, we confirm thefollowing.
1) Can robots with different structures provide the same
fetch and give task by referring to the database?
2) Can the wheelchair robot execute the automatic transportation task? Can the user move to the desired
location automatically by tapping the map on a tablet?
In the fetch and give task, the user does not specify theexact location of the object. Instead, the user just gives thename of the object to ROS-TMS, and ROS-TMS specifies
the position of the object by referring to the database inwhich the information of the objects, the robot, and the user
is registered beforehand. The ROS-TMS plans a secure robot
trajectory toward the object and a feasible grasping motion,then executes “move”, “grasp”, “move”, and “give” subtasks
according to the task execution machine, SMACH [30].
A. Fetch and give tasks by two types of service robots
1) Execution procedures for fetch and give task: We
conducted experiments for the fetch and give task by using
two types of service robots. In this task, the robot moves
to the position of the object desired by the user, grasps theobject, approaches the user, and passes the object to the user.
All the procedures are planned and controlled by ROSTMS. First, the task command received through the TMS UR
module is divided into several subtasks, “move”, ” grasp”,
and “give”, according to the task description in the database.
Each subtask is sent to the TMS RP module sequentially by
SMACH, shown on the left side of Fig. 12. The TMS RP
module produces detailed motion instructions, such as the
trajectory of movements or grasping motions by interpolating
the object position, the cabinet position in which the object
is placed, the current robot status, and the user position
stored in TMS DB. The motion instructions produced in the
TMS RP module are sent to the RMS RC module, translated
to the actual motion patterns by considering the structureof each robot, and executed by each robot. Due to thisarchitecture, the same motion instruction sets (a sequenceof subtasks) for a desired task can be repeatedly used, evenby robots with different structures. In addition, same subtasks
can be reused for other tasks repeatedly.
Here we explain the procedures for task executions inmore detail. Tasks, subtasks, their parameters, and execution
orders are stored in the database as a sequence of characters,
as shown in Table I. For example, get object task (ID=8001),
which is a fetch and give task, is composed of the move
subtask (ID=9001), the grasp subtask (ID=9002), and thegive subtask (ID=9003). In addition, each subtask is calledwith an object ID ($oid), user ID ($uid), and robot ID ($rid).
For the description of subtasks, we adopted the inverse Polishnotation and defined a delimiter with “(space)”, and serial
and parallel connections with “+” and “|”, respectively. Eachobject ID ($oid) or user ID ($uid) is translated to actual
coordinates of the positions stored in the database during thetask execution. When tasks such as “get object” are called,
the robot ID ($rid) must be set for a robot (2020: SmartPal,

2003: KXP, etc.) according to the user requests to performthe service tasks.
TABLE ITASK DESCRIPTIONS IN TMS DB
Type
Task
Task

ID
8001
8002

Name
get object
patrol
Type
Subtask
Subtask
Subtask
Subtask
Subtask

Code
9001$oid 9002$oid + 9003$uid +9001$rid 9006$oid 9007$oid | +
ID
9001
9002
9003
9006
9007

Name
move
grasp
give
sensing
random walk

In the case that $rid = 2002 (SmartPalV), $oid = 7001
(snack), and $uid = 1001 (the user), the input scripts about
the get object task for SMACH issued by TMS TS are as
follows:
smach.StateMachine.add(’move’,ServiceState(’rp_cmd’,rp_cmd,request = rp_cmdRequest(9001,2002,[7001])),transitions={’succeeded’:’grasp’})smach.StateMachine.add(’grasp’,ServiceState(’rp_cmd’,rp_cmd,request = rp_cmdRequest(9002,2002,[7001])),transitions={’succeeded’:’give’})smach.StateMachine.add(’give’,
ServiceState(’rp_cmd’,rp_cmd,request = rp_cmdRequest(9003,2002,[1001])),transitions={’succeeded’:’succeeded’,aborted’:’aborted’})2) Experimental results: Before the experiments using
the service robots, we confirmed that the actions produced
by ROS-TMS were safely executable by the Choreonoid
simulator. Fig. 12 shows the sequence of actions planned
for the fetch and give task explained in Section IV-A.1.
Fig. 12 shows the planned actions for each subtask. The
dotted lines on the floor show the desired trajectories planned
by TMS RP. These trajectories are so-called Voronoi edges,
which are the most distant lines from obstacles, and the robot
moves on these lines safely. The diagram on the left side ofeach figure shows the current execution status of subtasks
managed by SMACH.
Next, we conducted actual experiments using two typesof service robots, SmartPAL V and KXP. Fig. 13 shows that
SmartPAL V and KXP are performing the desired service
task with the same task descriptions issued by TMS RP. The
structure differences between the two robots are negated by
TMS RC. We can see that the user can receive the desired
object placed anywhere in Big Sensor Box if the object
position is registered in ROS-TMS using the intelligent
cabinet/refrigerator or the optical tracking system. If wecall the same task description by changing robot/user/object
IDs, ROS-TMS translates these IDs to the actual positions

by referring to the database and performs the desired task
appropriately.
Dotted lines

1 Move2 Graspwithout collisions with the environments or objects in theenvironments.
Through the experiments, we confirmed that the fetch
and give task can be performed with different types ofservice robots, sensors, user interfaces, and simulators by
the software platform ROS-TMS and the hardware platform
Big Sensor Box.
SmartPal,
bring me a snack

3 GiveFig. 12. Fetch and carry task performed by service robot (Simulation). 1.
Move to a shelf, 2. Grasp an object, 3. Move and give an object.

1

ObjectMove
Robot

Person1

ObjectPerson

Move

Robot

2

Grasp

2

Grasp

3

Give

3

Give

SmartPAL V

KXPFig. 13. Fetch and carry task performed by service robots (SmartPAL V
and KXP). 1. Move to a shelf, 2. Grasp an object, 3. Move and give an
object.

Fig. 14 shows the demonstration using the voice input
node in TMS UI. The user asks the system by voice, forexample, “SmartPal, bring me a snack”. Then the TMS UItransforms the voice input to the corresponding task description and parameters, and TMS RP and TMS RC execute thefetch and give task automatically through SmartPAL V. The
left figures of Fig. 14 show the robot motion and the rightfigures show the planned motions displayed in RViz.
The robot position is measured by the markers attached on
the body and the optical tracking system. The user position
is measured by using markers on glasses worn by the user.
The positions of objects and furniture are registered in the
database beforehand. The error and the sampling period of
the optical tracking system are less than 1 mm and 100 Hz,
respectively, and thus the robot can move and grasp an object

Fig. 14. Fetch and carry task performed by service robot in Big Sensor
Box according to voice input (Simulation and experiment)

Fig. 15 shows an example of a “patrol task” performedby the kobuki. The patrol task is a service task that moves
randomly and searches for a fallen person in the room. Thistask consists of subtasks “Move” (move to the center ofthe room), “Random walk” (move randomly), and “Sensing”(search for a fallen person by using an RGB-D camera),and includes the same subtask (Move) used in the fetch andgive task. The subtasks “Random walk” and “Sensing” areexecuted in parallel by SMACH. When a large object witha size similar to that of a human is detected, the task isterminated with the abort instruction, the robot stops at thecurrent position, and beeps to notify the abnormal detection.
This example suggests that the same subtasks can be utilized
repeatedly in several tasks, because the proposed system has
high reusability and the flexibility.
Random walk + Sensing

Fig. 15.

Patrol task performed by the kobuki

B. Automatic transportation task using a wheelchair robot
One of the key applications of the proposed platforms
ROS-TMS and Big Sensor Box is elderly care service ina care home. Therefore, we conducted experiments for an
automatic transportation task using the wheelchair robot wedeveloped. The aim of this experiment is to confirm thefollowing points:
1) A patient can move automatically and safely to thedesired location.

2) Vital data of a patient can be measured and stored in
the database so that a care giver can monitor the healthcondition.

Heart-rate Meditation Attention

The scenario in this experiment is as follows: a patientmoves from the sofa to the table, takes a plastic bottle, andmoves to the bed. More concretely, this scenario consists ofthe following sequences.
1) A patient calls the wheelchair robot by tapping thetablet, then the wheelchair robot comes to the sofa
automatically.
2) The patient transfers to the wheelchair robot.
3) The patient moves to the desk manually by using a
joystick on the armrest.
4) After taking a plastic bottle, the patient taps the bed
image displayed on the tablet and the wheelchair robot
transports him to the bed automatically.
Figs. 16 and 17 show the experimental results and thetarget position of the wheelchair robot displayed on thetablet, respectively. Fig. 18 shows the vital data, which areheart rate (hr), meditation level (med), and attention level(att) estimated by the electroencephalography (EEG) sensor.
These data are displayed on the Choreonoid simulator.
Through the experiments, it is confirmed that a patient
can move anywhere in Big Sensor Box automatically andmanually by the wheelchair robot while the vital data ismeasured, recorded, and monitored by the care giver.
1

2

3

4

Wheelchair robot

Fig. 18.

Vital data displayed on Choreonoid

V. CONCLUSIONSIn this paper, we introduced the software platform ROSTMS for an informationally structured environment. In addition, we proposed the hardware platform Big Sensor Box
for the informationally structured environment. Moreover,the hierarchical structure of ROS-TMS and the execution
procedure of service tasks in Big Sensor Box were explainedwith some examples.
Two kinds of service tasks, fetch and give task andautomatic transportation task by wheelchair robot, which areactivities occurring frequently in daily life, were demonstrated by using ROS-TMS and Big Sensor Box.
One of the main contributions of this study is that we built
a house-sized IoT environment based on ROS and showed thefeasibility of ROS-based sensor/robot control architectures
for realizing a CPS.
We plan to apply the proposed ROS-TMS and Big Sensor
Box to elderly care homes in the near future and to confirm
the practicality of the proposed platforms.
ACKNOWLEDGMENT

Fig. 16.
Service experiment performed by wheelchair robot. 1. Calla wheelchair robot by a tablet, 2. Get on a wheelchair robot, 3. Movemanually, 4. Move automatiaclly.

1

3

Current position

Target position

This research is supported by The Japan Science andTechnology Agency (JST) through its “Center of Innovation Science and Technology based Radical Innovation and
Entrepreneurship Program (COI Program).”
R EFERENCES

2

4

Fig. 17. GUI images for service experiment. 1. Call a wheelchair robot
by a tablet, 2. Get on a wheelchair robot, 3. Move manually, 4. Moveautomatiaclly.

[1] T. Sato, Y. Nishida, and H. Mizoguchi, “Robotic room: Symbiosis withhuman through behavior media,” Robotics and Autonomous Systems,
vol. 18, no. 1-2, pp. 185–194, 1996.
[2] J.-H. Lee, N. Ando, and H. Hashimoto, “Design policy of intelligent
space,” in Proceedings on 1999 IEEE International Conference on
Systems, Man, and Cybernetics, vol. 3, pp. 1077–1082 vol.3, 1999.
[3] A. P. Pentland, “Smart rooms,” Scientific American, vol. 274, no. 4,
pp. 54–62, 1996.
[4] R. A. Brooks, “The intelligent room project,” in Proceedings of the
2nd International Conference on Cognitive Technology (CT ’97), CT’97, (Washington, DC, USA), pp. 271–, IEEE Computer Society, 1997.
[5] J. A. Kientz, S. N. Patel, B. Jones, E. Price, E. D. Mynatt, andG. D. Abowd, “The georgia tech aware home,” in CHI ’08 Extended
Abstracts on Human Factors in Computing Systems, CHI EA ’08,
(New York, NY, USA), pp. 3675–3680, ACM, 2008.
[6] S. Sugano and Y. Shirai, “Robot design and environment design waseda robot-house project,” in Proceedings of International Joint
Conference SICE-ICASE, 2006, pp. I–31–I–34, Oct 2006.

[7] H. Noguchi, T. Mori, and T. Sato, “Automatic generation and connection of program components based on rdf sensor description in network
middleware,” in Proceedings of IEEE/RSJ International Conference onIntelligent Robots and Systems, pp. 2008–2014, 2006.
[8] K.-H. Park, Z. Bien, J.-J. Lee, B. K. Kim, J.-T. Lim, J.-O. Kim, H. Lee,D. H. Stefanov, D.-J. Kim, J.-W. Jung, et al., “Robotic smart house to
assist people with movement disabilities,” Autonomous Robots, vol. 22,
no. 2, pp. 183–198, 2007.
[9] Y. Kato, T. Izui, Y. Tsuchiya, M. Narita, M. Ueki, Y. Murakawa, andK. Okabayashi, “Rsi-cloud for integrating robot services with internet
services,” in Proceedings of IECON 2011 - 37th Annual Conference
on IEEE Industrial Electronics Society, pp. 2158–2163, 2011.
[10] H. Gross, C. Schroeter, S. Mueller, M. Volkhardt, E. Einhorn, A. Bley,
C. Martin, T. Langner, and M. Merten, “I’ll keep an eye on you: Home
robot companion for elderly people with cognitive impairment,” inProceedings of IEEE International Conference on Systems, Man, andCybernetics, pp. 2481–2488, 2011.
[11] M. Tenorth, A. Perzylo, R. Lafrenz, and M. Beetz, “The roboearth
language: Representing and exchanging knowledge about actions,objects, and environments,” in Proceedings of IEEE International
Conference on on Robotics and Automation, pp. 1284–1289, 2012.
[12] http://www.ros.org/.
[13] E. Aarts and R. Wichert, Ambient intelligence. Springer, 2009.
[14] http://www.companionable.net/.
[15] M. Chan, D. Esteve, C. Escriba, and E. Campo, “A review of smart
homes -present state and future challenges,” Computer Methods andPrograms in Biomedicine, vol. 91, no. 1, pp. 55 – 81, 2008.
[16] D. Stefanov, Z. Bien, and W.-C. Bang, “The smart house for older
persons and persons with physical disabilities: structure, technology
arrangements, and perspectives,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 12, pp. 228–250, June 2004.
[17] K. Ashton, “That ’internet of things’ thing,” RFiD Journal, vol. 22,
no. 7, pp. 97–114, 2009.
[18] NFS-07-504
https://www.nsf.gov/pubs/2007/
nsf07504/nsf07504.htm.
[19] https://github.com/irvs/ros_tms/wiki/.
[20] Y. Pyo, K. Nakashima, S. Kuwahata, R. Kurazume, T. Tsuji, K. Morooka, and T. Hasegawa, “Service robot system with an informationallystructured environment,” Robotics and Autonomous Systems, vol. 74,
no. Part A, pp. 148–165, 2015.
[21] R. G. Simmons, “Structured control for autonomous robots,” IEEE
Transactions on Robotics and Automation, vol. 10, no. 1, pp. 34–43,
1994.
[22] R. Simmons, R. Goodwin, K. Z. Haigh, S. Koenig, and J. O’Sullivan,“A layered architecture for office delivery robots,” in Proceedings ofthe first international conference on Autonomous agents, pp. 245–252,
ACM, 1997.
[23] R. G. Simmons, R. Goodwin, K. Z. Haigh, S. Koenig, J. O’Sullivan,
and M. M. Veloso, “Xavier: Experience with a layered robot architecture,” ACM Sigart Bulletin, vol. 8, no. 1-4, pp. 22–33, 1997.
[24] R. Alami, R. Chatila, S. Fleury, M. Ghallab, and F. Ingrand, “Anarchitecture for autonomy,” The International Journal of Robotics
Research, vol. 17, no. 4, pp. 315–337, 1998.
[25] R. Alami, S. Fleury, M. Herrb, F. Ingrand, and F. Robert, “Multi-robot
cooperation in the martha project,” Robotics & Automation Magazine,
IEEE, vol. 5, no. 1, pp. 36–47, 1998.
[26] T. Fong, C. Kunz, L. M. Hiatt, and M. Bugajska, “The humanrobot interaction operating system,” in Proceedings of the 1st ACM
SIGCHI/SIGART conference on Human-robot interaction, pp. 41–48,
ACM, 2006.
[27] J.-H. Kim, I.-B. Jeong, I.-W. Park, and K.-H. Lee, “Multi-layerarchitecture of ubiquitous robot system for integrated services,” International Journal of Social Robotics, vol. 1, no. 1, pp. 19–28, 2009.
[28] C.-L. Wu, C.-F. Liao, and L.-C. Fu, “Service-oriented smart-homearchitecture based on osgi and mobile-agent technology,” IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications andReviews, vol. 37, pp. 193–205, March 2007.
[29] S. Das, D. Cook, A. Battacharya, I. Heierman, E.O., and T.-Y. Lin,
“The role of prediction algorithms in the mavhome smart homearchitecture,” Wireless Communications, IEEE, vol. 9, pp. 77–84, Dec
2002.
[30] J. Bohren and S. Cousins, “The smach high-level executive [rosnews],” IEEE Robotics & Automation Magazine, vol. 17, no. 4, pp. 18–
20, 2010.