Sensor terminal ”Portable” for intelligent navigation of personal
mobility robots in informationally structured environment
Hiroyuki Yamada1,3 , Tomoki Hiramatsu1 , Masato Imai1 , Akihiro Kawamura2 , and Ryo Kurazume2
Abstract— This study proposes two kinds of systems forbuilding intelligent service robots, namely a sensor terminal
named ”Portable” for making various personal mobility robots
intelligent and a distributed sensor system for constructing
an informationally structured environment consisting of laser
range finders and active beacons. The sensor terminal Portable
is equipped with a laser range finder and a gyro. Two types of
personal mobility robot, namely standing type and wheelchair
type, are made intelligent by installing Portable to provide
navigation functions such as localization, obstacle detection,and path planning. The sensor system is mainly used toacquire position information about the personal mobility robots,
obstacles, and moving objects (e.g., people); this information isused by Portable for navigation. The obtained information istransmitted to the robot, allowing it to operate in a complicatedenvironment. The ability of the proposed system to navigate
personal mobility robots is verified in two real environments.

Omni-directional
mobiile robot

Wheelchair-type
mobility robot

Sensor terminal"Portable"
Standing-type
mobility robot

Fig. 1. Concept of sensor terminal ”Portable”. Portable can be installed invarious mobile robots to make them intelligent.

I. INTRODUCTION
The development of service robots, which support daily
activities, has expanded to prepare for an aging society.
Among such robots, the personal mobility vehicle (PMV),
which is expected to have high demand and contribute topreventive care, extends the physical ability of the elderly anddisabled people, giving them the opportunity to move theirown bodies more easily. The essential function of a PMV
is to take the user to the target location without harming
anyone. This is done by detecting obstacles, determining
PMV location using multiple kinds of sensors, and conducting path planning for complicated environments. The fastera PMV can move, the more frequently these processes mustbe performed. PMV control involves a lot of complicatedprocessing and requires high reliability. One solution is toperform these processes on a controller with high processing
capability mounted on the PMV, and another is to makethe environment intelligent. An intelligent environment isreferred to here as an informationally structured environment(ISE).
An ISE refers to a framework that supports mobility
robots. The sensors and processors are in the environment
rather than on the robots. An ISE acquires, analyzes, and
1 Hiroyuki Yamada, Tomoki Hiramatsu, and Masato Imai are with
Graduate School of Information Science and Electrical Engineering,
Kyushu University, Fukuoka 819-0395, Japan {yamada, hiramatsu,imai}@irvs.ait.kyushu-u.ac.jp
2 Akihiro Kawamura and Ryo Kurazume are with Faculty of Information
Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395,
Japan {kawamura, kurazume}@ait.kyushu-u.ac.jp
3 Hiroyuki Yamada is with Transportation System Control Research Department, Center for Technology Innovation - Controls, Hitachi, Ltd. Research and Development Group, Ibaraki 312-0034, Japanhiroyuki.yamada.qt@hitachi.com

manages environment information such as maps, obstacles,
and the locations of moving bodies, and gives this information to the robots. An ISE reduces the burden on robots
of acquiring various information about the surrounding environment, leaving robot processing capacity to other tasks.
This allows robots with a smaller processing capacity to be
used or the robots can perform more sophisticated tasks. In
addition, development resources can be allocated to robotspecific tasks, and it becomes easy to introduce a robot intoa real environment. The present authors previously developed
an ISE hardware platform named Big Sensor Box [1] and an
ISE software platform named ROS-TMS [1][2].
However, some information cannot be acquired using
sensors installed in the environment. For instance, the location of a moving object, which is essential environmental
information for a PMV, can be occluded by other moving
objects and the PMV itself, and thus not detected by sensors
in the environment. It is thus necessary for a PMV toacquire information about objects in its surroundings thatmay possibly move, even in an ISE.
The main task of a PMV is navigation. The main functions
necessary to achieve this are moving object detection, path
planning to the target location, and path following control.
Among these, detection and planning do not depend on the
PMV and are common technologies. With an ISE, a PMV
can be made intelligent by installing a general-purpose terminal that acquires information from environmental sensors,
detects some (but not all) moving objects, and plans a pathbased on this information (Fig. 1). A PMV requires somehardware and software to move itself along the planned
path. Furthermore, an ISE lowers the required processing
capability of the general-purpose terminal.

Wireless communication
antenna
Battery (inside)

Environmental Locations ofPMV, obstaclesdata
acquisition
Point cloud
Laser range
finder dataacquisition

Computing board(inside)

RTK-GNSS
data acqusition

Laser range finder

Rotary encoder
data acquisition

Fig. 2.

Portable.

Environment
map storage
Initial andtarget locations
input

The present study develops obstacle detection and path
planning functions for various kinds of PMV operating inan ISE, and makes two PMVs intelligent using small sensor
terminals. These PMVs are evaluated in an ISE that uses
laser range finders (LRFs) and active beacons.
II. R ELATED WORKThe goal of this study is to realize a robot that acts
effectively in a real environment and supports daily activities.
We focus on two scenarios for the robot: supporting the
movement of people who need nursing care in an indoor
living space and guiding visitors to the desired location in
an outdoor theme park. The experimental environments werethus a room that simulates an actual living space and thepremises of an actual theme park.
One feature of real environments is the occurrence of congestion, such as that caused by pedestrians. Methods for dealing with such congestion have been reported [3][4][5][6][7].
Kümmerle et al. proposed the SLAM system, which accurately creates large-scale urban area maps; SLAM wasable to navigate over a distance of more than 3 km using
LRFs and GPS in a crowded urban environment [3]. In
order to achieve long-term operation even in cases where
the data from two-dimensional LRFs are insufficient in busy
situations, Pérez et al. extended Monte Carlo localization
by integrating image information [4]. Jafari et al. proposeda method for people detection and tracking with a low
processing load by effectively utilizing depth information
obtained using RGB-D sensors [5].
Although satellite signals (e.g., those from GPS) are
important when navigating a robot outdoors, in urban areas,such signals are often obstructed by buildings and trees.
In such areas, for robust navigation, localization is often
achieved by matching precise maps with data from laser
scanners or vision sensors [8]. Schwesinger et al. demonstrated autonomous navigation over a distance of more than
12 km that was achieved by creating a large-scale highresolution map and detecting three-dimensional (3D) landmarks and obstacles using 3D LIDAR [8]. Map information
is important for robots in a real environment, and thus
map updating and maintenance methods for large-scale realenvironments have been proposed [9][10][11]. Pomerleau
et al. mapped dynamically changing environments, and byidentifying dynamic objects using a 3D laser scanner, extracted static geometry, allowing long-term maintenance withaccurate updating of the map [9].

Obstacle
detection

Localization

Latitude,longitude

Obstaclelocation

PMV
location

Rotational
speed
Map
Initial, target
locations

Generic functions

Fig. 3.

Motionplanning

Pathplanning

Targetspeed

Path

Actuator
control

PMV-specific function

Logical architecture of Portable.

Some studies have attempted to make robots run outdoors
using existing published maps, under the assumption thatthe cost of creating and maintaining large-scale maps is prohibitive [12][13]. When using such two-dimensional maps,
an alternative scan matching method is required because
there is no map information that can be used for matching.
The present study proposes an approach that makes thedevelopment and implementation of PMVs easier. Existing
techniques for autonomous navigation are implemented on asensor terminal, which can be installed to easily make various
robots intelligent. Furthermore, the processing required by
the sensor terminal is made as simple as possible by placing
sensors in the environment.
III. D EVELOPMENT OF SMALL SENSOR TERMINALS ANDINTELLIGENTIZATION OF VARIOUS ROBOTS

A. Small sensor unit Portable
We developed Portable, a small sensor terminal that canbe mounted on various robots to easily make them intelligent. Portable consists of an LRF (UST-20LX, Hokuyo),
an inertial measurement unit (myAHRS+, Hardkernel), abattery, a wireless communication system, and a computing
board (ODROID-XU4, Hardkernel). Portable is designed tobe compact so that it can be installed on any robot. Its
dimensions are 114 × 100 × 240 mm and its weight isapproximately 800 g. Portable is shown in Fig. 2.
Figure 3 shows the logical architecture of Portable.
Portable has generic functions that do not depend on the typeof PMV and a PMV-specific function. The generic functions
include acquisition of sensor data from the LRF, rotary encoder, and RTK-GNSS (Real-time kinematic global navigation satellite system) module directly connected to Portable,acquisition of environment information, map storage, input of
initial/target locations, obstacle detection, localization, path
planning, and motion planning. The PMV-specific function
is for controlling the actuators of the PMV and thus depends
on PMV hardware. Because there was a constraint that the
hardware on the PMV hardware cannot be changed, PMVspecific function is implemented in Portable. However, it
is also possible to have PMV-specific function performedby the PMV. Ideally, the interface between Portable and
the PMV, such as that for target speed, would be definedand PMV developers would implement the actuator control

GNSS
antenna
Portable

Right wheel
Left wheel
Rotary encoders

PortableFig. 4.

Emergency
stop switch

Standing type PMV and rotary encoders installed on wheels.

function. By operating according to the instructions fromPortable, a PMV can be guided to the target location without
hitting obstacles. PMV developers would no longer need todevelop basic guidance functions, giving them time to focus
on developing more unique functions that would increasevalue for users.
Each function is developed based on ROS. We use an
occupancy grid map for the environmental map, a particle
filter for obstacle detection, the extended Kalman filter for
localization, and the Dijkstra method based on a costmap
created from the environmental map and obstacle locations
for path planning. For a given path, the target velocity (translational and rotational speeds) of the PMV is determinedusing the dynamic window approach.
B. Intelligentization of standing-type PMV by Portable
In this study, two kinds of PMV are made intelligent,
namely standing type and wheelchair type. We targeted
the Ninebot by Segway miniPRO (Segway-Ninebot) as thestanding-type PMV (Fig. 4). In this PMV, the user can induce
translational motion by moving their own center of gravity
forward and backward, and can induce rotational motion
by tilting the handle held by the feet to the left or right.
We attached rotary encoders to the left and right wheelsof the miniPRO (Fig. 4), the Portable on the front of thehandle (Fig. 4), and a microcontroller for controlling therotation of the miniPRO at the lower part of the handle.
The target rotational speed is sent from the Portable to themicrocontroller to control rotation. Due to the characteristics
of a standing-type PMV, there is a safety concern if the robot
moves forward or backward against the user’s intention,and thus the user directly manipulates the PMV to control
translational speed. Therefore, in the Portable, the target
rotational speeds obtained from the generic functions are
ignored and the target rotational speed is computed by the
PMV-specific function based on the translational speed set
by the user. Only the target rotational speed is sent to the
PMV.
C. Intelligentization of wheelchair-type PMV by Portable
We targeted ChairBot (Sustainable Robotics) as thewheelchair-type PMV (Fig. 5). ChairBot is a modified electric wheelchair (MC 3000U, Suzuki) that controls the left
and right wheels via digital signals. In the actuator control,
which is a PMV-specific function, Portable calculates thetarget rotational speeds of the left and right wheels from thetarget translational and rotational speeds, and transmits theFig. 5.

Wheelchair-type PMV with Portable and GNSS antenna installed.

appropriate commands to the PMV. For use in an outdoor environment, we installed an RTK-GNSS module and a GNSS
antenna. Using a base station placed at a known location, theRTK method eliminates errors caused by ionospheric delay.
Excellent position measurement results can thus be obtained.
In this study, RTK-GNSS is used to improve robustness
by integrating different types of sensor data for localization(wheel odometry and LRF data).
IV. I NFORMATIONAL STRUCTURING OF REALENVIRONMENT

A. Development of P2 -sen and building ISE
We developed the Petit Petit Sensor Box (P2 -sen) (Fig. 7),
which consists of an LRF and a computing board. A P2 -sen
can track moving objects. The locations of moving objects
in an environment can be determined by arranging multiple
P2 -sen devices and integrating their data.
We constructed an ISE that can track moving objects using
the LRF of a P2 -sen. The following tracking algorithm wasused:1) Apply background subtraction based on a Gaussian
distribution to the scan data obtained from the LRF,
and extract data that may contain moving objects.
2) Clustering scan data by determining adjacent data as
the same object
3) Apply a particle filter to each cluster and estimate thelocations of moving objects
The above process was performed by a control PC to which
multiple P2 -sen devices were connected. The locations ofmoving objects obtained from P2 -sen were sent to Portablefrom the control PC and used for navigation control of the
PMV.
B. Building ISE using active beacons
Active beacons (Pozyx, Pozyx Labs), each comprising abase station and a mobile station, were used. By arranging
multiple base stations at known locations in the environment
and measuring the distance between a mobile station and thebase stations via wireless communication, an active beacon
can estimate the location of the mobile station based on the
distance from multiple known locations with an accuracyof about 10 cm. The mobile station was installed in the
PMV and transmitted its location to Portable. In Portable,
the Kalman filter was applied to the position output by the
beacon to improve stability.

1

Obstacle 2

Initial location
Plannedpath

Obstacle 2

Obstacle 1
Starting location

Target location

Kitchen2

Bed
Obstacle 1

Obstacle 2

Table

Target location

Floor plan

Obstacle 1
Re-planned path

Fig. 6.
Indoor experimental environment (Big Sensor Box)[1] andobstacles.
1

2

3

LRF
1

Computingboard

2

No coverFig. 7. P2 -sen, environmental map, and locations of two P2 -sen devices
in experimental environment.

4

V. E XPERIMENTS
A. Navigation experiment of standing-type PMV using
Portable and P2 -sen
We constructed an ISE in a real environment and conducted experiments to navigate a PMV using Portable. Thissection presents the results of constructing an ISE using P2 sen in an indoor environment that reproduces an actual living
space and navigating the standing-type PMV. Figure 6 showsthe experimental environment, named Big Sensor Box [1],the obstacles placed in the environment, and locations of the
obstacles. The ISE was constructed by placing two P2 -sen
devices in the environment, as shown in Fig. 7. The control
PC was collected to these devices and processed their data,
and then wirelessly transmitted the results to Portable on
the PMV. In this experiment, Portable estimated the PMV
location by integrating the locations obtained by the rotary
encoders and that transmitted from the control PC using theextended Kalman filter. Portable also detected obstacles using
its own LRF. The goal of the experiment was to navigate the
PMV from the initial location to the target location; however,
obstacle 1 was at a height that made it not detected by P2 sen.
Figure 8 shows the navigation results. First, the PMV
is at the initial location and there are no obstacles in the1detection range of the LRF mounted on Portable (Fig. 8 ⃝).
However, Portable knows the position of obstacle 2, obtained
from the ISE, and plans a path to avoid it. Next, Portabledetects obstacle 1 and re-plans the path to avoid the two2
obstacles (Fig. 8 ⃝).
The PMV then moves to the target
location according to the planned path.
Figure 9 shows the results of localization obtained using
P2 -sen and the rotary encoders and using only the rotary
encoders. As shown in Fig. 9, these localization results are
not smooth. This is considered to be due to the insufficientnumber of particles used in the particle filter for moving

5Fig. 8.
ISE.

Navigation of standing-type PMV using P2 -sen and Portable in

Wheel odometry only

P2-sen and wheel odometry

Fig. 9.

Localization results of standing-type PMV in ISE.

object tracking by P2 -sen. However, this did not cause
practical problems in navigating the PMV. The accuracy of
localization was improved by using information from P2 -sencompared to that obtained using only wheel odometry.
B. Navigation experiment of standing-type PMV using
Portable and active beacons
We conducted a PMV navigation experiment in a real
outdoor environment to simulate visitor guidance at a theme
park. In this experiment, we constructed an ISE with eightactive beacons placed at the entrance open space of thetheme park. The goal was for the PMV to navigate fromthe entrance gate to the first building. Portable performed
localization using wheel odometry and the active beacons.

Results of localization
Red : using active beacon onlyBlue : using active beacon andwheel odometry

Target location

Jumps data

Bridge10 m

Position of base stations

Base stations
Starting location

Fig. 10.
１

Experimental environment in theme park entrance area.

Localization results of standing-type PMV in ISE.
Results of localization
Red : using active beacon only
Light blue : using wheel odometry onlyBlue : using active beacon and wheel odometry

2

Target location

Base stations

Fig. 12.

Localized locationof PMV

Path planned by Portable

10 mStanding-type PMV

3

4

Fig. 13.

5

6

7

8

Target location

Localization results of standing-type PMV up to the bridge.

not possible to reach the bridge due to error accumulation.
Integrating information from the active beacons increased
accuracy. However, using only active beacons, the information was acquired intermittently or the values jumped insome situations. There seems to have been a problem with
the installation locations of the base stations. Various noise
sources in the environment, such as metal objects and radiosources, should also be considered. In this environment, thelocations and intervals of the base stations greatly affected
measurement accuracy.
C. Navigation experiment of wheelchair-type PMV using
Portable with RTK-GNSS

Fig. 11.
Navigation of standing-type PMV using active beacons andPortable in ISE.

Figure 10 shows the experimental environment and thenavigation path of the PMV. There is a bridge with a widthof about 4.5 m along the path. An accuracy of 20 to 30 cm
is required to pass over this bridge safely. The environment
map used in this experiment was precisely constructed usinga laser scanner at a time when there were no visitors.
In this experiment, PMV passed over the narrow bridge
and was able to navigate to the target location (Fig.11).
Figure 12 shows the actual movement path of the PMV andthe locations of the base stations of the active beacons. Part ofthe path is meandering because the speed set by the user andthe rotational speed calculated by Portable did not match. It
is thus necessary to review the algorithm used for calculatingrotation speed.
Figure 13 shows the results of localization from theentrance gate to the front of the bridge in the cases where
only wheel odometry is used, only the active beacon areused, and both are used. Using only wheel odometry, it was

We conducted a navigation experiment of the wheelchairtype PMV in an outdoor space to evaluate the localization
by Portable using rotary encoders and RTK-GNSS without
using environmental sensors. The experiment was conductedin an area with an open view of the sky that was sufficient
to obtain GNSS satellite signals. We set up the RTK basestation in the environment to receive correction data. Figure
14 shows the experimental environment. In this experiment,a map with virtual walls (red lines in Fig. 14) was used.
An obstacle was placed in the center of the environment, asshown in Fig. 15. Portable planned a path from the starting
location to the target location that avoided the virtual walls
and the obstacle.

Obstacle

Virtual wallFig. 14. Left: Experiment environment for wheelchair-type PMV. Right:
Map with virtual walls

1

Target location

Navigation in an ISE was also possible with the wheelchairtype PMV.
This study focused on evaluating the effectiveness of
Portable and thus congestion was not considered in theexperiments. Performance in a congested environment will
be evaluated in future experiments.

Planned path
Starting location2

Obstacle

ACKNOWLEDGMENT

Re-planned path3

The authors would like to thank Y. Kawauchi, T. Nakamura, T. Inoue, and O. Toyama at Living Robot Inc. fortechnical assistance with the experiments. This research was
supported by the Japan Science and Technology Agency(JST) through its ”Center of Innovation Science and Technology based Radical Innovation and Entrepreneurship Program(COI Program)” and was supported in part by a Grant-in-Aid
for Scientific Research (A) (26249029).

Re-planned path

4

R EFERENCES

Fig. 15.

Navigation experiment of wheelchair-type PMV using Portable.

RTK-GNSS and wheel odometry

Wheel odometry only

Fig. 16.

Trajectories of wheelchair-type PMV using RTK-GNSS.

Figure 15 shows the navigation results. First, the PMV
was at the starting location. Since there were no obstaclesin the detection area of the mounted LRF, the shortest target
1
path that avoided the virtual walls was planned (Fig. 15 ⃝).
Next, the obstacle was detected by the LRF, and the target2
path was re-planned to avoid it (Fig. 15 ⃝).
The PMV thenmoved to the target location according to the planned path3 ∼ ⃝).
4(Fig. 15 ⃝Figure 16 shows the results of localization obtained using
the RTK-GNSS module and the rotary encoders and using
only the rotary encoders. The two sets of results are similar
since ChairBot has a relatively small slip between the groundand the tires and its travel distance was short. However, byintegrating RTK-GNSS, it is possible to estimate the target
location more accurately. RTK-GNSS is thus beneficial for
long-distance movement.
VI. CONCLUSION
This study proposed the Portable framework that provides
functions required for an ISE and makes it easy to makevarious PMVs intelligent. The effectiveness of Portable wasconfirmed by basic experiments using two types of PMV,
namely standing type and wheelchair type. In addition, we
confirmed that a standing-type PMV equipped with Portable
can navigate a real environment that was made into an ISE.

[1] R. Kurazume, Y. Pyo, K. Nakashima, T. Tsuji, and A. Kawamura,“Feasibility study of iort platform ”big sensor box”,” in Proc. IEEE
International Conference on on Robotics and Automation (ICRA2017),
pp. 3664–3671, 2017.
[2] Y. Pyo, K. Nakashima, S. Kuwahata, R. Kurazume, T. Tsuji, K. Morooka, and T. Hasegawa, “Service robot system with an informationallystructured environmen,” Robotics and Autonomous Systems, vol. 74,
no. Part A, pp. 148–165, 2015.
[3] R. Kümmerle, M. Ruhnke, B. Steder, C. Stachniss, and W. Burgard,“A navigation system for robots operating in crowded urban environments,” in 2013 IEEE International Conference on Robotics andAutomation, pp. 3225–3232, May 2013.
[4] J. Pérez, F. Caballero, and L. Merino, “Integration of monte carlo
localization and place recognition for reliable long-term robot localization,” in 2014 IEEE International Conference on Autonomous Robot
Systems and Competitions (ICARSC), pp. 85–91, May 2014.
[5] O. H. Jafari, D. Mitzel, and B. Leibe, “Real-time rgb-d based people
detection and tracking for mobile robots and head-worn cameras,”in 2014 IEEE International Conference on Robotics and Automation(ICRA), pp. 5636–5643, May 2014.
[6] Y. Morales, E. Takeuchi, A. Carballo, W. Tokunaga, H. Kuniyoshi,
A. Aburadani, A. Hirosawa, Y. Nagasaka, Y. Suzuki, and T. Tsubouchi,“1km autonomous robot navigation on outdoor pedestrian paths ”running the tsukuba challenge 2007”;,” in 2008 IEEE/RSJ International
Conference on Intelligent Robots and Systems, pp. 219–225, Sept
2008.
[7] G. Ferrer, A. Garrell, and A. Sanfeliu, “Social-aware robot navigationin urban environments,” in 2013 European Conference on Mobile
Robots, pp. 331–336, Sept 2013.
[8] Dylan, Schwesinger, Armon, Shariati, Corey, Montella, and J. Spletzer,“A smart wheelchair ecosystem for autonomous navigation in urban
environments,” Autonomous Robots, vol. 41, pp. 519–538, Mar 2017.
[9] F. Pomerleau, P. Krüsi, F. Colas, P. Furgale, and R. Siegwart, “Longterm 3d map maintenance in dynamic environments,” in 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3712–
3719, May 2014.
[10] B. Suger, G. D. Tipaldi, L. Spinello, and W. Burgard, “An approach
to solving large-scale slam problems with a small memory footprint,”
in 2014 IEEE International Conference on Robotics and Automation(ICRA), pp. 3632–3637, May 2014.
[11] F. Ferri, M. Gianni, M. Menna, and F. Pirri, “Dynamic obstacles
detection and 3d map updating,” in 2015 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), pp. 5694–5699,
Sept 2015.
[12] P. Ruchti, B. Steder, M. Ruhnke, and W. Burgard, “Localization on
openstreetmap data using a 3d laser scanner,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 5260–
5265, May 2015.
[13] K. Irie, M. Sugiyama, and M. Tomono, “A dependence maximization
approach towards street map-based localization,” in 2015 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS),
pp. 3721–3728, Sept 2015.