Volume 3, Issue 2 June 2018

Special Contribution

Innovative management system for infrastructure byutilization of 3D point cloud data based on GIS platform
Dr. Hiroshi DOBASHI
Director, Maintenance and Traffic Management Department
The Metropolitan Expressway Co., Ltd.
E-mail: h.dobashi118@shutoko.jp
In recent years, much of the infrastructure that was constructed in the high-economic-growth period in Japan has begun
to age. At the same time, the number of maintenance engineers for this infrastructure has become insufficient due to adecrease in working-age population as well as budget deficits. With respect to the large change in the social environment
as described above, the advanced maintenance system of the infrastructure should be developed and required to improve
efficiency and accuracy of maintenance works on roads and other structures, based on the platform utilizing GIS(Geographic Information System) and the point cloud data. By using this innovative system, a variety of data related tostructures such as drawings, inspection results and repairs or reinforcing records are managed easily through a digital
map on the GIS. This developed system also offers the function to create 2-D and 3-D CAD data or FEM models, andsimulations of inspections and repair works, utilization of the 3D- point cloud data measured by an MMS (Mobile Mapping
System). In addition, the deformation of structures detected by using the 3D-point cloud data has been examined and theaccuracy of deformation verified. This paper describes the innovative management system based on the GIS platform tosupport maintenance work on infrastructures using the point cloud data.
Keywords: infrastructure, maintenance, point cloud data, GIS, MMS (Mobile Mapping System)
1.
Introduction
In recent years, much of the infrastructure such as roads, bridges and tunnels that was constructed in the high economic
growth period in Japan has begun to age. In addition, the number of inspection and maintenance engineers for thisinfrastructure will become insufficient based on the estimation of future working-age population by the Ministry ofInternal Affairs and Communications, as well as the issue of budget shortages. In order to solve the above issues, aninnovative management system for infrastructure based on the GIS platform has been developed.
The computerized system named as “InfraDoctor” which is a core technology of the innovative management system, will
be able to improve efficiency and accuracy of maintenance works on roads and structures, utilizing GIS and the point
cloud data. By using this system, a variety of data related to structures such as drawings, inspection results and repair orreinforcing records can be handled on a digital map of GIS. This system also has the ability to create 2-D and 3-D CAD data
and FEM models for structural analyses and deterioration prediction analyses, and to simulate the inspection and repair
works with utilization of the point cloud data measured by an MMS. Furthermore, inspection work can be carried out
efficiently with accuracy by detecting the change in displacement or damage of structures using point cloud datameasured over a certain period. In this paper, various functions of the “InfraDoctor” are introduced and innovative
management system for infrastructure named “i-DREAMs” (intelligence-Dynamic Revolution for Asset Management
system) which has been implemented by the Metropolitan Expressway Co., Ltd. since August 2017, is described.
2. Outline of InfraDoctor
The “InfraDoctor” has been developed to improve efficiency and accuracy of maintenance works on structures such as
roads, bridges, tunnels, buildings and so on, utilizing the GIS platform and 3D-point cloud data. This computerized system
consists of the following elements.
2.1 GIS platform
The “InfraDoctor” is based on the GIS platform. In the portal of the system, a digital map is shown in the browser on thecomputer screen. Through this digital map on the GIS platform, any data such as various ledgers can be searched foreasily and quickly.
3Volume 3, Issue 2 June 2018

The “maintenance ledger” has the information concerning the number, dimension and material of structures andappendages. The “inspection history ledger” has inspection-result records. The “repair history ledger” has records ofrepair and reinforcement works carried out based on the inspection results. All data in these ledgers has the coordinates
and is connected to the digital map. This data can also be searched by using various keywords such as the name, type,material and so on, of the structures. Using other information connected to the searched data, they can be sorted easilyand quickly for various purposes, such as confirming the quantity and locations of same-type structures or members,comparing between inspection results or repair work history, for example.
The point cloud data and the all-around video data which is described later, are also managed on the GIS platform andcan be shown on the computer screen by specifying the location from the digital map on the browser shown in Figure 1.

Figure 1 GIS browser with all-around video image

2.2 All-around video
All-around video images are captured using a vehicle called an“MMS” shown in Figure 2, while travelling. All-around video isrecorded as still images at every 4m of interval when the vehicle
with an MMS is travelling at 60 kilometers per hour. The
resolution of an image is about 12 mega pixels.
Using this all-around video, the condition of structures and theirsurroundings can be confirmed on a computer screen without
an actual investigation at the site. Because the all-around video
enables objects on a road to be seen from not only the directionthat the MMS is moving ahead but also any direction from thepoint of the MMS, the condition of the site can be grasped
quickly and precisely at the office. Even the back of a sign board
can be looked at, as seen in Figure 1, for instance.
Figure 2 Mobile Mapping System (MMS)
2.3 Three-dimensional point cloud data
Two laser scanners are installed on the MMS described above, and three-dimensional point cloud data are obtained while
the MMS is travelling at a speed of 40km/hr. These laser scanners emit a million of the laser beams for a second andcapture the position of an object which reflects the laser beams from the scanner. A laser beam can usually reach an
object about 800 meters away from the MMS. The MMS also equipped with a GNSS antenna and an IMU, which stands
for “Inertial Measurement Unit”, and the error of measured coordinates due to vibration or rotation of the vehicle arerevised using those devices. In general, the degree of an error for coordinates of an object is about 10 centimeters forabsolute value of position with respect to the surveying coordinate and a few millimeters for relative errors regarding the4Volume 3, Issue 2 June 2018

position of each point recorded in one measurement if the target is an expressway road bridge with a height of about
10m from the surface road. The degree of an error, however, is changed depending on the distance between the target
object and the laser scanner, velocity of the MMS, the condition of road surface which affects the vibration of MMS,
existing tall buildings or structures which interrupt or reflect radio waves for GPS antennae from satellites, and so on.
When a structure is modelled in three dimensions using the point cloud data, the vehicle with the MMS should be driven
several times around the target structures to cover its blind spot. Therefore, registration is required to integrate thosepoint cloud data groups measured separately. It has been confirmed that an error of a few tens of millimeters may occur
during a registration work. In order to solve this problem, a program which can carry out the registration work while
correcting the error automatically, is now under development and being prepared for commercialization.
The MMS is also equipped with some HD digital cameras and the color data of the image captured by those cameras canbe exported to each point in the point cloud data. As a result, the point cloud data look like a video image as seen in
Figure 3 and this makes it easier and more precise to recognize the shape and positional relationship of structures.
When the structures such as bridges are spanning over a wide river or existing in a mountainous area without roads
available for the MMS, all the necessary point cloud data of a target structure cannot be captured. In these cases, aportable scanner fixed on a tripod, a human carrying type scanner or drone equipped with a scanner can substitute andthe point cloud data captured by those different scanners can be integrated with that from the MMS.

Figure 3 GIS browser with point cloud data
3.

APPLICATIONS OF POINT CLOUD DATA

3.1 Detection of damage and displacement of structures
Since viaducts are above those intersections with heavy traffic or railroads, a remote inspection technique to detectdamages where a close visual inspection is difficult is highly recommended, especially in urban areas to protect people
from injury and cars from damages due to spalling and delamination of surface concrete. Therefore, a technology todetect spalling and delamination on the surface of concrete structures has been developed using the point cloud data,by taking the difference of positions between the measured points and a “reference surface” which is an average planeof the concrete surface created from the point cloud data. In addition, it is expected that the propagation of the damage
will be also detected by comparing the point cloud data measured in time series.
In a trial measurement, some delamination on the concrete surface with 3 or 4 millimeters of protrusion were detected
when measured by a scanner on the MMS about 10 meters away shown in Figure 4. In this trial, the point cloud datameasured in one measurement was used to eliminate the error induced by the registration. A further study is stillnecessary to obtain higher accuracy with this remote inspection method. Moreover, it is also possible to finddisplacement, deformation or lack of the appendages, structural members or the structure itself when subjected to a5Volume 3, Issue 2 June 2018

large external force, such as a large earthquake or tsunami, by comparing the point cloud data measured after the eventwith the one formerly recorded.
3.2 Three dimensional measurements

Figure 4 Detection of damage on concrete surface
It is possible to measure the dimension of and distance between structures in the 3D space created by the point clouddata on the GIS browser of the “InfraDoctor”. Using this function, it is possible to confirm the positional relationshipbetween structures where the traffic regulation is needed for measurement, and where inspectors cannot easily reach,or different organizations own. All the existing structures that are not described on the same drawing can be integrated
and the construction space can be checked three-dimensionally for inspection, repair and reinforcement work, as well as
both clearance between structures and the traffic clearance in this 3D space shown in Figure 5.
This digital platform also has a function to create a contour map by providing a color to each point sequentially different
depending on its position in the height direction of the pavement for instance. From this contour map, the inclination
and rutting depth on the pavement surface will be measured as shown in Figure 6, and the location of a ponding placeor flooding range can be confirmed. As a result, these data show a reference for determining whether the road surface isrequired to be repaved or not and are stored in the inspection database corresponding to the GIS map information.

Figure 5 Clearance check in the space bypoint cloud data

Figure 6 Contour of rutting depth on
pavement surface.

3.3 Creation of two-dimensional CAD drawing
By extracting the point cloud data in each direction of the cross-section, it is possible to create a CAD drawing in twodimensions. In conventional applications, a drawing is created by manually tracing the outline of the cross-section of thestructure on the point cloud data. On the other hand, the “InfraDoctor” can automatically generate outlines of the crosssection of the structure and complete a drawing much easier and faster than the conventional method. Since each pointof the point cloud data has a value in x, y, z-coordinates, the dimension and the distance of structures can be obtained atany section and a general drawing of a structure with the dimensions, as shown in Figure 7, can be generated semiautomatically in a short time.
6Volume 3, Issue 2 June 2018

In Japan, there are about 700,000 bridges with a length longer than 2 meters. However, it seems that the original drawingsof approximately 300,000 old bridges among them may have been lost. This current situation is surely hindering
maintenance work for bridges. In addition, since the appendages and the reinforcing members are installed on a structure
after the construction is completed, these drawings are not often well conserved. In these cases, creating drawings of thecurrent state of the structure using this computerized system will greatly contribute to the repair or retrofit design andefficient maintenance work.

Front view

Side view
Superstructure

Figure 7 2D-CAD drawing created from 3D-point cloud data (bridge pier)3.4 Creation of three-dimensional CAD model
Three-dimensional CAD model can be also
created in a short time by automaticallygenerating the planar or curved surface of astructure from the 3D-point cloud data, using
the “InfraDoctor” as shown in Figure 8. Thismodel will be used for three-dimensional
structural designs and analyses. In addition, athree-dimensional FEA (Finite Element
Analysis) model can be created from thethree-dimensional CAD data by meshingautomatically. Currently the development of
three dimensional FEA is remarkable and it
would be possible to estimate residual or
ultimate strength, and degree of damage ofstructures. Combining with the inspection
results which are used as the boundary
condition in the FEA analysis, evaluation of thestructure with high-precision can be carried
Figure 8 3D CAD model created from 3D-point cloud data
out. Therefore, this function will be a great
help to analyze the strength of a structure
effectively and to carry out a design of repair, retrofit or reinforcement, using the obtained structures in the current
situation with the damage or deformation caused by deterioration, natural disasters or other external loadings.
4. Smart Infrastructure Management System, i-DREAMs
To achieve the effective maintenance work with accuracy, various data made or obtained at the stage of survey, designand construction such as design statements, drawings, material data, methods or conditions of construction work,
inspection results including construction errors, repair records before operation, are very much required. Combining theConstruction Information Modeling (referred to as CIM) with “InfraDoctor”, the initial data required for the maintenancework will be transferred continuously from the design phase to the maintenance phase shown in Figure 9. As a result,comprehensive evaluation of deterioration and damage of the structures will be carried out based on the accumulated
data of the platform. In addition, it is possible to integrate the database of inspection, repair and reinforcement into the7Volume 3, Issue 2 June 2018use of ICT and IoT technologies on this platform.

i®

Smart Infrastructure Management System（ – DREAMs ）Design andConstruction

DIM,CIM(Design & Construction Information Management)

Lifecycle Management of Infrastructure- Computer aided construction using digital data.
- Application of 3D CAD drawing and construction simulation.
- Record of initial condition of structures such as material, construction condition,
quality control, and etc..

MIM(Maintenance Information Management)
Digital data management system based on GIS (InfraDoctor® )- Management of CIM data, inspection and repair record, 3D point cloud data, allaround video, etc. on GIS platform.
- Support inspection work by measuring displacement or condition changes.
- Support inspection planning work by inspection simulation or drawings.
- Create structural models and input data for performance evaluation analysis
using ledger data and 3D point cloud data considering construction conditions
and inspection data.

Maintenance

Timely repair and reinforcement
Repair/Reinforcement- Quick implementation of performance evaluation, deterioration evaluation andrepair/reinforcement design using analysis model and input data from the“InfraDoctor”.
- Achievement of timely repair and reinforcement.

Figure 9 Future development for smart infrastructure management system
Making use of this initial data from CIM and the boundary condition obtained by inspection, a deterioration prediction,
deterioration diagnosis and performance evaluation of structures with high-precision can be determined by making useof this initial data from CIM and the boundary condition obtained by inspection, and even by the structural analysis with
FEM models generated by “InfraDoctor”. Consequently, it is expected that proper repair and reinforcement at an
appropriate time will be carried out, and efficient and smart maintenance work on infrastructure can be achieved. Theseexperiences of infrastructure management can be applied not only to maintenance work but also to better design andconstruction. Furthermore, by processing the big data such as integrated data of traffic volume, axle weight andenvironmental conditions as well as inspection data, sensing data with the use of IoT (Internet of Things) utilizing AI
engine, the innovative platform of i-DREAMs (intelligence Dynamic Revolution of Asset Management system) makes it
possible to evaluate the damage and performance of structures efficiently with high accuracy shown in Figure 10.
SurveydesignDIM（Design Information Management）
Transfer data of survey and design

CIM（Construction Information Management）
Transfer and integrate initial data

Construction record

Material data

MIM（Maintenance Information Management）
Evaluation and analysis of soundness,diagnosis and estimation of deterioration
FEM
analysis
Estimation ofdeterioration

GIS Platform
Map
information

Damage estimation by AI

Structural
data

Inspection
and repair

 Completiondrawing Ledgers

 Inspection DB Repair DB

3D-Point cloud data

Monitoring To detect
deformation andcreate drawings

 Monitoring DBof the structures

Evaluationof repair
DB server

Surrounding
information

Accident anddisaster prevention

 Video image
 Pictures

 Hazard map Accident DB

ｾﾝｻｰ®
InfraDoctor

IoT
by 3D-point
cloud databy eddy current
flaw detection test

Repair planning andimplementation

by inspection
robotsby near-infrared
spectroscopy

by hammering
analysisby digital
image analysis

by monitoringbridges with sensor

Automatic
extraction

Figure 10 Innovative infrastructure management system (i-DREAMs)
8

Volume 3, Issue 2 June 2018

5. Summary
Maintenance work of infrastructure has become increasingly important due to ageing, deterioration and severe use ofstructures. Periodical close visual inspection of road structures with every five years was legalized four years ago by theJapanese Government. However, there still exist areas where inspection is very difficult, and the number of inspectors isstill limited due to a decrease in population, especially the working age population. Therefore, development of theadvanced and innovative inspection technologies is required for accurate and efficient inspection as well as for educationand training of inspectors.
For this reason, “InfraDoctor” has been developed as an advanced database system based on the GIS platform to support
the maintenance work on road and structures, integrating various ledger data, the inspection data, and repair andreinforcement records of structures, all-around video and point cloud data. By utilization of this platform, the operationand maintenance work will be more efficient and laborsaving without sacrificing accuracy. In addition, the seamless
infrastructure management system such as i-DREAMs combining with “InfraDoctor”, CIM and numerical analyses such as
FEM, is a more integrated system which enables efficient maintenance of infrastructure. As a result, timely andappropriate repair and reinforcement of structures following deterioration diagnosis and prediction connecting with theinitial data, the boundary condition such as inspection results and FEM analyses will be achieved.
The group companies of the Metropolitan Expressway Co., Ltd. are carrying out a trial operation of this platform to verify
applicability and accuracy for the maintenance work in practical use. With the results of trial operation, the system will
be further improved and implemented in the actual maintenance work. It is expected that this innovative GIS platform
will contribute to efficient management of infrastructure not only in Japan but also in various countries around the worldin the near future.
References:[1] DOBASHI, Hiroshi 2016. Infrastructure Management System Using ICT for Industry 4.0, Kensetsu IT Guide 2016,
Economic Research Association, Tokyo.
[2] DOBASHI, Hiroshi 2016. Seamless Infrastructure Management System Using ICT. Bridge and Foundation (Kyoryo toKiso), Vol.50. Kensetsu Tosyo, Tokyo.
[3] NAGATA, Y., et al., 2016. Maintenance of Infrastructures Using GIS and Point Cloud Data. 2nd IRF Asia Regional
Congress and Exhibition, Proceedings, Kuala Lumpur, Malaysia.

A brief CV of Dr. Hiroshi DOBASHI
Dr. DOBASHI is a director, Maintenance and Traffic Management Department of theMetropolitan Expressway Company limited, Japan. He is currently in charge of
maintenance work and traffic management service of the Metropolitan Expressway. He
has developed an innovative infrastructure management system to achievecomprehensive management of infrastructure. Especially he specializes in the field oftunnels. He has developed the shield tunnel enlargement method, the first of its kind in
the world.

9