The 8th International Conference on
Structural Health Monitoring of Intelligent Infrastructure
Brisbane, Australia | 5-8 December 2017

Development of Bridge Inspection Method without Temporary Scaffolding by
Using Optical Measurement Techniques
Keisuke KIMOTO 1, Kohei YAMAGUCHI 2, Toshihiro OKUMATSU 2,
Taiki KAWAMURA 2.Hiroshi MATSUDA 2

1

Keisoku Research Consultant CO., Japan, Email: kimoto@krcnet.co.jp 2 Nagasaki University, Japan

Abstract
This paper shows the development of inspecting method about bridge without temporary scaffold,using and applying the optical measurement techniques about Unmanned Aerial Vehicle andStructure from Motion. In late years UAV and SfM progress innovatively, they are becoming
common techniques. SfM is a technology for building a 3D model of the target structure only images
taken from multiview. By using the UAV, it is possible to acquire an image from a view point which
can not be confirmed normally, it can be more effective inspection.
Thereupon authors verified their techniques at the several bridges, and summarized the measurement
range, photography method, measurement precision and investigation time for future use.
1. Introduction
Japan has approximately 700,000 bridges with lengths of 2 m or more, including about 500,000 witha length of less than 15 m. In 2014, the Ministry of Land, Infrastructure, Transport and Tourism(MLIT) imposed an obligation to conduct periodic inspections of all of those bridges at a frequency
of once in 5 years by proximity visual inspection. However, the reality is that adequate inspection
cannot be performed because many of those bridges are managed by municipal governments which
are faced with financial problems and shortages of technical personnel accompanying low birth rates,
an aging population and population declines. As an additional problem, the percentage of bridges (2m or more) with ages exceeding 50 years since construction was 20% in 2016, but this is expected toincrease rapidly to 44% in 10 years. Thus, the development of more efficient bridge inspection
techniques has become an urgent issue. Moreover, in proximity visual inspections by theconventional techniques, it is necessary to approach the structure closely, but this requires a bridge
inspection vehicle or lift-type vehicle or temporary scaffolding and special work by rope access, andtraffic restrictions must be imposed during such work (Fig. 1). In this case, reduction of inspection
costs/labor is an issue.
Therefore, the authors considered
the possibility of labor-saving andhigher efficiency in inspections byapplying SfM (Structure from
Motion) technique to bridge
investigations. SfM has become ageneral technique as a result of
improvements
inpersonalcomputer (PC) performance andimage analysis technique in recent
years. As SfM is a technique for Fig. 1 Inspection of bridge pier by suspended scaffold and rope accessconstructing a 3D model of an object structures by using only images taken from multiple views, itwould appear that inspection work is possible on a PC, by indoor work, simply by photographing
images with an ordinary digital camera. Although remote measurement is also possible by using a
3D laser scanner, etc., the data acquired by this method is a point group and ultimately only
reproduces shape. SfM can reproduce shape, and can also acquire real color information. Since SfM
can grasp damage with high accuracy, SfM would seem to be an extremely effective technique forbridge inspections.
The use of unmanned aerial vehicles (UAV) has also spread rapidly in recent years, to the point where
UAV have been called an “industrial revolution in the sky.” By using UAV to photograph images
for SfM analysis, it is possible to acquire images from views that normally cannot be observed, and
it is thought that this will enable more effective inspections.
Therefore, we verified the effectiveness of damage investigation using UAV. The object of thisverification was a concrete main tower of a cable stayed bridge.
2. Applied Technologies
2.1 SfM (Structure from Motion)
SfM is a technique which makes it possible to construct a 3D model of a photographed object basedon images photographed from multiple views.
Unlike conventional photogrammetry, a characteristic point on the surface of the photographic
subject is taken from different directions, aiming at an overlap of 80%, and the positions of that point
and the camera, in other words, 3-dimensional coordinates (X, Y, Z), are automatically assigned insuccession by the stereo method from combinations of the photographed images (Fig. 2). The SfM
analysis generates a triangulated irregular network (TIN) of triangular plane elements with thosecoordinates as the tangent points based on the point group obtained in this manner, and constructs a
3D model. Here, a “characteristic point” means a point on the image and is a point with different
luminescence, i.e., color and brightness, compared with its surroundings. For characteristic points p
and q in regions (i, j) on two images, SfM calculates the sum total J of their differences ofluminescence T compared with the surrounding areas in the respectively regions, that is, the degreeof difference in color and brightness compared with the surroundings, and the points are judged tobe the same point when the value of J is within a certain judgment value (Fig. 3). Because thephotographed images are linked as texture in the generated TIN, a 3D model that faithfully
reproduces the photographic subject is constructed. Figure 4 shows an example of the constructionof a 3D model that was generated from photographic images by SfM analysis (partial excerpt).
It is also possible to give a scale to the generated TIN by setting targets with clear coordinate values
at 3 or more points in the object range, and photographing those targets in the image.

Characteristic
point
Camera position(X2,Y2,Z2)(2) Motion(3) 3-dimensional
position
Cameraposition(X1,Y1,Z1)(1) Correspondence
between images (matching)Fig. 2 Method of calculating characteristic point andcamera position

,

,

,

,

Fig. 3 Judgment of characteristic point

Images photographed from multiple views

Result of SfM analysis

Fig. 4 Example of construction of 3D model by SfM

2.2 Comparison of Analysis Software
At present, several software programs which perform SfM analysis are widely used. Therefore, we
compared the functions of the two types that seem to the most general, S software of A company andP software of B company. The results are summarized in Table 1. We also compared the 3D models
generated by the respective programs by using the same photographed images of a concrete structure.
As the result of the comparison, P software had many editing functions and high analytical flexibility,
but the texture quality was somewhat low. On the other hand, S software had the minimum functions
necessary for only generating models, with almost no editing functions, but a texture close with nearphotographic resolution could be pasted. Furthermore, the reproducibility of the edges of the structure
was also high (Fig. 5) because the TIN is generated by adjusting the edge line when performing
automatic analysis. Since the priorities in bridge inspections are the reproducibility of the object
structure and the degree of recognition of damage, the verification in this research was performed
using the S software.
Table 1 Outline of SfM analysis software programs
Name of program
Maker
Principle
Camera calibration
Point group editing
Measurement functions
Ortho-image output
Edge line confirmation

S
A Co.
SfM
No (parameter input is possible)
No
Distance (viewer software)
No
Yes

P
B Co.
SfM
Yes
Yes
Distance / Area / Accumulation
Yes
No

TIN generated by S software

TIN generated by P software

Can adequately distinguish texture.
TIN of structure edge is generated considering edge line.

Quality of texture is somewhat low.
Edge of structure is not accurately expressed.

Fig. 5 Comparison of reproducibility of edge of concrete structure2.3 UAV (Unmanned Aerial Vehicle)
UAV is a general name for small, unmanned aircraft. UAVs are already firmly established as ameasurement technique in the fields of surveying and civil engineering, in addition to aerial
photogrammetry, airborne LiDAR and ground measurements. By using UAVs, it is expected to bepossible to investigate dangerous locations and the actual site immediately after disasters, which had
been difficult in the past, and application to bridge inspections is also amply possible.
As shown in Fig. 6, a GPS antenna, gyro sensor and battery are mounted in a UAV, and automatic
flight is possible by setting the flight path in advance. When a camera is mounted on the UAV, theoperator manipulates the triaxial gimbal, which enables free vertical and lateral movement of thecamera, while constantly monitoring the photographed range from the ground, and the equipment
configuration makes it possible to trigger the shutter. Because the gimbal is controlled so as toautomatically maintain a horizontal attitude, images that reflect the intention of the photographer canbe acquired easily.
Table 2 summarizes the features of the UAV. In comparison with conventional radio-controlled
aircraft, the UAV is superior in terms of compact size, light weight and stability, and safer flight ispossible. Because it is motor-driven, it is comparatively quiet, but flight time depends on battery
capacity and is short, at about 20 min. Although a license is not required in order to operate a UAV,the safety of UAVs has been questioned recently. As this became a social issue, Japan’s Civil
Aeronautics Act was revised in December 2015 so that it is now necessary to apply in advance to the
Ministry of Land, Infrastructure, Transport and Tourism for flights in densely inhabited districts orover an altitude of 150 m, and for flights out of the operator’s range of vision.
The verification in this research was carried out giving full consideration to the safety of the UAV,
but there are also many restrictions for ensuring safe flight, as UAVs are easily affected by weather,radio waves and the like. Special care is also necessary in flights near roads, railroads, bridges, radio
towers and high voltage electric power lines, and an experienced operator is indispensable.
Compact and lightweight; easy to transport
Operator does not require license.
Operation is quiet (motor driven).
Flight is stable due to large number ofpropeller blades.
Easy to take aerial photographs
Short flight time (about 20 min)Difficult to use in strong winds (10 m/s or
faster)Approval is necessary for flights at altitudes
higher than 150 m and flights in densely
inhabited districts.
Flight out of the operator’s visual range is
prohibited.

GPS antenna
Gyro sensor

CameraAdvantages

Battery

Disadvantages

Table 2 Features of UAV

Triaxial gimbal

Fig. 6 Typical configuration of UAV (with camera
attached)

3. Verification at Long-Span Bridge
3.1 Outline
In bridge inspections in Nagasaki Prefecture, periodic inspections by proximity visual inspection arenormally carried out once in 5 years. However, inspections of bridges which are greatly different inbridge scale or structural characteristics and inspections of bridges with a large effect on the region(priority maintenance control bridges) are also performed once a year (hereinafter, annual inspection).
In annual inspections, the progress of deterioration of fixed observation points that can be observed
from the ground is confirmed.
Accordingly, by using UAV-SfM in annual inspections, it is possible to confirm the condition ofdamage safely, quickly and at low cost, and at a wider range of points which normally can only bechecked by proximity visual inspection using rope access or temporary scaffolding. In addition, since
the damage assessments are performed from images, it is also possible to make quantitative
judgments that are not dependent on the skill of the inspector, and comparisons over time are alsoeasy by accumulating digital data. Moreover, if UAV-SfM can be established as a new inspection
technique in place of proximity visual inspection, application to periodic inspections is also fully
possible.
In this research, an investigation using SfM was performed for a concrete main tower (4P) of the
Takashima-Hizen Bridge (cable stayed bridge) connecting Hoshika, Hizenmachi, Karatsu City, Saga
Prefecture and Kozakimen, Takashima-cho, Matsuura City, Nagasaki Prefecture, as shown in Figs.
7 and 8. The results were compared with the existing inspection results, and the required inspection
time was investigated. The following shows the data for the Takashima-Hizen Bridge.
Bridge name:Takashima-Hizen Bridge
Road name:Takashima-Hizen Line (General Prefectural Road)
Bridge length:
L = 1251.00 m
Total width: W = 11.50 m
Superstructure type: 5 span continuous steel 2 main girder, 5 span continuous cable stayed bridge,
4 span continuous steel 2 main girder
Substructure:
Abutments (2), piers (2)Year constructed: 2009
To Takashima
To Hizen

Object main tower 4P
Fig. 7 Takashima-Hizen Bridge

Fig. 8 Side elevation of Takashima-Hizen Bridge

3.2 Equipment Used
The specifications of the UAV and digital camera used in this research are shown in Table 3 andTable 4, respectively. The UAV used here was a Spreading Wings S900 manufactured by DJI witha maximum payload of 2.5 kg. This type was selected because strong winds were assumed above thesite of the bridge, as it is possible to mount a somewhat heavy camera on the UAV, and the UAV has
wind resistant stability. As the camera, a SONY mirrorless α7R was selected; this camera is equippedwith a full-size sensor and enables high pixel photography (36 x 106 pixels). The photograph in Fig.
9 shows the Spreading Wings S900 with the attached α7R camera.
Table 4 Specification of camera used

Table 3 Specification of UAV used
DJI Spreading Wings S900
PayloadWind resistant stability
Flight time
Flight distanceGPS

Mirrorless SONY α7R camera
Pixel number
7360 ×4912(36M)，
Sensor size
35 mm full size
Focal length
35mm
Weight
580 g (including lens)

2.5 kg8m/s
Upto 15 min
2 km
1 cycle3.3 Content of Measurements and Work Procedure

GPS antenna

Using the UAV with the mirrorless SONY α7R camera,
the sea side surface and the bridge axis side surface (45°direction) of the main tower were photographed (6measurement lines shown in Fig. 10). Althoughphotography with a resolution of 2 mm/pix is sufficient for
capturing a crack with a 0.2 mm width on a concrete
Triaxial gimbal
SONYα7R
surface, the target resolution was set at 1.5 mm/pix toacquire higher resolution images. (See the statement that
Fig. 9 UAV used
“In case of an image with digital resolution of 2 mm/pix, acrack with a width of 0.2 mm can be recognized on a PC monitor” in “Concrete Diagnosis
Technique ’16.) For photography with constant resolution, it is necessary to maintain a distance ofapproximately 10 m from the object surface, as shown in Fig. 11. Therefore, the UAV was flown by
a 2-person team consisting of a UAV operator and an image checker. While personally confirmed
the photographed images transmitted to a tablet in real time, the image checker determined thephotographing distance and resolution from the size of the photographed object relative to thephotographic angle of view, and gave instructions on the flight position to the operator (Fig. 12). In
order to generate a 3D model from the photographed images by SfM analysis, interval photography
was performed at 1 frame/sec (fps), assuming 80% overlap of all of the adjacent images. To give ascale to the generated TIN, 3D laser measurements were performed, and the coordinates of arbitrary
points were used as the target points. The procedure of the work is shown in Fig. 13.
⑤

⑥
③

④

Approx. 10 m②①

The photographs of ①② and ④⑤ were takenwith single flight
Fig. 10 Image of photography (6 measurement linesof outer side of main tower)

To photograph with a fixed focal
point, photography is performed so
as to keep a constant separation. →Instructions are given while
monitoring transmitted images.
Fig. 11 UAV flight plan

UAV

Operator
Image
checker

Measurement plan

UAV photography route plan,

Setting of target
points (surveying)Set targets near measurement

Photography ofimagesPhotograph images by using

Arrangement ofphotographs

Arrange photographs, andcarefully select photos for use

Site

Images received from imagetransfer position

Instructions on position while checking
photographed images
Indoor work

Fig. 12 System during photography

Analysis ofphotographs

3D modeling by SfMProcessing ofmodel

Output of ortho-images, etc.

Judgment ofInterpret/evaluate damage

Fig. 13 Work procedure
Fig. 14 Example of photographed image

3.4 Results of Photography
A total of 4 flights were carried out to photograph the 6 measurement lines of the sea side surface ofthe main tower. One flight was completed in about 12 min. Because interval photography was
performed at 1 fps, a total of 600 images were acquired. By using the UAV, it was also easy to obtain
images of the top part of the tower, which normally can only be checked by rope access, etc., and thecondition of the concrete surface could be recognized by enlarging the images. Figure 14 shows anexample of a photographed image of the top part of the tower, together with an enlarged view.
3.5 Procedure from SfM Analysis to Generation of Ortho-Image
Conventional photogrammetry was extremely laborious, as it is necessary to perform manualorientation work for the enormous number of images taken by the UAV, and much time was required
to match all of the images, but in SfM analysis, substantial labor-saving is possible because automaticorientation is performed using characteristic points. Furthermore, in photogrammetry, faithful
reproduction of shapes with depth was difficult, but with SfM, even images photographed frommultiple directions can be analyzed easily, and it is also possible to generate models of surfaces withdepth.
Figure 15 shows the analysis procedure and results. First, images that are blurred due to camera
shaking or are out-of-focus are eliminated from the photographed images. In UAV photography,
there are a large number overlapping photographs because interval photography is performed
continuously from take-off to landing of the UAV. Therefore, while eliminating poor-quality images,
images that secure the proper overlap are carefully selected. Next, these carefully-selected images
are analyzed by automatic orientation, and the characteristic points and camera position are calculated.

At this time, the rough shape is checked on the PC, and if a shape is not formed, the analysis isrepeated several times. A TIN is then constructed based on the calculated characteristic points, and a
3D model is generated by mapping, using the photographed image as the texture. Because this model
can be enlarged/reduced, moved and rotated on the PC, it is possible to examine the model from any
arbitrary view. The generated model can also be exported to other CG programs (in this research,
Autodesk 3dsmax was used), ortho-images can be output, and base drawings for damage
investigations can be generated. The time required for generation of a model by SfM is approximately
5 days.
TIN display
Camera position

Analysis is performed using carefullyselected images (approx. 350).

Easy operation by dedicatedCharacteristic
point

Ortho-image
Possible to generate damage map
overlaid on drawing.

3D model withmapping
Fig. 15 Procedure from SfM analysis to generation of ortho-image
Analysis result

3.6 Generation of Damage Map
Ortho-images with resolution of 2 mm/pix were generated from a 3D model with mapping, which
was created by SfM, and damage mapping was performed for items that could be read from theimages. The object items were cracks, exfoliation and exposure of rebars, water leakage and freelime, fallen out parts, and damage of repair/reinforcing materials as shown in Periodic Bridge
Inspection Procedure issued by MLIT. Among these, a tree-structure filter is sometimes used in
automatic judgments of the position and width of cracks, but in this research, a Crack Position/Width
Interpretation and Mapping System utilizing a
Crack Index (CI) was used. This system extracts
the crack position and width semi-automaticallyfrom the correlation between crack width and thetotal of characteristic values, CI, of the cracked
part.
Damage interpretation is performed on multiple
screens, as shown in Fig. 16. On the screen at theleft, damage is extracted from the ortho-image on
the lower screen, using the Crack Position/Width
Interpretation and Mapping System, and thisFig. 16 Condition of damage extraction work
damage is described on the screen. In this process,when the image resolution is 2 mm/pix, a range of approximately 3.5 m x 1.5 m is displayed on one
screen, and damage is searched in this range. At this time, a detailed interpretation is performed byenlarging the image on the PC so it is approximately on the same level as in visual interpretation.
The screen at the right displays the 3D model and is used in checking the object range in 3 dimensions
from various views. By searching for damage while observing the 3D model, it is also possible tocheck for deformation, such as differences in level and swelling, which cannot be seen in 2
dimensions. This is an extremely effective technique, in that the work be performed under calm
conditions indoors. It is also possible to convert the damage shown on the screen to CAD data andtransfer it to control charts, etc.
3.7 Discussion
No conspicuous damage of the main tower which was the object of this study was found as a resultof checking for damage by using the photographed images and 3D model. In this, however, the SfM
results were also compared with the results of visual inspection within the range possible.
In the SfM results, as shown in Fig. 17, it was possible to extract approximately the same cracks asin the visual investigation in FY 2014. However, even though fine cracks were extracted in detail inboth the visual investigation and the investigation by SfM, slight differences can be seen in thepositional relationship. The visual investigation was considered to be inferior in terms of positional
accuracy because the results were recorded in sketches, but when using SfM, positional
reproducibility was extremely high, as the images were recorded and stored at the same time.
Furthermore, while all of the visual inspection results are described as 0.1 mm in width, the SfM
results were generally on the order of 0.2 mm, but also included some cracks that were judged to be
0.5 mm. When cracks are extracted by using CI, there are deviations in the results due to the color or
soiling of the concrete surface. While it is conceivable that the results of this study were influenced
by soiling, evaluation is difficult with only this object range. For the crack width, it is considerednecessary to carry out continuing verification work focusing on bridges where clear true values canbe obtained, and clarify the relationship between the acquired images and crack width.
In conventional visual inspections, many parts are left to the skill and interpretation of multiple
personnel, and quantitative judgment is difficult, for example, in records of the position of damage,etc. However, in investigation by SfM, quantitative judgment is possible because damage
interpretation is performed based on images, and it is also possible extract 3D information such thecross section, etc. for arbitrary positions from the 3D model. Moreover, comparisons of changes overtime, the progress of deterioration, and other time-related phenomena can also be checked easily byacquiring the same data periodically. Because value-added date such as 3D models, images, etc. isaccumulated in this manner, it can be said that this investigation technique has great merit for future
operation and maintenance.
[0] <0.2 mm[1] 0.2 mm to <0.5 mm[2] 0.5 mm to < 1.0 mm[3] 1.0 mm to <2.0 mm

Same cracks

Result of visual inspection

Result of investigation by SfM
Fig. 17 Comparison of investigation results

3.8 Issues
Although investigation by UAV-SfM is an effective technique, as described above, the following
issues also exist.
Adequate consideration of the safety of the UAV is necessary. Particularly when investigating abridge, as was done in this study, it is important to be aware of weather conditions, etc., as winds
blow constantly. Although a large labor-saving and low cost are possible in field work if UAV-SfM
is used, the analysis work requires considerable time. Moreover, since the 3D model is constructedbased on characteristic points in SfM, certain parts cannot be analyzed, for example, uniformlypainted steel structures in which extraction of the characteristic points is difficult, members withrepeated patterns, thin members, etc. Furthermore, items such as lifted defects in concrete, whichmust be investigated by a contact technique, are difficult to interpret from images. It is thought that
this is the reason why investigation by images, in its current state, has been evaluated as unsuitable
as a substitute for proximity visual inspection.
4. Conclusion
Use of SfM technique to generate 3D models from images photographed employing a UAV andcreate damage maps is sufficiently applicable as an inspection technique, and can also realize a large
labor-saving and cost reduction in comparison with investigations in which temporary scaffolding iserected. Investigation using images still has not be recognized as a substitute for proximity visualinspection because its crack width detection accuracy is insufficient and it is not possible to detect
lifting, which requires investigation by a contact technique. However, a technique using images is
considered effective as a primary screening method. Primary screening to select the locations where
detailed inspection is necessary makes it possible to allocate time, budgets and personnel
appropriately, and thus enables effective, efficient inspections.
Application of UAVs in investigations can be expected in a variety of fields (river surveys, daminvestigations, etc.), and use in the rough design of countermeasures during disasters is also amply
possible. However, since safety must be secured when using a UAV, from the viewpoints of safety
and cost, UAVs are not necessarily effective in all cases. Depending on the required results, anappropriate plan should be prepared, and SfM analysis by manual photography, etc. should also be
used effectively. Moreover, because an as-built 3D model that faithfully reproduces the present
situation can be constructed by SfM analysis, expansion to BIM and CIM, while continuing thecurrent dissemination of this technique, is considered effective.
In the future, it will be necessary to carry out ongoing verification work and clarify solutions for
issues such as clarification of the relationship between crack width and photographic resolution,
improvement of detection accuracy by comparison with true values, and shortening of analysis time.
References
Ministry of Land, Infrastructure and Transport 2016.9,
http://www.mlit.go.jp/road/sisaku/yobohozen/28_3maint.pdf，p.27.

‘Annualreportonroad

maintenance’,

Keitaro Konuma, Syozo Nishimura, 2014.9, ‘Applicability of SfM technology to bridge inspection’ Proceedings of the69th Annual Conference of the Japan Society of Civil Engineers, Ⅵ-511
Nagasaki Pref. Government 2015.3., Priority Maintenance Bridge Detailed Inspection Report in Nagasaki Pref.
Japan Concrete Institute, 2016, ‘Concrete Diagnosis Technique’16’, pp.110-111
Syozo Nishimura, et al., 2013.1., ‘The Development and Evaluation of Telemetry in Bridge Maintenance’ Journal ofApplied Survey Technology Vol.24, pp.52-61