Development of a hybrid camera system for bridge inspection
S.NISHIMURA & K. KIMOTO
Keisoku Research Consultant Co., Ltd, Hiroshima, JapanS.KUSUHARA & S.KAWABATA
Honshu-Shikoku Bridge Expressway Co., Ltd, Kobe, Hyogo, Japan

A.ABE & T.OKAZAWA
Bridge Engineering Co., Ltd, Kobe, Hyogo, Japan

ABSTRACT: As the record means of the bridge inspection in recent years, it is absolutely imperative to use
technology of high resolution digital image and the data processing. However, structure of the bridge underneath is complex where columns and beams are crossed over. This makes bridge inspection quite difficultcompared to tunnel wall inspection, so that inspectors need to spend more time to identify where to take photos for the inspection. We have developed new inspection system aiming at high resolution photos for specificareas as well as a whole picture of objects such as building using 360 degrees camera. This is referred to system covering “micro to macro on any objects” for bridge inspection which will practically produce moreproductivity. Authors proofed to apply this camera system that was designed to inspect truss bridge and executed when the truss bridge was surveyed.

1 INTRODUCTION
In these days, acquisition of high resolution digital
images and processing techniques have been developed, which are applied to such as the tunnel wall
inspection. In this study, we applied these techniques
to huge bridge maintenances and management.
However, we predicted that it is difficult to specify
the taking a picture parts only with the individual detailed images, because huge bridges, especially truss
bridge, is complicated with various components,
compared with tunnel walls.
On the other hand, sphere image represented by
the street view of Google map is taken while runningvehicle with cameras, and we are able to grasp thesituation of a spot at an equal interval.
However, we are not able to grasp to detail for inspection of the structure because picture resolutionof the present sphere images is low.
By using sphere images and detailed images
taken at the same time, we built Images Research
System which use sphere images as an index, andexamine application possibility.
We applied experimentally this system to theAkashi Kaikyo Bridge with a higher efficiency andlaborsaving in image arrangement work. This paper describes the effectiveness of this System andremained problems.

Figure 1.1. Akashi Kaikyo Bridge

Figure 2.1. Location of the Akashi Kaikyo Bridge

2 THE OUTLINE OF THE AKASHI KAIKYOBRIDGEThe Akashi Kaikyo Bridge is the world’s longest
suspension bridge with 1991m of main span which
located between Kobe and Awaji Island. It wasopened to traffic in April, 1998 using the bridge
technology in the world highest level under a severe
design condition compared with foreign countries’one.

4 PICTURE RESOLUTION FOR INSPECTION
We have so far verified about resolution and possibility of checking crack width and detailed parts invarious investigations and research. Fig. 4.1 shows
the relationship the deficit and rust of a bolt thatbeen took a picture by digital camera from a long
distance and resolution that we can validate them. It
indicates that in order to checking the state of a boltwith a diameter of 30 mm, the resolution is requiredfor higher than 3 mm/pix.
5 TAKING IMAGE SYSTEM AND THE
PROJECTFig 2.2. Side elevation

3 SUMMARY OF THE INSPECTION OF THE
AKASHI KAIKYO BRIDGE AND PROBLEMS
Inspection parts of stiffing girder of the Akashi Kaikyo Bridge (Fig.3.1) are main truss, cross section
truss, steel deck plate and suspender anchorage.
Those are able to take images from wide inspection
way, upper (lower) inspection way, maintenance vehicle. However, on taking images of 4km bridge, effective specification of the photography / inspection
part is the problem. In this paper, we researched twopanels in about 4km of stiffing girders from wide inspection way and maintenance vehicle. We verified
a necessary condition and future problems in case ofwhole length stiffing girders. When we investigate astructure, we take many pictures and draw up a protocol. However, it is difficult to know shooting positions from those pictures after the investigation.
Moreover, in order to compare with the past them,
the pictures of the same angle is needed in manycases. In addition, in order to give information ofshooting position, we sometimes take a picture witha commercial camera carrying GPS. However, since
the picture didn’t have exact accuracy of locationand information of directions, we were not able tospecify the details of a structure.

We took pictures to investigate various structuralmembers of stiffing girder of the Akashi Kaikyo
Bridge by four methods shown in Fig.5.1. Fromwide inspection way, we took pictures, running vehicle which has a camera for taking sphere image(Ladybug3) and 16 industrial measurement cameras
for taking detailed image (Grasshopper) at 15 km/h.
Moreover, we experimentally set another one Grasshopper which has 5 mega pixels (hereinafter referredto as MP) and synchronized it with Ladybug3. Thus,
we identified shooting position of detailed image
which use sphere images as an index.
About outside parts which cannot be checked only
from inner stiffing girder, we used inspection gantry
that cameras were set on its floor face and telescopic
scaffold, and took images continually running. But,
since there are few pages, this paper describes only
the shooting method from wide inspection way.
High

1.5

Resolution
Low

3

10

Cross section truss
Main Truss

Inspection gantry

Wide inspection way

Figure 3.1. The outline of stiffing girder

Figure 4.1. Resolution-Visibility Chart30
mm/pixResolution[mm/pix]

2. Grasshopper (Taking detailed image)
2MP×16 cameras

20

1. Ladybug3 (Taking sphere image)
2MP× 6 cameras

12MP

f : 10mm(FishEye)
Ladybug3
f : 3.3mm
10DSLRcamera
Nikon
D700

f : 50mm Reflex
Digital Single-Lens
f : 120mm

03. Grasshopper
5MP（Taking detailed image）

51015202530 Distance[m]Distanc

Fig 5.3 Resolution of Ladybug3 and DSLR camera
20mm/pix
15mm/pix

4. DSLR camera
12MP（Taking detailed image）

Figure 5.1. Taking image with 4 kinds of method

10mm/pix

Ladybug3

5mm/pix

5.1 Taking image system in the stiffing girder
5.1.1 Taking sphere image system
On taking whole structural images, this researchused the system Ladybug3 (Point Grey ResearchCo.) that takes sphere images while running, which
is widely used for multiple researches.
This machine is compose of 6 cameras of 2MP,
takes 15 images in 1 second. It is possible to checkthe outline of the structure from the automaticallycomposed sphere images.
However, CCD size of camera is very small
6.9mm×5.2mm, and also lens is wide range3.3mm.
When the subject is located 7m, resolution is
10mm/pix, and it is difficult to grasp details of thesubject.
In addition, Fig.5.4 shows range of applicationsin the case using sphere camera in the Akashi Kaikyo Bridge. It indicates that when a picture is takenfrom wide inspection way located in the center,
since the resolution at main truss located in seaward
side is only 20 mm/pix, we can grasp only outline.

3mm/pix

35.5ｍ

Figure 5.4. Resolution of Ladybug3 in each distance

5.1.2 Taking detailed image system
We used Mobile shooting camera system (FULLCAP) as taking detailed images system. This system
makes it possible the continuation photography withwhich several industrial measurement cameras(Grasshopper, 2MP) synchronizes.
This investigation used 16 cameras with lens
(50~150 mm) so as to catch the details of member.
And we added the grasshopper (5MP).
The basic system consists of an industrial CCD
camera (2MP, 1 pixel = 4.4μm) and an uncompressed image recording system, FULLCAP. Whentaking image of track systems, it is necessary tounify the resolution per pixel by synchronizing morethan one camera.
Therefore, we constructed a taking and recording
system using industrial digital cameras connected
via IEEE 1394 that could photograph 15 frames per
second in order to taking images successively at 15km/h.

IEEE-1394b digital camera
Grasshopper(2MP)
Ladybug3

Stitching Layout
Figure 5.2. Specification of Ladybug3 and StitchingFig 5.5 The system of FULLCAP5.1.3 Taking image plan
Based on plans, we reproduced stiffing girder oftruss structure with 3DCG-model, and examinedcamera range and angle from measurement car inadvance. This examination was initial check withthe object, and was taken as the level which cancheck the lack of bolts, and generating of rust. Eachcamera was set in 3mm/pix referred to Fig. 5.6.
We constructed a jig to set cameras (Fig.5.6)
which is similar to structure of stiffing girders sothat taking images of stiffing girders could be done
efficiently from member components. This jig and
16 of industrial measurement cameras were set on
the upper measurement car so as to make resolution
in 3mm/pix, and took images continually running in
10 km per hour. From the prior examination byCG, since the trussed construction components was
14m in length, the lens was used so that each components might be caught with three cameras, and apicture was taken by arranging cameras.(Fig5.7)

5.2 Situation of taking image
Fig 5.8 shows the situation of measurement. PC tomeasure and control was set in the car.
When taking image by industrial measurement
cameras take sphere images and detailed images
which will be shown in monitors and stored in PC
with information at the same time (Fig.5.9). In addition, detailed images (5MP) will be shown in same
monitor, too. (Fig5.10)

Figure 5.8. The situation of measurement⑩⑯⑪⑫⑬⑭⑮

Detailed image (16 cameras)④⑤⑦①⑥⑧②Sphere image③⑨Figure 5.9. The situation in the measurement car

Figure 5.6 Camera position

A
カメラ①画像
Camera1

カメラ④画像
Camera4

カメラ⑦画像
Camera7

B
カメラ②画像
Camera2

カメラ⑤画像
Camera5

カメラ⑧画像
Camera8

カメラ③画像
Camera3

カメラ⑥画像
Camera6

カメラ⑨画像
Camera9

Figure 5.7 The examination of range and angle

Display sphere image (A)anddetailedimage(B,5MP) at the same time

Figure 5.10. The display example of 5MP image(Red flame shows the range of 5MP image.)6 RESULT AND EXAMINATION

6.2 Results

6.1 Sphere image
Fig.6.1 shows the images taken using Ladybug3. As
previously mentioned, Ladybug3 can take the images using 6 cameras equipped itself, and can makesphere image of those images. In addition, since thatcan display sphere images continuously on PC, wecan check the parts in every direction. Furthermore,
Ladybug3 can also display the images which wereremoved strains and the centroclinal images and soon.

Camera1

Camera4

Camera2

Camera5

Camera3

Camera6

Fig.6.2 shows images taken by FULLCAP were sufficient quality for inspection of bolts and rust both invertical components and horizontal components.
When taking pictures with a DSLR camera, thought
2 cameras captured one component, they were sufficient resolution images. The pictures taken frommaintenance vehicle were enough to check not only
bolts, but also inspection gantry rails and suspendersocket.
Table.6.3 shows images which the part located in
20 m from camera position with 4 kinds of cameras.
As we mentioned above, the resolution of capturing
sphere images by Ladybug3 is 20 mm/pix, and although we could check the form of parts, we werenot able to check the lack of bolts and the existenceof the generating of rust.
The resolution of the expanded parts taken by
5M camera and a DSLR camera D80 was high definition at 0.6mm/pix. However, D80 can catch larger area by one time. It is because that CCD size of aDSLR camera is three times as large as that of an industrial measurement camera. Moreover, since aDSLR camera has higher color reproduction characteristics, and easy to acquire information from images, we can check more smoothly than using images taken by industrial measurement camera.

Automatically compositeFigure 6.1. Sphere image

Table 6.3

The comparison table of the each image

Cam eraLadybug3

FULLCAP

Grasshopper

Pixels2MP2MP

5MP

DSLR(Nikon D80)
10MP

Taking image

Sphere Image

Detailed Image

Detailed Im age

Detailed Image

2448×2048

3872×2592

0.6mm/pix

0.6mm /pix

1600pix

Full Image

1200pix
Size

1616×123 2

1600×1200
2m

Extended Image

Resolution

21.5mm/pix

3.0mm/pixVertical member

Horizontal member

frames per second. If we take only the necessary images of the all area with the same method in this examination, we guess that the data capacity will be
about 1000GB. (1TB)
8 CONCLUSIONS

Fig 6.2

Detailed image

6.3 Comparison between the feature of a DSLR
camera and an industrial instrumentation
cameraTable.6.4 is comparison between the feature of a
DSLR camera and an industrial instrumentation
camera.
A DSLR camera has deep depth of field and cantake high quality pictures with focus. Besides,
color reproduction characteristics and the photography flexibility in backlighting is high, because of thesensitivity is high, and dynamic range is wide.
However, it is difficult to synchronize sphere image
camera with multiple DSLR camera because restriction of the number of cameras and frame. That is theproblem to consider the taking picture system.
On the other hand, an industrial measurement
camera is suitable for the continuation photography
because of the electronic shutter. However, the adjustable range of a lens and an iris diaphragm is narrow, and the operatively is low. Therefore, it takes
time to adjust camera angle and lens focus.
Whichever method we use, invisible areas exist
because truss is complicated with multiple components. Therefore, it is necessary to acquire images
aiming at important check point.
7 DATA CAPACITY
We took sphere and detailed images in 15 flames per
second while running at 10km/h. It is especially laborious work to extract the necessary images from alarge amount of images taken by industrial measurement camera, and the capacity of image data becomes huge. Based on the number and capacity oftaking images at 2 panels in this examination, we
examined about it if we inspect the all area of theAkashi Kaikyo Bridge. The even capacity of images
which took only 2 panels was huge ones (117GB).If
we apply the present system to inspecting the allarea of the Akashi Kaikyo Bridge, the data capacity
will be about 17000GB (17TB).It’s not efficient inview of keeping and analysis data.
Therefore, we considered about optimization ofspeed and number of taking image. When the running speed of measurement car is about 10km/h, it ispossible to catch the each target member in 1～2

Although some subject remained, we can confirm
the possibility of application through the experimentof using high resolution images for inspection of thebridge which has many a complex members.
Recent progress in digital camera techniques andimage processing software make it possible to takeimages in high resolution and to process easily.
Therefore, these methods are getting necessary tokeep records as a possible method when doing visualinspections. It is expected that improvement inspeed, reduction of labors, standardization, multifictionalization, reliability improvement, the visualization of a results, and electronization by using digital images in high resolution in the initial check.
We verified that this system which takes picturesby sphere image camera-Ladybugs3, specify the inspection parts, and then takes detailed image(FULLCAP) is able to apply for inspecting such as
long regular structural forms bridges. Additionally,in such bridge, it is necessary to adjust camera angle
and lens focus were adjusted sufficiently because it
takes pictures while running. However, accurate adjustment makes it possible to take images effectively.
Industrial measurement camera is inferior toDSLR cameras in camera angle, operability, flexibility. Considering the number of cameras and image
quality, the usage of Industrial measurement camera
is restrictive. Therefore, it is necessary to consider
about replacing industrial measurement cameras
with multiple DSLR cameras and synchronizing
with computer.
From this, it will be possible to take images withefficiency, and the quality of images will be higher.
In addition, this method will make it possible to reduce the data capacity because this method is able totake wider range image in 1 shooting than industrial
measurement cameras.
Furthermore, it is necessary to create a management system. This system is able to search the detailed images taken at over time and verify the stateof bolt and rust based on the sphere image taken asindex.
In addition, the subject remained about invisible
parts. Therefore, we considered a simply methodthat we took images continually with Ladybug3
while walking. In this method, we will use thesphere camera system while absorbing walking vibration with the vibration control device.

Table 6.4 Comparison between the feature of a DSLR camera and an industrial instrumentation camera

Blue：merit
Red：demerit

DSLR camera

industrial measurement camera

FeatureSensitivity

Depth of field

FeatureSensitivity

Depth of fieldLensLarge

High

Wide adjustable range

SmallLow

Small adjustable range

Iris diaphragmWide adjustable range

Wide adjustable range

Wide adjustable range

Wide adjustable range

Small adjustable range

Small adjustable range

Shutter

Mechanical shutter--

Electronic shutter

occurrence of smear-

Imaging device

Large

Wide dynamic range

Shallow

Small

Small dynamic range

DeepThis device makes it possible to take images stability and to store them at the same time while walking. Moreover, it is possible to display the walking
route on the map by this devise connected with GPSand watch the taken images easily and move them
freely around 360 degrees with mobile tablet PC.
For example, using sphere images taken before, wewill be going to check the damaged part easily at thenext research.

DSLR camera
Sphere camera
GPSFigure 8.2. Loading the sphere camera and multiple DSLR
camera on the car

REFERENCES
Taking sphere image while walking

Hirata.K et al. 2010: Investigation use of the Photogrammetry
which utilized the digital technology JAST vol.21, (2010)(CD-ROM). (in Japanese)
Kurashige.H et al, 2010: Development of On-Board Image
Measurement System for Actual Running and Application
to Wall Surface Surveys of Structures, IABMAS2010,Kusuhara.S et al. 2011: Development of a Hybrid Camera System for Bridge Inspection, Proceedings of the 66th Annual
Conference of the Japan Society of Civil Engineers, (CDROM) I-535 (in Japanese)

Taking the detailed image withthe high resolution camera at theNishimura.S et al. 2010: Development of On-Board Image
Measurement System for Actual Running and Application
to Wall Surface Surveys of Structures, JAST vol.21,(2010) (CD-ROM). (in Japanese)
Nishimura.S et al. 2011: The Measurement and Monitoring at
Gunkanjima-Island by using 3D laser scanner and Sphere
Camera, J. JSEM, (in Japanese)

Taking image system loading the vibration isolator
Display the sphere image
with the tablet PC

Figure8.3. Taking sphere image while walking