E-Journal of Advanced Maintenance Vol. 11 No. 2 (2019) 99-105
Japan Society of Maintenology

3D Environment Reconstruction Based on Images Obtained byReconnaissance Task in Fukushima Daiichi Nuclear Power StationToshihide HANARI1,* and Kuniaki KAWABATA1
1

Japan Atomic Energy Agency, 1-22 Nakamaru, Yamadaoka, Naraha-machi, Futaba-gun, Fukushima
979-0513, JapanABSTRACTThis paper describes a three-dimensional environment reconstruction obtained from multiple images, aiming todetermine the status of the reactor building of Fukushima Daiichi Nuclear Power Station. The Structure fromMotion was used to estimate the three-dimensional structure of the objects and camera trajectory. To improve
the feature detection achieved using the Structure from Motion, a contrast enhancement filter and unsharp
masking filter were applied to pre-process the images. We applied the Structure from Motion method to
pre-process multiple images obtained by reconnaissance inside the Primary Containment Vessel of Unit 3 using
a remotely operated vehicle. Moreover, to improve the visibility, a Multi-View Stereo technique was applied tothe data generated using the Structure from Motion. Herein, we report the results of the three-dimensional
environment reconstruction.
KEYWORDS

ARTICLE INFORMATION

3D reconstruction, Structure from Motion, multiple images,
decommissioning, Fukushima Daiichi Nuclear Power Station

Article history:
Received 23 October 2018
Accepted 30 August 20191.

Introduction

In March 2011, an accident occurred at Fukushima Daiichi Nuclear Power Station (NPS) ofTokyo Electric Power Company Holdings, Inc. (TEPCO). Subsequently, owing to the spread ofradioactive contamination, high-radiation environments arose within the reactor buildings. To protect
workers from radiation exposure, remotely operated machines including robots were utilized fordecommissioning tasks at Fukushima Daiichi NPS[1]. In decommissioning tasks performed byremotely operated machines, the fuel debris survey process in the Primary Containment Vessel (PCV)
is important because the retrieval process of fuel debris is determined from the survey results.
Similarly, the gathered data including images can be utilized for subsequent planning. Therefore,
three-dimensional (3D) structural information of a working environment is useful for intuitively
assessing the environmental state. Consequently, we applied a 3D environment reconstruction method
known as the Structure from Motion (SfM) [2].
This paper describes the generation of a 3D environment model based on multiple images
extracted from a video recorded via reconnaissance inside the PCV by an underwater Remotely
Operated Vehicle (ROV) at Fukushima Daiichi NPS. To improve the reconstruction performance, weapplied pre-processing to the target images, and post-processing was applied to the data generatedusing SfM to improve the visibility. The results of the 3D environment reconstruction are reported anddiscussed.

2.

3D environment reconstruction based on images

We first aimed to verify the suitability of the SfM method for assessing the working environment
in the decommissioning process. In this section, we describe the target images utilized herein and themethod applied to reconstruct a 3D model from the target images.

*Corresponding author, E-mail: hanari.toshihide@jaea.go.jp

ISSN-1883-9894/10 © 2010 – JSM and the authors. All rights reserved.

99

T. Hanari et al./ 3D Environment Reconstruction Based on Images
Obtained by Reconnaissance Task in Fukushima Daiichi Nuclear Power Station2.1. Target images
For a practical examination, we utilized image sources acquired during a reconnaissance task by
the underwater ROV at Fukushima Daiichi NPS. For this task, the field of view of the mounted
camera was limited, and lighting conditions was poor. The lighting conditions affected the quality of
the images and the resulting reconstruction performance. Therefore, to verify the performance of the3D reconstruction method and to investigate the influence of the lighting conditions, we utilized
multiple images extracted from a video available at the TEPCO website[3].
As a typical example, we utilized the video recorded inside the PCV of Unit 3 by an underwater
ROV[4] in July 2017[5]. Figure 1 shows the ROV used for the survey task (source: TEPCO). The
ROV has a diameter of 125 mm and a length of 300 mm and is equipped with a front camera with atilt axis and a rear camera without pan or tilt axes.
Figure 2 shows a schematic and an example image obtained in the PCV of Unit 3 (source:TEPCO). The ROV exploration was performed to examine the state inside the PCV, as shown in Fig.
2(a). We chose a video segment recorded near the bottom of the Control Rod Drive (CRD) housing
for application of the SfM method. The CRD housing and its supporting clamps were confirmed in thevideo segment, as shown in Fig. 2(b). All frames (30 frames per second) of the images in the video
were extracted.

Fig. 1. Underwater ROV[4] (source: TEPCO)

(a)

100

T. Hanari et al./ 3D Environment Reconstruction Based on Images
Obtained by Reconnaissance Task in Fukushima Daiichi Nuclear Power Station(b)Fig. 2. Underwater reconnaissance task inside the PCV of Unit 3 at Fukushima Daiichi NPS[5]: (a)schematic view, (b) image captured at the bottom of the CRD housing (source: TEPCO)

2.2. Image processing
We applied three image processing steps to the target images. First, pre-processing wasperformed to improve the image feature detection, including contrast enhancement (CE) and unsharp
masking (USM). Second, we applied the SfM method to the pre-processed images to obtain a sparsereconstruction and to estimate the camera trajectory. Finally, Multi-View Stereo (MVS) reconstruction
was applied as a post-processing step to improve the visibility. A flowchart of the image processing
applied to generate a 3D model from the images is shown in Fig. 3. In this section, the details of each
image processing step are given.

Fig. 3. Process flow for generating a 3D model from images

2.2.1. Pre-processing: CE and USM filters
The target images had a low contrast because only one light source was available for the ROV,
which operated in dark areas. From a preliminary examination of the image characteristics, we found
low contrast and blurring in the target images, which degraded the feature detection performance.
Therefore, to improve the image feature detection, we introduced CE and USM filters to pre-process
the target images extracted from the video. The CE filter exerts a contrast stretching effect on theintensity of each image pixel, acting as a linear translation operation that can be described as follows.

101T. Hanari et al./ 3D Environment Reconstruction Based on Images
Obtained by Reconnaissance Task in Fukushima Daiichi Nuclear Power Station,(1)where ICE (x,y) is the CE-filtered intensity value for each pixel position (x,y) of the image. Iorg(x,y), Imin,and Imax are the intensity value of the original image, the lower intensity limit, and the upper intensity
limit, respectively. Herein, the intensity values of the image are given as 8 bits (256 levels).
The USM filter sharpens the edges of the objects in the image as follows.
1. An edge image is generated from the original image.
2. The edge image is multiplied by a scaling constant.
3. The original image and edge image are combined (sharpened image).
This process is described by the following equation.

,(2)

where IUSM(x, y) is the USM-filtered intensity value, k is a scaling constant, and ∇2 is the Laplace
operator. The Laplace operator is a second-order differential operator and can be given as aconvolution based on the kernel shown in Fig. 4. To generate an edge image, the kernel is applied tothe original image, with a kernel window size of 3 pixels × 3 pixels.

Fig. 4. Kernel of the Laplace operator

By utilizing these two filters, the target images were pre-processed, and the image featuredetection performance of the SfM method was improved. These filters can also be applied tolow-contrast images acquired in similar dark areas.
2.2.2. 3D reconstruction: SfM
Herein, we utilized the SfM[2], a photogrammetry technique for 3D reconstruction based onimages. This method performs a sparse reconstruction of objects in the images and estimates thecamera trajectory. The SfM method generally extracts image features for use in computation. Forfeature detection, we utilized a Scale-Invariant Feature Transform(SIFT)[6]. The SIFT is a featuredetection algorithm that achieves robust feature detection under changes in scale and rotation as wellas in illumination among images. Feature detected using the SIFT algorithm has an orientation andmagnitude of a local image gradient on the detected point and is represented in a vector form called afeature vector.
The detected feature vectors were utilized for matching computation among the images. The
features matched among the images were assigned a coordinate in 3D space by a triangulation
operation, and each point was registered to a 3D point cloud. The matching and triangulation
processes were applied for sparse reconstruction and camera trajectory estimation. The camera
parameters, including the focal length and field of view, were estimated simultaneously. Herein, wedefine the sparse reconstructed model as the 3D point cloud obtained using the SfM method, camera102

T. Hanari et al./ 3D Environment Reconstruction Based on Images
Obtained by Reconnaissance Task in Fukushima Daiichi Nuclear Power Station

trajectory, and camera parameters.
2.2.2. Post-processing: MVS
The sparse reconstructed model obtained using the SfM provides a low-density 3D point cloud.
However, the structure of the low-density 3D point cloud is difficult to intuitively identify. Thus,
improved visibility is required to identify the structure of the working environment. To improve thevisibility, we applied an MVS[7] post-processing step to the sparse reconstructed model obtained via
the SfM process.
The MVS step produced a dense reconstruction based on the sparse reconstructed model, and amesh surface was generated based on the dense reconstructed model. Moreover, we added textures tothe generated mesh surfaces, finally creating a 3D model.

3.

Experimental results

We applied pre-processing, SfM processing, and post-processing to the images extracted from thevideo mentioned in Section 2.1. The video was recorded near the bottom of the CRD housing, asshown in Fig. 2(b). The video file was in the MP4 format, with a resolution of 1080 p (1920 pixels ×1080 pixels). We cropped right and left black masks of the video and exported the extracted images inthe JPEG format utilizing OpenCV[8], an open-source computer vision library. The resolution andcolor profile of the extracted images were 1456 pixels × 1080 pixels and 24-bit RGB (8 bits/channel),respectively. Part of the bottom of the CRD housing was observed in the video over approximately 6.7s, from 61.5 to 68.2 s, and the total number of extracted images was 203 frames. For pre-processing,the Imin and Imax values of the CE filter were set to 60 and 180, respectively. The scaling constant k for
the USM filter was 1, and the USM filter was applied to the images using the kernel illustrated in Fig.
4. We utilized OpenCV for these pre-processing steps. Furthermore, we utilized the software
VisualSFM[9][10] for the SfM computation and the open-source library OpenMVS[11] for executing
the MVS process. Figure 5 shows the extracted images and features detected using the SIFT algorithm.
The arrows in Fig. 5 indicates the detected feature vectors. As shown in Figs. 5(a) and 5(b), many
features were detected at the supporting clamps of the upper and lower sides in both images. However,some features were not detected on the CRD housing between the supporting clamps in the original
image shown in Fig. 5(a). In contrast, for the pre-processed image shown in Fig. 5(b), features weredetected on the CRD housing. The mean number of detected features in the pre-processed image wasapproximately 7-fold greater than that of the original image, confirming that the feature detection
performance was improved by the applied pre-processing.
Figure 6 shows a 3D environment reconstruction image obtained by the processing steps
described in Section 2.2. The sparse reconstruction result of the CRD housing, supporting clamps, andestimated camera trajectory are plotted in Fig. 6(a). Although we partly reconstructed the supporting
clamps as the sparse point cloud, we were not able to identify the structure of the CRD housing. As aresult, the sparse reconstructed model has a low density, and it is difficult to intuitively identify thestructure. To improve the visibility of the reconstructed model, the dense reconstruction process wasapplied to the sparse reconstructed model. Figure 6(b) shows the dense reconstruction result, whichexhibits an improved visibility compared with the sparse reconstructed model. The mesh surface
generated from the dense reconstructed model is shown in Fig. 6(c); finally, the 3D model shown inFig. 6(d) was obtained by adding textures to the generated mesh surfaces, utilizing the images. We canconfirm that the CRD housing and supporting clamps are roughly reconstructed.
The 3D structure of the generated model was superior to that obtained using unfiltered images.
However, some features on the upper and lower sides of the model were omitted due to errors in theestimated camera trajectory; thus, the matching calculations for the image features require furtherimprovement. We suggest that a feature matching process took into account sequential images would
improve the 3D modeling accuracy.

103T. Hanari et al./ 3D Environment Reconstruction Based on Images
Obtained by Reconnaissance Task in Fukushima Daiichi Nuclear Power Station

Fig. 5. Extracted images and detected features: (a) original image of the CRD housing in the PCV of Unit
3 and (b) pre-processed image

Fig. 6. Results of 3D reconstruction: (a) sparse reconstructed model, (b) dense reconstructed model, (c)generated mesh surface, and (d) generated 3D model

4.

Conclusion

This paper described the method for 3D environment reconstruction based on multiple images
obtained by the underwater ROV inside the PCV of Unit 3 at Fukushima Daiichi NPS. We confirmed

104

T. Hanari et al./ 3D Environment Reconstruction Based on Images
Obtained by Reconnaissance Task in Fukushima Daiichi Nuclear Power Stationthat 3D models of the CRD housing and supporting clamps could be generated using the SfM method
based on the pre-processed images and the MVS reconstruction method.
In future work, we will improve the SfM calculation algorithm to obtain a more accurate 3Dmodel.

References
Tokyo Electric Power Company Holdings: “Application of Robot Technology”, http://www.tepco.co.jp/en/decommision/principles/robot/index-e.html (Accessed October, 2018).
[2] H. C. Longuet-Higgins: “A computer algorithm for reconstructing a scene from two projections”, Nature,
Vol.293, No. 5828, pp.133-135 (1981).
[3] Tokyo Electric Power Company Holdings: “Photo & Video Library, Photo collection”,
https://photo.tepco.co.jp/en/index-e.html (Accessed October, 2018).
[4] Tokyo Electric Power Company Holdings: “Photo collection, Unit 3 PCV Internal Investigation at
Fukushima Daiichi NPS (remotely operated underwater vehicle)”, https://photo.tepco.co.jp/en/date/2017/201705-e/170525-01e.html (Accessed October, 2018).
[5] Tokyo Electric Power Company Holdings: “Photo collection, Progress of Unit 3 PCV internal investigation(Preliminary report of July 21 investigation)”, http://photo.tepco.co.jp/en/date/2017/201707-e/
170721-01e.html (Accessed October, 2018).
[6] D. G. Lowe: “Distinctive image features from scale-invariant keypoints”, Int. J. Comput. Vis., Vol. 60, No.
2, pp. 91-110 (2004).
[7] C. H. Esteban and F. Schmitt: “Silhouette and stereo fusion for 3D object modeling”, Comput. Vis. Image
Underst., Vol. 96, No.3, pp.367-392 (2004).
[8] OpenCV: Open Source Computer Vision Library, https://opencv.org/ (Accessed October, 2018).
[9] C. Wu: “Towards Linear-time Incremental Structure From Motion”, Proc. Int. Conf. 3D Vis., Washington,USA, Jun.29–Jul.1, ISBN 978-0-7695-5067-1 (2013).
[10] C. Wu, S. Agarwal, B. Curless, and S. M. Seitz: “Multicore Bundle Adjustment”, Proc. IEEE Conf. CVPR,
Providence, USA, Jun.20-25, pp.3057-3064 (2011).
[11] OpenMVS: open multi-view stereo reconstruction library, https://github.com/cdcseacave/openMVS(Accessed October, 2018).
[1]105