# An Operational Image-Based Digital Twin for Large-Scale Structures
## 3 Drone-Based Image Acquisition and Pre-Processing  
Images of a specific transition piece were collected at the factory of Bladt Industries using a DJI zemmouse P1 45M pixels camera onboard a DJI M300 RTK drone. Real-time kinematic positioning (RTK) was used to improve the precision of the GPS information saved in the metadata section of the images. The selected TP had sufficient space around it, making it easier to plan for a good flight path, where the drone during flight has a safe distance from both the selected TP and the surrounding TPs. Figure 2a,b show the drone flight paths and the actual drone and transition piece, respectively. The drone flight path was planned in a local Cartesian coordinate system which had the origin placed in the center of the CAD model of the tower. The number of flight points were selected high enough to ensure the necessary overlap between the images, in both the horizontal and in vertical directions. It was generally accepted, as a rule of thumb, that the minimum overlap between the photos was 60%, while the maximum angle difference between consecutive photos was 15 degrees [12]. The flight points were converted to a georeferenced coordinate system and uploaded to the drone. Figure 2a shows the drone flight paths, while the image acquisitions positions are represented by the blue dots. The images with improved quality were obtained when the camera on the drone was placed in a gimbal stabilizer and was pointing toward the TP during the flight. The drone images were used both for paint defect/damage detection and for performing the 3D model reconstruction. The TP was made of steel, which meant that specular reflection in the tower could produce shiny images that had patches with a color different from the surrounding areas. The TP in the images had very little texture; there were large areas with only slight differences in color. Figure 1: The flowchart shows the different steps needed to calculate the Digital Twin of a large-scale structure and subsequently update the physical structure based on this information. These issues made the 3D reconstruction of the TP very challenging. A possible solution to this challenge could be to optically project a pattern on the tower. Image key points can be used to predict whether the drone images can successfully be used in the model reconstruction process. Image key points were used in photogrammetry software, such as ContextCapture from the Bentley Institute [12], to find the \"interesting\" points in an image and to find the connection between images. The calculation of these keypoints for all of the images is one of the first steps in the model reconstruction process. Keypoints are spatial locations in the image that stand out; they can be an edge or a defect in a surface. Figure 3a,b and Figure 4 show the keypoints represented with green + signs in one of the images of the TP. These image keypoints were calculated using the speeded-up robust features (SURF) algorithm and the scale-invariant feature transform (SIFT) algorithm, as seen in Figures 3 and 4, respectively. Both of these robust feature descriptors are invariant to scale changes, blur, rotation, illumination changes, and affine transformations (see reference [13]). SIFT is better than SURF in different scale images, while SURF is a faster algorithm (see reference [14]). A good reconstruction is expected when the image keypoints are uniformly distributed; however, this is not the case for the example shown in Figure 3a. It is seen that there were very few SIFT and SURF keypoints on the TP, thus making the reconstruction more challenging. There were many keypoints on the highly textured ground close to the tower, which meant that these areas of the scene would be well-captured in the reconstructed model. Both the SIFT and SURF descriptors could find the areas where there were tower paint defects/damages, as seen in Figures 3b and 4b.