# An Operational Image-Based Digital Twin for Large-Scale Structures
## 6 Defect/Damage Mapping  
The paint defects/damages were mapped, using a novel approach, onto the reconstructed model or as an alternative to the CAD model. The reconstructed/CAD model, with all the defects/damages, could be used to provide an overview of the positioning of the defects/damages in 3D, thus making it possible to identify any systematic in which the defects/damages were placed. This identification could be used in the optimization of production. The AI algorithm identified the pixels in an image that corresponded to paint defects/damages; this 2D information was mapped to the 3D CAD or reconstructed model. The same paint defect/damage could often be seen from a slightly different angle in a number of different images. Each of these pixels with paint defects/damages was mapped to the reconstructed model, thereby increasing the paint defect/damage area on the tower. However, some of the mapped defect/damage points overlapped each other. Three different coordinate systems were used to perform the mapping (see Figure 12a). The world coordinate system (\(x\), \(y\), \(z\)) is shown in the figure, together with the three colored axes of the local camera coordinate system. Moving this coordinate system to the center of the camera sensor results in the third coordinate system. A meshed version of the CAD or reconstructed model was placed in the georeferenced coordinate system that was used during model reconstruction. The local camera coordinate system, where the \(x\)-, \(y\)-, and \(z\)-axis have the colors blue, red and black, respectively, is shown in Figure 12a. The camera angle measured from the optical axis of the camera determined the direction in which the camera is pointing. The green ray in the Figure 12 shows this direction. A simple pinhole camera model provided the relation between an object in the \"real world\" and the corresponding image captured by the camera. The effects of the light-collecting lens in the camera were added to the pinhole camera model. This \"extended\" pinhole camera model was used as shown in Equation (1) below. If \((x^{\prime},y^{\prime})\) are  
Figure 12: (**a**) The CAD model of the TP has been meshed with a triangular mesh and placed in a georeferenced coordinate system. This 3D world coordinates system is shown in the figure, together with a local camera coordinate system in which the \(x\)-, \(y\)-, and \(z\)-axis have the colors blue, red and black, respectively. The z-axis of this local camera coordinate system is the optical axis of the camera. The green ray shows the direction in which the camera is pointing. The red points near and on the tower are the 3D points corresponding to the paint defect/damage pixels, while the green points are the projected points onto the tower. (**b**) The figure zooms in on the projected points. the pixel values of the image, then the 3D world coordinates (\(x\), \(y\), \(z\)) of the 3D object can be calculated using the following equation:  
\[\left( \begin{array}{c}x\\ y\\ z\end{array} \right)=RotM\left( \begin{array}{c}(x^{\prime}-cx)\frac{DCT}{fx}\\ (y^{\prime}-cy)\frac{DCT}{fl}\\ DCT\end{array} \right)+\left( \begin{array}{c}cposx\\ cposy\\ cposz\end{array} \right) \tag{1}\]  
Here, (_cposx_, _cposy_, _cposz_) are the positions of the camera in world coordinates. _DCT_ is the distance from the camera to the tower, in the direction in which the camera is pointing, while _fx_ and _fl_ are the focal lengths of the camera expressed in pixels along the horizontal and vertical directions of the camera sensor, respectively. The two values _cx_ and _cy_ move the pixels values from the upper left corner to the center of the camera sensor. _RotM_ is the rotation matrix that transforms the local camera coordinates to world coordinates. In many applications, the image pixels are calculated from the 3D world coordinates of the object. This calculation is the reverse of the equation given in Equation (1).