# An Operational Image-Based Digital Twin for Large-Scale Structures
## 6 Defect/Damage Mapping
Parameters such as camera positions and rotation matrices can often be extracted from the photogrammetry software for every image used in the reconstruction. The simple pinhole camera model and the \"extended\" pinhole camera models calculating the image pixel values from the 3D world coordinates are explained in the course notes found in reference [16]. The effect of distortion is seen in the images from the drone. Vertical structures placed in the center of the image are reproduced as vertical structures, but when these structures were placed near the edges of the images, they tended to be slightly curved, showing signs of distortion. Radial and tangential lens distortion can be added to the \"extended: pinhole model provided in Equation (1). Particularly small lenses suffer from radial distortion when the light rays bend closer to the edges of a lens than when they bend closer to the center of a lens. Tangential distortion occurs when the lens and the image plane are not parallel. To account for radial and tangential distortions, the contributions to pixel values must be calculated. If the dimensionless pixel values _dlx_ and _dly_ used in Equation (1) are given by  
\[ \begin{array}{c}dlx=\frac{(x^{\prime}-cx)}{fx}\\ dly=\frac{(y^{\prime}-cy)}{fl}\end{array}  \tag{2}\]  
then the radial (_xrdis_, _yrdis_) and tangential distortions (_xtdis_, _ytdis_) are provided by the following set of equations:  
\[ \begin{array}{c}xrdis=dlx(k_{1}r^{2}+k_{2}r^{4}+k_{3}r^{6})\\ yrdis=dly(k_{1}r^{2}+k_{2}r^{4}+k_{3}r^{6})\\ xtdis=2p_{1}dx\,dly+p_{2}\big{(}r^{2}+2 dlx^{2}\big{)}\\ ytdis=p_{1}\big{(}r^{2}+2 dly^{2}\big{)}+2p_{2}dlx\,dly\end{array}  \tag{3}\]  
Here, \(r=\sqrt{dlx^{2}+dly^{2}}\), and \(k_{1}\), \(k_{2}\) and \(k_{3}\) are the radial distortion coefficients of the lens, while \(p_{1}\) and \(p_{2}\) are the tangential distortion coefficients of the lens. These distortion parameters can normally also be extracted from the photogrammetry software. It is seen that the magnitude of the distortion correction in Equations (2) and (3) will increase with the value of the radius \(r\). The distance _DCT_ from the camera to the tower needed in the equations above can be calculated because both the camera and the tower positions are known. The distortion of the paint damage pixels found from the AI routine are provided by Equation (3). The dimensionless pixel values dlx and dly are corrected by these values before they are used in Equations (1) and (2) to calculate the 3D world coordinates of the paint damages. The red points in Figure 12a,b near and on the tower are the world coordinates corresponding to the paint defect/damage pixels calculated using Equations (1)-(3). Not all of the points are on the tower (see Figure 12b), because depth information for all of the the pixel values in the image is needed in order to perform a precise image pixel to world coordinates transformation. The green points in the figure are calculated as a projection of the red points onto the tower. These green points represent the mapping of the paint defect/damage pixels found in one of the drone images onto the CAD model of the transition piece. Figure 13a,b shows the CAD model superimposed with the paint defects/damages found in all the drone images for this particular flight. It is seen that the defects/damages are not distributed uniformly, and there is a higher concentration of paint defects/damages near the bottom of the tower than in the middle and upper sections of the tower.