## 4 Three-Dimensional World Point Reconstruction and Structural Displacement Extraction
### Camera Pose Recovery  
The 6-DoF camera pose is defined by the camera extrinsic matrix \([R\,|\,t]\), where \(R\) is a 3 \(\times\) 3 orthogonal matrix representing the camera orientation and \(t\) is a 3 \(\times\) 1 vector representing the camera position in world coordinates. In the proposed method, the camera pose in each video frame is recovered based on the Perspective-\(n\)-Point (P\(n\)P) method followed by the Levenberg-Marquardt algorithm.  
The Perspective-\(n\)-Point (P\(n\)P) method estimates the pose of a calibrated camera from \(n\) 3-D world points with known positions and their corresponding 2-D image points. Although the P\(n\)P method requires a minimum of only four non-coplanar world-to-image point correspondences (three non-colinear correspondences if the points are coplanar) for obtaining the unique camera pose, more point correspondences can enhance the robustness and accuracy of the estimated camera pose due to the measurement noise in both world points and image points. The proposed method adopts an accurate and scalable solution to the P\(n\)P problem (ASP\(n\)P) [46] to estimate the camera-pose changes over time. The image and world point coordinates of 10 stationary reference targets (8 targets on the ground and 2 targets on the safety towers, which are identified as the blue crosses in Figure 7) are used as the world-to-image correspondences in each video frame. The image point coordinates are obtained as the target detection and tracking algorithm described in Section 3. The 3-D world point coordinates are extracted from a geo-referenced point cloud model of the building specimen and the background region of the test scene at the LHPOST6 (Figure 9). The point cloud model is generated based on the photogrammetry results from the same UAV and camera combination for the plan-view test video capture (i.e., the DJI Matrice 300 UAV with the DJI Zenmuse P1 camera) using a preprogrammed flight plan. The photogrammetry flight pattern is determined case by case in practice based on the complexity of the scene [47; 48]. For the present study, the image acquisition consists of 209 images in a lawnmower pattern using nadir imagery and oblique imagery (40 degrees off the vertical) with a target 75% overlap and sidelap at the roof level. Photogrammetric processing is performed using Agisoft Metashape 2.0.2 [43]. The resulting point cloud consists of around 480 million points with a nominal point spacing of 6 mm.  
As a non-iterative method for camera pose recovery, ASP_n_P obtains the optimal camera pose by solving a polynomial system derived from the first-order optimality condition for the image-to-world reprojection error [46]. However, non-iterative P_n_P solutions can still be improved by some iterative methods such as the Gauss-Newton algorithm or the Levenberg-Marquardt algorithm [49; 50]. In the proposed method, the Levenberg-Marquardt (LM) algorithm [51; 52] initialized with the ASP_n_P solution is applied as a nonlinear optimization process for the camera pose. The 6-DoF camera pose can be accurately estimated based on this ASP_n_P-LM process. Figure 10 shows the camera translational motion trajectory of the onboard camera for plan-view video capture during MID 15. According to the estimated camera motion trajectory, the UAV for plan-view video capture hovered at around 55 m above the shake table platen. The UAV had a drift of less than 0.2 m in both horizontal directions and a drift of less than 0.1 m in vertical direction.  
Figure 10: Camera translational motion trajectory for the plan-view video of MID 15.  
Figure 9: Point cloud model of the test scene at the LHPOST6 from photogrammetry and illustration of the 3-D world coordinates (the yellow dot represents the center of the shake table platen, which is the origin of the world coordinates, and the blue lines represent the XYZ coordinate axes). Note that \(Z=0\) is defined as the top of the shake table platen.