# Image Analysis Applications for Building Inter-Story Drift Monitoring
## 2 Methods
The image analysis techniques employed in this work include camera calibration, target tracking, and point projection. Camera calibration is used to estimate the parameters of the camera, including the intrinsic parameters and the extrinsic parameters. The intrinsic parameters include the focal lengths \(f_{x}\) and \(f_{y}\), the principal points \(c_{x}\) and \(c_{y}\), the distortion coefficients \(k_{1}\)\(to\)\(k_{6}\), \(p_{1}\), and \(p_{2}\). The extrinsic parameters include \(r_{x}\), \(r_{y}\), \(r_{z}\), \(t_{x}\), \(t_{y}\), and \(t_{z}\), which represent the geometric relationship between the camera and the global coordinate system. While the global coordinate system can be defined arbitrarily, it must be consistent throughout the entire process. It is suggested that the global coordinate system be aligned to the coordinate system used in the engineering drawing of the structure, so the engineers who will use this system have a consistent coordinate system when using the proposed system. The definition of the  
Figure 4: Demonstration of images taken by a camera that represent different inter-story responses and camera movement. (**a**) Un-deformed inter-story. (**b**) Response to \(u\). (**c**) Response to \(v\). (**d**) Response to torsion \(\theta\). (**e**) Camera pitch. (**f**) Camera yaw. (**g**) Camera roll. global coordinate system is especially important in the installation phase of the monitoring system (see Step 1 and 2 described below). Target tracking allows the defining of the tracking and reference points in the initial image (where the inter-story is un-deformed) and automatically and precisely determines where these points are in later images taken by the same camera, during which the inter-story may have deformed. Many types of target tracking methods have been developed and are widely employed, such as template matching [50], enhanced correlation correction [51], optical flow [52], feature-based image alignment [53], machine learning based methods [54], and many other variations of these methods. These methods have been thoroughly developed in the C++ open source computer vision library OpenCV [55,56]. As compared to other methods, the optical flow method is relatively fast and gives subpixel precision and is thus employed in this work. Given the 3D coordinate of an arbitrary point and the camera parameters, the point projection method calculates the theoretical position that this point should appear in the image. The aforementioned image analysis techniques have been well-developed by using OpenCV [55,56], and thus, the programming implementation developed in this study adopts this library. The technical details of this method can be found in many previous studies [55] and are not presented in this study. This work integrates the inter-story motion model (Figure 4b-d), camera movement model (Figure 4e-g), the aforementioned computer vision techniques, and an iterative process that solves the nonlinear simultaneous equations. The equations relate the inter-story responses, camera movements, camera parameters, image coordinates, and 3D coordinates of the reference and  
point \(c_{x}\) and \(c_{y}\), the distortion coefficients \(k_{1}\)_to_\(k_{6}\), \(p_{1}\), and \(p_{2}\), and the extrinsic parameters \(r_{x}^{t}\), \(r_{y}^{t}\), \(r_{z}^{t}\), \(t_{x}^{t}\), \(t_{y}^{t}\), and \(t_{z}^{t}\) that represent the geometric relationship between the camera and the global coordinate system, where the superscript \(t\) is zero at Step 3. The mathematical meanings of these parameters can be found in many computer vision textbooks [55], and a brief introduction is presented in [29]. While the mathematics of these parameters may seem complicated, the idea is simple: Once we have these parameters, we know the geometry of how any arbitrary 3D object in the global coordinate system would be projected onto the image through this camera. The software implementation of the camera calibration is also included in the software prototyped in this work.