# Image Analysis Applications for Building Inter-Story Drift Monitoring
## 2 Methods
By giving the image coordinates and the 3D coordinates of the reference and tracking points, the camera parameters are obtained by using the camera calibration technique [59]. In the prototyped software, the initial image that represents the initial un-deformed status is also obtained in this step. In this study, we use \(\mathbf{I}_{Ref(m)}^{t}\) and \(\mathbf{I}_{Tk(n)}^{t}\) to represent the image coordinates of the \(m\)th reference points and the \(n\)th tracking points at time \(t\), respectively. * Acquire the current image. By using the point tracking technique, the image coordinates of each reference point \(\mathbf{I}_{Ref(m)}^{t}\) and tracking points \(\mathbf{I}_{Tk(n)}^{t}\) are obtained. In this work, the optical flow algorithm as implemented in OpenCV [60] is used to track the points. * Guess the inter-story response and the camera movement. In general, the analyzed inter-story response and camera movement of the previous time step is taken as the initial guess. If it is the first time step of the analysis, the inter-story response and the camera movement are guessed as zeros. \(\mathbf{U}^{t}\) is used to represent the guessed inter-story response at time t, which is a 3-by-1 matrix \(\left\langle u^{t}\ v^{t}\ \theta^{t}\right\rangle^{\text{T}}\), and \(\mathbf{C}_{R}^{t}\) is used to represent the guessed camera movement, which is a 3-by-1 matrix \(\left\langle r_{pitch}^{t}\ r_{yaw}^{t}\ \ r_{roll}^{t}\right\rangle^{\text{T}}\). The superscript T represents matrix transpose. This study uses a 1-by-6 matrix \(\mathbf{d}^{t}\) to represent the combination of \(\mathbf{U}^{t}\) and \(\mathbf{C}_{R}^{t}\), as shown in the following Equation. \(\mathbf{d}^{t}\) is adjusted in Step 10. * Calculate the 3D coordinates of each of the tracking points \(\mathbf{X}_{Tk(n)}^{t}\) according to the guessed inter-story response \(\mathbf{U}^{t}\) using the inter-story model. The following equation is used to calculate the 3D coordinates of these points, where \(\mathbf{X}_{c}\) is the torsional center of the monitored story, which can be obtained from the modal analysis result of the structural analysis. If the structural analysis is not available, the torsional center can be replaced with the centroid of the monitored story. The inter-story drift of a point induced by its inter-story torsion is at the order of the torsion (in unit of radian) times the distance between the point and the torsional center. Since the quantity of inter-story torsion is typically small (i.e., normally much smaller than a thousandth radians), an inaccurately setting of torsional center, if that is about a few millimeters inaccurate, only induces a relatively small errors of displacements. \[\mathbf{X}_{Tk(m)}^{t}=\left(\left( \begin{array}{ccc}cos\theta^{t}&-sin\theta^ {t}&0\\ sin\theta^{t}&cos\theta^{t}&0\\ 0&0&1\end{array} \right)\left(\mathbf{X}_{Tk(m)}^{0}-\mathbf{X}_{c}\right)\right)+\mathbf{X }_{c}+\left( \begin{array}{c}u^{t}\\ v^{t}\\ 0\end{array} \right).\] (2)
* Calculate the extrinsic parameters \(r_{x}^{t}\), \(r_{y}^{t}\), \(r_{z}^{t}\), \(t_{x}^{t}\), \(t_{y}^{t}\), and \(t_{z}^{t}\) at time \(t\) according to the unintended camera movement. The unintended camera movement changes the geometric relationship  
\[k=\frac{1+k_{1}\big{(}x_{n}^{2}+y_{n}^{2}\big{)}+k_{2}\big{(}x_{n}^{2}+y_{n}^{2} \big{)}^{2}+k_{3}\big{(}x_{n}^{2}+y_{n}^{2}\big{)}^{3}}{1+k_{4}\big{(}x_{n}^{2}+y _{n}^{2}\big{)}+k_{5}\big{(}x_{n}^{2}+y_{n}^{2}\big{)}^{2}+k_{6}\big{(}x_{n}^{2}+ y_{n}^{2}\big{)}^{3}} \tag{9}\]  
\[\left( \begin{array}{c}x_{d}\\ y_{d}\end{array} \right)=\left( \begin{array}{cc}k+2p_{1}y_{n}+3p_{2}x_{n}&p_{ 2}y_{n}\\ p_{1}x_{n}&k+2p_{2}x_{n}+3p_{1}y_{n}\end{array} \right)\left( \begin{array}{c}x _{n}\\ y_{n}\end{array} \right), \tag{10}\]  
and  
\[\left( \begin{array}{c}x_{i}\\ y_{i}\\ 1\end{array} \right)=\left( \begin{array}{ccc}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{array} \right)\left( \begin{array}{c}x_{d}\\ y_{d}\\ 1\end{array} \right) \tag{11}\]  
* Compare the projected image coordinates and the actual image coordinates.