# Measurement of Bridge Vibration by UAVs Combined with CNN and KLT Optical-Flow Method
## 2 Methods
### KLT Optical-Flow Method
#### 2.1.2 KLT Optical-Flow Method  
In addition to the two assumptions, the KLT optical-flow method [13] also assumes the spatial consistency; that is, the adjacent pixels in the previous frame are also adjacent in the next frame.  
Suppose \(I_{1}(x,y)\) and \(I_{2}(x,y)\) are two adjacent images; a fixed size window, \(W\), centered on the position of a target point is established in the first image. All pixels in \(W\) move \(\left(d_{x},d_{y}\right)\) between the two images, and \((x,y)\) are the coordinates of pixels. Let \(\mathbf{d}=[d_{x}\;\;d_{x}]^{T}\), \(\mathbf{X}=[x\;\;y]^{T}\). The \(\mathbf{d}\) closest to the actual value can be obtained by minimizing the following expression:  
\[\epsilon=\iint\limits_{W}\left[I_{2}(\mathbf{X})-I_{1}(\mathbf{X}-\mathbf{d})\right]^{2} \omega(\mathbf{X})d\mathbf{X}. \tag{1}\]  
Figure 1: Technical flowchart.  
Figure 2: Motion of points.  
where \(\omega(\mathbf{X})\) is a weighting function. In the simplest case, \(\omega(\mathbf{X})=1\). Alternatively, \(\omega\) could be a Gaussian-like function to emphasize the central area of the window. Move the centers of \(I_{1}\) and \(I_{2}\) by \(\mathbf{d}/2\) to obtain  
\[\varepsilon=\iint\limits_{W}\left[I_{2}\left(\mathbf{X}+\frac{\mathbf{d}}{2}\right)-I_ {1}\left(\mathbf{X}-\frac{\mathbf{d}}{2}\right)\right]^{2}\omega(\mathbf{X})d\mathbf{X}. \tag{2}\]  
Set the partial derivative of \(\varepsilon\) with respect to \(\mathbf{d}\) as 0:  
\[2\iint\limits_{W}\left[I_{2}\left(\mathbf{X}+\frac{\mathbf{d}}{2}\right)-I_{1}\left( \mathbf{X}-\frac{\mathbf{d}}{2}\right)\right]\left[\frac{\partial I_{2}\left(\mathbf{X}+ \frac{\mathbf{d}}{2}\right)}{\partial\mathbf{d}}-\frac{\partial I_{1}\left(\mathbf{X}- \frac{\mathbf{d}}{2}\right)}{\partial\mathbf{d}}\right]\omega(\mathbf{X})d\mathbf{X}=0. \tag{3}\]  
The following formula can be obtained from [PERSON]'s expansion:  
\[I_{2}\left(\mathbf{X}+\frac{\mathbf{d}}{2}\right) \approx I_{2}(\mathbf{X})+\frac{d_{x}}{2}\frac{\partial I_{2}}{ \partial x}(\mathbf{X})+\frac{d_{y}}{2}\frac{\partial I_{2}}{\partial y}(\mathbf{X}). \tag{4}\] \[I_{1}\left(\mathbf{X}-\frac{\mathbf{d}}{2}\right) \approx I_{1}(\mathbf{X})-\frac{d_{x}}{2}\frac{\partial I_{1}}{ \partial x}(\mathbf{X})-\frac{d_{y}}{2}\frac{\partial I_{1}}{\partial y}(\mathbf{X}). \tag{5}\]  
Substitution of Equations (4) and (5) into Equation (3) leads to  
\[\iint\limits_{W}\left[I_{2}(\mathbf{X})-I_{1}(\mathbf{X})+\mathbf{p}^{\top}\mathbf{d}\right] \mathbf{p}(\mathbf{X})\omega(\mathbf{X})d\mathbf{X}=0. \tag{6}\]  
where  
\[\mathbf{p}=\left[\begin{array}{cc}\frac{\partial}{\partial x}\left(\frac{I_{1}+ I_{2}}{2}\right)&\frac{\partial}{\partial y}\;\left(\frac{I_{1}+I_{2}}{2} \right)\right]^{T}. \tag{7}\]  
The following equation can be obtained from Equation (6):  
\[Z\mathbf{d}=e. \tag{8}\]  
where \(Z=\iint_{W}\mathbf{p}(\mathbf{X})\mathbf{p}^{T}(\mathbf{X})\omega(\mathbf{X})d\mathbf{X}\), \(e=\iint_{W}[I_{1}(\mathbf{X})-I_{2}(\mathbf{X})]\mathbf{p}(\mathbf{X})\omega(\mathbf{X})d\mathbf{X}\).  
Equation (8) can be solved by an iterative method to obtain the value of \(\mathbf{d}\). When the value of \(e\) is less than the set threshold, the approximate solution of \(\mathbf{d}\) can be obtained.  
In this paper, the process of tracking the target point on the bridge model by the KLT optical-flow method is as follows:  
1. Select the target point in the initial image;
2. Based on the local template of each target point, the vector of the point between adjacent frames can be found [13];
3. The tracking effect is judged in each image to optimize the result for each target point.