## 4 Design of Multipoint Displacement Monitoring Algorithm for Bridge Structure
### The Method of Establishing the Space-Time Relationship of Image Sequence Data
#### 4.2.2 Target Tracking and Displacement Calculation  
In the previous research, the static image is used to extract the contour of the bridge structure at different times and under different load-damage conditions, and carry out stacking to obtain the deformation of the bridge structure. Given that this method has many manual interventions, the optical flow optical flow algorithm which is widely used in computer vision, was adopted for target tracking and displacement calculation in the static image sequence data collection [48; 49]. The algorithm using optical flow must satisfy two hypotheses: (1) constant brightness: The brightness of the same point does not change with time; (2) small motion: The position will not change drastically with over time, such as finding the derivative of grayscale relative to the position. In our research, both hypotheses were satisfied by the data collected by the NRS. The brightness of each point in the collected images remained constant because of the small data interval of the seven time sequences; the small motion was also fulfilled, and the bridge structure had a limited deformation for the monitoring object. Basic constraint equation. Consider the light intensity of a pixel\(f(x,y,t)\) on the first picture (where \(t\) represents its time dimension). It moves the distance (\(dx\), \(dy\)) to the next picture because it is the same pixel point, and according to the first assumption above, the light intensity of the pixel before and after the motion is constant. \[f(x,y,t)=f(x+dx,y+dy,t+dt) \tag{1}\]  
The Taylor expansion on the right end of Formula (1) is as follows:  
\[f(x+dx,y+dy,t+dt)=f(x,y,t)+\frac{\partial f}{\partial x}dx+\frac{\partial f}{ \partial y}dy+\frac{\partial f}{\partial t}dt+\varepsilon \tag{2}\]  
where \(\varepsilon\) represents the second order infinitesimal term, which could be ignored; then (2) is substituted into (1) and divided by \(dt\):  
\[\frac{\partial f}{\partial x}\frac{dx}{dt}+\frac{\partial f}{\partial y}\frac {dy}{dt}+\frac{\partial f}{\partial t}=0 \tag{3}\]  
Figure 16: Description of text bridge feature points. Let \(f_{x}=\frac{\partial f}{\partial x}\), \(f_{y}=\frac{\partial f}{\partial y}\), \(f_{t}=\frac{\partial f}{\partial t}\) represent the partial derivatives of the gray level of the pixels in the image along the \(x\), \(y\), and \(t\) directions. Summing up,  
\[f_{x}u+f_{y}v+f_{t}=0 \tag{4}\]  
where, \(f_{x}\), \(f_{y}\), and \(f_{t}\) can be obtained from the image data, and (\(u\), \(v\)) is the optical flow vector that must be solved. At this time, there is only one constraint equation but two unknowns. In this case, the exact values of \(u\) and \(v\) cannot be obtained. Constraints need to be introduced from another perspective. The introduction of constraints from different angles leads to different methods of optical flow field calculation. According to the difference between the theoretical basis and the mathematical method, they are divided into four kinds: Gradient-based method, matching-based method, energy-based method, phase-based method, and neurodynamic method. In addition to distinguishing the optical flow method according to different principles, the optical flow method can also be divided into dense optical flow and sparse optical flow according to the density of a two-dimensional vector in the optical flow field. Dense optical flow is a kind of image registration method that matches the image or a specific area point by point. It calculates the offset of all the points on the image to form a dense optical flow field. Through this dense optical flow field, pixel level image registration can be performed. In contrast to dense optical flow, sparse optical flow does not calculate every pixel of the image point by point. It usually needs to specify a group of points for tracking, which is better to have some obvious characteristics, such as the Harris corner, for a relatively stable and reliable tracking. The computation cost of sparse tracking is much less than that of dense tracking.