# The Dynamic Displacement Monitoring Method Using Unmanned Aerial Vehicles Based on Digital Close-Range Photogrammetry

Authors: Xianzhi Yu; Yongquan Ge; Tonglong Zhao; Guojian Zhang; Yingchun Liu; Chengxin Yu
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

How to carry out fast and reliable dynamic deformation monitoring of building structures is an important issue for engineers and technicians. Digital close-range photogrammetry technology has the advantages of noncontact, multipoint, and rapid monitoring, and the application of UAVs makes the monitoring task no longer limited by the size of the field, but the current technology can not guarantee the rapid deployment of UAVs in dynamic deformation monitoring, and these methods commonly have higher requirements for image quality, and cannot ensure the stability of accuracy. To this end, we propose a dynamic displacement monitoring method for UAVs based on digital close-range photogrammetry technology and optimize the data processing process for the problem that there is still moving when the UAV hovers. A building with an overhead corridor structure with a large flow of people was selected for the experiment. The experiment used ground digital cameras and drones to monitor separately to compare the monitoring results. The results show that the proposed method is simple and easy to use. The monitoring method can be deployed quickly, which is expected to be applied to emergency monitoring scenarios.


---

## Body

## 1. Introduction

On May 5, 2020, wind vibration occurred on the Humen Bridge (Figure1), when the natural wind speed was 10–16 m/s. The wind-induced vibration was diminished after removing the water horse on the bridge side. On May 6, the Humen Bridge vibrated again for an unknown cause. This Humen bridge passage appeared for the first time in 23 years, with nearly half a metre of wind vibration amplitude of the phenomenon. On May 12, experts published bridge vibration reasons: continuous set along the bridge across the fence water horse changed the steel box girder of aerodynamic shape. In particular, the wind environment conditions, the bridge of vortex vibration phenomenon and said the key bridge component is not an exception occurs, the bridge structure overall security. On May 16, the Humen Bridge was reopened to traffic. On May 18, 2021, an uncommon vibration occurred in the Shenzhen SEG Plaza (Figure 2) at the wind speed of about 10 m/s, and the market management department and the local government quickly decided to evacuate and suspend the operation. On July 15, an expert group organized by Shenzhen Municipal Housing and Construction Bureau came to a conclusion. The most direct cause of the sensible vibration of the SEG Plaza is the coupling of vortex-induced resonance caused by the wind of the mast and the change of dynamic characteristics of the building and the mast. Authorities removed the mast and restored the accumulated damage.Figure 1
The humen bridge.Figure 2
The shenzhen SEG plaza.It can be seen from these two cases that the dynamic deformation of buildings can reflect the safety performance of buildings to a certain extent. Therefore, engineers and technicians need to obtain the abnormal deformation of the structure before the accident and give early warning of the danger, to protect the safety of people’s lives and property. In addition to the need for construction of the low frequency and static deformation (such as building settlement and inclination) monitoring, the need for building dynamic deformation (such as building under strong wind and earthquake deformation) monitoring, and dynamic deformation always occurs quickly, so how to a rapid and reliable method for dynamic deformation monitoring of the building, apparently it is of great significance.Conventional deformation measurement technology mainly includes using the total station and other traditional instruments for deformation measurement, which is suitable for dynamic deformation measurement with high precision requirements, long deformation period, and low transformation rate. For buildings with the short deformation period and high deformation rate, displacement sensors and acceleration sensors can be installed. Currently, it is possible to install hundreds of sensors in a masonry structure and get data in real-time from different places performing continuous monitoring [1]. However, this is a contact measurement, and the sensor will cause irreversible damage to the monitored structure when installed. Using GNSS (Global Navigation Satellite System) to monitor the deformation of various large building structures, but the accuracy of this monitoring method and the integrity and availability of the measurement range is easily affected by the distribution of satellite systems. Monitoring cannot be implemented in places where the satellite signal is out of lock [2].Using the digital close-range photogrammetry technology to monitor the dynamic deformation of buildings can achieve high frequency, multi-point, and contactless observation and has been successfully applied to Bridges [3, 4], shuttle steel shelves [5], Masonry Walls in Seismic Oscillation Outdoors [6], Steel structure building [7], super high rise building [8], and so on. The technologies of measuring instrument is a digital camera, digital camera in these experiments showed high-frequency measurement and monitoring of the advantages of stable performance, but due to the digital camera can only be placed on the solid ground, so in the individual monitoring scene, this method also showed the perspective and monitoring field of constrained problem. Although the monitoring results can be modified through the Small acute Angle Method [9], in practice, the monitoring flexibility is still largely subject to the field environment. In [10], we proposed The Improved Zero-centered Motion Parallax Method with high-frequency monitoring without training data. However, in practical application, it is found that UAV still has movement problem after hovering, which has the influence on monitoring results. Therefore, we propose a data processing optimization method. To verify the validity of the method, we selected a building for the experiment. The elevated corridor of the building was taken as the primary research object. A Digital camera and a UAV were used to monitor their dynamic deformation. The experimental results show that the displacement trends and laws of the UAV monitoring results after data processing optimization are the same as those of the digital camera monitoring results, indicating that the data processing optimization method proposed in this paper is effective.


## 2. Related Works

With the development of UAV technology, it is possible to use UAVs to monitor the deformation of buildings [10]. At present, there are abundant research studies on UAV Assisted Structural Health Monitoring (UASHM) [11]. However, when a UAV monitor the target’s dynamic displacement, it will inevitably encounter the problem that the UAV still moves after hovering. To reduce the impact of this problem on the monitoring results, relevant scholars have carried out some targeted studies on the technologies used.Weng et al. proposed a homography-based approach to estimate the dynamic displacement of a structure from video taken by a UAV-mounted camera [12]. The transformation matrix is calculated by the coordinates of homonymous feature points in different images to weaken the influence of UAV movement on the monitoring results, but the technology is susceptible to the influence of illumination transformation. Hoskere et al. proposed the method of using a UAV to monitor the vibration of civil foundation buildings [13]. In order to improve the accuracy of UAV monitoring, they divided the monitoring area into monitoring areas one by one and used the ArUco marker to estimate the camera attitude to weaken the movement of the UAV. They use high-pass filters to remove low frequency vibrations. The study was eventually tested on the Little Golden Gate Bridge (suspension bridge) in Muhammad, Illinois and compared with the results recorded by an accelerometer mounted on the bridge, it was concluded that the method was effective in monitoring structural vibration frequencies. Catt et al. developed a UAV to carry digital image correlation cameras and showed a system that can measure deformations of structures [14]. The movement of the UAV also affects the monitoring of feature points, so this study by Catt et al. adopts the manual tracking method to reduce errors. The method proposed in this paper does not need specific marks, only the reference points and monitoring points can be identified. The monitoring method proposed in this paper can be implemented quickly, so it is expected to be used for emergency monitoring.


## 3. Measurements and Methods

### 3.1. Introduction of Measurements

The measuring instrument used in this experiment is a UAV (Figure3) and a digital camera (Figure 4). Tables 1 and 2 show the specific parameters of this UAV and digital camera.Figure 3
DJI phantom 4 pro.Figure 4
Digital cameraα350.Table 1
Parameters of the DJI phantom 4 pro [15].
CMOS size1 inchVideo resolution3840 × 2160 30 fpsFocal length8.8 mmAperturef/2.8–f/11Focus distance1 m∼∞Angular jitter±0.02°Hover precision (Visual positioning works)Vertical: ±0.1 mHorizontal: ±0.3 mTable 2
Parameters of the sony-α350 camera [16].
Model IDDSLR-A350Image sensorCCDImage sensor dimension23.5 × 15.7 mmImage dimension4592 × 3056(14.0 MP)Len typeMinolta/Sony AF DT 18–70 mm F3.5–5.6 (D)


### 3.2. The Zero-Centered Motion Parallax Method

As shown in Figure5, there are three virtual planes, which are the image plane, the object plane, and the reference plane. The camera sensor is located on the image plane, and all the monitoring points are located on the object plane, also called the monitored plane. All the reference points are located on the reference plane. The function of the reference plane is to eliminate systematic errors [10].Figure 5
The Z-MP method (part one).Suppose a monitoring point on the object plane move from position A to position B, which results in the displacementΔX of the X-axis and ΔZ of the Z-axis on the object plane. Meanwhile, the monitoring point from position a to position b, the X-axis, and the Z-axis also have displacements Δx and Δz on the reference plane.If the monitoring point on the object plane is moved fromA to B, its deformation and on the reference plane are(1)ΔXr=yfΔXiΔZr=yfΔZi,where f is the principal distance of the photo, where y is the distance of the image plane and the reference plane. ΔXi and ΔZi are the horizontal and vertical deformations of the monitoring point on the image plane. ΔXr and ΔZr are the horizontal and vertical deformations of the monitoring point on the reference plane.Although UAV can make observation Angle more flexible, it also brings more systematic error, and the most significant systematic error is the drift error of UAV hovering. For the system error caused by UAV hovering, adjustment of kinematic observations is proposed to weaken the system error.Strictly speaking, the photos before and after deformation are always taken under different elements of interior and exterior orientation, the interior orientation elements in a photo are the elements for restoring the shape of the photographic beam, and the exterior orientation in a photo are the elements for determining the position and orientation of the photographic beam in the space coordinate system [17]. In the experiment, we keep the UAV as stable as possible and set the digital camera in UAV mode to “manual.” Therefore, the error caused by the inconsistency of elements of interior and exterior orientation will have a slight impact on the monitoring results; we weaken the theoretical error to a certain extent by setting control points. To be exact, we weakened Δxp0; the specific process is as follows.On the reference plane, if corresponding monitoring points in the zero image and successive image arex1,z1 and x2,z2, compared with the ideal image which without errors of camera external and internal parameters, systematic errors of corresponding monitoring point are dx1,dz1 and dx2,dz2, respectively. The equations can be expressed as follows:(2)x2−x1−dx2−dx1−Δx=0z2−z1−dz2−dz1−Δz=0.If there are errors of camera external and internal parameters between the zero image and successive images, the control points located on the reference plane will generate parallaxΔxp as follows:(3)Δxp=x2−x1=dx2−dx1Δzp=z2−z1=dz2−dz1.The parallax and the control point must be caused by the errors of camera external and internal parameters in successive zero images, and we take parallax for example, it can be expressed as follows:(4)Δxp=x2−x1=dx2−dx1=−fYdXS2+dYS2Yx2+f+x22fdϕ2+x2z2fdω2−z2dk2+x2fdf2+dx02+fYdXS1+dYS1Yx1+f+x12fdϕ1+x1z1fdω1−z1dk1+x1fdf1+dx01.where dXS2,dYS2,dϕS2,… and dXS1,dYS1,dϕS1,… are errors of zero image and successive images. The detailed derivation process is shown in reference [18]. From equation (4), we notice x2=x1+Δxpx2=x1+Δxp, assume the difference between the errors of zero image and successive images as follows:(5)ΔXS=dxS2−dxs1ΔYS=dYS2−dYS1Δϕ=dϕ2−dϕ1Δω=dω2−dω1Δκ=dκ2−dκ1Δf=df2−dfΔx0=dx02−dx01.Then, equation (4) can be expressed as follows:(6)ΔxS=−fYΔXS−dYSYx1−fΔϕ−x12fΔϕ−x1z1fΔω+z1Δκ−Δffx1−Δx0−ΔxpdYS2Y−Δxp2x1fdϕ2−Δxpz1fdω2−Δzpx1fdω2+Δzpdκ2−Δxpdf2f.After sorting out equation (6), it can be expressed as follows:(7)Δxp=Δxp0+δxpΔxp0=−dYSY−Δffx1+z1Δκ+−fYΔXS−fΔϕ−Δx0−x12fΔϕ−x1z1fΔωδxp=−ΔxpdYs2Y−2Δxpx1fdϕ2−Δxpz1fdω2−Δzpx1fdω2+Δzpdω2+Δzpdκ2−df2fΔxp.Because motionΔxp0Δpx0 is caused by the change of camera external and internal parameters ΔXS,ΔZS,Δϕ,Δω,Δκ,Δf,Δx0ΔXs,ΔZs,Δϕ,Δω,Δκ,Δf,Δx0 in the successive and zero images, we can correct δxp0. with a sufficient number of control points. Differently, motion δxp. is caused by the interaction of Δxp. , Δzp, and camera external and internal parameters dZS2,dϕ2,dω2,dκ2,df2 in the successive image. However, and are different at each point, control points cannot be used to correct, so we only discuss as follows:(8)Δxp0=−ΔYSY−Δffx1+Δkz1+−fYΔXS−fΔϕ−Δx0−x12fΔϕ−x1z1fΔω.We can express equation (8) as follows:(9)Δxp0=ax+bz+c+dx2+exz.If there are more than five control points, each unknown coefficienta,b,c,d,e can be obtained according to their. We assume the correction of is, so the error equation is(10)v=ax+bz+c+dx2+exz−Δx.For convenience, we selected the linear part of the equation (10) for processing, as follows:(11)Δxp0=ax+bz+cΔzp0=a′x+b′z+c′.In this case, we only need three or more reference points to obtaina,b,c and a′,b′,c′. Take ΔPx0 as an example, When contains only occasional errors, equation (11) can be expressed as follows:(12)px′+v=ax′+bz′,where px′ is the differential coefficient of ΔPx0. The error equation is(13)V=ax′+bz′−px′.The equation of the composition method is(14)a∑x′2+b∑x′z′−∑x′px′=0,a∑x′z′+b∑z′2−∑z′px′=0.Calculate barycentric coordinates by control points on the reference plane, as follows:(15)xs=x−∑xnzs=z−∑zn.Because coordinates of control points are barycentric coordinates,∑xS∑zS=0 and the parallax coefficient in the X direction as follows:(16)a=∑zs′2∑xs′px′−∑xs′zs′∑zs′px′∑xs′2∑zs′2−∑xs′zs′∑xs′zs′,b=∑xs′2∑zs′px′−∑xs′zs′∑xs′px′∑xs′2∑zs′2−∑xs′zs′∑xs′zs′,where a=tanϕx and b=tanωz. Similarly, we can obtain the parallax coefficient a′ and b′ in the z direction. Then, we can obtain Δxp0 and Δzp0. Finally, we figure out the value of ΔXi as follows:(17)ΔXi≈Δxp0≈ax+bzΔZi≈Δzp0≈a′x+b′z.The displacement of the monitoring point on the reference plane after the first correction is(18)ΔXr=yfΔXiΔZr=yfΔZi.On the basis of the above work, we use reference points on the reference plane to further reduce the error, assume the coordinates of the 4 reference points are(19)Xnri,Ynri,i=1,2,3,4,where n is the photo number.If the UAV moves slightly and passively on a plane parallel to the object plane, the positions of the 4 reference points in the image coordinate system will inevitably change, and this change can be calculated under ideal conditions as follows:(20)Xnr¯=∑i=14Xnri4,Ynr¯=∑i=14Ynri4.We need to subtract these two valuesXnr¯ and Ynr¯ before we calculated the displacement of the monitoring point in the object plane.(21)ΔX0=YyΔXr−Xnr¯ΔZ0=YyΔZr−Znr¯.Finally, the displacementΔX0 and ΔZ0 of the final monitoring point can be obtained through the proportional conversion relationship as Figure 6 shows.Figure 6
The Z-MP method (part two).


## 3.1. Introduction of Measurements

The measuring instrument used in this experiment is a UAV (Figure3) and a digital camera (Figure 4). Tables 1 and 2 show the specific parameters of this UAV and digital camera.Figure 3
DJI phantom 4 pro.Figure 4
Digital cameraα350.Table 1
Parameters of the DJI phantom 4 pro [15].
CMOS size1 inchVideo resolution3840 × 2160 30 fpsFocal length8.8 mmAperturef/2.8–f/11Focus distance1 m∼∞Angular jitter±0.02°Hover precision (Visual positioning works)Vertical: ±0.1 mHorizontal: ±0.3 mTable 2
Parameters of the sony-α350 camera [16].
Model IDDSLR-A350Image sensorCCDImage sensor dimension23.5 × 15.7 mmImage dimension4592 × 3056(14.0 MP)Len typeMinolta/Sony AF DT 18–70 mm F3.5–5.6 (D)


## 3.2. The Zero-Centered Motion Parallax Method

As shown in Figure5, there are three virtual planes, which are the image plane, the object plane, and the reference plane. The camera sensor is located on the image plane, and all the monitoring points are located on the object plane, also called the monitored plane. All the reference points are located on the reference plane. The function of the reference plane is to eliminate systematic errors [10].Figure 5
The Z-MP method (part one).Suppose a monitoring point on the object plane move from position A to position B, which results in the displacementΔX of the X-axis and ΔZ of the Z-axis on the object plane. Meanwhile, the monitoring point from position a to position b, the X-axis, and the Z-axis also have displacements Δx and Δz on the reference plane.If the monitoring point on the object plane is moved fromA to B, its deformation and on the reference plane are(1)ΔXr=yfΔXiΔZr=yfΔZi,where f is the principal distance of the photo, where y is the distance of the image plane and the reference plane. ΔXi and ΔZi are the horizontal and vertical deformations of the monitoring point on the image plane. ΔXr and ΔZr are the horizontal and vertical deformations of the monitoring point on the reference plane.Although UAV can make observation Angle more flexible, it also brings more systematic error, and the most significant systematic error is the drift error of UAV hovering. For the system error caused by UAV hovering, adjustment of kinematic observations is proposed to weaken the system error.Strictly speaking, the photos before and after deformation are always taken under different elements of interior and exterior orientation, the interior orientation elements in a photo are the elements for restoring the shape of the photographic beam, and the exterior orientation in a photo are the elements for determining the position and orientation of the photographic beam in the space coordinate system [17]. In the experiment, we keep the UAV as stable as possible and set the digital camera in UAV mode to “manual.” Therefore, the error caused by the inconsistency of elements of interior and exterior orientation will have a slight impact on the monitoring results; we weaken the theoretical error to a certain extent by setting control points. To be exact, we weakened Δxp0; the specific process is as follows.On the reference plane, if corresponding monitoring points in the zero image and successive image arex1,z1 and x2,z2, compared with the ideal image which without errors of camera external and internal parameters, systematic errors of corresponding monitoring point are dx1,dz1 and dx2,dz2, respectively. The equations can be expressed as follows:(2)x2−x1−dx2−dx1−Δx=0z2−z1−dz2−dz1−Δz=0.If there are errors of camera external and internal parameters between the zero image and successive images, the control points located on the reference plane will generate parallaxΔxp as follows:(3)Δxp=x2−x1=dx2−dx1Δzp=z2−z1=dz2−dz1.The parallax and the control point must be caused by the errors of camera external and internal parameters in successive zero images, and we take parallax for example, it can be expressed as follows:(4)Δxp=x2−x1=dx2−dx1=−fYdXS2+dYS2Yx2+f+x22fdϕ2+x2z2fdω2−z2dk2+x2fdf2+dx02+fYdXS1+dYS1Yx1+f+x12fdϕ1+x1z1fdω1−z1dk1+x1fdf1+dx01.where dXS2,dYS2,dϕS2,… and dXS1,dYS1,dϕS1,… are errors of zero image and successive images. The detailed derivation process is shown in reference [18]. From equation (4), we notice x2=x1+Δxpx2=x1+Δxp, assume the difference between the errors of zero image and successive images as follows:(5)ΔXS=dxS2−dxs1ΔYS=dYS2−dYS1Δϕ=dϕ2−dϕ1Δω=dω2−dω1Δκ=dκ2−dκ1Δf=df2−dfΔx0=dx02−dx01.Then, equation (4) can be expressed as follows:(6)ΔxS=−fYΔXS−dYSYx1−fΔϕ−x12fΔϕ−x1z1fΔω+z1Δκ−Δffx1−Δx0−ΔxpdYS2Y−Δxp2x1fdϕ2−Δxpz1fdω2−Δzpx1fdω2+Δzpdκ2−Δxpdf2f.After sorting out equation (6), it can be expressed as follows:(7)Δxp=Δxp0+δxpΔxp0=−dYSY−Δffx1+z1Δκ+−fYΔXS−fΔϕ−Δx0−x12fΔϕ−x1z1fΔωδxp=−ΔxpdYs2Y−2Δxpx1fdϕ2−Δxpz1fdω2−Δzpx1fdω2+Δzpdω2+Δzpdκ2−df2fΔxp.Because motionΔxp0Δpx0 is caused by the change of camera external and internal parameters ΔXS,ΔZS,Δϕ,Δω,Δκ,Δf,Δx0ΔXs,ΔZs,Δϕ,Δω,Δκ,Δf,Δx0 in the successive and zero images, we can correct δxp0. with a sufficient number of control points. Differently, motion δxp. is caused by the interaction of Δxp. , Δzp, and camera external and internal parameters dZS2,dϕ2,dω2,dκ2,df2 in the successive image. However, and are different at each point, control points cannot be used to correct, so we only discuss as follows:(8)Δxp0=−ΔYSY−Δffx1+Δkz1+−fYΔXS−fΔϕ−Δx0−x12fΔϕ−x1z1fΔω.We can express equation (8) as follows:(9)Δxp0=ax+bz+c+dx2+exz.If there are more than five control points, each unknown coefficienta,b,c,d,e can be obtained according to their. We assume the correction of is, so the error equation is(10)v=ax+bz+c+dx2+exz−Δx.For convenience, we selected the linear part of the equation (10) for processing, as follows:(11)Δxp0=ax+bz+cΔzp0=a′x+b′z+c′.In this case, we only need three or more reference points to obtaina,b,c and a′,b′,c′. Take ΔPx0 as an example, When contains only occasional errors, equation (11) can be expressed as follows:(12)px′+v=ax′+bz′,where px′ is the differential coefficient of ΔPx0. The error equation is(13)V=ax′+bz′−px′.The equation of the composition method is(14)a∑x′2+b∑x′z′−∑x′px′=0,a∑x′z′+b∑z′2−∑z′px′=0.Calculate barycentric coordinates by control points on the reference plane, as follows:(15)xs=x−∑xnzs=z−∑zn.Because coordinates of control points are barycentric coordinates,∑xS∑zS=0 and the parallax coefficient in the X direction as follows:(16)a=∑zs′2∑xs′px′−∑xs′zs′∑zs′px′∑xs′2∑zs′2−∑xs′zs′∑xs′zs′,b=∑xs′2∑zs′px′−∑xs′zs′∑xs′px′∑xs′2∑zs′2−∑xs′zs′∑xs′zs′,where a=tanϕx and b=tanωz. Similarly, we can obtain the parallax coefficient a′ and b′ in the z direction. Then, we can obtain Δxp0 and Δzp0. Finally, we figure out the value of ΔXi as follows:(17)ΔXi≈Δxp0≈ax+bzΔZi≈Δzp0≈a′x+b′z.The displacement of the monitoring point on the reference plane after the first correction is(18)ΔXr=yfΔXiΔZr=yfΔZi.On the basis of the above work, we use reference points on the reference plane to further reduce the error, assume the coordinates of the 4 reference points are(19)Xnri,Ynri,i=1,2,3,4,where n is the photo number.If the UAV moves slightly and passively on a plane parallel to the object plane, the positions of the 4 reference points in the image coordinate system will inevitably change, and this change can be calculated under ideal conditions as follows:(20)Xnr¯=∑i=14Xnri4,Ynr¯=∑i=14Ynri4.We need to subtract these two valuesXnr¯ and Ynr¯ before we calculated the displacement of the monitoring point in the object plane.(21)ΔX0=YyΔXr−Xnr¯ΔZ0=YyΔZr−Znr¯.Finally, the displacementΔX0 and ΔZ0 of the final monitoring point can be obtained through the proportional conversion relationship as Figure 6 shows.Figure 6
The Z-MP method (part two).


## 4. The Elevated Corridor Experiment

The height difference of the elevated corridor is about 20m to 25m above the ground; the digital camera on the ground cannot observe it vertically. It can only monitor it from a low-angle as Figure7(a) shows, which cannot meet the requirement that the monitoring surface is parallel to the control surface, but the UAV do as Figure 7(b) shows.Figure 7
Ground digital cameras and UAV monitoring the elevated corridor from different angles. (a) Ground digital camera low-angle shot. (b) UAV vertical shot
(a)(b)


### 4.1. The Elevated Corridor Experiment Process

We have eight control points (C1 – C8) on the surface of the building, and on the ground, six monitoring points (U0–U5) were selected on the elevated corridor, as Figure8 shows.Figure 8
(a) C1 and C2 (b) C3 and C4 (c) Points layout of the experiment. C1–C8 are control points. C1–C4 are used for UAV, C5–C8 are used for Camera. U1–U5 are monitoring points.
(a)(b)(c)The UAV takes off and adjusts its altitude and attitude to ensure that all points are evenly distributed across the camera’s image; the UAV hovers and starts recording. In this process, the ground surveyor used a steel ruler to measure the relative lengths of 4 points (C5 – C8) on the ground, and measured the relative distance of 4 points (C1– C4) on the building surface with a total station as Figures8(a) and 8(b).


### 4.2. Results and Discussion

Figure9 shows monitoring result figures obtained from a digital camera for these points.Figure 9
: Monitoring results obtained from digital camera. (a) and (c) are deformation curves of monitoring points, (b) and (d) are deformation box plots of monitoring points.(a) Deformation of U0–U5 inX-axis (b) Deformation box plot of U0–U5 in X-axis (c) Deformation of U0–U5 in Z-axis (d) Deformation box plot of U0–U5 in Z-axis.
(a)(b)(c)(d)We can get the following results from Figure9:(1)
The monitoring points show an elastic trend, which we regard as vibration(2)
The vibration ranges and frequency of these monitoring points along theX-axis and Z-axis are similar, as Figures 9(a) and 9(c) show(3)
The displacement of the monitoring point on theX-axis is more dramatic than the displacement on the Z-axisFigure10 shows monitoring result obtained from UAV for these points. The upper and lower limits of the pixel deformation values of the monitored points in Figures 9 and 10 are different because the UAV is farther away from the object plane than the digital camera.Figure 10
Monitoring results obtained from UAV. (a) and (c) are deformation curves of monitoring points, (b) and (d) are deformation box plots of monitoring points.(a) Deformation of U0–U5 inX-axis (b) Deformation box plot of U0–U5 in X-axis (c) Deformation of U0–U5 in Z-axis (d) Deformation box plot of U0–U5 in Z-axis.
(a)(b)(c)(d)We can get some of the same results as the digital camera experiment:(1)
The deformation of each monitoring point is elastic.(2)
The vibration ranges and frequency of these monitoring points along theX-axis and Z-axis are similar. The deformation trends of each point on the X-axis and Z-axis are the same.(3)
The deformation interval of the monitoring point on theX axis is twice that on the Z-axis.There are few differences between the results of the two experiments; obtained the same results on the whole; that is, the monitored structure has good elasticity [19]. Based on experiments, we summarize the performance comparison of digital camera and UAV in Table 3.(1)
Hover error (DJI Phantom 4 Pro): Vertical: ±0.1 m, Horizontal: ±0.3 m (Visual positioning works).(2)
Maximum allowable wind speed (DJI Phantom 4 Pro): 10 m/s [15].(3)
The price of sensors and the UAVs with interchangeable sensors are high, generally.Table 3
Performance comparison of digital cameras and UAVs.
Digital camerasUAVsError sourcesLens distortionLens distortion measuring error hover error1 Windage2Measuring errorShutter jitter in manual modeFlexibilityInstall on stable groundHigh flexibilityAvoid crowdsExpansibilityChange lenses non-replaceable sensor hot shoe: GPS (global positioning system), flashlightChange lensesSensors3 [11]: RGB image/Video capturing device, LiDAR (light detection and ranging), NIR (near infrared), GPS, INS (inertial navigation system), geomagnetic sensors.


## 4.1. The Elevated Corridor Experiment Process

We have eight control points (C1 – C8) on the surface of the building, and on the ground, six monitoring points (U0–U5) were selected on the elevated corridor, as Figure8 shows.Figure 8
(a) C1 and C2 (b) C3 and C4 (c) Points layout of the experiment. C1–C8 are control points. C1–C4 are used for UAV, C5–C8 are used for Camera. U1–U5 are monitoring points.
(a)(b)(c)The UAV takes off and adjusts its altitude and attitude to ensure that all points are evenly distributed across the camera’s image; the UAV hovers and starts recording. In this process, the ground surveyor used a steel ruler to measure the relative lengths of 4 points (C5 – C8) on the ground, and measured the relative distance of 4 points (C1– C4) on the building surface with a total station as Figures8(a) and 8(b).


## 4.2. Results and Discussion

Figure9 shows monitoring result figures obtained from a digital camera for these points.Figure 9
: Monitoring results obtained from digital camera. (a) and (c) are deformation curves of monitoring points, (b) and (d) are deformation box plots of monitoring points.(a) Deformation of U0–U5 inX-axis (b) Deformation box plot of U0–U5 in X-axis (c) Deformation of U0–U5 in Z-axis (d) Deformation box plot of U0–U5 in Z-axis.
(a)(b)(c)(d)We can get the following results from Figure9:(1)
The monitoring points show an elastic trend, which we regard as vibration(2)
The vibration ranges and frequency of these monitoring points along theX-axis and Z-axis are similar, as Figures 9(a) and 9(c) show(3)
The displacement of the monitoring point on theX-axis is more dramatic than the displacement on the Z-axisFigure10 shows monitoring result obtained from UAV for these points. The upper and lower limits of the pixel deformation values of the monitored points in Figures 9 and 10 are different because the UAV is farther away from the object plane than the digital camera.Figure 10
Monitoring results obtained from UAV. (a) and (c) are deformation curves of monitoring points, (b) and (d) are deformation box plots of monitoring points.(a) Deformation of U0–U5 inX-axis (b) Deformation box plot of U0–U5 in X-axis (c) Deformation of U0–U5 in Z-axis (d) Deformation box plot of U0–U5 in Z-axis.
(a)(b)(c)(d)We can get some of the same results as the digital camera experiment:(1)
The deformation of each monitoring point is elastic.(2)
The vibration ranges and frequency of these monitoring points along theX-axis and Z-axis are similar. The deformation trends of each point on the X-axis and Z-axis are the same.(3)
The deformation interval of the monitoring point on theX axis is twice that on the Z-axis.There are few differences between the results of the two experiments; obtained the same results on the whole; that is, the monitored structure has good elasticity [19]. Based on experiments, we summarize the performance comparison of digital camera and UAV in Table 3.(1)
Hover error (DJI Phantom 4 Pro): Vertical: ±0.1 m, Horizontal: ±0.3 m (Visual positioning works).(2)
Maximum allowable wind speed (DJI Phantom 4 Pro): 10 m/s [15].(3)
The price of sensors and the UAVs with interchangeable sensors are high, generally.Table 3
Performance comparison of digital cameras and UAVs.
Digital camerasUAVsError sourcesLens distortionLens distortion measuring error hover error1 Windage2Measuring errorShutter jitter in manual modeFlexibilityInstall on stable groundHigh flexibilityAvoid crowdsExpansibilityChange lenses non-replaceable sensor hot shoe: GPS (global positioning system), flashlightChange lensesSensors3 [11]: RGB image/Video capturing device, LiDAR (light detection and ranging), NIR (near infrared), GPS, INS (inertial navigation system), geomagnetic sensors.


## 5. Discussion

### 5.1. Limitations

(1)
Although the method improves the deployment speed of monitoring, it is still necessary to set more than four reference points in practical applications. Otherwise, it is impossible to reduce the error and calculate the true displacement.(2)
The proposed data optimization method is only suitable for the slight movement of the common UAV. If the UAV has a large movement in the monitoring process, it is necessary to choose other processing methods.(3)
In the experiment, because only one UAV is used for monitoring, it is impossible to measure the displacement outside the plane, and the use of two UAVs to monitor at the same time from different perspectives will be expected to achieve the displacement of the measurement monitoring point in all directions.


### 5.2. Future Works

For the motion problems that still exist when drones hover, most of the studies used to eliminate this effect during data processing, which has been briefly introduced in the “Related Works” section. There are currently a certain number of studies to improve the stability of drones, such as Zhang Wei et al. propose various deep learning-based methods for machinery fault diagnosis [20, 21]. Lianghao Hua et al. propose a four-rotor UAV sensor fault diagnosis and fault-tolerant control based on a genetic algorithm. It is also pointed out that the fault adjustment method can be designed in the future to accelerate the response to the fault [22]. Zhong et al. propose a robust actuator fault detection and diagnosis (FDD) scheme for a quadrotor UAV (QUAV) in the presence of external disturbances. It is also pointed out that the estimates of actuator faults and external disturbances will be conducive to improving the system performance [23]. Optimizing the data processing process on the basis of improving the stability of the rotorcraft UAV is expected to make the monitoring results more accurate.


## 5.1. Limitations

(1)
Although the method improves the deployment speed of monitoring, it is still necessary to set more than four reference points in practical applications. Otherwise, it is impossible to reduce the error and calculate the true displacement.(2)
The proposed data optimization method is only suitable for the slight movement of the common UAV. If the UAV has a large movement in the monitoring process, it is necessary to choose other processing methods.(3)
In the experiment, because only one UAV is used for monitoring, it is impossible to measure the displacement outside the plane, and the use of two UAVs to monitor at the same time from different perspectives will be expected to achieve the displacement of the measurement monitoring point in all directions.


## 5.2. Future Works

For the motion problems that still exist when drones hover, most of the studies used to eliminate this effect during data processing, which has been briefly introduced in the “Related Works” section. There are currently a certain number of studies to improve the stability of drones, such as Zhang Wei et al. propose various deep learning-based methods for machinery fault diagnosis [20, 21]. Lianghao Hua et al. propose a four-rotor UAV sensor fault diagnosis and fault-tolerant control based on a genetic algorithm. It is also pointed out that the fault adjustment method can be designed in the future to accelerate the response to the fault [22]. Zhong et al. propose a robust actuator fault detection and diagnosis (FDD) scheme for a quadrotor UAV (QUAV) in the presence of external disturbances. It is also pointed out that the estimates of actuator faults and external disturbances will be conducive to improving the system performance [23]. Optimizing the data processing process on the basis of improving the stability of the rotorcraft UAV is expected to make the monitoring results more accurate.


## 6. Conclusions

Digital close-range photogrammetry technology has the advantages of noncontact, multipoint, and rapid monitoring, and the application of UAVs makes the monitoring task no longer limited by the size of the field. The paper proposes a dynamic displacement monitoring method for UAVs based on digital close-range photogrammetry technology and optimize the data processing process for the problem that there is still moving when the UAV hovers. The method proposed in this paper does not need a lot of preparation for monitoring targets in advance and has the characteristics of rapid deployment. The experimental results show that the monitoring results obtained by the UAV after data processing and optimization are the same as those obtained by the digital camera installed on the ground.



---

*Source: 1949213-2022-09-09.xml*