# Optimized Design of 3D Spatial Images Based on Kalman Filter Equation

Authors: Wei Shan
Journal: Advances in Mathematical Physics
 (2021)

---

## Abstract

This paper takes the advantageous ability of Kalman filter equation as a means to jointly realize the accurate and reliable extraction of 3D spatial information and carries out the research work from the extraction of 3D spatial position information from multisource remote sensing optical stereo image pairs, recovery of 3D spatial structure information, and joint extraction of 3D spatial information with optimal topological structure constraints, respectively. Taking advantage of the stronger effect capability of Wiener recovery and shorter computation time of Kalman filter recovery, Wiener recovery is combined with Kalman filter recovery (referred to as Wiener-Kalman filter recovery method), and the mean square error and peak signal-to-noise ratio of the recovered image of this method are comparable to those of Wiener recovery, but the subjective evaluation concludes that the recovered image obtained by the Wiener-Kalman filter recovery method is clearer. To address the problem that the Kalman filter recovery method has the advantage of short computation time but the recovery effect is not as good as the Wiener recovery method, an improved Kalman filter recovery algorithm is proposed, which overcomes the fact that the Kalman filter recovery only targets the rows and columns of the image matrix for noise reduction and cannot utilize the pixel point information among the neighboring rows and columns. The algorithm takes the first row of the matrix image as the initial parameter of the Kalman filter prediction equation and then takes the first row of the recovered image as the initial parameter of the second Kalman filter prediction equation. The algorithm does not need to estimate the degradation function of the degradation system based on the degraded image, and the recovered image presents the image edge detail information more clearly, while the recovery effect is comparable to that of the Wiener recovery and Wiener-Kalman filter recovery method, and the improved Kalman filter recovery method has stronger noise reduction ability compared with the Kalman filter recovery method. The problem that the remote sensing optical images are seriously affected by shadows and complex environment detail information when 3D spatial structure information is extracted and the data extraction feature edge is not precise enough and the structure information extraction is not stable enough is addressed. A global optimal planar segmentation method with graded energy minimization is proposed, which can realize the accurate and stable extraction of the topological structure of the top surface by combining the edge information of remote sensing optical images and ensure the accuracy and stability of the final extracted 3D spatial information.


---

## Body

## 1. Introduction

Due to the limitations of scientific and technological development, most of the data that people can obtain and process are two-dimensional data. However, with the rapid development of modern information technology and the expansion of computer graphics applications, how to accurately and quickly convert the real-world three-dimensional information into computer-processable data has become the goal of human efforts. 3D scanning technology can obtain 3D coordinates of object surface points and other information and belongs to a kind of stereo measurement technology [1]. Compared with traditional measurement technology, this technology can complete the measurement of complex objects with high accuracy, high speed, and significant time and cost savings and thus has important applications in the fields of 3D city modeling, reverse engineering, cultural relic’s protection, and reconstruction, large building construction, and building deformation monitoring. However, because the surface of complex ground objects can easily obscure the scanning lines, it is often impossible to obtain the point cloud data that completely covers the surface of the scanned object by one-stop scanning, and it is necessary to set up stations at different locations and multiview scanning to obtain the local point cloud data of each part of the target object and then obtain the complete point cloud data of the scanned object through the point cloud data stitching process [2, 3]. The stitching error exists in the process of point cloud stitching, and the error will gradually accumulate with the increase of the number of stitching stations, plus the measurement error of the station data, resulting in the stitching error getting larger and larger. Therefore, the overall stitching of multistation data can be based on the classical algorithm in dynamic positioning [4].The 3D reconstruction is one of the most popular directions in the field of computer vision, which is the application of transforming the acquired 2D image information of an object into 3D stereo information of the object in space. Therefore, the study of 3D reconstruction has academic significance and practical value. It is a multidisciplinary intersection research field and has great importance in computer image processing. Kalman filtering is used to seek a recursive estimation algorithm with the minimum mean square error as the best estimation criterion, and its basic idea is as follows: through the state-space model of signal and noise, the estimated value of the state variable is updated according to the estimated value of the previous moment and the observed value of the current moment, and the estimated value of this moment is found, which is suitable for real-time data processing. The essence of Kalman filtering is to reconstruct the state vector of the system from the measured values [5]. This article attempts to focus on the interaction between viewers based on the Kalman filter equation and three-dimensional space. This research can also enable image art works to better interact with viewers and artists, so that viewers can have a better and deeper experience of three-dimensional images and can also better participate in three-dimensional images, optimizing design [6, 7].The traditional image restoration methods need to estimate the degradation function of the degradation system, poor noise reduction ability, and long running time of the algorithm. In order to achieve the purpose of reducing the noise in the image, keeping the edge detail information of the image, and shortening the running time of the algorithm, this paper proposes an alternating Kalman filter image restoration algorithm, which is better than other methods through several experimental simulations. In the first chapter, this paper introduces the present and significance of the study of 3D spatial images based on Kalman filter equations and also explains the research framework of this paper. In Section2, the current status of image restoration at home and abroad is analyzed in detail, and problems such as the need to estimate the degradation function of the degradation system, the poor noise reduction capability, and the long-running time of the algorithm are addressed. In Section 3, an alternating Kalman filter image restoration algorithm is proposed. The mathematical model of multisource remote sensing optical stereo image for 3D spatial position information extraction is also analyzed in detail based on the Kalman filter equation model. The problem that the mathematical model of 3D spatial location information extraction is a hierarchical linear model rather than a standard linear model is addressed by the imbalance of the observations of multisource data. Section 4 shows that the method can effectively achieve the task of 3D spatial structure information recovery of image data through simulation and experimental results on real data. Under the guidance of key structure points, the accurate and reliable 3D spatial information extraction of buildings is achieved by combining multisource optical stereo image pairs and image data. Section 5 summarizes the research work done in this paper, points out the innovation points and shortcomings in the research, and looks forward to the next research content and the expected results to be achieved.


## 2. Related Work

The overall scanning of an object usually requires several stations, but the scanning of large scenes often requires dozens or even hundreds of stations, which makes the stitching of 3D point cloud data very complicated. Heo et al. proposed a dynamic stitching algorithm similar to the Kalman filter algorithm, which is based on the theory of instantaneous dynamics and its construction of a geometric distance function to a surface [8]. Nguyen et al. successively proposed a stitching algorithm based on the continuous Kalman filter equation to evaluate the alignment parameters of two rigid object point clouds, which is suitable for the alignment of large data volumes because it has less operations and higher operational efficiency [9]. The Kalman filter algorithm has the accuracy of at least second-order Taylor series expansion, its computational accuracy is higher than that of the extended Kalman filter algorithm; another advantage of this algorithm is that the computation of the transformation parameters changes as they change [10]. If these parameters are very different for the real scheme, this algorithm may not get better alignment results, and this drawback limits its application in large-scale transform. Bae et al. proposed the use of Kalman filter in multivariate remote sensing image union, alignment and fusion for the localization, and alignment errors of remote sensing images and their propagation errors, giving the computed union. The simulation experiments show that the algorithm is faster in convergence and higher in accuracy than other methods of joint data alignment executed separately [11].The search for image restoration methods that effectively reduce noise, retain clear edge detail information, and are computationally small has focused on Kalman filtering [12]. The shortcomings of the Wiener filter prompted the search for new methods that could directly design optimal filters in the time domain [13]. Huang and Sun proposed a dual implementation of the Kalman filter that uses sparse noisy acceleration measurements to estimate the unknown inputs and states of a linear state-space model [14]. Deng et al. proposed a Kalman filter-based principle for calculating distorted image sequence for calculating the initial guess of a deformed image sequence, which determines the initial guess for each image pixel and calculates the predicted value from the observed values [15]. Patra proposed a Kalman filter-based approach for neural network classification performance; a Kalman filter-based linear model is established as a postprocessing, which uses the target features; and the predicted linear combination of the output transforms the predicted output of the neural network into a value close to the desired output, and the validation results show that the Kalman filter-based linear model can improve the performance of the original neural network [16]. The distance between points in image data is often much larger than the plane fitting error, which means that the two are not on the same scale; then, the data cost, spatial smoothing cost, and labeling cost are put together to form a single-scale energy function like image segmentation, and it is difficult to make the three balanced by setting different weights [17, 18].Due to the limitations of the computer itself, the biological vision simulated by the computer has great limitations in the edge detection, target tracking, information processing effect, and feature extraction and real-time performance of the target [19]. At present, there are a large number of image processing algorithms; almost every algorithm has its own flaws. The technology of automatic key point matching, surface triangulation, binocular reconstruction, and three-dimensional point stitching is adopted. The complete three-dimensional structure of the object is generated through image point extraction, key point matching and reconstruction, triangulation, and data fusion [20]. For optical images, in order to give full play to their horizontal spatial position information extraction accuracy, rich semantic information, step structure feature positioning accuracy, and other advantageous capabilities of the prerequisite are the support of stable and reliable three-dimensional spatial structure a priori information. For airborne image data, after realizing the effective recovery of its carrying 3D spatial structure information, a stable and reliable optimal segmentation method of building top surface is needed to realize the effective extraction of 3D spatial structure information. Most of the existing image data segmentation methods need to carry out local feature analysis point by point in a passive way and then passively carry out region growing and clustering operations according to the local feature similarity of the data to achieve and are vulnerable to noise and wild value point images. For this reason, a global optimal segmentation method for image data needs to be implemented in a more proactive way with the aim of stable and reliable segmentation.


## 3. 3D Spatial Image Optimization Research Based on Kalman Filter Equation

### 3.1. Kalman Filter Equation Optimization Algorithm

Due to the large volume of the actual collected 3D laser point cloud data, it will be difficult to guarantee the fitting accuracy if the whole measurement data is fitted at one time. For this reason, this paper adopts segmented fitting of the central axis first and then uses the weighted overall least-squares method to fit the central axis of the station as the final stitching control condition. Introduce the concepts of probability theory and mathematical statistics. According to the prerequisite of Kalman filtering, the system noise is Gaussian white noise; that is, the presence of external interference and other factors makes the position and velocity in the dimensional space image conform to the Gaussian distribution; that is, they have their own expectation and variance and because position and velocity are both related.Fitting a discrete point cloud with a quadratic parametric surface presupposes parameterizing the point cloud data. First, the local tangent plane of the point cloud is determined. In this paper, a constrained least-squares plane is used, i.e., the center point of the tangent plane is directly taken as the point cloud shape centerx. Only the unit normal vector n of the tangent plane needs to be calculated so that the sum of squared distances from the adjacent points to the tangent plane is minimized. Each point Pi (i=1,2,⋯,N,N≥10) in the PD is connected to xi to obtain N directed line segments, and then, the dot product with the x vector, respectively, which is denoted as fxi, and the obtained du values are sorted, and the maximum value is denoted as fmax, and the minimum value is denoted as fmin, as in the following equation:
(1)∫fxdx=fxmax−fxifxmax−fxmin.Similarly, they parameter value of each point in PD can be obtained. The parameterization of the local discrete point cloud is realized by applying the obtained PD parameter values to the corresponding D. The equation of the quadratic parametric surface is expressed in the form of equation (2), where A is a 3×3 matrix consisting of vector elements Aij.
(2)fx,y=limM,N⟶∞∑i=1M∑j=1NAijxiyj.The component form of equation (2) can be expressed as
(3)fx,y=ax,y+bx,y+cx,y,A=aij,bij,cij.The other two-component elementsb and c of A are defined similarly to a. The values of x and y in F have been derived by the above point cloud parameterization method, and A consists of three vector elements a, b, and c. Now, assume that there are N+1 measurement points Fixi,yi,zi, and these points correspond to parameter values (xi, yi), i=0,1,2,⋯⋯,N. A is expressed by equation (4), which minimizes the sum of the squared Euclidean distances from each point being fitted to the quadratic parametric surface.
(4)A=FT+FFTF+FTxi,yi.The upper limit of model parameter concentration is set as the end condition of the cycle, and the model parameters are solved by randomly selected sample sets, and the mean valuem of the model parameters is used to represent the location distribution of the model parameters, where the concentration of the model parameters is expressed as the frequency of different parameter intervals. If the difference between the newly solved model parameters and the calculated model parameters is within the given threshold Δ, the two sets of parameters are considered to be consistent and the concentration of the consistent model parameters is increased; otherwise, the mean value of the new model parameters is recorded as the interval dispersion value to update the model parameter interval distribution. The above process of random extraction, calculation of model parameters, and comparison is cycled, and iterative statistics of model parameters are performed until the end-of-cycle condition is satisfied. The basic flow of the Kalman filtering method for 3D laser point clouds incorporating reflection-valued images and Kalman filtering is shown in Figure 1.Figure 1
Point cloud flow chart based on Kalman filter.Because each estimate builds on the previous one, the refinement of the estimates gradually increases with the input of low-resolution images. This process is not inconsistent with the state estimation of the Kalman filter, where the state of the Kalman filter is a discrete-time controlled process which governed by a linear stochastic difference equation.(5)Mn+1=An∗Mn+Bn∗Xn+Cn∗Yn+wn.Since low-resolution image sequences are null-question discontinuous, thus, regularization needs to be introduced to solve null-question continuous domain problems to avoid being trapped in local extrema. Therefore, a form of regularization is included into the cost function to constrain the solution space or stabilize the problem space. By imposing this constraint on the pathological problem, the problem is subsequently transformed into a fitness problem. Our proposed fast feedback model is also an improved example of a regularized constrained cost function as shown in equation (6). The λ is a weighting factor which is used to balance the two constraint terms to prevent disjoint minimization.
(6)∇fx=limM⟶∞∑i=1Mfxi−dxi2λ∫fnfxidx.In the image restoration process, different initializations can affect the recovery of Kalman filter. The Kalman filter initialization parameters often need to be determined based on a priori knowledge and multiple experiments. In this paper, the values of the observed noise matrix and the system noise matrix parameters that can achieve the desired results are finally determined after several simulations based on the a priori knowledge. Although the Kalman filter recovery method can retain the edge detail information of the recovered image better while reducing the noise, the former is not as good as the latter compared with the Wiener recovery method. In order to effectively reduce the noise in the image and also shorten the running time of the algorithm, an alternating Kalman filter image recovery algorithm is proposed in this paper. The filtering gain equation is shown in the following equation:(7)Kx=Ax,x−1∗βx+Ax,x−2∗βx−1.The estimated error variance array equation is shown in the following equation:(8)Zk=1−Ak+1,kβk−1Ak,k−1.The image points to the object, the essential characteristics of an existing or fictitious object. For the observation of an object, it has been selected, in the classical period selected for beauty. By selecting scenes and unique individuals, the modern rather expresses a scene and important events. The essential intuition or essential extraction of the object has actually extracted the object reproduction. The so-called intrinsic intuition is that even if it is realistic and figurative, it is essentially intuitive and the form presented by the work already has an imagination of the subject.


### 3.2. 3D Spatial Image Structure Information Extraction

The image pixel-based alignment does not need to extract the features of the image, it mainly uses the pixel grayscale of the image to make comparison, and the research on the alignment algorithm based on the grayscale distribution of the image is relatively in progress, so it is more mature in development. Usually, the grayscale matrix of a window of a certain size is extracted as a template in one image, and the matching block is searched in another image based on some or several similarity measures, and the relative displacement between the matched blocks is the displacement of these two images. When the templateM is searched in another image, the common similarity measure is normalized intercorrelation. Suppose the search map under the template cover is labeled as submap Cij; then, the normalized intercorrelation is shown in the following equation:
(9)Gi,j=limM,N⟶∞∑i=1M∑j=1NCijx,yGx,y.In the transform domain-based alignment method, because the technique of frequency domain alignment has a relatively high tolerance to noise and has a certain resistance to light changes, the common method to deal with the alignment problem down in the frequency domain is the phase correlation technique; to calculate the magnitude of the translation between the two images, letgm,ni+1 be the image of gm,ni after translation x0 and ni in the m and n directions, as in equation (10). Under some metric, the same physical feature will show similar or similar properties on both images, so given a point in one image, matching will always search for a point with similar properties on the other image to be used as the matching point. Compared with the observation point distance, the change of depth caused by the four convexities of the object surface is very slow, so the inspection number will also be slow, in other words, the parallax has continuity. The order between the matching points corresponding to the same polar line on the image is consistent, and there is no phenomenon that the feature sequence of the image crosses with the spatial coordinate sequence. These matching constraints not only improve the matching accuracy but also reduce the matching workload and thus improve the matching speed.
(10)gm,ni+1=gm−mi,n−nii,i⊆1,N.The selection objective is to randomly selectn points from the candidate reference point set Ω composed of N discrete points that are distributed as uniformly as possible in the test space where Ω is located. To achieve this purpose, the target image is first dissected into M=w×w (as shown in Figure 2(a)), and then, the [0,1] interval is divided into M subintervals, and the interval length of each interval is defined as Ni/∑Ni,i⊆1,M, where Ni is the number of the ith candidate point, as shown in Figure 2(b). When selecting control points, pay attention to the sequence of points and points. When selecting control points locally, do not follow blindly. Just place the points when you see corners, intersections, and other terrain. The result can only be that the local correction is better, but the overall view is not ideal. Spot density and unevenness, large amount, and disorder will not produce good results.Figure 2
Schematic diagram of control point selection method.The purpose of the uniform control point selection method is to select the control points that can fully reflect the target scene information to improve the accuracy and effectiveness of the related processing. However, the classical control point selection method mainly realizes the selection of uniformly distributed control points in a horizontal two-dimensional space. Obviously, for those control points used for 3D spatial information extraction, the uniform distribution of the entire 3D space should be considered in order to better reflect the uniform distribution of the point set. In addition, whether manually selected or automatically selected, ground control points or reference points are often corner points, intersection points, special shape center points, and other points with obvious geometric features in the scene.In order to better extract the correlation between point sets, it is necessary to obtain the information related to their image neighborhoods. Considering that reference points are mostly edge points, intersection points, corner points, etc., the neighborhood need not be taken as large. Here, we take the4∗4neighborhoods of the reference points and then classify the reference point set by the correlation between the neighborhoods. The correlation coefficient between neighbors is calculated as in the following equation:
(11)J=limM⟶∞∑i=1M∑j=1MXi,j−miYi,j−mj,where Xi,j and Yi,j are the grayscale values of the pixels at positions (i,j) in the neighboring image blocks I and J corresponding to any two control points to be selected; mi and mj are the grayscale mean values of the corresponding image blocks, respectively. The image data contains only 3D spatial coordinates without any topological structure information. For this reason, establishing a local neighborhood system is an important prerequisite for realizing local structure feature analysis of image data. In addition to the computed features (such as slope, flatness, roughness, and other features obtained by using local plane fitting), local shape analysis using eigenvalue features that can describe local structure shape information obtained by analyzing the covariance matrix of the local neighborhood point set using PCA technique is the most important method to perform local structure feature analysis of image point clouds. For a local neighborhood point set, the covariance matrix is shown in equation (12), where b is the number of points in the local neighborhood point set and A is taken as the mean point of the local neighborhood point set. The eigenvalues of the covariance matrix N are obtained by performing singular value decomposition.
(12)Mi,b=Mii=1b⊆1,N,N=limM⟶∞∑i=1bAi−avgABi−avgBT.In order to successfully implement the above algorithm, an effective estimation of the initial weight matrix is required. Obviously, the weight matrix is designed to balance the problem of unbalanced multisource image resolution due to differences in multisource imaging systems. For this case, in the case of two multisource optical images forming a stereo image pair, for example, most of the existing methods use empirical initial matrix assignment methods. Especially for large scene images and large inclination images, the resolution of each pixel in the image is not exactly the same, and this difference is not negligible when performing accurate 3D spatial position information extraction.


## 3.1. Kalman Filter Equation Optimization Algorithm

Due to the large volume of the actual collected 3D laser point cloud data, it will be difficult to guarantee the fitting accuracy if the whole measurement data is fitted at one time. For this reason, this paper adopts segmented fitting of the central axis first and then uses the weighted overall least-squares method to fit the central axis of the station as the final stitching control condition. Introduce the concepts of probability theory and mathematical statistics. According to the prerequisite of Kalman filtering, the system noise is Gaussian white noise; that is, the presence of external interference and other factors makes the position and velocity in the dimensional space image conform to the Gaussian distribution; that is, they have their own expectation and variance and because position and velocity are both related.Fitting a discrete point cloud with a quadratic parametric surface presupposes parameterizing the point cloud data. First, the local tangent plane of the point cloud is determined. In this paper, a constrained least-squares plane is used, i.e., the center point of the tangent plane is directly taken as the point cloud shape centerx. Only the unit normal vector n of the tangent plane needs to be calculated so that the sum of squared distances from the adjacent points to the tangent plane is minimized. Each point Pi (i=1,2,⋯,N,N≥10) in the PD is connected to xi to obtain N directed line segments, and then, the dot product with the x vector, respectively, which is denoted as fxi, and the obtained du values are sorted, and the maximum value is denoted as fmax, and the minimum value is denoted as fmin, as in the following equation:
(1)∫fxdx=fxmax−fxifxmax−fxmin.Similarly, they parameter value of each point in PD can be obtained. The parameterization of the local discrete point cloud is realized by applying the obtained PD parameter values to the corresponding D. The equation of the quadratic parametric surface is expressed in the form of equation (2), where A is a 3×3 matrix consisting of vector elements Aij.
(2)fx,y=limM,N⟶∞∑i=1M∑j=1NAijxiyj.The component form of equation (2) can be expressed as
(3)fx,y=ax,y+bx,y+cx,y,A=aij,bij,cij.The other two-component elementsb and c of A are defined similarly to a. The values of x and y in F have been derived by the above point cloud parameterization method, and A consists of three vector elements a, b, and c. Now, assume that there are N+1 measurement points Fixi,yi,zi, and these points correspond to parameter values (xi, yi), i=0,1,2,⋯⋯,N. A is expressed by equation (4), which minimizes the sum of the squared Euclidean distances from each point being fitted to the quadratic parametric surface.
(4)A=FT+FFTF+FTxi,yi.The upper limit of model parameter concentration is set as the end condition of the cycle, and the model parameters are solved by randomly selected sample sets, and the mean valuem of the model parameters is used to represent the location distribution of the model parameters, where the concentration of the model parameters is expressed as the frequency of different parameter intervals. If the difference between the newly solved model parameters and the calculated model parameters is within the given threshold Δ, the two sets of parameters are considered to be consistent and the concentration of the consistent model parameters is increased; otherwise, the mean value of the new model parameters is recorded as the interval dispersion value to update the model parameter interval distribution. The above process of random extraction, calculation of model parameters, and comparison is cycled, and iterative statistics of model parameters are performed until the end-of-cycle condition is satisfied. The basic flow of the Kalman filtering method for 3D laser point clouds incorporating reflection-valued images and Kalman filtering is shown in Figure 1.Figure 1
Point cloud flow chart based on Kalman filter.Because each estimate builds on the previous one, the refinement of the estimates gradually increases with the input of low-resolution images. This process is not inconsistent with the state estimation of the Kalman filter, where the state of the Kalman filter is a discrete-time controlled process which governed by a linear stochastic difference equation.(5)Mn+1=An∗Mn+Bn∗Xn+Cn∗Yn+wn.Since low-resolution image sequences are null-question discontinuous, thus, regularization needs to be introduced to solve null-question continuous domain problems to avoid being trapped in local extrema. Therefore, a form of regularization is included into the cost function to constrain the solution space or stabilize the problem space. By imposing this constraint on the pathological problem, the problem is subsequently transformed into a fitness problem. Our proposed fast feedback model is also an improved example of a regularized constrained cost function as shown in equation (6). The λ is a weighting factor which is used to balance the two constraint terms to prevent disjoint minimization.
(6)∇fx=limM⟶∞∑i=1Mfxi−dxi2λ∫fnfxidx.In the image restoration process, different initializations can affect the recovery of Kalman filter. The Kalman filter initialization parameters often need to be determined based on a priori knowledge and multiple experiments. In this paper, the values of the observed noise matrix and the system noise matrix parameters that can achieve the desired results are finally determined after several simulations based on the a priori knowledge. Although the Kalman filter recovery method can retain the edge detail information of the recovered image better while reducing the noise, the former is not as good as the latter compared with the Wiener recovery method. In order to effectively reduce the noise in the image and also shorten the running time of the algorithm, an alternating Kalman filter image recovery algorithm is proposed in this paper. The filtering gain equation is shown in the following equation:(7)Kx=Ax,x−1∗βx+Ax,x−2∗βx−1.The estimated error variance array equation is shown in the following equation:(8)Zk=1−Ak+1,kβk−1Ak,k−1.The image points to the object, the essential characteristics of an existing or fictitious object. For the observation of an object, it has been selected, in the classical period selected for beauty. By selecting scenes and unique individuals, the modern rather expresses a scene and important events. The essential intuition or essential extraction of the object has actually extracted the object reproduction. The so-called intrinsic intuition is that even if it is realistic and figurative, it is essentially intuitive and the form presented by the work already has an imagination of the subject.


## 3.2. 3D Spatial Image Structure Information Extraction

The image pixel-based alignment does not need to extract the features of the image, it mainly uses the pixel grayscale of the image to make comparison, and the research on the alignment algorithm based on the grayscale distribution of the image is relatively in progress, so it is more mature in development. Usually, the grayscale matrix of a window of a certain size is extracted as a template in one image, and the matching block is searched in another image based on some or several similarity measures, and the relative displacement between the matched blocks is the displacement of these two images. When the templateM is searched in another image, the common similarity measure is normalized intercorrelation. Suppose the search map under the template cover is labeled as submap Cij; then, the normalized intercorrelation is shown in the following equation:
(9)Gi,j=limM,N⟶∞∑i=1M∑j=1NCijx,yGx,y.In the transform domain-based alignment method, because the technique of frequency domain alignment has a relatively high tolerance to noise and has a certain resistance to light changes, the common method to deal with the alignment problem down in the frequency domain is the phase correlation technique; to calculate the magnitude of the translation between the two images, letgm,ni+1 be the image of gm,ni after translation x0 and ni in the m and n directions, as in equation (10). Under some metric, the same physical feature will show similar or similar properties on both images, so given a point in one image, matching will always search for a point with similar properties on the other image to be used as the matching point. Compared with the observation point distance, the change of depth caused by the four convexities of the object surface is very slow, so the inspection number will also be slow, in other words, the parallax has continuity. The order between the matching points corresponding to the same polar line on the image is consistent, and there is no phenomenon that the feature sequence of the image crosses with the spatial coordinate sequence. These matching constraints not only improve the matching accuracy but also reduce the matching workload and thus improve the matching speed.
(10)gm,ni+1=gm−mi,n−nii,i⊆1,N.The selection objective is to randomly selectn points from the candidate reference point set Ω composed of N discrete points that are distributed as uniformly as possible in the test space where Ω is located. To achieve this purpose, the target image is first dissected into M=w×w (as shown in Figure 2(a)), and then, the [0,1] interval is divided into M subintervals, and the interval length of each interval is defined as Ni/∑Ni,i⊆1,M, where Ni is the number of the ith candidate point, as shown in Figure 2(b). When selecting control points, pay attention to the sequence of points and points. When selecting control points locally, do not follow blindly. Just place the points when you see corners, intersections, and other terrain. The result can only be that the local correction is better, but the overall view is not ideal. Spot density and unevenness, large amount, and disorder will not produce good results.Figure 2
Schematic diagram of control point selection method.The purpose of the uniform control point selection method is to select the control points that can fully reflect the target scene information to improve the accuracy and effectiveness of the related processing. However, the classical control point selection method mainly realizes the selection of uniformly distributed control points in a horizontal two-dimensional space. Obviously, for those control points used for 3D spatial information extraction, the uniform distribution of the entire 3D space should be considered in order to better reflect the uniform distribution of the point set. In addition, whether manually selected or automatically selected, ground control points or reference points are often corner points, intersection points, special shape center points, and other points with obvious geometric features in the scene.In order to better extract the correlation between point sets, it is necessary to obtain the information related to their image neighborhoods. Considering that reference points are mostly edge points, intersection points, corner points, etc., the neighborhood need not be taken as large. Here, we take the4∗4neighborhoods of the reference points and then classify the reference point set by the correlation between the neighborhoods. The correlation coefficient between neighbors is calculated as in the following equation:
(11)J=limM⟶∞∑i=1M∑j=1MXi,j−miYi,j−mj,where Xi,j and Yi,j are the grayscale values of the pixels at positions (i,j) in the neighboring image blocks I and J corresponding to any two control points to be selected; mi and mj are the grayscale mean values of the corresponding image blocks, respectively. The image data contains only 3D spatial coordinates without any topological structure information. For this reason, establishing a local neighborhood system is an important prerequisite for realizing local structure feature analysis of image data. In addition to the computed features (such as slope, flatness, roughness, and other features obtained by using local plane fitting), local shape analysis using eigenvalue features that can describe local structure shape information obtained by analyzing the covariance matrix of the local neighborhood point set using PCA technique is the most important method to perform local structure feature analysis of image point clouds. For a local neighborhood point set, the covariance matrix is shown in equation (12), where b is the number of points in the local neighborhood point set and A is taken as the mean point of the local neighborhood point set. The eigenvalues of the covariance matrix N are obtained by performing singular value decomposition.
(12)Mi,b=Mii=1b⊆1,N,N=limM⟶∞∑i=1bAi−avgABi−avgBT.In order to successfully implement the above algorithm, an effective estimation of the initial weight matrix is required. Obviously, the weight matrix is designed to balance the problem of unbalanced multisource image resolution due to differences in multisource imaging systems. For this case, in the case of two multisource optical images forming a stereo image pair, for example, most of the existing methods use empirical initial matrix assignment methods. Especially for large scene images and large inclination images, the resolution of each pixel in the image is not exactly the same, and this difference is not negligible when performing accurate 3D spatial position information extraction.


## 4. Analysis of Results

### 4.1. Optimization Algorithm Analysis

Figure3 shows the data comparison of several image restoration methods. Observing Figure 3, it can be seen that in the same image, the Wiener-alternating Kalman filter recovery image has the smallest mean square error and the largest peak signal-to-noise ratio, and the Kalman filter recovery image has the largest mean square error and the smallest peak signal-to-noise ratio, and the other images are Wiener-Kalman filter recovery image, alternating Kalman filter recovery image in order. Therefore, the noise reduction ability in order of strength is Wiener-alternating Kalman filter recovery method, Wiener-Kalman filter recovery method, alternating Kalman filter recovery method, and Kalman filter recovery method.Figure 3
Data comparison of image restoration methods.The results of counting the feature points extracted from the reflection value images corresponding to each neighboring point cloud and the retained pairs of the same name are shown in Figure4. From Figure 4, it can be seen that due to the large geometric deformation of the global reflectance images, especially for the reflectance images generated from point clouds in narrow space, the Kalman filtering algorithm still extracts many incorrect feature points, but the homonymous point recognition by Kalman filtering equation retains enough feature point pairs for the subsequent stitching process, and the correct homonymous points among the neighboring stations are about 32.7%.Figure 4
Feature point extraction and eponymous point recognition statistics.The deviation of the control points of the first station relative to their initial coordinates before and after the splicing was calculated according to the conversion parameters of Kalman filter, and the calculation results are shown in Figure5. It can be seen that after the improvement of the splicing scheme, the accuracy of double-war splicing has been greatly improved, but with the increase of the number of splicing stations, the accumulation of splicing errors is serious, which leads to the unsatisfactory splicing effect of the later stations. The maximum value of the point deviation of the control point under the dual-station splicing is 0.29446719 m, the minimum value is 0.29282320 m, and the average value is 0.29375842 m. After its own closed Kalman filter, the maximum value of the point deviation of the control point is 0.00465297 m, the minimum value is 0.00077715 m, and the average value is 0.00209397 m. The results in Figure 5 show that Kalman filtering solves the problem of splicing error accumulation very well. Therefore, the point cloud stitching accuracy is significantly improved by adjusting the conversion parameters of each station by its own closed Kalman filter. The Kalman filter proposed in this paper is a stitching method based on the self-closing condition, which makes the data acquisition workload large and the point cloud stitching workload also large, which will be limited in practical applications; in addition, if the stitching of 3D image data of several kilometers or even tens of kilometers in length is performed, the error accumulation will be very serious and the stitching accuracy of this method will be difficult to guarantee.Figure 5
3D image control point coordinate change.Based on the projection-based central axis extraction, the global consistent central axis extraction and the normal central axis dynamic extraction methods are proposed to improve the accuracy of the central axis fitting of 3D images and provide references for the control conditions of point cloud stitching and the deformation monitoring of 3D images. A simple algorithm and generalized prior probability estimation method based on iterative statistics of model parameters are proposed, and the BAYSAC algorithm is improved to realize a high-precision discrete point cloud local surface fitting method by combining quadratic parametric surfaces, which provides a reference for its application in reverse engineering, deformation monitoring, 3D modeling, etc. The stitching of 3D point cloud data is realized by fusing reflection-valued images, introducing Kalman filtering into the stitching process of a large amount of point cloud data, and adding 3D image axis information as the stitching control condition, which effectively improves the problem of continuous accumulation of point cloud stitching errors and provides a guarantee for the subsequent processing and application of 3D laser point cloud data.


### 4.2. Optimization Design Analysis

Since there are limitations in machine wood, it is necessary to add bionics to computer processing, and we analyze the addition of the Ehrman filter mechanism to the 3D reconstruction next. Due to the selectivity of the Ehrman filter, the efficiency of 3D reconstruction can be greatly improved, and the experimental results are shown in Figure6. We can see the time saving from Figure 6. Due to the selectivity of the Ehrman filter, the total significant map of the image is reconstructed in 3D, and the time is shortened from 20212 ms to 11212 ms. Due to the different features that the image itself has, here are two other images with better results. Since this image is richer in information, the reconstructed 3D effect is also better, as discussed earlier. We can see that the time is shortened from 24821 ms to 3613 ms, so it is said that adding the Ehrman filter feature to the computer processing will be a great improvement. Some image reconstruction effects are good, some image reconstruction effects are not good, and in the 3D reconstruction with the attention mechanism, the result shows that the efficiency is improved.Figure 6
Comparison results of 3D reconstruction design optimization.Success rate is another quantitative representation of the accuracy performance of the optimal design algorithm, representing the coverage of the target in each frame of the image sequence by the optimal design algorithm. The center position error is judged mainly based on the center position of the target, ignoring the consideration of the target scale. The success rate, on the other hand, takes more into account the target frame covered by the rectangular frame of the optimal design optimization. Compared with the center position error, the success rate is judged on a more global basis. Figure7 gives the experimental results of the success rate of the improved Kalman filter algorithm and the other nine algorithms. From Figure 7, the success rate of KSTC algorithm is the highest for the jumping sequence, which indicates that the improved Kalman filter can successfully cope with the fast motion of the optimized design target and the blurred background. The improved Kalman filtering algorithm can successfully cope with the scale transformation of the target object in the FaceOcc2 sequence, and the improved Kalman filtering algorithm can cope with the local occlusion of the target object more effectively than the STC algorithm. In the jogging sequence, the STC algorithm fails to optimize the design, but the improved Kalman filter algorithm successfully optimizes the design to the target location and achieves the highest success rate.Figure 7
Success rate of optimization results.In order to verify the information optimization design effect of each method, the peak signal-to-noise ratio between the rasterized reference image without distorted data and the optimized design rasterized image is calculated. The specific experimental results are shown in Figure8.Figure 8
PSNR comparison of design optimization results by various methods of information.Considering the noise reduction capability, the clarity of the recovered image, and the time used by the algorithm, it is concluded from the above analysis that the mean square error of the Saturn and Peppers plots is 8.78% and 4.04% higher for the alternating Kalman filter recovery than the Wiener recovery, and the peak signal-to-noise ratio is basically the same. The calculation time of the alternating Kalman recovery is the percentage of the Wiener recovery time: Saturn, 1.7%, and Peppers, 1.9%. Therefore, the alternating Kalman filter recovery takes less than 1/50 of the Wiener recovery with approximately the same recovery effect. The alternating Kalman filter recovery has a significant advantage over the Wiener-Kalman filter recovery and Wiener-alternating Kalman filter recovery in terms of algorithm running time. The alternating Kalman filter combines the noise reduction ability, the clarity of the recovered image, and the time used by the algorithm, which is better than the other image recovery methods. However, as the noise level increases, the recovery effect of the alternating Kalman filter image recovery method becomes worse, but the recovery time does not change significantly.


## 4.1. Optimization Algorithm Analysis

Figure3 shows the data comparison of several image restoration methods. Observing Figure 3, it can be seen that in the same image, the Wiener-alternating Kalman filter recovery image has the smallest mean square error and the largest peak signal-to-noise ratio, and the Kalman filter recovery image has the largest mean square error and the smallest peak signal-to-noise ratio, and the other images are Wiener-Kalman filter recovery image, alternating Kalman filter recovery image in order. Therefore, the noise reduction ability in order of strength is Wiener-alternating Kalman filter recovery method, Wiener-Kalman filter recovery method, alternating Kalman filter recovery method, and Kalman filter recovery method.Figure 3
Data comparison of image restoration methods.The results of counting the feature points extracted from the reflection value images corresponding to each neighboring point cloud and the retained pairs of the same name are shown in Figure4. From Figure 4, it can be seen that due to the large geometric deformation of the global reflectance images, especially for the reflectance images generated from point clouds in narrow space, the Kalman filtering algorithm still extracts many incorrect feature points, but the homonymous point recognition by Kalman filtering equation retains enough feature point pairs for the subsequent stitching process, and the correct homonymous points among the neighboring stations are about 32.7%.Figure 4
Feature point extraction and eponymous point recognition statistics.The deviation of the control points of the first station relative to their initial coordinates before and after the splicing was calculated according to the conversion parameters of Kalman filter, and the calculation results are shown in Figure5. It can be seen that after the improvement of the splicing scheme, the accuracy of double-war splicing has been greatly improved, but with the increase of the number of splicing stations, the accumulation of splicing errors is serious, which leads to the unsatisfactory splicing effect of the later stations. The maximum value of the point deviation of the control point under the dual-station splicing is 0.29446719 m, the minimum value is 0.29282320 m, and the average value is 0.29375842 m. After its own closed Kalman filter, the maximum value of the point deviation of the control point is 0.00465297 m, the minimum value is 0.00077715 m, and the average value is 0.00209397 m. The results in Figure 5 show that Kalman filtering solves the problem of splicing error accumulation very well. Therefore, the point cloud stitching accuracy is significantly improved by adjusting the conversion parameters of each station by its own closed Kalman filter. The Kalman filter proposed in this paper is a stitching method based on the self-closing condition, which makes the data acquisition workload large and the point cloud stitching workload also large, which will be limited in practical applications; in addition, if the stitching of 3D image data of several kilometers or even tens of kilometers in length is performed, the error accumulation will be very serious and the stitching accuracy of this method will be difficult to guarantee.Figure 5
3D image control point coordinate change.Based on the projection-based central axis extraction, the global consistent central axis extraction and the normal central axis dynamic extraction methods are proposed to improve the accuracy of the central axis fitting of 3D images and provide references for the control conditions of point cloud stitching and the deformation monitoring of 3D images. A simple algorithm and generalized prior probability estimation method based on iterative statistics of model parameters are proposed, and the BAYSAC algorithm is improved to realize a high-precision discrete point cloud local surface fitting method by combining quadratic parametric surfaces, which provides a reference for its application in reverse engineering, deformation monitoring, 3D modeling, etc. The stitching of 3D point cloud data is realized by fusing reflection-valued images, introducing Kalman filtering into the stitching process of a large amount of point cloud data, and adding 3D image axis information as the stitching control condition, which effectively improves the problem of continuous accumulation of point cloud stitching errors and provides a guarantee for the subsequent processing and application of 3D laser point cloud data.


## 4.2. Optimization Design Analysis

Since there are limitations in machine wood, it is necessary to add bionics to computer processing, and we analyze the addition of the Ehrman filter mechanism to the 3D reconstruction next. Due to the selectivity of the Ehrman filter, the efficiency of 3D reconstruction can be greatly improved, and the experimental results are shown in Figure6. We can see the time saving from Figure 6. Due to the selectivity of the Ehrman filter, the total significant map of the image is reconstructed in 3D, and the time is shortened from 20212 ms to 11212 ms. Due to the different features that the image itself has, here are two other images with better results. Since this image is richer in information, the reconstructed 3D effect is also better, as discussed earlier. We can see that the time is shortened from 24821 ms to 3613 ms, so it is said that adding the Ehrman filter feature to the computer processing will be a great improvement. Some image reconstruction effects are good, some image reconstruction effects are not good, and in the 3D reconstruction with the attention mechanism, the result shows that the efficiency is improved.Figure 6
Comparison results of 3D reconstruction design optimization.Success rate is another quantitative representation of the accuracy performance of the optimal design algorithm, representing the coverage of the target in each frame of the image sequence by the optimal design algorithm. The center position error is judged mainly based on the center position of the target, ignoring the consideration of the target scale. The success rate, on the other hand, takes more into account the target frame covered by the rectangular frame of the optimal design optimization. Compared with the center position error, the success rate is judged on a more global basis. Figure7 gives the experimental results of the success rate of the improved Kalman filter algorithm and the other nine algorithms. From Figure 7, the success rate of KSTC algorithm is the highest for the jumping sequence, which indicates that the improved Kalman filter can successfully cope with the fast motion of the optimized design target and the blurred background. The improved Kalman filtering algorithm can successfully cope with the scale transformation of the target object in the FaceOcc2 sequence, and the improved Kalman filtering algorithm can cope with the local occlusion of the target object more effectively than the STC algorithm. In the jogging sequence, the STC algorithm fails to optimize the design, but the improved Kalman filter algorithm successfully optimizes the design to the target location and achieves the highest success rate.Figure 7
Success rate of optimization results.In order to verify the information optimization design effect of each method, the peak signal-to-noise ratio between the rasterized reference image without distorted data and the optimized design rasterized image is calculated. The specific experimental results are shown in Figure8.Figure 8
PSNR comparison of design optimization results by various methods of information.Considering the noise reduction capability, the clarity of the recovered image, and the time used by the algorithm, it is concluded from the above analysis that the mean square error of the Saturn and Peppers plots is 8.78% and 4.04% higher for the alternating Kalman filter recovery than the Wiener recovery, and the peak signal-to-noise ratio is basically the same. The calculation time of the alternating Kalman recovery is the percentage of the Wiener recovery time: Saturn, 1.7%, and Peppers, 1.9%. Therefore, the alternating Kalman filter recovery takes less than 1/50 of the Wiener recovery with approximately the same recovery effect. The alternating Kalman filter recovery has a significant advantage over the Wiener-Kalman filter recovery and Wiener-alternating Kalman filter recovery in terms of algorithm running time. The alternating Kalman filter combines the noise reduction ability, the clarity of the recovered image, and the time used by the algorithm, which is better than the other image recovery methods. However, as the noise level increases, the recovery effect of the alternating Kalman filter image recovery method becomes worse, but the recovery time does not change significantly.


## 5. Conclusion

The content of this chapter is linked with the stable and reliable extraction of topological structure information of the top surface of 3D image data and realizes the accurate and reliable extraction of 3D spatial information of images by combining remote sensing optical stereo image pairs and 3D image data. To reduce the noise in the degraded image, present the edge details of the recovered image clearly, and shorten the running time of the algorithm at the same time, an alternating Kalman filter recovery method is proposed in this paper. The alternating Kalman filter recovery method is compared with the Wiener recovery method, Kalman filter recovery method, Wiener-Kalman filter recovery method, and Wiener-alternating Kalman filter recovery method, and through multiple simulations, the images obtained by the alternating Kalman filter recovery method can clearly present the image edge detail information under the same noise, which is better than the joint Wiener recovery method and Wiener-Kalman filter recovery method for remote sensing. When optical images and 3D spatial information are extracted, the details of building structures in remote sensing optical images are disturbed by shadows and details of the surrounding environment, for which topological structure information needs to be provided by image data for constraint and compensation, while the image data extracts step features/edges imprecisely and the topological structure information extraction is unstable. To address the above problems, a global optimal segmentation method based on graded energy minimization of image data is proposed. By constructing a graded energy minimization objective function adapted to the image data, the global optimal extraction of topological structure information of the top surface of the image is stably achieved, and the accuracy of the joint extraction performance of the final image 3D spatial information is ensured. Finally, the accurate and reliable extraction of the top surface 3D spatial information of the image is realized under the support of the above innovative research results. Obviously, the research of the content of this paper effectively improves the utilization efficiency and performance of remote sensing multisource data for image 3D spatial information extraction, which has obvious economic and social value. The coefficients in the observation and prediction equations of Kalman filter are obtained through repeated experiments, so in future research, accurate equation coefficients will be obtained to improve the effect of alternate Kalman filter image recovery.



---

*Source: 7262871-2021-12-17.xml*