# Wireless Sensor System of UAV Infrared Image and Visible Light Image Registration Fusion

Authors: Lei Gu; Juan Meng
Journal: Journal of Electrical and Computer Engineering
 (2022)

---

## Abstract

The application of multisource sensors to drones requires high-quality images to ensure it. In simultaneous interpreting two or more multisensor images based on the same scene or target, the image obtained by the UAV sensor is limited by the imaging time and the shooting angle. The images obtained may not be aligned in the spatial position, thus affecting the fusion effect. Therefore, different sensor images must be registered before image fusion. During the shooting process of the drone imaging sensor, imaging angle, and environmental conditions, the obtained various sensor images will have rotation, translation, and other deformations in the spatial position so that they do not reach the spatial position. Therefore, it is impossible to directly perform image fusion directly. Therefore, before the multisensor image fusion, the image registration process must be completed to ensure that the two images are aligned in space. This paper analyzes the principles; based on the principles of the Powell search algorithm and improved walking algorithm, an algorithm combining Powell and improved walking algorithm is proposed. This paper also studies several traditional image neutrosophic fusions. The algorithm combines the fusion optimization algorithm proposed in this paper greatly reduces the calculation speed and improves the performance of the optimization algorithm and success rate.


---

## Body

## 1. Introduction

When multiple imaging sensors are used to shoot the same scene target, because the imaging sensor is affected by factors such as imaging time, shooting angle of view, and attitude, the obtained multisensor image has spatial displacement, distortion, and other deformations, so it needs to adjust the pixels before fusing different sensor images. Under multiview conditions and under different sensors, so that the same target in the image is aligned in space, and the translation and rotation between the images can be corrected. The geometric deformation of images and the accuracy and speed of image registration will directly affect the efficiency and quality of subsequent steps such as image fusion. With the development of sensor technology, aerial platform technology, communication technology, and optical technology, UAV imaging technology can develop rapidly and dynamically describe multiple types of views [1]. Despite the technology, airborne photoelectric imaging systems have also become an important means of obtaining aerial images and have received increasing attention. The airborne photoelectric imaging platform is an integrated photoelectric detection high-precision sensor device, such as a visible light camera device, an infrared image, and an infrared ranging device. UAVS with visible light cameras and infrared cameras make it widely used in aerial surveys, border patrols, geological surveys, unmanned surveillance, fire warning, on-site photoelectric imaging, equipment search, and rescue. The reconnaissance imaging mission instrument on the airborne electro-optical imaging platform is used to acquire target images. Infrared cameras use thermal radiation technology to look beyond the infrared wavelengths of the human eye to observe wavelength information and convert it into visible light information that can be mapped into an image. An infrared camera recognizes thermal targets and detects long distances. It has the characteristics of penetrating smoke and working day and night, but its image contrast is poor and the details are relatively rich. Visible light camera images have higher resolution. Since the differences and limitations of external images that depend on a single type of image for visible and infrared images are generally difficult to meet the actual needs of the project, how respective visible and infrared achieve the complementarity of information and the practical need for spatial correlation is a hot topic.After the development past two decades, a relatively complete theoretical research classification has been formed. At present, it is processed at the following level of fusion processing. Among them, pixel-level image fusion is currently the most widely studied and more commonly used technology. The pixel-level image fusion method requires that the fused image contains the complementary prominent feature, but it can also contain the spatial structure so that the fused image can reflect the comprehensive and rich information of the scene and the target. To obtain more information and more obvious images, it is necessary to introduce image fusion technology to fuse different types of image information to obtain high-quality, more comprehensive information-rich images [2]. The acquisition of the required imaging image by the airborne photoelectric imaging platform is affected by the hardware design of the imaging platform. When the drone is flying, it may be affected by wind, vibrations, refraction of light, and other factors resulting in the position, direction, size, and shape of the target in the image. There will be differences, and these will actually have a great impact on image fusion. Therefore, high-precision image registration is required before fusion. The image registration technology can match and spatially transform the same object in different perspectives and environments using different sensors. Its main purpose is to reduce or eliminate translation, rotation, and spatial transformation between images as much as possible and to obtain images with spatial consistency. Applying fusion technology to the UAV’s infrared target imaging platform system makes it easier to detect infrared target images. The end result is that it not only preserves the color and rough outline of the visible image but it makes the infrared target object stand out from the background brightness.Infrared and visible image fusion techniques are a hot topic in recent years in the research of multisensor fusion techniques. The existing infrared and visible light fusion techniques require registration before fusion due to the use of two cameras. However, the application effectiveness of the registration technique is yet to be improved. Therefore, Qiao T proposes a novel integrated multispectral sensor device for IR and visible light fusion by using a splitting prism to project coaxial light incident from the same lens onto an IR charge-coupled device (CCD) and a visible CCD, respectively. Quality evaluation metrics are used to analyze the simulation results. The experimental results show that the proposed sensor device is effective and feasible [3]. Alexander DC believes that recent observations encourage the use of ranking-based metrics. LambdaMART is the most advanced algorithm for learning rankings, and it relies on this metric. Despite its success, it does not have a principled regularization mechanism that relies on empirical methods to control model complexity and is therefore prone to overfitting. The low level is essentially a model complexity controller; most importantly, he proposes additional regularizers to constrain the potential representations of learning, which reflect user and item manifolds because they are based on primitive feature descriptors and preferences are defined. Finally, he also suggested using weighted variants of fines for similar products with large differences in ratings. His simple matrix completion setup evaluated the performance [4]. Unmanned aerial vehicles (UAVs) have become popular, and their use in agricultural monitoring is attracting increasing attention. Another class of agricultural UAVs has emerged, and their normal consumer-grade red/green/blue (RBG) band cameras have been modified to include near-infrared (NIR) bands, replacing one of the visible bands. This has reduced the cost of agricultural drones. However, few studies have evaluated the applicability of these modified UAV cameras in agricultural remote sensing. MUREFU used a modified UAV consumer-grade camera with blue/green/near-infrared (BGNIR) bands to evaluate its applicability in crop remote sensing monitoring. Green normalized vegetation index (GNDVI) processing maps from the UAV images were compared with actual ground data from geotagged images taken during the UAV flights. The visual comparison between ground and UAV images showed a positive correlation [5].The main purpose of this paper is to register and fuse infrared remote sensing. This article first outlines the basic principles of infrared and visible light images. Secondly, it briefly introduces the existing traditional registration and fusion algorithms and then proposes the registration and fusion algorithms in this paper. Finally, a comparison experiment with the existing algorithms using the proposed registration and fusion algorithm is carried out to analyze and verify the superiority of its registration fusion effect.


## 2. Proposed Method

### 2.1. Infrared and Visible Light Images

#### 2.1.1. Infrared Image

The development of thermal imaging technology began in the 1950s. At first, only thermal imagers based on unit devices can be developed. The field frequency is low and is limited to small-scale applications. It was not until the 1970s that the technology of medium- and long-wave HgCdTe (MCT) material and photoconductive multicomponent linear devices was mature that the thermal imager began to be mass-produced and equipped with the army. The development of infrared technology is marked by the development of infrared detectors. The detector realizes the function of the neural network on the focal plane and carries out logical processing according to the program to make the infrared whole machine intelligent. Infrared images are imaged according to different heat radiation capabilities between objects. Infrared image sensors present invisible infrared radiation radiated from the surface of objects in the scene as visible images [6, 7]. Due to the different heat radiation capabilities of various objects, the images can be distinguished from each other. It is also because of the imaging characteristics of the infrared image that it does not depend on the brightness of the external environment, has the characteristics of all-weather, and can be operated 24 hours a day without interruption. However, the clarity of infrared images is also affected by the sun’s light to a certain extent. At night, infrared imaging mainly relies on its own temperature for thermal radiation, and the image is blurred. During the day, the temperature difference between different objects is caused by the radiation and absorption of sunlight by objects larger; the image is sharper than at night [8, 9]. The image obtained by the infrared sensor is a grayscale image, lacking color and shadow. Because the infrared image is obtained by “measuring” the heat radiated by the object, it has poor resolution, low contrast, low signal-to-noise ratio, and blurred visual effect, and the gray distribution has no linear relationship with the target reflection characteristics. The way of image fusion is shown in Figure 1. Infrared thermal imaging technology is a passive, noncontact detection and recognition technology. It can use the infrared radiation characteristic image formed by the temperature difference or radiation difference between the target and the background or between each part of the target to find and identify the target. Its two basic functions are temperature measurement and night vision. The infrared detector and optical imaging objective lens are used to receive the infrared radiation energy distribution pattern of the measured target and reflect it on the photosensitive element of the infrared detector, so as to obtain the infrared thermal image, which corresponds to the thermal distribution field on the object surface.Figure 1
The way of image fusion.


#### 2.1.2. Visible Light Image

Visible light images are different images that use the object’s ability to reflect light visible to the human eye, the resolution is higher, and the infrared image has better detail expression ability and more comprehensive spectral information, but it also has some defects [10, 11]. Because visible light is imaged by the light reflected by the object, the imaging of visible light images is completely dependent on lighting, under the conditions of limited light, visible light images cannot display all information in the field of view [12, 13].


### 2.2. Image Preprocessing

#### 2.2.1. Image Desensitization

Image denoising is the main material properties, electronic components, circuit structure, and its working environment in the process of obtaining it; second, digital images may be contaminated by a variety of different noises [14, 15].There are two methods of image denoising: low-pass filtering; common transform, and Fourier transform [16, 17].


#### 2.2.2. Image Enhancement

Image enhancement is an indispensable content in preprocessing, to enhance the ability of image recognition and discrimination, improve the expression of its details, make the original visible details clearer, make blurred and unclear details clear and visible, and improve image quality [18, 19]. Also, a variety is summarized into two aspects: one is the enhancement in the spatial domain, and the other is the frequency domain. Enhancement of the spatial domain includes two types of point and field operations. The former is to perform linear or nonlinear processing on a single pixel, while the latter is to process a certain area of the image.


#### 2.2.3. Image Registration

Image registration needs to consider the external environment of the image as well as stitching and image fusion [20]. The registration work is the process of using the common objects in two images to find the spatial motion matrix between them through comparison and matching [21, 22]. Pixel-based registration methods and feature-based registration methods are the main registration methods at present [23, 24].(1) Pixel-based registration method. There are two commonly used pixel-based registration algorithms. The first is the gray-level image registration method, which is characterized in that the entire registration process does not need to detect image features and only completes the registration based on the gray information in the region; the other is a template registration method in which another sub-image is found in another image under the condition of a known template [25, 26].(2) Feature-based registration method. Feature-based registration method needs to extract relatively fixed features from two images and use the similarity between them to complete the registration. The process generally includes three steps: extracting features, generating descriptors, and finally completing the registration work based on calculating the similarity relationship between the feature descriptors [27, 28]. Because the two images are, respectively, captured by two sensors, the two images have different imaging principles, so the information expressed by the same target in the two is somewhat different. Infrared sensors rely on capturing infrared radiation emitted by objects to form images. The radiant ability of different objects is also very different. The smoother the surface, the less the radiant heat, and the darker the image; the visible light sensor is based on the visible light reflected by the object; the more visible light a smooth the surface reflects, the brighter the image will be [29]. The accuracy of registration seriously affects the quality of subsequent fusion result maps.


### 2.3. Image Registration Algorithm

#### 2.3.1. Gray Interpolation Technology

The gray value of nongrid points is estimated by using the gray value of surrounding grid points, that is to say, the gray value of nongrid points cannot be detected and must be estimated by interpolation method. In the process of transformation, the pixels of the floating image will not fall on the grid. An interpolation method is needed to estimate the intensity of grid points in the transformed image.(1) Nearest neighbor interpolation. Nearest neighbor interpolation is also called pixel copy interpolation. It is a relatively easy algorithm with low complexity. Point xi,yi uses it instead of (x, y).The corresponding formula is(1)Ix,y=Ixi,yi,xi,yi=argmindx,y,xi,yi,i=1,2,3,4.The calculation of the nearest neighbor interpolation algorithm is relatively simple, but because the gray value of the point closest to the point to be interpolated is selected, other points are ignored, so there is an error, which leads to inaccurate registration results.(2) Bilinear interpolation. To make up for the shortcomings of the nearest neighbor interpolation algorithm, researchers have proposed a new interpolation algorithm called the bilinear interpolation algorithm. Assume that there are four pixel points p1,p2,p3,p4 in the spatial range of pixel point P and the corresponding gray value, then gray value point p isIp(2)Ip=1−Δx1−ΔyI1+1−ΔxΔyI2+Δx1−ΔyI3+ΔxΔyI4,where Δx and Δy are the projections of the distance from p to p1 in the horizontal and vertical directions, respectively. The hierarchy of the image is shown in Figure 2.Figure 2
The hierarchy of the image.


#### 2.3.2. Powell and Improved Random Walk Hybrid Optimization Algorithm

The transmission speed of images is required to ensure real-time fusion. Combining the Powell algorithm with the improved random walk algorithm can speed up the moving speed of the initial point to the target position, fast convergence of the Powell algorithm, and the global optimization ability. The basic principle of the Powell and improved random walk hybrid algorithm is as follows: in each round of iteration, an initial point is selected, and the global best advantage found by the improved random walk algorithm is used as the initial point of the Powell algorithm, and let it serve as the initial point of the improved random walk algorithm, and carry out the next round of iterations until the the final best advantage is found.Specific steps of Powell and improved random walk algorithm are as follows:(1)
Set the number of iterationsn, search range, and select any initial point P within the range step λ.(2)
Iterate search using an improved random walk algorithm to find the current global optimal solutionp1.(3)
Calculate the sizes of the objective function valuesY and Y1 of p and p1 and compare their sizes. If Y<Y1, go to the next step; otherwise, go to the last step.(4)
Take the best advantagep1 obtained as the initial point of the Powell algorithm and perform an n-dimensional search to get its best advantage p2.(5)
Takep2 as the initial point of the improved random walk algorithm and return to step (2).(6)
Output the optimal solutionp1 and the objective function value Y1. Powell and improved random walk algorithms can make full use of their optimization ability and self-convergence speed to achieve the best parameter optimization and improve the overall registration level.


#### 2.3.3. Image Registration Based on Edge Region Extraction and Mutual Information

The relationship between the gray values of infrared and visible light images is very complicated. Some gray features of one image do not necessarily appear in other images, so the two images are not globally related. In infrared and visible images, a single gray value may correspond to multiple regions with low correlation gray values, so this part of the region must be removed during registration. However, this part of the area is generally difficult to find. There is a way to directly remove the area with the smallest gray change in each image, but the disadvantage of this method is that it is easy to remove the area with high correlation and small gray change. Because this kind of region is difficult to change the statistical correlation characteristics of the image, it will not have a great impact on the registration results. Based on the above theory, this paper uses the highly correlated regions of infrared and visible images for registration and uses the mutual information coefficient of fast calculation speed as the similarity measure function to achieve grayscale image registration.


### 2.4. Image Fusion Algorithm

#### 2.4.1. Traditional Fusion Algorithm

(1) Image fusion method based on lifting wavelet transform. After registration, the wavelet decomposition is lifted to obtain 3X, as shown in the following figure. According to the size of the required image, different layers can be decomposed. The lifting wavelet is to process different high- and low-frequency sub-bands, and finally use wavelet. The specific flowchart of image fusion based on lifting wavelet transform is shown in Figure 3.Figure 3
Flow chart of image fusion based on lifting wavelet transform.(2) PCA fusion method based on principal component analysis. The basic principle of PCA is as follows:LetY be an n-dimensional unit vector, mean μ = E(x), then the projection of X on Y is the inner product, and W=w1,w2,…,wnT is obtained by projection transformation, that is(3)W=YTX,thereby(4)X=YT−1W=∑i=1nwiyi.IfX achieves the smallest mean square error in m, the linear estimate of X is(5)x^=∑i=1mwiyi,1≤m≤n.The minimum mean square error at this time is(6)ε2m=∑i=m+1nEwiwiT=∑i=m+1nyiTRxyi,fU=rt+1y+∑m=0m−1yt+1+tyn,c=maxTt,Tt+1mU=∑n=1nπ+y,M=1ee1e2…t.SinceY is a standard orthogonal matrix, if you want to minimize the minimum mean square error, use the criterion function as follows:(7)J=∑i=m+1nyiTRxyi−∑i=m+1nλyiTyi−1,ht+1=dxt+xut,w1=q×n1.Becauseλi is the eigenvalue of Rx and λi is the eigenvector of the corresponding eigenvalue of Rx, we get(8)Rxwi=λiwi,SB=KSi,SN=minri−1rb,rb−1,hi=r,βr,mr,br|r∈ϕ.Substituting inε2m, the final minimum mean square error is obtained as follows:(9)ε2m=∑i=m+1nyiTRxwi=∑i=m+1nyiTλxyi=∑i=m+1nλi.Principle of the PCA, row vector, and feature vector obtained based on the maximum eigenvalue, as the first principal component corresponding feature vector, according to the obtained feature vector, determine the weight of the image to be fused and finally perform image fusion.To fuse the two images, the covariance matrix obtained is (m, n); the weight of one is m (m + n); the weight of the other image is n (m + n); and then the two weights are used for fusion.(3) Image fusion algorithm based on Laplace pyramid decomposition. The step of pyramid decomposition is as follows: suppose the source image M (x, y) is located at the bottom of the Laplace pyramid. The image above the source image is obtained by filtering the source image M with a 3 × 3 or 5 × 5 window Gaussian filter. The image layer gradually decreases from the bottom to the top, and the resolution gradually decreases. The expression formula of the Laplace pyramid decomposition of the image is(10)Mi=reduceMi−1,Mii,j=∑m=−22∑m=−22ϖm,nMi−12i+m,2j+n,d2=nm×a2,AN,P=tN,P+ϕ∑i=1MpM,Q×LA,N,A=BT+KT.The image obtained above the pyramid has a much lower resolution than the image below, and the size of the image gradually decreases. Therefore, it is necessary to use the grayscale interpolation technique to obtain an image of the same size as the image of this layer. The interpolation here is called the expand transform, which is the inverse transform of reduction. Performing k-expand transformations onM gives the image M, and the expression is(12)Mik=expandMi,k−1,Mii,j=4ϖm,nMik−1i+m2,j+n2,ϕ=−μg+γg+1,Q=mi+ni=pi+pi+yi,W=Gi+Bi=m+n.The Laplacian Pyramid Transform is an interpolation of each layer to the next. The principle of the image is decomposed at the same level to get to the top. Finally, for each Laplace pyramid, reconstruction is performed on the layer to obtain a new fused image.


#### 2.4.2. Fusion Algorithm Combining Color Space IHS Transform and Lifting Wavelet

So, when converting a visible light image into an infrared image, replacing the brightness part of the visible light with the brightness part of the infrared image is beneficial to enhance the object; After lifting the wavelet transform, integer pixels are obtained, and it does not depend on the Fourier transform, which can greatly improve the operation speed and save a lot of memory space. Visible light images are color images. Infrared images have a lot of brightness information. To take full advantage of the different characteristics of the two, the algorithm used must not only reflect the edge detail information. Therefore, the combination of can effectively retain the high-intensity image, and the algorithm has good real-time fusion and strong usability. The image fusion process based on IHS transform and lifting wavelet transform is shown in Figure4.(1)
Obtain components:I, H, and S(2)
Gray the image(3)
Enhance the wavelet to decompose the visible component and the grayscale infrared image and select the appropriate fusion rule to fuse the high- and low-frequency coefficients(4)
Perform inverse lifting wavelet transform obtained in step (3)(5)
Perform the newly obtained inverse IHS transform in step (4) and repeat step 1Figure 4
Image fusion process based on IHS transform and lifting wavelet transform.


## 2.1. Infrared and Visible Light Images

### 2.1.1. Infrared Image

The development of thermal imaging technology began in the 1950s. At first, only thermal imagers based on unit devices can be developed. The field frequency is low and is limited to small-scale applications. It was not until the 1970s that the technology of medium- and long-wave HgCdTe (MCT) material and photoconductive multicomponent linear devices was mature that the thermal imager began to be mass-produced and equipped with the army. The development of infrared technology is marked by the development of infrared detectors. The detector realizes the function of the neural network on the focal plane and carries out logical processing according to the program to make the infrared whole machine intelligent. Infrared images are imaged according to different heat radiation capabilities between objects. Infrared image sensors present invisible infrared radiation radiated from the surface of objects in the scene as visible images [6, 7]. Due to the different heat radiation capabilities of various objects, the images can be distinguished from each other. It is also because of the imaging characteristics of the infrared image that it does not depend on the brightness of the external environment, has the characteristics of all-weather, and can be operated 24 hours a day without interruption. However, the clarity of infrared images is also affected by the sun’s light to a certain extent. At night, infrared imaging mainly relies on its own temperature for thermal radiation, and the image is blurred. During the day, the temperature difference between different objects is caused by the radiation and absorption of sunlight by objects larger; the image is sharper than at night [8, 9]. The image obtained by the infrared sensor is a grayscale image, lacking color and shadow. Because the infrared image is obtained by “measuring” the heat radiated by the object, it has poor resolution, low contrast, low signal-to-noise ratio, and blurred visual effect, and the gray distribution has no linear relationship with the target reflection characteristics. The way of image fusion is shown in Figure 1. Infrared thermal imaging technology is a passive, noncontact detection and recognition technology. It can use the infrared radiation characteristic image formed by the temperature difference or radiation difference between the target and the background or between each part of the target to find and identify the target. Its two basic functions are temperature measurement and night vision. The infrared detector and optical imaging objective lens are used to receive the infrared radiation energy distribution pattern of the measured target and reflect it on the photosensitive element of the infrared detector, so as to obtain the infrared thermal image, which corresponds to the thermal distribution field on the object surface.Figure 1
The way of image fusion.


### 2.1.2. Visible Light Image

Visible light images are different images that use the object’s ability to reflect light visible to the human eye, the resolution is higher, and the infrared image has better detail expression ability and more comprehensive spectral information, but it also has some defects [10, 11]. Because visible light is imaged by the light reflected by the object, the imaging of visible light images is completely dependent on lighting, under the conditions of limited light, visible light images cannot display all information in the field of view [12, 13].


## 2.1.1. Infrared Image

The development of thermal imaging technology began in the 1950s. At first, only thermal imagers based on unit devices can be developed. The field frequency is low and is limited to small-scale applications. It was not until the 1970s that the technology of medium- and long-wave HgCdTe (MCT) material and photoconductive multicomponent linear devices was mature that the thermal imager began to be mass-produced and equipped with the army. The development of infrared technology is marked by the development of infrared detectors. The detector realizes the function of the neural network on the focal plane and carries out logical processing according to the program to make the infrared whole machine intelligent. Infrared images are imaged according to different heat radiation capabilities between objects. Infrared image sensors present invisible infrared radiation radiated from the surface of objects in the scene as visible images [6, 7]. Due to the different heat radiation capabilities of various objects, the images can be distinguished from each other. It is also because of the imaging characteristics of the infrared image that it does not depend on the brightness of the external environment, has the characteristics of all-weather, and can be operated 24 hours a day without interruption. However, the clarity of infrared images is also affected by the sun’s light to a certain extent. At night, infrared imaging mainly relies on its own temperature for thermal radiation, and the image is blurred. During the day, the temperature difference between different objects is caused by the radiation and absorption of sunlight by objects larger; the image is sharper than at night [8, 9]. The image obtained by the infrared sensor is a grayscale image, lacking color and shadow. Because the infrared image is obtained by “measuring” the heat radiated by the object, it has poor resolution, low contrast, low signal-to-noise ratio, and blurred visual effect, and the gray distribution has no linear relationship with the target reflection characteristics. The way of image fusion is shown in Figure 1. Infrared thermal imaging technology is a passive, noncontact detection and recognition technology. It can use the infrared radiation characteristic image formed by the temperature difference or radiation difference between the target and the background or between each part of the target to find and identify the target. Its two basic functions are temperature measurement and night vision. The infrared detector and optical imaging objective lens are used to receive the infrared radiation energy distribution pattern of the measured target and reflect it on the photosensitive element of the infrared detector, so as to obtain the infrared thermal image, which corresponds to the thermal distribution field on the object surface.Figure 1
The way of image fusion.


## 2.1.2. Visible Light Image

Visible light images are different images that use the object’s ability to reflect light visible to the human eye, the resolution is higher, and the infrared image has better detail expression ability and more comprehensive spectral information, but it also has some defects [10, 11]. Because visible light is imaged by the light reflected by the object, the imaging of visible light images is completely dependent on lighting, under the conditions of limited light, visible light images cannot display all information in the field of view [12, 13].


## 2.2. Image Preprocessing

### 2.2.1. Image Desensitization

Image denoising is the main material properties, electronic components, circuit structure, and its working environment in the process of obtaining it; second, digital images may be contaminated by a variety of different noises [14, 15].There are two methods of image denoising: low-pass filtering; common transform, and Fourier transform [16, 17].


### 2.2.2. Image Enhancement

Image enhancement is an indispensable content in preprocessing, to enhance the ability of image recognition and discrimination, improve the expression of its details, make the original visible details clearer, make blurred and unclear details clear and visible, and improve image quality [18, 19]. Also, a variety is summarized into two aspects: one is the enhancement in the spatial domain, and the other is the frequency domain. Enhancement of the spatial domain includes two types of point and field operations. The former is to perform linear or nonlinear processing on a single pixel, while the latter is to process a certain area of the image.


### 2.2.3. Image Registration

Image registration needs to consider the external environment of the image as well as stitching and image fusion [20]. The registration work is the process of using the common objects in two images to find the spatial motion matrix between them through comparison and matching [21, 22]. Pixel-based registration methods and feature-based registration methods are the main registration methods at present [23, 24].(1) Pixel-based registration method. There are two commonly used pixel-based registration algorithms. The first is the gray-level image registration method, which is characterized in that the entire registration process does not need to detect image features and only completes the registration based on the gray information in the region; the other is a template registration method in which another sub-image is found in another image under the condition of a known template [25, 26].(2) Feature-based registration method. Feature-based registration method needs to extract relatively fixed features from two images and use the similarity between them to complete the registration. The process generally includes three steps: extracting features, generating descriptors, and finally completing the registration work based on calculating the similarity relationship between the feature descriptors [27, 28]. Because the two images are, respectively, captured by two sensors, the two images have different imaging principles, so the information expressed by the same target in the two is somewhat different. Infrared sensors rely on capturing infrared radiation emitted by objects to form images. The radiant ability of different objects is also very different. The smoother the surface, the less the radiant heat, and the darker the image; the visible light sensor is based on the visible light reflected by the object; the more visible light a smooth the surface reflects, the brighter the image will be [29]. The accuracy of registration seriously affects the quality of subsequent fusion result maps.


## 2.2.1. Image Desensitization

Image denoising is the main material properties, electronic components, circuit structure, and its working environment in the process of obtaining it; second, digital images may be contaminated by a variety of different noises [14, 15].There are two methods of image denoising: low-pass filtering; common transform, and Fourier transform [16, 17].


## 2.2.2. Image Enhancement

Image enhancement is an indispensable content in preprocessing, to enhance the ability of image recognition and discrimination, improve the expression of its details, make the original visible details clearer, make blurred and unclear details clear and visible, and improve image quality [18, 19]. Also, a variety is summarized into two aspects: one is the enhancement in the spatial domain, and the other is the frequency domain. Enhancement of the spatial domain includes two types of point and field operations. The former is to perform linear or nonlinear processing on a single pixel, while the latter is to process a certain area of the image.


## 2.2.3. Image Registration

Image registration needs to consider the external environment of the image as well as stitching and image fusion [20]. The registration work is the process of using the common objects in two images to find the spatial motion matrix between them through comparison and matching [21, 22]. Pixel-based registration methods and feature-based registration methods are the main registration methods at present [23, 24].(1) Pixel-based registration method. There are two commonly used pixel-based registration algorithms. The first is the gray-level image registration method, which is characterized in that the entire registration process does not need to detect image features and only completes the registration based on the gray information in the region; the other is a template registration method in which another sub-image is found in another image under the condition of a known template [25, 26].(2) Feature-based registration method. Feature-based registration method needs to extract relatively fixed features from two images and use the similarity between them to complete the registration. The process generally includes three steps: extracting features, generating descriptors, and finally completing the registration work based on calculating the similarity relationship between the feature descriptors [27, 28]. Because the two images are, respectively, captured by two sensors, the two images have different imaging principles, so the information expressed by the same target in the two is somewhat different. Infrared sensors rely on capturing infrared radiation emitted by objects to form images. The radiant ability of different objects is also very different. The smoother the surface, the less the radiant heat, and the darker the image; the visible light sensor is based on the visible light reflected by the object; the more visible light a smooth the surface reflects, the brighter the image will be [29]. The accuracy of registration seriously affects the quality of subsequent fusion result maps.


## 2.3. Image Registration Algorithm

### 2.3.1. Gray Interpolation Technology

The gray value of nongrid points is estimated by using the gray value of surrounding grid points, that is to say, the gray value of nongrid points cannot be detected and must be estimated by interpolation method. In the process of transformation, the pixels of the floating image will not fall on the grid. An interpolation method is needed to estimate the intensity of grid points in the transformed image.(1) Nearest neighbor interpolation. Nearest neighbor interpolation is also called pixel copy interpolation. It is a relatively easy algorithm with low complexity. Point xi,yi uses it instead of (x, y).The corresponding formula is(1)Ix,y=Ixi,yi,xi,yi=argmindx,y,xi,yi,i=1,2,3,4.The calculation of the nearest neighbor interpolation algorithm is relatively simple, but because the gray value of the point closest to the point to be interpolated is selected, other points are ignored, so there is an error, which leads to inaccurate registration results.(2) Bilinear interpolation. To make up for the shortcomings of the nearest neighbor interpolation algorithm, researchers have proposed a new interpolation algorithm called the bilinear interpolation algorithm. Assume that there are four pixel points p1,p2,p3,p4 in the spatial range of pixel point P and the corresponding gray value, then gray value point p isIp(2)Ip=1−Δx1−ΔyI1+1−ΔxΔyI2+Δx1−ΔyI3+ΔxΔyI4,where Δx and Δy are the projections of the distance from p to p1 in the horizontal and vertical directions, respectively. The hierarchy of the image is shown in Figure 2.Figure 2
The hierarchy of the image.


### 2.3.2. Powell and Improved Random Walk Hybrid Optimization Algorithm

The transmission speed of images is required to ensure real-time fusion. Combining the Powell algorithm with the improved random walk algorithm can speed up the moving speed of the initial point to the target position, fast convergence of the Powell algorithm, and the global optimization ability. The basic principle of the Powell and improved random walk hybrid algorithm is as follows: in each round of iteration, an initial point is selected, and the global best advantage found by the improved random walk algorithm is used as the initial point of the Powell algorithm, and let it serve as the initial point of the improved random walk algorithm, and carry out the next round of iterations until the the final best advantage is found.Specific steps of Powell and improved random walk algorithm are as follows:(1)
Set the number of iterationsn, search range, and select any initial point P within the range step λ.(2)
Iterate search using an improved random walk algorithm to find the current global optimal solutionp1.(3)
Calculate the sizes of the objective function valuesY and Y1 of p and p1 and compare their sizes. If Y<Y1, go to the next step; otherwise, go to the last step.(4)
Take the best advantagep1 obtained as the initial point of the Powell algorithm and perform an n-dimensional search to get its best advantage p2.(5)
Takep2 as the initial point of the improved random walk algorithm and return to step (2).(6)
Output the optimal solutionp1 and the objective function value Y1. Powell and improved random walk algorithms can make full use of their optimization ability and self-convergence speed to achieve the best parameter optimization and improve the overall registration level.


### 2.3.3. Image Registration Based on Edge Region Extraction and Mutual Information

The relationship between the gray values of infrared and visible light images is very complicated. Some gray features of one image do not necessarily appear in other images, so the two images are not globally related. In infrared and visible images, a single gray value may correspond to multiple regions with low correlation gray values, so this part of the region must be removed during registration. However, this part of the area is generally difficult to find. There is a way to directly remove the area with the smallest gray change in each image, but the disadvantage of this method is that it is easy to remove the area with high correlation and small gray change. Because this kind of region is difficult to change the statistical correlation characteristics of the image, it will not have a great impact on the registration results. Based on the above theory, this paper uses the highly correlated regions of infrared and visible images for registration and uses the mutual information coefficient of fast calculation speed as the similarity measure function to achieve grayscale image registration.


## 2.3.1. Gray Interpolation Technology

The gray value of nongrid points is estimated by using the gray value of surrounding grid points, that is to say, the gray value of nongrid points cannot be detected and must be estimated by interpolation method. In the process of transformation, the pixels of the floating image will not fall on the grid. An interpolation method is needed to estimate the intensity of grid points in the transformed image.(1) Nearest neighbor interpolation. Nearest neighbor interpolation is also called pixel copy interpolation. It is a relatively easy algorithm with low complexity. Point xi,yi uses it instead of (x, y).The corresponding formula is(1)Ix,y=Ixi,yi,xi,yi=argmindx,y,xi,yi,i=1,2,3,4.The calculation of the nearest neighbor interpolation algorithm is relatively simple, but because the gray value of the point closest to the point to be interpolated is selected, other points are ignored, so there is an error, which leads to inaccurate registration results.(2) Bilinear interpolation. To make up for the shortcomings of the nearest neighbor interpolation algorithm, researchers have proposed a new interpolation algorithm called the bilinear interpolation algorithm. Assume that there are four pixel points p1,p2,p3,p4 in the spatial range of pixel point P and the corresponding gray value, then gray value point p isIp(2)Ip=1−Δx1−ΔyI1+1−ΔxΔyI2+Δx1−ΔyI3+ΔxΔyI4,where Δx and Δy are the projections of the distance from p to p1 in the horizontal and vertical directions, respectively. The hierarchy of the image is shown in Figure 2.Figure 2
The hierarchy of the image.


## 2.3.2. Powell and Improved Random Walk Hybrid Optimization Algorithm

The transmission speed of images is required to ensure real-time fusion. Combining the Powell algorithm with the improved random walk algorithm can speed up the moving speed of the initial point to the target position, fast convergence of the Powell algorithm, and the global optimization ability. The basic principle of the Powell and improved random walk hybrid algorithm is as follows: in each round of iteration, an initial point is selected, and the global best advantage found by the improved random walk algorithm is used as the initial point of the Powell algorithm, and let it serve as the initial point of the improved random walk algorithm, and carry out the next round of iterations until the the final best advantage is found.Specific steps of Powell and improved random walk algorithm are as follows:(1)
Set the number of iterationsn, search range, and select any initial point P within the range step λ.(2)
Iterate search using an improved random walk algorithm to find the current global optimal solutionp1.(3)
Calculate the sizes of the objective function valuesY and Y1 of p and p1 and compare their sizes. If Y<Y1, go to the next step; otherwise, go to the last step.(4)
Take the best advantagep1 obtained as the initial point of the Powell algorithm and perform an n-dimensional search to get its best advantage p2.(5)
Takep2 as the initial point of the improved random walk algorithm and return to step (2).(6)
Output the optimal solutionp1 and the objective function value Y1. Powell and improved random walk algorithms can make full use of their optimization ability and self-convergence speed to achieve the best parameter optimization and improve the overall registration level.


## 2.3.3. Image Registration Based on Edge Region Extraction and Mutual Information

The relationship between the gray values of infrared and visible light images is very complicated. Some gray features of one image do not necessarily appear in other images, so the two images are not globally related. In infrared and visible images, a single gray value may correspond to multiple regions with low correlation gray values, so this part of the region must be removed during registration. However, this part of the area is generally difficult to find. There is a way to directly remove the area with the smallest gray change in each image, but the disadvantage of this method is that it is easy to remove the area with high correlation and small gray change. Because this kind of region is difficult to change the statistical correlation characteristics of the image, it will not have a great impact on the registration results. Based on the above theory, this paper uses the highly correlated regions of infrared and visible images for registration and uses the mutual information coefficient of fast calculation speed as the similarity measure function to achieve grayscale image registration.


## 2.4. Image Fusion Algorithm

### 2.4.1. Traditional Fusion Algorithm

(1) Image fusion method based on lifting wavelet transform. After registration, the wavelet decomposition is lifted to obtain 3X, as shown in the following figure. According to the size of the required image, different layers can be decomposed. The lifting wavelet is to process different high- and low-frequency sub-bands, and finally use wavelet. The specific flowchart of image fusion based on lifting wavelet transform is shown in Figure 3.Figure 3
Flow chart of image fusion based on lifting wavelet transform.(2) PCA fusion method based on principal component analysis. The basic principle of PCA is as follows:LetY be an n-dimensional unit vector, mean μ = E(x), then the projection of X on Y is the inner product, and W=w1,w2,…,wnT is obtained by projection transformation, that is(3)W=YTX,thereby(4)X=YT−1W=∑i=1nwiyi.IfX achieves the smallest mean square error in m, the linear estimate of X is(5)x^=∑i=1mwiyi,1≤m≤n.The minimum mean square error at this time is(6)ε2m=∑i=m+1nEwiwiT=∑i=m+1nyiTRxyi,fU=rt+1y+∑m=0m−1yt+1+tyn,c=maxTt,Tt+1mU=∑n=1nπ+y,M=1ee1e2…t.SinceY is a standard orthogonal matrix, if you want to minimize the minimum mean square error, use the criterion function as follows:(7)J=∑i=m+1nyiTRxyi−∑i=m+1nλyiTyi−1,ht+1=dxt+xut,w1=q×n1.Becauseλi is the eigenvalue of Rx and λi is the eigenvector of the corresponding eigenvalue of Rx, we get(8)Rxwi=λiwi,SB=KSi,SN=minri−1rb,rb−1,hi=r,βr,mr,br|r∈ϕ.Substituting inε2m, the final minimum mean square error is obtained as follows:(9)ε2m=∑i=m+1nyiTRxwi=∑i=m+1nyiTλxyi=∑i=m+1nλi.Principle of the PCA, row vector, and feature vector obtained based on the maximum eigenvalue, as the first principal component corresponding feature vector, according to the obtained feature vector, determine the weight of the image to be fused and finally perform image fusion.To fuse the two images, the covariance matrix obtained is (m, n); the weight of one is m (m + n); the weight of the other image is n (m + n); and then the two weights are used for fusion.(3) Image fusion algorithm based on Laplace pyramid decomposition. The step of pyramid decomposition is as follows: suppose the source image M (x, y) is located at the bottom of the Laplace pyramid. The image above the source image is obtained by filtering the source image M with a 3 × 3 or 5 × 5 window Gaussian filter. The image layer gradually decreases from the bottom to the top, and the resolution gradually decreases. The expression formula of the Laplace pyramid decomposition of the image is(10)Mi=reduceMi−1,Mii,j=∑m=−22∑m=−22ϖm,nMi−12i+m,2j+n,d2=nm×a2,AN,P=tN,P+ϕ∑i=1MpM,Q×LA,N,A=BT+KT.The image obtained above the pyramid has a much lower resolution than the image below, and the size of the image gradually decreases. Therefore, it is necessary to use the grayscale interpolation technique to obtain an image of the same size as the image of this layer. The interpolation here is called the expand transform, which is the inverse transform of reduction. Performing k-expand transformations onM gives the image M, and the expression is(12)Mik=expandMi,k−1,Mii,j=4ϖm,nMik−1i+m2,j+n2,ϕ=−μg+γg+1,Q=mi+ni=pi+pi+yi,W=Gi+Bi=m+n.The Laplacian Pyramid Transform is an interpolation of each layer to the next. The principle of the image is decomposed at the same level to get to the top. Finally, for each Laplace pyramid, reconstruction is performed on the layer to obtain a new fused image.


### 2.4.2. Fusion Algorithm Combining Color Space IHS Transform and Lifting Wavelet

So, when converting a visible light image into an infrared image, replacing the brightness part of the visible light with the brightness part of the infrared image is beneficial to enhance the object; After lifting the wavelet transform, integer pixels are obtained, and it does not depend on the Fourier transform, which can greatly improve the operation speed and save a lot of memory space. Visible light images are color images. Infrared images have a lot of brightness information. To take full advantage of the different characteristics of the two, the algorithm used must not only reflect the edge detail information. Therefore, the combination of can effectively retain the high-intensity image, and the algorithm has good real-time fusion and strong usability. The image fusion process based on IHS transform and lifting wavelet transform is shown in Figure4.(1)
Obtain components:I, H, and S(2)
Gray the image(3)
Enhance the wavelet to decompose the visible component and the grayscale infrared image and select the appropriate fusion rule to fuse the high- and low-frequency coefficients(4)
Perform inverse lifting wavelet transform obtained in step (3)(5)
Perform the newly obtained inverse IHS transform in step (4) and repeat step 1Figure 4
Image fusion process based on IHS transform and lifting wavelet transform.


## 2.4.1. Traditional Fusion Algorithm

(1) Image fusion method based on lifting wavelet transform. After registration, the wavelet decomposition is lifted to obtain 3X, as shown in the following figure. According to the size of the required image, different layers can be decomposed. The lifting wavelet is to process different high- and low-frequency sub-bands, and finally use wavelet. The specific flowchart of image fusion based on lifting wavelet transform is shown in Figure 3.Figure 3
Flow chart of image fusion based on lifting wavelet transform.(2) PCA fusion method based on principal component analysis. The basic principle of PCA is as follows:LetY be an n-dimensional unit vector, mean μ = E(x), then the projection of X on Y is the inner product, and W=w1,w2,…,wnT is obtained by projection transformation, that is(3)W=YTX,thereby(4)X=YT−1W=∑i=1nwiyi.IfX achieves the smallest mean square error in m, the linear estimate of X is(5)x^=∑i=1mwiyi,1≤m≤n.The minimum mean square error at this time is(6)ε2m=∑i=m+1nEwiwiT=∑i=m+1nyiTRxyi,fU=rt+1y+∑m=0m−1yt+1+tyn,c=maxTt,Tt+1mU=∑n=1nπ+y,M=1ee1e2…t.SinceY is a standard orthogonal matrix, if you want to minimize the minimum mean square error, use the criterion function as follows:(7)J=∑i=m+1nyiTRxyi−∑i=m+1nλyiTyi−1,ht+1=dxt+xut,w1=q×n1.Becauseλi is the eigenvalue of Rx and λi is the eigenvector of the corresponding eigenvalue of Rx, we get(8)Rxwi=λiwi,SB=KSi,SN=minri−1rb,rb−1,hi=r,βr,mr,br|r∈ϕ.Substituting inε2m, the final minimum mean square error is obtained as follows:(9)ε2m=∑i=m+1nyiTRxwi=∑i=m+1nyiTλxyi=∑i=m+1nλi.Principle of the PCA, row vector, and feature vector obtained based on the maximum eigenvalue, as the first principal component corresponding feature vector, according to the obtained feature vector, determine the weight of the image to be fused and finally perform image fusion.To fuse the two images, the covariance matrix obtained is (m, n); the weight of one is m (m + n); the weight of the other image is n (m + n); and then the two weights are used for fusion.(3) Image fusion algorithm based on Laplace pyramid decomposition. The step of pyramid decomposition is as follows: suppose the source image M (x, y) is located at the bottom of the Laplace pyramid. The image above the source image is obtained by filtering the source image M with a 3 × 3 or 5 × 5 window Gaussian filter. The image layer gradually decreases from the bottom to the top, and the resolution gradually decreases. The expression formula of the Laplace pyramid decomposition of the image is(10)Mi=reduceMi−1,Mii,j=∑m=−22∑m=−22ϖm,nMi−12i+m,2j+n,d2=nm×a2,AN,P=tN,P+ϕ∑i=1MpM,Q×LA,N,A=BT+KT.The image obtained above the pyramid has a much lower resolution than the image below, and the size of the image gradually decreases. Therefore, it is necessary to use the grayscale interpolation technique to obtain an image of the same size as the image of this layer. The interpolation here is called the expand transform, which is the inverse transform of reduction. Performing k-expand transformations onM gives the image M, and the expression is(12)Mik=expandMi,k−1,Mii,j=4ϖm,nMik−1i+m2,j+n2,ϕ=−μg+γg+1,Q=mi+ni=pi+pi+yi,W=Gi+Bi=m+n.The Laplacian Pyramid Transform is an interpolation of each layer to the next. The principle of the image is decomposed at the same level to get to the top. Finally, for each Laplace pyramid, reconstruction is performed on the layer to obtain a new fused image.


## 2.4.2. Fusion Algorithm Combining Color Space IHS Transform and Lifting Wavelet

So, when converting a visible light image into an infrared image, replacing the brightness part of the visible light with the brightness part of the infrared image is beneficial to enhance the object; After lifting the wavelet transform, integer pixels are obtained, and it does not depend on the Fourier transform, which can greatly improve the operation speed and save a lot of memory space. Visible light images are color images. Infrared images have a lot of brightness information. To take full advantage of the different characteristics of the two, the algorithm used must not only reflect the edge detail information. Therefore, the combination of can effectively retain the high-intensity image, and the algorithm has good real-time fusion and strong usability. The image fusion process based on IHS transform and lifting wavelet transform is shown in Figure4.(1)
Obtain components:I, H, and S(2)
Gray the image(3)
Enhance the wavelet to decompose the visible component and the grayscale infrared image and select the appropriate fusion rule to fuse the high- and low-frequency coefficients(4)
Perform inverse lifting wavelet transform obtained in step (3)(5)
Perform the newly obtained inverse IHS transform in step (4) and repeat step 1Figure 4
Image fusion process based on IHS transform and lifting wavelet transform.


## 3. Experiments

### 3.1. Data Collection

In this paper, we selected images of different scenes from XenICs NV (Belgium), then generated infrared images of this photo, then used different algorithms to register and fuse them, and analyzed the registration fusion results.


### 3.2. Experimental Environment

Choose different given errors and iteration times. The experimental data is shown in Table1.Table 1
The experimental data.
Number of experimentsErrorsIteration timesMeasure functionTime (s)20.1103.64674.2330.1205.07415.0140.1305.09135.0650.3503.02315.4660.5503.07855.52


### 3.3. Objective Evaluation Indicators for Image Fusion

After the fusion processing is performed on the images captured by different sensors, desired requirements are also very important. This requires the evaluation of the obtained fusion image, and the evaluation method can be summarized into two kinds of subjective evaluation and objective evaluation. The positions of the initial and ending points in the article are shown in Table2.(1)
The average pixel value refers to the sum of all pixel values of the entire image to obtain the average value. It reflects the average brightness of the image.(2)
Image reflects the diversity of the information.(3)
Mutual information is used to represent the similarity between two images.(4)
Its ability is expressed to express details, that is, its clarity.(5)
The degree to which pixels in an image deviate from the average pixel is called discreteness.(6)
The spatial frequency reflects the changes of different pixel spaces.(7)
The root-mean-square error is the same as the mutual information entropy. It is a performance index obtained by computing two images and reflects the difference between the two images. To consider the effect of image fusion, this index is generally calculated for the fusion result map and the ideal image. When the value is smaller, it means that the difference between the two is smaller and the fusion quality is better.(8)
Structural similarity measures the similar relationship between two images. When the value is 1, the two are considered completely the same. For the evaluation of the fusion image quality, we choose to calculate the structural awaiting fusion. The higher the value, the better the fusion result quality.(9)
Also, the pixel difference between the two images is reflected when the value is higher.Table 2
The positions of the initial and ending points in the article.
Number of experimentsInitial pointEnd pointxyxy1002.331546.20142463.32458.56413494.21459.654745105.125410.1544By setting different step lengths and different iteration times, the experimental data obtained are shown in Table3.Table 3
By setting different step lengths and different iteration times the experimental data.
Number of experimentsXYTime (s)1−1−111.892425.343470.1944011.89


## 3.1. Data Collection

In this paper, we selected images of different scenes from XenICs NV (Belgium), then generated infrared images of this photo, then used different algorithms to register and fuse them, and analyzed the registration fusion results.


## 3.2. Experimental Environment

Choose different given errors and iteration times. The experimental data is shown in Table1.Table 1
The experimental data.
Number of experimentsErrorsIteration timesMeasure functionTime (s)20.1103.64674.2330.1205.07415.0140.1305.09135.0650.3503.02315.4660.5503.07855.52


## 3.3. Objective Evaluation Indicators for Image Fusion

After the fusion processing is performed on the images captured by different sensors, desired requirements are also very important. This requires the evaluation of the obtained fusion image, and the evaluation method can be summarized into two kinds of subjective evaluation and objective evaluation. The positions of the initial and ending points in the article are shown in Table2.(1)
The average pixel value refers to the sum of all pixel values of the entire image to obtain the average value. It reflects the average brightness of the image.(2)
Image reflects the diversity of the information.(3)
Mutual information is used to represent the similarity between two images.(4)
Its ability is expressed to express details, that is, its clarity.(5)
The degree to which pixels in an image deviate from the average pixel is called discreteness.(6)
The spatial frequency reflects the changes of different pixel spaces.(7)
The root-mean-square error is the same as the mutual information entropy. It is a performance index obtained by computing two images and reflects the difference between the two images. To consider the effect of image fusion, this index is generally calculated for the fusion result map and the ideal image. When the value is smaller, it means that the difference between the two is smaller and the fusion quality is better.(8)
Structural similarity measures the similar relationship between two images. When the value is 1, the two are considered completely the same. For the evaluation of the fusion image quality, we choose to calculate the structural awaiting fusion. The higher the value, the better the fusion result quality.(9)
Also, the pixel difference between the two images is reflected when the value is higher.Table 2
The positions of the initial and ending points in the article.
Number of experimentsInitial pointEnd pointxyxy1002.331546.20142463.32458.56413494.21459.654745105.125410.1544By setting different step lengths and different iteration times, the experimental data obtained are shown in Table3.Table 3
By setting different step lengths and different iteration times the experimental data.
Number of experimentsXYTime (s)1−1−111.892425.343470.1944011.89


## 4. Discussion

### 4.1. Analysis of Experimental Results of Image Registration Algorithm

#### 4.1.1. Analysis of Comparison Test Results of Optimization Algorithms

Since the optimal solution obtained directly based on the improved random walk algorithm is used as the initial point of the Powell algorithm each time, the probability of falling into a local extreme value can be effectively reduced. Powell algorithm settings: the initial point is set to (10, 0, 0, 0), and the number of iterations is 50, given errorε = 0.1; the parameter settings of the improved random walk algorithm: the initial step size is 5, and the search range is set as −100−10;5050;5050;−50−10T; and the parameter settings of Powell and the improved random walk optimization algorithm: the number of iterations is 150, and the search range is −100−10;5050;5050;−50−10T, and the initial step size is 5. The experimental results of the optimization algorithm are shown in Table 4 and Figure 5.Table 4
Experimental results of the optimization algorithm.
OptimizationΔxΔyΔθtMeasure functionCalculation time (s)Powell−49.2314−50.2352−1.13911.000−1.131310.131Improved walking algorithm−49.0237−50.2341−1.13561.002−1.131027.134Powell + improved walking algorithm−50.231−49.2314−1.13321.007−1.00235.211Figure 5
Experimental results of the optimization algorithm.It can be seen from Table4 and Figure 5 that the obtained affine transformation parameters are almost the same, and the measurement function is basically close, indicating that different initial points are set, and the three algorithms can obtain the best transformation parameters. The registration time of the algorithm combined with the improved random walk is greatly reduced, only 5.211 s, so it has good practicability. The three registration results are almost the same. Three sets of optimization algorithms were used to perform 50 sets of registration experiments. The average value of each set of experimental data was used to obtain the relevant results. Because it was the optimal solution obtained directly based on the improved random walk algorithm each time. Therefore, as the initial Powell algorithm, the probability of falling into local extremes is very high. The effect picture obtained by the algorithm in this paper is basically consistent with the original picture in the spatial position, and a higher registration consistency is achieved. The image registration parameters based on gradient information are shown in Table 5.Table 5
The image registration parameters based on gradient information.
ParameterX-axis translation/pixelY-axis translation/pixelNumber of iterationsRunning time (s)PSO-w7.65024.945416209.77CPSO8.08524.933817136.25QPSO7.88054.92001185.12The registration convergence curves of different optimization algorithms are compared, and the result is shown in Figure6.Figure 6
The registration convergence curves of different optimization algorithms are compared.However, highlight the characteristic content of infrared images and also include the structural characteristics of the visible light image, so the visual effect is the best. The comparison of different fusion methods is shown in Table6.Table 6
The comparison of different fusion methods.
AlgorithmInformation entropyStandard deviationMeanWavelet transform7.059032.9643151.8419NSCT7.128534.7858118.3729Paper algorithm7.197136.7598151.8419The comparison of the relevant results of the optimization algorithms is shown in Table7.Table 7
The comparison of the relevant results of the optimization algorithms.
OptimizationScaling factor,tMeasure functionCalculation time (s)Powell1.001−1.231310.231Improved random walk1.003−1.231027.234Powell + improved random walk1.008−1.10235.311


#### 4.1.2. Comparative Analysis with Traditional Methods

WhenD = 12, both the accuracy of the registration and the probability of success, the registration results of the translation and rotation images can reach the optimal value. Experiments were performed 50 times on the registration of the translated and rotated images with the traditional cross-correlation method, and the magnitude of the registration error value was compared with the result at D = 12. The comparison results with the traditional method are shown in Table 8 and Figure 7.Table 8
Comparison results with traditional methods.
Registration resultsTraditional methodEdge-region-based methodPan the imageRotate the imagePan the imageRotate the imageTRE16.2111.671.581.34Success rate01395100Figure 7
Compared with the traditional method.It can be seen from Table8 and Figure 7 that the traditional method based on mutual information cannot achieve accuracy. Within the registration, the success rate reached over 90%. The two sets of images were registered 50 times. Table 9 shows the registration results of the translational images with different D values.Table 9
Registration results of the translational images with differentD values.
Optimization resultD value2312X9.938.4313.13Y8.679.2315.34Θ0.23−0.020.05Table10 shows the registration results of the translation image d with different D values.Table 10
Registration results of the translation imaged with different D values.
Optimization resultD value2312X−1.250.64−0.29Y0.640.470.56θ3.231.236.34


### 4.2. Analysis of Experimental Results of Image Fusion Algorithm

#### 4.2.1. Analysis of Comparison Results of All Fusion Algorithms

To prove the superiority of the proposed improvement, in this paper, two traditional single fusion algorithms are used to perform principal component analysis and PCA fusion on the selected set of graphs, one of the two traditional algorithms is the substitution method, and the other is a wavelet transform-based fusion algorithm. It is analyzed and compared with objective indicators, specifically, the average gradient, spatial frequency, and structural similarity of the fusion result map are calculated to analyze the effect and quality of fusion. The comparison results of all fusion algorithms are shown in Table11 and Figure 8.Table 11
Analysis of comparison results of all fusion algorithms.
Fusion resultInformation entropyStandard deviationImage meanAverage gradientPCA fusion5.921623.1445109.13172.4231Laplace pyramid fusion5.929526.6453110.1353.1541IHS fusion7.21228.1351117.1453.2511Lifting wavelet fusion7.063428.2251111.15312.3521Algorithm proposed in this paper7.156729.0341120.1413.7911Figure 8
Analysis of comparison results of all fusion algorithms.According to Table11 and Figure 8, it is easy to get, the original algorithm results are better than other values and can achieve the desired results. The information entropy is 7.1567. Although it is slightly lower than the information entropy of the IHS fusion algorithm, the standard deviation is higher than other algorithms. Principal component analysis, PCA fusion, and Laplace pyramid fusion results are not clear, and there are distortions; the background of the image after color IHS fusion is blurred; the fusion image obtained by lifting the wavelet is clearer, but the brightness and color are weaker.


#### 4.2.2. Performance Analysis of Fusion Algorithm

A picture of a certain vehicle is selected, and fusion and lifting wavelet fusion are used to fuse this image with the algorithm proposed in this paper. The fusion result is shown in Figure9.Figure 9
Analysis of experimental results of fusion algorithm: (a) original, (b) IHS fusion, (c) lifting wavelet fusion, and (d) algorithm proposed in this paper.
(a)(b)(c)(d)It can be seen from Figure9 that after IHS fusion, the obtained fusion result has integrated color information and brightness information, but the shadow is not clear, and the contour information of some small buildings is lost; the fusion based on lifting wavelet combining the color and the brightness image makes the outline information of the building recognizable, but the overall is weakened; the fusion algorithm proposed in this paper not only has more detailed information than the IHS fusion result but also has more brightness information. The lifting wavelet fusion has enhanced, and the overall effect is more ideal than the fusion result of the former two. The registration results of the rotated image e at different D values are shown in Table 12.Table 12
The registration results of the rotated imagee at different D values.
Optimization resultD value2312X3.361.4616.16Y1.663.6615.64Θ0.66-0.060.05TRE is the result of the experiment and the root mean square of any point in the ideal state. The TRE when the picture rotates and translates is shown in Figure10.Figure 10
The TRE when the picture rotates and translates.The experimental data of different particle numbers and iteration times are shown in Table13.Table 13
The experimental data of different particle numbers and iteration times.
Number of experimentsNumber of particlesMutual information valueTime (s)1402.527458.9772204.529448.3773103.526436.321The four objective criteria of information entropy, standard deviation, and average gradient are used to objectively evaluate the fusion result, as shown in Figure11.Figure 11
Objectively evaluate the fusion result.Both the sift algorithm and the surf algorithm can detect feature points from infrared images and visible light images. Some data comparison between the sift algorithm and the surf algorithm is shown in Figure12.Figure 12
Some data comparison between the sift algorithm and the surf algorithm.The improved registration algorithm proposed in this paper can complete the registration well work. The registration effect is shown in Figure13.Figure 13
The registration effect.Figure14 shows the comparison of the weighted fusion method, the image fusion method based on Laplace transform, and the algorithm in this paper.Figure 14
Comparison of weighted fusion method, image fusion method based on Laplace transform, and the algorithm.


## 4.1. Analysis of Experimental Results of Image Registration Algorithm

### 4.1.1. Analysis of Comparison Test Results of Optimization Algorithms

Since the optimal solution obtained directly based on the improved random walk algorithm is used as the initial point of the Powell algorithm each time, the probability of falling into a local extreme value can be effectively reduced. Powell algorithm settings: the initial point is set to (10, 0, 0, 0), and the number of iterations is 50, given errorε = 0.1; the parameter settings of the improved random walk algorithm: the initial step size is 5, and the search range is set as −100−10;5050;5050;−50−10T; and the parameter settings of Powell and the improved random walk optimization algorithm: the number of iterations is 150, and the search range is −100−10;5050;5050;−50−10T, and the initial step size is 5. The experimental results of the optimization algorithm are shown in Table 4 and Figure 5.Table 4
Experimental results of the optimization algorithm.
OptimizationΔxΔyΔθtMeasure functionCalculation time (s)Powell−49.2314−50.2352−1.13911.000−1.131310.131Improved walking algorithm−49.0237−50.2341−1.13561.002−1.131027.134Powell + improved walking algorithm−50.231−49.2314−1.13321.007−1.00235.211Figure 5
Experimental results of the optimization algorithm.It can be seen from Table4 and Figure 5 that the obtained affine transformation parameters are almost the same, and the measurement function is basically close, indicating that different initial points are set, and the three algorithms can obtain the best transformation parameters. The registration time of the algorithm combined with the improved random walk is greatly reduced, only 5.211 s, so it has good practicability. The three registration results are almost the same. Three sets of optimization algorithms were used to perform 50 sets of registration experiments. The average value of each set of experimental data was used to obtain the relevant results. Because it was the optimal solution obtained directly based on the improved random walk algorithm each time. Therefore, as the initial Powell algorithm, the probability of falling into local extremes is very high. The effect picture obtained by the algorithm in this paper is basically consistent with the original picture in the spatial position, and a higher registration consistency is achieved. The image registration parameters based on gradient information are shown in Table 5.Table 5
The image registration parameters based on gradient information.
ParameterX-axis translation/pixelY-axis translation/pixelNumber of iterationsRunning time (s)PSO-w7.65024.945416209.77CPSO8.08524.933817136.25QPSO7.88054.92001185.12The registration convergence curves of different optimization algorithms are compared, and the result is shown in Figure6.Figure 6
The registration convergence curves of different optimization algorithms are compared.However, highlight the characteristic content of infrared images and also include the structural characteristics of the visible light image, so the visual effect is the best. The comparison of different fusion methods is shown in Table6.Table 6
The comparison of different fusion methods.
AlgorithmInformation entropyStandard deviationMeanWavelet transform7.059032.9643151.8419NSCT7.128534.7858118.3729Paper algorithm7.197136.7598151.8419The comparison of the relevant results of the optimization algorithms is shown in Table7.Table 7
The comparison of the relevant results of the optimization algorithms.
OptimizationScaling factor,tMeasure functionCalculation time (s)Powell1.001−1.231310.231Improved random walk1.003−1.231027.234Powell + improved random walk1.008−1.10235.311


### 4.1.2. Comparative Analysis with Traditional Methods

WhenD = 12, both the accuracy of the registration and the probability of success, the registration results of the translation and rotation images can reach the optimal value. Experiments were performed 50 times on the registration of the translated and rotated images with the traditional cross-correlation method, and the magnitude of the registration error value was compared with the result at D = 12. The comparison results with the traditional method are shown in Table 8 and Figure 7.Table 8
Comparison results with traditional methods.
Registration resultsTraditional methodEdge-region-based methodPan the imageRotate the imagePan the imageRotate the imageTRE16.2111.671.581.34Success rate01395100Figure 7
Compared with the traditional method.It can be seen from Table8 and Figure 7 that the traditional method based on mutual information cannot achieve accuracy. Within the registration, the success rate reached over 90%. The two sets of images were registered 50 times. Table 9 shows the registration results of the translational images with different D values.Table 9
Registration results of the translational images with differentD values.
Optimization resultD value2312X9.938.4313.13Y8.679.2315.34Θ0.23−0.020.05Table10 shows the registration results of the translation image d with different D values.Table 10
Registration results of the translation imaged with different D values.
Optimization resultD value2312X−1.250.64−0.29Y0.640.470.56θ3.231.236.34


## 4.1.1. Analysis of Comparison Test Results of Optimization Algorithms

Since the optimal solution obtained directly based on the improved random walk algorithm is used as the initial point of the Powell algorithm each time, the probability of falling into a local extreme value can be effectively reduced. Powell algorithm settings: the initial point is set to (10, 0, 0, 0), and the number of iterations is 50, given errorε = 0.1; the parameter settings of the improved random walk algorithm: the initial step size is 5, and the search range is set as −100−10;5050;5050;−50−10T; and the parameter settings of Powell and the improved random walk optimization algorithm: the number of iterations is 150, and the search range is −100−10;5050;5050;−50−10T, and the initial step size is 5. The experimental results of the optimization algorithm are shown in Table 4 and Figure 5.Table 4
Experimental results of the optimization algorithm.
OptimizationΔxΔyΔθtMeasure functionCalculation time (s)Powell−49.2314−50.2352−1.13911.000−1.131310.131Improved walking algorithm−49.0237−50.2341−1.13561.002−1.131027.134Powell + improved walking algorithm−50.231−49.2314−1.13321.007−1.00235.211Figure 5
Experimental results of the optimization algorithm.It can be seen from Table4 and Figure 5 that the obtained affine transformation parameters are almost the same, and the measurement function is basically close, indicating that different initial points are set, and the three algorithms can obtain the best transformation parameters. The registration time of the algorithm combined with the improved random walk is greatly reduced, only 5.211 s, so it has good practicability. The three registration results are almost the same. Three sets of optimization algorithms were used to perform 50 sets of registration experiments. The average value of each set of experimental data was used to obtain the relevant results. Because it was the optimal solution obtained directly based on the improved random walk algorithm each time. Therefore, as the initial Powell algorithm, the probability of falling into local extremes is very high. The effect picture obtained by the algorithm in this paper is basically consistent with the original picture in the spatial position, and a higher registration consistency is achieved. The image registration parameters based on gradient information are shown in Table 5.Table 5
The image registration parameters based on gradient information.
ParameterX-axis translation/pixelY-axis translation/pixelNumber of iterationsRunning time (s)PSO-w7.65024.945416209.77CPSO8.08524.933817136.25QPSO7.88054.92001185.12The registration convergence curves of different optimization algorithms are compared, and the result is shown in Figure6.Figure 6
The registration convergence curves of different optimization algorithms are compared.However, highlight the characteristic content of infrared images and also include the structural characteristics of the visible light image, so the visual effect is the best. The comparison of different fusion methods is shown in Table6.Table 6
The comparison of different fusion methods.
AlgorithmInformation entropyStandard deviationMeanWavelet transform7.059032.9643151.8419NSCT7.128534.7858118.3729Paper algorithm7.197136.7598151.8419The comparison of the relevant results of the optimization algorithms is shown in Table7.Table 7
The comparison of the relevant results of the optimization algorithms.
OptimizationScaling factor,tMeasure functionCalculation time (s)Powell1.001−1.231310.231Improved random walk1.003−1.231027.234Powell + improved random walk1.008−1.10235.311


## 4.1.2. Comparative Analysis with Traditional Methods

WhenD = 12, both the accuracy of the registration and the probability of success, the registration results of the translation and rotation images can reach the optimal value. Experiments were performed 50 times on the registration of the translated and rotated images with the traditional cross-correlation method, and the magnitude of the registration error value was compared with the result at D = 12. The comparison results with the traditional method are shown in Table 8 and Figure 7.Table 8
Comparison results with traditional methods.
Registration resultsTraditional methodEdge-region-based methodPan the imageRotate the imagePan the imageRotate the imageTRE16.2111.671.581.34Success rate01395100Figure 7
Compared with the traditional method.It can be seen from Table8 and Figure 7 that the traditional method based on mutual information cannot achieve accuracy. Within the registration, the success rate reached over 90%. The two sets of images were registered 50 times. Table 9 shows the registration results of the translational images with different D values.Table 9
Registration results of the translational images with differentD values.
Optimization resultD value2312X9.938.4313.13Y8.679.2315.34Θ0.23−0.020.05Table10 shows the registration results of the translation image d with different D values.Table 10
Registration results of the translation imaged with different D values.
Optimization resultD value2312X−1.250.64−0.29Y0.640.470.56θ3.231.236.34


## 4.2. Analysis of Experimental Results of Image Fusion Algorithm

### 4.2.1. Analysis of Comparison Results of All Fusion Algorithms

To prove the superiority of the proposed improvement, in this paper, two traditional single fusion algorithms are used to perform principal component analysis and PCA fusion on the selected set of graphs, one of the two traditional algorithms is the substitution method, and the other is a wavelet transform-based fusion algorithm. It is analyzed and compared with objective indicators, specifically, the average gradient, spatial frequency, and structural similarity of the fusion result map are calculated to analyze the effect and quality of fusion. The comparison results of all fusion algorithms are shown in Table11 and Figure 8.Table 11
Analysis of comparison results of all fusion algorithms.
Fusion resultInformation entropyStandard deviationImage meanAverage gradientPCA fusion5.921623.1445109.13172.4231Laplace pyramid fusion5.929526.6453110.1353.1541IHS fusion7.21228.1351117.1453.2511Lifting wavelet fusion7.063428.2251111.15312.3521Algorithm proposed in this paper7.156729.0341120.1413.7911Figure 8
Analysis of comparison results of all fusion algorithms.According to Table11 and Figure 8, it is easy to get, the original algorithm results are better than other values and can achieve the desired results. The information entropy is 7.1567. Although it is slightly lower than the information entropy of the IHS fusion algorithm, the standard deviation is higher than other algorithms. Principal component analysis, PCA fusion, and Laplace pyramid fusion results are not clear, and there are distortions; the background of the image after color IHS fusion is blurred; the fusion image obtained by lifting the wavelet is clearer, but the brightness and color are weaker.


### 4.2.2. Performance Analysis of Fusion Algorithm

A picture of a certain vehicle is selected, and fusion and lifting wavelet fusion are used to fuse this image with the algorithm proposed in this paper. The fusion result is shown in Figure9.Figure 9
Analysis of experimental results of fusion algorithm: (a) original, (b) IHS fusion, (c) lifting wavelet fusion, and (d) algorithm proposed in this paper.
(a)(b)(c)(d)It can be seen from Figure9 that after IHS fusion, the obtained fusion result has integrated color information and brightness information, but the shadow is not clear, and the contour information of some small buildings is lost; the fusion based on lifting wavelet combining the color and the brightness image makes the outline information of the building recognizable, but the overall is weakened; the fusion algorithm proposed in this paper not only has more detailed information than the IHS fusion result but also has more brightness information. The lifting wavelet fusion has enhanced, and the overall effect is more ideal than the fusion result of the former two. The registration results of the rotated image e at different D values are shown in Table 12.Table 12
The registration results of the rotated imagee at different D values.
Optimization resultD value2312X3.361.4616.16Y1.663.6615.64Θ0.66-0.060.05TRE is the result of the experiment and the root mean square of any point in the ideal state. The TRE when the picture rotates and translates is shown in Figure10.Figure 10
The TRE when the picture rotates and translates.The experimental data of different particle numbers and iteration times are shown in Table13.Table 13
The experimental data of different particle numbers and iteration times.
Number of experimentsNumber of particlesMutual information valueTime (s)1402.527458.9772204.529448.3773103.526436.321The four objective criteria of information entropy, standard deviation, and average gradient are used to objectively evaluate the fusion result, as shown in Figure11.Figure 11
Objectively evaluate the fusion result.Both the sift algorithm and the surf algorithm can detect feature points from infrared images and visible light images. Some data comparison between the sift algorithm and the surf algorithm is shown in Figure12.Figure 12
Some data comparison between the sift algorithm and the surf algorithm.The improved registration algorithm proposed in this paper can complete the registration well work. The registration effect is shown in Figure13.Figure 13
The registration effect.Figure14 shows the comparison of the weighted fusion method, the image fusion method based on Laplace transform, and the algorithm in this paper.Figure 14
Comparison of weighted fusion method, image fusion method based on Laplace transform, and the algorithm.


## 4.2.1. Analysis of Comparison Results of All Fusion Algorithms

To prove the superiority of the proposed improvement, in this paper, two traditional single fusion algorithms are used to perform principal component analysis and PCA fusion on the selected set of graphs, one of the two traditional algorithms is the substitution method, and the other is a wavelet transform-based fusion algorithm. It is analyzed and compared with objective indicators, specifically, the average gradient, spatial frequency, and structural similarity of the fusion result map are calculated to analyze the effect and quality of fusion. The comparison results of all fusion algorithms are shown in Table11 and Figure 8.Table 11
Analysis of comparison results of all fusion algorithms.
Fusion resultInformation entropyStandard deviationImage meanAverage gradientPCA fusion5.921623.1445109.13172.4231Laplace pyramid fusion5.929526.6453110.1353.1541IHS fusion7.21228.1351117.1453.2511Lifting wavelet fusion7.063428.2251111.15312.3521Algorithm proposed in this paper7.156729.0341120.1413.7911Figure 8
Analysis of comparison results of all fusion algorithms.According to Table11 and Figure 8, it is easy to get, the original algorithm results are better than other values and can achieve the desired results. The information entropy is 7.1567. Although it is slightly lower than the information entropy of the IHS fusion algorithm, the standard deviation is higher than other algorithms. Principal component analysis, PCA fusion, and Laplace pyramid fusion results are not clear, and there are distortions; the background of the image after color IHS fusion is blurred; the fusion image obtained by lifting the wavelet is clearer, but the brightness and color are weaker.


## 4.2.2. Performance Analysis of Fusion Algorithm

A picture of a certain vehicle is selected, and fusion and lifting wavelet fusion are used to fuse this image with the algorithm proposed in this paper. The fusion result is shown in Figure9.Figure 9
Analysis of experimental results of fusion algorithm: (a) original, (b) IHS fusion, (c) lifting wavelet fusion, and (d) algorithm proposed in this paper.
(a)(b)(c)(d)It can be seen from Figure9 that after IHS fusion, the obtained fusion result has integrated color information and brightness information, but the shadow is not clear, and the contour information of some small buildings is lost; the fusion based on lifting wavelet combining the color and the brightness image makes the outline information of the building recognizable, but the overall is weakened; the fusion algorithm proposed in this paper not only has more detailed information than the IHS fusion result but also has more brightness information. The lifting wavelet fusion has enhanced, and the overall effect is more ideal than the fusion result of the former two. The registration results of the rotated image e at different D values are shown in Table 12.Table 12
The registration results of the rotated imagee at different D values.
Optimization resultD value2312X3.361.4616.16Y1.663.6615.64Θ0.66-0.060.05TRE is the result of the experiment and the root mean square of any point in the ideal state. The TRE when the picture rotates and translates is shown in Figure10.Figure 10
The TRE when the picture rotates and translates.The experimental data of different particle numbers and iteration times are shown in Table13.Table 13
The experimental data of different particle numbers and iteration times.
Number of experimentsNumber of particlesMutual information valueTime (s)1402.527458.9772204.529448.3773103.526436.321The four objective criteria of information entropy, standard deviation, and average gradient are used to objectively evaluate the fusion result, as shown in Figure11.Figure 11
Objectively evaluate the fusion result.Both the sift algorithm and the surf algorithm can detect feature points from infrared images and visible light images. Some data comparison between the sift algorithm and the surf algorithm is shown in Figure12.Figure 12
Some data comparison between the sift algorithm and the surf algorithm.The improved registration algorithm proposed in this paper can complete the registration well work. The registration effect is shown in Figure13.Figure 13
The registration effect.Figure14 shows the comparison of the weighted fusion method, the image fusion method based on Laplace transform, and the algorithm in this paper.Figure 14
Comparison of weighted fusion method, image fusion method based on Laplace transform, and the algorithm.


## 5. Conclusions

(1)
UAV imaging technology for wireless sensor systems has been the focus of research. Aiming at the efficiency and accuracy of image registration requirements for UAV imaging platforms, Powell and an improved random walk algorithm are combined, and the optimization algorithm is experimentally verified. Not only can the improved random walk algorithm be used to find the global optimal solution, but Powell can be prevented from falling into the local extreme value, which greatly reduces the calculation speed and improves the performance of the optimization algorithm as a whole. Aiming at the low correlation between infrared images and visible light images, combining the edge area and mutual information can effectively solve the problem that heterogeneous images cannot be registered and effectively improve the registration success rate and accuracy. With the rapid development of imaging sensor technology and the increasing complexity of the application environment, process the information obtained by multiple imaging sensors and increase the usefulness of information. Carrying imaging sensors on UAVs makes UAVs widely used in military and civilian fields. The registration technology and fusion technology based on the drone platform not only can reduce the misjudgment rate of infrared targets but also can make it easier to detect infrared targets.(2)
Image registration improves the overall effect of wavelet fusion algorithms, principal component analysis, PCA fusion, and Laplace pyramid transform. More importantly, being faster than the traditional wavelet fusion processing, it has strong practicability when applied to drones.(3)
This paper has conducted in-depth research on infrared and visible light images and carried out the corresponding experimental verification. However, there are still areas that need improvement on specific issues. The future research work includes the following aspects: (1) the registration method based on the edge region and cross-correlation proposed in this paper may have errors in the selection of theD value during edge extraction. It can be studied from how to reduce the error as much as possible. (2) The fusion algorithm proposed in this paper is a fusion algorithm, and the future fusion of infrared and visible light images may be considered. (3) The experiments in this article are only simulated experiments on MATLAB; in the future, an imaging and transmission platform for drones can be built to achieve synchronization of image registration and fusion.



---

*Source: 9245014-2022-06-02.xml*