# Local Defogging Algorithm for Improving Visual Impact in Image Based on Multiobjective Optimization

Authors: Qiuju Lu
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

The preprocessing of images is required for many applications based on industry, social, and academic requirements. Researchers have developed a number of techniques to improve the visual effect of images and appropriately interpret visual effects. The accuracy of visuals is important in cyber security, military organization, police organizations, and forensics to detect the true story from the pictures. They search for evidence by digging deep into the network in search of evidence. If visuals are not clear, preprocessing of images is not done correctly, then it may lead to wrong interpretations. This paper proposes an image local defogging technique based on multiobjective optimization to improve the visual effect of the image as well as the information entropy. The multiobjective function is selected to establish the image reconstruction model based on multiple objectives. The model is utilized to reconstruct a single image to moderate the impact of noise and other interference factors in the original image. The color constancy model and effective detail intensity model are also devised for image enhancement to get the visual details. The atmospheric light value and transmittance are evaluated using a physical model of atmospheric scattering, and the guided filter is used to maximize the transmittance of a single image and improve the efficiency of image defogging. The dark channel priority method is used to realize the local defogging of a single image and to design the local defogging algorithm. Experiments verify the optimization effect of the proposed algorithm in terms of information entropy and container network interface (CNI) value. The tone restoration degree is good, and it improves the overall image quality. The image defogging effect of the proposed algorithm is verified with respect to subjective and objective levels to check the efficacy of the proposed multiobjective model.


---

## Body

## 1. Introduction

Ordinary optical imaging of outdoor scenes is often affected by foggy weather [1]. In foggy weather, the light reflected from the object’s surface will be scattered under the influence of suspended particles in the atmosphere before reaching the imaging equipment [2]. The degree of scattering is proportional to the type, size, shape, and degree of coagulation of dispersed particulates in the air, which means the fog concentration is proportional to light wavelength. [3]. When the fog is thick, the contrast of the image obtained by the imaging equipment is low, and the color is biased to gray white resulting in unclear identification of objects in the image [4]. This directly affects the normal work of most automatic image systems based on computer vision algorithms such as transportation, outdoor monitoring, and terrain detection system [5]. Therefore, the research of image defogging algorithm has an imperative need to apply this algorithm in real-time applications [6].Many authors have presented their respective works in the area of image processing for defogging the images. The authors proposed an environmental scattering model and retinex-based infrared image defogging technique in [7]. The atmospheric scattering model is used to rebuild the infrared image based on the resemblances between infrared and visible images in the degradation process in foggy conditions. Based on the difference in radiation, infrared images can differentiate targets from their environments, which works effectively in all weather conditions whereas textual details are provided by visual images. However, after defogging and restoration, the image often has the characteristics of low contrast and unclear details which are not conducive to direct observation by human eyes. In this case, to improve the contrast of the defogged image, the method employs the use of retinex model. The test results reveal that after algorithm processing, the signal-to-noise ratio (SNR) of infrared images can be improved, and this work can find a balance between calculation processing speed and defogging process, laying the groundwork for later embedded platforms to realize real-time video defogging. Both infrared and visible images process the intensity of the gray level in the image. In [8], authors propose a single-image defogging algorithm based on deep learning. Using a convolutional network (CNN), defogging is realized by learning the mapping relationship between YUV (Y represents brightness and UV represents chromaticity) channels of foggy image and clear image. The network structure consists of two identical feature modules mainly including multiscale convolution, convolution, and jump connection structure. The experimental results show that the image restored by the algorithm has a good processing effect irrespective of synthetic fog image data set or natural fog image data set, but the container network interface value is low indicating that there is less effective information in the image.In [9], authors propose an image dehazing method based on the second-order variation model. First, the atmospheric light value and the original transmittance map of the foggy image are estimated by the dark primary color prior method, and then, the nonlinear diffusion model is applied to the image. In the solution of the transmittance map, it is combined with the second-order variational model, the Hessian matrix variational model, the total generalized variational model, and the total curvature variational model. The results show that the edge of the image after processing is well maintained, and the image noise can be suppressed but the degree of tone restoration and the visual effect of the image are not good.In [10], dehazing models (H-LV model, H-HMV model, H-TGV model, and H-TCV model) are used to augment the image quality through defogging process. In order to improve the computational efficiency, the corresponding exchange direction multiplier algorithms are designed for the four models. By introducing auxiliary variables, the Lagrangian multipliers are continuously updated and iterated until the energy equation converges, and finally, the dehazed image is the output. Finally, the LIVE image defogging image database is used to verify the method experimentally.In [11], the local aerial elements of the image are extracted and then resortation is performed using the multiscale fusion method for fog-free image. In foggy images, each cluster becomes a line in RGB space, which the method may utilize to reconstruct digital elevation models and fog-free images. When dealing with cloudy images under complex sensing situations, these solutions are not ideal. Some researchers used convolution neural networks (CNN) to train foggy images. In [12], a defogging network of images was built using CNN. Fog-free images were extracted using the atmospheric scattering model. But this model was built based on a prior assumption like in traditional methods. In [13], a gated fusion defogging network (GFN) was proposed which is built through the sequence of operations such as contrast enhancement, gamma correction, and white balance. But the implementations of these operations were complicated [14].To progress the visual effect of the image, a local dehazing algorithm for a single image constructed on a multiobjective optimization algorithm is suggested in this paper to address the limitations in the existing literature. By analyzing the verification results at the subjective and objective levels, it can be seen that the images processed by the proposed algorithm have high sharpness and contrast and are superior to other contrast algorithms in both subjective and objective evaluations. The major contributions of the paper are as follows:(i)
To boost the visual effect and information entropy of an image, an image local defogging algorithm based on multiobjective optimization is proposed in this paper.(ii)
The proposed model is utilized to reconstruct a single image to shrink the effect of noise and other meddling factors in the original image. The color constancy model and effective detail intensity model are also used for image enhancement to get the better visual details.(iii)
According to the physical model of atmospheric scattering, the atmospheric light value and transmittance are estimated. The guided filter technique is used to increase transmittance and the effectiveness of image defogging by maximizing the transmittance of a single image.(iv)
The dark channel priority method is used to realize the local defogging of a single image and to design the local defogging algorithm.(v)
The proposed model improves the overall image quality. The image defogging effect of the proposed algorithm is verified with respect to subjective and objective levels to check the efficacy of the proposed multiobjective model.The rest of the paper is organized as follows: in Section 2, the single image reconstruction method based on a multiobjective optimization algorithm is elaborated. In Section 3, the model for image enhancement is discussed. In Section 4, local dehazing algorithm for a single image is described. In Section 5, experimental results and analysis is discussed in detail using the sampling method and evaluation indicators (objective and subjective quality evaluation). The last section 6 concludes the research study.


## 2. Single Image Reconstruction Model Based on Multiobjective Optimization

### 2.1. Selection of Multiobjective Function

The process of image defogging begins with determining the optimization target. The ultimate image processing effect is determined by the optimization target chosen. The task of image processing as well as the final image quality must be taken into account while choosing optimization objectives. The objectives of image processing differ due to the various tasks of image processing, in which an impact on the optimization targets has been chosen. The single image reconstruction processing optimization problem has many optimization objectives, making it a multi-objective optimization problem. In general, there will be conflicts between multiple targets in image processing, and there is no optimal solution for all targets at the same time. The improvement of one target performance is often at the cost of reducing the performance of one or more other targets [15].Considering the above problems, in order to effectively suppress noise in image reconstruction, the norm of the image is selected as the first objective function as shown in the following equation:(1)δl=∑k=1Nak+bk2,where l represents the peak value of the image; ak represents the structural component of the image; bk represents the texture component of the image; N represents the number of image pixels; k represents the gamma.Through the optimization of the objective function, the reconstructed image has the smallest peak globally, thereby reducing the interference of noise and effectively suppressing the artifacts caused by noise interference.According to the characteristics of the reconstructed image itself, each subimage is not in a continuous gray level, but only in a limited number of gray frequency bands. The smooth function [16] is chosen as the second objective function, as indicated in equation (2), to further remove the influence of statistical noise in the iterative process and the influence of statistical noise included in the original data, and to increase the quality of image reconstruction.(2)δr=12∑k∈KNh1γk+h2γk−1×ηk=12hTDh,where h1 and h2 represent the set of pixels belonging to the same subimage; D represent the smoothing matrix; ηk represent the image intensity.By minimizing the smoothing function represented in equation (2), there is a minimum nonuniformity between the pixels of the same subimage. On the one hand, to further strengthen the suppression of various noises and improve the quality of image reconstruction, the joint action with other objective functions is taken to increase the accuracy of image reconstruction, and the results are improved.In order to further effectively recover the target image from the noisy image and improve the image reconstruction effect, the log-likelihood function is used as the third objective function of the reconstruction model is shown in the following equation:(3)δm=minfzxϕiΔmaxPx+ϕjΔminPx,where ϕi and ϕj represent the noisy smooth block and the noisy nonsmooth block, respectively; fzx represents the log-likelihood estimation of the image block; ΔmaxPx represents the maximum variance estimate; and ΔminPy represents the minimum variance estimate.


### 2.2. Single Image Reconstruction Model

Considering the similarity of the objective functions,δl, δr, and δm, and the consistency of the optimization process, the above optimization objective functions can be combined into one objective function as shown in the following equation:(4)δk2=∑i,j=1Nδl+δr+δmij.Finally, in order to control the accuracy of image reconstruction, in the reconstruction process, the following constraints are set as given in the following equation:(5)Gh=∑i=1N∑j=1Mxij+yij−μ=0,where μ represents a preset constant used to control the accuracy of reconstruction.So far, a single image reconstruction modelSk based on multiobjective optimization can be obtained as given in the following equation:(6)Sk=maxδl,minδr,minδm,s,t,hδ=0.To summarize, the reconstruction of a single image based on a multiobjective optimization algorithm [17] can get better reconstruction results and reduce the influence of noise and other interference factors in the original image on the later image defogging effect.


## 2.1. Selection of Multiobjective Function

The process of image defogging begins with determining the optimization target. The ultimate image processing effect is determined by the optimization target chosen. The task of image processing as well as the final image quality must be taken into account while choosing optimization objectives. The objectives of image processing differ due to the various tasks of image processing, in which an impact on the optimization targets has been chosen. The single image reconstruction processing optimization problem has many optimization objectives, making it a multi-objective optimization problem. In general, there will be conflicts between multiple targets in image processing, and there is no optimal solution for all targets at the same time. The improvement of one target performance is often at the cost of reducing the performance of one or more other targets [15].Considering the above problems, in order to effectively suppress noise in image reconstruction, the norm of the image is selected as the first objective function as shown in the following equation:(1)δl=∑k=1Nak+bk2,where l represents the peak value of the image; ak represents the structural component of the image; bk represents the texture component of the image; N represents the number of image pixels; k represents the gamma.Through the optimization of the objective function, the reconstructed image has the smallest peak globally, thereby reducing the interference of noise and effectively suppressing the artifacts caused by noise interference.According to the characteristics of the reconstructed image itself, each subimage is not in a continuous gray level, but only in a limited number of gray frequency bands. The smooth function [16] is chosen as the second objective function, as indicated in equation (2), to further remove the influence of statistical noise in the iterative process and the influence of statistical noise included in the original data, and to increase the quality of image reconstruction.(2)δr=12∑k∈KNh1γk+h2γk−1×ηk=12hTDh,where h1 and h2 represent the set of pixels belonging to the same subimage; D represent the smoothing matrix; ηk represent the image intensity.By minimizing the smoothing function represented in equation (2), there is a minimum nonuniformity between the pixels of the same subimage. On the one hand, to further strengthen the suppression of various noises and improve the quality of image reconstruction, the joint action with other objective functions is taken to increase the accuracy of image reconstruction, and the results are improved.In order to further effectively recover the target image from the noisy image and improve the image reconstruction effect, the log-likelihood function is used as the third objective function of the reconstruction model is shown in the following equation:(3)δm=minfzxϕiΔmaxPx+ϕjΔminPx,where ϕi and ϕj represent the noisy smooth block and the noisy nonsmooth block, respectively; fzx represents the log-likelihood estimation of the image block; ΔmaxPx represents the maximum variance estimate; and ΔminPy represents the minimum variance estimate.


## 2.2. Single Image Reconstruction Model

Considering the similarity of the objective functions,δl, δr, and δm, and the consistency of the optimization process, the above optimization objective functions can be combined into one objective function as shown in the following equation:(4)δk2=∑i,j=1Nδl+δr+δmij.Finally, in order to control the accuracy of image reconstruction, in the reconstruction process, the following constraints are set as given in the following equation:(5)Gh=∑i=1N∑j=1Mxij+yij−μ=0,where μ represents a preset constant used to control the accuracy of reconstruction.So far, a single image reconstruction modelSk based on multiobjective optimization can be obtained as given in the following equation:(6)Sk=maxδl,minδr,minδm,s,t,hδ=0.To summarize, the reconstruction of a single image based on a multiobjective optimization algorithm [17] can get better reconstruction results and reduce the influence of noise and other interference factors in the original image on the later image defogging effect.


## 3. Single Image Enhancements

The image noise can be effectively suppressed by a single image reconstruction model constructed under the constraints of multiple objective functions. In order to further improve the image dehazing effect, color preservation effect and detail enhancement effect should also be considered [18]. As a result, image enhancement is realized, and the image visual effect is further adjusted starting from the two levels of color consistency and effective detail intensity.


### 3.1. Color Constancy Model

Color constancy is an adaptive characteristic of the human visual system (HVS), which means that the HVS can recognize the true color of an object under a wide range of different color lighting conditions. Under typical conditions, the human eye’s perception of the color of an object’s surface is closely tied to the object’s reflection qualities but has nothing to do with the spectral features that reach the human eye. That is, when the external lighting conditions change, the human eye’s color perception can still remain relatively unchanged to a certain extent, exhibiting color constancy, as shown in Figure1. In fact, the human eye can recognize the reflected color of the object surface even when the signals of different colors of the object surface to the retina are the same under different color lighting conditions. Imaging systems, unlike HVS, are unable to respond spectrally to changing lighting conditions, resulting in a color cast in the acquired image, or a shift in the entire image’s color range.Figure 1
Color constancy model.Under weather conditions such as fog and haze, the scattering of atmospheric particles leads to low contrast and color distortion in scene imaging. Therefore, the color constancy algorithm can be well applied to image dehazing. The nonphysical model method avoids solving the atmospheric scattering model by treating the ambient light component of the model as the illuminance component and estimating the reflection image independently of illuminance.


### 3.2. Effective Detail Intensity Model

The image after dehazing enhancement should have higher sharpness, which is manifested image with high edge dissimilarity. However, the dissimilarity of the image is not as high as possible, and the edge information after filtering out noise and the influence of the Halo effect exists. The Halo effect occurs at the edge of the object, especially at the sudden change of the depth of the field, which is manifested as widening and brightening at the edge of the object. The presence of the Halo effect has a significant impact on the dehazed image’s visual impression. According to the characteristics of Halo, the bright channel of the image is defined as(7)Ts=KGs+1⋅λs,where Gs represents the color channel of the image; λs represents the 7 × 7 small neighborhood-centered pixel point s.The Canny operator is used to detect the image’s edge, and the detected edge images are combined together to get the aggregate intensity detail of the image, which characterizes the intensity of the dehazed image after noise filtering, as shown in the following equation:(8)Zs=cnk−h∧z∂k,where cnk represents the correlation of noise scale; h∧z represents Gaussian noise; ∂k represents image signal-to-noise ratio.Since the dehazed image may have the Halo effect for a pixels′ that exists in the set of all edge pixels of the image. The summation is performed of the small neighborhood values of the corresponding pixel points in the bright channels of the image as given in the following equation:(9)θt=∑k=1Nhx,yetdt,where et the image luminance channel.According to equation (9), the brightness value of a single image is obtained to achieve image detail enhancement.


## 3.1. Color Constancy Model

Color constancy is an adaptive characteristic of the human visual system (HVS), which means that the HVS can recognize the true color of an object under a wide range of different color lighting conditions. Under typical conditions, the human eye’s perception of the color of an object’s surface is closely tied to the object’s reflection qualities but has nothing to do with the spectral features that reach the human eye. That is, when the external lighting conditions change, the human eye’s color perception can still remain relatively unchanged to a certain extent, exhibiting color constancy, as shown in Figure1. In fact, the human eye can recognize the reflected color of the object surface even when the signals of different colors of the object surface to the retina are the same under different color lighting conditions. Imaging systems, unlike HVS, are unable to respond spectrally to changing lighting conditions, resulting in a color cast in the acquired image, or a shift in the entire image’s color range.Figure 1
Color constancy model.Under weather conditions such as fog and haze, the scattering of atmospheric particles leads to low contrast and color distortion in scene imaging. Therefore, the color constancy algorithm can be well applied to image dehazing. The nonphysical model method avoids solving the atmospheric scattering model by treating the ambient light component of the model as the illuminance component and estimating the reflection image independently of illuminance.


## 3.2. Effective Detail Intensity Model

The image after dehazing enhancement should have higher sharpness, which is manifested image with high edge dissimilarity. However, the dissimilarity of the image is not as high as possible, and the edge information after filtering out noise and the influence of the Halo effect exists. The Halo effect occurs at the edge of the object, especially at the sudden change of the depth of the field, which is manifested as widening and brightening at the edge of the object. The presence of the Halo effect has a significant impact on the dehazed image’s visual impression. According to the characteristics of Halo, the bright channel of the image is defined as(7)Ts=KGs+1⋅λs,where Gs represents the color channel of the image; λs represents the 7 × 7 small neighborhood-centered pixel point s.The Canny operator is used to detect the image’s edge, and the detected edge images are combined together to get the aggregate intensity detail of the image, which characterizes the intensity of the dehazed image after noise filtering, as shown in the following equation:(8)Zs=cnk−h∧z∂k,where cnk represents the correlation of noise scale; h∧z represents Gaussian noise; ∂k represents image signal-to-noise ratio.Since the dehazed image may have the Halo effect for a pixels′ that exists in the set of all edge pixels of the image. The summation is performed of the small neighborhood values of the corresponding pixel points in the bright channels of the image as given in the following equation:(9)θt=∑k=1Nhx,yetdt,where et the image luminance channel.According to equation (9), the brightness value of a single image is obtained to achieve image detail enhancement.


## 4. Local Dehazing Algorithm for Single Image

According to the color constancy model and the effective detail intensity model discussed in Section3, the enhancement processing of the color and details of a single image is realized. Based on this, the local dehazing of a single image is studied.


### 4.1. Atmospheric Scattering Model

The reflected light of the target is absorbed and scattered by suspended particles in the environment in the presence of fog, haze, and other media. Simultaneously, external light, such as sunshine, is scattered by the scattering medium in the atmosphere, resulting in background light, making the camera’s brightness dim and fuzzy. The atmospheric scattering model, as shown in Figure2, is used to represent this physical process.Figure 2
Physical model of atmospheric scattering.According to Figure2, the atmospheric scattering physical model is represented as shown in the following equation:(10)Un=InhnFi1−wi,where Un is the observed hazy image; In is the natural haze-free image; Fi and wi represent the transmittance and atmospheric light values, respectively. Single image dehazing aims to recover a clear image from a hazy image. In this process, the atmospheric light value wi and transmittance Fi need to be estimated, and their expressions are given by the following equations:(11)wi=ln1−Δp×⋅pi,(12)Fi=x+x′Dt×θi,where pi is the average brightness of the sky area; Δp is the variation coefficient of the average brightness of the sky area; Dt is the Halo phenomenon generated after transmission.Assuming that the transmittance has local area consistency, equation (12) is improved, and the new transmittance expression can be obtained from the following equation:(13)Fi′=∑i=1Nxit×ht×θi,where ht is the local atmospheric light; xit is the initial transmission map.In reality, even in sunny weather, the atmosphere cannot contain any particles. When viewing distant objects, fog still exists, that is, there is a phenomenon of spatial perspective. If removing the fog completely will make the image look unreal and the image depth will be lost, therefore, a constantϕ is introduced in equation (13), 0<ϕ<1. Equation (13) can be further rewritten as the following equation:(14)Fi′=∫t0thtdt×θi.The advantage of this correction is that part of the fog covering distant objects can be retained, making the restored image more realistic and more in line with the characteristics of the human eye. The value ofϕ can be changed according to the actual situation. In this paper, the fixed value of ϕ is 0.95.


### 4.2. Improved Transmittance Optimization Based on Guided Filter

In the study of local dehazing of a single image, the ultimate goal is to remove the fog in the foggy image, so it can be regarded as a noise source. To remove the noise in the image, an appropriate image filter must be used in conjunction with the objective function. The median filter [13], mean filter [14], Gaussian filter [15], bilateral filter [16], and other image filters are common. These filters are widely used in image dehazing. In this paper, a new display image filter, namely image-guided filter, is selected in image dehazing. A local linear model is used to create the guiding filter. It works on the premise of limiting the filtered output image to the composition of a predefined guiding image. The guiding filter is similar to the bilateral filter and can maintain the edge information of the image. Moreover, because the guidance filter is more prominent in maintaining the edge information than the bilateral filter, its time complexity is low. It can increase the efficacy of image defogging.For the guiding imageq1 and the final output image q2, the guiding filter must finally have a local linear relationship, that is, one of the guiding images q1, pixel s obtains the final output image q2 through linear transformation in its domain ψk, which can be expressed as shown in the following equation:(15)q2=ψkbk+λk,where bk and λk both represent linear coefficients. Obviously, equation (15) is a local linear model. According to this local linear model, it can well explain the good image edge retention of the guided filter, that is, for the edge of the object in the image. Because the gray level on both sides of the edge of the image object changes greatly, the gradient operator is usually used to detect the edge of the image, and the local linear model can be obtained according to the following equation:(16)∇q2=∇q1×et.Equation (16) indicates that as long as the edge is detected in the guide image q1, the final output image q2 after the guide filter processing must have the corresponding edge in the guide image q1.In order to determine the linear coefficientsbk and λk in this local linear model, the input filtered image must be restricted, and the final output image q2 is represented by subtracting some unnecessary content (noise) from the input image and is as given in the following equation:(17)q2=q1−h∧z.Solve the cost function in a local fieldψk. The purpose of the cost function is to minimize the variance between the filtered input image q1 and the final output image q2 as much as possible. It is also necessary to maintain the local linear relationship between the guide image q1 and the final output image q2. Its expression is given in the following equation:(18)βq1,q2=D2×∑i=0Nq1t×ψkD2+∑i=0Nq2t×ψk.In equation (18), D2 is the regularization parameter used to constrain ψk. Since the guided filter is a linear model, the best solution to the cost function obtained by using the linear model is to use univariate linear regression [17], and the final linear coefficient is given by the following equations:(19)bk=ψk⋅βq1,q2×ϖk,(20)λk=P×q1t+q2tPq1,q2×τk+φ2,where ϖk and τk are the expectation and variance of all pixels of the guide image q1 in the neighborhood ψk, respectively; φ2 represents the expectation of the input image in the neighborhood ψk; P represents the total number of pixels in the neighborhood ψk of the guide image q1.Since the linear coefficientsbk and λk in the guided filter expression are obtained, the output of the guided filter can be calculated, but a pixel point s is included by multiple ψk neighborhoods, and different neighborhoods make the final output different. So here the expectation is calculated for all the output results of the ψk neighborhood containing the pixel point s, and its expression is shown in the following equation:(21)ψks=argminbk¯+λk¯2,where bk¯ and λk¯, respectively, represent the average linear coefficient of all the pixels s in the neighborhood of ψk.


### 4.3. Local Dehazing of Single Image Based on Dark Channel Priority

The dark channel priority dehazing method [18] is a dehazing method for outdoor natural sceneries that is based on the dark channel priority law. The dark channel priority law considers that most of the fog-free outdoor natural scene images are processed by the dark channel priority. The brightness of the pixels will be close to zero, and if there are a large number of bright pixels in the dark channel image, this brightness should come from the fog in the air or the sky [19, 20]. For the fogged original image, the initial transmission map xit and air color value Ct will be able to be obtained from the result of the dark channel prioritization. The higher the brightness in the transmission map, the better the passability of the scene color, which can also be understood as the closer to the viewpoint. Because the block calculation method is used in the dark channel processing, the initial transmission image has obvious squares, and at the same time, it cannot well conform to the geometric edge of the original image. Therefore, the dark channel priority method turns the process of transmission image optimization into the smallest one. The process of transforming the value function is shown in the following equation:(22)ϑx=logxit×Ct1−ρ2.For the optimization of this quadratic function, it can be converted into the solution of a linear system of equations, and the optimized transmission mapxi′t can be obtained by the following equation:(23)xi′t=expOi−L2.Combined with the transmittance optimization results in Section4.2, the transmittance optimization of a single image is realized. So far, the design of the local dehazing algorithm for a single image is completed.


## 4.1. Atmospheric Scattering Model

The reflected light of the target is absorbed and scattered by suspended particles in the environment in the presence of fog, haze, and other media. Simultaneously, external light, such as sunshine, is scattered by the scattering medium in the atmosphere, resulting in background light, making the camera’s brightness dim and fuzzy. The atmospheric scattering model, as shown in Figure2, is used to represent this physical process.Figure 2
Physical model of atmospheric scattering.According to Figure2, the atmospheric scattering physical model is represented as shown in the following equation:(10)Un=InhnFi1−wi,where Un is the observed hazy image; In is the natural haze-free image; Fi and wi represent the transmittance and atmospheric light values, respectively. Single image dehazing aims to recover a clear image from a hazy image. In this process, the atmospheric light value wi and transmittance Fi need to be estimated, and their expressions are given by the following equations:(11)wi=ln1−Δp×⋅pi,(12)Fi=x+x′Dt×θi,where pi is the average brightness of the sky area; Δp is the variation coefficient of the average brightness of the sky area; Dt is the Halo phenomenon generated after transmission.Assuming that the transmittance has local area consistency, equation (12) is improved, and the new transmittance expression can be obtained from the following equation:(13)Fi′=∑i=1Nxit×ht×θi,where ht is the local atmospheric light; xit is the initial transmission map.In reality, even in sunny weather, the atmosphere cannot contain any particles. When viewing distant objects, fog still exists, that is, there is a phenomenon of spatial perspective. If removing the fog completely will make the image look unreal and the image depth will be lost, therefore, a constantϕ is introduced in equation (13), 0<ϕ<1. Equation (13) can be further rewritten as the following equation:(14)Fi′=∫t0thtdt×θi.The advantage of this correction is that part of the fog covering distant objects can be retained, making the restored image more realistic and more in line with the characteristics of the human eye. The value ofϕ can be changed according to the actual situation. In this paper, the fixed value of ϕ is 0.95.


## 4.2. Improved Transmittance Optimization Based on Guided Filter

In the study of local dehazing of a single image, the ultimate goal is to remove the fog in the foggy image, so it can be regarded as a noise source. To remove the noise in the image, an appropriate image filter must be used in conjunction with the objective function. The median filter [13], mean filter [14], Gaussian filter [15], bilateral filter [16], and other image filters are common. These filters are widely used in image dehazing. In this paper, a new display image filter, namely image-guided filter, is selected in image dehazing. A local linear model is used to create the guiding filter. It works on the premise of limiting the filtered output image to the composition of a predefined guiding image. The guiding filter is similar to the bilateral filter and can maintain the edge information of the image. Moreover, because the guidance filter is more prominent in maintaining the edge information than the bilateral filter, its time complexity is low. It can increase the efficacy of image defogging.For the guiding imageq1 and the final output image q2, the guiding filter must finally have a local linear relationship, that is, one of the guiding images q1, pixel s obtains the final output image q2 through linear transformation in its domain ψk, which can be expressed as shown in the following equation:(15)q2=ψkbk+λk,where bk and λk both represent linear coefficients. Obviously, equation (15) is a local linear model. According to this local linear model, it can well explain the good image edge retention of the guided filter, that is, for the edge of the object in the image. Because the gray level on both sides of the edge of the image object changes greatly, the gradient operator is usually used to detect the edge of the image, and the local linear model can be obtained according to the following equation:(16)∇q2=∇q1×et.Equation (16) indicates that as long as the edge is detected in the guide image q1, the final output image q2 after the guide filter processing must have the corresponding edge in the guide image q1.In order to determine the linear coefficientsbk and λk in this local linear model, the input filtered image must be restricted, and the final output image q2 is represented by subtracting some unnecessary content (noise) from the input image and is as given in the following equation:(17)q2=q1−h∧z.Solve the cost function in a local fieldψk. The purpose of the cost function is to minimize the variance between the filtered input image q1 and the final output image q2 as much as possible. It is also necessary to maintain the local linear relationship between the guide image q1 and the final output image q2. Its expression is given in the following equation:(18)βq1,q2=D2×∑i=0Nq1t×ψkD2+∑i=0Nq2t×ψk.In equation (18), D2 is the regularization parameter used to constrain ψk. Since the guided filter is a linear model, the best solution to the cost function obtained by using the linear model is to use univariate linear regression [17], and the final linear coefficient is given by the following equations:(19)bk=ψk⋅βq1,q2×ϖk,(20)λk=P×q1t+q2tPq1,q2×τk+φ2,where ϖk and τk are the expectation and variance of all pixels of the guide image q1 in the neighborhood ψk, respectively; φ2 represents the expectation of the input image in the neighborhood ψk; P represents the total number of pixels in the neighborhood ψk of the guide image q1.Since the linear coefficientsbk and λk in the guided filter expression are obtained, the output of the guided filter can be calculated, but a pixel point s is included by multiple ψk neighborhoods, and different neighborhoods make the final output different. So here the expectation is calculated for all the output results of the ψk neighborhood containing the pixel point s, and its expression is shown in the following equation:(21)ψks=argminbk¯+λk¯2,where bk¯ and λk¯, respectively, represent the average linear coefficient of all the pixels s in the neighborhood of ψk.


## 4.3. Local Dehazing of Single Image Based on Dark Channel Priority

The dark channel priority dehazing method [18] is a dehazing method for outdoor natural sceneries that is based on the dark channel priority law. The dark channel priority law considers that most of the fog-free outdoor natural scene images are processed by the dark channel priority. The brightness of the pixels will be close to zero, and if there are a large number of bright pixels in the dark channel image, this brightness should come from the fog in the air or the sky [19, 20]. For the fogged original image, the initial transmission map xit and air color value Ct will be able to be obtained from the result of the dark channel prioritization. The higher the brightness in the transmission map, the better the passability of the scene color, which can also be understood as the closer to the viewpoint. Because the block calculation method is used in the dark channel processing, the initial transmission image has obvious squares, and at the same time, it cannot well conform to the geometric edge of the original image. Therefore, the dark channel priority method turns the process of transmission image optimization into the smallest one. The process of transforming the value function is shown in the following equation:(22)ϑx=logxit×Ct1−ρ2.For the optimization of this quadratic function, it can be converted into the solution of a linear system of equations, and the optimized transmission mapxi′t can be obtained by the following equation:(23)xi′t=expOi−L2.Combined with the transmittance optimization results in Section4.2, the transmittance optimization of a single image is realized. So far, the design of the local dehazing algorithm for a single image is completed.


## 5. Experimental Results and Analysis

The experimental investigation is carried out to validate the effectiveness of a single image local defogging technique based on a multiobjective optimization algorithm. This experiment is implemented on an ordinary PC with windows 10 operating system, core 2 Dual Core 2.8 GHz CPU, and 8 GB system memory. The suggested approach is compared against a single image defogging algorithm based on deep learning and an infrared image defogging algorithm based on the atmospheric scattering model and retinex.


### 5.1. Experimental Sample Set

The experimental samples from the RESIDE dataset are considered. It contains both indoor (ITS) and outdoor (OTS) synthetic training sets which are not labeled real hazy images (URHI). To create the training set, 6000 synthetic hazy images were randomly selected as labeled data, among which 3000 images from ITS, 3000 images from OTS, and 4000 images from URHI were collected. RESIDE dataset contains a test subset namely a synthetic test set (SOTS). It has 500 pairs of fog and sharp images for indoor and outdoor sets. To demonstrate the usefulness of the suggested approach, it is tested on the RESIDE dataset.


### 5.2. Evaluation Indicators

The image dehazing effect is analyzed from two aspects: subjective evaluation and objective evaluation. The objective evaluation index includes information entropy, CNI, and the degree of tone restoration, and the subjective evaluation index refers to the visual effect of image dehazing.(1)
Information entropy: it signifies the image’s information average value. The more the information entropy, the more clear is the image and has better quality.(2)
CNI is an indicator that is often used to assess the reliability and naturalness of an image. It is mainly used to judge the dehazing image, and the range is 0-1. The image nature is better when the CNI value is close to 1.(3)
The degree of tone restoration after the image is dehazed is represented by the histogram.


### 5.3. Objective Quality Evaluation

#### 5.3.1. Information Entropy

The image dehazing effects of the proposed approach, the infrared image dehazing algorithm based on the atmospheric scattering model and retinex, and the single image dehazing technique based on deep learning are compared using the information entropy as the evaluation index. The results are shown in Table1.Table 1
Information entropy comparison results.
Image typeThe proposed algorithmAtmospheric scattering models and retinexDeep learningIndoor image 17.536.876.75Indoor image 27.296.736.94Indoor image 37.346.257.01Indoor image 47.126.346.86Outdoor image 17.075.966.31Outdoor image 26.935.526.07Outdoor image 36.995.215.91Outdoor image 46.855.306.03In aspects of image defogging objective quality, Table1 compares the four defogging algorithms’ image defogging effects in terms of objective quality. Table 1 shows that the suggested technique has a higher information entropy than the infrared image defogging algorithm based on the atmospheric scattering model and retinex model. The single image defogging algorithm based on depth learning shows that the proposed algorithm is relative to the two traditional algorithms. The defogging result has more information and clearer image. Both indoor and outdoor images have a good defogging effect.


#### 5.3.2. CNI

The proposed algorithm, infrared image defogging algorithm based on atmospheric scattering model, retinex, and single image defogging algorithm based on depth learning are compared using CNI as the evaluation index. The results are shown in Table2.Table 2
CNI comparison results.
Image typeThe proposed algorithmAtmospheric scattering models and retinexDeep learningIndoor image 10.950.870.82Indoor image 20.970.830.79Indoor image 30.900.890.83Indoor image 40.890.790.75Outdoor image 10.880.720.65Outdoor image 20.860.750.70Outdoor image 30.820.690.67Outdoor image 40.750.650.62The proposed algorithm has a higher CNI value than the infrared image defogging algorithm based on atmospheric scattering model and retinex, as well as the single image defogging technique based on depth learning, as shown in Table2. The highest value is 0.97, and the CNI value is greater than 0.80. The CNI value of the two traditional algorithms is low; especially, the CNI value of deep learning algorithm has a lot of room to improve. According to the above analysis, it can be concluded that the fidelity and naturalness of the image after defogging with the proposed algorithm are better, and the image is more natural.


#### 5.3.3. Hue Reduction Degree

The image defogging effects of the proposed algorithm, the infrared image defogging algorithm based on atmospheric scattering model, retinex, and the single image defogging technique based on depth learning are compared using the hue restoration degree as the evaluation index.The image histogram shifts to the right as a result of the fog effect on the image. An effective image defogging algorithm should return the image to its original appearance. In other words, the shape of the histogram of the original image and the defogging image should be generally consistent. Figure3 shows the histogram of the original foggy image, the defogging image of the atmospheric scattering model and retinex algorithm, the defogging image of the depth learning algorithm, and the color component of the defogging image of the proposed algorithm.Figure 3
Comparison results of hue reduction degree. (a) Original image histogram. (b) Histogram of proposed algorithm. (c) Histogram of atmospheric and retinex Model. (d) Histogram of deep learning algorithm.
(a)(b)(c)(d)Figure3 shows that, when compared to the histograms in Figures 3(c) and 3(d), the histogram in Figure 3(b) maintains the shape of the original image histogram better. As a result, it can be stated that after defogging using the proposed methodology, the image tone restoration degree is higher, which has substantial benefits over the traditional algorithm.


### 5.4. Subjective Quality Evaluation

The above experiments have evaluated the image defogging effects of the three algorithms from an objective point of view. The image defogging effects of the proposed technique, the infrared image defogging algorithm based on atmospheric scattering model and retinex, and the single image defogging technique based on deep learning will next be compared from a subjective point of view. Figure4 shows the effect of image defogging.Figure 4
Comparison results of image defogging effect. (a) Defogging effect of indoor foggy image. (b) Defogging effect of outdoor foggy image.
(a)(b)By analyzing the defogging results of the image in Figure4, it can be found that the proposed algorithm has improved the image color, contrast, and clarity and maintained the real color of the image. Image saturation and color distortion are issues with the processing outputs of an infrared image defogging technique based on an atmospheric scattering model and retinex. The processing results of a single deep learning-based image defogging system exhibit issues with image detail loss and low image definition. By comprehensively analyzing the subjective and objective evaluation results of the above defogging algorithm, the effectiveness of the proposed algorithm is obtained. The defogging results are more in line with the visual characteristics of human eyes.


## 5.1. Experimental Sample Set

The experimental samples from the RESIDE dataset are considered. It contains both indoor (ITS) and outdoor (OTS) synthetic training sets which are not labeled real hazy images (URHI). To create the training set, 6000 synthetic hazy images were randomly selected as labeled data, among which 3000 images from ITS, 3000 images from OTS, and 4000 images from URHI were collected. RESIDE dataset contains a test subset namely a synthetic test set (SOTS). It has 500 pairs of fog and sharp images for indoor and outdoor sets. To demonstrate the usefulness of the suggested approach, it is tested on the RESIDE dataset.


## 5.2. Evaluation Indicators

The image dehazing effect is analyzed from two aspects: subjective evaluation and objective evaluation. The objective evaluation index includes information entropy, CNI, and the degree of tone restoration, and the subjective evaluation index refers to the visual effect of image dehazing.(1)
Information entropy: it signifies the image’s information average value. The more the information entropy, the more clear is the image and has better quality.(2)
CNI is an indicator that is often used to assess the reliability and naturalness of an image. It is mainly used to judge the dehazing image, and the range is 0-1. The image nature is better when the CNI value is close to 1.(3)
The degree of tone restoration after the image is dehazed is represented by the histogram.


## 5.3. Objective Quality Evaluation

### 5.3.1. Information Entropy

The image dehazing effects of the proposed approach, the infrared image dehazing algorithm based on the atmospheric scattering model and retinex, and the single image dehazing technique based on deep learning are compared using the information entropy as the evaluation index. The results are shown in Table1.Table 1
Information entropy comparison results.
Image typeThe proposed algorithmAtmospheric scattering models and retinexDeep learningIndoor image 17.536.876.75Indoor image 27.296.736.94Indoor image 37.346.257.01Indoor image 47.126.346.86Outdoor image 17.075.966.31Outdoor image 26.935.526.07Outdoor image 36.995.215.91Outdoor image 46.855.306.03In aspects of image defogging objective quality, Table1 compares the four defogging algorithms’ image defogging effects in terms of objective quality. Table 1 shows that the suggested technique has a higher information entropy than the infrared image defogging algorithm based on the atmospheric scattering model and retinex model. The single image defogging algorithm based on depth learning shows that the proposed algorithm is relative to the two traditional algorithms. The defogging result has more information and clearer image. Both indoor and outdoor images have a good defogging effect.


### 5.3.2. CNI

The proposed algorithm, infrared image defogging algorithm based on atmospheric scattering model, retinex, and single image defogging algorithm based on depth learning are compared using CNI as the evaluation index. The results are shown in Table2.Table 2
CNI comparison results.
Image typeThe proposed algorithmAtmospheric scattering models and retinexDeep learningIndoor image 10.950.870.82Indoor image 20.970.830.79Indoor image 30.900.890.83Indoor image 40.890.790.75Outdoor image 10.880.720.65Outdoor image 20.860.750.70Outdoor image 30.820.690.67Outdoor image 40.750.650.62The proposed algorithm has a higher CNI value than the infrared image defogging algorithm based on atmospheric scattering model and retinex, as well as the single image defogging technique based on depth learning, as shown in Table2. The highest value is 0.97, and the CNI value is greater than 0.80. The CNI value of the two traditional algorithms is low; especially, the CNI value of deep learning algorithm has a lot of room to improve. According to the above analysis, it can be concluded that the fidelity and naturalness of the image after defogging with the proposed algorithm are better, and the image is more natural.


### 5.3.3. Hue Reduction Degree

The image defogging effects of the proposed algorithm, the infrared image defogging algorithm based on atmospheric scattering model, retinex, and the single image defogging technique based on depth learning are compared using the hue restoration degree as the evaluation index.The image histogram shifts to the right as a result of the fog effect on the image. An effective image defogging algorithm should return the image to its original appearance. In other words, the shape of the histogram of the original image and the defogging image should be generally consistent. Figure3 shows the histogram of the original foggy image, the defogging image of the atmospheric scattering model and retinex algorithm, the defogging image of the depth learning algorithm, and the color component of the defogging image of the proposed algorithm.Figure 3
Comparison results of hue reduction degree. (a) Original image histogram. (b) Histogram of proposed algorithm. (c) Histogram of atmospheric and retinex Model. (d) Histogram of deep learning algorithm.
(a)(b)(c)(d)Figure3 shows that, when compared to the histograms in Figures 3(c) and 3(d), the histogram in Figure 3(b) maintains the shape of the original image histogram better. As a result, it can be stated that after defogging using the proposed methodology, the image tone restoration degree is higher, which has substantial benefits over the traditional algorithm.


## 5.3.1. Information Entropy

The image dehazing effects of the proposed approach, the infrared image dehazing algorithm based on the atmospheric scattering model and retinex, and the single image dehazing technique based on deep learning are compared using the information entropy as the evaluation index. The results are shown in Table1.Table 1
Information entropy comparison results.
Image typeThe proposed algorithmAtmospheric scattering models and retinexDeep learningIndoor image 17.536.876.75Indoor image 27.296.736.94Indoor image 37.346.257.01Indoor image 47.126.346.86Outdoor image 17.075.966.31Outdoor image 26.935.526.07Outdoor image 36.995.215.91Outdoor image 46.855.306.03In aspects of image defogging objective quality, Table1 compares the four defogging algorithms’ image defogging effects in terms of objective quality. Table 1 shows that the suggested technique has a higher information entropy than the infrared image defogging algorithm based on the atmospheric scattering model and retinex model. The single image defogging algorithm based on depth learning shows that the proposed algorithm is relative to the two traditional algorithms. The defogging result has more information and clearer image. Both indoor and outdoor images have a good defogging effect.


## 5.3.2. CNI

The proposed algorithm, infrared image defogging algorithm based on atmospheric scattering model, retinex, and single image defogging algorithm based on depth learning are compared using CNI as the evaluation index. The results are shown in Table2.Table 2
CNI comparison results.
Image typeThe proposed algorithmAtmospheric scattering models and retinexDeep learningIndoor image 10.950.870.82Indoor image 20.970.830.79Indoor image 30.900.890.83Indoor image 40.890.790.75Outdoor image 10.880.720.65Outdoor image 20.860.750.70Outdoor image 30.820.690.67Outdoor image 40.750.650.62The proposed algorithm has a higher CNI value than the infrared image defogging algorithm based on atmospheric scattering model and retinex, as well as the single image defogging technique based on depth learning, as shown in Table2. The highest value is 0.97, and the CNI value is greater than 0.80. The CNI value of the two traditional algorithms is low; especially, the CNI value of deep learning algorithm has a lot of room to improve. According to the above analysis, it can be concluded that the fidelity and naturalness of the image after defogging with the proposed algorithm are better, and the image is more natural.


## 5.3.3. Hue Reduction Degree

The image defogging effects of the proposed algorithm, the infrared image defogging algorithm based on atmospheric scattering model, retinex, and the single image defogging technique based on depth learning are compared using the hue restoration degree as the evaluation index.The image histogram shifts to the right as a result of the fog effect on the image. An effective image defogging algorithm should return the image to its original appearance. In other words, the shape of the histogram of the original image and the defogging image should be generally consistent. Figure3 shows the histogram of the original foggy image, the defogging image of the atmospheric scattering model and retinex algorithm, the defogging image of the depth learning algorithm, and the color component of the defogging image of the proposed algorithm.Figure 3
Comparison results of hue reduction degree. (a) Original image histogram. (b) Histogram of proposed algorithm. (c) Histogram of atmospheric and retinex Model. (d) Histogram of deep learning algorithm.
(a)(b)(c)(d)Figure3 shows that, when compared to the histograms in Figures 3(c) and 3(d), the histogram in Figure 3(b) maintains the shape of the original image histogram better. As a result, it can be stated that after defogging using the proposed methodology, the image tone restoration degree is higher, which has substantial benefits over the traditional algorithm.


## 5.4. Subjective Quality Evaluation

The above experiments have evaluated the image defogging effects of the three algorithms from an objective point of view. The image defogging effects of the proposed technique, the infrared image defogging algorithm based on atmospheric scattering model and retinex, and the single image defogging technique based on deep learning will next be compared from a subjective point of view. Figure4 shows the effect of image defogging.Figure 4
Comparison results of image defogging effect. (a) Defogging effect of indoor foggy image. (b) Defogging effect of outdoor foggy image.
(a)(b)By analyzing the defogging results of the image in Figure4, it can be found that the proposed algorithm has improved the image color, contrast, and clarity and maintained the real color of the image. Image saturation and color distortion are issues with the processing outputs of an infrared image defogging technique based on an atmospheric scattering model and retinex. The processing results of a single deep learning-based image defogging system exhibit issues with image detail loss and low image definition. By comprehensively analyzing the subjective and objective evaluation results of the above defogging algorithm, the effectiveness of the proposed algorithm is obtained. The defogging results are more in line with the visual characteristics of human eyes.


## 6. Conclusion

In this paper, a single image local defogging algorithm based on multiobjective optimization is proposed to improve the visual impacts of the image for better interpretations. The image reconstruction is realized by multiobjective optimization to consider multiple factors for improving the visual impacts of the images. By estimating the atmospheric light value, and improved transmittance, the transmittance of a single image is optimized. The dark channel priority method is also used to realize the local defogging of a single image. When compared to existing state-of-the-art methods, a significant number of experimental results suggest that the proposed technique has good defogging performance. The proposed method can process normal and foggy images as well.



---

*Source: 7200657-2022-07-30.xml*