# Partial Differential Equations-Based Iterative Denoising Algorithm for Movie Images

Authors: Pingli Sun; Chenxia Wang; Min Li; Lanqi Liu
Journal: Advances in Mathematical Physics
 (2021)

---

## Abstract

Film video noise can usually be defined as the error information visible on the video image, caused by the digital signal system. This distortion is inevitably present in the video obtained by various camera equipment. Noise reduction techniques are important preprocessing processes in many video processing applications, and its main goal is to reduce the noise contained in a video image while preserving as much of its edge and texture information as possible. In this paper, we describe in detail the principles of the space-time noise reduction filter, propose a 3D-filter algorithm for Gaussian noise, an improved 3D-filter algorithm based on the 3D-BDP (bloom-deep-split) filter for mixed noise, and a filter algorithm for luminance and color noise in low-brightness scenes. By dissecting the partial differential equation (PDE) denoising process, we establish a new iterative denoising algorithm. The partial differential equation method can be considered as the iterative denoising of the filter, and the first stage of the new algorithm uses wavelet-domain adaptive Wiener filter as the filtering base and achieves good results by adjusting the parameters. The proposed model in this paper is compared with the existing denoising model, and the analysis results show that the model proposed in this section can effectively remove multiplicative noise. The experimental report shows that the parameters set by the algorithm have some stability and can achieve good processing results for multiple images, which is an advantage over the partial differential equation method for denoising. The second stage of the algorithm uses the appropriate partial differential equation method to remove the pseudo-Gibbs in the first stage, which further improves the performance of the algorithm. After the image containing Gaussian noise is processed by the new algorithm, the pseudo-Gibbs effect, which often occurs in wavelet denoising, is eliminated, and the step effect, which occurs in partial differential equation denoising, is avoided; the details are better preserved, and the peak signal-to-noise ratio is improved, and a large number of experiments show that it is an effective denoising method.


---

## Body

## 1. Introduction

There are two ways to shoot movies: film shooting and digital camera. Digital cameras use CCD as the sensor, but a small percentage use CMOS, and the raw movie video is stored in digital format on tape, DVD, or hard disk. Digital cameras represent a trend in film shooting and production [1]. Linear CCD usually divides the internal electrodes of the CCD into groups, each group is called a phase, and the same clock pulse is applied. The number of phases required is determined by the internal structure of the CCD chip, and CCDs with different structures can meet the requirements of different occasions. Linear CCD is divided into single channel and double channel. It connects a photosensitive area array and a shift register scanning circuit. It is characterized by fast information processing speed, simple peripheral circuits, and easy real-time control. These films are stored digitally from the beginning and are less susceptible to external distortion, but on the other hand, the digital camera equipment used for digital cinematography is still in the development stage and inevitably introduces noise distortion during the shooting process [2]. Video noise can usually be defined as the error information visible on the video image, caused by the digital signal system. This distortion inevitably occurs in the video obtained by various camera equipment. For example, Gaussian noise easily appears in movies shot by digital cameras, pretzel noise easily appears in TV video reception, and grain noise and speckle noise easily appear in old movies playback [3]. The above noise level is usually expressed in terms of signal-to-noise ratio (SNR), and if the SNR is below a certain level, the noise gradually becomes visible in the shape of particles and even obscures the detailed information of the image in high-speed video, resulting in the degradation of image quality, and increases the entropy of the image, thus hindering the effective compression of the video, and the more typical noise is white noise and impulse noise, etc. [4].Noise reduction is an important preprocessing process in many video processing applications. Its main goal is to reduce the noise contained in the video image while preserving as much edge and texture information as possible. Typically, the classical noise reduction method is a linear filtering approach. Linear filtering can be used to smooth and denoise the image in the spatial and frequency domains, such as local averaging, multiframe averaging, low-pass filter, and Gaussian filter [5]. They act as constant coefficient templates for convolution in the spatial domain and can also be analyzed in the frequency domain using Fourier transform and wavelet transform methods. However, although these methods are effective in reducing noise in images, they lose much of the image edge and texture information in the denoising process because the edges and noise are in the high frequency part of the filter. These edge and texture information is very important for video images. The linear filtering method is difficult to treat edges and noise differently [6]. To overcome this drawback of linear filtering noise reduction, nonlinear space-time filtering noise reduction methods are often used to restore image sequences [7]. The research and development of nonlinear filters plays a very important role in promoting the progress of modern mathematics, modern signal processing, and communication. Nonlinear signal processing not only greatly promotes the classical linear signal processing theory, but also can better reflect the various characteristics of the natural world we live in [8]. At the same time, the interpenetration of nonlinear signal processing theory with other disciplines also promotes the development of corresponding interdisciplinary and fringe disciplines. Film filtering is to suppress the noise of the target film-image while preserving the details of the image as much as possible. It is an indispensable operation in filming preprocessing. The quality of its processing effect will directly affect the effectiveness and analysis of subsequent image processing and analysis and reliability. Nonlinear space-time noise processing technology is one of the important elements of nonlinear signal processing. It is mainly used in the smooth noise reduction and recovery of coding effect of digital video [9]. The most significant feature of spatiotemporal noise reduction filter is that it is itself a nonlinear three-dimensional signal processing process, which can reduce the spatial noise of each frame while taking advantage of the temporal information of the video sequence and effectively preserving the edge and texture information of the image [10]. The video processing based on spatiotemporal filtering will be targeted to repair the noise according to its specific characteristics, thus helping to solve some key problems in the field of digital TV and film picture quality improvement and effectively improve the performance of the signal system.In this paper, we study the image denoising method based on partial differential equation and the image coloring method based on variational differentiation. Image denoising is the most basic work in image processing. Noise largely blurs the detailed information of an image and makes the analysis and application of the image difficult. The partial differential equation-based image denoising method can remove the noise while protecting the edges so that the detailed information of the image is not destroyed, and this method is of great interest to scholars. Image coloring is the colorization of grayscale images, which can increase the visual effect of images and facilitate in-depth research and analysis of images. The traditional image coloring methods are color transfer-based coloring method and color diffusion-based coloring method. In addition, in order to solve the coupling problem between multiple channels in the image coloring process and reduce the color crossing phenomenon, the paper proposes an image coloring model based on natural vector full variation, which supports only a common edge direction for all channels and can better preserve the color edges of the image. In addition, the paper gives the existence of the solution of the proposed model, solves the proposed model using the primal pairwise algorithm, and also illustrates the convergence of the algorithm. Finally, it is shown experimentally that the proposed model is effective in preserving image edges and reducing color crossing for both structured and textured images.


## 2. Related Work

There are various ways for human beings to obtain information, such as hearing, seeing, and smelling, among which vision is an important way to obtain information [11]. As an important carrier of information perceived by vision, images occupy an important position in people’s life. In addition to expressing and conveying some information, some images are also appreciated and have great artistic value. In short, images occupy an important position in modern society. The processing of images has always been a concern of scholars, and at present, computers have been used to achieve a variety of processing of images such as denoising, coloring, segmentation, and compression [12]. With the continuous development of computers, artificial intelligence can dig out information that cannot be found by traditional methods through continuous learning or accomplish tasks that are difficult to be done in large quantities by traditional methods and obtain images that are more effective than traditional methods. Image processing technology is widely used in medicine, satellite image processing, automobile obstacle recognition, feature recognition, etc. and has greatly contributed to the development of science and technology. Before the actual processing of images, image denoising is required [13]. The presence of noise will affect the subsequent image processing results, such as the inability to obtain accurate edge information in image edge extraction and reduce the recognition ability of the model in image recognition, so image denoising is highly researchable [14]. Noise in images comes from the process of image generation and transmission, which is manifested as random and discrete pixel points in images. Image denoising can bring people clearer and higher-quality images.The research on image denoising has been developed to date with fruitful results [15]. This includes methods based on transform domain and spatial domain, vector-based diffusion, and tensor-based diffusion. In recent years, with the development of deep learning, image denoising has also shifted from traditional methods to deep learning, neural network-based methods. Nowadays, there are still many scholars working on the image denoising problem. Denoising methods based on image transform domain include wavelet transform and Fourier transform. Denoising methods are based on image space domain such as Wiener filter and adaptive median filter [16]. These methods can achieve the denoising effect, but at the same time, they will damage some edge information in the image and cannot meet people’s needs. Researchers creatively used partial differential equations (PDEs) to study image smoothing and enhancement, which opened up PDE-based image processing. Later, researchers developed a theoretical basis for PDEs to be used in image denoising [17]. By the 1990s, the PDE-based image processing method gradually became one of the popular methods in the field of image processing. At present, the method has achieved fruitful results.The earliest partial differential equation applied to image denoising is the heat conduction equation, which damages the image edges to some extent while denoising. Based on the heat conduction equation, researchers proposed an anisotropic diffusion model (i.e., the P-M model), which can preserve image boundaries well, but since the model is pathological, the uniqueness of the solution cannot be guaranteed. The researchers proposed the more stable regularized P-M model, which is an optimization of the P-M model. The researchers proposed a curvature change-based model that also achieved some denoising effect [18]. The researchers proposed a new denoising model based on the P-M equation, which is a second-order PDE model and its improvement model, which generates “step effect” in denoising, while the higher-order PDE model proposed by the researchers can effectively solve the problem. The researchers proposed a fourth-order PDE model (Y-K model) to eliminate the “step effect,” but the speckle phenomenon also appeared. The researchers modified the diffusion coefficient of the Y-K model, and the denoising effect was improved compared with the Y-K model.Researchers proposed a two-step algorithm for removing pretzel noise. The researchers proposed a denoising method for Poisson noise by maximum a posteriori estimation (MAP). In addition, many scholars have proposed fourth-order PDE models, fractional-order PDE models, and PDE-based “hybrid” denoising models, all of which have achieved certain denoising effects. Researchers have proposed a full variational- (TV-) based denoising model, which preserves the image edge information while denoising [19]. Based on this idea, various adaptive TV models have been proposed. Researchers have proposed a variational model that can effectively remove the step effect by smoothly approximating the function with higher-order derivatives. The full variational-based denoising models have been widely studied, and all of them have achieved the desired denoising effect in denoising. Besides, there are also denoising methods based on vector-based diffusion and tensor-based diffusion. The tensor diffusion model is proposed based on the model diffusing faster and slower in two orthogonal directions to achieve the denoising while preserving the edge features. The researchers proposed a complex diffusion model and a forward-backward diffusion model, both of which have satisfactory denoising effects. The researchers improved the structure tensor-based diffusion model, and the denoising effect is more satisfactory. In addition to the aforementioned denoising of grayscale images, the study of color image denoising is also of great importance. Most of the denoising methods and results for scalar images can be extended to vector-valued images, but it is not simple to apply the methods to each channel of the image independently. For the vector-valued image denoising problem, the definition of its vector TV can be divided into two categories: one is to calculate the TV channel-by-channel and then take the appropriate parametrization; the other is to use the Riemannian geometry approach.


## 3. Iterative Image Denoising Based on Partial Differential Equations

### 3.1. Noise Characteristic Analysis

Here, in order to settle the image denoising, we first to introduce the characteristics of the noise existing in the film-imaging, which is the key to produce the relative solving methods. Generally, noise is classified into additive noise and multiplicative noise according to the relationship between noise and pixels [20]. Additive noise is superimposed on the original video signal and is not related to the original signal, while multiplicative noise is expressed as the amplification or reduction of the grayscale of the original signal. Among them, the most common ones are Gaussian white noise and impulse noise. White noise is additive noise, the most common in practice, its characteristics are as follows: obey the Gaussian distribution, zero mean, flat power spectrum, and superimposed on the video signal, and the ideal Gaussian white noise should be uncorrelated with the signal, usually measured by the variance of the noise intensity. The Gaussian distribution probability density curve is shown in Equation (1), and its additive noise model is shown in Equation (2).
(1)q=e−x/σ22πσ,(2)I0=ni,j,tIi,j,t,whereI0 and ni,j,t are the original video signal and the noise signal, respectively, and Ii,j,t is the signal after being contaminated by noise.Burst impulse noise: as pepper and salt noise is the typical type of noise in this category, impulse noise differs from Gaussian noise in that it affects the original signal information more severely, causing the data on the covered pixels to be almost completely unavailable and visually more easily perceived by the human eye. It is also signal independent and does not affect other pixels of the image except for the covered pixel. Impulse noise is usually caused by the following: (1) noise caused by the transmission mechanism, (2) noise caused by the sensor, (3) noise caused by the storage mechanism, and (4) noise caused by the A/D conversion and the general model of impulse noise can be expressed as follows:
(3)Ii,j,t=0,i≤j,1,i>j,whereIi,j,t is the impulse noise point, which can be expressed as different impulse noise values, and the total probability of their occurrence is p, as depicted in Figure 1 for the two impulse noise values a and b, with occurrence probabilities p1 and p2, respectively, and p1+p2=p.Figure 1
Two noise probability density curves (a) Gaussian noise (b) impulsive noise.For Gaussian noise, several new nonlinear filtering noise reduction algorithms have been proposed, such as 3D threshold averaging filter, K-NN filter, and Rational filter, in addition to traditional 2D spatial filters, such as Gaussian filter, Wiener filter, and bidirectional filter. Since these methods somehow distinguish the image detail regions before smoothing the video noise, they improve the edge information of the image to some extent. The most common nonlinear denoising filter for impulsive burst noise is still the median filter. This technique is based on the following properties of images: noise tends to appear as isolated dots that correspond to a small number of pixels, while the original image information consists of areas with a large number of pixels and a large area. Median filtering effectively removes most of the burst noise when processing the whole frame, but it also destroys some detailed information in the image, such as fine edges and corner edges. In order to overcome these drawbacks, many scholars have made various improvements to the standard median filter in recent years and proposed filters such as center-weighted median and adaptive center-weighted median. In a general sense, median-based filters are selective filters whose output is always a certain value of the input sample, so the performance of these filters depends more on the recognition and judgment of the impulse noise, so the core of this type of noise restoration technology is how to design burst impulse detectors. In fact, the practical application value of filters that simply remove Gaussian or impulse noise is not high, and most of the videos have multiple characteristics of noise, so the key to designing noise reduction is to remove the mixed noise. Based on different applications, we analyze the characteristics of the mixed noise, try to use all the information available (in the time domain and spatial domain) without destroying the detailed parts of the natural image, and improve the objective (e.g., PSNR) and subjective quality of the video as much as possible, as shown in Figure2.Figure 2
Basic structure of noise reducer.


### 3.2. High-Order Partial Differential Denoising Model

Researchers proposed the following anisotropic diffusion model:(4)dkdt=divt∇u,(5)∇ui,j,tt=0=u0x,y,(6)du0x,ydt=0,where u0x,y is the noisy image, d and div denote the gradient operator and the scattering operator, respectively, du denotes the gradient mode of the image u, and gs is the monotonically decreasing function, which is usually chosen as follows:
(7)kt=tq2.The above equation wherek is the threshold is used to distinguish the flat region from the edge part of the image [21]. Since the diffusion coefficient varies with the gradient, the P-M model is the model of anisotropic diffusion. The model can effectively smooth the noise in the flat area of the image and effectively retain the image edge information in the edge part of the image, so it achieves the removal of noise while retaining the image edge. However, this model can produce “step effect” in the denoised image, which degrades the image quality, and the P-M model cannot guarantee the uniqueness of the solution, so it is a pathological model.In order to overcome the “step effect” of second-order PDE denoising, the researchers proposed the following fourth-order PDE denoising model:(8)dudt=∇ug∇u.The model retains the image edges while denoising, but the image denoising process is time-consuming with more iterations and causes speckle phenomenon when removing noise from smooth regions.Based on the above study, the researchers further proposed the classical full variational (TV) model (also known as ROF model) in image processing.(9)minu>iQu=∯u−q2,where Ω is the image region, f denotes the observed image, and u denotes the denoised image, where the first term is the regular term, which is used to ensure that the denoised image region has some discontinuities, the second term is the fidelity term, which prevents the denoised image from differing too much from the original image, and λ is the weighting coefficient, which is used to balance the regular and fidelity terms.Introducing the time auxiliary variablet, the solution by the gradient descent flow method has the following:
(10)dQudu=div∇u∇u.Ast increases, the image is slowly denoised and finally reaches a steady state. In the equation, the diffusion coefficient is du; then, the model can achieve fast diffusion in the flat area of the image, thus removing noise, and slower diffusion in the edge part of the image, thus preserving the image edge information. So the model can effectively remove the noise while preserving the image edge information. The ROF model and its improved models have been successfully applied to image denoising, image segmentation, image coloring, and other fields.Further, the researchers proposed an adaptive TV model.(11)minu>iTVu=∯m−q2.Among them,(12)kx=∇ug∇iG,i>1.The diffusion rate of different areas of the image can be controlled according to the image noise level. The model can preserve the detail information such as image edges while denoising, but the denoised image will still have the “step effect.” The researchers propose a generalized TV denoising model, where thei,j,t means the different distribution upon the dimensions.
(13)minikx=∮∇u/g∇iGpi,j,t,i>1.The denoising effect of the model is closely related to the value ofp. Only when a suitable value of p is chosen, the “step effect” will not occur. The full-variance-based denoising model has been widely studied and has achieved the ideal denoising effect in denoising.


### 3.3. Iterative Wavelet-Domain Adaptive Filtering

The strong steps of the image are preserved after several iterations. The diffusion process introduces edge detection, which makes the diffusion in the smooth domain preferable to the diffusion near the boundary, thus effectively preserving the edges. However, the denoising effect of the nonlinear diffusion equation established by Perona and Malik is sometimes not very good, and there are great difficulties in theory and practice. If the image is noisy, the Gaussian white noise introduces a very large theoretically unbounded gradient, so that the anisotropic smoothness introduced by the equation will sometimes give bad results, and some noise will be preserved without achieving the denoising purpose. Traditional iterative methods for image coloring include color transfer-based coloring methods and color diffusion-based coloring methods. In recent years, new coloring methods based on deep learning have also gradually gained the attention of scholars. This section focuses on several color diffusion-based coloring methods. Pixel points with similar grayscale values should have similar colors. In the paper, a small number of colored lines are first added to the image as the basis for coloring, and then, the following minimization equation is established:(14)Jx=∑ii>ng∗∇iUr+Nr,where r is the pixel point to be colored, Ur is the pixel value of r, Nr is the neighborhood of r, and Us denotes the chromaticity value of the sth neighboring pixel of r. wr is the weight function between pixel points r and s.
(15)∑ii>nwr∈Nr.Ifwrs is larger, the more similar the color of pixel point r is to pixel point s; otherwise, the more different the color of pixel point r is from s. wrs can be taken in two ways:
(16)ur⇒Ns−Nr+μr+μsσ2μr−μs,where wr and w2r are the mean and variance of the pixel luminance in the field with r as the center pixel, respectively. The process of solving Eq. is the process of color diffusion. The formula can be derived from the chromaticity component U. Similarly, the chromaticity component V can be obtained and then combined with the luminance component Y to obtain the final coloring image.This model better solves the problems we mentioned above, and we call it the Perona-Malik model, which has been subsequently improved and developed in various ways. So far, three main approaches have been developed in the field of PDE denoising: curvature-driven diffusion, tensor product diffusion, and variational methods. All these methods follow the same process, which is illustrated by the Perona-Malik model as an example, as shown in the following Eq.:(17)ur⇒ur−1⇒ur−1⇒.⋯⇒r>sus,where u0 denotes the noise image and uii=1,2,⋯,n denotes the denoised image.If the Perona-Malik model is used to denoise the image forn iterations, the whole process can be decomposed as Eq. The nth diffusion is performed on the (n−1)th denoised image un−1, and each diffusion is a denoising, and n diffusions are performed one after another to constitute the whole process of denoising by the PDE method. After the Perona-Malik model is converted to a benign state, there are several parameters to be chosen for the application. These parameters can be divided into two categories, those related to the partial differential equation itself and those related to the numerical solution of the partial differential equation, as shown in Figure 3, where u0 denotes the noise image and uii=1,2,⋯,n denotes the denoised image.Figure 3
Iterative process of partial differentiation.


## 3.1. Noise Characteristic Analysis

Here, in order to settle the image denoising, we first to introduce the characteristics of the noise existing in the film-imaging, which is the key to produce the relative solving methods. Generally, noise is classified into additive noise and multiplicative noise according to the relationship between noise and pixels [20]. Additive noise is superimposed on the original video signal and is not related to the original signal, while multiplicative noise is expressed as the amplification or reduction of the grayscale of the original signal. Among them, the most common ones are Gaussian white noise and impulse noise. White noise is additive noise, the most common in practice, its characteristics are as follows: obey the Gaussian distribution, zero mean, flat power spectrum, and superimposed on the video signal, and the ideal Gaussian white noise should be uncorrelated with the signal, usually measured by the variance of the noise intensity. The Gaussian distribution probability density curve is shown in Equation (1), and its additive noise model is shown in Equation (2).
(1)q=e−x/σ22πσ,(2)I0=ni,j,tIi,j,t,whereI0 and ni,j,t are the original video signal and the noise signal, respectively, and Ii,j,t is the signal after being contaminated by noise.Burst impulse noise: as pepper and salt noise is the typical type of noise in this category, impulse noise differs from Gaussian noise in that it affects the original signal information more severely, causing the data on the covered pixels to be almost completely unavailable and visually more easily perceived by the human eye. It is also signal independent and does not affect other pixels of the image except for the covered pixel. Impulse noise is usually caused by the following: (1) noise caused by the transmission mechanism, (2) noise caused by the sensor, (3) noise caused by the storage mechanism, and (4) noise caused by the A/D conversion and the general model of impulse noise can be expressed as follows:
(3)Ii,j,t=0,i≤j,1,i>j,whereIi,j,t is the impulse noise point, which can be expressed as different impulse noise values, and the total probability of their occurrence is p, as depicted in Figure 1 for the two impulse noise values a and b, with occurrence probabilities p1 and p2, respectively, and p1+p2=p.Figure 1
Two noise probability density curves (a) Gaussian noise (b) impulsive noise.For Gaussian noise, several new nonlinear filtering noise reduction algorithms have been proposed, such as 3D threshold averaging filter, K-NN filter, and Rational filter, in addition to traditional 2D spatial filters, such as Gaussian filter, Wiener filter, and bidirectional filter. Since these methods somehow distinguish the image detail regions before smoothing the video noise, they improve the edge information of the image to some extent. The most common nonlinear denoising filter for impulsive burst noise is still the median filter. This technique is based on the following properties of images: noise tends to appear as isolated dots that correspond to a small number of pixels, while the original image information consists of areas with a large number of pixels and a large area. Median filtering effectively removes most of the burst noise when processing the whole frame, but it also destroys some detailed information in the image, such as fine edges and corner edges. In order to overcome these drawbacks, many scholars have made various improvements to the standard median filter in recent years and proposed filters such as center-weighted median and adaptive center-weighted median. In a general sense, median-based filters are selective filters whose output is always a certain value of the input sample, so the performance of these filters depends more on the recognition and judgment of the impulse noise, so the core of this type of noise restoration technology is how to design burst impulse detectors. In fact, the practical application value of filters that simply remove Gaussian or impulse noise is not high, and most of the videos have multiple characteristics of noise, so the key to designing noise reduction is to remove the mixed noise. Based on different applications, we analyze the characteristics of the mixed noise, try to use all the information available (in the time domain and spatial domain) without destroying the detailed parts of the natural image, and improve the objective (e.g., PSNR) and subjective quality of the video as much as possible, as shown in Figure2.Figure 2
Basic structure of noise reducer.


## 3.2. High-Order Partial Differential Denoising Model

Researchers proposed the following anisotropic diffusion model:(4)dkdt=divt∇u,(5)∇ui,j,tt=0=u0x,y,(6)du0x,ydt=0,where u0x,y is the noisy image, d and div denote the gradient operator and the scattering operator, respectively, du denotes the gradient mode of the image u, and gs is the monotonically decreasing function, which is usually chosen as follows:
(7)kt=tq2.The above equation wherek is the threshold is used to distinguish the flat region from the edge part of the image [21]. Since the diffusion coefficient varies with the gradient, the P-M model is the model of anisotropic diffusion. The model can effectively smooth the noise in the flat area of the image and effectively retain the image edge information in the edge part of the image, so it achieves the removal of noise while retaining the image edge. However, this model can produce “step effect” in the denoised image, which degrades the image quality, and the P-M model cannot guarantee the uniqueness of the solution, so it is a pathological model.In order to overcome the “step effect” of second-order PDE denoising, the researchers proposed the following fourth-order PDE denoising model:(8)dudt=∇ug∇u.The model retains the image edges while denoising, but the image denoising process is time-consuming with more iterations and causes speckle phenomenon when removing noise from smooth regions.Based on the above study, the researchers further proposed the classical full variational (TV) model (also known as ROF model) in image processing.(9)minu>iQu=∯u−q2,where Ω is the image region, f denotes the observed image, and u denotes the denoised image, where the first term is the regular term, which is used to ensure that the denoised image region has some discontinuities, the second term is the fidelity term, which prevents the denoised image from differing too much from the original image, and λ is the weighting coefficient, which is used to balance the regular and fidelity terms.Introducing the time auxiliary variablet, the solution by the gradient descent flow method has the following:
(10)dQudu=div∇u∇u.Ast increases, the image is slowly denoised and finally reaches a steady state. In the equation, the diffusion coefficient is du; then, the model can achieve fast diffusion in the flat area of the image, thus removing noise, and slower diffusion in the edge part of the image, thus preserving the image edge information. So the model can effectively remove the noise while preserving the image edge information. The ROF model and its improved models have been successfully applied to image denoising, image segmentation, image coloring, and other fields.Further, the researchers proposed an adaptive TV model.(11)minu>iTVu=∯m−q2.Among them,(12)kx=∇ug∇iG,i>1.The diffusion rate of different areas of the image can be controlled according to the image noise level. The model can preserve the detail information such as image edges while denoising, but the denoised image will still have the “step effect.” The researchers propose a generalized TV denoising model, where thei,j,t means the different distribution upon the dimensions.
(13)minikx=∮∇u/g∇iGpi,j,t,i>1.The denoising effect of the model is closely related to the value ofp. Only when a suitable value of p is chosen, the “step effect” will not occur. The full-variance-based denoising model has been widely studied and has achieved the ideal denoising effect in denoising.


## 3.3. Iterative Wavelet-Domain Adaptive Filtering

The strong steps of the image are preserved after several iterations. The diffusion process introduces edge detection, which makes the diffusion in the smooth domain preferable to the diffusion near the boundary, thus effectively preserving the edges. However, the denoising effect of the nonlinear diffusion equation established by Perona and Malik is sometimes not very good, and there are great difficulties in theory and practice. If the image is noisy, the Gaussian white noise introduces a very large theoretically unbounded gradient, so that the anisotropic smoothness introduced by the equation will sometimes give bad results, and some noise will be preserved without achieving the denoising purpose. Traditional iterative methods for image coloring include color transfer-based coloring methods and color diffusion-based coloring methods. In recent years, new coloring methods based on deep learning have also gradually gained the attention of scholars. This section focuses on several color diffusion-based coloring methods. Pixel points with similar grayscale values should have similar colors. In the paper, a small number of colored lines are first added to the image as the basis for coloring, and then, the following minimization equation is established:(14)Jx=∑ii>ng∗∇iUr+Nr,where r is the pixel point to be colored, Ur is the pixel value of r, Nr is the neighborhood of r, and Us denotes the chromaticity value of the sth neighboring pixel of r. wr is the weight function between pixel points r and s.
(15)∑ii>nwr∈Nr.Ifwrs is larger, the more similar the color of pixel point r is to pixel point s; otherwise, the more different the color of pixel point r is from s. wrs can be taken in two ways:
(16)ur⇒Ns−Nr+μr+μsσ2μr−μs,where wr and w2r are the mean and variance of the pixel luminance in the field with r as the center pixel, respectively. The process of solving Eq. is the process of color diffusion. The formula can be derived from the chromaticity component U. Similarly, the chromaticity component V can be obtained and then combined with the luminance component Y to obtain the final coloring image.This model better solves the problems we mentioned above, and we call it the Perona-Malik model, which has been subsequently improved and developed in various ways. So far, three main approaches have been developed in the field of PDE denoising: curvature-driven diffusion, tensor product diffusion, and variational methods. All these methods follow the same process, which is illustrated by the Perona-Malik model as an example, as shown in the following Eq.:(17)ur⇒ur−1⇒ur−1⇒.⋯⇒r>sus,where u0 denotes the noise image and uii=1,2,⋯,n denotes the denoised image.If the Perona-Malik model is used to denoise the image forn iterations, the whole process can be decomposed as Eq. The nth diffusion is performed on the (n−1)th denoised image un−1, and each diffusion is a denoising, and n diffusions are performed one after another to constitute the whole process of denoising by the PDE method. After the Perona-Malik model is converted to a benign state, there are several parameters to be chosen for the application. These parameters can be divided into two categories, those related to the partial differential equation itself and those related to the numerical solution of the partial differential equation, as shown in Figure 3, where u0 denotes the noise image and uii=1,2,⋯,n denotes the denoised image.Figure 3
Iterative process of partial differentiation.


## 4. Numerical Experiments and Results Analysis

The methods for evaluating the effect of noise removal in images include two types: (1) the subjective method, i.e., judging whether the noise is removed cleanly by viewing the image directly with the human eye. This method is greatly influenced by subjective factors, and the judgment results vary from observer to observer. In addition, the subjective method cannot judge images with small differences and can only roughly evaluate the image denoising results. (2) Objective method, i.e., a specific value is obtained through certain evaluation indexes for judgment. The common image quality evaluation indexes are peak signal-to-noise ratio (PSNR), signal-to-noise ratio (SNR), and mean structural similarity (MSSIM). LetIi,j and Ki,j denote the original image and the processed image, respectively, and M∗N denotes the image size, then,
(1)
Peak signal-to-noise ratio (PSNR)(18)PSNR=2ln123∑j=1u∑i=1uIi,j,t−Ku(2)
Signal-to-noise ratio (SNR)(19)SNR=1232∑j=1u∑i=1uIi,j,t−KuWe assume that the noise variance is known and can otherwise be estimated from the best resolution subband by the method, the variance selected in the iterations is capped by the estimated variance, and the selected noise variance should not be larger than the actual situation each time; otherwise, it will destroy the real signal, and the methodological steps for the implementation are given below.(1)
Convolution is performed onU02 to obtain qij, and the size of the convolution kernel varies with the number of iterations and the level of decomposition(2)
Applying the Wiener filter reconstruction formula, the denoised image wavelet reconstruction coefficients are obtained as follows:(20)Ui,j=∑i5Ci(3)
Wavelet inversion is performed to obtain the first denoised imageIn the subsequent experiments, the are 5 iterations of the selected wavelets in order, and each wavelet transform is decomposed into five layers. The size of the square convolution window for each layer in the first iteration is noted asC1=3,5,7, and 7, 5, 3, 3, 3, and 3 are the sizes of the first-, second-, third-, fourth-, and fifth-layer windows in that order; similarly, the size of the window for each layer in the second iteration is noted as C2=3,5, the third as C3=3,5, the fourth as C1=3,3,3,3,3, and C5=1,3 for the fifth time. The standard deviation of noise for each iteration is recorded as σ=161, σ=122, σ=73, σ=5.54, and σ=45, in that order.In the following, the gradients in the 4 directions at the grid pointi,j are first given in uw denoting the image of w after Gaussian preprocessing, and then, the diffusion coefficients in the 4 directions are given as follows:
(21)∇1uri,j=uwi−1,j−uwi,j,∇2uri,j=uwi−2,j−uwi−1,j,∇3uri,j=uwi−3,j−uwi−2,j,⋯∇nuri,j=uwi−n,j−uwi−n+1,j.Next, we perform noise reduction on the movie images resolved using partial differential equations. Figure4 shows the plots of the coloring results of different methods for the texture image. Compared with other methods, the model proposed in this chapter has obvious advantages. For this method, all the image colors change significantly, such as the redundant colors in the right side and the bottom. We observe that the proposed model has better coloring results, which is due to the fact that the proposed model can preserve the contour lines of grayscale images while preserving the colored edges.Figure 4
Coloring result of denoising method.We have utilized two kinds of sequences to test the denoising effect: one is a sequence that is inherently noisy, and the other is a sequence with artificially added noise. The former cannot be evaluated using PSNR as an objective way because it does not have the help of a noise-free contaminated reference sequence, and here in this paper, we give comparison plots showing the subjective effect of the restoration, while the described noise estimator1 will be used to evaluate the difference between the noise variance in the restored sequence and the noisy sequence. The second test sequence is the impaired sequence with the specified Gaussian noise variance added, and the noise estimator described above also indirectly reflects the accuracy of the specified variance of the generated noise sequence. In this section, a commonly used “monitor” sequence in CIF format is used as the test subject, which is a surveillance video, and the subjective effect of noise restoration is shown in Figure5 using the 3D-BDP method proposed in this paper. The noise variance obtained by the noise estimation method is compared, the “monitor” sequence itself is not very noisy, and the noise variance obtained by the estimator is reduced by about 0.3.Figure 5
Comparison of noise variance before and after sequence repair.In order to obtain artificially generated noise sequences for experiments, Gaussian noise with specified variance and impulse noise with specified coverage can be added for YUV sequences, generating impulse noise with parameters for specified pixel coverage percentage and generating Gaussian noise with parameters for approximating the simulated noise variance with Rayleigh distribution. The generator can generate mixed noise. The noise variance obtained with the noise estimation method is compared with the noise sequence generation with the specified noise target variance of 5, 10, and 20, respectively, and the average noise variance obtained by the noise estimator is 4.8, 9.7, and 19.7, respectively, from Figure6, which reflects the accuracy of the noise sequence generator from one side.Figure 6
Comparison of noise variance before and after sequence repair.Each filter has a different focus, for example, the Rational filter is more suitable for removing impulse noise, the THAF filter is more suitable for low Gaussian noise, and the K-NN filter is more comprehensive. The proposed 3D-BDP filter (VI) is suitable for handling Gaussian noise, and the PSNR value of the restored sequence is significantly better than other filters because the 3D-BDP filter adopts different filtering strategies for regions with different characteristics in the motion sequence, such as motion region, detail region, and smooth region. In addition, the 3D-BDP filter also takes into account the noise variance and adjusts the parameters accordingly to obtain a better filtering effect. On the other hand, the 3D-BDP filter has an obvious drawback that it does not work well with impulse noise, but its modified 3D-BDPI filter (VII) adds a corresponding impulse noise detection and removal strategy to improve the impulse noise processing, but at the cost of a small performance degradation when dealing with Gaussian noise. The variance of the I and III noise and the restored sequences obtained by the noise estimator is shown in Figure7, with an average reduction of about 0.8 in the noise variance before and after restoration.Figure 7
Comparison of noise variance before and after I sequence repair.The IV sequence is a sequence in which the camera shakes, so that there is an overall motion in one frame, and the restoration results are compared as shown in the figure. It can be seen from the figure that the restored frame 20, which is in the stationary time period, is subjectively better than frames 69 to 71, because it is determined to be a stationary frame and is restored with the three-frame noise reduction algorithm, while frames 69, 70, and 71 are determined to be in global motion and can only be restored with the two-frame algorithm in order to reduce the global blur caused by motion. The variance of the noise and restored sequences obtained by the noise estimator is shown in Figure8, with an average reduction of about 1.6 before and after restoration.Figure 8
Noise variance after sequence repair.A new fourth-order partial differential equation image denoising model is proposed to address the step effect of the second-order partial differential equation in processing noisy images. The proposed model is first computed numerically using the finite difference method, and then MATLAB simulation experiments are conducted for Lena images, Cameraman images, and color Lena images and color Pepper images, respectively. The proposed model in this paper is compared with the existing denoising model, and the analysis results show that the model proposed in this section can effectively remove multiplicative noise. In this paper, the principle of the spatial noise reduction filter is described in detail, and the 3D-BDP filter algorithm is proposed for Gaussian noise, the improved 3D-BDPI filter algorithm is proposed for mixed noise based on the 3D-BDP filter, and the LLL filter algorithm is proposed for luminance and color noise in low brightness scenes. The LLL filter significantly improves the coding performance of the encoder and has a strong noise reduction effect in low luminance backgrounds, especially in night scenes. The research of the null-time noise reduction filter has great practicality in the field of fast video denoising. Nowadays, there is still a gap between the processing chip and the computational power of most video processing algorithms in reality, such as on mobile platforms (ARM platforms on cell phones), where the main frequency is only about one-tenth of that on PC platforms, and the algorithm speed is a factor that must be given priority. Therefore, how to combine the complexity with the restoration effect and the comprehensive design of denoising filters is still an issue that researchers in image and video restoration technology must face up to. For comparing with the previous method to filter the resolution of the movies, we, respectively, simulate different scenes in action movies, horror movies, and documentaries. As shown in Figure9, the methods used in this work show an obvious stable and rapid action in the real scenes. Especially, in the documentary, the traditional DPI methods show an obvious instability.Figure 9
Resolution contrast with different methods.


## 5. Conclusion

Image denoising is the most basic work in image processing. Noise largely blurs the detailed information of an image and brings difficulties to the analysis and application of the image. The partial differential equation-based image denoising method can remove the noise while protecting the edges so that the detailed information of the image is not destroyed, and the method has received much attention from scholars. By dissecting the denoising process of partial differential equation method (PDE), we establish a new iterative denoising algorithm. The partial differential equation method can be considered as the iterative denoising of the filter, and the first stage of the new algorithm uses wavelet-domain adaptive Wiener filter as the filter base and achieves good results by adjusting the parameters. The experimental report shows that the parameters set by the algorithm have certain stability and can achieve good results for multiple images, which is an advantage over the partial differential equation method for noise removal. In this paper, we analyze the denoising process of the PDE method, establish an iterative wavelet-domain adaptive Wiener filtering denoising model (IWAW), perform PDE postprocessing with pseudo-Gibbs removal on the denoised images of the IWAW method, and finally form the IWAWP denoising system with good denoising effect.



---

*Source: 8176746-2021-09-11.xml*