Image restoration model using Jaya‐Bat optimization‐enabled noise prediction map

IET Image ProcessingVolume 15, Issue 9 p. 1926-1939 ORIGINAL RESEARCH PAPEROpen Access Image restoration model using Jaya-Bat optimization-enabled noise prediction map Premnath S. P, Corresponding Author sppremnath959@gmail.com Department of Electronics and Communication Engineering, Sri Krishna College of Engineering and Technology, Kuniamuthur, Coimbatore, 641008 India Correspondence Premnath S.P, Sri Krishna College of Engineering and Technology, Kuniamuthur, Coimbatore 641008, India Email: sppremnath959@gmail.comSearch for more papers by this authorJ. Arokia Renjit, Department of Computer Science and Engineering, Jeppiaar Engineering College, Jeppiaar Nagar, Rajiv Gandhi Salai, Chennai, 600 119 IndiaSearch for more papers by this author Premnath S. P, Corresponding Author sppremnath959@gmail.com Department of Electronics and Communication Engineering, Sri Krishna College of Engineering and Technology, Kuniamuthur, Coimbatore, 641008 India Correspondence Premnath S.P, Sri Krishna College of Engineering and Technology, Kuniamuthur, Coimbatore 641008, India Email: sppremnath959@gmail.comSearch for more papers by this authorJ. Arokia Renjit, Department of Computer Science and Engineering, Jeppiaar Engineering College, Jeppiaar Nagar, Rajiv Gandhi Salai, Chennai, 600 119 IndiaSearch for more papers by this author First published: 06 March 2021 https://doi.org/10.1049/ipr2.12162AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Image restoration approaches are introduced to restore the latent clear images from the degraded images. However, the performance of the existing approaches remains an open problem, which may leads to the further development of advanced image restoration techniques. Therefore, an effective image restoration method is developed for restoring the input image from various noises, like impulse noise and random noise. The generation of pixel map, identification of noisy pixel, and the enhancement of pixel are the three major phases involved in the proposed method. Initially, the noisy pixel map generation is performed from the input image, and then the noisy pixels are identified based on deep convolutional neural network, which is trained by the proposed Jaya-Bat algorithm. The Jaya-Bat algorithm is developed by combing the Jaya optimization algorithm and Bat algorithm. Once the noisy pixels are identified, the pixel enhancement is done using the neuro fuzzy system. The experiment is carried out using Statlog (landsat satellite) dataset, and the developed method achieves the maximal peak signal to noise ratio of 51.03 dB, maximal structural similarity index of 0.848 for the image with random noise, and the maximal second derivative like measure of enhancement 62.96 dB with impulse noise, respectively. 1 INTRODUCTION The images taken in dim light is unsatisfactory, and noisy while increasing the ISO at short duration. Using the flash light may improve lighting, but still creates unwanted shadow or changing the tone in image [1, 2]. Image restoration plays a very important role in the field of computer vision, graphic design, remote sensing, agricultural buildings [3], thermal analysis [4] and the medical imaging [5]. The image restoration aims to recover the high-quality images from their degraded observation [6, 7]. In the image restoration, the degradation process is generally defined using linear transformation [7]. The degraded images are restored using various methods in signal processing environment. Here, the clean patches are predicted separately [8] or collaborating other same patches using non-local schemes [9]. Then, the computed patches in the image are returned to their raw positions and the overlapped patches get average to obtain the reconstructed image [10]. While denoising the image, the overlapping images are initially decomposed and denoising each patch separately and at last, the result is obtained by simple averaging [11, 12]. In the last few years, several pre-processing restoration approaches for HIS are introduced to achieve optimal results. The previous restoration approaches are broadly categorized into three. They are morphology or filter-based techniques, spectral and spatial methods, and the sparse representation and low-rank (SR-LR)-based methods. The filter-based techniques are the type of most practical and traditional HIS restoration approaches [13]. Non-local techniques are performed based on intrinsic similarities present in image patches. This leads to improve the development of various non-local image restoration approaches introduced in [9, 10, 14-16]. Non-local restoration capabilities are used by most of the research works for grouping similar images in image. In addition, the sparse representations are obtained based on sparsity methods [10, 11, 17]. Gaussian mixture models (GMM) is utilized in several signal processing tasks, like video applications, audio processing [18], image segmentation, and image denoising [19]. Nowadays, GMM is utilized to tackle the recovery issue caused in nonlinear and sparse computation, as GMM is the integration of linear estimation. Piecewise linear estimation (PLE) is another image restoration method employed for image patches. Expected patch log likelihood (EPLL) is introduced in [20] for weight initialization [10]. Introducing global model for the entire image is very hard because of the curse of dimensionality. Hence, several image restoration algorithms are introduced nowadays, for addressing modelling with patch priors [11]. This patch priors are easy to model, readily available and low dimensional. In [11], GMM, sparsity inspired model, and ICA are introduced to identify the same patches. In global GMM image restoration the similar patches are measured by Gaussian probability density function [10, 12]. In this research, the image restoration is carried out using the proposed Jaya-Bat optimization algorithm. The proposed approach involves three phases. The initial phase is generation of pixel map, the secondary phase is the identification of noisy pixel, and the tertiary phase is the enhancement of pixel. At first, the noisy pixel map generation is performed from the input image, and then the noisy pixels are identified based on Deep CNN, which is trained by the proposed optimization algorithm, termed Jaya-Bat algorithm. The Jaya-Bat algorithm is the combination of Jaya optimization algorithm and Bat algorithm. After the identification of noisy pixels, the pixel enhancement is done on the basis of fuzzy neuro system. The contribution of this research is the development of Jaya-Bat-based Deep CNN for eliminating the impulse and random noise from remote sensing images. The Jaya-Bat-based deep CNN is designed by integrating Jaya optimization algorithm and Bat algorithm. The developed method detects the noise in corrupted images into non-corrupted images. The paper is structured in the following manner: Section 2 discusses existing methods of image restoration with challenges of the methods that remains the motivation for the research. The proposed method of Jaya-Bat-based Deep CNN is demonstrated in Section 3, and Section 4 provides the results of developed methods. At last, concludes the research work in Section 5. 2 LITERATURE SURVEY Several methods related to image restoration are described, and analysed as follows: Papyan and Elad [11] developed multi-scale expected patch log likelihood (EPLL) method that imposes very small patch on various scale patches extracted from the image. This method attained better performance in both visually and quantitatively, however, denoising the noisy image is difficult. Zeng et al. [21] presented Average Image-induced non-local means (aviNLM) for image restoration. In this approach, the prior image was generated from the reconstructed image in every energy bin. Here, the performance was found better, but the deformation of tissue is very challenging. Niknejad et al. [10] modelled Gaussian mixture model (GMM) where the neighbourhood similar patches were derived based on multivariate Gaussian probability distribution with covariance and mean. However, zooming is very challenging from the randomly observed pixels. Teng et al. [13] developed adaptive morphological filtering (AMF) and the fusion-enabled restoration method. This method was very flexible enhancement component for local filters, but failed to consider HSI restoration for both pre-processing and post-processing areas. Zhang and Desrosiers [7] presented an approach for image restoration that integrates the global structure sparsity and non-local self-similarity in the single efficient model. Here, the performance was found better, but developing effective image regularization priors was very difficult. Shen et al. [2] developed multispectral joint image restoration framework for handling all the structure divergence and finding the edges and transitions. It was more robust and to tackle multispectral restoration issues. The method failed to consider other optimization algorithms for improving the system performance. Wang and Tao [1] modelled Non-local auto-encoder that uses self-similar information of natural images for the stability. This method achieved comparable denoising performance to other state-of-the-art methods and outperformed other leading super resolution methods, but the performance of image denoising was found poor. Peng and Cosman [22] developed depth estimation approach for underwater scenes using light absorption and image blurriness. This method estimates underwater depth scene accurately, but the variation of physical properties was difficult for restoring underwater images. Asif et al. [23] developed a deep learning (DL)-based image evaluation method (W-LDMM) for the effective image restoration. This method enhanced the quality of the endoscopy images used to examine internal body organs. Anyhow, the performance of this method was evaluated using small dataset. Wang et al. [24] developed a constrained iterative support shrinking algorithm with proximal linearization (C-ISSAPL) for enhancing the image restoration. This method had maximum convergence speed and easy to implement. Also, this method was applicable for natural image restoration and piecewise constant. This method was required more processing time. However, in this method, the calculation of box constraint was difficult. 3 PROPOSED JAYA-BAT ALGORITHM-BASED DEEP CONVOLUTIONAL NEURAL NETWORK FOR IMAGE RESTORATION This research primarily focuses to model a new method for image restoration using Jaya-Bat algorithm-based DCNN. Initially, the input image is fed to the noisy pixel map generation stage, where the noisy pixel map is effectively generated. The generated noisy pixel map is subjected to the noisy pixel identification stage, where the noisy pixels are identified based on deep convolutional neural network (deep CNN) classifier, which is trained by the Jaya-Bat optimization algorithm. Here, the proposed Jaya-Bat algorithm is the integration of Jaya optimization algorithm [25], and Bat Inspired algorithm [26]. Finally, the identified noisy pixels are sent to the pixel enhancement stage, where the enhancement of the pixel is carried out using the neuro fuzzy system. Figure 1 depicts the schematic view of the developed model for image restoration. FIGURE 1Open in figure viewerPowerPoint Block diagram of the proposed Jaya-Bat algorithm for image restoration Assume the input image J with size U × V , and the pixel location of the input image is represented as J ( x , y ) . Due to the noise present in image, certain part of the image is corrupted and the remaining pixels are considered as the noise free. The detected noisy pixels are very challenging, hence in this paper, the Deep CNN is introduced to identify the noisy pixel by considering the neighbourhood of the image. 3.1 Noisy pixel identification: Constructing the noisy pixel map using the DCNN classifier This DCNN is utilized for pixel identification. The DCNN is utilized for generating the prediction map for image pixels. The DCNN classifier [27] generates the prediction map based on input image. The architecture of DCNN and the algorithmic steps of the Jaya-Bat algorithm are described below. 3.1.1 Architecture of the DCNN The basic architecture of the DCNN [27] is discussed in this section with its architecture in Figure 2. The DCNN comprises of the number of convolutional (conv) layers, pooling (POOL) The architecture of the Deep CNN is deliberated in Figure 3 and the architecture of DCNN comprises of three layers, like pooling (POOL), convolutional (conv), and the fully connected (FC) layers. Among three layers of DCNN, each of the layers constitutes specific function. The main function of the conv layers is to generate the feature maps from the segments of the pre-processed image and these feature maps are further sub-sampled down in the pool layers. The third layer is FC layer, where the classification is progressed. The convolutional layer engages in mapping the input such that the input maps undergo convolution with the convolutional kernels in order to develop the output map The size of the output map is similar to the kernel number, and the size of the kernel matrix is, [3 × 3]. Thus, making it clear that the conv layers is the multilayer loop of input maps, kernel weights, and output maps. In the first conv layer, there are a number of inputs and outputs, whose size reduces in the successive conv layers such that the classification accuracy of the objects depends on the number of the layers in the DCNN. FIGURE 2Open in figure viewerPowerPoint Architecture of DCNN for the construction of prediction map FIGURE 3Open in figure viewerPowerPoint New pixel value computation Conv layers: The conv layer is utilized for extracting the features and provides pattern extraction using the feature vector obtained from the input images. The neurons are linked using the trainable weights and these trained weights are convoluted with the inputs to build the feature maps. Then, the outcome is subjected to non-linear activation function to ease complex functional mappings between input and the response variables. Let us assume the input to the deep CNN is G and hence, the output from the conv layer is given as, ( M c b ) i , j = B c b i , j + ∑ d = 1 ϖ 1 d − 1 ∑ k = − ϖ 1 l ϖ 1 l ∑ n = − ϖ 1 l ϖ 2 l ϑ c , d b k , n ∗ J i + k , j + n (1)where, the symbol * indicates the convolutional operator, which enables the pattern extraction from the outputs obtained from the adjacent conv layers, ( B c b ) i , j indicates the fixed feature maps. The output from previous ( b − 1 ) th layer forms the input to lth conv layer. The conv layer weight is denoted as, ϑ c , d b , which is the weights of bth conv layer and the bias of bth conv layer is denoted as, B c b . Let us consider d, k, and n as the notations of feature maps. ReLU layer: ReLU is the activation function for ensuring effectiveness and simplicity and the significance of ReLU layer is that DCNN with ReLU works faster dealing with large networks. The output from the ReLU layer when fed with the feature maps is given as, G c b = A f n M c b − 1 (2)where, A f u n ( ) refers to the activation function in bth layer. POOL layers: It is a non-parametric layer with no weights, and bias, undergoing a fixed operation. The importance of POOL layer is to mitigate the spatial dimensions of the input and minimizes the computational complexity. FC layers: The patterns generated using the pooling and the conv layers form the input to the fully connected layers that are subjected to the high-level reasoning. The output from the fully-connected layers is given as, H c b = δ G c b − 1 with M c b = ∑ d = 1 ϖ 1 d − 1 ∑ k = − ϖ 1 l ϖ 1 l ∑ n = − ϖ 1 l ϖ 2 l ϑ c , d l k , n ∗ J i + k , j + n (3)where, ( ϑ c , d l ) k , n denotes the weight. 3.1.2 Training of DCNN based on Jaya-Bat algorithm This section briefly explains the developed Jaya-Bat algorithm for image pixel identification. Here, the proposed Jaya-Bat is developed by integrating the Jaya optimization algorithm [25] and Bat algorithm [26]. The BA is motivated by echolocation characteristics of microbats. This method is very efficient for generating improved features to solve the multi-objective optimization problems. Moreover, it can solve the highly non-linear problems with complex constraints. Jaya is performed using candidate solutions, and this framework works independently of any parameters. The Jaya algorithm is simpler in operation; because it works in a single phase. Here, the update equation of Bat algorithm is modified using the update equation of Jaya optimization algorithm. The modification makes the solution update to be more efficient, and it further improves the performance of the proposed optimization algorithm. The update equation of Bat is expressed as, D r + 1 j = D r j + u r j (4)where, the term D r + 1 j denotes the jth bat position in ( r + 1 ) th iteration, D r j indicates the jth bat position in rth iteration, and u r j indicates the jth bat velocity in rth iteration. The velocity of the bat at rth iteration is given as, u r j = u r + 1 j − D r j − D ∗ p j (5) Substitute u r j in Equation (4), the solution becomes, D r + 1 j = D r j + u r + 1 j − D r j − D ∗ p j (6)where, D ∗ refer to the best position of the bat, p j signifies the frequency of the jth bat, and u r + 1 j indicates the velocity of the bat in ( r + 1 ) th iteration. Then, the above Equation (6) is modified using the Jaya for improving the effectiveness of approach and to solve several optimization problems. The standard equation of the Jaya optimization is given by, D r + 1 j = D r j + v 1 D ∗ − D r j − v 2 D worst − D r j (7)where, the term D r + 1 j indicates the value of jth variable in ( r + 1 ) th iteration, v1 and v2 refers to the random numbers ranging from 0 to 1. The worst candidate solution is represented as Dworst. Let us assume D r j is positive, D r + 1 j = D r j + v 1 ( D ∗ − D r j ) − v 2 ( D worst − D r j ) (8) D r + 1 j = D r j + v 1 D ∗ − v 1 D r j − v 2 D worst + v 2 D r j (9) D ∗ = 1 v 1 D r + 1 j − D r j + v 1 D r j + v 2 D w o r s t − v 2 D r j (10) Substituting Equation (10) in Equation (6) the final equation is, D r + 1 j = u r + 1 j v 1 + D r j v 1 − 1 + v 2 p j + v 2 D worst p j v 1 − p j (11)where, p j = p min + ( p max − p min ) α . Here, α denotes the random vector ranging from (0, 1), and f min is considered as zero and the value of f max tends to 0(1). The steps followed in the developed algorithm are illustrated below, Step 1: Initialization: The initial step of the proposed Jaya-Bat algorithm is the initialization of bat population in the search space as, B a t p o p u l a t i o n , D j ; 1 ≤ j ≤ a (12) where, a denotes the total bats, and D j refer to the location of jth bat in the search space, and D ∈ ϑ c , d b , ϑ c , d l , B c b . Step 2: Fitness function evaluation: The selection of the optimal location of the bat is performed based on minimization problem. The minimal value of the objective function describes the better solution and therefore, the solution with the minimum value of the error is chosen as the best solution. The error is determined as, MSE = 1 X ∑ h = 1 X H target − H c b (13) where, Htarget, and H c b are the estimated and target output of classifier. The term X denotes the total number of samples. Step 3: Update the solution based on Jaya-Bat algorithm: After computing the objective function, the solution undergoes the location update on the basis of Bat algorithm. While updating the solution, there exist two conditions, first based on pulse emission rate and the second based on loudness and the fitness. In the first condition, the random number is compared with the pulse emission rate, and if the random number exceeds the pulse emission rate, the position is updated based on random walk. In the second condition, if the random number is less than the pulse emission rate, then it is compared with the fitness and the loudness of the bat. If the random number exceeds the loudness and fitness exceeds the fitness of best solution, then the new solution is updated using Equation (22). After position update, the loudness decreases and the pulse emission rate increases for successive iterations. The evaluation of loudness and emission rate for the jth bat is expressed as, D r + 1 j = τ D r j (14) F r j = F 0 j 1 − exp − ω j (15) where, D r j represents the loudness in previous iteration, F r j is the emission rate, F 0 j is the initial emission rate, and τ and ω are the constants. Step 4: Ranking of bats based on fitness function: The solutions are ranked based on fitness and the solution with highest fitness measure is chosen as the best solution, D ∗ . Step 5: Termination: The process is continued for maximum number of iterations and stopped after the generation of global optimal solution. The pseudocode of the proposed Jaya-Bat algorithm is depicted in Algorithm 1. ALGORITHM 1 Pseudocode of proposed Jaya-Bat approach Input :Population of bats D j ; ( 1 ≤ j ≤ a ) Output :Best position of Bat D ∗ Start Initialization For ∀ a Calculate the pulse emission rate and the loudness While ( j < Maximum iterations ) Produce new solutions using equation (24) If ( Rand > F j ) { Update position based on random walk } End if Generate new solutions based on random fly If ( Rand < F j & & Fitness ( D j ) < Fitness ( D ∗ ) ) { Update the bat position Increment pulse emission rate F j decrement loudness D j } End if Rank the solution based on fitness Estimate the current best solution End While Produce the best solution D ∗ Stop 3.2 Restoration of noisy pixel based on statistical model After the identification of noisy pixel, the novel pixel values are estimated for corresponding noise pixels. When the noisy centre pixel is detected, the noisy pixel is removed using the below expression. J n p ( x , y ) = J n x , y i f M x , y = 1 J x , y otherwise (16)where, the term J n ( x , y ) refer to the new pixel image value. The selected noisy pixel is only applicable to compute the new pixel value. At first, the 3 × 3 window J ( u , v ) is generated, and then the matching is done with the noisy pixel of 3 × 3 window J ( u , v ) and the input image J ( x , y ) . From the matching output, the same matched pixels NS are considered for further processing. When NS is higher than threshold value T1, the novel pixel value is produced or else the noisy pixels are changed by the raw image in the selected 3 × 3 windows. The value of T1 is considered as 21. Here, the new pixel values are computed based on X b , Z, and B parameters, and is expressed below. E x , y = e X b , Z , B (17)where, the term Z refer to the absolute function of 3 × 3 window based on neighbour pixel and is given below, Z = abs J u , v − J u + x − 3 , v + y − 3 (18) The value of X b is derived by the absolute output of Equation (29) with prefixed window. The value of X b is initially set to zero in which b varies on the basis of selected prefixed window. X b = X b + Q 1 x , y ∗ Z ; 1 ≤ b ≤ ω (19) The rounding process is carried out for X b and is round by X b 8 , which is represented as g. After rounding, the sorting process is done for the result obtained from X b where the initial value is selected as E. E x , y = 1 − E − 1 / 4 E > 10 1 otherwise (20) The result of Equation (31) is compared with T2. Here, T2 value is set to 0.2. When the value of E ( x , y ) is higher than the second threshold value, the function is produced. R P x , y = s u m P α / q 1 s > T 3 E x , y o t h e r w i s e (21)where, the term α refer to the mean value of neighbourhood pixel, and the term q1 signifies the constant in which the q1 value is set to 4. The mean value is evaluated using the below expression, α = e Y a ∗ Y a / 0.001 + 2 β 2 (22)where, the term β signifies the standard deviation. When the output obtained from Equation (32) is compared with threshold value T3. Here, the T3 value is fixed to 23. If it is greater than T3, the new pixel value is generated using the equation given below, J n x , y = E P A E P B = ∑ J x , y . M x , y ∑ M x , y (23) Algorithm 2 depicts the pseudo code of novel pixel computation from noisy pixels. ALGORITHM 2 Pseudocode for computing new pixel Input: J k ( x , y ) Output: J n ( x , y ) Computation of parameter based on Jaya-Bat algorithm Procedure: Begin Selection of 3 × 3 window by considering the noise pixel as the centre pixel Matching the noisy pixel with input image Identification of same matched pixel, (Ns) If ( N s > 21 ) Identify E ( x , y ) If ( N s > 0.2 ) Find RP ( x , y ) else RP ( x , y ) = E ( x , y ) If RP ( x , y ) > 23 J n ( x , y ) = J ( x , y ) else Determination of new pixel values for noisy pixel J ( x , y ) else J n ( x , y ) = J ( x , y ) 3.3 Image enhancement based on neuro fuzzy system After the identification of new pixels, the image enhancement process is performed based on neuro fuzzy system. The image enhancement process is utilized for modifying input image by changing a visual impact. The enhancement image provides the detailed information about boundaries, edges with increased dynamic range. In this work, the enhancement procedure is carried out on the basis of noisy pixels. Once the noisy pixels position K are identified, that identified pixels s only applied for the enhancement process. After that, the new image matrix is produced on the basis of nearest neighbour for the noisy pixel is expressed by, J n 1 x , y ∀ K = 1 9 ∑ u = 1 1 ∑ v = 1 1 J n x + u , y + v (24) In order to solve the uncertainty issues, the generated new pixel values are subjected to the neuro fuzzy system. J n 2 x , y ∀ K = N fuzzy J n x , y (25)where, the term Nfuzzy signifies the neuro fuzzy system. Once the neuro fuzzy set model is generated, the new pixel J n ( x , y ) is considered as the input and the new image is generated using the noise pixel-based location. At last, the restored image is obtained from the below equation. J G x , y ∀ K = J n 1 x , y + ∀ K J n 2 x , y ∀ K 2 (26) The structure of ANFIS [28] composed of consequence and premise parts. The ANFIS is trained for determining the parameters related to the parts employed in optimization approach. Here, the ANFIS uses output and input data pairs in the training. The layers present in ANFIS are fuzzification, rule, normalization, defuzzification, and summation layer. Here, the fuzzification layer employs the membership functions for obtaining fuzzy clusters from the input values where the firing strengths are produced based on the membership values evaluated in the fuzzification, whereas the normalization layer is utilized to compute the firing strengths corresponding to every rule. The normalized value refers to the ratio of firing strength to ith rule to the total number of firing strengths. The defuzzification layer computes the weighted values for every node, while the summation layer summed the results obtained from the defuzzification layer. 4 DISCUSSION OF RESULTS The section demonstrates the results obtained by the developed Jaya-Bat-based DeepCNN model and the performance is evaluated based on the PSNR, SDME, and SSIM. 4.1 Experimental arrangement The experimentation of the proposed model is done in MATLAB, operating in the PC with Windows 10 OS and 2Gb RAM. 4.2 Dataset description The experimentation is done using Statlog (Landsat satellite) dataset [29], and the description of the datasets is given below: Statlog (Landsat satellite) Dataset: It is a multivariate dataset with integer attr