SAR image despeckling using deep CNN

IET Image ProcessingVolume 15, Issue 6 p. 1285-1297 ORIGINAL RESEARCH PAPEROpen Access SAR image despeckling using deep CNN Alicia Passah, Alicia Passah Department of Information Technology, North-Eastern Hill University, Shillong, MeghalayaSearch for more papers by this authorKhwairakpam Amitab, Khwairakpam Amitab Department of Information Technology, North-Eastern Hill University, Shillong, MeghalayaSearch for more papers by this authorDebdatta Kandar, Corresponding Author Debdatta Kandar dkandar@nehu.ac.in Department of Information Technology, North-Eastern Hill University, Shillong, Meghalaya Correspondence Debdatta Kandar, Department of Information Technology, North-Eastern Hill University, Shillong, Meghalaya. Email: dkandar@nehu.ac.inSearch for more papers by this author Alicia Passah, Alicia Passah Department of Information Technology, North-Eastern Hill University, Shillong, MeghalayaSearch for more papers by this authorKhwairakpam Amitab, Khwairakpam Amitab Department of Information Technology, North-Eastern Hill University, Shillong, MeghalayaSearch for more papers by this authorDebdatta Kandar, Corresponding Author Debdatta Kandar dkandar@nehu.ac.in Department of Information Technology, North-Eastern Hill University, Shillong, Meghalaya Correspondence Debdatta Kandar, Department of Information Technology, North-Eastern Hill University, Shillong, Meghalaya. Email: dkandar@nehu.ac.inSearch for more papers by this author First published: 18 January 2021 https://doi.org/10.1049/ipr2.12104Citations: 1AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Synthetic aperture radar (SAR) images are contaminated with noise called speckle that is multiplicative in nature. The presence of speckles in SAR images makes it impossible to understand and interpret for extensive range of applications. However, certain characteristics of SAR and the ability to function irrespective of weather conditions, are making such images worth processing in order to be able to extract relevant information. A despeckling model is proposed that uses deeper convolutional neural networks, which was never used before, as far as authors are concerned, for diminishing speckle in noisy SAR images. Multiple skip connections from the ResNet model are also employed in authors' proposed architecture. In order to maintain uniformity, a formula to be followed while applying skip connections is also derived. A hybrid loss function is developed to train the network more consistently to achieve the desired output. Experiments on simulated SAR images using the NWPU-RESISC benchmark are conducted and tested on real TerraSAR-X images and compared with the state-of-the-art techniques. Results show that the proposed method achieved considerable improvements compared to state-of-the-art methods with PSNRs 27.02, 24.60, and 22.01 for looks 10, 4, and 1, respectively. 1 INTRODUCTION Synthetic aperture radar is an all-weather remote sensing imaging radar used to construct two-dimensional images or three-dimensional recreation of objects present on the ground [1]. The ability of SAR to function in all types of weather circumstances has made it popular in diverse applications like military defense, disaster management, pollution detection, illegal mining detection, battlefield surveillance, exploration of resources, traffic monitoring, urban planning etc. [2]. The SAR modality has a number of characteristics that distinguish it from optical imagery. It is an active imaging system, in the sense that it has its own illumination, and is generally mounted on a spacecraft or aircraft and even in guided missiles these days. The SAR system captures the image by transmitting electromagnetic waves and processing the received echo to reconstruct the image of the target. These reflected echos are mostly interfered by signals resulting from uneven surfaces and mutual coupling arrays causing the system to add vague and irrelevant information termed as speckle noise on the reconstructed image. It may also be mentioned that the performance of a SAR radar can be characterised by its target detectability and imaging quality. But in the real environment, SAR's performance is greatly affected by the multi-path effect and clutter. To encounter such issues, researchers are looking for the development of low probability of intercept (LPI) signals [3], and such signals are characterised by their auto-correlation function or ambiguity function. As per studies, the desired signal should have a high peak side-lobe level (PSL) ratio and gradually decreasing side lobe level [4]. At the same time, one of the promising solutions is to have multiple antenna technologies like multiple input multiple output (MIMO) or phased array to encounter such environmental effects [5, 6]. However, collocated antenna systems give rise to mutual coupling issue between array elements, which ultimately degrades the performance of a SAR radar. Due to the coupling effect, the side lobes in the auto-correlation function gets perturbed leading to the reduction in PSL values, which affects the target detectability of SAR systems. The undesired signals resulting from array element mutual coupling degrades the measured data acquired by SAR radar. Hence the images produced from such corrupted signals are contaminated with unwanted effects such as artifacts and blurring [7, 8]. One possible solution to reduce this problem is by employing a distributed antenna arrangement, which may be at the cost of coherent signal processing gain. The collocated antenna can be arranged by taking advantage of the benefits of coherent processing. However, in this work, our primary focus is on the SAR image itself rather than the process of capturing it. Therefore, It may be worth mentioning that the images obtained by SAR inherently contain speckle noise, which ultimately degrades the quality and features of the images, thereby making the interpretation and understandings of SAR images difficult and therefore, a challenging task. The importance of SAR have led researchers to incorporate modern techniques like deep learning to improve the quality of images acquired by SAR. Hence, image denoising is still a dynamic subject since it is an essential step in a wide range of viable applications. Considering the presence of noise in SAR images, many researchers have proposed several filtering techniques, of which most of them are based on statistical methods such as Lee filter [9], Kuan filter [10], Frost filter [11], BM3D [12], SAR-BM3D [13]. However, due to their local processing nature, these methods resulted in blurring or artifacts and therefore failed to preserve essential features for subsequent processing [14]. Lately, deep learning have been known popularly for its ability to show certain improvements in diverse applications related to classification [15, 16], detection [17-19], segmentation [20, 21]. The use of deep learning has no doubt reduce the work load as it eliminates the process of handcrafting the necessary features for processing. This is because deep learning is known to automatically extract relevant features without the need of human intervention [22]. This has led researchers to diverge into deep learning and explore its application in SAR interpretation because handcrafting features from SAR images is nearly impossible as even humans could not recognise the images properly due to the presence of speckle noises. Few works on SAR interpretation using deep learning includes [23-29]. These works are all based on deep learning comprising of the convolutional neural network (CNN) specifically designed for SAR purposes and resulted in better accuracies as compared to conventional menthods. The improvements shown in these works have urged researchers in recent years to positively apply deep learning, particularly CNNs in SAR denoising as well. The performance of a CNN model usually depends on the way the network is tuned based on its configurations, which include the depth, number of units in each depth, form of non-linear functions, type of optimiser used, and hyperparameters such as learning rate, momentum, decay etc. Few state-of-the-art SAR denoising methods worth mentioning include the SAR-CNN [30] that first transformed the SAR image into a logarithmic space and then mapped it to a denoised image via a 17-layered CNN. Another similar work, referred to as SAR-DRN [31] uses dilated convolutions to learn a non-linear mapping between the noisy and clean SAR images. SAR-DRN also uses the skip connections to prevent the vanishing gradient problem and to retain relevant image information. ID-CNN [32] is another recent work whose architecture uses a division residual layer to calculate noise. It is an end-to-end based architecture that considers multiplicative noise without having to shift the image to a logarithmic space. These works have shown improvements in the field of SAR image denoising, and it may be observed that these works have inherited the properties of the famous ResNet architecture [33]. Motivated by these works, we have proposed a denoising architecture by exploring more on the behavior of ResNet and its usefulness in the application of SAR image despeckling. The following objectives are the main contributions of this paper. A deep CNN based end-to-end architecture for denoising SAR images is proposed. Quite a number of the famous ResNet's skip connections are utilised in the architecture so as to preserve the relevant features of the image and to prevent gradients from vanishing during backpropagation. A formula pertaining to the use of skip connections in the architecture is developed. A hybrid loss function is also developed and adopted in training the proposed model. The rest of the paper is organised as follows. Section 2 discusses the related works, which include the speckle noise, CNN, ResNet and state-of-the-art techniques used for denoising. Section 3 represents the proposed method describing the architecture and configurations used for denoising. Section 4 shows the experimental results and comparisons. Section 5 discusses the conclusion and future plan of work. 2 RELATED WORK In this section, the literature review related to denoising of SAR-images is presented. Firstly, we described the speckle noise, followed by the description of the convolutional neural network and ResNet. Finally, we present the relevant state-of-the-art techniques used for SAR image denoising. 2.1 Speckle noise In this sub-section, a brief explanation of the type of noise referred to as speckle, which is dominant in SAR imagery, is discussed. Speckle noises are the grainy features present in images that occurs due to the influence of the rough surfaces of various ground targets on the electromagnetic waves. Also, the ability of SAR to operate in any weather conditions like rain, lightning, fog etc. makes it prone to speckle noises. In other words, the reflected echo may be out of phase depending on the roughness of the target surface and the path in which the echo travels before reaching the SAR receiver. The existence of speckle noise in image acquired by SAR, causes difficulty in understanding the various aspects of the image, which in turn, resulted in performance deterioration for target surveillance. To be specific, the type or nature of the target in SAR images is hard to be examined. The existence of speckle noise in SAR images, however, makes it difficult to interpret or process them. Therefore, it becomes necessary to suppress the noise in SAR images in order to enhance the performance in various computer vision applications, as this may, in turn, lead to loss of relevant features from an image followed by an inaccurate prediction of targets. Therefore, speckle reduction in SAR data becomes challenging and is still an open field of research. The speckle that is present in SAR image is said to be multiplicative in nature [34], and the model of the noise is expressed as. y = n × x , (1)where x and y is a clean image and a noisy image, respectively, and n is the estimated speckle noise that follows a gamma distribution. Figure 1 shows a pictorial representation of Equation (1). It can be observed that the noisy image y results from multiplying the noise signal n to a clean image x. FIGURE 1Open in figure viewerPowerPoint Visual representation of the noise model in SAR image It is notable that the speckle in SAR images follows a G a m m a distribution [35], having probability density function as ρ ( n ) = L L n L − 1 exp ( − n L ) Γ ( L ) , (2)where n ≥ 0, L ≥ 1 and Γ ( . ) specifies the G a m m a function and L denotes the average number of looks that constitutes a SAR image, expressed as Equation (3). L = m e a n v a r i a n c e . (3) 2.2 Convolutional neural network In this subsection, one of the most attractive deep learning networks called convolutional neural networks, along with its different levels, are explained. CNN is a class of neural networks commonly used in computer vision applications. It is a deep learning network whose architecture consists typically of the following layers namely the input, convolutional, subsampling, fully connected, and the output. It is basically a two-stage process where the first stage comprising the convolutional layers and down sampling layers are mainly used for the extraction of relevant features from the given inputs, whereas, the second stage comprising of the multilayer perceptron or fully connected layer is basically a classifier. CNNs have shown promising outcomes in various fields such as image recognition [36, 37], classification [25, 26], object detection [38], face recognition etc., and require less pre-processing as compared to other algorithms [39]. The popularity of CNNs has prompted researchers to apply it in denoising techniques as well. Ongoing investigations that use CNN to denoise SAR images resulted in compelling performances as compared to other conventional filtering techniques [30, 32]. Thus, with this motivation, we proposed a CNN-based architecture to enhance the clarity of SAR images by eliminating the speckle noise. It is also noteworthy to highlight that the configurations of CNN models differ depending on the type of application it is meant for. Next, we briefly discuss the different basic computational layers of a CNN in the following subsection. 2.2.1 Computational layers of CNN The architecture of a CNN is organised as layers, whereby each layer performs computational operations on the inputs. A pictorial representation of a basic CNN architecture is presented in Figure 2. An input image is usually convolved to a set of feature maps with the help of convolutional layers whose dimensions are reduced using pooling layers. The final feature information is finally aggregated using the fully connected layer resulting in the final output. Some of the generally used mathematical models in the computational layers are discussed in the following [36]. FIGURE 2Open in figure viewerPowerPoint Basic CNN architecture [40] Consider an image x ε R h × w × c , where h is the height, w the width, and c is the number of channels. This image x along with a set of parameters W, acts as input to a block in order to generate a new image y ε R h ′ × w ′ × c ′ as output, that is, y = f ( x , W ) . The entire process of generating the output goes through the following mathematical computations at respective layers. 1) Convolution layer: This layer generates the convolution of the input x with a bank of filters W ε R h ∼ × w ∼ × c ∼ × c ′ , also called kernels, and adds a bias b ε R c ′ [41]. This computation can be formulated as y i ′ j ′ k ′ = f b k ′ + ∑ i = 1 h ∼ ∑ j = 1 w ∼ ∑ d = 1 c W i j d k ′ × x i ′ + i , j ′ + j , d , (4)where f ( . ) indicates a nonlinear activation function for which in the case of rectified linear unit (ReLU) [42] y i j k = m a x { 0 , x i j k } . (5) 2) Pooling or subsampling layer: This layer is used to downsample the output of the convolutional layer along each image channel in a h ∼ × w ∼ subwindow. One of such subsampling method is the max pooling method which can be written as [43]. y i ′ j ′ k = m a x 1 < i < h ∼ , 1 < j < w ∼ x i ′ + i , j ′ + j , k . (6) 3) Softmax layer:This layer, when applied, is usually used to implement multi-class logistic regression mainly for classification purposes and is primarily implemented just before the output layer as follows [44]. y i j k = e x p ( x i j k ) ∑ d = 1 c e x p ( x i j d ) . (7) 4) Loss function: Given the true label class l for an input x, this function is used to compute the error with respect to the output of the network, and based on this, the network parameters are re-adjusted until the error is reduced. One such function is the log loss function which is computed as [45]. y = − ∑ i = 1 h ∑ j = 1 w log x i j l . (8) 2.3 ResNet or residual learning One of the most pivotal works of recent years in computer vision called ResNet [33] is briefly explained in this section. ResNet, which is an acronym for Residual Network, is one of the known architectures of deep learning introduced in the year 2015. This model could train a network with 152 layers while at the same time it could handle the vanishing gradient issue, which normally occurs in deeper networks. The ability of ResNet to cope the vanishing gradient issue is because of the skip connections introduced in its architecture [33]. The skip connections also known as identity mappings does not require extra parameters. Instead, it only combines the output of the previous layer to the forthcoming layers. Figure 3 shows the basic building block of a ResNet model. As seen from Figure 3, the activation from the preceding layer is being appended to the activation of the next layer, and this small connection enabled ResNet to train deeper networks. ResNet has therefore, proved its capability to train deeper networks with less complexity compared to other architectures such as VggNet [46]. FIGURE 3Open in figure viewerPowerPoint Building block of ResNet [33] ResNet has recently been applied to various image denoising models such as ID-CNN [32], SAR-CNN [30], SAR-DRN [31]. The application of ResNet in these works have shown considerable improvement in performance, and this has motivated us towards exploring the ResNet along with its novel skip connections for possible application in SAR image denoising. The works in [30, 32, 47] have also utilised the skip connection from the ResNet architecture. However, they did not take full advantage of the characteristics offered by the traditional ResNet. 2.4 Existing noise filtering techniques In this subsection, we highlighted some of the noise filtering methods proposed in the literature. Concerning the existence of noise in images, many researchers have come up with several noise removing techniques from statistical to deep learning approaches. Lee filter [9] is a filtering technique that calculates the central pixel value based on the statistical distribution of the pixel values of a moving kernel. The lee filtering method works based on variance, in the sense that it performs smoothing operation only when the variance of the intended area is low, resulting in the ability to smooth noise and preserve edges adaptively. The filter results in blurring and may bring about the loss of detailed target information when applied to SAR images [48]. Another statistical method named Kuan filter [10] converts the noise model from multiplicative to additive model, and it resembles the Lee filter, except that it uses an alternate function for weighting. The filtered pixel replacement value is calculated based on the local statistics. Due to the local processing nature, the Kuan filter also results in artifacts and blurring, which, when applied to SAR images, might misinterpret its meaning [48]. Frost filter [11] is used to reduce noise, retaining the edge, and shape features by utilising exponentially damped convolution kernel. The trade-off of using a high value of damp factors resulted in better retaining of sharp edges. However, the noise reduction is less and vice versa. This trade-off makes Frost filter challenging when applied to SAR images. Another method named NLMeans [49] was brought forward, which follows a selective data-driven way of setting the weights by taking the average weight of the values of resembling pixels. The calculation of similar pixels is based upon the distance between the plot surrounding the central pixel and the plot surrounding a given nearby pixel. Due to the involvement of similar patch searching, the NLMeans filter has low computational efficiency resulting in its narrow application [31]. The BM3D [12] is a statistical image denoising method that clubs patches of an image into 3D arrays based on their resemblance and does combine filtering, thereby resulting in the estimation of 2D images for these grouped patches. SAR-BM3D [13] adopts the technique of BM3D for SAR image denoising by modifying the way of checking image patch resemblance based on probabilistic measure and taking into consideration the noise model, suppression of noise is performed based on the local linear minimum mean squared error. While the mechanism in [9, 10, 49] resulted in blurring and generation of artifacts, the techniques in [12, 13], are computationally intensive and therefore are less recommended for SAR systems [50]. Recent works using deep learning include the SAR-CNN [30], which map a SAR image to the corresponding noise image via a 17-layered CNN model. The predicted noise is then subtracted from the input image to obtain the filtered clean output. This technique, however, is a two-step process because the inputs are first log-transformed, and the output is then exponentiated back to the original image space. In comparison to SAR-BM3D, the SAR-CNN is capable of preserving more details in the despeckled image [30]. Another deep learning approach proposed for SAR image denoising is the SAR-DRN [31], which is a 7-layered network that uses dilated convolutions in place of the normal convolutions. It employs three skip connections: one from the first layer to the third layer and second from the fourth layer to the seventh layer, while the third one from the input directly to the output of the last layer. The SAR-DRN utilises the skip connections in order to prevent vanishing gradient and to maintain image details for better predictions. The key concept of SAR-DRN is the use of dilated convolutions resulting in larger receptive fields, preserving the size of the filter and the depth of the model. Though SAR-DRN outperforms other filters like SAR-BM3D, yet it produced artifacts contaminated images [50]. DnCNN [47] is another deep learning approach for image denoising that consists of 17 convolutional layers designed to handle Gaussian noise. It is an end-to-end mapping from the noisy image to the residual image irrespective of the gaussian noise model. The DnCNN incorporates residual learning, whereby the predicted residue is subtracted from the noisy input to generate the filtered image. The DnCNN, however requires two steps in producing the filtered image output as its model does not directly output the denoised image. One of the recent work named ID-CNN [32] was proposed for SAR image denoising whereby an 8-layered CNN is used to learn an end-to-end mapping from the noisy input to a clean output by strictly obeying multiplicative speckle model. Considering the noise model, the ID-CNN uses the division residual layer to divide the noisy input by the output of the last layer in order to predict a clean image. In addition to this, the ID-CNN combines the euclidean loss along with total variation loss as its loss function to train the model, and this resulted in the improvement of denoising accuracy. The ID-CNN, have shown better performance as compared to the above-mentioned techniques [32]. Table 1 shows some of the major properties of few existing deep learning based works on SAR image denoising. To further improve the performance of the state-of-art techniques, we proposed a model that uses a deeper network along with multiple skip connections and a hybrid loss function that are discussed in the next section. TABLE 1. Major properties of the existing deep-learning networks used for SAR image denoising SAR-CNN SAR-DRN ID-CNN Architecture 17-Layered 7-Layered 8-Layered No. of SC One Three One Loss function Custom MSE E + TV Type of convolution Normal Dilated Normal Noise model M A M Input log transform Yes No No E = euclidean, TV = total variation, M = multiplicative, A = additive, SC = skip connections 3 PROPOSED METHOD In this section, we present our proposed CNN for denoising of SAR images. Firstly, the architecture that we consider for our work incorporates the ResNet architecture, as it can be best used when a model needs to learn as many useful and relevant features as it can, especially when these features gets distorted in the process, as in the case of denoising of images, whereby important features may get suppressed along with the unwanted speckles. Therefore, our network enables the re-concatenation of the noise information to the later layers while retaining the image information with the help of skip connections. Also, these connections act as gradient highways when weights adjustment are back-propagated to the earlier layers resulting in faster convergence and non-vanishing gradient. The detailed architecture of the proposed model is discussed in the next subsection, followed by a discussion on the skip connections used in our architecture. 3.1 Proposed architecture Deeper networks are known to be significantly more proficient in creating deep representations by learning more abstract information about the input with the help of stacked layers. The benefit of a deep network is that it can learn a hierarchy of simple to sophisticated features. Therefore deeper the network, better is the feature portrayal. Also, considering the ImageNet [51] competitions' results, the winning networks became deeper and deeper from initially 8 layers [52] to lately 152 layers [33]. The competition has proved the capability of deeper networks and hence can be applied to various applications. The application of neural networks in SAR image denoising, deeper convolutional networks have not been explored despite its several benefits. Therefore in our proposed architecture, we have used deeper networks for denoising and the proposed method have shown considerable improvement in performance as compared to state-of-the-art techniques. The works in [30-32] have also used the famous property of ResNet called skip connections in their architectures but did not take advantage of the properties offered by ResNet in the use of numerous skip connections for deeper networks by considering deeper architectures. Therefore, the architecture of the proposed model consists of 24 convolutional layers with 64 feature maps Convolved (Conv) by a 3 × 3 kernel window. However, we have also incorporated 12 skip connections that skips layers based on a formula we derived. The use of this traditional skip connection approach enables the model to learn the residue without forgetting the relevant features or information in order to be able to predict a desired clear image. Figure 4 shows the proposed architecture of our method. The architecture is made up of two types of layers: Type I : Conv + ReLU Type II : Conv + BN + ReLU FIGURE 4Open in figure viewerPowerPoint Proposed architecture depicting the CNN layers with skip connections The first layer is made of Type I, and based on the strategy in [46], the kernel of size 3 × 3 is used for convolving the input to 64 features map. The 22 intermediate layers are made of Type II with 64 feature maps convolved by 3 × 3 kernels. The last layer is again made of Type I and works as a fully connected layer resulting in a single feature map with the help of a 3 × 3 kernel. In order to make the dimensions of the input and the output of each layer equivalent, we have added borders of zeros to every input of a convolutional layer. Turning to the activation function, the most used non-linear activation function in recent years is the ReLU [42] because of its simplicity and its linear-like behaviour that enables a network to be easily optimised. Therefore, we have also opted ReLU to be the activation function in every layer of our proposed network. To enable the layers in our network to independently learn a bit more by itself irrespective of what it learns from the previous layers, we have also used Batch Normalization (BNorm) [53] before every non-linear activation, except in the first and the last layer. 3.2 Skip connections With the increase in the depth of the proposed network, we have also incorporated skip connections in our network in order to prevent the gr