Frequency Regularized Deep Convolutional Dictionary Learning and Application to Blind Denoising

1 INTRODUCTION . Sparsity in a transform domain is an important and widely applicable property of natural images . This property can be exploited in a variety of tasks such as signal representation , feature extraction , and image processing . For instance , consider restoring an image from a degraded version ( noisy , blurry , or missing pixels ) . These inverse problems are generally ill-posed and require utilizing adequate prior knowledge , for which sparsity has proven extremely effective ( Mairal et al. , 2014 ) . In recent years , such problems have been tackled with deep neural network architectures that achieve superior performance but are not well-understood in terms of their building blocks . In this study , we are interested in utilizing the knowledge from classical signal processing and spare coding literature to introduce a learned framework which is interpretable and that can perform on-par with state-ofthe-art deep-learning methods . We choose to explore this method under the task of natural image denoising , in line with much of the recent literature ( Sreter & Giryes , 2018 ; Simon & Elad , 2019 ; Lecouat et al. , 2020 ) . As a benefit of this interpretability , we are able to extend the framework for a blind-denoising setting using ideas from signal processing . In sparse representation we seek to approximate a signal as a linear combination of a few vectors from a set of vectors ( usually called dictionary atoms ) . Olshausen & Field ( 1996 ) , following a neuroscientific perspective , proposed to adapt the dictionary to a set of training data . Later , dictionary learning combined with sparse coding was investigated in numerous applications ( Mairal et al. , 2009a ; Protter & Elad , 2008 ) . More specifically , for a set ofN image patches ( reshaped into column vectors ) X = [ x1 , · · · , xN ] ∈ Rm×N , we seek to find the dictionary D∗ ∈ Rm×k and the sparse representation Z∗ = [ z∗1 , · · · , z∗N ] ∈ Rk×N such that D∗ , Z∗ = arg min D , Z N∑ i=1 ‖zi‖0 subject to : Dzi = xi , ∀i = 1 , · · · , N. ( 1 ) This formulation is not tractable for large signals since minimizing the ` 0-pseudo-norm involves a combinatorial optimization ( Natarajan , 1995 ) . To address this complication , a popular technique is to relax the problem by using the ` 1-norm as a surrogate ( Sreter & Giryes , 2018 ) . When dealing with inverse problems such as denoising , learning the dictionary from the degraded signal has shown effective ( Toić & Frossard , 2011 ) . Let yi = xi + ni ∈ Rm represent the noisy signal where ni follows an additive white Gaussian distribution , N ( 0 , σ2nI ) . Then , the relaxed formulation can be written as min D , Z N∑ i=1 ‖zi‖1 s.t . N∑ i=1 1 2 ‖Dzi − yi‖22 ≤ or minD , Z N∑ i=1 1 2 ‖Dzi − yi‖22 + λ‖zi‖1 ( 2 ) where λ is a regularization parameter and is nontrivialy related to the representation error . We will refer to this as the basis-pursuit denoising ( BPDN ) formulation of dictionary learning . Many iterative algorithms have been proposed in the literature to solve this problem ( Mairal et al. , 2014 ) . A majority of these algorithms split the problem into a step updating the dictionary followed by a step solving for the sparse codes . Note that learning a dictionary over independent image patches neglects the dependencies between these patches . As a result , the models involving patch processing are inherently sub-optimal ( Batenkov et al. , 2017 ; Simon & Elad , 2019 ) . Although enforcing local priors on merged images ( Sulam & Elad , 2015 ) and utilizing self-similarity between patches ( Mairal et al. , 2009b ) have been proposed as ideas to mitigate this flaw , ideally a global shift-invariant model is more appropriate . By constraining the dictionary to have a Toeplitz structure , the Convolutional Sparse Coding ( CSC ) model has been introduced which replaces the local patch processing with a global convolution ( Grosse et al. , 2007 ; Papyan et al. , 2017 ) . Algorithms for solving the CSC model are also discussed in ( Moreau & Gramfort , 2019 ; Wohlberg , 2017 ) . In this study , we are interested in interpretable CSC-based deep-learning models . A metric known as the mutual-coherence is well known to be related to the representation capability of the dictionary and is of special concern in using the CSC model with natural images ( Simon & Elad , 2019 ) . We take an alternative route to Simon & Elad ( 2019 ) in addressing the mutual-coherence of CSC-based deep-learning models , which is both less computationally expensive and improves the denoising performance . We continue the discussion about CSC-based deep-learning models in Sec . 1.1 . Another important aspect of the sparse representation is the sparse coding algorithm . For a given signal y ∈ Rm and dictionary D , iterative soft-thresholding algorithm ( ISTA ) ( Beck & Teboulle , 2009 ) finds the solution to the BPDN functional , z∗ = arg minz 1/2 ‖Dz − y‖ 2 2 + λ‖z‖1 , by repeating the following iteration until a convergence criterion is reached : z ( k+1 ) = Sλη ( k ) ( zk − η ( k ) DT ( Dz ( k ) − y ) ) where Sθ ( x ) = sgn ( x ) ( |x| − θ ) + , θ ≥ 0 . ( 3 ) Here , η ( k ) is the step-size of the descent algorithm at iteration k. Note that performing sparse coding with an iterative method like ISTA for all patches is computationally exhausting and slow . To resolve this issue , Gregor & LeCun ( 2010 ) proposed to approximate the sparse coding via a learned differentiable encoder , dubbed LISTA . Further extensions of LISTA both in terms of practice and theory have been studied in the literature ( Wu et al. , 2019 ; Chen et al. , 2018 ) . More recently , using LISTA combined with dictionary learning has been a research highlight ( Sreter & Giryes , 2018 ; Simon & Elad , 2019 ; Lecouat et al. , 2020 ) . We refer to this type of models that leverages LISTA for convolutional dictionary learning as CDL models . 1.1 RELATED WORKS . In this study , we are interested in the CDL model that concatenates a LISTA network with a linear convolutional synthesis dictionary . Let D be a convolutional dictionary with M filters ( and their integer shifts ) . We denote the filters in D by d j where j ∈ { 1 , · · · , M } . Let Zi denote the sparse code for the data sample yi = xi + ni where i ∈ { 1 , 2 , · · · , N } and n ∼ N ( 0 , σ2nI ) . The corresponding subband signal to d j in Zi can be denoted as z j i . Then the convolutional dictionary learning problem is written as minimize dj , Zi N∑ i=1 1 2 ‖yi − M∑ j=1 dj ∗ zji ‖ 2 2 + λ M∑ j=1 ‖zji ‖1 . ( 4 ) Sreter & Giryes ( 2018 ) introduce the approximate convolutional sparse coding ( ACSC ) framework for “ task-driven convolutional sparse coding ” , combining a convolutional extension of LISTA with a linear convolutional decoder . The proposed framework offers a strategy for training an approximate convolutional sparse coding network and a corresponding convolutional dictionary in an end-to-end fashion . They demonstrate competitive performance against classical patch-based methods such as K-SVD ( Aharon et al. , 2006 ) , on image denoising and image inpainting . Our proposed baseline model ( CDLNet ) differs from the ACSC model by use of mean-subtraction preprocessing , employing small-strided convolutions , and imposing a norm-constraint on the synthesis dictionary . Simon & Elad ( 2019 ) extend the framework of Sreter & Giryes ( 2018 ) by considering the role of stride in the stable recovery of signals and proposed the “ CSCNet ” framework . They argue that the CSC model for image representation in a sparse domain is limited by the inclusion of “ smooth filters ” , which are required to represent the piecewise smooth characteristics of natural images . This limitation manifests itself in the maximum cross-correlation between atoms of the dictionary , known as the mutual-coherence . They empirically show that using relatively large stride , while processing shifted-duplicates of the input , improves denoising performance of the model . Although using large stride reduces the mutual coherence of the learned filters , all possible shifts of the image need to be processed and averaged , yielding a model very similar to patch-processing . We propose a frequency regularization strategy to mitigate the problem of smooth-varying filters which does not require shift-averaging . Note that the parameter λ in equation 4 depends on the desired sparsity , relative to the noise-level , and is directly related to the threshold values in ISTA . Sreter & Giryes ( 2018 ) propose to learn different thresholds for each channel , effectively changing the regularizer term in equation 4 to∑M j=1 ‖λjz j i ‖1 . Inspired by the benefit of minimax-concave ( MC ) penalty ( Selesnick , 2017 ) over ` 1 norm , Pokala et al . ( 2020 ) propose “ ConFirmNet ” where firm-thresholding function is used in the network . Kim & Park ( 2020 ) propose a signal adaptive threshold scheme for LISTA where the threshold is decreased if the previous estimate of an element is large . Mohan et al . ( 2020 ) explore the role of bias-vectors in popular deep-learning network ’ s convolution operators . They advocate for eliminating the biases completely to improve generalization in blinddenoising where there is mismatch between training and inference noise level . Isogawa et al . ( 2017 ) propose altering the biases of deep neural-networks by scaling them with the input noise standarddeviation . Their method is ultimately a non-blind denoising scheme as they use the ground-truth noise statistics during training and inference . In contrast , we propose a blind-denoising scheme that is motivated by the interpretation of the biases in LISTA as thresholds and employ a scaling by the noise variance ( in the last layer of LISTA ) , estimated from the input signal during training and inference . Performance of different denoising techniques on other noise distributions have also been studied in the literature , which is not the focus of this study ( Abdelhamed et al. , 2018 ; Plotz & Roth , 2017 ) . 1.2 CONTRIBUTION OF THIS STUDY . The unrolled convolutional sparse coding and dictionary learning frameworks have led to the field dubbed “ interpretable deep-learning ” . The networks constructed in such a way have the benefit of interpretability and decreased parameter count while performing quite closely to other state-of-theart deep-learning models . In this study we further extend such frameworks . We propose utilizing a strided convolutional dictionary with a fixed low-pass channel and a set of frequency-regularized learnt filters ( Section 2.2 ) . Our experimental results demonstrate that such frequency regularization and small stride leads to more interpretable dictionary filters than the prior work . Consequently , by limiting the number of low-pass atoms in the dictionary and using small-strided convolutions , we address the modeling assumptions associated with the convolutional sparse coding model ( Section 2.1.1 ) . Additionally , leveraging interpretability of our network , we propose to parameterize the softthresholding operator in LISTA such that the thresholds are proportional to the estimated input noiselevel for a given image ( Section 2.3 ) . Experimentally , we show improved denoising performance at reduced computational complexity compared to other frameworks ( Section 3.2 ) . Furthermore , our parameterization of the learned thresholds greatly improves robustness to noise-level mismatch between training and inference and increases the generalizability of the network ( Section 3.3 ) .

Summary: The paper proposes a denoising method with a neural network inspired from convolutional dictionary learning. In the proposed method, one atom of the dictionary is constrained to be a low frequency filters and all other atoms are to be high frequency atoms. The authors also propose to make the threshold depends on the noise level to better adapt to different noise level and to use strided convolution to reduce the computational cost of the method. The method is then evaluated on images from BSD68.