Frequency Regularized Deep Convolutional Dictionary Learning and Application to Blind Denoising

1 INTRODUCTION . Sparsity in a transform domain is an important and widely applicable property of natural images . This property can be exploited in a variety of tasks such as signal representation , feature extraction , and image processing . For instance , consider restoring an image from a degraded version ( noisy , blurry , or missing pixels ) . These inverse problems are generally ill-posed and require utilizing adequate prior knowledge , for which sparsity has proven extremely effective ( Mairal et al. , 2014 ) . In recent years , such problems have been tackled with deep neural network architectures that achieve superior performance but are not well-understood in terms of their building blocks . In this study , we are interested in utilizing the knowledge from classical signal processing and spare coding literature to introduce a learned framework which is interpretable and that can perform on-par with state-ofthe-art deep-learning methods . We choose to explore this method under the task of natural image denoising , in line with much of the recent literature ( Sreter & Giryes , 2018 ; Simon & Elad , 2019 ; Lecouat et al. , 2020 ) . As a benefit of this interpretability , we are able to extend the framework for a blind-denoising setting using ideas from signal processing . In sparse representation we seek to approximate a signal as a linear combination of a few vectors from a set of vectors ( usually called dictionary atoms ) . Olshausen & Field ( 1996 ) , following a neuroscientific perspective , proposed to adapt the dictionary to a set of training data . Later , dictionary learning combined with sparse coding was investigated in numerous applications ( Mairal et al. , 2009a ; Protter & Elad , 2008 ) . More specifically , for a set ofN image patches ( reshaped into column vectors ) X = [ x1 , · · · , xN ] ∈ Rm×N , we seek to find the dictionary D∗ ∈ Rm×k and the sparse representation Z∗ = [ z∗1 , · · · , z∗N ] ∈ Rk×N such that D∗ , Z∗ = arg min D , Z N∑ i=1 ‖zi‖0 subject to : Dzi = xi , ∀i = 1 , · · · , N. ( 1 ) This formulation is not tractable for large signals since minimizing the ` 0-pseudo-norm involves a combinatorial optimization ( Natarajan , 1995 ) . To address this complication , a popular technique is to relax the problem by using the ` 1-norm as a surrogate ( Sreter & Giryes , 2018 ) . When dealing with inverse problems such as denoising , learning the dictionary from the degraded signal has shown effective ( Toić & Frossard , 2011 ) . Let yi = xi + ni ∈ Rm represent the noisy signal where ni follows an additive white Gaussian distribution , N ( 0 , σ2nI ) . Then , the relaxed formulation can be written as min D , Z N∑ i=1 ‖zi‖1 s.t . N∑ i=1 1 2 ‖Dzi − yi‖22 ≤ or minD , Z N∑ i=1 1 2 ‖Dzi − yi‖22 + λ‖zi‖1 ( 2 ) where λ is a regularization parameter and is nontrivialy related to the representation error . We will refer to this as the basis-pursuit denoising ( BPDN ) formulation of dictionary learning . Many iterative algorithms have been proposed in the literature to solve this problem ( Mairal et al. , 2014 ) . A majority of these algorithms split the problem into a step updating the dictionary followed by a step solving for the sparse codes . Note that learning a dictionary over independent image patches neglects the dependencies between these patches . As a result , the models involving patch processing are inherently sub-optimal ( Batenkov et al. , 2017 ; Simon & Elad , 2019 ) . Although enforcing local priors on merged images ( Sulam & Elad , 2015 ) and utilizing self-similarity between patches ( Mairal et al. , 2009b ) have been proposed as ideas to mitigate this flaw , ideally a global shift-invariant model is more appropriate . By constraining the dictionary to have a Toeplitz structure , the Convolutional Sparse Coding ( CSC ) model has been introduced which replaces the local patch processing with a global convolution ( Grosse et al. , 2007 ; Papyan et al. , 2017 ) . Algorithms for solving the CSC model are also discussed in ( Moreau & Gramfort , 2019 ; Wohlberg , 2017 ) . In this study , we are interested in interpretable CSC-based deep-learning models . A metric known as the mutual-coherence is well known to be related to the representation capability of the dictionary and is of special concern in using the CSC model with natural images ( Simon & Elad , 2019 ) . We take an alternative route to Simon & Elad ( 2019 ) in addressing the mutual-coherence of CSC-based deep-learning models , which is both less computationally expensive and improves the denoising performance . We continue the discussion about CSC-based deep-learning models in Sec . 1.1 . Another important aspect of the sparse representation is the sparse coding algorithm . For a given signal y ∈ Rm and dictionary D , iterative soft-thresholding algorithm ( ISTA ) ( Beck & Teboulle , 2009 ) finds the solution to the BPDN functional , z∗ = arg minz 1/2 ‖Dz − y‖ 2 2 + λ‖z‖1 , by repeating the following iteration until a convergence criterion is reached : z ( k+1 ) = Sλη ( k ) ( zk − η ( k ) DT ( Dz ( k ) − y ) ) where Sθ ( x ) = sgn ( x ) ( |x| − θ ) + , θ ≥ 0 . ( 3 ) Here , η ( k ) is the step-size of the descent algorithm at iteration k. Note that performing sparse coding with an iterative method like ISTA for all patches is computationally exhausting and slow . To resolve this issue , Gregor & LeCun ( 2010 ) proposed to approximate the sparse coding via a learned differentiable encoder , dubbed LISTA . Further extensions of LISTA both in terms of practice and theory have been studied in the literature ( Wu et al. , 2019 ; Chen et al. , 2018 ) . More recently , using LISTA combined with dictionary learning has been a research highlight ( Sreter & Giryes , 2018 ; Simon & Elad , 2019 ; Lecouat et al. , 2020 ) . We refer to this type of models that leverages LISTA for convolutional dictionary learning as CDL models . 1.1 RELATED WORKS . In this study , we are interested in the CDL model that concatenates a LISTA network with a linear convolutional synthesis dictionary . Let D be a convolutional dictionary with M filters ( and their integer shifts ) . We denote the filters in D by d j where j ∈ { 1 , · · · , M } . Let Zi denote the sparse code for the data sample yi = xi + ni where i ∈ { 1 , 2 , · · · , N } and n ∼ N ( 0 , σ2nI ) . The corresponding subband signal to d j in Zi can be denoted as z j i . Then the convolutional dictionary learning problem is written as minimize dj , Zi N∑ i=1 1 2 ‖yi − M∑ j=1 dj ∗ zji ‖ 2 2 + λ M∑ j=1 ‖zji ‖1 . ( 4 ) Sreter & Giryes ( 2018 ) introduce the approximate convolutional sparse coding ( ACSC ) framework for “ task-driven convolutional sparse coding ” , combining a convolutional extension of LISTA with a linear convolutional decoder . The proposed framework offers a strategy for training an approximate convolutional sparse coding network and a corresponding convolutional dictionary in an end-to-end fashion . They demonstrate competitive performance against classical patch-based methods such as K-SVD ( Aharon et al. , 2006 ) , on image denoising and image inpainting . Our proposed baseline model ( CDLNet ) differs from the ACSC model by use of mean-subtraction preprocessing , employing small-strided convolutions , and imposing a norm-constraint on the synthesis dictionary . Simon & Elad ( 2019 ) extend the framework of Sreter & Giryes ( 2018 ) by considering the role of stride in the stable recovery of signals and proposed the “ CSCNet ” framework . They argue that the CSC model for image representation in a sparse domain is limited by the inclusion of “ smooth filters ” , which are required to represent the piecewise smooth characteristics of natural images . This limitation manifests itself in the maximum cross-correlation between atoms of the dictionary , known as the mutual-coherence . They empirically show that using relatively large stride , while processing shifted-duplicates of the input , improves denoising performance of the model . Although using large stride reduces the mutual coherence of the learned filters , all possible shifts of the image need to be processed and averaged , yielding a model very similar to patch-processing . We propose a frequency regularization strategy to mitigate the problem of smooth-varying filters which does not require shift-averaging . Note that the parameter λ in equation 4 depends on the desired sparsity , relative to the noise-level , and is directly related to the threshold values in ISTA . Sreter & Giryes ( 2018 ) propose to learn different thresholds for each channel , effectively changing the regularizer term in equation 4 to∑M j=1 ‖λjz j i ‖1 . Inspired by the benefit of minimax-concave ( MC ) penalty ( Selesnick , 2017 ) over ` 1 norm , Pokala et al . ( 2020 ) propose “ ConFirmNet ” where firm-thresholding function is used in the network . Kim & Park ( 2020 ) propose a signal adaptive threshold scheme for LISTA where the threshold is decreased if the previous estimate of an element is large . Mohan et al . ( 2020 ) explore the role of bias-vectors in popular deep-learning network ’ s convolution operators . They advocate for eliminating the biases completely to improve generalization in blinddenoising where there is mismatch between training and inference noise level . Isogawa et al . ( 2017 ) propose altering the biases of deep neural-networks by scaling them with the input noise standarddeviation . Their method is ultimately a non-blind denoising scheme as they use the ground-truth noise statistics during training and inference . In contrast , we propose a blind-denoising scheme that is motivated by the interpretation of the biases in LISTA as thresholds and employ a scaling by the noise variance ( in the last layer of LISTA ) , estimated from the input signal during training and inference . Performance of different denoising techniques on other noise distributions have also been studied in the literature , which is not the focus of this study ( Abdelhamed et al. , 2018 ; Plotz & Roth , 2017 ) . 1.2 CONTRIBUTION OF THIS STUDY . The unrolled convolutional sparse coding and dictionary learning frameworks have led to the field dubbed “ interpretable deep-learning ” . The networks constructed in such a way have the benefit of interpretability and decreased parameter count while performing quite closely to other state-of-theart deep-learning models . In this study we further extend such frameworks . We propose utilizing a strided convolutional dictionary with a fixed low-pass channel and a set of frequency-regularized learnt filters ( Section 2.2 ) . Our experimental results demonstrate that such frequency regularization and small stride leads to more interpretable dictionary filters than the prior work . Consequently , by limiting the number of low-pass atoms in the dictionary and using small-strided convolutions , we address the modeling assumptions associated with the convolutional sparse coding model ( Section 2.1.1 ) . Additionally , leveraging interpretability of our network , we propose to parameterize the softthresholding operator in LISTA such that the thresholds are proportional to the estimated input noiselevel for a given image ( Section 2.3 ) . Experimentally , we show improved denoising performance at reduced computational complexity compared to other frameworks ( Section 3.2 ) . Furthermore , our parameterization of the learned thresholds greatly improves robustness to noise-level mismatch between training and inference and increases the generalizability of the network ( Section 3.3 ) .

Summary: The paper proposes a new regularization for the dictionary in the learned convolutional sparse coding model of Sreter & Giryes '18. The main contribution is that the dictionary is regularized to be composed of 1) a fixed low-pass filter and 2) a set of learned filters to occupy the complementary high-frequency space. A second contribution is that the thresholding in the network is adjustable according to the estimated noise level in the image.