Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: ## Summary

The paper proposes a denoising method with a neural network inspired from convolutional dictionary learning. In the proposed method, one atom of the dictionary is constrained to be a low frequency filters and all other atoms are to be high frequency atoms. The authors also propose to make the threshold depends on the noise level to better adapt to different noise level and to use strided convolution to reduce the computational cost of the method. The method is then evaluated on images from BSD68.

----

*For extra citations, see bibtex at the end. Sorry if I ended up putting a lot of them but I feel the bibliography is a bit lacking.*

## Overall assessment

- A major weakness of this work is that it seems to be equivalent to doing CDL on a signal filtered with high pass filter $g^{-1} * y_i$ because $\theta_0 = 0$. Indeed, using parseval on the data fitting term and the equivalence between convolution and point wise multiplication in the frequency space, one can easily show that $z^1_i$ will correspond to the low frequency part of $y_i$ and as the filters $d^i$ all only have high frequency, they will try to reconstruct the high frequency part of $y_i$. This means that only keeping high frequency of $y_i$, one can use the classical CDL approach algorithm to recover the same method as the one proposed in this paper. To me, the method boils down to stating an equivalent model where the preprocessing is integrated in the model. If the authors think I am wrong, they could try to perform experiment 3.1 and show that there is significant difference between filtering $y$ and using the proposed method. At least, this point should be mentionned and discussed in the manuscript. Note that such approach of integrating multiple component can be related to the work on morphological component analysis (see `[Elad2005]`) and the integration of preprocessing step in CDL have recently been proposed with detrending in `[Lalanne2020]`.
- Apart from this point, the novelty of this work is not very significant. The adaptation of the thresholds with the input noise level in the context of denoising has been proposed in `[Isogawa2017]` and `[Ramzi2020]` and  the use of strided convolution are proposed in `Simon & Elad (2019)`.
- The effect of the stride on the denoising is not evaluated. This would be interesting to look at how the denoising performance change when changing the stride. In particular, how does FCDLNet compare with FCDLNet with `stride` $\in\\{1, 3, 4\\}$. Does it impact the performances a lot?
- It would also be interesting to study the impact of the noise level estimator. If the estimator is biased, how does it impact the performances? This is also related to Question 3, as I suppose if the estimator is biased but used for training, the thresholds $\nu$ can also be adapted to cope for this bias. This would be interesting to add such experiments.
- For the computational complexity, I would be interested to see comparison with modern convolutional sparse coding algorithm such as the one in `sporco` (`[Wohlberg2017]`) or the LGCD algorithm from `[Moreau2019]`, which scales to much larger images.
- The writing is not very clear and not always correct and there are many typos (see bellow).


## Some question

1. Could the authors comment on why the learned network is more interpretable than a classical network?
2. Why  does the authors change the constraint to the one in (7) ? This makes the model somewhat incomparable with other approaches as $d^j$ won't have the unit norm property. Moreover, I don't really see the point, as simply stating that $\\|d^j\\| \le 1$ also constrains appropriatly $\\|\tilde d^j\\|$ to be smaller than roughly $\frac{1}{\\|g\\|}$, which is a similar constraint but comparable to the original one.
3. When training the network, is the true noise level given as an input of the model or is it also estimated using the wavelet based estimator? This is unclear from the text and should be clarified.
4. What is the training time for the proposed model? This is not discussed, but I guess this is similar to the training time of other models.


## Minor comments, nitpicks and typos

- For the citation, when they are between parenthesis, could the authors use `\citep` to have proper formating.
- p.1: `Mairal et al. (2014)`: This is not an arXiv paper, the proper citation is `[Mairal2014a]`.
- p.1: The first paragraph could be improved a lot. It is unclear why this would be an inverse problem (there is no sensing matrix here, the dictionary correspond to the prior knowldge in the inverse problem literature). This is mainly denoising so the paragraph should be fixed to reflect this.
- p.1: `a linear combination of a collection of vectors`: this is simply linear representation. The sparse linear representation also promotes the usage of only a few atoms.
- p.1: `where $n_i \sim \mathcal N(0, \sigma^2 \bm I)$.` The authors should add in plain word that this `is an additive Gaussian white noise.`
- p.2: `is nontrivialy` -> `non trivialy`? What does it mean to be trivialy related? I would remove this as it is unclear.
- p.2: `the Convolutional Sparse Coding (CSC) model has been introduced`: the original work introducing such model is `[Grosse2007]`.
- p.3: `interpretabile` -> `interpretable`
- p.4: Note that there is a third option for training LISTA networks which is to use loss in Eq.(5) as a training loss, as it is done for instance in `[Ablin2019]`.
- p.5: `the Guassian distribution prior` -> `Guassian`
- Table.1: Could the authors highlight the leading method in the table?
- p.7: It is unclear whether the timing for FCDLNet is performed for full image denoising or on 128x128 patches.
- Figure.4: Could the authors add the error bars in this plot? As the gain is small between CDLNet and FCDLNet, it would be interesting to see the confidence here.

### References

```bibtex
@inproceedings{Ablin2019,
  title = {Learning Step Sizes for Unfolded Sparse Coding},
  booktitle = {Advances in {{Neural Information Processing Systems}} ({{NeurIPS}})},
  author = {Ablin, Pierre and Moreau, Thomas and Massias, Mathurin and Gramfort, Alexandre},
  year = {2019},
  pages = {13100--13110},
  address = {{Vancouver, BC, Canada}},
  archivePrefix = {arXiv},
  copyright = {All rights reserved},
  eprint = {1905.11071},
  eprinttype = {arxiv}
}

@article{Elad2005,
  title = {Simultaneous Cartoon and Texture Image Inpainting Using Morphological Component Analysis ({{MCA}})},
  author = {Elad, Michael and Starck, J. L. and Querre, P. and Donoho, D. L.},
  year = {2005},
  volume = {19},
  pages = {340--358},
  journal = {Applied and Computational Harmonic Analysis},
  number = {3},
  pmid = {16370462}
}

@article{Grosse2007,
  title = {Shift-{{Invariant Sparse Coding}} for {{Audio Classification}}},
  author = {Grosse, Roger and Raina, Rajat and Kwong, Helen and Ng, Andrew Y.},
  year = {2007},
  volume = {8},
  pages = {9},
  journal = {Cortex}
}

@article{Isogawa2017,
  title={Deep shrinkage convolutional neural network for adaptive noise reduction},
  author={Isogawa, Kenzo and Ida, Takashi and Shiodera, Taichiro and Takeguchi, Tomoyuki},
  journal={IEEE Signal Processing Letters},
  volume={25},
  number={2},
  pages={224--228},
  year={2017},
  publisher={IEEE}
}

@inproceedings{Lalanne2020,
  title = {Extraction of {{Nystagmus Patterns}} from {{Eye}}-{{Tracker Data}} with {{Convolutional Sparse Coding}}},
  booktitle = {2020 42nd {{Annual International Conference}} of the {{IEEE Engineering}} in {{Medicine}} \& {{Biology Society}} ({{EMBC}})},
  author = {Lalanne, Clement and Rateaux, Maxence and Oudre, Laurent and Robert, Matthieu P. and Moreau, Thomas},
  year = {2020},
  month = jul,
  pages = {928--931},
  publisher = {{IEEE}},
  address = {{Montreal, QC, Canada}},
}

@article{Mairal2014a,
  title = {Sparse {{Modeling}} for {{Image}} and {{Vision Processing}}},
  author = {Mairal, Julien and Bach, Francis and Ponce, Jean},
  year = {2014},
  volume = {8},
  pages = {85--283},
  journal = {Foundations and Trends\textregistered{} in Computer Graphics and Vision},
  number = {2-3}
}

@article{Moreau2019,
  title={Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals},
  author={Moreau, Thomas and Gramfort, Alexandre},
  journal={arXiv preprint arXiv:1901.09235},
  year={2019}
}

@inproceedings{Ramzi2020,
  title = {Wavelets in the {{Deep Learning Era}}},
  booktitle = {European {{Signal Processing Conference}} ({{EUSIPCO}})},
  author = {Ramzi, Zaccharie and Starck, Jean-Luc and Moreau, Thomas and Ciuciu, Philippe},
  year = {2020},
  month = jul,
  pages = {1417--1421},
}

@inproceedings{Tolooshams2018,
  title = {Scalable Convolutional Dictionary Learning with Constrained Recurrent Sparse Auto-Encoders},
  booktitle = {{{IEEE International Workshop}} on {{Machine Learning}} for {{Signal Processing}} ({{MLSP}})},
  author = {Tolooshams, Bahareh and Dey, Sourav and Ba, Demba},
  year = {2018},
  archivePrefix = {arXiv},
  eprint = {1807.04734v1},
  eprinttype = {arxiv}
}

@inproceedings{Wohlberg2017,
  title={SPORCO: A Python package for standard and convolutional sparse representations},
  author={Wohlberg, Brendt},
  booktitle={Proceedings of the 15th Python in Science Conference, Austin, TX, USA},
  pages={1--8},
  year={2017}
}

```
review_title: The method seems to be a small extension of CDLNet and the evaluation could be improved.

final_score: rating: 3: Clear rejection
review_content: The paper proposes a new regularization for the dictionary in the learned convolutional sparse coding model of Sreter & Giryes '18. The main contribution is that the dictionary is regularized to be composed of 1) a fixed low-pass filter and 2) a set of learned filters to occupy the complementary high-frequency space. A second contribution is that the thresholding in the network is adjustable according to the estimated noise level in the image. 

Comments:

- Presentation-wise, I didn't get the motivation of the work after reading the introduction. Everything before Sec. 1.2 is a narrative of the existing methods, and all of a sudden Sec. 1.2 states the proposed method, but what is the problem that the paper wants to address?

- The major contribution is the introduction of a fixed low-pass filter in the dictionary, but I don't see a clear justification as to why it is needed. If you want to model the DC component, why not simply use a bias term?

- The paper tells a story that the dictionary need to be incoherent, that some prior work enforces this by using large stride convolution, and that the proposed method does not need to use large stride. But ultimately there is no incoherence regularization for the high-frequency part of the dictionary in the proposed method. So how do you enforce those high-frequency filters to be incoherent?

- It is stated as a second main contribution of the paper that the thresholding in the network can be made to be dependent on the noise level in the image, so that the network is capable of performing blind denoising. However, the explanation for how it works (Sec. 2.3) is very sketchy. Why is the adaptive threshold only used in the last layer? Also, the threshold is set to be a multiple of noise variance, but how do you estimate the noise variance in practice? Where does the number 0.6745 come from?

Overall, the presentation of the paper needs major improvement to make the motivation as well as technical details clear. In addition, the technical contributions appear quite minor and are not fully justified: it is unclear how fixed low-pass filter compares with a bias term, and it is unclear how effective the adaptive threshold is in practice. 
review_title: Subpar in terms of significance and clarity, needs better development

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposed a convolutional dictionary learning (CDL) work for image denoising. Compared with existing CDLnet, the proposed method used stride convolutions with pre-defined low-pass channels  to improve the performance and reduce complexity. Results on image denoising showed comparable performance to the state-of-the-arts.


Strength:

1. The idea of constraining low-pass atoms in learned dictionary is interesting, which limits the space of learned atoms to have the desired behavior.

2. The proposed method parameterize the soft-thresholding operator in LISTA such that the thresholding is directly linked with the estimated image noise level.

Weaknesses:

1. The contribution of this paper is incremental. The proposed method is heavily based on existing CDLNet with some modifications. Although such modifications improved results a little bit, the motivation is not very clear and the implementation is empirical, e.g., Eq. (7).

2. The proposed method did not substantially improve the results. It improves the baseline CSCNet a little bit, but still not good as the state-of-the-art methods such as DnCNN. In addition, all the experiments are using synthetic data with ideal Gaussian noise. Actually image noise is more complex and is a mixture of Poisson and Gaussian. Under controlled environment, the proposed method still cannot outperform the state-of-the-art. The results on real image noise might be even worse.

Abdelhamed, Abdelrahman, Stephen Lin, and Michael S. Brown. "A high-quality denoising dataset for smartphone cameras." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

Plotz, Tobias, and Stefan Roth. "Benchmarking denoising algorithms with real photographs." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

3. More analysis of the proposed components are missing. For example, there is no ablation study for the proposed components.

review_title: Limited novelty. 

Meta-review: The paper proposes a deep learning approach to blind image denoising based on deep unrolling. In particular, the proposed network is derived from convolutional sparse coding algorithms, which are unrolled, untied across layers and learned from data. The paper proposes a frequency domain regularization scheme in which the filters consist of a single analytically defined low-pass filter and a large collection of filters which are constrained to reside in the mid-to-high frequency ranges. It also proposes to tie the thresholds in the soft-thresholding stages of the learned network to estimates of the noise variance, making the proposed scheme more robust to variations in the noise level. 

Pros and Cons:

[+] Having a single low-pass dictionary atom reduces redundancy (and potentially coherence) in the learned dictionary. This type of regularization may also reduce the time/data required to learn. 

[+/-] Using noise estimators and a noise adaptive threshold renders the model more robust to variations in the noise level. This is important, since in most denoising applications the noise level is not known a-priori. As the reviewers note, the idea of tuning thresholds in an unrolled sparse coding method based on the noise level is not a novelty of the paper; the novelty here is coupling this with a wavelet-based estimate of the noise level. 

[-] All three reviewers raise concerns regarding the novelty of the work compared to existing convolutional sparse coding-based neural networks. The structure of the network is similar; the main difference is the frequency restriction for learned atoms, which is enforced by prefiltering the learned atoms with a high-pass filter. 

[-] The paper is not entirely clear in its motivation and argumentation. Reducing the coherence of the learned dictionary makes sense from the perspective of certain worst case results from sparse approximation. However, the coherence is a worst case quantity; moreover, certain approaches to coherence control (e.g., using large stride) control coherence at the expense of the expressiveness of the dictionary, and hence may not actually improve its ability to provide sparse reconstructions of natural signals. The proposed frequency domain regularization is a sensible approach to controlling coherence, since low-frequency atoms will tend to be highly coherent, but would benefit from a crisper analytical motivation. 

[-] Reviewers found the experiments lacking in some regards. In particular, the paper only evaluates its proposals on synthetic experiments with Gaussian noise. While this is in line with some previous work on deep learning based denoising, more extensive and realistic experiments would have bolstered the paper's argument. 

Overall, the paper makes a sensible proposal regarding the adaptivity to unknown noise levels, and introduces a potentially useful frequency-domain restriction on the learned filters in a CSC network. However, the reviewers did not find that the paper made a clear argument for the significance of these proposals, and raised other concerns regarding the clarity and experiments. The consensus of the reviewers is to recommend rejection. 
Decision: Reject