Deep Belief Networks for Image Denoising
Authors: Mohammad Ali Keyvanrad, mohammad pezeshki, Mohammad Mehdi Homayounpour
ICLR 2014

Abstract: Deep Belief Networks which are hierarchical generative models are effective tools for feature representation and extraction. Furthermore, DBNs can be used in numerous aspects of Machine Learning such as image denoising. In this paper, we propose a novel method for image denoising which relies on the DBNs' ability in feature representation. This work is based upon learning of the noise behavior. Generally, features which are extracted using DBNs are presented as the values of the last layer nodes. We train a DBN a way that the network totally distinguishes between nodes presenting noise and nodes presenting image content in the last later of DBN, i.e. the nodes in the last layer of trained DBN are divided into two distinct groups of nodes. After detecting the nodes which are presenting the noise, we are able to make the noise nodes inactive and reconstruct a noiseless image. In section 4 we explore the results of applying this method on the MNIST dataset of handwritten digits which is corrupted with additive white Gaussian noise (AWGN). A reduction of 65.9% in average mean square error (MSE) was achieved when the proposed method was used for the reconstruction of the noisy images.


Peer review:
[Review]:
This paper presents an approach for image denoising, based on deep belief networks (DBN). The idea is to train a DBN on a training set consisting of both noising and non-noisy images. Then, the activity of top-hidden-layer units is compared between the noisy and non-noisy images, in order to identify units which are mostly involved in the modelling of noisy images. Denoising is then performed by inputing a noisy image, inferring the value of the top hidden units, fixing the 'noise' hidden units to its neutral value (i.e. its average value on the clean images) and then regenerating the input image. Experiments show that this approach has some success in performing denoising.

The main weakness of this paper is that no comparisons are made with other good denoising baselines. I would have at least expected a comparison with the denoising autoencoder work cited in this paper [6]. Also, denoising experiments on MNIST are not particularly compelling and too simplistic. 

Pros: 
- The presented idea is simple. 
- Results seem OK.

Cons: 
- The results are too preliminary, as no comparisons are made with a good denoising baseline (including the deep learning work on denoising, cited in this paper).
- Writing could be improved.

Other comment
- How is denoising performed in the 'reconstruction without eliminating any node'? Specifically, is this network trained on both noisy and non-noisy images, or only on clean images?