Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes a denoising auto-encoder where the input image is corrupted by adding noises to its Laplacian pyramid representation. Then a DAE is trained to predict the original data and learn a good representation of the data. By corrupting the Laplacian representation, which is multi-scale, the corruption of the image is not local and thus more robust representations are learned.

I personally like this idea. However, it seems a simple extension of the classical DAE. 

1. Is it possible to generalize this idea to other representations of the images such as wavelets or sift, or the representations learned by other neural networks? It seems that you can add corruptions to any image representations as long as you can reconstruct the image from the representation. Here, you can reconstruct from  Laplacian pyramid representation.

2. As an empirical work, the experiments in this work is rather small-scale, using only CIFAR10 and MNIST. That seems far from sufficient. 
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper provides a novel approach to learning useful representations with deep learning models. They formulate an entirely unsupervised framework based off autoencoders to accomplish this task. Their data differs from previous works by upsampling a noise-corrupted downsampling of the original inputs. The autoencoder is then trained to reconstruct the original image from this new data. Their experiments demonstrate that using this Laplacian pyramid scheme to generate noisy data leads to an autoencoder that learns better representations compared to a standard DAE. 
Overall, the work in this paper has the potential to be a contribution to ICLR but lacks experimental completeness and clarity. Moreover, the main contribution is a better denoising autoencoder, but in the grand scheme of representation learning, it is unclear how broad of a contribution this is. I would be willing to change my score, upon addressing the following details:
•	As mentioned above, the results as presented provide a better DAE. The paper would be much stronger if it also provided comparisons to more recent models in representation learning closer to state of the art. For instance, they choose BiGAN as a model for comparison, but these were developed over three years ago and are now outdated in favor of better GAN models.
•	The paper would be further strengthened with additional experiments of their representations being qualitatively better than previous models. Their example of image retrieval via nearest neighbor is quite limited when compared to the wealth of tasks GAN models can accomplish.
•	In general, when presenting qualitative results (Figures 3,4, and 5), additional examples should be put into supplementary materials so as to demonstrate the paper did not cherry pick.
•	The presentation of the quantitative results is peculiar. The paper chooses to combine their Laplacian DAE with the AET framework. As such, all the results tables should include numbers for AET alone and the conventional DAE with AET.
•	Lastly, the paper needs a revision for ease of readability, as there are a significant number of grammatical errors that make it hard to read at times. 
While the transfer learning experiments seem incomplete to me, that is not my area of expertise and I cannot judge how convincing that setup is as well as other reviewers. 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Denoising auto-encoder (DAE)  is a representation learning framework proposes in [1], which uses noisy input to reconstruct the clean “repaired” input. In the conventional DAE, the noise is directly added to the input space. This paper introduces a novel type of noise, which corrupts the Laplacian pyramid representation, and uses it to train a DAE. The effect of the proposed perturbation is to allow larger scale and semantically meaningful corruption, which may help to learn more transferable representation. Several experiments are conducted showing the effectiveness of the method
1) On MNIST, LapDAE provides better reconstruction images
2) On Cifar-10, LapDAE provides better image retrieval results
3) On Imagenet, combining with the transformation technique in [2], LapDAE achieve state-of-the-art result
4) On Pascal VOC,  combining with the transformation technique, LapDAE achieve state-of-the-art result on transfer learning
Overall, I find the idea natural and simple (simplicity of an approach is a good quality to me). At the same time, I also find the contribution a bit limited. The following are further comments and questions.

* From the experiments, standard DAE are harder to train comparing to the proposed LapDAE. In my opinion, this suggests that the local noise are harder to remove comparing to the Laplacian noise. It would be interesting to perform the following experiment to further understand the difference between DAE and LapDAE: train a DAE and LapDAE then 
a) Apply LapDAE to reconstruct an image where the random noise is added on the input space (as the standard DAE setting).
b) Apply DAE to reconstruct an image where the noise is added to the Laplacian pyramid representation.

* How does the corrupted set C been selected? Moreover, how does the selected layer in the Laplacian pyramid effects the performance? For example, is there a difference in terms of performance of the model when training on LPS4 versus LPS8?

* The transformation technique seems to be very helpful for the performance, is it possible to combine it with other benchmark methods like RotNet or Counting?

[1]  Vincent et al. 2010, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion 
[2] Zhang et al. 2019, AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data
review_title: Official Blind Review #2

Meta-review: The main idea proposed by the work is interesting. The reviewers had several concerns about applicability and the extent of the empirical work. The authors responded to all the comments, added more experiments, and as reviewer 2 noted, the method is interesting because of its ability to handle local noise. Despite the author's helpful responses, the ratings were not increased, and it is still hard to assess the exact extent of how the proposed approach improves over state of the art.   Because some concerns remained, and due to a large number of stronger papers, this paper was not accepted at this time.
Decision: Reject