After rebuttal:  Thanks for reporting the AlexNet results. The fact that they are not great is not so bad by itself, and as the authors mention, it would be interesting to understand why this happens. But the fact that these results  were not in the paper (and in fact still are not there) is disturbing. Moreover, some claims in the paper look wrong in the light of these results, for example: - "This suggests that our gains stem from the CC-GAN method rather than the use of a better architecture." - "Since discrimination of real/fake in-paintings is more closely related to the target task of object classification than extracting a feature representation suitable for in-filling, it is not surprising that we are able to exceed the performance of Pathak et al. (2016) on PASCAL classification."  These statements, and possibly other parts of the paper, have to be updated. I think the paper cannot be published in its current form. Perhaps after a revision.  -------- Initial review:  The paper demonstrates an application of generative adversarial networks (GAN) to unsupervised feature learning. The authors show that the representation learned by the discriminator of a conditional GAN trained for image inpainting performs well on image classification. As a side-effect, fairly convincing inpaintings are produced.  The proposed method combines two existing ideas: using the discriminator of a GAN as a feature learner [Radford et al. 2015] and performing unsupervised feature learning with image inpainting [Pathak et al. 2016]. Therefore conceptual novelty of the paper is limited. On the plus side, the authors implement their idea well and demonstrate state-of-the-art results on STL-10 and good results on Pascal VOC (although Pascal experiments are incomplete, see below). Overall, I am in the borderline mode, and I will gladly raise the score if the authors address my concerns regarding the experiments.  1) Experimental evaluation on Pascal VOC is not quite satisfactory. Comparison with prior work is unfair because the network architecture used by the authors (VGG) is different from the architecture used by all existing methods (AlexNet). It is great that the authors do not try to hide this fact in the paper, but I do not understand why the authors are not willing to simply run their method with AlexNet architecture, although two commenters asked them to do so. Such an experiment would strongly support authors’ claims. Current reasoning that “we thought it reasonable to use more current models while making the difference clear” is not convincing. It is great that better architectures lead to better results, but it is also very important to properly compare to prior work. On a related topic, Doersch et al. also tried using VGG architecture, would it be possible to compare to that? Yet another question: why are you not comparing to [Noroozi&Favaro, ECCV 2016] ? I would also like the authors to address the comment by Richard Zhang.  2) Qualitative inpainting results are incomplete: comparison with previous methods (for instance, [Pathak et al 2016]) is missing, and it is impossible to compare different versions of the proposed method because different images are used for different variants. I realize there may be too little space in the main paper to show all the results, but many more results should be shown in the supplementary material. Quantitative results are missing. Currently the inpainting results are just interesting pictures to look at, but they do not add as much to the paper as they could.
Output: en