Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes the use of a GAN to learn the distribution of image classes from an existing classifier, that is a nice and straightforward idea. From the point of view of forensic analysis of a classifier, it supposes a more principled strategy than a brute force attack based on the classification of a database and some conditional density estimation of some intermediate image features. Unfortunately, the experiments are inconclusive.  

Quality: The key question of the proposed scheme is the role of the auxiliary dataset. In the EMNIST experiment, the results for the “exact same” and “partly same” situations are good, but it seems that for the “mutually exclusive” situation the generated samples look like letters, not numbers, and raises the question on the interpolation ability of the generator. In the FaceScrub experiment is even more difficult to interpret the results, basically because we do not even know the full list of person identities. It seems that generated images contain only parts of the auxiliary images related to the most discriminative features of the given classifier. Does this imply that the GAN models a biased probability distribution of the image class? What is the result when the auxiliary dataset comes from a different kind of images? Due to the difficulty of evaluating GAN results, more experiments are needed to determine the quality and significance of this work.

Clarity: The paper is well structured and written, but Sections 1-4 could be significantly shorter to leave more space to additional and more conclusive experiments. Some typos on Appendix A should be corrected.

Originality: the paper is based on a very smart and interesting idea and a straightforward use of GANs. 

Significance: If additional simulations confirm the author’s claims, this work can represent a significant contribution to the forensic analysis of discriminative classifiers.

review_title: Nice idea, but still need some work

final_score: rating: 7: Good paper, accept
review_content: This paper considers a new problem : given a classifier f trained from D_tr and a set of auxillary samples from D_aux, find D_tr conditioned on label t*. Its solution is based on a new GAN: preImageGAN. Three settings of the similarity between auxillary distribution and training distribution is considered: exact same, partly same, mutually exclusive. Experiments show promising results in generating examples from the original training distribution, even in the "mutually exclusive" setting.

Quality: 
1. It is unclear to me if the generated distribution in the experiments is similar to the original distribution D_tr given y = t^*, either from inception accuracy or from pictorial illustration. Since we have hold out the training data, perhaps we can measure the distance between the generated distribution and D_tr given y = t^* directly.

2. It would be great if we can provide experiments quantifying the utility of the auxillary examples. For example, when they are completely noise, can we still get sensible generation of images? 

3. How does the experimental result of this approach compare with model attack? For example, we can imagine generating labels by e_t^* + epsilon, where epsilon is random noise. If we invert these random labels, do we get a distribution of examples from class t^*?

Clarity:
1. I think the key here is to first generate auxillary labels (as in Figure 2), then solve optimization problem (3) - this causes my confusion at first sight. (My first impression is that all labels, training or auxillary, are one-hot encoding - but this makes no sense since the dimension of f and y_aux does not match.)

Originality: I am not familiar with relevant literature - and I think the GAN formulation here is original.

Significance: I see this as a nice step towards inferring training data from trained classifiers. 


review_title: a nice paper, some details need to be clarified

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposed to learn a generative GAN model that generates the training data from the labels, given that only the black-box mapping $f$ from data to label is available, as well as an aux dataset that might and might not overlap with the training set. This approach can be regarded as a transfer learning version of ACGAN that generates data conditioned on its label.

Overall I feel it unclear to judge whether this paper has made substantial contributions. The performance critically relies on the structure of aux dataset and how the supervised model $f$ interacts with it. It would be great if the author could show how the aux dataset is partitioned according to the function $f, and what is the representative sample from aux dataset that maximizes a given class label. In Fig. 4, the face of Leonardo DiCaprio was reconstructed successfully, but is that because in the aux dataset there are other identities who look very similar to him and is classified as Leonardo, or it is because GAN has the magic to stitch characteristics of different face identities together?  Given the current version of the paper, it is not clear at all. From the results on EMNIST when the aux set and the training set are disjoint, the proposed model simply picks the most similar shapes as GAN generation, and is not that interesting. In summary, a lot of ablation experiments are needed for readers to understand the proposed method better.

The writing is ok but a bit redundant. For example, Eqn. 1 (and Eqn. 2) which shows the overall distribution of the training samples (and aux samples) as a linear combinations of the samples at each class, are not involved in the method. Do we really need Eqn. 1 and 2?
review_title: Need more ablation study to clarify the contribution

Meta-review: This paper addresses the very important problem of ensuring that sensitive training data remain private. It proposes an attack whereby the attacker can reconstruct information about the training data given only the trained classifier and an auxiliary dataset. If done well, such an attack would be a useful contribution that helps make discussion of differential privacy more complete. But as the reviewers pointed out, it's not clear from the paper whether the attack has succeeded. It works only when the auxiliary data is very similar to the training data, and it's not clear if it leaks information about the training set itself, or is just summarizing the auxiliary data. This work doesn't seem quite ready for publication, but could be a strong paper if it's convincingly demonstrated that information about the training set has been leaked.


Decision: Reject