Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a property inference attack against tsne plots wherein an attacker can train a classifier on tsne plots from surrogate models in order to estimate sensitive characteristics about datasets for which other tsne plots were generated.
strength_and_weaknesses: ### Pros:
1) The paper is very clearly presented.
2) The idea is interesting, and as far as I know, novel.
3) Experimentation is thorough.

### Cons/Comments:
1) The results are lukewarm in my eyes. Even in the comparatively easier setting of proportion classification, the proposed attack requires surrogate and target model pretrained on the same distribution, and several labeled tsne plots on which to train the attack model. And even then, the attack success on some datasets is fairly weak.
2) Am I correct that the target model and the surrogate model are both pretrained on the same data for all experiments? If so, why? It seems like a much more realistic setting is that the target model is trained on a distribution about which the attacker has little information. 
3) Many of the experimental settings seem a bit contrived. The more realistic setting (transfer setting in 5.5), is a step in the right direction, but here the authors only limit the number of tsne plots available to the attacker. Another constraint I would have liked to have seen explored in this setting is tsne plots generated only with a small amount of data. In fact, throughout the paper, it seems like the authors assume that that data on which to generate tsne plots with labeled proportions, can be easily found. This seems like a very strong assumption.

Overall, I'm a bit torn. The paper is a very interesting idea, and novel, but the results are a bit lukewarm and in somewhat contrived settings.
clarity,_quality,_novelty_and_reproducibility: The paper was very clear, and the work seems quite novel.
summary_of_the_review: I vote for acceptance because I think the idea is quite novel, and interesting. However, I would really like the authors to address my concerns/questions about the settings for the experiments. I would consider lowering my score if the questions are not sufficiently addressed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper "Property inference attacks against t-SNE plots" suggests a way to infer some properties of the data set from an unlabeled t-SNE plot of that data set. For example, if the data contain images of males and females, then the attack would aim to infer the fraction of males from the shape of the unlabeled t-SNE plot. To do this, the attacker generates multiple t-SNE plots of the data with varying fraction of males, and then trains a classifier to predict the fraction of males from the t-SNE plot (treated as image). This classifier can then be applied to the t-SNE plot under attack. The paper shows good performance on some of the datasets.
strength_and_weaknesses: In my opinion, the main strength of the paper is to suggest the framework of "attacks against t-SNE plots", which I believe is a novel concept and has not really been considered by anybody before. 

The main weakness is that the suggested attack is rather trivial; also, it can only work in very limited situations where the shape of the t-SNE plot is strongly influenced by the fraction of males (or other similar property) and when the attacker has access to the training ("shadow") labeled data and can generate multiple t-SNE plots with varying property values.

The paper also has some presentation issues but those can be easily fixed. Overall I am giving it a borderline reject score.

PRESENTATION ISSUES

* My understanding is that the "attack model" treats t-SNE plots as images. If this is correct, then I think it should be made more explicit and stated more clearly, e.g. in the end of Section 4, and also elsewhere. If t-SNE plots are images, then what is their size in pixels? What size (in pixels) are the dots of the scatter plot? Are these images black-and-white? Grayscale? Etc.

* The entire evaluation setup was not sufficiently clear to me from Section 4. E.g. what does 0.92 accuracy (beginning of Section 4.1) mean? This is the average over what exactly? Different random splits of the data? What is the true proportion of males in those splits? Were the splits generated such that the true proportion could be .3, .4, .5, .6, .7 with equal probability? Please clarify.

* I think the paper *NEEDS* a figure (preferably Figure 1), that would show a t-SNE of the CelebA dataset under attack, and example t-SNE plots generated from shadow data with .3, .4, .5, .6, .7 fractions. This figure would illustrate the attack and would show that it is indeed possible to "guess" the fraction of males in the CelebA dataset correctly, because the shadow t-SNE plots all look sufficiently different from each other. I was not able to find such figure either in the main text, or in the appendix.
clarity,_quality,_novelty_and_reproducibility: see above
summary_of_the_review: Interesting idea, but a rather trivial attack with very limited applicability. Borderline reject.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work proposes property inference attacks using t-SNE plots of data. While most work in the literature focuses on white-box or black-box model access, this threat model lies somewhere in the middle- predicting properties of hold-out data using 2-dimensional representations graphed onto images. The approach consists of data cleaning and using plots directly as input to a meta-classifier, which is trained by the adversary for property prediction. The authors demonstrate the robustness of their attack across different plot settings and propose a simple adaptation of their attack that can bypass potential defenses like noise addition in the latent space.
strength_and_weaknesses: ## Strengths

- This work introduces a new kind of threat model- information leaked by visualizations. It is common to use techniques like t-SNE or PCA to visualize high-dimensional data; often to demonstrate the effectiveness of an approach in learning useful representations. This work tackles the threat posed by the release of such plots- specifically, properties related to the data used to generate these plots.
- Evaluations across different pairs of datasets are useful in better assessing the practicality of this risk; fine-tuning-based baselines are also useful in evaluating just how much data is actually needed by the adversary to launch successful attacks. This also holds true for evaluations with different class-wise plots and density settings.
- The inclusion of adaptive attacks and evaluations with mixed datasets (Section 5.5) is also useful and further reinforces that the information leaked in this scenario is fairly robust across several configurations.

## Weaknesses
- The property inference attack here really focuses on the data used to generate the plot, not the actual training dataset/distribution itself. A victim is unlikely to use all of its data for t-SNE visualization (given its computational cost). A straightforward defense would thus be to sample the t-SNE plot data from the test set such that the "property" is different from what was used for training. In this case, even if an adversary can perfectly predict the property of the plot data, the information gleaned has utility no longer. Even if the adversary does learn this information, it is not very useful- definitely not for model auditing. The victim can always claim that the data used for plot generation was differently distributed from the data used for training (which is really what auditors would care about).

- Section 3.2: "...and get the image embeddings". How is the adversary aware of which layer's features were used for the embeddings? How robust is the adversary to settings where this information is not known, i.e. the victim and adversary use different layer outputs for the plots?

- Section 2: "....most assume that the adversary has white-box access to the model." This is not true- there are several works [1, 2, 4] in the literature that propose and evaluate black-box attacks. Similarly, the claim on "most of the works infer binary properties" is not true either. Several works have studied various kinds of graph-related properties [3], as well as direct regression of these values [2].

- As demonstrated in Tables 1 and 2, performance is near-random (and even worse) when the training and testing data are different. Although this can be alleviated by using some data from the target distribution (as demonstrated later in the paper), this shows how sensitive the attack is to know the exact data distribution. This might not be feasible for parties that might actually release these plots (like a company talking about their new ML model), and it is even more unclear if this attack would be practical in such a scenario. 

- Can plot rotation/scaling be a potential defense? It may not be perfect, but the adversary might suffer performance losses if trying to incorporate these invariances in its classifiers. Scaling the plot image, or rotating it, should not impact the utility of the t-SNE plot visualization and thus a victim could easily perform this post-processing to make things harder for the adversary.

### References
[1] Zhang, Wanrong, Shruti Tople, and Olga Ohrimenko. "Leakage of Dataset Properties in {Multi-Party} Machine Learning." USENIX Security Symposium. 2021.

[2] Suri, Anshuman, and David Evans. "Formalizing and Estimating Distribution Inference Risks." Privacy Enhancing Technologies Symposium. 2022.

[3] Zhang, Zhikun, et al. "Inference Attacks Against Graph Neural Networks." USENIX Security Symposium. 2022.

[4] Juarez, Marc, Samuel Yeom, and Matt Fredrikson. "Black-Box Audits for Group Distribution Shifts." arXiv preprint arXiv:2209.03620 (2022).
clarity,_quality,_novelty_and_reproducibility: __Clarity__: The paper is well written and presented for the most part (see minor comments below), and is easy to follow even without a substantial background in property inference. 

__Quality and Novelty__: While the problem itself is very unique and original, I am not convinced of the threat posed in the paper. The proposed techniques are standard to image classification and property inference- a CNN (serving as a meta-classifier here) trained for property prediction.

__Reproducibility__: The main attack idea is straightforward, and sufficient details are provided in the paper (and the Appendix) to be able to reproduce results in the paper.

### Minor comments
- Section 3.1: "Such plots are commonly published on...". Can the authors please include some examples or references for this? Personally, I have never seen t-SNE plots for data outside whitepapers. 

- Page 7, 'Different Target Properties' just like a baseline for accuracy for random guessing, the authors should also include a baseline for the case of regression. A straight-forward setting would be one where the adversary always predicts a constant value (0.5), or randomly picks a value in [0, 1].
summary_of_the_review: This work introduces a very new kind of threat model- one where the adversary tries to gain information about confidential data, given t-SNE plots generated with a model. The authors demonstrate how this leakage is very real, and fairly robust across various datasets, and configurations, and can be adapted to defenses with simple modifications. However, the practicality of the threat model is uncertain. The proposed attack cannot be used for model auditing (plausible deniability by the model trainer, as training data is different from plot-generation data), and it is unclear how the information leaked by the plots is of any use to the adversary. Even in the case that it is, the victim can always sample selectively from its test data, such that the leaked information is in no way relevant to the training distribution- like a 'honeypot' for the adversary.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper shows how to perform perform privacy attacks against t-SNE plots. The reviewers were positive on the identification of the attack vector. However, they were negative on the strength of the attack and the assumptions required to execute the attack/when it would be effective. There were also some concerns about certain figures in the paper, but unfortunately code was not available to investigate further. 
justification_for_why_not_higher_score: The attack was of relatively low efficacy and works best in restricted settings.
Decision: Reject