Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper introduces a framework to privatize sensitive attributes of data using adversarial representation learning. The proposed method consists of a “filter” that removes the sensitive attribute from the data representation, and a “generator” that replaces the removed sensitive attribute with a randomly sampled synthetic value. The authors argue that the second step done by the generator enhances privacy, and use experiments on real image data to verify their method and compare it with a baseline.

One important aspect of this work is that the privatization scheme is done regardless of the downstream learning task. The authors argue that all prior work (except for one) doesn’t have this property.

Some comments and questions:

-Can you provide more evidence as to why replacing the sensitive information with something else (which is what the generator does) is useful? In the third paragraph of introduction you mention this as an “assumption” but is this actually an assumption or observation?

-In your experiments I couldn’t find what the baseline is? Is it just the Huang et al. 2018 approach which is the same method without the generator? Are there any other methods you could compare your method with?

-In your experiments, why is it that in some cases when epsilon is increased, both privacy loss and utility score gets better? For e.g. see Age plot in Figure 1, epsilon=0.001 and epsilon=0.005 for the green curve. Shouldn’t we always have that increasing epsilon helps the privacy loss but hurts the utility score?

-In Eq. 5 & 6, Z_2 is probably missing inside the parentheses for g.

review_title: Weak Reject

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary: The paper presents a privacy-preserving transformation technique for image data. The main idea is to use adversarial representation learning to obfuscate sensitive attributes. 

Strengths:
1) The paper considers an important problem of preserving privacy on images and presents a heuristic approach to address this problem.
2) The idea of using adversarial representation learning for this problem seems reasonable. The prior work on this topic has  focused on removing information from representation, and the paper extends this to also generating new information.

Concerns:
1) The paper feels incremental; the idea utilized seems like a simple extension to the Generative adversarial privacy idea of Huang et al. 
2) The evaluation feels a little strange. The authors consider a set of attributes gender, lipstick, age, high cheekbones, mouth slightly open, heavy makeup, smiling (and split them as sensitive and utility). I did not follow the real motivation of using these set of attributes, why are they sensitive? A better dataset (usecase) and/or choice of attributes would have helped making a stronger case. Being an experimental paper, a stronger evaluation (say on multiple datasets) might also help.

Overall, while I feel that the problem is interesting, the paper presents just too little in terms of its technical contribution.

review_title: Review of "Adversarial representation learning for synthetic replacement of private attributes"

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This work proposes a privacy-preserving transformation mechanism based on adversarial representation learning. The proposed work is an extension of generative adversarial privacy (GAP). It replaces the filter with a generator that replaces the sensitive information with synthetic data instead of censoring it. 

Reason for score:
The contribution of the paper is incremental, and the experimental results are limited to one dataset and one baseline. More experiments need to be done to evaluate the consistency of the results and to study the generalizability of this work across various datasets.

More detailed comments:
-         In related work, please add more recent works in adversarial representation learning and explain the contribution of this work compared to the existing ones. One example includes:
Adversarial Learning of Privacy-Preserving and Task-Oriented Representations by Xiao et al.
-         It would be better if the authors theoretically prove the privacy-preserving of the proposed method.
-         This work is only evaluated on one dataset, also it is only compared to one baseline. The authors are expected to compare the proposed method with several recent works (some of them already mentioned in related work) and show how this method advances the state-of-the-art in this area.
-         Please explain how this work can scale on larger data or more high-quality images? More discussion on the efficiency of the method is also necessary compared to other existing techniques.
-         It would be better if the authors showed how the model preserves utility when generating  multiple synthetic attributes especially in cases where two or more attributes are highly correlated (e.g., heavy makeup and lipstick or mouth open and smiling, etc.)
review_title: interesting study, unconvincing experiments

Meta-review: This paper proposes a heuristic for removing privacy sensitive attributes and replacing them with sythetically generated ones. The technique is closely related to an existing work and, as pointed out in the reviews, the experimental evaluation is insufficient for properly evaluating the approach.
Decision: Reject