Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper introduces the problem of overlearning, which can be thought of as unintended transfer learning from a (victim) source model to a target task that the source model’s creator had not intended its model to be used for. The paper raises good points about privacy legislation limitations due to the fact that overlearning makes it impossible to foresee future uses of a given dataset.

Background on censoring is well developed, and helps position the submission. However, the relation to transfer learning is not sufficiently outlined in the introduction. In some ways, overlearning is a form of unintended transfer learning. In particular, the connection appears explicit in the case of model repurposing (Section 3.2). Editing the introduction to tease apart this relationship to transfer learning would help readers forge an intuition for what the paper considers.

While the de-censoring algorithm is intuitive, it is not clear what assumptions are being made because of ambiguous notation in Algorithm 1. How does Algorithm 1 train E_aux and C_aux? Does that step assume the adversary has knowledge of the algorithm use to train E and C? Descriptions of the experimental setup from Section 4.2 seem to indicate that this is the case. This is not necessarily an issue if the proposed attack is demonstrating a limitation of learning rather than a practical attack. 

Experiments overall show that censoring does not mitigate overlearning in a way that is robust to de-censoring. One aspect of the experiments that is unclear is the auxiliary dataset: what is the auxiliary dataset used by the adversary for each of the datasets considered in experiments? 

Question: how does simultaneous censoring of all layers affect overlearning and repurposing?

Question: what is the connection between learning more complex representations and overlearning? Is the intuition that if the representation is more complex, it is more likely to contain features useful to identify the sensitive attribute?

The paper is overall well-written. Some parts of the paper omit important details (perhaps due to space constraints?), but an editorial pass should address most of these. Detailed feedback:

1 / Explaining what model partitioning means in this context would help make the introduction more self-contained. 

1 / Is overlearning specific to attributes that raise fairness issues? Or is this phenomenon more general? The “Censoring representations” paragraph on page 2 seems to indicate that the phenomenon is more general.

5/ Is accuracy the best metric to report (e.g., for race attribute prediction) given that the random guessing figure suggests the data is not balanced across attribute values?

6/ Why is the effect not monotonic in Table 4? Were multiple runs averaged?
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: This paper highlights the problem of model overlearning - learning more than it is trained to do. Thus, there is leak of privacy and sensitive attributes of images during test/ inference time.

Pros:
1. The paper is well written and easy to follow

Cons:
1. There is very little novelty in this paper - the notion of overlearning is well established in the literature (Osia et al., 2018; Chi et al., 2018; Wang et al., 2018). This paper merely reinstates, what is being already told in the literature.

2. In fact, there are many defence mechanisms proposed in the literature, for example "Anonymizing k Facial Attributes via Adversarial Perturbations" IJCAI 2018 - where the authors are performing data perturbations to minimize overlearning. This paper does not suggest or propose any method for solving the issue of overlearning

In summary, this paper repeats a well established problem of overlearning, showing experiments that are already shown in literature with known datasets, and also NOT proposing a solution to minimize overlearning (as many papers already proposed in literature). 

3. Additionally, the experiments are very weak - the authors still perform experiments using LeNet variants and AlexNet, and for text using a textCNN. Why did the authors not perform experiments using more state-of the art CNN/RNN models. Did they not observe overlearning in these models?

4. As for section 4.4, it is pretty understood that lower layers of a DL model, learns very basic low-level features from the images such as edges, corner. Reinstating that, and calling it the reason for overlearning is not very convincing. 

As of now, I find the paper very weak, till a solution to avoid overlearning is not proposed as a part of this paper.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper demonstrates some limitations of censoring for privacy with respect to sensitive attributes. In particular, the authors show that censoring reduces, but does not eliminate, the ability of a neural network to infer private/sensitive attributes, e.g. to infer race from a model aiming to predict gender. Part of the proposed method is a component that performs de-censoring using an auxiliary dataset. The authors show that censoring strength often does reduce the ability to infer sensitive attributes, but also affects the ability to perform the main (non-sensitive) task; and in some cases, may actually increase ability to infer sensitive attributes. This type of work is important in that privacy is of growing importance, and so is the risk to privacy; this particular work is well carried out. 

One concern: In Sec. 3.1, there is an assumption of the availability of D_{aux}. How realistic is this assumption?
review_title: Official Blind Review #1

Meta-review: This paper introduces the problem of overlearning, which can be thought of as unintended transfer learning from a (victim) source model to a target task that the source model’s creator had not intended its model to be used for. The paper raises good points about privacy legislation limitations due to the fact that overlearning makes it impossible to foresee future uses of a given dataset.

Please incorporate the revisions suggested in the reviews to add clarity to the overlearning versus censoring confusion addressed by the reviewers.
Decision: Accept (Poster)