Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: - The authors propose the use of multiple adversaries over random subspaces of features in adversarial feature learning to produce censoring representations. They show that their idea is effective in reducing private information leakage, but this idea alone might not be signifcant enough as a contribution. 

- The idea of training multiple adversaries over random subspaces is very similar to the idea of random forests which help with variance reduction. Indeed judging from the large variance in the accuracy of predicting S in Table 1a-c for single adversaries, I suspect one of the main advantage of the current MARS method comes from variance reduction. The author also mentioned using high capacity networks as adversaries does not work well in practice in the introduction, and this could also be due to the high model variance of such high capacity networks.  

- The definition of S, the private information set, is not clear. There is no statement about it in the experiments section, and I assume S is the subject identity. But this makes the train-test split described in 4.1 rather odd, since there is no overlap of subjects in the train-test split. We need clarifications on these experimental details. 

- Judging from Figure 2 and Table 1, all the methods tested are not effective in hiding the private information S in the learned representation. Even though the proposed method works better, the prediction accuracies of S are still high.  

review_title: Improved performance in obtaining censoring representations, but contribution not quite signficant enough

final_score: rating: 6: Marginally above acceptance threshold
review_content: MARS is suggested to combine multiple adversaries with different roles.
Experiments show that it is suited to create censoring representations for increased anonymisation of data in the context of wearables.

Experiments a are satisfying and show good performance when compared to other methods.

It could be made clearer how significance is tested given the frequent usage of the term.

The idea is slightly novel, and the framework otherwise state-of-the-art.

The paper is well written, but can use some proof-reading.

Referencing is okay.
review_title: Multiple adversaries for increased privacy performs well

final_score: rating: 6: Marginally above acceptance threshold
review_content: The below review addresses the first revision of the paper. The revised version does address my concerns. The fact that the paper does not come with substantial theoretical contributions/justification still stands out.

---

The authors present a variant of the adversarial feature learning (AFL) approach by Edwards & Storkey. AFL aims to find a data representation that allows to construct a predictive model for target variable Y, and at the same time prevents to build a predictor for sensitive variable S. The key idea is to solve a minimax problem where the log-likelihood of a model predicting Y is maximized, and the log-likelihood of an adversarial model predicting S is minimized. The authors suggest the use of multiple adversarial models, which can be interpreted as using an ensemble model instead of a single model.

The way the log-likelihoods of the multiple adversarial models are aggregated does not yield a probability distribution as stated in Eq. 2. While there is no requirement to have a distribution here - a simple loss term is sufficient - the scale of this term differs compared to calibrated log-likelihoods coming from a single adversary. Hence, lambda in Eq. 3 may need to be chosen differently depending on the adversarial model. Without tuning lambda for each method, the empirical experiments seem unfair. This may also explain why, for example, the baseline method with one adversary effectively fails for Opp-L. A better comparison would be to plot the performance of the predictor of S against the performance of Y for varying lambdas. The area under this curve allows much better to compare the various methods.

There are little theoretical contributions. Basically, instead of a single adversarial model - e.g., a single-layer NN or a multi-layer NN - the authors propose to train multiple adversarial models on different views of the data. An alternative interpretation is to use an ensemble learner where each learner is trained on a different (overlapping) feature set. Though, there is no theoretical justification why ensemble learning is expected to better trade-off model capacity and robustness against an adversary. Tuning the architecture of the single multi-layer NN adversary might be as good?

In short, in the current experiments, the trade-off of the predictive performance and the effectiveness of obtaining anonymized representations effectively differs between the compared methods. This renders the comparison unfair. Given that there is also no theoretical argument why an ensemble approach is expected to perform better, I recommend to reject the paper.
review_title: Minor theoretical contribution; main focus on experiments

Meta-review: The reviewers tend to agree that the empirical results in this paper are good compared to the baselines. However, the paper in its current form is considered a bit too incremental. Some reviewers also suggested additional theory could help strengthen the paper.
Decision: Reject