Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: In this paper, the authors proposed a counter measure to protect collaborative training of DNN against the GAN attack in (Hitaj et al. 2017). The motivation of the paper is clear and so is the literature review. But for me the algorithm is not clearly defined and it is difficult to evaluate how the proposed procedure works. I am not saying that this is not the solution. I am just saying that the paper is not clear enough to say that it is (or it is not). From, my perspective this will make the paper a clear reject. 

I think the authors should explain a few things more clearly in order to make the paper foolproof. The first one seems to me the most clear problem with the approach proposed in the paper:

1 $\psi(c)$ defines the mapping from each class to a high dimensional vector that allows protection against the GAN attack. $\psi(c)$ is suppose to be private for each class (or user if each class belong only to one user). This is the key aspect in the paper. But if more than one user have the same class they will need to share this key. Furthermore, at test time, these keys need to be known by everyone, because the output of the neural network needs to be correlated against all keys to see which is the true label. Of course the keys can only be released after the training is completed. But the adversary can also claim to have examples from the class it is trying to attack and hence the legitimate user that generated the key will have to give the attacker the key from the training phase. For example, let assume the legitimate user only has ones from MNIST and declares that it only has one class. The attacker says it has two classes the same one that the legitimate user and some other label. In this case the legitimate user needs to share $\psi(c)$ with the attacker. Of course this sounds “fishy” and might be a way of finding who the attacker is, but there might be many cases in which it makes sense that two or more users shares the same labels and in a big system might be complicated to decide who has access to which key.

2 I do not understand the definition of $\phi(x)$. Is this embedding fixed for each user? Is this embedding the DNN? In Eq. 4 I would assume that $\phi(x)$ is the DNN and that it should be $\phi_\theta(x)$, because otherwise the equation does not make sense. But this is not clearly explained in the paper and Eq 4 makes no sense at all. In a way the solution to the maximization in Eq 4 is Theta=\infty. Also the term $\phi(x)$ is not mentioned in the paper after page 5. My take is that the authors want to maximize the inner product, but then the regularizer should go the other way around. 

3 In the paper in page 5 we can read: “Here, we emphasize the first reason why it is important to use l2-normalized class keys and embedding outputs: in this manner, the resulting classification score is by definition restricted to the range [-1; +1],” If I understand correctly the authors are dividing the inner product by ||$\psi(c)|| ||$\phi(x)||. I can see that we can easily divide by ||$\psi(c)||, but I cannot see how we can do dive by ||$\phi(x)||, if this term depends on \theta. If this term does not depend on \theta, then Eq 4 does not make sense.

To summarize, I have the impression that there are many elements in the paper that does not makes sense in the way that they are explained and that the authors need to tell the paper in a way that can be easily understood and replicated. I recommend the authors to run the paper by someone in their circle that could help them rewrite the paper in a way that is more accessible. 

review_title: The paper is unclear and needs more work

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper is a follow-up work to the CCS'2017 paper on the GAN-based attack on collaborative learning system where multiple users contribute their private and sensitive data to joint learning tasks. In order to avoid the potential risk of adversary's mimic based on information flow among distributed users, the authors propose to embed the class label into a multi-dimensional space, such that the joint learning is conducted over the embedding space without knowing the accurate representation of the classes. Under the assumption that the adversary can only generate fake and random class representations, they show their scheme is capable of hiding information from individual samples, especially over image data.

The paper is clearly written and easy to understand. The experiments show interesting results, which are particularly impressive with the face data. However, the reviewer feels the assumption on the adversary is generally too weak, such that slightly smarter adversary could circumvent the protection scheme and remain effective on sample recovery.

Basically, instead of randomly guessing the representations of the classes from other innocent users, the adversary could apply GAN to learn the representation based on the feedback from these users. This can be easily done by including the representations in the embedding space in the parameters in GAN for learning.

This paper could be an interesting work, if the authors address such enhanced attacks from the adversary and present protection results over their existing experimental settings.
review_title: The weak assumption on the adversary undermines the usefulness of the protection scheme

final_score: rating: 5: Marginally below acceptance threshold
review_content: Collaborative learning has been proposed as a way to learn over federated data while preserving privacy. However collaborative learning has been shown to be suscepti
ble to active attacks in which one of the participants uses a GAN to reveal information about another participant.

This paper proposes a collaborative learning framework (CLF) that mitigates the GAN attack. The framework involves using the neural net to learn a mapping of the inp
ut to a high-dimensional vector and computing the inner product of this vector to a random class-specific key (the final class prediction is the argmax of this inner product). The class-specific key can be chosen randomly by each participant. By choosing sufficiently long random keys, the probability of an attacker guessing the key can be reduced. Experiments on two datasets show that this scheme successfully avoids the GAN attack.
 
1. Some of the details of key sharing are not clear and would appear to be important for the scheme to work. For example, if participants have instances associated with the same class, then they would need to share the key. This would require a central key distribution scheme which would then allow the attacker to also get access to the key.

2. I would have  liked to see how the method works with an increasing fraction of adversarial participants (I could only see experiments with one adversary). Similarly, I would have liked to see experiments with and without the fixed dense layer to see its contribution to effective learning. 
review_title: Interesting idea to mitigate the GAN attack

Meta-review: While the reviewers feel there might be some merit to this work,  they find enough ambiguities and inaccuracies that I think this paper would be better served by a resubmission.
Decision: Reject