Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper looks into the three existing privacy protection mechanisms for edge-cloud collaborative inference systems and proposes an anti-defense method, dubbed Sensitive Feature Distillation (SFD), which simultaneously uses two existing techniques, shadow model and feature-based knowledge distillation, for attacking the privacy protection mechanisms. The core idea is to develop a shadow model for the attack-targeting model, which can produce the non-obfusticated and labeled training data for the privacy-protecting model, and then feeds this model to the knowledge distillation process to imitate or learn the behavior of the target model, so that the resulting model can produce the non-obfusticated (or purified) feature map, which can lead to the privacy leak. The authors claim that the proposed attack method can effectively extract the purified feature map from the intentionally obfusticated one and recover the private image for popular image datasets such as MNIST, CIFAR10, and CelebA. 
strengths_and_weaknesses: Strengths
* This paper tackles an interesting problem, the privacy protection in edge-cloud collaborative systems. 
* This paper identifies limitations of the existing solutions, which seem valid issues. 
* Writing quality is sufficient for nonknowledgable readers to follow and comprehend the idea. 

Weaknesses
* The proposed technique is based on narrow assumptions that much information is already available to the attackers and given. The examples are the model architecture, hyperparameters, and training data for shadow model (Ds). Moreover, the proposed technique narrows the target data type to be images. 
* The experiments focus on very early split points such as 1st or 2nd conv layer or 1st fc layer. This settings are favorable to the proposed technique since the existing techniques would be weaker to the model inverse attacks. On the other hand, prior works do not seem to split the model in such an early point in the model architecture. For instance, Table 2 in [Mireshghallah et al., 2020] reports that they run 4~11 layers in the edge while offloading the rest to the cloud. For fair comparison, I believe the methodology should be consistent among the baselines and proposed system. 
* While Section 5 offers the analysis for the implication of shadow model, the experiments still use quite small, simple, and outdated models such as VGG-13, VGG-16, and ResNet-18, which do not seem adequate for demonstrating the robustness on the different model architectures. 
questions: * Currently, the manuscript is written in such a way that there are various assumptions that seem to make the proposed technique work. Moreover, the experiments are performed in quite specific setups, which makes me question about the generality and practicality of proposed solution. Could you please elaborate on this?
* Especially, the model split point should play a significant role to evaluate the effectiveness of proposed solutions and make comparisons with the existing techniques. I believe, to be fair, the split points should be similar to what prior works used. Could you please comment on this?
* If I understood correctly, the attackers not only have model's behavior (functionality), model architecture, and hyperparameters a priori, but also have the labeled training data for ground truth that includes possible image inputs and corresponding intermediate feature map results. Should the attackers have all these conditions satisfied to try an attack, or are some of these conditions optional?  
limitations: Elaborated above in the weaknesses part.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: Existing works in ARL have developed different types of techniques for protecting sensitive information like sensitive attribute and sensitive input. For the sensitive input category, this authors develop an attack that can undermine the security of the existing defense techniques. Motivated by this observation, they introduces a new defense technique that perform favorably on their attack algorithms.
strengths_and_weaknesses: Strength -
- Sensitive feature distillation is an interesting and new idea in the ARL community.
- The experimental data highlights the vulnerability of the ARL approaches to protect inversion of inference data.

Weakness -
- The paper shouldn't call the attack techniques as model inversion since the goal is to infer sensitive information from the data used for prediction query and not the training data used.
- Wrong characterization of early MI attacks by Fredrikson et al. 2014 and Fredrikson et al. 2015 that extract sensitive attributes (Fredrikson et al. 2014) and reconstruction of original data (Fredrikson et al. 2015).
- While Mireshghallah et al. 2020 does add noise, it does not offer any differential privacy guarantee.
- Dropout Defense for preventing reconstruction of visual data has been explored in the context of convolution channel obfuscation [1].
- L249-250 assert that existing techniques do not leverage feature level data. I think this may not be true w.r.t the attacks used in [1].
- This paper does not compare with other classes of defense techinques such as [2,3,4,5,6].


1. Singh, A., Chopra, A., Garza, E., Zhang, E., Vepakomma, P., Sharma, V. and Raskar, R., 2021. DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12125-12135).
2. Rezaeifar, S., Voloshynovskiy, S., Asgari Jirhandeh, M. and Kinakh, V., 2022. Privacy-Preserving Image Template Sharing Using Contrastive Learning. Entropy, 24(5), p.643.
3. Osia, S.A., Shamsabadi, A.S., Sajadmanesh, S., Taheri, A., Katevas, K., Rabiee, H.R., Lane, N.D. and Haddadi, H., 2020. A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet of Things Journal, 7(5), pp.4505-4518.
4. Vepakomma, P., Singh, A., Gupta, O. and Raskar, R., 2020, November. NoPeek: Information leakage reduction to share activations in distributed deep learning. In 2020 International Conference on Data Mining Workshops (ICDMW) (pp. 933-942). IEEE.
5. Li, A., Guo, J., Yang, H., Salim, F.D. and Chen, Y., 2021, May. DeepObfuscator: Obfuscating intermediate representations with privacy-preserving adversarial learning on smartphones. In Proceedings of the International Conference on Internet-of-Things Design and Implementation (pp. 28-39).
6. Liu, S., Du, J., Shrivastava, A. and Zhong, L., 2019. Privacy adversarial network: representation learning for mobile data privacy. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(4), pp.1-18.
questions: How is your contribution 1  (L53-55) different from contribution 3 (L59-63)? I believe both refer to the experimental evaluation only.
limitations: The authors may wanna highlight limitations of their empirical evaluation but I do not see any major concern
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes an attack model for edge-cloud collaborative inference systems. More specifically, the authors develop a technique for evaluating defense mechanisms against model inversion attacks, where the attacker uses obfuscated features (computed on a trusted local edge device and to be fed to an untrusted cloud service for prediction) to recover users’ private data (i.e., the raw image). To this end, the authors propose a technique called Sensitive Feature Distillation (SFD) to sanitize obfuscated features generated by a defense technique, which are then fed to an Existing Model Inversion (EMI) attack to restore the original image. The proposed technique is evaluated against three defense techniques (adversarial learning, noise mask, dropout) on three datasets (MNIST, CIFAR10, CelebA), where it is shown to outperform a baseline EMI attack (owing to the SFD model).
strengths_and_weaknesses: I enjoyed reading this paper. The authors have developed a novel technique that can be used for evaluating defense techniques in edge–cloud collaborative inference systems. The proposed method is well-described and technically sound, and the black box attack setting makes this applicable to a wide range of scenarios. The evaluation results also reveal that current defense mechanisms cannot adequately defend against adversarial attacks, similar to many other domains.

One weakness of this type of inversion attack is that the recovered image tends to look more like a generic image of the same class (see the Figure 3b, specially the bottom rows) rather than the original image, and the used metrics somewhat fail to capture this distinction. So perhaps the privacy issue is less severe than what the actual numbers suggest. Nevertheless, the proposed technique is novel and succeeds in outperforming the baseline, which makes it a solid contribution for evaluating defense techniques.

Minor comments:
- Line 315: "Strong defence does not help, the performance of our attack increases along with the hyper-parameter increases", isn't the performance of the attack decreasing (lower SSIM) with larger hyperparameters?
- Line 301: defense -> defence, for consistency! :)
- Line 328: "When repeating the experiments on other datasets or other", typo?
- Section numbers in the checklist are unresolved.

questions: - I'm interested to see if an SFD trained on one dataset can transfer to another. This would further relax the constraints on the attacker since they don't always have access to data with the same distribution as the training dataset. Have the authors performed any such experiments, or do they have any insight on the robustness of the SFD in this setting?
- Figure 3b, the reconstructed images look more and more like generic images as you go down the rows. Is this also observed on other examples? Perhaps the authors could show some examples with the same image in all rows in the appendix.
limitations: The proposed method has very few constraints as it only requires blackbox access to the target model, and access to data with the same distribution as the training dataset for training the shadow model. This is discussed in good detail in the paper.
review_title: null

Meta-review: metareview: The paper presents a new model inversion attack for edge-cloud learning which is shown to overcome existing defenses against model inversion in such scenario. The attack is based on Sensitive Feature Distillation (SFD) which simultaneously uses two existing techniques, shadow model and feature-based knowledge distillation. The authors claim that the proposed attack method can effectively extract the purified feature map from the intentionally obfuscated one and recover the private image for popular image datasets such as MNIST, CIFAR10, and CelebA.
The proposed method is quite novel and its evaluation is solid. The method is based on certain assumptions and utilizes several techniques from related work in order to meet these assumptions, which makes the contribution rather incremental. The paper is well-written and the relationship to existing work is clearly presented and discussed.  
confidence: Less certain
Decision: Accept