Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies robust inference under the vertically split features setting, which appears in applications like e Internet-of-Things (IoT). The paper considers three types of threat models, including distributed adversarial attack, missing feature attack, and combined attack. The paper proposes a sparse recovery-based robust inference algorithm and provides recovery guarantees. The paper provides experimental results to show the superior performance of the proposed method.
strengths_and_weaknesses: Strengths:

(1) The problem of robust inference under the vertically split features setting is new and interesting.

(2) The proposed method has recovery theory and better empirical performance.

Weakness:

(1) The idea of sparse recovery-based algorithm design is not new.
questions: The recovery theorems rely on critical hyperparameters like delta, which is empirically tuned on a validation set according to the supplement. Is it possible to provide some theoretical guidance to choose such hyperparameters so that the certified recovery is always ensured?
limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper aims to improve the robustness of the deep learning model for collaborative inference. The authors propose a pre-processing-based defense method CoPur against inference phase attacks. The proposed method first leans the purified embedded features from the observed embedded features by using a generative model.Then, the purified embedded features are fed to the Fusion Center for prediction. This paper provide sufficient theoretical analyses to illustrate that the proposed method is able to recover the underlying true features exactly. Empirical results for targeted and non-targeted attacks are presented in this paper, which is used to demonstrate the effectiveness of the proposed method.
strengths_and_weaknesses: Strengths:
1. This paper proposes a certifiable defense to defense against inference phase attacks for the collaborative inference task. This method is based on the pre-processing strategy, which does not require much computing cost and does not change the original information of agents.
2. This paper provide sufficient theoretical analyses to demonstrate that the proposed method is certifiable. The authors analysis two cases (i.e., noiseless case and noisy case) and use them to illustrate that solving some relaxed objectives is able to recover the underlying true features exactly.
3. The authors design a non-target attack (i.e., distributed feature-flipping attack) against the collaborative inference model to evaluate the effect of the baseline defense models and the proposed defense model. This helps to evaluate the effectiveness of adversarial defenses.

Weaknesses:
1. The authors emphasize that in their considered inference setting, the features obtained by the agents are quiet different and the redundancy is less apparent. However, the authors seems not to introduce how to design a specific mechanism for this problem, and how the proposed method handles the issue of less redundancy in detail. In addition, the authors do not discuss the performances of the proposed method under different redundancy situations.
2. IoT applications are often communication-limited. The authors do not evaluate the communication requirements of the proposed method, and they do not show whether the improvement mechanism would increase the communication requirements.
3. Some closely related works are not discussed, and some closely related methods are not compared. Some import attacks are not used to evaluate the defense.
4. Writing and presentation need improvement. For example, the caption in Fig. 1 does not clearly describe the necessary information in the figure, such as what does the variable represent? In addition, \Omega_{adv} is the index for all corrupted features (\Omega = \Omega_{benign} U \Omega_{adv}), but in Eq. 3, it becomes the index for perturbed features, |\Omega_{adv}| in Eq. 3 should be \alpha.
questions: 1. Why are the well-known robust aggregation methods used for HFL not applicable? Please provide some further explanations.
2. The authors say they propose to improve robustness based on the underlying redundancy among the features held by the IoT devices. However，it is not very clear to me how the proposed method obtains and exploits redundant information, is it through an autoencoder? 
3. The method proposed by the authors seems to be mainly aimed at off-manifold attacks. Is the proposed method effective against on-manifold attacks [1]? 
4. PGD is also suitable for crafting non-targeted adversarial noise, why do the authors not consider the non-targeted PGD? In addition, the Autoattack [2] is a more powerful attack, the authors can use it to futher evaluate the effectiveness of the proposed defense.
5. The poposed method is a pre-processing-based method, but the authors do not compare it with other pre-processing-based adversarial defense methods, such as [3].
6. The authors do not consider the stronger adaptive attack [4]. For example, the used autoencoder may also be attacked. The authors should design adaptive attacks to more comprehensively evaluate the robustness of the proposed method.

[1]. Disentangling adversarial robustness and generalization. CVPR, 2019.
[2]. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML, 2020.
[3]. A self-supervised approach for adversarial robustness. CVPR, 2020.
[4]. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. ICML, 2018.


-------------------------------------------------------
After rebuttal

The authors solve most of my concerns, so I am willing to modify the score to borderline accept.
limitations: The authors do not consider some closely related works and some import attacks to further show the effectiveness of the proposed method.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work focuses on feature fusion in distributed machine learning scenarios. In this work, a distributed feature reconstruction method is proposed to recover noise / incomplete features committed by different local clients. The main idea is to decompose observed uncorrupted features into the underlying noise-free features and additional block-wise sparse noise corruption components. After that, a L2,1 signal recovery method is applied to estimate the noise-free feature profiles by enforcing the L2,1 norm sparsity regularisation over the estimated noise components. In parallel, the feature recovery process is accompanied with the learning of an AutoEncoder architecture performed over the recovered feature vector. The goal is to generate a smoothed feature representation from the recovered features for the final classification step. Theoretical and experimental study demonstrate the provably effectiveness of the proposed feature reconstruction method. 
strengths_and_weaknesses: Strong points:

1.  Recovering the noise-free feature profiles provides less noisy information for classification, compared to aligning noisy features to the subspace of normal data. 
2.  It is decent to make use of the block-wise sparsity structure of the noise components and formulate the feature recovery problem as the L2,1-regularized regression process. 
3. Theoretical study established for the noisy case helps clarify the practical implication of the proposed method. 

Weak points: 
1. Though it is interesting to see a novel feature-corruption-based attack method is proposed, the link between designing this attack algorithm and the feature recovery method (CoPur) is not clearly explained. 
2. How does the proposed CoPur method perform agains the distributed feature-flipping attack? Is it the combined attack in Table.2? 
3. How many malicious agents can the proposed CoPur defense method tolerate ? It is related to the break point of the proposed method. 
questions: Please find the questions in the second and third points in the listed weakness. 
limitations: Please find the listed weakness points.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposed a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold). This paper also gave theoretical results showing that the proposed method can robustly recover the true feature vector.  Extensive experiments showed the effectiveness of the proposed method. Besides, an untargeted distributed feature-flipping attack was proposed.

strengths_and_weaknesses: Pros
1. Theoretical analysis is given, showing that the recovered benign feature is close to the true feature with bounded error.
2. Feature-flipping attack is proposed to demonstrate the weakness of manifold projection defense.

Cons
1. Theoretical results are not adequate. 
2. Lack of experimental results.

questions: Q1: The main contribution (Eq.6) works out based on the prior knowledge that adversarial feature $e$ has a block-sparse structure. However, theoretical results didn't formalize the sparsity of $e$ clearly enough.

Q2: The proposed feature-flipping attack is not well-evaluated. Experiments showed that the proposed defense has better accuracy against this attack compared to other defenses. However, no experimental results are given to indicate that this attack is stronger than other attacks.

Q3: How sparsity of adversarial feature $e$ affects the robust accuracy is not evaluated.

Q4: Figure 2 shows that when amplification is low, CoPur has higher robust accuracy than Oracle. Can you explain why this phenomenon happens? If CoPur is applied to unattacked features, will it improve the standard accuracy? 

Q5: Why the solution to Eq.6 is not mentioned in the main text?

limitations: No limitations discussed in the paper. See questions.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper considers the problem of robust collaborative inference. This paper proposes a feature purification method for this problem. With the block-sparse structure, the authors propose a non-linear decomposition method to identify the corrupted agents. Experiments on the two datasets show the effectiveness of the proposed method. 
strengths_and_weaknesses: Firstly, I have to say that I’m not an expert in this area. The review could be misleading.

## Strengths:

1.This paper considers a practical and interesting problem, what if some agents are attacked and send corrupted features to the server.

2.This paper proposes a non-linear decomposition method for identifying poison agents, which is reasonable and should be effective.

## Weakness:

### Originality:

1.I want to know if this paper is the first time to study the problem of the robust collaborative inference, where there are both arbitrary agents and adversarial agents. The arbitrary agents are easy to identify. However, I’m afraid the proposed method achieves a similar performance to identify the adversarial agents compared with baselines.

2. From Eq.(5), the framework aims to find a combined feature $\boldsymbol{l}$ which is on the manifold and is near $\boldsymbol{h}$. The manifold projection could get a similar results for the adversarial sub-features. Could the authors discuss more about it? 

### Writtings:
1.After so many times of reading, I guess I understand this paper. The authors introduce their method in Section 2.3, which is very simple. However, it relies block-sparse structure which is detailed stated in Section4. This could cause confuse when understanding the proposed method.

2.The notations are confusing. For example, $\boldsymbol{h}$ and $\boldsymbol{l}$ both denote the feature. Why not use a letter (or with its variants)?

3. the citation format may be ICLR rather NeurIPS.

### Theoretical analysis:
1.This paper provides an extensive theoretical analysis. In fact, I suggest the authors discuss more what the analysis means. Compared with baselines, why CoPur could do better.

2.Could the authors give an intuitive explanation about the effect of the sparsity $\alpha$ on CoPur?


### Experiments:

1.From the ablation studies, CoPur achieves a better performance compared with the manifold projection, what if there are different $\Omega^{c}$ and different $\Omega_{adv}$?

2.More analysis is helpful, for example, The comparison on optimization efficiency.

questions: Please see weaknesses.
limitations: The authors discuss the limitations in Appendix. I have no other suggestions.
review_title: null

Meta-review: metareview: This paper focuses on improving the robustness of the model for collaborative inference. A pre-processing-based defense method is proposed against inference phase attacks. Both theoretical analyses and empirical evaluations are provided to demonstrate the effectiveness of the proposed method. 

In the review process, mixed reviews are achieved. Four reviewers are positive about this paper, while one reviewer is negative. Reviews appreciate the strengths of the paper: (1) the idea is interesting; (2) the pre-processing strategy does not require much computing cost; (3) the theoretical analysis is detailed and solid; (4) empirical evaluations are overall convincing. 

One reviewer who is negative about this paper points out major concerns: (1) theoretical results on the sparsity; (2) empirical evaluations on the sparsity; (3) the problems in the presentations of the paper. Major concerns have been addressed during the rebuttal. We suggest the author carefully merge the rebuttals in the final version.
confidence: Less certain
Decision: Accept