CoPur: Certifiably Robust Collaborative Inference via Feature Purification
Authors: Jing Liu
Chulin Xie
Oluwasanmi O Koyejo
Bo Li
Venue: NeurIPS 2022 Accept (2022)
Keywords: Robust Collaborative Inference
Feature Purification
Adversarial Machine Learning
Vertical Federated Inference
Robust Decomposition

Abstract: Collaborative inference leverages diverse features provided by different agents (e.g., sensors) for more accurate inference. A common setup is where each agent sends its embedded features instead of the raw data to the Fusion Center (FC) for joint prediction. In this setting, we consider the inference-time attacks when a small fraction of agents are compromised. The compromised agent either does not send embedded features to the FC, or sends arbitrarily embedded features. To address this, we propose a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as exploring the underlying redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold). We theoretically show that the proposed feature purification method can robustly recover the true feature vector, despite adversarial corruptions and/or incomplete observations. We also propose and test an untargeted distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as the features held by other agents, and is shown to be effective in attacking state-of-the-art defenses. Experiments on ExtraSensory and NUS-WIDE datasets show that CoPur significantly outperforms existing defenses in terms of robustness against targeted and untargeted adversarial attacks.

CoPur: Certifiably Robust Collaborative Inference via Feature Purification
Jing Liu Department of Computer Science University of Illinois at Urbana Champaign j $1129281llinois edu edu
Chulin Xie
Department of Computer Science University of Illinois Urbana Champaign chul inx2@i1 www.iiiiiii.....m.mm .dd
Oluwasanmi O Koyejo
Bo Li
University of Illinois at Urbana Champaign & Stanford University & Google Research canmi@stane edu
Department Computer Science University of Illinois at Urbana Champaign 1bo@illinois. edu
Abstract
Collaborative inference leverages diverse features provided by different agents (e.g., sensors) for more accurate inference. A common setup is where each agent sends its embedded features instead of the raw data to the Fusion Center (FC) for joint prediction. In this setting, consider inference phase attacks when a small fraction of agents compromised. The compromised agent either does not send embedded features to the FC or sends arbitrary embedded features. To address this. we propose a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as redundancy across the embedded features (by assuming the overall features lie manifold). We theoretically show that the proposed feature purification method robustly recover the true feature vector, despite adversarial corruptions and/or incomplete observations. We also propose and test an untargeted distributed feature-flipping attack, which is agnostic the model, training data, label, well as features held by other agents, and is shown to be effective in attacking slate-ofthhe-ae defenses. Experiments on ExtraSensory and NUS-W WDSE datasets show that CoPur significantly outperforms existing defenses in terms of robustness against targeted and untargeted adversarial attacks.
underlying lower dimensional
Introduction
Collaborative inference is an increasingly popular distributed prediction strategy that leverages diverse features provided by different agents (e.g.. sensors) for more accurate inference. For example, in context-aware music recommendation the app may wish recommend more appropriate music based on the user's 's behavioral context (e.g., sleeping, running) (Vaizman et al., 2017). Here, it can leverage diverse sensors from the user's smart phone and smart watch, e.g., phone accelerometer, gyroscope, watch accelerometer, location, audio, phone states (e.g., wifi, ringing mode), and environment sensing (c.g.. light). Thus, by combining information across sensors, much more accurate predictions can be made. In this work, we study the scenario where Internet-of of -ffenettof-Thnnnss (IoT) devices send embedded features (instead of the raw data) Fusion Center (FC) for collaborative inference. The IoT devices do not communicate with each other and may hold quite different raw data (e.g.. image, acoustic, Lidar data, etc). Nevertheless, as a result of distributed inference, the system vulnerable to attack, where a subset of sensors are corrupted by an adversary at inference time, e.g., due to the lack of effective security mechanisms, connection errors, sensor failures, among other issues this is also a
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

common concern in defense applications (Abdelzaher et al. 2018 Xie et al. 2020). Nevertheless, the goal of the system is robust inference based on the possibly incomplete and corrupted embedded features received from the IoT devices.
There have been number of works on robust parameter estimation in IoT settings, where the robustness apparently comes from redundancy among different sensors' linear measurements of the same parameter. For comprehensive overview, we refer the interested reader (Zhang et al.) 2018). However, in our considered inference setting, each IoTT device may hold quite different types of features, and the redundancy is less apparent. We also observe that loT applications are often communication-linn limited (Ko et al. 2018), thus, using lower- diveeniiensionall embeddings to communicate from each sensor a common and effective approach reduce the required communication overhead.
The problem we study here also applies to the inference phase of some Vertical Federated Learning (VFL.) frameworks (Chen al. 2020 Liu al. 2020), where server similarly combines different embedded features received from every agent to make a prediction. One difference between our setting and VFL is: the agents in VFL are other untrusted parties, if agent in VFL is malicious, it will likely attack during both training and inference; while in our setting, the agents (e.g.. loT sensors) are all owned by FC, and they can be well protected during training, and inference is more vulnerable. So far, there are no certifiable defenses for VFL against adversarial attacks during the inference phase. The well-known robust aggregation method (Yin al.
2018
Guerraoui et al. 2018
Blanchard et al. 2017] Fu et al. 2019 Pillutla et al. [2019] Fung et al.
2020 Chen et al. 2017 Xie
ct al. 2021 used for Horizontal Federated Learning (HFL) not applicable.
Other related works will be discussed in Section 4 We use the term 'device' and 'agent' interchange- ably in this paper. The main difficulty of defending against adversarial attacks in collaborative inference setting (as well as in VFL) is that there is no apparent redundancy across the agents. Though, intuitively, there is some mutual information among the features held by each agent. Even worse, the raw data held by the agents are not accessible by the FC/server to inspect, the attacker can perturb the compromised agents' embedded features with very large magnitude before sending them to the FC/server.
In this paper, we propose improve robustness based the underlying redundancy among the features held by 1oT devices. Specifically, we assume the combined features lie on an underlying lower dimensional manifold. We propose robust COIlaborative inference framework via feature PURification (CoPur), which is able to defend against many types of attacks during inference (see the attack taxonomy in Section 2.2 During the inference, the FC receives the corrupted and/or incomplete embedded features from agents (Figure Ib). Then CoPur purifies the embedded features based on the learned feature subspace (Figure (p.). Finally, it feeds the purified features to FC's trained global model for prediction. The detailed robust inference procedures will be described in Sections 2.3 We aim to answer the following questions: Can we make robust predictions based on corrupted and/or incomplete features? If yes, how many compromised devices can we tolerate?
Fusion Center
test Missing
Aest
PS X
Learned Feature Subspace
Agent xfme
Agent
Agent
Agent
(b)
Figure 1: (a) Fusion Center receives corrupted and incomplete embedded features; (b) Feature Purification.
Technical Contributions: Our main contributions include the following:
We propose certifiably robust collaborative inference procedure to defend against a variety of targeted or untargeted attacks (e.g., distributed adversarial attacks, missing- attack, and their combinations) during collaborative inference.
We propose a novel non-linear robust decomposition method that decomposes the potentially corrupted and incomplete feature vector into two parts: one lies (exactly or approximately) on the underlying feature manifold; the other with block-sparse structure. Theoretically, prove

---

that the proposed method can recover the true feature vector exactly (or approximately) despite corruptions and incomplete observations. This may be of independent interest.
We propose and test distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents. Empirical studies show that this new attack effectively attacks existing offth-of-th-arr defenses.
We conduct extensive experiments on ExtraSensory and NUS-WIDE datasets, and show that CoPur is significantly more robust than baselines against different types of attacks.
2
Robust collaborative inference via feature purification (CoPur)
We describe the system setting, the threat models, and the proposed robust inference procedures.
2.1 Collaborative inference and system setting
We first describe the basic framework of collaborative inference, which is also similar to the inference phase of some VFL frameworks (Chen et al., 2220; Liu et al. (2020). There are M agents, where each agent i holds partial raw data o! of the overall testing sample ot [arfest ....; ..."))) testt)] Also, each agent i has its own local feature extractor fa parameterized by 0g that maps its raw data vector aestt to the embedded feature vector h! _ fa 0z). The dimension of h is usually smaller than ztest. The FC only receives the embedded features h i 1.... M from the agents (not the raw test Then data af and concatenates them into long column vector hest & [h/rstt fest ....; MM the FC uses its pre-trained global model fe, to make the prediction.
While we do not focus on training, we will require AutoEncoder, trained on uncorrupted data (e.g.. during training) that can approximately capture the manifold of the uncorrupted embedded features, i.e., E (1*)) t' l2 I 8 for the true embedded features 1* of the testing instance, where D& is the decoder parameterized by Φ, and Ey the encoder parameterized by v.. Further, we assume that the global model for inference fa, is based on the output of the AutoEncoder and the Detailed examples of how to train such a model and the AutoEncoder can be found in Section of the supplemental material.
training labels, ie.. (D (E (f)))), y() jn_ b (E+)))))
2.2 Threat model
There are M agents which hold different parts of the feature of the same set of n training instances. We assume the training process well-protected (attackffree). However, during the inference, the attacker can attack a.M agents. The compromised agent either is not able to send features to the FC, or sends arbitrary embedded features to FC.
We use heenign to denote the overall embedded features provided by the (1 - -aM benign agents (these agents are indexed by Zeenign and [Ssenggn| (1 a ), and we use hade denote the overall embedded features sent by compromised agents (indexed by Padv). The set Ω = Phenign Radv is the index set of agents that provide features, and its complement, i.e., Ω denotes the compromised agents which do not provide features. We categorize the inference phase attacks
into the following three types:
Threat model A (distributed adversarial attack): All the compromised agents jointly send adver- sarial embedded features hadv. with the goal to mislead the prediction to the target label y
min E(Foo (1 (thades (benigg)), B)
hads
Threat model B (missing-feature attack): All the compromised agents do not send any feature to the FC. A such missing-feature attack may be due to damaged sensors or disconnection, and can still affect FC's prediction. The presence of missing-feature attack can be viewed as the FC observing only partial blocks (indexed by Ω) of the overall embedded features h, i.e., ha. Ω2 denotes the index set of agents that perform bassing-feature attacks. The attacker can compromise up to a. agents to perform such an attack:
min {fee, (hn). ), | S aM
f2"
Threat model C (combined attack): The attacker can also perform a combined adversarial and missing feature attack by providing hadv, indexed by Radv. and missing features 2
min (fhade, hbenign }), y 1), s.t. | + [aatvl S aM
(3)
n hade
Beyond these targeted attacks, the attacker can also perform untargeted attacks by maximizing the distance between the prediction and the ground truth label.
3

---

2.3 Inference procedures of CoPur
We propose an all-in-one solution to defend against various inference phase attacks defined in Section 2.2 Let Ω be the index set of the observed blocks. The observed ha can be decomposed as h2 o + eq where is the underlying uncorrupted features, and en models the corruptions on the observed set S?. Finding such decomposition seems hard, even if the learned AutoEncoder can capture the underlying feature subspace, i.e., (o(Ex(()) Fortunately, as we mentioned in Section en has block-sparse structure. We propose solve the following:
min
Σ 1[ 1. + 0} s.t. (o(E = IEN
(4)
where the indicator function 1(-) counts whether the block [h - 1 is nonzero. This black-sparsity minimization shares the same spirit the lo minimization in robust linear regression (Candes & Tao 2005
Mitra et al. 2013). However, it is NP-hard to solve, so we relax it to the &,, -norm:
min Σ l[
Qell
s.t. ()(E,(()) =1
(5)
Our theoretical analysis in Theorems] 1 2 shows that under certain conditions, solving the above relaxed objective is able to recover the underlying true features exactly.
But what if the learned AutoEncoder can only approximately reconstruct the underlying features, say ΣN (E)(E((()) Ujil22 8 In this case, further relax the constraints in Eq. 5 and solve:
min Σ 1 1]il12 EE
s.t. Σ (() 4:lla 8
Our theoretical analysis in Theorems & show that under certain conditions, solving Eq. 6 able to approximately recover the underlying true features, despite the corruptions and/or incomplete observations. In our implementation, embedded features on the manifold, i.e., ming Σjeo ||[h (e via gradient descend (It useful to notice that since Eq. fallowss I approximately lie on the manifold, its objective value will be slightly smaller than miny Lien [| - (E, (ll)) Then we apply Lagrange Multiplier
first try to find good initial point by simply searching the
method solve Eq. i.e., i = arg min: Σien 1| 12l + + ΣN M implementation details can be found in the supplemental.
(Es 1/112. More
After obtaining i, we feed (t) to FC's trained global model for the prediction.
3 Theoretical analysis of CoPur
We first analyze the 'noiseless case' where the learned AutoEncoder satisfies (E,) = 1" for the underlying uncorrupted feature vector 1". Next, we analyze the more challenging 'noisy case' where the AutoEncoder can only approximately reconstruct L* i.e., ΣM (E - -1]ill 8.
Proofs and additional discussions can be found in Section of the supplemental material.
3.1 Noiseless Case
To simplify the exposition, we first consider threat model B, i.e., the missing-feature attack.
Theorem 1. (Exact feature recovery under threat model B) Assume the trained AutoEncoder satisfies ( (t*)) = I" for the underlying uncorrupted feature vector I* For Vr', ,r',"" the range of the decoder (-) where r' of r", if (r' r")a + 0, then given hn 12 1 is the unique solution of Eq. and Eq. 4
The proof can be found in Section A.3 of the supplemental material, where we show that any feasible point that different from would have a larger objective value.
The condition (r' r")n A 0 required by Theorem I is intuitive and necessary. Suppose there are r' ن r but r/ = r/ then observing r'2 not able to tell whether it's from r' or r" In general, when the observed set Z gets larger, the required condition has better chance of being satisfied. we have complete observations, iee,, Ω {1, ..., M). the required condition (r' a o automatically holds.

---

Now we consider the more challenging case where we not only have missing features but also have corruptions on the observed features.
Theorem 2. (Exact feature recovery under threat model C) Assume the trained AutoEncoder satisfies (Eg = 1" for the underlying uncorrupted feature vector 1". For Vr', yr',"" the range of the decoder (+).) where r' * r", define v r' r", if for any partition {SSo. Sq of Q2 with
|8p - 4 < /21/2, 2, it holds that Eiesp ||0:|12 Lies, 110:112. then for any hn t + eq with Eico 1 (ee + 0} S q. l is the unique solution of Eq. and Eq.
The proof can be found in Section of the supplemental material. We first show that any feasible point of Eq. 5 that is different from 7" would have a larger objective value. Then we use contradiction to show that there does not exist any global optimal solution of Eq. 4 that is different from l".
Remark 1. Theorem 2 not only requires vo + 0 like Theorem | but also requires Eiesa lles >
Lies, ||0, This is due to the presence of corruptions in the observed blocks (indexed by Sa). When there is no corruption, i.e. q = 0, the corresponding set 5gg is empty and the required condition reduces to Eien ||v:||2 O, i.e., on j 0. Second, Theorem 2 is universal certification, where any subset Sq of the observations can be malicious, as long as |Sal
q. In practice, once the attacker chooses q agents to perform adversarial attacks, the set Sq gets fixed and corresponds to e, * Then, we only require Diesa |u:112 Dieso l/0/|lz to hold for this particular fixed set Sig. to guarantee 1" is the unique solution of Eq. This can be easily seen from the proof of Theorem 2 Third, the condition Eieso |e:112 Eiesa |lll1 prefers an to be not 'spiky'. Suppose r' and rd are the same on all the observed blocks (indexed by !2) except one, which means only one block of on is non-zero (i.e., un is spiky). If that non-zero block is in the set Sn, the required condition will not be met. Intuitively, if there are two instances differing in only one block of the observed features, and the attacker corrupts that block, the FC can not recover the original feature. In general, we prefer do to be not 'spiky,, thus the condition can hold when the number of corrupted blocks |Sal is sufficiently small. Such condition can be considered as the non-linear and missing-block extensions of the Range Space Property in robust linear regression literature (Flores) 2015 Liu et al. 2018), which we will discuss next. Finally, the fraction (g/ .M) of corrupted blocks that CoPur can tolerate not only depends on Ω2 but also depends the underlying feature subspace. It can tolerate up to 50% x [52] corrupted blocks there is enough redundancy among the agents, e.g., if every agent holds exactly the same embedded features, then every block of r' - r" p is the same, and q approaches 50%x [2))
Remark 2. To draw the connections to the Range Space Property in robust linear regression, let's consider a typical case where there is no non-linear activation function in the last layer of the AutoEncoder (other layers can have non-linear activation functions). Let Wend be the weight matrix of the last layer of the AutoEncoder. Note that the difference between any two vectors from the range of the decoder (-(-) is contained in the Range( W Wenn), i.e., v E Range W end (mn))0 O... Similar to the so called 'leverage constant' in robust linear regression (Flores 2015 Liu et al. 2018). we can define the Excs, [.enn=))lla
block-wisc 'incomplete' leverage constant Cn (Wend) min [S|= m Eieo [Wen.=]lll which between and and is monotonic increasing when the number of corrupted blocks q decreases. A sufficient condition for Eisso ||0:|e Eisso ||0:112 >> O to hold is ca, (Wend) >> 0.5. Lastly, very useful to note that [Sharon et al. (2009) designed an algorithm to calculate the leverage constant, which is possible to be extended to our block- wise incomplete leverage constant here. Consider the toy example again where every agent holds exactly the same embedded features, and g 0.1 x [S), then Co., (Wen) (end))= = 0.9.
Remark 3. It interesting to note that in the recent Generative model based compressive sensing work (Bora et al.) (017). they also impose conditions on the difference between any two vectors in the range of the decoder, though without missing blocks. But the conditions therein are quite different from ours due to the different purposes.
3.2 Noisy case
Now we analyze the 'noisy case" where the learned AutoEncoder can only approximately reconstruct
the underlying uncorrupted feature vector L" i.e., ΣM I| (E (I") - lell < 8. Again, to help the reader understand step-by-step, our analyses start from the threat model B.
Theorem 3. (Stable feature recovery under threat model B) Assume the trained AutoEncoder
satisfies Σ. (E)) - r']lll2 < 6 for the underlying uncorrupted feature vector t". Let i
5

---

be the solution of Eq. 6 Given the incomplete observations ho = li. where f2 is the index set of observed agent blocks. For Yr', r" in the range of the decoder (-).), define v == r", if holds
that Eien 10:122 > 28 for V/|u|12 > A, then (t) - Do (1(E) S..
The proof can be found in Section A.5 of the supplemental material.
Remark 4. First, note that EM
||: 2 v V | || When ||0||2 > A, we have
Σ ||e//12 > A. Again, we prefer v to be not 'spiky'. , thus the condition Σien ||0:12 > 26 can be
satisfied as long as [S) is large enough (assume A >> 26).
Now we consider the most challenging case where we not only have missing features but also have corruptions in the observed features.
Theorem 4. (Stable feature recovery under threat model C) Assume the trained AutoEncoder lelll I 8 for the underlying uncorrupted feature vector L"_ Let i be the solution of Eq. 6 Given the incomplete observations ha 12 + Eq. where 52 is the index set of observed agent blocks. For Vr', rr in the range of the decoder (.(.) where r' * r". define = r
satisfies Σ (E&
r", if for any partition {Sn, 3 of with [Sol = q. it holds that Eieso ||0:|12- - -ii |0::12 llla > 28 for v/ |/|l=||| A, then (Ex (D (1)) A long Eien 1 (e, * 0) S q.
The proof can be found Section A. the supplemental material.
Robust prediction. Recall that the server has learned a classifier Se, during the training. We can further smooth this classifier fer to get fa (e.g., via Randomized Smoothing (Cohen et al. 2019 Lecuyer et al., 2019 Li et al., 2018)) such that ( (t)) - (D - v) for 0| S A, then it is guaranteed that (. (i)) =
A. As we have (Ex - (E S (E
Remark& 5. The condition Eiesg |1::112 -iiss 110/122 > 28 required by Theorem 4 is stronger
than the condition Σien ||vil22 > 20 required by Theorem 3 This is due to the presence of corruption
in the observed blocks. As discussed in Remark 4 when ||r||22 > A, we have ΣM ||00 > A. Again, we prefer to be not 'spiky', thus Eien ||æ:l12 can have significant portion of mass
from E |11/ when the number of observed blocks |S]] is large. Further, when the number of corrupted blocks & = |Syp is small enough, we can have Eies, ||v:]|2 - -iies ||v:]la > 28. To draw the connections to the Range Space Property in robust linear regression, we continue on the discussions in Remark 2 Let's say Eteq ||0,1|2 = A', then a sufficient condition for ||0 > to hold is (co, (W Wand) - (1 -n, (Wend))] > 25/2'. Intuitively, the smaller number of corrupted blocks and smaller d (i.e., better recovered feature subspace) as well as larger observed set 92 (then A' approaches A), can lead smaller recovery error bound A.
Eies,
Related work and discussion
Handling missing data and outliers in 1oT. Sanyal & Zhang (018) model the underlying uncor- rupted loT data as a cow-dimensional linear subspace, i.e., low-rank, and try to clean the corrupted sensor data by projecting them onto the estimated linear subspace. There is guarantee of recovering the underlying uncorrupted sensor data. This approach is a linear version of the manifold projection method that will discuss in detail.
Kekatos & Giannakis T)0T)) studied the robust sensing problem where each sensor i has a linear measurement of the unknown parameter æ. i.e., b, A,æ,, where both b, and A; are known and some sensors are additionally corrupted by large measurement errors, but without any missing values. They showed that it is possible to recover the true parameter æ despite outlier corruptions. They also consider the scenario with dense inlier noise in the mcasurements, i.e., where the measurements b; only approximately lie on the range space of matrix A,, but their corresponding method does not have any recovery guarantee in such a noisy case.
There are also some works that study the scenario where the sensors observe different noisy versions of the same data vector, and they show it is possible to find a set of unattacked sensors based on the assumption that the fraction of attacked sensors is small, e.g., (Zhang et al.] 2018) or utilizing additional knowledge that some sensors are not attacked (Wilson Veeravalli 2016). We refer interested readers to the survey paper (Zhang al. 2018) for additional details this line of work.

---

Certified robustness for defending against adversarial attacks. In the adversarial machine learning literature, many defense methods have been proposed to deal with adversarial perturbations. We refer the reader to (Chakraborty et al. (2018) for a comprehensive survey. However, most of the existing defenses can only deal with small adversarial perturbations. Even worse, as new defenses are proposed, corresponding new attacks are also developed to adaptively attack them. Experimental demonstrations of a defense's s efficacy based on currently existing attacks do not provide general proof of security. Therefore, certifiably robust defenses are of great interest. Randomized Smoothing (RS) is a promising technique (Cohen et al., 2019; Lecuyer et al., 2019; Li et al. 2018) which smoothes base classifier to get robust classifier, such that the prediction of this robust classifier on adversarially perturbed input is the same as its prediction on the original input when the lp no-m of the adversarial perturbation is sufficiently bounded. Unfortunately, in setting, the adversarial perturbation on compromised devices' embedded features can be very large. Further, RS does not have robustness guarantees when there are missing features.
Another promising defense is Randomized Ablation (Levine & Feizi] 2020), which was proposed to defend against sparse adversarial attacks where small number of pixels in an image are changed to some arbitrary values in [O.1]. We extend this to the collaborative inference setting for comparison: the FC learns to make predictions based on randomly selected subsets of agent features during the training; and in the inference phase, FC randomly selects a subset of agents to make a prediction. The final prediction is by majority voting over the random trials. However, it is hard to design base classifier that is able to train on randomly ablated input feature:" Another limitation of this method is that its robust prediction (based randomly ablated features) may not be the same as the attack- free vanilla setting (i.e., the prediction of the vanilla classifier on the uncorrupted data). Further, to choose how many pixels to ablate, one needs knowledge of the number of corrupted pixels.
Manifold Projection vs. Feature Purification. In the centralized setting, several works (Meng & Chen 2017 lyas et al. 2017 Lindqvist et al., 2018) proposed to project the adversarial perturbed image onto the manifold of the normal data to defend against adversarial attacks. A natural question is: why do we propose a feature purification method, instead of using manifold projection? First, the manifold projection approach can not guarantee the recovery of the underlying features. In fact, the resulting projected features can be arbitrarily far from the underlying true features if the adversarial perturbation can be arbitrarily large. To make this concrete, consider the lincar manifold, i... subspace. Suppose the underlying true feature vector 1 is generated from the column space of a by r tall matrix U. We observe the corrupted feature vector h. which can be decomposed as 1 + e, where models the adversarial perturbations. Projecting h onto the column space of U will get Prh = Pul + Pue = + Pve, where Pt = Ú) So the recovery error is Pyh 1 Pyre. Apparently, if the adversarial perturbation vector e is not orthogonal this subspace (which is often the case), the recovery error would be proportional to the magnitude of the perturbation! It's hard to recover the underlying true feature even if we have perfect subspace, and the problem looks even more challenging when the underlying manifold is non-linear. Fortunately, in our collaborative inference setting, there is important prior information about the pattern of the corruptions that we can leverage: the perturbation e has a block-wise structure (each block corresponds to an agent), and the fraction of the perturbed blocks is small. Therefore, e has a block-sparse structure. By exploring this fact, our proposed feature purification method can provably recover the underlying feature vector exactly or with bounded error, even if the underlying manifold is highly non-linear and the magnitude of the perturbation is arbitrarily large.
Purifier Network vs. Feature Purification. Besides the Manifold-Projection based approaches, one recent work (Naseer et al., 2020) tries to train a purifier neural network that maps any corrupted feature h to the underlying true feature I, in the centralized setting. However, that method needs to generate the adversarial training examples based on the assumption that the magnitude of the adversarial perturbation e is bounded by small e (see Naseer et al. (2020 Eq.4)). which is different from our collaborative inference setting where the adversarial perturbation is unbounded. Further, in our collaborative inference setting, arbitrary a fraction of the blocks of feature I can be perturbed by arbitrarily large values. Based on the high (combinatorial) cardinality of the input space, it is highly doubtful that one can learn a magic purifier network that is able to map a combinatorial number of different block block-perturbation patterns (and with arbitrary unbounded perturbation values)
For images, the authors designed a color channel approach to encode the absence of information (i.e., NULL) at ablated pixels. Though a CNN-based classifier can be trained on such data, it still does not know such an encoding scheme of NULL.
7

---

to the underlying true feature I. In sharp contrast, the proposed method can guarantee to recover the underlying true feature exactly.
Empirical evaluation 5
We first describe the experimental setup and introduce a new attack in Section 5.1] then present experimental results in Section 5.2 More results can be found in Section B of the supplemental material.
5.1 Experimental setup
Datasets. We study the classification task on ExtraSensory (Vaizman et al. (017) and NUS- WIDE (Chua et al. 2009) datasets. ExtraSensory contains the measurements from diverse sensors of smart phone and smart watch. We divide the sensors into ten agents, which correspond to the phone accelerometer, gyroscope, magnetometer, watch accelerometer, compass, location, audio, phone states (e.g., battery, wifi, ringing mode), enviromment sensing (e.g., light, air pressure, humidity, and temperature), and time-of-day (c.g.. morning). There are naturally missing-fcature problems, e.g. the user may not permit the use of microphone and/or location. The original purpose of this dataset is to recognize the behavioral context of the user (c.g., sitting, walking, and running) and recommend more appropriate music. We the first 1721 samples from user for training, and the rest 465 samples for testing, with the binary label 'sitting' or not.
In NUS-WIDE, each sample has 634 image features, 1000 text features, and 5 different labels, i.e. "buildings", "grass", 'animal', 'water', 'person'. We split the features into four agents, where the 1st agent holds 360-d image features, 2nd agent holds the rest of the image features, and the remaining two agents each hold 500-d text features. We use 60000 samples for training, 1000 samples for testing targeted attacks, and 10000 samples for testing untargeted attacks.
Attack Setup. We perform the inference adversarial attacks, as well as their combination with the flastue attack (missing-feature attack alone is not very effective). For the adversarial attack, we test both targeted and untargeted attacks.
For the targeted attack, we adopt the commonly used Projected Gradient Descent (PGD) attack to generate adversarial perturbations on the embedded features held by malicious agents. However, since in this collaborative inference setting, the magnitude of the perturbation on the embedded features is allowed to be very large, we omit the projection step in PGD (or one can think of it as projecting onto very large norm ball as the maximum perturbation constraint). In the ExtraSensory dataset, the target label is simply the opposite of the true label, since it a binary classification. We set the PGD attack with a learning rate of 0.5 and 30 iterations so that it can successfully attack the unsecured model. While in the NUS- WIDE dataset, the target label is set to be 'grass'. We set the PGD attack with learning rate of 0.1 and 50 iterations. Note that PGD attack is a white- box attack, where the attacker tries to find arg minp.... {(fo, ((fo,((har ((hade: Abenign }), A) via using the gradient dé(fe, ((hader bbenign }), // /dhade where hade denotes the overall embedded features held by malicious agents and y is the target label. In practice, the attacker may not casily obtain such gradient information, which usually needs to know the trained global model fa, and heenign. Nevertheless, this attack serves as smart targeted perturbation on the embedded features to test the robustness of the defense methods.
Motivated by the discussions in Section agnostic to the model, training data, label, as well as features held by other agents. We call it distributed feature-flipping attack. In this attack, the malicious agent(s) simply flips the sign of its embedded feature vector L; and further amplifies its magnitude. Formally, during this inference phase untargeted attack, the malicious agent i provides the embedded features h, = Amplification L; to the FC, instead of providing the true embedded features 4, This attack can be easily understood in binary classification with a linear classifier since flipping the sign of the whole feature vector will likely flip the sign of the predicted label as well. To illustrate the effectiveness of this attack against the manifold projecccin based defenses, we can look it as adding the corruption vector e, = -11 + Amplification) x L; to the true embedded feature vector I, of i-th agent, which is malicious. So the overall perturbation vector e has many zero blocks corresponding to benign agents. Its non-zero blocks have values e; = -(1 + Amplification) x L, Vi corresponds to the malicious agents. So the inner product between perturbation vector and true embedded features I is -(1 + Amplification) Eierad ||. Such perturbation vector e introduces correlations with
propose and test an untargeted attack, which is
8

---

the underlying manifold of the uncorrupted embedded features. Therefore, projecting the corrupted embedded features onto the manifold is hard get rid of such corruptions. We will see such feature-flippng attack is very effective in attacking many existing defenses.
Baseline methods. We compare with 1) Unsecured: no defense. 2) Manifold Projection: project the corrupted embedded features h test onto the learned manifold via Autoencoder, i.e., (E (hteaf)) 3) Randomized Smoothing (Cohen et al. 2019), where randomly generated Gaussian noise is added to h test for the FC to make the prediction. the final prediction is by majority voting over 1000 such random trials. 4) Randomized Smoothing Block: since only some block(s) of the embedded features are corrupted, we further compare with an oracle version of Randomized Smoothing, which only adds random Gaussian noise the corrupted blocks of h fest
5) Randomized Ablation (Levine Feizi 2020) 6) Adversarial Training (AT) (Madry et al., 2018): standard ad adversarial training via using PGD attacks on h frasn to generate adversarial examples, but without using projection step to limit the magnitude of adversarial perturbations. For defending against the targeted attack, it knows the target label and leverages that to generate targeted adversarial examples. For defending against untargeted attack, it uses untargeted PGD attack to generate adversarial examples. 7) Block-wise Adversarial Training (BAT): since only some block(s) of h test will be corrupted, we further propose and test a block-wise version of AT: for each training sample's embedded features, we randomly select a block and generate adversarial perturbations (via corresponding targeted or untargeted PGD attack) on that block for adversarial training. 8) Oracle: where there is no attack. The implementation details of the baselines and the proposed CoPur can be found in Section C of the supplemental material.
5.2 Empirical results
We report the Robust Accuracy of each defense method, i.e., classification accuracy on the adversarial corupteed embedded features, against the distributed adversarial attack and its combination with the missing-featur attack. More experiments can be found in Section B of the supplemental material.
A) Adversarial attack. We first test the robustness of the defense methods under the targeted attack. Table shows the Robust Accuracy of each method against targeted PGD attacks on NUS-WIDE and ExtraSensory datasets, where the last agent is malicious. The Manifold Projection method (which has no recovery guarantee) shows some resistance on the WUD- dataset but is not robust on the ExtraSensory dataset. The proposed method achieves the best Robust Accuracy on both datasets.
In Figure 2b. we report the Robust Accuracy of the defense methods under the untargeted feature- flipping attack on the ExtraSensory dataset, with Amplification ranging from [1, 3, 5, 10, 20]. and the last three agents perform such attack. The Robust Accuracy of the compared methods drops sigmificantly when the Amplification gets larger. The proposed method significantly outperforms the baselines by a large margin.
Table 1: Robust Accuracy (%) of each defense against targeted PGD attacks on NUS-WIDE and ExtraSensory datasets, where the last agent is malicious.
Unsecured Manifold Proj Rand Smooth Rand Smooth Bleck Rand Ablation
Adv Train
Block Adv Train Proposed
NUS-WIDE
4.520.3
81,3#112
41.8=6.3
48.7=10.1
65.3+5,,3
1.0=
7.3+1122
83,7= 0.9
ExtraSensory/ | 40.41 16.9 $3.6 = 35.7
42.4 4: 17,9
42.0 ::: 17.8
59.61t 27.5
19.8 +: 34,1
0.0:t 0.0
79.0 5.0
We are also interested in knowing the robustness limit of each defense when there are more and more agents performing adversarial attacks. Figure 3 shows the Robust Accuracy of each defense method w.r.t. the index set of agents that perform untargeted adversarial attack (Amplification= on ExtraSensory dataset. The Robustness Accuracy of the Adversarial Training method (green solid line) significantly drops when there are more than two malicious agents. The proposed method keeps very high Robust Accuracy even when agents 8-10 perform attack together. When agents 7-10 perform attack together, the Robust Accuracy of the proposed method apparently drops, but it is still significantly better than other baseline methods.
B) Combined attack. We now test the combined attack on ExtraSensory dataset, where agent 7 which holds audio MFCC features, performs missing-feature attack, and agent 10 performs untargeted (Figure 2b) or targeted (Table adversarial attack. The proposed method significantly outperforms the baseline methods and maintains very high Robust Accuracy under both attacks.

---

Manifold Projection
Robuss 75
Rand Smooth
75
Smooth block
(proposed)
Rand Abletion
Robust
Adversoraal Thair
Block Train
Ampiification (5)
Amplification (b)
Figure 2: Robust Accuracy each defense method under the untargeted adversarial attack and combined attacks ExtraSensory dataset with 10 agents: (a) Agents 8,9,10 provide corrupted embedded features 1 = Amplification x 10 (b) Agent 7 performs missing-featur attack,
while agent 10 provides corrupted embedded features h10 AAplificationn L10-
85
Unsecured
80
Manifold Projection
Rand Smooth
75
Rand Smooth block
70
CoPur (proposed)
Rand Ablation
Robust
Adversarial Train
Block Adv Train
60
Oracle
55
50
Agent
Agent 9.10
Agent 8,9.10
Agent 7,8,9.10
Figure 3: Robust Accuracy (%) of each defense method w.r.t the index set of agents that perform untargeted adversarial attack (Amplification=10) ExtraSensory dataset.
Table 2: Robust Accuracy (% of each defense against combined targeted attacks (agent 7 performs missing-feature attack and agent 10 performs adversarial targeted attack) on ExtraSensory dataset.
Unsecured Manifold Proj Rand Smooth Rand Smooth Block Rand Ablation Adv Train Block Adv Train Proposed
38.3+ 15.7 52.6= 38.9
50.5+ 1.3
51.0+t 2.5
60.3+ 28.1 24.6 26.8
0.1 0.1
79.1 4.5
6
Conclusions
In this work, we proposed novel feature purification based robust collaborative inference framework to defend against variety attacks during inference, with theoretical guarantees. The proposed non-linear robust decomposition method and its theoretical analyses may have much wider impacts. We further validate the robustness of the proposed framework through extensive experiments on ExtraSensory and NUS-WIDE datasets. Our future work includes extending Algorithm of Sharon et al. 2009) to calculate the block-wise incomplete leverage constant introduced in Remark
Acknowledgments and Disclosure of Funding
This work is partially supported by NSF grant No. 1910100, NSF CNS No.2046726, NSF III 2046795, IIS 1909577, CCF 1934986 and NIFA award 2020-67021-32799 Award, and the Alfred P. Sloan Foundation. S.K. was supported by Google Inc. The authors thank the anonymous reviewers for their constructive suggestions. J.I., also thanks Dr. Pedro Cisneros- Velarde and Dr. Yu Ding for the helpful discussions regarding manifolds.
C3.ai DTI Award, a Jump Arches
10

---

References
Tarek Abdelzaher, Nora Ayanian, Tamer Basar, et al. Will distributed computing revolutionize peace? the emergence of battlefield iot. In 2018 IEEE 38th International Conference Distributed Computing Systems (ICDCS), pp. 1129-1138, 2018. doi: 10.1109/ICDCS. 2018.00112.
Peva Blanchard, Mahdi EI Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 118-128, 2017.
Ashish Bora, Ajil Jalal, Eric Price, and Alexandros Dimakis. Compressed sensing using generative models. In International Conference on Machine Learning, pp. 537-546. PMLR, 2017.
E.J. Candes and T. Tao. Decoding by linear programming. IEEE Transactions on Information Theory, 51(12))4203-422 2005. doi: 10.1 1000/100002 858979.
Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopad- hyay. Adversarial attacks and defences: A survey, arXiv preprint 1m 2018.
Tianyi Chen, Xiao Jin, Yuejiao Sun, and Wotao Yin. Vafl: a method of vertical asynchronous federated learning. arXiv preprint arXiv:2007.06081, 2020.
Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 1-2):1-25, 2017.
Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-Tao Zheng. Nus-wide: A real-world web image datahase from national university of singapore. In Proc. of ACM Conf. on Image and Video Retrieval (CIVR (09), Santorini, Greece., 2009. https://ms..oom.. www.ppuusseeee.... www..oo......uuus......... httpppppwass/11//eeear///////iis
-WDE...m.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 1310-1320. PMLR, 09-15 Jun 2019. URL http:tt//prccceeiinngg wttpeeeeeddnnn.mm.......m.. mlr. press/ w97/cohee19c.hog
Salvador Flores. Sharp non-asymptotic performance bounds for & and huber robust regression estimators. Test, 7464))796-812, 2015.
Shuhao Fu, Chulin Xie, Bo Li, and Qifeng Chen. Attack-resistant federated learning with residual- based reweighting. arXiv preprint arXiv: 1464, 2019.
Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. The limitations of federated learning in sybil settings. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020),, pp. 301-316, 2020.
Rachid Guerraoui, Sébastien Rouault, et al. The hidden vulnerability of distributed learning in byzantium. In International Conference Machine Learning, pp. 3521-3530. PMLR, 2018.
Andrew Ilyas, Ajil Jalal, Eirini Asteri, Constantinos Daskalakis, and Alexandros G. Dimakis. The robust manifold defense: Adversarial training using generative models. CoRR, abs/1772.00119, 2017. URL http:/// org/ awwp/as/.....////// 09196
Vassilis Kekatos and Georgios B Giannakis. From sparse signals to sparse residuals for robust sensing. IEEE Transactions on Signal Processing, 5977)3335-3368, 2011.
Jong Hwan Ko, Taesik Na, Mohammad Faisal Amir, and Saibal Mukhopadhyay. Edge-host partition- ing of deep neural networks with feature space encoding for resource-conssrainee internet-of-thing platforms. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1-6, 2018. doi: 1000/AVSS.201..666999
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 656-672. IEEE, 2019.

---

Alexander Levine and Soheil Feizi. Robustness certificates for sparse adversarial attacks by random- ized ablation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 4585-4593, 2020.
Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Second-order adversarial attack and certifiable robustness. 2018.
Blerta Lindqvist, Shridatt Sugrim, and Rauf Izmailov. Autogan: Robust classifier against adversarial attacks. CoRR, abs/18 12. .0430500, 2018. URL
03405
J. Liu, P. C. Cosman, and B. D. Rao. Robust linear regression via lo regularization. IEEE Trans.
Signal Process., 66(3):698-77) 713, Feb 2018. ISSN 1053-587X. doi: 10.1109/TSP2001.277779.
Yang Liu, Zhiqian Yi, and Tianjian Chen. Backdoor attacks and defenses in reature-partitioned collaborative learning. ArXiv, abs/2007.03608 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations, 2018. URL BteL:tttss:/opeeeriiee net mettfoorumidd-z22IBB22222
Dongyu Meng and Hao Chen. Magnet: A two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, pp. 135-147, New York, NY, USA. 2017. Association for Computing Machinery. ISBN 9781450349468. doi: 3134057
//doi. htt////oo.o////////// 10. 101110111113111555
Kaushik Mitra, Ashok Veeraraghavan, and Rama Chellappa. Analysis of sparse regularization based robust regression approaches. IEEE Transactions on Signal Processing, 61(5):1249-12577 2013, doi: 10.1109/TSP:211.22229999
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A sulf-supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 262-271, 2020.
Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. Robust aggregation for federated learning. arXiv preprint arXiv: 19992.2.14455 2019.
Sunny Sanyal and Puning Zhang. Improving quality of data: lot data aggregation using device 2018. doi: 10.1EE//CC
to device communications. IEEE Access, 2878640.
Yoav Sharon, John Wright, and Yi Ma. Minimum sum of distances estimator: Robustness and stability. In 2009 American Control Conference, pp. 524-530. IEEE, 2009.
Yonatan Vaizman, Katherine Ellis, and Gert Lanckriet. Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Computing, 16(4):62-74, 2017. doi: 10. 1.111//M...2011.3971111 BY-NC-SA 4.0.
ExtraSensory dataset: http://cextaasessrry.uudd...l License: CC
Craig Wilson and Venugopal Veeravalli. Mmse estimation in sensor network in the presence of an adversary. In 2016 IEEE International Symposium on Information Theory (ISIT), pp. 2479-2483, 2016. doi: 00.109//STT.2016.5417722
Chulin Xie, Keli Huang, Pin-Yo Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against federated learning. International Conference on Learning Representations, 2020.
Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. Cril: Certifiably robust federated learning against backdoor attacks. International Conference on Machine Learning. 2021.
Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robuss distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pp. 5650-5659. PMLR, 2018
Jiangfan Zhang, Rick S. Blum, and H. Vincent Poor. Approaches to secure inference in the internet of things: Performance bounds, algorithms, and effective attacks on iot sensor networks. IEEE Signal Processing Magazine, 35(5):00-63, 2018. doi: 10. 10L.10///P.220888822222
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Please Section E.2 of supplemental material.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Please see Section E.4 of supplemental material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set assumptions of all theoretical results? [Yes] Please see Section 3 of the main paper.
(b) Did you include complete proofs of all theoretical results? [Yes] Please see Section of supplemental material.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material as URL)? [Yes] Please see Section C.T of supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please Section of supplemental material.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g.. type GPUs, internal cluster, or cloud provider)? [Yes] Please see Section C.1 of supplemental material.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] License of ExtraSensory dataset is CC BY-NC-SA 4.0, but we can find license for NUS- WIDE dataset.
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [Yes] Please see Section C.I supplemental material.
(e) Did you discuss whether the data you are using/curating contains personally identifi- able information offensive content? [Yes] Please see Section C.I of supplemental material.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "_TbbzbOh5gC", "reviewer": "Reviewer_h8UV", "summary": "This paper considers the problem of robust collaborative inference. This paper proposes a feature purification method for this problem. With the block-sparse structure, the authors propose a non-linear decomposition method to identify the corrupted agents. Experiments on the two datasets show the effectiveness of the proposed method. ", "questions": "Please see weaknesses.", "limitations": "The authors discuss the limitations in Appendix. I have no other suggestions.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Firstly, I have to say that I’m not an expert in this area. The review could be misleading.\n\n## Strengths:\n\n1.This paper considers a practical and interesting problem, what if some agents are attacked and send corrupted features to the server.\n\n2.This paper proposes a non-linear decomposition method for identifying poison agents, which is reasonable and should be effective.\n\n## Weakness:\n\n### Originality:\n\n1.I want to know if this paper is the first time to study the problem of the robust collaborative inference, where there are both arbitrary agents and adversarial agents. The arbitrary agents are easy to identify. However, I’m afraid the proposed method achieves a similar performance to identify the adversarial agents compared with baselines.\n\n2. From Eq.(5), the framework aims to find a combined feature $\\boldsymbol{l}$ which is on the manifold and is near $\\boldsymbol{h}$. The manifold projection could get a similar results for the adversarial sub-features. Could the authors discuss more about it? \n\n### Writtings:\n1.After so many times of reading, I guess I understand this paper. The authors introduce their method in Section 2.3, which is very simple. However, it relies block-sparse structure which is detailed stated in Section4. This could cause confuse when understanding the proposed method.\n\n2.The notations are confusing. For example, $\\boldsymbol{h}$ and $\\boldsymbol{l}$ both denote the feature. Why not use a letter (or with its variants)?\n\n3. the citation format may be ICLR rather NeurIPS.\n\n### Theoretical analysis:\n1.This paper provides an extensive theoretical analysis. In fact, I suggest the authors discuss more what the analysis means. Compared with baselines, why CoPur could do better.\n\n2.Could the authors give an intuitive explanation about the effect of the sparsity $\\alpha$ on CoPur?\n\n\n### Experiments:\n\n1.From the ablation studies, CoPur achieves a better performance compared with the manifold projection, what if there are different $\\Omega^{c}$ and different $\\Omega_{adv}$?\n\n2.More analysis is helpful, for example, The comparison on optimization efficiency.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "WvzhdtzcEh", "reviewer": "Reviewer_d75c", "summary": "This paper aims to improve the robustness of the deep learning model for collaborative inference. The authors propose a pre-processing-based defense method CoPur against inference phase attacks. The proposed method first leans the purified embedded features from the observed embedded features by using a generative model.Then, the purified embedded features are fed to the Fusion Center for prediction. This paper provide sufficient theoretical analyses to illustrate that the proposed method is able to recover the underlying true features exactly. Empirical results for targeted and non-targeted attacks are presented in this paper, which is used to demonstrate the effectiveness of the proposed method.", "questions": "1. Why are the well-known robust aggregation methods used for HFL not applicable? Please provide some further explanations.\n2. The authors say they propose to improve robustness based on the underlying redundancy among the features held by the IoT devices. However，it is not very clear to me how the proposed method obtains and exploits redundant information, is it through an autoencoder? \n3. The method proposed by the authors seems to be mainly aimed at off-manifold attacks. Is the proposed method effective against on-manifold attacks [1]? \n4. PGD is also suitable for crafting non-targeted adversarial noise, why do the authors not consider the non-targeted PGD? In addition, the Autoattack [2] is a more powerful attack, the authors can use it to futher evaluate the effectiveness of the proposed defense.\n5. The poposed method is a pre-processing-based method, but the authors do not compare it with other pre-processing-based adversarial defense methods, such as [3].\n6. The authors do not consider the stronger adaptive attack [4]. For example, the used autoencoder may also be attacked. The authors should design adaptive attacks to more comprehensively evaluate the robustness of the proposed method.\n\n[1]. Disentangling adversarial robustness and generalization. CVPR, 2019.\n[2]. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML, 2020.\n[3]. A self-supervised approach for adversarial robustness. CVPR, 2020.\n[4]. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. ICML, 2018.\n\n\n-------------------------------------------------------\nAfter rebuttal\n\nThe authors solve most of my concerns, so I am willing to modify the score to borderline accept.", "limitations": "The authors do not consider some closely related works and some import attacks to further show the effectiveness of the proposed method.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper proposes a certifiable defense to defense against inference phase attacks for the collaborative inference task. This method is based on the pre-processing strategy, which does not require much computing cost and does not change the original information of agents.\n2. This paper provide sufficient theoretical analyses to demonstrate that the proposed method is certifiable. The authors analysis two cases (i.e., noiseless case and noisy case) and use them to illustrate that solving some relaxed objectives is able to recover the underlying true features exactly.\n3. The authors design a non-target attack (i.e., distributed feature-flipping attack) against the collaborative inference model to evaluate the effect of the baseline defense models and the proposed defense model. This helps to evaluate the effectiveness of adversarial defenses.\n\nWeaknesses:\n1. The authors emphasize that in their considered inference setting, the features obtained by the agents are quiet different and the redundancy is less apparent. However, the authors seems not to introduce how to design a specific mechanism for this problem, and how the proposed method handles the issue of less redundancy in detail. In addition, the authors do not discuss the performances of the proposed method under different redundancy situations.\n2. IoT applications are often communication-limited. The authors do not evaluate the communication requirements of the proposed method, and they do not show whether the improvement mechanism would increase the communication requirements.\n3. Some closely related works are not discussed, and some closely related methods are not compared. Some import attacks are not used to evaluate the defense.\n4. Writing and presentation need improvement. For example, the caption in Fig. 1 does not clearly describe the necessary information in the figure, such as what does the variable represent? In addition, \\Omega_{adv} is the index for all corrupted features (\\Omega = \\Omega_{benign} U \\Omega_{adv}), but in Eq. 3, it becomes the index for perturbed features, |\\Omega_{adv}| in Eq. 3 should be \\alpha.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "OcnYtglr8zu", "reviewer": "Reviewer_Zg9q", "summary": "This paper studies robust inference under the vertically split features setting, which appears in applications like e Internet-of-Things (IoT). The paper considers three types of threat models, including distributed adversarial attack, missing feature attack, and combined attack. The paper proposes a sparse recovery-based robust inference algorithm and provides recovery guarantees. The paper provides experimental results to show the superior performance of the proposed method.", "questions": "The recovery theorems rely on critical hyperparameters like delta, which is empirically tuned on a validation set according to the supplement. Is it possible to provide some theoretical guidance to choose such hyperparameters so that the certified recovery is always ensured?", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n(1) The problem of robust inference under the vertically split features setting is new and interesting.\n\n(2) The proposed method has recovery theory and better empirical performance.\n\nWeakness:\n\n(1) The idea of sparse recovery-based algorithm design is not new.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "HtuyaR6mxhK", "reviewer": "Reviewer_mjnb", "summary": "This work focuses on feature fusion in distributed machine learning scenarios. In this work, a distributed feature reconstruction method is proposed to recover noise / incomplete features committed by different local clients. The main idea is to decompose observed uncorrupted features into the underlying noise-free features and additional block-wise sparse noise corruption components. After that, a L2,1 signal recovery method is applied to estimate the noise-free feature profiles by enforcing the L2,1 norm sparsity regularisation over the estimated noise components. In parallel, the feature recovery process is accompanied with the learning of an AutoEncoder architecture performed over the recovered feature vector. The goal is to generate a smoothed feature representation from the recovered features for the final classification step. Theoretical and experimental study demonstrate the provably effectiveness of the proposed feature reconstruction method. ", "questions": "Please find the questions in the second and third points in the listed weakness. ", "limitations": "Please find the listed weakness points. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strong points:\n\n1.  Recovering the noise-free feature profiles provides less noisy information for classification, compared to aligning noisy features to the subspace of normal data. \n2.  It is decent to make use of the block-wise sparsity structure of the noise components and formulate the feature recovery problem as the L2,1-regularized regression process. \n3. Theoretical study established for the noisy case helps clarify the practical implication of the proposed method. \n\nWeak points: \n1. Though it is interesting to see a novel feature-corruption-based attack method is proposed, the link between designing this attack algorithm and the feature recovery method (CoPur) is not clearly explained. \n2. How does the proposed CoPur method perform agains the distributed feature-flipping attack? Is it the combined attack in Table.2? \n3. How many malicious agents can the proposed CoPur defense method tolerate ? It is related to the break point of the proposed method. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "9wxtj0SQ9sQ", "reviewer": "Reviewer_vVL3", "summary": "This paper proposed a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold). This paper also gave theoretical results showing that the proposed method can robustly recover the true feature vector.  Extensive experiments showed the effectiveness of the proposed method. Besides, an untargeted distributed feature-flipping attack was proposed.\n", "questions": "Q1: The main contribution (Eq.6) works out based on the prior knowledge that adversarial feature $e$ has a block-sparse structure. However, theoretical results didn't formalize the sparsity of $e$ clearly enough.\n\nQ2: The proposed feature-flipping attack is not well-evaluated. Experiments showed that the proposed defense has better accuracy against this attack compared to other defenses. However, no experimental results are given to indicate that this attack is stronger than other attacks.\n\nQ3: How sparsity of adversarial feature $e$ affects the robust accuracy is not evaluated.\n\nQ4: Figure 2 shows that when amplification is low, CoPur has higher robust accuracy than Oracle. Can you explain why this phenomenon happens? If CoPur is applied to unattacked features, will it improve the standard accuracy? \n\nQ5: Why the solution to Eq.6 is not mentioned in the main text?\n", "limitations": "No limitations discussed in the paper. See questions.\n", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros\n1. Theoretical analysis is given, showing that the recovered benign feature is close to the true feature with bounded error.\n2. Feature-flipping attack is proposed to demonstrate the weakness of manifold projection defense.\n\nCons\n1. Theoretical results are not adequate. \n2. Lack of experimental results.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept