Original: CoPur: Certifiably Robust Collaborative Inference via Feature Purification
Revised: CoPur: Certifiably Robust Collaborative Inference via Feature Purification

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Collaborative inference leverages diverse features provided by different agents(e.g., sensors) for more accurate inference.
Revised: Collaborative inference leverages diverse features provided by different agents (e.g., sensors) for more accurate inference.

Original: A common setup is where each agentsends its embedded features instead of the raw data to the Fusion Center (FC)for joint prediction.
Revised: A common setup is where each agent sends its embedded features instead of the raw data to the Fusion Center (FC) for joint prediction.

Original: In this setting, we consider inference phase attacks when asmall fraction of agents are compromised.
Revised: In this setting, we consider inference phase attacks when a small fraction of agents is compromised.

Original: The compromised agent either doesnot send embedded features to the FC, or sends arbitrary embedded features.
Revised: The compromised agent either does not send embedded features to the FC or sends arbitrary embedded features.

Original: Toaddress this, we propose a certifiably robust COllaborative inference framework viafeature PURification (CoPur), by leveraging the block-sparse nature of adversarialperturbations on the feature vector, as well as redundancy across the embeddedfeatures (by assuming the overall features lie on an underlying lower dimensionalmanifold).
Revised: To address this, we propose a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold).

Original: We theoretically show that the proposed feature purification methodcan robustly recover the true feature vector, despite adversarial corruptions and/orincomplete observations.
Revised: We theoretically show that the proposed feature purification method can robustly recover the true feature vector, despite adversarial corruptions and/or incomplete observations.

Original: We also propose and test an untargeted distributed featureflipping attack, which is agnostic to the model, training data, label, as well as thefeatures held by other agents, and is shown to be effective in attacking state-of-theart defenses.
Revised: We also propose and test an untargeted distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents, and is shown to be effective in attacking state-of-the-art defenses.

Original: Extensive experiments on ExtraSensory and NUS-WIDE datasetsshow that CoPur significantly outperforms existing defenses in terms of robustnessagainst targeted and untargeted adversarial attacks.
Revised: Experiments on ExtraSensory and NUS-WIDE datasets show that CoPur significantly outperforms existing defenses in terms of robustness against targeted and untargeted adversarial attacks.

Original:  Introduction
Revised:  Introduction

Original: Collaborative inference is an increasingly popular distributed prediction strategy that leverages diversefeatures provided by different agents (e g., sensors) for more accurate inference.
Revised: Collaborative inference is an increasingly popular distributed prediction strategy that leverages diverse features provided by different agents (e.g., sensors) for more accurate inference.

Original: For example, incontext-aware music recommendation, the app may wish to recommend more appropriate musicbased on  user’s behavioral context (e g., sleeping, running) (Vaizman et al., 2017).
Revised: For example, in context-aware music recommendation, the app may wish to recommend more appropriate music based on the user’s behavioral context (e.g., sleeping, running) (Vaizman et al., 2017).

Original: Here, it canleverage diverse sensors from the user’s smart phone and smart watch, e.g., phone accelerometer,gyroscope, watch accelerometer, location, audio, phone states (e.g., wifi, ringing mode), environmentsensing (e.g., light).
Revised: Here, it can leverage diverse sensors from the user’s smart phone and smart watch, e.g., phone accelerometer, gyroscope, watch accelerometer, location, audio, phone states (e.g., wifi, ringing mode), and environment sensing (e.g., light).

Original: Thus, by combining information across sensors, much more accurate predictionscan be made.
Revised: Thus, by combining information across sensors, much more accurate predictions can be made.

Original: In this work, we study the scenario where Internet-of-Things (IoT) devices sendembedded features (instead of the raw data) to a Fusion Center (FC) for collaborative inference.
Revised: In this work, we study the scenario where Internet-of-Things (IoT) devices send embedded features (instead of the raw data) to a Fusion Center (FC) for collaborative inference.

Original: The

Original:  IoT devices do not communicate with each other and may hold quite different raw data (e.g., image,acoustic, Lidar data, etc).
Revised: The IoT devices do not communicate with each other and may hold quite different raw data (e.g., image, acoustic, Lidar data, etc).

Original: Nevertheless, as a result of distributed inference, the system is vulnerableto attack, where a subset of sensors are corrupted by an adversary at inference time, e.g., due to thelack of effective security mechanisms, connection errors, sensor failures, among other issues – this isalso a common concern in defense applications (Abdelzaher et al., 2018).
Revised: Nevertheless, as a result of distributed inference, the system is vulnerable to attack, where a subset of sensors are corrupted by an adversary at inference time, e.g., due to the lack of effective security mechanisms, connection errors, sensor failures, among other issues – this is also a common concern in defense applications (Abdelzaher et al., 2018; Xie et al., 2020).

Original: Nevertheless, the goal ofthe system is robust inference based on the possibly incomplete and corrupted embedded featuresreceived from the IoT devices.
Revised: Nevertheless, the goal of the system is robust inference based on the possibly incomplete and corrupted embedded features received from the IoT devices.

Original: There have been a number of works on robust parameter estimation in IoT settings, where therobustness apparently comes from redundancy among different sensors’ linear measurements of the same parameter.
Revised: There have been a number of works on robust parameter estimation in IoT settings, where the robustness apparently comes from redundancy among different sensors’ linear measurements of the same parameter.

Original: For a comprehensive overview, we refer the interested reader to (Zhang et al.,2018).
Revised: For a comprehensive overview, we refer the interested reader to (Zhang et al., 2018).

Original: However, in our considered inference setting, each IoT device may hold quite different typesof features and the redundancy is less apparent.
Revised: However, in our considered inference setting, each IoT device may hold quite different types of features, and the redundancy is less apparent.

Original: We also observe that IoT applications are oftencommunication-limited (Ko et al., 2018), thus, using lower-dimensional embeddings to communicatefrom each sensor is a common and effective approach to reduce the required communication overhead.
Revised: We also observe that IoT applications are often communication-limited (Ko et al., 2018), thus, using lower-dimensional embeddings to communicate from each sensor is a common and effective approach to reduce the required communication overhead.

Original: The problem we study here also applies to the inference phase of some Vertical Federated Learning(VFL) frameworks (Chen et al., 2020; Liu et al., 2020), where a server similarly combines differentembedded features received from every agent to make a prediction.
Revised: The problem we study here also applies to the inference phase of some Vertical Federated Learning (VFL) frameworks (Chen et al., 2020; Liu et al., 2020), where a server similarly combines different embedded features received from every agent to make a prediction.

Original: One difference between oursetting and VFL is: the agents in VFL are other untrusted parties, if an agent in VFL is malicious,it will likely attack during both training and inference; while in our setting, the agents (e.g., IoTsensors) are all owned by FC, and they can be well protected during training, and inference is morevulnerable.
Revised: One difference between our setting and VFL is: the agents in VFL are other untrusted parties, if an agent in VFL is malicious, it will likely attack during both training and inference; while in our setting, the agents (e.g., IoT sensors) are all owned by FC, and they can be well protected during training, and inference is more vulnerable.

Original: So far, there are no certifiable defenses for VFL against adversarial attacks during theinference phase.
Revised: So far, there are no certifiable defenses for VFL against adversarial attacks during the inference phase.

Original: The well-known robust aggregation method (Yin et al., 2018; Guerraoui et al., 2018;
Revised: The well-known robust aggregation method (Yin et al., 2018; Guerraoui et al., 2018;

Original: Blanchard et al., 2017; Fu et al., 2019; Pillutla et al., 2019; Fung et al., 2020; Chen et al., 2017; Xieet al., 2019) used for Horizontal Federated Learning (HFL) is not applicable.
Revised: Blanchard et al., 2017; Fu et al., 2019; Pillutla et al., 2019; Fung et al., 2020; Chen et al., 2017; Xie et al., 2021) used for Horizontal Federated Learning (HFL) is not applicable.

Original: Other related works will be discussed in Section 4.
Revised: Other related works will be discussed in Section 4.

Original: We use the term ‘device’ and ’agent’ interchangeably in this paper.
Revised: We use the term ‘device’ and ’agent’ interchangeably in this paper.

Original: The main difficulty of defending against adversarial attacks in our collaborativeinference setting (as well as in VFL) is that there is no apparent redundancy across the agents.
Revised: The main difficulty of defending against adversarial attacks in our collaborative inference setting (as well as in VFL) is that there is no apparent redundancy across the agents.

Original: Though,intuitively, there is some mutual information among the features held by each agent.
Revised: Though, intuitively, there is some mutual information among the features held by each agent.

Original: Even worse, theraw data held by the agents are not accessible by the FC/server to inspect, the attacker can perturb thecompromised agents’ embedded features with  very large magnitude before sending  to the FC/server.
Revised: Even worse, the raw data held by the agents are not accessible by the FC/server to inspect, the attacker can perturb the compromised agents’ embedded features with a very large magnitude before sending them to the FC/server.

Original: In this paper, we propose to improve robustness based on the underlying redundancy among thefeatures held bythe IoT devices.
Revised: In this paper, we propose to improve robustness based on the underlying redundancy among the features held by IoT devices.

Original: Specifically, we assume the combined features lie on an underlyinglower dimensional manifold.
Revised: Specifically, we assume the combined features lie on an underlying lower dimensional manifold.

Original: We propose a robust COllaborative inference framework via feature
Revised: We propose a robust COllaborative inference framework via feature

Original: PURification (CoPur), which is able to defend against many types of attacks during inference (seethe attack taxonomy in Section 2.2).
Revised: PURification (CoPur), which is able to defend against many types of attacks during inference (see the attack taxonomy in Section 2.2).

Original: During the inference, the FC receives the corrupted and/orincomplete embedded features from agents (Figure 1a).
Revised: During the inference, the FC receives the corrupted and/or incomplete embedded features from agents (Figure 1a).

Original: Then CoPur purifies the embedded featuresbased on the learned feature subspace (Figure 1b).
Revised: Then CoPur purifies the embedded features based on the learned feature subspace (Figure 1b).

Original: Finally, it feeds the purified features to FC’strained global model for prediction.
Revised: Finally, it feeds the purified features to FC’s trained global model for prediction.

Original: The detailed robust inference procedures will be described in
Revised: The detailed robust inference procedures will be described in

Original: Sections 2.3.
Revised: Sections 2.3.

Original: We aim to answer the following questions: Can we make robust predictions based oncorrupted and/or incomplete features?
Revised: We aim to answer the following questions: Can we make robust predictions based on corrupted and/or incomplete features?

Original: If yes, how many compromised devices can we tolerate?
Revised: If yes, how many compromised devices can we tolerate?

Original: Technical Contributions : Our main contributions include the following:
Revised: Technical Contributions : Our main contributions include the following:

Original: • We propose a certifiably robust collaborative inference procedure to defend against a variety oftargeted or untargeted attacks (e.g., distributed adversarial attacks, missing-feature attack, and theircombinations) during collaborative inference.•
Revised: • We propose a certifiably robust collaborative inference procedure to defend against a variety of targeted or untargeted attacks (e.g., distributed adversarial attacks, missing-feature attack, and their combinations) during collaborative inference.

Original:  We propose a novel non-linear robust decomposition method that decomposes the potentiallycorrupted and incomplete feature vector into two parts: one lies (exactly or approximately) on theunderlying feature manifold;and the other with a block-sparse structure.
Revised: • We propose a novel non-linear robust decomposition method that decomposes the potentially corrupted and incomplete feature vector into two parts: one lies (exactly or approximately) on the underlying feature manifold; the other with a block-sparse structure.

Original: Theoretically, we provethat the proposed method can recover the true feature vector exactly (or approximately ) despitecorruptions and incomplete observations.
Revised: Theoretically, we prove that the proposed method can recover the true feature vector exactly (or approximately ) despite corruptions and incomplete observations.

Original: This may be of independent interest.
Revised: This may be of independent interest.

Original: • We propose and test a distributed feature-flipping attack, which is agnostic to the model, trainingdata, label, as well asthe features held by other agents.
Revised: • We propose and test a distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents.

Original: Empirical studies show that this new attackis very effective in attacking existing state-of-the-art defenses.•
Revised: Empirical studies show that this new attack effectively attacks existing state-of-the-art defenses.

Original:  We conduct extensive experiments on ExtraSensory and NUS-WIDE datasets and show that CoPuris significantly more robust than baselines against different types of attacks.
Revised: • We conduct extensive experiments on ExtraSensory and NUS-WIDE datasets, and show that CoPur is significantly more robust than baselines against different types of attacks.

Original:  Robust collaborative inference via feature purification (CoPur)
Revised:  Robust collaborative inference via feature purification (CoPur)

Original: We describe the system setting, the threat models and the proposed robust inference procedures.
Revised: We describe the system setting, the threat models, and the proposed robust inference procedures.

Original: 2.1 Collaborative inference; System setting
Revised: 2.1 Collaborative inference and system setting

Original: We first describe the basic framework of collaborative inference, which is also similar to the inferencephase of some VFL frameworks (Chen et al., 2020; Liu et al., 2020).
Revised: We first describe the basic framework of collaborative inference, which is also similar to the inference phase of some VFL frameworks (Chen et al., 2020; Liu et al., 2020).

Original: There are M agents, where eachagent i holds partial raw data x testi of the overall testing sample x test =
Revised: There are M agents, where each agent i holds partial raw data x testi of the overall testing sample x test =

Original: [ x test; ... ; x testM ] .
Revised: [ x test; ... ; x testM ] .

Original: Also, eachagent i has its own local feature extractor f i parameterized by θ i that maps its raw data vector x testito the embedded feature vector h testi ≜
Revised: Also, each agent i has its own local feature extractor f i parameterized by θ i that maps its raw data vector x testi to the embedded feature vector h testi ≜

Original: f i ( x testi ; θ i ) .
Revised: f i ( x testi ; θ i ) .

Original: The dimension of h testi is usually smaller thanx testi .
Revised: The dimension of h testi is usually smaller than x test i .

Original: The FC only receives the embedded features h testi , i = 1 , ..., M from the agents (not the rawdata x testi ), and concatenates them into a long column vector h test ≜ [ h test; h test; ... ; h testM ] .
Revised: The FC only receives the embedded features h test i , i = 1 , ..., M from the agents (not the raw data x testi ), and concatenates them into a long column vector h test ≜ [ h test; h test; ... ; h testM ] .

Original: Thenthe FC uses its pre-trained global model f θ 0 to make the prediction.
Revised: Then the FC uses its pre-trained global model f θ 0 to make the prediction.

Original: While we do not focus on training, we will require an AutoEncoder, trained on uncorrupted data (e.g.,during training) that can approximately capture the manifold of the uncorrupted embedded features,i.e. , (cid:80) Mi =1 ∥
Revised: While we do not focus on training, we will require an AutoEncoder, trained on uncorrupted data (e.g., during training) that can approximately capture the manifold of the uncorrupted embedded features, i.e. , (cid:80) Mi =1 ∥

Original: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the true embedded features l ∗ of the testing instance,where D ϕ is the decoder parameterized by ϕ , and E ψ is the encoder parameterized by ψ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the true embedded features l ∗ of the testing instance, where D ϕ is the decoder parameterized by ϕ , and E ψ is the encoder parameterized by ψ .

Original: Further, weassume that the global model for inference f θ 0 is based on the output of the AutoEncoder and thetraining labels, i.e. , {D ϕ ( E ψ ( h ( j ) )) , y ( j ) } nj =1 .
Revised: Further, we assume that the global model for inference f θ 0 is based on the output of the AutoEncoder and the training labels, i.e. , {D ϕ ( E ψ ( h ( j ) )) , y ( j ) } nj =1 .

Original: Detailed examples of how to train such a model andthe AutoEncoder can be found in Section D of the supplemental material.
Revised: Detailed examples of how to train such a model and the AutoEncoder can be found in Section D of the supplemental material.

Original: 2.2 Threat model
Revised: 2.2 Threat model

Original: There are M agents which hold different parts of the feature of the same set of n training instances.
Revised: There are M agents which hold different parts of the feature of the same set of n training instances.

Original: We assume the training process is well-protected (attack-free).
Revised: We assume the training process is well-protected (attack-free).

Original: However, during the inference, theattacker can attack αM agents.
Revised: However, during the inference, the attacker can attack αM agents.

Original: The compromised agent either is notbe able to send features to the FC, or sends arbitrary embedded features to FC.
Revised: The compromised agent either is not able to send features to the FC, or sends arbitrary embedded features to FC.

Original: We use h benign to denote the overall embedded features provided by the (1 − α )
Revised: We use h benign to denote the overall embedded features provided by the (1 − α )

Original: M benign agents(these agents are indexed by Ω benign and | Ω benign | = (1 − α ) M ), and we use h adv to denotethe overall embedded features sent by compromised agents (indexed by Ω adv ).
Revised: M benign agents (these agents are indexed by Ω benign and | Ω benign | = (1 − α ) M ), and we use h adv to denote the overall embedded features sent by compromised agents (indexed by Ω adv ).

Original: The set Ω =Ω benign ∪
Revised: The set Ω = Ω benign ∪

Original: Ω adv is the index set of agents that provide features, and its complement, i.e. , Ω c , denotesthe compromised agents which do not provide features.
Revised: Ω adv is the index set of agents that provide features, and its complement, i.e. , Ω c , denotes the compromised agents which do not provide features.

Original: We categorize the inference phase attacksinto the following three types:
Revised: We categorize the inference phase attacks into the following three types:

Original: Threat model A (distributed adversarial attack): All the compromised agents jointly send adversarial embedded features h adv , with the goal to mislead the prediction to the target label
Revised: Threat model A (distributed adversarial attack): All the compromised agents jointly send adversarial embedded features h adv , with the goal to mislead the prediction to the target label

Original: y A :
Revised: y A :

Original: [Equation]
Revised: [Equation]

Original: Threat model B (missing-feature attack): All the compromised agents do not send any feature tothe FC.
Revised: Threat model B (missing-feature attack): All the compromised agents do not send any feature to the FC.

Original: Such missing-feature attack may  due tothe damaged sensors or disconnection, and can stillaffect FC’s prediction.
Revised: A such missing-feature attack may be due to damaged sensors or disconnection, and can still affect FC’s prediction.

Original: The presence of  missing-feature attack can be viewed as the FC observingonly partial blocks (indexed by Ω ) of the overall embedded features h , i.e. , h Ω .
Revised: The presence of a missing-feature attack can be viewed as the FC observing only partial blocks (indexed by Ω ) of the overall embedded features h , i.e. , h Ω .

Original: Ω c denotes the indexset of agents that perform missing-feature attacks.
Revised: Ω c denotes the index set of agents that perform missing-feature attacks.

Original: The attacker can compromise up to αM agents toperform such  attack: 
Revised: The attacker can compromise up to αM agents to perform such an attack: min Ω

Original: [Equation]
Revised: [Equation]

Original: Threat model C (combined attack):
Revised: Threat model C (combined attack):

Original: The attacker can also perform a combined adversarial andmissing feature attack by providing h adv , indexed by Ω adv , and missing features Ω c :
Revised: The attacker can also perform a combined adversarial and missing feature attack by providing h adv , indexed by Ω adv , and missing features Ω c :

Original: [Equation]
Revised: [Equation]

Original: Beyond these targeted attacks, the attacker can also perform untargeted attacks by maximizing thedistance between the prediction and the ground truth label.
Revised: Beyond these targeted attacks, the attacker can also perform untargeted attacks by maximizing the distance between the prediction and the ground truth label.

Original: 2.3 Inference procedures of CoPur
Revised: 2.3 Inference procedures of CoPur

Original: We propose an all-in-one solution to defend against various inference phase attacks defined in
Revised: We propose an all-in-one solution to defend against various inference phase attacks defined in

Original: Section 2.2.
Revised: Section 2.2.

Original: Let Ω be the index set of the observed blocks.
Revised: Let Ω be the index set of the observed blocks.

Original: The observed h Ω can be decomposed ash Ω = l Ω + e Ω where l is the underlying uncorrupted features, and e Ω models the corruptions onthe observed set Ω .
Revised: The observed h Ω can be decomposed as h Ω = l Ω + e Ω where l is the underlying uncorrupted features, and e Ω models the corruptions on the observed set Ω .

Original: Finding such a decomposition seems hard, even if the learned AutoEncoder can capture the underlying feature subspace, i.e. , l = D ϕ
Revised: Finding such a decomposition seems hard, even if the learned AutoEncoder can capture the underlying feature subspace, i.e. , l = D ϕ

Original: ( E ψ ( l )) .
Revised: ( E ψ ( l )) .

Original: Fortunately, the perturbation e has ablock-wise structure (each block corresponds to an agent), and the fraction of the perturbed blocks issmall.
Revised: Fortunately, as we mentioned in

Original: Therefore, e Ω has a block-sparse structure.
Revised: Section 4, e Ω has a block-sparse structure.

Original: We propose to solve the following:
Revised: We propose to solve the following:

Original: [Equation]
Revised: [Equation]

Original: where the indicator function 1 {·} counts whether the block [ h − l ] i is nonzero.
Revised: where the indicator function 1 {·} counts whether the block [ h − l ] i is nonzero.

Original: This block-sparsityminimization shares the same spirit as the ℓ 0 minimization in robust linear regression (Candes & Tao,2005; Mitra et al., 2013).
Revised: This block-sparsity minimization shares the same spirit as the ℓ 0 minimization in robust linear regression (Candes & Tao, 2005; Mitra et al., 2013).

Original: However, it is NP-hard to solve, so we relax it to the ℓ 2 , 1 -norm:
Revised: However, it is NP-hard to solve, so we relax it to the ℓ 2 , 1 -norm:

Original: [Equation]
Revised: [Equation]

Original: Our theoretical analysis in Theorems 1 & 2 show that under certain conditions, solving the aboverelaxed objective is able to recover the underlying true features exactly.
Revised: Our theoretical analysis in Theorems 1 & 2 shows that under certain conditions, solving the above relaxed objective is able to recover the underlying true features exactly.

Original: But what if the learned AutoEncoder can only approximately reconstruct the underlying features, say137 (cid:80)
Revised: But what if the learned AutoEncoder can only approximately reconstruct the underlying features, say (cid:80)

Original: [Equation]
Revised: [Equation]

Original: =1 ∥ [ D ϕ [Equation] E ψ ( l)
Revised: l

Original: [Equation]
Revised: [Equation]

Original: − l ] i ∥ 2 ≤ δ ?
Revised: − l ] i ∥ 2 ≤ δ ?

Original: In this case, we further relax the constraints in Eq.
Revised: In this case, we further relax the constraints in Eq.

Original: 5 and solve:
Revised: 5 and solve:

Original: [Equation]
Revised: [Equation]

Original: Our theoretical analysis in Theorems 3 & 4 show that under certain conditions, solving Eq.
Revised: Our theoretical analysis in Theorems 3 & 4 show that under certain conditions, solving Eq.

Original: 6 is ableto approximately recover the underlying true features, despite the corruptions and/or incompleteobservations.
Revised: 6 is able to approximately recover the underlying true features, despite the corruptions and/or incomplete observations.

Revised: In our implementation, we first try to find a good initial point by simply searching the embedded features on the manifold, i.e., min l ′ (cid:80) i ∈

Revised: Ω ∥

Revised: [ h − D ϕ

Revised: ( E ψ ( l ′ ))] i ∥ 2 via gradient descend (It is useful to notice that since Eq. 6 allows l approximately lie on the manifold, its objective value will be slightly smaller than min l ′ (cid:80) i ∈

Revised: Ω ∥

Original: After obtaining ˆ l , we feed D ϕ ( E ψ (ˆ l )) to FC’s trained global model for the prediction.
Revised: [ h − D ϕ ( E ψ ( l ′ ))] i ∥ 2 ). Then we apply Lagrange Multiplier method to solve Eq.

Revised: 6, i.e., ˆ l = arg min l (cid:80) i ∈ Ω ∥

Revised: [ h − l ] i ∥ 2 + τ (cid:80) Mi

Revised: =1 ∥ [ D ϕ ( E ψ ( l )) − l ] i ∥ 2 .

Revised: More implementation details can be found in the supplemental.

Revised: After obtaining ˆ l , we feed D ϕ ( E ψ (ˆ l )) to FC’s trained global model for the prediction.

Original:  Theoretical analysis of CoPur
Revised:  Theoretical analysis of CoPur

Original: We first analyze the ‘noiseless case’ where the learned AutoEncoder satisfies D ϕ
Revised: We first analyze the ‘noiseless case’ where the learned AutoEncoder satisfies D ϕ

Original: ( E ψ ( l ∗ )) =
Revised: ( E ψ ( l ∗ )) =

Original: l ∗ forthe underlying uncorrupted feature vector l ∗ .
Revised: l ∗ for the underlying uncorrupted feature vector l ∗ .

Original: Next, we analyze the more challenging ‘noisy case’where the AutoEncoder can only approximately reconstruct l ∗ , i.e. , (cid:80) Mi
Revised: Next, we analyze the more challenging ‘noisy case’ where the AutoEncoder can only approximately reconstruct l ∗ , i.e. , (cid:80) M i =1 ∥

Original: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .

Original: Proofs and additional discussions can be found in Section A of the supplemental material.
Revised: Proofs and additional discussions can be found in Section A of the supplemental material.

Original: 3.1 Noiseless Case
Revised: 3.1 Noiseless Case

Original: To simplify the exposition, we first consider threat model B, i.e. , the missing-feature attack.
Revised: To simplify the exposition, we first consider threat model B, i.e. , the missing-feature attack.

Original: Theorem 1.
Revised: Theorem 1.

Original: (Exact feature recovery under threat model B) Assume the trained AutoEncodersatisfies D ϕ ( E ψ ( l ∗ )) =
Revised: (Exact feature recovery under threat model B) Assume the trained AutoEncoder satisfies D ϕ ( E ψ ( l ∗ )) =

Original: l ∗ for the underlying uncorrupted feature vector l ∗ .
Revised: l ∗ for the underlying uncorrupted feature vector l ∗ .

Original: For ∀ r ′ , r ′′ in the rangeof the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , if ( r ′ − r ′′ )
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , if ( r ′ − r ′′ )

Original: Ω ̸ = 0 , then given h Ω =
Revised: Ω ̸ = 0 , then given h Ω =

Original: l ∗ Ω , l ∗ is the uniquesolution of Eq.
Revised: l ∗ Ω , l ∗ is the unique solution of Eq.

Original: 5 and Eq. 4.
Revised: 5 and Eq. 4.

Original: The proof can be found in Section A of the supplemental material, where we show that any feasiblepoint that is different from l ∗ would have  larger objective value.
Revised: The proof can be found in Section A.3 of the supplemental material, where we show that any feasible point that is different from l ∗ would have a larger objective value.

Original: The condition ( r ′ − r ′′ ) Ω ̸ = 0 required by Theorem 1 is intuitive and necessary.
Revised: The condition ( r ′ − r ′′ ) Ω ̸ = 0 required by Theorem 1 is intuitive and necessary.

Original: Suppose thereare r ′ ̸ = r ′′ but r ′ Ω = r ′′ Ω , then observing r
Revised: Suppose there are r ′ ̸ = r ′′ but r ′ Ω = r ′′ Ω , then observing r

Original: Ω is not able to tell whether it’s from r ′ or r ′′ .
Revised: Ω is not able to tell whether it’s from r ′ or r ′′ .

Original: Ingeneral, when the observed set Ω gets larger, the required condition has  better chance to be satisfied.
Revised: In general, when the observed set Ω gets larger, the required condition has a better chance of being satisfied.

Original: If we have complete observations, i.e. , Ω = { 1 , 2 , ..., M } , the required condition ( r ′ − r ′′ ) Ω ̸ = 0automatically holds .
Revised: If we have complete observations, i.e. , Ω = { 1 , 2 , ..., M } , the required condition ( r ′ − r ′′ ) Ω ̸ =automatically holds .

Original: Now we consider the more challenging case where we not only have missing features, but also havecorruptions on the observed features.
Revised: Now we consider the more challenging case where we not only have missing features but also have corruptions on the observed features.

Original: Theorem 2.
Revised: Theorem 2.

Original: (Exact feature recovery under threat model C) Assume the trained AutoEncodersatisfies D ϕ
Revised: (Exact feature recovery under threat model C) Assume the trained AutoEncoder satisfies D ϕ

Original: ( E ψ ( l ∗ )) =
Revised: ( E ψ ( l ∗ )) =

Original: l ∗ for the underlying uncorrupted feature vector l ∗ .
Revised: l ∗ for the underlying uncorrupted feature vector l ∗ .

Original: For ∀ r ′ , r ′′ in the rangeof the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with| ¯ S Ω | = q < | Ω | / 2 , it holds that (cid:80) i ∈ S Ω
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with | ¯ S Ω | = q < | Ω | / 2 , it holds that (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 , then for any h Ω =
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 , then for any h Ω =

Original: l ∗ Ω + e ∗ Ω with166 (cid:80) i ∈
Revised: l ∗ Ω + e ∗ Ω with (cid:80) i ∈

Original: Ω 1 { e ∗ i ̸ = 0 } ≤ q , l ∗ is the unique solution of Eq.
Revised: Ω 1 { e ∗ i ̸ = 0 } ≤ q , l ∗ is the unique solution of Eq.

Original: 5 and Eq. 4.
Revised: 5 and Eq. 4.

Original: The proof can be found in Section A of the supplemental material.
Revised: The proof can be found in Section A.4 of the supplemental material.

Original: We first show that any feasiblepoint of Eq.
Revised: We first show that any feasible point of Eq.

Original: 5 that is different from l ∗ would have  larger objective value.
Revised: 5 that is different from l ∗ would have a larger objective value.

Original: Then we use contradictionto show that there does not exist any global optimal solution of Eq. 4 that is different from l ∗ .
Revised: Then we use contradiction to show that there does not exist any global optimal solution of Eq. 4 that is different from l ∗ .

Original: Remark 1.
Revised: Remark 1.

Original: Theorem 2 not only requires v Ω ̸ = 0 like Theorem 1, but also requires (cid:80)
Revised: Theorem 2 not only requires v Ω ̸ = 0 like Theorem 1, but also requires (cid:80)

Original: i ∈
Revised: i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 >171 (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 .
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 .

Original: This is due to the presence of corruptions in the observed blocks (indexed by ¯ S Ω ).
Revised: This is due to the presence of corruptions in the observed blocks (indexed by ¯ S Ω ).

Original: When there is no corruption, i.e. , q = 0 , the corresponding set ¯ S Ω is empty and the required conditionreduces to (cid:80) i ∈
Revised: When there is no corruption, i.e. , q = 0 , the corresponding set ¯ S Ω is empty and the required condition reduces to (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 > 0 , i.e. , v Ω ̸ = 0 .
Revised: ∥ v i ∥ 2 > 0 , i.e. , v Ω ̸ = 0 .

Original: Second, Theorem 2 is a universal certification, whereany subset ¯ S Ω of the observations can be malicious, as long as | ¯ S Ω | = q .
Revised: Second, Theorem 2 is a universal certification, where any subset ¯ S Ω of the observations can be malicious, as long as | ¯ S Ω | = q .

Original: In practice, once theattacker chooses q agents to perform adversarial attacks, the set ¯ S Ω gets fixed and corresponds toe ∗ i ̸ = 0 .
Revised: In practice, once the attacker chooses q agents to perform adversarial attacks, the set ¯ S Ω gets fixed and corresponds to e ∗ i ̸ = 0 .

Original: Then, we only require (cid:80) i ∈
Revised: Then, we only require (cid:80) i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 to hold for this particular fixed set ¯ S Ω ,to guarantee l ∗ is the unique solution of Eq.
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 to hold for this particular fixed set ¯ S Ω , to guarantee l ∗ is the unique solution of Eq.

Original: This can be easily seen from the proof of Theorem 2.
Revised: This can be easily seen from the proof of Theorem 2.

Original: Third, the condition (cid:80) i ∈ S Ω
Revised: Third, the condition (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 prefers v Ω to be not ‘spiky’.
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 prefers v Ω to be not ‘spiky’.

Original: Suppose r ′ and r ′′are the same on all the observed blocks (indexed by Ω ) except one, which means only one blockof v Ω is non-zero ( i.e. , v Ω is spiky).
Revised: Suppose r ′ and r ′′ are the same on all the observed blocks (indexed by Ω ) except one, which means only one block of v Ω is non-zero ( i.e. , v Ω is spiky).

Original: If that non-zero block is in the set ¯ S Ω , the required conditionwill not be met.
Revised: If that non-zero block is in the set ¯ S Ω , the required condition will not be met.

Original: Intuitively, if there are two instances differing in only one block of the observedfeatures, and the attacker corrupts that block, the FC can not recover the original feature.
Revised: Intuitively, if there are two instances differing in only one block of the observed features, and the attacker corrupts that block, the FC can not recover the original feature.

Original: In general,we prefer v Ω to be not ‘spiky’, thus the condition can hold when the number of corrupted blocks | ¯ S Ω |is sufficiently small.
Revised: In general, we prefer v Ω to be not ‘spiky’, thus the condition can hold when the number of corrupted blocks | ¯ S Ω | is sufficiently small.

Original: Such condition can be considered as the non-linear and missing-block extensionsof the Range Space Property in robust linear regression literature (Flores, 2015; Liu et al., 2018),which we will discuss next.
Revised: Such condition can be considered as the non-linear and missing-block extensions of the Range Space Property in robust linear regression literature (Flores, 2015; Liu et al., 2018), which we will discuss next.

Original: Finally, the fraction ( q/M ) of corrupted blocks that CoPur can toleratenot only depends on Ω , but also depends on the underlying feature subspace.
Revised: Finally, the fraction ( q/M ) of corrupted blocks that CoPur can tolerate not only depends on Ω , but also depends on the underlying feature subspace.

Original: It can tolerate up to50% ×| Ω | corrupted blocks if there is enough redundancy among the agents, e.g., if every agent holdsexactly the same embedded features, then every block of v = r ′ − r ′′ is  same, and q approaches50% ×| Ω | .
Revised: It can tolerate up to 50% ×| Ω | corrupted blocks if there is enough redundancy among the agents, e.g., if every agent holds exactly the same embedded features, then every block of v = r ′ − r ′′ is the same, and q approaches 50% ×| Ω | .

Original: Remark 2.
Revised: Remark 2.

Original: To draw the connections to the Range Space Property in robust linear regression, let’sconsider a typical case where there is no non-linear activation function in the last layer of the
Revised: To draw the connections to the Range Space Property in robust linear regression, let’s consider a typical case where there is no non-linear activation function in the last layer of the

Original: AutoEncoder (other layers can have non-linear activation functions).
Revised: AutoEncoder (other layers can have non-linear activation functions).

Original: Let W end be the weight matrixof the last layer of the AutoEncoder.
Revised: Let W end be the weight matrix of the last layer of the AutoEncoder.

Original: Note that the difference between any two vectors from the rangeof the decoder D ϕ ( · ) is contained in the Range ( W end ) , i.e. , v ∈ Range ( W end ) \ 0 .
Revised: Note that the difference between any two vectors from the range of the decoder D ϕ ( · ) is contained in the Range ( W end ) , i.e. , v ∈ Range ( W end ) \ 0 .

Original: Similar to the socalled ‘leverage constant’ in robust linear regression (Flores, 2015; Liu et al., 2018), we can define theblock-wise ‘incomplete’ leverage constant 
Revised: Similar to the so called ‘leverage constant’ in robust linear regression (Flores, 2015; Liu et al., 2018), we can define the block-wise ‘incomplete’ leverage constant [Equation]

Original: [Equation]
Revised: [Equation]

Original: which is between 0 and 1, and is monotonic increasing when the number of corrupted blocks qdecreases.
Revised: which is between 0 and 1, and is monotonic increasing when the number of corrupted blocks q decreases.

Original: A sufficient condition for (cid:80) i ∈ S Ω
Revised: A sufficient condition for (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 0 to hold is c Ω q ( W end ) > 0 .
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 0 to hold is c Ω q ( W end ) > 0 .

Original: Lastly, it is very useful to note that Sharon et al.
Revised: Lastly, it is very useful to note that Sharon et al.

Original: (2009) designed an algorithm to calculate theleverage constant, which is possible to be extended to our block-wise incomplete leverage constanthere.
Revised: (2009) designed an algorithm to calculate the leverage constant, which is possible to be extended to our block-wise incomplete leverage constant here.

Original: Consider the toy example again where every agent holds exactly the same embedded features,and q = 0 .
Revised: Consider the toy example again where every agent holds exactly the same embedded features, and q = 0 .

Original: 1 × | Ω | , then c Ω q ( W end ) = 0 .
Revised: 1 × | Ω | , then c Ω q ( W end ) = 0 .

Original: Remark 3.
Revised: Remark 3.

Original: It is interesting to note that in the recent Generative model based compressive sensingwork (Bora et al., 2017), they also impose conditions on the difference between any two vectors in therange of the decoder, though without missing blocks.
Revised: It is interesting to note that in the recent Generative model based compressive sensing work (Bora et al., 2017), they also impose conditions on the difference between any two vectors in the range of the decoder, though without missing blocks.

Original: But the conditions therein are quite differentfrom ours due to the different purposes.
Revised: But the conditions therein are quite different from ours due to the different purposes.

Original: 3.2 Noisy case
Revised: 3.2 Noisy case

Original: Now we analyze the ‘noisy case’ where the learned AutoEncoder can only approximately reconstructthe underlying uncorrupted feature vector l ∗ , i.e. , (cid:80) Mi
Revised: Now we analyze the ‘noisy case’ where the learned AutoEncoder can only approximately reconstruct the underlying uncorrupted feature vector l ∗ , i.e. , (cid:80) Mi

Original: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .
Revised: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .

Original: Again, to helpthe reader understand step-by-step, our analyses start from the threat model B.
Revised: Again, to help the reader understand step-by-step, our analyses start from the threat model B.

Original: Theorem 3.
Revised: Theorem 3.

Original: (Stable feature recovery under threat model B) Assume the trained AutoEncodersatisfies (cid:80) Mi 
Revised: (Stable feature recovery under threat model B) Assume the trained AutoEncoder satisfies (cid:80) Mi =1 ∥

Original: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .

Original: Let ˆ lbe the solution of Eq. 6.
Revised: Let ˆ l be the solution of Eq. 6.

Original: Given the incomplete observations h Ω =
Revised: Given the incomplete observations h Ω =

Original: l ∗ Ω , where Ω is the index set ofobserved agent blocks.
Revised: l ∗ Ω , where Ω is the index set of observed agent blocks.

Original: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) , define v = r ′ − r ′′ , if it holdsthat (cid:80) i ∈
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) , define v = r ′ − r ′′ , if it holds that (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ ( E ψ (ˆ l )) − D ϕ
Revised: ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ ( E ψ (ˆ l )) − D ϕ

Original: ( E ψ ( l ∗ ))
Revised: ( E ψ ( l ∗ ))

Original: ∥ 2 ≤ ∆ .
Revised: ∥ 2 ≤ ∆ .

Original: The proof can be found in Section A of the supplemental material.
Revised: The proof can be found in Section A.5 of the supplemental material.

Original: Remark 4.
Revised: Remark 4.

Original: First, note that (cid:80) Mi
Revised: First, note that (cid:80) Mi

Original: =1 ∥ v i ∥ 2 ≥ (cid:113)(cid:80)
Revised: =1 ∥ v i ∥ 2 ≥ (cid:113)(cid:80)

Original: Mi
Revised: Mi

Original: =1 ∥ v i ∥ 22 = ∥ v ∥ 2 .
Revised: =1 ∥ v i ∥ 22 = ∥ v ∥ 2 .

Original: When ∥ v ∥ 2 > ∆ , we have(cid:80) Mi =1 ∥ v i ∥ 2 > ∆ .
Revised: When ∥ v ∥ 2 > ∆ , we have (cid:80) Mi =1 ∥ v i ∥ 2 > ∆ .

Original: Again, we prefer v to be not ‘spiky’, thus the condition (cid:80) i ∈
Revised: Again, we prefer v to be not ‘spiky’, thus the condition (cid:80) i ∈

Original: Ω ∥ v i ∥ 2 > 2 δ can besatisfied as long as | Ω | is large enough (assume ∆ > 2 δ ).
Revised: Ω ∥ v i ∥ 2 > 2 δ can be satisfied as long as | Ω | is large enough (assume ∆ > 2 δ ).

Original: Now we consider the most challenging case where we not only have missing features, but also havecorruptions in the observed features.
Revised: Now we consider the most challenging case where we not only have missing features but also have corruptions in the observed features.

Original: Theorem 4.
Revised: Theorem 4.

Original: (Stable feature recovery under threat model C) Assume the trained AutoEncodersatisfies (cid:80) Mi 
Revised: (Stable feature recovery under threat model C) Assume the trained AutoEncoder satisfies (cid:80) Mi =1 ∥

Original: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .

Original: Letˆ l be the solution of Eq. 6.
Revised: Let ˆ l be the solution of Eq. 6.

Original: Given the incomplete observations h Ω =
Revised: Given the incomplete observations h Ω =

Original: l ∗ Ω + e ∗ Ω , where Ω is theindex set of observed agent blocks, define S Ω as the subset of Ω corresponds to e ∗ i = 0 ,
Revised: l ∗ Ω + e ∗ Ω , where Ω is the index set of observed agent blocks.

Original: and¯ S Ω = Ω \ S Ω . For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · )  , define v = r ′ − r ′′ , if  it holds that(cid:80) i ∈ S Ω
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with | ¯ S Ω | = q , it holds that (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ

Original: ( E ψ (ˆ l ))
Revised: ( E ψ (ˆ l ))

Original: − D ϕ
Revised: − D ϕ

Original: ( E ψ ( l ∗ ))
Revised: ( E ψ ( l ∗ ))

Original: ∥ 2 ≤ ∆  .
Revised: ∥ 2 ≤ ∆ as long as (cid:80) i ∈ Ω 1 { e ∗ i ̸ = 0 } ≤ q .

Original: The proof can be found in Section A of the supplemental material.
Revised: The proof can be found in Section A.6 of the supplemental material.

Original: Robust prediction.
Revised: Robust prediction.

Original: Recall that the server has learned a classifier f θ 0 during the training.
Revised: Recall that the server has learned a classifier f θ 0 during the training.

Original: We canfurther smooth this classifier f θ 0 to get f s (e.g., via Randomized Smoothing (Cohen et al., 2019;
Revised: We can further smooth this classifier f θ 0 to get f s (e.g., via Randomized Smoothing (Cohen et al., 2019;

Original: Lecuyer et al., 2019; Li et al., 2018))
Revised: Lecuyer et al., 2019; Li et al., 2018))

Original: such that f s ( D ϕ ( E ψ (ˆ l )))
Revised: such that f s ( D ϕ ( E ψ (ˆ l )))

Original: =
Revised: =

Original: f s ( D ϕ
Revised: f s ( D ϕ

Original: ( E ψ (ˆ l ))
Revised: ( E ψ (ˆ l ))

Original: + v ) for ∀∥ v ∥ 2 ≤∆ .
Revised: + v ) for ∀∥ v ∥ 2 ≤ ∆ .

Original: As we have ∥D ϕ ( E ψ (ˆ l ))
Revised: As we have ∥D ϕ ( E ψ (ˆ l ))

Original: − D ϕ
Revised: − D ϕ

Original: ( E ψ ( l ∗ ))
Revised: ( E ψ ( l ∗ ))

Original: ∥ 2 ≤ ∆ , then it is guaranteed that f s ( D ϕ ( E ψ (ˆ l )))
Revised: ∥ 2 ≤ ∆ , then it is guaranteed that f s ( D ϕ ( E ψ (ˆ l )))

Original: =
Revised: =

Original: f s ( D ϕ
Revised: f s ( D ϕ

Original: ( E ψ ( l ∗ ))) .
Revised: ( E ψ ( l ∗ ))) .

Original: Remark 5.
Revised: Remark 5.

Original: The condition (cid:80) i ∈
Revised: The condition (cid:80) i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ required by Theorem 4 is stronger thanthe condition (cid:80) i ∈
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ required by Theorem 4 is stronger than the condition (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 > 2 δ required by Theorem 3.
Revised: ∥ v i ∥ 2 > 2 δ required by Theorem 3.

Original: This is due to the presence of corruptionsin the observed blocks.
Revised: This is due to the presence of corruption in the observed blocks.

Original: As discussed in Remark 4, when ∥ v ∥ 2 > ∆ , we have (cid:80) Mi
Revised: As discussed in Remark 4, when ∥ v ∥ 2 > ∆ , we have (cid:80) Mi

Original: =1 ∥ v i ∥ 2 > ∆ .
Revised: =1 ∥ v i ∥ 2 > ∆ .

Original: Again, we prefer v to be not ‘spiky’, thus (cid:80) i ∈ Ω ∥ v i ∥ 2 can have a significant portion of massfrom (cid:80) Mi =1 ∥ v i ∥ 2 when the number of observed blocks | Ω | is large.
Revised: Again, we prefer v to be not ‘spiky’, thus (cid:80) i ∈ Ω ∥ v i ∥ 2 can have a significant portion of mass from (cid:80) Mi =1 ∥ v i ∥ 2 when the number of observed blocks | Ω | is large.

Original: Further, when the numberof corrupted blocks q = | ¯ S Ω | is small enough, we can have (cid:80) i ∈
Revised: Further, when the number of corrupted blocks q = | ¯ S Ω | is small enough, we can have (cid:80) i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ .
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ .

Original: To draw the connections to the Range Space Property in robust linear regression, we continueon the discussions in Remark 2.
Revised: To draw the connections to the Range Space Property in robust linear regression, we continue on the discussions in Remark 2.

Original: Let’s say (cid:80) i ∈
Revised: Let’s say (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 = ∆ ′ , then a sufficient condition for243 (cid:80) i ∈ S Ω
Revised: ∥ v i ∥ 2 = ∆ ′ , then a sufficient condition for (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ to hold is [ c Ω q ( W end ) − (1 − c Ω q ( W end ))] > 2 δ/ ∆ ′ .
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ to hold is [ c Ω q ( W end ) − (1 − c Ω q ( W end ))] > 2 δ/ ∆ ′ .

Original: Intuitively,the smaller number q of corrupted blocks and smaller δ ( i.e. , better recovered feature subspace) aswell as larger observed set Ω (then ∆ ′ approaches ∆ ), can lead to smaller recovery error bound ∆ .
Revised: Intuitively, the smaller number q of corrupted blocks and smaller δ ( i.e. , better recovered feature subspace) as well as larger observed set Ω (then ∆ ′ approaches ∆ ), can lead to smaller recovery error bound ∆ .

Original:  Related work and discussion
Revised:  Related work and discussion

Original: Handling missing data and outliers in IoT. Sanyal & Zhang (2018) model the underlying uncorrupted IoT data as a low-dimensional linear subspace, i.e. , low-rank, and try to clean the corruptedsensor data by projecting them onto the estimated linear subspace.
Revised: Handling missing data and outliers in IoT. Sanyal & Zhang (2018) model the underlying uncorrupted IoT data as a low-dimensional linear subspace, i.e. , low-rank, and try to clean the corrupted sensor data by projecting them onto the estimated linear subspace.

Original: There is no guarantee of recoveringthe underlying uncorrupted sensor data.
Revised: There is no guarantee of recovering the underlying uncorrupted sensor data.

Original: This approach is a linear version of the manifold projectionmethod that we will discuss in detail.
Revised: This approach is a linear version of the manifold projection method that we will discuss in detail.

Original: Kekatos & Giannakis (2011) studied the robust sensing problem where each sensor i has a linearmeasurement of the unknown parameter x , i.e. , b i
Revised: Kekatos & Giannakis (2011) studied the robust sensing problem where each sensor i has a linear measurement of the unknown parameter x , i.e. , b i

Original: =
Revised: =

Original: A i x , where both b i and A i are known andsome sensors are additionally corrupted by large measurement errors, but without any missing values.
Revised: A i x , where both b i and A i are known and some sensors are additionally corrupted by large measurement errors, but without any missing values.

Original: They showed that it is possible to recover the true parameter x despite outlier corruptions.
Revised: They showed that it is possible to recover the true parameter x despite outlier corruptions.

Original: They alsoconsider the scenario with dense inlier noise in the measurements, i.e. , where the measurements b ionly approximately lie on the range space of matrix
Revised: They also consider the scenario with dense inlier noise in the measurements, i.e. , where the measurements b i only approximately lie on the range space of matrix

Original: A i , but their corresponding method does nothave any recovery guarantee in such a noisy case.
Revised: A i , but their corresponding method does not have any recovery guarantee in such a noisy case.

Original: There are also some works that study the scenario where the sensors observe different noisy versionsof the same data vector, and they show it is possible to find a set of unattacked sensors based on theassumption that  fraction of attacked sensors is small, e.g., (Zhang et al., 2018) or utilizing additionalknowledge that some sensors are not attacked (Wilson & Veeravalli, 2016).
Revised: There are also some works that study the scenario where the sensors observe different noisy versions of the same data vector, and they show it is possible to find a set of unattacked sensors based on the assumption that the fraction of attacked sensors is small, e.g., (Zhang et al., 2018) or utilizing additional knowledge that some sensors are not attacked (Wilson & Veeravalli, 2016).

Original: We refer the interestedreaders to the survey paper (Zhang et al., 2018) for additional details on this line of work.
Revised: We refer interested readers to the survey paper (Zhang et al., 2018) for additional details on this line of work.

Original: Certified robustness for defending against adversarial attacks.
Revised: Certified robustness for defending against adversarial attacks.

Original: In the adversarial machine learningliterature, many defense methods have been proposed to deal with adversarial perturbations.
Revised: In the adversarial machine learning literature, many defense methods have been proposed to deal with adversarial perturbations.

Original: Werefer the reader to Chakraborty et al.
Revised: We refer the reader to Chakraborty et al.

Original: (2018) for a comprehensive survey.
Revised: (2018) for a comprehensive survey.

Original: However, most of theexisting defenses can only deal with small adversarial perturbations.
Revised: However, most of the existing defenses can only deal with small adversarial perturbations.

Original: Even worse, as new defensesare proposed, corresponding new attacks are also developed to adaptively attack them.
Revised: Even worse, as new defenses are proposed, corresponding new attacks are also developed to adaptively attack them.

Original: Experimentaldemonstrations of a defense’s efficacy based on currently existing attacks do not provide a generalproof of security.
Revised: Experimental demonstrations of a defense’s efficacy based on currently existing attacks do not provide general proof of security.

Original: Therefore, certifiably robust defense are of great interest.
Revised: Therefore, certifiably robust defenses are of great interest.

Original: Randomized Smoothing(RS) is a promising technique (Cohen et al., 2019; Lecuyer et al., 2019; Li et al., 2018) whichsmoothes a base classifier to get a robust classifier, such that the prediction of this robust classifer on adversarially perturbed input is the same as its prediction on the original input, when the ℓ p -normof the adversarial perturbation is sufficiently bounded.
Revised: Randomized Smoothing (RS) is a promising technique (Cohen et al., 2019; Lecuyer et al., 2019; Li et al., 2018) which smoothes a base classifier to get a robust classifier, such that the prediction of this robust classifier on adversarially perturbed input is the same as its prediction on the original input when the ℓ p -norm of the adversarial perturbation is sufficiently bounded.

Original: Unfortunately, in our setting, the adversarialperturbation on compromised devices’ embedded features can be very large.
Revised: Unfortunately, in our setting, the adversarial perturbation on compromised devices’ embedded features can be very large.

Original: Further, RS does nothave robustness guarantees when there are missing features.
Revised: Further, RS does not have robustness guarantees when there are missing features.

Original: Another promising defense is Randomized Ablation (Levine & Feizi, 2020), which was proposed todefend against sparse adversarial attacks where a small number of pixels in an image are changed tosome arbitrary values in [0,1].
Revised: Another promising defense is Randomized Ablation (Levine & Feizi, 2020), which was proposed to defend against sparse adversarial attacks where a small number of pixels in an image are changed to some arbitrary values in [0,1].

Original: We extend this to the collaborative inference setting for comparison:the FC learns to make predictions based on randomly selected subsets of agent features during thetraining; and in the inference phase, FC randomly select a subset of agents to make a prediction.
Revised: We extend this to the collaborative inference setting for comparison: the FC learns to make predictions based on randomly selected subsets of agent features during the training; and in the inference phase, FC randomly selects a subset of agents to make a prediction.

Original: Thefinal prediction is by majority voting over the random trials.
Revised: The final prediction is by majority voting over the random trials.

Original: However, it’s hard to design a baseclassifier that is able to train on randomly ablated input features 1 .
Revised: However, it is hard to design a base classifier that is able to train on randomly ablated input features 1 .

Original: Another limitation of this method isthat its robust prediction (based on randomly ablated features) may not be the same as the attack-freevanilla setting ( i.e. , the prediction of the vanilla classifier on the uncorrupted data).
Revised: Another limitation of this method is that its robust prediction (based on randomly ablated features) may not be the same as the attack-free vanilla setting ( i.e. , the prediction of the vanilla classifier on the uncorrupted data).

Original: Further, to choosehow many pixels to ablate, it needsthe knowledge of the number of corrupted pixels.
Revised: Further, to choose how many pixels to ablate, one needs knowledge of the number of corrupted pixels.

Original: Manifold Projection vs. Feature Purification.
Revised: Manifold Projection vs. Feature Purification.

Original: In the centralized setting, several works (Meng &
Revised: In the centralized setting, several works (Meng &

Original: Chen, 2017; Ilyas et al., 2017; Lindqvist et al., 2018) proposed to project the adversarial perturbedimage onto the manifold of the normal data to defend against adversarial attacks.
Revised: Chen, 2017; Ilyas et al., 2017; Lindqvist et al., 2018) proposed to project the adversarial perturbed image onto the manifold of the normal data to defend against adversarial attacks.

Original: A natural questionis: why do we propose a feature purification method, instead of using manifold projection?
Revised: A natural question is: why do we propose a feature purification method, instead of using manifold projection?

Original: First,the manifold projection approach can not guarantee  recovery of the underlying features.
Revised: First, the manifold projection approach can not guarantee the recovery of the underlying features.

Original: In fact, theresulting projected features can be arbitrarily far from the underlying true features if the adversarialperturbation can be arbitrarily large.
Revised: In fact, the resulting projected features can be arbitrarily far from the underlying true features if the adversarial perturbation can be arbitrarily large.

Original: To make this concrete, consider the linear manifold, i.e. ,subspace.
Revised: To make this concrete, consider the linear manifold, i.e. , subspace.

Original: Suppose the underlying true feature vector l is generated from the column space of ad by r tall matrix U .
Revised: Suppose the underlying true feature vector l is generated from the column space of a d by r tall matrix U .

Original: We observe the corrupted feature vector h , which can be decomposed asl + e , where e models the adversarial perturbations.
Revised: We observe the corrupted feature vector h , which can be decomposed as l + e , where e models the adversarial perturbations.

Original: Projecting h onto the column space of Uwill get P U
Revised: Projecting h onto the column space of U will get P U

Original: h
Revised: h

Original: = P U l + P U e =
Revised: = P U l + P U e =

Original: l + P U e ,
Revised: l + P U e ,

Original: where P U =
Revised: where P U =

Original: U ( U T U ) − 1 U T .
Revised: U ( U T U ) − 1 U T .

Original: So the recovery erroris P U 
Revised: So the recovery error is P U h − l

Original: h − l = P U e .
Revised: = P U e .

Original: Apparently, if the adversarial perturbation vector e is not orthogonal to thissubspace (which is often the case), the recovery error would be proportional to the magnitude of theperturbation!
Revised: Apparently, if the adversarial perturbation vector e is not orthogonal to this subspace (which is often the case), the recovery error would be proportional to the magnitude of the perturbation!

Original: It’s hard to recover the underlying true feature even if we have a perfect subspace, andthe problem looks even more challenging when the underlying manifold is non-linear.
Revised: It’s hard to recover the underlying true feature even if we have a perfect subspace, and the problem looks even more challenging when the underlying manifold is non-linear.

Original: Fortunately,in our collaborative inference setting, there is important prior information about the pattern ofthe corruptions that we can leverage: the perturbation e has a block-wise structure (each blockcorresponds to an agent), and the fraction of the perturbed blocks is small.
Revised: Fortunately, in our collaborative inference setting, there is important prior information about the pattern of the corruptions that we can leverage: the perturbation e has a block-wise structure (each block corresponds to an agent), and the fraction of the perturbed blocks is small.

Original: Therefore, e has ablock-sparse structure.
Revised: Therefore, e has a block-sparse structure.

Original: By exploring this fact, our proposed feature purification method can provablyrecover the underlying feature vector exactly or with bounded error, even if the underlying manifoldis highly non-linear and the magnitude of the perturbation is arbitrarily large.
Revised: By exploring this fact, our proposed feature purification method can provably recover the underlying feature vector exactly or with bounded error, even if the underlying manifold is highly non-linear and the magnitude of the perturbation is arbitrarily large.

Revised: Purifier Network vs. Feature Purification.

Original: Empirical evaluation
Revised: Besides the Manifold-Projection based approaches, one recent work (Naseer et al., 2020) tries to train a purifier neural network that maps any corrupted feature h to the underlying true feature l , in the centralized setting. However, that method needs to generate the adversarial training examples based on the assumption that the magnitude of the adversarial perturbation e is bounded by small ϵ (see Naseer et al. (2020, Eq.4)), which is different from our collaborative inference setting where the adversarial perturbation is unbounded. Further, in our collaborative inference setting, arbitrary α fraction of the blocks of feature l can be perturbed by arbitrarily large values.

Revised: Based on the high (combinatorial) cardinality of the input space, it is highly doubtful that one can learn a magic purifier network that is able to map a combinatorial number of different block-perturbation patterns (and with arbitrary unbounded perturbation values) to the underlying true feature l .

Revised: In sharp contrast, the proposed method can guarantee to recover the underlying true feature l exactly.

Revised:  Empirical evaluation

Original: We first describe  experimental setup and introduce a new attack in Section 5.1, then present experimental results in Section 5.2.
Revised: We first describe the experimental setup and introduce a new attack in Section 5.1, then present experimental results in Section 5.2.

Original: More results can be found in Section B of  supplemental material.
Revised: More results can be found in Section B of the supplemental material.

Original: 5.1 Experimental setup
Revised: 5.1 Experimental setup

Original: Datasets.
Revised: Datasets.

Original: We study the classification task on ExtraSensory (Vaizman et al., 2017) and NUSWIDE (Chua et al., 2009) datasets.
Revised: We study the classification task on ExtraSensory (Vaizman et al., 2017) and NUSWIDE (Chua et al., 2009) datasets.

Original: ExtraSensory contains the measurements from diverse sensorsof smart phone and smart watch.
Revised: ExtraSensory contains the measurements from diverse sensors of smart phone and smart watch.

Original: We divide the sensors into 10 agents, which correspond to phoneaccelerometer, gyroscope, magnetometer, watch accelerometer, compass, location, audio, phonestates (e.g., battery, wifi, ringing mode), environment sensing (e.g., light, air pressure, humidity, andtemperature), and time-of-day (e.g., morning).
Revised: We divide the sensors into ten agents, which correspond to the phone accelerometer, gyroscope, magnetometer, watch accelerometer, compass, location, audio, phone states (e.g., battery, wifi, ringing mode), environment sensing (e.g., light, air pressure, humidity, and temperature), and time-of-day (e.g., morning).

Original: There are naturally missing-feature problems, e.g., theuser may not permit the use of microphone and/or location.
Revised: There are naturally missing-feature problems, e.g., the user may not permit the use of microphone and/or location.

Original: The original purpose of this dataset isto recognize the behavioral context of the user (e.g., sitting, walking, and running) and recommendmore appropriate music.
Revised: The original purpose of this dataset is to recognize the behavioral context of the user (e.g., sitting, walking, and running) and recommend more appropriate music.

Original: We use  first 1721 samples from a user for training, and the rest 465 samplesfor testing, with the binary label ’sitting’ or not.
Revised: We use the first 1721 samples from a user for training, and the restsamples for testing, with the binary label ‘sitting’ or not.

Original: In NUS-WIDE, each sample has 634 image features, 1000 text features, and 5 different labels, i.e. ,‘buildings’, ‘grass’, ‘animal’, ‘water’, ‘person’.
Revised: In NUS-WIDE, each sample has 634 image features, 1000 text features, and 5 different labels, i.e. , ‘buildings’, ‘grass’, ‘animal’, ‘water’, ‘person’.

Original: We split the features into 4 agents, where the 1st agent holds 360-d image features, 2nd agent holds  rest of  image features, and the remaining 2 agentseach hold 500-d text features.
Revised: We split the features into four agents, where the 1st agent holds 360-d image features, 2nd agent holds the rest of the image features, and the remaining two agents each hold 500-d text features.

Original: We use 60000 samples for training, 1000 samples for testing targetedattack, and 10000 samples for testing untargeted attack.
Revised: We use 60000 samples for training, 1000 samples for testing targeted attacks, and 10000 samples for testing untargeted attacks.

Original: Attack Setup.
Revised: Attack Setup.

Original: We perform the inference adversarial attacks, as well as its combination with missingfeature attack (missing-feature attack alone is not very effective).
Revised: We perform the inference adversarial attacks, as well as their combination with the missing-feature attack (missing-feature attack alone is not very effective).

Original: For the adversarial attack, we testboth targeted and untargeted attacks.
Revised: For the adversarial attack, we test both targeted and untargeted attacks.

Original: For the targeted attack , we adopt the commonly used Projected Gradient Descent (PGD) attackto generate adversarial perturbations on the embedded features held by malicious agents.
Revised: For the targeted attack , we adopt the commonly used Projected Gradient Descent (PGD) attack to generate adversarial perturbations on the embedded features held by malicious agents.

Original: However,since in this collaborative inference setting, the magnitude of the perturbation on the embeddedfeatures is allowed to be very large, we omit the projection step in PGD (or one can think  it asprojecting onto a very large norm ball as the maximum perturbation constraint).
Revised: However, since in this collaborative inference setting, the magnitude of the perturbation on the embedded features is allowed to be very large, we omit the projection step in PGD (or one can think of it as projecting onto a very large norm ball as the maximum perturbation constraint).

Original: In the ExtraSensorydataset, the target label is simply the opposite of the true label, since it is  binary classification.
Revised: In the ExtraSensory dataset, the target label is simply the opposite of the true label, since it is a binary classification.

Original: We set the PGD attack with  learning rate  0.5 and 30 iterations, so that it can successfully attackthe unsecured model.
Revised: We set the PGD attack with a learning rate of 0.5 and 30 iterations so that it can successfully attack the unsecured model.

Original: While in the NUS-WIDE dataset, the target label is set to be ‘grass’.
Revised: While in the NUS-WIDE dataset, the target label is set to be ‘grass’.

Original: Weset the PGD attack with  learning rate  0.1 and 50 iterations.
Revised: We set the PGD attack with a learning rate of 0.1 and 50 iterations.

Original: Note that PGD attack is a whitebox attack, where the attacker tries to find arg min h adv ℓ ( f θ 

Revised: Note that PGD attack is a whitebox attack, where the attacker tries to find arg min h adv ℓ ( f θ

Original: ( { h adv , h benign } ) , y A ) via using thegradient dℓ ( f θ 

Revised: ( { h adv , h benign } ) , y A ) via using the gradient dℓ ( f θ

Original: { h adv , h benign } ) , y A ) /d
Revised: { h adv , h benign } ) , y A ) /d

Original: h adv , where h adv denotes the overall embedded featuresheld by malicious agents and
Revised: h adv , where h adv denotes the overall embedded features held by malicious agents and

Original: y A is the target label.
Revised: y A is the target label.

Original: In practice, the attacker may not easily obtainsuch gradient information, which usually needs to know the trained global model f
Revised: In practice, the attacker may not easily obtain such gradient information, which usually needs to know the trained global model f

Original: θ
Revised: θ

Original: and h benign .
Revised: and h benign .

Original: Nevertheless, this attack serves as a smart targeted perturbation on the embedded features to test therobustness of the defense methods.
Revised: Nevertheless, this attack serves as a smart targeted perturbation on the embedded features to test the robustness of the defense methods.

Original: Motivated by the discussions in Section 4, we propose and test an untargeted attack , which isagnostic to the model, training data, label, as well asthe features held by other agents.
Revised: Motivated by the discussions in Section 4, we propose and test an untargeted attack , which is agnostic to the model, training data, label, as well as features held by other agents.

Original: We call itdistributed feature-flipping attack .
Revised: We call it distributed feature-flipping attack .

Original: In this attack, the malicious agent(s) simply flips the sign of itsembedded feature vector l i and further amplifies its magnitude.
Revised: In this attack, the malicious agent(s) simply flips the sign of its embedded feature vector l i and further amplifies its magnitude.

Original: Formally, during this inference phaseuntargeted attack, the malicious agent i provides the embedded features h
Revised: Formally, during this inference phase untargeted attack, the malicious agent i provides the embedded features h

Original: i
Revised: i

Original: =
Revised: =

Original: − Amplification × l ito the FC, instead of providing the true embedded features l i .
Revised: − Amplification × l i to the FC, instead of providing the true embedded features l i .

Original: This attack can be easily understoodin the binary classification with  linear classifer, since flipping the sign of the whole feature vectorwill likely flip the sign of the predicted label as well.
Revised: This attack can be easily understood in binary classification with a linear classifier since flipping the sign of the whole feature vector will likely flip the sign of the predicted label as well.

Original: To illustrate the effectiveness of this attackagainst the manifold-projection based defenses, we can look it as adding the corruption vectore i = − (1 + Amplification ) × l i to the true embedded feature vector l i of i -th agent, which ismalicious.
Revised: To illustrate the effectiveness of this attack against the manifold-projection based defenses, we can look it as adding the corruption vector e i = − (1 + Amplification ) × l i to the true embedded feature vector l i of i -th agent, which is malicious.

Original: So the overall perturbation vector e has many zero blocks corresponding to benignagents.
Revised: So the overall perturbation vector e has many zero blocks corresponding to benign agents.

Original: Its non-zero blocks have values e i = − (1 + Amplification )
Revised: Its non-zero blocks have values e i = − (1 + Amplification )

Original: × l i , ∀ i corresponds to themalicious agents.
Revised: × l i , ∀ i corresponds to the malicious agents.

Original: So the inner product between perturbation vector e and true embedded featuresl is − (1 + Amplification ) (cid:80) i ∈
Revised: So the inner product between perturbation vector e and true embedded features l is − (1 + Amplification ) (cid:80) i ∈

Original: Ω adv ∥ l i ∥ 22 .
Revised: Ω adv ∥ l i ∥ 22 .

Original: Such perturbation vector e introduces correlations withthe underlying manifold of the uncorrupted embedded features.
Revised: Such perturbation vector e introduces correlations with the underlying manifold of the uncorrupted embedded features.

Original: Therefore, projecting the corruptedembedded features h onto the manifold is hard to get rid of such corruptions.
Revised: Therefore, projecting the corrupted embedded features h onto the manifold is hard to get rid of such corruptions.

Original: We will see suchfeature-flipping attack is very effective in attacking many existing defenses.
Revised: We will see such feature-flipping attack is very effective in attacking many existing defenses.

Original: Baseline methods.
Revised: Baseline methods.

Original: We compare with 1)
Revised: We compare with 1)

Original: Unsecured : no defense.
Revised: Unsecured : no defense.

Original: 2) Manifold Projection : project thecorrupted embedded features h test onto the learned manifold via Autoencoder, i.e. , D ϕ ( E ψ ( h test )) .3)
Revised: 2) Manifold Projection : project the corrupted embedded features h test onto the learned manifold via Autoencoder, i.e. , D ϕ ( E ψ ( h test )) .

Original:  Randomized Smoothing (Cohen et al., 2019), where randomly generated Gaussian noise is addedto h test for the FC to make the prediction, the final prediction is by majority voting over 1000 suchrandom trials.
Revised: 3) Randomized Smoothing (Cohen et al., 2019), where randomly generated Gaussian noise is added to h test for the FC to make the prediction, the final prediction is by majority voting over 1000 such random trials.

Original: Randomized Smoothing Block : since only some block(s) of the embedded featuresare corrupted, we further compare with an oracle version of Randomized Smoothing, which only addsrandom Gaussian noise to the corrupted blocks of h test .
Revised: Randomized Smoothing Block : since only some block(s) of the embedded features are corrupted, we further compare with an oracle version of Randomized Smoothing, which only adds random Gaussian noise to the corrupted blocks of h test .

Original: Randomized Ablation (Levine & Feizi,2020) 6)
Revised: Randomized Ablation (Levine & Feizi, 2020) 6)

Original: Adversarial Training (AT) (Madry et al., 2018): standard adversarial training via using
Revised: Adversarial Training (AT) (Madry et al., 2018): standard adversarial training via using

Original: PGD attacks on h train to generate adversarial examples, but without using projection step to limit themagnitude of adversarial perturbations.
Revised: PGD attacks on h train to generate adversarial examples, but without using projection step to limit the magnitude of adversarial perturbations.

Original: For defending against the targeted attack, it knows the targetlabel and leverages that to generate targeted adversarial examples.
Revised: For defending against the targeted attack, it knows the target label and leverages that to generate targeted adversarial examples.

Original: For defending against untargetedattack, it uses untargeted PGD attack to generate adversarial examples.
Revised: For defending against untargeted attack, it uses untargeted PGD attack to generate adversarial examples.

Original: 7) Block-wise Adversarial
Revised: 7) Block-wise Adversarial

Original: Training (BAT) : since only some block(s) of h test will be corrupted, we further propose and test ablock-wise version of AT: for each training sample’s embedded features, we randomly select a blockand generate adversarial perturbations (via corresponding targeted or untargeted PGD attack) on thatblock for adversarial training.
Revised: Training (BAT) : since only some block(s) of h test will be corrupted, we further propose and test a block-wise version of AT: for each training sample’s embedded features, we randomly select a block and generate adversarial perturbations (via corresponding targeted or untargeted PGD attack) on that block for adversarial training.

Original: 8) Oracle : where there is no attack.
Revised: 8) Oracle : where there is no attack.

Original: The implementation details of thebaselines and the proposed CoPur can be found in Section C of the supplemental material.
Revised: The implementation details of the baselines and the proposed CoPur can be found in Section C of the supplemental material.

Original: 5.2 Empirical results
Revised: 5.2 Empirical results

Original: We report the Robust Accuracy of each defense method, i.e. , classification accuracy on the adversarialcorrupted embedded features, against  distributed adversarial attack and its combination with missingfeature attack.
Revised: We report the Robust Accuracy of each defense method, i.e. , classification accuracy on the adversarial corrupted embedded features, against the distributed adversarial attack and its combination with the missing-feature attack.

Original: More experiments can be found in Section B of the supplemental material.
Revised: More experiments can be found in Section B of the supplemental material.

Original: A) Adversarial attack.
Revised: A) Adversarial attack.

Original: We first test the robustness of the defense methods under the targeted attack.
Revised: We first test the robustness of the defense methods under the targeted attack.

Original: Table 1 shows the Robust Accuracy of each method against targeted PGD attacks on NUS-WIDE and
Revised: Table 1 shows the Robust Accuracy of each method against targeted PGD attacks on NUS-WIDE and

Original: ExtraSensory datasets, where the last agent is malicious.
Revised: ExtraSensory datasets, where the last agent is malicious.

Original: The Manifold Projection method (whichhas no recovery guarantee) shows some resistance on the NUS-WIDE dataset, but is not robust on 
Revised: The Manifold Projection method (which has no recovery guarantee) shows some resistance on the NUS-WIDE dataset but is not robust on the

Original: ExtraSensory dataset.
Revised: ExtraSensory dataset.

Original: The proposed method achieves the best Robust Accuracy on both datasets.
Revised: The proposed method achieves the best Robust Accuracy on both datasets.

Original: In Figure 2a, we report the Robust Accuracy of the defense methods under the untargeted featureflipping attack on the ExtraSensory dataset, with Amplification ranging from [1, 3, 5, 10, 20], and thelast 3 agents perform such attack.
Revised: In Figure 2a, we report the Robust Accuracy of the defense methods under the untargeted featureflipping attack on the ExtraSensory dataset, with Amplification ranging from [1, 3, 5, 10, 20], and the last three agents perform such attack.

Original: The Robust Accuracies of the compared methods drop significantlywhen the Amplification gets larger.
Revised: The Robust Accuracy of the compared methods drops significantly when the Amplification gets larger.

Original: The proposed method significantly outperforms the baselineswith a large margin.
Revised: The proposed method significantly outperforms the baselines by a large margin.

Original: [Table]
Revised: [Table]

Original: combined attacks on ExtraSensory dataset with 10 agents: (a) Agents 8,9,10 provide corrupted embedded features h 8 , 9 , 10 = − Amplification × l 8 , 9 , 10 . (b) Agent 7 performs missing-feature attack, while agent 10 provides corrupted embedded features h 10 =
Revised: We are also interested in knowing the robustness limit of each defense when there are more and more agents performing adversarial attacks. Figure 3 shows the Robust Accuracy of each defense method w.r.t. the index set of agents that perform untargeted adversarial attack (Amplification=10) on ExtraSensory dataset.

Original: − Amplification × l 10 .
Revised: The Robustness Accuracy of the Adversarial Training method (green solid line) significantly drops when there are more than two malicious agents. The proposed method keeps very high Robust Accuracy even when agents 8-10 perform attack together. When agents 7perform attack together, the Robust Accuracy of the proposed method apparently drops, but it is still significantly better than other baseline methods.

Original: 393 B) Combined attack.
Revised: B) Combined attack.

Original: We now test the combined attack on ExtraSensory dataset, where agent 7which holds audio MFCC features, performs missing-feature attack, and agent 10 performs untargeted(Figure 2b) or targeted (Table 2) adversarial attack.
Revised: We now test the combined attack on ExtraSensory dataset, where agentwhich holds audio MFCC features, performs missing-feature attack, and agent 10 performs untargeted (Figure 2b) or targeted (Table 2) adversarial attack.

Original: The proposed method significantly outperformsthe baseline methods and maintains very high Robust Accuracy under both attacks.
Revised: The proposed method significantly outperforms the baseline methods and maintains very high Robust Accuracy under both attacks.

Original: [Table]
Revised: [Table]

Original:  Conclusions
Revised:  Conclusions

Original: In this work, we proposed a novel feature purification based robust collaborative inference frameworkto defend against a variety of attacks during inference, with theoretical guarantees.
Revised: In this work we proposed a novel feature purification based robust collaborative inference framework to defend against a variety of attacks during inference, with theoretical guarantees.

Original: The proposednon-linear robust decomposition method and its theoretical analyses may have much wider impacts.
Revised: The proposed non-linear robust decomposition method and its theoretical analyses may have much wider impacts.

Original: We further validate the robustness of the proposed framework through extensive experiments on
Revised: We further validate the robustness of the proposed framework through extensive experiments on

Original: ExtraSensory and NUS-WIDE datasets.
Revised: ExtraSensory and NUS-WIDE datasets.

Original: One of our future work is to extend Algorithm 1 of Sharonet al.
Revised: Our future work includes extending Algorithm 1 of Sharon et al.

Original: (2009) to calculate our block-wise incomplete leverage constant introduced in Remark2.
Revised: (2009) to calculate the block-wise incomplete leverage constant introduced in Remark

Original: Checklist
Revised: Acknowledgments and Disclosure of Funding

Original: [Table] For all authors...

Original: [Table] (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope? [Yes] [Table]
Revised: This work is partially supported by NSF grant No.1910100, NSF CNS No.2046726, NSF III 2046795,

Original: (b) Did you describe the limitations of your work?

Original: [Yes] Please see Section E of supplemental material.

Original: [Table]

Original: (c) Did you discuss any potential negative societal impacts of your work? [Yes] Please seeSection E of supplemental material. [Table] (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: IIS 1909577, CCF 1934986 and NIFA award 2020-67021-32799, a C3.ai DTI Award, a Jump Arches

Original: [Table] them? [Yes] [Table]
Revised: Award and the Alfred P. Sloan Foundation.

Original: If you are including theoretical results...

Original: [Table]

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] Please seeSection

Original: [Table]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] Please see Section Aof supplemental material.

Original: [Table]

Original: If you ran experiments... [Table] (a) Did you include the code, data, and instructions needed to reproduce the main exper- [Table]
Revised: S.K. was supported by

Original: imental results (either in the supplemental material or as a URL)?

Original: [Yes] Please seeSection C of supplemental material.

Original: [Table]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] Please see Section C of supplemental material.

Original: [Table] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes](d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)? [Yes] Please see Section C ofsupplemental material.4.
Revised: Google Inc.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators? [Yes] [Table]
Revised: The authors thank the anonymous reviewers for their constructive suggestions.

Original: (b) Did you mention the license of the assets?

Original: [Yes] License of ExtraSensory dataset is CCBY-NC-SA 4.0, but we can not find license for NUS-WIDE dataset.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [Yes] Please see Section C of supplemental material.
Revised: J.L. also thanks

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [Yes] Please see Section C of supplemental material. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: -Velarde for the helpful discussions regarding manifolds.

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]