Original: CoPur: Certifiably Robust Collaborative Inference via Feature Purification
Revised: CoPur: Certifiably Robust Collaborative Inference via Feature Purification

Original: Abstract
Revised: Abstract

Original: Collaborative inference leverages diverse features provided by different agents (e.g., sensors) for more accurate inference.
Revised: Collaborative inference leverages diverse features provided by different agents (e.g., sensors) for more accurate inference.

Original: A common setup is where each agent sends its embedded features instead of the raw data to the Fusion Center (FC) for joint prediction.
Revised: A common setup is where each agent sends its embedded features instead of the raw data to the Fusion Center (FC) for joint prediction.

Original: In this setting, we consider inference phase attacks when a small fraction of agents is compromised.
Revised: In this setting, we consider inference phase attacks when a small fraction of agents is compromised.

Original: The compromised agent either does not send embedded features to the FC or sends arbitrary embedded features.
Revised: The compromised agent either does not send embedded features to the FC or sends arbitrary embedded features.

Original: To address this, we propose a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold).
Revised: To address this, we propose a certifiably robust COllaborative inference framework via feature PURification (CoPur), by leveraging the block-sparse nature of adversarial perturbations on the feature vector, as well as redundancy across the embedded features (by assuming the overall features lie on an underlying lower dimensional manifold).

Original: We theoretically show that the proposed feature purification method can robustly recover the true feature vector, despite adversarial corruptions and/or incomplete observations.
Revised: We theoretically show that the proposed feature purification method can robustly recover the true feature vector, despite adversarial corruptions and/or incomplete observations.

Original: We also propose and test an untargeted distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents, and is shown to be effective in attacking state-of-the-art defenses.
Revised: We also propose and test an untargeted distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents, and is shown to be effective in attacking state-of-the-art defenses.

Original: Experiments on ExtraSensory and NUS-WIDE datasets show that CoPur significantly outperforms existing defenses in terms of robustness against targeted and untargeted adversarial attacks.
Revised: Experiments on ExtraSensory and NUS-WIDE datasets show that CoPur significantly outperforms existing defenses in terms of robustness against targeted and untargeted adversarial attacks.

Original:  Introduction
Revised:  Introduction

Original: Collaborative inference is an increasingly popular distributed prediction strategy that leverages diverse features provided by different agents (e.g., sensors) for more accurate inference.
Revised: Collaborative inference is an increasingly popular distributed prediction strategy that leverages diverse features provided by different agents (e.g., sensors) for more accurate inference.

Original: For example, in context-aware music recommendation, the app may wish to recommend more appropriate music based on the user’s behavioral context (e.g., sleeping, running) (Vaizman et al., 2017).
Revised: For example, in context-aware music recommendation, the app may wish to recommend more appropriate music based on the user’s behavioral context (e.g., sleeping, running) (Vaizman et al., 2017).

Original: Here, it can leverage diverse sensors from the user’s smart phone and smart watch, e.g., phone accelerometer, gyroscope, watch accelerometer, location, audio, phone states (e.g., wifi, ringing mode), and environment sensing (e.g., light).
Revised: Here, it can leverage diverse sensors from the user’s smart phone and smart watch, e.g., phone accelerometer, gyroscope, watch accelerometer, location, audio, phone states (e.g., wifi, ringing mode), and environment sensing (e.g., light).

Original: Thus, by combining information across sensors, much more accurate predictions can be made.
Revised: Thus, by combining information across sensors, much more accurate predictions can be made.

Original: In this work, we study the scenario where Internet-of-Things (IoT) devices send embedded features (instead of the raw data) to a Fusion Center (FC) for collaborative inference.
Revised: In this work, we study the scenario where Internet-of-Things (IoT) devices send embedded features (instead of the raw data) to a Fusion Center (FC) for collaborative inference.

Original: The IoT devices do not communicate with each other and may hold quite different raw data (e.g., image, acoustic,
Revised: The IoT devices do not communicate with each other and may hold quite different raw data (e.g., image, acoustic,

Original: Lidar data, etc).
Revised: Lidar data, etc).

Original: Nevertheless, as a result of distributed inference, the system is vulnerable to attack, where a subset of sensors are corrupted by an adversary at inference time, e.g., due to the lack of effective security mechanisms, connection errors, sensor failures, among other issues – this is also a common concern in defense applications (Abdelzaher et al., 2018; Xie et al., 2020).
Revised: Nevertheless, as a result of distributed inference, the system is vulnerable to attack, where a subset of sensors are corrupted by an adversary at inference time, e.g., due to the lack of effective security mechanisms, connection errors, sensor failures, among other issues – this is also a common concern in defense applications (Abdelzaher et al., 2018; Xie et al., 2020).

Original: Nevertheless, the goal of the system is robust inference based on the possibly incomplete and corrupted embedded features received from the IoT devices.
Revised: Nevertheless, the goal of the system is robust inference based on the possibly incomplete and corrupted embedded features received from the IoT devices.

Original: There have been a number of works on robust parameter estimation in IoT settings, where the robustness apparently comes from redundancy among different sensors’ linear measurements of the same parameter.
Revised: There have been a number of works on robust parameter estimation in IoT settings, where the robustness apparently comes from redundancy among different sensors’ linear measurements of the same parameter.

Original: For a comprehensive overview, we refer the interested reader to (Zhang et al., 2018).
Revised: For a comprehensive overview, we refer the interested reader to (Zhang et al., 2018).

Original: However, in our considered inference setting, each IoT device may hold quite different types of features, and the redundancy is less apparent.
Revised: However, in our considered inference setting, each IoT device may hold quite different types of features, and the redundancy is less apparent.

Original: We also observe that IoT applications are often communication-limited (Ko et al., 2018), thus, using lower-dimensional embeddings to communicate from each sensor is a common and effective approach to reduce the required communication overhead.
Revised: We also observe that IoT applications are often communication-limited (Ko et al., 2018), thus, using lower-dimensional embeddings to communicate from each sensor is a common and effective approach to reduce the required communication overhead.

Original: The problem we study here also applies to the inference phase of some Vertical Federated Learning (VFL) frameworks (Chen et al., 2020; Liu et al., 2020), where a server similarly combines different embedded features received from every agent to make a prediction.
Revised: The problem we study here also applies to the inference phase of some Vertical Federated Learning (VFL) frameworks (Chen et al., 2020; Liu et al., 2020), where a server similarly combines different embedded features received from every agent to make a prediction.

Original: One difference between our setting and VFL is: the agents in VFL are other untrusted parties, if an agent in VFL is malicious, it will likely attack during both training and inference; while in our setting, the agents (e.g., IoT sensors) are all owned by FC, and they can be well protected during training, and inference is more vulnerable.
Revised: One difference between our setting and VFL is: the agents in VFL are other untrusted parties, if an agent in VFL is malicious, it will likely attack during both training and inference; while in our setting, the agents (e.g., IoT sensors) are all owned by FC, and they can be well protected during training, and inference is more vulnerable.

Original: So far, there are no certifiable defenses for VFL against adversarial attacks during the inference phase.
Revised: So far, there are no certifiable defenses for VFL against adversarial attacks during the inference phase.

Original: The well-known robust aggregation method (Yin et al., 2018; Guerraoui et al., 2018;
Revised: The well-known robust aggregation method (Yin et al., 2018; Guerraoui et al., 2018;

Original: Blanchard et al., 2017; Fu et al., 2019; Pillutla et al., 2019; Fung et al., 2020; Chen et al., 2017; Xie et al., 2021) used for Horizontal Federated Learning (HFL) is not applicable.
Revised: Blanchard et al., 2017; Fu et al., 2019; Pillutla et al., 2019; Fung et al., 2020; Chen et al., 2017; Xie et al., 2021) used for Horizontal Federated Learning (HFL) is not applicable.

Original: Other related works will be discussed in Section 4.
Revised: Other related works will be discussed in Section 4.

Original: We use the term ‘device’ and ’agent’ interchangeably in this paper.
Revised: We use the term ‘device’ and ’agent’ interchangeably in this paper.

Original: The main difficulty of defending against adversarial attacks in our collaborative inference setting (as well as in VFL) is that there is no apparent redundancy across the agents.
Revised: The main difficulty of defending against adversarial attacks in our collaborative inference setting (as well as in VFL) is that there is no apparent redundancy across the agents.

Original: Though, intuitively, there is some mutual information among the features held by each agent.
Revised: Though, intuitively, there is some mutual information among the features held by each agent.

Original: Even worse, the raw data held by the agents are not accessible by the FC/server to inspect, the attacker can perturb the compromised agents’ embedded features with a very large magnitude before sending them to the
Revised: Even worse, the raw data held by the agents are not accessible by the FC/server to inspect, the attacker can perturb the compromised agents’ embedded features with a very large magnitude before sending them to the

Original: FC/server.
Revised: FC/server.

Original: In this paper, we propose to improve robustness based on the underlying redundancy among the features held by IoT devices.
Revised: In this paper, we propose to improve robustness based on the underlying redundancy among the features held by IoT devices.

Original: Specifically, we assume the combined features lie on an underlying lower dimensional manifold.
Revised: Specifically, we assume the combined features lie on an underlying lower dimensional manifold.

Original: We propose a robust COllaborative inference framework via feature
Revised: We propose a robust COllaborative inference framework via feature

Original: PURification (CoPur), which is able to defend against many types of attacks during inference (see the attack taxonomy in Section 2.2).
Revised: PURification (CoPur), which is able to defend against many types of attacks during inference (see the attack taxonomy in Section 2.2).

Original: During the inference, the FC receives the corrupted and/or incomplete embedded features from agents (Figure 1a).
Revised: During the inference, the FC receives the corrupted and/or incomplete embedded features from agents (Figure 1a).

Original: Then CoPur purifies the embedded features based on the learned feature subspace (Figure 1b).
Revised: Then CoPur purifies the embedded features based on the learned feature subspace (Figure 1b).

Original: Finally, it feeds the purified features to FC’s trained global model for prediction.
Revised: Finally, it feeds the purified features to FC’s trained global model for prediction.

Original: The detailed robust inference procedures will be described in
Revised: The detailed robust inference procedures will be described in

Original: Sections 2.3.
Revised: Sections 2.3.

Original: We aim to answer the following questions: Can we make robust predictions based on corrupted and/or incomplete features?
Revised: We aim to answer the following questions: Can we make robust predictions based on corrupted and/or incomplete features?

Original: If yes, how many compromised devices can we tolerate?
Revised: If yes, how many compromised devices can we tolerate?

Original: Technical Contributions : Our main contributions include the following:
Revised: Technical Contributions : Our main contributions include the following:

Original: • We propose a certifiably robust collaborative inference procedure to defend against a variety of targeted or untargeted attacks (e.g., distributed adversarial attacks, missing-feature attack, and their combinations) during collaborative inference.
Revised: • We propose a certifiably robust collaborative inference procedure to defend against a variety of targeted or untargeted attacks (e.g., distributed adversarial attacks, missing-feature attack, and their combinations) during collaborative inference.

Original: • We propose a novel non-linear robust decomposition method that decomposes the potentially corrupted and incomplete feature vector into two parts: one lies (exactly or approximately) on the underlying feature manifold; the other with a block-sparse structure.
Revised: • We propose a novel non-linear robust decomposition method that decomposes the potentially corrupted and incomplete feature vector into two parts: one lies (exactly or approximately) on the underlying feature manifold; the other with a block-sparse structure.

Original: Theoretically, we prove
Revised: Theoretically, we prove

Original: that the proposed method can recover the true feature vector exactly (or approximately ) despite corruptions and incomplete observations.
Revised: that the proposed method can recover the true feature vector exactly (or approximately ) despite corruptions and incomplete observations.

Original: This may be of independent interest.
Revised: This may be of independent interest.

Original: • We propose and test a distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents.
Revised: • We propose and test a distributed feature-flipping attack, which is agnostic to the model, training data, label, as well as features held by other agents.

Original: Empirical studies show that this new attack effectively attacks existing state-of-the-art defenses.
Revised: Empirical studies show that this new attack effectively attacks existing state-of-the-art defenses.

Original: • We conduct extensive experiments on ExtraSensory and NUS-WIDE datasets, and show that CoPur is significantly more robust than baselines against different types of attacks.
Revised: • We conduct extensive experiments on ExtraSensory and NUS-WIDE datasets, and show that CoPur is significantly more robust than baselines against different types of attacks.

Original:  Robust collaborative inference via feature purification (CoPur)
Revised:  Robust collaborative inference via feature purification (CoPur)

Original: We describe the system setting, the threat models, and the proposed robust inference procedures.
Revised: We describe the system setting, the threat models, and the proposed robust inference procedures.

Original: 2.1 Collaborative inference and system setting
Revised: 2.1 Collaborative inference and system setting

Original: We first describe the basic framework of collaborative inference, which is also similar to the inference phase of some VFL frameworks (Chen et al., 2020; Liu et al., 2020).
Revised: We first describe the basic framework of collaborative inference, which is also similar to the inference phase of some VFL frameworks (Chen et al., 2020; Liu et al., 2020).

Original: There are M agents, where each agent i holds partial raw data x testi of the overall testing sample x test =
Revised: There are M agents, where each agent i holds partial raw data x testi of the overall testing sample x test =

Original: [ x test; ... ; x testM ] .
Revised: [ x test; ... ; x testM ] .

Original: Also, each agent i has its own local feature extractor f i parameterized by θ i that maps its raw data vector x testi to the embedded feature vector h testi ≜
Revised: Also, each agent i has its own local feature extractor f i parameterized by θ i that maps its raw data vector x testi to the embedded feature vector h testi ≜

Original: f i ( x testi ; θ i ) .
Revised: f i ( x testi ; θ i ) .

Original: The dimension of h testi is usually smaller than x test i .
Revised: The dimension of h testi is usually smaller than x test i .

Original: The FC only receives the embedded features h test i , i = 1 , ..., M from the agents (not the raw data x testi ), and concatenates them into a long column vector h test ≜ [ h test; h test; ... ; h testM ] .
Revised: The FC only receives the embedded features h test i , i = 1 , ..., M from the agents (not the raw data x testi ), and concatenates them into a long column vector h test ≜ [ h test; h test; ... ; h testM ] .

Original: Then the FC uses its pre-trained global model f θ 0 to make the prediction.
Revised: Then the FC uses its pre-trained global model f θ 0 to make the prediction.

Original: While we do not focus on training, we will require an AutoEncoder, trained on uncorrupted data (e.g., during training) that can approximately capture the manifold of the uncorrupted embedded features, i.e. , (cid:80) Mi =1 ∥
Revised: While we do not focus on training, we will require an AutoEncoder, trained on uncorrupted data (e.g., during training) that can approximately capture the manifold of the uncorrupted embedded features, i.e. , (cid:80) Mi =1 ∥

Original: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the true embedded features l ∗ of the testing instance, where D ϕ is the decoder parameterized by ϕ , and E ψ is the encoder parameterized by ψ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the true embedded features l ∗ of the testing instance, where D ϕ is the decoder parameterized by ϕ , and E ψ is the encoder parameterized by ψ .

Original: Further, we assume that the global model for inference f θ 0 is based on the output of the AutoEncoder and the training labels, i.e. , {D ϕ ( E ψ ( h ( j ) )) , y ( j ) } nj =1 .
Revised: Further, we assume that the global model for inference f θ 0 is based on the output of the AutoEncoder and the training labels, i.e. , {D ϕ ( E ψ ( h ( j ) )) , y ( j ) } nj =1 .

Original: Detailed examples of how to train such a model and the AutoEncoder can be found in Section D of the supplemental material.
Revised: Detailed examples of how to train such a model and the AutoEncoder can be found in Section D of the supplemental material.

Original: 2.2 Threat model
Revised: 2.2 Threat model

Original: There are M agents which hold different parts of the feature of the same set of n training instances.
Revised: There are M agents which hold different parts of the feature of the same set of n training instances.

Original: We assume the training process is well-protected (attack-free).
Revised: We assume the training process is well-protected (attack-free).

Original: However, during the inference, the attacker can attack αM agents.
Revised: However, during the inference, the attacker can attack αM agents.

Original: The compromised agent either is not able to send features to the FC, or sends arbitrary embedded features to FC.
Revised: The compromised agent either is not able to send features to the FC, or sends arbitrary embedded features to FC.

Original: We use h benign to denote the overall embedded features provided by the (1 − α )
Revised: We use h benign to denote the overall embedded features provided by the (1 − α )

Original: M benign agents (these agents are indexed by Ω benign and | Ω benign | = (1 − α ) M ), and we use h adv to denote the overall embedded features sent by compromised agents (indexed by Ω adv ).
Revised: M benign agents (these agents are indexed by Ω benign and | Ω benign | = (1 − α ) M ), and we use h adv to denote the overall embedded features sent by compromised agents (indexed by Ω adv ).

Original: The set Ω = Ω benign ∪
Revised: The set Ω = Ω benign ∪

Original: Ω adv is the index set of agents that provide features, and its complement, i.e. , Ω c , denotes the compromised agents which do not provide features.
Revised: Ω adv is the index set of agents that provide features, and its complement, i.e. , Ω c , denotes the compromised agents which do not provide features.

Original: We categorize the inference phase attacks into the following three types:
Revised: We categorize the inference phase attacks into the following three types:

Original: Threat model A (distributed adversarial attack): All the compromised agents jointly send adversarial embedded features h adv , with the goal to mislead the prediction to the target label
Revised: Threat model A (distributed adversarial attack): All the compromised agents jointly send adversarial embedded features h adv , with the goal to mislead the prediction to the target label

Original: y A :
Revised: y A :

Original: [Equation]
Revised: [Equation]

Original: Threat model B (missing-feature attack): All the compromised agents do not send any feature to the FC.
Revised: Threat model B (missing-feature attack): All the compromised agents do not send any feature to the FC.

Original: A such missing-feature attack may be due to damaged sensors or disconnection, and can still affect FC’s prediction.
Revised: A such missing-feature attack may be due to damaged sensors or disconnection, and can still affect FC’s prediction.

Original: The presence of a missing-feature attack can be viewed as the FC observing only partial blocks (indexed by Ω ) of the overall embedded features h , i.e. , h Ω .
Revised: The presence of a missing-feature attack can be viewed as the FC observing only partial blocks (indexed by Ω ) of the overall embedded features h , i.e. , h Ω .

Original: Ω c denotes the index set of agents that perform missing-feature attacks.
Revised: Ω c denotes the index set of agents that perform missing-feature attacks.

Original: The attacker can compromise up to αM agents to perform such an attack: min Ω
Revised: The attacker can compromise up to αM agents to perform such an attack: min Ω

Original: [Equation]
Revised: [Equation]

Original: Threat model C (combined attack):
Revised: Threat model C (combined attack):

Original: The attacker can also perform a combined adversarial and missing feature attack by providing h adv , indexed by Ω adv , and missing features Ω c :
Revised: The attacker can also perform a combined adversarial and missing feature attack by providing h adv , indexed by Ω adv , and missing features Ω c :

Original: [Equation]
Revised: [Equation]

Original: Beyond these targeted attacks, the attacker can also perform untargeted attacks by maximizing the distance between the prediction and the ground truth label.
Revised: Beyond these targeted attacks, the attacker can also perform untargeted attacks by maximizing the distance between the prediction and the ground truth label.

Original: 2.3 Inference procedures of CoPur
Revised: 2.3 Inference procedures of CoPur

Original: We propose an all-in-one solution to defend against various inference phase attacks defined in
Revised: We propose an all-in-one solution to defend against various inference phase attacks defined in

Original: Section 2.2.
Revised: Section 2.2.

Original: Let Ω be the index set of the observed blocks.
Revised: Let Ω be the index set of the observed blocks.

Original: The observed h Ω can be decomposed as h Ω = l Ω + e Ω where l is the underlying uncorrupted features, and e Ω models the corruptions on the observed set Ω .
Revised: The observed h Ω can be decomposed as h Ω = l Ω + e Ω where l is the underlying uncorrupted features, and e Ω models the corruptions on the observed set Ω .

Original: Finding such a decomposition seems hard, even if the learned AutoEncoder can capture the underlying feature subspace, i.e. , l = D ϕ
Revised: Finding such a decomposition seems hard, even if the learned AutoEncoder can capture the underlying feature subspace, i.e. , l = D ϕ

Original: ( E ψ ( l )) .
Revised: ( E ψ ( l )) .

Original: Fortunately, as we mentioned in
Revised: Fortunately, as we mentioned in

Original: Section 4, e Ω has a block-sparse structure.
Revised: Section 4, e Ω has a block-sparse structure.

Original: We propose to solve the following:
Revised: We propose to solve the following:

Original: [Equation]
Revised: [Equation]

Original: where the indicator function 1 {·} counts whether the block [ h − l ] i is nonzero.
Revised: where the indicator function 1 {·} counts whether the block [ h − l ] i is nonzero.

Original: This block-sparsity minimization shares the same spirit as the ℓ 0 minimization in robust linear regression (Candes & Tao, 2005; Mitra et al., 2013).
Revised: This block-sparsity minimization shares the same spirit as the ℓ 0 minimization in robust linear regression (Candes & Tao, 2005; Mitra et al., 2013).

Original: However, it is NP-hard to solve, so we relax it to the ℓ 2 , 1 -norm:
Revised: However, it is NP-hard to solve, so we relax it to the ℓ 2 , 1 -norm:

Original: [Equation]
Revised: [Equation]

Original: Our theoretical analysis in Theorems 1 & 2 shows that under certain conditions, solving the above relaxed objective is able to recover the underlying true features exactly.
Revised: Our theoretical analysis in Theorems 1 & 2 shows that under certain conditions, solving the above relaxed objective is able to recover the underlying true features exactly.

Original: But what if the learned AutoEncoder can only approximately reconstruct the underlying features, say (cid:80)
Revised: But what if the learned AutoEncoder can only approximately reconstruct the underlying features, say (cid:80)

Original: [Equation]
Revised: [Equation]

Original: l
Revised: l

Original: [Equation]
Revised: [Equation]

Original: − l ] i ∥ 2 ≤ δ ?
Revised: − l ] i ∥ 2 ≤ δ ?

Original: In this case, we further relax the constraints in Eq.
Revised: In this case, we further relax the constraints in Eq.

Original: 5 and solve:
Revised: 5 and solve:

Original: [Equation]
Revised: [Equation]

Original: Our theoretical analysis in Theorems 3 & 4 show that under certain conditions, solving Eq.
Revised: Our theoretical analysis in Theorems 3 & 4 show that under certain conditions, solving Eq.

Original: 6 is able to approximately recover the underlying true features, despite the corruptions and/or incomplete observations.
Revised: 6 is able to approximately recover the underlying true features, despite the corruptions and/or incomplete observations.

Original: In our implementation, we first try to find a good initial point by simply searching the embedded features on the manifold, i.e., min l ′ (cid:80) i ∈
Revised: In our implementation, we first try to find a good initial point by simply searching the embedded features on the manifold, i.e., min l ′ (cid:80) i ∈

Original: Ω ∥
Revised: Ω ∥

Original: [ h − D ϕ
Revised: [ h − D ϕ

Original: ( E ψ ( l ′ ))] i ∥ 2 via gradient descend (It is useful to notice that since Eq. 6 allows l approximately lie on the manifold, its objective value will be slightly smaller than min l ′ (cid:80) i ∈
Revised: ( E ψ ( l ′ ))] i ∥ 2 via gradient descend (It is useful to notice that since Eq. 6 allows l approximately lie on the manifold, its objective value will be slightly smaller than min l ′ (cid:80) i ∈

Original: Ω ∥
Revised: Ω ∥

Original: [ h − D ϕ
Revised: [ h − D ϕ

Original: ( E ψ ( l ′ ))]
Revised: ( E ψ ( l ′ ))]

Original: i ∥ 2 ).
Revised: i ∥ 2 ).

Original: Then we apply Lagrange Multiplier method to solve Eq.
Revised: Then we apply Lagrange Multiplier method to solve Eq.

Original: 6, i.e., ˆ l = arg min l (cid:80) i ∈ Ω ∥
Revised: 6, i.e., ˆ l = arg min l (cid:80) i ∈ Ω ∥

Original: [ h − l ] i ∥ 2 + τ (cid:80) Mi
Revised: [ h − l ] i ∥ 2 + τ (cid:80) Mi

Original: =1 ∥ [ D ϕ ( E ψ ( l )) − l ] i ∥ 2 .
Revised: =1 ∥ [ D ϕ ( E ψ ( l )) − l ] i ∥ 2 .

Original: More implementation details can be found in the supplemental.
Revised: More implementation details can be found in the supplemental.

Original: After obtaining ˆ l , we feed D ϕ ( E ψ (ˆ l )) to FC’s trained global model for the prediction.
Revised: After obtaining ˆ l , we feed D ϕ ( E ψ (ˆ l )) to FC’s trained global model for the prediction.

Original:  Theoretical analysis of CoPur
Revised:  Theoretical analysis of CoPur

Original: We first analyze the ‘noiseless case’ where the learned AutoEncoder satisfies D ϕ
Revised: We first analyze the ‘noiseless case’ where the learned AutoEncoder satisfies D ϕ

Original: ( E ψ ( l ∗ )) =
Revised: ( E ψ ( l ∗ )) =

Original: l ∗ for the underlying uncorrupted feature vector l ∗ .
Revised: l ∗ for the underlying uncorrupted feature vector l ∗ .

Original: Next, we analyze the more challenging ‘noisy case’ where the AutoEncoder can only approximately reconstruct l ∗ , i.e. , (cid:80) M i =1 ∥
Revised: Next, we analyze the more challenging ‘noisy case’ where the AutoEncoder can only approximately reconstruct l ∗ , i.e. , (cid:80) M i =1 ∥

Original: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .

Original: Proofs and additional discussions can be found in Section A of the supplemental material.
Revised: Proofs and additional discussions can be found in Section A of the supplemental material.

Original: 3.1 Noiseless Case
Revised: 3.1 Noiseless Case

Original: To simplify the exposition, we first consider threat model B, i.e. , the missing-feature attack.
Revised: To simplify the exposition, we first consider threat model B, i.e. , the missing-feature attack.

Original: Theorem 1.
Revised: Theorem 1.

Original: (Exact feature recovery under threat model B) Assume the trained AutoEncoder satisfies D ϕ
Revised: (Exact feature recovery under threat model B) Assume the trained AutoEncoder satisfies D ϕ

Original: ( E ψ ( l ∗ )) =
Revised: ( E ψ ( l ∗ )) =

Original: l ∗ for the underlying uncorrupted feature vector l ∗ .
Revised: l ∗ for the underlying uncorrupted feature vector l ∗ .

Original: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , if ( r ′ − r ′′ )
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , if ( r ′ − r ′′ )

Original: Ω ̸ = 0 , then given h Ω =
Revised: Ω ̸ = 0 , then given h Ω =

Original: l ∗ Ω , l ∗ is the unique solution of Eq.
Revised: l ∗ Ω , l ∗ is the unique solution of Eq.

Original: 5 and Eq. 4.
Revised: 5 and Eq. 4.

Original: The proof can be found in Section A.3 of the supplemental material, where we show that any feasible point that is different from l ∗ would have a larger objective value.
Revised: The proof can be found in Section A.3 of the supplemental material, where we show that any feasible point that is different from l ∗ would have a larger objective value.

Original: The condition ( r ′ − r ′′ ) Ω ̸ = 0 required by Theorem 1 is intuitive and necessary.
Revised: The condition ( r ′ − r ′′ ) Ω ̸ = 0 required by Theorem 1 is intuitive and necessary.

Original: Suppose there are r ′ ̸ = r ′′ but r ′ Ω = r ′′ Ω , then observing r
Revised: Suppose there are r ′ ̸ = r ′′ but r ′ Ω = r ′′ Ω , then observing r

Original: Ω is not able to tell whether it’s from r ′ or r ′′ .
Revised: Ω is not able to tell whether it’s from r ′ or r ′′ .

Original: In general, when the observed set Ω gets larger, the required condition has a better chance of being satisfied.
Revised: In general, when the observed set Ω gets larger, the required condition has a better chance of being satisfied.

Original: If we have complete observations, i.e. , Ω = { 1 , 2 , ..., M } , the required condition ( r ′ − r ′′ ) Ω ̸ =automatically holds .
Revised: If we have complete observations, i.e. , Ω = { 1 , 2 , ..., M } , the required condition ( r ′ − r ′′ ) Ω ̸ =automatically holds .

Original: Now we consider the more challenging case where we not only have missing features but also have corruptions on the observed features.
Revised: Now we consider the more challenging case where we not only have missing features but also have corruptions on the observed features.

Original: Theorem 2.
Revised: Theorem 2.

Original: (Exact feature recovery under threat model C) Assume the trained AutoEncoder satisfies D ϕ
Revised: (Exact feature recovery under threat model C) Assume the trained AutoEncoder satisfies D ϕ

Original: ( E ψ ( l ∗ )) =
Revised: ( E ψ ( l ∗ )) =

Original: l ∗ for the underlying uncorrupted feature vector l ∗ .
Revised: l ∗ for the underlying uncorrupted feature vector l ∗ .

Original: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with | ¯ S Ω | = q < | Ω | / 2 , it holds that (cid:80) i ∈ S Ω
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with | ¯ S Ω | = q < | Ω | / 2 , it holds that (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 , then for any h Ω =
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 , then for any h Ω =

Original: l ∗ Ω + e ∗ Ω with (cid:80) i ∈
Revised: l ∗ Ω + e ∗ Ω with (cid:80) i ∈

Original: Ω 1 { e ∗ i ̸ = 0 } ≤ q , l ∗ is the unique solution of Eq.
Revised: Ω 1 { e ∗ i ̸ = 0 } ≤ q , l ∗ is the unique solution of Eq.

Original: 5 and Eq. 4.
Revised: 5 and Eq. 4.

Original: The proof can be found in Section A.4 of the supplemental material.
Revised: The proof can be found in Section A.4 of the supplemental material.

Original: We first show that any feasible point of Eq.
Revised: We first show that any feasible point of Eq.

Original: 5 that is different from l ∗ would have a larger objective value.
Revised: 5 that is different from l ∗ would have a larger objective value.

Original: Then we use contradiction to show that there does not exist any global optimal solution of Eq. 4 that is different from l ∗ .
Revised: Then we use contradiction to show that there does not exist any global optimal solution of Eq. 4 that is different from l ∗ .

Original: Remark 1.
Revised: Remark 1.

Original: Theorem 2 not only requires v Ω ̸ = 0 like Theorem 1, but also requires (cid:80)
Revised: Theorem 2 not only requires v Ω ̸ = 0 like Theorem 1, but also requires (cid:80)

Original: i ∈
Revised: i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 .
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 .

Original: This is due to the presence of corruptions in the observed blocks (indexed by ¯ S Ω ).
Revised: This is due to the presence of corruptions in the observed blocks (indexed by ¯ S Ω ).

Original: When there is no corruption, i.e. , q = 0 , the corresponding set ¯ S Ω is empty and the required condition reduces to (cid:80) i ∈
Revised: When there is no corruption, i.e. , q = 0 , the corresponding set ¯ S Ω is empty and the required condition reduces to (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 > 0 , i.e. , v Ω ̸ = 0 .
Revised: ∥ v i ∥ 2 > 0 , i.e. , v Ω ̸ = 0 .

Original: Second, Theorem 2 is a universal certification, where any subset ¯ S Ω of the observations can be malicious, as long as | ¯ S Ω | = q .
Revised: Second, Theorem 2 is a universal certification, where any subset ¯ S Ω of the observations can be malicious, as long as | ¯ S Ω | = q .

Original: In practice, once the attacker chooses q agents to perform adversarial attacks, the set ¯ S Ω gets fixed and corresponds to e ∗ i ̸ = 0 .
Revised: In practice, once the attacker chooses q agents to perform adversarial attacks, the set ¯ S Ω gets fixed and corresponds to e ∗ i ̸ = 0 .

Original: Then, we only require (cid:80) i ∈
Revised: Then, we only require (cid:80) i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 to hold for this particular fixed set ¯ S Ω , to guarantee l ∗ is the unique solution of Eq.
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 to hold for this particular fixed set ¯ S Ω , to guarantee l ∗ is the unique solution of Eq.

Original: This can be easily seen from the proof of Theorem 2.
Revised: This can be easily seen from the proof of Theorem 2.

Original: Third, the condition (cid:80) i ∈ S Ω
Revised: Third, the condition (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 prefers v Ω to be not ‘spiky’.
Revised: ∥ v i ∥ 2 > (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 prefers v Ω to be not ‘spiky’.

Original: Suppose r ′ and r ′′ are the same on all the observed blocks (indexed by Ω ) except one, which means only one block of v Ω is non-zero ( i.e. , v Ω is spiky).
Revised: Suppose r ′ and r ′′ are the same on all the observed blocks (indexed by Ω ) except one, which means only one block of v Ω is non-zero ( i.e. , v Ω is spiky).

Original: If that non-zero block is in the set ¯ S Ω , the required condition will not be met.
Revised: If that non-zero block is in the set ¯ S Ω , the required condition will not be met.

Original: Intuitively, if there are two instances differing in only one block of the observed features, and the attacker corrupts that block, the FC can not recover the original feature.
Revised: Intuitively, if there are two instances differing in only one block of the observed features, and the attacker corrupts that block, the FC can not recover the original feature.

Original: In general, we prefer v Ω to be not ‘spiky’, thus the condition can hold when the number of corrupted blocks | ¯ S Ω | is sufficiently small.
Revised: In general, we prefer v Ω to be not ‘spiky’, thus the condition can hold when the number of corrupted blocks | ¯ S Ω | is sufficiently small.

Original: Such condition can be considered as the non-linear and missing-block extensions of the Range Space Property in robust linear regression literature (Flores, 2015;
Revised: Such condition can be considered as the non-linear and missing-block extensions of the Range Space Property in robust linear regression literature (Flores, 2015;

Original: Liu et al., 2018), which we will discuss next.
Revised: Liu et al., 2018), which we will discuss next.

Original: Finally, the fraction ( q/M ) of corrupted blocks that CoPur can tolerate not only depends on Ω , but also depends on the underlying feature subspace.
Revised: Finally, the fraction ( q/M ) of corrupted blocks that CoPur can tolerate not only depends on Ω , but also depends on the underlying feature subspace.

Original: It can tolerate up to 50% ×| Ω | corrupted blocks if there is enough redundancy among the agents, e.g., if every agent holds exactly the same embedded features, then every block of v = r ′ − r ′′ is the same, and q approaches 50% ×| Ω | .
Revised: It can tolerate up to 50% ×| Ω | corrupted blocks if there is enough redundancy among the agents, e.g., if every agent holds exactly the same embedded features, then every block of v = r ′ − r ′′ is the same, and q approaches 50% ×| Ω | .

Original: Remark 2.
Revised: Remark 2.

Original: To draw the connections to the Range Space Property in robust linear regression, let’s consider a typical case where there is no non-linear activation function in the last layer of the
Revised: To draw the connections to the Range Space Property in robust linear regression, let’s consider a typical case where there is no non-linear activation function in the last layer of the

Original: AutoEncoder (other layers can have non-linear activation functions).
Revised: AutoEncoder (other layers can have non-linear activation functions).

Original: Let W end be the weight matrix of the last layer of the AutoEncoder.
Revised: Let W end be the weight matrix of the last layer of the AutoEncoder.

Original: Note that the difference between any two vectors from the range of the decoder D ϕ ( · ) is contained in the Range ( W end ) , i.e. , v ∈ Range ( W end ) \ 0 .
Revised: Note that the difference between any two vectors from the range of the decoder D ϕ ( · ) is contained in the Range ( W end ) , i.e. , v ∈ Range ( W end ) \ 0 .

Original: Similar to the so called ‘leverage constant’ in robust linear regression (Flores, 2015; Liu et al., 2018), we can define the block-wise ‘incomplete’ leverage constant
Revised: Similar to the so called ‘leverage constant’ in robust linear regression (Flores, 2015; Liu et al., 2018), we can define the block-wise ‘incomplete’ leverage constant

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which is between 0 and 1, and is monotonic increasing when the number of corrupted blocks q decreases.
Revised: which is between 0 and 1, and is monotonic increasing when the number of corrupted blocks q decreases.

Original: A sufficient condition for (cid:80) i ∈ S Ω
Revised: A sufficient condition for (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 0 to hold is c Ω q ( W end ) > 0 .
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 0 to hold is c Ω q ( W end ) > 0 .

Original: Lastly, it is very useful to note that Sharon et al.
Revised: Lastly, it is very useful to note that Sharon et al.

Original: (2009) designed an algorithm to calculate the leverage constant, which is possible to be extended to our block-wise incomplete leverage constant here.
Revised: (2009) designed an algorithm to calculate the leverage constant, which is possible to be extended to our block-wise incomplete leverage constant here.

Original: Consider the toy example again where every agent holds exactly the same embedded features, and q = 0 .
Revised: Consider the toy example again where every agent holds exactly the same embedded features, and q = 0 .

Original: 1 × | Ω | , then c Ω q ( W end ) = 0 .
Revised: 1 × | Ω | , then c Ω q ( W end ) = 0 .

Original: Remark 3.
Revised: Remark 3.

Original: It is interesting to note that in the recent Generative model based compressive sensing work (Bora et al., 2017), they also impose conditions on the difference between any two vectors in the range of the decoder, though without missing blocks.
Revised: It is interesting to note that in the recent Generative model based compressive sensing work (Bora et al., 2017), they also impose conditions on the difference between any two vectors in the range of the decoder, though without missing blocks.

Original: But the conditions therein are quite different from ours due to the different purposes.
Revised: But the conditions therein are quite different from ours due to the different purposes.

Original: 3.2 Noisy case
Revised: 3.2 Noisy case

Original: Now we analyze the ‘noisy case’ where the learned AutoEncoder can only approximately reconstruct the underlying uncorrupted feature vector l ∗ , i.e. , (cid:80) Mi
Revised: Now we analyze the ‘noisy case’ where the learned AutoEncoder can only approximately reconstruct the underlying uncorrupted feature vector l ∗ , i.e. , (cid:80) Mi

Original: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .
Revised: =1 ∥ [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ .

Original: Again, to help the reader understand step-by-step, our analyses start from the threat model B.
Revised: Again, to help the reader understand step-by-step, our analyses start from the threat model B.

Original: Theorem 3.
Revised: Theorem 3.

Original: (Stable feature recovery under threat model B) Assume the trained AutoEncoder satisfies (cid:80) Mi =1 ∥
Revised: (Stable feature recovery under threat model B) Assume the trained AutoEncoder satisfies (cid:80) Mi =1 ∥

Original: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .

Original: Let ˆ l be the solution of Eq. 6.
Revised: Let ˆ l be the solution of Eq. 6.

Original: Given the incomplete observations h Ω =
Revised: Given the incomplete observations h Ω =

Original: l ∗ Ω , where Ω is the index set of observed agent blocks.
Revised: l ∗ Ω , where Ω is the index set of observed agent blocks.

Original: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) , define v = r ′ − r ′′ , if it holds that (cid:80) i ∈
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) , define v = r ′ − r ′′ , if it holds that (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ ( E ψ (ˆ l )) − D ϕ
Revised: ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ ( E ψ (ˆ l )) − D ϕ

Original: ( E ψ ( l ∗ ))
Revised: ( E ψ ( l ∗ ))

Original: ∥ 2 ≤ ∆ .
Revised: ∥ 2 ≤ ∆ .

Original: The proof can be found in Section A.5 of the supplemental material.
Revised: The proof can be found in Section A.5 of the supplemental material.

Original: Remark 4.
Revised: Remark 4.

Original: First, note that (cid:80) Mi
Revised: First, note that (cid:80) Mi

Original: =1 ∥ v i ∥ 2 ≥ (cid:113)(cid:80)
Revised: =1 ∥ v i ∥ 2 ≥ (cid:113)(cid:80)

Original: Mi
Revised: Mi

Original: =1 ∥ v i ∥ 22 = ∥ v ∥ 2 .
Revised: =1 ∥ v i ∥ 22 = ∥ v ∥ 2 .

Original: When ∥ v ∥ 2 > ∆ , we have (cid:80) Mi
Revised: When ∥ v ∥ 2 > ∆ , we have (cid:80) Mi

Original: =1 ∥ v i ∥ 2 > ∆ .
Revised: =1 ∥ v i ∥ 2 > ∆ .

Original: Again, we prefer v to be not ‘spiky’, thus the condition (cid:80) i ∈
Revised: Again, we prefer v to be not ‘spiky’, thus the condition (cid:80) i ∈

Original: Ω ∥ v i ∥ 2 > 2 δ can be satisfied as long as | Ω | is large enough (assume ∆ > 2 δ ).
Revised: Ω ∥ v i ∥ 2 > 2 δ can be satisfied as long as | Ω | is large enough (assume ∆ > 2 δ ).

Original: Now we consider the most challenging case where we not only have missing features but also have corruptions in the observed features.
Revised: Now we consider the most challenging case where we not only have missing features but also have corruptions in the observed features.

Original: Theorem 4.
Revised: Theorem 4.

Original: (Stable feature recovery under threat model C) Assume the trained AutoEncoder satisfies (cid:80) Mi =1 ∥
Revised: (Stable feature recovery under threat model C) Assume the trained AutoEncoder satisfies (cid:80) Mi =1 ∥

Original: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .
Revised: [ D ϕ ( E ψ ( l ∗ )) − l ∗ ] i ∥ 2 ≤ δ for the underlying uncorrupted feature vector l ∗ .

Original: Let ˆ l be the solution of Eq. 6.
Revised: Let ˆ l be the solution of Eq. 6.

Original: Given the incomplete observations h Ω =
Revised: Given the incomplete observations h Ω =

Original: l ∗ Ω + e ∗ Ω , where Ω is the index set of observed agent blocks.
Revised: l ∗ Ω + e ∗ Ω , where Ω is the index set of observed agent blocks.

Original: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with | ¯ S Ω | = q , it holds that (cid:80) i ∈ S Ω
Revised: For ∀ r ′ , r ′′ in the range of the decoder D ϕ ( · ) where r ′ ̸ = r ′′ , define v = r ′ − r ′′ , if for any partition { S Ω , ¯ S Ω } of Ω with | ¯ S Ω | = q , it holds that (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ for ∀∥ v ∥ 2 > ∆ , then ∥D ϕ

Original: ( E ψ (ˆ l ))
Revised: ( E ψ (ˆ l ))

Original: − D ϕ
Revised: − D ϕ

Original: ( E ψ ( l ∗ ))
Revised: ( E ψ ( l ∗ ))

Original: ∥ 2 ≤ ∆ as long as (cid:80) i ∈
Revised: ∥ 2 ≤ ∆ as long as (cid:80) i ∈

Original: Ω 1 { e ∗ i ̸ = 0 } ≤ q .
Revised: Ω 1 { e ∗ i ̸ = 0 } ≤ q .

Original: The proof can be found in Section A.6 of the supplemental material.
Revised: The proof can be found in Section A.6 of the supplemental material.

Original: Robust prediction.
Revised: Robust prediction.

Original: Recall that the server has learned a classifier f θ 0 during the training.
Revised: Recall that the server has learned a classifier f θ 0 during the training.

Original: We can further smooth this classifier f θ 0 to get f s (e.g., via Randomized Smoothing (Cohen et al., 2019;
Revised: We can further smooth this classifier f θ 0 to get f s (e.g., via Randomized Smoothing (Cohen et al., 2019;

Original: Lecuyer et al., 2019; Li et al., 2018))
Revised: Lecuyer et al., 2019; Li et al., 2018))

Original: such that f s ( D ϕ ( E ψ (ˆ l )))
Revised: such that f s ( D ϕ ( E ψ (ˆ l )))

Original: =
Revised: =

Original: f s ( D ϕ
Revised: f s ( D ϕ

Original: ( E ψ (ˆ l ))
Revised: ( E ψ (ˆ l ))

Original: + v ) for ∀∥ v ∥ 2 ≤ ∆ .
Revised: + v ) for ∀∥ v ∥ 2 ≤ ∆ .

Original: As we have ∥D ϕ ( E ψ (ˆ l ))
Revised: As we have ∥D ϕ ( E ψ (ˆ l ))

Original: − D ϕ
Revised: − D ϕ

Original: ( E ψ ( l ∗ ))
Revised: ( E ψ ( l ∗ ))

Original: ∥ 2 ≤ ∆ , then it is guaranteed that f s ( D ϕ ( E ψ (ˆ l )))
Revised: ∥ 2 ≤ ∆ , then it is guaranteed that f s ( D ϕ ( E ψ (ˆ l )))

Original: =
Revised: =

Original: f s ( D ϕ
Revised: f s ( D ϕ

Original: ( E ψ ( l ∗ ))) .
Revised: ( E ψ ( l ∗ ))) .

Original: Remark 5.
Revised: Remark 5.

Original: The condition (cid:80) i ∈
Revised: The condition (cid:80) i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ required by Theorem 4 is stronger than the condition (cid:80) i ∈
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ required by Theorem 4 is stronger than the condition (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 > 2 δ required by Theorem 3.
Revised: ∥ v i ∥ 2 > 2 δ required by Theorem 3.

Original: This is due to the presence of corruption in the observed blocks.
Revised: This is due to the presence of corruption in the observed blocks.

Original: As discussed in Remark 4, when ∥ v ∥ 2 > ∆ , we have (cid:80) Mi
Revised: As discussed in Remark 4, when ∥ v ∥ 2 > ∆ , we have (cid:80) Mi

Original: =1 ∥ v i ∥ 2 > ∆ .
Revised: =1 ∥ v i ∥ 2 > ∆ .

Original: Again, we prefer v to be not ‘spiky’, thus (cid:80) i ∈ Ω ∥ v i ∥ 2 can have a significant portion of mass from (cid:80) Mi =1 ∥ v i ∥ 2 when the number of observed blocks | Ω | is large.
Revised: Again, we prefer v to be not ‘spiky’, thus (cid:80) i ∈ Ω ∥ v i ∥ 2 can have a significant portion of mass from (cid:80) Mi =1 ∥ v i ∥ 2 when the number of observed blocks | Ω | is large.

Original: Further, when the number of corrupted blocks q = | ¯ S Ω | is small enough, we can have (cid:80) i ∈
Revised: Further, when the number of corrupted blocks q = | ¯ S Ω | is small enough, we can have (cid:80) i ∈

Original: S Ω
Revised: S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ .
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ .

Original: To draw the connections to the Range Space Property in robust linear regression, we continue on the discussions in Remark 2.
Revised: To draw the connections to the Range Space Property in robust linear regression, we continue on the discussions in Remark 2.

Original: Let’s say (cid:80) i ∈
Revised: Let’s say (cid:80) i ∈

Original: Ω
Revised: Ω

Original: ∥ v i ∥ 2 = ∆ ′ , then a sufficient condition for (cid:80) i ∈ S Ω
Revised: ∥ v i ∥ 2 = ∆ ′ , then a sufficient condition for (cid:80) i ∈ S Ω

Original: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ to hold is [ c Ω q ( W end ) − (1 − c Ω q ( W end ))] > 2 δ/ ∆ ′ .
Revised: ∥ v i ∥ 2 − (cid:80) i ∈ ¯ S Ω ∥ v i ∥ 2 > 2 δ to hold is [ c Ω q ( W end ) − (1 − c Ω q ( W end ))] > 2 δ/ ∆ ′ .

Original: Intuitively, the smaller number q of corrupted blocks and smaller δ ( i.e. , better recovered feature subspace) as well as larger observed set Ω (then ∆ ′ approaches ∆ ), can lead to smaller recovery error bound ∆ .
Revised: Intuitively, the smaller number q of corrupted blocks and smaller δ ( i.e. , better recovered feature subspace) as well as larger observed set Ω (then ∆ ′ approaches ∆ ), can lead to smaller recovery error bound ∆ .

Original:  Related work and discussion
Revised:  Related work and discussion

Original: Handling missing data and outliers in IoT. Sanyal & Zhang (2018) model the underlying uncorrupted IoT data as a low-dimensional linear subspace, i.e. , low-rank, and try to clean the corrupted sensor data by projecting them onto the estimated linear subspace.
Revised: Handling missing data and outliers in IoT. Sanyal & Zhang (2018) model the underlying uncorrupted IoT data as a low-dimensional linear subspace, i.e. , low-rank, and try to clean the corrupted sensor data by projecting them onto the estimated linear subspace.

Original: There is no guarantee of recovering the underlying uncorrupted sensor data.
Revised: There is no guarantee of recovering the underlying uncorrupted sensor data.

Original: This approach is a linear version of the manifold projection method that we will discuss in detail.
Revised: This approach is a linear version of the manifold projection method that we will discuss in detail.

Original: Kekatos & Giannakis (2011) studied the robust sensing problem where each sensor i has a linear measurement of the unknown parameter x , i.e. , b i
Revised: Kekatos & Giannakis (2011) studied the robust sensing problem where each sensor i has a linear measurement of the unknown parameter x , i.e. , b i

Original: =
Revised: =

Original: A i x , where both b i and A i are known and some sensors are additionally corrupted by large measurement errors, but without any missing values.
Revised: A i x , where both b i and A i are known and some sensors are additionally corrupted by large measurement errors, but without any missing values.

Original: They showed that it is possible to recover the true parameter x despite outlier corruptions.
Revised: They showed that it is possible to recover the true parameter x despite outlier corruptions.

Original: They also consider the scenario with dense inlier noise in the measurements, i.e. , where the measurements b i only approximately lie on the range space of matrix
Revised: They also consider the scenario with dense inlier noise in the measurements, i.e. , where the measurements b i only approximately lie on the range space of matrix

Original: A i , but their corresponding method does not have any recovery guarantee in such a noisy case.
Revised: A i , but their corresponding method does not have any recovery guarantee in such a noisy case.

Original: There are also some works that study the scenario where the sensors observe different noisy versions of the same data vector, and they show it is possible to find a set of unattacked sensors based on the assumption that the fraction of attacked sensors is small, e.g., (Zhang et al., 2018) or utilizing additional knowledge that some sensors are not attacked (Wilson & Veeravalli, 2016).
Revised: There are also some works that study the scenario where the sensors observe different noisy versions of the same data vector, and they show it is possible to find a set of unattacked sensors based on the assumption that the fraction of attacked sensors is small, e.g., (Zhang et al., 2018) or utilizing additional knowledge that some sensors are not attacked (Wilson & Veeravalli, 2016).

Original: We refer interested readers to the survey paper (Zhang et al., 2018) for additional details on this line of work.
Revised: We refer interested readers to the survey paper (Zhang et al., 2018) for additional details on this line of work.

Original: Certified robustness for defending against adversarial attacks.
Revised: Certified robustness for defending against adversarial attacks.

Original: In the adversarial machine learning literature, many defense methods have been proposed to deal with adversarial perturbations.
Revised: In the adversarial machine learning literature, many defense methods have been proposed to deal with adversarial perturbations.

Original: We refer the reader to Chakraborty et al.
Revised: We refer the reader to Chakraborty et al.

Original: (2018) for a comprehensive survey.
Revised: (2018) for a comprehensive survey.

Original: However, most of the existing defenses can only deal with small adversarial perturbations.
Revised: However, most of the existing defenses can only deal with small adversarial perturbations.

Original: Even worse, as new defenses are proposed, corresponding new attacks are also developed to adaptively attack them.
Revised: Even worse, as new defenses are proposed, corresponding new attacks are also developed to adaptively attack them.

Original: Experimental demonstrations of a defense’s efficacy based on currently existing attacks do not provide general proof of security.
Revised: Experimental demonstrations of a defense’s efficacy based on currently existing attacks do not provide general proof of security.

Original: Therefore, certifiably robust defenses are of great interest.
Revised: Therefore, certifiably robust defenses are of great interest.

Original: Randomized Smoothing (RS) is a promising technique (Cohen et al., 2019; Lecuyer et al., 2019; Li et al., 2018) which smoothes a base classifier to get a robust classifier, such that the prediction of this robust classifier on adversarially perturbed input is the same as its prediction on the original input when the ℓ p -norm of the adversarial perturbation is sufficiently bounded.
Revised: Randomized Smoothing (RS) is a promising technique (Cohen et al., 2019; Lecuyer et al., 2019; Li et al., 2018) which smoothes a base classifier to get a robust classifier, such that the prediction of this robust classifier on adversarially perturbed input is the same as its prediction on the original input when the ℓ p -norm of the adversarial perturbation is sufficiently bounded.

Original: Unfortunately, in our setting, the adversarial perturbation on compromised devices’ embedded features can be very large.
Revised: Unfortunately, in our setting, the adversarial perturbation on compromised devices’ embedded features can be very large.

Original: Further, RS does not have robustness guarantees when there are missing features.
Revised: Further, RS does not have robustness guarantees when there are missing features.

Original: Another promising defense is Randomized Ablation (Levine & Feizi, 2020), which was proposed to defend against sparse adversarial attacks where a small number of pixels in an image are changed to some arbitrary values in [0,1].
Revised: Another promising defense is Randomized Ablation (Levine & Feizi, 2020), which was proposed to defend against sparse adversarial attacks where a small number of pixels in an image are changed to some arbitrary values in [0,1].

Original: We extend this to the collaborative inference setting for comparison: the FC learns to make predictions based on randomly selected subsets of agent features during the training; and in the inference phase, FC randomly selects a subset of agents to make a prediction.
Revised: We extend this to the collaborative inference setting for comparison: the FC learns to make predictions based on randomly selected subsets of agent features during the training; and in the inference phase, FC randomly selects a subset of agents to make a prediction.

Original: The final prediction is by majority voting over the random trials.
Revised: The final prediction is by majority voting over the random trials.

Original: However, it is hard to design a base classifier that is able to train on randomly ablated input features 1 .
Revised: However, it is hard to design a base classifier that is able to train on randomly ablated input features 1 .

Original: Another limitation of this method is that its robust prediction (based on randomly ablated features) may not be the same as the attack-free vanilla setting ( i.e. , the prediction of the vanilla classifier on the uncorrupted data).
Revised: Another limitation of this method is that its robust prediction (based on randomly ablated features) may not be the same as the attack-free vanilla setting ( i.e. , the prediction of the vanilla classifier on the uncorrupted data).

Original: Further, to choose how many pixels to ablate, one needs knowledge of the number of corrupted pixels.
Revised: Further, to choose how many pixels to ablate, one needs knowledge of the number of corrupted pixels.

Original: Manifold Projection vs. Feature Purification.
Revised: Manifold Projection vs. Feature Purification.

Original: In the centralized setting, several works (Meng &
Revised: In the centralized setting, several works (Meng &

Original: Chen, 2017; Ilyas et al., 2017; Lindqvist et al., 2018) proposed to project the adversarial perturbed image onto the manifold of the normal data to defend against adversarial attacks.
Revised: Chen, 2017; Ilyas et al., 2017; Lindqvist et al., 2018) proposed to project the adversarial perturbed image onto the manifold of the normal data to defend against adversarial attacks.

Original: A natural question is: why do we propose a feature purification method, instead of using manifold projection?
Revised: A natural question is: why do we propose a feature purification method, instead of using manifold projection?

Original: First, the manifold projection approach can not guarantee the recovery of the underlying features.
Revised: First, the manifold projection approach can not guarantee the recovery of the underlying features.

Original: In fact, the resulting projected features can be arbitrarily far from the underlying true features if the adversarial perturbation can be arbitrarily large.
Revised: In fact, the resulting projected features can be arbitrarily far from the underlying true features if the adversarial perturbation can be arbitrarily large.

Original: To make this concrete, consider the linear manifold, i.e. , subspace.
Revised: To make this concrete, consider the linear manifold, i.e. , subspace.

Original: Suppose the underlying true feature vector l is generated from the column space of a d by r tall matrix U .
Revised: Suppose the underlying true feature vector l is generated from the column space of a d by r tall matrix U .

Original: We observe the corrupted feature vector h , which can be decomposed as l + e , where e models the adversarial perturbations.
Revised: We observe the corrupted feature vector h , which can be decomposed as l + e , where e models the adversarial perturbations.

Original: Projecting h onto the column space of U will get P U
Revised: Projecting h onto the column space of U will get P U

Original: h
Revised: h

Original: = P U l + P U e =
Revised: = P U l + P U e =

Original: l + P U e ,
Revised: l + P U e ,

Original: where P U =
Revised: where P U =

Original: U ( U T U ) − 1 U T .
Revised: U ( U T U ) − 1 U T .

Original: So the recovery error is P U h − l
Revised: So the recovery error is P U h − l

Original: = P U e .
Revised: = P U e .

Original: Apparently, if the adversarial perturbation vector e is not orthogonal to this subspace (which is often the case), the recovery error would be proportional to the magnitude of the perturbation!
Revised: Apparently, if the adversarial perturbation vector e is not orthogonal to this subspace (which is often the case), the recovery error would be proportional to the magnitude of the perturbation!

Original: It’s hard to recover the underlying true feature even if we have a perfect subspace, and the problem looks even more challenging when the underlying manifold is non-linear.
Revised: It’s hard to recover the underlying true feature even if we have a perfect subspace, and the problem looks even more challenging when the underlying manifold is non-linear.

Original: Fortunately, in our collaborative inference setting, there is important prior information about the pattern of the corruptions that we can leverage: the perturbation e has a block-wise structure (each block corresponds to an agent), and the fraction of the perturbed blocks is small.
Revised: Fortunately, in our collaborative inference setting, there is important prior information about the pattern of the corruptions that we can leverage: the perturbation e has a block-wise structure (each block corresponds to an agent), and the fraction of the perturbed blocks is small.

Original: Therefore, e has a block-sparse structure.
Revised: Therefore, e has a block-sparse structure.

Original: By exploring this fact, our proposed feature purification method can provably recover the underlying feature vector exactly or with bounded error, even if the underlying manifold is highly non-linear and the magnitude of the perturbation is arbitrarily large.
Revised: By exploring this fact, our proposed feature purification method can provably recover the underlying feature vector exactly or with bounded error, even if the underlying manifold is highly non-linear and the magnitude of the perturbation is arbitrarily large.

Original: Purifier Network vs. Feature Purification.
Revised: Purifier Network vs. Feature Purification.

Original: Besides the Manifold-Projection based approaches, one recent work (Naseer et al., 2020) tries to train a purifier neural network that maps any corrupted feature h to the underlying true feature l , in the centralized setting.
Revised: Besides the Manifold-Projection based approaches, one recent work (Naseer et al., 2020) tries to train a purifier neural network that maps any corrupted feature h to the underlying true feature l , in the centralized setting.

Original: However, that method needs to generate the adversarial training examples based on the assumption that the magnitude of the adversarial perturbation e is bounded by small ϵ (see Naseer et al.
Revised: However, that method needs to generate the adversarial training examples based on the assumption that the magnitude of the adversarial perturbation e is bounded by small ϵ (see Naseer et al.

Original: (2020, Eq.4)), which is different from our collaborative inference setting where the adversarial perturbation is unbounded.
Revised: (2020, Eq.4)), which is different from our collaborative inference setting where the adversarial perturbation is unbounded.

Original: Further, in our collaborative inference setting, arbitrary α fraction of the blocks of feature l can be perturbed by arbitrarily large values.
Revised: Further, in our collaborative inference setting, arbitrary α fraction of the blocks of feature l can be perturbed by arbitrarily large values.

Original: Based on the high (combinatorial) cardinality of the input space, it is highly doubtful that one can learn a magic purifier network that is able to map a combinatorial number of different block-perturbation patterns (and with arbitrary unbounded perturbation values) to the underlying true feature l .
Revised: Based on the high (combinatorial) cardinality of the input space, it is highly doubtful that one can learn a magic purifier network that is able to map a combinatorial number of different block-perturbation patterns (and with arbitrary unbounded perturbation values) to the underlying true feature l .

Original: In sharp contrast, the proposed method can guarantee to recover the underlying true feature l exactly.
Revised: In sharp contrast, the proposed method can guarantee to recover the underlying true feature l exactly.

Original:  Empirical evaluation
Revised:  Empirical evaluation

Original: We first describe the experimental setup and introduce a new attack in Section 5.1, then present experimental results in Section 5.2.
Revised: We first describe the experimental setup and introduce a new attack in Section 5.1, then present experimental results in Section 5.2.

Original: More results can be found in Section B of the supplemental material.
Revised: More results can be found in Section B of the supplemental material.

Original: 5.1 Experimental setup
Revised: 5.1 Experimental setup

Original: Datasets.
Revised: Datasets.

Original: We study the classification task on ExtraSensory (Vaizman et al., 2017) and NUSWIDE (Chua et al., 2009) datasets.
Revised: We study the classification task on ExtraSensory (Vaizman et al., 2017) and NUSWIDE (Chua et al., 2009) datasets.

Original: ExtraSensory contains the measurements from diverse sensors of smart phone and smart watch.
Revised: ExtraSensory contains the measurements from diverse sensors of smart phone and smart watch.

Original: We divide the sensors into ten agents, which correspond to the phone accelerometer, gyroscope, magnetometer, watch accelerometer, compass, location, audio, phone states (e.g., battery, wifi, ringing mode), environment sensing (e.g., light, air pressure, humidity, and temperature), and time-of-day (e.g., morning).
Revised: We divide the sensors into ten agents, which correspond to the phone accelerometer, gyroscope, magnetometer, watch accelerometer, compass, location, audio, phone states (e.g., battery, wifi, ringing mode), environment sensing (e.g., light, air pressure, humidity, and temperature), and time-of-day (e.g., morning).

Original: There are naturally missing-feature problems, e.g., the user may not permit the use of microphone and/or location.
Revised: There are naturally missing-feature problems, e.g., the user may not permit the use of microphone and/or location.

Original: The original purpose of this dataset is to recognize the behavioral context of the user (e.g., sitting, walking, and running) and recommend more appropriate music.
Revised: The original purpose of this dataset is to recognize the behavioral context of the user (e.g., sitting, walking, and running) and recommend more appropriate music.

Original: We use the first 1721 samples from a user for training, and the restsamples for testing, with the binary label ‘sitting’ or not.
Revised: We use the first 1721 samples from a user for training, and the restsamples for testing, with the binary label ‘sitting’ or not.

Original: In NUS-WIDE, each sample has 634 image features, 1000 text features, and 5 different labels, i.e. , ‘buildings’, ‘grass’, ‘animal’, ‘water’, ‘person’.
Revised: In NUS-WIDE, each sample has 634 image features, 1000 text features, and 5 different labels, i.e. , ‘buildings’, ‘grass’, ‘animal’, ‘water’, ‘person’.

Original: We split the features into four agents, where the 1st agent holds 360-d image features, 2nd agent holds the rest of the image features, and the remaining two agents each hold 500-d text features.
Revised: We split the features into four agents, where the 1st agent holds 360-d image features, 2nd agent holds the rest of the image features, and the remaining two agents each hold 500-d text features.

Original: We use 60000 samples for training, 1000 samples for testing targeted attacks, and 10000 samples for testing untargeted attacks.
Revised: We use 60000 samples for training, 1000 samples for testing targeted attacks, and 10000 samples for testing untargeted attacks.

Original: Attack Setup.
Revised: Attack Setup.

Original: We perform the inference adversarial attacks, as well as their combination with the missing-feature attack (missing-feature attack alone is not very effective).
Revised: We perform the inference adversarial attacks, as well as their combination with the missing-feature attack (missing-feature attack alone is not very effective).

Original: For the adversarial attack, we test both targeted and untargeted attacks.
Revised: For the adversarial attack, we test both targeted and untargeted attacks.

Original: For the targeted attack , we adopt the commonly used Projected Gradient Descent (PGD) attack to generate adversarial perturbations on the embedded features held by malicious agents.
Revised: For the targeted attack , we adopt the commonly used Projected Gradient Descent (PGD) attack to generate adversarial perturbations on the embedded features held by malicious agents.

Original: However, since in this collaborative inference setting, the magnitude of the perturbation on the embedded features is allowed to be very large, we omit the projection step in PGD (or one can think of it as projecting onto a very large norm ball as the maximum perturbation constraint).
Revised: However, since in this collaborative inference setting, the magnitude of the perturbation on the embedded features is allowed to be very large, we omit the projection step in PGD (or one can think of it as projecting onto a very large norm ball as the maximum perturbation constraint).

Original: In the ExtraSensory dataset, the target label is simply the opposite of the true label, since it is a binary classification.
Revised: In the ExtraSensory dataset, the target label is simply the opposite of the true label, since it is a binary classification.

Original: We set the PGD attack with a learning rate of 0.5 and 30 iterations so that it can successfully attack the unsecured model.
Revised: We set the PGD attack with a learning rate of 0.5 and 30 iterations so that it can successfully attack the unsecured model.

Original: While in the NUS-WIDE dataset, the target label is set to be ‘grass’.
Revised: While in the NUS-WIDE dataset, the target label is set to be ‘grass’.

Original: We set the PGD attack with a learning rate of 0.1 and 50 iterations.
Revised: We set the PGD attack with a learning rate of 0.1 and 50 iterations.

Original: Note that PGD attack is a whitebox attack, where the attacker tries to find arg min h adv ℓ ( f θ 

Revised: Note that PGD attack is a whitebox attack, where the attacker tries to find arg min h adv ℓ ( f θ

Original: ( { h adv , h benign } ) , y A ) via using the gradient dℓ ( f θ 

Revised: ( { h adv , h benign } ) , y A ) via using the gradient dℓ ( f θ

Original: { h adv , h benign } ) , y A ) /d
Revised: { h adv , h benign } ) , y A ) /d

Original: h adv , where h adv denotes the overall embedded features held by malicious agents and
Revised: h adv , where h adv denotes the overall embedded features held by malicious agents and

Original: y A is the target label.
Revised: y A is the target label.

Original: In practice, the attacker may not easily obtain such gradient information, which usually needs to know the trained global model f
Revised: In practice, the attacker may not easily obtain such gradient information, which usually needs to know the trained global model f

Original: θ
Revised: θ

Original: and h benign .
Revised: and h benign .

Original: Nevertheless, this attack serves as a smart targeted perturbation on the embedded features to test the robustness of the defense methods.
Revised: Nevertheless, this attack serves as a smart targeted perturbation on the embedded features to test the robustness of the defense methods.

Original: Motivated by the discussions in Section 4, we propose and test an untargeted attack , which is agnostic to the model, training data, label, as well as features held by other agents.
Revised: Motivated by the discussions in Section 4, we propose and test an untargeted attack , which is agnostic to the model, training data, label, as well as features held by other agents.

Original: We call it distributed feature-flipping attack .
Revised: We call it distributed feature-flipping attack .

Original: In this attack, the malicious agent(s) simply flips the sign of its embedded feature vector l i and further amplifies its magnitude.
Revised: In this attack, the malicious agent(s) simply flips the sign of its embedded feature vector l i and further amplifies its magnitude.

Original: Formally, during this inference phase untargeted attack, the malicious agent i provides the embedded features h
Revised: Formally, during this inference phase untargeted attack, the malicious agent i provides the embedded features h

Original: i
Revised: i

Original: =
Revised: =

Original: − Amplification × l i to the FC, instead of providing the true embedded features l i .
Revised: − Amplification × l i to the FC, instead of providing the true embedded features l i .

Original: This attack can be easily understood in binary classification with a linear classifier since flipping the sign of the whole feature vector will likely flip the sign of the predicted label as well.
Revised: This attack can be easily understood in binary classification with a linear classifier since flipping the sign of the whole feature vector will likely flip the sign of the predicted label as well.

Original: To illustrate the effectiveness of this attack against the manifold-projection based defenses, we can look it as adding the corruption
Revised: To illustrate the effectiveness of this attack against the manifold-projection based defenses, we can look it as adding the corruption

Original: vector e i = − (1 + Amplification ) ×
Revised: vector e i = − (1 + Amplification ) ×

Original: l i to the true embedded feature vector l i of i -th agent, which is malicious.
Revised: l i to the true embedded feature vector l i of i -th agent, which is malicious.

Original: So the overall perturbation vector e has many zero blocks corresponding to benign agents.
Revised: So the overall perturbation vector e has many zero blocks corresponding to benign agents.

Original: Its non-zero blocks have values e i = − (1 + Amplification )
Revised: Its non-zero blocks have values e i = − (1 + Amplification )

Original: × l i , ∀ i corresponds to the malicious agents.
Revised: × l i , ∀ i corresponds to the malicious agents.

Original: So the inner product between perturbation vector e and true embedded features l is − (1 + Amplification ) (cid:80) i ∈
Revised: So the inner product between perturbation vector e and true embedded features l is − (1 + Amplification ) (cid:80) i ∈

Original: Ω adv ∥ l i ∥ 22 .
Revised: Ω adv ∥ l i ∥ 22 .

Original: Such perturbation vector e introduces correlations with the underlying manifold of the uncorrupted embedded features.
Revised: Such perturbation vector e introduces correlations with the underlying manifold of the uncorrupted embedded features.

Original: Therefore, projecting the corrupted embedded features h onto the manifold is hard to get rid of such corruptions.
Revised: Therefore, projecting the corrupted embedded features h onto the manifold is hard to get rid of such corruptions.

Original: We will see such feature-flipping attack is very effective in attacking many existing defenses.
Revised: We will see such feature-flipping attack is very effective in attacking many existing defenses.

Original: Baseline methods.
Revised: Baseline methods.

Original: We compare with 1)
Revised: We compare with 1)

Original: Unsecured : no defense.
Revised: Unsecured : no defense.

Original: 2) Manifold Projection : project the corrupted embedded features h test onto the learned manifold via Autoencoder, i.e. , D ϕ ( E ψ ( h test )) .
Revised: 2) Manifold Projection : project the corrupted embedded features h test onto the learned manifold via Autoencoder, i.e. , D ϕ ( E ψ ( h test )) .

Original: 3) Randomized Smoothing (Cohen et al., 2019), where randomly generated Gaussian noise is added to h test for the FC to make the prediction, the final prediction is by majority voting over 1000 such random trials.
Revised: 3) Randomized Smoothing (Cohen et al., 2019), where randomly generated Gaussian noise is added to h test for the FC to make the prediction, the final prediction is by majority voting over 1000 such random trials.

Original: Randomized Smoothing Block : since only some block(s) of the embedded features are corrupted, we further compare with an oracle version of Randomized Smoothing, which only adds random Gaussian noise to the corrupted blocks of h test .
Revised: Randomized Smoothing Block : since only some block(s) of the embedded features are corrupted, we further compare with an oracle version of Randomized Smoothing, which only adds random Gaussian noise to the corrupted blocks of h test .

Original: Randomized Ablation (Levine & Feizi, 2020) 6)
Revised: Randomized Ablation (Levine & Feizi, 2020) 6)

Original: Adversarial Training (AT) (Madry et al., 2018): standard adversarial training via using
Revised: Adversarial Training (AT) (Madry et al., 2018): standard adversarial training via using

Original: PGD attacks on h train to generate adversarial examples, but without using projection step to limit the magnitude of adversarial perturbations.
Revised: PGD attacks on h train to generate adversarial examples, but without using projection step to limit the magnitude of adversarial perturbations.

Original: For defending against the targeted attack, it knows the target label and leverages that to generate targeted adversarial examples.
Revised: For defending against the targeted attack, it knows the target label and leverages that to generate targeted adversarial examples.

Original: For defending against untargeted attack, it uses untargeted PGD attack to generate adversarial examples.
Revised: For defending against untargeted attack, it uses untargeted PGD attack to generate adversarial examples.

Original: 7) Block-wise Adversarial
Revised: 7) Block-wise Adversarial

Original: Training (BAT) : since only some block(s) of h test will be corrupted, we further propose and test a block-wise version of AT: for each training sample’s embedded features, we randomly select a block and generate adversarial perturbations (via corresponding targeted or untargeted PGD attack) on that block for adversarial training.
Revised: Training (BAT) : since only some block(s) of h test will be corrupted, we further propose and test a block-wise version of AT: for each training sample’s embedded features, we randomly select a block and generate adversarial perturbations (via corresponding targeted or untargeted PGD attack) on that block for adversarial training.

Original: 8) Oracle : where there is no attack.
Revised: 8) Oracle : where there is no attack.

Original: The implementation details of the baselines and the proposed CoPur can be found in Section C of the supplemental material.
Revised: The implementation details of the baselines and the proposed CoPur can be found in Section C of the supplemental material.

Original: 5.2 Empirical results
Revised: 5.2 Empirical results

Original: We report the Robust Accuracy of each defense method, i.e. , classification accuracy on the adversarial corrupted embedded features, against the distributed adversarial attack and its combination with the missing-feature attack.
Revised: We report the Robust Accuracy of each defense method, i.e. , classification accuracy on the adversarial corrupted embedded features, against the distributed adversarial attack and its combination with the missing-feature attack.

Original: More experiments can be found in Section B of the supplemental material.
Revised: More experiments can be found in Section B of the supplemental material.

Original: A) Adversarial attack.
Revised: A) Adversarial attack.

Original: We first test the robustness of the defense methods under the targeted attack.
Revised: We first test the robustness of the defense methods under the targeted attack.

Original: Table 1 shows the Robust Accuracy of each method against targeted PGD attacks on NUS-WIDE and
Revised: Table 1 shows the Robust Accuracy of each method against targeted PGD attacks on NUS-WIDE and

Original: ExtraSensory datasets, where the last agent is malicious.
Revised: ExtraSensory datasets, where the last agent is malicious.

Original: The Manifold Projection method (which has no recovery guarantee) shows some resistance on the NUS-WIDE dataset but is not robust on the
Revised: The Manifold Projection method (which has no recovery guarantee) shows some resistance on the NUS-WIDE dataset but is not robust on the

Original: ExtraSensory dataset.
Revised: ExtraSensory dataset.

Original: The proposed method achieves the best Robust Accuracy on both datasets.
Revised: The proposed method achieves the best Robust Accuracy on both datasets.

Original: In Figure 2a, we report the Robust Accuracy of the defense methods under the untargeted featureflipping attack on the ExtraSensory dataset, with Amplification ranging from [1, 3, 5, 10, 20], and the last three agents perform such attack.
Revised: In Figure 2a, we report the Robust Accuracy of the defense methods under the untargeted featureflipping attack on the ExtraSensory dataset, with Amplification ranging from [1, 3, 5, 10, 20], and the last three agents perform such attack.

Original: The Robust Accuracy of the compared methods drops significantly when the Amplification gets larger.
Revised: The Robust Accuracy of the compared methods drops significantly when the Amplification gets larger.

Original: The proposed method significantly outperforms the baselines by a large margin.
Revised: The proposed method significantly outperforms the baselines by a large margin.

Original: [Table]
Revised: [Table]

Original: We are also interested in knowing the robustness limit of each defense when there are more and more agents performing adversarial attacks.
Revised: We are also interested in knowing the robustness limit of each defense when there are more and more agents performing adversarial attacks.

Original: Figure 3 shows the Robust Accuracy of each defense method w.r.t.
Revised: Figure 3 shows the Robust Accuracy of each defense method w.r.t.

Original: the index set of agents that perform untargeted adversarial attack (Amplification=10) on ExtraSensory dataset.
Revised: the index set of agents that perform untargeted adversarial attack (Amplification=10) on ExtraSensory dataset.

Original: The Robustness Accuracy of the Adversarial Training method (green solid line) significantly drops when there are more than two malicious agents.
Revised: The Robustness Accuracy of the Adversarial Training method (green solid line) significantly drops when there are more than two malicious agents.

Original: The proposed method keeps very high Robust Accuracy even when agents 8-10 perform attack together.
Revised: The proposed method keeps very high Robust Accuracy even when agents 8-10 perform attack together.

Original: When agents 7perform attack together, the Robust Accuracy of the proposed method apparently drops, but it is still significantly better than other baseline methods.
Revised: When agents 7perform attack together, the Robust Accuracy of the proposed method apparently drops, but it is still significantly better than other baseline methods.

Original: B) Combined attack.
Revised: B) Combined attack.

Original: We now test the combined attack on ExtraSensory dataset, where agentwhich holds audio MFCC features, performs missing-feature attack, and agent 10 performs untargeted (Figure 2b) or targeted (Table 2) adversarial attack.
Revised: We now test the combined attack on ExtraSensory dataset, where agentwhich holds audio MFCC features, performs missing-feature attack, and agent 10 performs untargeted (Figure 2b) or targeted (Table 2) adversarial attack.

Original: The proposed method significantly outperforms the baseline methods and maintains very high Robust Accuracy under both attacks.
Revised: The proposed method significantly outperforms the baseline methods and maintains very high Robust Accuracy under both attacks.

Original: [Table]
Revised: [Table]

Original:  Conclusions
Revised:  Conclusions

Original: In this work we proposed a novel feature purification based robust collaborative inference framework to defend against a variety of attacks during inference, with theoretical guarantees.
Revised: In this work we proposed a novel feature purification based robust collaborative inference framework to defend against a variety of attacks during inference, with theoretical guarantees.

Original: The proposed non-linear robust decomposition method and its theoretical analyses may have much wider impacts.
Revised: The proposed non-linear robust decomposition method and its theoretical analyses may have much wider impacts.

Original: We further validate the robustness of the proposed framework through extensive experiments on
Revised: We further validate the robustness of the proposed framework through extensive experiments on

Original: ExtraSensory and NUS-WIDE datasets.
Revised: ExtraSensory and NUS-WIDE datasets.

Original: Our future work includes extending Algorithm 1 of Sharon et al.
Revised: Our future work includes extending Algorithm 1 of Sharon et al.

Original: (2009) to calculate the block-wise incomplete leverage constant introduced in Remark
Revised: (2009) to calculate the block-wise incomplete leverage constant introduced in Remark

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This work is partially supported by NSF grant No.1910100, NSF CNS No.2046726, NSF III 2046795,
Revised: This work is partially supported by NSF grant No.1910100, NSF CNS No.2046726, NSF III 2046795,

Original: IIS 1909577, CCF 1934986 and NIFA award 2020-67021-32799, a C3.ai DTI Award, a Jump Arches
Revised: IIS 1909577, CCF 1934986 and NIFA award 2020-67021-32799, a C3.ai DTI Award, a Jump Arches

Original: Award and the Alfred P. Sloan Foundation.
Revised: Award and the Alfred P. Sloan Foundation.

Original: S.K. was supported by
Revised: S.K. was supported by

Original: Google Inc.
Revised: Google Inc.

Original: The authors thank the anonymous reviewers for their constructive suggestions.
Revised: The authors thank the anonymous reviewers for their constructive suggestions.

Original: J.L. also thanks -Velarde for the helpful discussions regarding manifolds.
Revised: J.L. also thanks Dr. Pedro Cisneros-Velarde and Dr. Yu Ding for the helpful discussions regarding manifolds.