Original: zPROBE : Zero Peek Robustness Checks for Federated Learning
Revised: zPROBE : Zero Peek Robustness Checks for Federated Learning

Original: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Privacy-preserving federated learning allows multiple users to jointly train a model withcoordination of a central server.
Revised: Privacy-preserving federated learning allows multiple users to jointly train a model with coordination of a central server.

Original: The server only learns the final aggregation result andnot the individual user updates.
Revised: The server only learns the final aggregation result and not the individual user updates.

Original: However, keeping the individual updates private allowsmalicious users to perform Byzantine attacks and degrade the accuracy without beingdetected.
Revised: However, keeping the individual updates private allows malicious users to perform Byzantine attacks and degrade the accuracy without being detected.

Original: Best existing defenses against Byzantine workers rely on robust rank-basedstatistics, e.g., median, to find malicious updates.
Revised: Best existing defenses against Byzantine workers rely on robust rank-based statistics, e.g., median, to find malicious updates.

Original: However, implementing privacypreserving rank-based statistics is nontrivial and not scalable in the secure domain,as it requires sorting all individual updates.
Revised: However, implementing privacypreserving rank-based statistics is nontrivial and not scalable in the secure domain, as it requires sorting all individual updates.

Original: We establish the first private robustnesscheck that uses high break point rank-based statistics on aggregated model updates.
Revised: We establish the first private robustness check that uses high break point rank-based statistics on aggregated model updates.

Original: Byexploiting randomized clustering, we significantly improve the scalability of our defensewithout compromising privacy.
Revised: By exploiting randomized clustering, we significantly improve the scalability of our defense without compromising privacy.

Original: We leverage our statistical bounds in zero-knowledgeproofs to detect and remove malicious updates without revealing the private user updates.
Revised: We leverage our statistical bounds in zero-knowledge proofs to detect and remove malicious updates without revealing the private user updates.

Original: Our novel framework, zPROBE , enables Byzantine resilient and secure federatedlearning.
Revised: Our novel framework, zPROBE , enables Byzantine resilient and secure federated learning.

Original: Empirical evaluations demonstrate that zPROBE provides a low overheadsolution to defend against state-of-the-art Byzantine attacks while preserving privacy.
Revised: Empirical evaluations demonstrate that zPROBE provides a low overhead solution to defend against state-of-the-art Byzantine attacks while preserving privacy.

Original:  Introduction
Revised:  Introduction

Original: Federated learning (FL) has emerged as a popular paradigm for training a central model on a datasetdistributed amongst many parties, by sending model updates without requiring the parties to share theirdata.
Revised: Federated learning (FL) has emerged as a popular paradigm for training a central model on a dataset distributed amongst many parties, by sending model updates without requiring the parties to share their data.

Original: However, model updates in FL can be exploited by adversaries to infer properties of the users’ privatetraining data [1].
Revised: However, model updates in FL can be exploited by adversaries to infer properties of the users’ private training data [1].

Original: This lack of privacy prohibits the use of FL in many machine learning applications thatinvolve sensitive data such as healthcare information [2, 3] or financial transactions [4].
Revised: This lack of privacy prohibits the use of FL in many machine learning applications that involve sensitive data such as healthcare information [2, 3] or financial transactions [4].

Original: As such, existing FLschemes are augmented with privacy-preserving guarantees.
Revised: As such, existing FL schemes are augmented with privacy-preserving guarantees.

Original: Recently proposed secure aggregation protocolsuse cryptography [5, 6, 7] to ensure that the server does not learn individual user updates, but only a final aggregate with contribution from several users.
Revised: Recently proposed secure aggregation protocols use cryptography [5, 6, 7] to ensure that the server does not learn individual user updates, but only a final aggregate with contribution from several users.

Original: Hiding individual updates from the server opens a large attacksurface for malicious clients to send invalid updates that compromise the integrity of distributed training.
Revised: Hiding individual updates from the server opens a large attack surface for malicious clients to send invalid updates that compromise the integrity of distributed training.

Original: Byzantine attacks on FL are carried out by malicious clients who manipulate their local updates todegrade the model performance [8, 9, 10].
Revised: Byzantine attacks on FL are carried out by malicious clients who manipulate their local updates to degrade the model performance [8, 9, 10].

Original: Popular high-fidelity Byzantine-robust aggregation rules relyon rank-based statistics, e.g., trimmed mean [11, 12], median [11], mean around median [13, 14], andgeometric median [15, 16, 13].
Revised: Popular high-fidelity Byzantine-robust aggregation rules rely on rank-based statistics, e.g., trimmed mean [11, 12], median [11], mean around median [13, 14], and geometric median [15, 16, 13].

Original: These schemes require sorting of the individual model updates across users.
Revised: These schemes require sorting of the individual model updates across users.

Original: As such, using them in secure FL is nontrivial and unscalable to large number of users since the centralserver cannot access the (plaintext) value of user updates.
Revised: As such, using them in secure FL is nontrivial and unscalable to large number of users since the central server cannot access the (plaintext) value of user updates.

Original: In  we  rank-based statistics while preserving privacy.
Revised: In this work we address aforementioned challenges and provide high break point Byzantine tolerance using rank-based statistics while preserving privacy.

Original: We propose a median-based robustness check that derives athreshold for acceptable model updates using securely computed mean over random user clusters.
Revised: We propose a median-based robustness check that derives a threshold for acceptable model updates using securely computed mean over random user clusters.

Original: Our thresholds are dynamic and automatically change based on gradient distributions.
Revised: Our thresholds are dynamic and automatically change based on gradient distributions.

Original: Notably, we do not need accessto individual user updates or public datasets for our defense.
Revised: Notably, we do not need access to individual user updates or public datasets for our defense.

Original: We leverage the computed thresholds to identifyand filter malicious users in a privacy-preserving manner.
Revised: We leverage the computed thresholds to identify [Figure] and filter malicious users in a privacy-preserving manner.

Original: Our Byzantine-robust framework, zPROBE ,incorporates carefully crafted zero-knowledge proofs [17, 18] to check user behavior and identify possiblemalicious actions, including sending Byzantine updates or deviating from the secure aggregation protocol.
Revised: Our Byzantine-robust framework, zPROBE , incorporates carefully crafted zero-knowledge proofs [17, 18] to check user behavior and identify possible malicious actions, including sending Byzantine updates or deviating from the secure aggregation protocol.

Original: As such, zPROBE guarantees correct and consistent behavior in the challenging malicious threat model.
Revised: As such, zPROBE guarantees correct and consistent behavior in the challenging malicious threat model.

Original: We incorporate probabilistic optimizations in the design of zPROBE to minimize the overhead of ourzero-knowledge checks, without compromising security.
Revised: We incorporate probabilistic optimizations in the design of zPROBE to minimize the overhead of our zero-knowledge checks, without compromising security.

Original: By co-designing the robustness defense andcryptographic components of zPROBE , we are able to provide a scalable and low overhead solution for[Figure] private and robust FL.
Revised: By co-designing the robustness defense and cryptographic components of zPROBE , we are able to provide a scalable and low overhead solution for private and robust FL.

Original: Our construction is the first of its kind with cost that grows sub-linearly with respectto the number of clients.
Revised: Our construction is the first of its kind with cost that grows sub-linearly with respect to the number of clients.

Original: zPROBE performs an aggregation round on ResNet20 over CIFAR-10 withonly sub-second client compute time.
Revised: zPROBE performs an aggregation round on ResNet20 over CIFAR-10 with only sub-second client compute time.

Original: In summary, our contributions are:
Revised: In summary, our contributions are:

Original: • Developing a novel privacy-preserving robustness check based on rank-based statistics.
Revised: • Developing a novel privacy-preserving robustness check based on rank-based statistics.

Original: zPROBE is robust against various Byzantine attacks with 0 .
Revised: zPROBE is robust against various Byzantine attacks with 0 .

Original: 5 − 2 .
Revised: 5 − 2 .

Original: 8% higher accuracy compared to priorwork on private and robust FL. 
Revised: 8% higher accuracy compared to prior work on private and robust FL. •

Original: • Enabling private and robust aggregation in the malicious threat model by incorporating zero-knowledge proofs.
Revised: Enabling private and robust aggregation in the malicious threat model by incorporating zero-knowledge proofs.

Original: Our Byzantine-robust secure aggregation, for the first time, scalessub-linearly with respect to number of clients.
Revised: Our Byzantine-robust secure aggregation, for the first time, scales sub-linearly with respect to number of clients.

Original: • Leveraging probabilistic optimizations to reduce zPROBE overhead without compromising security, resulting in orders of magnitude client runtime reduction.
Revised: • Leveraging probabilistic optimizations to reduce zPROBE overhead without compromising security, resulting in orders of magnitude client runtime reduction.

Original:  Cryptographic Primitives
Revised:  Cryptographic Primitives

Original: Shamir Secret Sharing [19] is a method to distribute a secret s between n parties such that any t sharescan be used to reconstruct s , but any set of t − 1 or fewer shares reveal no information about the secret.
Revised: Shamir Secret Sharing [19] is a method to distribute a secret s between n parties such that any t shares can be used to reconstruct s , but any set of t − 1 or fewer shares reveal no information about the secret.

Original: Zero-Knowledge Proof (ZKP) is a cryptographic primitive between two parties, a prover P and a verifier
Revised: Zero-Knowledge Proof (ZKP) is a cryptographic primitive between two parties, a prover P and a verifier

Original: V , which allows P to convince V that a computation on P ’s private inputs is correct without revealingthe inputs.
Revised: V , which allows P to convince V that a computation on P ’s private inputs is correct without revealing the inputs.

Original: We use the Wolverine protocol [17] with highly efficient P in terms of runtime, memory usage,and communication.
Revised: We use the Wolverine protocol [17] with highly efficient P in terms of runtime, memory usage, and communication.

Original: Details of the Wolverine protocol are included in Appendix A.
Revised: Details of the Wolverine protocol are included in Appendix A.

Original: Secure FL Aggregation includes a server and n clients each holding a private vector of model updateswith l parameters u ∈ R l .
Revised: Secure FL Aggregation includes a server and n clients each holding a private vector of model updates with l parameters u ∈ R l .

Original: The server wishes to obtain the aggregate (cid:80) ni =1 u i without learning any of theindividual client updates.
Revised: The server wishes to obtain the aggregate (cid:80) ni =1 u i without learning any of the individual client updates.

Original: [5] and follow up [6] propose a secure aggregation protocol using low-overheadcryptographic primitives such as one-time pads.
Revised: [5] and follow up [6] propose a secure aggregation protocol using low-overhead cryptographic primitives such as one-time pads.

Original: Each pair of clients ( i,j ) agree on a random vector
Revised: Each pair of clients ( i,j ) agree on a random vector

Original: m i,j .
Revised: m i,j .

Original: User i adds m i,j to their input, and user j subtracts it from their input so the masks cancel out whenaggregated.
Revised: User i adds m i,j to their input, and user j subtracts it from their input so the masks cancel out when aggregated.

Original: To ensure privacy in case of dropout or network delays, each user adds an additional randommask r i .
Revised: To ensure privacy in case of dropout or network delays, each user adds an additional random mask r i .

Original: Users then create t -out-of- n Shamir shares of their masks and share them with other clients.
Revised: Users then create t -out-of- n Shamir shares of their masks and share them with other clients.

Original: Useri computes their masked input as follows:
Revised: User i computes their masked input as follows:

Original: [Equation]
Revised: [Equation]

Original: Once the server receives all masked inputs, it asks for shares of pairwise masks for dropped users and sharesof individual masks for surviving users (but never both) to reconstruct the aggregate value.
Revised: Once the server receives all masked inputs, it asks for shares of pairwise masks for dropped users and shares of individual masks for surviving users (but never both) to reconstruct the aggregate value.

Original: The constructionin [6] builds over [5] and improves the client runtime complexity to logarithmic scale rather than linear withrespect to the number of clients.
Revised: The construction in [6] builds over [5] and improves the client runtime complexity to logarithmic scale rather than linear with respect to the number of clients.

Original: Note that the secure aggregation of [6, 5] assumes the clients are semi-honestand do not deviate from the protocol.
Revised: Note that the secure aggregation of [6, 5] assumes the clients are semi-honest and do not deviate from the protocol.

Original: However, these assumptions are not suitable for our threat modelwhich involves malicious clients.
Revised: However, these assumptions are not suitable for our threat model which involves malicious clients.

Original: We propose an aggregation protocol that benefits from speedups in [6]and is augmented with zero-knowledge proofs for the challenging malicious setting as described below.Methodology Threat Model.
Revised: We propose an aggregation protocol that benefits from speedups in [6] and is augmented with zero-knowledge proofs for the challenging malicious setting as described below.

Original: We aim to protect the privacy of individual client updates as they leak information aboutclients’ private training data.

Original: No party should learn any information about a client’s update other thanthe contribution to an aggregate value with inputs from a large number of other clients.

Original: We also aim toprotect the central model against Byzantine attacks, i.e., when a malicious client sends invalid updatesto degrade the model performance.

Original: We consider a semi-honest server that follows the protocol but may

Original: [Table]
Revised: [Table]

Original: Generate key pair ( sk i ,pk i ) , sample b i a i,j ← KeyAgreement( sk i ,pk j ) m i,j ← PRG( a i,j ), r i ← PRG( b i ) { s skj } j ∈ U , ← SS( sk i ,t ), { s bj } j ∈ U ← SS( b i ,t )
Revised: Generate key pair ( sk i ,pk i ) , sample b i a i,j ← KeyAgreement( sk i ,pk j ) m i,j ← PRG( a i,j ), r i ← PRG( b i ) { s skj } j ∈ U , ← SS( sk i ,t ), { s bj } j ∈ U ← SS( b i ,t )

Original: Send
Revised: Send

Original: [Table] [Table]
Revised: [Table] [Table]

Original: i : v i ← u i + r i − (cid:88) 0 <j<i m i,j + (cid:88)
Revised: i : v i ← u i + r i − (cid:88) 0 <j<i m i,j + (cid:88)

Original: i<j ≤ U m i,j
Revised: i<j ≤ U m i,j

Original: [Table]
Revised: [Table]

Original: Agg ← (cid:88) i ∈
Revised: Agg ← (cid:88) i ∈

Original: U s v
Revised: U s v

Original: i − (cid:88) i ∈
Revised: i − (cid:88) i ∈

Original: U s r i + (cid:88)
Revised: U s r i + (cid:88)

Original: i ∈
Revised: i ∈

Original: U s ,j ∈ U
Revised: U s ,j ∈ U

Original: d m i,j
Revised: d m i,j

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ) for j
Revised: ) for j

Original: [Table]
Revised: [Table]

Original: U
Revised: U

Original: [Table]
Revised: [Table]

Original: m ki,
Revised: m ki,

Original: [Table]
Revised: [Table]

Original: PRG
Revised: PRG

Original: [Table]
Revised: [Table]

Original: ( a i,j ) ˆ v ki ← u ki +
Revised: ( a i,j ) ˆ v ki ← u ki +

Original: [Table]
Revised: [Table]

Original: r ki −
Revised: r ki −

Original: [Table]
Revised: [Table]

Original: 0 <
Revised: 0 <

Original: j<i ˆ m ki,j + (cid:88)
Revised: j<i ˆ m ki,j + (cid:88)

Original: i<j ≤ n ˆ m ki,j check = check ∧ (ˆ v ki = v ki )
Revised: i<j ≤ n ˆ m ki,j check = check ∧ (ˆ v ki = v ki )

Original: [Table]
Revised: [Table]

Original: check
Revised: check

Original: [Table]
Revised: [Table]

Original: return check try to  learn more information from the received data.
Revised: Threat Model. We aim to protect the privacy of individual client updates as they leak information about clients’ private training data. No party should learn any information about a client’s update other than the contribution to an aggregate value with inputs from a large number of other clients.

Revised: We also aim to protect the central model against Byzantine attacks, i.e., when a malicious client sends invalid updates to degrade the model performance.

Revised: We consider a semi-honest server that follows the protocol but may try to learn more information from the received data.

Original: We assume a portion of clients are malicious, i.e.,arbitrarily deviating from the protocol, or sending erroneous updates to cause divergence in the centralmodel.
Revised: We assume a portion of clients are malicious, i.e., arbitrarily deviating from the protocol, or sending erroneous updates to cause divergence in the central model.

Original: Notably, we assume the clients may: 1 perform Byzantine attacks by changing the value of theirmodel update to degrade central model performance, 2 use inconsistent update values in different stepsof the secure aggregation protocol, 3 perform the masked update computation in Eq.
Revised: Notably, we assume the clients may: 1 perform Byzantine attacks by changing the value of their model update to degrade central model performance, 2 use inconsistent update values in different steps of the secure aggregation protocol, 3 perform the masked update computation in Eq.

Original: 1 incorrectly orwith wrong values, and 4 use incorrect seed values in generating masks and shares.
Revised: 1 incorrectly or with wrong values, and 4 use incorrect seed values in generating masks and shares.

Original: To the best of ourknowledge, zPROBE is the first single-server framework with malicious clients that is resilient againstsuch an extensive attack surface, supports client dropouts, and does not require a public clean dataset.zPROBE Overview.
Revised: To the best of our knowledge, zPROBE is the first single-server framework with malicious clients that is resilient against such an extensive attack surface, supports client dropouts, and does not require a public clean dataset. zPROBE Overview.

Original: zPROBE comprises two main components, namely, secure aggregation, androbustness establishment.
Revised: zPROBE comprises two main components, namely, secure aggregation, and robustness establishment.

Original: We propose a new secure aggregation protocol for malicious clients in Sec. 3.1.
Revised: We propose a new secure aggregation protocol for malicious clients in Sec. 3.1.

Original: Our proposed method to establish robustness is detailed in Sec. 3.2.
Revised: Our proposed method to establish robustness is detailed in Sec. 3.2.

Original: We design an adaptive Byzantinedefense that finds the dynamic range of acceptable model updates per iteration.
Revised: We design an adaptive Byzantine defense that finds the dynamic range of acceptable model updates per iteration.

Original: Using the derived bounds,we perform a secure range check on client updates to filter Byzantine attackers.
Revised: Using the derived bounds, we perform a secure range check on client updates to filter Byzantine attackers.

Original: Our robustness checkis privacy-preserving and highly scalable.
Revised: Our robustness check is privacy-preserving and highly scalable.

Original: The proposed robust and private aggregation is performed inthree steps as illustrated in Fig.
Revised: The proposed robust and private aggregation is performed in three steps as illustrated in Fig.

Original: First the server clusters the clients randomly into c clusters.
Revised: First the server clusters the clients randomly into c clusters.

Original: Each clusterc j then performs zPROBE ’s secure aggregation protocol.
Revised: Each cluster c j then performs zPROBE ’s secure aggregation protocol.

Original: The server obtains the aggregate value α j andthe mean µ j = α j / | c j | for each cluster in plaintext.
Revised: The server obtains the aggregate value α j and the mean µ j = α j / | c j | for each cluster in plaintext.

Original: In the second step, the server uses the median λ ofall cluster means to compute a threshold θ for model updates.
Revised: In the second step, the server uses the median λ of all cluster means to compute a threshold θ for model updates.

Original: The values of median λ and threshold θare public, and broadcasted by the server to all clients.
Revised: The values of median λ and threshold θ are public, and broadcasted by the server to all clients.

Original: Each client i then provides a zero-knowledge proofattesting that their update is within the threshold from the median, i.e., abs ( u i − λ ) < θ .
Revised: Each client i then provides a zero-knowledge proof attesting that their update is within the threshold from the median, i.e., abs ( u i − λ ) < θ .

Original: This ensures thatclients are not performing Byzantine attacks on the central model (item 1 in threat model).
Revised: This ensures that clients are not performing Byzantine attacks on the central model (item 1 in threat model).

Original: Users thatfail to provide the proof are considered malicious and treated as dropped.
Revised: Users that fail to provide the proof are considered malicious and treated as dropped.

Original: The remaining users participatein a round of zPROBE secure aggregation and the server obtains the final aggregate result.
Revised: The remaining users participate in a round of zPROBE secure aggregation and the server obtains the final aggregate result.

Original: 3.1 zPROBE Secure Aggregation
Revised: 3.1 zPROBE Secure Aggregation

Original: Alg.
Revised: Alg.

Original: 1 shows the detailed steps for zPROBE ’s secure aggregation for n clients consisting of three rounds.
Revised: 1 shows the detailed steps for zPROBE ’s secure aggregation for n clients consisting of three rounds.

Original: In round 1, each client i generates a key pair ( sk i ,pk i ) , samples a random seed b i , and performs a keyagreement protocol [20] with client j to obtain a shared seed a i,j .
Revised: In round 1, each client i generates a key pair ( sk i ,pk i ) , samples a random seed b i , and performs a key agreement protocol [20] with client j to obtain a shared seed a i,j .

Original: The seeds are used to generate individualand pairwise masks using a pseudorandom generator (PRG).
Revised: The seeds are used to generate individual and pairwise masks using a pseudorandom generator (PRG).

Original: Each client then creates t -out-of- n Shamirshares (SS) of sk i and b i , and sends one share of each to every other client.
Revised: Each client then creates t -out-of- n Shamir shares (SS) of sk i and b i , and sends one share of each to every other client.

Original: In the second round, each client uses the masks generated in round one to compute masked updatesaccording to Eq. 1, which are then sent to the server.
Revised: In the second round, each client uses the masks generated in round one to compute masked updates according to Eq. 1, which are then sent to the server.

Original: All clients perform the ZKP authentication protocoldescribed in Sec.
Revised: All clients perform the ZKP authentication protocol described in Sec.

Original: 2 on their update.
Revised: 2 on their update.

Original: This ensures that clients use consistent update values across differentsteps (item 2 in threat model).
Revised: This ensures that clients use consistent update values across different steps (item 2 in threat model).

Original: In addition, each client proves, in zero-knowledge, that their sent value[Figure] v i is correctly computed as shown in Alg. 2.
Revised: In addition, each client proves, in zero-knowledge, that their sent value v i is correctly computed as shown in Alg. 2.

Original: Specifically, the circuit that is evaluated in zero-knowledgeexpands the generated seeds to masks, and computes the masked update using Eq. 1.
Revised: Specifically, the circuit that is evaluated in zero-knowledge expands the generated seeds to masks, and computes the masked update using Eq. 1.

Original: The value of check isthen opened by the client, and the server verifies that check =1 .
Revised: The value of check is then opened by the client, and the server verifies that check =1 .

Original: This ensures that the masks are correctlygenerated from seeds, and the masked update is correctly computed (item 3 in the threat model).
Revised: This ensures that the masks are correctly generated from seeds, and the masked update is correctly computed (item 3 in the threat model).

Original: Users thatfail to provide the proof are dropped in the next round and their update is not incorporated in aggregation.
Revised: Users that fail to provide the proof are dropped in the next round and their update is not incorporated in aggregation.

Original: We introduce optimizations in Sec 3.3 that allow the server to derive a bound q , for the number of modelupdates to be checked, such that the probability of detecting Byzantine updates is higher than a predefinedrate.
Revised: We introduce optimizations in Sec 3.3 that allow the server to derive a bound q , for the number of model updates to be checked, such that the probability of detecting Byzantine updates is higher than a predefined rate.

Original: The server samples q random parameters from client i , and performs the update correctness check(Alg. 2).
Revised: The server samples q random parameters from client i , and performs the update correctness check (Alg. 2).

Original: We note that clients are not motivated to modify the seeds for creating masks, since this resultsin uncontrollable, out-of-bound errors that can be easily detected by the server (item 4 in threat model).
Revised: We note that clients are not motivated to modify the seeds for creating masks, since this results in uncontrollable, out-of-bound errors that can be easily detected by the server (item 4 in threat model).

Original: We discuss the effect of using wrong seeds in Appendix B.
Revised: We discuss the effect of using wrong seeds in Appendix B.

Original: In round 3, the server performs unmasking by asking for shares of sk i for dropped users and shares ofb i for surviving users, which are then used to reconstruct the pairwise and individual masks for droppedand surviving users respectively.
Revised: In round 3, the server performs unmasking by asking for shares of sk i for dropped users and shares of b i for surviving users, which are then used to reconstruct the pairwise and individual masks for dropped and surviving users respectively.

Original: The server is then able to obtain the aggregate result.
Revised: The server is then able to obtain the aggregate result.

Original: 3.2 Establishing Robustness
Revised: 3.2 Establishing Robustness

Original: Deriving Dynamic Bounds.
Revised: Deriving Dynamic Bounds.

Original: To identify the malicious gradient updates, we adaptively find the valid rangefor acceptable gradients per iteration.
Revised: To identify the malicious gradient updates, we adaptively find the valid range for acceptable gradients per iteration.

Original: In this context, acceptable gradients are those that do not harm thecentral model’s convergence when included in the gradient aggregation.
Revised: In this context, acceptable gradients are those that do not harm the central model’s convergence when included in the gradient aggregation.

Original: To successfully identify suchgradients, we rely on the underlying assumption that benign model updates are in the majority whileharmful Byzantine updates form a minority of outlier values.
Revised: To successfully identify such gradients, we rely on the underlying assumption that benign model updates are in the majority while harmful Byzantine updates form a minority of outlier values.

Original: In the presence of outliers, the median canserve as a reliable baseline for in-distribution values [14].
Revised: In the presence of outliers, the median can serve as a reliable baseline for in-distribution values [14].

Original: In the secure FL setup, the true value of the individual updates is not revealed to the server.
Revised: In the secure FL setup, the true value of the individual updates is not revealed to the server.

Original: Calculatingthe median on the masked updates is therefore nontrivial since it requires sorting the values which incursextremely high overheads in secure domain.
Revised: Calculating the median on the masked updates is therefore nontrivial since it requires sorting the values which incurs extremely high overheads in secure domain.

Original: We circumvent this challenge by forming clusters of users,where our secure aggregation can be used to efficiently compute the average of their updates.
Revised: We circumvent this challenge by forming clusters of users, where our secure aggregation can be used to efficiently compute the average of their updates.

Original: The secure aggregation protects user’s individual updates, but reveals the final mean value for each cluster { µ 1 , µ 2 ,..., µ c }to the server.
Revised: The secure aggregation protects user’s individual updates, but reveals the final mean value for each cluster { µ 1 , µ 2 ,..., µ c } to the server.

Original: The server can thus easily compute the median ( λ ) on the mean of clusters in plaintext.
Revised: The server can thus easily compute the median ( λ ) on the mean of clusters in plaintext.

Original: Using the Central Limit Theorem for Sums, cluster means follow a normal distribution µ i ∼N ( µ , 1 √ n c σ )where µ and σ denote the mean and standard deviation of the original model updates and n c is the clustersize.
Revised: Using the Central Limit Theorem for Sums, cluster means follow a normal distribution µ i ∼N ( µ , 1 √ n c σ ) where µ and σ denote the mean and standard deviation of the original model updates and n c is the cluster size.

Original: We can thus use the standard deviation of the cluster means ( σ µ ) as a distance metric for markingoutlier updates.
Revised: We can thus use the standard deviation of the cluster means ( σ µ ) as a distance metric for marking outlier updates.

Original: The distance is measured from the median of means λ , which serves as an acceptablemodel update drawn from N ( µ , 1 √ n c σ ) .
Revised: The distance is measured from the median of means λ , which serves as an acceptable model update drawn from N ( µ , 1 √ n c σ ) .

Original: For a given update u i , we investigate Byzantine behavior bychecking | u i − λ | < θ , where θ = η. σ µ = η √ n c σ .
Revised: For a given update u i , we investigate Byzantine behavior by checking | u i − λ | < θ , where θ = η. σ µ = η √ n c σ .

Original: The value of η can be tuned based on cluster size ( n c )and the desired statistical bounds on the distance in terms of the standard deviation of model updates ( σ ).
Revised: The value of η can be tuned based on cluster size ( n c ) and the desired statistical bounds on the distance in terms of the standard deviation of model updates ( σ ).

Original: Specifically, assuming a higher bound on the portion of malicious users ϕ max , the server can automaticallyadjust η such that at most (1 − ϕ max ) · n of the users are marked as benign where n is the total user count.
Revised: Specifically, assuming a higher bound on the portion of malicious users ϕ max , the server can automatically adjust η such that at most (1 − ϕ max ) · n of the users are marked as benign where n is the total user count.

Original: Secure Robustness Check.
Revised: Secure Robustness Check.

Original: We use ZKPs to identify malicious clients that send invalid updates, withoutcompromising clients’ privacy.
Revised: We use ZKPs to identify malicious clients that send invalid updates, without compromising clients’ privacy.

Original: Our ZKP relies on the robustness metrics derived in Sec. 3.2, i.e., themedian of cluster means λ and the threshold θ .
Revised: Our ZKP relies on the robustness metrics derived in Sec. 3.2, i.e., the median of cluster means λ and the threshold θ .

Original: Clients ( P ) prove to the server ( V ) that their updatescomply with the robustness range check.
Revised: Clients ( P ) prove to the server ( V ) that their updates comply with the robustness range check.

Original: During the aggregation round in step 1, clients authenticate theirprivate updates, and the authenticated value is used in steps 2 and 3.
Revised: During the aggregation round in step 1, clients authenticate their private updates, and the authenticated value is used in steps 2 and 3.

Original: This ensures that consistent valuesare used across steps and clients can not change their update after learning λ and θ to fit in the robustnessthreshold.
Revised: This ensures that consistent values are used across steps and clients can not change their update after learning λ and θ to fit in the robustness threshold.

Original: In step 2, the server makes λ and θ public.
Revised: In step 2, the server makes λ and θ public.

Original: Inside ZKP, the clients’ updates u i are used in a
Revised: Inside ZKP, the clients’ updates u i are used in a

Original: Boolean circuit determining if | u i − λ | < θ as outlined in Alg.
Revised: Boolean circuit determining if | u i − λ | < θ as outlined in Alg.

Original: Invalid model updates that fail the rangecheck are dropped from the final aggregation round. 
Revised: Invalid model updates that fail the range check are dropped from the final aggregation round. [Figure] [Figure] [Figure]

Revised: [Table]

Original: 3.3 Probabilistic Optimizations
Revised: 3.3 Probabilistic Optimizations

Original: This section provides statistical bounds on the number of required checks to accurately detect maliciousclients.
Revised: This section provides statistical bounds on the number of required checks to accurately detect malicious clients.

Original: Using the derived bounds, we optimize our framework for minimum overhead, thereby ensuringscalability to large models.
Revised: Using the derived bounds, we optimize our framework for minimum overhead, thereby ensuring scalability to large models.

Original: Malicious clients can compromise their update, by sending updates with

Original: [Figure] [Figure] [Table] distance margins larger than the tolerable threshold θ , or sending incorrect masked updates (Eq. 1).
Revised: Malicious clients can compromise their update, by sending updates with distance margins larger than the tolerable threshold θ , or sending incorrect masked updates (Eq. 1).

Original: Assumethat a portion of model updates S m , are compromised.
Revised: Assume that a portion of model updates S m , are compromised.

Original: The probability of detecting a malicious updateis equivalent to finding at least one compromised parameter gradient:
Revised: The probability of detecting a malicious update is equivalent to finding at least one compromised parameter gradient:

Original: [Equation]
Revised: [Equation]

Original: where l is the total model parameter updates, and q denotes the number of per-user ZKP checks on modelupdates.
Revised: where l is the total model parameter updates, and q denotes the number of per-user ZKP checks on model updates.

Original: The above formulation confirms that it is indeed not necessary to perform ZKP checks on allparameter updates within the model.
Revised: The above formulation confirms that it is indeed not necessary to perform ZKP checks on all parameter updates within the model.

Original: Rather, q can be easily computed via Eq.
Revised: Rather, q can be easily computed via Eq.

Original: 2, such that the probabilityof detecting a compromised update is higher than a predefined rate: p> 1 − δ .
Revised: 2, such that the probability of detecting a compromised update is higher than a predefined rate: p> 1 − δ .

Original: Fig.
Revised: Fig.

Original: 2 shows the probabilityof detecting malicious users versus number of ZKP checks for a model with l =60 K parameters.
Revised: 2 shows the probability of detecting malicious users versus number of ZKP checks for a model with l =60 K parameters.

Original: As seen,zPROBE guarantees a failure rate lower than δ =0 .
Revised: As seen, zPROBE guarantees a failure rate lower than δ =0 .

Original: 005 with very few ZKP checks.
Revised: 005 with very few ZKP checks.

Original: Note that malicioususers are incentivized to attack a high portion of updates to increase their effect on the aggregated model’saccuracy.
Revised: Note that malicious users are incentivized to attack a high portion of updates to increase their effect on the aggregated model’s accuracy.

Original: We leverage Eq.
Revised: We leverage Eq.

Original: to derive the required number of correctness and robustness checks as describedin Alg. 2 and Alg. 3.
Revised: to derive the required number of correctness and robustness checks as described in Alg. 2 and Alg. 3.

Original: For each check, the server computes the bound q , then samples q random indices frommodel parameters for each client.
Revised: For each check, the server computes the bound q , then samples q random indices from model parameters for each client.

Original: The clients then provide ZKPs for the selected set of parameter indices.
Revised: The clients then provide ZKPs for the selected set of parameter indices.

Original:  Experiments
Revised:  Experiments

Original: Details of the benchmarked models, datasets, and defense implementation are provided in Appendix C.
Revised: Details of the benchmarked models, datasets, and defense implementation are provided in Appendix C.

Original: Byzantine Attacks.
Revised: Byzantine Attacks.

Original: We assume 25% of the clients are Byzantine, which is a common assumption inthe literature [10].
Revised: We assume 25% of the clients are Byzantine, which is a common assumption in the literature [10].

Original: Malicious users alter a portion S m of benign model updates according to a Byzantineattack scenario.
Revised: Malicious users alter a portion S m of benign model updates according to a Byzantine attack scenario.

Original: We evaluate three commonly observed Byzantine attacks: Sign Flip [8], Scaling [9], and
Revised: We evaluate three commonly observed Byzantine attacks: Sign Flip [8], Scaling [9], and

Original: Non-omniscient attack [10].
Revised: Non-omniscient attack [10].

Original: Attack details are included in Appendix D.
Revised: Attack details are included in Appendix D.

Original: Baseline Defenses.
Revised: Baseline Defenses.

Original: We present comparisons with prior work on robust and private FL, i.e., BREA [21]and EIFFeL [22].
Revised: We present comparisons with prior work on robust and private FL, i.e., BREA [21] and EIFFeL [22].

Original: While zPROBE is able to implement popular defenses based on rank-based statistics,
Revised: While zPROBE is able to implement popular defenses based on rank-based statistics,

Original: EIFFeL is limited to static thresholds and requires access to clean public datasets.
Revised: EIFFeL is limited to static thresholds and requires access to clean public datasets.

Original: BREA implementsmulti-Krum [23], but leaks pairwise distances of clients to the server.
Revised: BREA implements multi-Krum [23], but leaks pairwise distances of clients to the server.

Original: zPROBE achieves lower computationcomplexity compared to both works and higher accuracy 1 compared to EIFFeL. We also benchmark acommonly used aggregator which uses only the median of cluster means, and show that its accuracy isdrastically lower that zPROBE .
Revised: zPROBE achieves lower computation complexity compared to both works and higher accuracy 2 compared to EIFFeL. We also benchmark a commonly used aggregator which uses only the median of cluster means, and show that its accuracy is drastically lower that zPROBE .

Original: Additionally, we evaluate zPROBE when the training data is non-IIDand show our adaptive bounds outperform the state-of-the-art defense [24] 2 .
Revised: Additionally, we evaluate zPROBE when the training data is non-IID and show our adaptive bounds outperform the state-of-the-art defense [24] 3 .

Original: 4.1 Defense Performance
Revised: 4.1 Defense Performance

Original: IID Training Data.
Revised: IID Training Data.

Original: We evaluate zPROBE on various benchmarks using n =50 clients, randomly groupedinto c =7 clusters.
Revised: We evaluate zPROBE on various benchmarks using n =50 clients, randomly grouped into c =7 clusters.

Original: In Section 4.3 we present evaluations with different number of clients between 30 and 200 .
Revised: In Section 4.3 we present evaluations with different number of clients between 30 and 200 .

Original: Consistent with prior work [10], we assume malicious users compromise all model updates to maximize theaccuracy degradation.
Revised: Consistent with prior work [10], we assume malicious users compromise all model updates to maximize the accuracy degradation.

Original: Fig.
Revised: Fig.

Original: 3 demonstrates the convergence behavior of the FL scheme in the presence of
Revised: 3 demonstrates the convergence behavior of the FL scheme in the presence of

Original: Byzantine users with and without zPROBE defense.
Revised: Byzantine users with and without zPROBE defense.

Original: As seen, zPROBE successfully eliminates the effect of malicious model updates and recovers the ground-truth accuracy.
Revised: As seen, zPROBE successfully eliminates the effect of malicious model updates and recovers the ground-truth accuracy.

Original: We show evaluations of zPROBEaccuracy on other variants of the dataset and attack in Fig.
Revised: We show evaluations of zPROBE accuracy on other variants of the dataset and attack in Fig.

Original: 7 in Appendix E.
Revised: 7 in Appendix E.

Original: On the MNIST benchmark, thebyzantine attacks cause the central model’s accuracy to reduce to nearly random guess ( 10 . 2% - 11 .
Revised: On the MNIST benchmark, the byzantine attacks cause the central model’s accuracy to reduce to nearly random guess ( 10 . 2% - 11 .

Original: , without any defense.
Revised: , without any defense.

Original: zPROBE successfully thwarts the malicious updates, recovering benign accuracy within0 .
Revised: zPROBE successfully thwarts the malicious updates, recovering benign accuracy within 0 .

Original: 6% margin.
Revised: 6% margin.

Original: On F-MNIST, we recover the original ∼ 88% drop of accuracy caused by the attacks to0% - 2% drop.
Revised: On F-MNIST, we recover the original ∼ 88% drop of accuracy caused by the attacks to 0% - 2% drop.

Original: Finally, on CIFAR-10, the gap between benign training and the attacked model is reduced from 45% - 90% to only 3% - 7% .
Revised: Finally, on CIFAR-10, the gap between benign training and the attacked model is reduced from 45% - 90% to only 3% - 7% .

Original: Compared to EIFFeL [22], zPROBE achieves 1 .
Revised: Compared to EIFFeL [22], zPROBE achieves 1 .

Original: 5% , and 2 . 8% higheraccuracy when evaluated on the same attack applied to MNIST, FMNIST, and CIFAR-10, respectively.
Revised: 5% , and 2 . 8% higher accuracy when evaluated on the same attack applied to MNIST, FMNIST, and CIFAR-10, respectively.

Original: Non-IID Training Data.
Revised: Non-IID Training Data.

Original: Most recently, [24] show user clustering over existing robust aggregation methodscan adapt them to heterogeneous (non-IID) data.
Revised: Most recently, [24] show user clustering over existing robust aggregation methods can adapt them to heterogeneous (non-IID) data.

Original: We follow their training setup and hyperparameters todistribute the MNIST dataset unevenly across 25 users.
Revised: We follow their training setup and hyperparameters to distribute the MNIST dataset unevenly across 25 users.

Original: As shown in Tab. 1, zPROBE defense outperformsthe accuracies obtained by the various defenses evaluated in [24].
Revised: As shown in Tab. 1, zPROBE defense outperforms the accuracies obtained by the various defenses evaluated in [24].

Original: This performance boost we believe can beattributed to 1) the use of rank-based statistics to establish dynamic thresholds, and 2) the use of all benigngradients in the aggregation, rather than replacing all values with a robust aggregator, e.g., as in KRUM.
Revised: This performance boost we believe can be attributed to 1) the use of rank-based statistics to establish dynamic thresholds, and 2) the use of all benign gradients in the aggregation, rather than replacing all values with a robust aggregator, e.g., as in KRUM.

Original: 4.2 Runtime and Complexity Analysis
Revised: 4.2 Runtime and Complexity Analysis

Original: Tab. 2 summarizes the total runtime for clients in zPROBE for one round of federated training withn =50 , c =7 , and S m =0 .
Revised: Tab. 2 summarizes the total runtime for clients in zPROBE for one round of federated training with n =50 , c =7 , and S m =0 .

Original: 3 across different benchmarks.
Revised: 3 across different benchmarks.

Original: We use the secure aggregation protocol of [6]as our baseline, which does not provide security against malicious clients or robustness against Byzantineattacks.
Revised: We use the secure aggregation protocol of [6] as our baseline, which does not provide security against malicious clients or robustness against Byzantine attacks.

Original: We show the effect of S m on zPROBE runtime in Tab. 5 in Appendix F.
Revised: We show the effect of S m on zPROBE runtime in Tab. 5 in Appendix F.

Original: We also show the effectof increasing the number of clients n on zPROBE runtime in Tab. 6 in Appendix G.
Revised: We also show the effect of increasing the number of clients n on zPROBE runtime in Tab. 6 in Appendix G.

Original: In Tab. 2, we can see that as the underlying model gets much larger (growing ∼ 4 × in size from the
Revised: In Tab. 2, we can see that as the underlying model gets much larger (growing ∼ 4 × in size from the

Original: MNIST to CIFAR-10 tasks) zPROBE overhead grows a negligible amount.
Revised: MNIST to CIFAR-10 tasks) zPROBE overhead grows a negligible amount.

Original: This is a strong indicator ofthe scalability of our proposed secure aggregation.
Revised: This is a strong indicator of the scalability of our proposed secure aggregation.

Original: Alongside this, the probabilistic optimizations explainedin Sec.
Revised: Alongside this, the probabilistic optimizations explained in Sec.

Original: 3.3 become more beneficial as the model size increases.
Revised: 3.3 become more beneficial as the model size increases.

Original: Compared to a naive implementation where1 −S m parameters are checked, we achieve a speedup of 3 orders of magnitude in client and server runtime.[Table]
Revised: Compared to a naive implementation where 1 −S m parameters are checked, we achieve a speedup of 3 orders of magnitude in client and server runtime.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: We also provide the detailed breakdown of the runtime for various components of zPROBE in Fig. 4.
Revised: We also provide the detailed breakdown of the runtime for various components of zPROBE in Fig. 4.

Original: Results are gathered on CIFAR-10 dataset and ResNet-20 architecture with n =50 , c =7 , and S m =0 .
Revised: Results are gathered on CIFAR-10 dataset and ResNet-20 architecture with n =50 , c =7 , and S m =0 .

Original: Step 2 has very low overhead, even when only 30% of model updates are Byzantine, which requires morechecks.
Revised: Step 2 has very low overhead, even when only 30% of model updates are Byzantine, which requires more checks.

Original: The most significant operation in terms of percent increase from baseline is observed in Step 3 (R2),where the correctness of masked updates are checked (Alg. 1 Round 2 and Alg. 2).
Revised: The most significant operation in terms of percent increase from baseline is observed in Step 3 (R2), where the correctness of masked updates are checked (Alg. 1 Round 2 and Alg. 2).

Original: zPROBE enjoys a lowcommunication overhead as well, requiring only 2.1MB and 4.4MB of client and server communicationrespectively, for a round of aggregation over CIFAR-10.
Revised: zPROBE enjoys a low communication overhead as well, requiring only 2.1MB and 4.4MB of client and server communication respectively, for a round of aggregation over CIFAR-10.

Original: Overall, with sub-second performance on allbenchmarks examined, zPROBE provides an efficient full privacy-preserving androbust solution for FL.zPROBE Complexity.
Revised: Overall, with sub-second performance on all benchmarks examined zPROBE provides an efficient full privacy-preserving and

Revised: robust

Original:  In this section we present the complexity analysis of zPROBE runtime withrespect to number of clients n (with k =log n ) and model size l .
Revised: solution for FL. zPROBE Complexity. In this section we present the complexity analysis of zPROBE runtime with respect to number of clients n (with k =log n ) and model size l .

Original: • Client: Each client computation consists of performing key agreements with O ( k ) , generating pairwise masks with O ( k · l ) , creating t-out-of-k Shamir shares with O ( k 2 ) , performing correctness checks of Alg. with O ( k · l ) , and performing robustness checks of Alg. 3 with O ( l ) .
Revised: • Client: Each client computation consists of performing key agreements with O ( k ) , generating pairwise masks with O ( k · l ) , creating t-out-of-k Shamir shares with O ( k 2 ) , performing correctness checks of Alg.with O ( k · l ) , and performing robustness checks of Alg. 3 with O ( l ) .

Original: The complexity of client compute is therefore O (log 2 n + l · log n ) .
Revised: The complexity of client compute is therefore O (log 2 n + l · log n ) .

Original: • Server.
Revised: • Server.

Original: The server computation consist of reconstructing t-out-of-k shamir shares with O ( n · k 2 ) ,generating pairwise masks for dropped out clients with O ( n · k · l ) , performing correctness checks of Alg. 2with O ( n · l ) , and performing robustness checks of Alg. 3 with O ( n · l ) .
Revised: The server computation consist of reconstructing t-out-of-k shamir shares with O ( n · k 2 ) , generating pairwise masks for dropped out clients with O ( n · k · l ) , performing correctness checks of Alg.with O ( n · l ) , and performing robustness checks of Alg. 3 with O ( n · l ) .

Original: The overall complexity of servercompute is thus O ( n · log 2 n + n · l · log n ) .
Revised: The overall complexity of server compute is thus O ( n · log 2 n + n · l · log n ) .

Original:  We are unable to directly compare zPROBE ’s runtime numbers with previous private and robust FLmethods since their implementations are not publicly available.
Revised: [Figure] We are unable to directly compare zPROBE ’s runtime numbers with previous private and robust FL methods since their implementations are not publicly available.

Original: Instead, Tab. 3 presents a complexitycomparison between zPROBE , BREA [21], and EIFFeL [22] with respect to number of clients n (withk = log n ), model size l , and number of malicious clients m .
Revised: Instead, Tab. 3 presents a complexity comparison between zPROBE , BREA [21], and EIFFeL [22] with respect to number of clients n (with k = log n ), model size l , and number of malicious clients m .

Original: zPROBE enjoys a lower computationalcomplexity compared to both prior art for client and server.
Revised: zPROBE enjoys a lower computational complexity compared to both prior art for client and server.

Original: Specifically, the client runtime is quadraticand linear with number of clients in BREA and EIFFeL respectively, whereas logarithmic in zPROBE . Client Server BREA
Revised: Specifically, the client runtime is quadratic

Original: O ( n 2 l + nlk 2 )

Original: O (( n 3 + nl ) k 2 · log( k ))

Original: EIFFeL
Revised: EIFFeL

Original: O ( mnl ) O (( n + l ) nk 2 · log( k )+ m.l. min( n,m 2 )) zPROBE
Revised: , whereas logarithmic in zPROBE

Original: O ( k 2 + kl )
Revised: [Figure]

Original: O ( nk 2 + nlk ) BREA [21] and EIFFeL [22].
Revised: [Table]

Original: [Figure]
Revised: [Table]

Original: 4.3 Discussion
Revised: 4.3 Discussion

Original: We perform a sensitivity analysis to various attack parameters and FL configurations on F-MNIST.
Revised: We perform a sensitivity analysis to various attack parameters and FL configurations on F-MNIST.

Original: Numberof clients is set to n =50 with c =7 clusters, unless otherwise noted.
Revised: Number of clients is set to n =50 with c =7 clusters, unless otherwise noted.

Original: Sign flip attack [8] is applied to allmodel updates with κ =5 .
Revised: Sign flip attack [8] is applied to all model updates with κ =5 .

Original: As shown, zPROBE is largely robust to changes in the underlying attack ortraining configuration, consistently recovering the central models’ accuracy.
Revised: As shown, zPROBE is largely robust to changes in the underlying attack or training configuration, consistently recovering the central models’ accuracy.

Original: Portion of Compromised Updates S m .
Revised: Portion of Compromised Updates S m .

Original: We vary the portion of Byzantine model updates ( S m ) and showthe accuracy of the central model with and without zPROBE robustness checks in Fig. 5(a).
Revised: We vary the portion of Byzantine model updates ( S m ) and show the accuracy of the central model with and without zPROBE robustness checks in Fig. 5(a).

Original: Even whenonly a small portion of model updates are malicious, zPROBE ’s outlier detection can successfully recoverthe accuracy from random guess ( 10% ) to 97 .
Revised: Even when only a small portion of model updates are malicious, zPROBE ’s outlier detection can successfully recover the accuracy from random guess ( 10% ) to 97 .

Original: To worsen the central model’s accuracy, Byzantineworkers are incentivized to attack a high number of model updates.
Revised: To worsen the central model’s accuracy, Byzantine workers are incentivized to attack a high number of model updates.

Original: Attacking all model updates resultsin a 89 .
Revised: Attacking all model updates results in a 89 .

Original: 7% drop of accuracy when no defense is present.
Revised: 7% drop of accuracy when no defense is present.

Original: However, even when all model updates are compromised, zPROBE recovers the central model’s accuracy with less than 0 .
Revised: However, even when all model updates are compromised, zPROBE recovers the central model’s accuracy with less than 0 .

Original: 5% error margin.
Revised: 5% error margin.

Original: Attack Magnitude.
Revised: Attack Magnitude.

Original: We control the magnitude of the perturbation applied to model updates by changingthe parameter κ in various Byzantine attack scenarios.
Revised: We control the magnitude of the perturbation applied to model updates by changing the parameter κ in various Byzantine attack scenarios.

Original: Fig.
Revised: Fig.

Original: 5(b) shows the effect of the attack magnitudeon the central model’s accuracy and zPROBE ’s defense performance.
Revised: 5(b) shows the effect of the attack magnitude on the central model’s accuracy and zPROBE ’s defense performance.

Original: As seen, a higher perturbation iseasier to detect using our median-based robustness check.
Revised: As seen, a higher perturbation is easier to detect using our median-based robustness check.

Original: The Byzantine attack can cause an accuracydrop of ∼ 88% when no robustness check is applied.
Revised: The Byzantine attack can cause an accuracy drop of ∼ 88% when no robustness check is applied.

Original: However, zPROBE can largely recover the accuracydegradation, reducing the accuracy loss to 0 .
Revised: However, zPROBE can largely recover the accuracy degradation, reducing the accuracy loss to 0 .

Original: Aggregation Strategy.
Revised: Aggregation Strategy.

Original: Recall from Fig.
Revised: Recall from Fig.

Original: 1 that zPROBE uses the median of per-cluster means (inplaintext) to extract a threshold, which is then used in step 2 to filter out malicious clients.
Revised: 1 that zPROBE uses the median of per-cluster means (in plaintext) to extract a threshold, which is then used in step 2 to filter out malicious clients.

Original: Rather thanperforming steps 2 and 3 of zPROBE , an alternative robust aggregation rule may directly use the medianvalue to update the global model.
Revised: Rather than performing steps 2 and 3 of zPROBE , an alternative robust aggregation rule may directly use the median value to update the global model.

Original: Merely using the median for the final aggregation ignores beneficialupdates by benign users.
Revised: Merely using the median for the final aggregation ignores beneficial updates by benign users.

Original: This leads to a drastic accuracy degradation of 28 .
Revised: This leads to a drastic accuracy degradation of 28 .

Original: 6% compared to zPROBEwhich includes all gradients that pass the threshold.
Revised: 6% compared to zPROBE which includes all gradients that pass the threshold.

Original: A comparison between median-based aggregationand zPROBE is presented in Appendix H, Fig. 8.
Revised: A comparison between median-based aggregation and zPROBE is presented in Appendix H, Fig. 8.

Original: Number of Clients ( n ).
Revised: Number of Clients ( n ).

Original: Fig. 5(c) shows the convergence of zPROBE during training for variousn ∈ [30 , 200] .
Revised: Fig. 5(c) shows the convergence of zPROBE during training for various n ∈ [30 , 200] .

Original: We note that n is the subset of clients which are picked for an aggregation round.
Revised: We note that n is the subset of clients which are picked for an aggregation round.

Original: We pickthis range according to practical deployments of FL where the server picks a small fraction of all FL clientsfor each aggregation round [25].
Revised: We pick this range according to practical deployments of FL where the server picks a small fraction of all FL clients for each aggregation round [25].

Original: As seen, client count does not affect zPROBE convergence and thecentral model’s final accuracy.
Revised: As seen, client count does not affect zPROBE convergence and the central model’s final accuracy.

Original: Specifically, zPROBE can scale to n =200 , which is among the largeststudied user counts in robust and private FL.
Revised: Specifically, zPROBE can scale to n =200 , which is among the largest studied user counts in robust and private FL.

Original: Number of Clusters ( c ).
Revised: Number of Clusters ( c ).

Original: Fig. 5(d) shows the effect of number of clusters on accuracy.
Revised: Fig. 5(d) shows the effect of number of clusters on accuracy.

Original: The performance ofzPROBE is largely independent of the number of clusters, showing less than 0 .
Revised: The performance of zPROBE is largely independent of the number of clusters, showing less than 0 .

Original: 18% variation for differentc while the increase in latency is less than 8% .
Revised: 18% variation for different c while the increase in latency is less than 8% .

Original: The number of clusters can therefore be selected freelysuch that user privacy is ensured.
Revised: The number of clusters can therefore be selected freely such that user privacy is ensured.

Original: Recall from Fig.
Revised: Recall from Fig.

Original: 1 that cluster means in step 1 of zPROBE are revealedto the server.
Revised: 1 that cluster means in step 1 of zPROBE are revealed to the server.

Original: To analyze the privacy implications, we rely on the contemporary literature in model/gradientinversion which show that increasing the batch size, or equivalently number of users, ( > 100 [26] or> 48 [27]) reduces the effectiveness of such attacks.
Revised: To analyze the privacy implications, we rely on the contemporary literature in model/gradient inversion which show that increasing the batch size, or equivalently number of users, ( > 100 [26] or > 48 [27]) reduces the effectiveness of such attacks.

Original: We benchmark the SOTA attack by [26] to reconstruct user data from the aggregate.
Revised: We benchmark the SOTA attack by [26] to reconstruct user data from the aggregate.

Original: We use a small4 -layer model and a batch size of 10 to benefit the attacker.
Revised: We use a small 4 -layer model and a batch size of 10 to benefit the attacker.

Original: Fig.
Revised: Fig.

Original: 10(a),(b) in Appendix J show the efficacy of the inversion attack as the number of users in the aggregation varies.
Revised: 10(a),(b) in Appendix J show the efficacy of the inversion attack as the number of users in the aggregation varies.

Original: As seen, the reconstructions areunintelligible with > 4 users per cluster.
Revised: As seen, the reconstructions are unintelligible with > 4 users per cluster.

Original: More recently [28] quantify the user information leakage fromthe aggregate value using Mutual Information (MI).
Revised: More recently [28] quantify the user information leakage from the aggregate value using Mutual Information (MI).

Original: They show that MI reduces with more clients, startingto plateau around 10 - 20 users where the reconstructed image quality of the DLG attack [29] is severelyaffected.
Revised: They show that MI reduces with more clients, starting to plateau around 10 - 20 users where the reconstructed image quality of the DLG attack [29] is severely affected.

Original: Based on these results, our cluster size range of 7 to 30 can preserve user privacy.
Revised: Based on these results, our cluster size range of 7 to 30 can preserve user privacy.

Original: User Dropout.
Revised: User Dropout.

Original: zPROBE secure aggregation supports user dropouts, i.e., when a user is disconnectedamidst training iterations and/or in between zPROBE steps (see Fig. 1).
Revised: zPROBE secure aggregation supports user dropouts, i.e., when a user is disconnected amidst training iterations and/or in between zPROBE steps (see Fig. 1).

Original: We simulate the effect of userdropout on the training accuracy of zPROBE in Appendix I, Fig. 9.
Revised: We simulate the effect of user dropout on the training accuracy of zPROBE in Appendix I, Fig. 9.

Original: As shown, the changes in the trainingaccuracy are very subtle ( < 0 .
Revised: As shown, the changes in the training accuracy are very subtle ( < 0 .

Original: 13% difference) in the presence of random user dropouts.
Revised: 13% difference) in the presence of random user dropouts.

Original:  Related Work
Revised:  Related Work

Original: Secure Aggregation.
Revised: Secure Aggregation.

Original: Cryptographic techniques have been used in prior work for secure aggregation, e.g.,by using random masks to hide private updates [5, 6] during computation.
Revised: Cryptographic techniques have been used in prior work for secure aggregation, e.g., by using random masks to hide private updates [5, 6] during computation.

Original: A line of work [30, 31] considersa different trust model with non-colluding servers that receive secret shares of data and collaborate tocompute the aggregate.
Revised: A line of work [30, 31] considers a different trust model with non-colluding servers that receive secret shares of data and collaborate to compute the aggregate.

Original: However, realizing the non-colluding trust assumption can be challenging inpractice.
Revised: However, realizing the non-colluding trust assumption can be challenging in practice.

Original: Differential Privacy (DP) can provide complementary privacy guarantees to cryptographic methods(by protecting information leakage from output), and have been used in conjunction to reduce the requirednoise.
Revised: Differential Privacy (DP) can provide complementary privacy guarantees to cryptographic methods (by protecting information leakage from output), and have been used in conjunction to reduce the required noise.

Original: [7] combine threshold homomorphic encryption and differential privacy for private aggregation.
Revised: [7] combine threshold homomorphic encryption and differential privacy for private aggregation.

Original: Robust Aggregation.
Revised: Robust Aggregation.

Original: Prior work on Byzantine-robust aggregation sanitizes updates using robust statisticsor historical information.
Revised: Prior work on Byzantine-robust aggregation sanitizes updates using robust statistics or historical information.

Original: (Multi-)Krum [32, 23] selects updates with minimum Euclidean distance totheir neighbors.
Revised: (Multi-)Krum [32, 23] selects updates with minimum Euclidean distance to their neighbors.

Original: Coordinate-wise operations based on median and trimmed mean [11, 13] have also beenproposed that calculate the mean of values closest to the median for each coordinate.
Revised: Coordinate-wise operations based on median and trimmed mean [11, 13] have also been proposed that calculate the mean of values closest to the median for each coordinate.

Original: More recently,
Revised: More recently,

Original: AKSEL [14] defines an interval around the median and aggregates values within that interval.
Revised: AKSEL [14] defines an interval around the median and aggregates values within that interval.

Original: [15, 33]use the robustness of geometric median to provide a robust update rule.
Revised: [15, 33] use the robustness of geometric median to provide a robust update rule.

Original: Bulyan [16] augments prioraggregation rules to ensure all coordinates are agreed upon by a majority of user gradients.
Revised: Bulyan [16] augments prior aggregation rules to ensure all coordinates are agreed upon by a majority of user gradients.

Original: Several works propose applying robust aggregation over an accumulated history of gradients, assuming IIDdata.
Revised: Several works propose applying robust aggregation over an accumulated history of gradients, assuming IID data.

Original: [34] use the concentration of aggregated past gradients around the median to mark byzantine workers.
Revised: [34] use the concentration of aggregated past gradients around the median to mark byzantine workers.

Original: Similarly, [35] apply Byzantine-resilient aggregation rules on a weighted average of past gradients usinga momentum term.
Revised: Similarly, [35] apply Byzantine-resilient aggregation rules on a weighted average of past gradients using a momentum term.

Original: In lieu of using the median, centered clipping [36] iteratively scales the accumulatedgradients to ensure robust aggregation.
Revised: In lieu of using the median, centered clipping [36] iteratively scales the accumulated gradients to ensure robust aggregation.

Original: The aforesaid works focus on robust aggregation under IID data assumptions.
Revised: The aforesaid works focus on robust aggregation under IID data assumptions.

Original: [24] propose user clustering as an effective way to adapt previously proposed robust aggregationmethods, e.g., Krum, to heterogeneous (non-IID) data.
Revised: [24] propose user clustering as an effective way to adapt previously proposed robust aggregation methods, e.g., Krum, to heterogeneous (non-IID) data.

Original: zPROBE designs a new aggregation rule that 1)enables efficient execution in the secure domain and 2) achieves state-of-the-art accuracy compared to [24].
Revised: zPROBE designs a new aggregation rule that 1) enables efficient execution in the secure domain and 2) achieves state-of-the-art accuracy compared to [24].

Original: Robust and Secure Aggregation.
Revised: Robust and Secure Aggregation.

Original: RoFL [37] focuses on model poisoning, when malicious users tryto embed a backdoor in the model, without downgrading accuracy on benign data.
Revised: RoFL [37] focuses on model poisoning, when malicious users try to embed a backdoor in the model, without downgrading accuracy on benign data.

Original: RoFL uses Pedersencommitments to implement ZKP of norm bounds over model updates.
Revised: RoFL uses Pedersen commitments to implement ZKP of norm bounds over model updates.

Original: RoFL does not support dropoutsduring aggregation and the proposed l -norm bounds are unsuitable against Byzantine workers.
Revised: RoFL does not support dropouts during aggregation and the proposed l -norm bounds are unsuitable against Byzantine workers.

Original: zPROBEconsiders Byzantine attacks where the malicious parties send invalid updates to degrade the central model’saccuracy.
Revised: zPROBE considers Byzantine attacks where the malicious parties send invalid updates to degrade the central model’s accuracy.

Original: In this domain, BREA [21] relies on Shamir secret sharing and multi-Krum [23] for aggregation.
Revised: In this domain, BREA [21] relies on Shamir secret sharing and multi-Krum [23] for aggregation.

Original: However, BREA reveals the pairwise distances of client updates to the server, i.e., even one client’scollusion with the server would reveal all updates.
Revised: However, BREA reveals the pairwise distances of client updates to the server, i.e., even one client’s collusion with the server would reveal all updates.

Original: SHARE [38] incorporates secure averaging [5] on randomly clustered clients, and filters cluster averagesthrough robust aggregation.
Revised: SHARE [38] incorporates secure averaging [5] on randomly clustered clients, and filters cluster averages through robust aggregation.

Original: Any cluster with malicious clients detected will be dropped, resulting in lossof all benign updates.
Revised: Any cluster with malicious clients detected will be dropped, resulting in loss of all benign updates.

Original: SHARE’s mitigation is to repeat the random clustering several times for each epoch,resulting in increased computation and communication cost.
Revised: SHARE’s mitigation is to repeat the random clustering several times for each epoch, resulting in increased computation and communication cost.

Original: Moreover, reclustering compromises privacy asthe server observes different variations of cluster averages which can leak information about the user updates.
Revised: Moreover, reclustering compromises privacy as the server observes different variations of cluster averages which can leak information about the user updates.

Original: Most recently, EIFFeL [22] proposes a robust aggregation using Shamir shares of client updates, and secretshared non-interactive proofs (SNIP).
Revised: Most recently, EIFFeL [22] proposes a robust aggregation using Shamir shares of client updates, and secretshared non-interactive proofs (SNIP).

Original: EIFFeL does not support rank-based statistics for robustness checks,resulting in higher accuracy degradation, and requires access to a clean public dataset for defense parameters.
Revised: EIFFeL does not support rank-based statistics for robustness checks, resulting in higher accuracy degradation, and requires access to a clean public dataset for defense parameters.

Original:  Conclusion
Revised:  Conclusion

Original: This paper presented the zPROBE framework for low overhead, scalable, private, and robust FL in themalicious client setting.
Revised: This paper presented the zPROBE framework for low overhead, scalable, private, and robust FL in the malicious client setting.

Original: zPROBE ensures correct behavior from clients, and performs robustness checkson model updates, combining zero-knowledge proofs and secret sharing to provide robustness and privacyguarantees.
Revised: zPROBE ensures correct behavior from clients, and performs robustness checks on model updates, combining zero-knowledge proofs and secret sharing to provide robustness and privacy guarantees.

Original: zPROBE presents a paradigm shift from previous work by providing a private robustnessdefense relying on rank-based statistics with cost that grows sublinearly with respect to number of clients.
Revised: zPROBE presents a paradigm shift from previous work by providing a private robustness defense relying on rank-based statistics with cost that grows sublinearly with respect to number of clients.

Original: A Preliminary on ZKP
Revised: A Preliminary on ZKP

Original: zPROBE incorporates the Wolverine ZKP protocol [17].
Revised: zPROBE incorporates the Wolverine ZKP protocol [17].

Original: In Wolverine, value x known by P isauthenticated using information-theoretic message authentication codes (IT-MACs) [39] as follows: assume∆ is a global key sampled uniformly and is known only to V .
Revised: In Wolverine, value x known by P is authenticated using information-theoretic message authentication codes (IT-MACs) [39] as follows: assume ∆ is a global key sampled uniformly and is known only to V .

Original: V is given a uniform key K [ x ] and P isgiven the corresponding MAC tag M [ x ]= K [ x ]+∆ .x .
Revised: V is given a uniform key K [ x ] and P is given the corresponding MAC tag M [ x ]= K [ x ]+∆ .x .

Original: An authenticated value can be opened (verified)by P sending x and M [ x ] to V to check whether M [ x ] ?
Revised: An authenticated value can be opened (verified) by P sending x and M [ x ] to V to check whether M [ x ] ?

Original: =
Revised: =

Original: K [ x ]+∆ .x .
Revised: K [ x ]+∆ .x .

Original: Wolverine consists of an interactive offline phase (which is circuit and input independent), and an onlinephase that builds on a special case of a secure two-party computation where one party has no input.
Revised: Wolverine consists of an interactive offline phase (which is circuit and input independent), and an online phase that builds on a special case of a secure two-party computation where one party has no input.

Original: Theparties prepare authenticated values for P inputs, and the computation is represented as an arithmetic or
Revised: The parties prepare authenticated values for P inputs, and the computation is represented as an arithmetic or

Original: Boolean circuit which is evaluated gate-by-gate.
Revised: Boolean circuit which is evaluated gate-by-gate.

Original: All communication during the online phase is from the
Revised: All communication during the online phase is from the

Original: P to the V , therefore the circuit can be evaluated using only one round of communication.
Revised: P to the V , therefore the circuit can be evaluated using only one round of communication.

Original: At the endof the evaluation, P reveals the output which indicates the correctness of the proof.
Revised: At the end of the evaluation, P reveals the output which indicates the correctness of the proof.

Original: B Malicious Seed Modification
Revised: B Malicious Seed Modification

Original: Fig. 6(a) demonstrates the histogram of L ∞ norm of benign gradients, observed throughout trainingacross 100 users for CIFAR-10 dataset.
Revised: Fig. 6(a) demonstrates the histogram of L ∞ norm of benign gradients, observed throughout training across 100 users for CIFAR-10 dataset.

Original: As seen, majority of the gradient norms are bounded in [0 , 0 . 25] .
Revised: As seen, majority of the gradient norms are bounded in [0 , 0 . 25] .

Original: Masks are generated using seeds through a pseudorandom generator (PRG), and malicious users cannotcontrol the resulting error when changing the seed.
Revised: Masks are generated using seeds through a pseudorandom generator (PRG), and malicious users cannot control the resulting error when changing the seed.

Original: Fig.
Revised: Fig.

Original: 6(b) shows a histogram of mask values generatedfrom random seeds over 10000 runs.
Revised: 6(b) shows a histogram of mask values generated from random seeds over 10000 runs.

Original: As shown, changing the seed may cause unpredictable and drasticchanges in the mask.
Revised: As shown, changing the seed may cause unpredictable and drastic changes in the mask.

Original: By changing the random seed, the generated masks can vary anywhere between− 3 × 10 4 to 3 × 10 4 , which is much larger than the normal observed range for model updates.
Revised: By changing the random seed, the generated masks can vary anywhere between − 3 × 10 4 to 3 × 10 4 , which is much larger than the normal observed range for model updates.

Original: As such,when a malicious user changes the random seed from which the masks are generated, it can lead to easilyrecognizable errors in the gradient that raises alarms for the server.
Revised: As such, when a malicious user changes the random seed from which the masks are generated, it can lead to easily recognizable errors in the gradient that raises alarms for the server.

Original: Thus, in our threat model malicioususers are incentivized to use the correct seed when computing masks. 
Revised: Thus, in our threat model malicious users are incentivized to use the correct seed when computing masks. [Figure] [Figure]

Original: C Experimental Setup Details
Revised: C Experimental Setup Details

Original: Dataset and Models.
Revised: Dataset and Models.

Original: We consider three benchmarks commonly studied by prior work in secure FL.
Revised: We consider three benchmarks commonly studied by prior work in secure FL.

Original: Our first benchmark is a variant of LeNet5 [40] trained on the MNIST dataset [41], with 2 convolutionand 3 fully-connected layers, totaling 42 K parameters.
Revised: Our first benchmark is a variant of LeNet5 [40] trained on the MNIST dataset [41], with 2 convolution and 3 fully-connected layers, totaling 42 K parameters.

Original: Our second benchmark is the Fashion-MNIST[Figure] [Figure] (F-MNIST) dataset [42] trained on the LeNet5 architecture with 60 K parameters.
Revised: Our second benchmark is the Fashion-MNIST (F-MNIST) dataset [42] trained on the LeNet5 architecture with 60 K parameters.

Original: Finally, to showcase thescalability of our approach, we evaluate ResNet-20 [43] with 273 K parameters trained on the CIFAR-10dataset [44] which is among the biggest benchmarks studied in the secure FL literature [37, 22].
Revised: Finally, to showcase the scalability of our approach, we evaluate ResNet-20 [43] with 273 K parameters trained on the CIFARdataset [44] which is among the biggest benchmarks studied in the secure FL literature [37, 22].

Original: Table 4encloses the training hyperparameters for all models.
Revised: Tableencloses the training hyperparameters for all models.

Original: [Table]
Revised: [Figure]

Original: Implementation and Configuration.
Revised: Implementation and Configuration.

Original: zPROBE defense is implemented in Python and integrated in
Revised: zPROBE defense is implemented in Python and integrated in

Original: PyTorch to enable model training.
Revised: PyTorch to enable model training.

Original: We use the EMP-Toolkit [45] for implementation of zero-knowledgeproofs.
Revised: We use the EMP-Toolkit [45] for implementation of zero-knowledge proofs.

Original: We run all experiments on a 128GB RAM, AMD Ryzen 3990X CPU desktop.
Revised: We run all experiments on a 128GB RAM, AMD Ryzen 3990X CPU desktop.

Original: All reportedruntimes are averaged over 100 trials.
Revised: All reported runtimes are averaged over 100 trials.

Original: D Byzantine Attacks
Revised: D Byzantine Attacks

Original: We show the effectiveness of our robustness checks against three commonly used Byzantine attacks.
Revised: We show the effectiveness of our robustness checks against three commonly used Byzantine attacks.

Original: Here
Revised: Here

Original: U m denotes the malicious updates, where |U m | =
Revised: U m denotes the malicious updates, where |U m | =

Original: S m · l and l is the total number of model updates.
Revised: S m · l and l is the total number of model updates.

Original: • Sign Flip [8].
Revised: • Sign Flip [8].

Original: Malicious client flips the sign of the update: u = − κ.u,κ> 0 ( ∀ u ∈U m )• Scaling [9].
Revised: Malicious client flips the sign of the update: u = − κ.u,κ> 0 ( ∀ u ∈U m ) • Scaling [9].

Original: Malicious client scales the local gradients to increase the influence on the globalmodel: u = κ.u,κ> 0 ( ∀ u ∈U m )• Non-omniscient attack [10].
Revised: Malicious client scales the local gradients to increase the influence on the global model: u = κ.u,κ> 0 ( ∀ u ∈U m ) • Non-omniscient attack [10].

Original: Malicious clients construct their Byzantine update by addinga scaled Gaussian noise to their original update with mean µ and standard deviation σ :u = µ − κ.σ ( ∀ u ∈U m )
Revised: Malicious clients construct their Byzantine update by adding a scaled Gaussian noise to their original update with mean µ and standard deviation σ : u = µ − κ.σ ( ∀ u ∈U m )

Original: E zPROBE Test Accuracy
Revised: E zPROBE Test Accuracy

Original: [Figure]

Original: Fig.
Revised: Fig.

Original: 7 shows the test accuracy of zPROBE in face of different variations of Byzantine attacks anddatasets.
Revised: 7 shows the test accuracy of zPROBE in face of different variations of Byzantine attacks and datasets.

Original: The dataset is distributed evenly (IID) among n =50 clients.
Revised: The dataset is distributed evenly (IID) among n =50 clients.

Original: The server randomly clusters usersinto c =7 groups during each training round.
Revised: The server randomly clusters users into c =7 groups during each training round.

Original: We assume malicious users compromise all model updates|S m | =1 to maximize the accuracy degradation.
Revised: We assume malicious users compromise all model updates |S m | =1 to maximize the accuracy degradation.

Original: F Effect of S m on zPROBE Runtime
Revised: F Effect of S m on zPROBE Runtime

Original: [Figure]

Original: Tab. 5 summarizes the runtime of zPROBE versus the portion of attacked model updates.
Revised: Tab. 5 summarizes the runtime of zPROBE versus the portion of attacked model updates.

Original: By decreasing
Revised: By decreasing

Original: S m , zPROBE requires more checks to detect the outlier gradients as outlined in Eq. 2.
Revised: S m , zPROBE requires more checks to detect the outlier gradients as outlined in Eq. 2.

Original: Nevertheless,due to the optimizations in zPROBE robustness and correctness checks, we are still able to maintainsub-second runtime and sublinear growth with respect to number of ZKP checks necessary.
Revised: Nevertheless, due to the optimizations in zPROBE robustness and correctness checks, we are still able to maintain sub-second runtime and sublinear growth with respect to number of ZKP checks necessary.

Original: [Table]
Revised: [Table]

Original: G Effect of Number of Clients on zPROBE Runtime
Revised: G Effect of Number of Clients on zPROBE Runtime

Original: [Figure]

Original: Fig 6 shows the effect of increasing the number of clients on the performance on zPROBE .
Revised: Fig 6 shows the effect of increasing the number of clients on the performance on zPROBE .

Original: For theseexperiments, results are gathered on the F-MNIST dataset, with c =7 and |S m | =0 .
Revised: For these experiments, results are gathered on the F-MNIST dataset, with c =7 and |S m | =0 .

Original: As seen, althoughexceeding the sub-second performance as the number of clients scales up, zPROBE maintains sublineargrowth in runtime with respect to number of clients.
Revised: As seen, although exceeding the sub-second performance as the number of clients scales up, zPROBE maintains sublinear growth in runtime with respect to number of clients.

Original: [Table]
Revised: [Table]

Original: H Effect of Aggregation Method on Accuracy
Revised: H Effect of Aggregation Method on Accuracy

Original: zPROBE leverages the median of averaged model updates across user clusters to check whether theincoming updates are benign or Byzantine.
Revised: zPROBE leverages the median of averaged model updates across user clusters to check whether the incoming updates are benign or Byzantine.

Original: An alternative aggregation strategy is to directly use the medianof cluster means, rather than performing the subsequent per-user checks.
Revised: An alternative aggregation strategy is to directly use the median of cluster means, rather than performing the subsequent per-user checks.

Original: Fig.
Revised: Fig.

Original: 8 shows the test accuracy ofzPROBE as training progresses, when compared to the above baseline aggregation method that appliesthe coordinate-wise median of cluster means.
Revised: 8 shows the test accuracy of zPROBE as training progresses, when compared to the above baseline aggregation method that applies the coordinate-wise median of cluster means.

Original: As seen, compared to zPROBE , this baseline suffers from alarge accuracy degradation, since all information in benign user updates is lost by replacing the aggregationwith the median, which can potentially contain Byzantine workers.
Revised: As seen, compared to zPROBE , this baseline suffers from a large accuracy degradation, since all information in benign user updates is lost by replacing the aggregation with the median, which can potentially contain Byzantine workers.

Original: I Effect of User Dropout
Revised: I Effect of User Dropout

Original: [Figure]

Original: Fig 9 shows the effect of random user dropouts on zPROBE defense.
Revised: Fig 9 shows the effect of random user dropouts on zPROBE defense.

Original: As shown, the fluctuations in thecentral model’s test accuracy are negligible ( < 0 .
Revised: As shown, the fluctuations in the central model’s test accuracy are negligible ( < 0 .

Original: The robustness of zPROBE aggregation protocolto user dropouts is intuitive since the remaining users can carry on the training.
Revised: The robustness of zPROBE aggregation protocol to user dropouts is intuitive since the remaining users can carry on the training.

Original: [Figure]
Revised: [Figure]

Original: J Effect of Cluster Size on Inversion Attack
Revised: J Effect of Cluster Size on Inversion Attack

Original: [Figure]

Original: Fig.
Revised: Fig.

Original: 10 shows the effect of cluster size on gradient inversion attacks.
Revised: 10 shows the effect of cluster size on gradient inversion attacks.

Original: In Fig 10(a)
Revised: In Fig 10(a)

Original: we show the effectivenessof the attack [26] for different cluster sizes.
Revised: we show the effectiveness of the attack [26] for different cluster sizes.

Original: Fig 10(b) represents the reconstruction results from user datafor different number of users participating in the aggregation round.
Revised: Fig 10(b) represents the reconstruction results from user data for different number of users participating in the aggregation round.

Original: [Figure]
Revised: [Figure]