SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[44] Stefanie Warnat-Herresthal, Hartmut Schultze, Krishnaprasad Lingadahalli Shastry, Sathyanarayanan Manamohan, Saikat Mukherjee, Vishesh Garg, Ravi Sarveswara, Kristian Handler, Peter Pickkers, N Ahmad Aziz, et al. Swarm learning for decentralized and confidential clinical machine learning. Nature, 594(7862):265-270, 2021.
[45] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Generalized byzantine-tolerant sgd, 2018. URL https://arxiv.org/abs/1802.10116.
[46] Cong Xie, Sanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation, 2019. URL https://arxiv.org/abs/1903.03936.
[47] Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher A. Choquette-Choo, Peter Kairouz, H. Brendan McMahan, Jesse Rosenstock, and Yuanbo Zhang. Federated learning of gboard language models with differential privacy, 2023.
[48] Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pages 5650–5659. PMLR, 2018.
[49] Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, and and Pavlo Molchanov. See through Gradients: Image Batch Recovery via GradInversion. 2021. URL https://openaccess.thecvf.com/content/cvpr2021/html/yin_see_through_gradients_image_batch_recoveryVia_gradinversion_cvpr_2021_paper.html.
[50] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. iDLG: Improved Deep Leakage from Gradients. 2020.
[51] Yang Zhao, Jun Zhao, Linshan Jiang, Rui Tan, Dusit Niyato, Zengxiang Li, Lingjuan Lyu, and Yingbo Liu. Privacy-preserving blockchain-based federated learning for IoT devices. IEEE Internet of Things Journal, PP:1-1, 08 2020. doi: 10.1109/JIOT.2020.3017377.
[52] Ligeng Zhu and Song Han. Deep leakage from gradients. In Federated Learning, pages 17-31. Springer, Cham, 2020. doi: 10.1007/978-3-030-63076-8_2. URL https://link.springer.com/chapter/10.1007/978-3-030-63076-8_2.
The goal of our work is to provide a protocol that enables collaborative learning with guaranteed confidentiality of client data and fidelity of the trained model, even when both clients and server(s) can act maliciously. A potential positive impact of this work is increased privacy and accountability in machine learning systems. One potentially negative impact could be the degradation of performance (in terms of compute time, communication overhead, or additional storage) for legitimate users. However, as shown in our experimental results, we are still able to cater to 100s of users with a model size of 1 mln parameters.
B Additional Information
We further present additional information, experimental results, as well as a comparison between RSA, Centered Clipping with Momentum, and FL Trust.
B.1 Committee Size
The main protocol proceeds by first selecting a subset from the pool of peers which will be responsible for aggregating the updates of all the peers. This subset is termed the aggregation committee. To guarantee security, the size of the committee  $m$  has to be adjusted based on the number of corrupted parties. Let us denote the set of corrupted parties as  $\mathcal{B}$  with  $|\mathcal{B}| = b$ . If the committee members are selected randomly, then with probability  $p = b / n$ , a given committee member is an adversary. To ensure security in the malicious case, we need the aggregation committee to have an honest majority except with negligible probability (i.e. occurring with probability less than  $2^{-40}$  as in [33, 42]). We can assess the probability of this event by modeling the number of corrupted peers in a uniform sample as a binomial random variable  $X$  with bias  $p = \frac{b}{n}$  and  $m$  trials. In particular, we are interested in values of  $p$  and  $m$  for which  $Pr[X \geq n / 2] < 2^{-40}$ . These values can be computed from the cumulative density function of the binomial distribution. Assuming a  $10\%$  adversarial corruption threshold (i.e. setting  $p = 1 / 10$ ), we obtain a committee size of 46. We use this committee size for experiments with RSA and CC. With FLTrust, in order to accommodate secret share multiplications with Shamir secret sharing, we guarantee  $Pr[X \geq n / 3] < 2^{-40}$ , which gives a committee size of 121.
B.1.1 Tolerance of Committee Members Dropping
In general, our protocol requires that the number of adversaries in the aggregation committee be kept below a certain proportion in order to guarantee security. The committee size is chosen as the smallest number of parties such that (except with negligible probability) a random sample from the pool of clients has less than  $1/2$  adversarial proportion (in the case of RSA, CC), or less than  $1/3$  (in the case of FLT). To tolerate drop out of honest committee members, we simply need to select an increased committee size such that the proportion of adversaries in the committee stays beneath these thresholds even if some number of the honest parties drop out. In particular, if we choose a committee size which guarantees (except with negligible probability) that a random sample from the pool of clients has less than  $\frac{1}{2} - \frac{q}{2}$  adversarial proportion, where  $q$  is the proportion of tolerated dropouts from honest parties, we will guarantee that the adversarial proportion with reference to the number of committee members that stay online is at most  $1/2$ . We can find the necessary committee sizes by reasoning with the binomial distribution similarly to our original analysis of committee size. For example, to tolerate  $5\%$ ,  $10\%$ , and  $15\%$  dropout of honest committee members, RSA and CC would require committee sizes of 53, 60, and 69 respectively (compared to 46 with no dropout tolerance), and FLT would require 157, 218, and 326 respectively (compared to 121 with no dropout tolerance).
B.2 Building Blocks
Byzantine Robust Aggregation. In collaborative learning (e.g., federated learning), many clients submit model updates based on their local data. These local updates are aggregated to update the global model. In settings where clients are untrusted, some Byzantine or malicious clients may submit poisonous updates (which may take arbitrary values) with the aim of degrading the quality of the global model. Broadly speaking, Byzantine robust aggregation algorithms (often abbreviated to "robust aggregation"), guarantee that an accurate global model is trained as long as the proportion of
malicious clients is bounded by a certain threshold (e.g., the theoretical analysis in CC assumes a maximum of  $15\%$  malicious clients). Further formalization of this idea occurs in a variety of ways across different works of literature – our framework is intentionally modular, inheriting the guarantees of a given underlying algorithm. Our main contribution is augmenting robustness to malicious clients with the additional guarantee that aggregation is computed correctly and confidentially even in the presence of malicious servers / aggregation committee members. We use the cryptographic primitives reviewed below to achieve this guarantee.
Committee Election. Uniform election of committee members can be efficiently instantiated using coin-flipping [8]. A classical way to accomplish this is to have all peers generate a string of random bits locally. The peers then make a cryptographic commitment to their random bits and distribute it to all other peers. After all peers have made their commitments, the random bits are all revealed. The concatenation of all the random bits can then be used as input to a random oracle, whose outputs can be used to select the committee members uniformly at random. This method for uniform random committee election is secure as long as at least one peer behaves honestly during the commitment process. We leverage this to guarantee that the aggregation committee has an honest majority (or is 2/3 honest in the case of FLT) (see Appendix B.1 for details).
Verifiable Secret Sharing. To make our protocols secure in the presence of malicious adversaries, we require Verifiable Secret Sharing (VSS). A VSS scheme allows the secret owner with a secret  $s$ , to distribute shares of  $s$  among  $n$  parties with a threshold  $t$  such that (a) any group of  $t$  parties can reveal no information about  $s$  and (b) any  $t + 1$  parties can recover the correct value of  $s$ . In this work, we use Shamir secret sharing to instantiate VSS. Secrets are shared among members of the aggregation committee  $C$ . We make guarantees on the adversarial composition of  $C$ , and set  $t$  such that honest parties may perform computations necessary during DZKP and MPC protocols (see below), yet adversarial parties never gain access to enough shares to reveal or modify  $s$ .
Distributed Zero Knowledge Proofs. A malicious-secure zero knowledge proof protocol enables a prover in possession of a witness  $w$  to prove to a verifier that for some publicly known function  $f$ ,  $f(w)$  takes a particular value. It is guaranteed that the verifier learns no additional information about  $w$  other than what is implicitly revealed by  $f(w)$ , and that no malicious prover can convince the verifier that  $f(w)$  takes an incorrect value. A distributed zero-knowledge proof (DZKP) is a variation on this primitive, wherein the prover distributes secret shares of  $w$  among a set of verifiers. Leveraging the linear operations on secret shares enabled by this setting can admit particularly efficient zero-knowledge proofs (see e.g. [12]). In our implementations, we use DZKP protocols which assume that the set of verifiers has an honest majority.
Secure Multiparty Computation. A malicious-secure multiparty computation (MPC) protocol enables a group of parties  $P_{1}, P_{2}, \ldots, P_{n}$ , with respective private inputs  $x_{1}, x_{2}, \ldots, x_{n}$  to securely compute a function  $f$  and obtain output  $f(x_{1}, x_{2}, \ldots, x_{n})$ . In particular, it is guaranteed that no party learns any additional information about the inputs beyond what is implicit in the output, and it is guaranteed that  $f$  is computed correctly, even in the presence of parties that behave in arbitrarily malicious ways. In our implementations, we use MPC protocols which assume that the set of parties has an honest majority (2/3 honest in the case of FLT).
Composition. All of the building blocks listed above are secure under universal composability [13] and thus their compositions (i.e., using them together, either in sequence or in parallel) are secure. They can all be implemented under information theoretic security, although we used a pseudorandom generator to minimize the communication. That is, using these primitives together in concert in our protocol preserves the security guarantees afforded by the individual building blocks.
To provide a concrete sense of how the security guarantees and trust assumptions of the building blocks work together in the full protocol, we provide a step-by-step elaboration on the protocol below.
1. Committee Election All clients use the method described above to randomly select the aggregation committee  $C$  with malicious security. The analysis in Appendix B.1 guarantees that  $C$  has honest majority.
2. Client Local Computation. Each client computes  $F^C$  and  $F^P$  to obtain a preprocessed local model update given their data, and the global model parameters.
3. Verifiable Secret Sharing of Updates. Each client secret shares their update with threshold  $|C| / 2$  ( $|C| / 3$  for FLT), and sends a share to each committee member. Since  $C$  has honest majority, this guarantees that adversaries cannot alter or reveal the client updates.
4. DZKP of valid update. Clients prove to the committee that their updates are valid using a DZKP protocol that takes the secret shares as input. E.g. in P2P RSA, client updates must be binary-valued, so committee members create shares of a check value which is guaranteed to be 0 if the update was binary-valued, while leaking no further information (see Appendix B.4.1 for details). Here security follows from the security of the DZKP and the VSS schemes.
5. MPC for computing global updates. Committee members compute  $F^R$  in MPC, using client shares as input, to obtain a global model update. E.g. in P2P RSA, committee members sum the shares of all client updates using the standard secure addition protocol on Shamir secret shares. The committee members then reconstruct the shared sum to obtain the global update (correct reconstruction is guaranteed by VSS). Security follows from the security of the MPC and VSS schemes.
6. Global updates sent to clients. All committee members send the recovered value to all clients. Since  $C$  has an honest majority, the clients are guaranteed to recover the correct global update by accepting the majority result.
B.3 Security Proof
We provide a proof of Theorem 1 (malicious security of Figure 3) below.
Proof: We prove the security of the protocol by constructing a simulator interacting with the adversaries controlling a subset of the parties.
1 The simulator plays the role of coin flipping and return a uniform aggregation committee. If the committee contains more adversary than the allowed threshold, the simulator aborts.
The probability of simulator aborts in this step is negligible given the committee size and threshold.
2-4 The simulator obtains shares of  $\pmb{v}_i$  from the adversary and sends them random shares on behalf of the honest parties.
5 If  $F^P$  is not computationally surjective, The simulator plays the role of DZK to obtain the adversary's input  $\boldsymbol{u}_i$ . If  $F^P$  is computationally surjective, the simulator uses  $\boldsymbol{v}_i$  to compute some  $\boldsymbol{u}_i$ .
The simulator's running time is always polynomial in this step either because efficient extraction from DZK or because of the definition of computational surjectivity.
6 The simulator plays the role of DZK and check if  $\pmb{v}_i$  is in the image of  $F^P$  and aborts if it is not the case.
7 The simulator sends  $\pmb{u}_i$  to  $\mathcal{F}_{\mathrm{P2PL}}$  and gets back the new updates; it then plays the role of  $\mathcal{F}_{\mathrm{MPC}}$  and sends back the new updates to the adversary.
B.4 Instantiating Our Malicious Framework
B.4.1 Malicious-Secure P2P RSA.
Overview of Single-Server RSA. Single-server Byzantine-robust stochastic aggregation (RSA) [32] is a set of subgradient based algorithms for robust aggregation. The key component of the method is a regularization term incorporated into the objective function to make learning robust. To enable graceful handling of heterogeneous worker datasets, each client  $i$  maintains a local set of model parameters  $\boldsymbol{x}_i^k$  whilst working together to optimize the global model parameters  $\boldsymbol{w}^k$  at a step  $k$ . At each step, clients compute a parameter update which takes into account their local data, their prior local model, as well as the global model parameters. The server receives the local client updates and uses the regularized objective to obtain a robust aggregate update. Client and server updates, respectively, are given by the equations:
$$
\boldsymbol {x} _ {i} ^ {k + 1} = \boldsymbol {x} _ {i} ^ {k} - \eta^ {k} \left(\nabla F \left(\boldsymbol {x} _ {i} ^ {k}, \xi_ {i} ^ {k}\right) + \lambda \operatorname {s i g n} \left(\boldsymbol {x} _ {i} ^ {k} - \boldsymbol {w} ^ {k}\right)\right) \tag {1}
$$
$$
\boldsymbol {w} ^ {k + 1} = \boldsymbol {w} ^ {k} - \eta^ {k} \left(\nabla f _ {0} \left(\boldsymbol {w} ^ {k}\right) + \lambda \left(\sum_ {i \in [ n ]} \operatorname {s i g n} \left(\boldsymbol {w} ^ {k} - \boldsymbol {x} _ {i} ^ {k}\right)\right)\right) \tag {2}
$$
where  $\eta$  is a decaying learning rate hyper parameter,  $\xi$  is a sampling of the local client dataset,  $F(\cdot ,\cdot)$  is the loss function,  $f(\cdot ,\cdot)$  is the robust  $(\ell_2)$  regularization term,  $\lambda$  is a hyper parameter controlling the weighting of the robustness term, the sign is performed element-wise, and  $[n]$  is the set of clients.
Lifting RSA to the P2P setting. To cast RSA into our framework, we first observe that  $\sum_{i\in [n]}\mathrm{sign}(\pmb {w}^k -\pmb {x}_i^k)$  is the only term of the server's update that requires input from the clients. Thus we limit the work of the committee solely to computing this term, and the rest of the work is done locally. We instantiate RSA for our framework in Figure 7.
In the  $F^C$  (client update computation) part of the RSA protocol, each peer receives the global model parameters  $\pmb{w}^k$ . It computes local parameter update  $\pmb{x}_i^{k+1}$  based on the global model, the local model  $\pmb{x}_i^k$ , and the local gradient  $\nabla F$ . In the  $F^P$  (update preprocessing) part of the protocol, peers compute the sign of the difference between their local parameters and the global model parameters  $\mathrm{sign}(\pmb{w}^k - \pmb{u}_i)$ , resulting in a bit vector  $\pmb{v}_i$  (one bit per model parameter). In the  $F^R$  (aggregation) part of the protocol, the committee members receive secret shares of  $\mathrm{sign}(\pmb{w}^k - \pmb{x}_i^k)$  from each participant. We observe that RSA can be lifted to the malicious security model with high efficiency: it is provably computational surjective and the underlying MPC can be efficiently instantiated.
Computational Surjectivity. Recall that in RSA peer updates are the sign of the difference between each parameter of the local and global models (Figure 7). In other words, the  $F^P$  of RSA gives  $V = \{-1,1\}^d$ , where  $d$  is the number of parameters in the model. In the single-server model of RSA [32] and in Figure 7, poisonous peers can choose arbitrary  $\mathbf{u}_i$  before  $F^P$  is computed, which gives  $\mathbf{v}_i = \mathrm{sign}(\mathbf{w}^k - \mathbf{u}_i)$ . Now we are ready to show the computational surjectivity of this  $F^P$ .
Lemma 1  $F^P$  described in Figure 7 is a computationally surjective function.
Proof: Fix an arbitrary point  $\pmb{v} = (v_{1},\dots ,v_{d})\in V = \{-1,1\}^{d}$ . We can construct  $\pmb{u}\in U$  that  $F^P$  maps to  $\pmb{v}$  by first fixing some arbitrary  $\pmb{w}^{k} = (w_{1},\dots ,w_{d})$ , and letting  $\pmb{u} = (u_{1},\dots ,u_{d})$  such that
$$
u _ {j} = w _ {j} - v _ {j} \text {f o r e a c h} j \in [ d ].
$$
Clearly the  $F^P$  of RSA  $\pmb{v}_i = \mathrm{sign}(\pmb{w}^k - \pmb{u}_i)$  maps  $\pmb{u}$  to the arbitrary  $\pmb{v}$ . So  $F^P$  is computationally surjective.
Details of the cryptographic protocol. Thus, following Figure 3, it is sufficient for peers to prove in zero knowledge that their updates are in the set  $V = \{-1,1\}^{d}$ . This can be accomplished efficiently by having each peer represent their update as  $d$  shares of binary values.
The committee can verify that a shared  $x$  is binary-valued by constructing shares of  $x \cdot (1 - x)$  and revealing it to be zero. We implement this step efficiently by batching the binary-value DZK proofs together. That is, for every shared value  $x_{i}$ , parties uniformly sample a random value  $r_i$ , and locally construct shares of the sum  $\sum r_i \cdot (x_i \cdot (1 - x_i))$ . The parties then reconstruct the sum – if it is 0, then each of the  $(x_i \cdot (1 - x_i))$  components must have been 0 with all but negligible probability. For a more detailed treatment of this technique, see [12].
During the computation of  $F^R$ , the committee needs only to sum the shares and send out the reconstructed sum. The actual value of the summed updates in  $\{-1, 1\}$  is implicitly given by the sum of the binary values (if the sum of the binary values is  $x$ , simply take  $2x - m$ ). The updated global model parameters can then be obtained via local computation of Equation 2.
Computational Surjectivity. In RSA, peer updates are the sign of the difference between each parameter of the local and global models (Figure 7). The  $F^P$  of RSA gives  $V = \{-1,1\}^d$ , where  $d$  is the number of parameters in the model. In the single-server model of RSA [32] and in Figure 7, poisonous peers can choose arbitrary  $\pmb{u}_i$  before  $F^P$  is computed, which gives  $\pmb{v}_i = \mathrm{sign}(\pmb{w}^k - \pmb{u}_i)$ . Now we are ready to show the computational surjectivity of this  $F^P$ .
Lemma 2  $F^P$  described in Figure 7 is a computationally surjective function.
Figure 7: P2P Learning with RSA.  $F^R$  can be computed efficiently by performing only  $\sum_{i \in [n]} \boldsymbol{v}_i$  on the committee side. The rest of the terms are public, so the remainder of the update can be computed locally.
B.4.2 Malicious Secure P2P CC
Overview of Single-Server Centered Clipping. Centered Clipping [29] is a recent robust aggregation that ensures a high level robustness even when the noise distribution is not uni-modal (which is assumed in many prior works.) It also provides better robustness when corrupted updates at different rounds are correlated. Below we first discuss details of the algorithm and then how to express it in our framework.
Centered Clipping (no momentum): Given the training iteration  $k$ , globally shared model parameters  $w^k$ , local model parameters  $x_i^{k+1}$  in client  $i$ , and a radius  $\tau$ , CC using the  $\ell_2$ -norm computes an updated weight vector as follows:
$$
x _ {i} ^ {k + 1} = \left(x _ {i} ^ {k + 1} - w ^ {k}\right) \min  \left(1, \frac {\tau}{\left| \left| x _ {i} ^ {k + 1} - w ^ {k} \right| \right| _ {2}}\right) \tag {3}
$$
$$
w ^ {k + 1} = w ^ {k} + \frac {1}{n} \sum_ {i = 1} ^ {n} x _ {i} ^ {k + 1} \tag {4}
$$
In Equation (3), we clip the parameters for each client  $i$ , and then aggregate them in Equation (4).
Centered Clipping with Momentum: In addition to the above, each non-Byzantine client  $i$  first computes a gradient update  $\nabla F$  based on their mini-batch  $\xi_i^k$  and the current global weights  $w^k$ . Then, using the momentum parameter  $\beta$ , each client computes a momentum vector as shown in Equation 5 (executed before Equation (3) and Equation (4)):
$$
x _ {i} ^ {k + 1} = (1 - \beta) \nabla F \left(w ^ {k}, \xi_ {i} ^ {k}\right) + \beta x _ {i} ^ {k} \tag {5}
$$
Lifting CC to the P2P setting. We bring CC into the P2P setting by placing the momentum computation inside  $F^C$ , the clipping operation inside  $F^P$ , and the aggregation of clipped updates in  $F^R$ . The clipping operation is performed on individual client updates, and thus can be performed on the client side. Further, as in RSA we note that  $F^R$  is a linear function, and thus can be computed efficiently using the homomorphic addition and scalar multiplication properties of Shamir secret sharing.
Centered Clipping does not naturally give us a surjective  $F^P$ . Of note, if a corrupted peer supplies a value of  $v_i$  that is outside of the  $\tau$ -ball surrounding  $w$ , the global update will be computed incorrectly and the model fidelity guarantees will be broken. To avoid this possibility, we make a slight modification to the CC algorithm. Namely, we clip local updates using the  $\ell_{\infty}$  norm rather than the  $\ell_2$  norm. In other words, we clip the gradients to a  $\tau$ -box rather than a  $\tau$ -ball. The computation of the global update thus becomes
$$
\boldsymbol {w} _ {k + 1} = \boldsymbol {w} _ {k} + \frac {1}{m} \sum_ {i = 1} ^ {m} \min  (\tau , \max  (- \tau , x _ {i} - \boldsymbol {w} _ {k})) \tag {6}
$$
This modification admits a computationally surjective  $F^P$  with an efficient DZK proof that a client update is within the valid domain. In particular, we take  $V = [0, 2^\theta - 1]^d$ . Then in  $F^P$  we scale, round, and map clipped gradient updates to be within this domain. Here  $\theta$  is a public constant large enough to limit discretization error of local updates during scaling – in the present study we set  $\theta$  to 32 in order to align with 32-bit fixed-point numbers. Smaller values of  $\theta$  will increase protocol efficiency, at the expense of higher discretization error during rounding and mapping in  $F^P$  step 3. Computational surjectivity of this  $F^P$  follows from a similar argument to Lemma 2.