SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Olive Franzese\*†, Adam Dziedzic\*‡, Christopher A. Choquette-Choo³, Mark R. Thomas², Muhammad Ahmad Kaleem², Stephan Rabanser², Congyu Fang² Somesh Jha³,⁴, Nicolas Papernot², Xiao Wang¹
<sup>1</sup>Northwestern University, <sup>2</sup>University of Toronto and Vector Institute, <sup>3</sup>Google,
<sup>4</sup>University of Wisconsin-Madison, <sup>5</sup>CISPA
Collaborative machine learning (ML) is widely used to enable institutions to learn better models from distributed data. While collaborative approaches to learning intuitively protect user data, they remain vulnerable to either the server, the clients, or both, deviating from the protocol. Indeed, because the protocol is asymmetric, a malicious server can abuse its power to reconstruct client data points. Conversely, malicious clients can corrupt learning with malicious updates. Thus, both clients and servers require a guarantee when the other cannot be trusted to fully cooperate. In this work, we propose a peer-to-peer (P2P) learning scheme that is secure against malicious servers and robust to malicious clients. Our core contribution is a generic framework that transforms any (compatible) algorithm for robust aggregation of model updates to the setting where servers and clients can act maliciously. Finally, we demonstrate the computational efficiency of our approach even with 1-million parameter models trained by 100s of peers on standard datasets.
To leverage data that is located across different clients, service providers increasingly resort to collaborative forms of distributed machine learning. Rather than centralize the data on a single server, data remains on the owner's (client's) device(s), which could be a consumer's phone or bank/hospital's local data center. Take the canonical example of federated learning (FL) [34]. Rather than share data, clients instead send model updates to the server. Our work caters to settings where neither clients nor servers can be entirely trusted to faithfully participate in the Collaborative Learning (CL) protocol.
For example, consider if a group of banks wished to learn a better fraud detection model. Banks may not be able to directly share data [15] and further because banking is a competitive industry, it must be assumed that banks will deviate from the protocol if it serves their interest. On one hand, malicious server banks may breach the intuitive confidentiality of CL. A long line of work [9, 10, 24, 37, 43, 49, 50, 52] has shown that when the server acts maliciously, it can, for instance, construct model parameter values that exactly extract client data from (even aggregated) model updates. To protect client data from servers acting maliciously, it is thus paramount to design approaches to CL where no single server can have full control over the orchestration of the protocol. On the other hand, malicious client banks may entirely prevent learning by submitting poor updates. This may be intentional as in model poisoning attacks [2, 6, 45, 46] or unintentional if their dataset contained malformed data. Though a separate line of work [25, 26, 29, 30, 32, 38, 48] has studied how to robustly learn in the face of malicious updates (or data), there are none that have studied how to integrate such robust learning algorithms within a protocol that is secure to malicious servers. In this work, we design the first scheme that is robust to the harms of both malicious server(s) and clients, which are shown in Figure 1.
$^{\dagger}$ Correspondence to: nicholasfranzese2026@u.northwestern.edu and adam.dziedzic@cispa.de
$^{\ddagger}$ Project Lead. Work done while the author was at the University of Toronto and Vector Institute.
We observe that asymmetric power is the fundamental requirement for malicious servers to breach user data privacy. Thus, we design a fully-decentralized peer-to-peer (P2P) learning protocol where each participant (e.g., bank), or peer herein, can equally contribute to the role of the server aggregating updates (and of a client computing updates). Further, we ensure that no single peer has the power to orchestrate the protocol—instead, we elect a committee of peers to perform the aggregation at any given training round in a way that requires no central or trusted third party (see Section 3 for the full threat model). On the other hand, there is now a greater need for protection against malicious clients as the distributed nature may increase
the chances of intentional poisoning or bad data quality interfering with learning (e.g., due to fewer resources among some banks and/or competitive advantages). Thus, we ensure that our protocol can efficiently integrate with classical approaches for robustness against malicious clients, such as RSA [32], FL Trust (FLT) [14], or Centered Clipping (CC) [29]. Importantly, our work generalizes the setups of these works and introduces the general framework that adapts any (compatible) algorithm for robust aggregation of model updates to settings where servers and clients may behave maliciously.
To achieve this, our approach builds on cryptographic multi-party computation (MPC) protocols. This allows peers to collectively emulate the server's role while being robust against the collusion of a subset of these peers that may act maliciously. However, naively combining these with (insecure) robust aggregation techniques incurs prohibitive overhead because the server computation for robust aggregation, which must be securely computed in MPC, is almost always of a complexity that leads to a high multiplicative slowdown. We design a framework that modularizes the processing steps of robust aggregation so as to select the most suitable cryptographic building blocks for each one, leading to significant computational improvements. One such improvement is our proposed computational surjectivity. We show that aggregation algorithms with component functions satisfying this property can efficiently obtain security while still guaranteeing robustness against malicious peers; we also show that existing robustness algorithms satisfy this property, or can be tailored to do so.
1. We design the first collaborative learning protocol that operates under the malicious threat model and is robust to both malicious clients and servers. We provide a simulation-based proof of its cryptographic security.
2. We design our protocol as a generic compiler that can convert broad categories of robust aggregation algorithms to our improved security model efficiently. This modular approach enables practitioners obtain rigorous security guarantees while selecting the most appropriate model poisoning defense for their use case. To demonstrate our framework's flexibility, we generate malicious-secure protocols for three existing robust aggregation algorithms. We show empirically that the generated protocols retain their robustness guarantees.
3. We demonstrate the computational efficiency of our protocols. We benchmark our protocols up to 1 million parameter models, and thousands of peers. For example, we show that the aggregation step of our malicious-secure implementation of robust aggregation with RSA [32] obtains a per-round CPU time of roughly 46 seconds with  $10^{5}$  parameters when trained by 1000 peers.
Federated learning is perhaps the most studied collaborative learning framework [28, 35]. Most related to ours are variants based on Secure Aggregation (SecAgg) [11] that provide confidentiality of gradient transmission. However, existing work does not provide robust aggregation within SecAgg and is focused on the single-server setting, or additionally on their use for tighter differential privacy guarantees [16, 27, 41, 47]. In contrast, we focus solely on confidentiality in the distributed server setting with robust aggregation. Other works include CaPC [17] but this requires a trusted third party to reduce the computational overhead. We make no such assumptions. In Swarm Peer-2-Peer learning [44], participants can dynamically join or leave the collaboration and are enrolled via a Blockchain smart contract. There is no central party and each per-round server is dynamically elected
Figure 1: Motivation for P2P Learning. Current collaborative learning approaches are vulnerable to both client (denoted as  $C$  with data  $D$ ) and server attack vectors. Our framework tackles all of these vulnerabilities simultaneously.
METHOD	PROPERTY	UPDATE CONFIDENTIALITY	MALICIOUS CLIENTS	MALICIOUS SERVER	AGGREGATION COMMITTEE	ROBUST AGGREGATION
PREVENTED ATTACKS	PLAINTEXT INSPECTION	POISONING OR BACKDOORING [2, 6, 45, 46]	GRADIENT INVERSION [23, 50, 37] [43, 49, 50]	DATA RECONSTRUCTION [9, 10] OR DEGRADE UTILITY	MALFORMED DATA
SECAGG v1 [11]	✓	✗	✗	✗	✗	✗
SECAGG v2 [5]	✓	✗	✗	✗	✗	✗
CAPC [17]	✓	✗	✗	✓	✗	✗
SWARM P2P LEARNING [44]	✗	✗	✗	✓	✗	✗
BISCOTTI [39]	✓	✗	✗	✓	**	**
EIFFEL MS [20]	✓	✓	✓	✗	*	*
ACORN MS [4]	✓	✓	✓	✗	*	*
RS-P2P SHS (OURS)	✓	✗	✗	✓	✓	✓
RS-P2P MS (OURS)	✓	✓	✓	✓	✓	✓
Table 1: Comparison of Security Models between Aggregation Protocols. Robust aggregation provides protection against data poisoning by clients in the collaboration protocol. Update confidentiality guarantees that an individual updated from a client is not revealed. SHS denotes Semi-Honest Security while MS is Malicious Security. *Guarantees data integrigity, not robust aggregation of updates. **Only under a single robust aggregation protocol.
via Blockchain smart contracts. Crucially, Swarm Learning supports neither secure (confidentiality-preserving) nor robust aggregation—it uses standard parameter averaging.
Biscotti [39] incorporates robustness to poisoning by combining Multi-Krum [7] and secure aggregation through Shamir secret-sharing. Its core parts are a verification committee that runs robust update selection, and aggregation committee that computes the final model update. However, Biscotti only guarantees security in the semi-honest setting and is solely compatible with Multi-Krum, which is not always the preferable robustness algorithm [29]. Blockchain is also used as an alternative to the centralized aggregator in FL to deal with malicious participants or servers in [51]. The initial model is uploaded on the blockchain following which the participants train local models, then sign on hashes with their private keys, and upload the locally trained models to the blockchain. The validity of the uploaded models is verified with digital signatures and Multi-Krum. Algorand is used as the consensus algorithm in the blockchain system to update the global model. However, it uses a single leader for each training round and is compatible only with Multi-Krum.
Konstantinov and Lampert [31] present a distributed robust learning procedure that allows for robust learning from untrusted sources. Distributed Robust Learning (DRL) [22] is another approach to robust learning which uses a divide and conquer strategy. However, none of the papers achieves the two notions of robustness at the same time. Closest to our work are those that look to combine data integrity and confidentiality (security) [4, 20]. However, these works are crucially different from ours in that they perform checks on the underlying data of each client, not the update—then, these protocols drop clients with poor data. Because these approaches operate over a different input, they may be used simultaneously with ours.
Collaborative learning is conducted among a set of parties (herein, peers) performing one of two roles: a client (or worker) who performs learning on a local dataset, or a server that aggregates the many client updates. Our protocol differs in two main ways: first, it is conducted among a set of peers (parties) which can perform either role, and second, the role of the server is performed by a subset of peers termed the aggregation committee. To align with prior literature, we sometimes refer to peers as clients or servers when they are performing those respective roles. We consider a malicious threat model where clients and servers may perform arbitrary adversarial actions to interfere with the protocol. Malicious behavior in the two roles may include, but is not limited to the following.
1. Malicious Clients may attempt to (1) lower the quality of the trained model by sending distorted model updates. This may take the form of both (a) intentional model poisoning attacks, and (b) unintentional problems such as errors in computation, and skewed or incorrect local data sets. They may also attempt to (2) steal information about the other peers' data, i.e. break confidentiality, e.g. by colluding with other malicious peers and sharing transcripts of the protocol execution.
2. Malicious Servers / Committee Members may attempt to (1) reconstruct individual data points from the clients' updates, thus breaking data confidentiality, which can be achieved by arbitrarily modifying model parameters or colluding with other parties (Committee Members or Clients), (2) inappropriately change the shared model by e.g. omitting updates from selected clients, adding in bogus updates, or otherwise altering the global model updates.
We compose multiple cryptographic primitives, including secure committee election, verified secret sharing, distributed zero knowledge proofs, and secure multiparty computation. The assumptions and guarantees of the individual primitives are in Appendix B.2. Importantly, their composition is secure under universal composability [13]. Our overall protocol operates under the standard assumptions of authenticated point-to-point secure channels between peers and a bounded proportion of adversarial peers (see Appendix B.1 for details). The following are the formal guarantees of our protocol.
- Correctness of aggregation. Given a publicly known update aggregation function  $F^R$  and that clients submit local updates  $x_1, x_2, \ldots, x_n$ , the returned global update will be equal to  $F^R(x_1, x_2, \ldots, x_n)$ . See the following Section 4 for details.
- Confidentiality of client updates. During protocol execution, no peers gain information about any other's update  $x_{i}$  beyond what is implicitly revealed by the aggregate result  $F^{R}(x_{1},x_{2},\ldots ,x_{n})$ .
- Robustness to poisoning. An accurate model will be trained even if some subset of clients submit arbitrary poisonous updates. Our framework compiles existing robust aggregation algorithms into a stronger security model. Thus, the details of this guarantee depend on the underlying algorithm.
- Malicious (active) security. The above conditions hold even when a subset of parties actively perform arbitrary malicious behavior, including but not limited to: collusion between malicious peers, attempts to deviate from any part of the protocol, and submission of poisonous local updates.
Problem Setup. To construct a collaborative learning protocol that is robust against both malicious clients and servers, we must decentralize the task of update aggregation. Accordingly, P2P learning is conducted among a set of peers, who may be assigned the role of client or server.
4 Robust and Actively Secure Framework
Our framework efficiently lifts the robust aggregation algorithms (e.g., the aforementioned RSA, FLT, or CC) to the P2P learning setting with guaranteed malicious-secure (or, actively-secure) protocol fidelity. This security model guarantees both confidentiality and protocol fidelity against peers that may take arbitrary actions to disrupt the P2P learning protocol execution—fidelity is maintained by retaining the model fidelity guarantees of an underlying robust aggregation algorithm. Indeed, we previously mentioned that many algorithms provide model fidelity against poisonous adversaries in the single-server setting [7, 25, 29, 30, 32, 38]. Each algorithm makes different assumptions about the threat model, e.g., how many times a given malicious client can participate, what sort of malicious update they send, what the underlying data distribution is, etc. Thus, rather than pinning our framework on a single robustness algorithm, we propose a modular design that encompasses a broad class of such robust aggregation algorithms designed for the single-server setting.
4.1 Framework Design
In order to strengthen the security models of a broad class of robust aggregation algorithms, we design a modular template (Figure 2), which organizes aggregation algorithms in terms of three functions:
$$
F ^ {C}: \mathcal {D} \times \mathcal {S} \times \Omega \to U \quad F ^ {P}: U \to V \quad F ^ {R}: V ^ {m} \to \Omega
$$
The first function,  $F^C$ , represents the computation of client updates based on local data, state, and global model parameters; accordingly,  $\mathcal{D}$  is the space of possible client datasets,  $S$  is the space of local states,  $\Omega$  is the space of global model parameters, and  $U$  is the space of client updates. In the trusted single-server setting, each client computes  $F^C$  and sends their update  $\pmb{u}_i \in U$  to the server. Next comes the server's computation. We break the server's work into two parts: a preprocessing function  $F^P$  and an aggregation function  $F^R$ . The former transforms each client update to a preprocessed domain  $V$ , and the latter combines the preprocessed local updates into a global model update  $\pmb{w} \in \Omega$ . Our primary contribution is the design of a protocol that lifts any robust aggregation algorithm described in terms of these functions to a stronger security model. The security model in question
Trusted Single-Server Robust Aggregation
Public Functions: Single-server robust aggregation algorithms are defined by three functions:
-  $F^{C}(\cdot)$  - client-side update computation
-  $F^{P}(\cdot)$  - server-side update preprocessing
-  $F^{R}(\cdot)$  - server-side update aggregation.
Input: Global parameters  $\mathbf{w}$  from the previous round. Each client  $P_{i}$  has input data; all participants have local state  $\mathrm{st}$ .
Client update:
1. Each client  $P_{i}$  computes  $\pmb{u}_{i} \gets F^{C}(\mathrm{data},\mathrm{st},\pmb{w})$  and sends update  $\pmb{u}_{i}$  to the server.
Server preprocessing:
2. For each  $i\in [m]$ , the server obtains  $\pmb{v}_j$  for all  $j\in S_i$  and computes  $\pmb{v}_i\gets F^P (\{\pmb {u}_j\}_{j\in S_i})$
Server update:
3. Server computes  $\pmb{w} \gets F_R(\{\pmb{v}_i\}_{i \in [m]})$  and sends  $\pmb{w}$  to all clients.
Figure 2: Template for single-server robust aggregation.
is secure against malicious/active clients without relying on a trusted server, all while retaining the protection against poisoning attacks offered by the original algorithm.
Protocol Description. Peers carrying out a P2P Learning protocol (Figure 3) begin by randomly selecting an aggregation committee, the size of which is parameterized to guarantee an honest majority with all but negligible probability (see Appendix B for details). Since the committee is honest-majority, it can use secure MPC (secure Multi-Party Computation) and VSS (Verifiable Secret Sharing) schemes later in the protocol. All clients then compute local updates via  $F^C$  and preprocess those updates via  $F^P$ . Peers secret share their updates with VSS and pass shares to the aggregation committee. Each member of the committee receives a share of a local update from every peer. The committee uses distributed zero knowledge proofs to ensure that all updates are well-formed outputs of  $F^P$  — Section 4.2 discusses in detail how to do so with practical efficiency. Finally,  $F^R$  is computed by the aggregation committee by using the shares as input to a malicious-secure MPC protocol, and committee members send the resulting global model update to all peers. This protocol relies on users remaining online throughout, an assumption that may not always be practical. In Section 4.3, we discuss how to relax this.
Strengthened Security Model. In the single-server setting, the computation of  $F^R$  is handled by a single party. This makes it vulnerable to tampering – a malicious server may breach client confidentiality, omit updates from certain clients, modify updates, or simply make arbitrary changes to the global model. Our framework lifts aggregation algorithms to a security model where none of that is possible. Distributing the computation of  $F^R$  to an honest-majority committee equipped with malicious-secure MPC means that  $F^R$  is computed with guaranteed correctness and that no information about the local updates is leaked in the process. Further, using VSS guarantees that no committee member can breach the confidentiality of client updates before the computation of  $F^R$ , and that it is (with high probability) impossible to modify client updates before the computation of  $F^R$  without being caught. Further, since the committee is majority-honest, all peers can guarantee the received global update is correct by taking the majority result received from the committee members.
Obtaining Practical Efficiency. It is possible to strengthen the security model of almost any distributed computation by simply running it inside of a generalized MPC protocol, but doing so usually results in unbearable computational overhead since MPC substantially amplifies the cost of most operations. A key challenge that we surmount is strengthening security whilst maintaining the efficiency necessary to scale to real-world collaborative learning scenarios. The design choices we employ while formulating our protocol make this possible. For example, in applications of robust aggregation with a trusted single-server, the role of the server is typically executed by a data center with high compute capabilities. In such a setting it is beneficial to minimize client-side computation and shift the compute responsibility to the server wherever possible.
In contrast, collaborative learning with no trusted parties requires a committee to aggregate client updates, and operations performed in MPC by the committee are especially costly. Thus it becomes
Secure P2P Learning Against Malicious and Poisonous Adversaries
Protocol:
1. The clients randomly select an aggregation committee  $C \subset \{P_i\}_{i \in [m]}$ .
2. Each client  $P_{i}$  applies local computation  $\pmb{u}_{i} \gets F^{C}(\text{data}, \text{st}, \pmb{w})$ .
3. For each client  $P_{i}$ , compute  $\pmb{v}_{i} \gets F^{P}(\pmb{u}_{i})$ .
4.  $P_{i}$  secret shares  $\pmb{v}_{i}$  to obtain  $[\pmb{v}_i]$  and sends one share to each  $P_{j} \in C$ .
5. If  $F^P$  is not computationally surjective,  $P_i$  uses Distributed Zero Knowledge (DZK) to prove to the committee  $C$  that  $\pmb{v}_i$  is correctly computed from some  $\pmb{u}_i$  of  $P_i$ 's choice. Otherwise,  $P_i$  uses DZK to prove that  $\pmb{v}_i \in V$ .
6. If  $\operatorname{Domain}(F^R) \neq \operatorname{Image}(F^P)$ ,  $P_i$  uses DZK to prove to the committee  $C$  that  $\pmb{v}_i \in \operatorname{Image}(F^P)$ .
7. All committee members  $P_{j} \in C$  input shares  $[\pmb{v}_{i}]$  for all  $i \in [n]$  to a  $|C|$ -party computation protocol in order to compute  $\pmb{w} \gets F^{R}(\{\pmb{v}_{i}\}_{i \in [n]}')$ . Committee members send  $\pmb{w}$  to all clients.
Figure 3: Main protocol outline for the malicious setting.