# A Homomorphic Signcryption-Based Privacy Preserving Federated Learning Framework for IoTs

Authors: Weidong Du; Min Li; Yiliang Han; Xu An Wang; Zhaoying Wei
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Federated learning (FL) enables clients to train a machine learning model collaboratively by just aggregating their model parameters, which makes it very useful in empowering the IoTs with intelligence. To prevent privacy information leakage from parameters during aggregation, many FL frameworks use homomorphic encryption to protect client’s parameters. However, a secure federated learning framework should not only protect privacy of the parameters but also guarantee integrity of the aggregated results. In this paper, we propose an efficient homomorphic signcryption framework that can encrypt and sign the parameters in one go. According to the additive homomorphic property of our framework, it allows aggregating the signcryptions of parameters securely. Thus, our framework can both verify the integrity of the aggregated results and protect the privacy of the parameters. Moreover, we employ the blinding technique to resist collusion attacks between internal curious clients and the server and leverage the Chinese Remainder Theorem to improve efficiency. Finally, we simulate our framework in FedML. Extensive experimental results on four benchmark datasets demonstrate that our framework can protect privacy without compromising model performance, and our framework is more efficient than similar frameworks.


---

## Body

## 1. Introduction

Traditional machine learning (ML) uses huge amounts of data collected from various sources to train models. However, data sharing from different devices or organizations may disclose privacy information about the owners. To solve the dilemma between protecting data privacy and leveraging the AI benefits to these data sensitive domains, federated learning (FL) [1, 2] is proposed to train ML models without sharing data directly. Because FL can protect the privacy while utilizing data, it has great application prospect in many scenarios [3–6], especially in IoTs [7, 8].FL enables devices to collaboratively build a global ML model by only aggregating their model parameters with their data kept at their local storage. However, it still has the problem of privacy leakage. Existing researches [9–11] revealed that the exposed gradients still retain sensitive information about the training data. To avoid exposing gradients of a client during the training process, many privacy preserving frameworks based on cryptographic techniques have been proposed. Because homomorphic encryption (HE) allows aggregating the gradients through encryption envelope, privacy preserving FL frameworks based on HE have aroused many researchers’ interests [9, 12–14].However, because HE consists of complex cryptographic operations, these frameworks always bring heavy overhead. On the other hand, all clients in these frameworks share the same key pairs. If the server colludes with any client to get the secret key, the privacy protections will be invalid. Apart from efficiency and collusion attack issues, most privacy preserving FL frameworks ignore verifying the integrity of the aggregated results. However, a malicious or compromised server may forge the aggregated results to seduce clients to expose more sensitive information. Therefore, a privacy preserving FL framework should consider efficiency, privacy, and integrity.This paper aims to address these problems by designing an efficient homomorphic signcryption based privacy preserving framework for FL in IoTs. We compare our framework with others in Table1. Our contributions are summarized as follows:(i)
We design a homomorphic signcryption mechanism to encrypt and sign the clients’ gradients at the same time. According to its additive homomorphic property, our mechanism allows the server to aggregate the gradients and signatures securely and allows the clients to verify the integrity (correctness) of the aggregated results.(ii)
We employ the blinding technique to resist collusion attacks. Even ifn−2 clients try to collude with the server, they cannot steal privacy information of other clients’ parameters.(iii)
We combine the clipping and quantizing technique in Batchcrypt [13] and the Chinese Remainder Theorem to reduce the number of complex cryptographic operations and ciphertexts. Thus, our framework is more efficient in computation and communication than similar frameworks.(iv)
We present a comprehensive analysis for the security of our framework, which proves the privacy security of the gradients and the model as well as the integrity of the aggregated results. Finally, we simulate our framework in FedML on four benchmark datasets (and corresponding models): Federated MNIST, CINIC-10, CIFAR-10, and CIFAR-100. The experimental results demonstrate that our framework has high computation and communication efficiency.Table 1
Comparison of PPFLS.
PPFLUpdates privacyModel privacyCollusion resistantVerifiabilityModel supportedShokri & ShmatikovNoNoNoNoLinear & deep modelGeyer et al.NoNoNoNoLinear & deep modelBonawitz et al.YesNoYesNoLinear & deep modelPhong et al.YesYesNoNoLinear & deep modelBatchcryptYesYesYesNoLinear & deep modelMa et al.YesYesNoYesLinear & deep modelZheng et al.YesNoYesYesLinear modelXu et al.YesNoYesYesLinear & deep modelCRT-PaillierYesYesNoYesLinear & deep modelVFLYesYesYesYesLinear & deep modelOur frameworkYesYesYesYesLinear & deep modelWe note that, though our framework has the same property as VFL, our framework is more efficient both in computation and in communication, which will be proved in Section5.The rest of the paper is organized as follows: the preliminaries and related works are briefly introduced in Section2. The detailed procedures of our framework are presented in Section 3. The security analysis and experimental evaluation are provided in Sections 4 and 5, respectively. Finally, we conclude our paper in Section 6.


## 2. Preliminaries and Related Work

In this section, we introduce the preliminaries and related works of privacy preserving FL.


### 2.1. Paillier One-Way Trapdoor Permutation

In this paper, we employ the Paillier one-way trapdoor permutation [15] to realize the homomorphic signcryption mechanism. The details of the Paillier one-way trapdoor permutation are described as follows:(i)
Key generation: select two large prime numbersp and q randomly with gcdpq,p−1q−1=1 and compute the modulus N=pq and the least common multiple λ=lcmp−1,q−1. Select a random group generator g∈ℤN2∗, where the order of g divides n. The key pairs is N,g,λ.(ii)
Encryption: for any messagem<N2, split m into m1,m2 with m=m1+Nm2. The ciphertext of m is c=gm1m2NmodN2.(iii)
Decryption: the decryption process is as follows:(1)m1=LcλmodN2LgλmodN2modN,c′=cg−m1modN,m2=c′N−1modλmodN,m=m1+Nm2.TheL function is defined as Lu=u−1/N.


### 2.2. Batchcrypt

In Batchcrypt [13], the authors proposed a method to clip and quantize the gradients; it consists of two functions:(i)
α=dACIQs,V,v: C=compute the clipping threshold α according to the number s of gradients W, the maximum gradient V, and the minimum gradient v of each layer.(ii)
wi˜=Quantizewi,αi,n: quantize the gradient wi into r bits according to the scaled quantization range −nα,nα in case the sum of gradients from all clients overflows. Here, n denote the number of clients.Batchcrypt [13] allows clipping and quantizing the gradients into fixed bit width integers, which can reduce the number of parameters in encryption and decryption.


### 2.3. Chinese Remainder Theorem

Supposem0,m2,…,mK are K positive pairwise coprime integers. Let M=m1⋅m2⋯mK. Then there exists one unique integer satisfying the following group of congruences: y≡a1modm1,y≡a2modm2,…,y≡aKmodmK. Similarly, we can unpack y to get ai≡ymodmii=1,2,…,K. To ease description, we denote the packing function as y=CRTa1,a2,…,aK,m1,m2,…,mK and the unpacking function as a1,a2,…,aK=CRT_inversey,m1,m2,…,mK. For any two packed data yi and yj, the equation yi+yj≡ali+aljmodmll=1,2,…,K holds. Thus, we know that the CRT satisfies additive homomorphism.


### 2.4. Privacy Preserving Federated Learning

Many researches have been devoted to privacy preserving FL (PPFL), and they are mainly based on cryptographic methods. Shokri and Shmatikov [16] employed selective parameter update atop differential privacy to protect training record privacy. Geyer et al. [17] applied differential privacy directly to guarantee client level differential privacy. But these differential privacy based approaches reduced model accuracy significantly. Bonawitz et al. [18] leveraged secure aggregation to protect privacy in FL. Nevertheless, it exposes the trained model in the plaintext form. To address this problem, Phong et al. [9] proposed an additively homomorphic encryption based FL framework. However, it brought heavy computation and communication overhead because the encryption scheme involves complex cryptographic operations and large ciphertexts. To improve efficiency, Batchcrypt [13] tried to encode groups of quantized gradients large integers, but their method cannot defeat collusion attacks among internal curious clients and the server.


### 2.5. Verifiable Federated Learning

In FL, a malicious or compromised server may falsify aggregated results returned to the clients. Several methods have been proposed to solve this problem.Ma et al. [19] used bilinear aggregate signature [20] for integrity verification. But the verification process involves all clients. For a large number of clients, this process is time-consuming. Xu et al. [21] employed a homomorphic hash function to verify integrity, but it cannot protect the privacy of the jointly trained model. Zheng et al. [22] designed Helen, a framework utilized to secure multiparty computing protocol [23], to verify the correctness of the aggregated results. Nevertheless, the framework does not support training complex models such as neural networks. Zhang et al. (referred to as CRT-Paillier in Section 5) [24] combined bilinear aggregate signature and Paillier encryption [15] to protect privacy and integrity of the aggregated results. But they cannot resist collusion attacks between the server and internal curious clients. Fu et al. proposed VFL [25] by employing Lagrange interpolation to realize secure aggregation and check the integrity of the aggregated results, but the parameter splitting process is costly, making it unsuitable for large deep learning models.To summarize, among PPFL frameworks, many of them ignored verifying the correctness of the aggregated results. For verifiable frameworks, they either cannot apply to large nonlinear models, or cannot protect the privacy of the model, or cannot resist collusion attacks between the server and the clients. Our work is to address these problems.


## 2.1. Paillier One-Way Trapdoor Permutation

In this paper, we employ the Paillier one-way trapdoor permutation [15] to realize the homomorphic signcryption mechanism. The details of the Paillier one-way trapdoor permutation are described as follows:(i)
Key generation: select two large prime numbersp and q randomly with gcdpq,p−1q−1=1 and compute the modulus N=pq and the least common multiple λ=lcmp−1,q−1. Select a random group generator g∈ℤN2∗, where the order of g divides n. The key pairs is N,g,λ.(ii)
Encryption: for any messagem<N2, split m into m1,m2 with m=m1+Nm2. The ciphertext of m is c=gm1m2NmodN2.(iii)
Decryption: the decryption process is as follows:(1)m1=LcλmodN2LgλmodN2modN,c′=cg−m1modN,m2=c′N−1modλmodN,m=m1+Nm2.TheL function is defined as Lu=u−1/N.


## 2.2. Batchcrypt

In Batchcrypt [13], the authors proposed a method to clip and quantize the gradients; it consists of two functions:(i)
α=dACIQs,V,v: C=compute the clipping threshold α according to the number s of gradients W, the maximum gradient V, and the minimum gradient v of each layer.(ii)
wi˜=Quantizewi,αi,n: quantize the gradient wi into r bits according to the scaled quantization range −nα,nα in case the sum of gradients from all clients overflows. Here, n denote the number of clients.Batchcrypt [13] allows clipping and quantizing the gradients into fixed bit width integers, which can reduce the number of parameters in encryption and decryption.


## 2.3. Chinese Remainder Theorem

Supposem0,m2,…,mK are K positive pairwise coprime integers. Let M=m1⋅m2⋯mK. Then there exists one unique integer satisfying the following group of congruences: y≡a1modm1,y≡a2modm2,…,y≡aKmodmK. Similarly, we can unpack y to get ai≡ymodmii=1,2,…,K. To ease description, we denote the packing function as y=CRTa1,a2,…,aK,m1,m2,…,mK and the unpacking function as a1,a2,…,aK=CRT_inversey,m1,m2,…,mK. For any two packed data yi and yj, the equation yi+yj≡ali+aljmodmll=1,2,…,K holds. Thus, we know that the CRT satisfies additive homomorphism.


## 2.4. Privacy Preserving Federated Learning

Many researches have been devoted to privacy preserving FL (PPFL), and they are mainly based on cryptographic methods. Shokri and Shmatikov [16] employed selective parameter update atop differential privacy to protect training record privacy. Geyer et al. [17] applied differential privacy directly to guarantee client level differential privacy. But these differential privacy based approaches reduced model accuracy significantly. Bonawitz et al. [18] leveraged secure aggregation to protect privacy in FL. Nevertheless, it exposes the trained model in the plaintext form. To address this problem, Phong et al. [9] proposed an additively homomorphic encryption based FL framework. However, it brought heavy computation and communication overhead because the encryption scheme involves complex cryptographic operations and large ciphertexts. To improve efficiency, Batchcrypt [13] tried to encode groups of quantized gradients large integers, but their method cannot defeat collusion attacks among internal curious clients and the server.


## 2.5. Verifiable Federated Learning

In FL, a malicious or compromised server may falsify aggregated results returned to the clients. Several methods have been proposed to solve this problem.Ma et al. [19] used bilinear aggregate signature [20] for integrity verification. But the verification process involves all clients. For a large number of clients, this process is time-consuming. Xu et al. [21] employed a homomorphic hash function to verify integrity, but it cannot protect the privacy of the jointly trained model. Zheng et al. [22] designed Helen, a framework utilized to secure multiparty computing protocol [23], to verify the correctness of the aggregated results. Nevertheless, the framework does not support training complex models such as neural networks. Zhang et al. (referred to as CRT-Paillier in Section 5) [24] combined bilinear aggregate signature and Paillier encryption [15] to protect privacy and integrity of the aggregated results. But they cannot resist collusion attacks between the server and internal curious clients. Fu et al. proposed VFL [25] by employing Lagrange interpolation to realize secure aggregation and check the integrity of the aggregated results, but the parameter splitting process is costly, making it unsuitable for large deep learning models.To summarize, among PPFL frameworks, many of them ignored verifying the correctness of the aggregated results. For verifiable frameworks, they either cannot apply to large nonlinear models, or cannot protect the privacy of the model, or cannot resist collusion attacks between the server and the clients. Our work is to address these problems.


## 3. Homomorphic Signcryption-Based Privacy Preserving Federated Learning Framework for IoT

### 3.1. Threat Model and Design Goals

In our framework, we assume security threats from three different perspectives:(i)
Internal curious clients may try to steal privacy information about other clients by inspecting the ciphertexts transmitted during the training process. Here we should note that, because the ultimate goal of the clients in is to train a good ML model, the curious clients may collude with the server to steal private information about other clients, but they will not collude with the sever to tamper the aggregated results.(ii)
The server may try to steal the jointly trained model because of its great economic value. But the model should be the common property of the clients.(iii)
The server may return falsified aggregated results to clients driven by some unexpected motivations.Consequently, to enable a privacy preserving FL framework under the aforementioned threat model, the design goals are summarized as follows:(i)
Correctness: when the server and clients operate strictly according to the protocol, the aggregated results should be correct. This ensures that the clients achieve their goal and the final model has good performance.(ii)
Data privacy: data privacy means the privacy of a client’s training data and gradients. The server and curious clients cannot gather any private information about the data and gradients from messages they receive.(iii)
Model privacy: model privacy refers to the privacy of the jointly trained model. The server or any other party not in the framework cannot steal the model by inspecting the immediate data transmitted during the learning process.(iv)
Verifiability: all clients can verify the integrity of the aggregation of parameters so that clients can detect the malicious behavior if the server returns tampered aggregated results.


### 3.2. Overview

As illustrated in Figure1, our framework consists of 4 phases: initialization, model training, aggregation, and update. We summarize the 4 phases as follows:(i)
Initialization phase: the Public Key Generator (PKG) determines the model parameters and generates Paillier keypairs, Pseudo-Random GeneratorPRG, random seed S0, and K pairwise coprime large numbers mll=1,2,…,K for all clients.(ii)
Model training phase: all clients train the ML model locally clip and quantize the gradients intor bits long integers, pack them using CRT, mask them with blinding factors, and then signcrypt them.(iii)
Aggregation phase: the server aggregates the ciphertexts from all clients and distributes the result to them.(iv)
Update phase: each client firstly decrypts the aggregated ciphertexts and then verifies the integrity of the aggregated gradients. If the result is correct, it unpacks them into plain aggregated gradients and updates their local model accordingly. Otherwise, it would terminate the learning process.Figure 1
Overview of our framework.


### 3.3. Detailed Construction

#### 3.3.1. Initialization

For simplicity, we assume there aren clients, and each client is uniquely indexed by a number i. The set of client index is represented by index=1,2,…,n. In our framework, the PKG needs to initialize the model parameters and generate keys and PRG as follows:(1)
Initializing model parameters: the PKG specifies the architecture of the model, randomly initializes the model parameters asM0=Mii=1,2,…,T (T denotes the number of layers of the learning model), and specifies an appropriate learning rate η.(2)
Specifying packing parameters: the PKG generatesK positive integers mii=1,2,…,K that are pairwise coprime gcdmi,mj=1∀i≠j. We define M=∏i=1kmi. These integers are used to pack gradients.(3)
Specifying blinding parameters: the PKG randomly choosesn integer sets s1jj=1,2,…,M0,s2jj=1,2,…,M0,…,snjj=1,2,…,M0∈ℤN∗M0 such that ∑i=1nsij=0 for j=1,2,…,M0 (M0 stands for the number of parameters of the model) and then sends sijj=1,2,…,M0 to client i.(4)
Specifying PRG: the PKG sends each client the same pseudorandom generatorPRG⋅ and seed S0. The pseudorandom generator is used to synchronize random numbers among clients for signcrypting the gradients in each training round.(5)
Generating signcryption keys: the PKG sends each client a Paillier key pairN,g1,λ for encryption and decryption and a secret value g2x to sign the gradients, where N=pq,g1∈ℤN2∗, g2∈ℤN∗, and x∈N.


#### 3.3.2. Model Training Phase

The model training phase of clientii∈index consists of the following four steps:(1)
Local training: inR−th round of training, client i trains its local model on dataset Di and computes the gradients Gi.(2)
Packing: clienti sends the layer-wise maximum value Vij, minimum value vij, and the size sj of layer jj=1,2,…,T to the server. After receiving those layer-wise clipping parameters from all clients, the aggregator server calculates the clipping threshold αj=dACIQsj,MaxViji=1,2,…,n,inviji=1,2,…,n, where Max and Min compute the maximum and minimum of a set, respectively. After receiving the clipping thresholds, client i quantizes its gradients with Gi˜=QuantizeGi,αjj=1,2,…,T,n, client i partitions the quantized gradients into L=⌈Gi˜/K⌉ groups Pi=pi1=pi1ll=1,2,…,K,pi2=pi2ll=1,2,…,K,…,piL=piLll=1,2,…,K. If Gi˜ is not divisible by K, Gi˜ should be padded with 0 s. Then client i packs them into Wi=wi1,wi2,…,wiL with wij=CRTpij,mll=1,2,…,K and wij∈FM. After packing, the number of cryptographic operations and ciphertexts is greatly reduced; thus the computation and communication efficiency is improved.(3)
Blinding: to resist collusion attacks between curious clients and the aggregation server, clienti blinds wij by adding its blinding factors to get W˜i=w˜i1,w˜i2,…,w˜il=wi1+si1,wi2+si2,…,wiL+siL.(4)
Signcrypting: for each blinded gradientw˜ij∈W˜i, client i computes its signature σij=g2xw˜ijmodN and uses the PRG to synchronize with other clients a random number rR=SR=PRGSR−1 , where SR will be used as seed for the PRG in the next round of training. Finally, clients i compute a signcryption of w˜ij as cipherij=giw˜ijσijrRNmodN2.We summarize the steps of model training phase in Algorithm 1.Algorithm 1: Model training phase of client i.
Input: Training round R, model parameters MR−1, dataset Di,quantization bit width r, blinding factors sijj=1,2,…,M0, PRG, random seed SR−1, Paillier key pairs N,g1,λ, secret value g2xOutput: Signcryption of the masked gradients CipherijfunctionModelTrainingCompute gradientsGi based on MR−1 and DiSend layer-wise gradients Max-Min values and sizesVij,vij,sjj=1,2,…,T to the aggregation serverClipGi with corresponding threshold nαjj=1,2,…,T (Advance Scaling) and quantize them into r bitsBatch the quantized gradientsGi˜ layer by layer into Wi=wi1,wi2,…,wiL=EncodeGi˜BlindWi with sijj=1,2,…,M0 to compute W˜i=w˜i1,w˜i2,…,w˜il=wi1+si1,wi2+si2,…,wiL+siLSign each blinded gradientw˜ij∈W˜i with g2x, use the PRG to generate the synchronizing random number rR=SR=PRGsR−1, compute the signcryption cipherij=giw˜ijσijrRNmodN2.Sendcipherijj=1,2,…,T to the aggregation serverend function


#### 3.3.3. Aggregation Phase

(1)
Aggregation: after receiving the ciphertexts from all clients, the server aggregates them according to the layer indexjj=1,2,…,L. Because Paillier trapdoor permutation is additively homomorphic, the sever computes(2)cipherj=∏i=1ncipherij,=∏i=1ng1wij+sijg2xwij+sijrRN,=g1∑i=1nwij+∑i=1nsijg2x∑i=1nwij+x∑i=1nsijrRnNmodN2.Because∑i=1nsij=0 for i=1,2,…,n and j=1,2,…,|M0|, we have(3)cipherj=g1∑i=1nwijg2x∑i=1nwijrRnNmodN2.


#### 3.3.4. Update Phase

After receiving the aggregated ciphertextscipherjj=1,2,…,L, each client completes the update phase as follows.(1)
Decrypting: clienti firstly decrypts the j−th ciphertext:(4)Wj=LcipherjmodN2Lg1λmodN2modN2=∑i=1nwj+∑i=1nsj=∑i=1nwj.Then it computes the aggregated signature as follows:(5)cipher˜j=cipherjgi−WimodN,σj=cipher˜jN−1modλrR−nNmodN=g2x∑i=1nwijmodλmodN.(2)
Verifying: for each clienti, if the equation g2xWjmodλmodN≡σj holds, the aggregation results are correct. Otherwise, it is supposed to be falsified, and the learning process terminates.(3)
Unpacking: if the termination condition is not satisfied, clienti unpacks the plain aggregated gradients GR=∑i=1nGi=CRT_inverseW,m1,m2,⋯,mK, where W=W1,W2,…,WL=∑i=1nwi1,∑i=1nwi2,…,∑i=1nwiL.(4)
Updating: clienti updates the model parameters MR=MR−1−η/nGR and prepares for the next round of aggregation.We summarize the steps of update phase in Algorithm 2.Algorithm 2: Update phase of client i.
Input: Aggregated ciphertexts cipherjj=1,2,…,LOutput: Updated Model MRfunctionUpdateDecryptcipherjj=1,2,…,L to get W=Wjj=1,2,…,L=∑i=1nwij+sijj=1,2,…,L, compute cipher˜j=cipherjgi−WimodN and σj=cipher˜jN−1modλrR−nNmodN=g2x∑i=1nwijmodλmodNVerify the integrity of the aggregated gradients by checking if the equationg2xWjmodλmodN≡σj holds. If not, terminate the learning process.Decode the aggregated gradients to getGR=∑i=1nGi=DecodeW.Update model withMR=MR−1−η/nGR.end function


## 3.1. Threat Model and Design Goals

In our framework, we assume security threats from three different perspectives:(i)
Internal curious clients may try to steal privacy information about other clients by inspecting the ciphertexts transmitted during the training process. Here we should note that, because the ultimate goal of the clients in is to train a good ML model, the curious clients may collude with the server to steal private information about other clients, but they will not collude with the sever to tamper the aggregated results.(ii)
The server may try to steal the jointly trained model because of its great economic value. But the model should be the common property of the clients.(iii)
The server may return falsified aggregated results to clients driven by some unexpected motivations.Consequently, to enable a privacy preserving FL framework under the aforementioned threat model, the design goals are summarized as follows:(i)
Correctness: when the server and clients operate strictly according to the protocol, the aggregated results should be correct. This ensures that the clients achieve their goal and the final model has good performance.(ii)
Data privacy: data privacy means the privacy of a client’s training data and gradients. The server and curious clients cannot gather any private information about the data and gradients from messages they receive.(iii)
Model privacy: model privacy refers to the privacy of the jointly trained model. The server or any other party not in the framework cannot steal the model by inspecting the immediate data transmitted during the learning process.(iv)
Verifiability: all clients can verify the integrity of the aggregation of parameters so that clients can detect the malicious behavior if the server returns tampered aggregated results.


## 3.2. Overview

As illustrated in Figure1, our framework consists of 4 phases: initialization, model training, aggregation, and update. We summarize the 4 phases as follows:(i)
Initialization phase: the Public Key Generator (PKG) determines the model parameters and generates Paillier keypairs, Pseudo-Random GeneratorPRG, random seed S0, and K pairwise coprime large numbers mll=1,2,…,K for all clients.(ii)
Model training phase: all clients train the ML model locally clip and quantize the gradients intor bits long integers, pack them using CRT, mask them with blinding factors, and then signcrypt them.(iii)
Aggregation phase: the server aggregates the ciphertexts from all clients and distributes the result to them.(iv)
Update phase: each client firstly decrypts the aggregated ciphertexts and then verifies the integrity of the aggregated gradients. If the result is correct, it unpacks them into plain aggregated gradients and updates their local model accordingly. Otherwise, it would terminate the learning process.Figure 1
Overview of our framework.


## 3.3. Detailed Construction

### 3.3.1. Initialization

For simplicity, we assume there aren clients, and each client is uniquely indexed by a number i. The set of client index is represented by index=1,2,…,n. In our framework, the PKG needs to initialize the model parameters and generate keys and PRG as follows:(1)
Initializing model parameters: the PKG specifies the architecture of the model, randomly initializes the model parameters asM0=Mii=1,2,…,T (T denotes the number of layers of the learning model), and specifies an appropriate learning rate η.(2)
Specifying packing parameters: the PKG generatesK positive integers mii=1,2,…,K that are pairwise coprime gcdmi,mj=1∀i≠j. We define M=∏i=1kmi. These integers are used to pack gradients.(3)
Specifying blinding parameters: the PKG randomly choosesn integer sets s1jj=1,2,…,M0,s2jj=1,2,…,M0,…,snjj=1,2,…,M0∈ℤN∗M0 such that ∑i=1nsij=0 for j=1,2,…,M0 (M0 stands for the number of parameters of the model) and then sends sijj=1,2,…,M0 to client i.(4)
Specifying PRG: the PKG sends each client the same pseudorandom generatorPRG⋅ and seed S0. The pseudorandom generator is used to synchronize random numbers among clients for signcrypting the gradients in each training round.(5)
Generating signcryption keys: the PKG sends each client a Paillier key pairN,g1,λ for encryption and decryption and a secret value g2x to sign the gradients, where N=pq,g1∈ℤN2∗, g2∈ℤN∗, and x∈N.


### 3.3.2. Model Training Phase

The model training phase of clientii∈index consists of the following four steps:(1)
Local training: inR−th round of training, client i trains its local model on dataset Di and computes the gradients Gi.(2)
Packing: clienti sends the layer-wise maximum value Vij, minimum value vij, and the size sj of layer jj=1,2,…,T to the server. After receiving those layer-wise clipping parameters from all clients, the aggregator server calculates the clipping threshold αj=dACIQsj,MaxViji=1,2,…,n,inviji=1,2,…,n, where Max and Min compute the maximum and minimum of a set, respectively. After receiving the clipping thresholds, client i quantizes its gradients with Gi˜=QuantizeGi,αjj=1,2,…,T,n, client i partitions the quantized gradients into L=⌈Gi˜/K⌉ groups Pi=pi1=pi1ll=1,2,…,K,pi2=pi2ll=1,2,…,K,…,piL=piLll=1,2,…,K. If Gi˜ is not divisible by K, Gi˜ should be padded with 0 s. Then client i packs them into Wi=wi1,wi2,…,wiL with wij=CRTpij,mll=1,2,…,K and wij∈FM. After packing, the number of cryptographic operations and ciphertexts is greatly reduced; thus the computation and communication efficiency is improved.(3)
Blinding: to resist collusion attacks between curious clients and the aggregation server, clienti blinds wij by adding its blinding factors to get W˜i=w˜i1,w˜i2,…,w˜il=wi1+si1,wi2+si2,…,wiL+siL.(4)
Signcrypting: for each blinded gradientw˜ij∈W˜i, client i computes its signature σij=g2xw˜ijmodN and uses the PRG to synchronize with other clients a random number rR=SR=PRGSR−1 , where SR will be used as seed for the PRG in the next round of training. Finally, clients i compute a signcryption of w˜ij as cipherij=giw˜ijσijrRNmodN2.We summarize the steps of model training phase in Algorithm 1.Algorithm 1: Model training phase of client i.
Input: Training round R, model parameters MR−1, dataset Di,quantization bit width r, blinding factors sijj=1,2,…,M0, PRG, random seed SR−1, Paillier key pairs N,g1,λ, secret value g2xOutput: Signcryption of the masked gradients CipherijfunctionModelTrainingCompute gradientsGi based on MR−1 and DiSend layer-wise gradients Max-Min values and sizesVij,vij,sjj=1,2,…,T to the aggregation serverClipGi with corresponding threshold nαjj=1,2,…,T (Advance Scaling) and quantize them into r bitsBatch the quantized gradientsGi˜ layer by layer into Wi=wi1,wi2,…,wiL=EncodeGi˜BlindWi with sijj=1,2,…,M0 to compute W˜i=w˜i1,w˜i2,…,w˜il=wi1+si1,wi2+si2,…,wiL+siLSign each blinded gradientw˜ij∈W˜i with g2x, use the PRG to generate the synchronizing random number rR=SR=PRGsR−1, compute the signcryption cipherij=giw˜ijσijrRNmodN2.Sendcipherijj=1,2,…,T to the aggregation serverend function


### 3.3.3. Aggregation Phase

(1)
Aggregation: after receiving the ciphertexts from all clients, the server aggregates them according to the layer indexjj=1,2,…,L. Because Paillier trapdoor permutation is additively homomorphic, the sever computes(2)cipherj=∏i=1ncipherij,=∏i=1ng1wij+sijg2xwij+sijrRN,=g1∑i=1nwij+∑i=1nsijg2x∑i=1nwij+x∑i=1nsijrRnNmodN2.Because∑i=1nsij=0 for i=1,2,…,n and j=1,2,…,|M0|, we have(3)cipherj=g1∑i=1nwijg2x∑i=1nwijrRnNmodN2.


### 3.3.4. Update Phase

After receiving the aggregated ciphertextscipherjj=1,2,…,L, each client completes the update phase as follows.(1)
Decrypting: clienti firstly decrypts the j−th ciphertext:(4)Wj=LcipherjmodN2Lg1λmodN2modN2=∑i=1nwj+∑i=1nsj=∑i=1nwj.Then it computes the aggregated signature as follows:(5)cipher˜j=cipherjgi−WimodN,σj=cipher˜jN−1modλrR−nNmodN=g2x∑i=1nwijmodλmodN.(2)
Verifying: for each clienti, if the equation g2xWjmodλmodN≡σj holds, the aggregation results are correct. Otherwise, it is supposed to be falsified, and the learning process terminates.(3)
Unpacking: if the termination condition is not satisfied, clienti unpacks the plain aggregated gradients GR=∑i=1nGi=CRT_inverseW,m1,m2,⋯,mK, where W=W1,W2,…,WL=∑i=1nwi1,∑i=1nwi2,…,∑i=1nwiL.(4)
Updating: clienti updates the model parameters MR=MR−1−η/nGR and prepares for the next round of aggregation.We summarize the steps of update phase in Algorithm 2.Algorithm 2: Update phase of client i.
Input: Aggregated ciphertexts cipherjj=1,2,…,LOutput: Updated Model MRfunctionUpdateDecryptcipherjj=1,2,…,L to get W=Wjj=1,2,…,L=∑i=1nwij+sijj=1,2,…,L, compute cipher˜j=cipherjgi−WimodN and σj=cipher˜jN−1modλrR−nNmodN=g2x∑i=1nwijmodλmodNVerify the integrity of the aggregated gradients by checking if the equationg2xWjmodλmodN≡σj holds. If not, terminate the learning process.Decode the aggregated gradients to getGR=∑i=1nGi=DecodeW.Update model withMR=MR−1−η/nGR.end function


## 3.3.1. Initialization

For simplicity, we assume there aren clients, and each client is uniquely indexed by a number i. The set of client index is represented by index=1,2,…,n. In our framework, the PKG needs to initialize the model parameters and generate keys and PRG as follows:(1)
Initializing model parameters: the PKG specifies the architecture of the model, randomly initializes the model parameters asM0=Mii=1,2,…,T (T denotes the number of layers of the learning model), and specifies an appropriate learning rate η.(2)
Specifying packing parameters: the PKG generatesK positive integers mii=1,2,…,K that are pairwise coprime gcdmi,mj=1∀i≠j. We define M=∏i=1kmi. These integers are used to pack gradients.(3)
Specifying blinding parameters: the PKG randomly choosesn integer sets s1jj=1,2,…,M0,s2jj=1,2,…,M0,…,snjj=1,2,…,M0∈ℤN∗M0 such that ∑i=1nsij=0 for j=1,2,…,M0 (M0 stands for the number of parameters of the model) and then sends sijj=1,2,…,M0 to client i.(4)
Specifying PRG: the PKG sends each client the same pseudorandom generatorPRG⋅ and seed S0. The pseudorandom generator is used to synchronize random numbers among clients for signcrypting the gradients in each training round.(5)
Generating signcryption keys: the PKG sends each client a Paillier key pairN,g1,λ for encryption and decryption and a secret value g2x to sign the gradients, where N=pq,g1∈ℤN2∗, g2∈ℤN∗, and x∈N.


## 3.3.2. Model Training Phase

The model training phase of clientii∈index consists of the following four steps:(1)
Local training: inR−th round of training, client i trains its local model on dataset Di and computes the gradients Gi.(2)
Packing: clienti sends the layer-wise maximum value Vij, minimum value vij, and the size sj of layer jj=1,2,…,T to the server. After receiving those layer-wise clipping parameters from all clients, the aggregator server calculates the clipping threshold αj=dACIQsj,MaxViji=1,2,…,n,inviji=1,2,…,n, where Max and Min compute the maximum and minimum of a set, respectively. After receiving the clipping thresholds, client i quantizes its gradients with Gi˜=QuantizeGi,αjj=1,2,…,T,n, client i partitions the quantized gradients into L=⌈Gi˜/K⌉ groups Pi=pi1=pi1ll=1,2,…,K,pi2=pi2ll=1,2,…,K,…,piL=piLll=1,2,…,K. If Gi˜ is not divisible by K, Gi˜ should be padded with 0 s. Then client i packs them into Wi=wi1,wi2,…,wiL with wij=CRTpij,mll=1,2,…,K and wij∈FM. After packing, the number of cryptographic operations and ciphertexts is greatly reduced; thus the computation and communication efficiency is improved.(3)
Blinding: to resist collusion attacks between curious clients and the aggregation server, clienti blinds wij by adding its blinding factors to get W˜i=w˜i1,w˜i2,…,w˜il=wi1+si1,wi2+si2,…,wiL+siL.(4)
Signcrypting: for each blinded gradientw˜ij∈W˜i, client i computes its signature σij=g2xw˜ijmodN and uses the PRG to synchronize with other clients a random number rR=SR=PRGSR−1 , where SR will be used as seed for the PRG in the next round of training. Finally, clients i compute a signcryption of w˜ij as cipherij=giw˜ijσijrRNmodN2.We summarize the steps of model training phase in Algorithm 1.Algorithm 1: Model training phase of client i.
Input: Training round R, model parameters MR−1, dataset Di,quantization bit width r, blinding factors sijj=1,2,…,M0, PRG, random seed SR−1, Paillier key pairs N,g1,λ, secret value g2xOutput: Signcryption of the masked gradients CipherijfunctionModelTrainingCompute gradientsGi based on MR−1 and DiSend layer-wise gradients Max-Min values and sizesVij,vij,sjj=1,2,…,T to the aggregation serverClipGi with corresponding threshold nαjj=1,2,…,T (Advance Scaling) and quantize them into r bitsBatch the quantized gradientsGi˜ layer by layer into Wi=wi1,wi2,…,wiL=EncodeGi˜BlindWi with sijj=1,2,…,M0 to compute W˜i=w˜i1,w˜i2,…,w˜il=wi1+si1,wi2+si2,…,wiL+siLSign each blinded gradientw˜ij∈W˜i with g2x, use the PRG to generate the synchronizing random number rR=SR=PRGsR−1, compute the signcryption cipherij=giw˜ijσijrRNmodN2.Sendcipherijj=1,2,…,T to the aggregation serverend function


## 3.3.3. Aggregation Phase

(1)
Aggregation: after receiving the ciphertexts from all clients, the server aggregates them according to the layer indexjj=1,2,…,L. Because Paillier trapdoor permutation is additively homomorphic, the sever computes(2)cipherj=∏i=1ncipherij,=∏i=1ng1wij+sijg2xwij+sijrRN,=g1∑i=1nwij+∑i=1nsijg2x∑i=1nwij+x∑i=1nsijrRnNmodN2.Because∑i=1nsij=0 for i=1,2,…,n and j=1,2,…,|M0|, we have(3)cipherj=g1∑i=1nwijg2x∑i=1nwijrRnNmodN2.


## 3.3.4. Update Phase

After receiving the aggregated ciphertextscipherjj=1,2,…,L, each client completes the update phase as follows.(1)
Decrypting: clienti firstly decrypts the j−th ciphertext:(4)Wj=LcipherjmodN2Lg1λmodN2modN2=∑i=1nwj+∑i=1nsj=∑i=1nwj.Then it computes the aggregated signature as follows:(5)cipher˜j=cipherjgi−WimodN,σj=cipher˜jN−1modλrR−nNmodN=g2x∑i=1nwijmodλmodN.(2)
Verifying: for each clienti, if the equation g2xWjmodλmodN≡σj holds, the aggregation results are correct. Otherwise, it is supposed to be falsified, and the learning process terminates.(3)
Unpacking: if the termination condition is not satisfied, clienti unpacks the plain aggregated gradients GR=∑i=1nGi=CRT_inverseW,m1,m2,⋯,mK, where W=W1,W2,…,WL=∑i=1nwi1,∑i=1nwi2,…,∑i=1nwiL.(4)
Updating: clienti updates the model parameters MR=MR−1−η/nGR and prepares for the next round of aggregation.We summarize the steps of update phase in Algorithm 2.Algorithm 2: Update phase of client i.
Input: Aggregated ciphertexts cipherjj=1,2,…,LOutput: Updated Model MRfunctionUpdateDecryptcipherjj=1,2,…,L to get W=Wjj=1,2,…,L=∑i=1nwij+sijj=1,2,…,L, compute cipher˜j=cipherjgi−WimodN and σj=cipher˜jN−1modλrR−nNmodN=g2x∑i=1nwijmodλmodNVerify the integrity of the aggregated gradients by checking if the equationg2xWjmodλmodN≡σj holds. If not, terminate the learning process.Decode the aggregated gradients to getGR=∑i=1nGi=DecodeW.Update model withMR=MR−1−η/nGR.end function


## 4. Security Analysis

Theorem 1.
The privacy of the clients’ gradients is protected against internal curious clients.Proof.
Suppose an internal curious clienti intercepts a ciphertext from client l by intercepting the messages between client l and the server. Then client i has cipherlj=g1w˜ljσljrRNmodN2j=1,2,…,L. Though it can decrypt them to get w˜lj=wlj+slj and g2xwlj+slj, it cannot infer any information about the gradients wlj because the blinding factor slj is hidden from it. Therefore, our framework can protect the privacy of the clients’ gradients against internal curious clients.Theorem 2.
The privacy of the clients’ gradients is protected against collusion attack betweenn−2 clients.Proof.
Assume the set of collusion clients isAdv with |Adv|=2. For clients i,l∉Adv, their ciphertexts cipherij=g1w˜ijσijrRNmodN2 and cipherlj=g1w˜ljσljrRNmodN2j=1,2,…,L contain blinding factors sij and slj that are unknown to clients in Adv and the server. For any aggregated ciphertexts include cipherij and cipherlj, the situation is similar. Thus, even when the server colludes with n−2 clients, they cannot infer any information about other clients’ gradients.Theorem 3.
In our framework, each client can independently verify the correctness of the aggregated results, and the server cannot deceive the clients with tampered aggregated results.Proof.
If the clients receive the correct aggregated results, obviously the equationg2xWj=σjj=1,2,…,L holds. Assume a client receives forged aggregated results, and without loss of generality, we assume the sever falsified the aggregated results to(6)cipher^=g1Δ1g2Δ2∏i=1ncipherij=g1∑i=1nw˜ij+Δ1g2x∑i=1nw˜ij+Δ2rRnNmodN2.
If the malicious server tries to successfully fool the verification mechanism of our framework, it must make sure the following equation holds:g2∑i=1nxw˜ij+xΔ1=g2x∑i=1nw˜ij+Δ2; namely, the server should select Δ1 and Δ2 that satisfy xΔ1=Δ2. However, it is impossible because g2x is confidential to the server. Therefore, we prove that the server cannot deceive the clients with tampered aggregated results.


## 5. Performance Evaluation

We evaluate the performance of our framework in this section. Specifically, we first compare its accuracy against that of the original FL framework in four benchmark datasets: Federated MNIST, CINIC-10, CIFAR-10, and CIFAR-100. The models corresponding to the datasets are listed in Table2. Then we compare its computation and communication cost with that of the other two verifiable PPFL frameworks: VFL [25] and CRT-Paillier [24].Table 2
Datasets and corresponding models.
DatasetsModel structureModel parameters (#)Model size (MB)CINIC-10ResNet-5659132218.05CIFAR-100ResNet-5659132218.05Federated EMNISTCNN120659036.82CIFAR-10ResNet-18+ group normalization11232612342.79We should note that though ResNet-18 has less layers than ResNet-56; because of the group normalization operation, it has more parameters than ResNet-56.


### 5.1. Experimental Setup

Our evaluation experiments are conducted on a Dell T7920 workstation with 1 Intel Xeon Silver 4210R CPU and 32 GB RAM. The OS is Ubuntu 18.04. We employ the FedML [26] in its standalone simulation computing paradigms to build the baseline framework. In both our and CRT-Paillier frameworks, the open-sourced python-Paillier [27] is adopted as the Paillier HE implementation. In our experiments, according to VFL [25] and CRT-Paillier [24], the gradients in their frameworks are of 32-bit length, while the gradients in our framework are clipped and quantized into 16-bit length with the dACIQ and Quantize function. The Paillier key size in both our framework and CRT-Paillier is set to 2048 bits, just as recommended in NIST [28].


### 5.2. Discussion of Results

Accuracy: we compared the performance of our framework with that of the original FL framework (Figure2). For FMNIST, CINIC-10, CIFAR-10, and CIFAR-100, our framework can achieve an accuracy of 81.01%, 70.39%, 85.36%, and 65.60%, respectively, which is very close to that of the original FL, which has an accuracy of 80.99%, 70.49%, 85.81%, and 65.23%, respectively. The results demonstrate that our framework can guarantee the privacy of the learned model without compromising its performance. In fact, our quantized training sometimes has better results. Prior quantization work has observed a similar phenomenon [29], where the stochasticity introduced by quantization can reduce overfitting, similar to the function of dropout layer [30].Figure 2
Comparison of performance of original FL and our framework. (a) Accuracy on FMNIST, (b) accuracy on CINIC-10, (c) accuracy on CIFAR-10, and (d) accuracy on CIFAR-100.
(a)(b)(c)(d)Time cost, in Figure3, we compared the time cost of our framework with that of VFL and CRT-Paillier. In VFL, the degree of the interpolation polynomials is set to m=4 (VFL-4) and m=8 (VFL-8), respectively. It is known that the security level and cost of VFL are higher with larger m. In CRT-Paillier, because the plaintext in Paillier encryption should be less than the modulus N, the maximum group size is 60 to avoid overflow error. While the gradient in our framework quantized into 16 bits, the maximum batch size is 120.Figure 3
Comparison of time cost for VFL, CRT-Paillier, and our framework. (a) Encryption cost of a client, (b) decryption of a client, (c) total cost of a client, and (d) Aggregation cost of the server.
(a)(b)(c)(d)Figure3(a) shows that the encryption cost per client of all frameworks increases linearly with the amount of parameters. The encryption costs of our framework are 9.06 s for 2×105 parameters and 24.17 s for 11×105 parameters, respectively. In addition, the encryption costs of CRT-Paillier are 11.73 s and 35.55 s, VFL-8 are 33.83 s and 184.15 s, and VFL-4 are 12.19 s and 66.99 s. Though both our framework and CRT-Paillier employ Paillier encryption to protect privacy, our framework needs fewer encryption operations and takes less time because its batch size is larger. For VFL, though the interpolation is very fast, the parameter splitting costs are expensive. Thus, our framework is more efficient than VFL-8 and VFL-4.Figure3(b) presents the decryption cost of all frameworks. The decryption costs of our framework are 8.35 s for 2×105 parameters and 21.10 s for 11×105 parameters. The encryption costs of CRT-Paillier are 9.91 s and 43.21 s, VFL-8 are 2.22 s and 11.87 s, and VFL-4 are 0.91 s and 4.68 s. Because our framework and CRT-Paillier utilize the Paillier scheme to protect privacy, which involves exponentiations and modular multiplications with large integers, the decryption costs of both our framework and CRT-Paillier are a little higher than VFL. But our framework takes less time than CRT-Paillier because of its larger batch size.Figure3(c) displays the total time cost of a client. From the figure we know that the total cost of a client of our framework is less than both VFL and CRT-Paillier, and the gap grows linearly with the number of parameters. We should note that, as giant companies or organizations prefer large deep learning models to boost their performance, our framework is more suitable for large AI models.For the server, the overhead is caused by secure aggregation. Figure3(d) shows the aggregation cost of the server. Because of larger batching size, our framework needs fewer sum operations on the ciphertexts. Thus, our framework is more efficient than CRT-Paillier and VFL for the server.Communication cost: we compared the communication cost of our framework with that of VFL and CRT-Paillier in Figure4. Since we simulate different FL frameworks in the standalone computing paradigms of FedML, we use the ciphertext size exchanged between the server and clients as the metric for communication cost. In VFL-4, each gradient is randomly split into 4 parameters. Though the authors keep the number of parameters the same as the original model by employing CRT to batch parameters, the size of the batched results grows accordingly. Thus, the ciphertext expansion rate for VFL-4 is 4. Similarly, the ciphertext expansion factor of VFL-8 is 8. In CRT-Paillier framework, every 60 gradients are grouped together and then are encrypted to get a 2048×2 bit-length ciphertext, the ciphertext expansion factor is approximately 2048×260×32≈2.13. Similarly, the ciphertext expansion factor of our framework is approximately 2048×2120×32≈1.07. Therefore, our framework is more communication efficient than VFL and CRT-Paillier.Figure 4
Comparison of communication cost of VFL, CRT-Paillier, and our framework. (a) Traffic of a client; (b) traffic of the server.
(a)(b)


## 5.1. Experimental Setup

Our evaluation experiments are conducted on a Dell T7920 workstation with 1 Intel Xeon Silver 4210R CPU and 32 GB RAM. The OS is Ubuntu 18.04. We employ the FedML [26] in its standalone simulation computing paradigms to build the baseline framework. In both our and CRT-Paillier frameworks, the open-sourced python-Paillier [27] is adopted as the Paillier HE implementation. In our experiments, according to VFL [25] and CRT-Paillier [24], the gradients in their frameworks are of 32-bit length, while the gradients in our framework are clipped and quantized into 16-bit length with the dACIQ and Quantize function. The Paillier key size in both our framework and CRT-Paillier is set to 2048 bits, just as recommended in NIST [28].


## 5.2. Discussion of Results

Accuracy: we compared the performance of our framework with that of the original FL framework (Figure2). For FMNIST, CINIC-10, CIFAR-10, and CIFAR-100, our framework can achieve an accuracy of 81.01%, 70.39%, 85.36%, and 65.60%, respectively, which is very close to that of the original FL, which has an accuracy of 80.99%, 70.49%, 85.81%, and 65.23%, respectively. The results demonstrate that our framework can guarantee the privacy of the learned model without compromising its performance. In fact, our quantized training sometimes has better results. Prior quantization work has observed a similar phenomenon [29], where the stochasticity introduced by quantization can reduce overfitting, similar to the function of dropout layer [30].Figure 2
Comparison of performance of original FL and our framework. (a) Accuracy on FMNIST, (b) accuracy on CINIC-10, (c) accuracy on CIFAR-10, and (d) accuracy on CIFAR-100.
(a)(b)(c)(d)Time cost, in Figure3, we compared the time cost of our framework with that of VFL and CRT-Paillier. In VFL, the degree of the interpolation polynomials is set to m=4 (VFL-4) and m=8 (VFL-8), respectively. It is known that the security level and cost of VFL are higher with larger m. In CRT-Paillier, because the plaintext in Paillier encryption should be less than the modulus N, the maximum group size is 60 to avoid overflow error. While the gradient in our framework quantized into 16 bits, the maximum batch size is 120.Figure 3
Comparison of time cost for VFL, CRT-Paillier, and our framework. (a) Encryption cost of a client, (b) decryption of a client, (c) total cost of a client, and (d) Aggregation cost of the server.
(a)(b)(c)(d)Figure3(a) shows that the encryption cost per client of all frameworks increases linearly with the amount of parameters. The encryption costs of our framework are 9.06 s for 2×105 parameters and 24.17 s for 11×105 parameters, respectively. In addition, the encryption costs of CRT-Paillier are 11.73 s and 35.55 s, VFL-8 are 33.83 s and 184.15 s, and VFL-4 are 12.19 s and 66.99 s. Though both our framework and CRT-Paillier employ Paillier encryption to protect privacy, our framework needs fewer encryption operations and takes less time because its batch size is larger. For VFL, though the interpolation is very fast, the parameter splitting costs are expensive. Thus, our framework is more efficient than VFL-8 and VFL-4.Figure3(b) presents the decryption cost of all frameworks. The decryption costs of our framework are 8.35 s for 2×105 parameters and 21.10 s for 11×105 parameters. The encryption costs of CRT-Paillier are 9.91 s and 43.21 s, VFL-8 are 2.22 s and 11.87 s, and VFL-4 are 0.91 s and 4.68 s. Because our framework and CRT-Paillier utilize the Paillier scheme to protect privacy, which involves exponentiations and modular multiplications with large integers, the decryption costs of both our framework and CRT-Paillier are a little higher than VFL. But our framework takes less time than CRT-Paillier because of its larger batch size.Figure3(c) displays the total time cost of a client. From the figure we know that the total cost of a client of our framework is less than both VFL and CRT-Paillier, and the gap grows linearly with the number of parameters. We should note that, as giant companies or organizations prefer large deep learning models to boost their performance, our framework is more suitable for large AI models.For the server, the overhead is caused by secure aggregation. Figure3(d) shows the aggregation cost of the server. Because of larger batching size, our framework needs fewer sum operations on the ciphertexts. Thus, our framework is more efficient than CRT-Paillier and VFL for the server.Communication cost: we compared the communication cost of our framework with that of VFL and CRT-Paillier in Figure4. Since we simulate different FL frameworks in the standalone computing paradigms of FedML, we use the ciphertext size exchanged between the server and clients as the metric for communication cost. In VFL-4, each gradient is randomly split into 4 parameters. Though the authors keep the number of parameters the same as the original model by employing CRT to batch parameters, the size of the batched results grows accordingly. Thus, the ciphertext expansion rate for VFL-4 is 4. Similarly, the ciphertext expansion factor of VFL-8 is 8. In CRT-Paillier framework, every 60 gradients are grouped together and then are encrypted to get a 2048×2 bit-length ciphertext, the ciphertext expansion factor is approximately 2048×260×32≈2.13. Similarly, the ciphertext expansion factor of our framework is approximately 2048×2120×32≈1.07. Therefore, our framework is more communication efficient than VFL and CRT-Paillier.Figure 4
Comparison of communication cost of VFL, CRT-Paillier, and our framework. (a) Traffic of a client; (b) traffic of the server.
(a)(b)


## 6. Conclusion

In this paper, we have designed a preserving FL framework for IoT based on the homomorphic signcryption mechanism we designed. In our framework, each client can aggregate the gradients securely and verify the integrity of the aggregated results. Besides, our framework can also resist the collusion attacks between the server and at mostn−2 clients. Finally, experiments on four benchmark datasets show that our framework can protect the privacy and integrity of the learned model while guaranteeing its performance, and our framework is more efficient in computation and communication than existing similar frameworks. In future work, we will try to design more flexible privacy preserving framework that allows dynamic joining in and dropping out of clients.



---

*Source: 8380239-2022-09-22.xml*