# Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning
## 3 Method and Algorithm
### Paillier Federated Multi-Layer Perceptron (PFMLP)  
The basic structure of PFMLP is quite similar to FMLP. Since PFMLP needs to interact with KMC, the learning client should send a request to the KMC before training starts. The KMC confirms that each participant is online, and then generates key pairs and returns  
Figure 4: The architecture of a Paillier federated network. them to learning clients. After getting the key pairs, each learning client performs multi-party machine learning based on encrypted data. The flow chart of PFMLP is shown in Figure 5.  
Compared with FMLP, PFMLP adds another three parts: (1) Encryption and decryption operations in the learning clients; (2) Homomorphic operations in the computing server; and (3) Generation and distribution of key pairs in the key management center (KMC). In PFMLP, it contains learning clients, computing server, and KMC. The algorithm for learning clients is shown in Algorithm 2. ```
0: Dataset \(x\)
0: Model \(\theta_{final}\)
1: Request key pairs from KMC
2: Initialize the model parameters \(\theta\)
3:for\(i\) in iterationdo
4: Forward propagation: \(out_{i}=fp(x_{i},\theta_{i})\);
5: Compute loss: \(c_{i}=loss(f^{*}(x_{i}),out_{i})\);
6:if\(c_{i}<c\)then
7: Break
8:else
9: Back propagation: \(grad_{i}=bp(x_{i},\theta_{i},c_{i})\);
10: Use public key of client i to encrypt the gradient: \(Enc(grad_{i})=Enc_{\textit{Paillier}}(Public_{key},grad_{i})\);
11: Send \(Enc(grad_{i})\) to the computing server and receive: \(Enc(grad_{i_{max}})\);
12: Use private key of client i to decrypt the gradient: \(grad_{i}=Dec_{\textit{Paillier}}(Private_{key},Enc(grad_{i}))\);
13: Update: \(\theta_{i+1}=\theta_{i}-lr*grad_{\textit{new}}\);
14:endif
15:endfor
16:return the model with parameters \(\theta_{final}\);
```  
**Algorithm 2** PFMLP in the Learning Client  
The learning client does not immediately update the local model after calculating the gradient for each learning iteration. It homomorphically encrypts the gradient data and transmits it to the computing server, and then waits for the server to return the new encrypted gradient data after doing homomorphic operation. For the decryption phase, once the client decrypts the new encrypted gradient data, it can update the local model of each learning client with the new gradient. Thus, the new gradient contains other client's private data implicitly, in order to protect the data privacy indirectly. Figure 5: The flow chart of the Paillier federated multi-layer perceptron algorithm. Since PFMLP performs Paillier encryption on the gradient data, even if the computing server is compromised by a cracker, the leaked data only show the encrypted gradient data \(Enc(grad)\). Thus, the threat of inference attacks can be avoided. The algorithm for KMC is shown in Algorithm 3. In addition, its main functions are generating and distributing key pairs. That is, when it receives a request from a learning client, it generates a key pair and distributes it to the client. ```
1:requests
2:KeyPair
3:while listening request from Clients do
4:if receive a request from a client then
5: Generate a KeyPair;
6:return a KeyPair to the learning client;
7:endif
8:endwhile
```  
**Algorithm 3** PFMLP in KMC  
For the computing server, it performs homomorphic operations on the encrypted gradient data provided by each learning client. In addition, when the computing server receives a request from a client, it performs homomorphic operations on the encrypted data and returns the results to the client. The PFMLP algorithm for computing server is shown in Algorithm 4. Since the computing server does not obtain the key throughout the whole process, all of these guarantee the data privacy during the model training.