# Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning
## 4 Experiments and Results Analysis
### Accuracy Comparison  
For comparison, the PFMLP and MLP algorithms use the same network structure for model training while learning the same dataset. Supposing that there are two learning clients, we split each dataset into two subsets and distribute them to two learning clients.  
For the MNIST dataset, we select the first 4000 data as the train-set \(D^{mnist}\), and then divide the 4000 pieces of data into two parts, respectively: \(D^{mnist}_{I}=\{D^{mnist}_{1},\cdots,D^{mnist}_{200}\}\), \(D^{mnist}_{II}=\{D^{mnist}_{201},\cdots,D^{mnist}_{400}\}\). The testing data use 10,000 testing sets provided by MNIST.  
Meanwhile, we select 400 pieces of data from the metal fatigue strength dataset, and divide them into two equal subsets for model training. Supposing that the original data are \(D^{Fatigue}\), randomize the original data noted as \(D^{Fatigue^{\prime}}=random(D^{Fatigue})\). The two sub-datasets are \(D^{Fatigue^{\prime}}_{I}=\{D^{Fatigue^{\prime}}_{1},\cdots,D^{Fatigue^{\prime} }_{200}\}\), \(D^{Fatigue^{\prime}}_{II}=\{D^{Fatigue^{\prime}}_{201},\cdots,D^{Fatigue^{ \prime}}_{400}\}\). In addition, then we use 70% as the training set and the remaining 30% as the testing set. In addition, the experimental result is shown in Table 4.  
From Table 4, it shows that the model trained by PFMLP is more accurate than that of local MLP. The final model trained by PFMLP is almost equivalent to or even better than that of the model trained by MLP using all data from each client.  
The experiments on the MNIST dataset show that the model trained by PFMLP can reach an accuracy rate of 0.9252 on the testing set, while the model trained by MLP using all of the data of training set can reach an accuracy rate of 0.9245, just 0.007 lower than that of the PFMLP algorithm.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Data Subset** & **Algorithm** & **Accuracy** \\ \hline \multirow{4}{*}{MNIST} & \(D^{mnist}_{1}\) & MLP & 0.8333 \\  & \(D^{mnist}_{2}\) & MLP & 0.9033 \\  & \(D^{mnist}_{1}\) & MLP & 0.9245 \\  & \(D^{mnist}_{2}\) & PFMLP & 0.9252 \\ \hline \multirow{4}{*}{Fatigue} & \(D^{Fatigue^{\prime}}_{1}\) & MLP & 0.9013 \\  & \(D^{Fatigue^{\prime}}_{2}\) & MLP & 0.7833 \\ \cline{1-1}  & \(D^{Fatigue^{\prime}}_{1}\) & MLP & 0.8583 \\ \cline{1-1}  & \(D^{Fatigue^{\prime}}_{2}\) & PFMLP & 0.8167 \\ \cline{1-1}  & \(D^{Fatigue^{\prime}}_{2}\) & PFMLP & 0.8500 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison results of prediction accuracy of MLP and PFMLP on two datasets.  
Figure 6: The network deployment in the experimental environment.  
For the metal fatigue strength dataset, since models on each client learned from the same PFMLP, we can perform a weighted average of the results of the two experiments based on the amount of the testing set, and get a final prediction accuracy rate of 0.85. Compared with the MLP model with an accuracy rate of 0.858 after learning all the data, the accuracy rate has only decreased by 0.008.  
Therefore, from the experimental results on two datasets, it shows that the PFMLP algorithm can train a model with almost the same accuracy rate as the MLP on all data from multiple parties. Detailed results are shown in Figure 7.