# FREDY: Federated Resilience Enhanced with Differential Privacy
## 6 Conclusions  
PATE is a powerful privacy-preserving technique that enables the training of teacher models on local sensitive data whose outputs in public unlabeled data are noisy aggregated, and the final results act as labels and thus train a student model on these public data. Only the student model is released to the public, and therefore each teacher's sensitive data are kept private. However, PATE trains the teacher models sequentially and does not consider the federated learning scenario. In this paper, we proposed FREDY, a federated learning privacy-preserving framework that uses knowledge transfer from private data. FREDY trains teacher models in a federated manner on local sensitive data using the standard FedAvg algorithm as the aggregation function. At the last round of the federated training procedure, the teacher models do not upload their weights to the server and keep them to perform inference on public unlabeled data. The outputs of each teacher model's inference are noisy aggregated by adding Laplacian noise and keeping the most voted classes for each public unlabeled data point. The final aggregated labels along with the corresponding public data are used to train a student model. Similar to PATE, only the student model is available to the public community, while the teacher models are kept private. We show that our final student model is not vulnerable to membership inference attacks, making an attack model unable to determine whether a data point is part of the training data of the teacher models or not. In doing so, FREDY fortifies resilience not only against external threats, but also within its internal mechanisms, culminating in a comprehensive approach to maintaining data privacy and model integrity.  
Conceptualization, Z.A., T.-H.V., S.B. and T.Z.; methodology, Z.A., T.-H.V., A.V., S.B. and D.S.; software, Z.A., D.S. and K.P.; validation, Z.A., T.-H.V., A.V. and S.B.; formal analysis, Z.A., T.-H.V., A.V., S.B. and D.S.; investigation, Z.A., T.-H.V., A.V. and S.B.; writing--original draft preparation, Z.A., T.-H.V., A.V. and S.B.; writing--review and editing, Z.A., T.-H.V., A.V., S.B., K.P., D.S. and T.Z.; visualization, Z.A., T.-H.V., A.V., S.B. and K.P.; supervision, T.-H.V., A.V. and T.Z.; project administration, A.V. and T.Z.; funding acquisition, T.Z. All authors have read and agreed to the published version of the manuscript.  
The work presented in this document has received funding from the EU Horizon Europe research and innovation Programme under Grant Agreement No. 101070118.  
The datasets used for this article is available online at [[https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)]([https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)) and [[http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)]([http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)). Accessed on 10 June 2022.  
We acknowledge the equal contribution of all the authors.  
The authors declare no conflict of interest.  
The following abbreviations are used in this manuscript:  
\begin{tabular}{l l l} CNN & Convolutional Neural Network \\ FL & Federated Learning \\ IoT & Internet of Things \\ ML & Machine Learning \\ \end{tabular}  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & \(\epsilon\) & **Queries** &
\begin{tabular}{c} **Student** \\ **Convergence Time (s)** \\ \end{tabular} \\ \hline Baseline & - & - & 32.9 \\ \hline FREDY (ours) & 0.2 & 2000 & 33.6 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Computational overhead of FREDY with respect to student convergence time.