# FREDY: Federated Resilience Enhanced with Differential Privacy
## 2 Related Work and Backround
### Knowledge Transfer Frameworks  
The concept of PATE (Private Aggregation of Teacher Ensembles), introduced in [29], revolves around safeguarding the differential privacy of machine learning. The central idea posit that when two independently trained classifiers converge on a classification verdict, the decision itself does not divulge information about any individual training instance. This strategy employs a teachers-student model architecture, specifically designed to uphold the confidentiality of each entity's data. Numerous teacher models undergo training using distinct datasets, which could be sensitive, like medical records. Conversely, the student model is trained on publicly available data, encompassing labels generated through a noisy (Laplacian) voting process among all teachers. Subsequently, an aggregated teacher assimilates the predictions (votes) from each trained teacher concerning the student's unlabeled data for each sample and introduces random noise. The resultant most frequently endorsed predictions serve as labels to train the student model. Consequently, the student model, which might eventually become publicly accessible, internalizes consolidated insights drawn from teachers' private data, founded on their collective input, all while preserving their privacy.  
However, while PATE operates within a transfer learning framework, it overlooks the scenario of federated learning, where safeguarding privacy against malicious participants is paramount. In [48], the authors propose an innovative approach called FL-PATE that bridges the PATE transfer learning technique with the principles of federated learning. The FL-PATE framework unfolds in two distinct stages. Initially, teacher models undergo training via federated learning, executed across participant groups, and further fortified with differential privacy measures by introducing Gaussian noise to a dedicated layer within the teacher models, i.e., the last fully connected layer within a ResNet18 network. This strategic noise addition substantially mitigates its impact on model accuracy. Subsequently, the second stage involves training the student model using public datasets, the labels of which result from aggregating the outputs of the teacher models.  
Nevertheless, the original concept presented in [29] introduces PATE as a method for maintaining privacy, while FL-PATE employs PATE solely as a transfer learning approach, devoid of its privacy-preserving aspect. This deviation becomes evident through FL-PATE's utilization of Gaussian noise augmentation on the collective model's weights during the federated learning training phase, whereas PATE relies on Laplacian noise addition to the consensus of teacher predictions. In light of this, we put forward FREDY, an innovative federated learning framework that harnesses PATE as a central architecture, seamlessly integrating both transfer learning and privacy preservation techniques. That said, FREDY employs PATE's original Laplacian noise addition on the most voted teacher predictions to achieve privacy protection while simultaneously enhancing transfer learning. This distinctive dual utilization of PATE in FREDY sets it apart from both PATE and FL-PATE, making it a unique and promising approach that encompasses both transfer learning and privacy preservation within the federated learning paradigm.