Communication Efficient and Secure Federated Learning
Creators: name: ZHU, HANGYU
Publisher: name: University of Surrey
Year: 2021
Subjects: Federated Learning
Communication Efficient
Privacy Preservation
Homomorphic Encryption
Evolutionary Optimization
Machine Learning

description: Standard centralized machine learning applications require the participants to upload
description: their personal data to a central cloud for model training, which significantly harms
description: the users’ privacy. And federated learning is an emerging privacy preservation machine
description: learning paradigm proposed to alleviate this issue. It is inherently a distributed machine
description: learning framework and enables multiple users to collaboratively train a global model
description: without sharing their local training data, thus preventing individuals’ data from being
description: revealed.
description: However, federated learning often consumes much more communication resources than
description: centralized learning, since the model parameters are required to be uploaded and downloaded between clients and the server during the training process. To mitigate this issue,
description: we propose a multi-objective evolutionary federated learning framework to reduce the
description: communicated model size without obvious performance drop, thus, reducing the communication costs in federated learning. In addition, the modified SET algorithm is
description: adopted to improve the encoding scalability and further reduce the model size. This
description: approach is intrinsically an offline evolutionary optimization framework, and the performance of the reinitialized models at each generation will dramatically degrade, making
description: the offline optimization approach infeasible for real-time federated learning systems.
description: Therefore, we extend our previous offline method to propose a novel real-time federated evolutionary neural architecture search (NAS) framework named RT-FedEvoNAS.
description: By real-time or online federated evolutionary NAS, we mean that the neural network
description: models are already in use during the search process and each client only trains one
description: sub-model within one generation with the help of the proposed double sampling strategy. In addition, all the searched neural network models at the last generation are
description: already well trained and have no need to be re-trained from scratch again, making the
description: communication costs kept minimum.
description: Another challenge is that, federated learning cannot fully guarantee the local privacy
description: and the training data still have potential risk to be disclosed through the uploaded model parameters. To address this concern, homomorphic encryption (HE) is commonly
description: applied in federated learning by encrypting the model parameters on each client before
description: sending them to the server. However, most HE-based FL systems are not efficient enough and require a thrusted third party for key pair generation. Therefore, we propose
description: a distributed additive encryption and quantization federated deep learning framework,
description: in which the key pairs are generated between the server and clients without the help of
description: an extra thrusted third party. In addition, ternary gradient quantization and aggregation approximation strategy are adopted to simultaneously reduce the communication
description: and local computational costs.
description: Different from aforementioned methods focused on training parametric models in horizontal federated learning, our last work is located at learning non-parametric models in
description: vertical federated learning. Non-parametric models like gradient boosting decision trees
description: (GBDTs) are commonly used in the previous work of federated learning for vertically
description: partitioned data. But these approaches assume that all the training data labels are
description: stored on only one (guest) client, which is not applicable in the real world applications.
description: Therefore, we propose a secure vertical federated learning framework to train GBDT
description: with data labels distributed on multiple devices. A novel secure protocol is proposed
description: by setting source client and split client for node split, thus, preventing both data features and labels from being disclosed. Moreover, a partial differential privacy scheme
description: is introduced to add Gaussian noise upon the leaf weights before sending them to the
description: source clients for prediction protection.
description: All the proposed methods are empirically proven using both benchmark datasets and
description: real-world datasets. And the experimental results show that our approaches introduced
description: in this thesis work properly for building a secure and efficient federated learning system.