Federated Learning for Preserving Data Privacy
Francisco Herrera
Andalusian Research Institute in Data Science and Computational Intelligence
University of Granada - Spain
herrera@decsai.ugr.es

Outline•Artificial intelligence challenges•Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)•Federated Learning workflow•Federated Learning libraries•Case of study•Adversarial attacks. Proposal and Case of study•Concluding remarks. What’s next?

Artificial Intelligence Challenges
VolumeAI
PrivacyDistribution

Data
Integrity

Artificial Intelligence Challenges: Volume of data
• Current AI models, and specially deep learning models, arefuelled by vast amount of data.
• Although fine tuning reduces the amount of training data, westill need pre-trained models on big datasets.
• For instance, the amount of data and computational resourcesof some current language models for NLP are:

Model

Data

TokensParameters

BERT-base

13GB

250 billions

110M

GPT-3

570GB

300 billions

175M

Artificial Intelligence Challenges: Volume of data, different sources
Case of use• Data with sensitive information, as: emails, personalised recommendations or

health records. The data must be kept in their corresponding data owner silo.
• Data is stored in data silos, as the data stored by the healthcare industry.
• Data protected by legal regulations, as data from banks, telecom or hospitals

that cannot be shared.

Artificial Intelligence Challenges: Volume of data, different sources
Case of use• Data with sensitive information, as: emails, personalised recommendations or

health records. The data must be kept in their corresponding data owner silo.
• Data is stored in data silos, as the data stored by the healthcare industry.
• Data protected by legal regulations, as data from banks, telecom or hospitals

that cannot be shared.

These use cases would benefit from learning models fromdata silos, sharing models instead of dataArtificial Intelligence Challenges: Volume of data, different sources• Possible solution: Distributed Machine Learning• Weaknesses:• Communication costs. That huge amount of data must be communicated among thenodes of the distributed machine learning setting.
• Latency time. The latency time proportionally increase to the amount of data.
• Storage capacity. The storage of so huge amount of data requires the use of so muchstorage nodes, which makes wider the communication costs and latency time.
• Computation bottleneck. In those situations where the communication costs may beafforded, the bottleneck of the setting is on the computation time.

Artificial Intelligence Challenges
VolumeAI
Distribution

Privacy

Data
Integrity

Artificial Intelligence Challenges: Data Distribution
Independent and identically distributed randomvariables (IID)

same probability
distribution

Non-Independent and identically distributed randomvariables (Non-IID)

Artificial Intelligence Challenges: Data Distribution
Non-Independent and identically distributed random variables (Non-IID)• In a distributed scenario, we can set an IID distribution if we have access to the data.
• If the data is distributed in several data silos or devices (clients), we cannot make the
IID assumption [2,3].
• If we have a set of clients, let me consider:• Q: Distribution over the set of clients that own a set of data points (dataset).
• 𝑃𝑖 (𝑥, 𝑦): Local data distribution from the client i.

• If we consider each dataset (silo or device) as a random variable, we say that they

follow a Non-IID if they do not follow the same distribution. Or in other words, if there
are differences between 𝑃𝑖 and 𝑃𝑗 for different clients i and j (𝑃𝑖 ≠ 𝑃𝑗 ).
• [2] Sattler, F., Wiedemann, S., Müller, K. R., & Samek, W. (2019). Robust and communication-efficient federated learning from non-iid data. IEEE

transactions on neural networks and learning systems, 31(9), 3400-3413.
• [3] Hsieh, K., Phanishayee, A., Mutlu, O. & Gibbons, P.. (2020). The Non-IID Data Quagmire of Decentralized Machine Learning. Proceedings ofthe 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research119:4387-4398

Artificial Intelligence Challenges
VolumeAI
PrivacyDistribution

Data
Integrity

Artificial Intelligence Challenges: Privacy• Currently, people are increasingly aware about the relevance of preserving theprivacy and ownership of their personal data.
• People are demanding more AI-based services, but at the same time they are

reluctant to share their personal data.
• Likewise, several legal frames are coming up to regulate how data should bemanaged, shared and used, for instance the GDPR regulation of the EU.
• These new legislative landscape and the incessant growing concern on preserving

data privacy make more difficult to develop distributed machine learning
settings, which may hinder the progress of data-based AI.

Artificial Intelligence Challenges
VolumeAI
PrivacyDistribution

Data
Integrity

Artificial Intelligence Challenges: Data Integrity• The standard distributed machine learning requires to transfer data amongnodes.
• This exposes the data to be used by malicious agents to corrupt the learning modelby corrupting them.
• Likewise, the transferring of data may result in data leaks.
• Current AI recommendations as Trustworthy AI [4] asserts that AI systems

have to preserve data from illegitimate access.

• [4] European Commission. High-Level Expert Group on Articial Intelligence: Ethics guidelines for trustworthy AI (2019).

Artificial Intelligence Challenges
The standard AI-based systems or machine learning settings needs to:
1. Work with a vast amount of distributed sensitive data stored.

2. Work with data that follows a Non-IID distribution.
3. Preserve data privacy.
4. Protect the integrity of data and the learning model from malicious attacks.

Artificial Intelligence Challenges
Federated Learning
VolumeAI
PrivacyDistribution

Data
Integrity

Outline• Artificial intelligence challenges• Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)• Federated Learning workflow• Federated Learning libraries• Case of study• Adversarial attacks. Proposal and Case of study• Concluding remarks. What’s next?

Federated Learning. Definition
Federated Learning (FL) is a machine learning setting wheremultiple entities (clients) collaborate in solving a machinelearning problem, under the coordination of a central serveror service provider.
Each client’s raw data is stored locally and not exchanged ortransferred; instead local learning focused updates intended
for immediate aggregation are used to achieve the learning
objective [5].
• Key points:• Raw data is stored locally → Its privacy is protected.
• Raw data is not transferred → It prevents data leaks.

• Learning local models, aggregation of local models to get a global model[5] Kairouz, P. and McMahan, H. B. Advances and open problems in federated learning. Foundations andTrends® in Machine Learning, 14(1), 2021.

Federated Learning. Definition• In FL, there are two kind of nodes or clients [6]:• Data owners nodes {𝐶1 , 𝐶2 … , 𝐶𝑛 }: They own the raw data, i.e. the local datasets{𝐷1 , 𝐷2 … , 𝐷𝑛 }.

• Aggregation nodes {𝐺1 , 𝐺2 … , 𝐺𝑛 }: They aggregate the local learning models tolearn a global learning model from the data kept in the data owners node.

• FL aim. Learning a global learning model (𝜃, GLM) based on theaggregation of the local learning models (𝜃𝑖 , 𝐿𝐿𝑀𝑖 ) through several
rounds of learning.

[6] Rodríguez-Barroso, N., Stipcich, G., Jiménez-López, D., Ruiz-Millán, J. A., Martínez-Cámara, E., ... & Herrera,
F. (2020). Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework andmethodological guidelines for preserving data privacy. Information Fusion, 64, 270-292.

Federated Learning. Definition – Rounds of Learning• It is an iterative learning process composed of the following steps:
1.Each client i trains its 𝐿𝐿𝑀𝑖 on its local training data 𝐷𝑖𝑡 and it updates

the parameters of the 𝐿𝐿𝑀𝑖 , 𝜃𝑖𝑡 .

2.The clients send their updated parameters 𝜃𝑖𝑡 to the server.
3.The server computes the global parameters 𝜃 𝑡 by aggregating the local

parameters {𝜃1𝑡 , 𝜃2𝑡 , … , 𝜃𝑛𝑡 } of all the 𝐿𝐿𝑀𝑖 using an specific federated
aggregation operator Δ, formally𝜃 𝑡 = Δ 𝜃1𝑡 , 𝜃2𝑡 , … , 𝜃𝑛𝑡 .
4.After the aggregation, the LLM are updated with the aggregatedparameters:

𝜃𝑖𝑡+1 ← 𝜃𝑖𝑡 , ∀𝑖 ∈ {1, … , 𝑛}Federated Learning. Definition – Rounds of Learning

Source: https://blogs.aca-it.be/machine-learning-privacy/federated-learning/Federated Learning. Definition – Learning goal
• The rounds of learning are repeated until reach the learning goal.
• The learning goal is to minimise the following objective function:𝑛min 𝐹 𝜃 , 𝑤𝑖𝑡ℎ 𝐹 𝜃 ≔ ෍ 𝑤𝑖 𝐹𝑖 (𝜃)𝜃

𝑖=1Where:• n is the number of nodes or clients.
• 𝐹𝑖 is the local objective function for the i-th client fitted to each client’s
data.
• 𝑤𝑖 ≥ 0; σ𝑖 𝑤𝑖 = 1.

Federated Learning. Key Elements• FL is a specific machine learning setting, which incorporates some elements to thestandard workflow. In particular:• Data. It plays a central role in FL as in machine learning. The data stored in eachclient can follow a IID or a Non-IID distribution. In real FL scenarios, the Non-IID
distribution is the most likely.
• Learning model. It is composed of:• The Local Learning Models (LLM), which are locally trained in each client.
• The Global Learning Model (GLM), which is obtained by aggregating theparameters of LLMs. Hence, the GLM is not trained.

Federated Learning. Key Elements• Federated Aggregation Operator. It is the responsible of aggregating theparameters of the LLM in the server. It has to match the following requirements:
1. To assure a proper fusion of the LLM to optimise the objective function.
2.To facilitate a fast convergence of the objective function for reducing the numberof communication rounds among the clients and the server.

3.To be resilient and robust against clients with poor data quality andadversarial clients for preserving data integrity.

Federated Learning. Key Elements
Federated Averaging (FedAVG) [7]• So far, there are several aggregation

operators to optimise the objective functionof the GLM.
• The first one and the most used is FedAVG,

which is designed to optimise non-convex
objective functions commonly seen whentraining deep learning methods.

• [7] B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas,

Communication- Efficient Learning of Deep Networks from Decentralized
Data, in: Proceedings of the 20th International Conference on Artificial
Intelligence and Statistics, volume 54, 2017, pp. 1273–1282.
• Image source [5].

Federated Learning. Key Elements• Clients. They represent the nodes in a FL setting.
• The clients may be: IoT devices, mobile phones, self-driven cars, organisations.

Their main characteristic is that they own the data, and the data is not shared withany other node of the FL setting.
• Depending on the nature of the clients, we have:
Cross-device FL

Cross-silo FL

Federated Learning. Key Elements• Federated server. It orchestrates the iterative learning process.

The server participates in:
1.Receiving the trained parameters of the LLM.

2.Aggregating the trained parameters of each client model using the federated aggregation operator.
3.Updating all the learning model with the aggregated parameters.

•The server stores the GLM, which represents the aggregation of all the LLM, andhence the final model after the learning process.
•The GLM is used for predicting, but the prediction is done in each client.

Federated Learning. Key Elements• Communication• It is crucial in FL, since FL is a distributed learning process.
• It is one of the weakest element of FL, since the model parameters are shared withthe sever.
• If the parameters are not protected, an malicious agent can use them to reconstruct
the data of the clients by means model-inversion adversarial attacks [8].
• Strategies to mitigate the communication risks:•Differential privacy techniques to obfuscate the model parameters.
•Secure Multi-party Computation.

•Homomorphic Encryption.

Outline• Artificial intelligence challenges• Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)• Federated Learning workflow• Federated Learning libraries

Data.
Learning model
Federated Aggregation Operator
Clients
Federated server
Communication• Case of study• Adversarial attacks. Proposal and Case of study• Concluding remarks. What’s next?

Federated Learning. Categories• The FL setting extremely depends on the data distribution.
• Let consider:• 𝐷𝑖 : the data of the client i.
• X: it represents the feature space.
• Y: it represents the label space.
• I: it represents the sample ID space.
• (I, X, Y): it represents the dataset of the client i.

• Depending on how data is portioned among the clients, there are 3 categories of FL:•Horizontal FL•Vertical FL•Transfer FL

Federated Learning. Categories – Horizontal FL• The feature space (X) is shared among the clients.
• The label space (Y) is shared among the clients, although

there may be some differences in the label distribution(prior probability shift or unbalancedness).
• The example or ID space (I) is not shared, although it ispossible to exist some overlap.
• Formally:

𝑋𝑖 = 𝑋𝑗 , 𝑌𝑖 = 𝑌𝑗 , 𝐼𝑖 ≠ 𝐼𝑗 , ∀𝐷𝑖 , 𝐷𝑗 , 𝑖 ≠ 𝑗• Example: Two banks that serve two different regional

markets, they may share some users but their data
have the same or very similar features and labels.
Hence, they can collaboratively build a machinelearning model through horizontal FL.

Source: [9][9] Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., & Yu, H.
(2019). Federated learning. Synthesis Lectures on Artificial
Intelligence and Machine Learning, 13(3), 1-207.

Federated Learning. Categories – Vertical FL• The feature space (X) is not shared among the clients.
• The label space (Y) is not shared among the clients.
• The example or ID space (I) is shared.
• Formally:

𝑋𝑖 ≠ 𝑋𝑗 , 𝑌𝑖 ≠ 𝑌𝑗 , 𝐼𝑖 = 𝐼𝑗 , ∀𝐷𝑖 , 𝐷𝑗 , 𝑖 ≠ 𝑗• Example: When two companies provide different

services but they share a large amount of users (telco
and insurance companies). They may collaborate inbuilding a machine learning model through a Vertical
FL setting (different features and labels).

Source: [9][9] Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., & Yu, H.
(2019). Federated learning. Synthesis Lectures on Artificial
Intelligence and Machine Learning, 13(3), 1-207.

Federated Learning. Categories – Transfer FL• The feature space (X) is not shared among the clients.
• The label space (Y) is not shared among the clients.
• The example or ID space (I) is not shared.
• Formally:

𝑋𝑖 ≠ 𝑋𝑗 , 𝑌𝑖 ≠ 𝑌𝑗 , 𝐼𝑖 ≠ 𝐼𝑗 , ∀𝐷𝑖 , 𝐷𝑗 , 𝑖 ≠ 𝑗• Example:• Those scenarios where the mismatch among thefeature, label and instance spaces are large.
• Different organisations may collaboratively build
machine learning models, where each of them may be Source: [9]benefited from the data of the other party.
[9] Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., & Yu, H.
• It resembles the standard transfer learning scenario.
(2019). Federated learning. Synthesis Lectures on Artificial
Intelligence and Machine Learning, 13(3), 1-207.

Federated Learning. Architecture• FL is a distributed setting, so it can be deployed as other distributed architectures:• Peer-to-peer: All the nodes can be data owners and aggregation nodes.
• It is more common when there is not any agent with the enough reliability to be a server.
• Client-server: There is a difference among data owners (clients) and aggregation nodes (server).

Peer-to-Peer

Client-Server

Outline• Artificial intelligence challenges• Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)• Federated Learning workflow• Federated Learning libraries• Case of study• Adversarial attacks• Taxonomy• FL-IOWA-DQ• On-goingFederated Learning workflow

Source: [6]Rodríguez-Barroso, N., Stipcich, G., Jiménez-López, D., Ruiz-Millán, J. A., Martínez-Cámara, E., ... & Herrera, F. (2020). Federated Learning and Differential Privacy:
Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy. Information Fusion, 64, 270-292.

Federated Learning Libraries• Sherpa.ai: https://github.com/sherpaai/Sherpa.ai-Federated-Learning-Framework(The Sherpa.ai Federated Learning and Differential Privacy Framework is a project by Sherpa.ai, in collaboration with
the Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) research group fromthe University of Granada.

• Tensorflow: https://www.tensorflow.org/federated?hl=es-419• PySyft: https://github.com/OpenMined/PySyft• Fate: https://fate.fedai.org• Paddle Federated Learning: https://github.com/PaddlePaddle/PaddleFL

Outline• Artificial intelligence challenges• Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)• Federated Learning workflow• Federated Learning libraries• Case of study: Image classification with Sherpa.ai (EMNIST data set)
• Adversarial attacks. Proposal and Case of study• Concluding remarks. What’s next?

Case of Study: Image classification with Sherpa.ai• Dataset: EMNIST. The federated and extended version of the MNIST dataset.

Train set

Test set

Total

240,000

40,000280,000

https://github.com/sherpaai/Sherpa.ai-Federated-Learning-Framework

Case of Study: Image classification with Sherpa.ai• Classification model: feedforward network composed by two CNN layers with its

corresponding maxpoling layers.

https://github.com/sherpaai/Sherpa.ai-Federated-Learning-Framework

Case of Study: Image classification with Sherpa.ai• The development of the model is notcomplicated.
• Data loading• Distribution of the data among 5 clients following

an IID distribution.

https://github.com/sherpaai/Sherpa.ai-Federated-Learning-Framework

Case of Study: Image classification with Sherpa.ai• Deep learning image classification

model with Tensorflow.
• First CNN layer• Second CNN layer• Classification layer• Compilation of the model

https://github.com/sherpaai/Sherpa.aiFederated-Learning-Framework

Case of Study: Image classification with Sherpa.ai• Setting of the aggregation operator. In this case, we use the FedAVG aggregation

operator.
• The last step is the definition of the FA architecture with:• The classification model: model_builder.
• The data to process: federated_data.
• The aggregation operator: aggregator.

https://github.com/sherpaai/Sherpa.ai-Federated-Learning-Framework

Case of Study: Image classification with Sherpa.ai• The results of the federated model with an IID and a Non-IID data distribution issimilar to the results reached by the same classification model in a centralised
setting.
IID (Accuracy)

Non-IID (Accuracy)

Centralised setting

0,99040,9901

Federated setting

0,9921

0,9855• The configuration of the federated setting is:• Number of clients: 25.
• Rounds of learning: 10.
• Epochs: 5 per client.

Outline• Artificial intelligence challenges• Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)• Federated Learning workflow• Federated Learning libraries• Case of study: Image classification with Sherpa.ai (EMNIST data set)
• Adversarial attacks. Proposal and Case of study• Concluding remarks. What’s next?

Federated Learning: Adversarial Attacks• FL as machine learning is vulnerable to adversarial attacks.
• As a reminder, FL is featured by:• Its distributed nature.

• The inaccessibility of the training data, since it is sequestered in the clients.

• These two characteristics hinder the defence against adversarial attacks.

• The aim of the malicious agents (client or server) may be:•Misleading the behaviour of the FL model by poisoning the data or altering the parameters of themodel.
•Injecting a backdoor task without affecting the main learning task.
•Breaking data privacy by discovering if an instance is in the training data or any property of the
training data.

Federated Learning: Adversarial Attacks
Federated Model attacks: These attacks
are conducted in training time.
Inference attacks: These attacks are
conducted in inference or test time.

• Targeted. Their aim is to inject a second learning goal
into the model, as for instance recognising an specificimage face in a security system. These targeted attacks
can be conducted poisoning the data or the model.
• Untargeted. They only want to hinder the performanceof the FL model, and they are also known as byzantine
attacks. They can also be conducted in the model or in
the data.

Case of study: DDaBA against Byzantine attacks• Byzantine attacks are a type of untargeted adversarial attacks grounded in themaliciously alteration of the data or the model.
• We propose DDaBA (Dynamic Defense Against Byzantine Attacks), which is a robust

aggregation operator against Byzantine attacks regardless it is based on the
manipulation of the data or the model.
• We evaluate DDaBA in three different Byzantine Attacks:• Label-flipping: It consists of randomly flipping the labels of the examples.
• Out-of-distribution attack: It consists of introducing in the training dataset examples that do not follow the training

data distribution.
• Random weights: It consists of randomly generating the parameters of the local learning model.

Case of study: DDaBA against Byzantine attacks
DDaBA is based on:• The availability of a global validation subset in the FL server.
• The hypothesis is that a local learning model built upon altered dataset would

underperform on the global validation set, and we consider them as outliers.
• We define an induced OWA (ordered weighted averaging) operator to average thecontribution of each client according to its performance on the global validation set.
• Thanks to the induced OWA operator, DDaBA dynamically selects the clients to beaggregated in the global learning model (FL-IOWA-DQ operator)

Case of study: DDaBA against Byzantine attacks• We evaluate DDaBA on three federated datasets and we compare it with otherdefences from the state of the art.
• DDaBA shows in terms of Accuracy a strong performance independently the numberof adversarial clients (1/30, 5/30, 10/50).
• DDaBA allows to avoid the effect of adversarial clients since the results are the sameof higher when there are not any adversarial client.

Case of study: DDaBA against Byzantine attacks• We also evaluate DDaBA when there are a highnumber of adversarial clients.
• We see that a high number of adversarial clientsharms the performance of DDaBA.
• We propose the static version of DDaBA(SDaBA), which sets the proportion of clients to
not consider in the federated aggregation.
• SDaBA filters out those clients whose
performance on the validation set distances itself
from the best client in 𝛼 times.
• The results show that SDaBA can work in
scenarios with a high number of adversarial
clients.

Dataset: Federated EMNIST.
Adversarial clients: 10 out of 30 (33,33%).
Evaluation measure: Accuracy
𝛼 value: 0.25.

Outline• Artificial intelligence challenges• Federated Learning• Definition• Key elements• Categories• Architecture (client-server; peer-to-peer)• Federated Learning workflow• Federated Learning libraries• Case of study: Image classification with Sherpa.ai (EMNIST data set)
• Adversarial attacks. Proposal and Case of study• Concluding remarks. What’s next?

Concluding remarks• Artificial Intelligence challenges: Learning fromdata from different sources plus learning with privacy
and integrity lead us to focus on new machinelearning approaches. Federated Learning.
• Key Elements: Data. Learning model. Federated

Aggregation Operator. Clients. Federated server.
Communication.
• Taking decisions: Protection under adversarial tasks(low quality data, low quality models, attacks …)• Mitigation of Communications risk:•Differential privacy techniques to obfuscate themodel parameters. Secure Multi-party
Computation. Homomorphic Encryption.

Concluding remarks. Ongoing (What’s next?)• eXplainable Artificial Intelligence andFederated Learning• Personalized Federated Learning (per client)• Unsupervised Federated Learning• NLP under Federated Learning• Key elements for Vertical Federated Learning
Go ahead with our framework:
The Sherpa.ai Federated Learning and Differential Privacy
Framework55