# Federated Learning Algorithms to Optimize the Client and Cost Selections

Authors: Ali Alferaidi; Kusum Yadav; Yasser Alharbi; Wattana Viriyasitavat; Sandeep Kautish; Gaurav Dhiman
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

In recent years, federated learning has received widespread attention as a technology to solve the problem of data islands, and it has begun to be applied in fields such as finance, healthcare, and smart cities. The federated learning algorithm is systematically explained from three levels. First, federated learning is defined through the definition, architecture, classification of federated learning, and comparison with traditional distributed knowledge. Then, based on machine learning and deep learning, the current types of federated learning algorithms are classified, compared, and analyzed in-depth. Finally, the communication from the perspectives of cost, client selection, and aggregation method optimization, the federated learning optimization algorithms are classified. Finally, the current research status of federated learning is summarized. Finally, the three major problems and solutions of communication, system heterogeneity, and data heterogeneity faced by federated learning are proposed and expectations for the future.


---

## Body

## 1. Introduction

As digital technology enters a period of rapid development, technologies such as big data and artificial intelligence are ushering in explosive development [1, 2]. However, on the one hand, this has brought new opportunities for upgrading and reformation of traditional businesses [3–5]. On the other hand, it inevitably brings new challenges to data and network security, and the problem of data islands [6, 7] is one of the critical challenges. Vertically, the top giant companies in the industry monopolize a large amount of data and information, and it is often difficult for small companies to obtain this data, which leads to the continuous widening of the level and gap between enterprises. The occlusion and barrier of business make it difficult to realize the communication and integration of data and information, and joint modeling needs to overcome many obstacles.Federated Learning provides answers in response to the pain mentioned above currently facing the artificial intelligence industry. Federated learning is a concept first proposed by Google Research in 2016 [8–10]. This technology can complete joint modeling without data sharing. Specifically, the own data of each data owner (individual/enterprise/organization) will not leave the local area through the parameter exchange method under the encryption mechanism in the federal system [11].That is, without violating data privacy laws and regulations jointly establish a global sharing model, and the built model only serves local targets in their respective regions [12]. Although there are some similarities between federated learning [13–15] and distributed machine learning [16], federated learning has its characteristics in terms of application fields, system design, and optimization algorithms. When the amount of data is enormous, and the required computing resources are high, distributed machine learning (such as parameter servers) has obvious advantages. It stores independently identically distributed (IID) data or model parameters in each distributed node. On the Internet, the central server mobilizes data and computing resources to train the model jointly [17]. Due to the differences in clients’ geographical and temporal distribution, federated learning often has to deal with nonindependent and identically distributed (non-IID) data. This paper combines the status quo of federated learning, stratifies the federated learning system, and sorts out the relevant results of federated learning [18].To integrate data from multiple sources, the currently common practice is to use data preprocessing ETL (extract-transform-load) tools to move data from different sources to a relational database and deploy tasks with a tremendous amount of calculation to multiple machines improve computing efficiency and reduce task energy consumption.


## 2. Federated Learning

In 2016, Google Research Institute proposed a new artificial intelligence solution for federated learning when optimizing the keyboard input method for individual users. The scenario of federation learning is distributed multi-user (f1, f2,...., fn). Each user client has the current user data set (d1, d2... dn). Traditional deep learning collects these data together to get the summary data set d = u1 U u2 U un, and the training model MSUM is obtained. Participating users train a model MFED together in the federated learning method, while the user data di is kept locally and not externally transmitted. If there is a non-negative real number δ, the model accuracy VFED of MFED and the model accuracy VSUM of MSUM satisfy the following inequality:(1)VFED−VSUM<δ.It is said that the federated learning algorithm achievesδ-accuracy loss [4]. Federated learning allows the training model to have a certain degree of performance deviation but provides all participants data security and privacy protection. There are two commonly used frameworks for federated learning, one is the client-server architecture [8], and the other is the peer-to-peer network architecture [2]. In the client-server architecture, the training method of federated learning allows each data holder to train the model locally according to their conditions and rules, then aggregate the desensitization parameters to the central server for calculation, and then send back each data. The holder updates its local model until the global model is stable. When performing federated learning training in peer-to-peer network architecture, participants can communicate directly without using a third party, further improving security. Still, more computing operations are required for encryption and decryption [1–4]. Current research is more based on the framework of third-party servers. Therefore, this paper focuses on the federated learning process of the client-server architecture as shown in Figure 1.Figure 1
Structure of federated learning.


## 3. Federated Learning Process of Client-Server Architecture

The federated learning system is often built of data holders and central servers on a physical level. The quantity of local data or the number of features available to one data holder may not be sufficient to allow effective model training, necessitating the assistance of other data holder. The Federal Learning Center server functions similarly to a distributed machine learning server in that it gathers the gradients from each data bearer. It returns a new gradient after server-side aggregating procedures [5]. To ensure data privacy, the data holder’s training on local data is limited to local data in a cooperative modeling process of federated learning. Following desensitization, the gradient created by iterations is utilized as interactive data rather than local data and transferred to a third party. The trustworthy server then waits for the aggregated parameters to be returned before updating the model [8]. The client-server architecture’s federated learning mechanism is shown in Figure 2.Step 1. System initialization: first, the central server sends the modeling task, seeking to participate in the client. Next, the client data holder proposes a joint modeling vision based on its own needs. After reaching an agreement with other cooperative data holders, the joint modeling vision is established, and each data holder enters the collaborative modeling process. Finally, the central server releases the initial parameters to the data holders.Step 2. Partial calculation: after the joint modeling task is started and the system parameters are initialized, the data holders will be required to perform local calculations locally based on their data. After the analysis is completed, the gradients obtained from the local calculations are desensitized and uploaded in the global model.Step 3. Center aggregation: after receiving the calculation results from multiple data holders, the central server aggregates these calculated values. In the aggregation process, it is necessary to consider various issues such as efficiency, security, and privacy at the same time. For example, sometimes, due to the system’s heterogeneity, the central server may not wait for all data holders to upload but select a suitable subset of data holders as the collection target. To aggregate parameters safely, use certain encryption techniques to encrypt the parameters.Step 4. Model update: the central server updates the global model once based on the aggregation result and returns the updated model to the data holders participating in the modeling. The data holder updates the local model and starts the following partial calculation, and at the same time evaluates the performance of the updated model. When the performance is good enough, the training is terminated, and the joint modeling ends. The established global model will be kept on the central server for subsequent prediction or classification work.Figure 2
Federated learning process of client-server architecture [17].The above process is a typical federated learning process based on client-server architecture. However, not every federal learning task must strictly follow this process. Sometimes the process may be changed for different scenarios. For example, the communication frequency is appropriately reduced to ensure learning efficiency [6, 7], or in after aggregation, a logical judgment is added to judge the quality of the received local calculation results [2–5] to improve the robustness of the federated learning system [19].


## 4. Federated Vs Traditional Distributed Learning

Both federated learning and distributed machine learning based on the client-server architecture [8, 9, 20] are used to process distributed data. Still, they are different from distributed machine learning in terms of application fields, data attributes, and system components.


### 4.1. Application Areas

Large amounts of data or larger models often have higher requirements for computing resources. A single computing node can no longer meet the demand [10, 12]. Distributed machine learning distributes training data or model parameters to various computing or storage nodes and uses a central server to schedule nodes to accelerate model training. When the data has privacy-sensitive attributes, the central scheduling of distributed machine learning will bring a significant risk of privacy leakage to user data [21]. Federated learning permanently stores data locally, guarantee data privacy to the utmost extent compared to how information needs to be uploaded to the server.


### 4.2. Data Attributes

The primary purpose of machine learning is to find the probability distribution of data, which is relatively easy when the data set satisfies independent and identical distribution. However, the data processed by distributed machine learning and classic machine learning are often disconnected and identically distributed, while federated learning is different. Due to the differences in the client’s geographical location and time distribution, the raw data of the federated learning system is often nonindependent and identically distributed. At the same time, horizontal federated learning and vertical federated learning are also classified according to different attributes of client data. Therefore, the clients’ data characteristics and classification labels are quite different, and alignment work is required during training.


### 4.3. System Composition

In terms of physical composition, the federated learning system is similar to the distributed system, and both consist of a central server and multiple distributed nodes. In a distributed system, data calculation and model update is uniformly scheduled by the central server, and the data delay between nodes and the central server is relatively small. The model training time is mainly determined by the calculation time. In the federal system, each participant has equal status and can independently decide whether to participate in model training. And since distributed nodes are mostly clients with significant differences in computing capabilities, different network environments, and uncontrollable states, in system design, it is necessary to consider many factors such as data transmission delay, data nonindependent and identical distribution, and privacy and security. This requires the system to make adaptive changes to the federated learning algorithm [12–15]. Federated aggregation is an optimization algorithm that is different from distributed machine learning in a federated learning system. It provides new ideas for solving data nonindependent and identical distribution and reducing data heterogeneity. At the same time, because federated learning has excellent privacy protection capabilities, attention must be paid to applying encryption algorithms in all aspects of the system. The transmission of encrypted data, the calculation of the objective function loss, the analysis of the gradient and the information of the parameters of the transfer model all put forward new requirements for the traditional algorithms.


## 4.1. Application Areas

Large amounts of data or larger models often have higher requirements for computing resources. A single computing node can no longer meet the demand [10, 12]. Distributed machine learning distributes training data or model parameters to various computing or storage nodes and uses a central server to schedule nodes to accelerate model training. When the data has privacy-sensitive attributes, the central scheduling of distributed machine learning will bring a significant risk of privacy leakage to user data [21]. Federated learning permanently stores data locally, guarantee data privacy to the utmost extent compared to how information needs to be uploaded to the server.


## 4.2. Data Attributes

The primary purpose of machine learning is to find the probability distribution of data, which is relatively easy when the data set satisfies independent and identical distribution. However, the data processed by distributed machine learning and classic machine learning are often disconnected and identically distributed, while federated learning is different. Due to the differences in the client’s geographical location and time distribution, the raw data of the federated learning system is often nonindependent and identically distributed. At the same time, horizontal federated learning and vertical federated learning are also classified according to different attributes of client data. Therefore, the clients’ data characteristics and classification labels are quite different, and alignment work is required during training.


## 4.3. System Composition

In terms of physical composition, the federated learning system is similar to the distributed system, and both consist of a central server and multiple distributed nodes. In a distributed system, data calculation and model update is uniformly scheduled by the central server, and the data delay between nodes and the central server is relatively small. The model training time is mainly determined by the calculation time. In the federal system, each participant has equal status and can independently decide whether to participate in model training. And since distributed nodes are mostly clients with significant differences in computing capabilities, different network environments, and uncontrollable states, in system design, it is necessary to consider many factors such as data transmission delay, data nonindependent and identical distribution, and privacy and security. This requires the system to make adaptive changes to the federated learning algorithm [12–15]. Federated aggregation is an optimization algorithm that is different from distributed machine learning in a federated learning system. It provides new ideas for solving data nonindependent and identical distribution and reducing data heterogeneity. At the same time, because federated learning has excellent privacy protection capabilities, attention must be paid to applying encryption algorithms in all aspects of the system. The transmission of encrypted data, the calculation of the objective function loss, the analysis of the gradient and the information of the parameters of the transfer model all put forward new requirements for the traditional algorithms.


## 5. Federated Learning Classification

The island data of federated learning has different distribution characteristics. For each participant, the data they own can be represented by a matrix. Each row of the matrix represents each user or an independent research object, and each column represents a characteristic of the user or research object. At the same time, each row of data will have a label. People hope to learn a model to predict his label Y through his featureX for each user.In reality, different participants may be different companies or institutions. Moreover, people don’t want their data to be known by others. Still, people hope to jointly train a more robust model to predict the label Y. According to the data characteristics of federated learning (that is, the degree of data overlap between different participants), federated learning can be divided into horizontal federated learning [13], vertical federated learning [14], and transfer federated knowledge [15].When the user overlap of the two participants is minimal, but the user characteristics of the two data sets overlap more, the federated learning in this scenario is called horizontal federated learning. For example, the branches of a banking system in Shenzhen and Shanghai are the participants. The businesses on both sides are similar, and the characteristics of the collected user data are relatively identical. However, most of the users in the two branches are residents, and the user overlap is relatively small. Therefore, when a federated model is needed to classify users, it belongs to horizontal federated learning.On the other hand, when there is a lot of overlap between the users of the two participants. But the overlap of the user characteristics of the two data sets is relatively tiny. Federated learning in this scenario is called vertical federated learning. For example, two institutions in the same area, one institution has user’s consumption records; the other has user’s bank records. The two institutions have many overlapping users, but the recorded data characteristics are different. The two institutions want to aggregate through different encryption characteristics of users to train a more powerful federated learning model jointly. This type of machine learning model belongs to longitudinal federated learning.When the user overlap of the two participants is minimal, the user feature overlap of the two data sets is also relatively small, and some data still have missing labels. The federated learning in this scenario is called transfer federated learning. For example, two institutions in different regions have user consumption records in the area, and the other has bank records in the area. The two institutions have different users, and the data characteristics are also different. In this case, joint the trained machine learning model is transfer federated learning.Most of the current researches are based on horizontal federated learning and vertical federated learning. However, there is not much research in the field of transfer federated learning for the time being. Therefore, this article will focus on the algorithm types of horizontal federated learning and vertical federated learning. In horizontal federated learning, there are many overlapping dimensions of data features. The alignment is performed according to the overlapping dimensions, and the parts of the participant data with the same features but not the same users are taken out for joint training; vertical federated learning users have many overlapping dimensions, matching according to user IDs, and removing The part of the participant data that has the same user but not the same characteristics is jointly trained.


## 6. Features of Federated Learning Algorithm

Based on the above introduction to federated learning, the characteristics of the following federated learning algorithms are summarized.


### 6.1. Support Nonindependent and Identically Distributed Data

This is an essential feature of federated learning algorithms. The federated learning algorithm must perform well in nonindependent and identically distributed data. In the actual use of federated learning, the data quality and distribution of the data holder is uncontrollable. The data holder’s data cannot be required to meet independent and identical distribution [16], so the federated learning algorithm needs to support nonindependent and identically distributed data.


### 6.2. Communication Efficiency

The federated learning algorithm needs to consider the data holder’s system heterogeneity, improve communication efficiency, and reduce communication loss without losing accuracy or loss.


### 6.3. Fast Convergence

In the process of joint modeling, it is first necessary to ensure model convergence, and at the same time, it is essential to increase the convergence speed.


### 6.4. Security and Privacy

Data privacy security is an essential feature of federated learning, so security and privacy are requirements for federated gradient updates. Security and privacy can be carried out in the aggregation process through encryption and other methods and can also be reflected in stand-alone optimization.


### 6.5. Support for Complex Users

Complex users refer to many users themselves and the imbalance or deviation of user data. This is possible in the practical application of federated learning, and the federated optimization algorithm needs to have a good compatibility effect for this situation.


## 6.1. Support Nonindependent and Identically Distributed Data

This is an essential feature of federated learning algorithms. The federated learning algorithm must perform well in nonindependent and identically distributed data. In the actual use of federated learning, the data quality and distribution of the data holder is uncontrollable. The data holder’s data cannot be required to meet independent and identical distribution [16], so the federated learning algorithm needs to support nonindependent and identically distributed data.


## 6.2. Communication Efficiency

The federated learning algorithm needs to consider the data holder’s system heterogeneity, improve communication efficiency, and reduce communication loss without losing accuracy or loss.


## 6.3. Fast Convergence

In the process of joint modeling, it is first necessary to ensure model convergence, and at the same time, it is essential to increase the convergence speed.


## 6.4. Security and Privacy

Data privacy security is an essential feature of federated learning, so security and privacy are requirements for federated gradient updates. Security and privacy can be carried out in the aggregation process through encryption and other methods and can also be reflected in stand-alone optimization.


## 6.5. Support for Complex Users

Complex users refer to many users themselves and the imbalance or deviation of user data. This is possible in the practical application of federated learning, and the federated optimization algorithm needs to have a good compatibility effect for this situation.


## 7. Optimized Classification Method of Federated Learning Algorithm

Compared with distributed learning, federated learning has some unique attributes, as follows:(a)
The communication of federated learning is relatively slow and unstable.(b)
Participants in federated learning have heterogeneous devices, and different devices have different computing capabilities.(c)
Federated learning pays more attention to privacy and security. At present, most studies assume that the participants and the server are trustworthy. However, in real life, it may be untrustworthy.In implementing federated learning, it is necessary to consider how to optimize the federated learning algorithm to solve the existing practical problems. This article will introduce algorithms for optimizing federated learning from communication cost, client selection, and asynchronous aggregation. Before submitting the optimization algorithm, first, present the most traditional federated learning algorithm as FedAvg algorithm.The FedAvg algorithm [8] is currently the most commonly used federated learning optimization algorithm. Unlike the conventional optimization algorithm [7], it is essential idea is to use the local stochastic gradient descent method to optimize the local model on the data holder and perform aggregation operations on the central server-side [9]. The goal function is defined as follows:(2)fω∗=min1m∑n=1mEfω:x,xεn.Among them,M represents the number of data holders participating in the joint modeling; ω represents the current parameters of the model and represents the mean square error function. The FedAvg algorithm is a relatively basic federated optimization algorithm. Its deployment is relatively simple, and its application field is vast [6].


## 8. Cost Optimized FDA

Machine learning algorithms, incredibly complex deep learning algorithms, need to train many parameters during the training process. For example, CNN may need to train millions of parameters, and each update process needs to update millions of parameters; secondly, network communication. The status may also lead to remarkably high Communication costs, such as unstable network conditions and inconsistent speeds during parameter upload and download, leading to high model training costs for the entire algorithm. Therefore, it is necessary to consider optimizing the federated learning algorithm from the perspective of communication cost according to these characteristics [8]. The communication cost can be reduced from the following views.


### 8.1. Client Computing Cost

In the federated learning system, sometimes terminal nodes will only participate in federated learning training when Wi-Fi is available, or sometimes the network is in poor condition. In these cases, more calculations can be performed locally, thereby reducing the number of communications. Many algorithms optimize communication costs from this perspective. For example, optimizing the FedAvg algorithm [2], increasing the number of calculations of the local update parameters of each client in each iteration, and it was carried out with the Fed-SGD algorithm in which each round of server parameter update only requires one local update of the client. In contrast, the experiment shows through the MINSTCNN model test that the algorithm can significantly reduce the communication cost when the data is IID. However, when the information is non-IID, the algorithm can only slightly reduce the communication cost. Fed-Prox algorithm [9] more obvious optimization effect when the data is non-IID because the data and computing power of the joint training terminal participants are not balanced, so each During a parameter update, if the number of operations that different participants have to participate in is uniform, the computing resources of the client will not be fully utilized. To avoid this situation and optimize communication efficiency, the Fed-Prox algorithm can dynamically update the number of local calculations required by different clients in each round, making the algorithm more suitable for nonindependent and codistributed joint modeling scenarios. Federated optimization algorithm [10] and learn under the framework of vertical federated learning. FedAvg algorithm [12] and proved that the heterogeneity of the data would cause the convergence rate of federated learning to decrease.


### 8.2. Model Compression

Some optimization algorithms aim to reduce the number of parameters in each round of communication. For example, model compression techniques (such as quantization and subsampling) reduce the total number of parameters to be transmitted for each parameter update. Structured model [15] update round of parameter communication, the size of the model update parameters passed by the participants to the server reduces transmission. The structured update refers to uploading the model by defining the matrix structure of the uploaded model parameters in advance. The contour update means that the parameters of each update need to be compressed and encoded by the participants. The model is finally verified by the CIFAR-10 image algorithm. Experiments show that participation the more parties, the better the compression effect. Optimization of model parameter transfers from the server to the participants and reduced the data from the server to the client through loss compression and federated dropout. However, the number of parameters that need to be passed at the end and the cost of reducing communication costs is to reduce the model’s accuracy to a certain extent.When implementing federated learning, communication is a bottleneck. Reducing communication costs is a vital optimization link. Some optimizations are at the expense of increasing the local calculations of the participants, and some optimizations are to reduce the accuracy of the entire model. In the actual optimization process, you can decide which way to reduce the communication cost according to the actual situation and demand.


## 8.1. Client Computing Cost

In the federated learning system, sometimes terminal nodes will only participate in federated learning training when Wi-Fi is available, or sometimes the network is in poor condition. In these cases, more calculations can be performed locally, thereby reducing the number of communications. Many algorithms optimize communication costs from this perspective. For example, optimizing the FedAvg algorithm [2], increasing the number of calculations of the local update parameters of each client in each iteration, and it was carried out with the Fed-SGD algorithm in which each round of server parameter update only requires one local update of the client. In contrast, the experiment shows through the MINSTCNN model test that the algorithm can significantly reduce the communication cost when the data is IID. However, when the information is non-IID, the algorithm can only slightly reduce the communication cost. Fed-Prox algorithm [9] more obvious optimization effect when the data is non-IID because the data and computing power of the joint training terminal participants are not balanced, so each During a parameter update, if the number of operations that different participants have to participate in is uniform, the computing resources of the client will not be fully utilized. To avoid this situation and optimize communication efficiency, the Fed-Prox algorithm can dynamically update the number of local calculations required by different clients in each round, making the algorithm more suitable for nonindependent and codistributed joint modeling scenarios. Federated optimization algorithm [10] and learn under the framework of vertical federated learning. FedAvg algorithm [12] and proved that the heterogeneity of the data would cause the convergence rate of federated learning to decrease.


## 8.2. Model Compression

Some optimization algorithms aim to reduce the number of parameters in each round of communication. For example, model compression techniques (such as quantization and subsampling) reduce the total number of parameters to be transmitted for each parameter update. Structured model [15] update round of parameter communication, the size of the model update parameters passed by the participants to the server reduces transmission. The structured update refers to uploading the model by defining the matrix structure of the uploaded model parameters in advance. The contour update means that the parameters of each update need to be compressed and encoded by the participants. The model is finally verified by the CIFAR-10 image algorithm. Experiments show that participation the more parties, the better the compression effect. Optimization of model parameter transfers from the server to the participants and reduced the data from the server to the client through loss compression and federated dropout. However, the number of parameters that need to be passed at the end and the cost of reducing communication costs is to reduce the model’s accuracy to a certain extent.When implementing federated learning, communication is a bottleneck. Reducing communication costs is a vital optimization link. Some optimizations are at the expense of increasing the local calculations of the participants, and some optimizations are to reduce the accuracy of the entire model. In the actual optimization process, you can decide which way to reduce the communication cost according to the actual situation and demand.


## 9. Client Optimized FDA

The client devices of federated learning are heterogeneous, and the resources of different clients are limited. Usually, the client randomly chooses to participate in the model training process of federated learning. Therefore, in the federated learning and training process, some algorithms will consider the angle selected from the client for optimization.Different clients have different network speeds, computing power, etc., and the data distribution owned by each client is also unbalanced. If all clients are allowed to participate in the training process of federated learning, there will be participants with backward iterations. If some clients do not respond for a long time, the whole system may not complete the joint training. Therefore, it is necessary to consider how to choose the clients to participate in the training. The FedAvg algorithm randomly selects the clients participating in the training. However, when the network structure is complex, and the data is not independent and identically distributed, the FedAvg algorithm model does not necessarily perform well. The following two references introduce some optimization schemes.Fed-CS algorithm [19] is a greedy algorithm protocol mechanism to select the client with the highest model iteration efficiency for aggregation update in each update of joint training. Thus, optimize the convergence efficiency of the entire federated learning algorithm. Experiments show that the Fed-CS algorithm can achieve higher accuracy. Still, the disadvantage is that it can perform well only when the model is relatively basic, such as a virtual dynamic neural network. For the case where the network structure or the number of parameters is more complex, Fed-CS will reduce the efficiency of selecting the optimal aggregation client, increasing the number of communications and decreasing time efficiency.Hybrid-FL protocol [20] algorithm can process client data whose data set is non-IID and solve the problem of poor performance based on non-IID data on the FedAvg algorithm. The Hybrid-FL protocol enables the server to select some clients through the steps of the resource request to establish an approximately independent and identically distributed data set locally for training and iteration of federated learning. Furthermore, they show through experiments that Hybrid-FL has better accuracy performance for non-IID data type federated learning classification algorithms.


## 10. Result Evaluation

A classification model’s performance may be quantified in terms of True Positive and False Positive rates. The TP rate is the difference between the patterns recognized by a system and the total patterns detected by system. A simple mathematical representation is presented as follows.(3)TPR=tptp+fn.Similarly the True negative rateis shown in the following formula:(4)TNR=tntp+tn.False Positive rates occur when any system incorrectly classifies a normal outcome. The FP rate may be calculated in this experiment by dividing the total number of self-antigens by the number of false positives generated by the system.(5)FPR=fpfp+tn.Similarly false negative rate and accuracy calculated by(6)FNR=fntp+fn,accuracy=tp+tntp+tn+fp+fn.Table1 compares the simulation results; it compares the categorization accuracy rate for the old approach with the FedAvg algorithm. The generating function, also known as the activation threshold value, was adjusted to the greatest accuracy rate attainable with our system utilising the FedAvg algorithm approach during simulation.Table 1
Comparison of the simulation result of federated learning algorithm.
MethodsTPRTNRFPRFNRAccuracyTraditional algorithm64.5254.2342.1252.6975.24Fed-CS algorithm74.2564.2140.2845.2480.45Hybrid-FL79.4570.5230.5132.4686.47Fed-average algorithm84.2174.2523.4525.6490.45It compares the real positive rate for classification using the old technique to that utilising the FedAvg algorithm, as shown in Figure3. In simulation, the generating function, also known as the activation threshold value, was set to the greatest TPR rate feasible with our approach.Figure 3
True positive rate.As shown in Figure4, it compares the genuine negative rate for categorization using the old technique to the FedAvg algorithm. In simulation, the generating function, also known as the activated threshold value, was set to the greatest TNR rate attainable with our technique.Figure 4
True negative rate.It compares the false positive rate for classification using the old technique to the rate for classification using the FedAvg algorithm, as shown in Figure5. In simulation, the generating function, also known as the activation threshold value, was set to the algorithm’s minimum FPR rate, which is only attainable using the FedAvg algorithm approach.Figure 5
False positive rate.It compares the false negative rate for classification using the old technique to the rate for classification using the FedAvg algorithm, as shown in Figure6. In simulation, the generating function, also known as the activation threshold value, was set to the algorithm’s minimum FNR rate, which is only attainable using the FedAvg algorithm approach.Figure 6
False negative rate.It compares the classification accuracy of the old technique to the FedAvg algorithm, as shown in Figure7. The generating function, also known as the activation threshold value, was adjusted to the maximum accuracy feasible for our approach utilising the FedAvg algorithm technique during simulation.Figure 7
Accuracy.


## 11. Conclusion

This paper discusses the current development status of federated learning. From the perspective of federated learning algorithms, the federated learning-related algorithms are divided into federated optimization algorithms and federated machine learning algorithms. Related algorithms suitable for both central and decentralized federated learning structures are carried out. At the same time, the machine learning algorithm and the federated deep learning model under the federated learning framework will be summarized and discussed separately. In the process of federated algorithm optimization, the differences, advantages and disadvantages of existing federated optimization algorithms are discussed from the perspectives of reducing communication costs, selecting the best client, and optimizing model aggregation. Federated learning is still in a stage of rapid development. There has been a lot of research and discussion on the application of federated knowledge in practice, but there are still many problems and challenges in implementing federated learning. This work has been further extended to solve the real-life applications by using the recently developed optimization algorithms such as monarch butterfly optimization (MBO) [21], earthworm optimization algorithm (EWA) [22], elephant herding optimization (EHO) [23], moth search (MS) algorithm [24], Slime mould algorithm (SMA) [25], and Harris Hawks optimization (HHO) [26].



---

*Source: 8514562-2022-04-01.xml*