# Falafels: A tool for Estimating Federated Learning Energy Consumption via Discrete Simulation

Andrew Mary Huet de Barochez, Stéphan Plassart, Sébastien Monnet

► To cite this version:

Andrew Mary Huet de Barochez, Stéphan Plassart, Sébastien Monnet. Falafels: A tool for Estimating Federated Learning Energy Consumption via Discrete Simulation. The 39th International Conference on Advanced Information Networking and Applications (AINA-2025), Apr 2025, Barcelona, Spain. 10.1007/978-3-031-87769-8_13 . hal-04961398

HAL Id: hal-04961398

https://hal.science/hal-04961398v1

Submitted on 11 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Falafels: A tool for Estimating Federated Learning Energy Consumption via Discrete Simulation

Andrew Mary Huet de Barochez, Stéphan Plassart and Sébastien Monnet

**Abstract** The growth in computational power and data hungriness of Machine Learning has led to an important shift of research efforts towards the distribution of ML models on multiple machines, leading in even more powerful models. However, there exists many Distributed Artificial Intelligence paradigms and for each of them the platform and algorithm configurations play an important role in terms of training time and energy consumption. Many mathematical models and frameworks can respectively predict and benchmark this energy consumption, nonetheless, the former lacks of realism and extensibility while the latter suffers high run-times and actual power consumption. In this article, we introduce Falafels, an extensible tool that predicts the energy consumption and training time of — but not limited to — Federated Learning systems. It distinguishes itself with its discrete-simulator-based solution leading to nearly instant run-time and fast development of new algorithms. Furthermore, we show this approach permits the use of an evolutionary algorithm providing the ability to optimize the system configuration for a given machine learning workload.

**Key words:** Federated Learning, Distributed Systems, Energy Optimization, Discrete Simulation, Evolution Algorithm, Tool

## 1 Introduction

Since 2012, Artificial Intelligence (AI) computing power is growing exponentially [1]. As a consequence, Distributed Artificial Intelligence (DAI) [10], i.e., combining the computing power of multiple machines, is imperative. However, this ex-

Andrew Mary Huet de Barochez · Stéphan Plassart · Sébastien Monnet

LISTIC, Université Savoie Mont-Blanc, Annecy, France e-mail: {andrew.mary-huet-de-barochez, stephan.plassart, sebastien.monnet}@univ-smb.fr
ponential growth considerably worsen the impact of AI [15] on our planet's limited resources [18], therefore evaluating and reducing energy consumption of such systems is critical. To account for this consumption, we distinguish two complementary approaches:

* **Measuring** the electricity for the training, either with software-based solutions or with a watt-meter.

* **Estimating** the consumption via an energy model which takes as an input the execution time. This last can be retrieved a posteriori, or it can be predicted a priori by modelling the execution order.

In classical AI, studying and optimizing the model is the main way of reducing energy consumption [23, 28], but it is now insufficient in a context where the training workload is parallelized on multiple machines. Indeed, now communication cost must be taken into account, the platform on which the model will be trained must be sized and configured properly, and the algorithms used for machine coordination must be tested extensively as they play an important role on training time and energy consumption [24]. As a consequence, many parameters related to the system configuration exist and need to be chosen correctly.

A particularly interesting case of application is the Federated Learning (FL) paradigm [14] which leverages the computational power of several heterogeneous devices with limited computational power, bandwidth, memory storage, and availability. This aspect makes energy measurement and estimation more complex because the technique employed has to be adapted to each device, compared to an homogeneous cluster with the same machines. Furthermore, one can configure for example the applicative topology, the communication mechanism, the decentralization level, or the scheduling algorithm. This creates many combinations of parameters, making it difficult for measuring-based technologies as it requires to re-run the training for every combination, representing actual energy consumption and long run-times.

In this paper, we present Falafels<fnref n="1" />, a tool specifically built to study the Distributed Systems (DS) challenges of DAI. Falafels provides extensible building blocks for simulating and estimating the energy consumption of a FL system using a single-node. It is based on discrete simulation using the Simgrid framework<fnref n="2" />, which abstracts the computation workloads to focus on networking, task ordering and scheduling. Furthermore, Simgrid provides an energy model [9] that is used in Falafels in order to estimate the energy consumed by training machines and network devices. We believe that discrete-simulation-based tools are complementary to experimental frameworks because they have faster execution times, providing quick evaluations of distributed algorithms. The following contributions of this paper are:

1. We propose an extensible modeling of FL using the object oriented programming paradigm, providing seamless interoperability between networking and learning algorithms.

<footnote n="1">¹ https://github.com/PhoqueEberlue/falafels</footnote>

<footnote n="2">² https://simgrid.org/</footnote>
2. We introduce Falafels, a tool for simulating FL systems execution and predicting energy consumption of the machines and the network with a nearly instant run-time. This provides the ability to run fast experimentations of many configuration parameters and to develop new distributed algorithms for parallelizing efficiently machine learning tasks.

3. We demonstrate how optimization algorithms such as an evolutionary algorithm can be applied using our tool in order to find an optimal system configuration parameters for a given workload.

The remainder of this article is organized as follows. Sec. 2 details the principles and parameters of FL systems and provides references to related works concerning energy optimization in distributed systems. Sec. 3 introduces the developed tools and the state-of-the-art algorithms that have been implemented. Sec. 4 details the evolution algorithm and experimental setup used before Sec. 5 concludes the paper and gives some perspectives.

## 2 Background and Related work

Federated Learning (FL) is a trending domain introduced by Google [14], and its particularity of using many heterogeneous devices makes it challenging to find the best configurations for a use case. Due to the network topologies and algorithms available, [13] introduce a large number of different configurations. [2] separates FL systems in three categories.

The **Federation Architecture** includes the *federation type*; Cross-silo refers to collaborations between organizations owning large datasets and powerful machines, e.g, a cohort of hospitals [27]; Cross-device leverages the power of hundred or more Internet-of-Things (IoT) devices [21], however with limited and heterogeneous resources. *Decentralization scheme*; Centralized Federated Learning (CFL) is the classical scheme where a central server receives and aggregates the models; Semi-decentralized Federated Learning (SDFL) uses edge servers to perform pre-aggregations in the middle layers of the network [5]; Decentralized Federated Learning (DFL) can use any topology and often uses dynamic connections and role distribution [20].

**Network topology** refers to the degree of connection between the nodes. *Fully connected* networks provides highly reliable and robust networks as multiple points of connection between nodes prevent single-point-of-failures, however their scalability and flexibility are very low. *Partially connected* includes stars, rings, trees and random networks. They provide better scalability and maintainability but they might be prone to single-point-of-failures. *Node clustering* generally applies to SDFL (with edge and central servers) or DFL (with less organized clusters and no central server compared to SDFL). The advantages of this topology heavily depend on subclusters' topologies.

**Communication mechanism** describes the way nodes communicate between each others. For example, it can be the protocol (TCP/UDP/QUIC/WebSocket) or
higher level concepts (P2p/Gossip/Rendez-vous). It also describes the level of syn-
chronization between the rounds, for example in [7, 26], algorithms with asyn-
chronous rounds are implemented and lead to less idle clients with faster conver-
gence speed of the model.

Accounting energy consumption is done either via **estimating** or **measuring**.
[22, 16] implemented mathematical frameworks in order to **estimate** FL systems
consumption, they found out that communication efficiency and trainer population
size are important parameters when optimizing energy consumption. Preliminary
work have also been done concerning the carbon footprint of FL [17]. However
mathematical frameworks have limitations concerning the realness of the results
because of the lack of system considerations.

Many experimental frameworks exist [4, 11, 6], and energy consumption can be **measured** using benchmarking softwares or watt-meters. Nevertheless, the previous frameworks tend to be limited in realism and scalability, particularly when the FL system is simulated on a single-node. Flower [3] and ns3-fl [8] are two frameworks built to reproduce systems limitations in order to provide closer results between experimentation and production. The former proposes a flexible application where experiments can be executed both on a single-node and in real conditions on multiple nodes, and this without modifying the code. The latter paired the existing framework flsim [12] with the discrete network simulator ns-3 [19] in order to simulate the FL system and the network on a single-node.

Mathematical frameworks lack realism and extensibility while experimental
frameworks are constrained by the Machine Learning (ML) model training time,
thus, researching and evaluating new solutions for the DS challenges, e.g., imple-
menting a new fault-tolerance algorithm, can become tedious. From DS point of
view, we want to abstract the ML workload and focus on the distributed algorithms
themselves. To answer this problematic, we pushed the idea of ns3-fl where only
the network is discrete, to a fully discrete simulation framework called Falafels.
This led to nearly instant execution times, energy consumption is estimated which
reduces the impact of experimenting, and the experimentation of new distributed
algorithms becomes easier. This comes at the cost of the inability to know about
ML model accuracy, making it a complementary approach to existing frameworks.
Falafels currently only implements FL systems simulation, but the architecture
is flexible enough to embrace others paradigms such as Distributed Deep Learning
by reducing the granularity of computation and communication tasks as performed
in ASTRA-sim2.0 [25].

3 Falafels: federated learning with discrete simulation

In this section, we introduce the use cases, design goals, a formalization of
FL concepts and algorithms, and finally the core architecture of our framework
Falafels.
## 3.1 Use Cases

Existing tools have mostly focused on the machine learning aspect of FL, making them specialized in research towards model performance while neglecting systems related aspects. The goal of Falafels is to provide a convenient tool for analyzing and researching new network architectures and distributed algorithms for FL by mostly focusing on energy consumption rather than model performance. This tool aims to be complementary with the existing frameworks in order to find compromises in terms of model performance and energy consumption.

**Energy prediction for machines and network.** In DAI and FL in particular, networking can consume a lot of energy. The experiments must be able to predict the energy consumption from both the computations and the communications.

**Large scale network experiments.** The simulations need to be fast in order to conveniently iterate over results, and this statement should remain true even when working with large scale networks.

**Single-node execution.** Simulating a whole system on a single-node is very convenient for developing and debugging new distributed algorithms, it provides access to a global vision of the network, allowing to better understand the impacts of the new solutions.

**Reproducibility of experiments.** Providing an environment that can be reproduced by other researchers is crucial to propose quality research. Furthermore, reproducibility within a discrete event simulator has the advantage of having a deterministic history of network messages and task execution which is practical for debugging, compared to real experiments for which behaviours can diverge from multiple executions.

## 3.2 Design goals

Falafels focuses on use cases concerning DS problems applied to FL. However we do not want to restrict the design and implementation to the FL case, instead it should be open to other DAI paradigms for the future.

**Learning Algorithm extensibility.** The simulator should be extensible, adding new algorithms or modifying existing ones must be simple with a base level of abstraction for communications.

**Network topologies plurality.** Given the number of network topologies available in DS, we want to provide multiple networking algorithms built on common abstract primitives so that the learning algorithms are seamlessly compatible with any networking algorithm.

**Nodes can change role at run-time.** To permit research in dynamic and decentralized methods, nodes should be able to change their role at run-time. This is easier to work with this requirement from the start than adding it later during development.
## 3.3 Federated Learning modelling

Following the state-of-the-art in the FL domain (see Sec. 2), we identified the main algorithms and network topologies in order to support a broad part of existing implementations, while also providing a base for further extensions. Fig. 1 illustrates the different topologies implemented; the star topology is used mostly in CFL and it is rarely used in DFL; the ring topology is typically used in DFL; the hierarchical topology can be used both for edge computing (SDFL) and clustering (DFL).

<figure><img src="image_1.png" /><figcaption>**Fig. 1** Implemented network topologies. In a star, each trainer $T_x$ is connected to a central aggregator that orchestrates the training. The ring is unidirectional, packets flow in only one direction and multiple aggregators $A_y$ can be used. Hierarchical topology allows the connection of multiple subclusters to the central aggregator via hierarchical aggregators $HA_z$.</figcaption></figure>

The definition of an algorithm is done via a class which follows a finite-state machine behaviour. For example, the simple aggregator has 3 states and clearly defined transitions that will be triggered upon receiving network events, or when internal class conditions are met (see Fig. 2). For the asynchronous aggregator, the transition condition to the *aggregating* state is set in order to wait for a given proportion of the trainers.

<figure><img src="image_2.png" /><figcaption>**Fig. 2** Automaton representing the simple aggregator algorithm.</figcaption></figure>

The behaviour of networking algorithms is modelled in the same fashion. However, we need to specify how to connect to other nodes. Depending on its role, different behaviours should be implemented, e.g., a trainer will send a registration request to the aggregator, and the aggregator will listen to those requests (see Fig. 3). The communication logic is then executed in the *running* state: if the received packet is
targeted to the current node, it is sent to the role algorithm automaton, otherwise it
can be redirected to another node.

<figure><img src="image_3.png" /><figcaption>Fig. 3 Example of the network manager automaton for the star topology.</figcaption></figure>

**3.4 Core architecture**

As stated before, Falafels is based on discrete event simulation. The two main
discrete simulators renowned in the DS literature are *Simgrid* [9] and *ns-3* [19].
Although they both have an energy model, *Simgrid* seemed better for the follow-
ing reasons:

* *Simgrid* is simpler, more flexible and provides higher level abstraction which is handful to focus on the application level development.

* ns-3 simulate each packet in the network individually which is oversized for our use case and results in slower simulations (see ns3-fl run-times [8]).

The architecture is based on object oriented programming and revolves around
3 main classes: *Mediator*, *NetworkManager* and *Role*. Each of these classes lever-
ages polymorphism to provide base abstraction and extendability for future features.
For example, the simplified class diagram at Fig. 4 shows how a *NetworkManager*
provides basic *put* and *get* abstract functions that will be implemented differently
depending on the specific implementations of *NetworkManager*. Indeed, in a star
topology, making a broadcast is as simple as sending a packet to everyone, whereas
in a ring topology we want to implement a mechanism of redirection to make the
packet flow around the ring.

The same applies to the *Role* class, but this time there is one more layer of com-
plexity. Indeed, multiple layers of heritage are necessary to provide the three basic
roles (Aggregator, Trainer and Proxy), and to allow heritage for their derivatives.
This makes it possible to implement multiple algorithms of Aggregators with their
own execution function. This function overrides *Role's* run function which makes
each *Role* children class executable in the same way, but with different behaviours.

Furthermore, *Role* and *NetworkManager* have to be executed in parallel so that
networking tasks and ML tasks do not block each others. *Simgrid* proposes a
<figure><img src="image_4.png" /><figcaption>Fig. 4 Class diagram of the simulator architecture.</figcaption></figure>

model with three layers: the execution environment that contains the whole simulated system, the **Host** that represent a single physical machine, and the **Actor** which corresponds to a simulated thread on a given machine (see Fig. 5). Here, *Network-Manager* and *Role* run on two different **Actors** while remaining on the same **Host**, so the consumption is accounted for the same machine. This allows *Simgrid* to wait independently for events in *Role* and *NetworkManager*.

Finally, *Mediator* allows communication between *Role* and *NetworkManager* as they run on two different (simulated) threads. It is based on *MessageQueues*, an equivalent of channels between threads.

<figure><img src="image_5.png" /><figcaption>Fig. 5 Execution environment of the simulator in *Simgrid*. The actual simulations contains multiple hosts (with two actors in each), however only one host have been represented here for simplicity.</figcaption></figure>
# 4 Evolution algorithm

The evolution algorithm has been introduced to our experiment toolkit to test a large amount of simulation configurations without making any assumptions concerning their parameters. It can find optimal configurations by testing random parameters, thus converging as bad configurations will be filtered at each iteration based on a given criteria. The algorithm first initializes a group for each possible combination of *NetworkManager* and *Role*. In each of these groups, a given number of FL configurations will be created. Then the following pipeline gets executed for each group (see Fig. 6):

* Step 1: run the simulation for each FL configuration of the group;

* Step 2: sort the results by some criteria, e.g., simulation time or total energy consumption;

* Step 3: remove a proportion of the worse individuals;

* Step 4: clone the remaining individuals and apply mutations.

Two important choices have been made while designing the algorithm. Firstly, we separated the combinations into their respective pipeline. It was not the case originally: the combinations with great configurations from the start would take over and prevent the other combinations to converge. It made more sense to separate them so that they would all have a possibility to converge.

Secondly, we implemented mutation on single individuals, because using reproduction between two configurations would have been harder to implement. Indeed, reproduction between two platforms does not look intuitive. Instead we used mutations, so the algorithm randomly increases (or decreases) the number of machines, changes algorithms parameters, and swaps machines roles. To illustrate the last point, one aggregator may be running on a powerful machine during one iteration, and at the next one it could be affected to another machine with a slower profile.

<figure><img src="image_6.png" /><figcaption>**Fig. 6** The evolution algorithm implemented as an example for optimizing a platform for a given ML model. Each combination of network topology and role algorithm have its own pipeline, each pipeline contains multiple platform configuration strategies. The color gradient represent the individual performance of each strategy: the darkest are the best results.</figcaption></figure>
## Framework evaluation

**Experimental setup:** For reproducibility purpose, all parameters and execution scripts are available online <fnref n="3" />. Furthermore, many other experiments have been performed to compare the impact of platform configuration and are available on the online repository. For the workload, we choose the model used in the first FL paper [14], a multilayer-perceptron with 199,210 total parameters. The training task is expressed in Floating point operations per second (FLOPS) and is obtained by multiplying the number of parameters, the number of floating point operations, and the number of samples. The same applies for the aggregation task where a weighted arithmetic mean is applied with each trainer model. Concerning the platform, we used (have simulate) three types of computers: a workstation, a laptop, and a raspberry pi 4. We used a benchmarking software to determine the energy model of each machine.

**Results:** Fig. 7 shows the metrics of the best configuration in each group which represents a combination between a topology and a role algorithm. Here, the energy consumption is used as the optimization criteria, thus it can only decrease or keep steady. Several behaviours can be noticed, for example, increasing the number of machines can help lowering the consumption when the initial platform is under-scaled for the workload. Increasing the computational power of the platform (total GFLOPS) gives the best results to save energy when using asynchronous algorithms. The heterogeneity of the platform can be tampered by asynchronism, e.g., the most powerful machines do not have to wait for the slowest ones, which reduces their idle time that have a non-negligible impact on consumption.

<figure><img src="image_7.png" /><figcaption>**Fig. 7** Results of the evolution algorithm performed using the total energy consumption as the optimization criteria. The plot represents the best individual of each configuration group per generation.</figcaption></figure>

<footnote n="3">³ https://github.com/PhoqueEberlue/falafels/tree/740ac67ec300b8446693cfe0086491e3476b3abc/experiments/evolution/total_consumption_criteria</footnote>
## 5 Conclusion

We have presented Falafels, a simulator capable of predicting the run-time and energy consumption of a FL system. The nearly instant execution differentiate our framework from the others and make it possible to apply optimization algorithms in order to choose the most frugal platform in terms of energy consumption. However, the trade-off for fast executions is that the model convergence and accuracy cannot be known. That is why we believe that this tool could be complementary to existing frameworks by providing platform related optimizations, in the same way grid search is used to find optimal hyper-parameters.

As a future work, we aim to (a) extend the framework to other paradigms of DAI and (b) support switching between discrete simulation and real execution. This last point will help us to verify and calibrate the simulator, but will also create opportunities to investigate the impact of platform optimization for energy reduction on model performance. Lastly, using the framework to study the impact of faults and the implementation of fault-tolerance algorithms on the energy consumption would be interesting.

## References

1. Amodei, D., Hernandez, D.: Ai and compute. https://openai.com/index/ai-and-compute/ (2018)

2. Beltrán, E.T.M., Pérez, M.Q., Sánchez, P.M.S., Bernal, S.L., Bovet, G., Pérez, M.G., Pérez, G.M., Celdrán, A.H.: Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials (2023)

3. Beutel, D.J., Topal, T., Mathur, A., Qiu, X., Fernandez-Marques, J., Gao, Y., Sani, L., Li, K.H., Parcollet, T., de Gusmão, P.P.B., et al.: Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390 (2020)

4. Bonawitz, K.: Towards federated learning at scale: System design. arXiv preprint arXiv:1902.01046 (2019)

5. Briggs, C., Fan, Z., Andras, P.: Federated learning with hierarchical clustering of local updates to improve training on non-iid data. In: 2020 international joint conference on neural networks (IJCNN), pp. 1–9. IEEE (2020)

6. Caldas, S., Duddu, S.M.K., Wu, P., Li, T., Konečný, J., McMahan, H.B., Smith, V., Talwalkar, A.: Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097 (2018)

7. Chen, Y., Ning, Y., Slawski, M., Rangwala, H.: Asynchronous online federated learning for edge devices with non-iid data. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 15–24 (2020)

8. Ekaireb, E., Yu, X., Ergun, K., Zhao, Q., Lee, K., Huzaifa, M., Rosing, T.: ns3-fl: Simulating federated learning with ns-3. In: Proceedings of the 2022 Workshop on ns-3, pp. 97–104 (2022)

9. Heinrich, F.C., Cornebize, T., Degomme, A., Legrand, A., Carpen-Amarie, A., Hunold, S., Orgerie, A.C., Quinson, M.: Predicting the energy-consumption of mpi applications at scale using only a single node. In: 2017 IEEE international conference on cluster computing (CLUSTER), pp. 92–102 (2017)

10. Janbi, N., Katib, I., Mehmood, R.: Distributed artificial intelligence: Taxonomy, review, framework, and reference architecture. Intelligent Systems with Applications 18, 200,231 (2023)
11. Lai, F., Dai, Y., Singapuram, S., Liu, J., Zhu, X., Madhyastha, H., Chowdhury, M.: Fedscale: Benchmarking model and system performance of federated learning at scale. In: International conference on machine learning, pp. 11,814–11,827. PMLR (2022)

12. Li, L., Jun, W., Chengzhong, X.: Flsim: An extensible and reusable simulation framework for federated learning. In: Simulation Tools and Techniques - 12th EAI International Conference, SIMUtools 2020, Guiyang, China, August 28-29, 2020, Proceedings, Part I (2020)

13. Lo, S.K., Lu, Q., Zhu, L., Paik, H.Y., Xu, X., Wang, C.: Architectural patterns for the design of federated learning systems. Journal of Systems and Software **191**, 111,357 (2022)

14. McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics, pp. 1273–1282 (2017)

15. Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.M., Rothchild, D., So, D., Texier, M., Dean, J.: Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 (2021)

16. Pilla, L.L.: Scheduling algorithms for federated learning with minimal energy consumption. IEEE Transactions on Parallel and Distributed Systems **34**(4), 1215–1226 (2023)

17. Qiu, X., Parcollet, T., Fernandez-Marques, J., Gusmao, P.P., Gao, Y., Beutel, D.J., Topal, T., Mathur, A., Lane, N.D.: A first look into the carbon footprint of federated learning. Journal of Machine Learning Research **24**(129), 1–23 (2023)

18. Richardson, K., Steffen, W., Lucht, W., Bendtsen, J., Cornell, S.E., Donges, J.F., Drüke, M., Fetzer, I., Bala, G., Von Bloh, W., et al.: Earth beyond six of nine planetary boundaries. Science advances **9**(37), eadh2458 (2023)

19. Riley, G.F., Henderson, T.R.: The ns-3 network simulator. In: Modeling and tools for network simulation, pp. 15–34. Springer (2010)

20. Roy, A.G., Siddiqui, S., Pölsterl, S., Navab, N., Wachinger, C.: Braintorrent: A peer-to-peer environment for decentralized federated learning. arXiv preprint arXiv:1905.06731 (2019)

21. Savazzi, S., Nicoli, M., Rampa, V.: Federated learning with cooperating devices: A consensus approach for massive iot networks. IEEE Internet of Things Journal **7**(5), 4641–4654 (2020)

22. Savazzi, S., Rampa, V., Kianoush, S., Bennis, M.: An energy and carbon footprint analysis of distributed and federated learning. IEEE Transactions on Green Communications and Networking **7**(1), 248–264 (2022)

23. Sze, V., Chen, Y.H., Yang, T.J., Emer, J.S.: Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE **105**(12), 2295–2329 (2017)

24. Verbraeken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., Rellermeyer, J.S.: A survey on distributed machine learning. Acm computing surveys (csur) **53**(2), 1–33 (2020)

25. Won, W., Heo, T., Rashidi, S., Sridharan, S., Srinivasan, S., Krishna, T.: Astra-sim2. 0: Modeling hierarchical networks and disaggregated systems for large-model training at scale. In: 2023 IEEE International Symposium on Performance Analysis of Systems and Software (IS-PASS), pp. 283–294. IEEE (2023)

26. Xie, C., Koyejo, S., Gupta, I.: Asynchronous federated optimization. arXiv preprint arXiv:1903.03934 (2019)

27. Xu, J., Glicksberg, B.S., Su, C., Walker, P., Bian, J., Wang, F.: Federated learning for healthcare informatics. Journal of healthcare informatics research **5**, 1–19 (2021)

28. Yang, T.J., Chen, Y.H., Sze, V.: Designing energy-efficient convolutional neural networks using energy-aware pruning. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6071–6079 (2017)