## 5 Experimental Evaluation  
### Experimental Setup  
The relevant experiments of the UA-FedRL model were carried out on Ubuntu 18.04 desktop with AMD Ryzen 5 3500x cpu @ 3.6 Hhz, Nvidia RTX 3080, and 32 GBytes of RAM. Relevant codes were written in Python scripts using Python 3.8. To design a communication network, a Mininet emulator was utilized. The Mininet emulator is programmed in the Python language and is publicly accessible for researchers at the Mininet official website ([[http://mininet.org/](http://mininet.org/)]([http://mininet.org/](http://mininet.org/)), accessed on 1 August 2024). It uses the Linux kernel to create a real network with virtualized end-hosts, switches, routers, and links. It provides built-in support for the SDN architecture. Two commonly used datasets were selected to train local IoT devices. The local datasets were distributed with a non-IID distribution across IoT devices for creating heterogeneous scenarios. A Convolutional Neural Network (CNN) model was designed and distributed to all participating IoT devices for the purpose of training their local models. The architecture of this CNN comprised two convolutional layers, each with a kernel size of five. The optimization of this network was carried out using the SGD method, with a learning rate set at 0.01. For the UA-FedRL algorithm, multiple parameters were meticulously chosen to thoroughly investigate its performance under various conditions. The learning rate for UA-FedRL's \(\alpha\) was set at values of 0.1, 0.5, and 0.9. These values were selected to study the algorithm behavior under gradual, moderate, and rapid learning processes. Similarly, the discount factor \(\gamma\) was set at 0.1, 0.5, and 0.9 to analyze the agent's preference for immediate rewards as opposed to long-term gains. The episode counts \((T)\) were chosen as 20 and 100, in order to discern the relationship between the number of learning iterations and the overall performance of the algorithm. The UA-FedRL model was benchmarked against several state-of-the-art FL models, including FedAVG, FedProx, FedShare, and FedSGD. Default parameter values, simulation environment, and models' hyperparameter settings used for the experiments are presented in Table 2.  
The performance evaluation of our UA-FedRL method was conducted by simulating 100 client devices, each with varying resource capabilities. Five Raspberry Pi models were selected for this test due to their common use as IoT devices in real-world environments. These Raspberry Pi variants offered a range of CPU frequencies, from 700 MHz to 1.5 GHz, and varied battery capacities, from 60% to 100%. The Raspberry Pi 4 Model B was identified as the primary IoT device, while the remaining devices were categorized as stragglers. Detailed hardware specifications for each chosen device can be seen in Figure 5.  
A communication simulator was also introduced to both client- and server-side training to simulate communication between server and client. The Mininet 2.3.0 Python simulator was used to configure every client as a host, which is similar to IoT devices in a real-world environment. The model parameters from each host were then converted to a byte stream prior to being sent to the server. The byte stream was then divided into packets according to the size of the packet. These packets were then sent to the server