## 5 Performance Evaluation
the exploration-exploitation trade-off in the learning process, set to 1.0. \"Epsilon-Decay\" determines the rate at which epsilon (exploration factor) decreases over time, set to 0.95. \"Epsilon-min\" specifies the minimum value for epsilon during exploration, set to 0.05. \"Batch-size\" sets the number of samples used in each iteration of model training, set to 32. \"Train-starter\" defines the episode at which training begins, set to 200, and \"Reply-Memory\" used to store the agent experiences, is set to 100,000. **SUMO Simulation Environment:** As highlighted in our preliminary section, our study is evaluated through SUMO with smart navigation agents to create a unified experimental environment. SUMO supports the seamless simulation of real-world traffic conditions while concurrently incorporating the decision-making processes of the agents. It also provides two primary methods for network creation: using OpenStreetMap or utilizing the graphical network editor NetEdit. Our network is constructed using the NetEdit method, where we import network and demand traffic XML files, defining the network's topology and geometry. In Figure 4, we illustrate the graphical scenario (Figure 3(a)) alongside the corresponding network visualized within the SUMO simulator (Figure 3(b)). Our network consist of 15 edges, each with distinct properties such as speed limit and length. It is noteworthy that SUMO categorizes vehicles into two types. The first consists of randomly generated vehicles, following a specific probabilistic approach based on Origin-Destination (OD) points to simulate dynamic traffic congestion. These vehicles are marked as red and blue in Figure 3(b), representing trucks and regular cars, respectively. The second group comprises agent vehicles operating within the environment with the purpose of learning and selecting optimal actions. These agent vehicles are represented as yellow in Figure 3(b). To simulate real-time traffic conditions, agents start from various locations but share a common destination, denoted as \(E_{15}\) in Figure 3(a). Additionally, we utilize SUMO's randomTrip.py feature to generate random traffic within the network, specified as \([b-0-e-3600-period-3]\), which indicates that the simulation will run for 3600 s, generating a random vehicle every 3 s until the simulation ends. The simulation involves a total of 1200 non-intelligent vehicles and 5 intelligent agents. This setup ensures realistic traffic conditions while providing a controlled environment for evaluating the performance of intelligent agents. To validate the practicality of the FLDQN algorithm, we conduct several experiments. In each training step, the framework acquires environmental observations from SUMO and forwards them to the DRL agent. The DRL agent, based on these observations, evaluates traffic conditions and uses its policy neural network to make decisions. The algorithm then executes these actions, updating the state and progressing in SUMO until the simulation concludes. At the end of each simulation run, the reward is calculated and sent to the DRL agent for optimization. Figure 4: Figure **(a)** is graphical representation of our scenario. We see the actual network operating in the SUMO simulation in Figure **(b)**. Two types of vehicles are populated in the simulation: yellow ones represent our agent vehicles; red and blue ones are the randomly generated buses and normal cars.