## 4 Performance Evaluation
### Simulation Settings  
In subsequent sections, we detail our experimental setup and the default simulation parameters. Our primary focus is to evaluate the performance of the DOS-RL routing algorithm using three widely-accepted network performance metrics. Additionally, we compare its performance against two state-of-the-art routing protocols: OSPF and SDN-Q. For our evaluations, we establish a network with a varying number of wireless sensor nodes randomly distributed within a fixed-size simulation area. Each sensor node has the capability to choose multiple relay nodes for data forwarding. We installed traffic source applications on a varied number of nodes, ranging from 2, 4, 6, 8, to 10. This diverse setup allows for a comprehensive analysis of the DOS-RL routing algorithm's performance across various network scenarios. To determine the optimal paths from each source node to the target node, we employ the Q-learning algorithm to evaluate the set of all available loop-free paths. These paths  
Figure 3: DOS-RL SDWSN Architecture. are identified by the STP algorithm for a specified pair of source-destination nodes. Our experimentation begins by comparing the performance of different RL-based routing schemes. This includes the DOS-RL without shaped rewards, DOS-RL with shaped rewards (DOS-RL Shaped), and the traditional RL which utilizes a linear combination of multi-objectives based on the weighted sum approach (RL). The objective of our experiments is to observe and compare their performance on a pathfinding problem in a five by five dynamic Gridworld environment. Key performance indicators for evaluating these algorithms include their convergence speed by observing the average rewards collected; average energy consumed, which reveals how efficient the agent is in conserving energy while completing the task; the average episode length which shows the average number of steps taken by the agents to complete episodes; and finally, the average frequency of hitting obstacles which indicates how well the agents avoid obstacles in the environment. During experiments, the reward function of the RL includes three objectives: reducing the number of steps taken to reach the goal state, avoiding obstacles, and optimizing charging points (optimizing energy consumption). The DOS-RL with shaped rewards incorporates the additional shaping reward function based on the Manhattan distance to the goal state in order to guide the learning process and speed up the learning process. The experiment design integrates varied states such as states with obstacles, states with relatively higher energy consumption, recharging states, and normal states. The learning agent starts with a five-energy unit reservoir and loses energy by 0.1 units at each regular state transition. Encountering any of the dynamically selected obstacles (of a maximum of three after every 100 episodes) results in a penalty of 20 units. Conversely, navigating through a recharge point yields a reward of five units. An agent that runs out of energy while navigating incurs a penalty of 20 units. The agent's navigation starts at the state (0,0) with the ultimate objective of reaching the goal state (4,4) through the shortest and most energy-efficient path. This controlled setting allows us to examine the performance of these algorithms in a well-defined scenario. The values of all important parameter settings used during the experiment are summarized in Table 1.  
To evaluate the convergence speed of the algorithms, Figure 3(a) plots the learning curve for the average rewards collected and the average energy consumed per number of episodes in Figure 3(b). In Figure 3(c), we can see plots for the average episode length and the average frequency of hitting obstacles per number of episodes during the entire simulation time. In Figure 3(a), we can see how the poor performance of the RL algorithm is affected by its frequency of hitting obstacles during the first 100 episodes.