## 3 Design of SROA
### SROA Protocol Description
In Figure 5, for example, when \(\alpha_{2}\) is 0.2 and communication range is 1500 m, the residual energy variance for \(\alpha_{1}\) = 0.8 is 40% smaller than at \(\alpha_{1}\) = 0.2. The reason is that energy has a stronger impact on the reward function, hence influencing the routing decisions. It is apparent that the greater the value of \(\alpha_{1}\), the more probable it is that a node with more remaining energy will be selected as a forwarder. This is because the energy of the sensor nodes is well-distributed and the network lifetime may be prolonged. Correspondingly, Figures 6 and 7 depict the influence of \(\alpha_{1}\) and \(\alpha_{2}\) of SROA on the average end-to-end delay in a 300 node network with CRs of 1000 m and 1500 m. Comparing the two figures, we can find that the average end-to-end delay decreases with the expanding of CR. The reason is that the greater coverage of transmission makes less sensor nodes involved in the data forwarding, hence packets are routed by a shorter path which reduces the average end-to-end delay. It can be witnessed that an increase of \(\alpha_{2}\) promotes the protocol to choose the node that best balances the factors of residual energy and link quality as the forwarder. As a result, the protocol can converge on the path with the fewest hops, which not only improves energy efficiency but also minimizes end-to-end delay. It is also shown from each individual figure that increasing \(\alpha_{1}\) results in the larger end-to-end delay when \(\alpha_{2}\) is set. This is due to the even energy distribution, indicating that the SROA cannot select the shortest routing path. Specifically, the CR is 1500 m in Figure 7. When \(\alpha_{2}\) is set to 0.8 and \(\alpha_{1}\) is set to 0.2, the average delay is 6.61 s, which is approximately 24% less than that of \(\alpha_{2}\) = 0.2 and \(\alpha_{1}\) = 0.8. By comparing Figures 4-7, it can be seen that the increasing of the CR of sensor nodes decreases both the residual energy variance and the average end-to-end delay. While increasing \(\alpha_{2}\) reduces the end-to-end latency, it also increases the residual energy variance, resulting in a shorter network lifespan. As a result, it can be summarized that a greater value of \(\alpha_{1}\) makes the distributions of energy more uniform, nonetheless, this increases the average network latency. A higher value of \(\alpha_{2}\) indicates less end-to-end delay and greater residual energy variation. The values of \(\alpha_{1}\) and \(\alpha_{2}\) are weighted according to the scenario,

Figure 6: Average end-to-end delay of \(\alpha_{1}\) and \(\alpha_{2}\) (CR = 1000 m). We can observe from Figure 8 that QELAR and GEDAR with the on-site architecture consumes less energy than the original architecture, which reduces the total energy consumption by 25%. One of the reasons is that the data center deployed near the data source shortens the transmission distance thus reducing energy consumption. Besides, with the same architecture, the energy consumption of the QL-based QELAR protocol is lower. Due to its constant cost, it tends to choose the shortest path to forward data packets. In addition, it can also be seen that the Q-Learning-based QELAR protocol consumes less energy with the same architecture, because it usually chooses the more optimal path to forward data packets in a global view. The average end-to-end delay of the protocols with different architectures is shown in Figure 9. Corresponding to Figure 8, the end-to-end delay with the on-site architecture is significantly reduced. The reason is similar to Figure 8, with the on-site architecture, data packets are forwarded with fewer hops and there are fewer nodes participating in the route, thereby reducing the average end-to-end delay. As a result of the on-site deployment, the protocols with new architecture show apparent improvements in terms of total energy consumption and average end-to-end delay. Furthermore, the Q-Learning-based protocols also outperform classical protocols.