# Reinforcement Learning for Efficient Drone-Assisted Vehicle Routing
## 5 Computational Studies
### Results on the VRPD with Limited Flying Range  
The impact of restricted flying ranges was systematically explored by adjusting a crucial parameter denoted as \(\beta\). This parameter plays a vital role in defining the operational capabilities of the drone, specifically determining the maximum distance it can cover with a fully charged battery. The outcomes of these scenarios, detailed in Tables 2 and 3, provide detailed insight into how the system performs under limited flying distances. This specific variable is adopted based on empirical setups from [9]. In that study, \(\beta=0.6\) was tested as a moderately restrictive constraint, limiting drones to 60% of the largest distance between any two nodes, while \(\beta=1\) represented a more lenient scenario where drones could cover the entire maximum inter-node distance. By using these two values, conducted experiments align with established benchmarks, allowing for direct comparisons with previous results. Additionally, these \(\beta\) levels highlight how the proposed method adapts to different drone battery constraints: \(\beta=0.6\) simulates stricter capacity limitations, whereas \(\beta=1\) reflects drones capable of covering any route segment in a single trip. This setup helps evaluate how varying operational flexibility influences solution quality, route feasibility, and computational efficiency. **Scenario 1:**\(\beta=1\)  
In the first scenario, where \(\beta\) is set to 1, indicating the drone's maximum flight distance aligns with the largest distance between nodes, the results are showcased in Table 2.  
In smaller scenarios with 10 nodes, the Ensemble RL model outperforms Solver and Solver Lazy in terms of time and quality across all configurations. Compared to the Heuristic approach, Ensemble RL achieves a 1.04% gap in the first setup, showcasing nearly 60 times faster performance. The second configuration reveals a 2.03% gap with approximately a 16-fold speed boost, while the third setup shows a 2.3% gap with around 15 times faster speed. In larger-scale problems involving 20 and 50 nodes, Ensemble RL demonstrates superior solution efficiency and time performance compared to the Heuristic method in the initial configuration. However, in the second and third configurations for \(\mathcal{N}=20\), the gaps are 3.09% and 3.58%, respectively. For \(\mathcal{N}=50\), Ensemble RL outperforms the Heuristic method in all configurations. **Scenario 2:**\(\beta=0.6\)  
Transitioning to the second scenario is characterized by setting \(\beta\) to 0.6, thus constricting the drone's maximum flight distance to 60% of the largest distance between nodes. The results in Table 3 illuminate additional insights into the system's adaptability and efficiency under different operational constraints. Ensemble RL consistently outperformed Solver and Solver Lazy in our experiments with smaller instances across all configurations, demonstrating its reliable efficiency. Ensemble RL achieved a 1.28% gap compared to the Heuristic in the first setup. In the second configuration, Ensemble RL Sampling reduced the gap to 0.08%. In the third configuration, the Ensemble model showcased its ability to find a solution with only a 0.12% gap, and it did so with a significant approximately 10-fold increase in speed. Shifting to more significant instances with 20 nodes, Ensemble RL sustained its superiority over the Heuristic in the first setup, excelling in speed and achieving the best solutions. This outstanding performance extended to instances with 50 nodes. Ensemble RL consistently outperformed the Heuristic in both the first and second setups, highlighting its scalability and effectiveness in handling more intricate problem scenarios. Considering the third scenario, RL Sampling outperformed the Heuristic with only a 0.18% optimality gap. Under limited flight ranges (like when \(\beta=0.6\)), data shows that drones tend to stay closer to the trucks. This limits their ability to carry out independent missions.