# Reinforcement Learning for Efficient Drone-Assisted Vehicle Routing
## 2 Literature Review
### The VRPD  
The literature on the VRPD and its extensions has explored various aspects of optimizing time and cost savings. Ref. [6] conducted a worst-case analysis for the VRPD, proposing bounds for achieving maximum benefits in terms of time savings. Their analysis considered the same distance metrics for both trucks and drones, with no limitations on drone batteries and the ability to dispatch and retrieve drones from the same node. The results revealed that savings depended on the number of drones per truck and the drones' speed. Building upon [6]'s work, ref. [8] extended the analysis by considering limited batteries for drones, different distance metrics for trucks and drones, and demonstrated maximum time and cost savings based on bounds. Ref. [14] introduced two heuristics for the VRPD, involving construction and improvement phases to minimize makespan. In a further study, ref. [9] introduced en route operations, enabling drones and trucks to rendezvous at discrete points along the truck route and schedule new deliveries with negligible drone recharging time. They proposed a new Mixed-Integer Linear Programming (MILP) model and introduced valid inequalities to solve small instances for optimality. Additionally, they suggested a heuristic based on Variable Neighborhood Search (VNS) and Tabu Search (TS) concepts for large instances, which numerically proved the benefits of en route operations in reducing makespan. Notably, they assumed that drones could not be retrieved at the location where they were launched. Ref. [7] expanded on the work of [6] by proposing an arc-based integer programming model for the VRPD, reformulating it as a path-based model, and developing a branch-and-price algorithm for finding minimum logistics costs for up to 15 instances. Through sensitivity analyses, they concluded that a 10% logistics cost reduction could be achieved with drones with higher flying durations. In another study, ref. [15] proposed a new MILP model for solving the VRPD, allowing cyclic operations for drones, which could be launched and retrieved at the exact customer location, increasing complexity. The authors introduced valid inequalities and proposed a metaheuristic to divide problems into components, such as allocation and sequencing, which were further solved by the classical savings heuristic. They addressed problems with up to 100 customers. Beyond using heuristics and MILP models, ref. [16] tackled a similar problem to the VRPD using an encoder-decoder framework combined with RL. In their approach, deliveries were made exclusively by drones, while trucks acted as portable warehouses and charging stations. Their extensive experiments demonstrated that RL could produce satisfactory solutions in a significantly shorter time than heuristics. Moreover, many existing studies focus on specific operational scenarios, such as single-drone configurations or simplified battery constraints, and do not always consider advanced real-world applications like partial recharge or multi-objective routes. Drones have been extensively used in diverse fields, including telecommunications, healthcare, and urban surveillance outside classic truck-drone logistics. For example, ref. [17] present a MILP-based framework for drone routing and recharging as flying base stations, demonstrating how UAVs can serve mobile networks with grid-connected microgeneration constraints. In a broader survey, ref. [18] provides an exhaustive overview of optimization approaches for civil applications of UAVs, highlighting how routing variants extend to firefighting, emergency response, and beyond. In healthcare logistics, ref. [19] introduce a bi-objective optimization model for simultaneous pickup and delivery of medical supplies with drones,employing a modified NSGA-II algorithm to balance conflicting objectives. Likewise, ref. [20] proposes a decomposition-based approach to coordinate UAV tours for city-scale video surveillance, where drones even ride public buses for energy-efficient mobility.