## 5 Evaluation Tasks and Metrics
### Task Details
#### 5.1.1 AGV-Drone Collaborative Scheduling Task  
The AGV and drone collaborative scheduling task is formulated as a discrete-time Markov Decision Process (MDP), where the state, action, transition, and reward components are defined to capture the multi-agent coordination dynamics across three logistical zones: A (storage), B (processing), and C (buffer). An overview of the case is presented in Figure 2.  
The state space \(\mathcal{S}\) encodes a complete snapshot of the system status. Each state \(s\in\mathcal{S}\) is defined as a tuple \(s=(A,B,C,V,E,T)\), where \(A=(P_{A},T_{A})\) represents binary availability vectors for parts and trays at location A. The status of processing stations at location B is given by \(B\in\mathbb{R}^{n_{B}\times 4}\), storing information such as the assigned part/tray ID, completion time, and station status. Similarly, \(C\in\mathbb{R}^{n_{C}\times 3}\) records the contents of the buffer, including reservation flags. The vehicle fleet is captured by \(V\in\mathbb{R}^{n_{V}\times 5}\), where each row specifies a vehicle's location, current load, travel time, and type (AGV or drone). Battery levels are maintained as \(E\in[0,E_{\max}]^{n_{V}}\), and internal clocks as \(T\in\mathbb{R}^{n_{V}}\), tracking elapsed time per vehicle.  
The action space \(\mathcal{A}\) allows each vehicle to choose from a discrete set of operations: \(a\in\{0,1,2,3,4\}\), corresponding, respectively, to delivering a part from A to B, returning an empty tray from B to A, transferring an item from A to C, moving from C to B for processing, or initiating a charging procedure. The effects of an action are captured by the transition model \(P(s^{\prime}\mid s,a)\), which updates location, energy, and timing variables. Travel time for each path is computed as \(\Delta t=t_{\text{travel}}+t_{\text{wait}}\), where duration depends on the selected route, current congestion (indicated by a binary variable \(\mathbb{I}(\text{jam})\)), and vehicle type. Drones operate faster than AGVs, modeled via a scaling factor \(\alpha_{d}\). Processing actions at location B can fail with a probability \(p_{\text{fail}}\), and charging actions restore energy to \(E_{\max}\).  
The reward function \(R(s,a,s^{\prime})\) penalizes delay, distance, and task failures. Specifically, the immediate reward is computed as  
\[R=-(\alpha_{d}\cdot d+\alpha_{t}\cdot t+\alpha_{p}\cdot p_{\text{fail}}),\]  
where \(d\) is the travel distance, \(t\) is the task delay, and \(\alpha_{d}\), \(\alpha_{t}\), \(\alpha_{p}\) are weighting coefficients.  
Battery dynamics are governed by task-dependent consumption and recharging behaviors. A movement of distance \(d\) incurs energy loss \(E_{i}^{\prime}=E_{i}-\gamma\cdot d\), where \(\gamma\) is the energy-per-distance coefficient. Charging resets the level to \(E_{\text{max}}\). A vehicle can only execute a task if its current energy \(E_{i}\) exceeds a required threshold \(e_{\text{req}}\). Furthermore, the problem is subject to several operational constraints. Each vehicle can carry at most one part and one try simultaneously. Locations B and C have limited buffer and processing slots. Energy sufficiency is mandatory for movement, and processing at B requires available stations. The overall objective is to maximize completed part throughput while minimizing total energy consumption and cumulative task latency, yielding a trade-off between efficiency and speed under constrained coordination. The concrete parameters employed are shown in Table 1.