## 6 Experimental Results
### Results of the Mathematical Model
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Instance Name** & **Distance (m)** & **CPLEX Time (s)** & **Routes** \\ \hline \multirow{6}{*}{ESOCU\_R40} & \multirow{6}{*}{* 18,276.00} & & R1: Depot-10-11-12-13-23-20-Depot & R2: Depot-38-41-Depot \\  & & & R3: Depot-46-24-22A-27-28-Depot & R4: Depot-74-76-124-88-cs3-73-60B/2-Depot \\  & & & R5: Depot-119-1-113-125-8-Depot & R6: Depot-120-19-17-22C-21-Depot \\  & & & R7: Depot-42A-42C-154-53-153-Depot \\  & & & R8: Depot-43B-122-61B-64-57-44-Depot \\ \hline \multirow{6}{*}{ESOCU\_RC40} & \multirow{6}{*}{* 17,053.00} & & R1: Depot-2-8-10-11-13-15-20-Depot \\  & & & R2: Depot-19-17-css-125-13-115-Depot \\  & & & R3: Depot-38-35-119-3-25-28-39-Depot \\  & & & R4: Depot-49-50-120-21-24-22A-Depot \\  & & & R5: Depot-42B-57-60B/2-55-Depot \\  & & & R6: Depot-42C-152-153-Depot \\  & & & R7: Depot-58/1-58/1-61B-64-76-124-51-Depot \\ \hline \multirow{6}{*}{ESOCU\_C60} & \multirow{6}{*}{* 26,773.00} & & R1: Depot-3-25-115-1-36-37B-Depot \\  & & & R2: Depot-10-23-16-21-Depot \\  & & & R3: Depot-34-2-4-5-113-28-39-Depot \\  & & & R4: Depot-38-30-18-19-118-Depot \\  & & & R5: Depot-47-26-116-35-119-32-Depot \\  & & & R6: Depot-49-14-9-12-13-22C-Depot \\  & & & R7: Depot-56-61A/1-61B-65-70-71-Depot \\  & & & R8: Depot-59-60A/2-60B/2-43C-31-40-Depot \\  & & & R9: Depot-62-68-69-66-63-Depot \\  & & & R10: Depot-154-53-52-42C-Depot \\  & & & R11: Depot-42-A7-242-22A-Depot \\ \hline \multirow{6}{*}{ESOCU\_R60} & \multirow{6}{*}{* 26,159.00} & & R1: Depot-24-114-5-113-Depot \\  & & & R2: Depot-16-8-css-12-13-23-20-Depot \\  & & & R3: Depot-25-116-27-28-29-30-Depot \\  & & & R4: Depot-38-40-41-50-Depot \\  & & & R5: Depot-59-62-67-68-73-72-63-Depot \\  & & & R6: Depot-74-65-122-56-57-60A/2-60B/2-Depot \\  & & & R7: Depot-75-79-78-85-17-88-1 Depot \\  & & & R8: Depot-123-82-84-52-Depot \\  & & & R9: Depot-154-53-153-24A-Depot \\  & & & R10: Depot-37A-45-19-22D-33-36-37B-Depot \\  & & & R11: Depot-43A-44-43C-42B-Depot \\ \hline \multirow{6}{*}{ESOCU\_RC60} & \multirow{6}{*}{* 27,905.00} & & R1: Depot-13-14-10-16-23-Depot \\  & & & R2: Depot-25-26-116-27-28-Depot \\  & & & R3: Depot-29-30-31-38-119-Depot \\  & & & R4: Depot-34-114-5-113-8-4-Depot \\  & & & R5: Depot-45-24-22D-22A-33-36-37B-Depot \\  & & & R6: Depot-50-19-18-22C-21-Depot \\  & & & R7: Depot-52-65-56-57-122-61A/1-61B-Depot \\  & & & R8: Depot-77-79-88-cs3-60A/2-154-53-153-Depot \\  & & & R9: Depot-43A-62-68-73-63-59-Depot \\  & & & R10: Depot-43C-42B-40-Depot \\  & & & R11: Depot-60B/2-78-123-81-Depot \\ \hline \hline \end{tabular}  
* It is not the optimal solution. In large-scale problems, the mathematical model could not provide solutions in a reasonable time. The results were obtained by CPLEX in a three-hour time limit. Exact solutions obtained for problems with 5 and 10 customers.  
\end{table} 
Table 3: _Cont._Mathematical models provide an exact formulation of routing problems, such as the Electric Vehicle Routing Problem (EVRP), by explicitly defining constraints and objective functions. In contrast, Q-learning transforms the same problem into a reinforcement learning framework, representing it through actions, states, and rewards. While mathematical models aim to find an exact solution to the problem, the Q-learning algorithm approaches the objective function value by conducting a learning process based on rewards and penalties. In mathematical models, constraints are explicitly defined, and the solution is required to strictly adhere to them. However, in Q-learning, these constraints are indirectly learned through the reward function. While mathematical models yield exact solutions, Q-learning, as a learning-based approach, produces solutions that improve and converge over time. This feature is particularly advantageous for solving large-scale and complex problems where exact methods may become computationally prohibitive.