# Deep Q-Learning-Based Smart Scheduling of EVs for Demand Response in Smart Grids
## 4 Evaluation Results
As in the case of the L1 scenario, the policy learned in the case of the L2 scenario can make optimal decisions in most cases, regarding the allocation of electric vehicles to the charging stations. The tracking of the evolution of the percentage of optimal allocations of electric vehicles over episodes is shown in Figure 6.  
Examining the results obtained in the case of the L1 model, we notice that after 9500 episodes, the algorithm can optimally allocate EVs in 35.7% of the cases, reaching an optimal allocation of 70.8% after 160,000 episodes. Similarly, for the L2 model, an optimal EV allocation of 82.9% begins after 160,000 episodes. Consequently, in both models, the  
Figure 5: (**a**) The percentage of optimal/suboptimal allocations of EVs for the policy learned in the case of the charging scenario; (**b**) The percentage of optimal/suboptimal allocations of EVs for the policy learned in the case of the discharging scenario. percentage of optimal allocations for EVs increases with each passing episode throughout the learning process, which demonstrates the effectiveness of the learned policies. The final step of the learned-policy evaluation involves assessing the effectiveness of EV allocations at charging stations in each time slot throughout the scheduling period. Figure 7 presents the curve resulted by scheduling EVs at charging stations, together with the target energy curve provided by the grid operator, as well as the baseline curve in the case of charge and discharge scenarios. Analyzing these graphs, we notice that for both learned models (i.e., L1 and L2), the curve obtained from scheduling EVs closely follows the target energy curve provided by the grid operator. Also, when we schedule the charging/discharging of electric vehicles with our method, the resulting energy curve is much closer to the target energy curve. EV scheduling for charging/discharging significantly improves the charging/discharging capacity compared to the baseline scenario, where EV charging/discharging is conducted without any prior appointment. In the charging scenario, the variation between the energy required to charge electric vehicles and the energy supplied by the network operator in each time slot does not exceed a maximum deviation of 6 kW, indicating an optimal allocation of electric vehicles at charging stations. This high level of alignment between the two curves is reflected by the value of the Pearson correlation coefficient of 0.9926, which indicates a strong similarity between the energy curves. In the discharge scenario, the situation is similar. In this case,  
Figure 6: (**a**) The evolution of the percentage of optimal allocations of electric vehicles over episodes for the L1 model; (**b**) The evolution of the percentage of optimal allocations of electric vehicles over episodes for the L2 model. Figure 7: (**a**) Comparison between the microgrid energy curve and the resulting curve after scheduling EVs in the case of the L1 model; (**b**) Comparison between the microgrid energy curve and the resulting curve after scheduling EVs in the case of the L2 model. the Pearson correlation coefficient shall be 0.9937, and the variation between the energy discharged by electric vehicles and the network operator's energy requirements during each time slot does not exceed a maximum deviation of 4 kW.