## 3 Proposed RL-HCPV
\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Elements**} & **Electricity Bill** & \multirow{2}{*}{**Electricity Bill with V2H and RL**} & **Electricity Bill with without V2H with RL** & **Electricity Bill with V2H and RL** \\ \hline PV generation & 10 kw & 10 kw & 10 kw & 10 kw \\ Max. load & 1.4 kw & 1 kw & 1.2 kw & 0.8 kw \\ ToU tariff & 00.18 SAR & 00.18 SAR & 00.18 SAR & 00.18 SAR \\ RTP tariff & 00.31–00.47 SAR & 00.31–00.40 SAR & 00.31–00.38 SAR & 00.31–00.36 SAR \\ Penalty & 0.05 SAR & NA & 0.05 SAR & NA \\ Total electricity charge & 34.8 SAR & 17.4 SAR & 25.2 SAR & 14.7 SAR \\  & 9.28 \$ & 4.64 \$ & 6.72 \$ & 3.92 \$ \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of electricity bills with and without V2H and RL. Figure 9: Dependence on ToU/RTP with/without V2H. **(a)** Case 1 during sunny days **(b)** Case 1 during cloudy days **(c)** Case 2 during sunny days **(d)** Case 2 during cloudy days

### Analysis of the Computational Capabilities

A mathematical HEMS model typically consists of discrete and continuous binary variables. The MILP problem related to HEMS was calculated using a number of heuristic algorithms, including differential evolution (DE), binary particle swarm (BPS), particle swarm optimization (PSO), binary learning differential evolution (BLDE), and genetic algorithm (GA). However, the computing demands of these approaches and algorithms are high and require tremendous processing power to meet the requirements for the best solution. As a result, the ideal convergence of these methodologies and algorithms is considered unstable. Hence, this study utilized DL and RL to overcome these issues. Thus, Table 8 compares the obtained DL and RL results with the GA algorithm used to analyse our proposed model's productivity and effectiveness. The reduced cost statistics and average computational time of GA have been obtained for scheduled power optimizations. The result indicates that the RL generates noticeably better computational results in less time. ## 5 Conclusions

This study proposed an RL-HCPV system for intelligently managing the DR of a smart home. The proposed model aims to make optimal decisions during on-peak and off-peak power supply to minimize costs and ensure sustainable electricity for the appliances. Several DL algorithms have been applied to the real-time data to deal with the uncertainty of PV generation, appliance power consumption, prices, EV availability, and SOC. DL algorithms have examined each uncertainty parameter to determine the most accurate model. Consequently, ANN is applied for the PV generation, ToU, and RTP tariffs. A multi-target regression is applied to predict EV availability and SOC. In addition, LSTM is applied to predict appliance energy consumption. Finally, the predicted data are transferred into the RL algorithm, the fuzzy Q-learning model, for efficient DR decision-making. Moreover, simulations demonstrate that:

* When the V2H unit is included, the proposed RL-HCPV algorithm can minimize the reliance on grid prices by 38% on sunny days and 24% on cloudy days;
* Using both V2H and RL is a most effective way to lower the electricity bill;
* RL generates noticeably better computational results in less time. Future analyses should consider consumer-friendly actions, such as selling surplus electricity to the grid to boost user revenue. In addition, smart MGs require an autonomous sustainable power supply plan for household appliances. [PERSON] planned the presented idea and wrote the manuscript with support from S.A.b.S. and B.A.Z., who contributed to the original draft preparation, supervised the findings and final revisions. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. The authors declare no conflict of interest.