## 3 Proposed RL-HCPV
#### 3.2.2 Case2:

Smart MG with V2H unit

Depending on the link between the overall power usage, the price of electricity, and the Q-values, the smart MG, in this instance, is made up of PV solar panels, SB, V2H units, and appliances. All of the executed activities are depicted in Figure 6b. First, the RL-HCPV checks the EV availability and SOC efficiencies. Thus, the V2H discharging mode is initiated when appliance power demand is high, and PV power output is low, typically between 7:00 and 9:00 p.m. when prices are also high. Thus, the current state is \(s_{t}\) = V2H discharge; low; high; expensive; expensive. Figure 6: (a) Actions during Case 1. (b) Actions during Case 2. ### Performance of the Proposed RL-HCPV

* _Performance of the DL Algorithms_

The performance of the LSTM model for PV generation is shown in Figure 7a. In addition, the hourly prediction of the appliances' consumption using ANN compared with actual consumption is shown in Figure 7b. The multi-target regression for the hourly EV availability and SOC prediction is compared with the actual power generation, as shown in Figure 7c,d. Finally, the performance of the ANN model for the price tariff is shown in Figure 7e,f. The blue line in these graphs reflects the hourly predicted data, whereas the orange line represents the actual values. The actual and predicted data are similar, while slight inaccuracies can be observed in certain time intervals. In summary, the accuracy of the applied DL algorithms is listed in Table 6. As a result, the suggested DL algorithms for each uncertainty parameter generate precise and realistic predicted values, that can help in the decision-making process for DR management. Figure 7: Actual and predicted (**a**) PV generation; (**b**) appliance consumption; (**c**) EV SOC; (**d**) EV availability; (**e**) ToU pricing; and (**f**) RTP. * Performance comparison with/without V2H

While implementing the model, the main focus was minimizing costs and customer discomfort by ensuring sustainable power delivery. This is achieved by comparing the appliance consumption in various situations to the ToU and RTP tariffs. The hourly appliance power consumption shown in Figure 9 illustrates the effectiveness of the proposed RL-HCPV algorithm. As displayed in this figure, when the V2H unit is not included, the dependence on grid prices grows from roughly seven hours on sunny days to 15 h on cloudy days. In contrast, when the V2H unit is included, the dependence on grid prices on sunny and cloudy days is around three hours. The figure shows that when the V2H unit is incorporated, the suggested solution decreases reliance on grid costs by 38% on sunny days and 24% on cloudy days. Thus, integrating the V2H unit reduces reliance on the utility, indicating a significant reduction in electricity costs. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Uncertainty Parameters** & **DL Algorithm** & **MAE** & **MSE** \\ \hline PV generation & LSTM & 0.37 & 0.27 \\ Appliance consumption & ANN & 0.21 & 0.22 \\ EV SOC and availability & Multi-target Regression & 0.31 & 0.15 \\ ToU tariffs & ANN & 0.37 & 0.32 \\ RTP tariffs & ANN & 0.0013 & 0.0066 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accuracy of the DL algorithms. Figure 8: The convergence of the Q-value. ### Electricity Bill Analysis

According to the Saudi Electricity Company (SEC), the energy consumption is calculated using the following equation:

\[\frac{load}{30\ days}\times actual\ days \tag{35}\]

Table 7 compares the prices for a day of 14 KW consumption in four scenarios. The comparison shows that including the V2H unit can remove the penalties and significantly reduce the bill compared to other cases that do not include the V2H. Furthermore, when using RL, scheduling consumption when costs are the lowest significantly impacts RTP, resulting in lower costs than in other situations. Therefore, using both V2H and RL is the most effective way to lower the electricity bill.