###### Abstract
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Ref.** & **Algorithm** & **Prediction** & **ML Type** & **Real-Time** \\ \hline
[19] & Two classification problems and one regression model & Predict the EV availability to provide (V2H) services & DL & - \\ \hline
[41] & A stochastic programming approach based on Wasserstein distance metric and K-medoids & Manage the residential PV solar power uncertainty & AI & - \\ \hline
[42] & Regression & Predicted consumed energy in Madrid & DL & - \\ \hline
[43] & Binary particle swarm optimization (BPSO) & Minimizes the energy cost, peak to average ratio (PAR), and customer dissatisfaction & AI & Real-time appliance consumption \\ \hline
[33] & Energy balance control algorithm & Characterize the source details utilizing energy demand & AI & - \\ \hline
[44] & Twin delayed deep deterministic policy gradient (TD3) learning approach & DR management strategy & Deep RL & Real-time DRM \\ \hline
[45] & Controller algorithm & Prediction for the trip & AI & - \\ \hline
[46] & artificial neural network (ANN) & Day-ahead solar PV generation and household load & RL & - \\ \hline
[47] & Double-stage mixed-integer linear programming (MILP) [neural network] & Optimal energy trading in real-time and day-ahead energy signals & RL & Real-time energy markets \\ \hline
[48] & Long short-term memory (LSTM) [Type of recurrent neural network] & Short-term prediction of PV power production & RL & - \\ \hline  & A stochastic model predictive control strategy consists of a neural network, recurrent neural network (RNN), radial basis function neural network & Short-term prediction of PV power production & RL & - \\ \hline
[49] & Network (RNN) and convolutional neural network (CNN) & & &
\begin{tabular}{c} Operates in real-time and thus \\ minimizes the effects of the gap \\ between the forecasted and real \\ data on the home energy \\ management system performance \\ by updating its control decisions \\ and the forecast data as the \\ stochastic parameters are realized \\ in each time step \\ \hline
[50] & MILP [neural network] & Day-ahead load scheduling provides optimal DR, cost-reduction, and PV self-consumption & RL & - \\ \hline
[51] & A stochastic adaptive dynamic programming (ADP) & Reduces the energy cost & AI &
\begin{tabular}{c} Includes the status of appliances \\ and occupants in real-time \\ for execution \\ \end{tabular} \\ \hline
[52] & A model predictive HEMS &
\begin{tabular}{c} Optimizes the charge and \\ discharge profile of an EV and the \\ operation of an HPWH to \\ minimize excess electricity \\ considering the disconnection of \\ the in-vehicle storage battery \\ \end{tabular} & AI & - \\ \hline
[53] & Q-learning &
\begin{tabular}{c} Reduces the electricity bill within \\ the consumer comfort level and \\ the appliance \\ operation characteristics \\ \end{tabular} & DL and RL & - \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._Despite extensive research on residential energy management using the RL method, real-time smart energy scheduling algorithms, considering the operation of multiple smart home appliances, PV solar generation, SB, and V2H units, have not been studied. Thus, this study proposes a home centralized photovoltaic (HCPV) technique that uses deep learning (DL) and a model-free RL algorithm, the fuzzy Q-learning algorithm. The following are the main contributions of this paper:  
* The proposed RL-HCPV model optimizes the energy usage of a smart home equipped with PV solar panels, stationary batteries, EVs, smart appliances, and V2H units;
* Various DL algorithms predict PV power generation, appliance consumption, and prices one hour in advance. Then, an RL algorithm is employed to make efficient power decisions based on the predicted data. Finally, the RL-HCPV utilizes fuzzy Q-learning to make optimal decisions during the on-peak and off-peak power supply to reduce costs and provide sustainable power for the appliances.