###### Abstract
Moreover, the V2H discharging mode can utilize the excess energy from the EV battery more efficiently. For example, in V2H, residences consume the excess energy from the EV battery through appliances. Therefore, the HEMS manages smart appliances and EVs. In addition, a smart meter is connected to the HEMS to support the bi-directional information exchange between the customers and the utility service provider. Moreover, the HEMS can rely on a prediction algorithm to determine the charging and discharging periods. The HEMS charges the EV battery for cost reduction when the price is low. In contrast, the HEMS schedules the discharge of the EV battery to power home appliances, and avoids purchasing energy from the utility service provider when the price is high. Furthermore, this strategy serves as an emergency energy source during a power outage. Other studies have indicated that EVs and V2H technologies were expected to be used as domestic electricity storage systems [11]. Thus, it is crucial to promote PV power self-consumption. In addition, the authors identified V2H as a technology that uses an EV battery as a controlled voltage source to supply home loads during power outages. As a result, EVs provide an uninterrupted power supply, increasing grid reliability. The study compared V2H and V2G, demonstrating that V2G has a complex infrastructure, that increases losses due to location distance. Therefore, the EV application in V2H is more promising in terms of its controlling strategy. V2H technology has gone through several optimization procedures [12]. Conventional optimization methods, including model-based linear and dynamic programming, have been used in different applications for integrating V2H into HEMSs, such as scheduling EV charging and discharging [13], economic load reduction [14], and load shifting [15]. However, these optimization methods cannot adapt to the environment's stochastic nature, including unpredictable driving patterns, load profiles, and pricing. The global search technique, that uses genetic algorithms [16] and swarm intelligence [17] to resolve power management issues, is another model-based optimization strategy. However, the application of these methods in real-time systems is limited due to extensive computations, which makes them slow. Thus, data-driven approaches, including machine learning (ML), are being introduced to offer more advanced services [18; 19; 20]. Furthermore, reinforcement learning (RL) algorithms provide superior alternative energy decisions, control, and management solutions due to their outstanding ability to make decisions and resolve issueswithout prior knowledge of the environment. Additionally, before being activated online for any driving patterns, load profiles, or dynamic electricity pricing, RL models may be tested offline for a generation profile and general load [12; 21]. Moreover, one direction of the reward formulation used in [22] for deep RL to control battery deterioration is a non-parametric reward function. The evaluation indicated training effectiveness, optimality, and flexibility across different common driving tests. According to an independent statistical analysis by the US Energy Information Administration (EIA) in 2021, the transportation sector consumes 28% of the energy produced annually, most of which comes from fossil fuels. Moreover, residential energy consumption reaches 27% [23]. The analysis reveals an existing CO\({}_{2}\) concentration of 286 ppm, predicting an annual increase of 2 ppm. According to another study, if drastic measures are not taken, some regions could experience increased CO\({}_{2}\) emissions of up to 700 ppm by 2100 [10]. Therefore, it is crucial to radically change fossil fuel usage to decrease CO\({}_{2}\) emissions in the atmosphere. A major challenge to achieving this goal is decarbonizing transportation. Therefore, electrifying transport to overcome this challenge is the most promising option.