## 2 The Assessment of the Energy Management System for HRES
\begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt} p{142.3 pt}} \hline \hline
**Methods** & **Advantages** & **Disadvantages** \\ \hline \multirow{6}{*}{fuzzy logic control (FLC)} & -Following the rule basis and membership functions (MF), easy to understand. & -Trial-and-error method for determining the MFs, time-consuming and not optimal performance. \\  & -Insensitive to variation of the parameters. & -Greater number of variables makes it more complex to optimize the MFs. \\  & -Do not need a good model of the system and training process. & -Its “black box” nature and the network instruction problem lead to a lack of rules for determining the structure (cell and layers). \\ \multirow{4}{*}{ANN} & -Able to learn and to process parallel data. & -Its “black box” nature and the network instruction problem lead to a lack of rules for determining the structure (cell and layers). \\  & -Generalization skills and design do not depend on system parameters. & -Historical data provides a need for the learning and tuning process. \\  & -Fast response capacities compared to the conventional method. & -The number of data set used to train the ANN defines the optimality. \\ \hline \multirow{6}{*}{adaptive neuro-fuzzy inference system (ANFIS)} & -Has the inference ability of FLC and able to learn and process parallel data as ANN. & -More input variables lead to a more complex structure. \\  & -Applies neural learning rules to define and tune the MF of the fuzzy logic. & -More input variables lead to a more complex structure. \\ \hline \multirow{6}{*}{reinforcement learning (RL)} & -Conducts learning without prior knowledge. & -Long-time convergence for large real-world problem if not good initialization. \\  & -Can be combined with ANN for deep RL to solve the continuous state-space control problems. & -Long-time convergence for large real-world problem if not good initialization. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The advantages and disadvantages of some recently developed methods. calculate the value estimation and associated the pairs of state and action. Thus, the DRL method has been rapidly applied in robotics [27], building HVAC control [28], hybrid electric vehicles [29], etc. Some researchers have studied the use of RL and DRL energy management systems for hybrid electric vehicles and smart building [30; 31]. However, few publications study on the energy management of the HRES. [PERSON] (2013) proposed a two step ahead Q-learning method for defining the battery scheduling in a wind system, while [PERSON], [PERSON], and [PERSON] (2014) [32] developed a-three-step-ahead Q-learning for controlling the battery in a solar system. A novel online energy management technique using RL was developed in reference [33], which can learn and give the minimum power consumption without prior information on the workload. Additionally, a single agent system based on Q-learning has been developed by [PERSON] (2016) for energy management of a solar system [34]. Finally, a fuzzy reward function has been introduced based on the Q-learning algorithm by [PERSON] (2017) [35] to enhance the learning efficiency for controlling the power flow between components including PV, a battery, the local consumer, and a desalination unit for water supply. A multi-agent system (MAS) includes a set of agents which interact with each other and with their environment. Due to its feature of solving complex problems in a more computationally efficient manner compared to a single-agent system, many researchers have used it to solve energy management problems [36]. A MAS-based system was considered in a grid-connected microgrid for optimal operation [37]. Additionally, a MAS-based intelligent EMS for the islanded microgrid is designed in reference [38] to balance the energy among the generators, batteries and loads. an autonomous multi-agent system for optimally managing the buying and selling power has been proposed by [PERSON] (2012) [39].