## 5 Simulations
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Device**} & \multicolumn{6}{c}{**Parameters**} \\ \cline{2-7}  & \(a\) & \(b\) & \(c\) & \(d\) & \(e\) & \(f\) \\ \hline Gas boiler (GB) & 0.038 & 2.011 & 65 & - & - & - \\ Combined heat and power (CHP) & 0.0065 & 1.21 & 2 & 0.003 & 4 & 0.61 \\ \hline \end{tabular}
\end{table} 
Table 5: Parameters of operating cost. \begin{table}
\begin{tabular}{c c} \hline
**Learning Parameters** & **Value** \\ \hline Discounted factor & 0.7 \\ Learning rate & 0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Selection of Q learning parameters. The output of photovoltaic in ERC1 and ERC2 can be depicted in Figure 7 as follows:  
The energy routing design for both electrical power and thermal power can be obtained by using the proposed algorithm. The voltages of the ports connected with connection lines can be obtained which is illustrated in Figure 8.  
Figure 8: _Cont._  
Figure 6: Users’ demands in a single day. (**a**) Electrical power demands of ERC6 and ERC7; (**b**) heat demands of ERC5 and ERC7. Figure 7: Power output of photovoltaic in a single day. (**a**) PV output of ERC1 in a day; (**b**) PV output of ERC2 in a day. As shown in Figure 8, the voltage deviation on a transmission line reflects the amount of electrical power that can be transmitted through the line. More power can be transmitted under a larger voltage difference between two ports. According to Figure 8, the voltage of each ports can be obtained, therefore the power transmitting on each connection line can be calculated according to Equation (9) and the energy-management-based energy routing strategy is shown in Figure 9.  
Figure 8: Voltages of the ports connected with connection lines. (**a**) Voltage of both ends of connection line \(l_{1-3}\); (**b**) Voltage of both ends of connection line \(l_{2-3}\); (**c**) Voltage of both ends of connection line \(l_{2-4}\); (**d**) Voltage of both ends of connection line \(l_{3-5}\); (**e**) Voltage of both ends of connection line \(l_{5-7}\); (**f**) Voltage of both ends of connection line \(l_{5-6}\); (**g**) Voltage of both ends of connection line \(l_{4-6}\); (**h**) Voltage of both ends of connection line \(l_{6-7}\). As shown in Figure 9a, the electrical power outputs of different devices are illustrated and the electrical demands of ERC6 and ERC7 are plotted by dotted lines. During 0:00-8:00 and 16:00-24:00, the output power of PV arrays is zero, and CHP is used to produce electricity. The output power of PV increases over time; therefore, from 9:00-15:00, renewable energy is given priority in supplying electricity to meet users' demand due to its low operating and environmental costs. From 12:00-14:00, the total output power of two PV arrays is able to meet the whole electrical power demand, consequently, CHP is not involved in the process of supplying energy. While at the other times in a day, due to environmental factors, the output of PV fails to meet the total electrical power demand. Therefore, CHP is used to compensate the difference between supply and demand. As can be seen in Figure 9b, CHP and GB are used to supply thermal power to users of other energy regions. In most hours of a day, only CHP is used to produce heat. What is worthy of mentioning, in order to ensure the efficient operation of CHP, the ratio between its output electrical power and thermal power is limited to a certain range which is defined to be 100-300% in this paper. During 10:00-12:00, the increase of PV output leads to a reduction of electrical power from CHP. As a consequence, the thermal output of CHP is restricted, and GB is forced to generate more heat to satisfy users' demands. From 12:00-14:00, CHP is not involved in the energy supplying process, and GB is used to transmit heat to users. In Figure 9c,d, energy routing paths for different hours are shown, the negative power means that the direction of power flow is contradicted with the prescribed direction.