## 5 Discussion of Simulations and Results
### Implementing a GDCAC-Based Approach on the Test System  
In this section, the feasibility of the GDCAC-based EM approach is studied by using Scenario 1, as proposed in Section 5.1. In our simulation in Scenario 1, 20,000 iterations are set for every NRGenCO and DisCO to bid in EM. Among the 20,000 iterations, the first 15,000 iterations (training iterations) are used to train every NRGenCO and DisCO to perceive the ability of distinguishing optimal bidding strategies under the exploration and exploitation policy; the last 5000 iterations (decision-making iterations) are used to help every NRGenCO and DisCO with decision-making under the greedy policy. After 20,000 iterations of the bidding process, a Nash index is adopted in this paper to test whether the obtained bidding strategies of all NRGenCOs and DisCOs under any EM state reach Nash Equilibrium (NE) or not. Similar to the NE testing method in [2], the Nash index is equal to 1 when the NE under an EM state is reached and otherwise is equal to 0. Because the EM state space is continuous, we cannot display the process of reaching NE under every given EM state in this paper. Figure 5 demonstrates the Nash indices obtained after each iteration in a case in which the state variables 1 and 2 are equal to 20 and 30 MW, respectively. From Figure 5, it can be seen that occurrences of NE during the first 15,000 iterations are very sparse, and the EM can reach NE during the last 5000 iterations. The reasons for this phenomenon are: (1) every NRGenCO and DisCO has not yet accumulated enough experience to make the optimal bidding decision during the first 15,000 iterations; (2) the exploration and exploitation policy may make every NRGenCO and DisCO choose sub-optimal bids during the first 15,000 iterations; (3) 15,000 iterations of the training process are enough for every NRGenCO and DisCO to perceive the ability of distinguishing optimal bidding strategies and to make the optimal bidding decision under greedy policy during the last 5000 iterations. Figure 4: Fuzzy sets for state variable 2 in the fuzzy Q-learning-based EM model. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**Scenarios** & **Participants** & **EM State Set** & **Action Set** & **-** & **fi** & **ff** & **fi** & \(\mathbf{e_{1}}\) & \(\mathbf{e_{2}}\) & \(m\) \\  & & **(MW)** & & & & & & & & & \\ \hline \multirow{2}{*}{Scenario 1} & All NRGenCOs & \([0,20]\) and \([0,30]\) & \([1,3]\) & - & 0.5 & 0.1 & 0.1 & 4 & 6 & 1 \\  & All DisCOs & & \((0,1)\) & - & 0.5 & 0.1 & 0.1 & 4 & 6 & 1 \\ \hline \multirow{2}{*}{Scenario 2} & NRGen1 & \multirow{2}{*}{Other NRGenCOs} & \([U_{\text{gl}},U_{\text{glg}},\ldots,U_{\text{glggggg}}]\) & 0.1 & 0.5 & & & & & \\  & Other NRGenCOs & \([0,20]\) and \([0,30]\) & \([1,3]\) & - & 0.5 & 0.1 & 0.1 & 4 & 6 & 1 \\  & All DisCOs & & \((0,1)\) & - & 0.5 & 0.1 & 0.1 & 4 & 6 & 1 \\ \hline \multirow{2}{*}{Scenario 3} & All NRGenCOs & \([0,20]\) and \([0,30]\) & \(\{U_{\text{glg}},U_{\text{glggg}},\ldots,U_{\text{glggggg}}\}\) & 0.1 & 0.5 & - & - & - & - & - \\  & All DisCOs & \([0,20]\) and \([0,30]\) & \(\{U_{\text{glg}},U_{\text{glggg}},\ldots,U_{\text{glgggggg}}\}\) & 0.1 & 0.5 & - & - & - & - & - \\ \hline \multirow{2}{*}{Note: \(\text{U}_{\text{gl}}\) represents the median of interval \([1,1.02)\), \(\text{U}_{\text{glggg}}\) represents the median of interval \([1.02,1.04)\), \(\text{U}_{\text{glggg}}\) represents the median of interval \([2.98,3]\); \(\text{U}_{\text{dl}}\) represents the median of interval \((0,0.01)\), \(\text{U}_{\text{dl}2}\) represents the median of interval \([0.01,0.02)\), \(\text{U}_{\text{dl}100}\) represents the median of interval \([0.99,1]\). Other parameters of fuzzy Q-learning algorithm can be found in [2].  
\end{table} 
Table 3: Related information about the three scenarios.