## 5 Discussion of Simulations and Results  
### Data and Assumptions  
In this Section, by using Matlab R2014a software, our proposed approach is implemented on IEEE 30-bus test system [2; 6] with six NRGenCOs and 20 DisCOs. There are two additional wind farms connected to buses 7 and 10 [2], and the output power of the wind farms connected to bus 7 and 10 lies within the range of [0, 20] MW and [0, 30] MW, respectively. Figure 2 shows the schematic structure of the test system. Parameters of NRGenCOs' and DisCOs' bid functions are shown in Tables 1 and 2 [2], respectively.  
In order to verify the superiority of our proposed method, the GDCAC-based method and the fuzzy Q-learning-based method [2] are implemented on this test system. There are three scenarios set in this paper for simulation and comparison.  
In Scenario 1, every NRGenCO and DisCO searches for its optimal bidding strategy by using the GDCAC algorithm. In Scenario 2, NRGenCO1 searches for its optimal bidding strategy by using the fuzzy Q-learning algorithm, and other NRGenCOs and DisCOs using the GDCAC algorithm. In Scenario 3, every NRGenCO and DisCO searches for its optimal bidding strategy by using the fuzzy Q-learning algorithm.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Bus** & **NRGenCO** & \(\mathbf{a_{i}}\) **(103 \$/MW2h)** & \(\mathbf{b_{i}}\) **(103 \$/MWh)** & \(\mathbf{P_{gl,min}}\) **(MW)** & \(\mathbf{P_{gl,max}}\) **(MW)** \\ \hline
1 & NRGenCO1 & 0.2 & 20 & 0 & 80 \\
2 & NRGenCO2 & 0.175 & 17.5 & 0 & 80 \\
13 & NRGenCO3 & 0.625 & 10 & 0 & 50 \\
22 & NRGenCO4 & 0.0834 & 32.5 & 0 & 55 \\
23 & NRGenCO5 & 0.25 & 30 & 0 & 30 \\
27 & NRGenCO6 & 0.25 & 30 & 0 & 30 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Parameters of NRGenCOs’ bid functions.  
Figure 2: Diagram of the test system. Note: For the sake of simplicity, here it is assumed that the maximum congestion constraint in all transmission lines is 25 MW.  
In our simulation, the output powers of the two wind farms together constitute the 2-dimensional state vector. Each fuzzy Q-learning-based agent (whose step-by-step learning procedure and method for fuzzy set definition can be found in [2]) defines three triangular fuzzy sets for the state variable 1, and 4 triangular fuzzy sets for state variable 2, as shown in Figures 3 and 4. Table 3 presents the state and action sets of every NRGenCO and DisCO while taking Scenarios 1, 2, and 3 into consideration. The related parameters of the GDCAC algorithm and fuzzy Q-learning algorithm [2], which use the \(\varepsilon-greedy\) method to balance exploration and exploitation, are also listed in Table 3.  
In the Gauss radial basis function, we set the central point parameter matrix corresponding to state variable 1 and 2, which is expressed as follows:  
\[Cp=\left[\begin{array}{cccc}(0,0)&(0,6)&\ldots&(0,30)\\ (4,0)&(4,6)&\ldots&(4,30)\\ \ldots&\ldots&\ldots&\ldots\\ (20,0)&(20,6)&\ldots&(20,30)\end{array}\right]. \tag{41}\]  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Bus** & **DisCO** & \(\mathbf{c_{j}\) (\(10^{3}\) 5 / MW\({}^{2}\)h)} & \(\mathbf{d_{j}\) (\(10^{3}\) 5 / MWh)} & \(\mathbf{P_{d_{j},min}\) (MW)} & \(\mathbf{P_{d_{j},max}\) (MW)} \\ \hline  
6\) & \(15.6\) \\ \hline \hline \end{tabular} Note: In the 4 th column, parameter labeled by \({}^{***}\) are slightly adjusted from [2] in order to ensure that all DisCOs do not lose in competition because of their obvious differencein revenue parameters from other DisCOs.  
\end{table}
Table 2: Parameters of DisCOs’ bid functions.  
Figure 3: Fuzzy sets for state variable 1 in fuzzy Q-learning-based EM model.