## 3 Definitions  
In our proposed EM, an agent, by using GDCAC algorithm, can adaptively adjust its bidding strategy (action) during repeated interactions with other participants until it obtains its maximum profit (under any EM environment state). In order to apply the GDCAC algorithm for bilateral spot EM modeling considering renewable power penetration, we use definitions similar to those in [2], organized as follows:  
1. Iteration: since the market is assumed to be cleared on an hour-ahead basis, we consider each hour as an iteration [2]. Moreover, just like in [2], time differences between hours such as demand preference, generation ramping constraints, number of participants, etc. are neglected. The purpose of doing this is to test whether the proposed modeling approach can automatically converge to the Nash equilibrium (NE) or not under the condition of no other external interference.
2. State variable: in iteration \(t\), the predicted power production of each RPGenCO can be defined as one state variable of the EM environment [2]. Due to the intermittent and random nature of the renewable power production, the \(v\)th state variable, representing the predicted power production of RPGenCO \(v\), is a random variable, and can be represent as: \[x_{v,t}=P_{rv,t}\quad v=1,2,\ldots,N_{g2},\] (12) where \(x_{v,t}\) randomly changes within a continuous interval of scalar values over time [2]: \[x_{v,t}\in[P_{rv,\min},P_{rv,\max}]\quad v=1,2,\ldots,N_{g2}.\] (13)Hence, in iteration \(t\), all state variables together constitute a state vector:  
\[x_{t}=(x_{1,t},x_{2,t},\ldots,x_{0,t},\ldots,x_{N_{g2},t})\in\mathbf{X}, \tag{14}\]  
where \(\mathbf{X}\subset\mathbf{R}^{N_{g2}}\) represents the continuous state space of the EM environment.  
(3) Action variable: the bidding strategy of every NRGenCO and DisCO is defined as one action variable of an agent. The \(i\)th action variable, representing the bidding strategy of NRGenCO \(i\), in iteration \(t\) is:  
\[u_{gi,t}=k_{gi,t}\quad i=1,2,\ldots,N_{g1}. \tag{15}\]  
The \(j\)th action variable, representing the bidding strategy of DisCO \(j\), in iteration \(t\) is:  
\[u_{dj,t}=k_{dj,t}\quad j=1,2,\ldots,N_{d}. \tag{16}\]  
All \(u_{gi,t}\)s and \(u_{dj,t}\)s can be adjusted, by the corresponding agent, within continuous intervals of scalar values over time, because an agent may not be able to achieve its maximum profit when selecting bidding strategies within a discrete action set.  
\[u_{gi,t}\in[k_{gi,\min},k_{gi,\max}]\quad i=1,2,\ldots,N_{g1} \tag{17}\]  
\[u_{dj,t}\in[k_{dj,\min},k_{dj,\max}]\quad j=1,2,\ldots,N_{d} \tag{18}\]  
(4) Reward: In iteration \(t\), NRGenCO \(i\)'s (\(i=1,2,\ldots,N_{g1}\)) reward is:  
\[r_{gi,t}=LMP_{gi,t}P_{gi,t}-(\frac{1}{2}a_{i}P_{gi,t}{}^{2}+b_{i}P_{gi,t}). \tag{19}\]  
DisCO \(j\)'s (\(j=1,2,\ldots,N_{d}\) ) reward in iteration \(t\) is:  
\[r_{dj,t}=(-\frac{1}{2}c_{j}P_{dj,t}{}^{2}+d_{i}P_{dj,t})-LMP_{dj,t}P_{dj,t}, \tag{20}\]  
where \(LMP_{gi,t}(LMP_{dj,t})\) is the LMP of the bus connecting NRGenCO\(i\) (DisCO\(j\)), and \(\frac{1}{2}a_{i}P_{gi,t}{}^{2}+b_{i}P_{gi,t}\) (\(-\frac{1}{2}c_{j}P_{dj,t}{}^{2}+d_{i}P_{dj,t}\)) is the cost (revenue) of NRGenCO \(i\) (DisCO \(j\)) when its dispatched power production (power demand) is \(P_{gi,t}\) (\(P_{dj,t}\)).  
Based on experiencing these received rewards over enough iterations, an agent in EM can gradually adjust its actions until it obtains the corresponding optimal action:  
\[u_{gi,t}^{(optimal)}\in[k_{gi,\min},k_{gi,\max}]\quad i=1,2,\ldots,N_{g1}\ \text{or}\ u_{dj,t}^{(optimal)}\in[k_{dj,\min},k_{dj,\max}]\quad j=1,2, \ldots,N_{d},\]  
which brings the most profit under any state (\(\mathbf{x}_{t}\in\mathbf{X}\)) of the EM environment. Hence, \(u_{gi,t}\), \(u_{dj,t}\) (\(i=1\), 2, \(\ldots,N_{g1}\); \(j=1,2,\ldots,N_{d}\)) and _LMPs_ are changing dynamically over iterations, which may or may not be constant under the same values of the state vector \(\mathbf{x}_{t}\) after enough iterations.