# MADDPG-Based Security Situational Awareness for Smart Grid with Intelligent Edge
## 4 Problem Formulation  
In the framework of SSA with edge computing, we define the situational elements obtained by an edge agent in time slot \(t\) as \(O(t)=\{O_{1}(t),\ldots,O_{k}(t),\ldots,O_{K}(t)\}\), where \(O_{k}(t)\) denotes the situational elements obtained by edge agent \(k\) in time slot \(t\) and \(K\) is the number of edge agents. For edge agent \(k,k=1,2,\ldots,K\), we divide the situational elements in time slot \(t\) into three dimensions. The first dimension is the information of the external environment elements, here denoted as \(O_{k}^{ex}(t)\), \(O_{k}^{ex}(t)=\{O_{k}^{exPot}(t),O_{k}^{exLog}(t)\}\). Note that \(O_{k}^{exPot}(t)\) is the traffic information of ports. Typically, the port traffic information is continuous. Therefore, we set the number of ports here as \(P(t)\), then \(O_{k}^{exPot}(t)=\{O_{k1}^{exPot}(t),O_{k2}^{exPot}(t),\ldots,O_{kP(t)}^{exPot }(t)\}\). For each of the available ports,  
\[O_{kp(t)}^{exPot}(t)\in[0,1],\ p(t)=\{1,2,\ldots,P(t)\}. \tag{1}\]  
The power terminals have the function of recording operation logs, and similarly, where \(O_{k}^{exLog}(t)\) indicates log information such as overvoltage or overcurrent signals, alarm events, etc. In contrast to port traffic information, there are only two states of events, happening or not. Accordingly, let the total number of events extracted from the logs be \(E(t)\), then \(O_{k}^{exLog}(t)=\{O_{k1}^{exLog}(t),O_{k2}^{exLog}(t),\ldots,O_{kE(t)}^{exLog }(t)\}\). For each type of event,  
\[O_{k(t)}^{exLog}(t)\in\{0,1\},\ e(t)=\{1,2,\ldots,E(t)\}. \tag{2}\]  
The second dimension is the internal environment elements, including internal nodes vulnerability information, denoted as \(O_{k}^{in}(t)\), and we define the number of internal power terminal nodes as \(N(t)\). As vulnerability of a node characterizes a weak component in the grid, the vulnerability measurement protects against a chain of breakdowns. Consequently, we adopt continuous variables to signify the vulnerability of nodes. Thus, the internal environment elements of the edge agent \(k\) at time \(t\) is represented as \(O_{k}^{in}(t)=\{O_{k1}^{in}(t),O_{k2}^{in}(t),\ldots,O_{kN(t)}^{in}(t)\}\). For each specific node, the vulnerability is denoted as  
\[O_{kn(t)}^{in}(t)\in[0,1],\ n(t)=\{1,2,\ldots,N(t)\}. \tag{3}\]  
The third dimension is the possible attacks, denoted by \(O_{k}^{At}(t)\). Considering the attacks that the smart grid has encountered, we regard them as known attacks and form an attack library to determine the new attacks. Meanwhile, the known attacks in this library work as possible attacks that the grid will encounter. Therefore, we define the categories of possible attacks as \(A(t)\). Thus \(O_{k}^{At}(t)=\{O_{k1}^{At}(t),O_{k2}^{At}(t),\ldots,O_{kA(t)}^{At}(t)\}\). Considering that the possible attacks can be blocked in-process, we define continuous variables to represent information about the situation's elements where a particular possible attack is encountered, denoted as  
\[O_{ka(t)}^{At}(t)\in[0,1],\ a(t)=\{1,2,\ldots,A(t)\}. \tag{4}\]  
Situational elements under edge computing provide an essential basis for situational decisions. However, the defense tools are generally distinct for different threats. When a threat is detected, we should immediately activate the defense mechanism to suppress and offset the attack's impact on smart grids. However, these defense mechanisms come at a cost. \(\mathcal{C}_{kp(t)}^{exPot}(t)\), \(\mathcal{C}_{k(t)}^{exLog}(t)\), \(\mathcal{C}_{kn(t)}^{in}(t)\), and \(\mathcal{C}_{kn(t)}^{At}(t)\) represent the cost of processing anomalies traffic on port \(p(t)\), alarm event \(e(t)\) in the logs, restoring internally vulnerable nodes \(n(t)\), and defending against possible attack's category \(a(t)\) at time slot \(t\), respectively. Here, \(\mathcal{A}_{bk}^{p(t)}\), \(\mathcal{A}_{bk}^{e(t)}\), \(\mathcal{A}_{bk}^{n(t)}\), and \(\mathcal{A}_{bk}^{a(t)}\) denote the anomalous situational behaviors detected by the edge agent \(k\) at time slot \(t\), respectively.