## 3. Game Model  
### Game Definition  
The game is defined as seven element-tuples:  
\[Game=(G,\ S,\ A,\ D,\ P,R,\gamma) \tag{15}\]  
where \(G=(nodes,\ edges)\) is the network, \(S=\{s_{1},\ s_{2},\ \ldots,\ s_{S}\}\) is the set of states, \(A=\{a_{1},a_{2},\ldots,a_{A}\ \}\) and \(D=\{d_{1},d_{2},\ldots,d_{D}\}\) are the sets of attack actions and defense actions, respectively; \(P:S\times A\times D\times S\rightarrow[0,1]\) is the state transition probability function; \(R:S\times A\times D\to RV\) is the reward function, \(RV\) is reward space, and \(\gamma\in[0,1]\) is the discount factor.  
The game represents an ad hoc network with high-value and low-value devices. The nodes represent the devices on the network. The nodes are connected by edges. An edge can run one or more service links and the ability of the service to respond to requests (e.g., bandwidth available to the service) is denoted by the weight \(\beta\). The ability of a service processes a request (e.g., CPU time available to the service) is denoted by the weight \(v\). These two values are node-specific, since the network consists of high- and low-value nodes. Equations (16) and (17) formally describe the nodes and edges, respectively:  
\[nodes\subseteq\left\{\beta,v\ \Big{|}\ \beta\in\mathbb{N}^{|services|},v\in \mathbb{N}^{|services|}\right\} \tag{16}\]  
\[edges\subseteq\{(x,y)|x,y\in nodes,\ x\
eq y\} \tag{17}\]  
Since both players, the attacker and the defender, can only interact with the nodes, the state of all the nodes, at a given time, represents the state of the network. A node state \(n\) is defined in Equation (18):  
\[n\in\left\{b,\ v\ \Big{|}\ b\in\{0,1\}^{|services|},\ v\in\{0,1\}^{|services|} \right\}^{|nodes|} \tag{18}\]  
where \(b\) is the state of the available bandwidth (1 = service is bandwidth available, 0 = bandwidth is insufficient to handle request) and \(v\) is the state of available CPU resources (1 = cannot execute request, 0 = insufficient CPU resource to execute request).  
At the initial stage of the game the attacker has an available number of bot resources to attack a device on the network. The number of bot resource is defined as \(p\in\mathbb{N}\), which they can spend to flood the bandwidth or exhaust the resources of one or multiple targeted devices. In response, the defender can allocate more CPU resources to the device or increase the bandwidth to the compromised service.  
The game ends after a fixed number of rounds. In an actual defense scenario, the interactions will persist until either the attacker ceases his attack or the defender blocks all the attacking bots. However, this game focuses on what the defender can learn to mitigate the ongoing attack by intelligently distributing the resources available to him. Thus, the definition of game state is described below in Equation (19):  
\[<n,\ r>\ \in S \tag{19}\]  
where \(n\) is the state of all nodes, and \(r\) is the number of rounds left in the game.  
We have defined the rules of the game to represent a distributed denial-of-service attack. The goal of a denial-of-service attack is to prevent the legitimate use of a service. A distributed denial-of-service attack (DDOS) attains this goal by instructing a large number of entities (generally bots) to attack a service [36].  
An attack generally consists of flooding the victim with malicious traffic, thus leaving the service with insufficient bandwidth to handle real requests. Another type of attack consists of sending malformed or resource intensive requests that confuse an application on the target device and force it to consume a large amount of processing/CPU time to handle the request. The machine's inability to handle the request also has the result of making the service unavailable [37].