# Optimal Network Defense Strategy Selection Method: A Stochastic Differential Game Model

Authors: Yan Mi; Hengwei Zhang; Hao Hu; Jinglei Tan; Jindong Wang
Journal: Security and Communication Networks
 (2021)

---

## Abstract

In a real-world network confrontation process, attack and defense actions change rapidly and continuously. The network environment is complex and dynamically random. Therefore, attack and defense strategies are inevitably subject to random disturbances during their execution, and the transition of the network security state is affected accordingly. In this paper, we construct a network security state transition model by referring to the epidemic evolution process, use Gaussian noise to describe random effects during the strategy execution, and introduce a random disturbance intensity factor to describe the degree of random effects. On this basis, we establish an attack-defense stochastic differential game model, propose a saddle point equilibrium solution method, and provide an algorithm to select the optimal defense strategy. Our method achieves real-time defense decision-making in network attack-defense scenarios with random disturbances and has better real-time performance and practicality than current methods. Results of a simulation experiment show that our model and algorithm are effective and feasible.


---

## Body

## 1. Introduction

With the rapid development of network information technology, network attack means emerge endlessly, and network security has a profound impact on people’s daily life [1]. In 2019, more than 900 million network attacks were found in the world, 270 million URLs were identified as malicious URLs by web antivirus components, and 19.8% of computer users were attacked by malicious software at least once [2]. Network security is facing major threats and severe challenges. However, static defense technology represented by intrusion detection and antivirus software is difficult to cope with complex and changeable network attacks. How to analyze network attack behavior, deploy security defense system, and enhance system defense capability has become an urgent problem to be solved [3].In selecting a defense strategy, we need to consider not only the effectiveness of the strategy, but also the cost constraints and attack strategies. Selecting the optimal defense strategy under the constraints of much complex information is the key to network defense [4]. As a decision-making theory in a conflict environment [5], game theory can help players select optimal strategies under complex conditions. In addition, with the basic characteristics of objective opposition, a noncooperative relationship, and strategic interdependence, game theory closely matches the fundamental characteristics of network attack and defense [6]. Therefore, the research of network attack and defense confrontation using game theory has become a hot spot in recent years and has achieved rich results [7–10].In the process of attack and defense, the actions of the attacker and the defender are generally not simultaneous, so it is of better research value to use the dynamic game theory to build the network attack and defense model.As a typical dynamic game, signaling game has attracted the attention of researchers for its extraordinary ability to accurately describe the critical role of intelligence information in the strategy selection of both attack and defense sides [11]. For stealthy link flooding attacks, Aydeger et al. [12] used a signaling game to perform modeling and, by solving a Bayesian Nash equilibrium, developed appropriate defense strategies and improved their effectiveness. Chen et al. [13] constructed a multistage signaling game model, analyzed spear-phishing attacks on networks, considered defense effectiveness and strategy cost, and proposed an optimal strategy selection algorithm. Shen et al. [14] analyzed the intrusion detection mechanism of Internet of things (IoT) with multistage signaling game and calculated the best malware detection strategy to inhibit the spread of malware.Considering that there is only one state in multistage games, some researcher combined Markov decision process with game model to realize the state transition of the game. Considering the influence of network attack on the power grid system, Li et al. [15] analyzed the network security state assessment from the perspective of defenders and put forward the network security state assessment method with Markov game model as the core. Tan et al. [9] proposed an attack-defense game model for moving targets based on the attack surface and detection surface and built multilevel and multistate features in the process of attack and defense confrontation by using Markov robust game model. On this basis, an optimal defense strategy selection algorithm for moving targets was proposed. Sahabandu et al. [16] adopted Markov game to model Dynamic Information Flow Tracking (DIFT) and Advanced Persistent Threats (APTs) as a two-person zero-sum game model and proposed Q-learning algorithm to solve the problem of APTs attack detection. Li et al. [17] considered the optimal decision-making method of moving target defense and used Markov game method to calculate the optimal strategy in time and space, thus improving the value of the decision.Multistage game models can represent network attack and defense to some extent, but discrete attack-defense processes are far from applicable to actual attack-defense confrontations. As a theoretical method to study conflict confrontations and decision-making processes in continuous time [18], differential game can model and analyze the rapid and continuous change of the network attack-defense process. Huang and Zhu [19] used a differential game framework to analyze antivirus mechanisms on complex networks and proposed a distributed antivirus algorithm to minimize the overall network cost. M. Li and S. Li [20] established a differential game model in wireless sensor networks, studied network IP defense problems, and proposed an optimal defense strategy based on limited resources. Feng et al. [21] used differential game theory to model the attack-defense interaction process of an objective function attack (OFA), introduced threat factors to describe the threats in the system operating environment, and determined the optimal operating node parameters through numerical calculations. Yang et al. [22] studied the repair problem of APT attack and used differential game to allocate available repair resources to insecure hosts, so as to reduce the potential loss of the organization. Aiming at the attack decision-making problem in honeynet, Ren and Zhang [23] considered the inherent characteristics and functional interaction between ordinary nodes and honeypots, established a differential game model, and finally obtained the optimal dynamic strategies for honeynet system and attacker.Considering the randomness of network system states and attack-defense strategies, and their effects on the states of the game system, Orojloo and Azgomi [24] proposed to assess network security using a stochastic game model with the two phases of intrusion and interruption. Liu et al. [25] analyzed the impact of threat propagation on the security of a network system, evaluated system states through a stochastic game model, and provided enhanced defense strategies based on the Nash equilibrium. Kandoussi et al. [26] combined stochastic game theory with attack graphs to model attack-defense interactions, analyzed potential attack paths, and deployed corresponding security measures.Although differential game models can solve continuous real-time decision-making problems for attack-defense confrontation, they do not consider the influence of uncertain factors in confrontation processes. Attack-defense strategies are inevitably affected by fluctuations in equipment performance and changes in network environments during strategy execution. Therefore, the effectiveness of strategies fluctuates. Stochastic game models describe the influence of random factors in the game process through state jumps, but current models adopt staged game methods, which provide limited guidance for the continuous real-time selection of a network defense strategy.To deal with the inability of stochastic games at continuous real-time decision-making and the lack of random disturbances in differential games, we construct an attack-defense stochastic differential game model. By solving and analyzing the saddle point equilibrium, we design an algorithm to select an optimal defense strategy and verify the effectiveness of the model and method through simulation experiments. Our main contributions are as follows. (1) We propose a network attack-defense stochastic differential game model, consider the influence of random factors in the execution of attack-defense strategies, and change the intensity factor of random disturbance and characterize the difference in the random disturbance effects on the players. Our model is more suitable than network attack-defense models based on a classic differential game for real-world network attack and defense scenarios. (2) To describe the selection of attack-defense strategies and changes in payoffs, we construct an attack-defense decision-making control function and payoff integral function. Based on the Hamilton-Jacobi-Bellman (HJB) equation, we solve the saddle point equilibrium strategy, obtain the optimal dynamic strategy trajectory description equation, and design an algorithm for real-time selection of an optimal defense. Our model has better timeliness, relevance, and practicality than the existing methods. (3) We conduct simulation experiments. Based on the numerical analysis on the number of nodes in different security states and attack-defense optimal dynamic strategies, we provide decision-making recommendations for network defense scenarios with random disturbances (experimental code and data are available athttps://github.com/kaster-hn/Network-attack-defense-stochastic-differential-game-algorithm).


## 2. Attack-Defense Stochastic Differential Game Model

### 2.1. Analysis of Network Security State Transition

The security states of nodes in a network change continuously due to virus invasion, penetration, and the network’s own restoring function, which is similar to the spread and immune process of infectious diseases. Therefore, we can use the susceptible-infectious-recovered (SIR) dynamic model to study the transition of the network security state. We use the nodes in a network system to represent the individuals of the SIR model, expand that model, and construct a network security state transition model, namely, the normal-infected-restored-malfunctioned (NIRM) model, with four network node security states. Changes of node states are affected by the outcome of the confrontation between attack and defense strategies.Normal stateN: A network node functions normally. Due to its inherent vulnerability, it may be attacked.Infected stateI: A network node has been infected by the attack strategy but can still function normally; the attacker can also use this node to attack neighboring nodes.Restored stateR: A network node has been effectively protected by the defender. The node possesses immunity, and the attacker fails to infect it.Malfunctioned stateM: A network node cannot continue to provide services, because its quality of service deteriorates severely after attack.The network security state transition model has five state transition paths:N⟶I: If the attacker successfully invades or penetrates a node in the normal state N, then the node transitions from state N to state I, and the attacker can use it to attack neighboring nodes.N⟶R: A node in the normal state N discovers its own vulnerabilities, removes them, and forms an immune function that prevents an attack.I⟶N: The defender identifies and modifies an infected node and transitions it to a normal node; however, due to different network environments and node configurations, the defender cannot identify all vulnerabilities. The attacker can use different strategies to attack again.I⟶R: The defender identifies infected nodes, removes all vulnerabilities, and acquires immunity, so the attacker cannot attack again.I⟶M: The infected node is destroyed by the attacker, resulting in a serious deterioration of its quality of service and the loss of its service function. The node can no longer be used to attack neighboring nodes.Denote the total time of a network attack byT, and the total number of nodes by Q. Let the number of the nodes in states N, I, R, and M at time t be xNt, xIt, xRt, and xMt, respectively, ∀t∈t0,T, xNt,xIt,xRt,xMt≥0 and. xNt,xIt,xRt,xMt=Q.According to the above five state transition scenarios, the state transition of a node is carried out around statesN and I, because these two states can be restored and infected. The infection action is analyzed first. A node in state I can only infect nodes directly connected to it. Therefore, the more nodes connected to state I, the faster the growth rate of infected nodes. Assuming that the nodes in the network are distributed with density σ, the number of nodes connected to a node is σπr2, where r is the network distance between two nodes. When r = 1, two nodes are directly connected. Therefore, the number of nodes directly connected to an infected node in state I is σπ. At time t, the proportion of normal nodes is xNt/Q, and hence, the number of normal nodes adjacent to an infected node is σπxItxNt/Q. Assume that the infection rate is va, and this is related to the proportion of infected nodes in the network. Let va=αxIt/Q, where α is an infection rate coefficient. Then, in an attack-defense confrontation, if the defense strategy fails, the number of nodes transitioning from state N to state I is vaσπxItxNt/Q, and the number transitioning from state I to state M is vaxIt. The nodes are restored through the defense strategy, which, if successful, can have two results: entering the normal state and entering the restored state. The specific state depends on the network structure and the strategies adopted by the attacker and defender. Assume that the probability of entering the restored state is a constant μ, and the restoring rate vd is related to the ratio of normal to restored nodes in the network. Let vd=βxNt+xIt/Q, where β is a restoring rate coefficient. Then, in the attack-defense confrontation, if the defense strategy is successful, the numbers of nodes transitioning from state I to N, I to R, and N to R are 1−μvdxIt, μvdxIt, and μvdxNt, respectively.Defenders can be divided into those of high-, medium-, and low-intensity types, denoted byDH, DM, and DL, respectively, with corresponding defense strategy strengths of sDH¯,sDM¯,sDL¯∈0,1. The defender adopts a hybrid strategy, selecting its defense strategy at time t according to the probability vector PDt=pDHt,pDMt,pDLt. The expected utilities of high-, medium-, and low-intensity defense strategies are dht=pDHtsDH¯, dmt=pDMtsDM¯, and dlt=pDLtsDL¯, respectively. So, at time t, the overall expected utility of the defender is dt=dht+dmt+dlt. Attackers are similarly divided into those of high-, medium-, and low-intensity types, denoted by AH, AM, and AL, respectively, with corresponding attack strategy strengths denoted by sAH¯,sAM¯,sAL¯∈0,1. At time t, the attacker adopts a hybrid strategy, PAt=pAHt,pAMt,pALt. The expected utilities of high-, medium-, and low-intensity attack strategies are aht=pAHtsAH¯, amt=pAMtsAM¯, and alt=pALtsAL¯, respectively. So, the overall expected utility of the attacker at time t is at=aht+amt+alt.However, in a real-world network attack-defense confrontation, the network environment changes constantly and randomly. The defender always has certain information errors in its observation of the network environment. At the same time, in the execution of defense strategies, the basic network quality of service, performance of network nodes, and defense equipment are dynamic and may vary randomly within a certain range; i.e., they are not static. Therefore, an actual defense utility is a random variable. Suppose that, at timet, the actual defense utility of defender type i is dit=pDitsDi¯+δDdωDit,i∈H,M,L, where δD is the factor of random disturbance intensity, ωDit reflects one-dimensional Brownian motion and is distributed as N0,ρDsDi2t, dωDit represents random disturbance and is distributed as N0,ρDsDi2Δt, and ρD is the coefficient of variation. The random disturbances received by the defender mainly come from the fluctuation of the basic network environment and equipment performance. The intensity of random disturbance received by different types of defenders is basically the same and is denoted by δD. The high-intensity strategy is more complex, requires more resources in execution, is affected by other processes in the system, and has a higher execution cost. Therefore, its actual utility will be more instable, i.e., with greater variance. Assume that the variance and defense strength are positively correlated as ρDsDi2Δt. Then, the actual defense utility of the defender at time t is dactt=dHt+dMt+dLt.Similarly, the actual attack utility is affected by uncertain factors. In particular, due to the asymmetry of information in a network attack-defense confrontation, attack information tends to be more concealed and difficult to observe. The grasp of game information such as attack strategies and attack revenue calculations is incomplete and uncertain. The calculation of actual attack utility has higher uncertainty; i.e., the attacker receives greater random disturbance intensity. Suppose that, at timet, the actual attack utilities by various types of attackers are ait=pAitsAi¯+δAdωAit,i∈H,M,L, where δA is the random disturbance intensity factor, ωAit is one-dimensional Brownian motion distributed as N0,ρAsAi2t, dωAit is distributed as N0,ρAsAi2Δt, and ρA is the coefficient of variation. The attack strategy is subjected to the same intensity of random disturbance, δA, during execution. From the above analysis, δA>δD. The actual attack utility of the attacker at time t is aactt=aHt+aMt+aLt.The result of the attack-defense confrontation determines the state transition of the nodes, and the difference between the actual utilities of the attack-defense hybrid strategies,τt=aactt−dactt, is used to represent the confrontation result. Note that τt∈0,1. The attack is successful if τt≥0, and the defense is successful if τt<0. Using attack-defense results, τt, to analyze the state transition paths, we can obtain the parameters τNI, τNR, τIN, τIR, and τIM to describe the likelihoods of state transitions N⟶I, N⟶R, I⟶N, I⟶R, and I⟶M, respectively:(1)τNI=τIM=0,τt<0,τt,τt≥0;τNR=τIN=τIR=τt,τt<0,0,τt≥0.Based on the above analysis, the node state transition equations are obtained as(2)x˙N=−τNItvaσπxItxNtQ−μτNRtvdxNt+1−μτINtvdxIt,x˙I=τNItvaσπxItxNtQ−τIMtvaxIt−1−μτINtvdxIt−μτIRvdxIt,x˙R=μτNRtvdxNt+μτIRvdxIt,x˙M=τIMtvaxIt,∀t∈t0,T,xNt+xIt+xRt+xMt=Q.


### 2.2. Definition of Attack-Defense Stochastic Differential Game Model

Definition 1.
The network attack-defense stochastic differential game modelADSDG can be expressed as a nine-tuple, ADSDG=ℜ,ℑ,P,t,x,δ,P,θ,U.①
ℜ=ℜD,ℜA is a set of game players, where ℜD and ℜA represent defenders and attackers, respectively.②
ℑ=ℑD,ℑA is the type space of the defender and attacker. It is the private information of the players. ℜD=Ai|i=1,2,…,m and ℜA=Aj|j=1,2,…,n.③
℘=DG,AG is the action space of players. DG=ζ1,ζ2,…,ζg and AG=ξ1,ξ2,…,ξk are the action sets of the defender and attacker, respectively. g,k≥1.④
t is the moment of the attack-defense game, t∈t0,T.⑤
xt=xNt,xIt,xRt,xMt|xNt+xIt+xRt+xMt=Q is the state variable of nodes in the network. xNt,xIt,xRt,xMt represent the number of nodes in the normal, infected, restored, and malfunctioned states, respectively, at time t. Q is the total number of nodes.⑥
δ=δD,δA is the random disturbance intensity factor, which indicates how much the random disturbance affects the actual utility of the strategy. Random disturbances received by the defender mainly come from the influence of the basic network environment and equipment performance. The attacker is not only affected by the above random disturbances. The asymmetry of attack and defense information increases the difficulty of calculating the actual attack utility, which leads to a higher random disturbance intensity, i.e., δA>δD.⑦
P=PDt,PAt is the dynamic strategy of the attacker and defender at time t. PDt=pDit|1≤i≤m represents the hybrid strategy of the defender at time t, where pDit is the probability that the defender chooses defense strategy type i, and ∑i=1mpDit=1. Similarly, PAt=pAjt|1≤j≤n represents the hybrid strategy of the attacker at time t, pAjt is the probability of the attacker choosing attack strategy type i, and ∑j=1npAjt=1.⑧
θ=θN,θI,θR,θM is the state transition function, where θN=dxNt/dt=x˙N, θI=dxIt/dt=x˙I, θR=dxRt/dt=x˙R, θM=dxMt/dt=x˙M.⑨
L=LD,LA is the set of payoff functions of the defender and attacker. In the stochastic differential game model, a payoff function is L=∫t0Tgt,xt,PAt,PDtdt+qxt, where g:0,T×ℝ4×PA×PD↦ℝ represents the instantaneous payoffs of the players at time t, and q:xT↦ℝ represents the final payoff at the end of the game.


## 2.1. Analysis of Network Security State Transition

The security states of nodes in a network change continuously due to virus invasion, penetration, and the network’s own restoring function, which is similar to the spread and immune process of infectious diseases. Therefore, we can use the susceptible-infectious-recovered (SIR) dynamic model to study the transition of the network security state. We use the nodes in a network system to represent the individuals of the SIR model, expand that model, and construct a network security state transition model, namely, the normal-infected-restored-malfunctioned (NIRM) model, with four network node security states. Changes of node states are affected by the outcome of the confrontation between attack and defense strategies.Normal stateN: A network node functions normally. Due to its inherent vulnerability, it may be attacked.Infected stateI: A network node has been infected by the attack strategy but can still function normally; the attacker can also use this node to attack neighboring nodes.Restored stateR: A network node has been effectively protected by the defender. The node possesses immunity, and the attacker fails to infect it.Malfunctioned stateM: A network node cannot continue to provide services, because its quality of service deteriorates severely after attack.The network security state transition model has five state transition paths:N⟶I: If the attacker successfully invades or penetrates a node in the normal state N, then the node transitions from state N to state I, and the attacker can use it to attack neighboring nodes.N⟶R: A node in the normal state N discovers its own vulnerabilities, removes them, and forms an immune function that prevents an attack.I⟶N: The defender identifies and modifies an infected node and transitions it to a normal node; however, due to different network environments and node configurations, the defender cannot identify all vulnerabilities. The attacker can use different strategies to attack again.I⟶R: The defender identifies infected nodes, removes all vulnerabilities, and acquires immunity, so the attacker cannot attack again.I⟶M: The infected node is destroyed by the attacker, resulting in a serious deterioration of its quality of service and the loss of its service function. The node can no longer be used to attack neighboring nodes.Denote the total time of a network attack byT, and the total number of nodes by Q. Let the number of the nodes in states N, I, R, and M at time t be xNt, xIt, xRt, and xMt, respectively, ∀t∈t0,T, xNt,xIt,xRt,xMt≥0 and. xNt,xIt,xRt,xMt=Q.According to the above five state transition scenarios, the state transition of a node is carried out around statesN and I, because these two states can be restored and infected. The infection action is analyzed first. A node in state I can only infect nodes directly connected to it. Therefore, the more nodes connected to state I, the faster the growth rate of infected nodes. Assuming that the nodes in the network are distributed with density σ, the number of nodes connected to a node is σπr2, where r is the network distance between two nodes. When r = 1, two nodes are directly connected. Therefore, the number of nodes directly connected to an infected node in state I is σπ. At time t, the proportion of normal nodes is xNt/Q, and hence, the number of normal nodes adjacent to an infected node is σπxItxNt/Q. Assume that the infection rate is va, and this is related to the proportion of infected nodes in the network. Let va=αxIt/Q, where α is an infection rate coefficient. Then, in an attack-defense confrontation, if the defense strategy fails, the number of nodes transitioning from state N to state I is vaσπxItxNt/Q, and the number transitioning from state I to state M is vaxIt. The nodes are restored through the defense strategy, which, if successful, can have two results: entering the normal state and entering the restored state. The specific state depends on the network structure and the strategies adopted by the attacker and defender. Assume that the probability of entering the restored state is a constant μ, and the restoring rate vd is related to the ratio of normal to restored nodes in the network. Let vd=βxNt+xIt/Q, where β is a restoring rate coefficient. Then, in the attack-defense confrontation, if the defense strategy is successful, the numbers of nodes transitioning from state I to N, I to R, and N to R are 1−μvdxIt, μvdxIt, and μvdxNt, respectively.Defenders can be divided into those of high-, medium-, and low-intensity types, denoted byDH, DM, and DL, respectively, with corresponding defense strategy strengths of sDH¯,sDM¯,sDL¯∈0,1. The defender adopts a hybrid strategy, selecting its defense strategy at time t according to the probability vector PDt=pDHt,pDMt,pDLt. The expected utilities of high-, medium-, and low-intensity defense strategies are dht=pDHtsDH¯, dmt=pDMtsDM¯, and dlt=pDLtsDL¯, respectively. So, at time t, the overall expected utility of the defender is dt=dht+dmt+dlt. Attackers are similarly divided into those of high-, medium-, and low-intensity types, denoted by AH, AM, and AL, respectively, with corresponding attack strategy strengths denoted by sAH¯,sAM¯,sAL¯∈0,1. At time t, the attacker adopts a hybrid strategy, PAt=pAHt,pAMt,pALt. The expected utilities of high-, medium-, and low-intensity attack strategies are aht=pAHtsAH¯, amt=pAMtsAM¯, and alt=pALtsAL¯, respectively. So, the overall expected utility of the attacker at time t is at=aht+amt+alt.However, in a real-world network attack-defense confrontation, the network environment changes constantly and randomly. The defender always has certain information errors in its observation of the network environment. At the same time, in the execution of defense strategies, the basic network quality of service, performance of network nodes, and defense equipment are dynamic and may vary randomly within a certain range; i.e., they are not static. Therefore, an actual defense utility is a random variable. Suppose that, at timet, the actual defense utility of defender type i is dit=pDitsDi¯+δDdωDit,i∈H,M,L, where δD is the factor of random disturbance intensity, ωDit reflects one-dimensional Brownian motion and is distributed as N0,ρDsDi2t, dωDit represents random disturbance and is distributed as N0,ρDsDi2Δt, and ρD is the coefficient of variation. The random disturbances received by the defender mainly come from the fluctuation of the basic network environment and equipment performance. The intensity of random disturbance received by different types of defenders is basically the same and is denoted by δD. The high-intensity strategy is more complex, requires more resources in execution, is affected by other processes in the system, and has a higher execution cost. Therefore, its actual utility will be more instable, i.e., with greater variance. Assume that the variance and defense strength are positively correlated as ρDsDi2Δt. Then, the actual defense utility of the defender at time t is dactt=dHt+dMt+dLt.Similarly, the actual attack utility is affected by uncertain factors. In particular, due to the asymmetry of information in a network attack-defense confrontation, attack information tends to be more concealed and difficult to observe. The grasp of game information such as attack strategies and attack revenue calculations is incomplete and uncertain. The calculation of actual attack utility has higher uncertainty; i.e., the attacker receives greater random disturbance intensity. Suppose that, at timet, the actual attack utilities by various types of attackers are ait=pAitsAi¯+δAdωAit,i∈H,M,L, where δA is the random disturbance intensity factor, ωAit is one-dimensional Brownian motion distributed as N0,ρAsAi2t, dωAit is distributed as N0,ρAsAi2Δt, and ρA is the coefficient of variation. The attack strategy is subjected to the same intensity of random disturbance, δA, during execution. From the above analysis, δA>δD. The actual attack utility of the attacker at time t is aactt=aHt+aMt+aLt.The result of the attack-defense confrontation determines the state transition of the nodes, and the difference between the actual utilities of the attack-defense hybrid strategies,τt=aactt−dactt, is used to represent the confrontation result. Note that τt∈0,1. The attack is successful if τt≥0, and the defense is successful if τt<0. Using attack-defense results, τt, to analyze the state transition paths, we can obtain the parameters τNI, τNR, τIN, τIR, and τIM to describe the likelihoods of state transitions N⟶I, N⟶R, I⟶N, I⟶R, and I⟶M, respectively:(1)τNI=τIM=0,τt<0,τt,τt≥0;τNR=τIN=τIR=τt,τt<0,0,τt≥0.Based on the above analysis, the node state transition equations are obtained as(2)x˙N=−τNItvaσπxItxNtQ−μτNRtvdxNt+1−μτINtvdxIt,x˙I=τNItvaσπxItxNtQ−τIMtvaxIt−1−μτINtvdxIt−μτIRvdxIt,x˙R=μτNRtvdxNt+μτIRvdxIt,x˙M=τIMtvaxIt,∀t∈t0,T,xNt+xIt+xRt+xMt=Q.


## 2.2. Definition of Attack-Defense Stochastic Differential Game Model

Definition 1.
The network attack-defense stochastic differential game modelADSDG can be expressed as a nine-tuple, ADSDG=ℜ,ℑ,P,t,x,δ,P,θ,U.①
ℜ=ℜD,ℜA is a set of game players, where ℜD and ℜA represent defenders and attackers, respectively.②
ℑ=ℑD,ℑA is the type space of the defender and attacker. It is the private information of the players. ℜD=Ai|i=1,2,…,m and ℜA=Aj|j=1,2,…,n.③
℘=DG,AG is the action space of players. DG=ζ1,ζ2,…,ζg and AG=ξ1,ξ2,…,ξk are the action sets of the defender and attacker, respectively. g,k≥1.④
t is the moment of the attack-defense game, t∈t0,T.⑤
xt=xNt,xIt,xRt,xMt|xNt+xIt+xRt+xMt=Q is the state variable of nodes in the network. xNt,xIt,xRt,xMt represent the number of nodes in the normal, infected, restored, and malfunctioned states, respectively, at time t. Q is the total number of nodes.⑥
δ=δD,δA is the random disturbance intensity factor, which indicates how much the random disturbance affects the actual utility of the strategy. Random disturbances received by the defender mainly come from the influence of the basic network environment and equipment performance. The attacker is not only affected by the above random disturbances. The asymmetry of attack and defense information increases the difficulty of calculating the actual attack utility, which leads to a higher random disturbance intensity, i.e., δA>δD.⑦
P=PDt,PAt is the dynamic strategy of the attacker and defender at time t. PDt=pDit|1≤i≤m represents the hybrid strategy of the defender at time t, where pDit is the probability that the defender chooses defense strategy type i, and ∑i=1mpDit=1. Similarly, PAt=pAjt|1≤j≤n represents the hybrid strategy of the attacker at time t, pAjt is the probability of the attacker choosing attack strategy type i, and ∑j=1npAjt=1.⑧
θ=θN,θI,θR,θM is the state transition function, where θN=dxNt/dt=x˙N, θI=dxIt/dt=x˙I, θR=dxRt/dt=x˙R, θM=dxMt/dt=x˙M.⑨
L=LD,LA is the set of payoff functions of the defender and attacker. In the stochastic differential game model, a payoff function is L=∫t0Tgt,xt,PAt,PDtdt+qxt, where g:0,T×ℝ4×PA×PD↦ℝ represents the instantaneous payoffs of the players at time t, and q:xT↦ℝ represents the final payoff at the end of the game.


## 3. Equilibrium Solution and Selection of Optimal Defense Strategy

### 3.1. Equilibrium Solution of Attack-Defense Stochastic Differential Game

To facilitate the solution of the attack-defense stochastic differential game, the game’s equilibrium solution is divided into two parts: quantifying payoffs and solving the saddle point equilibrium strategy. Payoff quantification is the basis for solving the saddle point equilibrium strategy, whose method is related to the result of the equilibrium solution and affects the selection of the optimal defense strategy.


#### 3.1.1. Payoff Quantification

Based on the analysis in Section2.1 and the definition of the attack-defense stochastic differential game model in Section 2.2, the following method of payoff quantification is proposed. When a node experiences a state transition, the attacker and defender are affected accordingly. The reward coefficient of a node transitioning from state N to state I is denoted by r1, representing the damage to neighboring nodes after the node is infected. The reward coefficient of a node transitioning from state N or I to state R is denoted by r2, representing immunity of the node after being restored, which may reduce the expected loss. The reward coefficient of the node from state I to state M is denoted by r3, representing the loss of the node’s service function. The reward coefficient of the node from state I to state N is denoted by r4. The statistical average approach is used to define the reward coefficients r1,r2,r3,r4∈0,10. In addition, when a node transitions to state M, a new normal node needs to be established, with terminal cost cMTxMT, where cMT≥0.Therefore, the rewards for the defender and attacker, respectively, at timet are(3)rD=r2μτNRtvdxNt+μτIRvdxIt+r41−μτINtvdxIt−r3τIMtvaxIt−r1τNItvaσπxItxNtQ,rA=r1τNItvaσπxItxNtQ+r3τIMtvaxIt−r2μτNRtvdxNt+μτIRvdxIt−r41−μτINtvdxIt.The attacker and defender will incur costs associated with strategy execution. These are related to utilities. The higher a strategy’s utility, the greater the cost. Denote the cost coefficients of high-, medium-, and low-intensity defense strategies bycDH, cDM, and cDL, respectively, and those of high-, medium-, and low-intensity attack strategies by cAH, cAM, and cAL, respectively, where ciH>ciM>ciL,i∈D,A. By referring to [27], the strategy costs of defense and attack, respectively, at time t are established as(4)cD=12dH2cDH+dM2cDM+dL2cDLxNt+xRt,cA=12aH2cAH+aM2cAM+aL2cALxIt.Through the above analysis and comprehensive consideration of the costs and rewards of attack-defense strategies, the payoff functions of the attacker and defender can be obtained as(5)LDPAt,PDt=∫t0TgDt,xt,PAt,PDdt+qDxT=∫t0Tr2μτNRtvdxNt+μτIRvdxIt+r41−μτINtvdxIt−r3τIMtvaxIt−r1τNItvaσπxItxNtQ−1/2dH2cDH+dM2cDM+dL2cDLxNt+xRtdt−cMTxMT,LAPAt,PDt=∫t0TgAt,xt,PAt,PDtdt+qAxT=∫t0Tr1τNItvaσπxItxNtQ+r3τIMtvaxIt−r2μτNRtvdxNt+μτIRvdxIt−r41−μτINtvdxIt−1/2aH2cAH+aM2cAM+aL2cALxItdt,dH=pDHsDH¯+δDdWDH,dM=pDMsDM¯+δDdWDM,dL=pDLsDL¯+δDdWDL,aH=pAHsAH¯+δAdWAH,aM=pAMsAM¯+δAdWAM,aL=pALsAL¯+δAdWAL.


#### 3.1.2. Solution of Saddle Point Equilibrium Strategy

For a network attack-defense stochastic differential game modelADSDG, both the attacker and defender want to maximize their payoff by choosing optimal strategies. Let PDt∗,PAt∗ represent the optimal dynamic strategies of the defender and attacker.Definition 2.
If the strategy setPDt∗,PAt∗ constitutes a Nash equilibrium solution of the model ADSDG, then the following equations exist:(6)PDt∗=maxPDt∫t0TgDt,xt∗,PAt∗,PDtdt+qDxT∗,PAt∗=maxPAt∫t0TgAt,xt∗,PAt,PDt∗dt+qAxT∗,x˙t∗=θt,xt∗,PAt∗,PDt∗.
PA∗t,PD∗t is referred to as the saddle equilibrium strategy in the attack-defense stochastic differential game, or the saddle point strategy in short.Theorem 1.
For the attack-defense stochastic differential game modelADSDG, if there is a costate function λit:t0,T×ℝ4⟶ℝ,i∈D,A, then a saddle point PDt∗,PAt∗ strategy exists if the following conditions are met, where P−i∗t denotes the collection of controls of all players except player i.(7)ddtx∗t=θt,x∗t,PD∗t,PA∗t,x∗t0=xt0,Pi∗t=argmaxPitθt,x∗t,Pit,P−i∗tλit+git,x∗t,Pit,P−i∗t,ddtλit=−∂∂x∗θt,x∗t,Pi∗t,P−i∗tλit+git,x∗t,Pi∗t,P−i∗t,λiT=∂qix∗T∂x∗.Proof.
In Theorem1, when the attacker chooses the optimal strategy PA∗t, the defender forms an optimal control problem. According to Pontryagin’s maximum principle, the existence of the function λDt can be proved, and the existence of the function λAt can similarly be proved. Therefore, Theorem 1 is proved: there is a saddle point strategy in the attack-defense stochastic differential game ADSDG.
The solution process for the saddle point strategy is as follows. The Hamiltonian function of the defender is constructed as(8)HDt,λDt,xt,PAt,PDt=gDt,xt,PDt,PAt+∑i∈N,I,R,M∂λD∂xiθit,xt,PDt,PAt=τNRtr2−λDxN+λDxRμvdxNt+τIRtr2−λDxI+λDxRμvdxIt+τINtr4+λDxN−λDxI1−μvdxIt+τNIt−r1−λDxN+λDxIvaσπxItxNtQ+τIMt−r3−λDxI+λDxMvaxIt−12dH2cDH+dM2cDM+dL2cDLxNt+xRt.
Then, the costate functionλDt is calculated. λDt=λDxitT=λDxNt,λDxIt,λDxRt,λDxMtT.(9)ddtλDxNt=−∂∂xNtHDt,λt,xt,PAt,PDt=−τNRtμβr2−λDxN+λDxRxRt+2xNtQ−τIRtμβr2−λDxI+λDxRxItQ−τINt1−μβr4+λDxN−λDxIxItQ−τNItα−r1−λDxN+λDxIσπxI2tQ2+12dH2cDH+dM2cDM+dL2cDL,λDxNt=0,(10)ddtλDxIt=−∂∂xItHDt,λt,xt,PAt,PDt=−τIRtμβr2−λDxI+λDxRxNt+xRtQ−τINt1−μβr4+λDxN−λDxIxNt+xRtQ−2τNItα−r1−λDxN+λDxIσπxItxNtQ2−2τIMtα−r3−λDxI+λDxMxItQ,λDxIT=0,(11)ddtλDxRt=−∂∂xRtHDt,λt,x,PAt,PDt=−τNRtμβr2−xNt+λDxRxNtQ−τIRtμβr2−λDxI+λDxRxItQ−τINt1−μβr4+xNt−λDxIxItQ+1/2dH2cDH+dM2cDM+dL2cDL,λDxRT=0,(12)ddtλxMt=−∂∂xMtHDt,λt,xt,PAt,PDt=0,λxMT=−cMT.
Let∂HD∗/∂pDit=0. Then, pDit∗ is obtained as(13)pDit∗=r1+λDxN−λDxIσπvaxItxNt/Q+r3+λDxI−λDxMvaxIt−cDixNt+xRtδDidWDisDi¯cDixNt+xRt,τt≥0,r2−λDxN+λDxRμvdxNt+r2−λDxI+λDxRμvdxIt+r4−λDxI+λDxN1−μvdxIt−cDixNt+xRtδDidWDisDi¯cDixNt+xRt,τt<0.
λAt can be similarly obtained. Let ∂HA∗/∂pAit=0. Then, pAit∗ is obtained as(14)pAit∗=r1−λAxN+λAxIσπvaxItxNt/Q+r3−λAxI+λAxMvaxIt−cAixItδAidWAisAi¯cAixIt,τt≥0,r2+λAxN−λAxRμvdxNt+r2+λAxI−λAxRμvdxIt+r4+λAxI−λAxN1−μvdxIt−cAixItδDidWDisAi¯cAixIt,τt<0.
In equations (13) and (14), i∈H,M,L, and τt=aactt−dactt represents the result of the attack-defense confrontation. pDit∗ and pAit∗ have different results according to τt, indicating that the optimal strategies of the two sides are interdependent.Algorithm 1: Algorithm to select optimal defense strategy in attack-defense differential game.
Input: Attack-defense stochastic differential game modelADSDGOutput: Optimal defense strategypD∗tBEGINInitializeADSDG=ℜ,ℑ,℘,t,x,δ,P,θ,L/∗Initialize the stochastic differential game model∗/InitializepDHt=1 and pAHt=1/∗For the given initial attack-defense strategies, all adopt high-intensity strategies∗/Calculate,τNI, τNR, τIN, τIR, and τIM/∗Calculate the result of attack-defense confrontation according to the initial strategy∗/FORt = 0 to TSubstituteτNI, τNR, τIN, τIR, and τIM in equations (2) and (9)–(14)/∗Calculate the current statext=xNt,xIt,xRt,xMt and costate values of the node, λD and λA∗/Calculate the defender’s payoffgDt,xt,PAt,PDtCalculate the attacker’s payoffgAt,xt,PAt,PDtCalculate the optimal strategies of the defender and attacker,pDt∗ and pAt∗Calculate the result of the attack-defense confrontation,τNI, τNR, τIN, τIR, and τIMENDFORReturnpDt∗END


### 3.2. Algorithm to Select Optimal Defense Strategy

Algorithm to select optimal defense strategy in attack-defense differential game in Algorithm1.


### 3.3. Comparative Analysis of Results

From the aspects of game type, player type, real-time performance of decision-making, random disturbance, equilibrium solution, and specific application, we compare and analyze the results of this paper and related results, as shown in Table1.Table 1
Comparison of models and methods.
LiteratureGame typePlayer typeReal-time performance of decision-makingRandom disturbanceEquilibrium solutionApplication[12]Single-stage game2—NoSimpleStrategy selection[14]Multistage game2PoorNoDetailedIntrusion detection[19]Differential game2Real-timeNoDetailedStrategy selection[21]Differential game2Real-timeNoSimpleStrategy selection[24]Stochastic game2PoorYesSimpleSafety analysis[25]Stochastic game2PoorYesDetailedSituation assessmentThis paperStochastic differential gamenReal-timeYesDetailedStrategy selectionIn a network attack-defense confrontation, due to differences between the attacker and defender in terms of technology, ability, and tendency of strategy choice, it is more realistic to divide players into different types in the analysis. We divide both the attacker and defender into high, medium, and low types. The number of types of players can be extended ton, so the model can be extended to more application scenarios. However, the literatures lack a clear classification of player types and only elaborate on their action spaces. Real-time performance of decision-making refers to the effective time of one decision-making, and this has a great impact on the effectiveness of decision-making. In a staged game, the execution of a decision lasts for one stage. This way of decision-making has poor real-time performance and prevents timely responses to environment changes. By making decisions in continuous time, the stochastic differential game ensures that decision-making can adapt to changes in the network environment and has good real-time performance. Random disturbance considers the influence of uncertain factors during the attack-defense process. In an actual attack-defense confrontation, the network environment and structure are dynamically changing, and the performance of equipment varies within a certain range, which will disturb the strategy execution. In addition, the attacker and defender hide their information, making the game information incomplete for both parties. These uncertainties are unavoidable. The proposed stochastic differential game model introduces random disturbances in the strategy execution process, making the game model reflect the actual situation more realistically. Equilibrium solution refers to whether the process of solving game equilibrium is detailed. The equilibrium solution of a stochastic differential game changes in real time, and the solution process is complex. This paper elaborates upon the detailed solving process, which can improve the practical application capability of the model.


## 3.1. Equilibrium Solution of Attack-Defense Stochastic Differential Game

To facilitate the solution of the attack-defense stochastic differential game, the game’s equilibrium solution is divided into two parts: quantifying payoffs and solving the saddle point equilibrium strategy. Payoff quantification is the basis for solving the saddle point equilibrium strategy, whose method is related to the result of the equilibrium solution and affects the selection of the optimal defense strategy.


### 3.1.1. Payoff Quantification

Based on the analysis in Section2.1 and the definition of the attack-defense stochastic differential game model in Section 2.2, the following method of payoff quantification is proposed. When a node experiences a state transition, the attacker and defender are affected accordingly. The reward coefficient of a node transitioning from state N to state I is denoted by r1, representing the damage to neighboring nodes after the node is infected. The reward coefficient of a node transitioning from state N or I to state R is denoted by r2, representing immunity of the node after being restored, which may reduce the expected loss. The reward coefficient of the node from state I to state M is denoted by r3, representing the loss of the node’s service function. The reward coefficient of the node from state I to state N is denoted by r4. The statistical average approach is used to define the reward coefficients r1,r2,r3,r4∈0,10. In addition, when a node transitions to state M, a new normal node needs to be established, with terminal cost cMTxMT, where cMT≥0.Therefore, the rewards for the defender and attacker, respectively, at timet are(3)rD=r2μτNRtvdxNt+μτIRvdxIt+r41−μτINtvdxIt−r3τIMtvaxIt−r1τNItvaσπxItxNtQ,rA=r1τNItvaσπxItxNtQ+r3τIMtvaxIt−r2μτNRtvdxNt+μτIRvdxIt−r41−μτINtvdxIt.The attacker and defender will incur costs associated with strategy execution. These are related to utilities. The higher a strategy’s utility, the greater the cost. Denote the cost coefficients of high-, medium-, and low-intensity defense strategies bycDH, cDM, and cDL, respectively, and those of high-, medium-, and low-intensity attack strategies by cAH, cAM, and cAL, respectively, where ciH>ciM>ciL,i∈D,A. By referring to [27], the strategy costs of defense and attack, respectively, at time t are established as(4)cD=12dH2cDH+dM2cDM+dL2cDLxNt+xRt,cA=12aH2cAH+aM2cAM+aL2cALxIt.Through the above analysis and comprehensive consideration of the costs and rewards of attack-defense strategies, the payoff functions of the attacker and defender can be obtained as(5)LDPAt,PDt=∫t0TgDt,xt,PAt,PDdt+qDxT=∫t0Tr2μτNRtvdxNt+μτIRvdxIt+r41−μτINtvdxIt−r3τIMtvaxIt−r1τNItvaσπxItxNtQ−1/2dH2cDH+dM2cDM+dL2cDLxNt+xRtdt−cMTxMT,LAPAt,PDt=∫t0TgAt,xt,PAt,PDtdt+qAxT=∫t0Tr1τNItvaσπxItxNtQ+r3τIMtvaxIt−r2μτNRtvdxNt+μτIRvdxIt−r41−μτINtvdxIt−1/2aH2cAH+aM2cAM+aL2cALxItdt,dH=pDHsDH¯+δDdWDH,dM=pDMsDM¯+δDdWDM,dL=pDLsDL¯+δDdWDL,aH=pAHsAH¯+δAdWAH,aM=pAMsAM¯+δAdWAM,aL=pALsAL¯+δAdWAL.


### 3.1.2. Solution of Saddle Point Equilibrium Strategy

For a network attack-defense stochastic differential game modelADSDG, both the attacker and defender want to maximize their payoff by choosing optimal strategies. Let PDt∗,PAt∗ represent the optimal dynamic strategies of the defender and attacker.Definition 2.
If the strategy setPDt∗,PAt∗ constitutes a Nash equilibrium solution of the model ADSDG, then the following equations exist:(6)PDt∗=maxPDt∫t0TgDt,xt∗,PAt∗,PDtdt+qDxT∗,PAt∗=maxPAt∫t0TgAt,xt∗,PAt,PDt∗dt+qAxT∗,x˙t∗=θt,xt∗,PAt∗,PDt∗.
PA∗t,PD∗t is referred to as the saddle equilibrium strategy in the attack-defense stochastic differential game, or the saddle point strategy in short.Theorem 1.
For the attack-defense stochastic differential game modelADSDG, if there is a costate function λit:t0,T×ℝ4⟶ℝ,i∈D,A, then a saddle point PDt∗,PAt∗ strategy exists if the following conditions are met, where P−i∗t denotes the collection of controls of all players except player i.(7)ddtx∗t=θt,x∗t,PD∗t,PA∗t,x∗t0=xt0,Pi∗t=argmaxPitθt,x∗t,Pit,P−i∗tλit+git,x∗t,Pit,P−i∗t,ddtλit=−∂∂x∗θt,x∗t,Pi∗t,P−i∗tλit+git,x∗t,Pi∗t,P−i∗t,λiT=∂qix∗T∂x∗.Proof.
In Theorem1, when the attacker chooses the optimal strategy PA∗t, the defender forms an optimal control problem. According to Pontryagin’s maximum principle, the existence of the function λDt can be proved, and the existence of the function λAt can similarly be proved. Therefore, Theorem 1 is proved: there is a saddle point strategy in the attack-defense stochastic differential game ADSDG.
The solution process for the saddle point strategy is as follows. The Hamiltonian function of the defender is constructed as(8)HDt,λDt,xt,PAt,PDt=gDt,xt,PDt,PAt+∑i∈N,I,R,M∂λD∂xiθit,xt,PDt,PAt=τNRtr2−λDxN+λDxRμvdxNt+τIRtr2−λDxI+λDxRμvdxIt+τINtr4+λDxN−λDxI1−μvdxIt+τNIt−r1−λDxN+λDxIvaσπxItxNtQ+τIMt−r3−λDxI+λDxMvaxIt−12dH2cDH+dM2cDM+dL2cDLxNt+xRt.
Then, the costate functionλDt is calculated. λDt=λDxitT=λDxNt,λDxIt,λDxRt,λDxMtT.(9)ddtλDxNt=−∂∂xNtHDt,λt,xt,PAt,PDt=−τNRtμβr2−λDxN+λDxRxRt+2xNtQ−τIRtμβr2−λDxI+λDxRxItQ−τINt1−μβr4+λDxN−λDxIxItQ−τNItα−r1−λDxN+λDxIσπxI2tQ2+12dH2cDH+dM2cDM+dL2cDL,λDxNt=0,(10)ddtλDxIt=−∂∂xItHDt,λt,xt,PAt,PDt=−τIRtμβr2−λDxI+λDxRxNt+xRtQ−τINt1−μβr4+λDxN−λDxIxNt+xRtQ−2τNItα−r1−λDxN+λDxIσπxItxNtQ2−2τIMtα−r3−λDxI+λDxMxItQ,λDxIT=0,(11)ddtλDxRt=−∂∂xRtHDt,λt,x,PAt,PDt=−τNRtμβr2−xNt+λDxRxNtQ−τIRtμβr2−λDxI+λDxRxItQ−τINt1−μβr4+xNt−λDxIxItQ+1/2dH2cDH+dM2cDM+dL2cDL,λDxRT=0,(12)ddtλxMt=−∂∂xMtHDt,λt,xt,PAt,PDt=0,λxMT=−cMT.
Let∂HD∗/∂pDit=0. Then, pDit∗ is obtained as(13)pDit∗=r1+λDxN−λDxIσπvaxItxNt/Q+r3+λDxI−λDxMvaxIt−cDixNt+xRtδDidWDisDi¯cDixNt+xRt,τt≥0,r2−λDxN+λDxRμvdxNt+r2−λDxI+λDxRμvdxIt+r4−λDxI+λDxN1−μvdxIt−cDixNt+xRtδDidWDisDi¯cDixNt+xRt,τt<0.
λAt can be similarly obtained. Let ∂HA∗/∂pAit=0. Then, pAit∗ is obtained as(14)pAit∗=r1−λAxN+λAxIσπvaxItxNt/Q+r3−λAxI+λAxMvaxIt−cAixItδAidWAisAi¯cAixIt,τt≥0,r2+λAxN−λAxRμvdxNt+r2+λAxI−λAxRμvdxIt+r4+λAxI−λAxN1−μvdxIt−cAixItδDidWDisAi¯cAixIt,τt<0.
In equations (13) and (14), i∈H,M,L, and τt=aactt−dactt represents the result of the attack-defense confrontation. pDit∗ and pAit∗ have different results according to τt, indicating that the optimal strategies of the two sides are interdependent.Algorithm 1: Algorithm to select optimal defense strategy in attack-defense differential game.
Input: Attack-defense stochastic differential game modelADSDGOutput: Optimal defense strategypD∗tBEGINInitializeADSDG=ℜ,ℑ,℘,t,x,δ,P,θ,L/∗Initialize the stochastic differential game model∗/InitializepDHt=1 and pAHt=1/∗For the given initial attack-defense strategies, all adopt high-intensity strategies∗/Calculate,τNI, τNR, τIN, τIR, and τIM/∗Calculate the result of attack-defense confrontation according to the initial strategy∗/FORt = 0 to TSubstituteτNI, τNR, τIN, τIR, and τIM in equations (2) and (9)–(14)/∗Calculate the current statext=xNt,xIt,xRt,xMt and costate values of the node, λD and λA∗/Calculate the defender’s payoffgDt,xt,PAt,PDtCalculate the attacker’s payoffgAt,xt,PAt,PDtCalculate the optimal strategies of the defender and attacker,pDt∗ and pAt∗Calculate the result of the attack-defense confrontation,τNI, τNR, τIN, τIR, and τIMENDFORReturnpDt∗END


## 3.1.1. Payoff Quantification

Based on the analysis in Section2.1 and the definition of the attack-defense stochastic differential game model in Section 2.2, the following method of payoff quantification is proposed. When a node experiences a state transition, the attacker and defender are affected accordingly. The reward coefficient of a node transitioning from state N to state I is denoted by r1, representing the damage to neighboring nodes after the node is infected. The reward coefficient of a node transitioning from state N or I to state R is denoted by r2, representing immunity of the node after being restored, which may reduce the expected loss. The reward coefficient of the node from state I to state M is denoted by r3, representing the loss of the node’s service function. The reward coefficient of the node from state I to state N is denoted by r4. The statistical average approach is used to define the reward coefficients r1,r2,r3,r4∈0,10. In addition, when a node transitions to state M, a new normal node needs to be established, with terminal cost cMTxMT, where cMT≥0.Therefore, the rewards for the defender and attacker, respectively, at timet are(3)rD=r2μτNRtvdxNt+μτIRvdxIt+r41−μτINtvdxIt−r3τIMtvaxIt−r1τNItvaσπxItxNtQ,rA=r1τNItvaσπxItxNtQ+r3τIMtvaxIt−r2μτNRtvdxNt+μτIRvdxIt−r41−μτINtvdxIt.The attacker and defender will incur costs associated with strategy execution. These are related to utilities. The higher a strategy’s utility, the greater the cost. Denote the cost coefficients of high-, medium-, and low-intensity defense strategies bycDH, cDM, and cDL, respectively, and those of high-, medium-, and low-intensity attack strategies by cAH, cAM, and cAL, respectively, where ciH>ciM>ciL,i∈D,A. By referring to [27], the strategy costs of defense and attack, respectively, at time t are established as(4)cD=12dH2cDH+dM2cDM+dL2cDLxNt+xRt,cA=12aH2cAH+aM2cAM+aL2cALxIt.Through the above analysis and comprehensive consideration of the costs and rewards of attack-defense strategies, the payoff functions of the attacker and defender can be obtained as(5)LDPAt,PDt=∫t0TgDt,xt,PAt,PDdt+qDxT=∫t0Tr2μτNRtvdxNt+μτIRvdxIt+r41−μτINtvdxIt−r3τIMtvaxIt−r1τNItvaσπxItxNtQ−1/2dH2cDH+dM2cDM+dL2cDLxNt+xRtdt−cMTxMT,LAPAt,PDt=∫t0TgAt,xt,PAt,PDtdt+qAxT=∫t0Tr1τNItvaσπxItxNtQ+r3τIMtvaxIt−r2μτNRtvdxNt+μτIRvdxIt−r41−μτINtvdxIt−1/2aH2cAH+aM2cAM+aL2cALxItdt,dH=pDHsDH¯+δDdWDH,dM=pDMsDM¯+δDdWDM,dL=pDLsDL¯+δDdWDL,aH=pAHsAH¯+δAdWAH,aM=pAMsAM¯+δAdWAM,aL=pALsAL¯+δAdWAL.


## 3.1.2. Solution of Saddle Point Equilibrium Strategy

For a network attack-defense stochastic differential game modelADSDG, both the attacker and defender want to maximize their payoff by choosing optimal strategies. Let PDt∗,PAt∗ represent the optimal dynamic strategies of the defender and attacker.Definition 2.
If the strategy setPDt∗,PAt∗ constitutes a Nash equilibrium solution of the model ADSDG, then the following equations exist:(6)PDt∗=maxPDt∫t0TgDt,xt∗,PAt∗,PDtdt+qDxT∗,PAt∗=maxPAt∫t0TgAt,xt∗,PAt,PDt∗dt+qAxT∗,x˙t∗=θt,xt∗,PAt∗,PDt∗.
PA∗t,PD∗t is referred to as the saddle equilibrium strategy in the attack-defense stochastic differential game, or the saddle point strategy in short.Theorem 1.
For the attack-defense stochastic differential game modelADSDG, if there is a costate function λit:t0,T×ℝ4⟶ℝ,i∈D,A, then a saddle point PDt∗,PAt∗ strategy exists if the following conditions are met, where P−i∗t denotes the collection of controls of all players except player i.(7)ddtx∗t=θt,x∗t,PD∗t,PA∗t,x∗t0=xt0,Pi∗t=argmaxPitθt,x∗t,Pit,P−i∗tλit+git,x∗t,Pit,P−i∗t,ddtλit=−∂∂x∗θt,x∗t,Pi∗t,P−i∗tλit+git,x∗t,Pi∗t,P−i∗t,λiT=∂qix∗T∂x∗.Proof.
In Theorem1, when the attacker chooses the optimal strategy PA∗t, the defender forms an optimal control problem. According to Pontryagin’s maximum principle, the existence of the function λDt can be proved, and the existence of the function λAt can similarly be proved. Therefore, Theorem 1 is proved: there is a saddle point strategy in the attack-defense stochastic differential game ADSDG.
The solution process for the saddle point strategy is as follows. The Hamiltonian function of the defender is constructed as(8)HDt,λDt,xt,PAt,PDt=gDt,xt,PDt,PAt+∑i∈N,I,R,M∂λD∂xiθit,xt,PDt,PAt=τNRtr2−λDxN+λDxRμvdxNt+τIRtr2−λDxI+λDxRμvdxIt+τINtr4+λDxN−λDxI1−μvdxIt+τNIt−r1−λDxN+λDxIvaσπxItxNtQ+τIMt−r3−λDxI+λDxMvaxIt−12dH2cDH+dM2cDM+dL2cDLxNt+xRt.
Then, the costate functionλDt is calculated. λDt=λDxitT=λDxNt,λDxIt,λDxRt,λDxMtT.(9)ddtλDxNt=−∂∂xNtHDt,λt,xt,PAt,PDt=−τNRtμβr2−λDxN+λDxRxRt+2xNtQ−τIRtμβr2−λDxI+λDxRxItQ−τINt1−μβr4+λDxN−λDxIxItQ−τNItα−r1−λDxN+λDxIσπxI2tQ2+12dH2cDH+dM2cDM+dL2cDL,λDxNt=0,(10)ddtλDxIt=−∂∂xItHDt,λt,xt,PAt,PDt=−τIRtμβr2−λDxI+λDxRxNt+xRtQ−τINt1−μβr4+λDxN−λDxIxNt+xRtQ−2τNItα−r1−λDxN+λDxIσπxItxNtQ2−2τIMtα−r3−λDxI+λDxMxItQ,λDxIT=0,(11)ddtλDxRt=−∂∂xRtHDt,λt,x,PAt,PDt=−τNRtμβr2−xNt+λDxRxNtQ−τIRtμβr2−λDxI+λDxRxItQ−τINt1−μβr4+xNt−λDxIxItQ+1/2dH2cDH+dM2cDM+dL2cDL,λDxRT=0,(12)ddtλxMt=−∂∂xMtHDt,λt,xt,PAt,PDt=0,λxMT=−cMT.
Let∂HD∗/∂pDit=0. Then, pDit∗ is obtained as(13)pDit∗=r1+λDxN−λDxIσπvaxItxNt/Q+r3+λDxI−λDxMvaxIt−cDixNt+xRtδDidWDisDi¯cDixNt+xRt,τt≥0,r2−λDxN+λDxRμvdxNt+r2−λDxI+λDxRμvdxIt+r4−λDxI+λDxN1−μvdxIt−cDixNt+xRtδDidWDisDi¯cDixNt+xRt,τt<0.
λAt can be similarly obtained. Let ∂HA∗/∂pAit=0. Then, pAit∗ is obtained as(14)pAit∗=r1−λAxN+λAxIσπvaxItxNt/Q+r3−λAxI+λAxMvaxIt−cAixItδAidWAisAi¯cAixIt,τt≥0,r2+λAxN−λAxRμvdxNt+r2+λAxI−λAxRμvdxIt+r4+λAxI−λAxN1−μvdxIt−cAixItδDidWDisAi¯cAixIt,τt<0.
In equations (13) and (14), i∈H,M,L, and τt=aactt−dactt represents the result of the attack-defense confrontation. pDit∗ and pAit∗ have different results according to τt, indicating that the optimal strategies of the two sides are interdependent.Algorithm 1: Algorithm to select optimal defense strategy in attack-defense differential game.
Input: Attack-defense stochastic differential game modelADSDGOutput: Optimal defense strategypD∗tBEGINInitializeADSDG=ℜ,ℑ,℘,t,x,δ,P,θ,L/∗Initialize the stochastic differential game model∗/InitializepDHt=1 and pAHt=1/∗For the given initial attack-defense strategies, all adopt high-intensity strategies∗/Calculate,τNI, τNR, τIN, τIR, and τIM/∗Calculate the result of attack-defense confrontation according to the initial strategy∗/FORt = 0 to TSubstituteτNI, τNR, τIN, τIR, and τIM in equations (2) and (9)–(14)/∗Calculate the current statext=xNt,xIt,xRt,xMt and costate values of the node, λD and λA∗/Calculate the defender’s payoffgDt,xt,PAt,PDtCalculate the attacker’s payoffgAt,xt,PAt,PDtCalculate the optimal strategies of the defender and attacker,pDt∗ and pAt∗Calculate the result of the attack-defense confrontation,τNI, τNR, τIN, τIR, and τIMENDFORReturnpDt∗END


## 3.2. Algorithm to Select Optimal Defense Strategy

Algorithm to select optimal defense strategy in attack-defense differential game in Algorithm1.


## 3.3. Comparative Analysis of Results

From the aspects of game type, player type, real-time performance of decision-making, random disturbance, equilibrium solution, and specific application, we compare and analyze the results of this paper and related results, as shown in Table1.Table 1
Comparison of models and methods.
LiteratureGame typePlayer typeReal-time performance of decision-makingRandom disturbanceEquilibrium solutionApplication[12]Single-stage game2—NoSimpleStrategy selection[14]Multistage game2PoorNoDetailedIntrusion detection[19]Differential game2Real-timeNoDetailedStrategy selection[21]Differential game2Real-timeNoSimpleStrategy selection[24]Stochastic game2PoorYesSimpleSafety analysis[25]Stochastic game2PoorYesDetailedSituation assessmentThis paperStochastic differential gamenReal-timeYesDetailedStrategy selectionIn a network attack-defense confrontation, due to differences between the attacker and defender in terms of technology, ability, and tendency of strategy choice, it is more realistic to divide players into different types in the analysis. We divide both the attacker and defender into high, medium, and low types. The number of types of players can be extended ton, so the model can be extended to more application scenarios. However, the literatures lack a clear classification of player types and only elaborate on their action spaces. Real-time performance of decision-making refers to the effective time of one decision-making, and this has a great impact on the effectiveness of decision-making. In a staged game, the execution of a decision lasts for one stage. This way of decision-making has poor real-time performance and prevents timely responses to environment changes. By making decisions in continuous time, the stochastic differential game ensures that decision-making can adapt to changes in the network environment and has good real-time performance. Random disturbance considers the influence of uncertain factors during the attack-defense process. In an actual attack-defense confrontation, the network environment and structure are dynamically changing, and the performance of equipment varies within a certain range, which will disturb the strategy execution. In addition, the attacker and defender hide their information, making the game information incomplete for both parties. These uncertainties are unavoidable. The proposed stochastic differential game model introduces random disturbances in the strategy execution process, making the game model reflect the actual situation more realistically. Equilibrium solution refers to whether the process of solving game equilibrium is detailed. The equilibrium solution of a stochastic differential game changes in real time, and the solution process is complex. This paper elaborates upon the detailed solving process, which can improve the practical application capability of the model.


## 4. Simulation Experiment and Analysis

### 4.1. Description of Experimental Environment

The simulation experiment verifies the effectiveness of the proposed network attack-defense stochastic differential game model and optimal defense strategy selection method. The Scalable Simulation Framework (SSFNet) [28–30] simulation tool is used, and parameters are set to simulate network attack and defense scenarios. The number of nodes in the simulation experiment is set to Q = 1000. Referring to [31] and MIT’s attack-defense classification [32], the attack-defense action information is given in Tables 2 and 3 .Table 2
Description of attack actions.
No.Attack actionAttack strengthAttack typeAverage strength1Install trojan0.9AH0.822Attack SSH on web server0.823Install listener program0.744Send abnormal data to buffer0.5AM0.455LPC to LSASS process0.456SQL injection0.47FTP host attack0.35AL0.38Apache chunk overflow0.39Oracle TNS listener0.25Table 3
Description of defense actions.
No.Defense actionDefense strengthDefense typeAverage strength1Increase password complexity0.8DH0.762Uninstall delete trojan0.83Change account and password0.74Restart FTP server0.6DM0.55Renew root data0.56Correct homepage0.57Code randomization0.48Identify code injection0.3DL0.2679Add physical resource0.310Delete suspicious account0.2


### 4.2. Experimental Analysis

MATLAB 2018a is used to conduct numerical experiments. The transitions of the numbers of nodes with different security states in the network are recorded. The optimal dynamic strategies for the attacker and defender under different random disturbance intensities are calculated. On this basis, the influence of random disturbance and dynamic strategy on the transitions of node states is analyzed.The relevant parameters in the experiment are initialized as follows. The node density isσ=10. Based on the data in Tables 2 and 3, the attacker’s high-, medium-, and low-strategy strengths are sAH¯=0.82,sAM¯=0.45,sAL¯=0.3, respectively, and the defender’s high-, medium-, and low-strategy strengths are sDH¯=0.76,sDM¯=0.5,sDL¯=0.267, respectively. The strategy cost factors of the attacker are cAH=4.4,cAM=2.4,cAL=1.5. The strategy cost factors of the defender are cDH=4.1,cDM=2.7,cDL=1.3. The reward coefficients for node state transition are r1=4,r2=7,r3=10,r4=4. The infection rate coefficient is α=0.7. The restoring rate coefficient is β=0.7. The restored state transition probability is μ=0.8. The coefficients of variation are ρA=ρD=2. The total number of nodes is Q=1000. The number of initial infected nodes is xI0=10. For initial dynamic strategies, pDHt=1 and pAHt=1, indicating that the attacker and defender adopt the highest-intensity strategies at the beginning of the game.The attack-defense confrontation is simulated forδD=δA=0; δD=0.01, δA=0.03; and δD=0.06, δA=0.08. The experimental results are shown in Figures 1–4 .Figure 1
The numbers of nodes in different security states. (a)δD=δA=0. (b)δD=0.01, δA=0.03. (c)δD=0.06, δA=0.08.
(a)(b)(c)Figure 2
The trajectories of attack and defense strategies whenδD=δA=0.Figure 3
The trajectories of attack and defense strategies whenδD=0.01, δA=0.08.Figure 4
The trajectories of attack and defense strategies whenδD=0.06, δA=0.08.


#### 4.2.1. Analysis of Number of Transitioned Nodes in Different Security Sates

Figure1 shows plots of the trends of the number of nodes in all states with time under different random intensity disturbances. The horizontal axis is time t. The N, I, R, and M curves on the vertical axis indicate the number of nodes in the normal, infected, restored, and malfunctioned states, respectively.(1)
Some common points in the above three figures are observed. In the early stages of network attack-defense confrontation, normal nodes can quickly transition to the infected state. At aboutt=4.7s, the number of infected nodes reaches the maximum, because, in the initial stage of attack-defense confrontation, there are many normal nodes, with high strategy execution costs, while the number of infected nodes is small, with relatively low-strategy execution costs. So, the attacker will choose a high-intensity attack strategy, while the defender is subject to cost constraints and cannot respond to the attack strategy in time, resulting in the infection of a large number of normal nodes. Similarly, when the number of infected nodes is large, the defender can transition some of the infected nodes to the normal and restored states. As the game progresses, the number of nodes will eventually approach 500 in both the restored and malfunctioned states, and the number in the infected and malfunctioned states will eventually approach zero.(2)
It can be seen from the above figures that, for random disturbance intensities ofδD=δA=0, the curves of changes of the numbers of nodes tend to level off after t=66s; for δD=0.01 and δA=0.03, these curves tend to level off after t=33s; and for δD=0.06 and δA=0.08, they tend to level off after t=23s. Therefore, when the intensity of random disturbance increases, the time required for the number of nodes to stabilize decreases. The reason is as follows. After a normal node is infected, the attacker will quickly turn it into a malfunctioned state, so that the number of infected nodes can be kept at a relatively low level, and the cost of executing the attack is lower. However, due to the large number of nodes on the defender side, defense strategy execution cost is relatively high. The attacker eventually has stronger antidisturbance ability. Therefore, when the intensity of random disturbance increases, the node infection rate increases, and it takes less time for the game to reach a steady state.(3)
The following can be found from the above analysis. The influence of random factors can speed up the network attack-defense process, but the final numbers of various nodes are not changed much by random disturbances, indicating that the model has a certain antidisturbance ability. Random disturbances have a great impact on the game process. The defender should reduce these as much as possible to ensure that network changes do not deviate from its own expectations. The attacker and defender are constrained by costs. When one side has a large number of nodes, a strategy will often incur more costs, providing opportunity to the other side. Therefore, the defender should take measures to reduce defense costs, in order to preserve more normal nodes in the confrontation.


#### 4.2.2. Trajectory Analyses of Optimal Dynamic Strategies of the Attacker and Defender

Figures2–4 show the trajectory of the dynamic strategies of the attacker and defender over time under random disturbances of different intensities. The horizontal axis represents time t. The vertical axes pDHt and pDMt, respectively, represent the probabilities that the defender chooses high- and medium-intensity defense strategies. pAHt and pAMt, respectively, represent the probabilities of the attacker choosing high- and medium-intensity attack strategies. τt=aactt−dactt represents the result of the attack-defense confrontation with τt≥0 representing a successful attack, and τt<0 represents a successful defense.(1)
According to Figure2, the game can be divided into stages of defense, stalemate, and penetration. The defense stage occurs at t∈0,7, when the attacker and defender tend to adopt high-intensity strategies, and the game process is more intense. The attacker tries to use a high-intensity strategy in the early stage of the game to cause as much loss as possible to the defender and transitions some normal nodes into the infected state. The defender uses a high-intensity strategy to reduce losses and turns some nodes into the restored state. After a period of time, due to the high execution costs of high-intensity strategies, the attacker and defender reduce the probabilities of using them. The attacker and defender enter a stalemate stage at t∈7,63, both adopt medium-intensity strategies, and both can achieve a balance between attack and defense. At t∈22,26, the attacker attempts a high-intensity attack. Although the defender adopts certain countermeasures, it is successfully attacked, and some normal nodes are infected. After t=63s, the game enters the penetration stage, in which the attacker and defender tend to adopt low-intensity strategies, because it is difficult for either to reverse the current situation, and low-intensity strategies can reduce costs. However, the attacker has the advantage of a higher overall cost-effectiveness ratio, and infected nodes will gradually turn into the malfunctioned state.(2)
It can be seen from Figure3 that, in the case of relatively weak random disturbances, the trajectories of the dynamic strategies of the attacker and defender are similar to those without random disturbance. The defense stage exists at t∈0,7, the stalemate stage at t∈7,26, and the penetration stage at t>26s. The main intensities of strategies in the defense, stalemate, and penetration stages are high, medium, and low, respectively. However, under the influence of random factors, the elapsed time of stalemate is significantly reduced, from 56 s to 19 s. In this stage, the two sides have achieved certain results. The attacker turns the infected nodes into the malfunctioned state, and the defender turns the normal nodes into the restored state.(3)
According to Figure4, when the random disturbances are strong, the strategy trajectories of the attacker and defender change significantly, but the three stages of the game still exist. The stalemate stage exists at t∈7,17. It can be found that the increase of random disturbances has the greatest impact on the stalemate stage of the game process, whose time noticeably shortens. In the stalemate stage, the defender always tends to adopt a conservative strategy, i.e., turning the normal nodes into the restored state to ensure their safety, rather than restoring the infected nodes to reduce losses. During the game process, the attacker turns the infected nodes into the malfunctioned state. As a result, in the penetration stage, the number of restored nodes is always larger, while the number of infected nodes is smaller, so that the attacker can rely on the cost advantage to gradually turn the infected nodes to the malfunctioned state.(4)
Figures2–4 and the above analysis indicate that, in the stalemate stage, the defender mainly adopts a medium-intensity strategy. However, the stalemate stage has a great impact in the entire game process and lasts a long time. In this stage, the attacker may adopt a surprise attack strategy. Therefore, the defender must try to increase the cost-effectiveness ratio of the medium-intensity strategy to ensure its advantage in the stalemate stage and try to achieve the maximum defense effectiveness at the minimum cost. The defender should also increase the upper limit of the strategy strength as much as possible, reduce the gap between it and the attacker, and reduce the loss when defense is impossible.


## 4.1. Description of Experimental Environment

The simulation experiment verifies the effectiveness of the proposed network attack-defense stochastic differential game model and optimal defense strategy selection method. The Scalable Simulation Framework (SSFNet) [28–30] simulation tool is used, and parameters are set to simulate network attack and defense scenarios. The number of nodes in the simulation experiment is set to Q = 1000. Referring to [31] and MIT’s attack-defense classification [32], the attack-defense action information is given in Tables 2 and 3 .Table 2
Description of attack actions.
No.Attack actionAttack strengthAttack typeAverage strength1Install trojan0.9AH0.822Attack SSH on web server0.823Install listener program0.744Send abnormal data to buffer0.5AM0.455LPC to LSASS process0.456SQL injection0.47FTP host attack0.35AL0.38Apache chunk overflow0.39Oracle TNS listener0.25Table 3
Description of defense actions.
No.Defense actionDefense strengthDefense typeAverage strength1Increase password complexity0.8DH0.762Uninstall delete trojan0.83Change account and password0.74Restart FTP server0.6DM0.55Renew root data0.56Correct homepage0.57Code randomization0.48Identify code injection0.3DL0.2679Add physical resource0.310Delete suspicious account0.2


## 4.2. Experimental Analysis

MATLAB 2018a is used to conduct numerical experiments. The transitions of the numbers of nodes with different security states in the network are recorded. The optimal dynamic strategies for the attacker and defender under different random disturbance intensities are calculated. On this basis, the influence of random disturbance and dynamic strategy on the transitions of node states is analyzed.The relevant parameters in the experiment are initialized as follows. The node density isσ=10. Based on the data in Tables 2 and 3, the attacker’s high-, medium-, and low-strategy strengths are sAH¯=0.82,sAM¯=0.45,sAL¯=0.3, respectively, and the defender’s high-, medium-, and low-strategy strengths are sDH¯=0.76,sDM¯=0.5,sDL¯=0.267, respectively. The strategy cost factors of the attacker are cAH=4.4,cAM=2.4,cAL=1.5. The strategy cost factors of the defender are cDH=4.1,cDM=2.7,cDL=1.3. The reward coefficients for node state transition are r1=4,r2=7,r3=10,r4=4. The infection rate coefficient is α=0.7. The restoring rate coefficient is β=0.7. The restored state transition probability is μ=0.8. The coefficients of variation are ρA=ρD=2. The total number of nodes is Q=1000. The number of initial infected nodes is xI0=10. For initial dynamic strategies, pDHt=1 and pAHt=1, indicating that the attacker and defender adopt the highest-intensity strategies at the beginning of the game.The attack-defense confrontation is simulated forδD=δA=0; δD=0.01, δA=0.03; and δD=0.06, δA=0.08. The experimental results are shown in Figures 1–4 .Figure 1
The numbers of nodes in different security states. (a)δD=δA=0. (b)δD=0.01, δA=0.03. (c)δD=0.06, δA=0.08.
(a)(b)(c)Figure 2
The trajectories of attack and defense strategies whenδD=δA=0.Figure 3
The trajectories of attack and defense strategies whenδD=0.01, δA=0.08.Figure 4
The trajectories of attack and defense strategies whenδD=0.06, δA=0.08.


### 4.2.1. Analysis of Number of Transitioned Nodes in Different Security Sates

Figure1 shows plots of the trends of the number of nodes in all states with time under different random intensity disturbances. The horizontal axis is time t. The N, I, R, and M curves on the vertical axis indicate the number of nodes in the normal, infected, restored, and malfunctioned states, respectively.(1)
Some common points in the above three figures are observed. In the early stages of network attack-defense confrontation, normal nodes can quickly transition to the infected state. At aboutt=4.7s, the number of infected nodes reaches the maximum, because, in the initial stage of attack-defense confrontation, there are many normal nodes, with high strategy execution costs, while the number of infected nodes is small, with relatively low-strategy execution costs. So, the attacker will choose a high-intensity attack strategy, while the defender is subject to cost constraints and cannot respond to the attack strategy in time, resulting in the infection of a large number of normal nodes. Similarly, when the number of infected nodes is large, the defender can transition some of the infected nodes to the normal and restored states. As the game progresses, the number of nodes will eventually approach 500 in both the restored and malfunctioned states, and the number in the infected and malfunctioned states will eventually approach zero.(2)
It can be seen from the above figures that, for random disturbance intensities ofδD=δA=0, the curves of changes of the numbers of nodes tend to level off after t=66s; for δD=0.01 and δA=0.03, these curves tend to level off after t=33s; and for δD=0.06 and δA=0.08, they tend to level off after t=23s. Therefore, when the intensity of random disturbance increases, the time required for the number of nodes to stabilize decreases. The reason is as follows. After a normal node is infected, the attacker will quickly turn it into a malfunctioned state, so that the number of infected nodes can be kept at a relatively low level, and the cost of executing the attack is lower. However, due to the large number of nodes on the defender side, defense strategy execution cost is relatively high. The attacker eventually has stronger antidisturbance ability. Therefore, when the intensity of random disturbance increases, the node infection rate increases, and it takes less time for the game to reach a steady state.(3)
The following can be found from the above analysis. The influence of random factors can speed up the network attack-defense process, but the final numbers of various nodes are not changed much by random disturbances, indicating that the model has a certain antidisturbance ability. Random disturbances have a great impact on the game process. The defender should reduce these as much as possible to ensure that network changes do not deviate from its own expectations. The attacker and defender are constrained by costs. When one side has a large number of nodes, a strategy will often incur more costs, providing opportunity to the other side. Therefore, the defender should take measures to reduce defense costs, in order to preserve more normal nodes in the confrontation.


### 4.2.2. Trajectory Analyses of Optimal Dynamic Strategies of the Attacker and Defender

Figures2–4 show the trajectory of the dynamic strategies of the attacker and defender over time under random disturbances of different intensities. The horizontal axis represents time t. The vertical axes pDHt and pDMt, respectively, represent the probabilities that the defender chooses high- and medium-intensity defense strategies. pAHt and pAMt, respectively, represent the probabilities of the attacker choosing high- and medium-intensity attack strategies. τt=aactt−dactt represents the result of the attack-defense confrontation with τt≥0 representing a successful attack, and τt<0 represents a successful defense.(1)
According to Figure2, the game can be divided into stages of defense, stalemate, and penetration. The defense stage occurs at t∈0,7, when the attacker and defender tend to adopt high-intensity strategies, and the game process is more intense. The attacker tries to use a high-intensity strategy in the early stage of the game to cause as much loss as possible to the defender and transitions some normal nodes into the infected state. The defender uses a high-intensity strategy to reduce losses and turns some nodes into the restored state. After a period of time, due to the high execution costs of high-intensity strategies, the attacker and defender reduce the probabilities of using them. The attacker and defender enter a stalemate stage at t∈7,63, both adopt medium-intensity strategies, and both can achieve a balance between attack and defense. At t∈22,26, the attacker attempts a high-intensity attack. Although the defender adopts certain countermeasures, it is successfully attacked, and some normal nodes are infected. After t=63s, the game enters the penetration stage, in which the attacker and defender tend to adopt low-intensity strategies, because it is difficult for either to reverse the current situation, and low-intensity strategies can reduce costs. However, the attacker has the advantage of a higher overall cost-effectiveness ratio, and infected nodes will gradually turn into the malfunctioned state.(2)
It can be seen from Figure3 that, in the case of relatively weak random disturbances, the trajectories of the dynamic strategies of the attacker and defender are similar to those without random disturbance. The defense stage exists at t∈0,7, the stalemate stage at t∈7,26, and the penetration stage at t>26s. The main intensities of strategies in the defense, stalemate, and penetration stages are high, medium, and low, respectively. However, under the influence of random factors, the elapsed time of stalemate is significantly reduced, from 56 s to 19 s. In this stage, the two sides have achieved certain results. The attacker turns the infected nodes into the malfunctioned state, and the defender turns the normal nodes into the restored state.(3)
According to Figure4, when the random disturbances are strong, the strategy trajectories of the attacker and defender change significantly, but the three stages of the game still exist. The stalemate stage exists at t∈7,17. It can be found that the increase of random disturbances has the greatest impact on the stalemate stage of the game process, whose time noticeably shortens. In the stalemate stage, the defender always tends to adopt a conservative strategy, i.e., turning the normal nodes into the restored state to ensure their safety, rather than restoring the infected nodes to reduce losses. During the game process, the attacker turns the infected nodes into the malfunctioned state. As a result, in the penetration stage, the number of restored nodes is always larger, while the number of infected nodes is smaller, so that the attacker can rely on the cost advantage to gradually turn the infected nodes to the malfunctioned state.(4)
Figures2–4 and the above analysis indicate that, in the stalemate stage, the defender mainly adopts a medium-intensity strategy. However, the stalemate stage has a great impact in the entire game process and lasts a long time. In this stage, the attacker may adopt a surprise attack strategy. Therefore, the defender must try to increase the cost-effectiveness ratio of the medium-intensity strategy to ensure its advantage in the stalemate stage and try to achieve the maximum defense effectiveness at the minimum cost. The defender should also increase the upper limit of the strategy strength as much as possible, reduce the gap between it and the attacker, and reduce the loss when defense is impossible.


## 4.2.1. Analysis of Number of Transitioned Nodes in Different Security Sates

Figure1 shows plots of the trends of the number of nodes in all states with time under different random intensity disturbances. The horizontal axis is time t. The N, I, R, and M curves on the vertical axis indicate the number of nodes in the normal, infected, restored, and malfunctioned states, respectively.(1)
Some common points in the above three figures are observed. In the early stages of network attack-defense confrontation, normal nodes can quickly transition to the infected state. At aboutt=4.7s, the number of infected nodes reaches the maximum, because, in the initial stage of attack-defense confrontation, there are many normal nodes, with high strategy execution costs, while the number of infected nodes is small, with relatively low-strategy execution costs. So, the attacker will choose a high-intensity attack strategy, while the defender is subject to cost constraints and cannot respond to the attack strategy in time, resulting in the infection of a large number of normal nodes. Similarly, when the number of infected nodes is large, the defender can transition some of the infected nodes to the normal and restored states. As the game progresses, the number of nodes will eventually approach 500 in both the restored and malfunctioned states, and the number in the infected and malfunctioned states will eventually approach zero.(2)
It can be seen from the above figures that, for random disturbance intensities ofδD=δA=0, the curves of changes of the numbers of nodes tend to level off after t=66s; for δD=0.01 and δA=0.03, these curves tend to level off after t=33s; and for δD=0.06 and δA=0.08, they tend to level off after t=23s. Therefore, when the intensity of random disturbance increases, the time required for the number of nodes to stabilize decreases. The reason is as follows. After a normal node is infected, the attacker will quickly turn it into a malfunctioned state, so that the number of infected nodes can be kept at a relatively low level, and the cost of executing the attack is lower. However, due to the large number of nodes on the defender side, defense strategy execution cost is relatively high. The attacker eventually has stronger antidisturbance ability. Therefore, when the intensity of random disturbance increases, the node infection rate increases, and it takes less time for the game to reach a steady state.(3)
The following can be found from the above analysis. The influence of random factors can speed up the network attack-defense process, but the final numbers of various nodes are not changed much by random disturbances, indicating that the model has a certain antidisturbance ability. Random disturbances have a great impact on the game process. The defender should reduce these as much as possible to ensure that network changes do not deviate from its own expectations. The attacker and defender are constrained by costs. When one side has a large number of nodes, a strategy will often incur more costs, providing opportunity to the other side. Therefore, the defender should take measures to reduce defense costs, in order to preserve more normal nodes in the confrontation.


## 4.2.2. Trajectory Analyses of Optimal Dynamic Strategies of the Attacker and Defender

Figures2–4 show the trajectory of the dynamic strategies of the attacker and defender over time under random disturbances of different intensities. The horizontal axis represents time t. The vertical axes pDHt and pDMt, respectively, represent the probabilities that the defender chooses high- and medium-intensity defense strategies. pAHt and pAMt, respectively, represent the probabilities of the attacker choosing high- and medium-intensity attack strategies. τt=aactt−dactt represents the result of the attack-defense confrontation with τt≥0 representing a successful attack, and τt<0 represents a successful defense.(1)
According to Figure2, the game can be divided into stages of defense, stalemate, and penetration. The defense stage occurs at t∈0,7, when the attacker and defender tend to adopt high-intensity strategies, and the game process is more intense. The attacker tries to use a high-intensity strategy in the early stage of the game to cause as much loss as possible to the defender and transitions some normal nodes into the infected state. The defender uses a high-intensity strategy to reduce losses and turns some nodes into the restored state. After a period of time, due to the high execution costs of high-intensity strategies, the attacker and defender reduce the probabilities of using them. The attacker and defender enter a stalemate stage at t∈7,63, both adopt medium-intensity strategies, and both can achieve a balance between attack and defense. At t∈22,26, the attacker attempts a high-intensity attack. Although the defender adopts certain countermeasures, it is successfully attacked, and some normal nodes are infected. After t=63s, the game enters the penetration stage, in which the attacker and defender tend to adopt low-intensity strategies, because it is difficult for either to reverse the current situation, and low-intensity strategies can reduce costs. However, the attacker has the advantage of a higher overall cost-effectiveness ratio, and infected nodes will gradually turn into the malfunctioned state.(2)
It can be seen from Figure3 that, in the case of relatively weak random disturbances, the trajectories of the dynamic strategies of the attacker and defender are similar to those without random disturbance. The defense stage exists at t∈0,7, the stalemate stage at t∈7,26, and the penetration stage at t>26s. The main intensities of strategies in the defense, stalemate, and penetration stages are high, medium, and low, respectively. However, under the influence of random factors, the elapsed time of stalemate is significantly reduced, from 56 s to 19 s. In this stage, the two sides have achieved certain results. The attacker turns the infected nodes into the malfunctioned state, and the defender turns the normal nodes into the restored state.(3)
According to Figure4, when the random disturbances are strong, the strategy trajectories of the attacker and defender change significantly, but the three stages of the game still exist. The stalemate stage exists at t∈7,17. It can be found that the increase of random disturbances has the greatest impact on the stalemate stage of the game process, whose time noticeably shortens. In the stalemate stage, the defender always tends to adopt a conservative strategy, i.e., turning the normal nodes into the restored state to ensure their safety, rather than restoring the infected nodes to reduce losses. During the game process, the attacker turns the infected nodes into the malfunctioned state. As a result, in the penetration stage, the number of restored nodes is always larger, while the number of infected nodes is smaller, so that the attacker can rely on the cost advantage to gradually turn the infected nodes to the malfunctioned state.(4)
Figures2–4 and the above analysis indicate that, in the stalemate stage, the defender mainly adopts a medium-intensity strategy. However, the stalemate stage has a great impact in the entire game process and lasts a long time. In this stage, the attacker may adopt a surprise attack strategy. Therefore, the defender must try to increase the cost-effectiveness ratio of the medium-intensity strategy to ensure its advantage in the stalemate stage and try to achieve the maximum defense effectiveness at the minimum cost. The defender should also increase the upper limit of the strategy strength as much as possible, reduce the gap between it and the attacker, and reduce the loss when defense is impossible.


## 5. Conclusion

The actual network attack and defense confrontations often occur continuously and in real time and are affected by random disturbance factors. Dynamic network attack and defense analysis models based on game theory mainly include staged and differential game models. The staged game model cannot meet the requirements of network real-time decision-making, and the differential game model does not consider uncertain factors in the continuous real-time attack-defense process. Therefore, it is difficult to meet the requirements of real-time defense decision-making in network attack and defense scenarios with random disturbances. This study considers random disturbances in network attack-defense confrontations, describes the transition process of the network security states by differential equations, constructs a network attack-defense stochastic differential game model, and proposes a method to select the optimal defense strategy based on real-time analysis of network attack-defense confrontation under the influence of random disturbances. The simulation experiment verifies the effectiveness and rationality of the proposed model and method, which can provide a reference for the selection of active defense strategy under random disturbances.



---

*Source: 5594697-2021-08-24.xml*