# Study on Stochastic Differential Game Model in Network Attack and Defense

Authors: Xiaotong Xu; Gaocai Wang; Jintian Hu; Yuting Lu
Journal: Security and Communication Networks
 (2020)

---

## Abstract

In recent years, evolutionary game theory has been gradually applied to analyze and predict network attack and defense for maintaining cybersecurity. The traditional deterministic game model cannot accurately describe the process of actual network attack and defense due to changing in the set of attack-defense strategies and external factors (such as the operating environment of the system). In this paper, we construct a stochastic evolutionary game model by the stochastic differential equation with Markov property. The evolutionary equilibrium solution of the model is found and the stability of the model is proved according to the knowledge of the stochastic differential equation. And we apply the explicit Euler numerical method to analyze the evolution of the strategy selection of the players for different problem situations. The simulation results show that the stochastic evolutionary game model proposed in this paper can get a steady state and obtain the optimal defense strategy under the action of the stochastic disturbance factor. In addition, compared with other kinds of literature, we can conclude that the return on security investment of this model is better, and the strategy selection of the attackers and defenders in our model is more suitable for actual network attack and defense.


---

## Body

## 1. Introduction

With the development of the Internet, the security of the network and the privacy of users have been greatly disturbed. Therefore, the issues of cybersecurity have caused people’s high attention. The security of the Internet has become one of the important factors hindering the development of information technology. It is impossible to guarantee the security of cyberspace by relying on some passive defense measures in the increasing complexity of the network environment. Therefore, it is especially necessary to find new technologies that can detect the potential danger of network environment and take defense measures.In the network attack and defense, intruders can carry out an intrusion and the computer network can resist attack, which is similar to the process of the evolutionary game. Therefore, quite a lot of research studies have established a network attack and defense game model to select the optimal strategy [1–6]. The study of game theory first appeared in the field of economics research. In 1944, John von Neumann and Oskar Morgenstern proposed “game theory and economics,” which received wide attention [7]. Evolutionary game is a theory that combines game theory with the dynamic evolution process. It adopts the evolutionary theory of biology based on traditional game theory. The development of evolutionary game theory in various fields can be attributed to Smith (1973) and Price (1974) [8], who proposed the basic concept: Evolutionary Stable Strategy (ESS). Among them, the participants are the bounded rationality (between completely rational and incompletely rational). The players between groups constantly correct, imitate, and improve during the evolution process. They gradually tend to a certain stability strategy and eventually reach a state of equilibrium in the game. And players get the best strategy (to maximize their profits) in this state. In the field of cybersecurity, the traditional evolutionary game model does not consider the external environment and strategy mutation, which leads to the limitation of the evolution trend. The prejudgment of network attack and defense is also not accurate enough. Therefore, researchers tried to further improve the effectiveness of the model and more accurately describe the evolutionary game of attack and defense by using stochasticity [9–25]. In the stochastic game of this paper, the attackers will try to interfere or destroy the network environment. The defenders (network environment) can enhance the defensive ability by increasing defensive investments. Based on the principle of bounded rationality, the players gradually evolve into a stable state by learning and improving. The accuracy of the defenders’ choice of the optimal strategy has been effectively improved and the security of cyberspace has been guaranteed.The main contributions of this paper are as follows.(1)
The network attack and defense stochastic game model is constructed under incompletely rational conditions. We use stochastic differential equations to consider the randomness caused by external factors in the process of attack-defense. And we construct the stochastic replication dynamic equation to further accurately describe the evolution of the network attack and defense strategy.(2)
The attack lethality coefficient is used to describe the impact of different attack strategies on players. Furthermore, the model proposed in this paper is compared with other kinds of literature by the defenders’ payoffs, which further proves that the model proposed in this paper is more suitable for the actual situation of the network attack and defense.(3)
The selection algorithm of the optimal defense strategy under this model is designed. This algorithm can provide effective support for active defense in the process of network attack and defense.The remainder of this paper is organized as follows. Related work is discussed in Section2. In Section 3, the network attack and defense stochastic differential game model and corresponding concepts are described and analyzed. In Section 4, the stochastic differential game optimal defense strategy algorithm is introduced. Simulation experiments and results analysis are presented in Section 5. Finally, this paper is concluded in Section 6.


## 2. Related Work

The application of evolutionary game theory in cybersecurity has become a study boom in recent years. In the actual attack and defense process, the change of the system operating environment and the disturbance of other external factors have stochasticity. Therefore, researchers began to introduce stochastic evolutionary game theory into the study of cybersecurity. There are two main aspects of concern: the first is to consider the offensive and defensive process as a random jump between multistates. The other is to construct a stochastic evolutionary game using stochastic differential equations.In the analysis of the vulnerability of the network environment, the authors in [9] studied the security and reliability issues of software and hardware services. They built the Markov chain to construct a stochastic evolution alliance game to evaluate the optimal strategy, and the model can be applied to various defensive scenarios in the cloud computing network. From the perspective of attack and defense, the authors in [10] utilized the game model to find network vulnerability state and established the mapping relationship between attack and defense states. They quantified the level of network vulnerability and proposed a hidden Markov model. On this basis, they accurately inferred the attacking intent using the Viterbi algorithm. Govaert et al. modeled system dynamics as a discrete-time Markov process [11], which identifies equilibrium states and periods. And, in any initial state, it can converge to a balanced state for a limited time. The above literature [9–11] utilizes the Markov process, which can accurately characterize the stochastic behavior of the network system and the interrelationship between components. And it is convenient for calculating various safety targets. Wang et al. established an attack and defense game model based on stochastic Petri nets [12], which can analyze and evaluate the attack success rate, average attack time, vulnerable nodes, and potential attack paths of the target network. He et al. [13] defined the Stochastic Colored Petri Nets (SCPN) based on the Internet of Things (IoT) when studying the offensive and defensive scenarios of the smart home and obtained the game model of security situational awareness. It can effectively predict the attacker’s potential attack strategy and achieve the purpose of promoting defense strategy selection. Talukder et al. [14] found that it is necessary to construct a suitable model to express the spread of threats in mobile IP. Talukder proposed four common mobile IP attacks and used SCPN as model, which effectively reduced the probability of successful attackers. To assess the risk of intrusion, El Bouchti and Nahhal [15] introduced the process and rules of constructing an SCPN model using attack trees and showed how to transform and analyze the attack tree in Stochastic Game Nets (SGN). Fanti et al. [16] proposed a network model of satellite base station (SBS) affected by attack and defense. The optimal defense strategy was obtained by calculating the Nash equilibrium, and the model was able to obtain the evolution equilibrium state under the stochastic game rules. The above literature [12–16] has a strong dynamic analysis ability for the concurrency, asynchrony, and uncertainty of the system. It has the advantages of less modeling language and intuitive graphical representation that can describe the state and behavior of the system. It has some functions that other methods do not have, such as system description, security analysis, and system testing. It can be accomplished graphically in the system model framework. However, these methods do not consider the issue of the participants’ payoffs and costs.Huang et al. [17] found that the attack and defense strategy usually changes dynamically and continuously. Therefore, Huang used the Ito stochastic differential equation to construct a stochastic evolutionary game from the perspective of the actual attack and defense. The model accurately shows the evolution process of attack and defense by analyzing the continuous game evolution. The process of path discovery was modeled as a noncooperative stochastic evolution game when Wang et al. [18] studied radio network security. It was carried out by distributed strategy learning at each stage of the game process, which effectively bypassed the malicious nodes of the hybrid attack strategy. Wei et al. [19] designed the optimal load shedding technique to quantify the physical impact of the coordinated attack. For the interaction between attackers and defenders, the stochastic game model is proposed to select the optimal defense strategy and protect the network. The above literature [17–19] uses stochastic differential equations to describe the stochastic evolution process. It considers the direct impact of network security incidents and can effectively prevent malicious threats.In addition, Riehl and Cao [20] introduced a hierarchical approximation algorithm while studying the stochastic evolutionary games. It can search the required strategies in stochastic evolutionary games and find the optimal results of the network attack and defense. Liu et al. [21] integrated multiple network security elements (such as assets, threats, and vulnerabilities) of multisensor mobile phones into standard data sets to improve the awareness of network security. The Nash equilibrium of the hybrid strategy is calculated by the stochastic game model, and the security status of the network is evaluated effectively, comprehensively, and accurately. Subbulakshmi et al. [22] constructed a stochastic evolutionary game model to analyze the destructive techniques related to radio networks. The model evaluated the optimal solution to improve network performance. Kumar et al. [23] proposed a stochastic alliance game to realize data distribute in-vehicle network physical systems (VCPS) when studying the safety and comfort of the in-vehicle network. Vehicles can access various resources from the cloud environment. These resources help people find optimized strategy selection by transmitting short-range, medium-range, and remote information. Arfaoui et al. [24] proposed a stochastic game model to balance network performance and security. The model is more efficient than the basic algorithm in terms of network lifecycle and throughput. Chen and Yeh [25] discussed the robustness of noncooperative evolutionary game strategies from the perspective of stochastic Nash Equilibrium and then explored the application of stochastic evolutionary game theory.In summary, the researchers regard the attack and defense process as the process of random hopping in multistate when using SCPN for modeling. It can better describe the offensive and defensive processes, but it is difficult to avoid the conditions that need to satisfy the complete information. Researchers constructed stochastic differential models with incomplete information by using stochastic differential equations, which can effectively describe the network attack and defense process. However, most literature is limited to a specific network environment for attack and defense confrontation, which leads to low versatility. Aiming at the existing research results, this paper studies and proposes a stochastic differential game model of network attack and defense that introduces the stochastic differential equation, and the model has Markov property. Based on the network attack and defense scenario, the evolution trend of the behavior strategies of the network attack and defense groups is analyzed. We find the optimal defense strategy and effectively analyze the behaviors of the attack and defense strategy on this basis.


## 3. Network Attack and Defense Stochastic Differential Game Model

The attack and defense groups choose different strategies for the game based on the incomplete rationality of attackers and defenders. Both sides constantly try to adjust and improve their decision-making methods in the process of attack and defense and form a new situation of the game finally. This process also highlights the dynamic equilibrium of evolutionary game theory.


### 3.1. Model Definition

The players will always suffer from some uncertain factors in the actual network attack and defense. Therefore, this paper defines the Network Attack-Defense Stochastic Differential Game Model (NADSDGM).Definition 1.
The network attack and defense stochastic differential game model is defined as a quaternion modelNADSDGM=N,S,U,τ, where we have the following:(a)
N=NA,ND represents the players in the attack and defense evolution game, that is, the participants who adopt strategies in the game. The participants have different meanings in different environments. They can represent individuals and can also represent a team or a group of multiple teams. Among them, NA are the attackers and ND are the defenders (defense system).(b)
S=SA,SD represents the set of strategies of the players in the game; it is the tool and means for the players to play the game, where SA=SA1,SA2,…,SAm represents the set of attack strategies and SD=SD1,SD2,…,SDn represents the set of defense strategies. For the attackers, there are m strategies for attacking. Correspondingly, there are n strategies for implementing defense by the defenders.(c)
U=UA,UD is the set of payoff functions of the players. UA=UA0,UA1. UA0 represents the payoffs of the attackers when the attack is not performed, and UA1 represents the payoffs of the successful attack by the attackers. UD=UD0,UD1. UD0 represents the expected payoffs when the defenders do not make a defensive investment, and UD1 represents the expected payoffs of the defenders after defensive investment.(d)
τ=τ0,τ1 indicates stochasticity. Among them, τ0=0 indicates that the model does not use stochastic disturbance factors. τ1=1 indicates that the model uses stochastic disturbance factors.


### 3.2. Parameter Quantization

In the analysis of the attack and defense evolution game, we first define some relevant parameters to be convenient for the quantification of the payoffs.Definition 2.
Attack costCA: this indicates the financial and material resources that the attackers need to perform the attack.
The defenders do not invest in a defensive strategy, whether the attackers can successfully implement the attack depends only on the defenders’ system vulnerabilities, and the attack cost at this time isCA0. When the defenders make defensive investments, it will increase the attacking difficulty of the attackers, and the attack cost is CA1 . Obviously, CA0<CA1.Definition 3.
Incentive mechanism remuneration R: this represents the third-party regulator’s reward for the defenders.
In today’s information age, the degree of possession of information resources and monopoly determine benefits. The main reason why the target network is attacked is that the information is not public and opaque, and the attackers want to obtain certain information through the attack. Therefore, social regulators use incentive mechanisms to motivate defenders to properly publish information and share resources without harming their interests. The society that benefits from this will also reward defenders. To reduce the damage caused by the defense system being attacked, the defenders choose appropriate public information to receive social rewards. The more beneficial the public information is to society, the more rewards the incentives will generate.R represents the remuneration for the incentive mechanism. The remuneration is R0 when the defenders do not make a defensive investment. When the defenders make defensive investments, the remuneration is R1.Definition 4.
Penalty cost G: this means that the third-party regulator punishes attackers who have committed attacks.
Internet attacks can lead to a series of cybersecurity issues, such as the users’ data being leaked and the network’s services being forced to be interrupted. It affects people’s daily work and life and even affects the country’s safety in case of seriousness. Therefore, it is the responsibility of the third-party regulatory authority to punish the attackers for violating the cybersecurity.G is used to indicate the punishment of the attackers by the supervisor. When the defenders do not make a defensive investment, the attackers receive the penalty of G0. Correspondingly, when the defenders take defensive investments, the attackers receive the penalty of G1.Definition 5.
Attack lethality coefficientλ and defender loss l.
In actual network attack and defense, for different attack strategies, the defenders’ loss is affected by the lethality of the attack. Assume that the total loss caused by a network attack to the defenders isl. The more lethal the attack strategy is, the less likely the defenders are to resist successfully, and the greater the loss suffered, that is, the greater the loss suffered by the defenders. On the contrary, the weaker the lethality of the attack strategy, the smaller the loss suffered by the defenders.
Let attack lethality coefficient be(1)λ=λk=1,k=m,e−1/k,1≤k<m,0,k=0.
Among them,m represents the attack dangerous level (m∈N∗), and the lethality coefficient of attack changes under the influence of the dangerous degree of attack strategy.
We definel=λL. When the attack dangerous level is not enough to hurt the defenders, the loss of the defenders is 0. When the attack dangerous level is at 1≤k≤m, the defenders can take remedial measures in time to reduce part of the loss. At this time, the attack lethality coefficient is e−1/k, and the defenders’ loss is l=e−1/kL. When the attack dangerous level is large enough, it can be regarded as attack lethality coefficient being 1 and defenders’ loss being l=L.Definition 6.
Total return E: this represents the total return that the attackers can obtain from a successful attack.
When the defenders adopt defensive investment strategies, the payoffs of the attackers’ successful attack areP1E−CA1−G1. When the defenders do not adopt defensive investment strategies, the payoffs of the attackers’ successful attack are P0E−CA0−G0.
Additionally, the defenders as the target network are also capable of a certain defense. But to protect their infrastructure and information assets from harm, defenders can choose to increase their defensive investments against network attacks. Assume that the defenders’ original defensive infrastructure and informational assets are collectively called the original assetV0, and the investment cost per time is Vadd. Therefore, before and after the defensive investment by the defenders, the losses caused by the network attack are P0l and P1l, respectively. When the attackers successfully attack, if the defenders choose defensive investment strategies, their expected payoffs are V0−Vadd−P1l+R1. And if the defenders do not adopt a defense investment strategy, the expected payoffs are V0−P0l+R0. On the contrary, when the attackers do not take any attack, the attackers’ expected payoffs are 0. And the defenders’ expected payoffs before and after the defensive investment strategy are V0+R0 and V0−Vadd+R1.
The main parameters and descriptions involved above are shown in Table1.Table 1
Main parameters and descriptions.
ParameterDescriptionCA0Attackers’ required attack cost before defensive investmentCA1Attackers’ required attack cost after defensive investmentP0Probability of successful attack before defensive investmentP1Probability of successful attack after defensive investmentV0Defenders’ original defensive ability (original assets)VaddDefenders’ increased defensive ability (defensive investment)RThe social rewards for defenders’ public informationGThe attackers are punished for the attacklThe loss suffered by the defenders after being attackedEThe total return from a successful attack by the attackers


### 3.3. Stochastic Differential Equation

Assume that, in the process of attack and defense games, the proportion of attack strategies adopted by the attackers’ groups isx, and the proportion of adopting nonattack strategies is 1−x. The proportion of defensive investment strategies and nondefensive investment strategies in the defenders’ group is y and 1−y, respectively. Using the above parameters, the payoff matrix of the network attack and defense evolutionary game model is shown in Table 2.Table 2
Differential game expectation payoff matrix.
Defender/attackerAttackNo attackInvestmentV0−Vadd−P1l+R1 and P1E−CA1−G1V0−Vadd+R1, 0No investmentV0−P0l+R0 and P0E−CA0−G0V0+R0, 0UseUD1 to indicate the expected payoffs of the defenders when the defenders choose to invest in the defense strategy, and UD0 indicates the expected payoffs of the defenders when the defenders do not invest in the defense strategy. From the above payoff matrix, we can know(2)UD1=xV0−Vadd−P1l+R1+1−xV0−Vadd+R1=−xP1λL+V0−Vadd+R1,(3)UD0=xV0−P0l+R0+1−xV0+R0=−xP0λL+V0+R0.UseUD¯ to indicate the average payoffs of the defenders, which can be obtained by equations (2) and (3)(4)UD¯=yUD1+1−yUD0=yxλLP0−P1+R1−R0−Vadd−xλP0L+V0+R0.Correspondingly,UA1 indicates the expected payoffs of the attackers when the attackers adopt the attack strategy, and UA0 indicates the payoffs of the attackers when the attackers do not adopt the attack strategy; that is,(5)UA1=yP1E−CA1−G1+1−yP0E−CA0−G0=−yP0E−P1E1+G1−G0+C1−C0+P0E−C0−G0,(6)UA0=0.UseUA¯ to indicate the average payoffs of the attackers, which is available from equation (5) and (6)(7)UA¯=xUA1+1−xUA0=−xyP0E−P1E1+G1−G0+C1−C0+xP0E−C0−G0.According to the above analysis, the replication dynamic equation of the offensive and defensive evolution game model is obtained. From (5) and (7), the attackers’ replication dynamic equation is(8)dxt=xUA1−UA¯dt=x1−x−yP0E−P1E1+G1−G0+C1−C0P0E−C0−G0dt.The defenders’ replication dynamic equation is obtained by equation (2) and (4)(9)dyt=yUD1−UD¯dt=y1−yxλLP0−P1+R1−R0−Vadddt.In order to characterize stochastic disturbance factors, the common method is to add a stochastic disturbance after replication dynamic equation. It satisfies the Gaussian hypothesis and obeys the normal distribution, which can reflect the stochastic effects caused by many tiny factors. Common Markov processes include Poisson process and Wiener process, and white noise has become a kind of stochastic disturbance commonly used in system analysis [17]. Therefore, this paper uses the white noise process as a stochastic disturbance in the game process, and (8) and (9) are modified to obtain(10)dxt=x1−xV0−Vadd+R1−V0+R0+λLP0−P1ydt+τx1−xydθ1t,dyt=y1−yP0E−CA0−G0+P1E−CA1−G1−P0E−CA0−G0xdt+τy1−yxdθ2t,dθ1tdθ2t=dt,where dθ1tdθ2t represents the Wiener process; it has Markov property.


## 3.1. Model Definition

The players will always suffer from some uncertain factors in the actual network attack and defense. Therefore, this paper defines the Network Attack-Defense Stochastic Differential Game Model (NADSDGM).Definition 1.
The network attack and defense stochastic differential game model is defined as a quaternion modelNADSDGM=N,S,U,τ, where we have the following:(a)
N=NA,ND represents the players in the attack and defense evolution game, that is, the participants who adopt strategies in the game. The participants have different meanings in different environments. They can represent individuals and can also represent a team or a group of multiple teams. Among them, NA are the attackers and ND are the defenders (defense system).(b)
S=SA,SD represents the set of strategies of the players in the game; it is the tool and means for the players to play the game, where SA=SA1,SA2,…,SAm represents the set of attack strategies and SD=SD1,SD2,…,SDn represents the set of defense strategies. For the attackers, there are m strategies for attacking. Correspondingly, there are n strategies for implementing defense by the defenders.(c)
U=UA,UD is the set of payoff functions of the players. UA=UA0,UA1. UA0 represents the payoffs of the attackers when the attack is not performed, and UA1 represents the payoffs of the successful attack by the attackers. UD=UD0,UD1. UD0 represents the expected payoffs when the defenders do not make a defensive investment, and UD1 represents the expected payoffs of the defenders after defensive investment.(d)
τ=τ0,τ1 indicates stochasticity. Among them, τ0=0 indicates that the model does not use stochastic disturbance factors. τ1=1 indicates that the model uses stochastic disturbance factors.


## 3.2. Parameter Quantization

In the analysis of the attack and defense evolution game, we first define some relevant parameters to be convenient for the quantification of the payoffs.Definition 2.
Attack costCA: this indicates the financial and material resources that the attackers need to perform the attack.
The defenders do not invest in a defensive strategy, whether the attackers can successfully implement the attack depends only on the defenders’ system vulnerabilities, and the attack cost at this time isCA0. When the defenders make defensive investments, it will increase the attacking difficulty of the attackers, and the attack cost is CA1 . Obviously, CA0<CA1.Definition 3.
Incentive mechanism remuneration R: this represents the third-party regulator’s reward for the defenders.
In today’s information age, the degree of possession of information resources and monopoly determine benefits. The main reason why the target network is attacked is that the information is not public and opaque, and the attackers want to obtain certain information through the attack. Therefore, social regulators use incentive mechanisms to motivate defenders to properly publish information and share resources without harming their interests. The society that benefits from this will also reward defenders. To reduce the damage caused by the defense system being attacked, the defenders choose appropriate public information to receive social rewards. The more beneficial the public information is to society, the more rewards the incentives will generate.R represents the remuneration for the incentive mechanism. The remuneration is R0 when the defenders do not make a defensive investment. When the defenders make defensive investments, the remuneration is R1.Definition 4.
Penalty cost G: this means that the third-party regulator punishes attackers who have committed attacks.
Internet attacks can lead to a series of cybersecurity issues, such as the users’ data being leaked and the network’s services being forced to be interrupted. It affects people’s daily work and life and even affects the country’s safety in case of seriousness. Therefore, it is the responsibility of the third-party regulatory authority to punish the attackers for violating the cybersecurity.G is used to indicate the punishment of the attackers by the supervisor. When the defenders do not make a defensive investment, the attackers receive the penalty of G0. Correspondingly, when the defenders take defensive investments, the attackers receive the penalty of G1.Definition 5.
Attack lethality coefficientλ and defender loss l.
In actual network attack and defense, for different attack strategies, the defenders’ loss is affected by the lethality of the attack. Assume that the total loss caused by a network attack to the defenders isl. The more lethal the attack strategy is, the less likely the defenders are to resist successfully, and the greater the loss suffered, that is, the greater the loss suffered by the defenders. On the contrary, the weaker the lethality of the attack strategy, the smaller the loss suffered by the defenders.
Let attack lethality coefficient be(1)λ=λk=1,k=m,e−1/k,1≤k<m,0,k=0.
Among them,m represents the attack dangerous level (m∈N∗), and the lethality coefficient of attack changes under the influence of the dangerous degree of attack strategy.
We definel=λL. When the attack dangerous level is not enough to hurt the defenders, the loss of the defenders is 0. When the attack dangerous level is at 1≤k≤m, the defenders can take remedial measures in time to reduce part of the loss. At this time, the attack lethality coefficient is e−1/k, and the defenders’ loss is l=e−1/kL. When the attack dangerous level is large enough, it can be regarded as attack lethality coefficient being 1 and defenders’ loss being l=L.Definition 6.
Total return E: this represents the total return that the attackers can obtain from a successful attack.
When the defenders adopt defensive investment strategies, the payoffs of the attackers’ successful attack areP1E−CA1−G1. When the defenders do not adopt defensive investment strategies, the payoffs of the attackers’ successful attack are P0E−CA0−G0.
Additionally, the defenders as the target network are also capable of a certain defense. But to protect their infrastructure and information assets from harm, defenders can choose to increase their defensive investments against network attacks. Assume that the defenders’ original defensive infrastructure and informational assets are collectively called the original assetV0, and the investment cost per time is Vadd. Therefore, before and after the defensive investment by the defenders, the losses caused by the network attack are P0l and P1l, respectively. When the attackers successfully attack, if the defenders choose defensive investment strategies, their expected payoffs are V0−Vadd−P1l+R1. And if the defenders do not adopt a defense investment strategy, the expected payoffs are V0−P0l+R0. On the contrary, when the attackers do not take any attack, the attackers’ expected payoffs are 0. And the defenders’ expected payoffs before and after the defensive investment strategy are V0+R0 and V0−Vadd+R1.
The main parameters and descriptions involved above are shown in Table1.Table 1
Main parameters and descriptions.
ParameterDescriptionCA0Attackers’ required attack cost before defensive investmentCA1Attackers’ required attack cost after defensive investmentP0Probability of successful attack before defensive investmentP1Probability of successful attack after defensive investmentV0Defenders’ original defensive ability (original assets)VaddDefenders’ increased defensive ability (defensive investment)RThe social rewards for defenders’ public informationGThe attackers are punished for the attacklThe loss suffered by the defenders after being attackedEThe total return from a successful attack by the attackers


## 3.3. Stochastic Differential Equation

Assume that, in the process of attack and defense games, the proportion of attack strategies adopted by the attackers’ groups isx, and the proportion of adopting nonattack strategies is 1−x. The proportion of defensive investment strategies and nondefensive investment strategies in the defenders’ group is y and 1−y, respectively. Using the above parameters, the payoff matrix of the network attack and defense evolutionary game model is shown in Table 2.Table 2
Differential game expectation payoff matrix.
Defender/attackerAttackNo attackInvestmentV0−Vadd−P1l+R1 and P1E−CA1−G1V0−Vadd+R1, 0No investmentV0−P0l+R0 and P0E−CA0−G0V0+R0, 0UseUD1 to indicate the expected payoffs of the defenders when the defenders choose to invest in the defense strategy, and UD0 indicates the expected payoffs of the defenders when the defenders do not invest in the defense strategy. From the above payoff matrix, we can know(2)UD1=xV0−Vadd−P1l+R1+1−xV0−Vadd+R1=−xP1λL+V0−Vadd+R1,(3)UD0=xV0−P0l+R0+1−xV0+R0=−xP0λL+V0+R0.UseUD¯ to indicate the average payoffs of the defenders, which can be obtained by equations (2) and (3)(4)UD¯=yUD1+1−yUD0=yxλLP0−P1+R1−R0−Vadd−xλP0L+V0+R0.Correspondingly,UA1 indicates the expected payoffs of the attackers when the attackers adopt the attack strategy, and UA0 indicates the payoffs of the attackers when the attackers do not adopt the attack strategy; that is,(5)UA1=yP1E−CA1−G1+1−yP0E−CA0−G0=−yP0E−P1E1+G1−G0+C1−C0+P0E−C0−G0,(6)UA0=0.UseUA¯ to indicate the average payoffs of the attackers, which is available from equation (5) and (6)(7)UA¯=xUA1+1−xUA0=−xyP0E−P1E1+G1−G0+C1−C0+xP0E−C0−G0.According to the above analysis, the replication dynamic equation of the offensive and defensive evolution game model is obtained. From (5) and (7), the attackers’ replication dynamic equation is(8)dxt=xUA1−UA¯dt=x1−x−yP0E−P1E1+G1−G0+C1−C0P0E−C0−G0dt.The defenders’ replication dynamic equation is obtained by equation (2) and (4)(9)dyt=yUD1−UD¯dt=y1−yxλLP0−P1+R1−R0−Vadddt.In order to characterize stochastic disturbance factors, the common method is to add a stochastic disturbance after replication dynamic equation. It satisfies the Gaussian hypothesis and obeys the normal distribution, which can reflect the stochastic effects caused by many tiny factors. Common Markov processes include Poisson process and Wiener process, and white noise has become a kind of stochastic disturbance commonly used in system analysis [17]. Therefore, this paper uses the white noise process as a stochastic disturbance in the game process, and (8) and (9) are modified to obtain(10)dxt=x1−xV0−Vadd+R1−V0+R0+λLP0−P1ydt+τx1−xydθ1t,dyt=y1−yP0E−CA0−G0+P1E−CA1−G1−P0E−CA0−G0xdt+τy1−yxdθ2t,dθ1tdθ2t=dt,where dθ1tdθ2t represents the Wiener process; it has Markov property.


## 4. Optimal Defense Strategy Selection

In this section, the evolutionary equilibrium solution and stability analysis of the stochastic equation are firstly proved, and then the optimal strategy selection algorithm is given.


### 4.1. Evolutionary Equilibrium Solution

Because the stochastic game model proposed in this paper is composed of nonlinear Ito stochastic differential equations, the analytical solution of the equations cannot be obtained directly. Therefore, in this section, we first prove that the stochastic differential equation presented in this paper has a unique solution (i.e., satisfying the local Lipschitz condition and the linear growth condition [26–29]). And in the following Section 5.2, we use the explicit Euler numerical method to find the solution, so as to obtain the corresponding evolutionary equilibrium solution of attack and defense.Theorem 1.
The parameters in Table1 are known, for x∈0,1, y∈0,1, q∈0,1, t∈0,T, and τ=1; equation (10) has a unique solution.Proof.
We rewrite equation (10)(11)dxt=f1x,ydt+g1x,ydθ1t,(12)dyt=f2x,ydt+g2x,ydθ2t.
Among them,(13)f1x,y=x1−xV0−Vadd+R1−V0+R0+λLP0−P1y,f2x,y=1−yP0E−CA0−G0+P1E−CA1−G1−P0E−CA0−G0x,g1x,y=x1−xy,g2x,y=y1−yx.
Obviously,f1x,y, f2x,y, g1x,y, and g2x,y are continuous on 0,1×0,1.
For equation (11), we first verify that it satisfies the local Lipschitz condition. For any x and x∗ in 0,1, then(14)f1x,y−f1x∗,y=x1−xV0−Vadd+R1−V0+R0+λLP0−P1y−x∗1−x∗V0−Vadd+R1−V0+R0+λLP0−P1y=V0−Vadd+R1−V0+R0+λLP0−P1yx1−x−x∗1−x∗=V0−Vadd+R1−V0+R0+λLP0−P1y⋅x−x∗⋅1−x−x∗≤x−x∗V0−Vadd+R1−V0+R0+λLP0−P1y⋅1−x−x∗≤x−x∗V0−Vadd+R1−V0+R0+λLP0−P1.
Therefore,(15)f1x,y−f1x∗,y2≤M1x−x∗2,where M1=V0−Vadd+R1−V0+R0+λLP0−P12 is the positively constant.
In addition,(16)g1x,y−g1x∗,y2=x1−xy−x∗1−x∗y2=x−x∗1−x−x∗y2=x−x∗2⋅1−x−x∗2⋅y2≤x−x∗2.
LetM=maxM1,1; then(17)maxf1x,y−f1x∗,y2,g1x,y−g1x∗,y2≤M⋅x−x∗2.
Therefore, wheny∈0,1, equation (11) satisfies the local Lipschitz condition.
Next, we verify that equation (11) satisfies the condition of linear growth. For any x in 0,1, then(18)f1x,y2=x1−xV0−Vadd+R1−V0+R0+λLP0−P1y2=x1−x2⋅V0−Vadd+R1−V0+R0+λLP0−P1y2≤x1−x2⋅V0−Vadd+R1−V0+R0+λLP0−P12≤1+x2⋅V0−Vadd+R1−V0+R0+λLP0−P12=B11+x2,where B1=V0−Vadd+R1−V0+R0+λLP0−P12 is the positively constant.
Forg1x,y2, we construct function kx=x21−x2y2/1+x2.
By derivingx, we can get k′x=2x1−x1−2x−x3/1+x22y2. Because of x∈0,1, y∈0,1, so let k′x>0; we can get 2x1−x1−2x−x3>0. Therefore, let η∈0,1; when 0<x<η<1, zx is an increasing function. When 0<η<x<1, kx is an decreasing function.
Consequently,(19)max0≤x≤1kx=kη=η21−η2y21+η2≤η21−η21+η2,namely x21−x2y2/1+x2≤η21−η2/1+η2; that is g1x,y2≤B21+x2.
Among them,B2=η21−η2/1+η2. Let B=maxB1,B2; then(20)maxf1x,y2,g1x,y2≤B1+x2.
Therefore, equation (11) satisfies the condition of linear growth.
Well, giveny∈0,1, equation (11) has a unique solution x. Similarly, given x∈0,1, equation (12) has a unique solution y.
In summary, equation (10) has a unique solution.


### 4.2. Evolutionary Stability Analysis

For the stochastic game model constructed, the stability of the game model is proved according to the conclusion described in [26–29], that is, the expected operation of the Ito integral and the exchangeable property of the integral operation.Theorem 2.
The parameters in Table1 are known, for x∈0,1, y∈0,1, q∈0,1, t∈0,T, and τ=1; the zero solution of equation (10) is stable in the sense of mean square exponential.Proof.
Equations (11) and (12) are expressed as integral equations as(21)xt=x0+∫0tf1x,ydq+∫0tg1x,ydθ1q,yt=y0+∫0tf2x,ydq+∫0tg2x,ydθ2q,where x0 and y0 are the values when t=0.
Letmx,y=1/x02eztExt2. Among them, EΘ is the expectation and z is the positively constant.
Then,(22)mx,y=1x02ezt⋅Ext2=1x02ezt⋅Ex0+∫0tf1x,ydq+∫0tg1x,ydθ1q2≤1x02ezt⋅Ex02+∫0Tf1x,ydq2+2x0∫0Tf1x,ydq+2x0∫0Tg1x,ydθ1q+2∫0Tf1x,ydq∫0Tg1x,ydθ1q≤1x02ezt⋅x02+T16V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL2+T2x0V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL+4x0⋅E∫0Tdθ1q=1x02ezt⋅x02+T2V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL⋅18V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL+x0.
Thereby,(23)Ext2≤Nx02e−μt.
Among them,(24)N=x02+T2V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL⋅18V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL+x0>0,μ=z>0.
That is, for anyX0, there is a constant of μ>0, N>0, and when 0≤t≤T, there is Ext2≤Nx02e−μt; then the zero solution of equation (11) can be called the stability on the mean square exponential.
Similarly, for the zero solution of equation (12), the mean square exponential is also stable.


### 4.3. The Optimal Defense Strategy Selection Algorithm

In the process of the network attack and defense, the attackers and the defenders play opposite to each other. Each player in the game is constantly testing, adjusting, and improving in the game to maximize their expected returns. Under the guidance of this principle, both attacker’s strategy and defenders’ strategy will gradually tend to balance. Neither party will try to change this strategy because the party that does not tend to balance will be reducing payoffs. That is to say, the strategy of achieving balance at this time is the optimal strategy. The specific Algorithm1 is described as follows.Algorithm 1: Optimal defense strategy selection algorithm.
InputNA, ND who participated in the game and host node information.Output Attack strategy SA∗, optimal defense strategy SD∗.Begin(1)
InitializeNADSDGM=N,S,U,τ /∗ Initialize stochastic evolutionary game model∗/(2)
Constructx, y/∗ Construct the group probability of the selected strategy set of both attack and defense ∗/(3)
Constructing a stochastic evolution game matrix between attack and defense(4)
Construct the stochastic differential equation of the attackers and defenders, and see equation (10) for details.(5)
Numerical analysis of the equation using the explicit Euler equation(6)
BackSA∗, SD∗End


## 4.1. Evolutionary Equilibrium Solution

Because the stochastic game model proposed in this paper is composed of nonlinear Ito stochastic differential equations, the analytical solution of the equations cannot be obtained directly. Therefore, in this section, we first prove that the stochastic differential equation presented in this paper has a unique solution (i.e., satisfying the local Lipschitz condition and the linear growth condition [26–29]). And in the following Section 5.2, we use the explicit Euler numerical method to find the solution, so as to obtain the corresponding evolutionary equilibrium solution of attack and defense.Theorem 1.
The parameters in Table1 are known, for x∈0,1, y∈0,1, q∈0,1, t∈0,T, and τ=1; equation (10) has a unique solution.Proof.
We rewrite equation (10)(11)dxt=f1x,ydt+g1x,ydθ1t,(12)dyt=f2x,ydt+g2x,ydθ2t.
Among them,(13)f1x,y=x1−xV0−Vadd+R1−V0+R0+λLP0−P1y,f2x,y=1−yP0E−CA0−G0+P1E−CA1−G1−P0E−CA0−G0x,g1x,y=x1−xy,g2x,y=y1−yx.
Obviously,f1x,y, f2x,y, g1x,y, and g2x,y are continuous on 0,1×0,1.
For equation (11), we first verify that it satisfies the local Lipschitz condition. For any x and x∗ in 0,1, then(14)f1x,y−f1x∗,y=x1−xV0−Vadd+R1−V0+R0+λLP0−P1y−x∗1−x∗V0−Vadd+R1−V0+R0+λLP0−P1y=V0−Vadd+R1−V0+R0+λLP0−P1yx1−x−x∗1−x∗=V0−Vadd+R1−V0+R0+λLP0−P1y⋅x−x∗⋅1−x−x∗≤x−x∗V0−Vadd+R1−V0+R0+λLP0−P1y⋅1−x−x∗≤x−x∗V0−Vadd+R1−V0+R0+λLP0−P1.
Therefore,(15)f1x,y−f1x∗,y2≤M1x−x∗2,where M1=V0−Vadd+R1−V0+R0+λLP0−P12 is the positively constant.
In addition,(16)g1x,y−g1x∗,y2=x1−xy−x∗1−x∗y2=x−x∗1−x−x∗y2=x−x∗2⋅1−x−x∗2⋅y2≤x−x∗2.
LetM=maxM1,1; then(17)maxf1x,y−f1x∗,y2,g1x,y−g1x∗,y2≤M⋅x−x∗2.
Therefore, wheny∈0,1, equation (11) satisfies the local Lipschitz condition.
Next, we verify that equation (11) satisfies the condition of linear growth. For any x in 0,1, then(18)f1x,y2=x1−xV0−Vadd+R1−V0+R0+λLP0−P1y2=x1−x2⋅V0−Vadd+R1−V0+R0+λLP0−P1y2≤x1−x2⋅V0−Vadd+R1−V0+R0+λLP0−P12≤1+x2⋅V0−Vadd+R1−V0+R0+λLP0−P12=B11+x2,where B1=V0−Vadd+R1−V0+R0+λLP0−P12 is the positively constant.
Forg1x,y2, we construct function kx=x21−x2y2/1+x2.
By derivingx, we can get k′x=2x1−x1−2x−x3/1+x22y2. Because of x∈0,1, y∈0,1, so let k′x>0; we can get 2x1−x1−2x−x3>0. Therefore, let η∈0,1; when 0<x<η<1, zx is an increasing function. When 0<η<x<1, kx is an decreasing function.
Consequently,(19)max0≤x≤1kx=kη=η21−η2y21+η2≤η21−η21+η2,namely x21−x2y2/1+x2≤η21−η2/1+η2; that is g1x,y2≤B21+x2.
Among them,B2=η21−η2/1+η2. Let B=maxB1,B2; then(20)maxf1x,y2,g1x,y2≤B1+x2.
Therefore, equation (11) satisfies the condition of linear growth.
Well, giveny∈0,1, equation (11) has a unique solution x. Similarly, given x∈0,1, equation (12) has a unique solution y.
In summary, equation (10) has a unique solution.


## 4.2. Evolutionary Stability Analysis

For the stochastic game model constructed, the stability of the game model is proved according to the conclusion described in [26–29], that is, the expected operation of the Ito integral and the exchangeable property of the integral operation.Theorem 2.
The parameters in Table1 are known, for x∈0,1, y∈0,1, q∈0,1, t∈0,T, and τ=1; the zero solution of equation (10) is stable in the sense of mean square exponential.Proof.
Equations (11) and (12) are expressed as integral equations as(21)xt=x0+∫0tf1x,ydq+∫0tg1x,ydθ1q,yt=y0+∫0tf2x,ydq+∫0tg2x,ydθ2q,where x0 and y0 are the values when t=0.
Letmx,y=1/x02eztExt2. Among them, EΘ is the expectation and z is the positively constant.
Then,(22)mx,y=1x02ezt⋅Ext2=1x02ezt⋅Ex0+∫0tf1x,ydq+∫0tg1x,ydθ1q2≤1x02ezt⋅Ex02+∫0Tf1x,ydq2+2x0∫0Tf1x,ydq+2x0∫0Tg1x,ydθ1q+2∫0Tf1x,ydq∫0Tg1x,ydθ1q≤1x02ezt⋅x02+T16V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL2+T2x0V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL+4x0⋅E∫0Tdθ1q=1x02ezt⋅x02+T2V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL⋅18V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL+x0.
Thereby,(23)Ext2≤Nx02e−μt.
Among them,(24)N=x02+T2V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL⋅18V0−Vadd+R1−V0+R0+−P1L−λL+P0L−λL+x0>0,μ=z>0.
That is, for anyX0, there is a constant of μ>0, N>0, and when 0≤t≤T, there is Ext2≤Nx02e−μt; then the zero solution of equation (11) can be called the stability on the mean square exponential.
Similarly, for the zero solution of equation (12), the mean square exponential is also stable.


## 4.3. The Optimal Defense Strategy Selection Algorithm

In the process of the network attack and defense, the attackers and the defenders play opposite to each other. Each player in the game is constantly testing, adjusting, and improving in the game to maximize their expected returns. Under the guidance of this principle, both attacker’s strategy and defenders’ strategy will gradually tend to balance. Neither party will try to change this strategy because the party that does not tend to balance will be reducing payoffs. That is to say, the strategy of achieving balance at this time is the optimal strategy. The specific Algorithm1 is described as follows.Algorithm 1: Optimal defense strategy selection algorithm.
InputNA, ND who participated in the game and host node information.Output Attack strategy SA∗, optimal defense strategy SD∗.Begin(1)
InitializeNADSDGM=N,S,U,τ /∗ Initialize stochastic evolutionary game model∗/(2)
Constructx, y/∗ Construct the group probability of the selected strategy set of both attack and defense ∗/(3)
Constructing a stochastic evolution game matrix between attack and defense(4)
Construct the stochastic differential equation of the attackers and defenders, and see equation (10) for details.(5)
Numerical analysis of the equation using the explicit Euler equation(6)
BackSA∗, SD∗End


## 5. Simulation Results and Analysis

In this section, we first set up a network experimental environment. Due to the nonlinearity of the stochastic game model, the model is simulated by the explicit Euler numerical method.


### 5.1. Experimental Environment

We deploy a network topology environment to simulate the network attack and defense evolution game model proposed in this paper. The validity of the model is proved by analyzing the evolutionary stability strategy.As shown in Figure1, in the network topology environment, attack host A is located on the external network and it is used to simulate a variety of attack strategies of attackers. The intranet contains three servers, namely, MySQL Server B, Web Server C, and FTP Server D. The internal network is isolated from the external network by the firewall.Figure 1
Network topology environment.Since the firewall separates the internal network from the external network, the external host can only access Web Server C and FTP Server D through the network. In the intranet, MySQL Server B, Web Server C, and FTP Server D can access each other by using user rights. The Nessus vulnerability scanner is used to perform vulnerability scanning on three server nodes in the network. The server node information is shown in Table3.Table 3
Server node information.
Host/IPOSServerVulnerability IDB 172.16.3.2LinuxMySQLCVE-2018-10757C 172.16.3.3LinuxsshCVE-2016-10012D 172.16.3.4LinuxftpCVE-2016-9499Through the analysis of the vulnerability and attack behavior of each host node in the network, combined with the China National Vulnerability Database of Information Security (CNNVD), the network attack and defense strategies are designed in the experiment, as shown in Tables4 and 5. Assuming that the network attack strategy is SA1 and SA2, the strategy SA1 has a high cost, high attack effectiveness, and strong pertinence. Strategy SA2 has low cost and low attack effectiveness, which can be considered as not attacking. In addition, assuming that the network defense strategy is SD1 and SD2 when defending against external attacks, the defenders can increase the cost to take defensive investments or can rely on the existing defense ability to passively defend.Table 4
Atomic attack information.
ID/nameNetwork attack strategySA1SA21 Remote buffer overflow2 Buffer error√3 Install Web Listener program√4 Install delete Trojan√5 Trying to steal account√6 FTP server information disclosure√7 Homepage attack√8 Check Point ZoneAlarm9 LPC to LSASS process√10 SQL injection vulnerability√Table 5
Defense strategy information.
ID/nameNetwork defense strategySD1SD21 Install MySQL patches√2 Uninstall delete Trojan√3 Install sshd patches√4 Limit packets from ports5 Delete suspicious account√6 Restart database server√√7 Install ftp patches√8 Repair database9 Close homepage√10 Add physical recourse√


### 5.2. Explicit Euler Numerical Results

For equation (10), we use the explicit Euler numerical method to simulate it [27]. N is the number of iterations, T is the game time, and the average step size is H=T/N.LetN (0, 1) denote the standard normal distribution and divide t∈0,T into N∈N+ equal parts; that is, interval 0,T is divided into 0=t0<t1<⋯<tN−1<tN=T, the average step size is H=T/N, and the node is tn=nH.The Wiener increment isΔwitn=witn−witn−1∼HN0,1, i = (1, 2), n = (1, 2, ..., N), assuming xn=xtn, yn=ytn, Δwitn=witn, and the explicit Euler iteration formula is(25)xn+1=xn+xn1−xn−1V0−Vadd+R1−V0+R0+λLP0−P1yn+τxn1−xnyn⋅Δw1n,yn+1=yn+yn1−ynP0E−CA0−G0+P1E−CA1−G1−P0E−CA0−G0xn+τyn1−ynxn⋅Δw2n.


### 5.3. Attack-Defense Simulation and Analysis

Stochastic evolutionary game is a kind of stochastic theory which combines game theory analysis with dynamic evolutionary process analysis. In the following, according to the problem situation ofx and y, multiple simulation experiments are carried out on the constructed network environment. From the obtained simulation results, the dynamic evolution law of attackers x and defenders y can be analyzed intuitively; the prediction of attack and defense strategies can be realized. And the evolutionary stability strategy is found, that is, the optimal defense strategy in this state. In the simulation experiment, it is assumed that τ=0 indicates the evolution of the attack and defense strategy without considering the stochastic disturbance factor. It is too ideal and the stochastic disturbance in the actual attack and defense is not solved. τ=1 indicates that the game evolution after considering the stochastic disturbance factor is more realistic and more effective.The problem situation isx = 0.4, y = 0.7; that is, the attackers in the group select the hybrid strategy SA1,SA2 with the probability of {0.4, 0.6}, and the defenders in the group select the hybrid strategy SD1,SD2 with the probability of {0.7, 0.3}. It can be seen from Figure 2 that, after continuous evolution, the probability of the attackers selecting the strategy SA1 gradually tends to 0 and the probability that the defenders select the strategy SD1 gradually tends to 1. Both of them reach an evolutionarily stable state. The optimal defense strategy at this time is SD1. Therefore, in this situation, the defenders belong to a more active state of defense. The defense groups are willing to adopt defensive investment strategies for its vulnerability, and it is gradually increasing. The attacker groups gradually turn to the passive state of not taking the attack. The network environment is safer.Figure 2
Group evolution trend whenx = 0.4 and y = 0.7.Figure3 shows the experimental results obtained when the problem situation is x = 0.5 and y = 0.6. The situation indicates that the attackers in the group select the hybrid strategy SA1,SA2 with the probability of {0.5, 0.5} and the defenders select the hybrid strategy SD1,SD2 with the probability of {0.6, 0.4}. As shown in Figure 3, after continuous evolution, the probability that the attackers finally select the attack strategy SA1 gradually tends to 1 and the probability that the defenders select the defense strategy SD1 gradually tends to 1. Both of them reach an evolutionarily stable state, and the optimal defense strategy at this time is SD1. Analysis of the situation at this moment shows that the attackers and the defenders are actively adopting strategies to participate in the game; the network environment is in a relatively fierce state.Figure 3
Group evolution trend whenx = 0.5 and y = 0.6.The problem situation isx = 0.4 and y = 0.3; that is, the attackers in the group select the hybrid strategy SA1,SA2 with the probability of {0.4, 0.6}, and the defenders select the hybrid strategy SD1,SD2 with the probability of {0.3, 0.7}. After continuous evolution, the probability that the attackers finally select the attack strategy SA1 gradually tends to 0 and the probability that the defenders select the defense strategy SD1 gradually approaches 0. Both of them reach an evolutionarily stable state. The optimal defense strategy at this moment is SD2. Figure 4 is a figure of experimental results in the situation of this problem. Analysis of the situation currently shows that although the network environment is relatively stable, the state of both offense and defense is relatively negative.Figure 4
Group evolution trend whenx = 0.4 and y = 0.3.The problem situation isx = 0.7 and y = 0.2; that is, the attackers select the hybrid strategy SA1,SA2 with the probability of {0.7, 0.3}, and the defenders select the hybrid strategy SD1,SD2 with the probability of {0.2, 0.8}. The experimental results obtained in this situation are shown in Figure 5. It can be observed from Figure 5 that, after continuous evolution, the probability that the attackers finally select the attack strategy SA1 gradually tends to 1 and the probability that the defenders select the defense strategy SD1 gradually approaches 0. Both of them reach an evolutionarily stable state. At this moment, the optimal defense strategy is SD2. In summary, the analysis of the situation at this time shows that the defenders choose defensive investment strategies with a small probability. It is more passive in the offensive and defensive confrontation, and the attackers gradually adopt effective attack strategies; the overall network environment is paralyzed.Figure 5
Group evolution trend whenx = 0.7 and y = 0.2.


### 5.4. The Attack Dangerous Level Analysis

Figure6 shows the effect of the attack dangerous level on attack strategies. As we can see from Figure 6, when the attack strategy SA1 is not dangerous enough to hurt the defenders (that is, when k = 0 or 1), after the evolution equilibrium is reached, the probability that the attackers continue to select the strategy SA1 is about 0. That is to say, the attackers tend not to adopt the strategy SA1. When the attack strategy SA1 is more dangerous (that is, when k = 55), the defenders suffered losses but were not fatal. After the evolution equilibrium is reached, the probability that the attackers continue to select the strategy SA1 is about 0.3. When the attack dangerous level is k = m, the strategy SA1 is lethal to the defenders. At this time, the attackers’ payoffs increase; the probability that the attackers continue to select the strategy SA1 is about 0.9.Figure 6
Impact of the attack dangerous level on attack strategies.Figure7 is the effect of the attack dangerous level on the defense strategy. As shown in Figure 7, regardless of how many times the attackers use the strategy SA1, the defenders actively select the strategy SD1 to respond. However, when k = 55, after about 0.2 h, the probability that the defenders choose the strategy SD1 to deal with is gradually less than 1. This is because the strategy SA1 does less damage to the defenders, and the attackers gradually choose not to adopt the strategy SA1. Accordingly, the defenders also began to show that they did not adopt the strategy SD1.Figure 7
Impact of the attack dangerous level on defense strategies.


### 5.5. Comparison Consequence with Other Literatures

Compared with other kinds of literature, we introduce the concept of the Return on Security Investment (ROSI) to measure the effectiveness of the attack and defense game model. ROSI is an important benchmark to decide the optimal security investment level; researchers have used ROSI to measure the benefits of defenders. According to the Sonnenreich equation [30], we can get ROSI of attack and defense game model. Figure 8 is a comparison of ROSI. As shown in Figure 8, we can draw a conclusion that ROSI of literature [4] and this paper are better and more suitable for the real network attack and defense environment.Figure 8
ROSI comparison.In addition, we also made a comprehensive comparison with some typical research results; as shown in Table6, we can see that the traditional game model constructed in [1] is dynamic but not as good as the evolutionary game. The literature [4] adopts the evolutionary game. It has good versatility, but it is difficult to accurately describe the evolution process of attack and defense because the model does not consider stochasticity. The literature [6] adopts dynamic detection game, which improves the APT (Advanced Persistent Threats) detection performance in the dynamic games and has better data protection ability, but it does not consider the influence of stochasticity on strategy and its application field is data protection. The literature [12] regards the offensive and defensive evolution game as the random jump process of multistate, but the condition of complete information is challenging to meet in the actual network attack and defense. The literature [9] considers stochasticity, but the model has a small scope of application and its versatility in general. In this paper, the stochasticity of the model is considered based on the condition of incomplete information, and the model is constructed by using stochastic differential equations, which improves the effectiveness of the model.Table 6
Comparison consequence with other literatures.
LiteratureGame typeBehavioral informationModel versatilityModel accuracyConcrete application[1]Dynamic gameIncomplete informationGeneralGeneralStrategy selection[6]Dynamic gameIncomplete informationGoodGoodData protection[4]Evolutionary gameIncomplete informationGoodGeneralSecurity defense[12]Stochastic gameComplete informationGoodGeneralSecurity defense[9]Stochastic evolutionary gameIncomplete informationGeneralGoodStrategy selectionThis paperStochastic differential gameIncomplete informationGoodGoodStrategy selection


## 5.1. Experimental Environment

We deploy a network topology environment to simulate the network attack and defense evolution game model proposed in this paper. The validity of the model is proved by analyzing the evolutionary stability strategy.As shown in Figure1, in the network topology environment, attack host A is located on the external network and it is used to simulate a variety of attack strategies of attackers. The intranet contains three servers, namely, MySQL Server B, Web Server C, and FTP Server D. The internal network is isolated from the external network by the firewall.Figure 1
Network topology environment.Since the firewall separates the internal network from the external network, the external host can only access Web Server C and FTP Server D through the network. In the intranet, MySQL Server B, Web Server C, and FTP Server D can access each other by using user rights. The Nessus vulnerability scanner is used to perform vulnerability scanning on three server nodes in the network. The server node information is shown in Table3.Table 3
Server node information.
Host/IPOSServerVulnerability IDB 172.16.3.2LinuxMySQLCVE-2018-10757C 172.16.3.3LinuxsshCVE-2016-10012D 172.16.3.4LinuxftpCVE-2016-9499Through the analysis of the vulnerability and attack behavior of each host node in the network, combined with the China National Vulnerability Database of Information Security (CNNVD), the network attack and defense strategies are designed in the experiment, as shown in Tables4 and 5. Assuming that the network attack strategy is SA1 and SA2, the strategy SA1 has a high cost, high attack effectiveness, and strong pertinence. Strategy SA2 has low cost and low attack effectiveness, which can be considered as not attacking. In addition, assuming that the network defense strategy is SD1 and SD2 when defending against external attacks, the defenders can increase the cost to take defensive investments or can rely on the existing defense ability to passively defend.Table 4
Atomic attack information.
ID/nameNetwork attack strategySA1SA21 Remote buffer overflow2 Buffer error√3 Install Web Listener program√4 Install delete Trojan√5 Trying to steal account√6 FTP server information disclosure√7 Homepage attack√8 Check Point ZoneAlarm9 LPC to LSASS process√10 SQL injection vulnerability√Table 5
Defense strategy information.
ID/nameNetwork defense strategySD1SD21 Install MySQL patches√2 Uninstall delete Trojan√3 Install sshd patches√4 Limit packets from ports5 Delete suspicious account√6 Restart database server√√7 Install ftp patches√8 Repair database9 Close homepage√10 Add physical recourse√


## 5.2. Explicit Euler Numerical Results

For equation (10), we use the explicit Euler numerical method to simulate it [27]. N is the number of iterations, T is the game time, and the average step size is H=T/N.LetN (0, 1) denote the standard normal distribution and divide t∈0,T into N∈N+ equal parts; that is, interval 0,T is divided into 0=t0<t1<⋯<tN−1<tN=T, the average step size is H=T/N, and the node is tn=nH.The Wiener increment isΔwitn=witn−witn−1∼HN0,1, i = (1, 2), n = (1, 2, ..., N), assuming xn=xtn, yn=ytn, Δwitn=witn, and the explicit Euler iteration formula is(25)xn+1=xn+xn1−xn−1V0−Vadd+R1−V0+R0+λLP0−P1yn+τxn1−xnyn⋅Δw1n,yn+1=yn+yn1−ynP0E−CA0−G0+P1E−CA1−G1−P0E−CA0−G0xn+τyn1−ynxn⋅Δw2n.


## 5.3. Attack-Defense Simulation and Analysis

Stochastic evolutionary game is a kind of stochastic theory which combines game theory analysis with dynamic evolutionary process analysis. In the following, according to the problem situation ofx and y, multiple simulation experiments are carried out on the constructed network environment. From the obtained simulation results, the dynamic evolution law of attackers x and defenders y can be analyzed intuitively; the prediction of attack and defense strategies can be realized. And the evolutionary stability strategy is found, that is, the optimal defense strategy in this state. In the simulation experiment, it is assumed that τ=0 indicates the evolution of the attack and defense strategy without considering the stochastic disturbance factor. It is too ideal and the stochastic disturbance in the actual attack and defense is not solved. τ=1 indicates that the game evolution after considering the stochastic disturbance factor is more realistic and more effective.The problem situation isx = 0.4, y = 0.7; that is, the attackers in the group select the hybrid strategy SA1,SA2 with the probability of {0.4, 0.6}, and the defenders in the group select the hybrid strategy SD1,SD2 with the probability of {0.7, 0.3}. It can be seen from Figure 2 that, after continuous evolution, the probability of the attackers selecting the strategy SA1 gradually tends to 0 and the probability that the defenders select the strategy SD1 gradually tends to 1. Both of them reach an evolutionarily stable state. The optimal defense strategy at this time is SD1. Therefore, in this situation, the defenders belong to a more active state of defense. The defense groups are willing to adopt defensive investment strategies for its vulnerability, and it is gradually increasing. The attacker groups gradually turn to the passive state of not taking the attack. The network environment is safer.Figure 2
Group evolution trend whenx = 0.4 and y = 0.7.Figure3 shows the experimental results obtained when the problem situation is x = 0.5 and y = 0.6. The situation indicates that the attackers in the group select the hybrid strategy SA1,SA2 with the probability of {0.5, 0.5} and the defenders select the hybrid strategy SD1,SD2 with the probability of {0.6, 0.4}. As shown in Figure 3, after continuous evolution, the probability that the attackers finally select the attack strategy SA1 gradually tends to 1 and the probability that the defenders select the defense strategy SD1 gradually tends to 1. Both of them reach an evolutionarily stable state, and the optimal defense strategy at this time is SD1. Analysis of the situation at this moment shows that the attackers and the defenders are actively adopting strategies to participate in the game; the network environment is in a relatively fierce state.Figure 3
Group evolution trend whenx = 0.5 and y = 0.6.The problem situation isx = 0.4 and y = 0.3; that is, the attackers in the group select the hybrid strategy SA1,SA2 with the probability of {0.4, 0.6}, and the defenders select the hybrid strategy SD1,SD2 with the probability of {0.3, 0.7}. After continuous evolution, the probability that the attackers finally select the attack strategy SA1 gradually tends to 0 and the probability that the defenders select the defense strategy SD1 gradually approaches 0. Both of them reach an evolutionarily stable state. The optimal defense strategy at this moment is SD2. Figure 4 is a figure of experimental results in the situation of this problem. Analysis of the situation currently shows that although the network environment is relatively stable, the state of both offense and defense is relatively negative.Figure 4
Group evolution trend whenx = 0.4 and y = 0.3.The problem situation isx = 0.7 and y = 0.2; that is, the attackers select the hybrid strategy SA1,SA2 with the probability of {0.7, 0.3}, and the defenders select the hybrid strategy SD1,SD2 with the probability of {0.2, 0.8}. The experimental results obtained in this situation are shown in Figure 5. It can be observed from Figure 5 that, after continuous evolution, the probability that the attackers finally select the attack strategy SA1 gradually tends to 1 and the probability that the defenders select the defense strategy SD1 gradually approaches 0. Both of them reach an evolutionarily stable state. At this moment, the optimal defense strategy is SD2. In summary, the analysis of the situation at this time shows that the defenders choose defensive investment strategies with a small probability. It is more passive in the offensive and defensive confrontation, and the attackers gradually adopt effective attack strategies; the overall network environment is paralyzed.Figure 5
Group evolution trend whenx = 0.7 and y = 0.2.


## 5.4. The Attack Dangerous Level Analysis

Figure6 shows the effect of the attack dangerous level on attack strategies. As we can see from Figure 6, when the attack strategy SA1 is not dangerous enough to hurt the defenders (that is, when k = 0 or 1), after the evolution equilibrium is reached, the probability that the attackers continue to select the strategy SA1 is about 0. That is to say, the attackers tend not to adopt the strategy SA1. When the attack strategy SA1 is more dangerous (that is, when k = 55), the defenders suffered losses but were not fatal. After the evolution equilibrium is reached, the probability that the attackers continue to select the strategy SA1 is about 0.3. When the attack dangerous level is k = m, the strategy SA1 is lethal to the defenders. At this time, the attackers’ payoffs increase; the probability that the attackers continue to select the strategy SA1 is about 0.9.Figure 6
Impact of the attack dangerous level on attack strategies.Figure7 is the effect of the attack dangerous level on the defense strategy. As shown in Figure 7, regardless of how many times the attackers use the strategy SA1, the defenders actively select the strategy SD1 to respond. However, when k = 55, after about 0.2 h, the probability that the defenders choose the strategy SD1 to deal with is gradually less than 1. This is because the strategy SA1 does less damage to the defenders, and the attackers gradually choose not to adopt the strategy SA1. Accordingly, the defenders also began to show that they did not adopt the strategy SD1.Figure 7
Impact of the attack dangerous level on defense strategies.


## 5.5. Comparison Consequence with Other Literatures

Compared with other kinds of literature, we introduce the concept of the Return on Security Investment (ROSI) to measure the effectiveness of the attack and defense game model. ROSI is an important benchmark to decide the optimal security investment level; researchers have used ROSI to measure the benefits of defenders. According to the Sonnenreich equation [30], we can get ROSI of attack and defense game model. Figure 8 is a comparison of ROSI. As shown in Figure 8, we can draw a conclusion that ROSI of literature [4] and this paper are better and more suitable for the real network attack and defense environment.Figure 8
ROSI comparison.In addition, we also made a comprehensive comparison with some typical research results; as shown in Table6, we can see that the traditional game model constructed in [1] is dynamic but not as good as the evolutionary game. The literature [4] adopts the evolutionary game. It has good versatility, but it is difficult to accurately describe the evolution process of attack and defense because the model does not consider stochasticity. The literature [6] adopts dynamic detection game, which improves the APT (Advanced Persistent Threats) detection performance in the dynamic games and has better data protection ability, but it does not consider the influence of stochasticity on strategy and its application field is data protection. The literature [12] regards the offensive and defensive evolution game as the random jump process of multistate, but the condition of complete information is challenging to meet in the actual network attack and defense. The literature [9] considers stochasticity, but the model has a small scope of application and its versatility in general. In this paper, the stochasticity of the model is considered based on the condition of incomplete information, and the model is constructed by using stochastic differential equations, which improves the effectiveness of the model.Table 6
Comparison consequence with other literatures.
LiteratureGame typeBehavioral informationModel versatilityModel accuracyConcrete application[1]Dynamic gameIncomplete informationGeneralGeneralStrategy selection[6]Dynamic gameIncomplete informationGoodGoodData protection[4]Evolutionary gameIncomplete informationGoodGeneralSecurity defense[12]Stochastic gameComplete informationGoodGeneralSecurity defense[9]Stochastic evolutionary gameIncomplete informationGeneralGoodStrategy selectionThis paperStochastic differential gameIncomplete informationGoodGoodStrategy selection


## 6. Conclusion

Nowadays, the analysis method based on the traditional dynamic game cannot meet the actual demand. In this paper, we construct a stochastic differential game model in network attack and defense by using stochastic differential equations based on Markov property. In different problem situations, the attackers and defenders will eventually tend to a stable state via continuous evolution. Compared with the strategy model without considering stochastic factors, it is proved that the model proposed in this paper is more suitable for the actual network attack and defense.By comparison, we can intuitively find that the theoretical analysis is consistent with the conclusions obtained by the simulation experiment, which proves the significance of the attack and defense evolutionary game model proposed in this paper. Compared with other related kinds of literature, we can conclude that the return on security investment of this model is better. Applying the model to the actual network environment can provide the choice of the defenders’ optimal defense strategy and have a certain positive effect on the maintenance of cybersecurity.



---

*Source: 3417039-2020-06-08.xml*