# Game Theoretical Method for Anomaly-Based Intrusion Detection

Authors: Zhiyong Wang; Shengwei Xu; Guoai Xu; Yongfeng Yin; Miao Zhang; Dawei Sun
Journal: Security and Communication Networks
 (2020)

---

## Abstract

In this paper, the game theoretical analysis method is presented to provide optimal strategies for anomaly-based intrusion detection systems (A-IDS). A two-stage game model is established to represent the interactions between the attackers and defenders. In the first stage, the players decide to do actions or keep silence, and in the second stage, attack intensity and detection threshold are considered as two important strategic variables for the attackers and defenders, respectively. The existence, uniqueness, and explicit computation of the Nash equilibrium are analyzed and obtained by considering six different scenarios, from which the optimal detection and attack actions are provided. Numerical examples are provided to validate our theoretical results.


---

## Body

## 1. Introduction

Nowadays, network devices and communication services are vulnerable to various kinds of intrusion attacks, such as DoS/DDoS, false data injection, and botnet attacks. The intrusion attacks tend to be more intelligent and the unexpected attack modes arise frequently. Consequently, great challenges are brought into network security control and management. As one of the most important techniques to tackle with various attacks, anomaly-based intrusion detection system (A-IDS) has been widely adopted in almost all kinds of network environments [1, 2]. An anomaly-based intrusion detector attempts to estimate the normal behavior with a profile and generates an anomaly alarm once the profile collected from real-time observation exceeds a predefined threshold [2].In an intrusion detection system, the attacker and defender can naturally be regarded as two players who try to maximize their payoffs, respectively, by executing certain optimal strategies. Thus, the game theoretical method is an effective tool which enables a defender to earn the maximum payoff (or the minimum loss) while fighting with the attacks. A number of results on game theory-based intrusion detection methods have been reported for different network environments and security requirements. Excellent surveys about this topic can be found in [3–6]. In [7], two-player noncooperative strategic game models are established for some general intrusion detection problems and Nash equilibriums are analyzed explicitly. In [8–11], game theoretical intrusion detection methods are investigated to solve the security resource allocation problems of large-scale heterogeneous networks. Note that, in [8–10], it is assumed that the defender scan always correctly identify the malicious behaviors of the attackers without any errors, while such an assumption may not be satisfied in some cases. For example, for intelligent APT attacks, the attackers often disguise themselves as no attack happens, which may make the detector to not always preciously identify the malicious actions. To handle these uncertainties, Bayesian games are considered in intrusion detection by updating the defender’s belief to her/his opponent based on the past behaviors [12–15]. The main idea of Bayesian game-based intrusion detection is to use probability to represent the uncertainties and further use Bayesian iteration to update the dynamics. For self-organizing ad hoc networks, some nodes may be malicious and how to detect the malicious actions is an important work. Some strategic games are presented to stimulate the cooperation among distinct regular nodes, based on which the hidden malicious nodes can be detected [16–21]. In [22], a two-player Stackelberg stochastic game is analyzed for achieving the best response against the intrusion. In [23, 24], game theory-based analysis methods for distributed intrusion detection are proposed, where consensus-based distributed detection method is presented and then game analysis is provided for the optimal defense and attack strategies. In [25], the privacy defense problem is also considered in the collaborative security scheme design problem by using the game theoretical analysis method. In [26], a differential game model is established to analyze the dynamic process of the attack and defense.In a game between an attacker and a defender, the rational attacker will not launch an attack otherwise she/he can get a positive payoff. Moreover, the attack intensity needs be chosen to maximize her/his positive payoff. On the contrary, the defender will perform a defense action to resist the attack according to a similar rule. In an A-IDS, a predefined detection threshold needs be cautiously determined. In general, a higher threshold with a larger normal coverage area will result in a smaller false alarm rate but a larger missing alarm rate. Note that the missing alarm rate is also closely related to the attack intensity. More specifically, larger attack intensity will cause a lower missing alarm rate. Though attack intensity and detection threshold are two important factors affecting the false and missing alarm rates, which correspond to the payoffs of attackers and defenders in an intrusion detection game, they are seldom considered in the aforementioned results. In most of the aforementioned works, the false and missing alarm rates are assumed to be known constants and only binary actions “do” or “not do” are considered in their game models. In [11], the detection threshold and attack intensity are considered, while the focus is mainly related to distributed resource allocation of the heterogeneous networks.Motivated by the limitations mentioned above in the literature, a more realistic two-stage form of the intrusion detection game model is presented in this paper. The attack intensity and detection threshold are considered as two strategic variables. In the first stage, the attackers and defenders make decisions on whether the attack and defense actions should be executed, respectively. Once the attack/monitoring actions are decided to be executed, optimal attack intensity and detection threshold are determined to maximize their utilities in the second stage. The existence and uniqueness of the Nash equilibrium are discussed for the first stage of our presented game model under different scenarios, when the strategic variables of the second stage are restricted to certain regions. Then, the optimal attack intensity and detection threshold are derived for each scenario, correspondingly.The contributions of this paper can be summarized as follows:(1)
A two-stage game model is presented for anomaly-based intrusion detection confrontation. In contrast to the existing work, where only binary actions “do” or “not do” are considered in the game model, the attack intensity and the detection threshold are considered as two key strategic variables, and the false and missing alarm rates are the functions of the attack intensity and the detection threshold, instead of being assumed to be constant. The two stages of the game model are tightly coupled with each other and thus the game model is more complex.(2)
The existence, uniqueness, and calculation of Nash equilibriums are discussed. Based on the results, optimal selections of attack intensity and detection threshold for achieving the maximum payoffs of the attackers and defenders are provided. The results provide a new method to determine the detection threshold in the defense, from the perspectives of the optimization and confrontation. So, the presented game model and Nash equilibrium solution give a more realistic theoretical analysis framework for the anomaly-based security detection.The rest of this paper is organized as follows. In Section2, some definitions are introduced and a two-stage game model of the A-IDS is presented. In Section 3, the Nash equilibrium of the proposed game model is analyzed. Simulation results are given to show the effectiveness of our game theoretical analysis methods in Section 4, followed by the conclusions of the paper summarized in Section 5.


## 2. A Two-Stage Intrusion Detection Game Model

Suppose that there is a network unit vulnerable to intrusion attacks. Typical examples for such a unit include a software system, network equipment, and a communication channel. Here, we adopt similar attack and A-IDS detection models as that in [11]. The strategic form of two-player noncooperative game is given in Table 1. UA and UD denote the payoffs of the attacker and the defender, respectively.Table 1
Strategic form of the local game.
MonitorNot monitorAttackUAx,y=qx,ysxW−c1W−uxW,UDx,y=−qx,ysxW−c2WUAx,y=sxW−c1W−uxW,UDx,y=−sxWNot attackUAx,y=0,UDx,y=−pyc3W−c2WUAx,y=0,UDx,y=0In the following, we give the physical meanings of the corresponding variables in Table1. The variable x denotes the attack intensity, for example, the number of attack packets in a DoS/DDoS attack, or the number of bogus packets in a DNS cache poisoning attack or jamming strength in a communication attack, or the magnitude of false data injection. It is assumed that x∈x¯,x¯, where 1≥x¯>x¯>0. The function sx∈ℜ is used to represent the extent of damage to the security of the unit, when it is suffered from an attack with intensity x. It is natural to consider sx as a strictly increasing function such that ∂sx/∂x>0 and sx∈s¯,s¯ with 1≥s¯>s¯>0. The term c1W+uxW, where c1∈0,1 is a constant, W is the security asset of the unit, and ux is a strictly increasing function, denotes the cost of launching the attack. The variable y denotes the detection threshold. It is assumed that y∈y¯,y¯ with y¯>y¯>0 and a larger y corresponds to a larger coverage area for normal behavior. The function p denotes the false alarm rate, i.e., it represents the probability that an alarm is generated though no attack is activated. Obviously, p is determined completely by the threshold y and py is a strictly decreasing function in this paper. The function q denotes the missing alarm rate, i.e., it represents the probability that no alarm is generated though an attack is executed. The function q is determined by both attack intensity x and threshold y. It can be easily derived that q is strictly decreasing and increasing with respect to x and y, respectively. The parameters c2∈0,1 and c3∈0,1 are two constants.Clearly, the game model described in Table1 contains the following two stages. In the first stage, the optimal strategy set “Attack/Not attack” and “Monitor/Not monitor” needs be determined by the attacker and defender. Then, both players proceed to the next stage to select optimal attack intensity x and detection threshold y. For better understanding, the two-stage pure-strategic intrusion detection game model with one attacker and one defender is described in Table 2 in a more rigorous way.Table 2
Two-stage pure-strategic intrusion detection game.
PlayersAttacker, defenderStrategy setsAttacker: Attack, not attack, attack intensityxDefender: Monitor, not monitor, detection thresholdyConstraintsx∈x¯,x¯, x¯>x¯>0, y∈y¯,y¯, y¯>y¯>0, sx∈s¯,s¯, 1≥s¯>s¯>0,∂sx/∂x>0, ∂ux/∂x>0, ∂py/∂y<0, ∂qx,y/∂x<0, ∂qx,y/∂y>0,p∈p¯,p¯, q∈q¯,q¯, 1>p¯>p¯>0, 1>q¯>q¯>0, c1,c2,c3∈0,1PayoffsUA, UD (see Table 1)Game targetThe players choose their strategies to maximize their payoffsUA, UDRemark 1.
The attack and detection models are similar to that in [11], while the results of [11] mainly consider the attack and defense resource allocation problem for heterogeneous distributed networks. In this paper, we consider the confrontation problem for one network unit, as expressed by the game model in Table 2. Thus, it is essentially different from the work in [11]. Besides, we establish a two-stage game model by considering the attack intensity and detection threshold as the key strategic variables, which is also different from the existing works.


## 3. Nash Equilibrium Analysis of the Game

As mentioned in Section2, the attacker/defender needs to decide whether to launch an attack/to monitor the unit or keep silence in the first stage of the presented game model. For simplicity, an extra assumption is imposed that if the payoffs of a player choosing to perform the action and to keep silence are the same, she/he will keep silence. In other words, the attacker/defender tends to do nothing if she/he cannot earn larger payoffs by launching an attack/monitoring. Note that the value of W has no impact on the analysis of Nash equilibrium (hereinafter referred to as NE) of the game from Table 1. Thus, without loss of generality, we set W=1.Denote the feasible set ofx and y by π with π∈x¯,x¯×y¯,y¯. For convenience in later analysis, π is divided into the following subsets:(1)π1=x,y∈π:sx−c1−ux≤0,π2=x,y∈π:sx−c1−ux≥0,−qx,ysx−c2≤−sx,π3=x,y∈π:qx,ysx−c1−ux≥0,−qx,ysx−c2≥−sx,π4=x,y∈π:qx,ysx−c1−ux<0,sx−c1−ux>0,−qx,ysx−c2>−sx.It can be readily shown thatπ1∪π2∪π3∪π4=π and π1∪π2∪π3∩π4=∅. The results of NE for the game as described in Tables 1 and 2 will be obtained from the following scenarios. In Scenario L.1, only one subset of π1, π2, π3, and π4 is nonempty. In Scenarios L.2∼L.5, π4 is empty while at least two subsets of π1, π2, and π3 are nonempty. In Scenario L.6, π4 and at least one subset of π1, π2, and π3 are nonempty. Clearly, there is no overlap between any two scenarios and the six scenarios include all the possibilities. In the following, the sufficient and necessary conditions on x and y for the existence and uniqueness of NE are first derived for Scenarios L.1∼L.6, respectively. Then, the optimal values of x and y, denoted by x∗ and y∗, are further provided.For convenient expression in what follows, two variablesx′ and x″ are first defined, i.e.,(2)x′=argxmaxqx,y¯sx−ux−c1,s.t.x∈x¯,x¯,(3)x″=argxmaxsx−c1−ux,s.t.x∈x¯,x¯.The optimization problems presented by (2) and (3) can be solved by classical optimization methods such as the gradient method and Lagrangian multiplier method [27].Scenario L.1.
Only one of the subsetsπ1, π2, π3, and π4 is nonempty.
The following conclusions can be drawn.Theorem 1.
In Scenario L.1, the NE of the game, as described in Table1, is derived as follows:(1)
If only the subsetπ1≠∅, “not attack, not monitor” is the unique NE(2)
If only the subsetπ2≠∅, “attack, not monitor” is the unique NE and x∗=x″(3)
If only the subsetπ3≠∅, “attack, monitor” is the unique NE and x∗=x′, y∗=y¯(4)
If only the subsetπ4≠∅, no NE existsProof.
Firstly, the strategy combination “attack, not monitor” will not be the NE. This is because,−pyc3−c2<0, the defender tends to “not monitor” the unit to earn zero payoff:(1)
If onlyπ1≠∅, we have qx,ysx−c1−ux<sx−c1−ux≤0. This indicates that the attacker has no incentive to launch an attack either. Therefore, “not attack, not monitor” is the unique NE.(2)
If onlyπ2≠∅, as the payoff of the attacker sx−c1−ux is positive for any attack intensity x, the attacker will select “attack.” Besides, the defender will never get more payoffs when she/he selects “monitor” as −qx,ysx−c2≤−sx for an arbitrary threshold y. Thus, the defender will select “monitor.” The optimal attack intensity x∗ should be derived by maximizing the payoff of the attack; therefore, x∗=x″ based on (3).(3)
If onlyπ3≠∅, the attacker will always select “attack.” This is because for any attack intensity x and detection threshold y the payoff of the attacker satisfies sx−c1−ux>qx,ysx−c1−ux≥0. Since the payoff of the defender satisfies −qx,ysx−c2≥−sx for an arbitrary y, the defender will select “monitor.” Then, for the defender, the optimal threshold is computed by(4)y∗=argymax−qx,ysx−c2,s.t.x∈x¯,x¯,y∈y¯,y¯.Based on the property that∂qx,y/∂y>0 in Table 2, we have y∗=y¯. Then, the optimal attack intensity is given by x∗=x′ based on (2).(4)
If onlyπ4≠∅, “attack, monitor” cannot be the NE since qx,ysx−c1−ux<0. Meanwhile, “attack, not monitor” is not the NE because sx−qx,ysx>c2 indicates that the defender will selects “monitor”.Moreover,since sx−c1−ux>0, “not attack, not monitor” cannot be the NE, either. Combining with the result derived in the beginning that “not attack, monitor” cannot be the NE, it is concluded that no NE exists.Remark 2.
From Theorem1, the payoffs of the two players are, respectively, expressed as UA=sx∗−c1−ux∗ and UD=−sx∗ in (2) in Scenario L.1. It implies that the attacker obtains positive payoff while the defender loses certain security asset in this scenario. On the contrary, the payoffs of two players are, respectively, expressed as UA=qx∗,y¯sx∗−ux∗−c1 and UD=−qx∗,y¯sx∗−c2 in (3) in Scenario L.1. Similar to (2) in Scenario L.1, the attacker earns positive payoff while the defender loses certain security asset. Nevertheless, different from (2) in Scenario L.1, the defender compensates for part of the loss by executing monitoring action in this scenario as q<1. Thus, the payoff earned by the attacker decreases.
As discussed previously, Scenarios L.2∼L.5 cover the possibilities thatπ4 is empty while at least two subsets of π1, π2, and π3 are nonempty. Details are given as below.Scenario L.2.
π1≠∅, π2≠∅, and π3=π4=∅.
The following results about the NE for this scenario can be shown.Theorem 2.
In Scenario L.2, the strategy combination “attack, not monitor” is the unique NE andx∗=x″.Proof.
The subsetπ2≠∅ indicates that there exists an x such that the payoff of the attacker sx−ux−c1 is positive. Thus, the attacker will select the strategy “attack.” Besides, the payoff of the defender satisfies −qx,ysx−c2≤−sx for any threshold y, so the defender will select “not monitor.” Besides, the optimal attack intensity is given by x∗=x″.Scenario L.3.
π1≠∅, π3≠∅, and π2=π4=∅.
Main results for this scenario are formally stated in the following theorem.Theorem 3.
In Scenario L.3, the strategy combination “attack, monitor” is the unique NE if and only ifqx′,y¯sx′−c1−ux′>0. The optimal attack intensity and detection threshold are x∗=x′ and y∗=y¯.Proof.
Necessity: if “attack, monitor” is the unique NE, then from (2) and (4), there are x∗=x′ and y∗=y¯. The payoff of the attacker with x∗ and y∗ must be positive; thus, qx′,y¯sx′−c1−ux′>0.
Sufficiency: sinceqx′,y¯sx′−c1−ux′>0, the attacker can earn a positive maximum payoff if the defender selects the strategy “monitor” and y∗=y¯. Thus, the attacker will select to “attack” and y∗=y¯. As q<1 and ∂qx,y/∂y>0, there is sx′−c1−ux′>qx′,ysx′−c1−ux′≥qx′,y¯sx′−c1−ux′>0 for y∈y¯,y¯. It follows that x′∉π1 and x′,y∈π3 for y∈y¯,y¯. From the definition of π3, it can be concluded that −qx′,ysx′−c2≥−sx′ for y∈y¯,y¯. This indicates that no matter how the threshold is selected, the defender will earn larger payoff when she/he selects the strategy “monitor” rather than “not monitor.” Clearly, the defender will select “monitor” and the optimal threshold is set as y∗=y¯ from (4). Therefore, the strategy combination “attack, monitor” is the unique NE and x∗=x′ and y∗=y¯.Scenario L.4.
π2≠∅, π3≠∅, and π1=π4=∅.
The following conclusions can be drawn for this scenario.Theorem 4.
In Scenario L.4,(1)
If and only if−qx″,y¯sx″−c2≤−sx″, “attack, not monitor” is the NE and x∗=x″(2)
If and only if−qx′,y¯sx′−c2>−sx′, “attack, monitor” is the NE and x∗=x′ and y∗=y¯Proof.
(1)
Necessity: under the strategy combination “attack, not monitor”, the attacker will selectx″ as the optimal attack intensity. If −qx″,y¯sx″−c2>−sx″, the defender will select “monitor” to earn larger payoffs, which is a contradiction to the premise that “attack, not monitor” is the NE. Thus, the necessity is shown.Sufficiency: from the definitions ofπ2 and π3, the attacker can always earn positive maximum payoff when s/he selects “attack.” As ∂q/∂y>0, there is(5)−qx″,ysx″−c2≤−qx″,y¯sx″−c2≤−sx″.This means when the attacker selectsx∗=x″, the defender never earn larger payoffs than she/he does nothing no matter how the threshold is set. Thus, “attack, not monitor” is the NE and x∗=x″. The sufficiency is shown.(2)
Necessity: under the strategy combination “attack, monitor,” the defender and attacker will selecty¯ and x′ as the optimal detection threshold and attack intensity from (4) and (2). If −qx″,y¯sx″−c2≤−sx″, then similar to (5), there is(6)−qx′,ysx′−c2≤−qx′,y¯sx′−c2≤−sx′.
This means the defender never earns larger payoffs than she/he does nothing, which is a contradiction to the premise that “attack, monitor” is the NE. Thus, the necessity is shown.
Sufficiency: the attacker always selects “attack” from the definitions ofπ2 and π3. If the attacker selects x∗=x′, since −qx′,y¯sx′−c2>−sx′, the defender will select “monitor” to obtain larger payoffs than “not monitor” and the optimal detection threshold is y¯ from (4). Meanwhile, when the defender selects “monitor” and y∗=y¯, from (2), the attack will select “attack” and x∗=x′ to earn the maximum positive payoff. Thus, the sufficiency is shown.
Based on Theorem4, the uniqueness of the NE for Scenario L.4 can also be concluded.Corollary 1.
In Scenario L.4,(1)
If and only if−qx″,y¯sx″−c2≤−sx″ and −qx′,y¯sx′−c2≤−sx′, “attack, not monitor” is the unique NE and x∗=x″(2)
If and only if−qx″,y¯sx″−c2>−sx″ and −qx′,y¯sx′−c2>−sx′, “attack, monitor” is the unique NE and x∗=x′ and y∗=y¯Proof.
From Theorem4, “attack, not monitor” and “attack, monitor” are the only two possible NEs. Clearly, “attack, not monitor” is the unique NE if an extra condition holds, i.e., −qx″,y¯sx″−c2≤−sx″. Then, “attack, monitor” will not be the NE. Similarly, “attack, monitor” is the unique NE if the extra condition −qx′,y¯sx′−c2>−sx′ holds. Then, “attack, not monitor” is not the NE. Therefore, Corollary 1 can be concluded.Scenario L.5.
π1≠∅, π2≠∅, π3≠∅, and π4=∅.
Different from Scenario L.4, there existsx∈x¯,x¯ belonging to π1 such that sx−c1−ux≤0. Since the attacker can always find an x such that she/he earns a positive payoff, the strategy combination “not attack, not monitor” cannot be the NE in this scenario. The main results about the NE in this scenario can be formally stated in the following theorem.Theorem 5.
In Scenario L.5,(1)
If and only if−qx″,y¯sx″−c2≤−sx″, “attack, not monitor” is the NE and x∗=x″(2)
If and only ifqx′,y¯sx′−c1−ux′>0 and −qx′,y¯sx′−c2>−sx′, “attack, monitor” is the NE and x∗=x′ and y∗=y¯(3)
If and only if−qx″,y¯sx″−c2≤−sx″ and qx′,y¯sx′−c1−ux′≤0 or −qx′,y¯sx′−c2≤−sx′, “attack, not monitor” is the unique NE and x∗=x″(4)
If and only ifqx′,y¯sx′−c1−ux′>0, −qx′,y¯sx′−c2>−sx′, and −qx″,y¯sx″−c2>−sx″, “attack, monitor” is the unique NE and x∗=x′ and y∗=y¯Proof.
(1)
The proof is similar to that of (1) in Theorem4 and is omitted here.(2)
Different from Scenario L.4, there existsx∈x¯,x¯ belonging to π1 such that(7)qx,ysx−c1−ux<sx−c1−ux≤0,as qx,y<1. Thus, compared to (2) in Theorem 4, an extra condition qx′,y¯sx′−c1−ux′>0 needs be added to ensure that “attack, monitor” still be the NE. The remaining proof is similar to that of (2) in Theorem 4 and is omitted here.(3) and (4) By following similar analysis in the proof of Corollary1, the uniqueness of the NE in this case can also be concluded.
In contrast to previous scenarios,π4 and at least one subset of π1, π2, and π3 are nonempty in Scenario L.6 as described below.Scenario L.6.
π1∪π2∪π3≠∅, and π4≠∅.
From (4) in Theorem1, there is no NE if only π4≠∅. Besides, if π4=∅ is replaced by π4≠∅ for (1)–(3) in Scenario L.1 and Scenarios L.2–L.5, the NEs will never belong to π4. This is because all the strategy combinations driven by x and y within π4 are inconsistent with the obtained NE in Theorems 1–5. Hence, x∗,y∗ of the NE for Scenario L.6 will belong to π1, π2, or π3. Moreover, the conditions for the derived NEs in Theorems 1–5 are still necessary. Therefore, to analyze the NE in Scenario L.6, we only need to verify whether the results in Theorems 1–5 are still correct if the subset π4 is changed to be nonempty. The following conclusions will be shown.Theorem 6.
In Scenario L.6, the NE for the game as described in Table1 is derived as follows:(1)
Ifπ1≠∅ and π2=π3=∅, no NE exists(2)
If {π2≠∅, π1=π3=∅} or {π1≠∅, π2≠∅, π3=∅}, the results in (1) in Theorem 4 hold true and “attack, not monitor” is the unique NE(3)
If {π3≠∅, π1=π2=∅} or {π1≠∅, π3≠∅, π2=∅}, the results in Theorem 3 hold true(4)
If {π2≠∅, π3≠∅, π1=∅} or {π1≠∅, π2≠∅, π3≠∅}, the results in Theorem 5 hold trueProof.
(1)
As there exists anx such that the payoff of the attacker sx−c1−ux is positive, “not attack, not monitor” is no longer the NE if π4=∅ is replaced by π4≠∅ for (1) in Scenario L.1, i.e., π1≠∅, π2=π3=∅, and π4≠∅. It can be easily shown that other strategy combinations cannot be the NE either.(2)
Ifπ4=∅ is replaced by π4≠∅ for (2) in Scenario L.1, there exists feasible x and y such that −qx,ysx−c2>−sx. Thus, an extra condition −qx″,y¯sx″−c2≤−sx″ is required with comparison to (2) in Theorem 1 to ensure that “attack, not monitor” still be the NE. If π4=∅ is replaced by π4≠∅ for Scenario L.2, by following similar analysis in the proofs of Theorem 2 and (1) in Theorem 4, we can show that the results in (1) in Theorem 4 are true.(3)
Whenπ3≠∅, π1=π2=∅, and π4≠∅, there exist feasible x and y such that qx,ysx−c1−ux<0. Thus, an extra condition qx′,y¯sx′−c1−ux′>0 is required with comparison to (3) in Theorem 1 to ensure that “attack, monitor” still be the NE. When π1≠∅, π3≠∅, π2=∅, and π4≠∅, based on the proof of Theorem 3 and the definitions of π3 and π4, it can be shown that the results of Theorem 3 are still true.(4)
Firstly, ifπ4 is changed to be nonempty in Scenario L.4, x and y belonging to π4 will have no influence on the results of (1) in Theorem 4. As the results of (1) in Theorem 5 are the same as that of (1) in Theorem 4, (1) in Theorem 5 holds true in this case. Besides, an extra condition qx′,y¯sx′−c1−ux′>0 is required with comparison to (2) in Theorem 4 to ensure “attack, monitor” be the NE since there exist x and y such that qx,ysx−c1−ux<0 from the definition of π4. Thus, the results in (2) in Theorem 5 are true. The uniqueness of the NE can also be verified from (3) and (4) in Theorem 5. Secondly, if all the subsets are nonempty, i.e., π1≠∅, π2≠∅, π3≠∅, and π4≠∅, it can be easily shown that the feasible values of x and y belonging to π4 have no influence on the results of Theorem 5.Remark 3.
It can be seen from (3) in Theorem1, Theorem 3, (2) in Theorem 4, and (2) in Theorem 5 that once the defender decides to monitor in (3) in Scenario L.1, Scenario L.3, (2) in Scenario L.4, (2) in Scenario L.5, and (4) and (5) in Scenario L.6, she/he will always select y¯ as the optimal threshold y∗.Remark 4.
In this paper, we assume that the attackers are completely rational, while this assumption may not be satisfied in some scenarios. However, based on our method, we present an optimal defense strategy for the worst case. That is, we can guarantee that the maximum damage in the worst case can be minimized by our method.


## 4. Simulation Studies

In this section, simulation results are provided to validate the theoretical results as presented above. In A-IDS, a profile is generally selected to cause distinctions between normal and abnormal states. Such a profile is normally described by a random variable in many cases. Here, we assume it follows a Gaussian distribution with zero mean under normal states. Similar assumptions can be seen in many intrusion detection application areas such as network traffic detection and Kalman filtering-based anomaly detection. Let the intensity of the attack be denoted asx. Other parameters in simulation are chosen as x¯=y¯=0.1, x¯=y¯=2, s=0.5x, u=0.1x, and c3=0.2. The false alarm rate and missing alarm rate can be expressed by(8)p=∫y∞e−z2/8dz/22π,q=∫−∞ye−z−x2/8dz/2π,respectively. Parameters c1 and c2 are used to represent the costs of the attacker and the defender, respectively.Case 1.
We first selectc1∈0,0.2 and c2=0.2. Then, it can be calculated by (1) that(a)
Ifc1∈0,0.04, there are π2≠∅, π3≠∅, and π1=π4=∅, which corresponds to Scenario L.4(b)
Ifc1∈0.04,0.08, there are π1≠∅, π2≠∅, π3≠∅, and π4=∅, which corresponds to Scenario L.5(c)
Ifc1∈0.08,0.2, all the four subsets are nonempty, which corresponds to Scenario L.6
Then, it can be checked whether the inequality conditions in Theorems4 and 5 and (4) in Theorem 6 are satisfied for the above three scenarios, as given in Table 3. ‘IC 4.1’, ‘IC 4.2’, ‘IC 5.1’, and ‘IC 5.2’ refer to the inequality conditions in (1) and (2) in Theorem 4 and (1) and (2) Theorem 5, respectively. It is worth noting that the inequality conditions in (4) in Theorem 6 are the same as those in Theorem 5. From the theoretical analysis given in Section 2, the following conclusions on the NEs can be drawn:(a)
Based on (2) in Theorem4, “attack, monitor” is the unique NE if c1∈0,0.04 and c2=0.2.(b)
Based on (2) in Theorem5, “attack, monitor” is the unique NE if c1∈0.04,0.08 and c2=0.2.(c)
Based on (4) in Theorem6 and (2) in Theorem 5, “attack, monitor” is still the unique NE if c1∈0.08,0.2and c2=0.2. However, no NE exists if c1∈0.12,0.2, c2=0.2. This result can be verified by observing the payoff of the attacker (UA) with respect to c1, as shown in Figure 1. UA decreases as c1 increases. Besides, UA will approach zero when c1 tends to 0.12, which indicates that the NE is broken.Table 3
The results showing whether the inequality conditions in Theorems4 and 5 and (4) in Theorem 6 are satisfied when c1∈0,0.2 and c2=0.2.
c1∈0,0.04c1∈0.04,0.08c1∈0.08,0.12c1∈0.12,0.2Scenario L.4Scenario L.5Scenario L.6Scenario L.6IC 4.1×IC 5.1×IC 5.1×IC 5.1×IC 4.2√IC 5.2√IC 5.2√IC 5.2×Figure 1
Payoff of the attackerUA with respect to c1 if c2 is fixed as c2=0.2.Case 2.
In this case, we fixc1 as c1=0.1, while let c2 vary within the interval 0,0.2. It can be calculated that(a)
Ifc2∈0,0.04, there are π1≠∅, π3≠∅, π4≠∅, and π2=∅, which corresponds to Scenario L.6(b)
Ifc2∈0.04,0.2, all the four subsets are nonempty, which also corresponds to Scenario L.6
Similarly, Table4 is given to show whether the inequality conditions in Theorems 3 and 5 are satisfied, where ‘IC 3’ refers to the inequality condition in Theorem 3. Then, the following conclusions on the NEs can be drawn:(a)
Based on Theorem3 and (3) in Theorem 6, “attack, monitor” is the unique NE if c1=0.1 and c2∈0,0.04(b)
Based on (2) in Theorem5 and (4) in Theorem 6, “attack, monitor” is the unique NE if c1=0.1 and c2∈0.04,0.2
Therefore, “attack, monitor” is always the unique NE ifc1=0.1, c2∈0,0.2. Besides, from Theorem 3 and (2) Theorem 5, it can be calculated that the payoff of the attacker (UA) is equal to 0.024 if c2∈0,0.2. It indicates that the attacker has the motivation to launch the attack. The performance of the defender’s payoff (UD) with respect to c2 is shown in Figure 2. Clearly, the defender loses some security asset as UD<0. Moreover, the lost security asset will increase as the defense cost c2 increases.
At last, we make some comparisons with the existing methods in [7–15], where attack intensity and detection threshold are scarcely considered and majority of them assume that the false and missing alarm rates, and the game model of detection problem can be modelled as Table 5.
It can be seen that, without considering the attack intensity and detection threshold, the payoffs of the game model will be reduced to be constant and the Nash equilibrium analysis can be easily done. From the definition of the Nash equilibrium, it can be calculated that ifq+c2>1, (Attack, Monitor) will be the unique NE. Though the existing analysis methods in [7–15] can determine the optimal action strategies, while our results can further determine the optimal explicit attack intensity and detection threshold, different results can be obtained. First, the existing work considers only the strategy do or not do; thus, the one-stage game model, as expressed in Table 3, is established to help analyze the optimal actions, while we further consider the attack intensity and detection threshold in the game model, as these two parameters are two key strategies used for the defender and the attacker. Moreover, we establish a more detailed two-stage game model to consider both the action do or not do and the attack intensity and detection threshold. Based on the experimental results, we can see that the attack intensity and detection threshold play an important role in the determination of the Nash equilibrium. Intuitively, for the game in Table 3, the NE are completely determined by the parameter x and y; however, this conclusion seems not to make sense as the false alarm rate and other parameters have no any effect on the Nash equilibrium. Alternatively, for our game model, we can see that all parameters will jointly determine the Nash equilibriumthus, our analysis results are more realistic. In practical, the false and missing alarm rates are not constant, as the attacks are always dynamically changing. In A-IDS methods, the false and missing alarm rates are commonly determined by the attack intensity and detection threshold. Our method just considers this real scenario and establishes a more explicit game model, based on which the optimal strategies are completely determined.Table 4
The results about the inequality conditions in Theorems3 and 5 with c1=0.1 and c2∈0,0.2.
c2∈0,0.04c2∈0.04,0.2Scenario L.4Scenario L.5IC 3√IC 5.1×IC 5.2√Figure 2
Payoff of the defenderUD with respect to c2 if c1 is fixed as c1=0.1.Table 5
Strategic form of the game in existing work.
MonitorNot monitorAttackUA=qW−c1WUA=W−c1WUD=−qW−c2WUD=−WNot attackUA=0UA=0UD=−pc3W−c2WUD=0


## 5. Conclusion

For anomaly-based intrusion detection system, we present a game theoretical analysis method to provide the optimal strategies. We first establish a more realistic game model by considering the attack intensity and detection threshold as two strategies for the players. The necessary and sufficient conditions, for which strategies are the Nash equilibriums, are presented. Simulation studies are provided to validate our theoretical results. The results provide a new method to determine the detection threshold in the security defense. In the future, some more research work could be considered, for example, the game theoretical analysis method for specific scenarios such as Internet of Things and DoS/DDoS attacks. Besides, dynamic game analysis is also an interesting topic for dynamic security confrontation process, for example, Stackelberg game analysis can be adopted to solve the sequential problem of the attack and defense actions.



---

*Source: 8824163-2020-09-04.xml*