# Defending against the Advanced Persistent Threat: An Optimal Control Approach

Authors: Pengdeng Li; Xiaofan Yang; Qingyu Xiong; Junhao Wen; Yuan Yan Tang
Journal: Security and Communication Networks
 (2018)

---

## Abstract

The new cyberattack pattern of advanced persistent threat (APT) has posed a serious threat to modern society. This paper addresses the APT defense problem, that is, the problem of how to effectively defend against an APT campaign. Based on a novel APT attack-defense model, the effectiveness of an APT defense strategy is quantified. Thereby, the APT defense problem is modeled as an optimal control problem, in which an optimal control stands for a most effective APT defense strategy. The existence of an optimal control is proved, and an optimality system is derived. Consequently, an optimal control can be figured out by solving the optimality system. Some examples of the optimal control are given. Finally, the influence of some factors on the effectiveness of an optimal control is examined through computer experiments. These findings help organizations to work out policies of defending against APTs.


---

## Body

## 1. Introduction

Nowadays, the daily operation of most organizations, ranging from large enterprises and financial institutions to government sectors and military branches, depends largely on computers and networks. However, this dependency renders the organizations vulnerable to a wide range of cyberattacks. Traditional cyberattacks include computer viruses, worms, and spyware. Conventional cyber defense measures including firewall and intrusion detection turn out to be effective in withstanding these cyberattacks [1, 2].The cybersecurity landscape has changed drastically over the past few years. A new type of cyberattack—advanced persistent threat (APT)—has posed an unprecedentedly serious threat to modern society. According to report, many high-profile organizations have experienced APTs [3], and the number of APTs has been increasing rapidly [4]. Compared with traditional cyberattacks, APTs exhibit two distinctive characteristics: (a) The attacker of an APT is a well-resourced and well-organized group, with the goal of stealing as many sensitive data as possible from a specific organization. (b) Based on meticulous reconnaissance, the attacker is going to launch a preliminary advanced social engineering attack on a few target users to gain footholds in the organization and then to gain access to critical information stealthily and slowly [5–7]. Due to these characteristics, APTs can evade traditional detection, causing tremendous damage to organizations. To date, the detection of APTs is far from mature [8, 9]. Consequently, the APT defense problem, that is, the problem of how to effectively defend against APTs, has become a major concern in the field of cybersecurity.As a branch of applied mathematics, optimal control theory aims to solve a class of optimization problems in which, subject to a set of dynamic constraints, we seek to find a function (control) so that an objective functional is optimized [10, 11]. In real world applications, the set of dynamic constraints represents a dynamic environment, a control represents a time-varying strategy, and the objective functional represents an index to be maximized or minimized. Optimal control theory has been successfully applied to some aspects of cybersecurity [12–19]. To our knowledge, the APT defense problem has yet to be addressed in the framework of optimal control theory. To model the problem as an optimal control problem, we have to formulate an APT defense strategy as a control, characterize the state evolution of an organization as a set of dynamic constraints, and quantify the effectiveness of an APT defense strategy as an objective functional. The key to the modeling process is to accurately characterize the state evolution of an organization by employing the epidemic modeling technique [20].Individual-level epidemic models refer to epidemic models in which the state evolution of each individual in a population is characterized by one or a few separate differential equations. As compared with the coarse-fined state-level epidemic models [21–26] and the intermediate degree-level epidemic models [27–33], the finest individual-level epidemic models can characterize spreading processes more accurately, because they can perfectly accommodate the network topology. The individual-level epidemic modeling technique has been successfully applied to areas such as the epidemic spreading [34–37], the malware spreading [38–43], and the rumor spreading [44]. In particular, a number of APT attack-defense models have recently been proposed by employing this technique [45–48].This paper focuses on the APT defense problem. Based on a novel individual-level APT attack-defense model, the effectiveness of an APT defense strategy is quantified. On this basis, the APT defense problem is modeled as an optimal control problem, in which an optimal control represents a most effective APT defense strategy. The existence of an optimal control to the optimal control problem is proved, and an optimality system for the optimal control problem is derived. Therefore, an optimal control can be figured out by solving the optimality system. Some examples of the optimal control are presented. Finally, the influence of some factors on the effectiveness of an optimal control is examined through computer simulations. To our knowledge, this is the first time the APT defense problem is dealt with in this way. These findings help organizations to work out policies of defending against APTs.The remaining materials are organized in this fashion. Section2 models the APT defense problem as an optimal control problem. Section 3 studies the optimal control problem. Some most effective APT defense strategies are given in Section 4. Section 5 discusses the influence of different factors on the optimal effectiveness. This work is closed by Section 6.


## 2. The Modeling of the APT Defense Problem

The goal of this paper is to solve the following problem.The APT Defense Problem. Defend an organization against APTs in an effective way.To achieve the goal, we have to model the problem. The modeling process consists of the following four steps.Step 1.
Introduce preliminary terminologies and notations.Step 2.
Establish an APT attack-defense model.Step 3.
Quantify the effectiveness of an APT defense strategy.Step 4.
Model the APT defense problem as an optimal control problem.Now, let us proceed by following this four-step procedure.


### 2.1. Preliminary Terminologies and Notations

Consider an organization with a set ofN computer systems labeled 1,2,…,N. Let G=(V,E) denote the access network of the organization, where (a) each node stands for a system, that is, V={1,2,…,N}, and (b) (i,j)∈E if and only if system i has access to system j. Let A=aijN×N denote the adjacency matrix for the network, that is, aij=1 or 0 according to (i,j)∈E or not.Suppose an APT campaign to the organization starts at timet=0 and terminates at time t=T. Suppose at any time t∈[0,T] every node in the organization is eithersecure, that is, under the defender’s control, orcompromised, that is, under the attacker’s control. Let Xi(t)=0 and 1 denote the event that node i is secure and compromised at time t, respectively. The vector(1)Xt=X1t,X2t,…,XNtstands for thestate of the organization at time t. Let Si(t) and Ci(t) denote the probability of the event that node i is secure and compromised at time t, respectively. That is,(2)Sit=Pr⁡Xit=0,Cit=Pr⁡Xit=1.As Si(t)+Ci(t)≡1, the vector(3)Ct=C1t,…,CNtTstands for theexpected state of the organization at time t.From the attacker’s perspective, each secure node in the organization is subject to the external attack. Letai denote the cost per unit time for attacking a secure node i. The vector(4)a=a1,…,aNstands for anattack strategy. Additionally, each secure node is vulnerable to all the neighboring compromised nodes.From the defender’s perspective, each secure node in the organization is protected from being compromised. Letxi(t) denote the cost per unit time for protecting the secure node i at time t. The vector-valued function(5)xt=x1t,…,xNt,0≤t≤T,stands for aprevention strategy. Additionally, each compromised node in the organization is recovered. Let yi(t) denote the cost per unit time for recovering the compromised node i at time t. The vector-valued function(6)yt=y1t,…,yNt,0≤t≤T,stands for arecovery strategy. We refer to the vector-valued function(7)ut=xt,yt=x1t,…,xNt,y1t,…,yNt,0≤t≤Tas anAPT defense strategy.


### 2.2. An APT Attack-Defense Model

For fundamental knowledge on differential dynamical systems, see [49]. For our purpose, let us impose a set of hypotheses as follows.(H1)
Due to the external attack and prevention, a secure nodei gets compromised at time t at the average rate ai/xi(t). The rationality of this hypothesis lies in that the average rate is proportional to the attack cost per unit time and is inversely proportional to the prevention cost per unit time.(H2)
Due to the internal infection and prevention, a secure nodei gets compromised at time t at the average rate β∑j=1N‍ajiCj(t)/xi(t), where β>0 is a constant, which we refer to as theinfection force. The rationality of this assumption lies in that the average rate is proportional to the probability of each neighboring node being compromised and is inversely proportional to the prevention cost per unit time.(H3)
Due to the recovery, a compromised nodei becomes secure at time t at the average rate yi(t). The rationality of this assumption lies in that the average rate is proportional to the recovery cost per unit time.According to these hypotheses, the state transitions of a node are shown in Figure1. Hence, the time evolution of the expected state of the organization obeys the following dynamical system:(8)dCitdt=1xitai+β∑j=1NajiCjt1-Cit-yitCit,0≤t≤T,i=1,…,N.Figure 1
The diagram of state transitions of a node under the hypotheses (H1)–(H3).We refer to the model as theAPT attack-defense model.The APT attack-defense model can be written in matrix-vector notation as(9)dCtdt=FCt,ut,0≤t≤T.


### 2.3. The Effectiveness of an APT Defense Strategy

The defender’s goal is to find the most effective APT defense strategy. To achieve the goal, we have to quantify the effectiveness of an APT defense strategy. For this purpose, let us introduce an additional set of hypotheses as follows.(H4)
The prevention cost per unit time is bounded from above byx¯ and from below by x_>0, and the recovery cost per unit time is bounded from above by y¯ and from below by y_>0. That is, the admissible set of APT defense strategies is given by(10)U=u∈L20,T2N∣0<x_≤xit≤x¯,0<y_≤yit≤y¯,0≤t≤T,1≤i≤N,whereL2[0,T] denote the set of all the Lebesgue square integrable functions defined on the interval [0,T] [50].(H5)
The amount of losses caused by a compromised nodei in the infinitesimal time interval [t,t+dt] is widt, where wi=∑j=1N‍aij stands for the out-degree of node i in the network. The rationality of this hypothesis lies in that the more nodes a node has access to, the more serious the consequence when it is compromised [51, 52].According to the hypotheses, the expected loss of the organization in the time horizon[0,T] when implementing an APT defense strategy u=x,y is(11)Lossu=∑i=1N∫0TwiCitdt,and the overall cost for implementing the APT defense strategy is(12)Costu=∑i=1N∫0Txit+yitdt.Hence, the effectiveness of the APT defense strategy u can be measured by the quantity (13)Ju=Lossu+Costu=∑i=1N∫0TwiCitdt+∑i=1N∫0Txit+yitdt.Obviously, the smaller this quantity, the more effective the APT defense strategy. Let(14)LCt,ut=∑i=1NwiCit+xit+yit.Then(15)Ju=∫0TLCt,utdt.


### 2.4. The Modeling of the APT Defense Strategy

Based on the previous discussions, the APT defense problem boils down to the following optimal control problem:(16)minu∈UJu=∫0TLCt,utdt,subjecttodCtdt=FCt,ut,0≤t≤T,C0=C0.Here, each control stands for an APT defense strategy, the objective functional stands for the effectiveness of an APT defense strategy, the set of constraints stands for the time evolution of the expected state of the organization, an optimal control stands for a most effective APT defense strategy, and the optimal value stands for the effectiveness of a most effective APT defense strategy.


## 2.1. Preliminary Terminologies and Notations

Consider an organization with a set ofN computer systems labeled 1,2,…,N. Let G=(V,E) denote the access network of the organization, where (a) each node stands for a system, that is, V={1,2,…,N}, and (b) (i,j)∈E if and only if system i has access to system j. Let A=aijN×N denote the adjacency matrix for the network, that is, aij=1 or 0 according to (i,j)∈E or not.Suppose an APT campaign to the organization starts at timet=0 and terminates at time t=T. Suppose at any time t∈[0,T] every node in the organization is eithersecure, that is, under the defender’s control, orcompromised, that is, under the attacker’s control. Let Xi(t)=0 and 1 denote the event that node i is secure and compromised at time t, respectively. The vector(1)Xt=X1t,X2t,…,XNtstands for thestate of the organization at time t. Let Si(t) and Ci(t) denote the probability of the event that node i is secure and compromised at time t, respectively. That is,(2)Sit=Pr⁡Xit=0,Cit=Pr⁡Xit=1.As Si(t)+Ci(t)≡1, the vector(3)Ct=C1t,…,CNtTstands for theexpected state of the organization at time t.From the attacker’s perspective, each secure node in the organization is subject to the external attack. Letai denote the cost per unit time for attacking a secure node i. The vector(4)a=a1,…,aNstands for anattack strategy. Additionally, each secure node is vulnerable to all the neighboring compromised nodes.From the defender’s perspective, each secure node in the organization is protected from being compromised. Letxi(t) denote the cost per unit time for protecting the secure node i at time t. The vector-valued function(5)xt=x1t,…,xNt,0≤t≤T,stands for aprevention strategy. Additionally, each compromised node in the organization is recovered. Let yi(t) denote the cost per unit time for recovering the compromised node i at time t. The vector-valued function(6)yt=y1t,…,yNt,0≤t≤T,stands for arecovery strategy. We refer to the vector-valued function(7)ut=xt,yt=x1t,…,xNt,y1t,…,yNt,0≤t≤Tas anAPT defense strategy.


## 2.2. An APT Attack-Defense Model

For fundamental knowledge on differential dynamical systems, see [49]. For our purpose, let us impose a set of hypotheses as follows.(H1)
Due to the external attack and prevention, a secure nodei gets compromised at time t at the average rate ai/xi(t). The rationality of this hypothesis lies in that the average rate is proportional to the attack cost per unit time and is inversely proportional to the prevention cost per unit time.(H2)
Due to the internal infection and prevention, a secure nodei gets compromised at time t at the average rate β∑j=1N‍ajiCj(t)/xi(t), where β>0 is a constant, which we refer to as theinfection force. The rationality of this assumption lies in that the average rate is proportional to the probability of each neighboring node being compromised and is inversely proportional to the prevention cost per unit time.(H3)
Due to the recovery, a compromised nodei becomes secure at time t at the average rate yi(t). The rationality of this assumption lies in that the average rate is proportional to the recovery cost per unit time.According to these hypotheses, the state transitions of a node are shown in Figure1. Hence, the time evolution of the expected state of the organization obeys the following dynamical system:(8)dCitdt=1xitai+β∑j=1NajiCjt1-Cit-yitCit,0≤t≤T,i=1,…,N.Figure 1
The diagram of state transitions of a node under the hypotheses (H1)–(H3).We refer to the model as theAPT attack-defense model.The APT attack-defense model can be written in matrix-vector notation as(9)dCtdt=FCt,ut,0≤t≤T.


## 2.3. The Effectiveness of an APT Defense Strategy

The defender’s goal is to find the most effective APT defense strategy. To achieve the goal, we have to quantify the effectiveness of an APT defense strategy. For this purpose, let us introduce an additional set of hypotheses as follows.(H4)
The prevention cost per unit time is bounded from above byx¯ and from below by x_>0, and the recovery cost per unit time is bounded from above by y¯ and from below by y_>0. That is, the admissible set of APT defense strategies is given by(10)U=u∈L20,T2N∣0<x_≤xit≤x¯,0<y_≤yit≤y¯,0≤t≤T,1≤i≤N,whereL2[0,T] denote the set of all the Lebesgue square integrable functions defined on the interval [0,T] [50].(H5)
The amount of losses caused by a compromised nodei in the infinitesimal time interval [t,t+dt] is widt, where wi=∑j=1N‍aij stands for the out-degree of node i in the network. The rationality of this hypothesis lies in that the more nodes a node has access to, the more serious the consequence when it is compromised [51, 52].According to the hypotheses, the expected loss of the organization in the time horizon[0,T] when implementing an APT defense strategy u=x,y is(11)Lossu=∑i=1N∫0TwiCitdt,and the overall cost for implementing the APT defense strategy is(12)Costu=∑i=1N∫0Txit+yitdt.Hence, the effectiveness of the APT defense strategy u can be measured by the quantity (13)Ju=Lossu+Costu=∑i=1N∫0TwiCitdt+∑i=1N∫0Txit+yitdt.Obviously, the smaller this quantity, the more effective the APT defense strategy. Let(14)LCt,ut=∑i=1NwiCit+xit+yit.Then(15)Ju=∫0TLCt,utdt.


## 2.4. The Modeling of the APT Defense Strategy

Based on the previous discussions, the APT defense problem boils down to the following optimal control problem:(16)minu∈UJu=∫0TLCt,utdt,subjecttodCtdt=FCt,ut,0≤t≤T,C0=C0.Here, each control stands for an APT defense strategy, the objective functional stands for the effectiveness of an APT defense strategy, the set of constraints stands for the time evolution of the expected state of the organization, an optimal control stands for a most effective APT defense strategy, and the optimal value stands for the effectiveness of a most effective APT defense strategy.


## 3. A Theoretical Analysis of the Optimal Control Problem

For fundamental knowledge on optimal control theory, see [10, 11]. This section is devoted to studying the optimal control problem (16).


### 3.1. The Existence of an Optimal Control

As an optimal control to problem (16) represents a most effective APT defense strategy, it is critical to show that the problem does have an optimal control. For this purpose, we need the following lemma [11].Lemma 1.
Problem (16) has an optimal control if the following five conditions hold simultaneously. (C1)
U is closed and convex.(C2)
There isu∈U such that the adjunctive dynamical system is solvable.(C3)
F(C,u) is bounded by a linear function in C.(C4)
L(C,u) is convex on U.(C5)
L(C,u)≥c1uρ+c2 for some vector norm ·, ρ>1,c1>0 and c2.Next, let us show that the five conditions in Lemma1 indeed hold.Lemma 2.
The admissible setU is closed.Proof.
Letu=(x1,…,xN,y1,…,yN) be a limit point of U, and let u(n)=x1(n),…,xN(n),y1(n),…,yN(n),n=1,2,…, be a sequence of points in U that approaches u. As L2[0,T]2N is complete, we have u∈L2(0,T)2N. Hence, the claim follows from the observation that(17)x_≤xit=limn→∞xint≤x¯,y_≤yit=limn→∞yint≤y¯,0≤t≤T,1≤i≤N.Lemma 3.
The admissible setU is convex.Proof.
Letu(1)=x1(1),…,xN(1),y1(1),…,yN(1), u(2)=x1(2),…,xN(2),y1(2),…,yN(2)∈U, 0<η<1. As L2[0,T]2N is a real vector space, we get (1-η)u(1)(t)+ηu(2)(t)∈L2[0,T]2N. So, the claim follows from the observation that (18)x_≤1-ηxi1t+ηxi2t≤x¯,y_≤1-ηyi1t+ηyi2t≤y¯,0≤t≤T,1≤i≤N.Lemma 4.
There isu∈U such that the associated adjunctive dynamical system is solvable.Proof.
Consider the adjunctive dynamical system(19)dCtdt=FCt,u¯,0≤t≤T,where u(t)≡u¯=(x¯,…,x¯,y¯,…,y¯). As F(C,u¯) is continuously differentiable, the claim follows from the Continuation Theorem for Differential Dynamical Systems [49].Lemma 5.
F ( C , u ) is bounded by a linear function in C.Proof.
The claim follows from the observation that, for0≤t≤T, i=1,2,…,N,(20)1xitai+β∑j=1NajiCjt1-Cit-yitCit≤1x_ai+β∑j=1NajiCjt-y_Cit.Lemma 6.
L ( C , u ) is convex on U.Proof.
Letu(1),u(2)∈U, 0<η<1. Then(21)LC,1-ηu1+ηu2=1-ηLC,u1+ηLC,u2.Lemma 7.
One hasL(C,u)≥1/max⁡x¯,y¯u22.Proof.
We have(22)LC,u=∑i=1NwiCi+xi+yi≥∑i=1Nxi+yi≥∑i=1Nxi2x¯+yi2y¯≥1max⁡x¯,y¯u22.Based on Lemmas1–7, we get the following result.Theorem 8.
Problem (16) has an optimal control.This theorem guarantees that there is a most effective APT defense strategy.


### 3.2. The Optimality System

It is known that the optimality system for an optimal control problem offers a method for numerically solving the problem. This subsection is intended to present the optimality system for problem (16). For this purpose, consider the corresponding Hamiltonian(23)HCt,ut,λt=∑i=1NwiCit+xit+yit+∑i=1Nλit1xitai+β∑j=1NajiCjt1-Cit-yitCit,where λ=(λ1,…,λN)T is the adjoint.Theorem 9.
Supposeu∗ is an optimal control to problem (16) and C∗ is the solution to the adjunctive dynamical system with u=u∗. Then, there exists λ∗ with λ∗(T)=0 such that, for 0≤t≤T, 1≤i≤N,(24)dλi∗tdt=-wi+yi∗tλi∗t+λi∗txi∗tai+β∑j=1NajiCj∗t-β∑j=1Naij1-Cj∗tλj∗txj∗t,xi∗t=max⁡min⁡λi∗tai+β∑j=1NajiCj∗t1-Ci∗t1/2,x¯,x_,yi∗t=y_,λi∗tCi∗t<1,y¯,λi∗tCi∗t>1.Proof.
According to the Pontryagin Minimum Principle [10], there exists λ∗ such that(25)dλi∗tdt=-∂HC∗t,u∗t,λ∗t∂Ci,0≤t≤T,1≤i≤N.Thus, the first N equations in the claim follow by direct calculations. As the terminal cost is unspecified and the final state is free, the transversality condition λ∗(T)=0 holds. By using the optimality condition (26)u∗t=arg⁡minu∈UHC∗t,ut,λ∗t,0≤t≤T.we get (a) either(27)∂HC∗t,u∗t,λ∗t∂xi=1-λi∗txi∗t2ai+β∑i=1NajiCj∗t1-Ci∗t=0or xi∗(t)=x_ or xi∗(t)=x¯ and (b)(28)yi∗t=argminy_≤yit≤y¯1-λi∗tCi∗tyit=y_,λi∗tCi∗t<1,y¯,λi∗tCi∗t>1.Combining the above discussions, we get the optimality system for problem (16) as follows.(29)dCitdt=1xitai+β∑j=1NajiCjt1-Cit-yitCit,dλitdt=-wi+yitλit+λitxitai+β∑j=1NajiCjt-β∑j=1Naij1-Cjtλjtxjt,xit=max⁡min⁡λitai+β∑j=1NajiCjt1-Cit1/2,x¯,x_,yit=y_,λitCit<1,y¯,λitCit>1,where C(0)=C0, λ(T)=0, 0≤t≤T, 1≤i≤N.Applying the forward-backward Euler scheme to the optimality system, we can obtain an optimal control to problem (16), that is, a most effective APT defense strategy.


## 3.1. The Existence of an Optimal Control

As an optimal control to problem (16) represents a most effective APT defense strategy, it is critical to show that the problem does have an optimal control. For this purpose, we need the following lemma [11].Lemma 1.
Problem (16) has an optimal control if the following five conditions hold simultaneously. (C1)
U is closed and convex.(C2)
There isu∈U such that the adjunctive dynamical system is solvable.(C3)
F(C,u) is bounded by a linear function in C.(C4)
L(C,u) is convex on U.(C5)
L(C,u)≥c1uρ+c2 for some vector norm ·, ρ>1,c1>0 and c2.Next, let us show that the five conditions in Lemma1 indeed hold.Lemma 2.
The admissible setU is closed.Proof.
Letu=(x1,…,xN,y1,…,yN) be a limit point of U, and let u(n)=x1(n),…,xN(n),y1(n),…,yN(n),n=1,2,…, be a sequence of points in U that approaches u. As L2[0,T]2N is complete, we have u∈L2(0,T)2N. Hence, the claim follows from the observation that(17)x_≤xit=limn→∞xint≤x¯,y_≤yit=limn→∞yint≤y¯,0≤t≤T,1≤i≤N.Lemma 3.
The admissible setU is convex.Proof.
Letu(1)=x1(1),…,xN(1),y1(1),…,yN(1), u(2)=x1(2),…,xN(2),y1(2),…,yN(2)∈U, 0<η<1. As L2[0,T]2N is a real vector space, we get (1-η)u(1)(t)+ηu(2)(t)∈L2[0,T]2N. So, the claim follows from the observation that (18)x_≤1-ηxi1t+ηxi2t≤x¯,y_≤1-ηyi1t+ηyi2t≤y¯,0≤t≤T,1≤i≤N.Lemma 4.
There isu∈U such that the associated adjunctive dynamical system is solvable.Proof.
Consider the adjunctive dynamical system(19)dCtdt=FCt,u¯,0≤t≤T,where u(t)≡u¯=(x¯,…,x¯,y¯,…,y¯). As F(C,u¯) is continuously differentiable, the claim follows from the Continuation Theorem for Differential Dynamical Systems [49].Lemma 5.
F ( C , u ) is bounded by a linear function in C.Proof.
The claim follows from the observation that, for0≤t≤T, i=1,2,…,N,(20)1xitai+β∑j=1NajiCjt1-Cit-yitCit≤1x_ai+β∑j=1NajiCjt-y_Cit.Lemma 6.
L ( C , u ) is convex on U.Proof.
Letu(1),u(2)∈U, 0<η<1. Then(21)LC,1-ηu1+ηu2=1-ηLC,u1+ηLC,u2.Lemma 7.
One hasL(C,u)≥1/max⁡x¯,y¯u22.Proof.
We have(22)LC,u=∑i=1NwiCi+xi+yi≥∑i=1Nxi+yi≥∑i=1Nxi2x¯+yi2y¯≥1max⁡x¯,y¯u22.Based on Lemmas1–7, we get the following result.Theorem 8.
Problem (16) has an optimal control.This theorem guarantees that there is a most effective APT defense strategy.


## 3.2. The Optimality System

It is known that the optimality system for an optimal control problem offers a method for numerically solving the problem. This subsection is intended to present the optimality system for problem (16). For this purpose, consider the corresponding Hamiltonian(23)HCt,ut,λt=∑i=1NwiCit+xit+yit+∑i=1Nλit1xitai+β∑j=1NajiCjt1-Cit-yitCit,where λ=(λ1,…,λN)T is the adjoint.Theorem 9.
Supposeu∗ is an optimal control to problem (16) and C∗ is the solution to the adjunctive dynamical system with u=u∗. Then, there exists λ∗ with λ∗(T)=0 such that, for 0≤t≤T, 1≤i≤N,(24)dλi∗tdt=-wi+yi∗tλi∗t+λi∗txi∗tai+β∑j=1NajiCj∗t-β∑j=1Naij1-Cj∗tλj∗txj∗t,xi∗t=max⁡min⁡λi∗tai+β∑j=1NajiCj∗t1-Ci∗t1/2,x¯,x_,yi∗t=y_,λi∗tCi∗t<1,y¯,λi∗tCi∗t>1.Proof.
According to the Pontryagin Minimum Principle [10], there exists λ∗ such that(25)dλi∗tdt=-∂HC∗t,u∗t,λ∗t∂Ci,0≤t≤T,1≤i≤N.Thus, the first N equations in the claim follow by direct calculations. As the terminal cost is unspecified and the final state is free, the transversality condition λ∗(T)=0 holds. By using the optimality condition (26)u∗t=arg⁡minu∈UHC∗t,ut,λ∗t,0≤t≤T.we get (a) either(27)∂HC∗t,u∗t,λ∗t∂xi=1-λi∗txi∗t2ai+β∑i=1NajiCj∗t1-Ci∗t=0or xi∗(t)=x_ or xi∗(t)=x¯ and (b)(28)yi∗t=argminy_≤yit≤y¯1-λi∗tCi∗tyit=y_,λi∗tCi∗t<1,y¯,λi∗tCi∗t>1.Combining the above discussions, we get the optimality system for problem (16) as follows.(29)dCitdt=1xitai+β∑j=1NajiCjt1-Cit-yitCit,dλitdt=-wi+yitλit+λitxitai+β∑j=1NajiCjt-β∑j=1Naij1-Cjtλjtxjt,xit=max⁡min⁡λitai+β∑j=1NajiCjt1-Cit1/2,x¯,x_,yit=y_,λitCit<1,y¯,λitCit>1,where C(0)=C0, λ(T)=0, 0≤t≤T, 1≤i≤N.Applying the forward-backward Euler scheme to the optimality system, we can obtain an optimal control to problem (16), that is, a most effective APT defense strategy.


## 4. Some Most Effective APT Defense Strategies

In this section, we give some most effective APT defense strategies by solving the optimality system (29). For ease in observation, let us introduce two functions as follows. For an admissible control u to problem (16), define thecumulative effectiveness (CE) as(30)CEt;u=∑i=1N∫0twiCisds+∑i=1N∫0txis+yisds,0≤t≤T,and define thesuperposed control (SC) as(31)SCt;u=∑i=1Nxit+yit,0≤t≤T.Obviously, we have CE(T;u)=J(u).For some optimal control problems, let us give the cumulative effectiveness and superposed control for an optimal control.Example 10.
Consider problem (16) in which G is a scale-free network with N=100 nodes which is generated by executing the algorithm given in [53], T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. An optimal control to the problem is obtained by solving the optimality system (29). Figure 2 plots the cumulative effectiveness and superposed control for the optimal control. For comparison purpose, the cumulative effectiveness and superposed control for three admissible static controls are also shown in Figure 2.Figure 2
The cumulative effectiveness and superposed control for the optimal control and a few static controls in Example10.
(a) (b)Example 11.
Consider problem (16) in which G is a small-world network with N=100 nodes which is generated by executing the algorithm given in [54], T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. An optimal control to the problem is obtained by solving the optimality system (29). Figure 3 depicts the cumulative effectiveness and superposed control for the optimal control. For comparison purpose, the cumulative effectiveness and superposed control for three admissible static controls are also shown in Figure 3.Figure 3
The cumulative effectiveness and superposed control for the optimal control and a few static controls in Example11.
(a) (b)Example 12.
Consider problem (16) in which G is a realistic network given in [55], T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. An optimal control to the problem is obtained by solving the optimality system (29). Figure 4 exhibits the cumulative effectiveness and superposed control for the optimal control. For comparison purpose, the cumulative effectiveness and superposed control for three admissible static controls are also shown in Figure 4.Figure 4
The cumulative effectiveness and superposed control for the optimal control and a few static controls in Example12.
(a) (b)It is seen from the above three examples that a most effective APT defense strategy is significantly superior to any static APT defense strategy in terms of the effectiveness. This observation justifies our method. Additionally, the superposed control drops rapidly to a lower value.


## 5. Further Discussions

This section is devoted to examining the influence of different factors on the optimal effectiveness of an admissible APT defense strategy. For ease in understanding these influences, let us introduce three quantities as follows. For an optimal controlu∗ to problem (16), let OL∗, OC∗, and OJ∗ denote the corresponding expected loss, overall cost, and effectiveness, respectively. That is,(32)OL∗=Lossu∗,OC∗=Costu∗,OJ∗=OL∗+OC∗=Ju∗.


### 5.1. The Bounds on the Admissible Controls

Definitely, the four bounds on the admissible controls affect the optimal effectiveness of an admissible APT defense strategy. Now, let us examine these influences.Example 13.
Consider a set of problems (16) in which G is the scale-free network generated in Example 10, T=20, β=0.001, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. (a)
Lety_=0.1, y¯=0.7. Figures 5(a)–5(c) exhibit the influence of x_ and x¯ on OL∗, OC∗, and OJ∗, respectively.(b)
Letx_=0.1, x¯=0.7. Figures 5(d)–5(f) display the influence of y_ and y¯ on OL∗, OC∗, and OJ∗, respectively.Figure 5
The influence of the four bounds onOL∗, OC∗, and OJ∗ in Example 13.
(a) (b) (c) (d) (e) (f)Example 14.
Consider a set of problems (16) in which G is the small-world network generated in Example 11, T=20, β=0.001, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. (a)
Lety_=0.1, y¯=0.7. Figures 6(a)–6(c) exhibit the influence of x_ and x¯ on OL∗, OC∗, and OJ∗, respectively.(b)
Letx_=0.1, x¯=0.7. Figures 6(d)–6(f) display the influence of y_ and y¯ on OL∗, OC∗, and OJ∗, respectively.Figure 6
The influence of the four bounds onOL∗, OC∗, and OJ∗ in Example 14.
(a) (b) (c) (d) (e) (f)Example 15.
Consider a set of problems (16) in which G is the realistic network given in Example 12, T=20, β=0.001, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. (a)
Lety_=0.1, y¯=0.7. Figures 7(a)–7(c) exhibit the influence of x_ and x¯ on OL∗, OC∗, and OJ∗, respectively.(b)
Letx_=0.1, x¯=0.7. Figures 7(d)–7(f) display the influence of y_ and y¯ on OL∗, OC∗, and OJ∗, respectively.
The following conclusions are drawn from the above three examples.(a)
With the increase of the two lower bounds,OL∗ goes down, but OC∗ and OJ∗ go up. In practice, the two lower bounds should be chosen carefully so that a balance between the expected loss and the overall cost is achieved.(b)
The influence of the two upper bounds onOL∗, OC∗, and OJ∗ is almost negligible.Figure 7
The influence of the four bounds onOL∗, OC∗, and OJ∗ in Example 15.
(a) (b) (c) (d) (e) (f)


### 5.2. The Network Topology

Obviously, the topology of the network in an organization affects the optimal effectiveness of an admissible APT defense strategy. Now, let us inspect this influence.Example 16.
Consider a set of problems (16) in which G∈Gi:1≤i≤7, where Gi is a scale-free network with N=100 nodes and a power-law exponent of γi=2.7+0.1×i, T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. Figure 8 displays the influence of the power-law exponent on OL∗, OC∗, and OJ∗, respectively.
It is seen from this example that, with the increase of the power-law exponent of a scale-free network,OL∗ and OJ∗ decline, but OC∗ inclines. It is well known that the heterogeneity of a scale-free network increases with its power-law exponent. Therefore, a homogeneously mixed access network is better in terms of the optimal defense effectiveness.Figure 8
The influence of the power-law exponent of a scale-free network onOL∗, OC∗, and OJ∗ in Example 16.
(a) (b) (c)Example 17.
Consider a set of problems(P) in which G∈Gi:1≤i≤5, where Gi is a small-world network with N=100 nodes and an edge-rewiring probability of pi=0.1×i, T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. Figure 9 exhibits the influence of the edge-rewiring probability on OL∗, OC∗, and OJ∗, respectively.
It is seen from this example that, with the increase of the randomness of a small-world network,OL∗, OC∗, and OJ∗ rise rapidly. Hence, a randomly connected access network is better from the perspective of the optimal defense effectiveness.Figure 9
The influence of the edge-rewiring probability of a small-world network onOL∗, OC∗, and OJ∗ in Example 17.
(a) (b) (c)


## 5.1. The Bounds on the Admissible Controls

Definitely, the four bounds on the admissible controls affect the optimal effectiveness of an admissible APT defense strategy. Now, let us examine these influences.Example 13.
Consider a set of problems (16) in which G is the scale-free network generated in Example 10, T=20, β=0.001, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. (a)
Lety_=0.1, y¯=0.7. Figures 5(a)–5(c) exhibit the influence of x_ and x¯ on OL∗, OC∗, and OJ∗, respectively.(b)
Letx_=0.1, x¯=0.7. Figures 5(d)–5(f) display the influence of y_ and y¯ on OL∗, OC∗, and OJ∗, respectively.Figure 5
The influence of the four bounds onOL∗, OC∗, and OJ∗ in Example 13.
(a) (b) (c) (d) (e) (f)Example 14.
Consider a set of problems (16) in which G is the small-world network generated in Example 11, T=20, β=0.001, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. (a)
Lety_=0.1, y¯=0.7. Figures 6(a)–6(c) exhibit the influence of x_ and x¯ on OL∗, OC∗, and OJ∗, respectively.(b)
Letx_=0.1, x¯=0.7. Figures 6(d)–6(f) display the influence of y_ and y¯ on OL∗, OC∗, and OJ∗, respectively.Figure 6
The influence of the four bounds onOL∗, OC∗, and OJ∗ in Example 14.
(a) (b) (c) (d) (e) (f)Example 15.
Consider a set of problems (16) in which G is the realistic network given in Example 12, T=20, β=0.001, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. (a)
Lety_=0.1, y¯=0.7. Figures 7(a)–7(c) exhibit the influence of x_ and x¯ on OL∗, OC∗, and OJ∗, respectively.(b)
Letx_=0.1, x¯=0.7. Figures 7(d)–7(f) display the influence of y_ and y¯ on OL∗, OC∗, and OJ∗, respectively.
The following conclusions are drawn from the above three examples.(a)
With the increase of the two lower bounds,OL∗ goes down, but OC∗ and OJ∗ go up. In practice, the two lower bounds should be chosen carefully so that a balance between the expected loss and the overall cost is achieved.(b)
The influence of the two upper bounds onOL∗, OC∗, and OJ∗ is almost negligible.Figure 7
The influence of the four bounds onOL∗, OC∗, and OJ∗ in Example 15.
(a) (b) (c) (d) (e) (f)


## 5.2. The Network Topology

Obviously, the topology of the network in an organization affects the optimal effectiveness of an admissible APT defense strategy. Now, let us inspect this influence.Example 16.
Consider a set of problems (16) in which G∈Gi:1≤i≤7, where Gi is a scale-free network with N=100 nodes and a power-law exponent of γi=2.7+0.1×i, T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. Figure 8 displays the influence of the power-law exponent on OL∗, OC∗, and OJ∗, respectively.
It is seen from this example that, with the increase of the power-law exponent of a scale-free network,OL∗ and OJ∗ decline, but OC∗ inclines. It is well known that the heterogeneity of a scale-free network increases with its power-law exponent. Therefore, a homogeneously mixed access network is better in terms of the optimal defense effectiveness.Figure 8
The influence of the power-law exponent of a scale-free network onOL∗, OC∗, and OJ∗ in Example 16.
(a) (b) (c)Example 17.
Consider a set of problems(P) in which G∈Gi:1≤i≤5, where Gi is a small-world network with N=100 nodes and an edge-rewiring probability of pi=0.1×i, T=20, β=0.001, x_=y_=0.1, x¯=y¯=0.7, ai=0.1, 0≤i≤N, and Ci(0)=0.1,0≤i≤N. Figure 9 exhibits the influence of the edge-rewiring probability on OL∗, OC∗, and OJ∗, respectively.
It is seen from this example that, with the increase of the randomness of a small-world network,OL∗, OC∗, and OJ∗ rise rapidly. Hence, a randomly connected access network is better from the perspective of the optimal defense effectiveness.Figure 9
The influence of the edge-rewiring probability of a small-world network onOL∗, OC∗, and OJ∗ in Example 17.
(a) (b) (c)


## 6. Concluding Remarks

This paper has addressed the APT defense problem, that is, the problem of how to effectively defend against APTs. By introducing an APT attack-defense model and quantifying the effectiveness of an APT defense strategy, we have modeled the APT defense problem as an optimal control problem in which an optimal control represents a most effective APT defense strategy. Through theoretical study, we have presented the optimality system for the optimal control problem. This implies that an optimal control can be derived by solving the optimality system. The influence of some factors on the optimal effectiveness of an APT defense strategy has been examined.There are many relevant problems to be resolved. The expected loss and overall cost of an APT defense strategy should be appropriately balanced to adapt to specific application scenarios. In practice, the implementation of a recommended defense strategy needs a great effort; the security level of all the systems in an organization must be labeled accurately [6], the defense budget must be made, and the robustness of the defense strategy must be evaluated. As the topology of the access network in an organization may well vary with time, the approach proposed in this work should be adapted to time-varying networks [56–59]. It is of practical importance to deal with the APT defense problem in the game-theoretical framework, where the attacker is strategic [60–63].



---

*Source: 2975376-2018-02-27.xml*