# Stochastic Differential Game-Based Malware Propagation in Edge Computing-Based IoT

Authors: Li Miao; Shuai Li
Journal: Security and Communication Networks
 (2021)

---

## Abstract

Internet of Things (IoT) has played an important role in our daily life since its emergence. The applications of IoT cover from the traditional devices to intelligent equipment. With the great potential of IoT, there comes various kinds of security problems. In this paper, we study the malware propagation under the dynamic interaction between the attackers and defenders in edge computing-based IoT and propose an infinite-horizon stochastic differential game model to discuss the optimal strategies for the attackers and defenders. Considering the effect of stochastic fluctuations in the edge network on the malware propagation, we construct the Itô stochastic differential equations to describe the propagation of the malware in edge computing-based IoT. Subsequently, we analyze the feedback Nash equilibrium solutions for our proposed game model, which can be considered as the optimal strategies for the defenders and attackers. Finally, numerical simulations show the effectiveness of our proposed game model.


---

## Body

## 1. Introduction

Recently, a rapidly increasing number of physical devices and sensors are connecting to the Internet at an unprecedented rate. It has led to the emergence of the Internet of Things (IoT). By deploying smart devices and sensors to collect and analyze the physical data, the IoT can monitor and control the physical environment [1]. IoT has brought great convenience to our daily life in the past few years. For example, the IoT has been widely used in intelligent transportation, smart home appliances, smart healthcare, and other fields [2, 3].Since the IoT devices typically have limited resources, it is common to forward the physical data to the cloud computing platform, which will need extra bandwidth or cause data security problem. With the advance of IoT, edge computing has been introduced to address the above issues [4–6]. Generally, edge computing provides powerful computing resources at the edge of the Internet and is close to the IoT devices [7, 8]. Edge computing has relieved the pressure of bandwidth and overcome the latency issue. However, edge computing environment is an open ecosystem and the IoT devices with limited resources are more vulnerable to be attacked [9]. Then, the existing defense mechanisms based on cloud computing cannot be used to edge computing because of the geographically dispersed nature of IoT devices. Thus, how to effectively design defense mechanisms to defend against attackers has become a serious problem that desperately needs to be solved. In this paper, we pay attention to the security problem of malware propagation based on the stochastic differential game; in this framework, we try to model an optimal defense strategy for IoT devices.In edge computing-based IoT, attackers want to infect more IoT devices with malware to gain illegal gains using the attack strategy, while the defenders want to minimize the damage caused by IoT devices infected with malware using the defense strategy. Meanwhile, the IoT devices join or exit the network randomly, which can affect the stability of the edge network. The dynamic interaction between the attackers and defenders leads to the propagation of the malware, and the influence of network instability can be considered as the stochastic elements. In this paper, we propose an infinite-horizon stochastic differential game model to research the malware propagation among IoT devices under the dynamic interaction between attackers and defenders in edge computing-based IoT, considering the stochastic fluctuations in the network. The main contributions of our proposed scheme are as follows:(1
)Firstly, we use the infinite-horizon stochastic differential game to model the malware propagation under the dynamic interaction between the attackers and defenders in edge computing-based IoT.(2)
Secondly, the Itô stochastic differential equation is used to characterize the effect of stochastic fluctuations of the edge network on the malware propagation.(3)
Finally, we discuss the feedback Nash equilibrium solutions for our proposed game model, which can be considered as the optimal strategies for both the attackers and defenders.This paper is organized as follows. Section2 introduces related works. Section 3 discusses the security problem of the attackers and defenders in a stochastic differential game theory. The feedback Nash equilibrium solutions for our proposed game model are analyzed in Section 4. Numerical simulations are given in Section 5. Finally, we conclude this paper in Section 6.


## 2. Related Works

Malware propagation problem is one of the most fundamental problems, for which many kinds of research have been proposed in the literature [10–15]. Malware propagation means that the infected legitimate nodes are able to contaminate other noninfected legitimate nodes, in addition to the attack nodes [16]. The edge users achieve shared interactions through smart applications in edge computing-based IoT, which increase the probability of malware download.Generally, there are two complementary classes of methods to defend against malware threat: detection-based method and prevention-based method. Tobias et al. [10] proposed a novel malware detection approach that used the compression-based graph mining, in which the characteristic behaviors were extracted by the quantitative data flow graphs to derive the detection accuracy. TaeGuen et al. [11] discussed the malware characteristics through the feature vector generation methods and proposed a multimodal deep neural network malware detection model for android applications. The advantage of this method is more suitable for the dynamic environments. Dehghantanha et al. [12] studied the malware detection problem in IoT using the deep learning based method, which provided a new direction for further research. Since various IoT device vulnerabilities, Indre et al. [13] created a system that could detect and prevent malicious connections based on machine learning to enhance network security. Lan et al. [14] researched the propagation of epidemic in complex networks and proposed a dynamic prevention model with a time-varying community network. They considered the subnets of the network as communities and investigated the process of the malware. Khouzani et al. [15] searched the propagation of malware in a battery-constrained mobile device, considering the fact that malware can control the rate of killing the infectives and the scanning rate of the infectives. The maximum damage caused by the malware was quantified with the optimal control theory, through which the network damage can be minimized by adjusting the relevant parameters.In addition to successful defense mechanisms to defend against malware threat, another effective defense scheme should consider both the limited resources and the dynamic characteristic of network. In recent years, game theory has been used to solve the decision making between the IoT devices and attackers [17, 18]. Game theory provides a mathematical method for the problems that different players compete with each other or with contradictory goals. Similarly, an effective security scheme in edge computing-based IoT depends not only on the successful defense strategies but also on the attackers’ behaviors.Spyridopoulos et al. [19] proposed a game-based security model to solve the malware dissemination prevention problem and analyzed the optimal defense strategy for the defender to minimize the damage of malware and the security cost with the optimal strategy. Quang et al. [20] modeled the problem of defending against attackers in IoT networks as a Bayesian game of incomplete information and showed that there was a threshold for the frequency of active attackers. Liao et al. [21] designed a zero-sum stochastic game to analyze the effect of malware in IoT and obtained the optimal defense strategy by the feedback Nash equilibrium solutions for the game model. Sedjelmaci et al. [22] presented a game-based detection technology for IoT device, which can not only activate the anomaly detection technology but also balance the energy consumption. Kaur et al. [23] proposed a stochastic game net security model, which combined the advantages of game theory and stochastic Petri nets. Shen et al. [24] proposed a multistage privacy-preserved game model for malware detection in fog-cloud-based IoT networks. In [24], the optimal detection strategy was attained under the consideration of privacy leakage of IoT devices, and the proposed detection scheme overcame the problem of limited resources of IoT devices.Nevertheless, none of the above research considered the stochastic characteristic of edge network.. In this paper, we introduce an infinite-horizon stochastic differential game to analyze the malware propagation problem in edge computing-based IoT, in which the stochastic characteristic of edge networks is considered.


## 3. System Model

In this section, we will use the infinite-horizon stochastic differential game to model the malware propagation under the dynamic interaction between the attackers and defenders. An infinite-horizon stochastic differential game involves anm-dimensional vector-valued stochastic differential equation(1)dxs=fxs,u1s,…,unsds+gxsdBs,xt0=x0,which describes the evolution of the state and n objective functions(2)maxuiEt0∫t0∞hixs,u1s,…,unsexp−rs−t0ds,fori=1,…,n,where Et0· denotes the expectation operation taken at time t0, gxs is an m×l matrix, Bs is an l-dimensional Brownian motion, and the initial state x0 is given in [25].We consider an edge computing-based IoT environment withN IoT devices. Figure 1 shows the architecture of edge computing-based IoT.Figure 1
The architecture of edge computing-based IoT.We first use the SEIRS model [26] to describe the spread of the malware in edge computing-based IoT. Like the SEIRS model, we divide the IoT devices into susceptible, exposed, infective, and recovered classes. The devices in the infectious state show that the device has been infected by the malware and the susceptible device is prone to be infected, but not infected. The exposed IoT device shows that it has been infected but not yet infectious, and the device in the recovery state represents that it has been immune to malicious attacks. We use St, Et, It, and Rt to denote the number of them at time t∈t0,∞, respectively (that is, St+Et+It+Rt=N).Letβ denote the rate of transmitting malware between a susceptible and an infectious IoT device, σ denote the rate of exposed IoT devices becoming infectious, γ denote the rate of infectious IoT devices becoming recovered, ξ denote recovered IoT devices becoming susceptible, u0t denote the number of IoT devices from susceptible to exposed caused by the attacker strategy at time t∈t0,∞, and u1t denote the number of IoT devices from infectious to recovered caused by the defender strategy at time t∈t0,∞. As shown in Figure 2, due to the dynamic interaction between the attackers and defenders, the spread of the malware in edge computing-based IoT can be described as the following differential equations:(3)dStdt=−βStItN+ξRt−u0t,dEtdt=βStItN−σEt+u0t,dItdt=σEt−γIt−u1t,dRtdt=γIt−ξRt+u1t.Figure 2
The spread of the malware.In edge computing-based IoT, the parametersβ, σ, γ, and ξ also may fluctuate because of the effect of stochastic fluctuations of the edge network on the malware propagation. To characterize the fluctuation of the parameters β, σ, γ, and ξ, we work on a complete probability space Ω,F,Ftt≥0,ℙ with a filtration Ftt≥0 satisfying the usual conditions [27]. By the central limit theorem, the fluctuation of the parameters β, σ, γ, and ξ follows a normal distribution. Then, we may replace the parameters β, σ, γ, and ξ by β⟶β+δ1dB1t/dt, σ⟶σ+δ2dB2t/dt, γ⟶γ+δ3dB3t/dt, and ξ⟶ξ+δ4dB4t/dt, respectively, where Bit is standard Brownian motion defined on the complete probability space with Bit0=0 and δi is a positive constant describing the intensity of the fluctuation for i=1,2,3,4. Thus, the differential equations (3) can be rewritten as the following It ô stochastic differential equations:(4)dSt=−βStItN+ξRt−u0t−δ1StItdB1tN+δ4RtdB4t,dEt=βStItN−σEt+u0t+δ1StItdB1tN−δ2EtdB2t,dIt=σEt−γIt−u1t+δ2EtdB2t−δ3ItdB3t,dRt=γIt−ξRt+u1t+δ3ItdB3t−δ4RtdB4t.As mentioned in Section1, in edge computing-based IoT, attackers want to make malware infect more IoT devices to gain illegal gains using the attack strategy, while defenders want to reduce the damage caused by IoT devices infected with malware using the defense strategy. More accurately, the aim of the attackers includes maximizing the number of infectious IoT devices and the number of IoT devices from susceptible to exposed and reducing the payoff of the attack strategy; the aim of the defenders includes minimizing the number of infectious IoT devices and the number of IoT devices from susceptible to exposed and reducing the payoff of the defense strategy. Inspired by Alpcan and Başar [28], the payoff of the attack strategy and the defense strategy can be described as u0t2/2c0 and u1t2/2c1, respectively, where c0 and c1 are positive constants. Thus, the objective functions of the attackers and defenders can be formulated as(5)maxu0Et0∫t0∞a0It+b0StIt−u0t22c0exp−rt−t0dt,maxu1Et0∫t0∞−a1It−b1StIt−u1t22c1exp−rt−t0dt,where a0 and a1 are positive constants, b0 and b1 are constants, and r is the discount factor. Note that a0 represents the benefits of each infectious IoT devices to the defenders while a1 represents the losses of each infectious IoT devices to the defenders; c0 describes that the payoff of the attack strategy is proportional to the number of IoT devices from susceptible to exposed caused by the attack strategy while c1 describes that the payoff of the defense strategy is proportional to the number of IoT devices from infectious to recovered caused by the attack strategy.In summary, the malware propagation under the dynamic interaction between the attackers and defenders can be formulated as the infinite-horizon stochastic differential game:(6)maxuiEt0∫t0∞hixs,u0s,u1sexp−rs−t0ds,fori=0,1,subject to the stochastic dynamics(7)dxs=fxs,u0s,u1sds+gxsdBs,xt0=x0,where(8)xs=Ss,Es,Is,RsT,Bs=B1s,B2s,B3s,B4sT,(9)h0xs,u0s,u0s=a0Is+b0SsIs−u0s22c0,h1xs,u1s,u1s=−a1Is−b1SsIs−u1s22c1,(10)fxs,u0s,u1s=−βSsIsN+ξRs−u0sβSsIsN−σEs+u0sσEs−γIs−u1sγIs−ξRs+u1s,(11)gxs=−α100α4α1−α2000α2−α3000α3−α4,for α1=δ1SsIs/N, α2=δ2Es, α3=δ3Is, and α4=δ4Rs.


## 4. Nash Equilibrium Solution

In this section, we will discuss the feedback Nash equilibrium solutions for game 6-7 to obtain the optimal strategies for the defender and attackers. Each participant is assumed to be rational and the decision making of each participant depends on their own objective functions in this game. The feedback Nash equilibrium solutions for game 1-2 can be characterized by the following theorem [25].Theorem 1.
Ann-tuple of strategies ui∗=ϕi∗x:i∈N provides a feedback Nash equilibrium solution to game 1-2 if there exist continuously twice differentiable functions Wix:Rm⟶R, i∈N, satisfying the following set of partial differential equations:(12)rWix−12∑h,k=1mΩhkxWxhxkix=maxuihix,ϕ1∗x,…,ϕi−1∗x,uix,ϕi+1∗x,…,ϕn∗x+Wxixfx,ϕ1∗x,…,ϕi−1∗x,uix,ϕi+1∗x,…,ϕn∗x=hix,ϕ1∗x,…,ϕn∗x+Wxixfx,ϕ1∗x,…,ϕn∗x,fori=1,…,n,where Ωxs=gxsgxsT denotes the covariance matrix with its element in row h and column k denoted by Ωhkxs.To obtain the feedback Nash equilibrium solutions for game 6-7, we consider the alternative problem(13)maxuiEt0∫t0∞hixs,u0s,u1sexp−rs−t0ds,i=0,1,subject to the stochastic dynamics(14)dxs=fxs,u0s,u1sds+gxsdBs,xt=xt,where hixs,u0s,u1s for i=0,1, fxs,u0s,u1s, and gxs are given by equations (9)–(11).Invoking Theorem1, we obtain two feedback strategies u0∗=ϕ0∗x and u1∗=ϕ1∗x constituting the feedback Nash equilibrium solutions for game 13-14, if there exist continuously twice differentiable functions Wix:R4⟶R, i=0,1, satisfying the following set of partial differential equations:(15)rW0x−12∑h,k=14ΩhkxWxhxk0x=maxu0h0x,u0,ϕ1∗+Wx0xfx,u0,ϕ1∗,=h0x,ϕ0∗,ϕ1∗+Wx0xfx,ϕ0∗,ϕ1∗,rW1x−12∑h,k=14ΩhkxWxhxk1x=maxu1h1x,u0,ϕ1∗+Wx1xfx,ϕ0∗,u1,=h1x,ϕ0∗,ϕ1∗+Wx0xfx,ϕ0∗,ϕ1∗,where Ωxs=gxsgxsT is given by(16)α12+α32−α120−α42−α12α12+α22−α2200−α22α22+α32−α32−α420−α32α32+α42,for α1=δ1SsIs/N, α2=δ2Es, α3=δ3Is, and α4=δ4Rs.Applying the maximization operator in equation (15), we obtain the feedback Nash equilibrium solutions for game 13-14:(17)ϕ0∗x=c0WE0x−WS0x,ϕ1∗x=c1WR1x−WI1x.Substitutingϕ1∗x and ϕ2∗x in equations (17) into (15), we obtain the following proposition upon solving equation (15).Proposition 1.
The set of partial differential equation (15) admits a solution:(18)W0x=a0σEσ−rγ−r+a0Iγ−r−a02c0σ22rσ−r2γ−r2−a0a1c1rγ−r2,W1x=a1σEσ−rγ−r+a1Iγ−r+a0a1c0σ2rσ−r2γ−r2+a12c12rγ−r2,where (19)a0βσ+b0σ−rγ−r=0,a1βσ+b1σ−rγ−r=0.Proof.
By equation (18), we have(20)Wx0x=0,a0σσ−rγ−r,a0γ−r,0,Wx1x=0,a1σσ−rγ−r,a1γ−r,0,Wxhxk0x=Wxhxk1x=0,for1≤h,k≤4.
Combining equations (17) and (20), we obtain(21)ϕ0∗x=a0c0σσ−rγ−r,ϕ1∗x=a1c1γ−r.
Substituting equations (20) and (21) into equation (15), we obtain(22)a0βσ+b0σ−rγ−r=0,a1βσ+b1σ−rγ−r=0.
Thus, this proposition holds.According to the proof of Proposition1, the feedback Nash equilibrium solutions for game 6-7 is given by equation (21). In other words, the optimal strategies for the defenders and attackers are derived. The optimal state for game 6-7 describes the propagation of the malware in edge computing-based IoT when both the attackers and defenders adopt the optimal strategy. Substituting (21) into (7), we obtain the optimal state for game 6-7, i.e.,(23)dSt=−βStItN+ξRt−a0c0σσ−rγ−r−δ1StItdB1tN+δ4RtdB4t,dEt=βStItN−σEt+a0c0σσ−rγ−r+δ1StItdB1tN−δ2EtdB2t,dIt=σEt−γIt−a1c1γ−r+δ2EtdB2t−δ3ItdB3t,dRt=γIt−ξRt+a1c1γ−r+δ3ItdB3t−δ4RtdB4t,St0=S0,Et0=E0,It0=I0,Rt0=R0.


## 5. Numerical Simulations

In this section, we discuss the implementation of the stochastic game algorithm which is given in Table1 and analyze the proposed infinite-horizon stochastic differential game model by simulations.Algorithm 1: The stochastic game algorithm for the defenders and attackers (the optimal strategies for the defenders and attackers).
Input:xST,Et,It,Rt;Output: optimal state trajectoryxs and optimal strategies ϕ0∗x,ϕ1∗x;(1)
Initialize game parametersN,δ1,δ2,δ3,δ4,a0,b0,c0,a1b1,c1, and r;(2)
Set the rate parametersβ,σ,γ,ξ;(3)
Fort = 0 to T;(4)
If the infected nodesxIt>0(5)
Response the defense mechanism;(6)
Calculate the optimal state trajectory in equation (4);(7)
Calculate the optimal strategies in equation (21);(8)
Analyze the influence of parametersa0,c0,a1,c1 on the optimal strategies;(9)
Else(10)
t = t + 1;(11)
End if(12)
Updatexs and store it;(13)
End for;(14)
Returnxx,ϕ0∗x,ϕ1∗x.The algorithm is divided into two parts. One is the “feedback Nash equilibrium of defenders” part, which is used to calculate the optimal defense strategies during the attacks. The other is the “feedback Nash equilibrium of attackers,” which is used to calculate the optimal attack strategies. The time and space complexity is O (n), respectively, because the proposed algorithm should be solved in a finite time horizon [0, T] for all the attackers and defenders. Besides, all the functions need to be invoked at each time.We assume that the number of IoT devices isN=10000 and consider the time horizon to be T=20 minutes. The rest of the related simulation parameters are shown in Table 1.Table 1
Values of parameters.
βσγξ0.2490.1470.8700.012δ1δ2δ3δ40.1870.1100.6520.009a0b0c0a10.9930.3480.4370.295b1c1rN3.5540.3510.34610000Figure3 shows the optimal trajectory xt with time t. It can be seen that the number of the susceptible devices is rapidly decreased with the time variation, while the number of the infected devices increases at the beginning and then gradually decreases to zero. The dynamic evolution of the number of the exposed devices is similar to that of the infected devices. In addition, the number of the recovered devices is increased with the time variation. It means that defenders can respond their defense mechanism against attackers, which is consistent with the practical network environment, where the infected devices are always recovered and the exposed devices always exist.Figure 3
The state trajectoryxt.Based on equation (21), we discuss the optimal strategies of the attackers and defenders in Figure 4. As shown in the results, the variation of the optimal defense strategy is increased with the time variation while the variation of optimal strategy of the attackers is decreased and then tends to be stable.Figure 4
The optimal strategy with given parameters.Figure5 shows the change of the optimal strategy of the attackers with a0 and c0 while Figure 6 shows the change of the optimal strategy of the defenders with a1 and c1, where a0 represents the benefits of each infectious IoT device to the defenders while a1 represents the losses of each infectious IoT devices to the defenders; c0 describes that the payoff of the attack strategy is proportional to the number of IoT devices from susceptible to exposed caused by the attack strategy while c1 describes that the payoff of the defense strategy is proportional to the number of IoT devices from infectious to recovered caused by the attack strategy. It can It can be seen that the level of the optimal strategy of attackers increases with the increase of a0 and c0 while the level of the optimal strategy of defenders increases with the increase of a1 and c1. As shown in the results, the level of the optimal strategy of defenders grows faster.Figure 5
The change of the optimal strategy of the attackers witha0 and c0.Figure 6
The change of the optimal strategy of the defenders witha1 and c1.The comparison of proposed model with the existing model [23] is shown in Figure 7; it can be seen that the number of infected devices in both models is rapidly increased with the time variation and then decreased. The number of infected devices in the proposed model is less than that of the comparative model, which means that the proposed security strategy is more effective and more suitable for IoT environment.Figure 7
The comparison between the proposed model and the comparative model.


## 6. Conclusions

In this paper, we have proposed an infinite-horizon stochastic differential game model to study the malware propagation problem under the dynamic interaction between the attackers and defenders in the edge computing-based IoT environment that is composed byN IoT devices and to maximize the profit for both the attackers and defenders. In terms of model construction, we assumed that the states of IoT devices were infected, susceptible, exposed, and recovered and considered the effect of the stochastic fluctuations of the network on the state of the IoT devices. By solving the feedback Nash equilibrium solutions for our proposed game model, we obtained the optimal strategies for both the attackers and defenders. Based on the simulations results, it can be seen that the proposed model can prevent the malware propagation in edge computing-based IoT. In future work, we will apply this model to other resource-constrained environments.



---

*Source: 8896715-2021-02-23.xml*