# Privacy-Aware Online Task Offloading for Mobile-Edge Computing

Authors: Dali Zhu; Ting Li; Haitao Liu; Jiyan Sun; Liru Geng; Yinlong Liu
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

Mobile edge computing (MEC) has been envisaged as one of the most promising technologies in the fifth generation (5G) mobile networks. It allows mobile devices to offload their computation-demanding and latency-critical tasks to the resource-rich MEC servers. Accordingly, MEC can significantly improve the latency performance and reduce energy consumption for mobile devices. Nonetheless, privacy leakage may occur during the task offloading process. Most existing works ignored these issues or just investigated the system-level solution for MEC. Privacy-aware and user-level task offloading optimization problems receive much less attention. In order to tackle these challenges, a privacy-preserving and device-managed task offloading scheme is proposed in this paper for MEC. This scheme can achieve near-optimal latency and energy performance while protecting the location privacy and usage pattern privacy of users. Firstly, we formulate the joint optimization problem of task offloading and privacy preservation as a semiparametric contextual multi-armed bandit (MAB) problem, which has a relaxed reward model. Then, we propose a privacy-aware online task offloading (PAOTO) algorithm based on the transformed Thompson sampling (TS) architecture, through which we can (1) receive the best possible delay and energy consumption performance, (2) achieve the goal of preserving privacy, and (3) obtain an online device-managed task offloading policy without requiring any system-level information. Simulation results demonstrate that the proposed scheme outperforms the existing methods in terms of minimizing the system cost and preserving the privacy of users.


---

## Body

## 1. Introduction

In the recent years, with the advent of 5G network, as well as the fast popularization of mobile devices, a myriad of new applications is emerging, such as augmented reality (AR)/virtual reality (VR) [1, 2], online 3D games [3], and connected cars [4]. Specifically, the recent Cisco Annual Internet Report expects that the number of global mobile devices will grow from 8.8 billion in 2018 to 13.1 billion by 2023 and the vast majority of mobile data traffic (99%) will originate from these mobile devices [5]. However, due to their limited computing units and battery energy, mobile devices struggle to resist to such traffic explosion and become unable to meet the stringent requirements of computing-demanding and latency-sensitive applications.To get rid of such limitations, a novel paradigm of mobile edge computing (MEC) [6] is proposed as an extension of remote-centralized clouds [7] by the European Telecommunications Standards Institute (ETSI). The key idea beneath MEC is to deploy computing and storage resources from the core network to the radio access network (RAN) in the fifth generation (5G) networks [8]. In such computing paradigm, computation tasks will be offloaded to nearby MEC servers via wireless channels by mobile devices, which can meet the requirements of computing intensive applications and achieve ultrashort processing latency.Despite the benefits, MEC still has shortcomings in terms of security and privacy leakage [9]. For example, the location privacy and usage pattern privacy problem [10] are investigated in this paper, which are related to the MEC task offloading feature. Intuitively, when a mobile device is to obtain optimal offloading performance, it tends to offload all its tasks to the MEC server. Accordingly, an honest-but-curious MEC server can infer the location privacy and usage pattern privacy of users who are privacy sensitive, which may prevent these users from accessing the MEC system if not properly addressed. Although these two privacy issues have been extensively studied in other fields, one challenge still needs to be addressed in MEC systems, which is how to protect both the location privacy and usage pattern privacy while minimizing the delay and energy consumption cost.Most existing task offloading schemes that want to achieve optimal system performance, such as [11, 12], largely ignore these privacy problems. And current privacy-preserving techniques of cloud computing are not always applicable for the MEC system, such as the works in [13, 14]. Therefore, the more challenging problem is how to prevent unintentional leakage of user’s privacy while still maintaining the optimal delay and energy consumption performance. The most related works probably are [10, 15], which studied the optimization of delay and energy consumption cost while considering both location privacy and usage pattern privacy. The former scheme formulates this problem as a constrained Markov decision process (CMDP), and the latter one applies a Dyan-Q architecture based on the CMDP to achieve a better privacy-aware offloading policy. However, both of them are system-level solutions. They rely on the assumption of the wireless channel power gain that is formulated as a Markov chain model, in which some system-side information should be known in advance. Such assumption is not relevant to infrastructure-free scenarios, such as individual combat in military scenarios, forest fire rescue [16], and heterogeneous IoT [17] which are more applicable user-level schemes.In order to minimize system cost (e.g., latency and energy consumption) and protect user’s privacy without requiring any system-level information as a prior knowledge, we propose a device-level and privacy-preserving task offloading scheme for the MEC system. This scheme is based on a semiparametric contextual multi-armed bandit (MAB) problem, which can address the trade-offs inherent in the sequential decision problem and overcome the challenges of lacking system-side information. To the best of our knowledge, this user-level scheme is the first to be proposed to solve the privacy problem of MEC. The location privacy and usage pattern privacy of users will be preserved in this paper. An online-learning algorithm will be proposed to make adaptive task offloading decisions under dynamic network environment. The main contributions of this paper are summarized as follows:(1)
MAB-based problem modelling. We study a joint optimization problem of task offloading and privacy preservation in the MEC system. And then, this problem is transformed as a semiparametric contextual MAB problem to overcome the challenge of unknown network dynamics, which can utilize the contextual feature vector to describe user-side information for decoupling the time dependency(2)
Privacy-aware optimal offloading decision. We propose a privacy-aware online learning algorithm, called PAOTO (privacy-aware online task offloading), to make device-level task offloading decisions while protecting user’s location privacy and usage pattern privacy. By utilizing the transformed Thompson sampling (TS) architecture, we can make adaptive task offloading decisions at the user-side perspective(3)
Extensive simulation-based performance evaluation. We carry out simulations to demonstrate the effectiveness of the proposed algorithm. The results show that the PAOTO algorithm performs close-to-optimal and far better than the newly proposed Dyna-Q algorithm in [15].The remaining parts of this paper are organized as follows. In Section2, we discuss the motivation and related works. We will formally describe the system model and problem formulation in Section 3. Next, we present the algorithm design and simulation evaluation in Sections 4 and 5, respectively. Finally, we draw some conclusions and highlight the direction for future work in Section 6.


## 2. Motivation and Related Work

### 2.1. Motivation

The problem of determining the privacy-aware and user-level task offloading decisions for mobile devices requires solving two important challenges: (1) how to best prevent leakage of user’s privacy while still maintaining the optimal delay and energy consumption performance and (2) how to design a task offloading policy that can online determine the optimal execution platform (i.e., local processing unit, MEC servers, or buffer) for users at the user-side perspective?To address the first challenge, we propose a privacy metric to jointly quantify the location and usage pattern privacy and utilize a semiparametric MAB to incorporate the privacy metric into the performance model. This can strike a balance between the privacy-preserving level and system cost (e.g., processing latency and energy consumption cost). Previous works require system-level information to design an optimal task offloading strategy, but this is not applicable to infrastructure-free scenarios (e.g., individual combat in military scenarios, forest fire rescue and heterogeneous IoT). We address this second challenge by utilizing the contextual feature vector in the contextual MAB model to describe user-side information and applying the Thompson sampling (TS) algorithm to estimate and learn the performance model based on the contextual information.


#### 2.1.1. Characterizing Privacy Metric

Recently, MEC has been increasing in popularity but issues relating to the security and privacy in the MEC system still has shortcomings. On one hand, some security issues such as authentication, private data storage, and intrusion detection have received attentions but these security issues are inherited from the conventional cloud computing framework and are less relevant to the key technologies in the MEC system. On the other hand, based on the simulation results and considered setting in [10], we can find that the privacy problems relating to MEC unique wireless task offloading technology remains less explored, which are user’s location privacy and usage pattern privacy.According to [10], the offloading pattern can be observed as a mobile device may offload all its tasks to the MEC server when the wireless channel state is good, while it will not offload any tasks otherwise. Accordingly, the honest but curious MEC server (it may be controlled by adversary) can be based on the offloading pattern and historical statistics to obtain the number of tasks offloaded in each period. Hence, the wireless channel condition (it is only assumed as good or bad and can be extended to the multistate case) and user’s actual usage pattern can be inferred by adversary. Specifically, the wireless channel condition is highly related to the distance between the user and the MEC server. If a mobile user communicates with multiple MEC servers, the location privacy may be inferred by these MEC servers based on the surveillance of the wireless channel state. The user’s location privacy is leaked. Moreover, when someone’s office is near the AP, its wireless channel state may be always good and it may always offload all its tasks to the MEC server. Thus, the adversary can obtain the total number of tasks offloaded that is determined by the user’s actual device usage pattern. The user’s usage pattern privacy may be leaked.Particularly, it is very important for privacy-sensitive users to solve the problem of leaking location privacy and usage pattern privacy that are induced into the unique wireless task offloading feature in the MEC system. If they are note properly addressed, it may prevent these privacy-sensitive users from accessing the MEC system. Significantly, although these two privacy problems have already been studied in other system, protecting user’s location privacy and usage pattern privacy while minimizing delay and energy consumption cost in the MEC system still poses a critical challenge.Therefore, to address this challenge, it is desirable to design a metric to jointly quantify the location and usage pattern privacy. Next, we formulate the task offloading and privacy preservation problem as a contextual MAB problem with a semiparametric reward model based on processing latency, energy consumption cost, and this privacy metric. This is aiming to strike a balance between the privacy-preserving level and system cost. That is, according to problem formulation and proposed algorithm, we can obtain the optimal delay and energy consumption performance while protecting user’s location and usage pattern privacy, which can be seen in Sections3 and 4.


#### 2.1.2. User-Level Task Offloading

With mobile data traffic growing explosively, the mobile devices with limited resources cannot meet the stringent requirements of computing-demanding and latency-sensitive applications. Therefore, designing a desirable task offloading strategy of the MEC system has attracted tremendous attention in the industry and academia. This strategy can determine the optimal task execution platform for the user, executing in the local processing unit, offloading to MEC server or queueing in the buffer.Many previous works (e.g., [10, 15]) on task offloading generally assume that the system-side information is always available. Such assumption is more applicable to the infrastructure-assisted edge computing scenarios where the infrastructure (e.g., an access point or base station) is available for obtaining system-side information in advance [18]. However, some infrastructure-free scenarios, such as individual combat in military scenarios, forest fire rescue and heterogeneous IoT, are not suitable for previous system-level solutions, because these mobile devices in infrastructure-free scenarios are operating in a scattered manner and the system-side network information is missing for them. Especially, if they want to explore system-level information in advance, it may cause additional system cost, such as scarce bandwidth usage and additional energy consumption cost.In this case, it is desirable to design a user-level task offloading strategy for overcoming the challenge of lacking the system-side information. In response to the challenge that the system-level information may not be readily available in some infrastructure-free scenarios, we propose an online task offloading scheme at the user perspective. In this scheme, the user-side information will be described as contextual feature vector and the Thompson sampling (TS) algorithm will be applied to estimate and learn the performance model based on the contextual information. It can adaptively decide where to execute the offloaded task for the mobile user without any system level information. It can be seen in Section4 for details.


### 2.2. Related Work

In recent years, the task offloading strategies have attracted significant efforts to minimize total delay and energy consumption cost in MEC systems. For example, Xu et al. proposed an online algorithm based on Lyapunov optimization and Gibbs sampling, which jointly optimized dynamic service caching and task offloading to reduce computation latency while keeping energy consumption low [19]. Wei et al. studied the problem of task offloading and channel resource allocation based on MEC in 5G ultra-dense networks (UDN) [20]. The authors formulated task offloading as an integer nonlinear programming problem and proposed an efficient task offloading and channel resource allocation scheme based on differential evolution algorithm. Dab et al. proposed a joint radio resource allocation and task assignment strategy based on a Q-learning algorithm to minimize the energy consumption cost under both the latency and device’s computation resource constraints [21]. Li and Cai discussed the incentive mechanism design for collaborative task offloading in the MEC network [22]. They proposed an online truthful mechanism integrating computation and communication resource allocation to address social welfare maximization problem by considering each task’s specific requirements in terms of data size, delay, and preference. However, none of the works mentioned above considered user’s privacy issues.There are a few works considering both task offloading and privacy preservation. For example, He et al. identified a new privacy vulnerability caused by the wireless offloading feature of MEC-enabled IoT. To address this vulnerability, the authors developed an offloading strategy for MEC-enabled IoT, which can learn a good offloading strategy while protecting the devices’ location privacy [23]. However, the extra prior information was required. In [24], Zhang et al. proposed a strategy that can achieve an efficient task scheduling policy on edge while ensuring privacy. In [25], Zhou proposed a novel context-aware task allocation framework for mobile crowdsensing in the scenario of edge computing. The task allocation was performed in both the cloud computing layer and the edge computing layer. In the cloud layer, authors proposed a privacy-preserving and contextual online learning algorithm to manage the participants’ reputation. But this scheme was implemented at the system level and required a priori network information.Besides, He et al. identified location privacy and usage pattern privacy issues, which are induced by the wireless task offloading feature of MEC [10]. To address these privacy issues, authors proposed a constrained Markov decision process- (CMDP-) based privacy-aware task offloading scheduling algorithm to achieve the best possible system performance while protecting user’s privacy. Min and Wan proposed a reinforcement learning- (RL-) based privacy-aware offloading scheme, which enables the IoT device to make the task offloading decisions and protect both the user location privacy and the usage pattern privacy for the MEC system [15]. Nevertheless, both the works were implemented at the system level. They all need to explore system-level information in advance, which is difficult to obtain and may cause additional system cost, such as scarce bandwidth usage and energy consumption of network devices.In general, none of the aforementioned works consider both task offloading and privacy protection problem at the user level. These aforementioned studies mainly face two challenges. First, they only consider the simple task offloading strategies for minimizing total delay and energy consumption cost in MEC systems. However, the privacy issues related to the task offloading pattern were ignored in their works, which may be very important for the privacy-sensitive users. Second, some works considering both task offloading and privacy preservation were all implemented at the system level. That is, these works generally assume that the system-side information is always available. However, this is applicable for the infrastructure-assisted scenarios where the infrastructure (e.g., an access point (AP)) is available for obtaining system-side information in advance. For the infrastructure-free scenarios (e.g., individual combat in military scenarios, forest fire rescue, and heterogeneous IoT), these mobile devices will operate in a decentralized manner and the system-side information is difficult to obtain and may cause additional system cost.To conquer this challenge, we propose a novel privacy-aware task offloading scheme based on an online learning algorithm that just requires device-level information and it can achieve the best possible system performance while protecting the user’s privacy.


## 2.1. Motivation

The problem of determining the privacy-aware and user-level task offloading decisions for mobile devices requires solving two important challenges: (1) how to best prevent leakage of user’s privacy while still maintaining the optimal delay and energy consumption performance and (2) how to design a task offloading policy that can online determine the optimal execution platform (i.e., local processing unit, MEC servers, or buffer) for users at the user-side perspective?To address the first challenge, we propose a privacy metric to jointly quantify the location and usage pattern privacy and utilize a semiparametric MAB to incorporate the privacy metric into the performance model. This can strike a balance between the privacy-preserving level and system cost (e.g., processing latency and energy consumption cost). Previous works require system-level information to design an optimal task offloading strategy, but this is not applicable to infrastructure-free scenarios (e.g., individual combat in military scenarios, forest fire rescue and heterogeneous IoT). We address this second challenge by utilizing the contextual feature vector in the contextual MAB model to describe user-side information and applying the Thompson sampling (TS) algorithm to estimate and learn the performance model based on the contextual information.


### 2.1.1. Characterizing Privacy Metric

Recently, MEC has been increasing in popularity but issues relating to the security and privacy in the MEC system still has shortcomings. On one hand, some security issues such as authentication, private data storage, and intrusion detection have received attentions but these security issues are inherited from the conventional cloud computing framework and are less relevant to the key technologies in the MEC system. On the other hand, based on the simulation results and considered setting in [10], we can find that the privacy problems relating to MEC unique wireless task offloading technology remains less explored, which are user’s location privacy and usage pattern privacy.According to [10], the offloading pattern can be observed as a mobile device may offload all its tasks to the MEC server when the wireless channel state is good, while it will not offload any tasks otherwise. Accordingly, the honest but curious MEC server (it may be controlled by adversary) can be based on the offloading pattern and historical statistics to obtain the number of tasks offloaded in each period. Hence, the wireless channel condition (it is only assumed as good or bad and can be extended to the multistate case) and user’s actual usage pattern can be inferred by adversary. Specifically, the wireless channel condition is highly related to the distance between the user and the MEC server. If a mobile user communicates with multiple MEC servers, the location privacy may be inferred by these MEC servers based on the surveillance of the wireless channel state. The user’s location privacy is leaked. Moreover, when someone’s office is near the AP, its wireless channel state may be always good and it may always offload all its tasks to the MEC server. Thus, the adversary can obtain the total number of tasks offloaded that is determined by the user’s actual device usage pattern. The user’s usage pattern privacy may be leaked.Particularly, it is very important for privacy-sensitive users to solve the problem of leaking location privacy and usage pattern privacy that are induced into the unique wireless task offloading feature in the MEC system. If they are note properly addressed, it may prevent these privacy-sensitive users from accessing the MEC system. Significantly, although these two privacy problems have already been studied in other system, protecting user’s location privacy and usage pattern privacy while minimizing delay and energy consumption cost in the MEC system still poses a critical challenge.Therefore, to address this challenge, it is desirable to design a metric to jointly quantify the location and usage pattern privacy. Next, we formulate the task offloading and privacy preservation problem as a contextual MAB problem with a semiparametric reward model based on processing latency, energy consumption cost, and this privacy metric. This is aiming to strike a balance between the privacy-preserving level and system cost. That is, according to problem formulation and proposed algorithm, we can obtain the optimal delay and energy consumption performance while protecting user’s location and usage pattern privacy, which can be seen in Sections3 and 4.


### 2.1.2. User-Level Task Offloading

With mobile data traffic growing explosively, the mobile devices with limited resources cannot meet the stringent requirements of computing-demanding and latency-sensitive applications. Therefore, designing a desirable task offloading strategy of the MEC system has attracted tremendous attention in the industry and academia. This strategy can determine the optimal task execution platform for the user, executing in the local processing unit, offloading to MEC server or queueing in the buffer.Many previous works (e.g., [10, 15]) on task offloading generally assume that the system-side information is always available. Such assumption is more applicable to the infrastructure-assisted edge computing scenarios where the infrastructure (e.g., an access point or base station) is available for obtaining system-side information in advance [18]. However, some infrastructure-free scenarios, such as individual combat in military scenarios, forest fire rescue and heterogeneous IoT, are not suitable for previous system-level solutions, because these mobile devices in infrastructure-free scenarios are operating in a scattered manner and the system-side network information is missing for them. Especially, if they want to explore system-level information in advance, it may cause additional system cost, such as scarce bandwidth usage and additional energy consumption cost.In this case, it is desirable to design a user-level task offloading strategy for overcoming the challenge of lacking the system-side information. In response to the challenge that the system-level information may not be readily available in some infrastructure-free scenarios, we propose an online task offloading scheme at the user perspective. In this scheme, the user-side information will be described as contextual feature vector and the Thompson sampling (TS) algorithm will be applied to estimate and learn the performance model based on the contextual information. It can adaptively decide where to execute the offloaded task for the mobile user without any system level information. It can be seen in Section4 for details.


## 2.1.1. Characterizing Privacy Metric

Recently, MEC has been increasing in popularity but issues relating to the security and privacy in the MEC system still has shortcomings. On one hand, some security issues such as authentication, private data storage, and intrusion detection have received attentions but these security issues are inherited from the conventional cloud computing framework and are less relevant to the key technologies in the MEC system. On the other hand, based on the simulation results and considered setting in [10], we can find that the privacy problems relating to MEC unique wireless task offloading technology remains less explored, which are user’s location privacy and usage pattern privacy.According to [10], the offloading pattern can be observed as a mobile device may offload all its tasks to the MEC server when the wireless channel state is good, while it will not offload any tasks otherwise. Accordingly, the honest but curious MEC server (it may be controlled by adversary) can be based on the offloading pattern and historical statistics to obtain the number of tasks offloaded in each period. Hence, the wireless channel condition (it is only assumed as good or bad and can be extended to the multistate case) and user’s actual usage pattern can be inferred by adversary. Specifically, the wireless channel condition is highly related to the distance between the user and the MEC server. If a mobile user communicates with multiple MEC servers, the location privacy may be inferred by these MEC servers based on the surveillance of the wireless channel state. The user’s location privacy is leaked. Moreover, when someone’s office is near the AP, its wireless channel state may be always good and it may always offload all its tasks to the MEC server. Thus, the adversary can obtain the total number of tasks offloaded that is determined by the user’s actual device usage pattern. The user’s usage pattern privacy may be leaked.Particularly, it is very important for privacy-sensitive users to solve the problem of leaking location privacy and usage pattern privacy that are induced into the unique wireless task offloading feature in the MEC system. If they are note properly addressed, it may prevent these privacy-sensitive users from accessing the MEC system. Significantly, although these two privacy problems have already been studied in other system, protecting user’s location privacy and usage pattern privacy while minimizing delay and energy consumption cost in the MEC system still poses a critical challenge.Therefore, to address this challenge, it is desirable to design a metric to jointly quantify the location and usage pattern privacy. Next, we formulate the task offloading and privacy preservation problem as a contextual MAB problem with a semiparametric reward model based on processing latency, energy consumption cost, and this privacy metric. This is aiming to strike a balance between the privacy-preserving level and system cost. That is, according to problem formulation and proposed algorithm, we can obtain the optimal delay and energy consumption performance while protecting user’s location and usage pattern privacy, which can be seen in Sections3 and 4.


## 2.1.2. User-Level Task Offloading

With mobile data traffic growing explosively, the mobile devices with limited resources cannot meet the stringent requirements of computing-demanding and latency-sensitive applications. Therefore, designing a desirable task offloading strategy of the MEC system has attracted tremendous attention in the industry and academia. This strategy can determine the optimal task execution platform for the user, executing in the local processing unit, offloading to MEC server or queueing in the buffer.Many previous works (e.g., [10, 15]) on task offloading generally assume that the system-side information is always available. Such assumption is more applicable to the infrastructure-assisted edge computing scenarios where the infrastructure (e.g., an access point or base station) is available for obtaining system-side information in advance [18]. However, some infrastructure-free scenarios, such as individual combat in military scenarios, forest fire rescue and heterogeneous IoT, are not suitable for previous system-level solutions, because these mobile devices in infrastructure-free scenarios are operating in a scattered manner and the system-side network information is missing for them. Especially, if they want to explore system-level information in advance, it may cause additional system cost, such as scarce bandwidth usage and additional energy consumption cost.In this case, it is desirable to design a user-level task offloading strategy for overcoming the challenge of lacking the system-side information. In response to the challenge that the system-level information may not be readily available in some infrastructure-free scenarios, we propose an online task offloading scheme at the user perspective. In this scheme, the user-side information will be described as contextual feature vector and the Thompson sampling (TS) algorithm will be applied to estimate and learn the performance model based on the contextual information. It can adaptively decide where to execute the offloaded task for the mobile user without any system level information. It can be seen in Section4 for details.


## 2.2. Related Work

In recent years, the task offloading strategies have attracted significant efforts to minimize total delay and energy consumption cost in MEC systems. For example, Xu et al. proposed an online algorithm based on Lyapunov optimization and Gibbs sampling, which jointly optimized dynamic service caching and task offloading to reduce computation latency while keeping energy consumption low [19]. Wei et al. studied the problem of task offloading and channel resource allocation based on MEC in 5G ultra-dense networks (UDN) [20]. The authors formulated task offloading as an integer nonlinear programming problem and proposed an efficient task offloading and channel resource allocation scheme based on differential evolution algorithm. Dab et al. proposed a joint radio resource allocation and task assignment strategy based on a Q-learning algorithm to minimize the energy consumption cost under both the latency and device’s computation resource constraints [21]. Li and Cai discussed the incentive mechanism design for collaborative task offloading in the MEC network [22]. They proposed an online truthful mechanism integrating computation and communication resource allocation to address social welfare maximization problem by considering each task’s specific requirements in terms of data size, delay, and preference. However, none of the works mentioned above considered user’s privacy issues.There are a few works considering both task offloading and privacy preservation. For example, He et al. identified a new privacy vulnerability caused by the wireless offloading feature of MEC-enabled IoT. To address this vulnerability, the authors developed an offloading strategy for MEC-enabled IoT, which can learn a good offloading strategy while protecting the devices’ location privacy [23]. However, the extra prior information was required. In [24], Zhang et al. proposed a strategy that can achieve an efficient task scheduling policy on edge while ensuring privacy. In [25], Zhou proposed a novel context-aware task allocation framework for mobile crowdsensing in the scenario of edge computing. The task allocation was performed in both the cloud computing layer and the edge computing layer. In the cloud layer, authors proposed a privacy-preserving and contextual online learning algorithm to manage the participants’ reputation. But this scheme was implemented at the system level and required a priori network information.Besides, He et al. identified location privacy and usage pattern privacy issues, which are induced by the wireless task offloading feature of MEC [10]. To address these privacy issues, authors proposed a constrained Markov decision process- (CMDP-) based privacy-aware task offloading scheduling algorithm to achieve the best possible system performance while protecting user’s privacy. Min and Wan proposed a reinforcement learning- (RL-) based privacy-aware offloading scheme, which enables the IoT device to make the task offloading decisions and protect both the user location privacy and the usage pattern privacy for the MEC system [15]. Nevertheless, both the works were implemented at the system level. They all need to explore system-level information in advance, which is difficult to obtain and may cause additional system cost, such as scarce bandwidth usage and energy consumption of network devices.In general, none of the aforementioned works consider both task offloading and privacy protection problem at the user level. These aforementioned studies mainly face two challenges. First, they only consider the simple task offloading strategies for minimizing total delay and energy consumption cost in MEC systems. However, the privacy issues related to the task offloading pattern were ignored in their works, which may be very important for the privacy-sensitive users. Second, some works considering both task offloading and privacy preservation were all implemented at the system level. That is, these works generally assume that the system-side information is always available. However, this is applicable for the infrastructure-assisted scenarios where the infrastructure (e.g., an access point (AP)) is available for obtaining system-side information in advance. For the infrastructure-free scenarios (e.g., individual combat in military scenarios, forest fire rescue, and heterogeneous IoT), these mobile devices will operate in a decentralized manner and the system-side information is difficult to obtain and may cause additional system cost.To conquer this challenge, we propose a novel privacy-aware task offloading scheme based on an online learning algorithm that just requires device-level information and it can achieve the best possible system performance while protecting the user’s privacy.


## 3. System Model and Problem Formulation

### 3.1. System Model

In this section, the task offloading model will be presented. As illustrated in Figure1, we consider a scenario in which the mobile user/device communicates with the MEC server through the access point (e.g., Wi-Fi or 5G base station) via the wireless channel. For ease of exposition, the bandwidth constraint of the wireless channel is not considered in this paper and we will consider it in the next work. The mobile device has computing-intensive computation tasks that are required to be completed as soon as possible. Due to its limited battery energy and computing capabilities, the mobile device can offload some computation tasks to the MEC server, which has powerful computing capabilities. As such, the mobile device has three ways to process these computation tasks, that is, computing in the local processing unit, offloading to the MEC server through the transmission unit, and queuing in the buffer for processing in the next time slot.Figure 1
An illustration of task offloading in the MEC system.Without loss of generality, we assume that the task offloading policies are made in a slotted structure and its timeline is discretized into time slotst∈T=1,2,⋯,T. At each time slot t, the mobile user will newly generate Nt computation tasks to the mobile device, denoted by a set of ℕ=1,2,⋯,Nmaxt (Nmaxt is the maximum possible number of generated tasks), which depends on the user’s usage pattern. And the bt∈B=1,2,⋯,bmax (with the maximum buffer size bmax) can be denoted as the number of tasks in the buffer at time slot t.A widely used three-parameter model [26] can be used to describe each task n, denoted by a set of Nt=1,2,⋯,Nt+bt. The three-parameter model consists of the input data size λn (bits), computation intensity δn (CPU cycles/bit), and maximum allowed latency τn (seconds). Whereupon, the computation demand for each task can be obtained by mn=λn∗δn (CPU cycles). In each time slot t, all the (Nt+bt) tasks (including the newly generated Nt tasks and the bt tasks in the buffer) will be either locally executed, buffered, or remotely offloaded according to the proposed task offloading policy. More specifically, the mobile device will explore the optimal offloading policy for each task.Based on [10], in order to minimize computing delay and energy consumption, the mobile device tends to offload all its tasks to the MEC server if the wireless channel state is good and processes all its tasks locally if the wireless channel state is bad. Under such circumstances, the user’s location and usage pattern privacy are easily spied by the attacker. Hence, the proposed task offloading policy in this paper takes the privacy preservation into account. And the wireless channel power gain will not be assumed as the Markov model, which allows the proposed task offloading policy to be executed on the device level. More explicitly, the mobile device can only observe its local information (e.g., the number of tasks and the computation demand of each task) but the system-side information is not observable. Key parameter notations in this paper are listed in Table 1 for ease of reference.Table 1
List of main parameter notations.
NotationDefinitionNtThe number of newly generated tasks at time slottn∈NtTasks to be offloadedλnInput data size of taskn (bits)δnComputation intensity of taskn (CPU cycles/bit)mnComputation demand for taskn (CPU cycles)bt∈BTasks in the bufferi∈I=l,s,bOffloading execution platforms (local, MEC, and buffer)γiAvailable computing capability in execution platformidn,itProcessing delay of each taskDtTotal processing delay for processing all tasksen,itEnergy consumption of each taskEtTotal energy consumptionqtThe number of tasks offloaded to the MEC serverξWeighting factor of location privacyKThe metric of the usage pattern privacyΔtThe difference betweenNt and qtPtPrivacy metric


### 3.2. Problem Formulation

We focus on privacy-aware and user-level task offloading optimization problems in this paper. In this section, we firstly formulate the task offloading decision making and then the model system cost (including processing latency and energy consumption cost) and privacy level as the performance metrics. Finally, the objective function will be presented.


#### 3.2.1. Task Offloading Decision Making

To maintain satisfactory quality of service, an available and reliable task offloading policy should be considered. And the tasks can be dynamically offloaded to the three different positionsi by the mobile device, denoted by I=l,s,b, where l, s, and b represent the local processing unit, MEC server, and buffer, respectively. At each time slot t, the mobile device (also called the earner or operator) makes the task offloading decision for each task n. Here, we design a binary indicator xn,it to denote the dynamic task offloading decision variable; let xn,it=1 if the task n∈Nt is offloaded to platform i∈I at time slot t and xn,it=0 otherwise. Note that at a given time slot t, each task n can be offloaded to only one execution platform (l, s, or b). We have the following constraints for xn,it:
(1)xn,it∈0,1,∀t,n,i,(2)xn,lt+xn,st+xn,bt=1,∀n,t,(3)∑n∈Nt∪B∑i∈Ixn,it=Nt+bt,∀t.Equation (1) indicates that whether offloading the task nin platform i in the time slot t. Equation (2) indicates that only one of xn,lt, xn,st, and xn,bt for task n in the time slot t can be nonzero. Equation (3) indicates that the (Nt+bt) tasks will be offloaded to execution platform i in the time slot t. Based on the above definition, the system cost model (including processing latency and energy consumption) and privacy model will be further described.


#### 3.2.2. System Cost Model

Similar to [27, 28], we consider a system cost that accounts for processing latency and energy consumption cost, which are associated with the task offloading. They depend on both the tasks and the processing platforms where the tasks are computed.(1) Processing Latency. In this paper, total processing latency consists of three parts, i.e., queuing delay in buffer, computing delay in either the local processing unit or the MEC server. For ease of exposition, we assume that the queuing delay in the buffer can be converted to computing delay and the buffer can be treated as a microprocessor, which has much lower computing capability than the local processing unit. In our system, each task n∈Nt∪B will be offloaded to execution platform i∈I by the mobile device in time slot t. We use γi to denote the available computing capability (i.e., CPU cycles per second) of execution platform i for task processing at time slot t. Then, the processing delay dn,it of each task n can be expressed as follows:
(4)dn,it=mntγi,where mnt is the computation demand of each task n at time slot t. Therefore, given the task offloading decision xn,it, the total processing latency required to process (Nt+bt) tasks within time slot t can be further expressed as follows:
(5)Dt=∑n=0Nt+bt∑i∈Idn,itxn,it=∑n=0Nt+bt∑i∈Imntγixn,it.(2) Energy Consumption. Task offloading will consume the energy of the mobile device, whose battery storage capacity is rather limited. Thereby, further investigation on how to minimize the total energy consumption of the mobile device is one of the objectives of this paper. The energy consumption cost of the mobile device may include the CPU cycles, transmitting energy and electric energy. They are associated with the tasks executed in the local processing unit, offloaded to the MEC server, and queued in the buffer. To better characterize these energy consumption costs, we let the en,it be the energy consumption in time slot t for offloading task n to execution platform i (i∈I=b,l,s). Thus, when considering the task offloading decision xn,it, the overall energy consumption at time slot t can be expressed as follows:
(6)Et=∑n=0Nt+bt∑i∈Ien,itxn,it,


#### 3.2.3. Privacy Model

As more and more people enjoy the benefits of MEC, the location privacy and usage pattern privacy of MEC have become a major concern. According to the simulation results and considered setting in [10], we can observe the offloading pattern that the mobile device may offload all its tasks to the MEC server when the wireless channel state is good, while it will not offload any tasks otherwise. For simplicity, it is assumed that the wireless channel states are only good and bad in this work. It can be extended to the multistate case. The wireless channel gain is highly related to the distance between the user and the MEC server. Thus, the honest-but-curious MEC server (it may be controlled by adversary) can infer not only the wireless channel state but also the distance to the mobile device based on the offloading pattern and historical statistics.Accordingly, when the mobile device communicates with multiple MEC servers, its location information may be jointly inferred by these MEC servers. Besides, if a mobile device always maintains a good channel state (e.g., its office near the base station), it will always offload all its tasks to the MEC server. The total number of tasks is highly related to the user’s usage pattern (i.e., user’s app running if a certain pattern exists in the number of tasks generated by the app), which may be very important for the privacy-sensitive users. Hence, the MEC server may be able to infer the personal information of the user through monitoring the total number of offloading tasks and analyzing the historical statistics.Hence, from the privacy perspective, we propose a metric to jointly quantify the location and usage pattern privacy and strike a balance between the privacy-preserving level and system cost. Firstly, the total number of tasks offloaded to the MEC server at the end of time slott is defined as qt and we have
(7)qt=∑n=0Nt+btxn,st,∀t.Then, the privacy metric ofPt can be obtained by
(8)Pt=IΔt=0⋅K+IΔt≠0⋅Iqt=0⋅ξ+Iqt≠0⋅ξ^Δt,where the I represents the indicator function that equals 1 if the statement is true and 0 otherwise; Δt indicates the difference between Nt and qt, and it has Δt=Nt−qt; ξ and ξ^ are the weighting factors reflecting the importance of the location privacy over the usage pattern privacy in different situations; K∈1,Nt denotes the metric of usage pattern privacy, which is the number of dummy tasks. The dummy tasks may sacrifice some system performance but will increase the privacy level, and the proposed algorithm will balance them.The first term of equation (8) represents that if the mobile device offloaded all its tasks to the MEC server (Δt=0), in order to protect the usage pattern privacy, it will continue to offload K dummy tasks to the MEC server to confuse the attacker. As such, the attacker cannot pinpoint the number of tasks actually generated by the user. According to the second term of equation (8), there are two situations correspond to the Δt≠0. In the first situation, qt=0 denotes that the tasks either queued in the buffer or processed locally otherwise. In order to protect the location privacy, the mobile device needs to offload ξ tasks (which is queuing in the buffer, 0≤ξ≤bt) to the MEC server for preventing the attacker from inferring the wireless channel status. In the second situation of Δt≠0, some tasks are offloaded to the MEC server (qt≠0) and the privacy level can be achieved by ξ^/Δt. It denotes the importance of the location privacy over the usage pattern privacy ξ^, which will increase as Δt decreases.


#### 3.2.4. Objective Function

In order to achieve a desirable trade-off between the system cost (i.e., computing delay and energy consumption) and the user’s privacy level, we design different weightsωdelayt, ωenergyt, and ωprivacyt to indicate the different preference device. These weights also can convert the privacy level and system cost into the same dimension. Thus, the objective of this paper is to achieve robust minimization of a weighted sum of the privacy level and system cost for the mobile device. Based on [29], given a finite time horizon T, the problem can be formulated as
(9)min∑t=0TωdelaytDt+ωenergytEt+ωprivacyt1Pt,s.t.1−3,From the mobile device perspective, it is difficult for them to explore the system-wide information (e.g., the wireless channel states and resource availability) in advance and it may need extremely expensive energy cost. Therefore, devising a device-level adaptive privacy-preserving task offloading policy is highly desirable, in which the future system-level information will not be needed.


## 3.1. System Model

In this section, the task offloading model will be presented. As illustrated in Figure1, we consider a scenario in which the mobile user/device communicates with the MEC server through the access point (e.g., Wi-Fi or 5G base station) via the wireless channel. For ease of exposition, the bandwidth constraint of the wireless channel is not considered in this paper and we will consider it in the next work. The mobile device has computing-intensive computation tasks that are required to be completed as soon as possible. Due to its limited battery energy and computing capabilities, the mobile device can offload some computation tasks to the MEC server, which has powerful computing capabilities. As such, the mobile device has three ways to process these computation tasks, that is, computing in the local processing unit, offloading to the MEC server through the transmission unit, and queuing in the buffer for processing in the next time slot.Figure 1
An illustration of task offloading in the MEC system.Without loss of generality, we assume that the task offloading policies are made in a slotted structure and its timeline is discretized into time slotst∈T=1,2,⋯,T. At each time slot t, the mobile user will newly generate Nt computation tasks to the mobile device, denoted by a set of ℕ=1,2,⋯,Nmaxt (Nmaxt is the maximum possible number of generated tasks), which depends on the user’s usage pattern. And the bt∈B=1,2,⋯,bmax (with the maximum buffer size bmax) can be denoted as the number of tasks in the buffer at time slot t.A widely used three-parameter model [26] can be used to describe each task n, denoted by a set of Nt=1,2,⋯,Nt+bt. The three-parameter model consists of the input data size λn (bits), computation intensity δn (CPU cycles/bit), and maximum allowed latency τn (seconds). Whereupon, the computation demand for each task can be obtained by mn=λn∗δn (CPU cycles). In each time slot t, all the (Nt+bt) tasks (including the newly generated Nt tasks and the bt tasks in the buffer) will be either locally executed, buffered, or remotely offloaded according to the proposed task offloading policy. More specifically, the mobile device will explore the optimal offloading policy for each task.Based on [10], in order to minimize computing delay and energy consumption, the mobile device tends to offload all its tasks to the MEC server if the wireless channel state is good and processes all its tasks locally if the wireless channel state is bad. Under such circumstances, the user’s location and usage pattern privacy are easily spied by the attacker. Hence, the proposed task offloading policy in this paper takes the privacy preservation into account. And the wireless channel power gain will not be assumed as the Markov model, which allows the proposed task offloading policy to be executed on the device level. More explicitly, the mobile device can only observe its local information (e.g., the number of tasks and the computation demand of each task) but the system-side information is not observable. Key parameter notations in this paper are listed in Table 1 for ease of reference.Table 1
List of main parameter notations.
NotationDefinitionNtThe number of newly generated tasks at time slottn∈NtTasks to be offloadedλnInput data size of taskn (bits)δnComputation intensity of taskn (CPU cycles/bit)mnComputation demand for taskn (CPU cycles)bt∈BTasks in the bufferi∈I=l,s,bOffloading execution platforms (local, MEC, and buffer)γiAvailable computing capability in execution platformidn,itProcessing delay of each taskDtTotal processing delay for processing all tasksen,itEnergy consumption of each taskEtTotal energy consumptionqtThe number of tasks offloaded to the MEC serverξWeighting factor of location privacyKThe metric of the usage pattern privacyΔtThe difference betweenNt and qtPtPrivacy metric


## 3.2. Problem Formulation

We focus on privacy-aware and user-level task offloading optimization problems in this paper. In this section, we firstly formulate the task offloading decision making and then the model system cost (including processing latency and energy consumption cost) and privacy level as the performance metrics. Finally, the objective function will be presented.


### 3.2.1. Task Offloading Decision Making

To maintain satisfactory quality of service, an available and reliable task offloading policy should be considered. And the tasks can be dynamically offloaded to the three different positionsi by the mobile device, denoted by I=l,s,b, where l, s, and b represent the local processing unit, MEC server, and buffer, respectively. At each time slot t, the mobile device (also called the earner or operator) makes the task offloading decision for each task n. Here, we design a binary indicator xn,it to denote the dynamic task offloading decision variable; let xn,it=1 if the task n∈Nt is offloaded to platform i∈I at time slot t and xn,it=0 otherwise. Note that at a given time slot t, each task n can be offloaded to only one execution platform (l, s, or b). We have the following constraints for xn,it:
(1)xn,it∈0,1,∀t,n,i,(2)xn,lt+xn,st+xn,bt=1,∀n,t,(3)∑n∈Nt∪B∑i∈Ixn,it=Nt+bt,∀t.Equation (1) indicates that whether offloading the task nin platform i in the time slot t. Equation (2) indicates that only one of xn,lt, xn,st, and xn,bt for task n in the time slot t can be nonzero. Equation (3) indicates that the (Nt+bt) tasks will be offloaded to execution platform i in the time slot t. Based on the above definition, the system cost model (including processing latency and energy consumption) and privacy model will be further described.


### 3.2.2. System Cost Model

Similar to [27, 28], we consider a system cost that accounts for processing latency and energy consumption cost, which are associated with the task offloading. They depend on both the tasks and the processing platforms where the tasks are computed.(1) Processing Latency. In this paper, total processing latency consists of three parts, i.e., queuing delay in buffer, computing delay in either the local processing unit or the MEC server. For ease of exposition, we assume that the queuing delay in the buffer can be converted to computing delay and the buffer can be treated as a microprocessor, which has much lower computing capability than the local processing unit. In our system, each task n∈Nt∪B will be offloaded to execution platform i∈I by the mobile device in time slot t. We use γi to denote the available computing capability (i.e., CPU cycles per second) of execution platform i for task processing at time slot t. Then, the processing delay dn,it of each task n can be expressed as follows:
(4)dn,it=mntγi,where mnt is the computation demand of each task n at time slot t. Therefore, given the task offloading decision xn,it, the total processing latency required to process (Nt+bt) tasks within time slot t can be further expressed as follows:
(5)Dt=∑n=0Nt+bt∑i∈Idn,itxn,it=∑n=0Nt+bt∑i∈Imntγixn,it.(2) Energy Consumption. Task offloading will consume the energy of the mobile device, whose battery storage capacity is rather limited. Thereby, further investigation on how to minimize the total energy consumption of the mobile device is one of the objectives of this paper. The energy consumption cost of the mobile device may include the CPU cycles, transmitting energy and electric energy. They are associated with the tasks executed in the local processing unit, offloaded to the MEC server, and queued in the buffer. To better characterize these energy consumption costs, we let the en,it be the energy consumption in time slot t for offloading task n to execution platform i (i∈I=b,l,s). Thus, when considering the task offloading decision xn,it, the overall energy consumption at time slot t can be expressed as follows:
(6)Et=∑n=0Nt+bt∑i∈Ien,itxn,it,


### 3.2.3. Privacy Model

As more and more people enjoy the benefits of MEC, the location privacy and usage pattern privacy of MEC have become a major concern. According to the simulation results and considered setting in [10], we can observe the offloading pattern that the mobile device may offload all its tasks to the MEC server when the wireless channel state is good, while it will not offload any tasks otherwise. For simplicity, it is assumed that the wireless channel states are only good and bad in this work. It can be extended to the multistate case. The wireless channel gain is highly related to the distance between the user and the MEC server. Thus, the honest-but-curious MEC server (it may be controlled by adversary) can infer not only the wireless channel state but also the distance to the mobile device based on the offloading pattern and historical statistics.Accordingly, when the mobile device communicates with multiple MEC servers, its location information may be jointly inferred by these MEC servers. Besides, if a mobile device always maintains a good channel state (e.g., its office near the base station), it will always offload all its tasks to the MEC server. The total number of tasks is highly related to the user’s usage pattern (i.e., user’s app running if a certain pattern exists in the number of tasks generated by the app), which may be very important for the privacy-sensitive users. Hence, the MEC server may be able to infer the personal information of the user through monitoring the total number of offloading tasks and analyzing the historical statistics.Hence, from the privacy perspective, we propose a metric to jointly quantify the location and usage pattern privacy and strike a balance between the privacy-preserving level and system cost. Firstly, the total number of tasks offloaded to the MEC server at the end of time slott is defined as qt and we have
(7)qt=∑n=0Nt+btxn,st,∀t.Then, the privacy metric ofPt can be obtained by
(8)Pt=IΔt=0⋅K+IΔt≠0⋅Iqt=0⋅ξ+Iqt≠0⋅ξ^Δt,where the I represents the indicator function that equals 1 if the statement is true and 0 otherwise; Δt indicates the difference between Nt and qt, and it has Δt=Nt−qt; ξ and ξ^ are the weighting factors reflecting the importance of the location privacy over the usage pattern privacy in different situations; K∈1,Nt denotes the metric of usage pattern privacy, which is the number of dummy tasks. The dummy tasks may sacrifice some system performance but will increase the privacy level, and the proposed algorithm will balance them.The first term of equation (8) represents that if the mobile device offloaded all its tasks to the MEC server (Δt=0), in order to protect the usage pattern privacy, it will continue to offload K dummy tasks to the MEC server to confuse the attacker. As such, the attacker cannot pinpoint the number of tasks actually generated by the user. According to the second term of equation (8), there are two situations correspond to the Δt≠0. In the first situation, qt=0 denotes that the tasks either queued in the buffer or processed locally otherwise. In order to protect the location privacy, the mobile device needs to offload ξ tasks (which is queuing in the buffer, 0≤ξ≤bt) to the MEC server for preventing the attacker from inferring the wireless channel status. In the second situation of Δt≠0, some tasks are offloaded to the MEC server (qt≠0) and the privacy level can be achieved by ξ^/Δt. It denotes the importance of the location privacy over the usage pattern privacy ξ^, which will increase as Δt decreases.


### 3.2.4. Objective Function

In order to achieve a desirable trade-off between the system cost (i.e., computing delay and energy consumption) and the user’s privacy level, we design different weightsωdelayt, ωenergyt, and ωprivacyt to indicate the different preference device. These weights also can convert the privacy level and system cost into the same dimension. Thus, the objective of this paper is to achieve robust minimization of a weighted sum of the privacy level and system cost for the mobile device. Based on [29], given a finite time horizon T, the problem can be formulated as
(9)min∑t=0TωdelaytDt+ωenergytEt+ωprivacyt1Pt,s.t.1−3,From the mobile device perspective, it is difficult for them to explore the system-wide information (e.g., the wireless channel states and resource availability) in advance and it may need extremely expensive energy cost. Therefore, devising a device-level adaptive privacy-preserving task offloading policy is highly desirable, in which the future system-level information will not be needed.


## 3.2.1. Task Offloading Decision Making

To maintain satisfactory quality of service, an available and reliable task offloading policy should be considered. And the tasks can be dynamically offloaded to the three different positionsi by the mobile device, denoted by I=l,s,b, where l, s, and b represent the local processing unit, MEC server, and buffer, respectively. At each time slot t, the mobile device (also called the earner or operator) makes the task offloading decision for each task n. Here, we design a binary indicator xn,it to denote the dynamic task offloading decision variable; let xn,it=1 if the task n∈Nt is offloaded to platform i∈I at time slot t and xn,it=0 otherwise. Note that at a given time slot t, each task n can be offloaded to only one execution platform (l, s, or b). We have the following constraints for xn,it:
(1)xn,it∈0,1,∀t,n,i,(2)xn,lt+xn,st+xn,bt=1,∀n,t,(3)∑n∈Nt∪B∑i∈Ixn,it=Nt+bt,∀t.Equation (1) indicates that whether offloading the task nin platform i in the time slot t. Equation (2) indicates that only one of xn,lt, xn,st, and xn,bt for task n in the time slot t can be nonzero. Equation (3) indicates that the (Nt+bt) tasks will be offloaded to execution platform i in the time slot t. Based on the above definition, the system cost model (including processing latency and energy consumption) and privacy model will be further described.


## 3.2.2. System Cost Model

Similar to [27, 28], we consider a system cost that accounts for processing latency and energy consumption cost, which are associated with the task offloading. They depend on both the tasks and the processing platforms where the tasks are computed.(1) Processing Latency. In this paper, total processing latency consists of three parts, i.e., queuing delay in buffer, computing delay in either the local processing unit or the MEC server. For ease of exposition, we assume that the queuing delay in the buffer can be converted to computing delay and the buffer can be treated as a microprocessor, which has much lower computing capability than the local processing unit. In our system, each task n∈Nt∪B will be offloaded to execution platform i∈I by the mobile device in time slot t. We use γi to denote the available computing capability (i.e., CPU cycles per second) of execution platform i for task processing at time slot t. Then, the processing delay dn,it of each task n can be expressed as follows:
(4)dn,it=mntγi,where mnt is the computation demand of each task n at time slot t. Therefore, given the task offloading decision xn,it, the total processing latency required to process (Nt+bt) tasks within time slot t can be further expressed as follows:
(5)Dt=∑n=0Nt+bt∑i∈Idn,itxn,it=∑n=0Nt+bt∑i∈Imntγixn,it.(2) Energy Consumption. Task offloading will consume the energy of the mobile device, whose battery storage capacity is rather limited. Thereby, further investigation on how to minimize the total energy consumption of the mobile device is one of the objectives of this paper. The energy consumption cost of the mobile device may include the CPU cycles, transmitting energy and electric energy. They are associated with the tasks executed in the local processing unit, offloaded to the MEC server, and queued in the buffer. To better characterize these energy consumption costs, we let the en,it be the energy consumption in time slot t for offloading task n to execution platform i (i∈I=b,l,s). Thus, when considering the task offloading decision xn,it, the overall energy consumption at time slot t can be expressed as follows:
(6)Et=∑n=0Nt+bt∑i∈Ien,itxn,it,


## 3.2.3. Privacy Model

As more and more people enjoy the benefits of MEC, the location privacy and usage pattern privacy of MEC have become a major concern. According to the simulation results and considered setting in [10], we can observe the offloading pattern that the mobile device may offload all its tasks to the MEC server when the wireless channel state is good, while it will not offload any tasks otherwise. For simplicity, it is assumed that the wireless channel states are only good and bad in this work. It can be extended to the multistate case. The wireless channel gain is highly related to the distance between the user and the MEC server. Thus, the honest-but-curious MEC server (it may be controlled by adversary) can infer not only the wireless channel state but also the distance to the mobile device based on the offloading pattern and historical statistics.Accordingly, when the mobile device communicates with multiple MEC servers, its location information may be jointly inferred by these MEC servers. Besides, if a mobile device always maintains a good channel state (e.g., its office near the base station), it will always offload all its tasks to the MEC server. The total number of tasks is highly related to the user’s usage pattern (i.e., user’s app running if a certain pattern exists in the number of tasks generated by the app), which may be very important for the privacy-sensitive users. Hence, the MEC server may be able to infer the personal information of the user through monitoring the total number of offloading tasks and analyzing the historical statistics.Hence, from the privacy perspective, we propose a metric to jointly quantify the location and usage pattern privacy and strike a balance between the privacy-preserving level and system cost. Firstly, the total number of tasks offloaded to the MEC server at the end of time slott is defined as qt and we have
(7)qt=∑n=0Nt+btxn,st,∀t.Then, the privacy metric ofPt can be obtained by
(8)Pt=IΔt=0⋅K+IΔt≠0⋅Iqt=0⋅ξ+Iqt≠0⋅ξ^Δt,where the I represents the indicator function that equals 1 if the statement is true and 0 otherwise; Δt indicates the difference between Nt and qt, and it has Δt=Nt−qt; ξ and ξ^ are the weighting factors reflecting the importance of the location privacy over the usage pattern privacy in different situations; K∈1,Nt denotes the metric of usage pattern privacy, which is the number of dummy tasks. The dummy tasks may sacrifice some system performance but will increase the privacy level, and the proposed algorithm will balance them.The first term of equation (8) represents that if the mobile device offloaded all its tasks to the MEC server (Δt=0), in order to protect the usage pattern privacy, it will continue to offload K dummy tasks to the MEC server to confuse the attacker. As such, the attacker cannot pinpoint the number of tasks actually generated by the user. According to the second term of equation (8), there are two situations correspond to the Δt≠0. In the first situation, qt=0 denotes that the tasks either queued in the buffer or processed locally otherwise. In order to protect the location privacy, the mobile device needs to offload ξ tasks (which is queuing in the buffer, 0≤ξ≤bt) to the MEC server for preventing the attacker from inferring the wireless channel status. In the second situation of Δt≠0, some tasks are offloaded to the MEC server (qt≠0) and the privacy level can be achieved by ξ^/Δt. It denotes the importance of the location privacy over the usage pattern privacy ξ^, which will increase as Δt decreases.


## 3.2.4. Objective Function

In order to achieve a desirable trade-off between the system cost (i.e., computing delay and energy consumption) and the user’s privacy level, we design different weightsωdelayt, ωenergyt, and ωprivacyt to indicate the different preference device. These weights also can convert the privacy level and system cost into the same dimension. Thus, the objective of this paper is to achieve robust minimization of a weighted sum of the privacy level and system cost for the mobile device. Based on [29], given a finite time horizon T, the problem can be formulated as
(9)min∑t=0TωdelaytDt+ωenergytEt+ωprivacyt1Pt,s.t.1−3,From the mobile device perspective, it is difficult for them to explore the system-wide information (e.g., the wireless channel states and resource availability) in advance and it may need extremely expensive energy cost. Therefore, devising a device-level adaptive privacy-preserving task offloading policy is highly desirable, in which the future system-level information will not be needed.


## 4. Algorithm Design

In this section, we focus on the privacy preserving task offloading problem in the MEC-enabled network and propose a device-level privacy-aware online learning scheme to minimize the objective in equation (9) for the mobile device without knowing the system-side information.Firstly, we transform the information-constrained multiobjective optimization problem to a contextual multi-armed bandit (MAB) problem [30] with a semiparametric reward model. Then, we propose a privacy-aware online task offloading (PAOTO) algorithm which can accommodate the network dynamics at the device level and learn the optimal offloading policy for the mobile device while maintaining the user’s privacy.


### 4.1. Problem Transformation

In this work, we focus on the device-level and privacy-aware task offloading problem, which is a typical sequential decision problem. For decoupling the time dependency, we formulate this problem as a contextual MAB problem with a relaxed, semiparametric reward model in [30]. It is an extended version of the conventional contextual MAB that has a linear reward model [31]. Both versions can utilize the contextual feature vector to indicate the use-side information for overcoming the challenges of lacking future system information. However, why we use the contextual MAB with a relaxed, semiparametric reward model is that the privacy metric in our model is difficult to formulate as a linear reward model. The semiparametric reward model can provide a more relaxed reward model, and this proof can be found in the literature [30].Accordingly, in order to learn the network dynamics and take the privacy protection into consideration, the problem in this paper can be transformed as a semiparametric contextual MAB problem [13], which can address the tradeoffs inherent in the sequential decision problem and has a relaxed, semiparametric reward model. This model can be described as
(10)E∑n∈Ntrn,it∣Ft−1=νt+∑n∈Ntbt⊤μ¯n,it,where rn,it is the received cost of offloading task n to execution platform i; νt is a nonparametric component; bt is a current contextual feature vector; μ¯n,it is a fixed but unknown underlying expectation of the feature vector μn,it; the Ft−1 is the union of historical information and bt. Furthermore, it has assumptions about the upper bound of some parameters, which is bt2≤1,μ¯n,it2≤1,νt2≤1, and ⋅2 denotes the L2 norm.When the computation tasks arrive, the task offloading decision can be executed for each task by the mobile device. Nonetheless, only the device-side status information can observable, which can be described as a contextual feature vectorbt=ωdelaytMt,ωenergytSt∈ℝ2Nmaxt+bmax for arriving tasks. More specifically, Mt∈ℝNmaxt+bmax denotes the computation demand vector of tasks. The first Nt+bt values of Mt are corresponding computation demand mnt of each task n, and the remaining values are 0; St∈ℝNmaxt+bmax is a transition vector, which denotes the number of tasks in time slot t. The first Nt+bt values of St are 1, and the remainder are 0. According to the system cost (including processing latency and energy consumption cost) defined in Section 3, we transform them as a feature vector μn,it=1/γit,en,it∈ℝ2Nmaxt+bmax to better learn the network uncertainty and resource availability, which is related to μ¯n,it. Besides, the privacy level in this task offloading policy will be formulated as the aggregated nonparametric component νt=ωprivacyt1/Ptbased on the reward model of contextual MAB in equation (10). The reason of this is that it cannot be directly formulated as a linear component like other metrics (such as computing delay and energy consumption). We assume that it can be calculated when all task decisions are completed at the end of t.Hereinafter, we defineHt−1=Aτ,rAττ,bτ, τ=1,2,⋯,t−1 as the historical observations until t−1, where Aτ represents the set of actions for all tasks at time slot τ and rAττ denotes the total received cost at time slot τ. And the Ft−1=Ht−1,bt can be denoted as the union of historical information Ht−1 and the current contextual feature vector bt. Given that Ft−1, we assume that the expectation of the total received cost rAtt can be decomposed into a time-invariant linear component ∑n=0Nt+btbt⊤μ¯n,Att (associated with processing delay and energy consumption cost) and a nonparametric component νt (associated with the privacy-preserving level). Therefore, according to equation (10), we have
(11)ErAtt∣Ft−1=E∑n=0Nt+btrn,it∣Ft−1=νt+∑n=0Nt+btbt⊤μ¯n,it=ωprivacyt1Pt+∑n=0Nt+btbt⊤μ¯n,it.The task offloading scheme needs to select an execution platformi (or called an arm at MAB) for every task n at time slot t. Specifically, we let ant∈At denote the choice for every task and let the optimal action to be an∗t∈A∗t based on equation (10). Additionally, it must be noted that the nonparametric component νt in equation (11) depends on time and historical information, but not on the current action [30]. Hence, the optimal received cost rn,it of each task can be obtained by minimum bt⊤μ¯n,it and we can achieve the optimal offloading decision of each task by an∗t=argminibt⊤μ¯n,it. Indeed, the privacy level νt will have an impact on the aggregated received cost of all tasks at the end of time slot t and this aggregated received cost will be used to update the contextual feature vector for the next interval t.Beyond that, the regret at time slott is defined as the difference between the average cost of the optimal choices and the universal choices for all tasks and it does not depend on νt either. Hence, the regret can be expressed as
(12)Regrett=bt⊤μ¯A∗t−bt⊤μ¯At.Moreover, given a finite time horizonT, the total regret can be described as
(13)Rt=∑t=1Tregrett=∑t=1Tbt⊤μ¯A∗t−bt⊤μ¯At.This regret is used to evaluate the effectiveness of task offloading decision making based on the online learning of system-level information.


### 4.2. Privacy-Aware Online Task Offloading Algorithm

In order to minimize the total system cost (e.g., delay and energy consumption cost) and protect user’s privacy without exploring any system-level information, a novel PAOTO algorithm is proposed in this work. In particular, this algorithm keeps the framework of the Thompson sampling (TS) with a semiparameter reward model [32]. Its key idea is to estimate and learn the device’s performance by selecting different actions over time based on the contextual information. At the same time, the privacy metric can be abstracted into the semiparametric reward model.In the proposed PAOTO algorithm, the mobile device will learn the network information while executing the task offloading policy. As time goes by, the mobile device learns abundant information and it can estimate how to offload theseNt+bt tasks for achieving the optimal system cost and privacy-preserving level. According to the aforementioned MAB transformation of our problem, it is known that the optimal offloading decision and the received cost rn,it of each task mainly depend on the current contextual feature vector bt and the fixed but unknown feature vector μ¯n,it. Through the previous trial and error, the underlying relationship between the feature vectors and received cost will be learned by the mobile device. The νt is related to the privacy metric Pt, and it can be achieved after all tasks are offloaded at the end of time slot t. Hence, we let μ^n,it denote the estimate of the feature vector μn,it and Bit represent the cumulative contextual vector. The estimate of feature vector μ^n,it and the cumulative contextual vector Bit can be denoted as
(14)μ^n,it=Id+Σ∧t+Σt−1∑τ=1t−12XτrAττ=Bit−1∑τ=1t−12XτrAττ,Bit=Id+Σ^t+Σt=Id+∑τ=1t−1XτXτ⊤+∑τ=1t−1EXτXτ⊤∣Fτ−1,where Xτ=bτ−Ebτ∣Fτ−1; I is a d dimensional identity matrix, where d=2Nmaxt+bmax.For ease of exposition, theEbτ∣Fτ−1 in Xτ can be denoted as b¯τ and it can be calculated as
(15)b¯τ=E∑n=1Nt+bt∑i=13Ianτ=ibτ∣Fτ−1=∑n=1Nt+bt∑i=13πn,iτbτ,where πn,iτ=ℙanτ=i∣Fτ−1 is the probability of offloading task n to the ith execution platform at time τ.Besides, we can calculate the covarianceEXτXτ⊤∣Fτ−1 as follows:
(16)EXτXτ⊤∣Fτ−1=∑n=1Nt+bt∑i=13πn,iτbτ−b¯τbτ−b¯τ⊤.Accordingly, the mobile device can continuously explore and then gather the relationship between the feature vector of each task and the system cost of the chosen execution platform. Then, it also measures the corresponding privacy level to estimate which execution platform is likely to give the minimum system cost while maintaining a good privacy level.In this paper, the TS-based online learning algorithm will be applied to learn the underlying relation between the feature vector and received cost. Hence, we should construct a distributional likelihood function to sample the estimated cost. Firstly, the standard deviation of the estimated costr^n,it=bt⊤μ^n,it can be defined as s^n,it=bt⊤Bit−1bt and the standard deviation of the sampling cost r~n,it=bt⊤μ~n,it can be denoted as s~n,it=vbt⊤Bit−1bt=vs^n,it, where v=2R+66dlogT/δ is a control parameter.According to Bayes’ theoremPBi∣A∝PBiPA∣Bi, we have:
(17)Prbt⊤μ~n,itr~n,it∝Prr~n,itbt⊤μ~n,itPrbt⊤μ~n,it.Based on the TS algorithm in [31], if the prior for received cost rn,it=bt⊤μ¯n,it at time slot t is given by Nbt⊤μ^n,it,v2bt⊤Bit−1bt, it is easy to compute the posterior distribution at time slot t+1, i.e., Nbt+1⊤μ^n,it+1,v2bt+1⊤Bit+1−1bt+1 (details of this computation can be seen in Appendix A.1 of [31]). Hence, at every time step t, we can use this Gaussian likelihood function Nbt⊤μ^n,it,v2bt⊤Bit−1bt to sample the cost r~n,it for offloaded task n at execution platform i in our algorithm. Then, the sampling cost r~n,it will be used to estimate the performance of offloaded task n at execution platform i and finally the execution platform that has minimum r~n,it.Hence, guided by the problem transformation and key vectors mentioned above, we introduce the PAOTO algorithm in Algorithm1.Algorithm 1: Privacy-aware online task offloading (PAOTO) algorithm.
Input:Nt, Nmaxt, bt, bmax, mnt, i∈I;Output: total received cost rAtt.1:Initialization: Initialize the cumulative contextual vector Bi=Id, where Id is a 2Nmaxt+bmax dimensional identity matrix, the cumulative contextual system cost yi=0d, and a control parameter v=2R+66dlogT/δ,δ∈0,1.2:End initialization3:fort=1,2,⋯,Tdo4:forn=1,2,⋯,Nt+btdo5:fori=1,2,3do6:Compute the estimated feature vector μ^n,it=Bi−1yi.7:Sample the cost r~n,it independently for each task n and each execution platform i from the Gaussian distributionNbt⊤μ^n,it,v2bt⊤Bit−1bt.8:Compute the probability of offloading task n to the i-th execution platform πn,it=ℙant=i∣Ft−1.9:End for10:Select the offloading execution platform ant=argminir~n,it.11:Record the selection ant into offloading decisions vector At.12:End for13:Compute the aggregated received cost rAtt via At, Equation (5), (6), (8) and (9).14:UpdateB and y:15:BAt⟵BAt+bt−b¯tbt−b¯t⊤+∑n=1Nt+bt∑i=1i=3πn,itbt−b¯tbt−b¯t⊤16:yAt⟵yAt+2bt−b¯trAtt.17:End forAlgorithm1 gives the details of exploring the optimal solution that can make an adaptive task offloading decision and preserve the privacy of users. It estimates the offloading cost rn,it of each task based on context information bt and performance feature vector μn,it and selects the best offloading action based on the minimum received cost rn,it. At the same time, it calculates the offloading probability πn,it to fit the MAB problem with a semiparametric reward model for privacy preservation. At the end, it utilizes total received cost rAtt to update cumulative contextual vector B and cumulative contextual system cost y corresponding to the decisions vector At of all tasks at every time slot t.


## 4.1. Problem Transformation

In this work, we focus on the device-level and privacy-aware task offloading problem, which is a typical sequential decision problem. For decoupling the time dependency, we formulate this problem as a contextual MAB problem with a relaxed, semiparametric reward model in [30]. It is an extended version of the conventional contextual MAB that has a linear reward model [31]. Both versions can utilize the contextual feature vector to indicate the use-side information for overcoming the challenges of lacking future system information. However, why we use the contextual MAB with a relaxed, semiparametric reward model is that the privacy metric in our model is difficult to formulate as a linear reward model. The semiparametric reward model can provide a more relaxed reward model, and this proof can be found in the literature [30].Accordingly, in order to learn the network dynamics and take the privacy protection into consideration, the problem in this paper can be transformed as a semiparametric contextual MAB problem [13], which can address the tradeoffs inherent in the sequential decision problem and has a relaxed, semiparametric reward model. This model can be described as
(10)E∑n∈Ntrn,it∣Ft−1=νt+∑n∈Ntbt⊤μ¯n,it,where rn,it is the received cost of offloading task n to execution platform i; νt is a nonparametric component; bt is a current contextual feature vector; μ¯n,it is a fixed but unknown underlying expectation of the feature vector μn,it; the Ft−1 is the union of historical information and bt. Furthermore, it has assumptions about the upper bound of some parameters, which is bt2≤1,μ¯n,it2≤1,νt2≤1, and ⋅2 denotes the L2 norm.When the computation tasks arrive, the task offloading decision can be executed for each task by the mobile device. Nonetheless, only the device-side status information can observable, which can be described as a contextual feature vectorbt=ωdelaytMt,ωenergytSt∈ℝ2Nmaxt+bmax for arriving tasks. More specifically, Mt∈ℝNmaxt+bmax denotes the computation demand vector of tasks. The first Nt+bt values of Mt are corresponding computation demand mnt of each task n, and the remaining values are 0; St∈ℝNmaxt+bmax is a transition vector, which denotes the number of tasks in time slot t. The first Nt+bt values of St are 1, and the remainder are 0. According to the system cost (including processing latency and energy consumption cost) defined in Section 3, we transform them as a feature vector μn,it=1/γit,en,it∈ℝ2Nmaxt+bmax to better learn the network uncertainty and resource availability, which is related to μ¯n,it. Besides, the privacy level in this task offloading policy will be formulated as the aggregated nonparametric component νt=ωprivacyt1/Ptbased on the reward model of contextual MAB in equation (10). The reason of this is that it cannot be directly formulated as a linear component like other metrics (such as computing delay and energy consumption). We assume that it can be calculated when all task decisions are completed at the end of t.Hereinafter, we defineHt−1=Aτ,rAττ,bτ, τ=1,2,⋯,t−1 as the historical observations until t−1, where Aτ represents the set of actions for all tasks at time slot τ and rAττ denotes the total received cost at time slot τ. And the Ft−1=Ht−1,bt can be denoted as the union of historical information Ht−1 and the current contextual feature vector bt. Given that Ft−1, we assume that the expectation of the total received cost rAtt can be decomposed into a time-invariant linear component ∑n=0Nt+btbt⊤μ¯n,Att (associated with processing delay and energy consumption cost) and a nonparametric component νt (associated with the privacy-preserving level). Therefore, according to equation (10), we have
(11)ErAtt∣Ft−1=E∑n=0Nt+btrn,it∣Ft−1=νt+∑n=0Nt+btbt⊤μ¯n,it=ωprivacyt1Pt+∑n=0Nt+btbt⊤μ¯n,it.The task offloading scheme needs to select an execution platformi (or called an arm at MAB) for every task n at time slot t. Specifically, we let ant∈At denote the choice for every task and let the optimal action to be an∗t∈A∗t based on equation (10). Additionally, it must be noted that the nonparametric component νt in equation (11) depends on time and historical information, but not on the current action [30]. Hence, the optimal received cost rn,it of each task can be obtained by minimum bt⊤μ¯n,it and we can achieve the optimal offloading decision of each task by an∗t=argminibt⊤μ¯n,it. Indeed, the privacy level νt will have an impact on the aggregated received cost of all tasks at the end of time slot t and this aggregated received cost will be used to update the contextual feature vector for the next interval t.Beyond that, the regret at time slott is defined as the difference between the average cost of the optimal choices and the universal choices for all tasks and it does not depend on νt either. Hence, the regret can be expressed as
(12)Regrett=bt⊤μ¯A∗t−bt⊤μ¯At.Moreover, given a finite time horizonT, the total regret can be described as
(13)Rt=∑t=1Tregrett=∑t=1Tbt⊤μ¯A∗t−bt⊤μ¯At.This regret is used to evaluate the effectiveness of task offloading decision making based on the online learning of system-level information.


## 4.2. Privacy-Aware Online Task Offloading Algorithm

In order to minimize the total system cost (e.g., delay and energy consumption cost) and protect user’s privacy without exploring any system-level information, a novel PAOTO algorithm is proposed in this work. In particular, this algorithm keeps the framework of the Thompson sampling (TS) with a semiparameter reward model [32]. Its key idea is to estimate and learn the device’s performance by selecting different actions over time based on the contextual information. At the same time, the privacy metric can be abstracted into the semiparametric reward model.In the proposed PAOTO algorithm, the mobile device will learn the network information while executing the task offloading policy. As time goes by, the mobile device learns abundant information and it can estimate how to offload theseNt+bt tasks for achieving the optimal system cost and privacy-preserving level. According to the aforementioned MAB transformation of our problem, it is known that the optimal offloading decision and the received cost rn,it of each task mainly depend on the current contextual feature vector bt and the fixed but unknown feature vector μ¯n,it. Through the previous trial and error, the underlying relationship between the feature vectors and received cost will be learned by the mobile device. The νt is related to the privacy metric Pt, and it can be achieved after all tasks are offloaded at the end of time slot t. Hence, we let μ^n,it denote the estimate of the feature vector μn,it and Bit represent the cumulative contextual vector. The estimate of feature vector μ^n,it and the cumulative contextual vector Bit can be denoted as
(14)μ^n,it=Id+Σ∧t+Σt−1∑τ=1t−12XτrAττ=Bit−1∑τ=1t−12XτrAττ,Bit=Id+Σ^t+Σt=Id+∑τ=1t−1XτXτ⊤+∑τ=1t−1EXτXτ⊤∣Fτ−1,where Xτ=bτ−Ebτ∣Fτ−1; I is a d dimensional identity matrix, where d=2Nmaxt+bmax.For ease of exposition, theEbτ∣Fτ−1 in Xτ can be denoted as b¯τ and it can be calculated as
(15)b¯τ=E∑n=1Nt+bt∑i=13Ianτ=ibτ∣Fτ−1=∑n=1Nt+bt∑i=13πn,iτbτ,where πn,iτ=ℙanτ=i∣Fτ−1 is the probability of offloading task n to the ith execution platform at time τ.Besides, we can calculate the covarianceEXτXτ⊤∣Fτ−1 as follows:
(16)EXτXτ⊤∣Fτ−1=∑n=1Nt+bt∑i=13πn,iτbτ−b¯τbτ−b¯τ⊤.Accordingly, the mobile device can continuously explore and then gather the relationship between the feature vector of each task and the system cost of the chosen execution platform. Then, it also measures the corresponding privacy level to estimate which execution platform is likely to give the minimum system cost while maintaining a good privacy level.In this paper, the TS-based online learning algorithm will be applied to learn the underlying relation between the feature vector and received cost. Hence, we should construct a distributional likelihood function to sample the estimated cost. Firstly, the standard deviation of the estimated costr^n,it=bt⊤μ^n,it can be defined as s^n,it=bt⊤Bit−1bt and the standard deviation of the sampling cost r~n,it=bt⊤μ~n,it can be denoted as s~n,it=vbt⊤Bit−1bt=vs^n,it, where v=2R+66dlogT/δ is a control parameter.According to Bayes’ theoremPBi∣A∝PBiPA∣Bi, we have:
(17)Prbt⊤μ~n,itr~n,it∝Prr~n,itbt⊤μ~n,itPrbt⊤μ~n,it.Based on the TS algorithm in [31], if the prior for received cost rn,it=bt⊤μ¯n,it at time slot t is given by Nbt⊤μ^n,it,v2bt⊤Bit−1bt, it is easy to compute the posterior distribution at time slot t+1, i.e., Nbt+1⊤μ^n,it+1,v2bt+1⊤Bit+1−1bt+1 (details of this computation can be seen in Appendix A.1 of [31]). Hence, at every time step t, we can use this Gaussian likelihood function Nbt⊤μ^n,it,v2bt⊤Bit−1bt to sample the cost r~n,it for offloaded task n at execution platform i in our algorithm. Then, the sampling cost r~n,it will be used to estimate the performance of offloaded task n at execution platform i and finally the execution platform that has minimum r~n,it.Hence, guided by the problem transformation and key vectors mentioned above, we introduce the PAOTO algorithm in Algorithm1.Algorithm 1: Privacy-aware online task offloading (PAOTO) algorithm.
Input:Nt, Nmaxt, bt, bmax, mnt, i∈I;Output: total received cost rAtt.1:Initialization: Initialize the cumulative contextual vector Bi=Id, where Id is a 2Nmaxt+bmax dimensional identity matrix, the cumulative contextual system cost yi=0d, and a control parameter v=2R+66dlogT/δ,δ∈0,1.2:End initialization3:fort=1,2,⋯,Tdo4:forn=1,2,⋯,Nt+btdo5:fori=1,2,3do6:Compute the estimated feature vector μ^n,it=Bi−1yi.7:Sample the cost r~n,it independently for each task n and each execution platform i from the Gaussian distributionNbt⊤μ^n,it,v2bt⊤Bit−1bt.8:Compute the probability of offloading task n to the i-th execution platform πn,it=ℙant=i∣Ft−1.9:End for10:Select the offloading execution platform ant=argminir~n,it.11:Record the selection ant into offloading decisions vector At.12:End for13:Compute the aggregated received cost rAtt via At, Equation (5), (6), (8) and (9).14:UpdateB and y:15:BAt⟵BAt+bt−b¯tbt−b¯t⊤+∑n=1Nt+bt∑i=1i=3πn,itbt−b¯tbt−b¯t⊤16:yAt⟵yAt+2bt−b¯trAtt.17:End forAlgorithm1 gives the details of exploring the optimal solution that can make an adaptive task offloading decision and preserve the privacy of users. It estimates the offloading cost rn,it of each task based on context information bt and performance feature vector μn,it and selects the best offloading action based on the minimum received cost rn,it. At the same time, it calculates the offloading probability πn,it to fit the MAB problem with a semiparametric reward model for privacy preservation. At the end, it utilizes total received cost rAtt to update cumulative contextual vector B and cumulative contextual system cost y corresponding to the decisions vector At of all tasks at every time slot t.


## 5. Simulation Results

In this section, extensive simulations are conducted to evaluate the performance of the proposed PAOTO algorithm under different scenarios. We build our simulations in Python 3.6. The implementations are conducted on a Lenovo desktop PC equipped with Intel(R) core (TM) i7-4500U CPU @1.80 GHz processor and 12.0 GB (11.7 GB available) RAM. The simulation settings, algorithm benchmarks, and performance evaluation are elaborated below.


### 5.1. Simulation Settings

In our simulation environment, we consider a MEC system, in which the access point is deployed with the MEC server. TheNt computing tasks are randomly generated by the mobile device at every time slot t, where the maximum of Nt is in the range from 10 to 60. The maximum buffer capacity bmax of the mobile device can be set to 10. The length of each time slot is 1 s. The total computation capacity for MEC server γs is uniformly distributed in (10, 15) GHz. In order to accommodate the dynamics, we assume that the computation capacity of the mobile device γl is determined randomly from 1 to 3 GHz and the converted computation capacity of buffer γb is uniformly distributed in (0.1, 0.15) GHz. Based on [33], the data size of each task λn is distributed in (300 K, 800 K) bits and the computation intensity δn is taken randomly within (250, 1000) CPU cycles/bit. Thus, we can get the required CPU cycles of a computing task mn by mn=λn∗δn (CPU cycles). Besides, the energy consumption for transmitting one task to the MEC server is uniformly distributed in (0.1, 0.5) J and the mobile device consumes 0.8 J to 3 J to locally compute one task and 0.5 J to 1 J to buffer one task. The weights of processing delay, energy consumption, and privacy metric, which are ωdelayt, ωenergyt and ωprivacyt,respectively, can be dynamically set by the users according to the user’s preferences and the running application demands. In our simulation, we set them to 1, 1, and 10, respectively. The control parameter v of the PAOTO algorithm is usually set to 1.


### 5.2. Benchmarks

The simulations are carried out based on the above setting. In order to better manifest the advantages and effectiveness of the proposed algorithm, two typical benchmarks are implemented for comparison with the PAOTO algorithm, which are presented as follows:(1)
Random offloading algorithm: the random offloading algorithm is chosen as one of the baselines, which will arrange the offloading in a random way. This is the method for the resource-constrained mobile device to decentralize computing tasks. However, it does consider the privacy preservation and performance optimization. The purpose of this benchmark is to evaluate the necessity of the proposed algorithm(2)
Dyna-Q algorithm: we implemented the Dyna-Q algorithm as one of the benchmarks in our simulations. The implementation details may be slightly different from that of [15], but the main framework is the same. The Dyna-Q in [15] is a reinforcement learning- (RL-) based privacy-aware offloading scheme. It is an improvement of the Q-learning method, combining the model-independent and model-dependent methods. But, it requires more system-level information (e.g., assumption of the Markov model) than the proposed algorithm. As the most state-of-the-art and relevant scheme to our works, the implementation of Dyna-Q can bring more reliable performance guarantees for our algorithm evaluation(3)
No privacy scenario: the scenario that does not consider privacy protection is also used as one of our baselines. According to [10], when mobile devices do not consider privacy protection but focuses solely on optimizing delay and energy consumption, the optimal latency and energy consumption performance can be obtained. Comparing with a scenario that does not consider privacy protection, it can reflect that the proposed algorithm will compromise the system performance in order to protect user’s privacyAs such, our algorithms are comparing the performance with these two benchmarks for analysis and these values match those used in previous works.


### 5.3. Numerical Results

In this section, the numerical results are presented to evaluate the effectiveness of the proposed algorithm. The weighted sum-cost, privacy level, computing delay, and energy consumption cost of the PAOTO algorithm in a period of time are compared with the two benchmarks to evaluate the performance of the proposed algorithm.


#### 5.3.1. The First Set of Simulations

In the first set of simulations, we randomly generate some tasks for the mobile device at each time slott, which are the same for the three algorithms. The number of newly generated tasks are taken randomly within [5, 30]. The task offloading policy is executed for each task per round. Since the number of tasks in each round is dynamic, the simulation results are averaged for each task. The results of average weighted sum-cost, privacy level, computing delay, and energy consumption cost are reported as the following and the results are plotted at every 100 time slots.As shown in Figure2, we trace the average weighted sum-cost of the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm at each time slot t. It can be seen that the PAOTO algorithm can obtain lower system cost for each task with about 23.0% reduction comparing to the Dyna-Q algorithm and about 50.1% to the random offloading at the 1000th time slot. However, compared to another scenario that the privacy is not considered, the proposed PAOTO algorithm has a higher system cost. This shows that the PAOTA algorithm has compromised the cost in order to protect privacy, which obtains the suboptimal solution.Figure 2
The simulation result comparison of time average and task average weighted sum-cost for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.According to Figure3, we can see that the proposed algorithm achieves better performance of the privacy-preserving level comparing to the random offloading and Dyna-Q algorithm. The proposed algorithm improves 4.8% and 19.1% of the privacy level compared with the Dyna-Q scheme and random offloading algorithm, respectively, at the 2000th time slot. The performance comparison of computing delay and energy consumption cost also verifies the improvement of the proposed algorithm, which are shown in Figures 4 and 5, respectively. For instance, compared to Dyna-Q, the computing delay and the energy consumption cost of the PAOTO algorithm decrease by 22.5% and 25.4%, respectively, at the 1000th time slot. It is a pity that compared to the scenarios that privacy is not considered (i.e., the optimal solution), the latency and energy consumption performance of the proposed algorithm is slightly worse. Because it sacrifices some performance in order to preserve privacy. Besides, the simulation results of the random offloading scheme are very poor, which further proves that it is of great significance to study the task offloading and preserve the privacy of the users in the MEC system. Given these facts in the first set of simulations, it can be observed that the PAOTO algorithm outperforms the other two benchmarks and it obtains suboptimal task offloading performance while protecting user’s privacy.Figure 3
The simulation result comparison of the time average and task average privacy level for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm.Figure 4
The simulation result comparison of the time average and task average computing delay for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.Figure 5
The simulation result comparison of the time average and task average energy consumption cost for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.


#### 5.3.2. The Second Set of Simulations

In the second set of simulations, we investigate the performance of the proposed algorithm with different maximum number of input computation tasksNmaxt, which ranged from 10 to 60. The data size of each task is uniformly distributed in (300, 800 K) bits. These simulation results in the second set are averaged over the first 2000 time slots. As shown in Figures 6–8, the PAOTO algorithm can get a lower average weighted sum-cost, computing delay, and the energy consumption cost than the other two benchmarks. And the improvements of these performances (histogram difference) increase as the Nmaxt increases from 10 to 60. For instance, when the number of computing tasks is 20, compared with Dyna-Q, the average sum-cost, computing delay, and the energy consumption cost of the PAOTO algorithm increase by 28.1%, 44.7%, and 28.6%, respectively. Whereas, when the number of computing tasks is 60, they are 29.3%, 51.6%, and 37.2%, respectively. The reason is that as the total number of tasks increases, the Dyna-Q algorithm requires more time to learn, and the random offloading does not have any performance optimization effects, but the proposed algorithm has stable processing efficiency to obtain a lower cost.Figure 6
The simulation result comparison of time average weighted sum-cost for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Figure 7
The simulation result comparison of time average computing delay for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Figure 8
The simulation result comparison of time average energy consumption for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Additionally, as shown in Figure9, with the increment of the number of tasks, the privacy level of the proposed algorithm will increase significantly but the privacy level of the Dyna-Q algorithm will decrease slightly. It is also because the processing efficiency of the Dyna-Q algorithm will decrease as the number of tasks increases. Besides, the privacy level of the random offloading algorithm is not affected by the number of tasks. Hence, the simulation results of the second set validate that the PAOTO algorithm has superior and stable system performance and privacy-preserving level for increasing computing-intensive tasks.Figure 9
The simulation result comparison of the time average privacy level for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.From the two set of simulations mentioned above, it can be seen that the PAOTO algorithm meets the objective of this paper that receives the close-to-optimal delay and energy consumption performance for MD while protecting the user’s privacy. And it has a significant performance improvement comparing to the other two benchmarks.


#### 5.3.3. The Third Set of Simulations

In order to analyze the effect of different key parameters (i.e.,δ and K) on the PAOTO algorithm, the weighted sum-cost is plotted under different values of v and the number of dummy tasks K. First, the parameter v in the PAOTO algorithm is associated with the standard deviation of the sampling, where v=2R+66dlogT/δ,δ∈0,1. Thus, we set the values of v as 0.1, 1, 5, 10, 15, and 20. As shown in Figure 10, we can observe that the values of v and the average weighted sum-cost of the PAOTO algorithm are positively correlated when v≥1, such as the curves v=1, v=5, and v=10. As the value of v is larger, the convergence of the PAOTO algorithm becomes worse. However, when v<1, the average cost of the PAOTO algorithm will increase with the decrease of v, such as curves v=1 and v=0.1. The reason is the cost tradeoff in the theoretical bound, and there are different effects before and after reaching the bound.Figure 10
Performance comparison with different values ofv.Second, the different number of dummy tasksK that is related to privacy metric Pt is simulated for weighted sum-cost of the proposed algorithm. As shown in Figure 11, K can be set as 1, 5, 10, and 15. Then, we can observe that as K increases, the weighted sum-cost will also increase. However, as K is larger, the increment of the weighted sum-cost will decrease. According to equations (8) and (9) in Section 3, K is directly proportional to the privacy metric Pt and the privacy metric Pt is inversely proportional to the weighted sum-cost. When the number of dummy tasks K increases, the system cost will increase at the beginning. Nevertheless, taking dummy tasks into the privacy metric can restrict the increment of the weighted sum-cost. That is, there is a tradeoff between the system cost and the privacy metric.Figure 11
Performance comparison with different values ofK.


## 5.1. Simulation Settings

In our simulation environment, we consider a MEC system, in which the access point is deployed with the MEC server. TheNt computing tasks are randomly generated by the mobile device at every time slot t, where the maximum of Nt is in the range from 10 to 60. The maximum buffer capacity bmax of the mobile device can be set to 10. The length of each time slot is 1 s. The total computation capacity for MEC server γs is uniformly distributed in (10, 15) GHz. In order to accommodate the dynamics, we assume that the computation capacity of the mobile device γl is determined randomly from 1 to 3 GHz and the converted computation capacity of buffer γb is uniformly distributed in (0.1, 0.15) GHz. Based on [33], the data size of each task λn is distributed in (300 K, 800 K) bits and the computation intensity δn is taken randomly within (250, 1000) CPU cycles/bit. Thus, we can get the required CPU cycles of a computing task mn by mn=λn∗δn (CPU cycles). Besides, the energy consumption for transmitting one task to the MEC server is uniformly distributed in (0.1, 0.5) J and the mobile device consumes 0.8 J to 3 J to locally compute one task and 0.5 J to 1 J to buffer one task. The weights of processing delay, energy consumption, and privacy metric, which are ωdelayt, ωenergyt and ωprivacyt,respectively, can be dynamically set by the users according to the user’s preferences and the running application demands. In our simulation, we set them to 1, 1, and 10, respectively. The control parameter v of the PAOTO algorithm is usually set to 1.


## 5.2. Benchmarks

The simulations are carried out based on the above setting. In order to better manifest the advantages and effectiveness of the proposed algorithm, two typical benchmarks are implemented for comparison with the PAOTO algorithm, which are presented as follows:(1)
Random offloading algorithm: the random offloading algorithm is chosen as one of the baselines, which will arrange the offloading in a random way. This is the method for the resource-constrained mobile device to decentralize computing tasks. However, it does consider the privacy preservation and performance optimization. The purpose of this benchmark is to evaluate the necessity of the proposed algorithm(2)
Dyna-Q algorithm: we implemented the Dyna-Q algorithm as one of the benchmarks in our simulations. The implementation details may be slightly different from that of [15], but the main framework is the same. The Dyna-Q in [15] is a reinforcement learning- (RL-) based privacy-aware offloading scheme. It is an improvement of the Q-learning method, combining the model-independent and model-dependent methods. But, it requires more system-level information (e.g., assumption of the Markov model) than the proposed algorithm. As the most state-of-the-art and relevant scheme to our works, the implementation of Dyna-Q can bring more reliable performance guarantees for our algorithm evaluation(3)
No privacy scenario: the scenario that does not consider privacy protection is also used as one of our baselines. According to [10], when mobile devices do not consider privacy protection but focuses solely on optimizing delay and energy consumption, the optimal latency and energy consumption performance can be obtained. Comparing with a scenario that does not consider privacy protection, it can reflect that the proposed algorithm will compromise the system performance in order to protect user’s privacyAs such, our algorithms are comparing the performance with these two benchmarks for analysis and these values match those used in previous works.


## 5.3. Numerical Results

In this section, the numerical results are presented to evaluate the effectiveness of the proposed algorithm. The weighted sum-cost, privacy level, computing delay, and energy consumption cost of the PAOTO algorithm in a period of time are compared with the two benchmarks to evaluate the performance of the proposed algorithm.


### 5.3.1. The First Set of Simulations

In the first set of simulations, we randomly generate some tasks for the mobile device at each time slott, which are the same for the three algorithms. The number of newly generated tasks are taken randomly within [5, 30]. The task offloading policy is executed for each task per round. Since the number of tasks in each round is dynamic, the simulation results are averaged for each task. The results of average weighted sum-cost, privacy level, computing delay, and energy consumption cost are reported as the following and the results are plotted at every 100 time slots.As shown in Figure2, we trace the average weighted sum-cost of the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm at each time slot t. It can be seen that the PAOTO algorithm can obtain lower system cost for each task with about 23.0% reduction comparing to the Dyna-Q algorithm and about 50.1% to the random offloading at the 1000th time slot. However, compared to another scenario that the privacy is not considered, the proposed PAOTO algorithm has a higher system cost. This shows that the PAOTA algorithm has compromised the cost in order to protect privacy, which obtains the suboptimal solution.Figure 2
The simulation result comparison of time average and task average weighted sum-cost for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.According to Figure3, we can see that the proposed algorithm achieves better performance of the privacy-preserving level comparing to the random offloading and Dyna-Q algorithm. The proposed algorithm improves 4.8% and 19.1% of the privacy level compared with the Dyna-Q scheme and random offloading algorithm, respectively, at the 2000th time slot. The performance comparison of computing delay and energy consumption cost also verifies the improvement of the proposed algorithm, which are shown in Figures 4 and 5, respectively. For instance, compared to Dyna-Q, the computing delay and the energy consumption cost of the PAOTO algorithm decrease by 22.5% and 25.4%, respectively, at the 1000th time slot. It is a pity that compared to the scenarios that privacy is not considered (i.e., the optimal solution), the latency and energy consumption performance of the proposed algorithm is slightly worse. Because it sacrifices some performance in order to preserve privacy. Besides, the simulation results of the random offloading scheme are very poor, which further proves that it is of great significance to study the task offloading and preserve the privacy of the users in the MEC system. Given these facts in the first set of simulations, it can be observed that the PAOTO algorithm outperforms the other two benchmarks and it obtains suboptimal task offloading performance while protecting user’s privacy.Figure 3
The simulation result comparison of the time average and task average privacy level for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm.Figure 4
The simulation result comparison of the time average and task average computing delay for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.Figure 5
The simulation result comparison of the time average and task average energy consumption cost for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.


### 5.3.2. The Second Set of Simulations

In the second set of simulations, we investigate the performance of the proposed algorithm with different maximum number of input computation tasksNmaxt, which ranged from 10 to 60. The data size of each task is uniformly distributed in (300, 800 K) bits. These simulation results in the second set are averaged over the first 2000 time slots. As shown in Figures 6–8, the PAOTO algorithm can get a lower average weighted sum-cost, computing delay, and the energy consumption cost than the other two benchmarks. And the improvements of these performances (histogram difference) increase as the Nmaxt increases from 10 to 60. For instance, when the number of computing tasks is 20, compared with Dyna-Q, the average sum-cost, computing delay, and the energy consumption cost of the PAOTO algorithm increase by 28.1%, 44.7%, and 28.6%, respectively. Whereas, when the number of computing tasks is 60, they are 29.3%, 51.6%, and 37.2%, respectively. The reason is that as the total number of tasks increases, the Dyna-Q algorithm requires more time to learn, and the random offloading does not have any performance optimization effects, but the proposed algorithm has stable processing efficiency to obtain a lower cost.Figure 6
The simulation result comparison of time average weighted sum-cost for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Figure 7
The simulation result comparison of time average computing delay for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Figure 8
The simulation result comparison of time average energy consumption for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Additionally, as shown in Figure9, with the increment of the number of tasks, the privacy level of the proposed algorithm will increase significantly but the privacy level of the Dyna-Q algorithm will decrease slightly. It is also because the processing efficiency of the Dyna-Q algorithm will decrease as the number of tasks increases. Besides, the privacy level of the random offloading algorithm is not affected by the number of tasks. Hence, the simulation results of the second set validate that the PAOTO algorithm has superior and stable system performance and privacy-preserving level for increasing computing-intensive tasks.Figure 9
The simulation result comparison of the time average privacy level for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.From the two set of simulations mentioned above, it can be seen that the PAOTO algorithm meets the objective of this paper that receives the close-to-optimal delay and energy consumption performance for MD while protecting the user’s privacy. And it has a significant performance improvement comparing to the other two benchmarks.


### 5.3.3. The Third Set of Simulations

In order to analyze the effect of different key parameters (i.e.,δ and K) on the PAOTO algorithm, the weighted sum-cost is plotted under different values of v and the number of dummy tasks K. First, the parameter v in the PAOTO algorithm is associated with the standard deviation of the sampling, where v=2R+66dlogT/δ,δ∈0,1. Thus, we set the values of v as 0.1, 1, 5, 10, 15, and 20. As shown in Figure 10, we can observe that the values of v and the average weighted sum-cost of the PAOTO algorithm are positively correlated when v≥1, such as the curves v=1, v=5, and v=10. As the value of v is larger, the convergence of the PAOTO algorithm becomes worse. However, when v<1, the average cost of the PAOTO algorithm will increase with the decrease of v, such as curves v=1 and v=0.1. The reason is the cost tradeoff in the theoretical bound, and there are different effects before and after reaching the bound.Figure 10
Performance comparison with different values ofv.Second, the different number of dummy tasksK that is related to privacy metric Pt is simulated for weighted sum-cost of the proposed algorithm. As shown in Figure 11, K can be set as 1, 5, 10, and 15. Then, we can observe that as K increases, the weighted sum-cost will also increase. However, as K is larger, the increment of the weighted sum-cost will decrease. According to equations (8) and (9) in Section 3, K is directly proportional to the privacy metric Pt and the privacy metric Pt is inversely proportional to the weighted sum-cost. When the number of dummy tasks K increases, the system cost will increase at the beginning. Nevertheless, taking dummy tasks into the privacy metric can restrict the increment of the weighted sum-cost. That is, there is a tradeoff between the system cost and the privacy metric.Figure 11
Performance comparison with different values ofK.


## 5.3.1. The First Set of Simulations

In the first set of simulations, we randomly generate some tasks for the mobile device at each time slott, which are the same for the three algorithms. The number of newly generated tasks are taken randomly within [5, 30]. The task offloading policy is executed for each task per round. Since the number of tasks in each round is dynamic, the simulation results are averaged for each task. The results of average weighted sum-cost, privacy level, computing delay, and energy consumption cost are reported as the following and the results are plotted at every 100 time slots.As shown in Figure2, we trace the average weighted sum-cost of the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm at each time slot t. It can be seen that the PAOTO algorithm can obtain lower system cost for each task with about 23.0% reduction comparing to the Dyna-Q algorithm and about 50.1% to the random offloading at the 1000th time slot. However, compared to another scenario that the privacy is not considered, the proposed PAOTO algorithm has a higher system cost. This shows that the PAOTA algorithm has compromised the cost in order to protect privacy, which obtains the suboptimal solution.Figure 2
The simulation result comparison of time average and task average weighted sum-cost for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.According to Figure3, we can see that the proposed algorithm achieves better performance of the privacy-preserving level comparing to the random offloading and Dyna-Q algorithm. The proposed algorithm improves 4.8% and 19.1% of the privacy level compared with the Dyna-Q scheme and random offloading algorithm, respectively, at the 2000th time slot. The performance comparison of computing delay and energy consumption cost also verifies the improvement of the proposed algorithm, which are shown in Figures 4 and 5, respectively. For instance, compared to Dyna-Q, the computing delay and the energy consumption cost of the PAOTO algorithm decrease by 22.5% and 25.4%, respectively, at the 1000th time slot. It is a pity that compared to the scenarios that privacy is not considered (i.e., the optimal solution), the latency and energy consumption performance of the proposed algorithm is slightly worse. Because it sacrifices some performance in order to preserve privacy. Besides, the simulation results of the random offloading scheme are very poor, which further proves that it is of great significance to study the task offloading and preserve the privacy of the users in the MEC system. Given these facts in the first set of simulations, it can be observed that the PAOTO algorithm outperforms the other two benchmarks and it obtains suboptimal task offloading performance while protecting user’s privacy.Figure 3
The simulation result comparison of the time average and task average privacy level for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm.Figure 4
The simulation result comparison of the time average and task average computing delay for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.Figure 5
The simulation result comparison of the time average and task average energy consumption cost for the PAOTO algorithm, random offloading algorithm, Dyna-Q algorithm, and no privacy scenario.


## 5.3.2. The Second Set of Simulations

In the second set of simulations, we investigate the performance of the proposed algorithm with different maximum number of input computation tasksNmaxt, which ranged from 10 to 60. The data size of each task is uniformly distributed in (300, 800 K) bits. These simulation results in the second set are averaged over the first 2000 time slots. As shown in Figures 6–8, the PAOTO algorithm can get a lower average weighted sum-cost, computing delay, and the energy consumption cost than the other two benchmarks. And the improvements of these performances (histogram difference) increase as the Nmaxt increases from 10 to 60. For instance, when the number of computing tasks is 20, compared with Dyna-Q, the average sum-cost, computing delay, and the energy consumption cost of the PAOTO algorithm increase by 28.1%, 44.7%, and 28.6%, respectively. Whereas, when the number of computing tasks is 60, they are 29.3%, 51.6%, and 37.2%, respectively. The reason is that as the total number of tasks increases, the Dyna-Q algorithm requires more time to learn, and the random offloading does not have any performance optimization effects, but the proposed algorithm has stable processing efficiency to obtain a lower cost.Figure 6
The simulation result comparison of time average weighted sum-cost for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Figure 7
The simulation result comparison of time average computing delay for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Figure 8
The simulation result comparison of time average energy consumption for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.Additionally, as shown in Figure9, with the increment of the number of tasks, the privacy level of the proposed algorithm will increase significantly but the privacy level of the Dyna-Q algorithm will decrease slightly. It is also because the processing efficiency of the Dyna-Q algorithm will decrease as the number of tasks increases. Besides, the privacy level of the random offloading algorithm is not affected by the number of tasks. Hence, the simulation results of the second set validate that the PAOTO algorithm has superior and stable system performance and privacy-preserving level for increasing computing-intensive tasks.Figure 9
The simulation result comparison of the time average privacy level for the PAOTO algorithm, random offloading algorithm, and Dyna-Q algorithm under different numbers of input computation tasks.From the two set of simulations mentioned above, it can be seen that the PAOTO algorithm meets the objective of this paper that receives the close-to-optimal delay and energy consumption performance for MD while protecting the user’s privacy. And it has a significant performance improvement comparing to the other two benchmarks.


## 5.3.3. The Third Set of Simulations

In order to analyze the effect of different key parameters (i.e.,δ and K) on the PAOTO algorithm, the weighted sum-cost is plotted under different values of v and the number of dummy tasks K. First, the parameter v in the PAOTO algorithm is associated with the standard deviation of the sampling, where v=2R+66dlogT/δ,δ∈0,1. Thus, we set the values of v as 0.1, 1, 5, 10, 15, and 20. As shown in Figure 10, we can observe that the values of v and the average weighted sum-cost of the PAOTO algorithm are positively correlated when v≥1, such as the curves v=1, v=5, and v=10. As the value of v is larger, the convergence of the PAOTO algorithm becomes worse. However, when v<1, the average cost of the PAOTO algorithm will increase with the decrease of v, such as curves v=1 and v=0.1. The reason is the cost tradeoff in the theoretical bound, and there are different effects before and after reaching the bound.Figure 10
Performance comparison with different values ofv.Second, the different number of dummy tasksK that is related to privacy metric Pt is simulated for weighted sum-cost of the proposed algorithm. As shown in Figure 11, K can be set as 1, 5, 10, and 15. Then, we can observe that as K increases, the weighted sum-cost will also increase. However, as K is larger, the increment of the weighted sum-cost will decrease. According to equations (8) and (9) in Section 3, K is directly proportional to the privacy metric Pt and the privacy metric Pt is inversely proportional to the weighted sum-cost. When the number of dummy tasks K increases, the system cost will increase at the beginning. Nevertheless, taking dummy tasks into the privacy metric can restrict the increment of the weighted sum-cost. That is, there is a tradeoff between the system cost and the privacy metric.Figure 11
Performance comparison with different values ofK.


## 6. Conclusions

In this paper, we investigated joint task offloading and privacy preservation for the small-size and low-power mobile devices without any system-level network information in the MEC system. The objective is to minimize a weighted sum of the computing delay, energy consumption cost, and reciprocal of the privacy metric. In particular, the joint optimization problem has been formulated as a contextual MAB problem with a semiparametric reward model to accommodate network dynamics, in which the privacy metric is taken into account. Subsequently, a privacy-aware online task offloading (PAOTO) algorithm is proposed to explore the balance between the optimal system cost and the privacy level. The simulation results show that the proposed algorithm can provide near-optimal solutions in a short computing time. In the future, we will extend our work to the scenarios that have multiple MEC servers with distinct computing capability and take the bandwidth constraint into account.



---

*Source: 6622947-2021-06-11.xml*