# Private Computing Offloading in Edge Cloud via Collaborative Online Learning

Authors: Lin Wang; Lei Yang; Mingchuan Zhang; Jianxin Zhang; Zhibin Cao; Qingtao Wu
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Computing offloading based on mobile edge computing (MEC) for mobile devices (MDs) has received great attentions in recent years. Strategy selection is an extremely important part of computing offloading, so how to make an optimal decision quickly and accurately during the computing offloading is a difficult point. Furthermore, MDs are likely to leak personal privacy when interacting with edge cloud, and there is also an issue about commercial privacy leakage between different cloud service suppliers. In this paper, we propose the privacy-protected edge cloud computing offloading (EPCO) algorithm based on online learning to improve the efficiency of computing offloading while ensuring the privacy of system users. Simultaneously, EPCO also supports different MDs customize their privacy level. We prove that adding privacy protection mechanism is almost no effect on the convergence of the algorithm. The simulation results validate our conclusion using a real-world dataset.


---

## Body

## 1. Introduction

Mobile devices (MDs) have become extremely popular in recent years due to their mobility and convenience [1, 3]. Meanwhile, the functionality of the application for MDs becomes increasingly powerful [3], which leads to lack of local resources of MDs, such as computing resources, storage, and energy [4, 5]. To this end, computing offloading for MDs has emerged. Researchers proposed mobile cloud computing (MCC) that source starvation can be resolved by sending computing tasks of MDs to remote cloud for execution [6, 7]. However, since cloud servers are often far away from MDs, the data needs to be transmitted for a long distance, which results in a long response time. To this end, much research in recent years has focused on mobile edge computing (MEC) [8], which sends computing tasks to edge cloud servers (ECSs) [9]. ECSs are typically deployed around MDs, which enables a short physical distance between MDs and servers, resulting a shorter latency [10]. The work of this paper is based on the edge cloud network.Strategy selection is an important part of computing offloading between ECSs and MDs [1]. When a MD decides to offload its computing tasks to an ECS, it must first make a decision to select an optimal server for computing offloading. Researchers have used game theory in past research to solve the problem of selecting servers for computing offloading [11], which was also significantly effective at the time. However, with the increasing demands of users on the quality of network services and the challenges of big data [12, 13], most of the previous studies are outdated. In recent years, online learning algorithms have been greatly developed and used in various fields to help improve system’s performance [14–18]. Therefore, we consider using online learning algorithms to solve strategy selection problem of computing offloading. Furthermore, not only the efficiency of computing offloading should be considered, but the privacy of system users should also be concerned. However, few researches involve the above two aspects.Privacy protection is an important part of the computing offloading [19]. The privacy issues we consider include the following two parts: the privacy of the MDs and the service suppliers. On the one hand, a MD’s privacy may be exposed during data transmission or forwarding if there is a malicious third party involved. The malicious third party can infer the characteristics of the MD by accessing the computing offloading records. For example, a large amount of computation indicates the importance of the MD and the distance item exposes the location of the MD. With multiple of these side information, it is possible to identify a user in real world. For instance, He et al. [19] proposed a privacy-aware task offloading algorithm, which enabled low delay and energy consumption while maintaining an appropriate level of privacy. Min et al. [20] proposed a privacy-aware offloading algorithm that can improve the offloading performance, save energy, and enable privacy of healthcare IoT devices. Although these studies focus on the privacy issue in offloading, they only avoid the possibility of privacy leakage through some transmission method, so the privacy protection effect of these method is limited. On the other hand, since there is commercial competition between service suppliers, the privacy between them should also be considered. Therefore, privacy protection is another important part of the computing offloading. However, the research that considers both strategy selection and privacy protection is barely known.To overcome above challenge, we consider introducing differential privacy into our computing offloading scenario. Differential privacy which proposed by Dwork et al. [21] has received great attention in the field of privacy protection in recent years. Differential privacy uses random noise to ensure that the private information of the individual will not be disclosed when the result of a query requests to disclose visible information. Zhang et al. and Hassan et al. [22, 23] use differential privacy techniques to address the risk of privacy leakage in their systems. This paper is motivated by the unresolved privacy risks in computing offloading scenarios. The privacy of mobile device users, such as location and device usage, may be leaked through data exchange during the offloading process, which is a potential privacy breach risk for users. According to the research of He et al. [19] and Min et al. [20], privacy risk has indeed become an important issue in computing offloading. Although they paid attention to the privacy issues in computing offloading, these algorithms only avoided the possibility of privacy leakage through a certain transmission method and did not fundamentally solve the problem of privacy risks. Therefore, we introduce differential privacy technology and propose an algorithm EPCO that protects the privacy of multiparty users in computing offloading. In this paper, the system structure is shown in Figure 1. For instance, a healthcare device needs to compute a large amount of monitoring data, so part of the computing tasks should be offloaded to the edge cloud for computing. The device sends the encrypted offloading data to the edge cloud, and then, each service supplier in the edge cloud gives an optimal offloading plan through online learning algorithms, and finally, the device makes a decision. Our main contributions are as follows:
(i)
In this paper, we propose EPCO algorithm for MD and edge cloud to perform computing offloading based on online learning and differential privacy technology(ii)
EPCO preserves the privacy of both MDs and service suppliers. Moreover, we support different MDs to customize their own privacy protection levels. We proved it through theoretical derivation(iii)
We verified the theory through simulation experiments. The results show that EPCO guarantees the efficiency of computing offloading while protecting the privacy of system usersFigure 1
The structure of edge cloud in this paper.


## 2. Related Work

In this section, we introduce related work from two aspects: optimal offloading and privacy management.


### 2.1. Optimal Offloading

Computing offloading alleviates the limitation of MDs’ resources by sending computing tasks to the remote cloud for execution [24]. There has been a lot of research work on computing offloading in the past two decades [25–27]. Selecting ECSs for MDs is an indispensable part of computing offloading [28]. Previous decisions about server selection usually used the game theory method, which mainly concerned about energy conservation, network environment perception, and so on. Jošilo and Dán [29] made the decision to choose the wireless access points for mobile users during the computing offloading process. They proved that there is a Nash equilibrium in the model they propose, which can maximize the benefits of all users. However, this solution requires multiple interactions between users and computing resources to obtain better results, which is very unfriendly to time-sensitive applications. Barrameda and Samaan [30] considered to use tree execution dependency trees to enhance the accuracy, which is an important technical indicator in computing offloading. However, this solution is designed for a central cloud with a large number of computing resources. First, it needs to use a large amount of additional computing to run the algorithm, and secondly, the service response time cannot be guaranteed. Although these studies have solved some specific problems, they are gradually unable to adapt to the more complicated situation such as the challenges of big data and the personalized needs of users.In recent years, online learning has been used in various fields to help improve system performance. Shahrampour et al. [31] used online learning in object recognition to help identify the current object through historical information from other modes. Sakulkar and Krishnamachari [32] proposed two online learning algorithms to help them solve the power allocation problem modelled as a Markov decision process. We consider applying online learning to the scenario of computing offloading, which can help us further improve the efficiency of computing offloading. Cao and Cai [33] used machine learning technology to solve the decision problems of MD for achieving Nash equilibrium points in the noncooperative game that they proposed, which shows the prospect of machine learning in computing offloading research. Therefore, these studies bring us new ideas for strategy selection of computing offloading. We consider that if we try to apply online learning theory and technology to the scenario of computing offloading, this can help us further improve the efficiency of computing offloading.


### 2.2. Privacy Management

As more and more people pay attention to personal privacy, the issue of user privacy protection should also be considered in computing offloading. Differential privacy is a popular research in recent years in terms of privacy protection. Differential privacy was first proposed by Dwork [34] and gave provable differential privacy protection. In recent years, people have realized the importance of privacy, and privacy has been used in various research fields to protect users’ privacy [35, 36]. Shin et al. [37] proposed a novel matrix factorization algorithm that guarantees per-user privacy under local differential privacy. In addition, they reduced communication overhead between the server and users by dimensionality reduction. Piao et al. [38] proposed an algorithm that can reduce query sensitivity and improved the effectiveness of published data. The above researches on differential privacy only provide same level of privacy protection, which is not practical in many applications. Dobbe et al. [39] proposed a customized local differential privacy mechanism to solve the privacy protection problem in multiagent distributed optimization problems. They proposed an approach for determining sensitivity, and they derived analytical bounds for some quadratic problems. The customizable ideas mentioned here have been adopted by us. In this paper, we allow different MDs to customize their privacy protection levels.Since few researchers have paid attention to the privacy protection of computing offloading before, there are not many related research contributions, but we will continue to pay attention to the research progress in this area.


## 2.1. Optimal Offloading

Computing offloading alleviates the limitation of MDs’ resources by sending computing tasks to the remote cloud for execution [24]. There has been a lot of research work on computing offloading in the past two decades [25–27]. Selecting ECSs for MDs is an indispensable part of computing offloading [28]. Previous decisions about server selection usually used the game theory method, which mainly concerned about energy conservation, network environment perception, and so on. Jošilo and Dán [29] made the decision to choose the wireless access points for mobile users during the computing offloading process. They proved that there is a Nash equilibrium in the model they propose, which can maximize the benefits of all users. However, this solution requires multiple interactions between users and computing resources to obtain better results, which is very unfriendly to time-sensitive applications. Barrameda and Samaan [30] considered to use tree execution dependency trees to enhance the accuracy, which is an important technical indicator in computing offloading. However, this solution is designed for a central cloud with a large number of computing resources. First, it needs to use a large amount of additional computing to run the algorithm, and secondly, the service response time cannot be guaranteed. Although these studies have solved some specific problems, they are gradually unable to adapt to the more complicated situation such as the challenges of big data and the personalized needs of users.In recent years, online learning has been used in various fields to help improve system performance. Shahrampour et al. [31] used online learning in object recognition to help identify the current object through historical information from other modes. Sakulkar and Krishnamachari [32] proposed two online learning algorithms to help them solve the power allocation problem modelled as a Markov decision process. We consider applying online learning to the scenario of computing offloading, which can help us further improve the efficiency of computing offloading. Cao and Cai [33] used machine learning technology to solve the decision problems of MD for achieving Nash equilibrium points in the noncooperative game that they proposed, which shows the prospect of machine learning in computing offloading research. Therefore, these studies bring us new ideas for strategy selection of computing offloading. We consider that if we try to apply online learning theory and technology to the scenario of computing offloading, this can help us further improve the efficiency of computing offloading.


## 2.2. Privacy Management

As more and more people pay attention to personal privacy, the issue of user privacy protection should also be considered in computing offloading. Differential privacy is a popular research in recent years in terms of privacy protection. Differential privacy was first proposed by Dwork [34] and gave provable differential privacy protection. In recent years, people have realized the importance of privacy, and privacy has been used in various research fields to protect users’ privacy [35, 36]. Shin et al. [37] proposed a novel matrix factorization algorithm that guarantees per-user privacy under local differential privacy. In addition, they reduced communication overhead between the server and users by dimensionality reduction. Piao et al. [38] proposed an algorithm that can reduce query sensitivity and improved the effectiveness of published data. The above researches on differential privacy only provide same level of privacy protection, which is not practical in many applications. Dobbe et al. [39] proposed a customized local differential privacy mechanism to solve the privacy protection problem in multiagent distributed optimization problems. They proposed an approach for determining sensitivity, and they derived analytical bounds for some quadratic problems. The customizable ideas mentioned here have been adopted by us. In this paper, we allow different MDs to customize their privacy protection levels.Since few researchers have paid attention to the privacy protection of computing offloading before, there are not many related research contributions, but we will continue to pay attention to the research progress in this area.


## 3. System Model

### 3.1. Computing Offloading

Consider an edge cloud network with a setM=1,⋯,M of mobile device and an edge cloud manager (ECM) that manages a set V=1,⋯,V of service suppliers. Each supplier has a set S=1,⋯,S of servers that can provide computing service. Each mobile device i∈M has a task that has been determined to perform computing offloading. As shown in Figure 1, at each slot t, a MD sends a d-dimensional context in X≔0,1d denoted by xt to all suppliers, where xt is added to Laplace noise based on the privacy protection requirements of different MDs. By receiving xt, the ECM first broadcasts it to all suppliers. Each supplier then selects an optimal ECS and sends the information of this ECS to the ECM. The ECM provides the MD with the optimal ECS in this network that denoted by st. Subsequently, the MD decides whether to perform computing offloading.Eachst∈S has a two-dimensional vector denoted by wt=wtp,wtq, where wtp and wtq denote the reward in the price and reliability for performing computing offloading, respectively. wtp and wtq are given by wtp=ystpxt+πtp and wtq=ystqxt+πtq, respectively, where ystkxt, k∈p,q denotes the expected reward of selecting ECS s in k given context x. πtk, k∈p,q is a random noise, which satisfies Eπtks1:t,x1:t,π1:t−1p,π1:t−1q=0. We assume that πtk, k∈p,q is conditionally 1-sub-Gaussian. Formally, this means that
(1)∀λ∈ℝEexpλπtks1:t,x1:t,π1:t−1p,π1:t−1q≤expλ22.Lety∗px≔maxs∈Syspx and y∗qx≔maxs∈Sysqx denote the expected reward of a ECS s in the price and the reliability for context x, respectively. Let s∗x denote the optimal ECS for the context x.Assumption 1.
We assume that for allk∈p,q, s∈S and x, x′∈X, yskx satisfy the following condition:
(2)yskx−yskx′≤Lx−x′α,
whereL>0, 0<α≤1. Assumption 1 means that if the offloading price and reliability of two ESCs are similar, it is expected that the cost of their offloading are similar.
Initially, the MD does not know any reward of ECSs. The MD learns the reward of ECSs over time. In order to evaluate the performance of our method, we define the 2D regret of the ECS as the tupleRpT,RqT, where
(3)RkT≔∑t=1Ty∗kxt−∑t=1Tystkxt,k∈p,q.
WhenRpT=OTβ1 and RqT=OTβ2, we consider that the 2D regret is OTmaxβ1,β2.


### 3.2. Differential Privacy

Definition 2 (differential privacy).
An algorithmG has ε differential privacy if there is only one entry different in all pairs D,D′∈ℝd, and all set of outcomes R∈RangeG.
(4)PGD∈RPGD′∈R≤expε.This definition mentioned above applies only to the identical level of privacy protection used by all suppliers. We now consider that each supplieri in our system specifies its own privacy εi.Definition 3 (local differential privacy).
An algorithmG~ has M nodes in the system, and we say that the algorithm G~ is εi locally private for node i,i=1,⋯,M if for any Ri∈RangeG~i it satisfies that
(5)PG~iD1,⋯,Di,⋯,DM∈RiPG~iD1,⋯,D′i,⋯,DM∈Ri≤expεi.And we say that the algorithmG~ is ε1,⋯,εM-differentially private, if G~ is εi-differentially locally private for all suppliers, where s=1,⋯,M.Definition 4 (sensitivity).
The sensitivity of the functiony:X↦Rd is as follows:
(6)Δf=maxX,X′X−X′1=1fX−fX′1.Definition 5 (sensitivity).
The sensitivity of the functionh:G↦Rd is as follows:
(7)Φi=maxGi,G′iGi−G′i1=1gGi−gG′i1.


### 3.3. The Learning Algorithm

In this section, we detail our proposed EPCO as shown in Algorithm1 (EPCO(1)), Algorithm 2 (EPCO(2)), and Algorithm 3 (EPCO(3)). Since the computing offloading decision of each ECS for different MDs has stochastic distributions, we decide to let our proposed system learn an ECS’s performance by online learning method. According to the sample mean reward of each ECS for the same context vector update, service suppliers learn the performance of each ECS. In order to understand the EPCO, we divide it into three algorithms which are named as EPCO(1), EPCO(2), and EPCO(3), respectively. As shown in Figure 2, the MDs run EPCO(1) to customize their privacy protection level. ECM runs EPCO(2) to interact with the MD and send the best option among all agents to the MD for decisions. Service suppliers run EPCO(3) to select the optimal ECS and interact with the ECM.Figure 2
Algorithm mechanism.First, we analyse the privacy problem of MDs. Since different MDs have different requirements for privacy protection, MDs are allowed to customize the privacy level of each user. In order to protect personal privacy when MDs send computing information, the information is added with a noise which is drawn from the Laplace distribution in EPCO(1). Then, we discuss the privacy issues of the suppliers. When suppliers have selected optimal ECSs for MDs, they send the information to ECM. Since any supplier can access to this information in ECM, the Laplace mechanism is used in EPCO(2) to protect the privacy of service suppliers.Algorithm 1: EPCO(1).
Input:ẋ.Output:x.1: MD is ready for computing offloading;2:fẋ=ẋ+LapΔf/ε;3: Setx=fẋ;4: Sendx to the edge cloud;In this paper, the context spaceX is divided into md identical hypercubes with side length n−l. Let C denote the subspace context space of X. For ECS s and each c∈C, EPCO maintains a counter Ns,c recording the number of times that s was selected for the context that belongs to c. When a MD needs to perform a computing offloading, it first sends a message to the ECSs containing information about the computing task. In order to protect the privacy of the MD, it adds Laplace noise to this information in EPCO(1). Upon each context data of a MD arrival, the suppliers first identify to which subspace c the context belongs. Then, each service supplier first calculates the indices for the rewards (line 5 in EPCO(3)), which is given as follows:
(8)hs,ck≔y^s,ck+μs,c+ηsk,k∈p,q,where y^s,ck=ys,ck/Ns,c estimates the sample mean of the reward for the selection of s in subspace c. y^s,cp and y^s,cq denote the price objective and reliability objective, respectively. μs,c=2Sn,T/Ns,c, where Sn,T=1+2log4SndT3/2 denotes the uncertainty of the reward estimate, which is commonly used to tradeoff exploration and exploitation in online learning [40]. ηsk is a random noise obeying the Gaussian distribution. Then, the Upper Confidence Bound (UCB) for yskx is hs,ck+b for context x in subspace c, where b≔Ldα/2n−α denotes the uncertainty due to context partition. Its main purpose is to inflate the reward of ECSs that are seldomly selected, which is more conducive to exploring more suitable servers than just servers with high estimated reward.Algorithm 2: EPCO(2).
Input:xi.Output:s∗.1: Receivexi from the MD;2: Broadcastxi to each service suppliers;3: Receive optimal ECSs from all service suppliers;4: Setsi∗=argmaxv∈VJv,s;5: Sendsi∗ to the MD;6: Observe the decision of the MD and send it to the service suppliersAlgorithm 3: EPCO(3).
Input:xi,γ.Output:s∗.1: Initialize:C⊆X,Ns=0,∀s∈Si;2: Receivexi from the ECM;3:fort=1,2,⋯,T,xt∈cdo4: Computehs,ck,k∈p,q via (4);5:Is,ck≔hs,ck+ηsk,k∈p,q6:ifμsp∗,c+ηsk>γbthen7:  Setsp∗=sp∗;8:else9:  Find the candidate optimal set of ECSsS∗ via (5);10:  Setsp∗=argmaxs∈S∗gs,cq;11:end if12: Sends∗ and hs∗,c to the ECM;13: Receivewt=wtp,wq,t;14:y^st,ck⟵y^st,ckNst,ck+wtk/Nst,ck+1,k∈p,q15:Nst,ck⟵Nst,ck+116:t⟵t+1;17:end forWe add Laplace noiseηsk to the index function to protect the privacy of ECSs. When μsp∗,c+ηsk≤γb, it means that the confidence of sp∗ is high, and EPCO(3) calculates the candidate set of the optimal ECSs, which is given as follows:
(9)S∧∗≔s∈S:hs,cp≥y^sp∗,cp−μsp∗,c−ηsk−2b=s∈S:y^s,cp≥y^sp∗,cp−μsp∗,c−ηsk−μs,c∗−2b.Whenμsp∗,c+ηsk≥γb, EPCO(3) just set sp∗=argmaxs∈Shs,cp to improve the confidence of sp∗ (lines 6-7 in EPCO(3)). Simultaneously, an optimal ECS s^ is selected by the exponential mechanism (lines 12-15 in EPCO(2)). We use hs,c to denote the total reward of ECS s that can be compared, which is given as follows:
(10)Js,c=ψy^s,cp+1−ψy^s,cq,ψ∈0,1,whereψ represents a MD’s preference and is adjusted according to the actual needs of a MD. For example, if a MD requires strict service payment, the value of ψ is larger. However, if a MD requires strict reliability, the value of ψ is relatively small. We select the ECS with the highest total reward and sent it to the MD (line 12 in EPCO(3)). Finally, according to the computing offloading decision of the MD (line 7 in EPCO(2)), the service suppliers update the sample mean reward and the counter (lines 14-16 in EPCO(3)).


## 3.1. Computing Offloading

Consider an edge cloud network with a setM=1,⋯,M of mobile device and an edge cloud manager (ECM) that manages a set V=1,⋯,V of service suppliers. Each supplier has a set S=1,⋯,S of servers that can provide computing service. Each mobile device i∈M has a task that has been determined to perform computing offloading. As shown in Figure 1, at each slot t, a MD sends a d-dimensional context in X≔0,1d denoted by xt to all suppliers, where xt is added to Laplace noise based on the privacy protection requirements of different MDs. By receiving xt, the ECM first broadcasts it to all suppliers. Each supplier then selects an optimal ECS and sends the information of this ECS to the ECM. The ECM provides the MD with the optimal ECS in this network that denoted by st. Subsequently, the MD decides whether to perform computing offloading.Eachst∈S has a two-dimensional vector denoted by wt=wtp,wtq, where wtp and wtq denote the reward in the price and reliability for performing computing offloading, respectively. wtp and wtq are given by wtp=ystpxt+πtp and wtq=ystqxt+πtq, respectively, where ystkxt, k∈p,q denotes the expected reward of selecting ECS s in k given context x. πtk, k∈p,q is a random noise, which satisfies Eπtks1:t,x1:t,π1:t−1p,π1:t−1q=0. We assume that πtk, k∈p,q is conditionally 1-sub-Gaussian. Formally, this means that
(1)∀λ∈ℝEexpλπtks1:t,x1:t,π1:t−1p,π1:t−1q≤expλ22.Lety∗px≔maxs∈Syspx and y∗qx≔maxs∈Sysqx denote the expected reward of a ECS s in the price and the reliability for context x, respectively. Let s∗x denote the optimal ECS for the context x.Assumption 1.
We assume that for allk∈p,q, s∈S and x, x′∈X, yskx satisfy the following condition:
(2)yskx−yskx′≤Lx−x′α,
whereL>0, 0<α≤1. Assumption 1 means that if the offloading price and reliability of two ESCs are similar, it is expected that the cost of their offloading are similar.
Initially, the MD does not know any reward of ECSs. The MD learns the reward of ECSs over time. In order to evaluate the performance of our method, we define the 2D regret of the ECS as the tupleRpT,RqT, where
(3)RkT≔∑t=1Ty∗kxt−∑t=1Tystkxt,k∈p,q.
WhenRpT=OTβ1 and RqT=OTβ2, we consider that the 2D regret is OTmaxβ1,β2.


## 3.2. Differential Privacy

Definition 2 (differential privacy).
An algorithmG has ε differential privacy if there is only one entry different in all pairs D,D′∈ℝd, and all set of outcomes R∈RangeG.
(4)PGD∈RPGD′∈R≤expε.This definition mentioned above applies only to the identical level of privacy protection used by all suppliers. We now consider that each supplieri in our system specifies its own privacy εi.Definition 3 (local differential privacy).
An algorithmG~ has M nodes in the system, and we say that the algorithm G~ is εi locally private for node i,i=1,⋯,M if for any Ri∈RangeG~i it satisfies that
(5)PG~iD1,⋯,Di,⋯,DM∈RiPG~iD1,⋯,D′i,⋯,DM∈Ri≤expεi.And we say that the algorithmG~ is ε1,⋯,εM-differentially private, if G~ is εi-differentially locally private for all suppliers, where s=1,⋯,M.Definition 4 (sensitivity).
The sensitivity of the functiony:X↦Rd is as follows:
(6)Δf=maxX,X′X−X′1=1fX−fX′1.Definition 5 (sensitivity).
The sensitivity of the functionh:G↦Rd is as follows:
(7)Φi=maxGi,G′iGi−G′i1=1gGi−gG′i1.


## 3.3. The Learning Algorithm

In this section, we detail our proposed EPCO as shown in Algorithm1 (EPCO(1)), Algorithm 2 (EPCO(2)), and Algorithm 3 (EPCO(3)). Since the computing offloading decision of each ECS for different MDs has stochastic distributions, we decide to let our proposed system learn an ECS’s performance by online learning method. According to the sample mean reward of each ECS for the same context vector update, service suppliers learn the performance of each ECS. In order to understand the EPCO, we divide it into three algorithms which are named as EPCO(1), EPCO(2), and EPCO(3), respectively. As shown in Figure 2, the MDs run EPCO(1) to customize their privacy protection level. ECM runs EPCO(2) to interact with the MD and send the best option among all agents to the MD for decisions. Service suppliers run EPCO(3) to select the optimal ECS and interact with the ECM.Figure 2
Algorithm mechanism.First, we analyse the privacy problem of MDs. Since different MDs have different requirements for privacy protection, MDs are allowed to customize the privacy level of each user. In order to protect personal privacy when MDs send computing information, the information is added with a noise which is drawn from the Laplace distribution in EPCO(1). Then, we discuss the privacy issues of the suppliers. When suppliers have selected optimal ECSs for MDs, they send the information to ECM. Since any supplier can access to this information in ECM, the Laplace mechanism is used in EPCO(2) to protect the privacy of service suppliers.Algorithm 1: EPCO(1).
Input:ẋ.Output:x.1: MD is ready for computing offloading;2:fẋ=ẋ+LapΔf/ε;3: Setx=fẋ;4: Sendx to the edge cloud;In this paper, the context spaceX is divided into md identical hypercubes with side length n−l. Let C denote the subspace context space of X. For ECS s and each c∈C, EPCO maintains a counter Ns,c recording the number of times that s was selected for the context that belongs to c. When a MD needs to perform a computing offloading, it first sends a message to the ECSs containing information about the computing task. In order to protect the privacy of the MD, it adds Laplace noise to this information in EPCO(1). Upon each context data of a MD arrival, the suppliers first identify to which subspace c the context belongs. Then, each service supplier first calculates the indices for the rewards (line 5 in EPCO(3)), which is given as follows:
(8)hs,ck≔y^s,ck+μs,c+ηsk,k∈p,q,where y^s,ck=ys,ck/Ns,c estimates the sample mean of the reward for the selection of s in subspace c. y^s,cp and y^s,cq denote the price objective and reliability objective, respectively. μs,c=2Sn,T/Ns,c, where Sn,T=1+2log4SndT3/2 denotes the uncertainty of the reward estimate, which is commonly used to tradeoff exploration and exploitation in online learning [40]. ηsk is a random noise obeying the Gaussian distribution. Then, the Upper Confidence Bound (UCB) for yskx is hs,ck+b for context x in subspace c, where b≔Ldα/2n−α denotes the uncertainty due to context partition. Its main purpose is to inflate the reward of ECSs that are seldomly selected, which is more conducive to exploring more suitable servers than just servers with high estimated reward.Algorithm 2: EPCO(2).
Input:xi.Output:s∗.1: Receivexi from the MD;2: Broadcastxi to each service suppliers;3: Receive optimal ECSs from all service suppliers;4: Setsi∗=argmaxv∈VJv,s;5: Sendsi∗ to the MD;6: Observe the decision of the MD and send it to the service suppliersAlgorithm 3: EPCO(3).
Input:xi,γ.Output:s∗.1: Initialize:C⊆X,Ns=0,∀s∈Si;2: Receivexi from the ECM;3:fort=1,2,⋯,T,xt∈cdo4: Computehs,ck,k∈p,q via (4);5:Is,ck≔hs,ck+ηsk,k∈p,q6:ifμsp∗,c+ηsk>γbthen7:  Setsp∗=sp∗;8:else9:  Find the candidate optimal set of ECSsS∗ via (5);10:  Setsp∗=argmaxs∈S∗gs,cq;11:end if12: Sends∗ and hs∗,c to the ECM;13: Receivewt=wtp,wq,t;14:y^st,ck⟵y^st,ckNst,ck+wtk/Nst,ck+1,k∈p,q15:Nst,ck⟵Nst,ck+116:t⟵t+1;17:end forWe add Laplace noiseηsk to the index function to protect the privacy of ECSs. When μsp∗,c+ηsk≤γb, it means that the confidence of sp∗ is high, and EPCO(3) calculates the candidate set of the optimal ECSs, which is given as follows:
(9)S∧∗≔s∈S:hs,cp≥y^sp∗,cp−μsp∗,c−ηsk−2b=s∈S:y^s,cp≥y^sp∗,cp−μsp∗,c−ηsk−μs,c∗−2b.Whenμsp∗,c+ηsk≥γb, EPCO(3) just set sp∗=argmaxs∈Shs,cp to improve the confidence of sp∗ (lines 6-7 in EPCO(3)). Simultaneously, an optimal ECS s^ is selected by the exponential mechanism (lines 12-15 in EPCO(2)). We use hs,c to denote the total reward of ECS s that can be compared, which is given as follows:
(10)Js,c=ψy^s,cp+1−ψy^s,cq,ψ∈0,1,whereψ represents a MD’s preference and is adjusted according to the actual needs of a MD. For example, if a MD requires strict service payment, the value of ψ is larger. However, if a MD requires strict reliability, the value of ψ is relatively small. We select the ECS with the highest total reward and sent it to the MD (line 12 in EPCO(3)). Finally, according to the computing offloading decision of the MD (line 7 in EPCO(2)), the service suppliers update the sample mean reward and the counter (lines 14-16 in EPCO(3)).


## 4. Regret Analysis

In this section, we prove that the 2D regret of EPCO is sublinear functions ofT. The regret RkT is due to selecting suboptimal ECSs from Si by time T.Let(11)RckT≔∑t=1Ty∗kxct−∑t=1Tyεsδkxct,k∈p,qdenotes the regret for objectivek in round Tc. The best fixed ECS is denoted by y∗kx, y∗kx=maxs∈Sy∗kx, and k∈p,q. Then, we have the total regret for selecting suboptimal ECSs
(12)RkT=∑c∈CRckT.Then, the corresponding expected regret is given as follows:(13)ERkT=∑c∈CERckT.Leth¯s,ckt≔y^s,ckt−μs,ct−ηskt and h¯s,ckt≔y^s,ckt+μs,ct+ηskt denote lower and upper bounds for k∈p,q, respectively. Then, h¯s,ckt−b and h¯s,ckt+b are the lower and upper confidence bounds, respectively. Let
(14)Qs,ck≔⋃t=1NcTyskxst∉h¯s,ckt−b,h¯s,ckt+bdenote that the service supplier is not confident about the reward estimate by timeT with the context x in subspace c. Then, we partition the regret into following and bound them, respectively.
(15)ERck1T=ERck1TQPQ+ERck1TQ~PQ~≤Amaxk1NcTPQ+ERck1TQ~,where Q~ denotes the complement of event Q and Ak is the maximum difference between the expected reward in optimal server and other server for objective k. We use s to denote the server selected in EPCO(3) algorithm, s∗ to denote the optimal server, and s^ to denote the server whose index is highest. Next, we will bound the items in Equation (15). We first bound PQ.Lemma 6.
For anyc∈C, we have the following:
(16)PQc≤1ndT.Proof.
LetWs,ckt denote the random reward of server s in objective k in round t. We know
(17)Wsk,ct=yskxct+πckt+ηck,ysk,ct=∑l=1t−1Wsk,clIscl=sNs,ct.We define upper and lower bounds of the random reward as follows:(18)W^sk,ct=y^skxct+πckt+ηck,W˘sk,ct=y˘skxct+πckt+ηck.Let(19)y^skxct=∑l=1t−1W^sk,cl1scl=sNs,ct,y˘skxct=∑l=1t−1W˘sk,cl1scl=sNs,ct.Then, we have the following:(20)L^s,ckt≔y^skxct−μs,ct−ηck,U^s,ckt≔y^skxct+μs,ct+ηck,L˘s,ckt≔y˘skxct−μs,ct−ηck,U˘s,ckt≔y˘skxct+μs,ct+ηck.Since whenNs.ct=0, Pyskxct∈L^s,ckt−b,U^s,ckt+b=0, so below, we only focus on the case of Ns.ct>0, which can be expressed as follows:
(21)yskyct∈Ls,ckt−b,Us,ckt+b⊂yskyct∈L^s,ckt−b,U^s,ckt+b∪yskyct∈L˘s,ckt−b,U˘s,ckt+b.From the Hölder continuity, we have the following derivation:(22)y^skxct=supx∈cysky=yskx′,y^s,ck−yskxct≤Lx′−xctα=Ld·1n2α=Ldnα.Then, integrating the above derivation, we have the following:(23)yskxct≤y^s,ck≤yskxct+Ldnα,(24)yskxct−Ldnα≤y˘skxct≤yskxct.Using Equation (23) and Equation (24), the question can be expressed as follows:
(25)yskxct∈L^s,ckt−b,U^s,ckt+b⊂y^skxct∈L^s,ckt,U^s,ckt,yskxct∈L˘s,ckt−b,U˘s,ckt+b⊂y˘skxct∈L˘s,ckt,U˘s,ckt.Thus, plugging Equation (23) and Equation (24) into Equation (21), we obtain the following:
(26)yskxct∈Ls,ckt−b,Us,ckt+b⊂y^sk∈L^s,ckt,U^s,ckt∪y˘sk∈L˘s,ckt,U˘s,cktp.Using Equation (26), we have the following:
(27)PU~≤P⋃t=1NcTy^sk∈L^s,ckt,U^s,ckt+P⋃t=1NcTy˘sk∈L˘s,ckt,U˘s,ckt.Using the concentration inequality, the right side of the above inequality is bounded as follows:(28)PU~≤12SndT.Using the union bound, we have the following:(29)PU~≤12ndT,PU~≤1pndT.Using the result of Lemma10, PU and PU~ can be bounded as follows:
(30)PU=P∪k∈p,qUck=md·PUc≤1T,PU~≥1−1T.Lemma 7.
Under Assumption1, ys∗tpxct and ystpxct are generated by EPCO(3) algorithm. On event U~, we have the following:
(31)ys∗tpxct−ystpxct≤Us,cpt−Ls,cpt+2γ+2b.Proof.
There are two cases here. Whenμs^p∗,c+ηsk≤γb, we have the following:
(32)Us,cpt≥Ls^,cp−2b≥Us^,cpt−μs^p∗,c−2ηsk−2b≥Us^,cpt−2γ+1b.
Whenμs^p∗,c+ηsk≥γb, we have the following:
(33)Us,cpt=Us∧∗,cpt≥Us∧∗,cpt−2γ+1b.
According to the above two cases, we obtain the following:(34)Us,cpt≥Us∧∗,cpt−2γ+1b≥Us∗,cpt−2γ+1b,(35)Us∧∗,cpt≥Us∗,cpt.
On eventU~, we have the following:
(36)ys∗,cpxct≤Us∗,cpt+b≤Us∧∗,cpt+2γ+1b+b,(37)ys,cpxct≤Us,cpt−b.
Combined with Equations (34)–(37), we obtain the following:
(38)ys∗,cpxct−ys,cpxct≤Us,cpt−Ls,cpt+2γ+2b.Lemma 8.
Under Assumption1, ys∗tqxct and ystqxct are generated by EPCO(3) algorithm. On event U~, we have the following:
(39)ys∗tqxct−ystqxct≤Us,cqt−Ls,cqt+2b.Proof.
We know that whenμs^p∗,c+ηsk≤γb holds, all servers S∧∗ are in interval Ls∧∗,cpt−2b,Us∧∗,cpt. Then, we show that Us∗,cpt also satisfies this condition. On event U~, we have the following:
(40)ys∗,cpxct∈Ls∗,cpt−b,Us∗,cpt+b,(41)ys∧∗,cpxct∈Ls∧∗,cpt−b,Us∧∗,cpt+b,(42)ys∗tqxct≥ys∧∗,cpxct.
By Equations (40)–(42), we obtain the following:
(43)Us∗,cp≥ys∗,cpxct−b≥ys∧∗,cpxct−b≥Ls∧∗,cpt−2b.Since the selected servers is in Ls∧∗,cpt−2b,Us∧∗,cp, we have Us,cq≥Us∗,cq. Using this result, we obtain the following:
(44)ys,cq≥Us,cq−b,(45)ys∗,cq≤Us∗,cq+b≤Us,cq+b.From Equation (44) and Equation (45), we have the following:
(46)ys∗,cq−ys,cq≤Us,cq−Ls,cq+2b.Lemma 9.
Under Assumption1, μs^p∗,c are generated by EPCO(3) algorithm, and the upper limit of the number of rounds for μs^p∗,c+ηsk>γb is as follows:
(47)S2Sn,Tγb−ηsk2+1.Proof.
Since(48)μs^p∗,c=2Sn,TNs∧∗,c,Ns∧∗,c=2Sn,Tμs^p∗,c<2Sn,Tγb−ηsk2.And each such event increases the value ofNs∧∗,c by one. The number of rounds for μs^p∗,c+ηsk>γb is bounded by 2Sn,T/γb−ηsk2+1. Summing all the servers together obtains the final result.Lemma 10.
Under Assumption1, Rcpt is generated by EPCO(3) algorithm. On event U~, we have the following:
(49)Rcpt≤SAmaxp+2Bn,TSNct+2γ+2zNct,where Bn,T≔22Sn,T.Lemma 11.
Under Assumption1, Rcqt is generated by EPCO(3) algorithm. On event U~, we have for all c∈C(50)Rcqt≤SAmaxq2Sn,Tγ2z2+1+2zNct+2Bn,TSNc.More detailed proof of Lemma10 and Lemma 11 is presented in [41] (see Lemma 10 and Lemma 11 in [41]).Theorem 12.
Under Assumption1, Rkt is generated by EPCO(3) algorithm, and we have for any k∈p,q as follows:
(51)PRkt<ξkt≥1−,∀t∈1,⋯,T,
where(52)ξpt=ndSAmaxp+2Bn,TSndt+2γ+2zt,ξqt=ndSAmaxq2Sn,Tγ2z2+1+2zt+2Bn,TSndt.Proof.
Combining Equation (12), Lemma 10, and Lemma 11, we have the following:
(53)Rpt≤ndSAmaxp+2Bn,T∑c∈CSNct+2γ+2zNct,≤ndSAmaxp+2Bn,TSndt+2γ+2zt,Rqt≤ndSAmaxq2Sn,Tγ2z2+1+2zNct+2Bn,T∑c∈CSNc≤ndSAmaxq2Sn,Tγ2z2+1+2zt+2Bn,TSndt.Theorem 13.
Under Assumption1, Rkt is generated by EPCO(3) algorithm, and m and γ satisfy n=T1/3ρ+d and γ>0, and we have the following:
(54)ERpT≤Amaxp+2dSAmaxqTd/3ρ+d+2γ+2Ldρ/2T2ρ+d/3ρ+d+2d/2+12Bn,TST1.5ρ+d/3ρ+d,ERqT≤2d/2+12Bn,TST1.5ρ+d/3ρ+d+Amaxq+2Ldρ/2+AmaxqS21+2ρ+dBn,Tγ2L2dρT2ρ+d/3ρ+d+2dAmaxqSTd/3ρ+d.Proof.
According to the Theorem 12 and Equation (15), ERkT is bounded as follows:
(55)ERkT≤ERkTU~+∑c∈CAmaxkNctPU≤ERkTU~+∑c∈CAmaxkNctT=ERkTU~+Amaxk
Then, we obtain the following:(56)ERpT≤ξpt+Amaxp,ERqT≤ξqt+Amaxq.
Whenn=T1/3ρ+d, we have the following:
(57)ERpT≤Amaxp+2dSAmaxqTd/3ρ+d+2γ+2Ldρ/2T2ρ+d/3ρ+d+2d/2+12Bn,TST1.5ρ+d/3ρ+d,ERqT≤2d/2+12Bn,TST1.5ρ+d/3ρ+d+Amaxq+2Ldρ/2+AmaxqS21+2ρ+dBn,Tγ2L2dρT2ρ+d/3ρ+d+2dAmaxqSTd/3ρ+d.
The result shows that not only the regret of EPCO is sublinear, which isOT2ρ+d/3ρ+d, but also added privacy differential mechanism does not affect its convergence.


## 5. Differentially Private

The privacy protection mechanism applied in this paper usesdifferential privacy mechanism, which is originally introduced by Dwork et al. [21].Theorem 14.
EPCO(1) preservesεi,0-differential privacy for MD i where
(58)εi=Φi∑t=1T1ρit,
and each MDi of the noise ηit is independently selected according to the Laplace distribution, where the density function is pηit=1/2ρitexp−ηit/ρit for i=1,⋯,M and t=1,⋯,T.Proof.
We first show thatG~i is locally εi,0-differential privacy for MD i. Now we start by studying the quantity of interest
(59)PG~iTD1,⋯,Di,⋯,DM∈WPG~iTD1,⋯,D′i,⋯,DM∈Wof MD i. We use a random variable Wit, for t=1,⋯,T, to denote the output of G~iT with input D1,⋯,Di,⋯,DM and W′it to denote the output of G~iT with input D1,⋯,D′i,⋯,DM.According to the definition above, we can rewrite the issue as follows:(60)PWi1=wi1,⋯,WiT=wiTPW′i1=wi1,⋯,W′iT=wiT.We useWt to denote the tuple W1t,⋯,WMt. Then, we have the following:
(61)PWi1=wi1,⋯,WiT=wiTPW′i1=wi1,⋯,W′iT=wiT=PW1=w1,⋯,WT=wTPW′1=w1,⋯,W′T=wT=∏t=1TPWt=wtWτ=wτ,τ<tPW′t=wtW′τ=wτ,τ<t.We form this process into a Markov chain where the random vectorat denotes the Lagrangian ν1t,⋯,νMt, which is presented in [39] (see Theorem 3.1 in [39]). Then, we have the following relationship:
(62)PWt=wtWτ=wτ,τ<tPW′t=wtW′τ=wτ,τ<t=PWt=wtat−1=νt−1PW′t=wta′t−1=ν′t−1=PWt=wtat−1=νt−1PW′t=wta′t−1=νt−1≤expyνit−1−yν′it−1ρit≤expΦiρit.WithΦi in Definition 5, we obtain the following:
(63)PGiTD1,⋯,Di,⋯,DM∈WPGiTD1,⋯,D′i,⋯,DM∈W=∏t=1TPWt=wtWτ=wτ,τ<tPW′t=wtW′τ=wτ,τ<t≤∏t=1TexpΦiρit=exp∑t=1TΦiρit.This result proves the privacy guarantee in Equation (58).Thus, Theorem14 proves that our proposed EPCO(1) can guarantee the MD’s privacy and different MDs have different privacy protection levels.Theorem 15.
EPCO(1) preservesε,0-differential privacy for the contextual information of ECSs.Proof.
LetD=h1,⋯,hT denote true information of a supplier and D′ denote a dataset which differs from D in only one data. We define the reward that adds noise as It. Then, for different suppliers s1 and s2, we have the following:
(64)PMs1,t=ItPMs2,t=It=exp−ε′hs1,t−It/Δfexp−ε′hs2,t−It/Δf=expε′Δfhs2,t−It−hs1,t−It≤expε′Δfhs1,t−hs2,t=expε′Δfhs1,t−hs2,t1≤expε′.
Employing Theorem 3.6 in [42] (see page 32 in [42]), Theorem 15 is proved.Therefore, Theorem 15 proves that service supplier fails to extract information about ECSs in ECM by the rewards. In summary, Theorem14 and Theorem 15 prove that EPCO can preserve both privacy of MDs and service suppliers synchronously. In addition, EPCO also supports different MDs to customize their privacy protection levels.


## 6. Simulation Results

In order to verify the efficiency and privacy of our proposed algorithm, we conducted the following simulation experiments with real-world datasets [43]. We compare EPCO with P-UCB1, S-UCB1 [44], and MOC-MAB [41] algorithms. We use Python 3.6 version to implement these algorithms. We run these algorithms on an Acer computer with Intel(R) Core(TM) i5-4460 @ 3.2 GHz and 8 GB RAM. The operating system is Windows 10 Professional. We set L, α, and γ to 1. We give the sets P=0.2,0.4,0.6 and R=0.1,0.25,0.5,1. We let the time horizon T=105 and T=106 to satisfy different experiments. We take X=0,12, and the context is chosen randomly from X at each round. We use 6 arms in each algorithm and every algorithm runs 20 times. We take the average result of the simulation experiment.


### 6.1. Analysis of Regret

Figure3 shows the change of the regret of the algorithm in price and reliability objectives over time. The simulation results show that the regret of EPCO is at a lower position to both price and reliability objectives. But its regret is not the lowest, which is slightly higher than the MOC-MAB algorithm. Because we added a privacy protection mechanism to EPCO, it affected the convergence of the algorithm, but this effect was almost negligible.Figure 3
Regret of multiobjective.
(a)
Regret of price objective(b)
Regret of reliability objective


### 6.2. Analysis of Rewards

We compare the rewards of these algorithms in two objectives, and the results are shown in Figure4. Compared with other multiobjective optimization algorithms, EPCO performs relatively well. Figure 4(a) shows that the EPCO algorithm performs well in price objective, outperforming other algorithms, but slightly inferior to the MOC-MAB algorithm. Figure 4(b) shows that the EPCO algorithm also performs well in reliability objective. This is because although our algorithm has high efficiency, due to the addition of a privacy protection mechanism, a small part of the error is caused, which affects the reward of different objectives.Figure 4
Reward of multiobjective.
(a)
Reward of price objective(b)
Reward of reliability objective


### 6.3. Analysis of Privacy Protection

The effect of different privacy protection levels on the performance of the EPCO algorithm is shown in Figures5 and 6. We set the privacy parameter ε to 0.25, 0.5, 0.75, and 1, respectively, representing different levels of privacy protection. Figure 5 shows the relationship between different privacy levels and regret. Obviously, when the value of ε is larger, the regret of the EPCO algorithm is smaller. This is because as the value of ε increases, the availability of data increases, resulting in a smaller regret. Similarly, in Figure 6, as the value of ε increases, the availability of data also increases, resulting in a larger reward for the EPCO algorithm.Figure 5
The relationship between regret and privacy.
(a)
Impact on price objective(b)
Impact on reliability objectiveFigure 6
The relationship between reward and privacy.
(a)
Impact on price objective(b)
Impact on reliability objective


### 6.4. Discussion

The simulation results show that EPCO’s regret is at a low level in P-UCB1, S-UCB1, and MOC-MAB, because P-UCB1 and S-UCB1 are not suitable for multiobjective optimization. The regret of MOC-MAB is lower than that of EPCO. This is due to the extra work done by EPCO to protect the privacy of users. Withε taking 0.25, 0.5, 0.75, and 1.0, respectively, the regret and reward EPCO also produced different consequences. We noticed that when the value of the privacy parameter ε is larger, the regret of EPCO is smaller, the reward and privacy leakage is larger. This is because a larger ε has little effect on the calculation performance of the algorithm. Although there is a difference in privacy leakage values from the data point of view, this difference is almost impossible to detect in real scenarios; that is to say, regardless of whether the privacy parameter ε is set to 0.1 or 0.5, this means that user privacy is almost impossible to be leaked.


### 6.5. Lessons Learned

The purpose of our simulation experiment is to verify the performance of EPCO from three perspectives, namely, regret, reward, and privacy. We used the Python programming language to implement the EPCO algorithm and used real-world datasets for operations. By comparing with three multiobjective optimization algorithms from the perspectives of regret and reward, namely, P-UCB1, S-UCB1, and MOC-MAB, it is concluded that EPCO does a good job in these two aspects. Meanwhile, it is concluded that EPCO can also effectively protect the privacy of users in the system through the method of privacy leakage. This is because differential privacy mechanism integrates well with online learning algorithms. No matter whetherε is set to 0.1 or 1.0, there is no trending change in the performance of EPCO. That is to say, theoretically, the value of b can be adjusted as large as possible within a certain range, which can not only ensure the privacy of users but also minimize the impact of privacy protection mechanisms on the calculation performance of the algorithm.


## 6.1. Analysis of Regret

Figure3 shows the change of the regret of the algorithm in price and reliability objectives over time. The simulation results show that the regret of EPCO is at a lower position to both price and reliability objectives. But its regret is not the lowest, which is slightly higher than the MOC-MAB algorithm. Because we added a privacy protection mechanism to EPCO, it affected the convergence of the algorithm, but this effect was almost negligible.Figure 3
Regret of multiobjective.
(a)
Regret of price objective(b)
Regret of reliability objective


## 6.2. Analysis of Rewards

We compare the rewards of these algorithms in two objectives, and the results are shown in Figure4. Compared with other multiobjective optimization algorithms, EPCO performs relatively well. Figure 4(a) shows that the EPCO algorithm performs well in price objective, outperforming other algorithms, but slightly inferior to the MOC-MAB algorithm. Figure 4(b) shows that the EPCO algorithm also performs well in reliability objective. This is because although our algorithm has high efficiency, due to the addition of a privacy protection mechanism, a small part of the error is caused, which affects the reward of different objectives.Figure 4
Reward of multiobjective.
(a)
Reward of price objective(b)
Reward of reliability objective


## 6.3. Analysis of Privacy Protection

The effect of different privacy protection levels on the performance of the EPCO algorithm is shown in Figures5 and 6. We set the privacy parameter ε to 0.25, 0.5, 0.75, and 1, respectively, representing different levels of privacy protection. Figure 5 shows the relationship between different privacy levels and regret. Obviously, when the value of ε is larger, the regret of the EPCO algorithm is smaller. This is because as the value of ε increases, the availability of data increases, resulting in a smaller regret. Similarly, in Figure 6, as the value of ε increases, the availability of data also increases, resulting in a larger reward for the EPCO algorithm.Figure 5
The relationship between regret and privacy.
(a)
Impact on price objective(b)
Impact on reliability objectiveFigure 6
The relationship between reward and privacy.
(a)
Impact on price objective(b)
Impact on reliability objective


## 6.4. Discussion

The simulation results show that EPCO’s regret is at a low level in P-UCB1, S-UCB1, and MOC-MAB, because P-UCB1 and S-UCB1 are not suitable for multiobjective optimization. The regret of MOC-MAB is lower than that of EPCO. This is due to the extra work done by EPCO to protect the privacy of users. Withε taking 0.25, 0.5, 0.75, and 1.0, respectively, the regret and reward EPCO also produced different consequences. We noticed that when the value of the privacy parameter ε is larger, the regret of EPCO is smaller, the reward and privacy leakage is larger. This is because a larger ε has little effect on the calculation performance of the algorithm. Although there is a difference in privacy leakage values from the data point of view, this difference is almost impossible to detect in real scenarios; that is to say, regardless of whether the privacy parameter ε is set to 0.1 or 0.5, this means that user privacy is almost impossible to be leaked.


## 6.5. Lessons Learned

The purpose of our simulation experiment is to verify the performance of EPCO from three perspectives, namely, regret, reward, and privacy. We used the Python programming language to implement the EPCO algorithm and used real-world datasets for operations. By comparing with three multiobjective optimization algorithms from the perspectives of regret and reward, namely, P-UCB1, S-UCB1, and MOC-MAB, it is concluded that EPCO does a good job in these two aspects. Meanwhile, it is concluded that EPCO can also effectively protect the privacy of users in the system through the method of privacy leakage. This is because differential privacy mechanism integrates well with online learning algorithms. No matter whetherε is set to 0.1 or 1.0, there is no trending change in the performance of EPCO. That is to say, theoretically, the value of b can be adjusted as large as possible within a certain range, which can not only ensure the privacy of users but also minimize the impact of privacy protection mechanisms on the calculation performance of the algorithm.


## 7. Conclusion

We have proposed a privacy-protected algorithm and EPCO algorithm, for two types of users of computing offloading. One is for MDs whose privacy is protected by a customizable differential privacy mechanism. The other is for ECS suppliers whose privacy is protected by using a common differential privacy mechanism. In addition, we proved no significant impact on the performance of the EPCO algorithm with our privacy protection mechanisms. Simultaneously, an online learning algorithm is introduced to improve the computing offloading efficiency and a detailed theoretical proof of the method is given. The simulation results verify the effectiveness of the EPCO algorithm.



---

*Source: 7444916-2022-07-14.xml*