# A Secure and Verifiable Outsourcing Scheme for Assisting Mobile Device Training Machine Learning Model

Authors: Cheng Li; Li Yang; Jianfeng Ma
Journal: Wireless Communications and Mobile Computing
 (2020)

---

## Abstract

In smart applications such as smart medical equipment, more data needs to be processed and trained locally and near the local end to prevent privacy leaks. However, the storage and computing capabilities of smart devices are limited, so some computing tasks need to be outsourced; concurrently, the prevention of malicious nodes from accessing user data during outsourcing computing is required. Therefore, this paper proposes EVPP (efficient, verifiable, and privacy-preserving), which is a computing outsourcing scheme used in the training process of machine learning models. The edge nodes outsource the complex computing process to the edge service node. First, we conducted a certain amount of testing to confirm the parts that need to be outsourced. In this solution, the computationally intensive part of the model training process is outsourced. Meanwhile, a random encryption perturbation is performed on the outsourced training matrix, and verification factors are introduced to ensure the verifiability of the results. In addition, the system can generate verifiable evidence that can be generated to build a trust mechanism when a malicious service node is found. At the same time, this paper also discusses the application of the scheme in other algorithms in order to be better applied. Through the analysis of theoretical and experimental data, it can be shown that the scheme proposed in this paper can effectively use the computing power of the equipment.


---

## Body

## 1. Introduction

With the development of the Internet of Things, 5G communication networks, AI technology, and the construction of intelligent facilities that promote the development of mobile devices, connected cars, and smart wearable devices, concurrently, a large amount of data has also been generated that is processed by different companies and servers. Data are collected on various cloud computing platforms for various data analyses and mining. It is expected that by 2020, an average person will generate approximately 250 million bytes of data per day [1], which may come from mobile phone sensors, smart wearable devices, and so on.Abundant data require intelligent terminal processing, calculations, storage, etc. [2]; however, the storage and computing capabilities of smart devices are limited, and more data is being continuously collected, transmitted, and calculated. The transmission capabilities and data storage capabilities have become increasingly powerful, but in the face of a geometrically increasing amount of data, it is also difficult to meet users’ requirements for data processing capabilities and transmission quality. Furthermore, the transmission of these data in the network will definitely apply great pressure to the network.The traditional centralized computing architecture based on a cloud center [3–5] has been unable to meet the requirements of modern devices and applications for low latency, high efficiency, and low-cost applications. In some special scenarios, such as smart healthcare [6, 7], identity recognition [8], and smart homes [9], all have high requirements on time and accuracy. Transferring data to cloud servers will raise latency, but running artificial intelligence algorithms such as machine learning and deep learning locally will bring an additional consumption of computing and power to the device.Research on data outsourcing has received widespread attention, whether it is data outsourcing to cloud servers or other nodes with greater computing power. Similarly, more and more algorithms need to be calculated on the user side with the development of machine learning, Internet of Things devices (especially the wide application of wearable devices), network technology, and so on. Therefore, it is a good choice to outsource some calculations in machine learning algorithms. Neto et al. [10] used mobile devices for health monitoring and outsourced data to improve the practical application problems caused by the limited resources of mobile devices. However, there is usually a long distance between the user and the cloud (This not only includes the distance in space; in fact, the user also needs to pass through many nodes in the network to the cloud. These nodes may not be able to be transmitted in time, or there are unstable factors such as jitter in the network.), so it will cause time consumption and some unstable factors, which are often not what users want to see. Therefore, to avoid the problem that a large amount of data cannot be processed in time, Neto et al. used near-user service nodes to perform data outsourcing calculations in the study and achieved certain results.Therefore, the application of edge computing [11] technology is used to outsource the calculation of data to edge nodes that are close and satisfy the computing power to reduce the computing and processing pressure of the device and reduce the delay in data transmission. At the same time, to reduce the pressure of network transmission, some data needs to be processed locally, such as the basic operations including simple data cleaning and partial data processing; simultaneously, to avoid a lengthy time, it is necessary to seek auxiliary computing nodes in the model training process on the near device-side due to the limitation of the network with a high delay and high network pressure.Data in medical, health, and other applications is exploding. Therefore, outsourcing data to the cloud has become the choice of many users, but it also brings security and privacy risks. Therefore, Li et al. [12] proposed an anonymous authorization scheme to ensure the confidentiality and authenticity of the data. Ding et al. [13] studied the data access control scheme under ciphertext, which can also perform effective calculations while ensuring flexible access control. Similarly, when applying edge computing to model training for local devices and nodes, data security and privacy issues cannot be ignored [12, 14, 15]. For example, in the application scenarios of user data collection such as smart medical devices and smart bracelets, the local device continuously accesses the user’s geographic location, physical characteristics (including heart rate, stride, voiceprint, and other characteristics), or medical characteristics, which is apart from the collection and processing data by these devices that include a large number of user’s privacy characteristics. As mentioned earlier, the local device’s computing and processing capabilities cannot process and return results in a timely manner. To avoid the leakage of users’ private data and to ensure that the calculation results are obtained in a timely and effective manner, advancements are needed.Based on the above issues, as shown in Figure1, this paper uses edge computing to solve data processing and computing problems in the construction of intelligent facilities, such as the Internet of Things, ensuring high availability of data and effectively reducing network pressure and network delays. Concurrently, it will combine existing artificial intelligence and machine learning algorithms; the machine learning algorithm training process is “local+edge” for effective and safe training, and finally, the machine learning algorithm model is obtained. EVPP (efficient, verifiable, and privacy-preserving) is proposed which is an outsourcing algorithm for device-to-edge machine learning model training. This algorithm is a good compromise between privacy-preserving and execution efficiency. For example, deep learning is adjusted and compressed to reduce complexity, and high-complexity computing tasks are deployed at the edge of the network. The device only needs to perform some relatively simple operations to complete the entire model training process to appropriately reduce the network time delay via the effective use of computing resources. While ensuring the security of the data and the correctness of the calculation results, the outsourced data is encrypted and replaced, and the existence of malicious service nodes is taken into consideration to ensure the correctness of the calculation results of rational computing nodes. At the same time, we also conducted theoretical analysis and experiments to explain which calculations need to be outsourced and how to reasonably perform outsourcing calculations, so as to better perform outsourcing calculations and optimization in practical applications. This paper also adds a trust mechanism to further increase the security of the system.Figure 1
Schematic diagram of data processing and calculation using edge computing.The contributions of this paper are as follows:(1)
To solve the high calculation and high storage pressure caused by local machine learning algorithms on the device (especially mobile devices), the method called EVPP is proposed to outsource the computing part of the training process(2)
To solve the problems of high latency and network transmission pressure in outsourced computing, a near-local outsourcing algorithm is proposed in conjunction with edge computing, and concurrently, a cryptographic device is designed to solve the privacy and security problems brought by data outsourcing; a random matrix calculation scheme is introduced to randomly perturb the calculation data(3)
To prevent the dishonest outsourced computing nodes from affecting the training process, a trust mechanism with the arbitration function is proposed, which can guarantee the correctness of the calculation results of rational outsourced computing nodesThe organizational structure of this paper is as follows: Section1 briefly introduces the related research, Section 2 will further describe the problems and challenges studied in this paper, and Section 3 will discuss the scheme and its algorithms in detail. Section 4 will focus on the security and performance proofs of the proposed goals in this paper. In Section 5, we will discuss the application of the scheme in other machine learning algorithms and related issues in related fields. In the last section, the scheme will be summarized, and future research directions will be discussed.


## 2. Related Work

Smarter healthcare [6, 7, 16], urban transportation [17, 18], connected cars [19], social networks [20], and other scenarios are increasingly applying machine learning algorithms for prediction and analysis. In these application scenarios, the data are outsourced to the cloud, which has strong computing power, so that resource-constrained devices can use the cloud center to complete various complex computing tasks and better serve users [21, 22].However, with the advent of the Internet of Everything, the edge devices of the network longer generate abundant data, which is not suitable for processing in the cloud center from the perspective of computing or network transmission. Applying edge computing technology is a good choice to solve this problem. At the same time, machine learning algorithms with a higher computational complexity can be applied in the cloud, but it is not realistic to apply them directly on the edge of the network. Zhang et al. [23] proposed the “OpenEI” open-edge intelligent framework to “marginalize” the model training process.At the edge of the network, in order to meet the requirements of delay and efficiency, complex computing tasks must be outsourced to edge nodes with strong computing capabilities, but these nodes are often untrustworthy. To ensure the security and privacy of the data, the data must be encrypted and calculated in the ciphertext domain. This work has become more mature in cloud computing. For example, secure multiparty computing can be applied [24–26], including homomorphic encryption [27], differential privacy [28], and attribute-based encryption [29].Rahulamathavan et al. [30] proposed an SVM classification scheme for outsourcing using Pailler homomorphic encryption technology. In 2018, Li et al. [31] found that the research of Rahulamathavan et al. had problems in terms of soundness and security, so they proposed a more secure solution. In 2015, Liu et al. [32, 33] published two articles on machine learning algorithm outsourcing. They conducted research on the security and privacy issues faced after the data was outsourced to the cloud server and conducted research and experimental analysis on the SVM and gradient descent algorithm, respectively. Li et al. [34] solved the outsourcing security challenge under the malicious model and used homomorphic encryption and security obfuscation circuit technology to implement the secure outsourcing calculation of the ID3 decision tree algorithm. Under the semihonest model, Zhang et al. [35] designed a privacy-protected single-layer perceptron training scheme. This scheme uses a secure two-party calculation to implement a lightweight algorithm to ensure that the participants cannot know the input data and the safety of the calculation results. At the same time, Li et al. [5] also noted that the confidentiality of the classifier is also very important. They proposed the POCC solution to solve the confidentiality of the data and classifier in the cloud. Liu et al. [36] designed a complete and secure outsourcing solution for the KNN algorithm to ensure the safety of data in the process of publishing, storing, applying, requesting, and returning results.Most of the above studies are for cloud computing. In order to reduce the computational and storage costs of network edge devices and return the results required by users in a timely and effective manner, relevant studies have begun to consider applying edge computing to solve this problem. Wu et al. [37] designed a safe and lightweight query scheme (LPEQ) using edge computing technology. This scheme satisfies the requirements of security under the semihonest model. Tao et al. [38] proposed the outlet solution to the difficulties in the ability of wearable devices to process complex data, which uses nearby mobile edge resources to solve context dependence, scattered resources, and resource dynamics, to ensure that the Internet of Things, especially wearable devices, is timely and effective in terms of data. But these solutions are not friendly to edge devices with low computing and storage capabilities. In related research, technologies like homomorphic encryption are mostly used. These encryption schemes may also bring certain computing pressure to some mobile devices and wearable devices.Since determinant and matrix calculations are widely used in the fields of science and engineering, especially in various AI algorithms, there have been many studies on outsourcing calculations of determinants and matrixes [29, 39–41]. Salinas et al. [29] proposed a large-scale deterministic secure outsourcing computing solution. The client can effectively verify the correctness of the calculation results of the outsourced data. Chen et al. [42] proposed a scheme for scrambling the original matrix data using diagonal matrix multiplication to ensure the security of the data; subsequently, it was improved in Zhou et al. [43], and it must be further improved in terms of security and result verification. Hu et al. [44] proposed a matrix outsourcing inversion matrix scheme that can be applied to cloud computing and other scenarios, which effectively reduces the computational complexity of the client. Therefore, these solutions are worthy of reference for solving the safety and efficiency problems of equipment in outsourcing calculation.


## 3. Problem Description and Research Goals

### 3.1. Research Goals and Challenges

In this paper, to better solve the computing and privacy issues of edge devices and ensure the security and accuracy of computing, four research goals are proposed.(1)
Privacy. These data contain tremendous user identity information, privacy data, etc. The collection and processing of these data are extremely prone to leakage of information. Therefore, the outsourcing of data computing needs to ensure the privacy of the data.(2)
Verifiability. Due to the instability of the system, network, or computing nodes, the nodes to which the data are outsourced should be assumed to be incompletely trusted or even malicious. They may steal or peek at the user’s data; furthermore, the operation may not be performed in accordance with the protocol at all, and the wrong calculation result for the user is returned, leading to the failure of the entire data training. Therefore, the calculation result of the data should be verifiable.(3)
High Efficiency. In the whole process, the user’s calculation amount in the outsourced calculation process should be lower than the entire operation performed by the device itself; otherwise, the outsourced operation will be useless.(4)
Accuracy. This requires the design of the entire system to ensure that the calculation results can be guaranteed under the premise of the correct operation at each stage.


### 3.2. System Model

To ensure the security and availability of the system and achieve the research goals proposed in Section3.1, this paper designs an outsourcing model training scheme based on edge computing, as shown in Figure 2.Figure 2
Training model of the outsourcing model based on edge computing.In the solution, the system is divided into three layers (the cloud computing problem is not considered here), which are the sensor node layer (or data acquisition layer), the edge node layer, and the edge service layer.The sensor node layer is responsible for collecting data, but because of its poor computing and storage capabilities, the collected data cannot be calculated, organized, and stored. The collected data must be transmitted to the edge node layer of other networks for processing. For example, smartphones are implanted with several sensors, and these sensors transmit data to the mobile phone’s computing unit for processing and storage. To ensure the availability of data and the security of users’ data, the edge node layer mainly guarantees the data cleaning, calculation, and storage tasks of sensor devices. Concurrently, to ensure the timeliness and accuracy of calculations, it also performs some calculation tasks to offload the edge service layer. The main task of the edge service layer is to assist the devices in the edge node layer to perform collaborative computing. However, due to the difficulty in ensuring security, there are the following risks: (1) the layer may peep and steal data, leading to information leakage, and (2) it may not complete the computing task as agreed, causing the computing task of the edge nodes to fail.As shown in Figure2, the solution proposed in this paper includes edge service nodes that need to outsource computing tasks and edge service nodes that assist edge nodes to outsource the computing tasks. Edge service nodes include two parts that assist users in key generation calculations and an edge server that assists users in computing tasks. What these two edge servers know are intermediate values, so the security of the system is guaranteed. The specific proof is proved in Hypotheses 7 and 8.
(1)
The edge nodes send a request to the edge service node to assist it in calculating the inverse matrix of the matrix for subsequent steps(2)
The edge service node returns the value of the invertible matrix of the edge nodes(3)
The edge nodes send the data that needs to outsource calculation to the edge service node(4)
The edge service node returns the calculated result to the edge nodes. It is worth noting that in this process, the transmitted data is all ciphertextIt is worth noting that in the system, users should be guaranteed to be legal; secondly, the credibility (honesty) data of each node is stored and queried, which requires a trusted third party or a public verification system (for example, government, authority, and blockchain). The preparation work for system construction and operation can be done with reference to [45–47] to ensure the safety and reliability of the system.


### 3.3. Linear Regression and Gradient Descent

There are many optimization and learning algorithms in machine learning and deep learning. Most of these algorithms are based on matrix calculations and training models. During the training phase, the device performs a large number of matrix multiplications and additions. Through the analysis of the corresponding algorithm, it is not difficult to find that the number of multiplication operations is higher than that of addition operations. At the same time, the computational complexity of multiplication is higher than that of addition.Linear regression and gradient descent methods are more common methods. Therefore, in order to facilitate the description of the scheme, this paper uses gradient descent to optimize the model in the linear regression problem and finally obtain the training model. At the same time, the main ideas of the scheme are described.Given then sample set X,Y where the ith sample Xi contains d features, that is, Xi=x1,x2,⋯,xd, adjust the objective function hXi=x1∙w1+x2∙w2+⋯+xd∙wd=Xi∙wT, where w=w1,w2,⋯,wd. Adjust the parameters through the training process to yield the appropriate to make Jiw=1/2n∑i=1nhXi−Yi2 the smallest, that is,Yi≈hXi.The gradient descent method is widely used in machine learning to solve optimization problems. When targeting linear regression problems, for thejth feature of Xi, the weight is wj=wj−α∂Jiw/∂wi=wj−α1/n∑i=1nfXi−YiXij, and by representing vectors as a matrix, w can be expressed as w∶=w−αXT×X×w−Y[25].Among them,α is the learning rate or step size, which is a fixed value, and this parameter determines the convergence degree of the algorithm; Y is a vector of n×l dimensions, that is, a given data tag set.In the gradient descent method, because all the samples are used for training at one time, it will cause pressure on the memory and calculations, so∣B∣ samples are selected for small batch training. In the formula, ∣B∣ is the amount of data, and w∶=w−α1/∣B∣XBT×XB×w−YB[25].Therefore, this paper decomposes the matrix calculations with abundant calculations. Among them, suppose(1)∆=XBT×XB×w−YB=XBT×XB×w+−YB=XBT×XB×w+XBT×−YB.That is, the update formula can be expressed as(2)w∶=w−α1B∆.


### 3.4. System Framework

The main idea of our solution is shown in Figure3, which includes the following five parts.Step 1 (outsourced data generation algorithm).
The client constructs a reversible matrixD for scrambling the data matrix, generating a random matrix, and randomly generating a verification matrix. Concurrently, the client has a training data set. The sample set X contains n samples xi. Each sample set can be represented as an m-dimension vector, and the tag set is represented as Y.Figure 3
The main design ideas of the scheme.The client calculates the confusion matrix formsC1, C2, C3, and C4 corresponding to the sample set X and its transposed matrix XT, tag set Y, and initialization weight matrix w and sends the data and corresponding calculation rules fC′ to the edge service layer for calculation. At the same time, the matrix verification block is calculated and saved to facilitate subsequent result verification work.Step 2 (outsourcing data calculation algorithm).
The edge service layer node outputs the calculation resultΔ∗ according to the outsourcing calculation rule fC′ sent by the client and sends the calculated result back to the client.Step 3 (training result generation algorithm).
The client receives the calculation resultΔ∗ sent back by the edge server layer node and performs a recovery operation (Step 4) based on the information held locally to obtain the calculation result ∆. Then, the result is brought into Formula (1) and calculated, obtaining wt. By comparison, wt<wt−1 indicates that the function has not reached the convergence value, and the scrambling operation is performed and returns to Step 2 to continue training; wt>wt−1 indicates that the function has reached the convergence value, which terminates this calculation task and enters Step 5.Step 4 (data verification algorithm).
The client receives the calculation result∆∗ returned by the edge server layer node, extracts the verification matrix block V∗ from it, and compares it with VV∗=?. When they are equal, the result indicates that the edge service layer node has performed the calculation operation correctly; otherwise, it indicates that it has not faithfully calculated the outsourcing task; the client retains the test evidence, publishes the evidence and the identity of the edge service layer node, and executes Step 5 to find new computing nodes.Step 5 (end the calculation task).
When the function reaches the convergence value, the client sendsW0 to the edge service layer node. When the edge service layer node receives the message, it knows that the calculation task is terminated, and the edge service layer node clears all relevant data. (This step may be performed in the following two situations: (1) the protocol execution process is normally completed, and (2) it is found that the edge server does not faithfully execute the calculation protocol. Therefore, it is not identified in Figure 3.)


## 3.1. Research Goals and Challenges

In this paper, to better solve the computing and privacy issues of edge devices and ensure the security and accuracy of computing, four research goals are proposed.(1)
Privacy. These data contain tremendous user identity information, privacy data, etc. The collection and processing of these data are extremely prone to leakage of information. Therefore, the outsourcing of data computing needs to ensure the privacy of the data.(2)
Verifiability. Due to the instability of the system, network, or computing nodes, the nodes to which the data are outsourced should be assumed to be incompletely trusted or even malicious. They may steal or peek at the user’s data; furthermore, the operation may not be performed in accordance with the protocol at all, and the wrong calculation result for the user is returned, leading to the failure of the entire data training. Therefore, the calculation result of the data should be verifiable.(3)
High Efficiency. In the whole process, the user’s calculation amount in the outsourced calculation process should be lower than the entire operation performed by the device itself; otherwise, the outsourced operation will be useless.(4)
Accuracy. This requires the design of the entire system to ensure that the calculation results can be guaranteed under the premise of the correct operation at each stage.


## 3.2. System Model

To ensure the security and availability of the system and achieve the research goals proposed in Section3.1, this paper designs an outsourcing model training scheme based on edge computing, as shown in Figure 2.Figure 2
Training model of the outsourcing model based on edge computing.In the solution, the system is divided into three layers (the cloud computing problem is not considered here), which are the sensor node layer (or data acquisition layer), the edge node layer, and the edge service layer.The sensor node layer is responsible for collecting data, but because of its poor computing and storage capabilities, the collected data cannot be calculated, organized, and stored. The collected data must be transmitted to the edge node layer of other networks for processing. For example, smartphones are implanted with several sensors, and these sensors transmit data to the mobile phone’s computing unit for processing and storage. To ensure the availability of data and the security of users’ data, the edge node layer mainly guarantees the data cleaning, calculation, and storage tasks of sensor devices. Concurrently, to ensure the timeliness and accuracy of calculations, it also performs some calculation tasks to offload the edge service layer. The main task of the edge service layer is to assist the devices in the edge node layer to perform collaborative computing. However, due to the difficulty in ensuring security, there are the following risks: (1) the layer may peep and steal data, leading to information leakage, and (2) it may not complete the computing task as agreed, causing the computing task of the edge nodes to fail.As shown in Figure2, the solution proposed in this paper includes edge service nodes that need to outsource computing tasks and edge service nodes that assist edge nodes to outsource the computing tasks. Edge service nodes include two parts that assist users in key generation calculations and an edge server that assists users in computing tasks. What these two edge servers know are intermediate values, so the security of the system is guaranteed. The specific proof is proved in Hypotheses 7 and 8.
(1)
The edge nodes send a request to the edge service node to assist it in calculating the inverse matrix of the matrix for subsequent steps(2)
The edge service node returns the value of the invertible matrix of the edge nodes(3)
The edge nodes send the data that needs to outsource calculation to the edge service node(4)
The edge service node returns the calculated result to the edge nodes. It is worth noting that in this process, the transmitted data is all ciphertextIt is worth noting that in the system, users should be guaranteed to be legal; secondly, the credibility (honesty) data of each node is stored and queried, which requires a trusted third party or a public verification system (for example, government, authority, and blockchain). The preparation work for system construction and operation can be done with reference to [45–47] to ensure the safety and reliability of the system.


## 3.3. Linear Regression and Gradient Descent

There are many optimization and learning algorithms in machine learning and deep learning. Most of these algorithms are based on matrix calculations and training models. During the training phase, the device performs a large number of matrix multiplications and additions. Through the analysis of the corresponding algorithm, it is not difficult to find that the number of multiplication operations is higher than that of addition operations. At the same time, the computational complexity of multiplication is higher than that of addition.Linear regression and gradient descent methods are more common methods. Therefore, in order to facilitate the description of the scheme, this paper uses gradient descent to optimize the model in the linear regression problem and finally obtain the training model. At the same time, the main ideas of the scheme are described.Given then sample set X,Y where the ith sample Xi contains d features, that is, Xi=x1,x2,⋯,xd, adjust the objective function hXi=x1∙w1+x2∙w2+⋯+xd∙wd=Xi∙wT, where w=w1,w2,⋯,wd. Adjust the parameters through the training process to yield the appropriate to make Jiw=1/2n∑i=1nhXi−Yi2 the smallest, that is,Yi≈hXi.The gradient descent method is widely used in machine learning to solve optimization problems. When targeting linear regression problems, for thejth feature of Xi, the weight is wj=wj−α∂Jiw/∂wi=wj−α1/n∑i=1nfXi−YiXij, and by representing vectors as a matrix, w can be expressed as w∶=w−αXT×X×w−Y[25].Among them,α is the learning rate or step size, which is a fixed value, and this parameter determines the convergence degree of the algorithm; Y is a vector of n×l dimensions, that is, a given data tag set.In the gradient descent method, because all the samples are used for training at one time, it will cause pressure on the memory and calculations, so∣B∣ samples are selected for small batch training. In the formula, ∣B∣ is the amount of data, and w∶=w−α1/∣B∣XBT×XB×w−YB[25].Therefore, this paper decomposes the matrix calculations with abundant calculations. Among them, suppose(1)∆=XBT×XB×w−YB=XBT×XB×w+−YB=XBT×XB×w+XBT×−YB.That is, the update formula can be expressed as(2)w∶=w−α1B∆.


## 3.4. System Framework

The main idea of our solution is shown in Figure3, which includes the following five parts.Step 1 (outsourced data generation algorithm).
The client constructs a reversible matrixD for scrambling the data matrix, generating a random matrix, and randomly generating a verification matrix. Concurrently, the client has a training data set. The sample set X contains n samples xi. Each sample set can be represented as an m-dimension vector, and the tag set is represented as Y.Figure 3
The main design ideas of the scheme.The client calculates the confusion matrix formsC1, C2, C3, and C4 corresponding to the sample set X and its transposed matrix XT, tag set Y, and initialization weight matrix w and sends the data and corresponding calculation rules fC′ to the edge service layer for calculation. At the same time, the matrix verification block is calculated and saved to facilitate subsequent result verification work.Step 2 (outsourcing data calculation algorithm).
The edge service layer node outputs the calculation resultΔ∗ according to the outsourcing calculation rule fC′ sent by the client and sends the calculated result back to the client.Step 3 (training result generation algorithm).
The client receives the calculation resultΔ∗ sent back by the edge server layer node and performs a recovery operation (Step 4) based on the information held locally to obtain the calculation result ∆. Then, the result is brought into Formula (1) and calculated, obtaining wt. By comparison, wt<wt−1 indicates that the function has not reached the convergence value, and the scrambling operation is performed and returns to Step 2 to continue training; wt>wt−1 indicates that the function has reached the convergence value, which terminates this calculation task and enters Step 5.Step 4 (data verification algorithm).
The client receives the calculation result∆∗ returned by the edge server layer node, extracts the verification matrix block V∗ from it, and compares it with VV∗=?. When they are equal, the result indicates that the edge service layer node has performed the calculation operation correctly; otherwise, it indicates that it has not faithfully calculated the outsourcing task; the client retains the test evidence, publishes the evidence and the identity of the edge service layer node, and executes Step 5 to find new computing nodes.Step 5 (end the calculation task).
When the function reaches the convergence value, the client sendsW0 to the edge service layer node. When the edge service layer node receives the message, it knows that the calculation task is terminated, and the edge service layer node clears all relevant data. (This step may be performed in the following two situations: (1) the protocol execution process is normally completed, and (2) it is found that the edge server does not faithfully execute the calculation protocol. Therefore, it is not identified in Figure 3.)


## 4. System Solutions

In this section, the application scenario of EVPP is introduced in detail. The solution takes the gradient descent method as an example to achieve the four goals required by the previously described.


### 4.1. Encryption and Decryption Methods for Outsourced Data

In this section, we will describe the construction, encryption, and decryption processes of Formulas (1) and (2) in the scheme. To ensure the security of the data and the simplicity of the result verification, the training data is encrypted. The edge nodes encrypt m×n dataX, m×l data, and n×l data w to ensure data security and then operate the matrix according to the following methods.The edge nodes randomly generatem×t order matrixes M1 and M3; n×t order matrixes M2 and M4; four randomly generated t order matrixes V1, V2, V3, and V4; randomly select the diagonal matrix R; and construct the reversible matrix D. Finally, we can obtain the outsourcing matrix:
(3)XTn+t×m+t′=XTD−1M20V2,(4)wn+t×i+t′=RwTM30V3,(5)Yn+t×i+t′=RYTM40V4,(6)Xm+t×n+t′=DXM10V1.The construction process of the invertible matrixD will be described in detail [43].The invertible matrixD=D1+D2 and the matrixes D1, D2, and D are all square matrixes of order m×m. The matrix D1 is a random diagonal matrix, in whicha11=0 and other diagonal position element values are randomly selected from the edge nodes. In the matrix D2=PTQ, where P=p1,p2,⋯,pm and Q=1,q1,q2,⋯,qm−1. D will be shown below as an invertible matrix.Proof.
First, the matrixesD1, D2, and D are all square matrixes of m×m.
Second,D1 is a diagonal matrix, D2=PTQ, and D=D1+D2. Therefore,
(7)D=p1q1p1q2p1⋯qm−1p1p2q1p2+d1q2p2⋯qm−1p2p3q1p3q2p1+d3⋯qm−1p3⋮⋮⋮⋱⋮pmq1pmq2pm⋯qm−1pm+dm.
Subtract the first row from 2 tom of the matrix D to yield
(8)D=p1q1p1q2p1⋯qm−1p10d10⋯000d3⋯0⋮⋮⋮⋱⋮000⋯dm.
Finally, the unit matrixI can be obtained from the above matrix through a further elementary transformation, so it can be easily obtained that the matrix D must be an invertible matrix. The matrix inversion process can be performed with reference to [43].
Next, we will demonstrate how to ensure that the calculation results are recoverable and verifiable in the matrix. For the formula∆=XBT×XB×w+XBT×−YB, the method of matrix block construction can be obtained: ∆′=XT′×X′×w′+XT′×−Y′).
Multiply two block matrixes, for example,(9)C2×C1=XTD−1M20V2T×DXM10V1=XTD−1DXXTD−1M1+M2V10V2V1.
It is not difficult to see from the result of the matrix multiplication that the upper left part of the matrix is equivalent to solvingXT×X; the edge nodes can calculate detV=detV1V2V3−V2V4 as the verification factor for the outsourced calculation, which indirectly proves the correctness of the calculation result.


### 4.2. Description of Scheme

In this section, Algorithms1 and 2 will be described according to the algorithm described in Section 4.1, and the initialization process of the scheme has been completed. Use f to represent the calculation rules for outsourced data. In this paper, f uses ∆ of Section 3.4 to define the calculation rules.Step 1 (outsourced data generation algorithm).
The client constructs a reversible matrixD for scrambling the data matrix, generates a random matrix M1, M2, M3, M4, and randomly generates a verification matrix V1, V2, V3, V4; R is a diagonal matrix. Concurrently, the client has a training data set. The sample set X contains n samples xi. Each sample set can be represented as a d-dimension vector, and the tag set is represented as Y.Outsource the invertible matrixD to obtain D−1.The client can obtain the sample setX and its transposed matrix DT, the tag set Y, and the initial w confusion matrixes C2, C1, C3, and C4 through Algorithm 1. The constructed calculation rules and confusion matrix are sent to the edge service node for outsourced calculation.Algorithm 1: Outsourced data generation algorithm.
Input: Keyk, Invertible matrix D1, PT, Q, Random matrixes (M1, M2, M3, and M4),Random verification matrixes (V1, V2, V3, and V4), Sample setX, Tag setY, and theinitialization vectorw .Output: Outsourcing matrixC2 , C1, C3, and C4.1: CalculateD=D1+D2;2: Go to the inverse matrix algorithm is outsourced to obtain the inverse matrixD−1;3: The edge computing node calculates the key matrixK←kI;4: Initialize the vector for scrambling (perturbation)w′←Kw and tag set Y′←KY5: Design calculation rulesf;6: According to the calculation rules, solve the verification factor detV;7: Construct the sample setX′←DX and its transpose matrix XT′←XTD−18: returnC2←XT′M2V2, C1←X′M1V1, C3←w′M3V3, C4←Y′M4V4Step 2 (outsourcing data calculation algorithm).
The edge service layer node outputs the calculation result∆∗ according to the outsourcing calculation rulefC′ sent by the client (the calculation rule is based on the gradient descent method for linear regression) and sends the calculation result to the client.Algorithm 2: Outsourcing data calculation algorithm.
Input: Keyk, MatrixC2 , C1, C3, C4, and Calculation Rulef.Output: Calculation result∆∗.1: Calculate∆∗=C2C1C3+C2−C4;2: Send the calculated result∆∗ to the edge computing node.Step 3 (training result generation algorithm).
The client receives the calculation result∆∗ returned by the edge server layer node and executes Algorithm 3. In this section, to better explain the application process of the algorithm, Formula (1) will be taken as an example.Algorithm 3: Training result generation algorithm.
1:wt∗←∆∗, extract submatrix part of the matrix wt∗;2: Go to Algorithm4;3: ifwt′=wt∗ then4: The results returned by outsourced calculations are true;5: ifwt′<wt∗ then6:  The function does not reach the convergence value and generates a new validationfactorVt to replace the validation factor in ∆∗;7:  Perform scrambling operation to generate∆∗←wt′ ;8:  Go to Step 2;9: else10:  The function reaches a convergence value;11: end if12: end if13:wt=K−1Wt′ ;14: Go to Step 5.It is worth noting that in Algorithm3, Algorithm 4 is not necessarily executed every time for verification, because Algorithm 4 will bring the calculation overhead of the edge device. At the same time, the frequency of Algorithm 4 execution is related to the probability of finding dishonest service nodes. When Algorithm4 is executed every time, the scheme in this paper can be applied to the malicious model.Step 4 (data verification algorithm).
After the client receives the calculation result∆∗ returned by the edge server layer node, it verifies the calculation result.Algorithm 4: Data verification algorithm.
Input: Keyk, Calculation result ∆∗.Output: Validation resultVV∗=?.1: Extract validation matrix blocksV∗←∆∗;2: Calculation det(V∗);3: if det(V)= det(V∗) then4: return “True”;5: else6: Go to Step 5 and Step 6;7: Find new computing nodes and perform model training tasks;8: end ifStep 5 (end the calculation task).
When the function reaches the convergence value or the edge service node calculation task fails, the edge calculation node executes this step algorithm.Algorithm 5: End the calculation task.
1: The client will sendW0 to the edge server node;2: When the edge service layer node receivesW0, it learns that the computing task is terminated;3: The edge service layer node deletes the computing data related to the training task.When the edge computing node checks and finds that there is an error in the calculation result returned by the edge service node, Algorithm6 is executed. This algorithm is used to generate evidence that the edge service node has not faithfully performed the model training task according to the protocol, thereby announcing that the node is untrustworthy and building a system trust mechanism.When other nodes verify the security of the edge service node, the verification matrix is extracted from the evidenceEu→s, and the corresponding results are obtained according to the calculation rules to determine whether the evidence is valid. When the verification node records the verification result, a trust record is built locally.Algorithm 6: Evidence generation and adjudication algorithm.
Input: Keyk, Validation matrixes V2 , V1, V3, V4, and Calculation Rulef.Output: EvidenceE.1: Generate evidence’s signatureS←HV1∣V2∣V3∣V4∣V∗HuIDHsID ;2: Generate evidenceEu→s←V1∣V2∣V3∣V4∣V∗∣f∣S,


## 4.1. Encryption and Decryption Methods for Outsourced Data

In this section, we will describe the construction, encryption, and decryption processes of Formulas (1) and (2) in the scheme. To ensure the security of the data and the simplicity of the result verification, the training data is encrypted. The edge nodes encrypt m×n dataX, m×l data, and n×l data w to ensure data security and then operate the matrix according to the following methods.The edge nodes randomly generatem×t order matrixes M1 and M3; n×t order matrixes M2 and M4; four randomly generated t order matrixes V1, V2, V3, and V4; randomly select the diagonal matrix R; and construct the reversible matrix D. Finally, we can obtain the outsourcing matrix:
(3)XTn+t×m+t′=XTD−1M20V2,(4)wn+t×i+t′=RwTM30V3,(5)Yn+t×i+t′=RYTM40V4,(6)Xm+t×n+t′=DXM10V1.The construction process of the invertible matrixD will be described in detail [43].The invertible matrixD=D1+D2 and the matrixes D1, D2, and D are all square matrixes of order m×m. The matrix D1 is a random diagonal matrix, in whicha11=0 and other diagonal position element values are randomly selected from the edge nodes. In the matrix D2=PTQ, where P=p1,p2,⋯,pm and Q=1,q1,q2,⋯,qm−1. D will be shown below as an invertible matrix.Proof.
First, the matrixesD1, D2, and D are all square matrixes of m×m.
Second,D1 is a diagonal matrix, D2=PTQ, and D=D1+D2. Therefore,
(7)D=p1q1p1q2p1⋯qm−1p1p2q1p2+d1q2p2⋯qm−1p2p3q1p3q2p1+d3⋯qm−1p3⋮⋮⋮⋱⋮pmq1pmq2pm⋯qm−1pm+dm.
Subtract the first row from 2 tom of the matrix D to yield
(8)D=p1q1p1q2p1⋯qm−1p10d10⋯000d3⋯0⋮⋮⋮⋱⋮000⋯dm.
Finally, the unit matrixI can be obtained from the above matrix through a further elementary transformation, so it can be easily obtained that the matrix D must be an invertible matrix. The matrix inversion process can be performed with reference to [43].
Next, we will demonstrate how to ensure that the calculation results are recoverable and verifiable in the matrix. For the formula∆=XBT×XB×w+XBT×−YB, the method of matrix block construction can be obtained: ∆′=XT′×X′×w′+XT′×−Y′).
Multiply two block matrixes, for example,(9)C2×C1=XTD−1M20V2T×DXM10V1=XTD−1DXXTD−1M1+M2V10V2V1.
It is not difficult to see from the result of the matrix multiplication that the upper left part of the matrix is equivalent to solvingXT×X; the edge nodes can calculate detV=detV1V2V3−V2V4 as the verification factor for the outsourced calculation, which indirectly proves the correctness of the calculation result.


## 4.2. Description of Scheme

In this section, Algorithms1 and 2 will be described according to the algorithm described in Section 4.1, and the initialization process of the scheme has been completed. Use f to represent the calculation rules for outsourced data. In this paper, f uses ∆ of Section 3.4 to define the calculation rules.Step 1 (outsourced data generation algorithm).
The client constructs a reversible matrixD for scrambling the data matrix, generates a random matrix M1, M2, M3, M4, and randomly generates a verification matrix V1, V2, V3, V4; R is a diagonal matrix. Concurrently, the client has a training data set. The sample set X contains n samples xi. Each sample set can be represented as a d-dimension vector, and the tag set is represented as Y.Outsource the invertible matrixD to obtain D−1.The client can obtain the sample setX and its transposed matrix DT, the tag set Y, and the initial w confusion matrixes C2, C1, C3, and C4 through Algorithm 1. The constructed calculation rules and confusion matrix are sent to the edge service node for outsourced calculation.Algorithm 1: Outsourced data generation algorithm.
Input: Keyk, Invertible matrix D1, PT, Q, Random matrixes (M1, M2, M3, and M4),Random verification matrixes (V1, V2, V3, and V4), Sample setX, Tag setY, and theinitialization vectorw .Output: Outsourcing matrixC2 , C1, C3, and C4.1: CalculateD=D1+D2;2: Go to the inverse matrix algorithm is outsourced to obtain the inverse matrixD−1;3: The edge computing node calculates the key matrixK←kI;4: Initialize the vector for scrambling (perturbation)w′←Kw and tag set Y′←KY5: Design calculation rulesf;6: According to the calculation rules, solve the verification factor detV;7: Construct the sample setX′←DX and its transpose matrix XT′←XTD−18: returnC2←XT′M2V2, C1←X′M1V1, C3←w′M3V3, C4←Y′M4V4Step 2 (outsourcing data calculation algorithm).
The edge service layer node outputs the calculation result∆∗ according to the outsourcing calculation rulefC′ sent by the client (the calculation rule is based on the gradient descent method for linear regression) and sends the calculation result to the client.Algorithm 2: Outsourcing data calculation algorithm.
Input: Keyk, MatrixC2 , C1, C3, C4, and Calculation Rulef.Output: Calculation result∆∗.1: Calculate∆∗=C2C1C3+C2−C4;2: Send the calculated result∆∗ to the edge computing node.Step 3 (training result generation algorithm).
The client receives the calculation result∆∗ returned by the edge server layer node and executes Algorithm 3. In this section, to better explain the application process of the algorithm, Formula (1) will be taken as an example.Algorithm 3: Training result generation algorithm.
1:wt∗←∆∗, extract submatrix part of the matrix wt∗;2: Go to Algorithm4;3: ifwt′=wt∗ then4: The results returned by outsourced calculations are true;5: ifwt′<wt∗ then6:  The function does not reach the convergence value and generates a new validationfactorVt to replace the validation factor in ∆∗;7:  Perform scrambling operation to generate∆∗←wt′ ;8:  Go to Step 2;9: else10:  The function reaches a convergence value;11: end if12: end if13:wt=K−1Wt′ ;14: Go to Step 5.It is worth noting that in Algorithm3, Algorithm 4 is not necessarily executed every time for verification, because Algorithm 4 will bring the calculation overhead of the edge device. At the same time, the frequency of Algorithm 4 execution is related to the probability of finding dishonest service nodes. When Algorithm4 is executed every time, the scheme in this paper can be applied to the malicious model.Step 4 (data verification algorithm).
After the client receives the calculation result∆∗ returned by the edge server layer node, it verifies the calculation result.Algorithm 4: Data verification algorithm.
Input: Keyk, Calculation result ∆∗.Output: Validation resultVV∗=?.1: Extract validation matrix blocksV∗←∆∗;2: Calculation det(V∗);3: if det(V)= det(V∗) then4: return “True”;5: else6: Go to Step 5 and Step 6;7: Find new computing nodes and perform model training tasks;8: end ifStep 5 (end the calculation task).
When the function reaches the convergence value or the edge service node calculation task fails, the edge calculation node executes this step algorithm.Algorithm 5: End the calculation task.
1: The client will sendW0 to the edge server node;2: When the edge service layer node receivesW0, it learns that the computing task is terminated;3: The edge service layer node deletes the computing data related to the training task.When the edge computing node checks and finds that there is an error in the calculation result returned by the edge service node, Algorithm6 is executed. This algorithm is used to generate evidence that the edge service node has not faithfully performed the model training task according to the protocol, thereby announcing that the node is untrustworthy and building a system trust mechanism.When other nodes verify the security of the edge service node, the verification matrix is extracted from the evidenceEu→s, and the corresponding results are obtained according to the calculation rules to determine whether the evidence is valid. When the verification node records the verification result, a trust record is built locally.Algorithm 6: Evidence generation and adjudication algorithm.
Input: Keyk, Validation matrixes V2 , V1, V3, V4, and Calculation Rulef.Output: EvidenceE.1: Generate evidence’s signatureS←HV1∣V2∣V3∣V4∣V∗HuIDHsID ;2: Generate evidenceEu→s←V1∣V2∣V3∣V4∣V∗∣f∣S,


## 5. System Analysis

### 5.1. Security

The security of the solution is considered from the following aspects: data security and privacy, the correctness of the calculation results, and the trust mechanism of edge service nodes.


#### 5.1.1. The Correctness of Calculation Results

When calculating a block matrix, it can be divided into two parts: matrix addition and matrix multiplication for analysis and consideration.First, verify the correctness of matrix multiplication.In the calculation ofXT′×X′, the result is
(10)XTD−1DXXTD−1M1+M2V10V2V1.And the result in the upper right corner of the matrix is not difficult to see asXT×X. For a further description of the calculation with w, it can be simplified to XTD−1DXRwT⋯0V2V1V3.SinceR is a diagonal matrix, it is assumed that k is randomly selected as the diagonal element value of the matrix. The matrix element of XT×X is xij. The calculation result of Formula (1) indicates
(11)∑j=1n∑i−1nx1jkwi1⋯∑j=1n∑i−1nx1jkwil⋮⋱⋮∑j=1n∑i−1nxnjkwi1⋯∑j=1n∑i−1nxnjkwilIt is not difficult to see that multiplying the matrix by1/kI can restore the original data.The correctness of the matrix addition is relatively simple and will not be repeated; the reader can prove it by himself.


#### 5.1.2. Proof of Algorithm Security

In this section, the security of this scheme is demonstrated under five assumptions of insecurity.Hypothesis 1.
Malicious users obtain data through intermediate parameters.In the embodiment, for∆=XBT×XB×w+XBT×−YB, the data used for the matrixes X, Y, and w are used for the matrixes randomly perturbing invertible matrix operation, and the elements in the invertible matrix D are randomly generated and have no correlation. There is a transformation of XTD−1DX, which guarantees data security and recovery. Therefore, the service edge nodes cannot guess any information data from the matrix, to ensure the privacy of the data. At the same time, both the target matrix and the intermediate parameters are the result of addingK to the disturbance, and only users who master K can restore the data.In the training process, the learning rate (step size) is determined by the edge nodes, and the initialized parameters are random. Except for the calculations necessary for the outsourcing calculation, the other is done locally by the edge nodes. Therefore, the program also ensures the safety system model in the training process, the iterative training process is completed, and the edge service node only grasps the intermediate value, thus ensuring the safety training model.The scheme ensures the security of data during the exchange process. This paper only describes the solution with a linear regression model. When other machine learning algorithms are needed, only the calculation process needs to be improved.Hypothesis 2.
The malicious edge server can recover the intermediate parameter through the inverse matrix.In the solution described in this paper, all user data is added with disturbance and encapsulation, and edge servers cannot know the true meaning of their calculation data. Here, we assume a more powerful enemy that can conspire with the edge server that computes the inverse matrixD−1′ (this is not truly an inverse matrix). In this case, although the malicious server obtains the intermediate value of the relevant inverse matrix D−1, it cannot recover the accurate inverse matrix information, and it also cannot restore the original data.After generating the inverse matrix, the inverse matrix returned by the server contains the data disturbance. Therefore, except for the owner of the data, it is difficult for others to restore the original matrix and its inverse matrix.[44] has carried out detailed proof of the security analysis and will not repeat them here.Hypothesis 3.
The two edge servers conspire to get user data.In this method, the biggest threat is that the two edge service providers recover the data by obtaining the inverse matrixD−1 information of the user, and other methods cannot recover the data. This is similar to Hypothesis 8, due to a matrix calculated by one of the servers. The matrix contains inverse matrix and disturbance information, and to restore the true inverse matrix, the adversary needs to grasp the parameters of its disturbance information. Therefore, the final inverse matrix D−1 cannot be obtained between these two edge servers, so this scheme is already safe under this assumption.Hypothesis 4.
Trust mechanism of edge service nodes.In the system, this paper introduces an arbitration mechanism. When the edge nodes detect that the calculation result returned by the edge service node is abnormal, that is, the returned result is inconsistent with the verification result, then the edge nodes consider labelling the edge service node as malicious. The edge nodes verify the information, including the returned resultv, the raw data used to generate the verification results (V2, V1, V3, and V4), and the relevant identity information HIDu∣HIDs of the edge nodes and the service node, and then generate evidence and publish it to other nodes. When each node in the system receives the evidence, it performs a test. If it is indeed proved that the edge service node has not performed calculation tasks according to the agreement, it will be added to the blacklist, and the data will no longer be outsourced to the node. Concurrently, to ensure the long-term effectiveness of the system, you can look for trusted authorities (for example, government agencies) to store evidence and maintain node information in the system.Hypothesis 5.
Adversary obtains enough ciphertext for analysis.The scheme in this paper is to protect the security and privacy of the sample dataX, tag data Y, and parameter (weight) information w. Ensure that the adversary can assist the user in the calculation task and does not obtain any data information.In the scheme, the invertible matrixD and its inverse matrix D−1 are the key to ensure the computability and safety of X and XT. It has been proved in Hypotheses 7 and 8 that it can guarantee the security of the data. However, long-term use of the same set of matrixes (including the invertible matrix D and its inverse matrix D−1) is a security risk. The adversary may recover the original data contained in the disturbance data by collecting a large number of matrixes (for example, C1 and C2). In turn, it threatens the security of the program. Therefore, the frequency of use of D and D−1 can be adjusted to achieve better data security. The safest solution is to use different D and D−1 each time. Of course, this will increase the computational cost of the initialization phase (Algorithms 1 and 2). The computational cost of the initialization phase will be discussed in the experimental part.Hypothesis 6.
Discussion on scheme security model.When introducing Algorithm3, we have already mentioned that there is a certain probability of detecting dishonest edge service nodes. Since Algorithm 4 requires the edge device to run locally, it will cause additional computational costs. Assuming that the malicious edge service node is strong enough, it can know when the edge nodes perform the verification of outsourcing results. But this will increase the computational cost of the edge nodes. Although in the scheme in this paper the edge nodes can choose a smaller dimension of the verification matrix, it still has computational costs.At the same time, we designed evidence generation and verification algorithms in the scheme. Suppose that the rational edge service node in the system is detected to be evil, and the evidence of evil will be retained in the system. Other edge nodes can verify the evidence in the system. If the evidence is true, the malicious edge service node will lose the trust of the edge nodes, so that the edge nodes will not choose to submit outsourced computing tasks to it.Therefore, once the edge service node loses the outsourced task of the edge nodes, it will not receive the corresponding reward. So, for the rational node, it will still abide by the computing protocol. However, this also increases the computational cost of edge nodes. So, our purpose is to effectively reduce the computational cost of edge nodes, so our scheme is more used for semihonest models.


### 5.2. Analysis of Computational Complexity and Time Cost

In this section, we will discuss the computational overhead of matrix addition and multiplication operations commonly used in machine learning. Of course, matrix computing is used not only in the field of machine learning but also in many fields of scientific computing. Therefore, the final conclusion is generally applicable. For the convenience of description, it is assumed that twon∗n matrixes A, B need to be added and multiplied (subtraction and division operations can be transformed into addition or multiplication operations).
(i)
Addition. During the addition operation, we first need to traverse the elements in A by row and the elements in B by column. Since both A and B are n∗n matrixes, it is not difficult to see that the time complexity of addition is On2.(ii)
Multiply. During the multiplication operation, each row of A needs to be multiplied with each column of B, and then, the results are added. So, the time complexity of multiplication is On3.Next, we also conducted corresponding experiments in Python and Java.In Figure4, we use fixed calculation rounds to calculate the time cost of matrix addition and multiplication in different dimensions. By observing the experimental results, it is not difficult to find that although the execution efficiency is different in different operating environments, the trend of computing costs is the same. This is also consistent with our theoretical analysis results.Figure 4
Fixed matrix size (take 10 as an example).
(a)
Python(b)
JavaIn Figure5, we fix the dimension of the matrix to 50 and increase the number of calculation execution rounds each time (here is the process of iterative calculation during the training process of the simulated machine learning model). It is not difficult to find that with the increase of execution rounds, the cost overhead of multiplication increases greatly, while the trend of increase in the cost overhead of addition is relatively gentle, because, with the increase of calculation rounds, the gap of single calculation overhead is enlarged.Figure 5
Fixed number of iterations (take the matrix with a dimension of 50 as an example).
(a)
Python(b)
JavaTherefore, through the above theoretical and experimental results, we can know that the matrix multiplication calculation cost is higher than the addition. In the field of machine learning, the size of the matrix (here refers to a small data set, because mobile devices and wearable devices cannot handle, store, and calculate large data sets) and the number of iterations are often relatively large. Therefore, it is necessary to determine which calculation overheads are needed to ensure that the device effectively offloads the calculation amount.


### 5.3. System Performance Analysis

For the performance of the system, this paper uses a real data set for comparison experiments. In the calculation process, the scheme uses a block matrix for calculation, which ensures that the calculation result is the same as the original calculation result. According to the characteristics of the block matrix calculation, it can be seen that the accuracy of the training model is not affected. Therefore, it is not difficult to see through a theoretical analysis that the calculation results generated by using this solution are consistent with the original calculation results. So, the comparison of the calculation results will not be performed here.The experimental parameters are as follows: CPU I5-2450M (2.5 GHz), 8 GB of memory, and Windows 10 64-bit operating system. The system is implemented using Python language, and because some edge devices may not be equipped with a GPU, this paper does not use a GPU to accelerate processing.In this paper, we first use the Boston house price prediction data set for experiments. The data set is507∗13. During the experiment, 400 pieces of data are selected for training. The average of 15 test results is used to determine the final experimental data results. The experimental test results are shown in Figure 6: the method proposed in this paper takes 24.76 ms, and the general training process takes 30.15 ms. From the results, it can be seen that the method in this paper is approximately 18% faster than the general method (In this paper, it refers to the general machine learning algorithm, that is, linear regression algorithm.).Figure 6
Comparison of time consumptions for the Boston house price prediction data set.The feature dimensions of the Boston house price prediction data set are small, as there are only 13 features. Afterward, this paper randomly generates a data set of size 500, 600, 700, etc., which has 300 features and 200 training rounds. The experimental results obtained are shown in Figure7.Figure 7
Comparison of the efficiencies of the proposed scheme and the general method.It can be seen from Figure7 that the solution proposed in this paper can effectively reduce the calculation amount of edge computing nodes. Among devices with limited resources, the system will effectively reduce the computing pressure of the device and alleviate the consumption of local resources.From the experimental results, the scheme in this paper saves time by approximately 20%, when compared with the unoptimized scheme. The machine learning algorithm used in this paper is relatively basic and has a small number of calculations. The results have a certain advantage in terms of time consumption. Since the main computing tasks in the training process are outsourced to other service nodes, appropriate adjustments can be applied to machine learning algorithms with more complex training processes. In general, the solution in this paper is suitable for application scenarios where the edge device cannot execute or is difficult to handle.At the same time, as shown in Figure8, this paper also compares with homomorphic encryption. By using a homomorphic encryption scheme to perform addition operations and multiplication operations of the same order of magnitude, this method can be seen to be more suitable for mobile devices in terms of efficiency. It is worth noting that it is because the addition homomorphic encryption time based on Paillier [48] cryptography runs far longer than does the scheme in this paper during the addition operation of the same order of magnitude (data is not shown in the figure).Figure 8
Comparison of the efficiencies of this scheme and RSA multiplication homomorphic encryption.In the initialization phase of the algorithm, the initialization time of the multiplicative homomorphic encryption based on RSA is approximately 45.4 ms, while the initialization time of the additive homomorphic encryption based on Paillier is approximately 1687.8 ms; the initialization time of the method in this paper varies with the amount of data formed by the matrix. When processing the same amount of data, the scheme in this paper takes the shortest time at this stage.Figure9 is a comparison of the data processing time required at the edge nodes. In the scheme in this paper, Algorithm 1 requires edge node calculation and generation, and the matrix inversion process can also be outsourced. Therefore, we calculate the computational cost of Algorithm 1. At the same time, we also compared the computational cost of the additive homomorphic encryption scheme in the encryption process (because this process is consistent with the purpose of Algorithm 1 in this paper). It is not difficult to find from Figure 9 that the program execution efficiency in this paper is higher and the trend is more gradual.Figure 9
Comparison of data processing time required on edge nodes.Through the experimental comparison and theoretical analysis, it can be seen that, by comparing with the scheme without any encryption and data perturbation, the scheme in this paper significantly improves the execution efficiency and other aspects, as well as ensured the security and accuracy of the data. Compared with the homomorphic encryption system, the security of this solution is weaker than the homomorphic encryption technology, but the execution efficiency is more suitable for devices with lower computing capabilities. This solution can enable low computing power equipment to process larger machine learning algorithms, while ensuring their safety and accuracy.As shown in Table1, we also compared the performance of various aspects. First, we will divide it into five levels from low to high. Because encryption technology will increase the system’s computational overhead, it also has an advantage in security. On the contrary, in the case of not using cryptographic schemes or using data perturbation schemes, performance will be more prominent, but it will bring a series of problems such as security and fidelity. Therefore, we evaluated and compared the schemes in the table.Table 1
Comparison of various performance indicators of the scheme.
Computational complexityRequired computing powerSecurityData fidelityGeneral outsourcing training programLowLowLowHighDifferential privacyMediumMediumHigherLowBased on homomorphic encryption [49]HighHighHighHighEVPP (our scheme)LowerLowerHigherHighAlthough the outsourcing scheme using homomorphic encryption has advantages in security and fidelity schemes, its complexity is higher. The scheme of using differential privacy will bring some loss in data accuracy. The solution proposed in this paper is slightly weaker in terms of security, but it ensures that the accuracy of the data is not lost. That can effectively protect user privacy and save power on mobile devices.


## 5.1. Security

The security of the solution is considered from the following aspects: data security and privacy, the correctness of the calculation results, and the trust mechanism of edge service nodes.


### 5.1.1. The Correctness of Calculation Results

When calculating a block matrix, it can be divided into two parts: matrix addition and matrix multiplication for analysis and consideration.First, verify the correctness of matrix multiplication.In the calculation ofXT′×X′, the result is
(10)XTD−1DXXTD−1M1+M2V10V2V1.And the result in the upper right corner of the matrix is not difficult to see asXT×X. For a further description of the calculation with w, it can be simplified to XTD−1DXRwT⋯0V2V1V3.SinceR is a diagonal matrix, it is assumed that k is randomly selected as the diagonal element value of the matrix. The matrix element of XT×X is xij. The calculation result of Formula (1) indicates
(11)∑j=1n∑i−1nx1jkwi1⋯∑j=1n∑i−1nx1jkwil⋮⋱⋮∑j=1n∑i−1nxnjkwi1⋯∑j=1n∑i−1nxnjkwilIt is not difficult to see that multiplying the matrix by1/kI can restore the original data.The correctness of the matrix addition is relatively simple and will not be repeated; the reader can prove it by himself.


### 5.1.2. Proof of Algorithm Security

In this section, the security of this scheme is demonstrated under five assumptions of insecurity.Hypothesis 1.
Malicious users obtain data through intermediate parameters.In the embodiment, for∆=XBT×XB×w+XBT×−YB, the data used for the matrixes X, Y, and w are used for the matrixes randomly perturbing invertible matrix operation, and the elements in the invertible matrix D are randomly generated and have no correlation. There is a transformation of XTD−1DX, which guarantees data security and recovery. Therefore, the service edge nodes cannot guess any information data from the matrix, to ensure the privacy of the data. At the same time, both the target matrix and the intermediate parameters are the result of addingK to the disturbance, and only users who master K can restore the data.In the training process, the learning rate (step size) is determined by the edge nodes, and the initialized parameters are random. Except for the calculations necessary for the outsourcing calculation, the other is done locally by the edge nodes. Therefore, the program also ensures the safety system model in the training process, the iterative training process is completed, and the edge service node only grasps the intermediate value, thus ensuring the safety training model.The scheme ensures the security of data during the exchange process. This paper only describes the solution with a linear regression model. When other machine learning algorithms are needed, only the calculation process needs to be improved.Hypothesis 2.
The malicious edge server can recover the intermediate parameter through the inverse matrix.In the solution described in this paper, all user data is added with disturbance and encapsulation, and edge servers cannot know the true meaning of their calculation data. Here, we assume a more powerful enemy that can conspire with the edge server that computes the inverse matrixD−1′ (this is not truly an inverse matrix). In this case, although the malicious server obtains the intermediate value of the relevant inverse matrix D−1, it cannot recover the accurate inverse matrix information, and it also cannot restore the original data.After generating the inverse matrix, the inverse matrix returned by the server contains the data disturbance. Therefore, except for the owner of the data, it is difficult for others to restore the original matrix and its inverse matrix.[44] has carried out detailed proof of the security analysis and will not repeat them here.Hypothesis 3.
The two edge servers conspire to get user data.In this method, the biggest threat is that the two edge service providers recover the data by obtaining the inverse matrixD−1 information of the user, and other methods cannot recover the data. This is similar to Hypothesis 8, due to a matrix calculated by one of the servers. The matrix contains inverse matrix and disturbance information, and to restore the true inverse matrix, the adversary needs to grasp the parameters of its disturbance information. Therefore, the final inverse matrix D−1 cannot be obtained between these two edge servers, so this scheme is already safe under this assumption.Hypothesis 4.
Trust mechanism of edge service nodes.In the system, this paper introduces an arbitration mechanism. When the edge nodes detect that the calculation result returned by the edge service node is abnormal, that is, the returned result is inconsistent with the verification result, then the edge nodes consider labelling the edge service node as malicious. The edge nodes verify the information, including the returned resultv, the raw data used to generate the verification results (V2, V1, V3, and V4), and the relevant identity information HIDu∣HIDs of the edge nodes and the service node, and then generate evidence and publish it to other nodes. When each node in the system receives the evidence, it performs a test. If it is indeed proved that the edge service node has not performed calculation tasks according to the agreement, it will be added to the blacklist, and the data will no longer be outsourced to the node. Concurrently, to ensure the long-term effectiveness of the system, you can look for trusted authorities (for example, government agencies) to store evidence and maintain node information in the system.Hypothesis 5.
Adversary obtains enough ciphertext for analysis.The scheme in this paper is to protect the security and privacy of the sample dataX, tag data Y, and parameter (weight) information w. Ensure that the adversary can assist the user in the calculation task and does not obtain any data information.In the scheme, the invertible matrixD and its inverse matrix D−1 are the key to ensure the computability and safety of X and XT. It has been proved in Hypotheses 7 and 8 that it can guarantee the security of the data. However, long-term use of the same set of matrixes (including the invertible matrix D and its inverse matrix D−1) is a security risk. The adversary may recover the original data contained in the disturbance data by collecting a large number of matrixes (for example, C1 and C2). In turn, it threatens the security of the program. Therefore, the frequency of use of D and D−1 can be adjusted to achieve better data security. The safest solution is to use different D and D−1 each time. Of course, this will increase the computational cost of the initialization phase (Algorithms 1 and 2). The computational cost of the initialization phase will be discussed in the experimental part.Hypothesis 6.
Discussion on scheme security model.When introducing Algorithm3, we have already mentioned that there is a certain probability of detecting dishonest edge service nodes. Since Algorithm 4 requires the edge device to run locally, it will cause additional computational costs. Assuming that the malicious edge service node is strong enough, it can know when the edge nodes perform the verification of outsourcing results. But this will increase the computational cost of the edge nodes. Although in the scheme in this paper the edge nodes can choose a smaller dimension of the verification matrix, it still has computational costs.At the same time, we designed evidence generation and verification algorithms in the scheme. Suppose that the rational edge service node in the system is detected to be evil, and the evidence of evil will be retained in the system. Other edge nodes can verify the evidence in the system. If the evidence is true, the malicious edge service node will lose the trust of the edge nodes, so that the edge nodes will not choose to submit outsourced computing tasks to it.Therefore, once the edge service node loses the outsourced task of the edge nodes, it will not receive the corresponding reward. So, for the rational node, it will still abide by the computing protocol. However, this also increases the computational cost of edge nodes. So, our purpose is to effectively reduce the computational cost of edge nodes, so our scheme is more used for semihonest models.


## 5.1.1. The Correctness of Calculation Results

When calculating a block matrix, it can be divided into two parts: matrix addition and matrix multiplication for analysis and consideration.First, verify the correctness of matrix multiplication.In the calculation ofXT′×X′, the result is
(10)XTD−1DXXTD−1M1+M2V10V2V1.And the result in the upper right corner of the matrix is not difficult to see asXT×X. For a further description of the calculation with w, it can be simplified to XTD−1DXRwT⋯0V2V1V3.SinceR is a diagonal matrix, it is assumed that k is randomly selected as the diagonal element value of the matrix. The matrix element of XT×X is xij. The calculation result of Formula (1) indicates
(11)∑j=1n∑i−1nx1jkwi1⋯∑j=1n∑i−1nx1jkwil⋮⋱⋮∑j=1n∑i−1nxnjkwi1⋯∑j=1n∑i−1nxnjkwilIt is not difficult to see that multiplying the matrix by1/kI can restore the original data.The correctness of the matrix addition is relatively simple and will not be repeated; the reader can prove it by himself.


## 5.1.2. Proof of Algorithm Security

In this section, the security of this scheme is demonstrated under five assumptions of insecurity.Hypothesis 1.
Malicious users obtain data through intermediate parameters.In the embodiment, for∆=XBT×XB×w+XBT×−YB, the data used for the matrixes X, Y, and w are used for the matrixes randomly perturbing invertible matrix operation, and the elements in the invertible matrix D are randomly generated and have no correlation. There is a transformation of XTD−1DX, which guarantees data security and recovery. Therefore, the service edge nodes cannot guess any information data from the matrix, to ensure the privacy of the data. At the same time, both the target matrix and the intermediate parameters are the result of addingK to the disturbance, and only users who master K can restore the data.In the training process, the learning rate (step size) is determined by the edge nodes, and the initialized parameters are random. Except for the calculations necessary for the outsourcing calculation, the other is done locally by the edge nodes. Therefore, the program also ensures the safety system model in the training process, the iterative training process is completed, and the edge service node only grasps the intermediate value, thus ensuring the safety training model.The scheme ensures the security of data during the exchange process. This paper only describes the solution with a linear regression model. When other machine learning algorithms are needed, only the calculation process needs to be improved.Hypothesis 2.
The malicious edge server can recover the intermediate parameter through the inverse matrix.In the solution described in this paper, all user data is added with disturbance and encapsulation, and edge servers cannot know the true meaning of their calculation data. Here, we assume a more powerful enemy that can conspire with the edge server that computes the inverse matrixD−1′ (this is not truly an inverse matrix). In this case, although the malicious server obtains the intermediate value of the relevant inverse matrix D−1, it cannot recover the accurate inverse matrix information, and it also cannot restore the original data.After generating the inverse matrix, the inverse matrix returned by the server contains the data disturbance. Therefore, except for the owner of the data, it is difficult for others to restore the original matrix and its inverse matrix.[44] has carried out detailed proof of the security analysis and will not repeat them here.Hypothesis 3.
The two edge servers conspire to get user data.In this method, the biggest threat is that the two edge service providers recover the data by obtaining the inverse matrixD−1 information of the user, and other methods cannot recover the data. This is similar to Hypothesis 8, due to a matrix calculated by one of the servers. The matrix contains inverse matrix and disturbance information, and to restore the true inverse matrix, the adversary needs to grasp the parameters of its disturbance information. Therefore, the final inverse matrix D−1 cannot be obtained between these two edge servers, so this scheme is already safe under this assumption.Hypothesis 4.
Trust mechanism of edge service nodes.In the system, this paper introduces an arbitration mechanism. When the edge nodes detect that the calculation result returned by the edge service node is abnormal, that is, the returned result is inconsistent with the verification result, then the edge nodes consider labelling the edge service node as malicious. The edge nodes verify the information, including the returned resultv, the raw data used to generate the verification results (V2, V1, V3, and V4), and the relevant identity information HIDu∣HIDs of the edge nodes and the service node, and then generate evidence and publish it to other nodes. When each node in the system receives the evidence, it performs a test. If it is indeed proved that the edge service node has not performed calculation tasks according to the agreement, it will be added to the blacklist, and the data will no longer be outsourced to the node. Concurrently, to ensure the long-term effectiveness of the system, you can look for trusted authorities (for example, government agencies) to store evidence and maintain node information in the system.Hypothesis 5.
Adversary obtains enough ciphertext for analysis.The scheme in this paper is to protect the security and privacy of the sample dataX, tag data Y, and parameter (weight) information w. Ensure that the adversary can assist the user in the calculation task and does not obtain any data information.In the scheme, the invertible matrixD and its inverse matrix D−1 are the key to ensure the computability and safety of X and XT. It has been proved in Hypotheses 7 and 8 that it can guarantee the security of the data. However, long-term use of the same set of matrixes (including the invertible matrix D and its inverse matrix D−1) is a security risk. The adversary may recover the original data contained in the disturbance data by collecting a large number of matrixes (for example, C1 and C2). In turn, it threatens the security of the program. Therefore, the frequency of use of D and D−1 can be adjusted to achieve better data security. The safest solution is to use different D and D−1 each time. Of course, this will increase the computational cost of the initialization phase (Algorithms 1 and 2). The computational cost of the initialization phase will be discussed in the experimental part.Hypothesis 6.
Discussion on scheme security model.When introducing Algorithm3, we have already mentioned that there is a certain probability of detecting dishonest edge service nodes. Since Algorithm 4 requires the edge device to run locally, it will cause additional computational costs. Assuming that the malicious edge service node is strong enough, it can know when the edge nodes perform the verification of outsourcing results. But this will increase the computational cost of the edge nodes. Although in the scheme in this paper the edge nodes can choose a smaller dimension of the verification matrix, it still has computational costs.At the same time, we designed evidence generation and verification algorithms in the scheme. Suppose that the rational edge service node in the system is detected to be evil, and the evidence of evil will be retained in the system. Other edge nodes can verify the evidence in the system. If the evidence is true, the malicious edge service node will lose the trust of the edge nodes, so that the edge nodes will not choose to submit outsourced computing tasks to it.Therefore, once the edge service node loses the outsourced task of the edge nodes, it will not receive the corresponding reward. So, for the rational node, it will still abide by the computing protocol. However, this also increases the computational cost of edge nodes. So, our purpose is to effectively reduce the computational cost of edge nodes, so our scheme is more used for semihonest models.


## 5.2. Analysis of Computational Complexity and Time Cost

In this section, we will discuss the computational overhead of matrix addition and multiplication operations commonly used in machine learning. Of course, matrix computing is used not only in the field of machine learning but also in many fields of scientific computing. Therefore, the final conclusion is generally applicable. For the convenience of description, it is assumed that twon∗n matrixes A, B need to be added and multiplied (subtraction and division operations can be transformed into addition or multiplication operations).
(i)
Addition. During the addition operation, we first need to traverse the elements in A by row and the elements in B by column. Since both A and B are n∗n matrixes, it is not difficult to see that the time complexity of addition is On2.(ii)
Multiply. During the multiplication operation, each row of A needs to be multiplied with each column of B, and then, the results are added. So, the time complexity of multiplication is On3.Next, we also conducted corresponding experiments in Python and Java.In Figure4, we use fixed calculation rounds to calculate the time cost of matrix addition and multiplication in different dimensions. By observing the experimental results, it is not difficult to find that although the execution efficiency is different in different operating environments, the trend of computing costs is the same. This is also consistent with our theoretical analysis results.Figure 4
Fixed matrix size (take 10 as an example).
(a)
Python(b)
JavaIn Figure5, we fix the dimension of the matrix to 50 and increase the number of calculation execution rounds each time (here is the process of iterative calculation during the training process of the simulated machine learning model). It is not difficult to find that with the increase of execution rounds, the cost overhead of multiplication increases greatly, while the trend of increase in the cost overhead of addition is relatively gentle, because, with the increase of calculation rounds, the gap of single calculation overhead is enlarged.Figure 5
Fixed number of iterations (take the matrix with a dimension of 50 as an example).
(a)
Python(b)
JavaTherefore, through the above theoretical and experimental results, we can know that the matrix multiplication calculation cost is higher than the addition. In the field of machine learning, the size of the matrix (here refers to a small data set, because mobile devices and wearable devices cannot handle, store, and calculate large data sets) and the number of iterations are often relatively large. Therefore, it is necessary to determine which calculation overheads are needed to ensure that the device effectively offloads the calculation amount.


## 5.3. System Performance Analysis

For the performance of the system, this paper uses a real data set for comparison experiments. In the calculation process, the scheme uses a block matrix for calculation, which ensures that the calculation result is the same as the original calculation result. According to the characteristics of the block matrix calculation, it can be seen that the accuracy of the training model is not affected. Therefore, it is not difficult to see through a theoretical analysis that the calculation results generated by using this solution are consistent with the original calculation results. So, the comparison of the calculation results will not be performed here.The experimental parameters are as follows: CPU I5-2450M (2.5 GHz), 8 GB of memory, and Windows 10 64-bit operating system. The system is implemented using Python language, and because some edge devices may not be equipped with a GPU, this paper does not use a GPU to accelerate processing.In this paper, we first use the Boston house price prediction data set for experiments. The data set is507∗13. During the experiment, 400 pieces of data are selected for training. The average of 15 test results is used to determine the final experimental data results. The experimental test results are shown in Figure 6: the method proposed in this paper takes 24.76 ms, and the general training process takes 30.15 ms. From the results, it can be seen that the method in this paper is approximately 18% faster than the general method (In this paper, it refers to the general machine learning algorithm, that is, linear regression algorithm.).Figure 6
Comparison of time consumptions for the Boston house price prediction data set.The feature dimensions of the Boston house price prediction data set are small, as there are only 13 features. Afterward, this paper randomly generates a data set of size 500, 600, 700, etc., which has 300 features and 200 training rounds. The experimental results obtained are shown in Figure7.Figure 7
Comparison of the efficiencies of the proposed scheme and the general method.It can be seen from Figure7 that the solution proposed in this paper can effectively reduce the calculation amount of edge computing nodes. Among devices with limited resources, the system will effectively reduce the computing pressure of the device and alleviate the consumption of local resources.From the experimental results, the scheme in this paper saves time by approximately 20%, when compared with the unoptimized scheme. The machine learning algorithm used in this paper is relatively basic and has a small number of calculations. The results have a certain advantage in terms of time consumption. Since the main computing tasks in the training process are outsourced to other service nodes, appropriate adjustments can be applied to machine learning algorithms with more complex training processes. In general, the solution in this paper is suitable for application scenarios where the edge device cannot execute or is difficult to handle.At the same time, as shown in Figure8, this paper also compares with homomorphic encryption. By using a homomorphic encryption scheme to perform addition operations and multiplication operations of the same order of magnitude, this method can be seen to be more suitable for mobile devices in terms of efficiency. It is worth noting that it is because the addition homomorphic encryption time based on Paillier [48] cryptography runs far longer than does the scheme in this paper during the addition operation of the same order of magnitude (data is not shown in the figure).Figure 8
Comparison of the efficiencies of this scheme and RSA multiplication homomorphic encryption.In the initialization phase of the algorithm, the initialization time of the multiplicative homomorphic encryption based on RSA is approximately 45.4 ms, while the initialization time of the additive homomorphic encryption based on Paillier is approximately 1687.8 ms; the initialization time of the method in this paper varies with the amount of data formed by the matrix. When processing the same amount of data, the scheme in this paper takes the shortest time at this stage.Figure9 is a comparison of the data processing time required at the edge nodes. In the scheme in this paper, Algorithm 1 requires edge node calculation and generation, and the matrix inversion process can also be outsourced. Therefore, we calculate the computational cost of Algorithm 1. At the same time, we also compared the computational cost of the additive homomorphic encryption scheme in the encryption process (because this process is consistent with the purpose of Algorithm 1 in this paper). It is not difficult to find from Figure 9 that the program execution efficiency in this paper is higher and the trend is more gradual.Figure 9
Comparison of data processing time required on edge nodes.Through the experimental comparison and theoretical analysis, it can be seen that, by comparing with the scheme without any encryption and data perturbation, the scheme in this paper significantly improves the execution efficiency and other aspects, as well as ensured the security and accuracy of the data. Compared with the homomorphic encryption system, the security of this solution is weaker than the homomorphic encryption technology, but the execution efficiency is more suitable for devices with lower computing capabilities. This solution can enable low computing power equipment to process larger machine learning algorithms, while ensuring their safety and accuracy.As shown in Table1, we also compared the performance of various aspects. First, we will divide it into five levels from low to high. Because encryption technology will increase the system’s computational overhead, it also has an advantage in security. On the contrary, in the case of not using cryptographic schemes or using data perturbation schemes, performance will be more prominent, but it will bring a series of problems such as security and fidelity. Therefore, we evaluated and compared the schemes in the table.Table 1
Comparison of various performance indicators of the scheme.
Computational complexityRequired computing powerSecurityData fidelityGeneral outsourcing training programLowLowLowHighDifferential privacyMediumMediumHigherLowBased on homomorphic encryption [49]HighHighHighHighEVPP (our scheme)LowerLowerHigherHighAlthough the outsourcing scheme using homomorphic encryption has advantages in security and fidelity schemes, its complexity is higher. The scheme of using differential privacy will bring some loss in data accuracy. The solution proposed in this paper is slightly weaker in terms of security, but it ensures that the accuracy of the data is not lost. That can effectively protect user privacy and save power on mobile devices.


## 6. Discussion on the Application of the Scheme in Other Algorithms

### 6.1. Analysis of Application Background

In the field of scientific computing, matrix operations are often used. In the field of machine learning, the optimal value cannot be obtained due to the results of one calculation. Therefore, it is necessary for the user (device) to go back to the calculation so that the calculation result can better approach the optimal value. This is often referred to as the model training process.In general equipment, such as personal computers and notebooks, training machine learning models are generally acceptable in terms of computing costs, power consumption, data storage, and so on. With the development of Internet of Things technology, more and more smartphones, wearable devices, etc., have been applied. However, the common problems of these devices are (1) the power storage capacity is weak, and they need to work continuously for a longer time; (2) the computing power of these devices is weak; and (3) the storage space of the device is limited. Therefore, this limits the data processing and computing capabilities of the device.Next, we will introduce the application scenarios of our scheme. Taking the smart bracelet as an example, users often use it to monitor the heart rate. If the user needs to monitor the heart rate status frequently or continuously, the power consumption of the device will be increased first, and secondly, the amount of data generated will increase as the monitoring frequency increases. For users with abnormal heart function, they may need to monitor heart rate changes in real time and be able to prompt the user in case of an abnormal heart rate. In this case, we need to get results as soon as possible while protecting user data.Therefore, our scheme can be applied to training user models, which includes initial training of user models and later adjustment of models. The solution can reduce the computing pressure of the device and improve efficiency, while protecting user data privacy as much as possible.


### 6.2. Application in Other Machine Learning Algorithms

Next, we will take the logistic regression algorithm as an example to briefly introduce how our scheme is applied to other algorithms.First, we give the expression of the logistic regression [50] model:
(12)y=11+e−wTx.Because, in the early stage of model training or application process, the value ofw is not optimal, we need to train w∗ to find the value that is most similar to w. We still use a gradient descent algorithm for optimization. Among them, N is the data or sample size. We only need to adjust Algorithm 1, and the implementation details will be given in Algorithm 13. Here, we only need to formulate the calculation rules according to Formula (13). The other algorithms are the same:
(13)wt−1=wt+α∑n=1Nyn−11+e−wtTxnxn.Algorithm 7: Outsourcing matrix generation algorithm (logistic regression).
Input: Keyk, Invertible matrix D1, PT, Q, Random matrixes (M1, M2, M3, and M4),Random verification matrixes (V1, V2, V3, and V4), Sample setX, Tag setY, and theinitialization vectorw.Output: Outsourcing matrixC2 , C1, C3, and C4.1: CalculateD=D1+D2;2: Go to the inverse matrix algorithm is outsourced to obtain the inverse matrixD−1;3: The edge computing node calculates the key matrixK←kI;4: Initialize the vector for scrambling (perturbation)w′←wTD and tag set Y′←αYD5: Design calculation rulesf=C4−C2/1+e−C3C4;6: According to the calculation rules, solve the verification factor detV;7: Construct the sample setX1′←D−1X and its transpose matrix X2′←αX8: returnC2←X2′M2V2, C1←X1′M1V1, C3←w′M3V3, C4←Y′M4V4


## 6.1. Analysis of Application Background

In the field of scientific computing, matrix operations are often used. In the field of machine learning, the optimal value cannot be obtained due to the results of one calculation. Therefore, it is necessary for the user (device) to go back to the calculation so that the calculation result can better approach the optimal value. This is often referred to as the model training process.In general equipment, such as personal computers and notebooks, training machine learning models are generally acceptable in terms of computing costs, power consumption, data storage, and so on. With the development of Internet of Things technology, more and more smartphones, wearable devices, etc., have been applied. However, the common problems of these devices are (1) the power storage capacity is weak, and they need to work continuously for a longer time; (2) the computing power of these devices is weak; and (3) the storage space of the device is limited. Therefore, this limits the data processing and computing capabilities of the device.Next, we will introduce the application scenarios of our scheme. Taking the smart bracelet as an example, users often use it to monitor the heart rate. If the user needs to monitor the heart rate status frequently or continuously, the power consumption of the device will be increased first, and secondly, the amount of data generated will increase as the monitoring frequency increases. For users with abnormal heart function, they may need to monitor heart rate changes in real time and be able to prompt the user in case of an abnormal heart rate. In this case, we need to get results as soon as possible while protecting user data.Therefore, our scheme can be applied to training user models, which includes initial training of user models and later adjustment of models. The solution can reduce the computing pressure of the device and improve efficiency, while protecting user data privacy as much as possible.


## 6.2. Application in Other Machine Learning Algorithms

Next, we will take the logistic regression algorithm as an example to briefly introduce how our scheme is applied to other algorithms.First, we give the expression of the logistic regression [50] model:
(12)y=11+e−wTx.Because, in the early stage of model training or application process, the value ofw is not optimal, we need to train w∗ to find the value that is most similar to w. We still use a gradient descent algorithm for optimization. Among them, N is the data or sample size. We only need to adjust Algorithm 1, and the implementation details will be given in Algorithm 13. Here, we only need to formulate the calculation rules according to Formula (13). The other algorithms are the same:
(13)wt−1=wt+α∑n=1Nyn−11+e−wtTxnxn.Algorithm 7: Outsourcing matrix generation algorithm (logistic regression).
Input: Keyk, Invertible matrix D1, PT, Q, Random matrixes (M1, M2, M3, and M4),Random verification matrixes (V1, V2, V3, and V4), Sample setX, Tag setY, and theinitialization vectorw.Output: Outsourcing matrixC2 , C1, C3, and C4.1: CalculateD=D1+D2;2: Go to the inverse matrix algorithm is outsourced to obtain the inverse matrixD−1;3: The edge computing node calculates the key matrixK←kI;4: Initialize the vector for scrambling (perturbation)w′←wTD and tag set Y′←αYD5: Design calculation rulesf=C4−C2/1+e−C3C4;6: According to the calculation rules, solve the verification factor detV;7: Construct the sample setX1′←D−1X and its transpose matrix X2′←αX8: returnC2←X2′M2V2, C1←X1′M1V1, C3←w′M3V3, C4←Y′M4V4


## 7. Conclusion

With the continuous improvement in mobile device capabilities and the need for low-latency applications, computing tasks will increasingly be migrated locally, but this also brings issues such as device energy consumption, occupied device computing, and storage. Therefore, outsourcing data to a near local end can reduce network transmission delays and reduce pressure on mobile devices. However, the security and privacy issues arising during the outsourcing process cannot be ignored. Therefore, this paper proposed EVPP: a secure data outsourcing computing solution based on matrix operations. In order to ensure that complex computing tasks can be effectively offloaded, we conducted theoretical and experimental analysis. Through the analysis of the results, we determined which calculation operations should be outsourced, effectively reducing the computing pressure of edge nodes. The outsourcing matrix adds a lightweight verification factor so that the verification process does not place excessive computing pressure on the mobile device. In terms of the trust mechanism, when the device finds a malicious service node in the system, it can verify that the data generates arbitration evidence so that other nodes in the system can verify and avoid sending outsourced tasks to malicious nodes. Through theoretical analysis and experimental comparison, it can be verified that the scheme exhibits certain improvements in efficiency, safety, and correctness and can be applied to practical applications. Because the solution in this paper only optimizes the training process, it has certain limitations. To better reduce the complexity of machine learning models for equipment training in edge environments and, at the same time, due to the insufficient data volume of a single mobile device, how to build a distributed algorithm is also a problem worth considering. Therefore, the next step will be to combine federal learning [51, 52] to study the problem of collaboration between multiple nodes in a distributed system.



---

*Source: 8825623-2020-11-17.xml*