# Practical Privacy Preserving-Aided Disease Diagnosis with Multiclass SVM in an Outsourced Environment

Authors: Ruoli Zhao; Yong Xie; Xingxing Jia; Hongyuan Wang; Neeraj Kumar
Journal: Security and Communication Networks
 (2022)

---

## Abstract

With the rapid development of cloud computing and machine learning, using outsourced stored data and machine learning model for training and online-aided disease diagnosis has a great application prospect. However, training and diagnosis in an outsourced environment will cause serious challenges to the privacy of data. At present, many scholars have proposed privacy preserving machine learning schemes and made a lot of progress, but there are still great challenges in security and low client load. In this paper, we propose a complete privacy preserving outsourced multiclass SVM training and aided disease diagnosis scheme. We design some efficient basic operation algorithms for encrypted data. Then, we design an efficient and privacy preserving SVM model training protocol using the basic operation algorithms. We propose a secure maximum finding algorithm and secure comparison algorithm. Then, we design an efficient online-aided disease diagnosis scheme based on the BFV cryptosystem and blinding technique. Detailed security analysis proves that our scheme can protect the privacy of each participant. The experimental results illustrate that our proposed scheme significantly reduces the computation overhead compared with the previous similar works. Our proposed scheme completes most of the operations of aided disease diagnosis by the cloud servers and the client only needs to complete a small amount of encryption and decryption operations. The overall computation overhead is 0.175 s, and the efficiency of online aided disease diagnosis is improved by 85.4%. At the same time, our proposed scheme provides multiclass diagnosis results, which can better assist doctors in their treatment.


---

## Body

## 1. Introduction

Machine learning (ML) uses the computer system to build mathematical models on sample data with statistical methods and makes predictions or decisions without being explicitly programmed. Now, ML has shown significant advantages in the field of disease diagnosis and brings more and more convenience to the prevention and treatment of diseases.With the rapid development of cloud computing technology, cloud service providers (CSP) have high-quality computation and huge storage space, which can provide data processing, model training, diagnosis services and deployment, and other intelligent solutions based on machine learning. In this context, the local clients will outsource their medical data and machine learning models to CSP without having to build their own large-scale infrastructure and computing resources. The cloud can train a machine learning model and provide aided disease diagnosis service by using the outsourced medical data and machine learning models, which can help improve doctors’ diagnosis, treatment decisions and provide patients an online disease diagnosis service. A typical cloud platform machine learning system architecture is shown in Figure1.Figure 1
A typical cloud platform machine learning system architecture.However, the security and privacy of outsourced data will be threatened by various threats, making people afraid to use the service of CSP. The security and privacy threats are mainly reflected in the leakage of the data, the machine learning model of the model owners, the users’ request, and diagnosis results. As we all know, the leakage of medical information may cause irreversible losses or become a major event. Therefore, the security and privacy preserving of model training and diagnosis based on cloud computing have become a major challenge.To address the abovementioned challenges, many scholars have proposed various schemes, such as a secure outsourced classification based on logistic regression model [1], an electronic medical disease risk prediction scheme based on naive Bayes model [2], and other secure disease prediction schemes based on machine learning technology [3–5]. As a machine learning algorithm with high computational efficiency and nice predictive accuracy, the support vector machine (SVM) has achieved high classification accuracy and efficiency in the medical field [6, 7]. However, the existing privacy preserving SVM schemes mainly implement secure prediction [8–11], and there are few privacy preserving SVM schemes for secure training. Most of the existing privacy preserving SVM schemes are designed for binary classification, which can only determine whether the patient has the disease [12], but cannot deal with the multiclass of the disease. In addition, multiclass SVM requires more computation, which will reduce the efficiency [13].To solve the abovementioned problems, we propose an efficient and privacy preserving online disease diagnosis scheme based on the SVM algorithm. In our scheme, we can achieve multi-class SVM training on the encrypted outsourced data from multiple data owners and provide users with privacy preserving disease diagnosis. In summary, our contributions are as follows:(1)
Efficient and secure basic operation algorithms: Based on the Paillier cryptosystem, we design several basic operation algorithms to realize the secure outsourced data storage and computation, including secure aggregation algorithm, secure multiplication algorithm, and so on. These secure computation algorithms are the building blocks for our proposed training protocol.(2)
Completing machine learning process under privacy preserving: Aiming at the general machine learning process and the goal of privacy preserving, we propose a privacy preserving outsourced multiclass SVM model training and online-aided disease diagnosis scheme. Different from the existing privacy preserving schemes that only support training or diagnosis, our proposed scheme extends the function of privacy preserving machine learning system.(3)
Efficient and secure online aided disease diagnosis: Based on the BFV cryptosystem, we design a secure maximum finding algorithm and secure comparison algorithm. We provide an efficient and privacy preserving aided disease diagnosis scheme. Experimental results illustrate that our proposed scheme significantly reduces the computation cost than the existing similar schemes, which is suitable for practical application scenarios where a large number of users request diagnosis at the same time.(4)
Low overhead for local client: For a local client, the client only needs to perform encryption and decryption operations in our proposed scheme, which reduces the storage and computation overhead of the local client to the greatest extent and makes full use of the computation power of the cloud servers.The remainder of this paper is organized as follows. In Section2, we review some related works. In Section 3, we review the Paillier cryptosystem, BFV cryptosystem, and SVM algorithm as preliminaries. In Section 4, we make a system overview. Then, we propose our scheme in Section 5. In Section 6, we analyze the security of our proposed scheme. In Section 7, we make a performance evaluation. Finally, we conclude this paper in Section 8.


## 2. Related Work

In this section, we summarize the privacy preserving machine learning schemes in recent years.With the development of big data era, machine learning has been widely used in many fields. Among them, the application of machine learning in the field of intelligent disease diagnosis has developed rapidly. Disease diagnosis schemes based on various machine learning classification algorithms have been proposed [14–17]. However, at the same time, the problem of privacy disclosure in the machine learning process is becoming more and more serious. So, many scholars have carried out the research studies on privacy preserving machine learning.Triastcyn and Faltings [18] proposed the Bayesian differential privacy, considered the distribution of data and provided a more practical privacy guarantee. Laur et al. [19] proposed a privacy preserving scheme of support vector machine based on secure multiparty computation. For each training or testing phase, their scheme involves multiple parties holding encrypted data and secret sharing obtained during training. Based on additive homomorphic encryption, Mandal and Gong [20] designed a privacy preserving scheme that performs gradient descent on data owners and cloud server. They achieved secure linear and logistic regression model training. Shen et al. [21] used blockchain technology to establish a secure and reliable data sharing platform among multiple data providers and constructed a privacy preserving support vector machine training scheme based on the Paillier cryptosystem. However, in their scheme, the data provider needs to interact with the cloud server to complete the computation. The computation cost of the data provider is large. Liu et al. [22] proposed a privacy preserving clinical decision support system using the naive Bayes (NB) classifier. The BGV homomorphic encryption system significantly improved the performance. In work [23], a framework for securely and efficiently outsourcing decision tree inference was proposed. Tan et al. [24] proposed a system for privacy-preserving machine learning that implements all operations on the GPU, which makes full use of the computing power of GPU. Zheng et al. [25] combined random permutation and arithmetic secret sharing by the compute-after-permutation technique and built a privacy-preserving machine learning framework. Li et al. [26] proposed a verifiable privacy-preserving machine learning prediction scheme for the edge-enhanced HCPSs, which outputs the verifiable prediction results for users without privacy leakage. Ma et al. [27] designed a lightweight privacy-preserving medical diagnosis mechanism on edge called LPME.Among them, the SVM algorithm is a research hotspot and has been widely used in different data mining and machine learning schemes. Most of the existing privacy preserving SVM schemes are based on three main privacy preserving technologies: differential privacy (DP), secure multi-party computation (SMC), and homomorphic encryption (HE). DP can significantly improve the calculation and communication efficiency, but the cost is to sacrifice the accuracy of the model by adding random noise [28, 29]. Zhang et al. [30] proposed a general differential privacy model fitting method based on the genetic algorithm, but it reduces the decision accuracy of the model. SMC alleviates the limitation of computing but requires more interaction between participants. This leads to expensive communication overhead [31, 32]. Yu et al. [33] first proposed a privacy preserving SVM classification method based on vertically segmented data. They use SMC technology to obtain the global model, so as to protect the local privacy data and hide the classification model. However, this method requires at least three parties to participate in the calculation, which is complex and inefficient. HE can directly calculate the encrypted data, but it also requires a lot of computing costs [34, 35]. Bajard et al. [36] uses HE technology to protect the decision model and medical data, but it needs high computational load. Therefore, it is necessary to design an efficient and secure SVM scheme for cloud online disease diagnosis service. Wang et al. [37] proposed an efficient privacy preserving outsourced SVM scheme for Internet of medical things deployment, which protected training data privacy and guaranteed the security of the trained SVM model.In this paper, we propose a new privacy preserving scheme for training and disease diagnosis of the multiclass SVM algorithm. We make a comparison analysis with the schemes in [38–40]. The experimental results demonstrate that our scheme has more practical application values.


## 3. Preliminaries

In this section, we describe some techniques as the basis of our scheme, including the Paillier cryptosystem, BFV cryptosystem, and SVM algorithm.


### 3.1. Paillier Cryptosystem

In the training phase, the data are encrypted by the Paillier cryptosystem [41]. The Paillier cryptosystem is a public key cryptosystem with additive homomorphic operation. We will introduce the Paillier cryptosystem as follows.(i)
Key generation: Set the security parameterk. Choose two big primes p,q, p=q=k,n=p⋅q,λ=l cmp−1,q−1, λ is the Carmichael function of n. Choose a random number g∈Zn2∗, and gcdLgλmodn2,n=1,Lx=x−1/n. The public key is pk=n,g. The private key is sk=λ.(ii)
Encryption: Givenm∈Zn. The message m will be encrypted with pk. The ciphertext is expressed as c=Epkm=gmrnmodn2, where r∈Zn∗ is a random number.(iii)
Decryption: According to the key generation stage and Carmichael’s theorem,gλ≡1modn. So gλ=kn+1. Then, m=Dskc=Lcλmodn2/Lgλmodn2modn.(iv)
Homomorphic computation: Given two ciphertextsEpkm1,Epkm2 under the same public key pk. The homomorphic computations are defined as Epkm1+m2=Epkm1⋅Epkm2,Epkm1⋅m2=Epkm1m2.


### 3.2. BFV Cryptosystem

In the prediction phase, the data are encrypted by the BFV cryptosystem [34]. BFV cryptosystem is a leveled-FHE public key cryptosystem based on RLWE, which can support unlimited times additive homomorphic operation and limited times multiplicative homomorphic operation.(i)
Key generation: Generate a polynomials=Zx/xd+1. The private key is defined as sk=s. Then, generate a polynomial from ciphertext polynomial space (polynomial s), a=Zqx/xd+1. The polynomial a is used to generate public key. Define a noise polynomial e←χ. The notation χ expresses the Gaussian distribution. The public key is pk=−a⋅s+eq,a.(ii)
Encryption: The messagem∈Rt. Define p0=pk0,p1=pk1,u←χ,e1←χ,e2←χ. The ciphertext c is computed as c=p0⋅u+t⋅e2+m,p1⋅u+t⋅e1.(iii)
Decryption: To decrypt the ciphertextc, define c0=p0⋅u+t⋅e2+m,c1=p1⋅u+t⋅e1. The message m is computed as m=c0+c1⋅smodt.(iv)
Homomorphic computation: BFV cryptosystem supports ciphertext batch processing. Define twoz-dimensional vectors encrypted under public key pk, Epkx1,x2,…,xz,Epky1,y2,…,yz. The homomorphic computations are defined as follows:(1)Epkx1+y1,x2+y2,…,xz+yz=Epkx1,x2,…,xz+Epky1,y2,…,yz,Epkx1+⋅y1,x2⋅y2,…,xz⋅yz=Epkx1,x2,…,xz⋅Epky1,y2,…,yz.


### 3.3. SVM Algorithm

SVM is a classical supervised learning algorithm to solve two kinds of classification problems. The SVM algorithm will find the best hyperplane. The classifier is a decision functionfX=W⋅X+b, fX≥0 expresses positive class and fX<0 expresses negative class.There are two training methods for the SVM model: one is based on the SMO algorithm and the other is based on the gradient descent algorithm. Because the operation steps of the SMO algorithm are more complex, which makes a lot computation costs when using encrypted data. Therefore, we choose gradient descent to realize the privacy preserving SVM model training. In the SVM model training process based on the gradient descent method, the objective functionLX=1/2W2+C∑i=1nmax0,1−yiW⋅X+b=1/2W2+C∑i=1nloss needs to be minimized. When yiW⋅X+b≥1, it means that the classification is correct. The loss=0 and the parameters do not need to be updated. When yiW⋅X+b<1, it means that the classification is incorrect. The loss=1−yiW⋅X+b and the parameters need to be updated.


## 3.1. Paillier Cryptosystem

In the training phase, the data are encrypted by the Paillier cryptosystem [41]. The Paillier cryptosystem is a public key cryptosystem with additive homomorphic operation. We will introduce the Paillier cryptosystem as follows.(i)
Key generation: Set the security parameterk. Choose two big primes p,q, p=q=k,n=p⋅q,λ=l cmp−1,q−1, λ is the Carmichael function of n. Choose a random number g∈Zn2∗, and gcdLgλmodn2,n=1,Lx=x−1/n. The public key is pk=n,g. The private key is sk=λ.(ii)
Encryption: Givenm∈Zn. The message m will be encrypted with pk. The ciphertext is expressed as c=Epkm=gmrnmodn2, where r∈Zn∗ is a random number.(iii)
Decryption: According to the key generation stage and Carmichael’s theorem,gλ≡1modn. So gλ=kn+1. Then, m=Dskc=Lcλmodn2/Lgλmodn2modn.(iv)
Homomorphic computation: Given two ciphertextsEpkm1,Epkm2 under the same public key pk. The homomorphic computations are defined as Epkm1+m2=Epkm1⋅Epkm2,Epkm1⋅m2=Epkm1m2.


## 3.2. BFV Cryptosystem

In the prediction phase, the data are encrypted by the BFV cryptosystem [34]. BFV cryptosystem is a leveled-FHE public key cryptosystem based on RLWE, which can support unlimited times additive homomorphic operation and limited times multiplicative homomorphic operation.(i)
Key generation: Generate a polynomials=Zx/xd+1. The private key is defined as sk=s. Then, generate a polynomial from ciphertext polynomial space (polynomial s), a=Zqx/xd+1. The polynomial a is used to generate public key. Define a noise polynomial e←χ. The notation χ expresses the Gaussian distribution. The public key is pk=−a⋅s+eq,a.(ii)
Encryption: The messagem∈Rt. Define p0=pk0,p1=pk1,u←χ,e1←χ,e2←χ. The ciphertext c is computed as c=p0⋅u+t⋅e2+m,p1⋅u+t⋅e1.(iii)
Decryption: To decrypt the ciphertextc, define c0=p0⋅u+t⋅e2+m,c1=p1⋅u+t⋅e1. The message m is computed as m=c0+c1⋅smodt.(iv)
Homomorphic computation: BFV cryptosystem supports ciphertext batch processing. Define twoz-dimensional vectors encrypted under public key pk, Epkx1,x2,…,xz,Epky1,y2,…,yz. The homomorphic computations are defined as follows:(1)Epkx1+y1,x2+y2,…,xz+yz=Epkx1,x2,…,xz+Epky1,y2,…,yz,Epkx1+⋅y1,x2⋅y2,…,xz⋅yz=Epkx1,x2,…,xz⋅Epky1,y2,…,yz.


## 3.3. SVM Algorithm

SVM is a classical supervised learning algorithm to solve two kinds of classification problems. The SVM algorithm will find the best hyperplane. The classifier is a decision functionfX=W⋅X+b, fX≥0 expresses positive class and fX<0 expresses negative class.There are two training methods for the SVM model: one is based on the SMO algorithm and the other is based on the gradient descent algorithm. Because the operation steps of the SMO algorithm are more complex, which makes a lot computation costs when using encrypted data. Therefore, we choose gradient descent to realize the privacy preserving SVM model training. In the SVM model training process based on the gradient descent method, the objective functionLX=1/2W2+C∑i=1nmax0,1−yiW⋅X+b=1/2W2+C∑i=1nloss needs to be minimized. When yiW⋅X+b≥1, it means that the classification is correct. The loss=0 and the parameters do not need to be updated. When yiW⋅X+b<1, it means that the classification is incorrect. The loss=1−yiW⋅X+b and the parameters need to be updated.


## 4. System Overview

In this section, we will introduce our system model, security goals, and threat model.


### 4.1. System Model

Our system model should achieve the privacy preserving training and online disease diagnosis process. Therefore, our system model is designed as shown in Figure2.Figure 2
System model.There are six participants in our system model, which are trusted authority (TA), medical centers (MCs), cloud storage server (CSS), cloud computation server (CCS), diagnosis service provider (DSP), and users.(i)
Trusted authority (TA): TA is the fully trusted party of the whole system, which is used to generate and distribute keys for other participants in the system. After initialization, TA will stay offline.(ii)
Medical centers (MCs): Each MC has its own local medical data. To reduce the local storage cost, MCs will outsource the medical data to CSS for storage.(iii)
Cloud storage server (CSS): CSS has the ability to store and manage outsourced data. CSS can perform privacy preserving computation with its powerful computation power.(iv)
Cloud computation server (CCS): CCS assists CSS to complete privacy preserving computation.(v)
Diagnosis service provider (DSP): DSP wants to train a machine learning model on the outsourced data from MCs and provides online aided disease diagnosis for users. Due to the limited computation and communication ability, DSP will outsource the training and diagnosis to CSS.(vi)
Users: Users are patients or doctors who have unlabeled samples and want to get the diagnosis results. The users will send encrypted diagnosis requests to CSS and obtain the encrypted results. The users can decrypt the results with own private key.


### 4.2. Security Goals

In order to meet the security requirements of outsourced training and diagnosis, our scheme will achieve the following security goals.(i)
Medical data privacy: The outsourced data of MCs will not be leaked to other participants in the whole machine learning process.(ii)
Model privacy: Other participants cannot learn any useful information about the model of DSP.(iii)
Users privacy: The diagnosis requests and results of users will not be acquired by other participants.(iv)
Intermediate results privacy: In the execution of protocols, any participant will not infer other participants’ sensitive information through the intermediate results.In our scheme, the training and diagnosis processes are completed by CSS and CCS. All participants are semi-honest (or honest-but-curious). Specifically, they will honestly implement the secure computation protocols, but they will try to analyze the sensitive data and intermediate results to infer the useful information of other participants. Like the previous works, we assume that CSS and CCS will not collude. Because CSS and CCS belong to different commercial companies, they will not collude with each other for their own reputation.


### 4.3. Threat Model

In this paper, we will define three attacks in our system model.(i)
Eavesdropping attack: This attack means that an adversary can eavesdrop and analyze data during the data transmission. The data transmission includes outsourcing process and the interaction between participants in protocol implementation.(ii)
Honest-but-curious attack: All participants will implement the protocol honestly, but they will infer the useful information during the execution of protocols.(iii)
Client-collusion attack: In the training and diagnosis process, some clients may collude to analyze the useful information of other participants.


## 4.1. System Model

Our system model should achieve the privacy preserving training and online disease diagnosis process. Therefore, our system model is designed as shown in Figure2.Figure 2
System model.There are six participants in our system model, which are trusted authority (TA), medical centers (MCs), cloud storage server (CSS), cloud computation server (CCS), diagnosis service provider (DSP), and users.(i)
Trusted authority (TA): TA is the fully trusted party of the whole system, which is used to generate and distribute keys for other participants in the system. After initialization, TA will stay offline.(ii)
Medical centers (MCs): Each MC has its own local medical data. To reduce the local storage cost, MCs will outsource the medical data to CSS for storage.(iii)
Cloud storage server (CSS): CSS has the ability to store and manage outsourced data. CSS can perform privacy preserving computation with its powerful computation power.(iv)
Cloud computation server (CCS): CCS assists CSS to complete privacy preserving computation.(v)
Diagnosis service provider (DSP): DSP wants to train a machine learning model on the outsourced data from MCs and provides online aided disease diagnosis for users. Due to the limited computation and communication ability, DSP will outsource the training and diagnosis to CSS.(vi)
Users: Users are patients or doctors who have unlabeled samples and want to get the diagnosis results. The users will send encrypted diagnosis requests to CSS and obtain the encrypted results. The users can decrypt the results with own private key.


## 4.2. Security Goals

In order to meet the security requirements of outsourced training and diagnosis, our scheme will achieve the following security goals.(i)
Medical data privacy: The outsourced data of MCs will not be leaked to other participants in the whole machine learning process.(ii)
Model privacy: Other participants cannot learn any useful information about the model of DSP.(iii)
Users privacy: The diagnosis requests and results of users will not be acquired by other participants.(iv)
Intermediate results privacy: In the execution of protocols, any participant will not infer other participants’ sensitive information through the intermediate results.In our scheme, the training and diagnosis processes are completed by CSS and CCS. All participants are semi-honest (or honest-but-curious). Specifically, they will honestly implement the secure computation protocols, but they will try to analyze the sensitive data and intermediate results to infer the useful information of other participants. Like the previous works, we assume that CSS and CCS will not collude. Because CSS and CCS belong to different commercial companies, they will not collude with each other for their own reputation.


## 4.3. Threat Model

In this paper, we will define three attacks in our system model.(i)
Eavesdropping attack: This attack means that an adversary can eavesdrop and analyze data during the data transmission. The data transmission includes outsourcing process and the interaction between participants in protocol implementation.(ii)
Honest-but-curious attack: All participants will implement the protocol honestly, but they will infer the useful information during the execution of protocols.(iii)
Client-collusion attack: In the training and diagnosis process, some clients may collude to analyze the useful information of other participants.


## 5. Proposed Scheme

In this section, we describe the proposed scheme in detail. Our scheme mainly includes system initialization, privacy preserving machine learning training, and online disease diagnosis.In order to accurately describe our proposed scheme, we give the description of used notations in Table1.Table 1
Notation and definition.
NotationDefinitionlxThe key length ofxPKP,SKPPaillier public-private key pair of CCSPKB,SKBBFV public-private key pair of CCSPKPC,SKPCPaillier public-private key pair of CSSPKBC,SKBCBFV public-private key pair of CSSPKPD,SKPDPaillier public-private key pair of DSPidiThe authentication ofMCiEThe precision of floating point numbersaiThe private key ofMCixPKThe ciphertext ofx under PKLClassification numbersdThe degree of polynomial


### 5.1. System Initialization

In the system initialization phase, TA generates system parameters and distributes the parameters for MCs, CSS, CCS, and DSP, respectively. TA sends the parameters through the secure communication channel. Then, TA will stay offline. We assume that there arem MCs in our system. Because the Paillier cryptosystem and BFV cryptosystem can only encrypt integers, the floating point numbers and negative numbers should be converted into integers. Therefore, all participants should make data conversion before encrypting their sensitive information.


#### 5.1.1. Generate System Parameters

(1)
Generate a public-private key pairPKP=NP,g,SKP of the Paillier cryptosystem and a public-private key pair PKB,SKB of the BFV cryptosystem. The BFV plaintext space is NB. The public keys are public and the private keys are sent to the CCS.(2)
Generate a public-private key pairPKPC,SKPC of the Paillier cryptosystem and a public-private key pair PKBC,SKBC of the BFV cryptosystem for CSS. The BFV plaintext space is NBC. The public keys are public and the private keys are sent to CSS.(3)
Generate a public-private key pairPKPD,SKPD of the Paillier cryptosystem for DSP. The public key is public and the private key is sent to DSP.(4)
Generate a random integerω∈NP. TA randomly splits ω to m integers, satisfying ω1+ω2+⋯+ωm=ω and sends ωi to MCi. Then, generate two lists H and H′. Each list has m random integers, H=n1,n2,…,nm,ni∈NP,H′=n1′,n2′,…,nm′,ni′∈NP. Each element in H and H′ represents the ID of each MC. When MCi sends authentication idi to CSS, MCi will hide gai and ωi with ni and ni′, respectively. The ni,ni′ is sent to MCi. H and H′ are sent to CSS.


#### 5.1.2. Data Conversion

In the machine learning application scenario, data and model parameters contain floating point numbers and negative numbers.For a floating point numberx, we enlarge x to x⋅2E (E is the precision of floating point numbers). For example, given a floating point number x=3.61 and the precision E=20, we can convert x into an integer x′=3785359. For a negative number y, we divide the plaintext space N (N is expressed the plaintext space of the Paillier or BFV cryptosystem) into two parts because all variables and intermediate results in the process of training and prediction are much smaller than N/2. An integer in 0,N/2 represents a positive integer and N/2,N−1 represents a negative integer. When encrypting the negative integer y, it is converted to encrypt N−y. If y is both a floating point number and a negative number, y is first converted into a negative integer.


### 5.2. Privacy Preserving Machine Learning Training

The privacy preserving machine learning training process is completed by CSS and CCS. We assume that the amount of outsourced data isn.


#### 5.2.1. Local Data Outsourcing

To protect the privacy of MCs’ local data, MCs will encrypt the data before outsourcing. The outsourcing process ofMCii=1,…,m is as follows.(1)
MCi generates a random integer ai∈ZNP. Computing pki=gaimodNP2 as public key and the private key is ski=ai.(2)
Computinghi=gai+ωmodNP2.(3)
For each plaintext data, such asx, MCi will make a data conversion as mentioned in Section 5.1. Then, compute xMCi=gxrNP+hi to encrypt and outsource the encrypted data to CSS for storage.


#### 5.2.2. Secure Basic Building Blocks for Training

To complete the privacy preserving outsourced training, we construct some algorithms as basic building blocks based on the Paillier cryptosystem: secure data aggregation (Block_1), secure multiplication algorithm (Block_2), secure inner product algorithm (Block_3), secure scalar multiplication of vector algorithm (Block_4), and secure symbol judgment algorithm (Block_5). The algorithms will be executed with CSS and CCS.(1) Secure data aggregation algorithm (Block_1). CSS needs to aggregate MCs’ outsourced data before starting machine learning training. The algorithm works as follows and is described in Algorithm 1.(1)
CSS sends a training request toMCi, i=1,2,…,m.(2)
After receiving the training request,MCi computes idi=pki+ni,ωi+ni′ as authentication (The idi indicates that CSS is allowed to use the outsourced data of MCi for training) and sends to CSS.(3)
CSS obtains thepki,ωi of MCi through the idi and computes ω=ω1+ω2+⋯+ωm. It should be noted that ω can be obtained only after all MCs have sent their authentication. Then, CSS computes hi=gai+ωmodNP2 and completes the aggregation.(2). Secure multiplication algorithm (Block_2). Given two encrypted integers xPKP and yPKP, the algorithm needs to compute x⋅yPKP. The algorithm works as follows and is described in Algorithm 2.(1)
CSS generates two random integersR1,R2 and R1,R2∈ZNP. Then, it computes by applying the additive homomorphism, obtaining the following results.(2)x+R1PKP=xPKP⋅gR1⋅,y+R2PKP=yPKP⋅gR2⋅,Then, sending them to CCS.(2)
CCS generates a random integerT,T∈ZNP. It decrypts y+R2PKP by using SKP. Then, it encrypts y+R2+TmodNPC with PKPC to get y+R2+TPKPC. Computing xT+R1TPKP=x+R1PKPT and encrypting T with PKP. Sending y+R2+TPKPC,xT+R1TPKP and TPKP to CSS.(3)
CSS decryptsy+R2+TPKPC with SKPC and computes y+T. Then, computing by applying the additive homomorphism, obtaining the following results.(3)xy+xTPKP=xPKPy+T,xy−R1TPKP=xy+xTPKP⋅xT+R1TPKP−1,R1TPKP=TPKPR1.Computing the result,(4)xyPKP=xy−R1TPKP⋅R1TPKP.(3). Secure inner product algorithm (Block_3). Given two encrypted vectors XPKP,YPKP. The algorithm will compute X⋅YPKP and is described in Algorithm 3.Algorithm 1: Secure data aggregation (Block_1).
Input: the authentication and outsourced data ofMCi.Output: the training data.CSS:(1)
Send a training request toMCi, i=1,2,…,m.MCs:(2)
fori=1⟶m:MCi sends idi=pki+ni,ωi+ni′ to CSSendfor CCS:(3)
CSS obtainspki,ωi, i=1,2,…,m(4)
Computeω=ω1+ω2+⋯+ωm(5)
fori=1⟶m:Computehi=gai+ωmodNP2For each outsourced data ofMCi, such as xMCi,ComputexPKP=xMCi−hi to complete aggregateendforAlgorithm 2: Secure multiplication (Block_2).
Input:xPKP,yPKPOutput:xyPKPCSS:(1)
R1,R2∈ZNP(2)
x+R1PKP=xPK1⋅gR1y+R2PKP=yPKP⋅gR2(3)
Sendx+R1PKP,y+R2PKP to CCS.CCS:(4)
T∈ZNP(5)
Decrypty+R2PKP(6)
xT+R1TPKP=x+R1PKPT(7)
Encrypty+R2+TmodNPC with PKPC(8)
EncryptT with PKP(9)
SendxT+R1TPKP,y+R2+TPKPC and TPKP to CSS.CSS:(10)
Decrypty+R2+TPKPC with SKPC and Compute y+T(11)
xy+xTPKP=xPKPy+T(12)
xy−R1TPKP=xy+xTPKP⋅xT+R1TPKP−1(13)
R1TPKP=TPKPR1(14)
xyPKP=xy−R1TPKP⋅R1TPKPAlgorithm 3: Secure inner product (Block_3).
Input:XPKP,YPKPOutput:X⋅YPKPCSS:(1)
DefineX⋅YPKP=1PKP.(2)
fori=1⟶X.length:X⋅YPKP⋅=Block_1xiPKP,yiPKPendfor(4). Secure scalar multiplication of vector algorithm (Block_4). Given a encrypted vector XPKP and a encrypted integer yPKP, the algorithm will compute y⋅XPKP and is described in Algorithm 4.Algorithm 4: Secure scalar multiplication of vector (Block_4).
Input:XPKP,yPKPOutput:y⋅XPKPCSS:(1)
Definey⋅XPKP=1,1,…,1PKP.(2)
fori=1⟶X.length:yxiPKP=Block_1yPKP,xiPKPendforAlgorithm 5: Secure symbol judgment (Block_5).
Input:xPKPOutput:judgeCSS:(1)
Choose a random integerr,lr<lNP/2(2)
x⋅rPKP=xPKPr(3)
Sendx⋅rPKP to CCSCCS:(4)
Decryptx⋅rPKP(5)
ifx⋅r≥0:judge=1,else:judge=0(6)
Encryptjudge with PKPC(7)
SendjudgePKPC to CSS(5). Secure symbol judgment algorithm (Block_5). Given an encrypted integer xPKP, the algorithm will compute the sign of xPKP. Let judge=1 if x≥0 else judge=0. The algorithm works as follows and is described in Algorithm 5.(1)
CSS chooses a random integerr,lr<lNP/2. Then, it computes x⋅rPKP=xPKPr by applying the additive homomorphism and sends x⋅rPKP to CCS.(2)
CCS decryptsx⋅rPKP. Let judge=1 if x⋅r≥0 else judge=0. Then, it sends judgePKPC to CSS.(3)
CSS decrypts and obtains the symboljudge.


#### 5.2.3. Privacy Preserving Outsourced Training with Multiclass SVM

In this section, we construct a privacy preserving outsourced training protocol to train a multiclass SVM model using the proposed building blocks. DSP outsources the training task to CSS and CSS completes the aggregation of outsourced data. Then, CSS and CCS complete the model training. After finishing the training, CSS transformsW1PKP,…WLPKP into W1PKPD,…WLPKPD. To achieve the transformation, we use the algorithm proposed in reference [38].For multiclass SVM training, there are two methods: one to rest (ovr) and one to one (ovo). In order to improve the efficiency and reduce the number of iterations, we choose the ovr method for training. We need to constructL binary SVM classifiers, each of which corresponds to one classification. The process is described in Algorithm 6.Algorithm 6: Secure multiclass SVM training.
Input: outsourced data of MCsX1PKP,y1PKP,…,XnPKP,ynPKP,iterationsT, learning rate learnrate,regularization parameterzOutput:L encrypted binary SVM classifiers parametersW1PKP,…WLPKP(1)
fork=1⟶L:for it=1⟶T:gradPKP=WkPKPfori=1⟶n:(2)
Wk⋅XiPKP=Block_3WkPKP,XiPKP(3)
tmp=Block_2yiPKP,Wk⋅XiPKP⋅1PKP−1(4)
if Block_5tmp==0:yi⋅XiPKP=Block_4yi,XigradPKP⋅=Wj⋅yi⋅XiPKPz⋅NP−1endfor(5)
WPKP⋅=gradPKPlearnrate⋅NP−1endforendfor(6)
returnW1PKP,…,WLPKP


### 5.3. Privacy Preserving Online-Aided Disease Diagnosis

In this section, our proposed scheme consists of four steps: diagnosis outsourcing, secret diagnosis request generation, diagnosis values computation, and diagnosis result generation. The privacy preserving online-aided disease diagnosis is completed by CSS and CCS.


#### 5.3.1. Diagnosis Outsourcing

To reduce the computation and communication overhead, DSP outsources the SVM model parameters to CSS and authorizes CSS to provide diagnosis service for users.The SVM parameters of DSP are expressed asW1,W2,…,WL (There are L classifiers),(5)Wi=w1i,w2i,…,wt+1i,i=1,2,…,L.ForWi and the corresponding class result classi, DSP generates a t+1-dimensional random integer vector Ri=R1i,R2i,…,Rt+1i and a random integer ri, lRji=lri<lNBC/2,lRji<lNB/2. Then, DSP computes Wi+Ri and classi+ri to hide the parameters class results.According to the combination of subtraction ofL random integer vectors, DSP constructs a combination table. The combination table has CL2 values, as shown in Table 2. The values in combination table are used to eliminate the blinding factors in subsequent computation.Table 2
Combination table.
1,2: index=1⋅L+2…L−1,L: index=L−1⋅L+LR1−R2…RL−1−RLDSP encryptsWi+Ri with PKB, classi+ri with PKBC, ri with PKP and all values of combination table with PKBC. Then, DSP sends them as the outsourced parameters to CSS. After receiving the outsourced parameters, CSS decrypts classi+riPKBC and the combination table with SKBC. CSS computes as follows:(6)classiPKP=classi+rPKPi⋅riPKP−1,i=1,2,…,L.


#### 5.3.2. Secret Diagnosis Request Generation

Foruseri, the symptom is expressed as Xi=x1i,x2i,…,xti,1 (The last 1 is added to facilitate the computation of vector inner product). The useri generates a t+1-dimensional random integer vector Ti=T1i,T2i,…,Tji,…,Tt+1i and lTji<lNBC/2,lTji<lNB/2. Then, useri hides plaintext symptoms Xi+Ti.Theuseri encrypts symptom Xi+Ti with PKB and encrypts Ti with PKBC. Let S as the secret prediction request of useri.(7)S=Xi+TiPKB,TiPKBC.Then, theuseri sends S to CSS.


#### 5.3.3. Diagnosis Value Computation

In our proposed diagnosis scheme, it is a multiclassification problem, so it is necessary to compute the diagnosis value of each classification. After receiving the secret prediction requestS, CSS decrypts TiPKBC with SKBC. Then, it computes Xi+TiPKB−Ti by the homomorphic operation of the BFV cryptosystem.According to the decision functionfX=W⋅X+b of the SVM algorithm, a diagnosis value needs to be computed by one multiplication homomorphic operation and one addition homomorphic operation. Because the BFV encryption algorithm supports ciphertext packaging, batch operation can be realized and the computation efficiency is significantly improved. The process is described in Algorithm 7.Algorithm 7: Diagnosis value computation.
Input:XiPKB, Wj+RjPKB, j=1,2,…,LOutput:L encrypted diagnosis valuesW1+R1XiPKB,…,WL+RLXiPKBCSS:(1)
forj=1⟶L:Wj+RjXiPKB=Wj+RjPKB⋅XiPKBendfor


#### 5.3.4. Diagnosis Result Generation

After computing the diagnosis values, CSS obtainsL encrypted diagnosis values and each value corresponds to a class result. Then, CSS needs to select the classification corresponding to the maximum value from the L encrypted values as the diagnosis result.Therefore, we design a secure maximum find protocol and a secure comparison algorithm. In this process, CSS and CCS jointly execute the protocol.(1). Secure maximum finding. CSS sets an initial maximum position pos=1. Then, CSS executes L cycles and each cycle executes a secure comparison algorithm to continuously update the pos value.AfterL cycles, CSS obtains the final diagnosis result classposPKP and converts classposPKP into classposPKuseri under the public key PKuseri of useri. To achieve the transformation, we use the algorithm proposed in literature [38]. Then, CSS sends classposPKuseri to useri. The useri decrypts the encrypted result with SKuseri. The process is described in Algorithm 8.Algorithm 8: Secure maximum finding.
Input:L diagnosis values and corresponding class resultsWj+RjXiPKB,classj,j=1,2,…,L;initialpos=1Output:classposPKuseriCSS:forj=2⟶L:(1)
judge=SCWpos+RposXiPKB,Wj+RjXiPKB(2)
if judge=1:pos=iendfor(3)
TransformclassposPKP into classposPKuseri with CCS(4)
SendclassposPKuseri to useri.useri:(5)
useri decrypts classposPKuseri with SKuseri.(2). Secure comparison (SC). For the i-th cycle, CSS computes Δpos−i=Wpos+RposXiPKB−Wj+RjXiPKB. Then, according to pos and j, computing index=pos⋅L+j. The index corresponds to the value Rpos−Rjindex in the combination table and computing as follows:(8)XiRpos−RjindexPKB=XiPKB⋅Rpos−Rjindex,Δpos−j′PKB=Δpos−jPKB−XiRpos−RjindexPKB.At this time,Δpos−j′PKB has eliminated Rpos−Rj⋅Xi in Δpos−jPKB.CSS choosest+1 equal random integers r′,R′=r′,…,r′ and lr′<lNB/2. Computing Δpos−j′PKB=Δpos−j′PKB⋅R′. Then, CSS chooses t+1 different random integers, R″=r1″,…,rt+1″,lr1″=⋯=lrt+1″<lNB/2 and computing Δpos−j′PKB=Δpos−j′PKB+R″. Summing all elements in R″ to get Rcss and encrypting it with PKP. CSS sends Δpos−j′PKB,RcssPKP to CCS.CCS decryptsΔpos−j′PKB with SKB and RcssPKP with SKP, w1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″. Then, summing each dimension, s=sumw1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″.CCS removesRcss from s by computing s−RcssmodNB. Let judge=1 if s>NB/2, else judge=0.CCS encryptsjudge with PKBC and sends it to CSS. CSS decrypts it and if judge=1, updates the value of pos.The process is described in Algorithm9.Algorithm 9: Secure comparison (SC).
Input:Wpos+RposXiPKB,Wj+RjXiPKBOutput:judgeCSS:(1)
Δpos−jPKB=Wpos+RposXiPKB−Wj+RjXiPKB(2)
index=pos⋅L+j(3)
XiRpos−RjindexPKB=XiPKB⋅Rpos−Rjindex(4)
Δpos−j′PKB=Δpos−jPKB−XiRpos−RjindexPKB(5)
GenerateR′.(6)
Δpos−j′PKB=Δpos−j′PKB⋅R′(7)
GenerateR″=r1″,r2″,…,rt+1″.(8)
Δpos−j′PKB=Δpos−j′PKB+R″(9)
Rcss=r1′+r2′+⋯+rt+1′(10)
SendΔpos−i′PKB,RcssPKP to CCS.CCS:(11)
DecryptΔpos−i′PKB,RcssPKP.(12)
s=sumw1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″(13)
s=s−RcssmodNB(14)
ifs>NB/2:judge=1else:judge=0(15)
Encryptjudge. Send it to CSSCSS:(16)
DecryptjudgePKBC


## 5.1. System Initialization

In the system initialization phase, TA generates system parameters and distributes the parameters for MCs, CSS, CCS, and DSP, respectively. TA sends the parameters through the secure communication channel. Then, TA will stay offline. We assume that there arem MCs in our system. Because the Paillier cryptosystem and BFV cryptosystem can only encrypt integers, the floating point numbers and negative numbers should be converted into integers. Therefore, all participants should make data conversion before encrypting their sensitive information.


### 5.1.1. Generate System Parameters

(1)
Generate a public-private key pairPKP=NP,g,SKP of the Paillier cryptosystem and a public-private key pair PKB,SKB of the BFV cryptosystem. The BFV plaintext space is NB. The public keys are public and the private keys are sent to the CCS.(2)
Generate a public-private key pairPKPC,SKPC of the Paillier cryptosystem and a public-private key pair PKBC,SKBC of the BFV cryptosystem for CSS. The BFV plaintext space is NBC. The public keys are public and the private keys are sent to CSS.(3)
Generate a public-private key pairPKPD,SKPD of the Paillier cryptosystem for DSP. The public key is public and the private key is sent to DSP.(4)
Generate a random integerω∈NP. TA randomly splits ω to m integers, satisfying ω1+ω2+⋯+ωm=ω and sends ωi to MCi. Then, generate two lists H and H′. Each list has m random integers, H=n1,n2,…,nm,ni∈NP,H′=n1′,n2′,…,nm′,ni′∈NP. Each element in H and H′ represents the ID of each MC. When MCi sends authentication idi to CSS, MCi will hide gai and ωi with ni and ni′, respectively. The ni,ni′ is sent to MCi. H and H′ are sent to CSS.


### 5.1.2. Data Conversion

In the machine learning application scenario, data and model parameters contain floating point numbers and negative numbers.For a floating point numberx, we enlarge x to x⋅2E (E is the precision of floating point numbers). For example, given a floating point number x=3.61 and the precision E=20, we can convert x into an integer x′=3785359. For a negative number y, we divide the plaintext space N (N is expressed the plaintext space of the Paillier or BFV cryptosystem) into two parts because all variables and intermediate results in the process of training and prediction are much smaller than N/2. An integer in 0,N/2 represents a positive integer and N/2,N−1 represents a negative integer. When encrypting the negative integer y, it is converted to encrypt N−y. If y is both a floating point number and a negative number, y is first converted into a negative integer.


## 5.1.1. Generate System Parameters

(1)
Generate a public-private key pairPKP=NP,g,SKP of the Paillier cryptosystem and a public-private key pair PKB,SKB of the BFV cryptosystem. The BFV plaintext space is NB. The public keys are public and the private keys are sent to the CCS.(2)
Generate a public-private key pairPKPC,SKPC of the Paillier cryptosystem and a public-private key pair PKBC,SKBC of the BFV cryptosystem for CSS. The BFV plaintext space is NBC. The public keys are public and the private keys are sent to CSS.(3)
Generate a public-private key pairPKPD,SKPD of the Paillier cryptosystem for DSP. The public key is public and the private key is sent to DSP.(4)
Generate a random integerω∈NP. TA randomly splits ω to m integers, satisfying ω1+ω2+⋯+ωm=ω and sends ωi to MCi. Then, generate two lists H and H′. Each list has m random integers, H=n1,n2,…,nm,ni∈NP,H′=n1′,n2′,…,nm′,ni′∈NP. Each element in H and H′ represents the ID of each MC. When MCi sends authentication idi to CSS, MCi will hide gai and ωi with ni and ni′, respectively. The ni,ni′ is sent to MCi. H and H′ are sent to CSS.


## 5.1.2. Data Conversion

In the machine learning application scenario, data and model parameters contain floating point numbers and negative numbers.For a floating point numberx, we enlarge x to x⋅2E (E is the precision of floating point numbers). For example, given a floating point number x=3.61 and the precision E=20, we can convert x into an integer x′=3785359. For a negative number y, we divide the plaintext space N (N is expressed the plaintext space of the Paillier or BFV cryptosystem) into two parts because all variables and intermediate results in the process of training and prediction are much smaller than N/2. An integer in 0,N/2 represents a positive integer and N/2,N−1 represents a negative integer. When encrypting the negative integer y, it is converted to encrypt N−y. If y is both a floating point number and a negative number, y is first converted into a negative integer.


## 5.2. Privacy Preserving Machine Learning Training

The privacy preserving machine learning training process is completed by CSS and CCS. We assume that the amount of outsourced data isn.


### 5.2.1. Local Data Outsourcing

To protect the privacy of MCs’ local data, MCs will encrypt the data before outsourcing. The outsourcing process ofMCii=1,…,m is as follows.(1)
MCi generates a random integer ai∈ZNP. Computing pki=gaimodNP2 as public key and the private key is ski=ai.(2)
Computinghi=gai+ωmodNP2.(3)
For each plaintext data, such asx, MCi will make a data conversion as mentioned in Section 5.1. Then, compute xMCi=gxrNP+hi to encrypt and outsource the encrypted data to CSS for storage.


### 5.2.2. Secure Basic Building Blocks for Training

To complete the privacy preserving outsourced training, we construct some algorithms as basic building blocks based on the Paillier cryptosystem: secure data aggregation (Block_1), secure multiplication algorithm (Block_2), secure inner product algorithm (Block_3), secure scalar multiplication of vector algorithm (Block_4), and secure symbol judgment algorithm (Block_5). The algorithms will be executed with CSS and CCS.(1) Secure data aggregation algorithm (Block_1). CSS needs to aggregate MCs’ outsourced data before starting machine learning training. The algorithm works as follows and is described in Algorithm 1.(1)
CSS sends a training request toMCi, i=1,2,…,m.(2)
After receiving the training request,MCi computes idi=pki+ni,ωi+ni′ as authentication (The idi indicates that CSS is allowed to use the outsourced data of MCi for training) and sends to CSS.(3)
CSS obtains thepki,ωi of MCi through the idi and computes ω=ω1+ω2+⋯+ωm. It should be noted that ω can be obtained only after all MCs have sent their authentication. Then, CSS computes hi=gai+ωmodNP2 and completes the aggregation.(2). Secure multiplication algorithm (Block_2). Given two encrypted integers xPKP and yPKP, the algorithm needs to compute x⋅yPKP. The algorithm works as follows and is described in Algorithm 2.(1)
CSS generates two random integersR1,R2 and R1,R2∈ZNP. Then, it computes by applying the additive homomorphism, obtaining the following results.(2)x+R1PKP=xPKP⋅gR1⋅,y+R2PKP=yPKP⋅gR2⋅,Then, sending them to CCS.(2)
CCS generates a random integerT,T∈ZNP. It decrypts y+R2PKP by using SKP. Then, it encrypts y+R2+TmodNPC with PKPC to get y+R2+TPKPC. Computing xT+R1TPKP=x+R1PKPT and encrypting T with PKP. Sending y+R2+TPKPC,xT+R1TPKP and TPKP to CSS.(3)
CSS decryptsy+R2+TPKPC with SKPC and computes y+T. Then, computing by applying the additive homomorphism, obtaining the following results.(3)xy+xTPKP=xPKPy+T,xy−R1TPKP=xy+xTPKP⋅xT+R1TPKP−1,R1TPKP=TPKPR1.Computing the result,(4)xyPKP=xy−R1TPKP⋅R1TPKP.(3). Secure inner product algorithm (Block_3). Given two encrypted vectors XPKP,YPKP. The algorithm will compute X⋅YPKP and is described in Algorithm 3.Algorithm 1: Secure data aggregation (Block_1).
Input: the authentication and outsourced data ofMCi.Output: the training data.CSS:(1)
Send a training request toMCi, i=1,2,…,m.MCs:(2)
fori=1⟶m:MCi sends idi=pki+ni,ωi+ni′ to CSSendfor CCS:(3)
CSS obtainspki,ωi, i=1,2,…,m(4)
Computeω=ω1+ω2+⋯+ωm(5)
fori=1⟶m:Computehi=gai+ωmodNP2For each outsourced data ofMCi, such as xMCi,ComputexPKP=xMCi−hi to complete aggregateendforAlgorithm 2: Secure multiplication (Block_2).
Input:xPKP,yPKPOutput:xyPKPCSS:(1)
R1,R2∈ZNP(2)
x+R1PKP=xPK1⋅gR1y+R2PKP=yPKP⋅gR2(3)
Sendx+R1PKP,y+R2PKP to CCS.CCS:(4)
T∈ZNP(5)
Decrypty+R2PKP(6)
xT+R1TPKP=x+R1PKPT(7)
Encrypty+R2+TmodNPC with PKPC(8)
EncryptT with PKP(9)
SendxT+R1TPKP,y+R2+TPKPC and TPKP to CSS.CSS:(10)
Decrypty+R2+TPKPC with SKPC and Compute y+T(11)
xy+xTPKP=xPKPy+T(12)
xy−R1TPKP=xy+xTPKP⋅xT+R1TPKP−1(13)
R1TPKP=TPKPR1(14)
xyPKP=xy−R1TPKP⋅R1TPKPAlgorithm 3: Secure inner product (Block_3).
Input:XPKP,YPKPOutput:X⋅YPKPCSS:(1)
DefineX⋅YPKP=1PKP.(2)
fori=1⟶X.length:X⋅YPKP⋅=Block_1xiPKP,yiPKPendfor(4). Secure scalar multiplication of vector algorithm (Block_4). Given a encrypted vector XPKP and a encrypted integer yPKP, the algorithm will compute y⋅XPKP and is described in Algorithm 4.Algorithm 4: Secure scalar multiplication of vector (Block_4).
Input:XPKP,yPKPOutput:y⋅XPKPCSS:(1)
Definey⋅XPKP=1,1,…,1PKP.(2)
fori=1⟶X.length:yxiPKP=Block_1yPKP,xiPKPendforAlgorithm 5: Secure symbol judgment (Block_5).
Input:xPKPOutput:judgeCSS:(1)
Choose a random integerr,lr<lNP/2(2)
x⋅rPKP=xPKPr(3)
Sendx⋅rPKP to CCSCCS:(4)
Decryptx⋅rPKP(5)
ifx⋅r≥0:judge=1,else:judge=0(6)
Encryptjudge with PKPC(7)
SendjudgePKPC to CSS(5). Secure symbol judgment algorithm (Block_5). Given an encrypted integer xPKP, the algorithm will compute the sign of xPKP. Let judge=1 if x≥0 else judge=0. The algorithm works as follows and is described in Algorithm 5.(1)
CSS chooses a random integerr,lr<lNP/2. Then, it computes x⋅rPKP=xPKPr by applying the additive homomorphism and sends x⋅rPKP to CCS.(2)
CCS decryptsx⋅rPKP. Let judge=1 if x⋅r≥0 else judge=0. Then, it sends judgePKPC to CSS.(3)
CSS decrypts and obtains the symboljudge.


### 5.2.3. Privacy Preserving Outsourced Training with Multiclass SVM

In this section, we construct a privacy preserving outsourced training protocol to train a multiclass SVM model using the proposed building blocks. DSP outsources the training task to CSS and CSS completes the aggregation of outsourced data. Then, CSS and CCS complete the model training. After finishing the training, CSS transformsW1PKP,…WLPKP into W1PKPD,…WLPKPD. To achieve the transformation, we use the algorithm proposed in reference [38].For multiclass SVM training, there are two methods: one to rest (ovr) and one to one (ovo). In order to improve the efficiency and reduce the number of iterations, we choose the ovr method for training. We need to constructL binary SVM classifiers, each of which corresponds to one classification. The process is described in Algorithm 6.Algorithm 6: Secure multiclass SVM training.
Input: outsourced data of MCsX1PKP,y1PKP,…,XnPKP,ynPKP,iterationsT, learning rate learnrate,regularization parameterzOutput:L encrypted binary SVM classifiers parametersW1PKP,…WLPKP(1)
fork=1⟶L:for it=1⟶T:gradPKP=WkPKPfori=1⟶n:(2)
Wk⋅XiPKP=Block_3WkPKP,XiPKP(3)
tmp=Block_2yiPKP,Wk⋅XiPKP⋅1PKP−1(4)
if Block_5tmp==0:yi⋅XiPKP=Block_4yi,XigradPKP⋅=Wj⋅yi⋅XiPKPz⋅NP−1endfor(5)
WPKP⋅=gradPKPlearnrate⋅NP−1endforendfor(6)
returnW1PKP,…,WLPKP


## 5.2.1. Local Data Outsourcing

To protect the privacy of MCs’ local data, MCs will encrypt the data before outsourcing. The outsourcing process ofMCii=1,…,m is as follows.(1)
MCi generates a random integer ai∈ZNP. Computing pki=gaimodNP2 as public key and the private key is ski=ai.(2)
Computinghi=gai+ωmodNP2.(3)
For each plaintext data, such asx, MCi will make a data conversion as mentioned in Section 5.1. Then, compute xMCi=gxrNP+hi to encrypt and outsource the encrypted data to CSS for storage.


## 5.2.2. Secure Basic Building Blocks for Training

To complete the privacy preserving outsourced training, we construct some algorithms as basic building blocks based on the Paillier cryptosystem: secure data aggregation (Block_1), secure multiplication algorithm (Block_2), secure inner product algorithm (Block_3), secure scalar multiplication of vector algorithm (Block_4), and secure symbol judgment algorithm (Block_5). The algorithms will be executed with CSS and CCS.(1) Secure data aggregation algorithm (Block_1). CSS needs to aggregate MCs’ outsourced data before starting machine learning training. The algorithm works as follows and is described in Algorithm 1.(1)
CSS sends a training request toMCi, i=1,2,…,m.(2)
After receiving the training request,MCi computes idi=pki+ni,ωi+ni′ as authentication (The idi indicates that CSS is allowed to use the outsourced data of MCi for training) and sends to CSS.(3)
CSS obtains thepki,ωi of MCi through the idi and computes ω=ω1+ω2+⋯+ωm. It should be noted that ω can be obtained only after all MCs have sent their authentication. Then, CSS computes hi=gai+ωmodNP2 and completes the aggregation.(2). Secure multiplication algorithm (Block_2). Given two encrypted integers xPKP and yPKP, the algorithm needs to compute x⋅yPKP. The algorithm works as follows and is described in Algorithm 2.(1)
CSS generates two random integersR1,R2 and R1,R2∈ZNP. Then, it computes by applying the additive homomorphism, obtaining the following results.(2)x+R1PKP=xPKP⋅gR1⋅,y+R2PKP=yPKP⋅gR2⋅,Then, sending them to CCS.(2)
CCS generates a random integerT,T∈ZNP. It decrypts y+R2PKP by using SKP. Then, it encrypts y+R2+TmodNPC with PKPC to get y+R2+TPKPC. Computing xT+R1TPKP=x+R1PKPT and encrypting T with PKP. Sending y+R2+TPKPC,xT+R1TPKP and TPKP to CSS.(3)
CSS decryptsy+R2+TPKPC with SKPC and computes y+T. Then, computing by applying the additive homomorphism, obtaining the following results.(3)xy+xTPKP=xPKPy+T,xy−R1TPKP=xy+xTPKP⋅xT+R1TPKP−1,R1TPKP=TPKPR1.Computing the result,(4)xyPKP=xy−R1TPKP⋅R1TPKP.(3). Secure inner product algorithm (Block_3). Given two encrypted vectors XPKP,YPKP. The algorithm will compute X⋅YPKP and is described in Algorithm 3.Algorithm 1: Secure data aggregation (Block_1).
Input: the authentication and outsourced data ofMCi.Output: the training data.CSS:(1)
Send a training request toMCi, i=1,2,…,m.MCs:(2)
fori=1⟶m:MCi sends idi=pki+ni,ωi+ni′ to CSSendfor CCS:(3)
CSS obtainspki,ωi, i=1,2,…,m(4)
Computeω=ω1+ω2+⋯+ωm(5)
fori=1⟶m:Computehi=gai+ωmodNP2For each outsourced data ofMCi, such as xMCi,ComputexPKP=xMCi−hi to complete aggregateendforAlgorithm 2: Secure multiplication (Block_2).
Input:xPKP,yPKPOutput:xyPKPCSS:(1)
R1,R2∈ZNP(2)
x+R1PKP=xPK1⋅gR1y+R2PKP=yPKP⋅gR2(3)
Sendx+R1PKP,y+R2PKP to CCS.CCS:(4)
T∈ZNP(5)
Decrypty+R2PKP(6)
xT+R1TPKP=x+R1PKPT(7)
Encrypty+R2+TmodNPC with PKPC(8)
EncryptT with PKP(9)
SendxT+R1TPKP,y+R2+TPKPC and TPKP to CSS.CSS:(10)
Decrypty+R2+TPKPC with SKPC and Compute y+T(11)
xy+xTPKP=xPKPy+T(12)
xy−R1TPKP=xy+xTPKP⋅xT+R1TPKP−1(13)
R1TPKP=TPKPR1(14)
xyPKP=xy−R1TPKP⋅R1TPKPAlgorithm 3: Secure inner product (Block_3).
Input:XPKP,YPKPOutput:X⋅YPKPCSS:(1)
DefineX⋅YPKP=1PKP.(2)
fori=1⟶X.length:X⋅YPKP⋅=Block_1xiPKP,yiPKPendfor(4). Secure scalar multiplication of vector algorithm (Block_4). Given a encrypted vector XPKP and a encrypted integer yPKP, the algorithm will compute y⋅XPKP and is described in Algorithm 4.Algorithm 4: Secure scalar multiplication of vector (Block_4).
Input:XPKP,yPKPOutput:y⋅XPKPCSS:(1)
Definey⋅XPKP=1,1,…,1PKP.(2)
fori=1⟶X.length:yxiPKP=Block_1yPKP,xiPKPendforAlgorithm 5: Secure symbol judgment (Block_5).
Input:xPKPOutput:judgeCSS:(1)
Choose a random integerr,lr<lNP/2(2)
x⋅rPKP=xPKPr(3)
Sendx⋅rPKP to CCSCCS:(4)
Decryptx⋅rPKP(5)
ifx⋅r≥0:judge=1,else:judge=0(6)
Encryptjudge with PKPC(7)
SendjudgePKPC to CSS(5). Secure symbol judgment algorithm (Block_5). Given an encrypted integer xPKP, the algorithm will compute the sign of xPKP. Let judge=1 if x≥0 else judge=0. The algorithm works as follows and is described in Algorithm 5.(1)
CSS chooses a random integerr,lr<lNP/2. Then, it computes x⋅rPKP=xPKPr by applying the additive homomorphism and sends x⋅rPKP to CCS.(2)
CCS decryptsx⋅rPKP. Let judge=1 if x⋅r≥0 else judge=0. Then, it sends judgePKPC to CSS.(3)
CSS decrypts and obtains the symboljudge.


## 5.2.3. Privacy Preserving Outsourced Training with Multiclass SVM

In this section, we construct a privacy preserving outsourced training protocol to train a multiclass SVM model using the proposed building blocks. DSP outsources the training task to CSS and CSS completes the aggregation of outsourced data. Then, CSS and CCS complete the model training. After finishing the training, CSS transformsW1PKP,…WLPKP into W1PKPD,…WLPKPD. To achieve the transformation, we use the algorithm proposed in reference [38].For multiclass SVM training, there are two methods: one to rest (ovr) and one to one (ovo). In order to improve the efficiency and reduce the number of iterations, we choose the ovr method for training. We need to constructL binary SVM classifiers, each of which corresponds to one classification. The process is described in Algorithm 6.Algorithm 6: Secure multiclass SVM training.
Input: outsourced data of MCsX1PKP,y1PKP,…,XnPKP,ynPKP,iterationsT, learning rate learnrate,regularization parameterzOutput:L encrypted binary SVM classifiers parametersW1PKP,…WLPKP(1)
fork=1⟶L:for it=1⟶T:gradPKP=WkPKPfori=1⟶n:(2)
Wk⋅XiPKP=Block_3WkPKP,XiPKP(3)
tmp=Block_2yiPKP,Wk⋅XiPKP⋅1PKP−1(4)
if Block_5tmp==0:yi⋅XiPKP=Block_4yi,XigradPKP⋅=Wj⋅yi⋅XiPKPz⋅NP−1endfor(5)
WPKP⋅=gradPKPlearnrate⋅NP−1endforendfor(6)
returnW1PKP,…,WLPKP


## 5.3. Privacy Preserving Online-Aided Disease Diagnosis

In this section, our proposed scheme consists of four steps: diagnosis outsourcing, secret diagnosis request generation, diagnosis values computation, and diagnosis result generation. The privacy preserving online-aided disease diagnosis is completed by CSS and CCS.


### 5.3.1. Diagnosis Outsourcing

To reduce the computation and communication overhead, DSP outsources the SVM model parameters to CSS and authorizes CSS to provide diagnosis service for users.The SVM parameters of DSP are expressed asW1,W2,…,WL (There are L classifiers),(5)Wi=w1i,w2i,…,wt+1i,i=1,2,…,L.ForWi and the corresponding class result classi, DSP generates a t+1-dimensional random integer vector Ri=R1i,R2i,…,Rt+1i and a random integer ri, lRji=lri<lNBC/2,lRji<lNB/2. Then, DSP computes Wi+Ri and classi+ri to hide the parameters class results.According to the combination of subtraction ofL random integer vectors, DSP constructs a combination table. The combination table has CL2 values, as shown in Table 2. The values in combination table are used to eliminate the blinding factors in subsequent computation.Table 2
Combination table.
1,2: index=1⋅L+2…L−1,L: index=L−1⋅L+LR1−R2…RL−1−RLDSP encryptsWi+Ri with PKB, classi+ri with PKBC, ri with PKP and all values of combination table with PKBC. Then, DSP sends them as the outsourced parameters to CSS. After receiving the outsourced parameters, CSS decrypts classi+riPKBC and the combination table with SKBC. CSS computes as follows:(6)classiPKP=classi+rPKPi⋅riPKP−1,i=1,2,…,L.


### 5.3.2. Secret Diagnosis Request Generation

Foruseri, the symptom is expressed as Xi=x1i,x2i,…,xti,1 (The last 1 is added to facilitate the computation of vector inner product). The useri generates a t+1-dimensional random integer vector Ti=T1i,T2i,…,Tji,…,Tt+1i and lTji<lNBC/2,lTji<lNB/2. Then, useri hides plaintext symptoms Xi+Ti.Theuseri encrypts symptom Xi+Ti with PKB and encrypts Ti with PKBC. Let S as the secret prediction request of useri.(7)S=Xi+TiPKB,TiPKBC.Then, theuseri sends S to CSS.


### 5.3.3. Diagnosis Value Computation

In our proposed diagnosis scheme, it is a multiclassification problem, so it is necessary to compute the diagnosis value of each classification. After receiving the secret prediction requestS, CSS decrypts TiPKBC with SKBC. Then, it computes Xi+TiPKB−Ti by the homomorphic operation of the BFV cryptosystem.According to the decision functionfX=W⋅X+b of the SVM algorithm, a diagnosis value needs to be computed by one multiplication homomorphic operation and one addition homomorphic operation. Because the BFV encryption algorithm supports ciphertext packaging, batch operation can be realized and the computation efficiency is significantly improved. The process is described in Algorithm 7.Algorithm 7: Diagnosis value computation.
Input:XiPKB, Wj+RjPKB, j=1,2,…,LOutput:L encrypted diagnosis valuesW1+R1XiPKB,…,WL+RLXiPKBCSS:(1)
forj=1⟶L:Wj+RjXiPKB=Wj+RjPKB⋅XiPKBendfor


### 5.3.4. Diagnosis Result Generation

After computing the diagnosis values, CSS obtainsL encrypted diagnosis values and each value corresponds to a class result. Then, CSS needs to select the classification corresponding to the maximum value from the L encrypted values as the diagnosis result.Therefore, we design a secure maximum find protocol and a secure comparison algorithm. In this process, CSS and CCS jointly execute the protocol.(1). Secure maximum finding. CSS sets an initial maximum position pos=1. Then, CSS executes L cycles and each cycle executes a secure comparison algorithm to continuously update the pos value.AfterL cycles, CSS obtains the final diagnosis result classposPKP and converts classposPKP into classposPKuseri under the public key PKuseri of useri. To achieve the transformation, we use the algorithm proposed in literature [38]. Then, CSS sends classposPKuseri to useri. The useri decrypts the encrypted result with SKuseri. The process is described in Algorithm 8.Algorithm 8: Secure maximum finding.
Input:L diagnosis values and corresponding class resultsWj+RjXiPKB,classj,j=1,2,…,L;initialpos=1Output:classposPKuseriCSS:forj=2⟶L:(1)
judge=SCWpos+RposXiPKB,Wj+RjXiPKB(2)
if judge=1:pos=iendfor(3)
TransformclassposPKP into classposPKuseri with CCS(4)
SendclassposPKuseri to useri.useri:(5)
useri decrypts classposPKuseri with SKuseri.(2). Secure comparison (SC). For the i-th cycle, CSS computes Δpos−i=Wpos+RposXiPKB−Wj+RjXiPKB. Then, according to pos and j, computing index=pos⋅L+j. The index corresponds to the value Rpos−Rjindex in the combination table and computing as follows:(8)XiRpos−RjindexPKB=XiPKB⋅Rpos−Rjindex,Δpos−j′PKB=Δpos−jPKB−XiRpos−RjindexPKB.At this time,Δpos−j′PKB has eliminated Rpos−Rj⋅Xi in Δpos−jPKB.CSS choosest+1 equal random integers r′,R′=r′,…,r′ and lr′<lNB/2. Computing Δpos−j′PKB=Δpos−j′PKB⋅R′. Then, CSS chooses t+1 different random integers, R″=r1″,…,rt+1″,lr1″=⋯=lrt+1″<lNB/2 and computing Δpos−j′PKB=Δpos−j′PKB+R″. Summing all elements in R″ to get Rcss and encrypting it with PKP. CSS sends Δpos−j′PKB,RcssPKP to CCS.CCS decryptsΔpos−j′PKB with SKB and RcssPKP with SKP, w1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″. Then, summing each dimension, s=sumw1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″.CCS removesRcss from s by computing s−RcssmodNB. Let judge=1 if s>NB/2, else judge=0.CCS encryptsjudge with PKBC and sends it to CSS. CSS decrypts it and if judge=1, updates the value of pos.The process is described in Algorithm9.Algorithm 9: Secure comparison (SC).
Input:Wpos+RposXiPKB,Wj+RjXiPKBOutput:judgeCSS:(1)
Δpos−jPKB=Wpos+RposXiPKB−Wj+RjXiPKB(2)
index=pos⋅L+j(3)
XiRpos−RjindexPKB=XiPKB⋅Rpos−Rjindex(4)
Δpos−j′PKB=Δpos−jPKB−XiRpos−RjindexPKB(5)
GenerateR′.(6)
Δpos−j′PKB=Δpos−j′PKB⋅R′(7)
GenerateR″=r1″,r2″,…,rt+1″.(8)
Δpos−j′PKB=Δpos−j′PKB+R″(9)
Rcss=r1′+r2′+⋯+rt+1′(10)
SendΔpos−i′PKB,RcssPKP to CCS.CCS:(11)
DecryptΔpos−i′PKB,RcssPKP.(12)
s=sumw1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″(13)
s=s−RcssmodNB(14)
ifs>NB/2:judge=1else:judge=0(15)
Encryptjudge. Send it to CSSCSS:(16)
DecryptjudgePKBC


## 5.3.1. Diagnosis Outsourcing

To reduce the computation and communication overhead, DSP outsources the SVM model parameters to CSS and authorizes CSS to provide diagnosis service for users.The SVM parameters of DSP are expressed asW1,W2,…,WL (There are L classifiers),(5)Wi=w1i,w2i,…,wt+1i,i=1,2,…,L.ForWi and the corresponding class result classi, DSP generates a t+1-dimensional random integer vector Ri=R1i,R2i,…,Rt+1i and a random integer ri, lRji=lri<lNBC/2,lRji<lNB/2. Then, DSP computes Wi+Ri and classi+ri to hide the parameters class results.According to the combination of subtraction ofL random integer vectors, DSP constructs a combination table. The combination table has CL2 values, as shown in Table 2. The values in combination table are used to eliminate the blinding factors in subsequent computation.Table 2
Combination table.
1,2: index=1⋅L+2…L−1,L: index=L−1⋅L+LR1−R2…RL−1−RLDSP encryptsWi+Ri with PKB, classi+ri with PKBC, ri with PKP and all values of combination table with PKBC. Then, DSP sends them as the outsourced parameters to CSS. After receiving the outsourced parameters, CSS decrypts classi+riPKBC and the combination table with SKBC. CSS computes as follows:(6)classiPKP=classi+rPKPi⋅riPKP−1,i=1,2,…,L.


## 5.3.2. Secret Diagnosis Request Generation

Foruseri, the symptom is expressed as Xi=x1i,x2i,…,xti,1 (The last 1 is added to facilitate the computation of vector inner product). The useri generates a t+1-dimensional random integer vector Ti=T1i,T2i,…,Tji,…,Tt+1i and lTji<lNBC/2,lTji<lNB/2. Then, useri hides plaintext symptoms Xi+Ti.Theuseri encrypts symptom Xi+Ti with PKB and encrypts Ti with PKBC. Let S as the secret prediction request of useri.(7)S=Xi+TiPKB,TiPKBC.Then, theuseri sends S to CSS.


## 5.3.3. Diagnosis Value Computation

In our proposed diagnosis scheme, it is a multiclassification problem, so it is necessary to compute the diagnosis value of each classification. After receiving the secret prediction requestS, CSS decrypts TiPKBC with SKBC. Then, it computes Xi+TiPKB−Ti by the homomorphic operation of the BFV cryptosystem.According to the decision functionfX=W⋅X+b of the SVM algorithm, a diagnosis value needs to be computed by one multiplication homomorphic operation and one addition homomorphic operation. Because the BFV encryption algorithm supports ciphertext packaging, batch operation can be realized and the computation efficiency is significantly improved. The process is described in Algorithm 7.Algorithm 7: Diagnosis value computation.
Input:XiPKB, Wj+RjPKB, j=1,2,…,LOutput:L encrypted diagnosis valuesW1+R1XiPKB,…,WL+RLXiPKBCSS:(1)
forj=1⟶L:Wj+RjXiPKB=Wj+RjPKB⋅XiPKBendfor


## 5.3.4. Diagnosis Result Generation

After computing the diagnosis values, CSS obtainsL encrypted diagnosis values and each value corresponds to a class result. Then, CSS needs to select the classification corresponding to the maximum value from the L encrypted values as the diagnosis result.Therefore, we design a secure maximum find protocol and a secure comparison algorithm. In this process, CSS and CCS jointly execute the protocol.(1). Secure maximum finding. CSS sets an initial maximum position pos=1. Then, CSS executes L cycles and each cycle executes a secure comparison algorithm to continuously update the pos value.AfterL cycles, CSS obtains the final diagnosis result classposPKP and converts classposPKP into classposPKuseri under the public key PKuseri of useri. To achieve the transformation, we use the algorithm proposed in literature [38]. Then, CSS sends classposPKuseri to useri. The useri decrypts the encrypted result with SKuseri. The process is described in Algorithm 8.Algorithm 8: Secure maximum finding.
Input:L diagnosis values and corresponding class resultsWj+RjXiPKB,classj,j=1,2,…,L;initialpos=1Output:classposPKuseriCSS:forj=2⟶L:(1)
judge=SCWpos+RposXiPKB,Wj+RjXiPKB(2)
if judge=1:pos=iendfor(3)
TransformclassposPKP into classposPKuseri with CCS(4)
SendclassposPKuseri to useri.useri:(5)
useri decrypts classposPKuseri with SKuseri.(2). Secure comparison (SC). For the i-th cycle, CSS computes Δpos−i=Wpos+RposXiPKB−Wj+RjXiPKB. Then, according to pos and j, computing index=pos⋅L+j. The index corresponds to the value Rpos−Rjindex in the combination table and computing as follows:(8)XiRpos−RjindexPKB=XiPKB⋅Rpos−Rjindex,Δpos−j′PKB=Δpos−jPKB−XiRpos−RjindexPKB.At this time,Δpos−j′PKB has eliminated Rpos−Rj⋅Xi in Δpos−jPKB.CSS choosest+1 equal random integers r′,R′=r′,…,r′ and lr′<lNB/2. Computing Δpos−j′PKB=Δpos−j′PKB⋅R′. Then, CSS chooses t+1 different random integers, R″=r1″,…,rt+1″,lr1″=⋯=lrt+1″<lNB/2 and computing Δpos−j′PKB=Δpos−j′PKB+R″. Summing all elements in R″ to get Rcss and encrypting it with PKP. CSS sends Δpos−j′PKB,RcssPKP to CCS.CCS decryptsΔpos−j′PKB with SKB and RcssPKP with SKP, w1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″. Then, summing each dimension, s=sumw1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″.CCS removesRcss from s by computing s−RcssmodNB. Let judge=1 if s>NB/2, else judge=0.CCS encryptsjudge with PKBC and sends it to CSS. CSS decrypts it and if judge=1, updates the value of pos.The process is described in Algorithm9.Algorithm 9: Secure comparison (SC).
Input:Wpos+RposXiPKB,Wj+RjXiPKBOutput:judgeCSS:(1)
Δpos−jPKB=Wpos+RposXiPKB−Wj+RjXiPKB(2)
index=pos⋅L+j(3)
XiRpos−RjindexPKB=XiPKB⋅Rpos−Rjindex(4)
Δpos−j′PKB=Δpos−jPKB−XiRpos−RjindexPKB(5)
GenerateR′.(6)
Δpos−j′PKB=Δpos−j′PKB⋅R′(7)
GenerateR″=r1″,r2″,…,rt+1″.(8)
Δpos−j′PKB=Δpos−j′PKB+R″(9)
Rcss=r1′+r2′+⋯+rt+1′(10)
SendΔpos−i′PKB,RcssPKP to CCS.CCS:(11)
DecryptΔpos−i′PKB,RcssPKP.(12)
s=sumw1posx1i−w1jx1ir′+r1″,…,wt+1posxt+1i−wt+1jxt+1ir′+rt+1″(13)
s=s−RcssmodNB(14)
ifs>NB/2:judge=1else:judge=0(15)
Encryptjudge. Send it to CSSCSS:(16)
DecryptjudgePKBC


## 6. Security Analysis

In this section, we analyze the security of the proposed scheme. The focus is on the outsourced data of MCs, the SVM model parameters of DSP, the symptoms, and diagnosis results of users.


### 6.1. Security Analysis of Training

In the training phase, the outsourced data of MCs and the SVM model parameters of DSP need privacy preserving. The training protocol is composed of building blocks designed in Section5.2.2, which are completed by CSS and CCS. According to the threat models proposed in Section 4.3, we analyze the security of the training protocol.


#### 6.1.1. Eavesdropping Attack

The data transmission process in the training phase includes that MCs outsources the encrypted data to CSS and the interactions of training protocol between CSS and CCS.In the outsourcing process, the data ofMCi have been encrypted. MCi combines the system public key PKP, parameter ω, and its own public key gai to ensure that the data are hidden while encrypting. Suppose an adversary obtains the private key SKP and eavesdrops when MCs outsource their data to CSS. Because the data of MCi have been encrypted, such as xMCi=gxrNP+hi, the adversary cannot obtain any useful information. Similarly, the authentications are also hidden by random numbers. In the training protocol execution process, CSS and CCS will interact and the transformed data have been encrypted and hidden the real values with random numbers. The adversary also cannot obtain any useful information.


#### 6.1.2. Honest-But-Curious Attack

During the training phase, CSS and CCS will get some intermediate results from the proposed building blocks in Section5.2.2.In the Block_2, CSS hidesx,y with R1,R2 by homomorphic operation before sending them to CCS. Then, CCS sends xT+R1TPKP,y+R2+TPKPC and TPKP to CSS after computing. Therefore, both CSS and CCS cannot learn any useful information about x,y. Because the Block_3 and Block_4 are designed based on the Block_2, we will not analyze them. In the Block_5, CSS hides x with r and sending x⋅rPKP to CCS. CCS can only know the symbol of x, but cannot obtains the real value of x. CCS only returns the result judge (0 or 1) to CSS. Through the abovementioned analysis, CSS and CCS cannot learn any useful information in the training process.


#### 6.1.3. Client-Collusion Attack

For MCs, eachMCi only know its own ωi. Therefore, if m−1 MCs collude with each other to steal the privacy of another MC, they cannot learn any useful information.


### 6.2. Security Analysis of Disease Diagnosis

In the diagnosis phase, the SVM parameters of DSP, the symptomXi and the diagnosis result classpos of useri need privacy preserving. The diagnosis process consists of diagnosis outsourcing, secret diagnosis request generation, diagnosis value computation, and diagnosis result generation. Therefore, we conduct security analysis on the main steps by the threat model.


#### 6.2.1. Eavesdropping Attack

The data transmission process includes that DSP outsourcesWi+RiPKB,classi+riPKBC,riPKP,i=1,2,…,L and R1−R2PKBC,…,RL−1−RLPKBC to CSS, useri sends request S to CSS and the interaction of diagnosis process between CSS and CCS.Through the encrypted data of outsourcing process, it can be seen that the adversary(CCS) can only decryptWi+RiPKB and riPKP with SKB and SKP. However, the adversary cannot learn Wi because of the Ri and the ri do not contain any useful information. When useri sends S to CSS, the symptom Xi may be eavesdropped and decrypted by the adversary, but Xi is hidden by random numbers. In the interaction of SC algorithm between CSS and CCS, all transmitted data are hidden by random numbers and ciphertext state, so the adversary cannot learn any useful information.


#### 6.2.2. Honest-But-Curious Attack

In the diagnosis value computation process, CSS can only obtain theL encrypted diagnosis values under PKB and does not know the corresponding classification meaning. The whole process is executed in the ciphertext state, so CSS cannot learn any useful information. The process of diagnosis result generation consists of secure maximum finding protocol and secure comparison algorithm. When CSS and CCS execute the secure comparison algorithm, CSS computes the difference between the two encrypted vectors to be compared. The obtained difference vector can confuse the positive and negative of the two numbers on each dimension of the original two vectors. At the same time, random integers are used to hide the difference vector. After decrypting the difference vector, CCS can eliminate the random number only after summing. During this process, CSS and CCS cannot obtain any useful information.After CSS and CCS execute secure maximum finding protocol, CSS obtains the diagnosis resultclassposPKP. When performing key conversion on classposPKP, CSP hides classpos with a random integer R. Then, sending classpos+RPKP to CCS. CCS can decrypt it. However, because there is a random integer hidden, CCS cannot obtain classpos.


#### 6.2.3. Client-Collusion Attack

For all users, they can only get the diagnosis results and cannot get any other information. Therefore, our proposed scheme can resist the client-collusion attack.


## 6.1. Security Analysis of Training

In the training phase, the outsourced data of MCs and the SVM model parameters of DSP need privacy preserving. The training protocol is composed of building blocks designed in Section5.2.2, which are completed by CSS and CCS. According to the threat models proposed in Section 4.3, we analyze the security of the training protocol.


### 6.1.1. Eavesdropping Attack

The data transmission process in the training phase includes that MCs outsources the encrypted data to CSS and the interactions of training protocol between CSS and CCS.In the outsourcing process, the data ofMCi have been encrypted. MCi combines the system public key PKP, parameter ω, and its own public key gai to ensure that the data are hidden while encrypting. Suppose an adversary obtains the private key SKP and eavesdrops when MCs outsource their data to CSS. Because the data of MCi have been encrypted, such as xMCi=gxrNP+hi, the adversary cannot obtain any useful information. Similarly, the authentications are also hidden by random numbers. In the training protocol execution process, CSS and CCS will interact and the transformed data have been encrypted and hidden the real values with random numbers. The adversary also cannot obtain any useful information.


### 6.1.2. Honest-But-Curious Attack

During the training phase, CSS and CCS will get some intermediate results from the proposed building blocks in Section5.2.2.In the Block_2, CSS hidesx,y with R1,R2 by homomorphic operation before sending them to CCS. Then, CCS sends xT+R1TPKP,y+R2+TPKPC and TPKP to CSS after computing. Therefore, both CSS and CCS cannot learn any useful information about x,y. Because the Block_3 and Block_4 are designed based on the Block_2, we will not analyze them. In the Block_5, CSS hides x with r and sending x⋅rPKP to CCS. CCS can only know the symbol of x, but cannot obtains the real value of x. CCS only returns the result judge (0 or 1) to CSS. Through the abovementioned analysis, CSS and CCS cannot learn any useful information in the training process.


### 6.1.3. Client-Collusion Attack

For MCs, eachMCi only know its own ωi. Therefore, if m−1 MCs collude with each other to steal the privacy of another MC, they cannot learn any useful information.


## 6.1.1. Eavesdropping Attack

The data transmission process in the training phase includes that MCs outsources the encrypted data to CSS and the interactions of training protocol between CSS and CCS.In the outsourcing process, the data ofMCi have been encrypted. MCi combines the system public key PKP, parameter ω, and its own public key gai to ensure that the data are hidden while encrypting. Suppose an adversary obtains the private key SKP and eavesdrops when MCs outsource their data to CSS. Because the data of MCi have been encrypted, such as xMCi=gxrNP+hi, the adversary cannot obtain any useful information. Similarly, the authentications are also hidden by random numbers. In the training protocol execution process, CSS and CCS will interact and the transformed data have been encrypted and hidden the real values with random numbers. The adversary also cannot obtain any useful information.


## 6.1.2. Honest-But-Curious Attack

During the training phase, CSS and CCS will get some intermediate results from the proposed building blocks in Section5.2.2.In the Block_2, CSS hidesx,y with R1,R2 by homomorphic operation before sending them to CCS. Then, CCS sends xT+R1TPKP,y+R2+TPKPC and TPKP to CSS after computing. Therefore, both CSS and CCS cannot learn any useful information about x,y. Because the Block_3 and Block_4 are designed based on the Block_2, we will not analyze them. In the Block_5, CSS hides x with r and sending x⋅rPKP to CCS. CCS can only know the symbol of x, but cannot obtains the real value of x. CCS only returns the result judge (0 or 1) to CSS. Through the abovementioned analysis, CSS and CCS cannot learn any useful information in the training process.


## 6.1.3. Client-Collusion Attack

For MCs, eachMCi only know its own ωi. Therefore, if m−1 MCs collude with each other to steal the privacy of another MC, they cannot learn any useful information.


## 6.2. Security Analysis of Disease Diagnosis

In the diagnosis phase, the SVM parameters of DSP, the symptomXi and the diagnosis result classpos of useri need privacy preserving. The diagnosis process consists of diagnosis outsourcing, secret diagnosis request generation, diagnosis value computation, and diagnosis result generation. Therefore, we conduct security analysis on the main steps by the threat model.


### 6.2.1. Eavesdropping Attack

The data transmission process includes that DSP outsourcesWi+RiPKB,classi+riPKBC,riPKP,i=1,2,…,L and R1−R2PKBC,…,RL−1−RLPKBC to CSS, useri sends request S to CSS and the interaction of diagnosis process between CSS and CCS.Through the encrypted data of outsourcing process, it can be seen that the adversary(CCS) can only decryptWi+RiPKB and riPKP with SKB and SKP. However, the adversary cannot learn Wi because of the Ri and the ri do not contain any useful information. When useri sends S to CSS, the symptom Xi may be eavesdropped and decrypted by the adversary, but Xi is hidden by random numbers. In the interaction of SC algorithm between CSS and CCS, all transmitted data are hidden by random numbers and ciphertext state, so the adversary cannot learn any useful information.


### 6.2.2. Honest-But-Curious Attack

In the diagnosis value computation process, CSS can only obtain theL encrypted diagnosis values under PKB and does not know the corresponding classification meaning. The whole process is executed in the ciphertext state, so CSS cannot learn any useful information. The process of diagnosis result generation consists of secure maximum finding protocol and secure comparison algorithm. When CSS and CCS execute the secure comparison algorithm, CSS computes the difference between the two encrypted vectors to be compared. The obtained difference vector can confuse the positive and negative of the two numbers on each dimension of the original two vectors. At the same time, random integers are used to hide the difference vector. After decrypting the difference vector, CCS can eliminate the random number only after summing. During this process, CSS and CCS cannot obtain any useful information.After CSS and CCS execute secure maximum finding protocol, CSS obtains the diagnosis resultclassposPKP. When performing key conversion on classposPKP, CSP hides classpos with a random integer R. Then, sending classpos+RPKP to CCS. CCS can decrypt it. However, because there is a random integer hidden, CCS cannot obtain classpos.


### 6.2.3. Client-Collusion Attack

For all users, they can only get the diagnosis results and cannot get any other information. Therefore, our proposed scheme can resist the client-collusion attack.


## 6.2.1. Eavesdropping Attack

The data transmission process includes that DSP outsourcesWi+RiPKB,classi+riPKBC,riPKP,i=1,2,…,L and R1−R2PKBC,…,RL−1−RLPKBC to CSS, useri sends request S to CSS and the interaction of diagnosis process between CSS and CCS.Through the encrypted data of outsourcing process, it can be seen that the adversary(CCS) can only decryptWi+RiPKB and riPKP with SKB and SKP. However, the adversary cannot learn Wi because of the Ri and the ri do not contain any useful information. When useri sends S to CSS, the symptom Xi may be eavesdropped and decrypted by the adversary, but Xi is hidden by random numbers. In the interaction of SC algorithm between CSS and CCS, all transmitted data are hidden by random numbers and ciphertext state, so the adversary cannot learn any useful information.


## 6.2.2. Honest-But-Curious Attack

In the diagnosis value computation process, CSS can only obtain theL encrypted diagnosis values under PKB and does not know the corresponding classification meaning. The whole process is executed in the ciphertext state, so CSS cannot learn any useful information. The process of diagnosis result generation consists of secure maximum finding protocol and secure comparison algorithm. When CSS and CCS execute the secure comparison algorithm, CSS computes the difference between the two encrypted vectors to be compared. The obtained difference vector can confuse the positive and negative of the two numbers on each dimension of the original two vectors. At the same time, random integers are used to hide the difference vector. After decrypting the difference vector, CCS can eliminate the random number only after summing. During this process, CSS and CCS cannot obtain any useful information.After CSS and CCS execute secure maximum finding protocol, CSS obtains the diagnosis resultclassposPKP. When performing key conversion on classposPKP, CSP hides classpos with a random integer R. Then, sending classpos+RPKP to CCS. CCS can decrypt it. However, because there is a random integer hidden, CCS cannot obtain classpos.


## 6.2.3. Client-Collusion Attack

For all users, they can only get the diagnosis results and cannot get any other information. Therefore, our proposed scheme can resist the client-collusion attack.


## 7. Performance Evaluation

In this section, we implemented our scheme and evaluated the performance of training and diagnosis.Our experimental environment is shown in Table3.Table 3
Experimental environment.
Operating systemWindows 10CPUIntel (R) Core(TM)i7-10510U, 1.80 GHz, 2.30 GHzMemory8 GProgram languageC++In our experiments, we evaluated our proposed scheme with a real dataset from UCI machine learning library called dermatology. The dermatology dataset is a multiclassification dataset with 6 categories and 34 symptoms.


### 7.1. Privacy Preserving Machine Learning Training Evaluation

#### 7.1.1. Effect of Key Length on Computation Overhead

The key length in cryptosystem has a great impact on efficiency and security. Therefore, we tested the data encryption time and main building blocks time (Block_1 and Block_3), which have high computation overhead. The test results are shown in Table4.Table 4
Computation overhead under different key length.
Key length (bit)Data encryption (s)Block_1 (s)Block_3 (s)l=2564.15e10−42.08e10−50.094l=5122.52e10−35.0e10−50.417l=10240.01532.1e10−42.52l=20480.1039.9e10−417.2From Table4, it can be seen that the increase of key length has a great impact on the computation overhead. Based on the experimental results and security considerations, the key length of the Paillier cryptosystem is set to 1024 bit in the training phase.


#### 7.1.2. Privacy Preserving Multiclass SVM Training Analysis

In order to meet the requirements of data encryption, we convert all floating-point numbers to integers. The conversion accuracyE of floating-point numbers has a great impact on the accuracy of the SVM model. We tested the accuracy of the SVM model under different E values; the results are shown in Figure 3.Figure 3
The influence of precision.Through the abovementioned experimental analysis, it can be seen that the larger theE, the higher the accuracy of the model. With the increase of E, the accuracy of the model tends to be stable. When E=20, the accuracy of the model is the highest. At the same time, we also used the gradient descent method to train the SVM model in the plaintext state. We compared the accuracy with the model trained in ciphertext state and the results are shown in Table 5.Table 5
Comparison analysis of model accuracy.
DatasetPlaintext stateOur proposed schemeDermatology98.61%98.61%Through the abovementioned experimental analysis, it can be seen that the accuracy of our proposed scheme is the same as the plaintext state (98.61%). Therefore, it is verified that our proposed scheme is correct and available.


### 7.2. Privacy Preserving Online-Aided Disease Diagnosis Evaluation

We implemented our proposed scheme by using SEAL library in the diagnosis phase.


#### 7.2.1. Noise Effect of BFV Cryptosystem

When using the BFV cryptosystem for homomorphic operation, the influence of noise needs to be considered. The noise of ciphertext will be increased when the multiplication homomorphic operation is carried out. If the noise is too large after computation, the correct result cannot be obtained after decryption.Therefore, the BFV cryptosystem in SEAL will set thenoise budget during initialization. If the noise budget is greater than 0 after the computation, it can be decrypted correctly. The value of noise budget is related to the setting of parameters. We evaluated the influence of poly module degree d on the encryption time, the change of noise budget after homomorphic operation, the computation time and whether the decryption result is correct. The results are shown in Table 6. It can be seen that the noise consumption of the BFV cryptosystem is relatively large when performing multiplication homomorphism, so the BFV cryptosystem can only perform multiplication homomorphism for a limited number of times. When computing the diagnosis values, only one inner product operation and one addition operation are required. Therefore, it is completely feasible to use the BFV cryptosystem.Table 6
The influence of poly modulus degreed on noise budget.
dEncryption time (s)Initial noise budget (bit)Noise budget (bit) after operationComputation time (s)Decryption result (correct or wrong)20480.013200.012✕40960.023900.031✕81920.69110640.123✓327681.1787617132.268✓We comprehensively consider the encryption time and computation time and ensure that the computation results can be decrypted correctly. The parameter we set ispoly module degreed=8192.


#### 7.2.2. Influence of Different Classification Numbers on Computation Overhead

When using the BFV cryptosystem to encrypt data, multiple plaintext data can be packaged and encrypted into a ciphertext. The number of classifications isL.We tested the impact of differentL on useri and DSP. The results are shown in Figure 4(a). With the increase of L, the encryption time of DSP is gradually increasing, and the encryption time of useri can be considered as unchanged. We also tested the impact of different L on the diagnosis values computation of CSS. The results are shown in Figure 4(b). With the continuous increase of L, the computation time for CSS is also increasing. The process of generating diagnosis result is jointly completed by CSS and CCS. We tested the effect of different L on the diagnosis result generation. The results are shown in Figure 4(c). With the continuous increase of L, the time for CSS and CCS is also increasing.Figure 4
Influence of different classification numbers on computation overhead. (a) Data encryption, (b) diagnosis values computation, and (c) diagnosis result generation.
(a)(b)(c)


#### 7.2.3. Comparison Analysis of Secret Diagnosis Request Generation and Diagnosis Values Computation

In our proposed scheme, secret diagnosis request generation can be regarded as data encryption ofuseri and diagnosis value computation can be regarded as homomorphic operation. We compared with the other three privacy preserving schemes. The results are shown in Table 7.Table 7
Comparison analysis.
SchemesData encryption (s)Diagnosis values computation (s)Total time (s)Reference [38], l=5120.6580.0050.663Reference [39], l=5120.570.5521.122Reference [40], l=5120.0041.0921.096Ours0.0080.0960.104Through the comparison analysis, it can be seen that the time of data encryption in our proposed scheme is significantly reduced compared with [38, 39]. In the computation of decision function, our scheme has significantly reduced the computational cost compared with the scheme in [39, 40]. At the same time, it can be seen from the total time that our proposed scheme is significantly lower than the other three schemes.Next, we make further analysis. The names of participants may be slightly different in different schemes. In order to facilitate analysis, we divided participants into cloud server and client. We compared the computation overhead of cloud server and client, respectively. The results are shown in Tables8 and 9.Table 8
Comparison analysis of the cloud server.
SchemesData encryption (s)Diagnosis value computation (s)Total time (s)Reference [38], l=512000Reference [39], l=5120.570.0010.571Reference [40], l=51201.0921.092Ours00.0960.096Table 9
Comparison analysis of the client.
SchemesData encryption (s)Diagnosis value computation (s)Total time (s)Reference [38], l=5120.6580.0050.663Reference [39], l=51200.5510.551Reference [40], l=5120.00400.004Ours0.00800.008In our proposed scheme, the client only needs to encrypt the data and can be offline after uploading the data to the cloud server. The cloud server only needs to compute the decision function. This model reduces the computation overhead of the client to the greatest extent and performs privacy preserving computation through the powerful computing power of the cloud server. In scheme [38], the cloud server does not participate in the whole process, so it brings heavy computation overhead to the client. In scheme [39], the computation of the diagnosis values needs to be completed by the cloud server and the client. Therefore, it not only brings heavy computation overhead to the client but also requires the client to always stay online in this process.


#### 7.2.4. Comparison Analysis of Diagnosis Result Generation

In our proposed scheme, after CSS completes the diagnosis values computation, it will jointly execute the secure protocol with CCS to generate the diagnosis result. We continued to make comparison analysis with schemes in [38–40]. The results are shown in Table 10.Table 10
Comparison analysis of diagnosis result generation.
SchemesCloud server computation (s)Client computation (s)Diagnosis result generation (s)Reference [38], l=5121.5840.0971.681Reference [39], l=5120.0070.0410.048Reference [40], l=5120.10100.101Ours0.07100.071Through the comparison analysis in Table10, it can be seen that the computation time of our proposed scheme is significantly lower than the scheme in references [38, 40]. In our proposed scheme, the client does not need to participate in the process of diagnosis result generation. The schemes in references [38, 39] require the participation of the client, which brings heavy computation overhead to the client.


#### 7.2.5. Comprehensive Comparison Analysis

We made a comparison analysis of the whole privacy preserving online disease diagnosis process. It is divided into the secret diagnosis request generation (data encryption), diagnosis value computation, and diagnosis result generation. The results are shown in Table11.Table 11
Comprehensive comparison analysis.
Data encryption (s)Diagnosis value computation (s)Diagnosis result generation (s)Total time (s)Reference [38], l=5120.6580.0051.6812.389Reference [39], l=5120.570.5520.0481.17Reference [40], l=5120.0041.0920.1011.197Ours0.0080.0960.0710.175Through the comparison analysis in Table11, the total time of our proposed scheme is significantly lower than the schemes in references [38–40]. Considering that in the actual application scenario, a large number of users will constantly initiate secret diagnosis requests. It is very important to be able to quickly respond to the diagnosis results for users. Therefore, our scheme has more practical application value. Then, we made a summary as shown in Table 12.Table 12
Scheme summary.
SchemesMultidata ownersTrainingMulticlassificationLow client overheadReference [38]✕✕✓✕Reference [39]✕✕✓✕Reference [40]✕✕✓✓Reference [37]✓✓✕✓Ours✓✓✓✓


## 7.1. Privacy Preserving Machine Learning Training Evaluation

### 7.1.1. Effect of Key Length on Computation Overhead

The key length in cryptosystem has a great impact on efficiency and security. Therefore, we tested the data encryption time and main building blocks time (Block_1 and Block_3), which have high computation overhead. The test results are shown in Table4.Table 4
Computation overhead under different key length.
Key length (bit)Data encryption (s)Block_1 (s)Block_3 (s)l=2564.15e10−42.08e10−50.094l=5122.52e10−35.0e10−50.417l=10240.01532.1e10−42.52l=20480.1039.9e10−417.2From Table4, it can be seen that the increase of key length has a great impact on the computation overhead. Based on the experimental results and security considerations, the key length of the Paillier cryptosystem is set to 1024 bit in the training phase.


### 7.1.2. Privacy Preserving Multiclass SVM Training Analysis

In order to meet the requirements of data encryption, we convert all floating-point numbers to integers. The conversion accuracyE of floating-point numbers has a great impact on the accuracy of the SVM model. We tested the accuracy of the SVM model under different E values; the results are shown in Figure 3.Figure 3
The influence of precision.Through the abovementioned experimental analysis, it can be seen that the larger theE, the higher the accuracy of the model. With the increase of E, the accuracy of the model tends to be stable. When E=20, the accuracy of the model is the highest. At the same time, we also used the gradient descent method to train the SVM model in the plaintext state. We compared the accuracy with the model trained in ciphertext state and the results are shown in Table 5.Table 5
Comparison analysis of model accuracy.
DatasetPlaintext stateOur proposed schemeDermatology98.61%98.61%Through the abovementioned experimental analysis, it can be seen that the accuracy of our proposed scheme is the same as the plaintext state (98.61%). Therefore, it is verified that our proposed scheme is correct and available.


## 7.1.1. Effect of Key Length on Computation Overhead

The key length in cryptosystem has a great impact on efficiency and security. Therefore, we tested the data encryption time and main building blocks time (Block_1 and Block_3), which have high computation overhead. The test results are shown in Table4.Table 4
Computation overhead under different key length.
Key length (bit)Data encryption (s)Block_1 (s)Block_3 (s)l=2564.15e10−42.08e10−50.094l=5122.52e10−35.0e10−50.417l=10240.01532.1e10−42.52l=20480.1039.9e10−417.2From Table4, it can be seen that the increase of key length has a great impact on the computation overhead. Based on the experimental results and security considerations, the key length of the Paillier cryptosystem is set to 1024 bit in the training phase.


## 7.1.2. Privacy Preserving Multiclass SVM Training Analysis

In order to meet the requirements of data encryption, we convert all floating-point numbers to integers. The conversion accuracyE of floating-point numbers has a great impact on the accuracy of the SVM model. We tested the accuracy of the SVM model under different E values; the results are shown in Figure 3.Figure 3
The influence of precision.Through the abovementioned experimental analysis, it can be seen that the larger theE, the higher the accuracy of the model. With the increase of E, the accuracy of the model tends to be stable. When E=20, the accuracy of the model is the highest. At the same time, we also used the gradient descent method to train the SVM model in the plaintext state. We compared the accuracy with the model trained in ciphertext state and the results are shown in Table 5.Table 5
Comparison analysis of model accuracy.
DatasetPlaintext stateOur proposed schemeDermatology98.61%98.61%Through the abovementioned experimental analysis, it can be seen that the accuracy of our proposed scheme is the same as the plaintext state (98.61%). Therefore, it is verified that our proposed scheme is correct and available.


## 7.2. Privacy Preserving Online-Aided Disease Diagnosis Evaluation

We implemented our proposed scheme by using SEAL library in the diagnosis phase.


### 7.2.1. Noise Effect of BFV Cryptosystem

When using the BFV cryptosystem for homomorphic operation, the influence of noise needs to be considered. The noise of ciphertext will be increased when the multiplication homomorphic operation is carried out. If the noise is too large after computation, the correct result cannot be obtained after decryption.Therefore, the BFV cryptosystem in SEAL will set thenoise budget during initialization. If the noise budget is greater than 0 after the computation, it can be decrypted correctly. The value of noise budget is related to the setting of parameters. We evaluated the influence of poly module degree d on the encryption time, the change of noise budget after homomorphic operation, the computation time and whether the decryption result is correct. The results are shown in Table 6. It can be seen that the noise consumption of the BFV cryptosystem is relatively large when performing multiplication homomorphism, so the BFV cryptosystem can only perform multiplication homomorphism for a limited number of times. When computing the diagnosis values, only one inner product operation and one addition operation are required. Therefore, it is completely feasible to use the BFV cryptosystem.Table 6
The influence of poly modulus degreed on noise budget.
dEncryption time (s)Initial noise budget (bit)Noise budget (bit) after operationComputation time (s)Decryption result (correct or wrong)20480.013200.012✕40960.023900.031✕81920.69110640.123✓327681.1787617132.268✓We comprehensively consider the encryption time and computation time and ensure that the computation results can be decrypted correctly. The parameter we set ispoly module degreed=8192.


### 7.2.2. Influence of Different Classification Numbers on Computation Overhead

When using the BFV cryptosystem to encrypt data, multiple plaintext data can be packaged and encrypted into a ciphertext. The number of classifications isL.We tested the impact of differentL on useri and DSP. The results are shown in Figure 4(a). With the increase of L, the encryption time of DSP is gradually increasing, and the encryption time of useri can be considered as unchanged. We also tested the impact of different L on the diagnosis values computation of CSS. The results are shown in Figure 4(b). With the continuous increase of L, the computation time for CSS is also increasing. The process of generating diagnosis result is jointly completed by CSS and CCS. We tested the effect of different L on the diagnosis result generation. The results are shown in Figure 4(c). With the continuous increase of L, the time for CSS and CCS is also increasing.Figure 4
Influence of different classification numbers on computation overhead. (a) Data encryption, (b) diagnosis values computation, and (c) diagnosis result generation.
(a)(b)(c)


### 7.2.3. Comparison Analysis of Secret Diagnosis Request Generation and Diagnosis Values Computation

In our proposed scheme, secret diagnosis request generation can be regarded as data encryption ofuseri and diagnosis value computation can be regarded as homomorphic operation. We compared with the other three privacy preserving schemes. The results are shown in Table 7.Table 7
Comparison analysis.
SchemesData encryption (s)Diagnosis values computation (s)Total time (s)Reference [38], l=5120.6580.0050.663Reference [39], l=5120.570.5521.122Reference [40], l=5120.0041.0921.096Ours0.0080.0960.104Through the comparison analysis, it can be seen that the time of data encryption in our proposed scheme is significantly reduced compared with [38, 39]. In the computation of decision function, our scheme has significantly reduced the computational cost compared with the scheme in [39, 40]. At the same time, it can be seen from the total time that our proposed scheme is significantly lower than the other three schemes.Next, we make further analysis. The names of participants may be slightly different in different schemes. In order to facilitate analysis, we divided participants into cloud server and client. We compared the computation overhead of cloud server and client, respectively. The results are shown in Tables8 and 9.Table 8
Comparison analysis of the cloud server.
SchemesData encryption (s)Diagnosis value computation (s)Total time (s)Reference [38], l=512000Reference [39], l=5120.570.0010.571Reference [40], l=51201.0921.092Ours00.0960.096Table 9
Comparison analysis of the client.
SchemesData encryption (s)Diagnosis value computation (s)Total time (s)Reference [38], l=5120.6580.0050.663Reference [39], l=51200.5510.551Reference [40], l=5120.00400.004Ours0.00800.008In our proposed scheme, the client only needs to encrypt the data and can be offline after uploading the data to the cloud server. The cloud server only needs to compute the decision function. This model reduces the computation overhead of the client to the greatest extent and performs privacy preserving computation through the powerful computing power of the cloud server. In scheme [38], the cloud server does not participate in the whole process, so it brings heavy computation overhead to the client. In scheme [39], the computation of the diagnosis values needs to be completed by the cloud server and the client. Therefore, it not only brings heavy computation overhead to the client but also requires the client to always stay online in this process.


### 7.2.4. Comparison Analysis of Diagnosis Result Generation

In our proposed scheme, after CSS completes the diagnosis values computation, it will jointly execute the secure protocol with CCS to generate the diagnosis result. We continued to make comparison analysis with schemes in [38–40]. The results are shown in Table 10.Table 10
Comparison analysis of diagnosis result generation.
SchemesCloud server computation (s)Client computation (s)Diagnosis result generation (s)Reference [38], l=5121.5840.0971.681Reference [39], l=5120.0070.0410.048Reference [40], l=5120.10100.101Ours0.07100.071Through the comparison analysis in Table10, it can be seen that the computation time of our proposed scheme is significantly lower than the scheme in references [38, 40]. In our proposed scheme, the client does not need to participate in the process of diagnosis result generation. The schemes in references [38, 39] require the participation of the client, which brings heavy computation overhead to the client.


### 7.2.5. Comprehensive Comparison Analysis

We made a comparison analysis of the whole privacy preserving online disease diagnosis process. It is divided into the secret diagnosis request generation (data encryption), diagnosis value computation, and diagnosis result generation. The results are shown in Table11.Table 11
Comprehensive comparison analysis.
Data encryption (s)Diagnosis value computation (s)Diagnosis result generation (s)Total time (s)Reference [38], l=5120.6580.0051.6812.389Reference [39], l=5120.570.5520.0481.17Reference [40], l=5120.0041.0920.1011.197Ours0.0080.0960.0710.175Through the comparison analysis in Table11, the total time of our proposed scheme is significantly lower than the schemes in references [38–40]. Considering that in the actual application scenario, a large number of users will constantly initiate secret diagnosis requests. It is very important to be able to quickly respond to the diagnosis results for users. Therefore, our scheme has more practical application value. Then, we made a summary as shown in Table 12.Table 12
Scheme summary.
SchemesMultidata ownersTrainingMulticlassificationLow client overheadReference [38]✕✕✓✕Reference [39]✕✕✓✕Reference [40]✕✕✓✓Reference [37]✓✓✕✓Ours✓✓✓✓


## 7.2.1. Noise Effect of BFV Cryptosystem

When using the BFV cryptosystem for homomorphic operation, the influence of noise needs to be considered. The noise of ciphertext will be increased when the multiplication homomorphic operation is carried out. If the noise is too large after computation, the correct result cannot be obtained after decryption.Therefore, the BFV cryptosystem in SEAL will set thenoise budget during initialization. If the noise budget is greater than 0 after the computation, it can be decrypted correctly. The value of noise budget is related to the setting of parameters. We evaluated the influence of poly module degree d on the encryption time, the change of noise budget after homomorphic operation, the computation time and whether the decryption result is correct. The results are shown in Table 6. It can be seen that the noise consumption of the BFV cryptosystem is relatively large when performing multiplication homomorphism, so the BFV cryptosystem can only perform multiplication homomorphism for a limited number of times. When computing the diagnosis values, only one inner product operation and one addition operation are required. Therefore, it is completely feasible to use the BFV cryptosystem.Table 6
The influence of poly modulus degreed on noise budget.
dEncryption time (s)Initial noise budget (bit)Noise budget (bit) after operationComputation time (s)Decryption result (correct or wrong)20480.013200.012✕40960.023900.031✕81920.69110640.123✓327681.1787617132.268✓We comprehensively consider the encryption time and computation time and ensure that the computation results can be decrypted correctly. The parameter we set ispoly module degreed=8192.


## 7.2.2. Influence of Different Classification Numbers on Computation Overhead

When using the BFV cryptosystem to encrypt data, multiple plaintext data can be packaged and encrypted into a ciphertext. The number of classifications isL.We tested the impact of differentL on useri and DSP. The results are shown in Figure 4(a). With the increase of L, the encryption time of DSP is gradually increasing, and the encryption time of useri can be considered as unchanged. We also tested the impact of different L on the diagnosis values computation of CSS. The results are shown in Figure 4(b). With the continuous increase of L, the computation time for CSS is also increasing. The process of generating diagnosis result is jointly completed by CSS and CCS. We tested the effect of different L on the diagnosis result generation. The results are shown in Figure 4(c). With the continuous increase of L, the time for CSS and CCS is also increasing.Figure 4
Influence of different classification numbers on computation overhead. (a) Data encryption, (b) diagnosis values computation, and (c) diagnosis result generation.
(a)(b)(c)


## 7.2.3. Comparison Analysis of Secret Diagnosis Request Generation and Diagnosis Values Computation

In our proposed scheme, secret diagnosis request generation can be regarded as data encryption ofuseri and diagnosis value computation can be regarded as homomorphic operation. We compared with the other three privacy preserving schemes. The results are shown in Table 7.Table 7
Comparison analysis.
SchemesData encryption (s)Diagnosis values computation (s)Total time (s)Reference [38], l=5120.6580.0050.663Reference [39], l=5120.570.5521.122Reference [40], l=5120.0041.0921.096Ours0.0080.0960.104Through the comparison analysis, it can be seen that the time of data encryption in our proposed scheme is significantly reduced compared with [38, 39]. In the computation of decision function, our scheme has significantly reduced the computational cost compared with the scheme in [39, 40]. At the same time, it can be seen from the total time that our proposed scheme is significantly lower than the other three schemes.Next, we make further analysis. The names of participants may be slightly different in different schemes. In order to facilitate analysis, we divided participants into cloud server and client. We compared the computation overhead of cloud server and client, respectively. The results are shown in Tables8 and 9.Table 8
Comparison analysis of the cloud server.
SchemesData encryption (s)Diagnosis value computation (s)Total time (s)Reference [38], l=512000Reference [39], l=5120.570.0010.571Reference [40], l=51201.0921.092Ours00.0960.096Table 9
Comparison analysis of the client.
SchemesData encryption (s)Diagnosis value computation (s)Total time (s)Reference [38], l=5120.6580.0050.663Reference [39], l=51200.5510.551Reference [40], l=5120.00400.004Ours0.00800.008In our proposed scheme, the client only needs to encrypt the data and can be offline after uploading the data to the cloud server. The cloud server only needs to compute the decision function. This model reduces the computation overhead of the client to the greatest extent and performs privacy preserving computation through the powerful computing power of the cloud server. In scheme [38], the cloud server does not participate in the whole process, so it brings heavy computation overhead to the client. In scheme [39], the computation of the diagnosis values needs to be completed by the cloud server and the client. Therefore, it not only brings heavy computation overhead to the client but also requires the client to always stay online in this process.


## 7.2.4. Comparison Analysis of Diagnosis Result Generation

In our proposed scheme, after CSS completes the diagnosis values computation, it will jointly execute the secure protocol with CCS to generate the diagnosis result. We continued to make comparison analysis with schemes in [38–40]. The results are shown in Table 10.Table 10
Comparison analysis of diagnosis result generation.
SchemesCloud server computation (s)Client computation (s)Diagnosis result generation (s)Reference [38], l=5121.5840.0971.681Reference [39], l=5120.0070.0410.048Reference [40], l=5120.10100.101Ours0.07100.071Through the comparison analysis in Table10, it can be seen that the computation time of our proposed scheme is significantly lower than the scheme in references [38, 40]. In our proposed scheme, the client does not need to participate in the process of diagnosis result generation. The schemes in references [38, 39] require the participation of the client, which brings heavy computation overhead to the client.


## 7.2.5. Comprehensive Comparison Analysis

We made a comparison analysis of the whole privacy preserving online disease diagnosis process. It is divided into the secret diagnosis request generation (data encryption), diagnosis value computation, and diagnosis result generation. The results are shown in Table11.Table 11
Comprehensive comparison analysis.
Data encryption (s)Diagnosis value computation (s)Diagnosis result generation (s)Total time (s)Reference [38], l=5120.6580.0051.6812.389Reference [39], l=5120.570.5520.0481.17Reference [40], l=5120.0041.0920.1011.197Ours0.0080.0960.0710.175Through the comparison analysis in Table11, the total time of our proposed scheme is significantly lower than the schemes in references [38–40]. Considering that in the actual application scenario, a large number of users will constantly initiate secret diagnosis requests. It is very important to be able to quickly respond to the diagnosis results for users. Therefore, our scheme has more practical application value. Then, we made a summary as shown in Table 12.Table 12
Scheme summary.
SchemesMultidata ownersTrainingMulticlassificationLow client overheadReference [38]✕✕✓✕Reference [39]✕✕✓✕Reference [40]✕✕✓✓Reference [37]✓✓✕✓Ours✓✓✓✓


## 8. Conclusion

In this paper, we propose an efficient and privacy preserving outsourced multiclass SVM training and online-aided disease diagnosis scheme. We design some secure basic operation algorithms for machine learning training over the outsourced data from multiple data owners. We achieve a privacy preserving multiclass SVM training based on the basic operation algorithms. In the diagnosis phase, we achieve a privacy preserving multiclass diagnosis through our proposed the secure maximum find algorithm and secure comparison algorithm. Security analysis proves that our proposed scheme ensures that outsourced data, model parameters, users’ symptoms, and diagnosis results will not be leaked. Experimental evaluation illustrates that our proposed scheme significantly reduces the computation overhead. In the future, we will study more efficient and privacy preserving machine learning schemes.



---

*Source: 7751845-2022-10-12.xml*