# Noninteractive Lightweight Privacy-Preserving Auditing on Images in Mobile Crowdsourcing Networks

Authors: Juan Zhang; Changsheng Wan; Chunyu Zhang; Xiaojun Guo; Yongyong Chen
Journal: Security and Communication Networks
 (2020)

---

## Abstract

To determine whether images on the crowdsourcing server meet the mobile user’s requirement, an auditing protocol is desired to check these images. However, before paying for images, the mobile user typically cannot download them for checking. Moreover, since mobiles are usually low-power devices and the crowdsourcing server has to handle a large number of mobile users, the auditing protocol should be lightweight. To address the above security and efficiency issues, we propose a novel noninteractive lightweight privacy-preserving auditing protocol on images in mobile crowdsourcing networks, called NLPAS. Since NLPAS allows the mobile user to check images on the crowdsourcing server without downloading them, the newly designed protocol can provide privacy protection for these images. At the same time, NLPAS uses the binary convolutional neural network for extracting features from images and designs a novel privacy-preserving Hamming distance computation algorithm for determining whether these images on the crowdsourcing server meet the mobile user’s requirement. Since these two techniques are both lightweight, NLPAS can audit images on the crowdsourcing server in a privacy-preserving manner while still enjoying high efficiency. Experimental results show that NLPAS is feasible for real-world applications.


---

## Body

## 1. Introduction

Recently, mobile crowdsourcing systems have been widely deployed all over the world, which collect and process data through widely available mobile devices [1]. As discussed in [2], since data typically originate from third parties, falsified data may be reported to the crowdsourcing server. Therefore, to ensure data trust, it is important to check whether the data collected by workers meet the mobile user’s requirement before using it [2]. We call this sort of protocol the auditing protocol. At the same time, to avoid economic loss, mobile users may not be allowed to download images before paying for them. Therefore, to check images before downloading them, the auditing protocol should have the noninteractive feature. Moreover, as shown in [2], data uploaded by participants may contain their private and sensitive information. Therefore, the auditing protocol should have the privacy-preserving feature. Finally, due to limited resources of mobile users, the auditing protocol should be quite efficient. Therefore, a noninteractive lightweight privacy-preserving auditing scheme is needed for the mobile user to check images before downloading them.Regardless of the technology implemented, a typical “noninteractive lightweight privacy-preserving auditing system (NLPAS)” includes three entities: the “crowdsourcing server (CS)” which stores images, the “mobile user (MU)” who audits images stored on the crowdsourcing server before downloading them, and the worker who collects images and uploads them to the CS. In practice, these entities are involved in two processes (i.e., the uploading process and the auditing process). During the uploading process, the worker collects images and uploads them to the CS. During the auditing process, the MU audits images stored on the CS and then determines whether to download them.Security has vital significance for NLPAS. To avoid economic loss, the CS is not willing to transport images to the MU before the latter pays for them. On the other hand, the MU is not willing to pay for images before he/she can make sure that these images really meet the requirement. To handle this dilemma, it is reasonable to design a privacy-preserving auditing protocol, which allows the MU to check whether these images meet the requirement before downloading them. Unfortunately, the current security protocols for crowdsourcing systems (i.e., [3–40]) only consider how the CS checks images uploaded by workers. This leads to two issues. First, to earn more money, the CS is not willing to check images and provide true information to the MU. Second, the requirements of multiple MUs may vary, and the CS may not know them. For example, one MU may be interested in the hill in the image, while another MU may be interested in the lion in the image. In this case, it is impossible for the CS to know the requirements of multiple MUs. So, to make sure the images on the CS meet the requirements of the MU, it is desired to design a privacy-preserving auditing protocol for mobile crowdsourcing systems.Efficiency is another serious concern for NLPAS. Due to limited resources of mobile devices, the MU is seriously concerned about the high computation cost arising from running the auditing protocol. At the same time, the CS will have to handle a lot of auditing requests from multiple MUs, and it is seriously concerned about the computation cost too. So, the newly designed auditing protocol should be lightweight. Taking both security and efficiency into account, we aim to design a noninteractive privacy-preserving lightweight auditing protocol on images in the mobile crowdsourcing system, which extracts features from images and then determines whether these features meet the MU’s requirement. An auditing protocol for the mobile crowdsourcing system should fulfill the following requirements:(1)
Privacy Preserving. When auditing images on the CS, the MU must be ensured that his/her requirement is not leaked to the CS or a third-party adversary. Otherwise, the CS may forge wrong information to pass the auditing, resulting in economic loss of the MU. At the same time, the CS must be ensured that the features of images are not leaked to the MU or a third-party adversary too. Otherwise, it will result in economic loss of the CS.(2)
Content Privacy. During auditing, the CS should be ensured that the MU or a third-party adversary cannot extract content of images from exchanged information. Otherwise, the adversary may steal content of images stored on the CS, resulting in economic loss of the CS.(3)
Auditing. After the auditing process, the MU should be able to determine whether the content in an image meets his/her requirement.(4)
Computation Cost. The CS and the MU should be ensured that their computation costs are low when running auditing algorithms.(5)
Accuracy. The CS and the MU should be ensured that the accuracy of the model used for extracting features from images during auditing is high.Obviously, designing an auditing protocol for NLPAS is a nontrivial task, as the MU has to determine whether the images stored on the CS meet the requirement without downloading them. Recently, security protocols for mobile crowdsourcing systems have focused on image-checking techniques run by the CS. However, there is no protocol considering the image-checking technique established by the MU. Moreover, when focusing on this topic, we notice that there is no security scheme which can be directly used for satisfying all the above requirements. We will present the detailed analysis for arriving at this conclusion in the next section. This becomes a more serious issue, since more and more mobile crowdsourcing systems are being deployed. Motivated by this observation, in this paper, we mainly make three contributions:(1)
We present a comprehensive set of requirements for auditing protocols in mobile crowdsourcing systems and show some security and efficiency problems of current data checking protocols in mobile crowdsourcing systems.(2)
We propose a novel auditing protocol called NLPAS, which can check whether the images stored on the CS meet the MU’s requirement. However, different from current data checking protocols for mobile crowdsourcing systems, NLPAS allows the MU instead of the CS to check images in a privacy-preserving manner. By doing so, the MU can determine whether the images meet the requirement before paying for them. At the same time, the CS can make sure that those images will not be leaked. To fulfill all these requirements, we will first introduce the “binary convolutional neural network (BCNN)” technique [41] to the newly designed auditing protocol for extracting features of images, and then we will design a novel privacy-preserving Hamming distance computation [42] technique for determining whether the images meet the MU’s requirements. Since these two techniques are quite lightweight, the computation cost of NLPAS can be reduced significantly.(3)
We analyze the security of NLPAS, showing it satisfies requirements (1), (2), and (3) in Section1. And, we evaluate the efficiency of NLPAS, showing it satisfies requirements (4) and (5) in Section 1. We organize the remainder of this paper as follows. First, we investigate the related work in Section 2. Second, we describe the NLPAS protocol in Section 3, followed by security analysis and efficiency evaluation in Sections 4 and 5, respectively. Finally, in Section 6, we draw our conclusions.


## 2. Related Work

### 2.1. Data Trust

Data trust is an essential security problem in mobile crowdsourcing systems. Due to openness, workers in mobile crowdsourcing systems may have different security abilities, resulting in low data trust [3]. Recently, a lot of papers have been published, focusing on data trust in crowdsourcing systems, as illustrated below.


#### 2.1.1. Voting-Based Data Checking

This sort of scheme takes observed results with the most observers as the true data [4–6]. For example, in [4], the authors allowed a number of observers to report their evaluation results on data and took the one as true data if most of the evaluation results were positive. Similarly, the authors in [5] aimed to find conflicts of data obtained from multiple workers, which could be seen as a variation of the voting-based data checking scheme.


#### 2.1.2. Context Information-Based Data Checking

This sort of scheme uses context information such as location information to determine whether the data are true or not. For example, in [7], the authors required workers to upload location information together with data, while the CS used location information to determine whether the data were true or not. Similarly, the authors in [8, 9] used the worker’s trajectory for determining whether the data were true or not.


#### 2.1.3. Statistics-Based Data Checking

This sort of scheme [10–13] uses statistic methods for evaluating whether the data are true or not. For example, the authors in [10] used maximum likelihood estimation for checking data. And, the authors in [11] used maximum a posteriori (MAP) estimation for determining whether the data were true or not.


#### 2.1.4. Gold Data Set-Based Data Checking

This sort of scheme uses a gold data set for checking data uploaded by workers [14, 15]. For example, in [15], the authors assigned fully trusted workers with a gold data set for checking data uploaded from other workers.


#### 2.1.5. Data Redundancy Checking

This sort of scheme aims to address data redundancy issues [16, 17]. For example, in [16], the authors designed a scheme to find redundancy data from multiple workers and used redundancy data for estimating missing values.From the above analysis, it can be seen that the existing schemes mainly focus on data checking performed by the CS and workers. And, they mainly consider whether the data are correct. However, existing schemes did not allow the mobile users to check data before downloading it. Moreover, existing schemes only focus on the correctness of data and do not consider whether the data match the MU’s requirement since requirements from multiple mobile users may vary. This leads to a serious issue: The mobile user may waste money on nonmatched data. Therefore, an auditing scheme performed by the MU before data downloading is desired.


### 2.2. Data Privacy

Data privacy is another important problem in mobile crowdsourcing systems. First, if the data uploaded by workers are leaked, the CS and workers may lose money. More importantly, if the leaked data contain privacy of workers, the adversary may cause them harm. Second, if the data of mobile users are leaked, the adversary may deduce valuable information about mobile users [3]. Therefore, data privacy is a serious concern in crowdsourcing systems. Papers about data privacy in mobile crowdsourcing systems are illustrated below.


#### 2.2.1. Encryption

This sort of scheme aims to encrypt data before uploading them [18–21]. For example, in [18], a homomorphic identity-based encryption algorithm was designed for protecting data uploaded by workers.


#### 2.2.2. Differential Privacy

This sort of scheme adds perturbation to data [22–25]. For example, in [25], the authors added random perturbation to data uploaded by workers.


#### 2.2.3. Location Privacy

This sort of scheme aims to protect location information of workers [26–31]. For example, in [26], the authors used k-anonymity for providing location privacy.


#### 2.2.4. Personal Information Privacy

This sort of scheme aims to protect personal information of workers and mobile users [32–35]. For example, in [35], the authors defined multiple privacy level of personal information.Recently, several new techniques such as blockchain and fog computing have been introduced to mobile crowdsourcing networks [36–40]. For example, in [36], the authors used blockchain for user authorization. And, in [38], the authors used fog computing for data aggregation and task allocation. Finally, the features of existing schemes are listed in Table 1.Table 1
Features of existing schemes.
Data trustPrivacy preservingNoninteractive[4–6]Voting-based data checking╳╳[7–9]Context information-based data checking╳╳[10–13]Statistics-based data checking╳╳[14, 15]Gold data set-based data checking╳╳[16, 17]Data redundancy checking╳╳[18–21]╳Encryption╳[22–25]╳Differential privacy╳[26–31]╳Location privacy╳[32–35]╳Personal information privacy╳[36]Blockchain-based authorization╳╳[37]╳Blockchain-based data privacy╳[38]╳Fog-based data privacy╳From Table1, it can be seen that the existing schemes can provide data trust and privacy protection in various ways. However, most schemes only consider one feature, either data trust or privacy preserving. And, no scheme provides both features. Moreover, all existing schemes do not support the noninteractive feature. This leads to a dilemma: on the one hand, to provide privacy protection, the MU cannot get data before paying for it; on the other hand, the MU has to check data before paying for it, to determine whether the data meet the requirement.Furthermore, for images, this dilemma becomes more serious since the data volume of images is much larger than that of traditional texts. To handle this dilemma, it is desired to design a noninteractive lightweight privacy-preserving auditing protocol on images in mobile crowdsourcing networks, which allows the MU to efficiently determine whether the images meet the requirements without knowing anything about these images.


## 2.1. Data Trust

Data trust is an essential security problem in mobile crowdsourcing systems. Due to openness, workers in mobile crowdsourcing systems may have different security abilities, resulting in low data trust [3]. Recently, a lot of papers have been published, focusing on data trust in crowdsourcing systems, as illustrated below.


### 2.1.1. Voting-Based Data Checking

This sort of scheme takes observed results with the most observers as the true data [4–6]. For example, in [4], the authors allowed a number of observers to report their evaluation results on data and took the one as true data if most of the evaluation results were positive. Similarly, the authors in [5] aimed to find conflicts of data obtained from multiple workers, which could be seen as a variation of the voting-based data checking scheme.


### 2.1.2. Context Information-Based Data Checking

This sort of scheme uses context information such as location information to determine whether the data are true or not. For example, in [7], the authors required workers to upload location information together with data, while the CS used location information to determine whether the data were true or not. Similarly, the authors in [8, 9] used the worker’s trajectory for determining whether the data were true or not.


### 2.1.3. Statistics-Based Data Checking

This sort of scheme [10–13] uses statistic methods for evaluating whether the data are true or not. For example, the authors in [10] used maximum likelihood estimation for checking data. And, the authors in [11] used maximum a posteriori (MAP) estimation for determining whether the data were true or not.


### 2.1.4. Gold Data Set-Based Data Checking

This sort of scheme uses a gold data set for checking data uploaded by workers [14, 15]. For example, in [15], the authors assigned fully trusted workers with a gold data set for checking data uploaded from other workers.


### 2.1.5. Data Redundancy Checking

This sort of scheme aims to address data redundancy issues [16, 17]. For example, in [16], the authors designed a scheme to find redundancy data from multiple workers and used redundancy data for estimating missing values.From the above analysis, it can be seen that the existing schemes mainly focus on data checking performed by the CS and workers. And, they mainly consider whether the data are correct. However, existing schemes did not allow the mobile users to check data before downloading it. Moreover, existing schemes only focus on the correctness of data and do not consider whether the data match the MU’s requirement since requirements from multiple mobile users may vary. This leads to a serious issue: The mobile user may waste money on nonmatched data. Therefore, an auditing scheme performed by the MU before data downloading is desired.


## 2.1.1. Voting-Based Data Checking

This sort of scheme takes observed results with the most observers as the true data [4–6]. For example, in [4], the authors allowed a number of observers to report their evaluation results on data and took the one as true data if most of the evaluation results were positive. Similarly, the authors in [5] aimed to find conflicts of data obtained from multiple workers, which could be seen as a variation of the voting-based data checking scheme.


## 2.1.2. Context Information-Based Data Checking

This sort of scheme uses context information such as location information to determine whether the data are true or not. For example, in [7], the authors required workers to upload location information together with data, while the CS used location information to determine whether the data were true or not. Similarly, the authors in [8, 9] used the worker’s trajectory for determining whether the data were true or not.


## 2.1.3. Statistics-Based Data Checking

This sort of scheme [10–13] uses statistic methods for evaluating whether the data are true or not. For example, the authors in [10] used maximum likelihood estimation for checking data. And, the authors in [11] used maximum a posteriori (MAP) estimation for determining whether the data were true or not.


## 2.1.4. Gold Data Set-Based Data Checking

This sort of scheme uses a gold data set for checking data uploaded by workers [14, 15]. For example, in [15], the authors assigned fully trusted workers with a gold data set for checking data uploaded from other workers.


## 2.1.5. Data Redundancy Checking

This sort of scheme aims to address data redundancy issues [16, 17]. For example, in [16], the authors designed a scheme to find redundancy data from multiple workers and used redundancy data for estimating missing values.From the above analysis, it can be seen that the existing schemes mainly focus on data checking performed by the CS and workers. And, they mainly consider whether the data are correct. However, existing schemes did not allow the mobile users to check data before downloading it. Moreover, existing schemes only focus on the correctness of data and do not consider whether the data match the MU’s requirement since requirements from multiple mobile users may vary. This leads to a serious issue: The mobile user may waste money on nonmatched data. Therefore, an auditing scheme performed by the MU before data downloading is desired.


## 2.2. Data Privacy

Data privacy is another important problem in mobile crowdsourcing systems. First, if the data uploaded by workers are leaked, the CS and workers may lose money. More importantly, if the leaked data contain privacy of workers, the adversary may cause them harm. Second, if the data of mobile users are leaked, the adversary may deduce valuable information about mobile users [3]. Therefore, data privacy is a serious concern in crowdsourcing systems. Papers about data privacy in mobile crowdsourcing systems are illustrated below.


### 2.2.1. Encryption

This sort of scheme aims to encrypt data before uploading them [18–21]. For example, in [18], a homomorphic identity-based encryption algorithm was designed for protecting data uploaded by workers.


### 2.2.2. Differential Privacy

This sort of scheme adds perturbation to data [22–25]. For example, in [25], the authors added random perturbation to data uploaded by workers.


### 2.2.3. Location Privacy

This sort of scheme aims to protect location information of workers [26–31]. For example, in [26], the authors used k-anonymity for providing location privacy.


### 2.2.4. Personal Information Privacy

This sort of scheme aims to protect personal information of workers and mobile users [32–35]. For example, in [35], the authors defined multiple privacy level of personal information.Recently, several new techniques such as blockchain and fog computing have been introduced to mobile crowdsourcing networks [36–40]. For example, in [36], the authors used blockchain for user authorization. And, in [38], the authors used fog computing for data aggregation and task allocation. Finally, the features of existing schemes are listed in Table 1.Table 1
Features of existing schemes.
Data trustPrivacy preservingNoninteractive[4–6]Voting-based data checking╳╳[7–9]Context information-based data checking╳╳[10–13]Statistics-based data checking╳╳[14, 15]Gold data set-based data checking╳╳[16, 17]Data redundancy checking╳╳[18–21]╳Encryption╳[22–25]╳Differential privacy╳[26–31]╳Location privacy╳[32–35]╳Personal information privacy╳[36]Blockchain-based authorization╳╳[37]╳Blockchain-based data privacy╳[38]╳Fog-based data privacy╳From Table1, it can be seen that the existing schemes can provide data trust and privacy protection in various ways. However, most schemes only consider one feature, either data trust or privacy preserving. And, no scheme provides both features. Moreover, all existing schemes do not support the noninteractive feature. This leads to a dilemma: on the one hand, to provide privacy protection, the MU cannot get data before paying for it; on the other hand, the MU has to check data before paying for it, to determine whether the data meet the requirement.Furthermore, for images, this dilemma becomes more serious since the data volume of images is much larger than that of traditional texts. To handle this dilemma, it is desired to design a noninteractive lightweight privacy-preserving auditing protocol on images in mobile crowdsourcing networks, which allows the MU to efficiently determine whether the images meet the requirements without knowing anything about these images.


## 2.2.1. Encryption

This sort of scheme aims to encrypt data before uploading them [18–21]. For example, in [18], a homomorphic identity-based encryption algorithm was designed for protecting data uploaded by workers.


## 2.2.2. Differential Privacy

This sort of scheme adds perturbation to data [22–25]. For example, in [25], the authors added random perturbation to data uploaded by workers.


## 2.2.3. Location Privacy

This sort of scheme aims to protect location information of workers [26–31]. For example, in [26], the authors used k-anonymity for providing location privacy.


## 2.2.4. Personal Information Privacy

This sort of scheme aims to protect personal information of workers and mobile users [32–35]. For example, in [35], the authors defined multiple privacy level of personal information.Recently, several new techniques such as blockchain and fog computing have been introduced to mobile crowdsourcing networks [36–40]. For example, in [36], the authors used blockchain for user authorization. And, in [38], the authors used fog computing for data aggregation and task allocation. Finally, the features of existing schemes are listed in Table 1.Table 1
Features of existing schemes.
Data trustPrivacy preservingNoninteractive[4–6]Voting-based data checking╳╳[7–9]Context information-based data checking╳╳[10–13]Statistics-based data checking╳╳[14, 15]Gold data set-based data checking╳╳[16, 17]Data redundancy checking╳╳[18–21]╳Encryption╳[22–25]╳Differential privacy╳[26–31]╳Location privacy╳[32–35]╳Personal information privacy╳[36]Blockchain-based authorization╳╳[37]╳Blockchain-based data privacy╳[38]╳Fog-based data privacy╳From Table1, it can be seen that the existing schemes can provide data trust and privacy protection in various ways. However, most schemes only consider one feature, either data trust or privacy preserving. And, no scheme provides both features. Moreover, all existing schemes do not support the noninteractive feature. This leads to a dilemma: on the one hand, to provide privacy protection, the MU cannot get data before paying for it; on the other hand, the MU has to check data before paying for it, to determine whether the data meet the requirement.Furthermore, for images, this dilemma becomes more serious since the data volume of images is much larger than that of traditional texts. To handle this dilemma, it is desired to design a noninteractive lightweight privacy-preserving auditing protocol on images in mobile crowdsourcing networks, which allows the MU to efficiently determine whether the images meet the requirements without knowing anything about these images.


## 3. NLPAS: The Protocol

### 3.1. Preliminaries

#### 3.1.1. Binary Convolutional Neural Networks

A convolutional neural network [41] is a kind of neural networks whose forward propagation operation can be expressed as Y=fW⊗X, where Y is the output tensor of the operation, f⋅ is a nonlinear function, W and X are the weight tensor and the activation tensor generated by the previous neural network layer, and ⊗ is the convolution operation. A binary convolutional neural network is a kind of convolutional neural networks, whose weights w∈W and activations x∈X are 1 bit instead of the floating point. And, the forward propagation operation can be expressed as Y=fW⊗X=fmBW⊗nBX=fmnBW⊗BX, where m and n are integers, and each bW∈BW is computed from the corresponding w∈W as follows:(1)bW=sgnw=1,w≥0,−1,w<0.Similarly, eachbX∈BX is computed from the corresponding x∈X as follows:(2)bX=sgnx=1,x≥0,−1,x<0.For bitwisebW and bX, the BW⊗BX operation can be efficiently computed using the XNOR-bitcount algorithm defined in [43].Moreover, since thesgn function is not differentiable during backward propagation, Bengio et al. used the clip function instead of sgn as follows: clip⋅=max−1,min1,⋅ [44]. By using the clip function, the binary convolutional neural network can use the same gradient descent algorithm as that of traditional neural networks to update parameters during training.


#### 3.1.2. Hamming Distance

The Hamming distance [42] is defined below.Given twon-bit binary vectors a⇀=a1,a2,⋯,an and b⇀=b1,b2,⋯,bn, where ai∈0,1,i=1,⋯,n and bi∈0,1,i=1,⋯,n, the Hamming distance of a⇀ and b⇀ is defined as(3)da⇀,b⇀=∑i=1nai⊗bi.The above definition shows that the Hamming distance is the total number of different bits betweena⇀ and b⇀. In other words, to compute the Hamming distance, we need to count the different bits between a⇀ and b⇀.


### 3.2. System Model

The main purpose of NLPAS is to determine whether the image on the CS meets the requirement of the MU in a privacy-preserving manner.The main idea of NLPAS can be divided into two parts, namely, the feature extracting part and the Hamming distance computation part. For the feature extracting part, the MU first defines a binary convolutional neural network and trains it using a data set according to the user’s requirement. Then, the MU extracts a binary vector (a⇀) from a template image using this binary convolutional neural network, which is used as the requirement feature. Finally, the MU sends the trained binary convolutional neural network to the CS, and the latter extracts a binary vector (b⇀) from the image to be audited using this trained network, which is used as the auditing feature. Since the binary convolutional neural network is quite lightweight, NLPAS can achieve high efficiency.For the Hamming distance computation part, the MU and the CS hide the two input vectors (a⇀ and b⇀) in a carrier number, respectively. And then, all operations for counting different bits between these two vectors are based on five basic mathematical operations, namely, addition, subtraction, multiplication, division, and modulo operations. Since these five basic mathematical operations are quite lightweight, our scheme can achieve high efficiency.Based on the feature extracting and Hamming distance computation techniques, the MU compares the Hamming distance ofa⇀ and b⇀ with a threshold to determine whether the interested image on the CS meets the requirement.The system model of NLPAS is shown in Figure1, which includes three phases as described below. And, the notations are listed in Table 2.Figure 1
System model of NLPAS.Table 2
Notations in this paper.
NotationDescriptionmodeBinary convolutional neural network model trained by the MUSKAPrivate parameters of the MUPKAPublic parameters of NLPASa⇀=a1,⋯,ai,⋯,anFeature vector of the MUb⇀=b1,⋯,bi,⋯,bnFeature vector of the CSnLength of the two binary vectorsa⇀ and b⇀lSecurity strength of NLPAScipherA=C,DCiphertexts generated by the MUC=c1,c2,⋯,cnThe first set of random numbers thata⇀ is hidden inD=d1,d2,⋯,dnThe second set of random numbers thata⇀ is hidden incipherB=E,FCiphertexts generated by the CSda⇀,b⇀Hamming distance ofa⇀ and b⇀ei,i=1,2,⋯,nThe first set of random numbers thatda⇀,b⇀ is hidden infi,i=1,2,⋯,nThe second set of random numbers thatda⇀,b⇀ is hidden inthresThreshold for determining whether the image on the CS meets the MU’s requirementwPrime number for counting different bits ina⇀ and b⇀gPrime number used as a carrierP=p1,p2,⋯,pnThe first set of random numbers for hidinggQ=q1,q2,⋯,qnThe second set of random numbers for hidinggS=s1,s2,⋯,snThe third set of random numbers for hidinggT=t1,t2,⋯,tnThe fourth set of random numbers for hidinggV=vi=pi+qig,i=1,2,⋯,nThe first set of bases for hiding vectorsU=ui=si+tig,i=1,2,⋯,nThe second set of bases for hiding vectorsJ,KTransitional values for extracting the hamming distanceX,YValues that contain the hamming distance


#### 3.2.1. The Initialization Phase

During this phase, the MU defines a binary convolutional neural network model (mode), trains it, and extracts a binary vector a⇀ from the template image using themode. Then, the MU generates public and private cryptographic parameters for the NLPAS system. These cryptographic parameters will be used for hiding vectors and extracting results in the following hiding phase and extracting phase. The initialization algorithm is described as follows.SKA,PKA⟵Initn,l. This algorithm is run by the MU for generating system parameters for NLPAS. It takes as input the length of input vectors (i.e., n) and the security strength of NLPAS (i.e., l-bit) and outputs the set of private and public cryptographic parameters (i.e., SKA and PKA).Then, the MU sends the public parameter (PKA) and the trained model (mode) to the CS. And, the latter extracts a binary vector b⇀ from the interested image stored on it.After the initialization phase, the MU holds (SKA,PKA,a⇀,mode) and the CS holds PKA,b⇀,mode.


#### 3.2.2. The Hiding Phase

When the MU wants to compute the Hamming distanceda⇀,b⇀ where a⇀ is known only by the MU and b⇀ is known only by the CS, it establishes the hiding process, by running the HidingA algorithm and sending the results to the CS. The HidingA algorithm is described below.cipherA⟵HidingAa⇀,SKA,PKA. This algorithm is run by the MU for hiding the binary vector a⇀ into a ciphertext. It takes as inputs the MU’s binary vector (i.e., a⇀), the MU’s private parameter (i.e., SKA), and the MU’s public parameter (i.e., PKA) and outputs the ciphertext (i.e., cipherA).Upon receiving the ciphertext (i.e.,cipherA), the CS injects its vector (i.e., b⇀) into cipherA using the InjectingB algorithm and gets the updated ciphertext cipherB. Then, the CS sends cipherB back to the MU, and the Hamming distance of vectors a⇀ and b⇀ is included in cipherB. The InjectingB algorithm is described below.cipherB⟵InjectingBb⇀,cipherA,PKA. This algorithm is run by the CS for injecting the binary vector b⇀ into cipherA. It takes as inputs the CS’s binary vector (i.e., b⇀), the MU’s ciphertext (i.e., cipherA), and the MU’s public parameter (i.e., PKA) and outputs the updated ciphertext (i.e., cipherB).After the hiding phase, the MU getscipherB, and the Hamming distance of a⇀ and b⇀ is hidden in cipherB for being extracted in the following extracting phase.


#### 3.2.3. The Extracting Phase

After receiving the updated ciphertext (i.e.,cipherB) from the CS, the MU extracts the Hamming distance from cipherB using the Extra algorithm, which is described below.da⇀,b⇀⟵ExtracipherB,SKA,PKA. This algorithm is run by the MU for extracting the Hamming distance from the updated ciphertext (i.e., cipherB). It takes as inputs the updated ciphertext cipherB, the MU’s private parameter (i.e., SKA), and the MU’s public parameter (i.e., PKA) and outputs the Hamming distance of a⇀ and b⇀ (i.e., da⇀,b⇀).After the extracting phase, the MU gets the Hamming distance ofa⇀ and b⇀ (i.e., da⇀,b⇀). Then, the MU sets a threshold value thres. If da⇀,b⇀≥thres, the interested image stored on the CS does not match the MU’s requirement. Otherwise, the interested image on the CS matches the MU’s requirement.In the above system model, the MU’s vectora⇀ is hidden in cipherA using the HidingA algorithm, which cannot be known by the CS. At the same time, the CS’s vector b⇀ is hidden in cipherB using the InjectingB algorithm, which cannot be known by the MU. Therefore, NLPAS can achieve the privacy-preserving goal described in Section 1.In the above system model, the CS’s vectorb⇀ is hidden in cipherB using the InjectingB algorithm, which cannot be known by the MU. Therefore, the MU only knows the Hamming distance between a⇀ and b⇀ and does not know b⇀ and the corresponding interested image. Therefore, NLPAS can achieve the content privacy goal described in Section 1.


### 3.3. Construction

The construction of NLPAS is a tuple (Init, HidingA, InjectingB, Extra) of probabilistic polynomial time algorithms as shown in Figure 2, and the details are defined below.Figure 2
Construction of NLPAS.SKA,PKA⟵Initn,l. The MU runs this algorithm for generating system parameters for NLPAS as follows. First, the MU generates a l-bit prime number w for counting different bits. Second, the MU generates a large prime number g with the length 2l+2+log2n as the carrier of NLPAS. Third, the MU generates four sets of positive random numbers for hiding g, namely, P=p1,p2,⋯,pn, Q=q1,q2,⋯,qn, S=s1,s2,⋯,sn, and T=t1,t2,⋯,tn, where ∑i=1npi<w−n/2, ∑i=1nsi<w−n/2. Fourth, the MU computes two sets of bases for hiding vectors as follows: V=vi=pi+qig,i=1,2,⋯,n and U=ui=si+tig,i=1,2,⋯,n. Finally, the MU gets SKA=g,P,Q,S,T,V,U and PKA=w.cipherA⟵HidingAa⇀,SKA,PKA. The MU runs this algorithm for hiding the binary vector a⇀=a1,⋯,ai,⋯,an into a ciphertext as follows. The MU computes ci=vi+w and di=ui for each ai=1 in a⇀. Otherwise, ci=vi and di=ui+w. Then, the MU gets cipherA=C=c1,c2,⋯,cn,D=d1,d2,⋯,dn.cipherB⟵InjectingBb⇀,cipherA,PKA. The CS runs this algorithm for injecting the binary vector b⇀=b1,⋯,bi,⋯,bn into cipherA as follows. First, the CS computes ei=ci and fi=wdi for each bi=1 in b⇀. Otherwise, ei=wci and fi=di. Second, the CS computes E=∑i=1nei and F=∑i=1nfi. Finally, the CS gets cipherB=E,F.da⇀,b⇀⟵ExtracipherB,SKA,PKA. The MU runs this algorithm for extracting the Hamming distance from the updated ciphertext (i.e., cipherB) as follows. First, the MU computes J=Emodg and K=Fmodg. Second, the MU computes X=J−Jmodw2/w2 and Y=K−Kmodw2/w2. Third, the MU computes da⇀,b⇀=X+Y.In the above construction, NLPAS uses only a few simple mathematical operations (i.e., addition, subtraction, multiplication, division, and modulo operations) instead of time-consuming cryptographic operations such as modular exponentiation. Therefore, it enjoys high efficiency. We will further evaluate the efficiency of NLPAS in Section5.


## 3.1. Preliminaries

### 3.1.1. Binary Convolutional Neural Networks

A convolutional neural network [41] is a kind of neural networks whose forward propagation operation can be expressed as Y=fW⊗X, where Y is the output tensor of the operation, f⋅ is a nonlinear function, W and X are the weight tensor and the activation tensor generated by the previous neural network layer, and ⊗ is the convolution operation. A binary convolutional neural network is a kind of convolutional neural networks, whose weights w∈W and activations x∈X are 1 bit instead of the floating point. And, the forward propagation operation can be expressed as Y=fW⊗X=fmBW⊗nBX=fmnBW⊗BX, where m and n are integers, and each bW∈BW is computed from the corresponding w∈W as follows:(1)bW=sgnw=1,w≥0,−1,w<0.Similarly, eachbX∈BX is computed from the corresponding x∈X as follows:(2)bX=sgnx=1,x≥0,−1,x<0.For bitwisebW and bX, the BW⊗BX operation can be efficiently computed using the XNOR-bitcount algorithm defined in [43].Moreover, since thesgn function is not differentiable during backward propagation, Bengio et al. used the clip function instead of sgn as follows: clip⋅=max−1,min1,⋅ [44]. By using the clip function, the binary convolutional neural network can use the same gradient descent algorithm as that of traditional neural networks to update parameters during training.


### 3.1.2. Hamming Distance

The Hamming distance [42] is defined below.Given twon-bit binary vectors a⇀=a1,a2,⋯,an and b⇀=b1,b2,⋯,bn, where ai∈0,1,i=1,⋯,n and bi∈0,1,i=1,⋯,n, the Hamming distance of a⇀ and b⇀ is defined as(3)da⇀,b⇀=∑i=1nai⊗bi.The above definition shows that the Hamming distance is the total number of different bits betweena⇀ and b⇀. In other words, to compute the Hamming distance, we need to count the different bits between a⇀ and b⇀.


## 3.1.1. Binary Convolutional Neural Networks

A convolutional neural network [41] is a kind of neural networks whose forward propagation operation can be expressed as Y=fW⊗X, where Y is the output tensor of the operation, f⋅ is a nonlinear function, W and X are the weight tensor and the activation tensor generated by the previous neural network layer, and ⊗ is the convolution operation. A binary convolutional neural network is a kind of convolutional neural networks, whose weights w∈W and activations x∈X are 1 bit instead of the floating point. And, the forward propagation operation can be expressed as Y=fW⊗X=fmBW⊗nBX=fmnBW⊗BX, where m and n are integers, and each bW∈BW is computed from the corresponding w∈W as follows:(1)bW=sgnw=1,w≥0,−1,w<0.Similarly, eachbX∈BX is computed from the corresponding x∈X as follows:(2)bX=sgnx=1,x≥0,−1,x<0.For bitwisebW and bX, the BW⊗BX operation can be efficiently computed using the XNOR-bitcount algorithm defined in [43].Moreover, since thesgn function is not differentiable during backward propagation, Bengio et al. used the clip function instead of sgn as follows: clip⋅=max−1,min1,⋅ [44]. By using the clip function, the binary convolutional neural network can use the same gradient descent algorithm as that of traditional neural networks to update parameters during training.


## 3.1.2. Hamming Distance

The Hamming distance [42] is defined below.Given twon-bit binary vectors a⇀=a1,a2,⋯,an and b⇀=b1,b2,⋯,bn, where ai∈0,1,i=1,⋯,n and bi∈0,1,i=1,⋯,n, the Hamming distance of a⇀ and b⇀ is defined as(3)da⇀,b⇀=∑i=1nai⊗bi.The above definition shows that the Hamming distance is the total number of different bits betweena⇀ and b⇀. In other words, to compute the Hamming distance, we need to count the different bits between a⇀ and b⇀.


## 3.2. System Model

The main purpose of NLPAS is to determine whether the image on the CS meets the requirement of the MU in a privacy-preserving manner.The main idea of NLPAS can be divided into two parts, namely, the feature extracting part and the Hamming distance computation part. For the feature extracting part, the MU first defines a binary convolutional neural network and trains it using a data set according to the user’s requirement. Then, the MU extracts a binary vector (a⇀) from a template image using this binary convolutional neural network, which is used as the requirement feature. Finally, the MU sends the trained binary convolutional neural network to the CS, and the latter extracts a binary vector (b⇀) from the image to be audited using this trained network, which is used as the auditing feature. Since the binary convolutional neural network is quite lightweight, NLPAS can achieve high efficiency.For the Hamming distance computation part, the MU and the CS hide the two input vectors (a⇀ and b⇀) in a carrier number, respectively. And then, all operations for counting different bits between these two vectors are based on five basic mathematical operations, namely, addition, subtraction, multiplication, division, and modulo operations. Since these five basic mathematical operations are quite lightweight, our scheme can achieve high efficiency.Based on the feature extracting and Hamming distance computation techniques, the MU compares the Hamming distance ofa⇀ and b⇀ with a threshold to determine whether the interested image on the CS meets the requirement.The system model of NLPAS is shown in Figure1, which includes three phases as described below. And, the notations are listed in Table 2.Figure 1
System model of NLPAS.Table 2
Notations in this paper.
NotationDescriptionmodeBinary convolutional neural network model trained by the MUSKAPrivate parameters of the MUPKAPublic parameters of NLPASa⇀=a1,⋯,ai,⋯,anFeature vector of the MUb⇀=b1,⋯,bi,⋯,bnFeature vector of the CSnLength of the two binary vectorsa⇀ and b⇀lSecurity strength of NLPAScipherA=C,DCiphertexts generated by the MUC=c1,c2,⋯,cnThe first set of random numbers thata⇀ is hidden inD=d1,d2,⋯,dnThe second set of random numbers thata⇀ is hidden incipherB=E,FCiphertexts generated by the CSda⇀,b⇀Hamming distance ofa⇀ and b⇀ei,i=1,2,⋯,nThe first set of random numbers thatda⇀,b⇀ is hidden infi,i=1,2,⋯,nThe second set of random numbers thatda⇀,b⇀ is hidden inthresThreshold for determining whether the image on the CS meets the MU’s requirementwPrime number for counting different bits ina⇀ and b⇀gPrime number used as a carrierP=p1,p2,⋯,pnThe first set of random numbers for hidinggQ=q1,q2,⋯,qnThe second set of random numbers for hidinggS=s1,s2,⋯,snThe third set of random numbers for hidinggT=t1,t2,⋯,tnThe fourth set of random numbers for hidinggV=vi=pi+qig,i=1,2,⋯,nThe first set of bases for hiding vectorsU=ui=si+tig,i=1,2,⋯,nThe second set of bases for hiding vectorsJ,KTransitional values for extracting the hamming distanceX,YValues that contain the hamming distance


### 3.2.1. The Initialization Phase

During this phase, the MU defines a binary convolutional neural network model (mode), trains it, and extracts a binary vector a⇀ from the template image using themode. Then, the MU generates public and private cryptographic parameters for the NLPAS system. These cryptographic parameters will be used for hiding vectors and extracting results in the following hiding phase and extracting phase. The initialization algorithm is described as follows.SKA,PKA⟵Initn,l. This algorithm is run by the MU for generating system parameters for NLPAS. It takes as input the length of input vectors (i.e., n) and the security strength of NLPAS (i.e., l-bit) and outputs the set of private and public cryptographic parameters (i.e., SKA and PKA).Then, the MU sends the public parameter (PKA) and the trained model (mode) to the CS. And, the latter extracts a binary vector b⇀ from the interested image stored on it.After the initialization phase, the MU holds (SKA,PKA,a⇀,mode) and the CS holds PKA,b⇀,mode.


### 3.2.2. The Hiding Phase

When the MU wants to compute the Hamming distanceda⇀,b⇀ where a⇀ is known only by the MU and b⇀ is known only by the CS, it establishes the hiding process, by running the HidingA algorithm and sending the results to the CS. The HidingA algorithm is described below.cipherA⟵HidingAa⇀,SKA,PKA. This algorithm is run by the MU for hiding the binary vector a⇀ into a ciphertext. It takes as inputs the MU’s binary vector (i.e., a⇀), the MU’s private parameter (i.e., SKA), and the MU’s public parameter (i.e., PKA) and outputs the ciphertext (i.e., cipherA).Upon receiving the ciphertext (i.e.,cipherA), the CS injects its vector (i.e., b⇀) into cipherA using the InjectingB algorithm and gets the updated ciphertext cipherB. Then, the CS sends cipherB back to the MU, and the Hamming distance of vectors a⇀ and b⇀ is included in cipherB. The InjectingB algorithm is described below.cipherB⟵InjectingBb⇀,cipherA,PKA. This algorithm is run by the CS for injecting the binary vector b⇀ into cipherA. It takes as inputs the CS’s binary vector (i.e., b⇀), the MU’s ciphertext (i.e., cipherA), and the MU’s public parameter (i.e., PKA) and outputs the updated ciphertext (i.e., cipherB).After the hiding phase, the MU getscipherB, and the Hamming distance of a⇀ and b⇀ is hidden in cipherB for being extracted in the following extracting phase.


### 3.2.3. The Extracting Phase

After receiving the updated ciphertext (i.e.,cipherB) from the CS, the MU extracts the Hamming distance from cipherB using the Extra algorithm, which is described below.da⇀,b⇀⟵ExtracipherB,SKA,PKA. This algorithm is run by the MU for extracting the Hamming distance from the updated ciphertext (i.e., cipherB). It takes as inputs the updated ciphertext cipherB, the MU’s private parameter (i.e., SKA), and the MU’s public parameter (i.e., PKA) and outputs the Hamming distance of a⇀ and b⇀ (i.e., da⇀,b⇀).After the extracting phase, the MU gets the Hamming distance ofa⇀ and b⇀ (i.e., da⇀,b⇀). Then, the MU sets a threshold value thres. If da⇀,b⇀≥thres, the interested image stored on the CS does not match the MU’s requirement. Otherwise, the interested image on the CS matches the MU’s requirement.In the above system model, the MU’s vectora⇀ is hidden in cipherA using the HidingA algorithm, which cannot be known by the CS. At the same time, the CS’s vector b⇀ is hidden in cipherB using the InjectingB algorithm, which cannot be known by the MU. Therefore, NLPAS can achieve the privacy-preserving goal described in Section 1.In the above system model, the CS’s vectorb⇀ is hidden in cipherB using the InjectingB algorithm, which cannot be known by the MU. Therefore, the MU only knows the Hamming distance between a⇀ and b⇀ and does not know b⇀ and the corresponding interested image. Therefore, NLPAS can achieve the content privacy goal described in Section 1.


## 3.2.1. The Initialization Phase

During this phase, the MU defines a binary convolutional neural network model (mode), trains it, and extracts a binary vector a⇀ from the template image using themode. Then, the MU generates public and private cryptographic parameters for the NLPAS system. These cryptographic parameters will be used for hiding vectors and extracting results in the following hiding phase and extracting phase. The initialization algorithm is described as follows.SKA,PKA⟵Initn,l. This algorithm is run by the MU for generating system parameters for NLPAS. It takes as input the length of input vectors (i.e., n) and the security strength of NLPAS (i.e., l-bit) and outputs the set of private and public cryptographic parameters (i.e., SKA and PKA).Then, the MU sends the public parameter (PKA) and the trained model (mode) to the CS. And, the latter extracts a binary vector b⇀ from the interested image stored on it.After the initialization phase, the MU holds (SKA,PKA,a⇀,mode) and the CS holds PKA,b⇀,mode.


## 3.2.2. The Hiding Phase

When the MU wants to compute the Hamming distanceda⇀,b⇀ where a⇀ is known only by the MU and b⇀ is known only by the CS, it establishes the hiding process, by running the HidingA algorithm and sending the results to the CS. The HidingA algorithm is described below.cipherA⟵HidingAa⇀,SKA,PKA. This algorithm is run by the MU for hiding the binary vector a⇀ into a ciphertext. It takes as inputs the MU’s binary vector (i.e., a⇀), the MU’s private parameter (i.e., SKA), and the MU’s public parameter (i.e., PKA) and outputs the ciphertext (i.e., cipherA).Upon receiving the ciphertext (i.e.,cipherA), the CS injects its vector (i.e., b⇀) into cipherA using the InjectingB algorithm and gets the updated ciphertext cipherB. Then, the CS sends cipherB back to the MU, and the Hamming distance of vectors a⇀ and b⇀ is included in cipherB. The InjectingB algorithm is described below.cipherB⟵InjectingBb⇀,cipherA,PKA. This algorithm is run by the CS for injecting the binary vector b⇀ into cipherA. It takes as inputs the CS’s binary vector (i.e., b⇀), the MU’s ciphertext (i.e., cipherA), and the MU’s public parameter (i.e., PKA) and outputs the updated ciphertext (i.e., cipherB).After the hiding phase, the MU getscipherB, and the Hamming distance of a⇀ and b⇀ is hidden in cipherB for being extracted in the following extracting phase.


## 3.2.3. The Extracting Phase

After receiving the updated ciphertext (i.e.,cipherB) from the CS, the MU extracts the Hamming distance from cipherB using the Extra algorithm, which is described below.da⇀,b⇀⟵ExtracipherB,SKA,PKA. This algorithm is run by the MU for extracting the Hamming distance from the updated ciphertext (i.e., cipherB). It takes as inputs the updated ciphertext cipherB, the MU’s private parameter (i.e., SKA), and the MU’s public parameter (i.e., PKA) and outputs the Hamming distance of a⇀ and b⇀ (i.e., da⇀,b⇀).After the extracting phase, the MU gets the Hamming distance ofa⇀ and b⇀ (i.e., da⇀,b⇀). Then, the MU sets a threshold value thres. If da⇀,b⇀≥thres, the interested image stored on the CS does not match the MU’s requirement. Otherwise, the interested image on the CS matches the MU’s requirement.In the above system model, the MU’s vectora⇀ is hidden in cipherA using the HidingA algorithm, which cannot be known by the CS. At the same time, the CS’s vector b⇀ is hidden in cipherB using the InjectingB algorithm, which cannot be known by the MU. Therefore, NLPAS can achieve the privacy-preserving goal described in Section 1.In the above system model, the CS’s vectorb⇀ is hidden in cipherB using the InjectingB algorithm, which cannot be known by the MU. Therefore, the MU only knows the Hamming distance between a⇀ and b⇀ and does not know b⇀ and the corresponding interested image. Therefore, NLPAS can achieve the content privacy goal described in Section 1.


## 3.3. Construction

The construction of NLPAS is a tuple (Init, HidingA, InjectingB, Extra) of probabilistic polynomial time algorithms as shown in Figure 2, and the details are defined below.Figure 2
Construction of NLPAS.SKA,PKA⟵Initn,l. The MU runs this algorithm for generating system parameters for NLPAS as follows. First, the MU generates a l-bit prime number w for counting different bits. Second, the MU generates a large prime number g with the length 2l+2+log2n as the carrier of NLPAS. Third, the MU generates four sets of positive random numbers for hiding g, namely, P=p1,p2,⋯,pn, Q=q1,q2,⋯,qn, S=s1,s2,⋯,sn, and T=t1,t2,⋯,tn, where ∑i=1npi<w−n/2, ∑i=1nsi<w−n/2. Fourth, the MU computes two sets of bases for hiding vectors as follows: V=vi=pi+qig,i=1,2,⋯,n and U=ui=si+tig,i=1,2,⋯,n. Finally, the MU gets SKA=g,P,Q,S,T,V,U and PKA=w.cipherA⟵HidingAa⇀,SKA,PKA. The MU runs this algorithm for hiding the binary vector a⇀=a1,⋯,ai,⋯,an into a ciphertext as follows. The MU computes ci=vi+w and di=ui for each ai=1 in a⇀. Otherwise, ci=vi and di=ui+w. Then, the MU gets cipherA=C=c1,c2,⋯,cn,D=d1,d2,⋯,dn.cipherB⟵InjectingBb⇀,cipherA,PKA. The CS runs this algorithm for injecting the binary vector b⇀=b1,⋯,bi,⋯,bn into cipherA as follows. First, the CS computes ei=ci and fi=wdi for each bi=1 in b⇀. Otherwise, ei=wci and fi=di. Second, the CS computes E=∑i=1nei and F=∑i=1nfi. Finally, the CS gets cipherB=E,F.da⇀,b⇀⟵ExtracipherB,SKA,PKA. The MU runs this algorithm for extracting the Hamming distance from the updated ciphertext (i.e., cipherB) as follows. First, the MU computes J=Emodg and K=Fmodg. Second, the MU computes X=J−Jmodw2/w2 and Y=K−Kmodw2/w2. Third, the MU computes da⇀,b⇀=X+Y.In the above construction, NLPAS uses only a few simple mathematical operations (i.e., addition, subtraction, multiplication, division, and modulo operations) instead of time-consuming cryptographic operations such as modular exponentiation. Therefore, it enjoys high efficiency. We will further evaluate the efficiency of NLPAS in Section5.


## 4. Security Analysis

In this section, we first show that NLPAS is correct and then analyze the security of NLPAS according to the security requirements described in Section1 (i.e., privacy preserving, content privacy, and auditing).


### 4.1. Correctness

In the construction in Section3.3, we use X for counting the bits where ai=1 and bi=0. Similarly, we use Y for counting the bits where ai=0 and bi=1. Therefore, the Hamming distance of a⇀ and b⇀ can be computed as da⇀,b⇀=X+Y.In this section, we shall show thatX can really be used for counting the bits where ai=1 and bi=0. And, the meaning of Y can be analyzed in a similar way.We start analyzing the meaning ofX from the variable ei as follows. First, according to the InjectingB algorithm, ei can be written as(4)ei=ci,bi=1,wci,bi=0.Second, taking the value ofci in the HidingA algorithm into consideration, ei can be further written as(5)ei=vi+w,ai=1,bi=1,vi,ai=0,bi=1,wvi+w,ai=1,bi=0,wvi,ai=0,bi=0.Third, taking the value ofvi in the Init algorithm into consideration, ei can be written as(6)ei=pi+qig+w,ai=1,bi=1,pi+qig,ai=0,bi=1,wpi+qig+w,ai=1,bi=0,wpi+qig,ai=0,bi=0.Fourth, considering all the four conditions (i.e.,ai=1,bi=1, ai=1,bi=0, ai=0,bi=1, and ai=0,bi=0) together, we can compute(7)E=∑i=1nei=∑ai=1,bi=1pi+qig+w+∑ai=1,bi=0wpi+qig+w+∑ai=0,bi=1pi+qig+∑ai=0,bi=0wpi+qig=∑ai=1,bi=01w2+∑ai=1,bi=11+∑bi=0piw+∑ai=0,bi=1pi+∑bi=1qi+∑bi=0wqig.Fifth, since the length ofw is l-bit, the length of ∑ai=1,bi=01w2 should be no more than log2n+2l. Since ∑i=1npi<w−n/2, the length of ∑ai=1,bi=11+∑bi=0piw should be no more than 2l−1, and the length of ∑ai=0,bi=1pi should be no more than l−1. Therefore, the length of ∑ai=1,bi=01w2 + ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi should be no more than log2n+2l+2. That is to say, ∑ai=1,bi=01w2 + ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi<g. So, we get(8)J=Emodg=∑ai=1,bi=01w2+∑ai=1,bi=11+∑bi=0piw+∑ai=0,bi=1pi.Sixth, since the length of∑ai=1,bi=11+∑bi=0piw is no more than 2l−1 and the length of ∑ai=0,bi=1pi is no more than l−1, the length of ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi is no more than 2l−1. That is to say, ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi<w2. So, we get Jmodw2 = ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi.Finally, we get(9)X=J−Jmodw2w2=∑ai=1,bi=01w2w2=∑ai=1,bi=01.This is really the total number of bits whereai=1 and bi=0. Similarly, we can learn that Y is the total number of bits where ai=0 and bi=1. Therefore, the Hamming distance of a⇀ and b⇀ is da⇀,b⇀=X+Y.From the above discussion, we can see that the main idea of privacy-preserving Hamming distance computation includes two points. First, we hide the information ofa⇀ and b⇀ in a big prime g. Second, the different bits (e.g., ai=1 and bi=0) are counted in an independent part of E, e.g., ∑ai=1,bi=01w2, which can be extracted using several modulo operations.


### 4.2. Privacy Preserving

The privacy-preserving requirement is to ensure that the adversary cannot extracta⇀ or b⇀ transmitted in the hiding phase.We first consider the privacy-preserving requirement fora⇀, where the adversary can be anybody who is able to get cipherA including the CS.From theHidingA algorithm defined in Section 3.3, it can be seen that ai∈a⇀ is hidden in ci and di. Since pi, qi, si, ti, and g are random numbers known only by the MU, ci and di are random numbers too.Moreover, since the length ofg is much longer than pi, si, and w, the lengths of ci and di are determined by qig and tig. Since qi and ti are random numbers, the lengths of ci and di are randomly determined by qi and ti, regardless of the value of ai (i.e., 0 or 1). Therefore, for ai=1 (i.e., ci=vi+w=pi+qig+w) and aj=0 (i.e., cj=vj=pj+qjg), it may be ci<cj. Similarly, for ai=1 and aj=0, it may be di>dj. That is to say, the set of ci and di including w may or may not be bigger than the set of ci and di without w. So, the adversary cannot get the value of ai from ci and di by determining that a bigger random ci represents 1, or a smaller random di represents 1. In other words, without knowing the set of random secrets pi,qi,si,ti,g, the adversary can extract ai from ci or di only with a negligible probability.Furthermore, if the length ofa⇀ is n, the probability that the adversary can get a⇀ is 1/2n.We then consider the privacy-preserving requirement forb⇀, where the adversary can be anybody who is able to get cipherB including the MU.From theInjectingB algorithm defined in Section 3.3, it can be seen that bi is hidden in E and F using the addition operations. Knowing the result of addition, the adversary can extract bi only with a negligible probability. Therefore, the privacy of b⇀ is ensured by the addition operation.Moreover, assuming the MU is the adversary who wants to extractb⇀ from cipherB=E,F, the MU has to solve the two equations  E=∑ai=1,bi=01w2+ ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi + ∑bi=1qi+∑bi=0wqig and F=∑ai=0,bi=11w2 + ∑ai=0,bi=01+∑bi=1siw + ∑ai=1,bi=0si + ∑bi=0ti+∑bi=1wtig. Since the MU knows pi,qi,si,ti,g,w, these two equations can be treated as two linear equations with n unknown numbers (i.e., b1,⋯,bn). Therefore, when n>2, there are a number of solutions for them. In addition, for n-bit b⇀, the number of solutions is 1/2n. That is to say, the probability that the MU can extract b⇀ from cipherB=E,F is 1/2n.From the above discussion, it can be seen that the adversary cannot extracta⇀ or b⇀ transmitted in the hiding phase. Therefore, NLPAS can achieve the privacy-preserving goal.


### 4.3. Content Privacy

The content privacy requirement is to ensure that the adversary cannot extract content of the interested image stored on the CS fromcipherB. The content of the interested image is included in b⇀. Since the adversary cannot extract b⇀ from cipherB as shown in the previous subsection, the content of the interested image stored on the CS cannot be extracted. Therefore, NLPAS can achieve the content privacy goal.


### 4.4. Auditing

The auditing requirement is to ensure that the MU can determine whether the content in the image stored on the CS meets the MU’s requirement. This is ensured by the Hamming distance. If the Hamming distance ofa⇀ and b⇀ is smaller than the threshold, the MU can determine that the content in the image is the one he/she needs. Otherwise, the content in the image is not needed by the MU. Moreover, the correctness of Hamming distance computation is ensured in Section 4.1.


## 4.1. Correctness

In the construction in Section3.3, we use X for counting the bits where ai=1 and bi=0. Similarly, we use Y for counting the bits where ai=0 and bi=1. Therefore, the Hamming distance of a⇀ and b⇀ can be computed as da⇀,b⇀=X+Y.In this section, we shall show thatX can really be used for counting the bits where ai=1 and bi=0. And, the meaning of Y can be analyzed in a similar way.We start analyzing the meaning ofX from the variable ei as follows. First, according to the InjectingB algorithm, ei can be written as(4)ei=ci,bi=1,wci,bi=0.Second, taking the value ofci in the HidingA algorithm into consideration, ei can be further written as(5)ei=vi+w,ai=1,bi=1,vi,ai=0,bi=1,wvi+w,ai=1,bi=0,wvi,ai=0,bi=0.Third, taking the value ofvi in the Init algorithm into consideration, ei can be written as(6)ei=pi+qig+w,ai=1,bi=1,pi+qig,ai=0,bi=1,wpi+qig+w,ai=1,bi=0,wpi+qig,ai=0,bi=0.Fourth, considering all the four conditions (i.e.,ai=1,bi=1, ai=1,bi=0, ai=0,bi=1, and ai=0,bi=0) together, we can compute(7)E=∑i=1nei=∑ai=1,bi=1pi+qig+w+∑ai=1,bi=0wpi+qig+w+∑ai=0,bi=1pi+qig+∑ai=0,bi=0wpi+qig=∑ai=1,bi=01w2+∑ai=1,bi=11+∑bi=0piw+∑ai=0,bi=1pi+∑bi=1qi+∑bi=0wqig.Fifth, since the length ofw is l-bit, the length of ∑ai=1,bi=01w2 should be no more than log2n+2l. Since ∑i=1npi<w−n/2, the length of ∑ai=1,bi=11+∑bi=0piw should be no more than 2l−1, and the length of ∑ai=0,bi=1pi should be no more than l−1. Therefore, the length of ∑ai=1,bi=01w2 + ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi should be no more than log2n+2l+2. That is to say, ∑ai=1,bi=01w2 + ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi<g. So, we get(8)J=Emodg=∑ai=1,bi=01w2+∑ai=1,bi=11+∑bi=0piw+∑ai=0,bi=1pi.Sixth, since the length of∑ai=1,bi=11+∑bi=0piw is no more than 2l−1 and the length of ∑ai=0,bi=1pi is no more than l−1, the length of ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi is no more than 2l−1. That is to say, ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi<w2. So, we get Jmodw2 = ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi.Finally, we get(9)X=J−Jmodw2w2=∑ai=1,bi=01w2w2=∑ai=1,bi=01.This is really the total number of bits whereai=1 and bi=0. Similarly, we can learn that Y is the total number of bits where ai=0 and bi=1. Therefore, the Hamming distance of a⇀ and b⇀ is da⇀,b⇀=X+Y.From the above discussion, we can see that the main idea of privacy-preserving Hamming distance computation includes two points. First, we hide the information ofa⇀ and b⇀ in a big prime g. Second, the different bits (e.g., ai=1 and bi=0) are counted in an independent part of E, e.g., ∑ai=1,bi=01w2, which can be extracted using several modulo operations.


## 4.2. Privacy Preserving

The privacy-preserving requirement is to ensure that the adversary cannot extracta⇀ or b⇀ transmitted in the hiding phase.We first consider the privacy-preserving requirement fora⇀, where the adversary can be anybody who is able to get cipherA including the CS.From theHidingA algorithm defined in Section 3.3, it can be seen that ai∈a⇀ is hidden in ci and di. Since pi, qi, si, ti, and g are random numbers known only by the MU, ci and di are random numbers too.Moreover, since the length ofg is much longer than pi, si, and w, the lengths of ci and di are determined by qig and tig. Since qi and ti are random numbers, the lengths of ci and di are randomly determined by qi and ti, regardless of the value of ai (i.e., 0 or 1). Therefore, for ai=1 (i.e., ci=vi+w=pi+qig+w) and aj=0 (i.e., cj=vj=pj+qjg), it may be ci<cj. Similarly, for ai=1 and aj=0, it may be di>dj. That is to say, the set of ci and di including w may or may not be bigger than the set of ci and di without w. So, the adversary cannot get the value of ai from ci and di by determining that a bigger random ci represents 1, or a smaller random di represents 1. In other words, without knowing the set of random secrets pi,qi,si,ti,g, the adversary can extract ai from ci or di only with a negligible probability.Furthermore, if the length ofa⇀ is n, the probability that the adversary can get a⇀ is 1/2n.We then consider the privacy-preserving requirement forb⇀, where the adversary can be anybody who is able to get cipherB including the MU.From theInjectingB algorithm defined in Section 3.3, it can be seen that bi is hidden in E and F using the addition operations. Knowing the result of addition, the adversary can extract bi only with a negligible probability. Therefore, the privacy of b⇀ is ensured by the addition operation.Moreover, assuming the MU is the adversary who wants to extractb⇀ from cipherB=E,F, the MU has to solve the two equations  E=∑ai=1,bi=01w2+ ∑ai=1,bi=11+∑bi=0piw + ∑ai=0,bi=1pi + ∑bi=1qi+∑bi=0wqig and F=∑ai=0,bi=11w2 + ∑ai=0,bi=01+∑bi=1siw + ∑ai=1,bi=0si + ∑bi=0ti+∑bi=1wtig. Since the MU knows pi,qi,si,ti,g,w, these two equations can be treated as two linear equations with n unknown numbers (i.e., b1,⋯,bn). Therefore, when n>2, there are a number of solutions for them. In addition, for n-bit b⇀, the number of solutions is 1/2n. That is to say, the probability that the MU can extract b⇀ from cipherB=E,F is 1/2n.From the above discussion, it can be seen that the adversary cannot extracta⇀ or b⇀ transmitted in the hiding phase. Therefore, NLPAS can achieve the privacy-preserving goal.


## 4.3. Content Privacy

The content privacy requirement is to ensure that the adversary cannot extract content of the interested image stored on the CS fromcipherB. The content of the interested image is included in b⇀. Since the adversary cannot extract b⇀ from cipherB as shown in the previous subsection, the content of the interested image stored on the CS cannot be extracted. Therefore, NLPAS can achieve the content privacy goal.


## 4.4. Auditing

The auditing requirement is to ensure that the MU can determine whether the content in the image stored on the CS meets the MU’s requirement. This is ensured by the Hamming distance. If the Hamming distance ofa⇀ and b⇀ is smaller than the threshold, the MU can determine that the content in the image is the one he/she needs. Otherwise, the content in the image is not needed by the MU. Moreover, the correctness of Hamming distance computation is ensured in Section 4.1.


## 5. Efficiency Evaluation

As shown in Section3, NLPAS includes two parts: feature extracting using the binary convolutional neural network and similarity computation using the privacy-preserving Hamming distance. Therefore, we mainly evaluate the computation costs consumed in these two parts. Moreover, for the feature extracting part, we will evaluate the accuracy of the trained model (i.e., mode).


### 5.1. Accuracy

To provide a benchmark of efficiency evaluation, we used the MNIST data set [45] and LeNet [46] for comparing the accuracy of the binary convolutional neural network with that of the full-precision convolutional neural network.MNIST [45] is a data set of handwritten digits, which contains a training set of 60,000 examples and a test set of 10,000 examples. And, all examples in the training and test data sets are 28×28 binary images.LeNet [46] is a convolutional neural network with three convolutional layers, two subsampling layers, two full connection layers, an input layer, and an output layer.For implementation, we used the BMXnet [47], which provided basic binarization operations for convolutional neural networks. After experimentation, we got the results as shown in Table 3.Table 3
Accuracy comparison.
AccuracyModel sizeBinary LeNet0.970.2 MBFull-precision LeNet0.994.6 MBFrom Table3, it can be seen that(1)
The accuracy of the binary LeNet is slightly lower than that of the full-precision LeNet. The accuracy reduced by using the binary LeNet is around0.99−0.97/0.99≈2%.(2)
The model size of the binary LeNet is much lower than that of the full-precision LeNet. The memory saved by binary LeNet is around4.6−0.2/4.6≈95.7%.In other words, by using the binary convolutional neural network instead of the traditional full-precision convolutional neural network, the accuracy is only slightly reduced, but the memory is largely saved. Therefore, the binary convolutional neural network is quite suitable for the mobile crowdsourcing network, where mobile devices are with limited storage resources. The above evaluation shows that NLPAS fulfills the fifth requirement listed in Section1 (i.e., the accuracy requirement).


### 5.2. Computation Costs

The computation cost of NLPAS includes the time cost consumed by the binary LeNet model and those consumed by mathematical operations. To test these time costs, we conducted our experiment on a laptop with an Intel i7-4770hq processor and an ubuntu-18.04 operating system. Then, we used OPENSSL [48] as the cryptographic library.For the binary LeNet, we take the features extracted by the last full-connection layer as the input vectors (i.e.,a⇀ and b⇀). Therefore, the vector length is n=84 [46]. To provide a basic security level, we set l=256, and the length of g is log2n+2l+2=521. To make sure ∑i=1npi<w−n/2 and ∑i=1nsi<w−n/2, we set the lengths of pi and si to be 500 bit. Then, we set the lengths of qi and ti to be 683 bit, so that the lengths of vi and ui are around 1024 bit.After the initial settings, we can count the mathematical operations in the hiding and extracting phases as listed in Table4. From Table 4, it can be seen that all mathematical operations are run over 1024 bit and 512 bit fields.Table 4
Number of mathematical operations in NLPAS.
HidingAInjectingBExtra+1024−bit841660×1024−bit0840mod1024−bit002mod512−bit002÷512−bit002−512−bit002+512−bit002Then, we tested the time costs consumed by these mathematical operations on the above laptop, and the average results of running them for 1,000,000 times are shown in Table5. From Table 5, it can be seen that the time costs of mathematical operations are at the μs level.Table 5
Time costs of mathematical operations (unit:μs).
+1024−bit0.22×1024−bit1.69mod1024−bit2.19mod512−bit0.98÷512−bit1.03−512−bit0.16+512−bit0.14Taking the results in Table5 into Table 4, we can get the computation costs of algorithms in NLPAS, as shown in Table 6. From Table 6, it can be seen that the computation cost of mathematical operations on the MU (i.e., time costs of HidingA and Extra) is much lower than that on the CS (i.e., InjectingB). Therefore, NLPAS is suitable for mobile crowdsourcing networks, where MU is with limited computation resources.Table 6
Computation costs of algorithms in NLPAS (unit:μs).
HidingAInjectingBExtra18.48178.489.00The time costs of the binary LeNet and the full-precision LeNet are shown in Table7, where the results are average values of running the feature extracting process for 1,000,000 times. From Table 7, it can be seen that the computation cost of feature extracting in NLPAS can be largely reduced by using the binary convolutional neural network instead of the full-precision convolutional neural network.Table 7
Computation costs of algorithms in NLPAS (unit:μs).
Binary LeNetFull-precision LeNet56.61435.2The above evaluation shows that NLPAS fulfills the fourth requirement listed in Section1 (i.e., the computation cost requirement).


### 5.3. Implementation of NLPAS

To make sure that NLPAS can work well, we implemented it. In our experimental environment, there were one laptop and one computer. The laptop acts as the MU, and the computer acts as the CS. The result shows that the total running time in the auditing protocol is approximately 0.3 ms. Therefore, NLPAS is feasible for being deployed in the real world.


## 5.1. Accuracy

To provide a benchmark of efficiency evaluation, we used the MNIST data set [45] and LeNet [46] for comparing the accuracy of the binary convolutional neural network with that of the full-precision convolutional neural network.MNIST [45] is a data set of handwritten digits, which contains a training set of 60,000 examples and a test set of 10,000 examples. And, all examples in the training and test data sets are 28×28 binary images.LeNet [46] is a convolutional neural network with three convolutional layers, two subsampling layers, two full connection layers, an input layer, and an output layer.For implementation, we used the BMXnet [47], which provided basic binarization operations for convolutional neural networks. After experimentation, we got the results as shown in Table 3.Table 3
Accuracy comparison.
AccuracyModel sizeBinary LeNet0.970.2 MBFull-precision LeNet0.994.6 MBFrom Table3, it can be seen that(1)
The accuracy of the binary LeNet is slightly lower than that of the full-precision LeNet. The accuracy reduced by using the binary LeNet is around0.99−0.97/0.99≈2%.(2)
The model size of the binary LeNet is much lower than that of the full-precision LeNet. The memory saved by binary LeNet is around4.6−0.2/4.6≈95.7%.In other words, by using the binary convolutional neural network instead of the traditional full-precision convolutional neural network, the accuracy is only slightly reduced, but the memory is largely saved. Therefore, the binary convolutional neural network is quite suitable for the mobile crowdsourcing network, where mobile devices are with limited storage resources. The above evaluation shows that NLPAS fulfills the fifth requirement listed in Section1 (i.e., the accuracy requirement).


## 5.2. Computation Costs

The computation cost of NLPAS includes the time cost consumed by the binary LeNet model and those consumed by mathematical operations. To test these time costs, we conducted our experiment on a laptop with an Intel i7-4770hq processor and an ubuntu-18.04 operating system. Then, we used OPENSSL [48] as the cryptographic library.For the binary LeNet, we take the features extracted by the last full-connection layer as the input vectors (i.e.,a⇀ and b⇀). Therefore, the vector length is n=84 [46]. To provide a basic security level, we set l=256, and the length of g is log2n+2l+2=521. To make sure ∑i=1npi<w−n/2 and ∑i=1nsi<w−n/2, we set the lengths of pi and si to be 500 bit. Then, we set the lengths of qi and ti to be 683 bit, so that the lengths of vi and ui are around 1024 bit.After the initial settings, we can count the mathematical operations in the hiding and extracting phases as listed in Table4. From Table 4, it can be seen that all mathematical operations are run over 1024 bit and 512 bit fields.Table 4
Number of mathematical operations in NLPAS.
HidingAInjectingBExtra+1024−bit841660×1024−bit0840mod1024−bit002mod512−bit002÷512−bit002−512−bit002+512−bit002Then, we tested the time costs consumed by these mathematical operations on the above laptop, and the average results of running them for 1,000,000 times are shown in Table5. From Table 5, it can be seen that the time costs of mathematical operations are at the μs level.Table 5
Time costs of mathematical operations (unit:μs).
+1024−bit0.22×1024−bit1.69mod1024−bit2.19mod512−bit0.98÷512−bit1.03−512−bit0.16+512−bit0.14Taking the results in Table5 into Table 4, we can get the computation costs of algorithms in NLPAS, as shown in Table 6. From Table 6, it can be seen that the computation cost of mathematical operations on the MU (i.e., time costs of HidingA and Extra) is much lower than that on the CS (i.e., InjectingB). Therefore, NLPAS is suitable for mobile crowdsourcing networks, where MU is with limited computation resources.Table 6
Computation costs of algorithms in NLPAS (unit:μs).
HidingAInjectingBExtra18.48178.489.00The time costs of the binary LeNet and the full-precision LeNet are shown in Table7, where the results are average values of running the feature extracting process for 1,000,000 times. From Table 7, it can be seen that the computation cost of feature extracting in NLPAS can be largely reduced by using the binary convolutional neural network instead of the full-precision convolutional neural network.Table 7
Computation costs of algorithms in NLPAS (unit:μs).
Binary LeNetFull-precision LeNet56.61435.2The above evaluation shows that NLPAS fulfills the fourth requirement listed in Section1 (i.e., the computation cost requirement).


## 5.3. Implementation of NLPAS

To make sure that NLPAS can work well, we implemented it. In our experimental environment, there were one laptop and one computer. The laptop acts as the MU, and the computer acts as the CS. The result shows that the total running time in the auditing protocol is approximately 0.3 ms. Therefore, NLPAS is feasible for being deployed in the real world.


## 6. Conclusions

In this paper, we have proposed a noninteractive lightweight privacy-preserving auditing protocol on images in mobile crowdsourcing networks called NLPAS. NLPAS allows the mobile user to audit images stored on the crowdsourcing server without downloading them. Moreover, to achieve high efficiency, this paper introduced the binary convolutional neural network technique to the newly proposed auditing protocol and designed a novel privacy-preserving Hamming distance computation algorithm using basic mathematical operations. Experimental results show that NLPAS is feasible for real-world applications.In this paper, we mainly focused on the privacy-preserving issue of the newly designed auditing protocol for mobile crowdsourcing networks. However, several more issues are to be addressed in the future. First, NLPAS does not consider the integrity of transmitted messages. Therefore, a new security protocol is needed to prevent these messages from being tampered by adversaries. Second, NLPAS used the binary convolutional neural network for extracting a binary vector from images. However, in many scenarios, feature vectors may be extracted using full-precision neural networks, which are not binarized. Therefore, a new technique is needed to convert the full-precision feature vector to a binarized one. To address these issues, future works are needed.



---

*Source: 8827364-2020-07-25.xml*