# Achieving Private and Fair Truth Discovery in Crowdsourcing Systems

Authors: Zhenya Wang; Xiang Cheng; Sen Su; Longhan Wang
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Nowadays, crowdsourcing has witnessed increasing popularity as it can be adopted to solve many challenging question-answering tasks. One of the most significant problems in crowdsourcing is truth discovery, which aims to find reliable information from conflict answers provided by different workers. Despite the effectiveness for providing reliable aggregated results, existing works on truth discovery either fall short of preserving the workers’ privacy or fail to consider the unfairness issue in their design. In light of this deficiency, we propose a novel private and fair truth discovery approach called PFTD, which is implemented by two non-colluding cloud servers and leverages the Paillier cryptosystem. This approach not only preserves the privacy of the answers of each worker, but also addresses the unfairness issue in crowdsourcing. Extensive experiments conducted on both real and synthetic datasets demonstrate the effectiveness of our proposed PFTD approach.


---

## Body

## 1. Introduction

The proliferation of information techniques [1] is pushing forward the paradigm of crowdsourcing [2–6], where a crowd of participant workers submit their answers for the tasks released by a data requester. However, the answers yielded by the workers are usually unreliable due to various reasons, such as poor sensor quality, lack of sensor calibration, background noise, and even the intent to deceive [7]. To this end, truth discovery, which aims to unearth the ground truth by taking into account the reliability degree of each participating worker, has received wide attention recently [8–12]. The common principle of truth discovery approaches is that a worker will be assigned a higher weight if their answer is closer to the aggregated results, and the answer of a worker will be counted more in the aggregation procedure if they have a higher weight [12].Despite the effectiveness of truth discovery, the privacy of the workers has always been an acute concern in crowdsourcing systems, since the answers may reveal sensitive information. For example, aggregating the medical data from workers helps discover the effects of new drugs but may also disclose their health conditions. Accordingly, a series of privacy-preserving truth discovery approaches have been proposed [13–22], where secure multiparty computation (MPC) [23–26] or differential privacy [15, 27–31] are employed to preserve the privacy of the participant workers.In addition to privacy concerns, unfairness issues may also arise in crowdsourcing systems. Due to various factors such as personal preference and cultural background, answers are highly prone to the worker’s biases [32, 33]. For instance, in a recent crowdsourcing study conducted on Amazon Mechanical Turk (AMT) [34] for assessment of recidivism prediction [35], the inferred truth is discriminated against the black defendants [35]. For this reason, Li et al. [22] proposed a fair truth discovery approach FairTD, which iteratively estimates worker’s bias, variance, weights, and truth and removes a particular bias to alleviate the discrimination.Our target in this paper is to develop a truth discovery approach that not only provides privacy protection for the answers of each worker, but also ensures fairness of the inferred truth. Conceptually one natural idea is to achieve this goal would be to run the FairTD [22] approach in the encrypted domain. However, there are several challenges in doing so. Firstly, FairTD involves many arithmetic operations such as addition and multiplication. Although we can encrypt the workers’ answers with a fully homomorphic encryption (FHE) cryptosystem [36] which allows these operations fulfilled in the encrypted domain, such a solution suffers from poor scalability due to the notoriously high inherent computation cost of FHE [37] and the iterative estimating process in FairTD, which hinders its real applications in (almost) all practical use cases. Secondly, in certain circumstances, simply applying cryptographic primitives to FairTD is unable to provide sufficient privacy protection for the participant workers (explained in Section 5.1, Step C1). Thirdly, fair truth estimating in FairTD requires selecting a bias from the constructive set with probability, which is non-trivial in the encrypted domain. Therefore, a tailored approach to our problem is needed to ensure privacy and fairness simultaneously.In light of this need, we propose a private and fair truth discovery approach named PFTD for crowdsourcing [22], which follows the two non-colluding cloud servers model like many prior arts [13, 17, 18, 20, 38–41]. In PFTD, we exploit the Paillier cryptosystem [42, 43] for encryption, which is a widely adopted partially homomorphic cryptosystem due to its additively homomorphic property and significantly higher efficiency compared to FHE. To prevent the cloud server from inferring the estimated truth while calculating the statistical disparity, we leverage the idea of random permutation to decouple the comparison results with the task ID. Moreover, since some necessary algorithmic operations in FairTD, such as multiplication, division, and sign checking, are not supported by the original Paillier cryptosystem, we bridge the technique of random perturbation with the Paillier cryptosystem in an elegant manner to fulfill these operations without impacting the calculation results. Furthermore, to reduce the disparity of the estimated truth, we design a method to securely select a constructive bias from the encrypted constructive set with probability. Through the interaction of the two cloud servers, the fair truth can be estimated without compromising the privacy of the participant workers. Last but not least, since all the ciphertext-based calculations are shifted to the cloud side, the computation and communication overhead of each participant worker is substantially reduced in PFTD.The main contributions of this paper are summarized as follows:(i)
We present a private and fair truth discovery (PFTD) approach for privately inferring the fair truth based on the answers submitted by the participant workers in crowdsourcing systems.(ii)
To calculate the statistical disparity without disclosing the transformed truth at each iteration, we design a private probability calculating method where random permutation is leveraged to decouple the comparison results with the task ID.(iii)
To fulfill the necessary algorithmic operations of FairTD in the encrypted domain, we integrate the idea of random perturbation with the Paillier cryptosystem, and design methods for secure multiplication, secure division, and secure sign checking. Moreover, we develop a method for securely selecting a constructive bias from the encrypted constructive set.(iv)
Privacy analysis shows that our PFTD approach can provide privacy protection for the workers’ sensitive information. Extensive experiments conducted on both synthetic and real datasets demonstrate that PFTD can effectively alleviate discrimination with low communication costs.The rest of this paper is organized as follows. In Section2, we discuss the related work. Section 3 provides preliminaries, including the FairTD approach and the Paillier additive homomorphic cryptosystem. Then, we introduce the problem formulation in Section 4. The details of our PFTD approach, along with its privacy analysis, are given in Section 5. Next, Section 6 presents the experiments. Finally, Section 7 draws the conclusions.


## 2. Related Work

As a hot topic for conflict resolution in crowdsourcing, the privacy issue in truth discovery has attracted significant attention, and various approaches based on cryptography [44–49] or differential privacy have been investigated. Miao et al. [16] proposed the first privacy-preserving approach called PPTD, which relies on the threshold homomorphic cryptosystem [50] to protect the confidentiality of the worker’s answers and weights. Xu et al. [51] leveraged the additive homomorphic privacy-preserving data aggregation and super-increasing sequence techniques to design an efficient privacy-preserving truth discovery approach. Zheng et al. [21] devised a privacy-aware truth discovery that integrates splitting encryption [41, 52] with homomorphic encryption to achieve higher worker side efficiency. Zheng et al. [17] proposed the first secure system framework for encrypted confidence-aware truth discovery [53] by combining garbled circuit [25] and homomorphic encryption. Miao et al. [20] adopted the two-server model and designed a lightweight truth discovery approach based on homomorphic encryption. Tang et al. [18] followed the two-server model and leveraged the garbled circuit [25] to design the first non-interactive approach. Li et al. [15] developed a two-layer mechanism for truth discovery where the privacy of the workers is guaranteed by deferentially private noise perturbation [54]. Tang et al. [13] presented a truth discovery approach based on additive homomorphic encryption and data perturbation technology to provide privacy protections for the workers.The last few years have witnessed an explosion of interest in algorithmic fairness. The mathematical definition of fairness generally falls into two categories: (1) group fairness which aims to achieve equality among populations differentiated by their protected attributes [55]; (2) individual fairness [55] which is based on the intuition that similar individuals should be treated similarly. In this paper, we follow [22] and consider the statistical disparity, a kind of group fairness. Existing approaches enforcing non-discrimination constraints in machine learning can be categorized into three groups: (1) preprocessing: eliminating the bias in the training data before training [56–59]; (2) in-processing: adjusting the machine learning model during the training process [22, 60–64]; (3) postprocessing: modifying a previously trained machine learning model to fulfill fairness requirements [65]. The FairTD [22] is an in-processing approach and the only known work focusing on the unfairness issue in crowdsourcing scenarios.Apart from the works above exploring privacy and fairness in isolation, some exceptions linking the two topics have also emerged recently. Jagielski et al. [66] and Mozannar et al. [67] proposed two simple yet effective algorithms that guarantee differential privacy [54] and equalized odds [65]. Lyu et al. [68] proposed the FDPDDL framework, which enables parties to derive deep learning models in a fair and private manner. Tran et al. [69] exploited Lagrangian duality to design neural networks that can accommodate fairness constraints while guaranteeing the privacy of sensitive attributes. Xu et al. [70] and Ding et al. [71] proposed training privacy-preserving and fair logistic regression model by making use of the functional mechanism [72]. Besides, a recent line of work has investigated the negative impact of differential privacy on algorithmic fairness [73–75]. In contrast to the above studies, our PFTD approach is the first work achieving private and fair truth discovery in crowdsourcing to the best of our knowledge.


## 3. Preliminaries

This section reviews the general procedure of fair truth discovery and the additive homomorphic cryptosystem. Refer to Table1 for a list of the frequently used notations in this paper.Table 1
Frequently used notations.
NotationsDescriptionMThe number of protected attributesNThe number of unprotected attributesA=aThe protected attributesA=a¯The unprotected attributesKThe number of workerspkThek-th workerxkThe answer vector of workerpkx∗The ground truth vectorwkThe weight of thek-th workerbkThe bias of thek-th workerσk2The variance of thek-th worker


### 3.1. Fair Truth Discovery

Truth discovery aims to aggregate answers collected from the participant workers to infer the ground truth (or truth) [9, 12, 22, 53]. In this paper, we take into account the unfairness issues inherent in crowdsourcing and resort to the FairTD approach in [22] for its effectiveness.We now formalize the definition of fair truth discovery. LetT=T1,…,TM;TM+1,…,TM+N be the tasks released by the data requester, where the first M tasks are associated with the protected attributes (i.e., the attributes in terms of which the prediction results may discriminate against, e.g., gender and race), while the last N tasks are associated with the unprotected attributes. These M+N tasks are allocated to a crow of participant workers P=p1,p2,…,pK. Let xk=xk1,…,xkM;xkM+1,…,xkM+N denote the answer provided by worker pk, where xkm,m∈1,M+N is the answer of worker pk to task Tm. The fair truth discovery aims to infer the truth x∗=x∗1,…,x∗M;x∗M+1,…,x∗M+N that satisfies θ-disparity based on the answers provided by the workers.Definition 1.
(θ-disparity [22, 55]). Let x∗ be the truth inferred from the worker answers by a truth discovery algorithm ℱ. Then, the inferred truth x∗ satisfies θ-disparity if Prx∗=1|A=a−Prx∗=1|A=a¯≤θ, where A=a and A=a¯ are the protected attributes and unprotected attributes, respectively. Here, θ∈0,1 is the predefined disparity threshold, which is used to bound the difference in the positive rate of the inferred truth between the protected group and unprotected group. The smaller θ is, the stronger fairness is guaranteed. When θ=1, there is no fairness requirement on the inferred truth.
Following [22], we model the answer provided by worker pk regarding the m-th task as xkm=x^m+N0,σk2+bkA=a, where x^m is the underline answer of the m-th task, N0,σk2 is a normal distribution with variance σk2 used to model the answer error around the bias, and bkA=a is the bias of worker pk on the protected attributes A=a. Following the literature [22, 32, 33], we assume the bias and the variance are independent of each other. We now describe the main procedure of FairTD (interested readers can refer to [22] for its details). Basically speaking, FairTD iteratively estimates the worker bias, variance, weights, and the fair truth and dynamically selects worker bias to be removed from the answers during truth inference. The FairTD approach starts with random initialization of worker bias, variance, and the estimated truth and then iteratively conducts the following steps until convergence.(1)
Constructive set building: the constructive set is composed of a series of constructive biases and is built asC=bkA=a|∀pk∈P,ckA=a<0, where ckA=a=bkA=a×ΨA=a is the disparity impact score of worker pk, and ΨA=a is the statistical disparity defined as(1)ΨA=a=Prx∗=1|A=a−Prx∗=1|A=a¯.(2)
Constructive bias selecting: a bias is randomly selected from the constructive set with probabilitypk=ckA=a/∑blA=a∈CclA=a.(3)
Fair truth estimating: the fair truth is estimated as(2)x∗m=xsm×ws+∑pk∈P\psxkm−bkA=a×wk,wherewk=1/σk2/∑wk∈W1/σk2 is the weight of worker pk, and ps is the worker whose bias is selected in the above step.(4)
Bias and variance updating: the worker bias and variance is updated as follows:(3)bkA=a=1TA=a∑tm∈TA=axkm−x∗m,σk2=1M+N∑a∈A∑tm∈TA=axkm−x∗m−bkA=a2,whereTA=a is a subset of tasks associated with the protected attributes A=a [22].


### 3.2. Partially Homomorphic Cryptosystem

The partially homomorphic cryptosystem is a probabilistic asymmetric encryption scheme for restricted computation over the ciphertexts. In this paper, we adopt the widely used Paillier additive homomorphic cryptosystem [42, 76], which consists of three algorithms:(i)
The key generation algorithmpk,sk=Genkeysize that generates the private/public key pair given a security parameter keysize.(ii)
The encryption algorithmc=Ench,pk that maps a plaintext h to a ciphertext c using the public key pk.(iii)
The decryption algorithmh=Decc,sk that decrypts a ciphertext c to a plaintext h using the secret key sk.Unlike multiplicatively homomorphic cryptosystems such as ElGamal and RSA [77], Paillier supports additive homomorphic encryption; i.e., multiplying two ciphertexts generates a new ciphertext such that its decryption yields the sum of the two original plaintexts. In the rest of this paper, we omit the public key pk in the encryption algorithm and represent Ench,pk as h for simplicity. Let ℳ denote the message space and h1,h2 be two plaintexts; the Paillier homomorphic cryptosystem has the following properties:(i)
Homomorphic addition: givenh1,h2∈ℳ, it holds that h1+h2=h1h2.(ii)
Homomorphic multiplication with constant: givenh1∈ℳ and a constant α, it holds that α⋅h1=h1α.Note that the data to be encrypted in PFTD may not be an integer, and we round it by multiplying a parameterR (a magnitude of 10) and at last the original data will be recovered by dividing the same parameter.


## 3.1. Fair Truth Discovery

Truth discovery aims to aggregate answers collected from the participant workers to infer the ground truth (or truth) [9, 12, 22, 53]. In this paper, we take into account the unfairness issues inherent in crowdsourcing and resort to the FairTD approach in [22] for its effectiveness.We now formalize the definition of fair truth discovery. LetT=T1,…,TM;TM+1,…,TM+N be the tasks released by the data requester, where the first M tasks are associated with the protected attributes (i.e., the attributes in terms of which the prediction results may discriminate against, e.g., gender and race), while the last N tasks are associated with the unprotected attributes. These M+N tasks are allocated to a crow of participant workers P=p1,p2,…,pK. Let xk=xk1,…,xkM;xkM+1,…,xkM+N denote the answer provided by worker pk, where xkm,m∈1,M+N is the answer of worker pk to task Tm. The fair truth discovery aims to infer the truth x∗=x∗1,…,x∗M;x∗M+1,…,x∗M+N that satisfies θ-disparity based on the answers provided by the workers.Definition 1.
(θ-disparity [22, 55]). Let x∗ be the truth inferred from the worker answers by a truth discovery algorithm ℱ. Then, the inferred truth x∗ satisfies θ-disparity if Prx∗=1|A=a−Prx∗=1|A=a¯≤θ, where A=a and A=a¯ are the protected attributes and unprotected attributes, respectively. Here, θ∈0,1 is the predefined disparity threshold, which is used to bound the difference in the positive rate of the inferred truth between the protected group and unprotected group. The smaller θ is, the stronger fairness is guaranteed. When θ=1, there is no fairness requirement on the inferred truth.
Following [22], we model the answer provided by worker pk regarding the m-th task as xkm=x^m+N0,σk2+bkA=a, where x^m is the underline answer of the m-th task, N0,σk2 is a normal distribution with variance σk2 used to model the answer error around the bias, and bkA=a is the bias of worker pk on the protected attributes A=a. Following the literature [22, 32, 33], we assume the bias and the variance are independent of each other. We now describe the main procedure of FairTD (interested readers can refer to [22] for its details). Basically speaking, FairTD iteratively estimates the worker bias, variance, weights, and the fair truth and dynamically selects worker bias to be removed from the answers during truth inference. The FairTD approach starts with random initialization of worker bias, variance, and the estimated truth and then iteratively conducts the following steps until convergence.(1)
Constructive set building: the constructive set is composed of a series of constructive biases and is built asC=bkA=a|∀pk∈P,ckA=a<0, where ckA=a=bkA=a×ΨA=a is the disparity impact score of worker pk, and ΨA=a is the statistical disparity defined as(1)ΨA=a=Prx∗=1|A=a−Prx∗=1|A=a¯.(2)
Constructive bias selecting: a bias is randomly selected from the constructive set with probabilitypk=ckA=a/∑blA=a∈CclA=a.(3)
Fair truth estimating: the fair truth is estimated as(2)x∗m=xsm×ws+∑pk∈P\psxkm−bkA=a×wk,wherewk=1/σk2/∑wk∈W1/σk2 is the weight of worker pk, and ps is the worker whose bias is selected in the above step.(4)
Bias and variance updating: the worker bias and variance is updated as follows:(3)bkA=a=1TA=a∑tm∈TA=axkm−x∗m,σk2=1M+N∑a∈A∑tm∈TA=axkm−x∗m−bkA=a2,whereTA=a is a subset of tasks associated with the protected attributes A=a [22].


## 3.2. Partially Homomorphic Cryptosystem

The partially homomorphic cryptosystem is a probabilistic asymmetric encryption scheme for restricted computation over the ciphertexts. In this paper, we adopt the widely used Paillier additive homomorphic cryptosystem [42, 76], which consists of three algorithms:(i)
The key generation algorithmpk,sk=Genkeysize that generates the private/public key pair given a security parameter keysize.(ii)
The encryption algorithmc=Ench,pk that maps a plaintext h to a ciphertext c using the public key pk.(iii)
The decryption algorithmh=Decc,sk that decrypts a ciphertext c to a plaintext h using the secret key sk.Unlike multiplicatively homomorphic cryptosystems such as ElGamal and RSA [77], Paillier supports additive homomorphic encryption; i.e., multiplying two ciphertexts generates a new ciphertext such that its decryption yields the sum of the two original plaintexts. In the rest of this paper, we omit the public key pk in the encryption algorithm and represent Ench,pk as h for simplicity. Let ℳ denote the message space and h1,h2 be two plaintexts; the Paillier homomorphic cryptosystem has the following properties:(i)
Homomorphic addition: givenh1,h2∈ℳ, it holds that h1+h2=h1h2.(ii)
Homomorphic multiplication with constant: givenh1∈ℳ and a constant α, it holds that α⋅h1=h1α.Note that the data to be encrypted in PFTD may not be an integer, and we round it by multiplying a parameterR (a magnitude of 10) and at last the original data will be recovered by dividing the same parameter.


## 4. Problem Formulation

Our system consists of three types of entities: the data requester, the participant workers, and the cloud servers. The data requester is a party who releases the crowdsourcing tasks via the crowdsourcing platform. The participant workers are the individuals willing to participate in the crowdsourcing and provide their respective answers. We consider the cloud entity consists of two cloud serversSA and SB, which are assigned with different roles:(1)
CloudSA: a cloud server with abundant resources that handles the majority of calculations in the encrypted domain.(2)
CloudSB: a cloud server that provides security services, such as public/private key generation, encryption, and decryption of the perturbed values.Note that such a two-server model has been widely adopted in recent privacy-aware literature [13, 17, 18, 20, 38–41], and we believe our adoption of this model to be in this trend.Like most of the prior arts [78–81], we consider the semi-honest model, where all the entities follow the protocol exactly as specified, yet they may keep all the protocol transcripts to infer private information related to the participating workers. Additionally, we assume there is no collusion between cloud SA and SB, which means the two servers will not collude outside the designed protocols. Note that the scenario where the participant workers maliciously manipulate their answers to disrupt the protocol is not the focus of this paper, and it can be addressed by using off-the-shelf techniques such as zero-knowledge proof [82–84]. We also remark that all the communications between the entities are secured via a secure transport protocol, such as SSL [85] or IPSec [86].The goal we target in this paper is to infer the fair truthx∗ accurately while satisfying the following desiderata:(i)
Privacy. The weights, answers, bias, and variance of each worker should not be disclosed to any entities in the system.(ii)
Fairness. The truthx∗ inferred based on the answers provided by the participant workers satisfies θ-disparity (Definition 1).


## 5. PFTD: Private and Fair Truth Discovery

Our approach follows the system workflow illustrated in Figure1. At a high level, the data requester first releases the crowdsourcing tasks to the participant workers. Then, the workers encrypt their answers with the public key generated by cloud SB and submit the encrypted answers to cloud SA. Next, cloud SA and SB collaboratively run the private and fair truth discovery (PFTD) approach to obtain the encrypted fair truth. Finally, cloud SB decrypts the fair truth and forward it to the data requester.Figure 1
System workflow of our PFTD approach.Our PFTD approach starts with the random initialization of the truthx∗=x∗1,…,x∗M;x∗M+1,…,x∗M+N, the worker bias bkA=ak=1K, and the worker variance σk2k=1K on cloud SA. After the initialization, the cloud SA encrypts them with the public key received from cloud SB. Subsequently, the two cloud servers collaboratively execute the following four subroutines to infer the fair truth: secure constructive set building, secure constructive bias selecting, secure fair truth estimating, and secure bias and variance updating. Following [22], we only consider binary labels (i.e., the inferred truth is binary) in this paper. Note that our results can be easily extended to non-binary labeling tasks. In what follows, we elaborate on the four subroutines in Section 5.1, Section 5.2, Section 5.3, and Section 5.4, respectively.


### 5.1. Secure Constructive Set Building

Step C1. In this step, cloudSA calculates the statistical disparity ΨA=a in the encrypted domain:(4)ΨA=a=Prx∗=1|A=a−Prx∗=1|A=a¯.Note that the valuesx∗mm=1M+N of the estimated truth obtained at each iteration may probably be floating-point numbers (see (2)). The common practice in the plaintext domain is to transform the estimated truth to binary data according to the following rule [22]:(5)x∗m>0.5=1,ifx∗m>0.5,0,otherwise.Then the two probabilities are calculated as follows:(6)Prx∗=1|A=a=∑m=1Mx∗m>0.5M,Prx∗=1|A¯=a=∑m=M+1M+Nx∗m>0.5N,Algorithm 1: Private probability calculating.
(i)
Input: the encrypted estimated truth x∗;(ii)
Output: the encrypted probability Prx∗=1|A=a;(1)
CloudSA generates a set of random numbers γmm=1M;(2)
CloudSA computes x˜∗mm=1M=x∗m+γmm=1M and 0.5˜mm=1M=0.5+γmm=1M;(3)
CloudSA permutes the pairs of x˜∗mm=1M and 0.5˜mm=1M as illustrated in Figure 2 and sends the permuted pairs to cloud SB;(4)
CloudSB decrypts x˜∗mm=1M and 0.5˜mm=1M and compares each pair of x˜∗m and 0.5˜m;(5)
CloudSB calculates ∑m=1Mx˜∗m>0.5˜m and sends the summation back to cloud SA;(6)
CloudSA calculates the encrypted probability Prx∗=1|A=a according to equations (8)–(11);(7)
return the encrypted probability Prx∗=1|A=aOne natural idea to adapt the above transformation to the encrypted domain may arise, i.e., directly run a secure comparison protocol [87, 88] to determine whether the value of the encrypted truth x∗m is greater than 0.5. However, the transformed values of the estimated truth will be inferred via the comparison results and thus incur privacy leakage. To this end, we design a private probability calculating method to compute the two encrypted probabilities Prx∗=1|A=a and Prx∗=1|A=a¯ without leaking the transformed values of the estimated truth.Figure 2
Illustration of the random permutation. The pairs ofx˜∗mm=1M and 0.5˜m=1M are randomly permuted to decouple with the task ID.Without loss of generality, we show how to calculatePrx∗=1|A=a. Firstly, cloud SA generates a set of random numbers γmm=1M and computes(7)x˜∗mm=1M=x∗m+γmm=1M,0.5˜mm=1M=0.5m+γmm=1M.Then, cloudSA randomly permutes the pairs of x˜∗mm=1M and 0.5˜mm=1M to decouple them with the task ID. Specifically, given the tasks Tmm=1M and the corresponding pair of numbers, cloud SA randomly confounds the order of the pair of numbers and then sends them to cloud SB. In this process, what the cloud SB receives are a series of pairs of numbers in a randomly permuted order. Therefore, cloud SB is unable to identify which task that a specific pair of numbers correspond to. Besides, since the number of pairs where x˜∗m is greater than 0.5˜m remains unchanged after permutation, the random permutation has no impact on the calculation result of Prx∗=1|A=a . Figure 2 illustrates the process of random permutation.Once receiving the permuted pairs ofx˜∗mm=1M and 0.5˜mm=1M, cloud SB compares them, computes the summation ∑m=1Mx˜∗m>0.5˜m, and sends the encrypted results back to cloud SA. Finally, cloud SA calculates the encrypted probability Prx∗=1|A=a as follows:(8)Prx∗=1|A=a,(9)=∑m=1Mx∗m>0.5M(10)=∑m=1Mx∗m>0.5m1/M,(11)=∑m=1Mx˜∗m>0.5˜m1/M.The procedure of private probability calculating method is given in Algorithm 1.Subsequently, based on the additively homomorphic property of the Paillier cryptosystem, cloudSA can calculate the encrypted statistical disparity ΨA=a as follows:(12)ΨA=a=Prx∗=1|A=a−Prx∗=1|A=a¯,=Prx∗=1|A=aPrx∗=1|A=a¯−1.Step C2. In this step, cloudSA calculates ckA=a=bkA=a⋅ΨA=a, i.e., the ciphertext of the disparity impact score for worker pk. Since homomorphic multiplication is not supported by the Paillier cryptosystem, we develop a method of secure multiplication via the interaction of the two cloud servers. In particular, cloud SA first generates two random numbers α1 and α2 and subtracts these two random numbers from bkA=a and ΨA=a to get the respective perturbed values:(13)b˜kA=a=bkA=a⋅−α1=bkA=a−α1,Ψ˜A=a=ΨA=a⋅−α2=ΨA=a−α2.Subsequently, cloudSA sends b˜kA=a and Ψ˜A=a to cloud SB. Then, cloud SB decrypts b˜kA=a and Ψ˜A=a and calculates the product b˜kA=a⋅Ψ˜A=a. Next, the encrypted result b˜kA=a⋅Ψ˜A=a is sent back to cloud SA. Finally, the encryption of the disparity impact score ckA=a can be obtained as follows:(14)ckA=a=bkA=a⋅ΨA=a,=b˜kA=a⋅Ψ˜A=abkA=aα1ΨA=aα2−α1⋅α2,=b˜kA=a⋅Ψ˜A=aα1⋅bkA=aα2⋅ΨA=a−α1⋅α2,=bkA=a−α1⋅ΨA=a−α2,α1⋅bkA=aα2⋅ΨA=a−α1⋅α2,=bkA=a−α1⋅ΨA=a−α2+α1⋅bkA=a+α2⋅ΨA=a−α1⋅α2.Step C3. AlthoughckA=a has been obtained, cloud SA is unable to build the encrypted constructive set C=bkA=a|∀pk∈P,ckA=a<0 directly, since checking the sign of ckA=a (i.e., whether ckA=a is positive or negative) is non-trivial. To this end, we propose a method for secure sign checking. Specifically, cloud SA first perturbs ckA=a with a random positive number βk and sends the perturbed result c˜kA=a=βk⋅ckA=a=ckA=aβk to cloud SB. Then cloud SB decrypts the received βk⋅ckA=a and determines whether c˜kA=a is greater or less than zero. Finally, the determining result sgnc˜kA=a is sent back to cloud SA and used to construct the encrypted constructive set as follows:(15)C=bkA=a|∀pk∈P,sgnc˜kA=a<0.Step S1. In this step, cloudSA calculates the encrypted probability pk=ckA=a/∑bkA=a∈CckA=a to securely select a bias from the encrypted constructive set with it. Note that the encrypted numerator and denominator of pk can be computed as(16)pknum=ckA=a=ckA=a−1,pkden=∑bkA=a∈CckA=a=∏bkA=a∈CckA=a−1.Since the Paillier cryptosystem does not support homomorphic division, we design a method cloudSA that interacts with SB to get the encrypted probability pk. In particular, cloud SA first generates two random numbers α3 and α4 and sends the perturbed value p˜knum=α3⋅pknum=pknumα3 and p˜kden=α4⋅pkden=pkdenα4 to cloud SB. Once receiving p˜knum and p˜kden, cloud SB decrypts them, calculates p˜knum/p˜kden=α3⋅pknum/α4⋅pkden, and sends the encrypted result p˜knum/p˜kden=α3⋅pknum/α4⋅pkden back to cloud SA. Finally, the encrypted probability pk can be calculated as follows:(17)pk=pknumpkden=p˜knump˜kdenα4/α3=α3⋅pknumα4⋅pkdenα4/α3.Step S2. In this step, cloudSA securely selects an encrypted bias bkA=a from the encrypted constructive set C with the probability pk. To achieve this, we design a method of secure selecting with probability. In particular, cloud SA first samples a threshold probability ps from the uniform distribution Uni0,1 and encrypts it. Then, cloud SA initializes the accumulative probability Pacc=0 that is used to store the sum of the probabilities traversed so far and encrypt it. Next, the encrypted accumulative probability Pacc is updated as Pacc=Pacc+pk=Paccpk and compared with the encrypted accept probability ps via the interaction with cloud SB. Finally, cloud SA selects the encrypted bias bkA=a, which corresponds to the first probability pk that makes the accumulated probability Pacc greater than ps. The details of the secure selecting with probability are presented in Algorithm 2.


### 5.2. Secure Constructive Bias Selecting

Algorithm 2: Secure selecting with probability
(i)
Input: the encrypted constructive set C, the encrypted probabilities pkk=1K;(ii)
Output: the securely selected encrypted bias bsA=a;(1)
CloudSA generates a threshold probability ps∼Uni0,1 and encrypts it;(2)
cloudSA initializes the accumulative probability Pacc=0 and encrypts it;(3)
k=1;(4)
repeat(5)
cloudSA updates the encrypted accumulative probability Pacc=Pacc+pk=Paccpk ;(6)
cloudSA generates a random number αk, calculates p˜s=ps+αk=psαk, and P˜acc=Pacc+αk=Paccαk and sends the results to cloud SB;(7)
cloudSB decrypts p˜s and P˜acc, compares p˜s=ps+αk and P˜acc=Pacc+αk, and returns the comparison result to cloud SA;(8)
k=k+1;(9)
until cloud SA receives from cloud SB that P˜acc>p˜s ;(10)
return the selected bias bsA=a=bkA=a


### 5.3. Secure Fair Truth Estimating

Step T1. In this step, cloudSA computes the encrypted weight wk=1/σk2/∑pk∈P1/σk2. Since the numerator 1/σk2 is the division of 1 and σk2, a similar way in Step S1 can be applied to calculate 1/σk2. Specifically, cloud SA first generates two random numbers α5 and α6. Then, the encrypted α5 and α6⋅σk2 are sent to cloud SB. Next, cloud SB decrypts the received α5 and α6⋅σk2, calculates α5/α6⋅σk2, and sends the encrypted result α5/α6⋅σk2 back to cloud SA. Finally, 1/σk2 can be calculated as(18)1σk2=α5α6⋅σk2α6/α5=α6α5⋅α5α6⋅σk2.Once obtaining1/σk2, the encrypted denominator ∑pk∈P1/σk2 can be calculated via the additively homomorphic property of the Paillier cryptosystem as follows:(19)∑pk∈P1/σk2=∏pk∈P1/σk2.Note that the weightwk is the division of 1/σk2 and ∑pk∈P1/σk2. Therefore, the method of calculating division can also be applied to get wk. We omit the details here for brevity.Step T2. In this step, cloudSA updates the encrypted fair truth x∗m as follows:(20)x∗m=ws⋅xsm+∑wk∈W\wswk⋅xkm−bkA=a,=ws⋅xsm∑wk∈W\wswk⋅xkm−bkA=a,=ws⋅xsm∏wk∈W\wswk⋅xkm−bkA=a.Here,ws is the worker whose bias is securely selected in Step S2. Besides, the calculation of the two products ws⋅xsm and wk⋅xkm−bkA=a can be fulfilled following the similar way of calculating ckA=a=rkA=a⋅ΨA=a in Step C2.


### 5.4. Secure Bias and Variance Updating

Step BV1. In this step, cloudSA updates the encrypted bias bkA=a as(21)bkA=a=1TA=a∑tk∈TA=axkm−x∗m,=∑tk∈TA=axkm−x∗m1/TA=a,=∏tk∈TA=axkm−x∗m1/TA=a,=∏tk∈TA=axkmx∗m−11/TA=a,whereTA=a is a subset of tasks associated with A = a [22].Step BV2. In this step, cloud updates the encrypted varianceσk2 as follows:(22)σk2=1M∑a∈A∑tm∈TA=axkm−x∗m−bkA=a2,(23)=∑a∈A∑tm∈TA=axkm−x∗m−bkA=a21/M,(24)=∏a∈A∏tm∈TA=axkm−x∗m−bkA=a21/M.Since the square operation is not supported by the Paillier cryptosystem, the itemxkm−x∗m−bkA=a2 in (24) cannot be calculated directly. Therefore, cloud SA interacts with cloud SB to obtain it. In particular, cloud SA first generates a random number α7 and calculates(25)xkm−x∗k−bkA=a+α7,=xkmx∗m−1bkA=a−1α7.Then, the resultxkm−x∗m−bkA=a+α7 is sent to cloud SB for decryption. Next, cloud SB calculates xkm−x∗m−bkA=a+α72 and sends the encrypted result(26)SQU˜=xkm−x∗m−bkA=a+α72,=xkm−x∗m−bkA=a2,xkm−x∗m−bkA=a2⋅α7α72.back to cloudSA. Finally, xkm−x∗m−bkA=a2 is calculated by cloud SA as follows:(27)xkm−x∗m−bkA=a2=,SQU˜xkm−x∗m−bkA=a2⋅α7−1α72−1.The above four subroutines are repeated iteratively until the convergence criterion is satisfied. It is worth noting that the data requester and the workers are not required to stay online during the execution of PFTD.


### 5.5. Privacy Analysis

Theorem 1.
Our PFTD approach is secure under semi-honest and non-colluding cloud serverSA and SB in that both of them learn nothing about the sensitive information of the participant workers.Proof. Given the semantic security of the Paillier cryptosystem [42, 76], the privacy guarantee of our PFTD approach can be proved by analyzing its concrete workflow, based on the modular sequential composition theorem [89, 90].Firstly, cloudSA receives the answers encrypted by the semantically secure Paillier cryptosystem, for which SA does not have the private key for decryption. Then, cloud SA operates over the ciphertexts of the answers and interact with cloud SB when necessary. Throughout the interaction with SB, what the cloud SA learns are the following ciphertexts: ∑m=1Mx˜∗m>0.5˜, ∑m=m+1M+Nx˜∗m>0.5˜, ΨA=a, ckA=a, bkA=a, C, wk, x∗m, bkA=a, σk2. According to the semantic security of the Paillier cryptosystem [42, 76], cloud SA is unable to infer the workers’ private information.Throughout the interaction withSA, what the cloud SB learns are the following perturbed values: x˜∗mm=1M+N, 0.5˜mm=1M+N, b˜kA=a, Ψ˜A=a, c˜kA=a,p˜knum,p˜kden, α6⋅σk2, and xkm−x∗m−bkA=a+α7. Since there is no collusion between SA and SB, cloud SB is unable to infer the private information of each worker.In summary, the weights, answers, bias, and variance of each participant worker will not be disclosed to other entities in our PFTD approach.


## 5.1. Secure Constructive Set Building

Step C1. In this step, cloudSA calculates the statistical disparity ΨA=a in the encrypted domain:(4)ΨA=a=Prx∗=1|A=a−Prx∗=1|A=a¯.Note that the valuesx∗mm=1M+N of the estimated truth obtained at each iteration may probably be floating-point numbers (see (2)). The common practice in the plaintext domain is to transform the estimated truth to binary data according to the following rule [22]:(5)x∗m>0.5=1,ifx∗m>0.5,0,otherwise.Then the two probabilities are calculated as follows:(6)Prx∗=1|A=a=∑m=1Mx∗m>0.5M,Prx∗=1|A¯=a=∑m=M+1M+Nx∗m>0.5N,Algorithm 1: Private probability calculating.
(i)
Input: the encrypted estimated truth x∗;(ii)
Output: the encrypted probability Prx∗=1|A=a;(1)
CloudSA generates a set of random numbers γmm=1M;(2)
CloudSA computes x˜∗mm=1M=x∗m+γmm=1M and 0.5˜mm=1M=0.5+γmm=1M;(3)
CloudSA permutes the pairs of x˜∗mm=1M and 0.5˜mm=1M as illustrated in Figure 2 and sends the permuted pairs to cloud SB;(4)
CloudSB decrypts x˜∗mm=1M and 0.5˜mm=1M and compares each pair of x˜∗m and 0.5˜m;(5)
CloudSB calculates ∑m=1Mx˜∗m>0.5˜m and sends the summation back to cloud SA;(6)
CloudSA calculates the encrypted probability Prx∗=1|A=a according to equations (8)–(11);(7)
return the encrypted probability Prx∗=1|A=aOne natural idea to adapt the above transformation to the encrypted domain may arise, i.e., directly run a secure comparison protocol [87, 88] to determine whether the value of the encrypted truth x∗m is greater than 0.5. However, the transformed values of the estimated truth will be inferred via the comparison results and thus incur privacy leakage. To this end, we design a private probability calculating method to compute the two encrypted probabilities Prx∗=1|A=a and Prx∗=1|A=a¯ without leaking the transformed values of the estimated truth.Figure 2
Illustration of the random permutation. The pairs ofx˜∗mm=1M and 0.5˜m=1M are randomly permuted to decouple with the task ID.Without loss of generality, we show how to calculatePrx∗=1|A=a. Firstly, cloud SA generates a set of random numbers γmm=1M and computes(7)x˜∗mm=1M=x∗m+γmm=1M,0.5˜mm=1M=0.5m+γmm=1M.Then, cloudSA randomly permutes the pairs of x˜∗mm=1M and 0.5˜mm=1M to decouple them with the task ID. Specifically, given the tasks Tmm=1M and the corresponding pair of numbers, cloud SA randomly confounds the order of the pair of numbers and then sends them to cloud SB. In this process, what the cloud SB receives are a series of pairs of numbers in a randomly permuted order. Therefore, cloud SB is unable to identify which task that a specific pair of numbers correspond to. Besides, since the number of pairs where x˜∗m is greater than 0.5˜m remains unchanged after permutation, the random permutation has no impact on the calculation result of Prx∗=1|A=a . Figure 2 illustrates the process of random permutation.Once receiving the permuted pairs ofx˜∗mm=1M and 0.5˜mm=1M, cloud SB compares them, computes the summation ∑m=1Mx˜∗m>0.5˜m, and sends the encrypted results back to cloud SA. Finally, cloud SA calculates the encrypted probability Prx∗=1|A=a as follows:(8)Prx∗=1|A=a,(9)=∑m=1Mx∗m>0.5M(10)=∑m=1Mx∗m>0.5m1/M,(11)=∑m=1Mx˜∗m>0.5˜m1/M.The procedure of private probability calculating method is given in Algorithm 1.Subsequently, based on the additively homomorphic property of the Paillier cryptosystem, cloudSA can calculate the encrypted statistical disparity ΨA=a as follows:(12)ΨA=a=Prx∗=1|A=a−Prx∗=1|A=a¯,=Prx∗=1|A=aPrx∗=1|A=a¯−1.Step C2. In this step, cloudSA calculates ckA=a=bkA=a⋅ΨA=a, i.e., the ciphertext of the disparity impact score for worker pk. Since homomorphic multiplication is not supported by the Paillier cryptosystem, we develop a method of secure multiplication via the interaction of the two cloud servers. In particular, cloud SA first generates two random numbers α1 and α2 and subtracts these two random numbers from bkA=a and ΨA=a to get the respective perturbed values:(13)b˜kA=a=bkA=a⋅−α1=bkA=a−α1,Ψ˜A=a=ΨA=a⋅−α2=ΨA=a−α2.Subsequently, cloudSA sends b˜kA=a and Ψ˜A=a to cloud SB. Then, cloud SB decrypts b˜kA=a and Ψ˜A=a and calculates the product b˜kA=a⋅Ψ˜A=a. Next, the encrypted result b˜kA=a⋅Ψ˜A=a is sent back to cloud SA. Finally, the encryption of the disparity impact score ckA=a can be obtained as follows:(14)ckA=a=bkA=a⋅ΨA=a,=b˜kA=a⋅Ψ˜A=abkA=aα1ΨA=aα2−α1⋅α2,=b˜kA=a⋅Ψ˜A=aα1⋅bkA=aα2⋅ΨA=a−α1⋅α2,=bkA=a−α1⋅ΨA=a−α2,α1⋅bkA=aα2⋅ΨA=a−α1⋅α2,=bkA=a−α1⋅ΨA=a−α2+α1⋅bkA=a+α2⋅ΨA=a−α1⋅α2.Step C3. AlthoughckA=a has been obtained, cloud SA is unable to build the encrypted constructive set C=bkA=a|∀pk∈P,ckA=a<0 directly, since checking the sign of ckA=a (i.e., whether ckA=a is positive or negative) is non-trivial. To this end, we propose a method for secure sign checking. Specifically, cloud SA first perturbs ckA=a with a random positive number βk and sends the perturbed result c˜kA=a=βk⋅ckA=a=ckA=aβk to cloud SB. Then cloud SB decrypts the received βk⋅ckA=a and determines whether c˜kA=a is greater or less than zero. Finally, the determining result sgnc˜kA=a is sent back to cloud SA and used to construct the encrypted constructive set as follows:(15)C=bkA=a|∀pk∈P,sgnc˜kA=a<0.Step S1. In this step, cloudSA calculates the encrypted probability pk=ckA=a/∑bkA=a∈CckA=a to securely select a bias from the encrypted constructive set with it. Note that the encrypted numerator and denominator of pk can be computed as(16)pknum=ckA=a=ckA=a−1,pkden=∑bkA=a∈CckA=a=∏bkA=a∈CckA=a−1.Since the Paillier cryptosystem does not support homomorphic division, we design a method cloudSA that interacts with SB to get the encrypted probability pk. In particular, cloud SA first generates two random numbers α3 and α4 and sends the perturbed value p˜knum=α3⋅pknum=pknumα3 and p˜kden=α4⋅pkden=pkdenα4 to cloud SB. Once receiving p˜knum and p˜kden, cloud SB decrypts them, calculates p˜knum/p˜kden=α3⋅pknum/α4⋅pkden, and sends the encrypted result p˜knum/p˜kden=α3⋅pknum/α4⋅pkden back to cloud SA. Finally, the encrypted probability pk can be calculated as follows:(17)pk=pknumpkden=p˜knump˜kdenα4/α3=α3⋅pknumα4⋅pkdenα4/α3.Step S2. In this step, cloudSA securely selects an encrypted bias bkA=a from the encrypted constructive set C with the probability pk. To achieve this, we design a method of secure selecting with probability. In particular, cloud SA first samples a threshold probability ps from the uniform distribution Uni0,1 and encrypts it. Then, cloud SA initializes the accumulative probability Pacc=0 that is used to store the sum of the probabilities traversed so far and encrypt it. Next, the encrypted accumulative probability Pacc is updated as Pacc=Pacc+pk=Paccpk and compared with the encrypted accept probability ps via the interaction with cloud SB. Finally, cloud SA selects the encrypted bias bkA=a, which corresponds to the first probability pk that makes the accumulated probability Pacc greater than ps. The details of the secure selecting with probability are presented in Algorithm 2.


## 5.2. Secure Constructive Bias Selecting

Algorithm 2: Secure selecting with probability
(i)
Input: the encrypted constructive set C, the encrypted probabilities pkk=1K;(ii)
Output: the securely selected encrypted bias bsA=a;(1)
CloudSA generates a threshold probability ps∼Uni0,1 and encrypts it;(2)
cloudSA initializes the accumulative probability Pacc=0 and encrypts it;(3)
k=1;(4)
repeat(5)
cloudSA updates the encrypted accumulative probability Pacc=Pacc+pk=Paccpk ;(6)
cloudSA generates a random number αk, calculates p˜s=ps+αk=psαk, and P˜acc=Pacc+αk=Paccαk and sends the results to cloud SB;(7)
cloudSB decrypts p˜s and P˜acc, compares p˜s=ps+αk and P˜acc=Pacc+αk, and returns the comparison result to cloud SA;(8)
k=k+1;(9)
until cloud SA receives from cloud SB that P˜acc>p˜s ;(10)
return the selected bias bsA=a=bkA=a


## 5.3. Secure Fair Truth Estimating

Step T1. In this step, cloudSA computes the encrypted weight wk=1/σk2/∑pk∈P1/σk2. Since the numerator 1/σk2 is the division of 1 and σk2, a similar way in Step S1 can be applied to calculate 1/σk2. Specifically, cloud SA first generates two random numbers α5 and α6. Then, the encrypted α5 and α6⋅σk2 are sent to cloud SB. Next, cloud SB decrypts the received α5 and α6⋅σk2, calculates α5/α6⋅σk2, and sends the encrypted result α5/α6⋅σk2 back to cloud SA. Finally, 1/σk2 can be calculated as(18)1σk2=α5α6⋅σk2α6/α5=α6α5⋅α5α6⋅σk2.Once obtaining1/σk2, the encrypted denominator ∑pk∈P1/σk2 can be calculated via the additively homomorphic property of the Paillier cryptosystem as follows:(19)∑pk∈P1/σk2=∏pk∈P1/σk2.Note that the weightwk is the division of 1/σk2 and ∑pk∈P1/σk2. Therefore, the method of calculating division can also be applied to get wk. We omit the details here for brevity.Step T2. In this step, cloudSA updates the encrypted fair truth x∗m as follows:(20)x∗m=ws⋅xsm+∑wk∈W\wswk⋅xkm−bkA=a,=ws⋅xsm∑wk∈W\wswk⋅xkm−bkA=a,=ws⋅xsm∏wk∈W\wswk⋅xkm−bkA=a.Here,ws is the worker whose bias is securely selected in Step S2. Besides, the calculation of the two products ws⋅xsm and wk⋅xkm−bkA=a can be fulfilled following the similar way of calculating ckA=a=rkA=a⋅ΨA=a in Step C2.


## 5.4. Secure Bias and Variance Updating

Step BV1. In this step, cloudSA updates the encrypted bias bkA=a as(21)bkA=a=1TA=a∑tk∈TA=axkm−x∗m,=∑tk∈TA=axkm−x∗m1/TA=a,=∏tk∈TA=axkm−x∗m1/TA=a,=∏tk∈TA=axkmx∗m−11/TA=a,whereTA=a is a subset of tasks associated with A = a [22].Step BV2. In this step, cloud updates the encrypted varianceσk2 as follows:(22)σk2=1M∑a∈A∑tm∈TA=axkm−x∗m−bkA=a2,(23)=∑a∈A∑tm∈TA=axkm−x∗m−bkA=a21/M,(24)=∏a∈A∏tm∈TA=axkm−x∗m−bkA=a21/M.Since the square operation is not supported by the Paillier cryptosystem, the itemxkm−x∗m−bkA=a2 in (24) cannot be calculated directly. Therefore, cloud SA interacts with cloud SB to obtain it. In particular, cloud SA first generates a random number α7 and calculates(25)xkm−x∗k−bkA=a+α7,=xkmx∗m−1bkA=a−1α7.Then, the resultxkm−x∗m−bkA=a+α7 is sent to cloud SB for decryption. Next, cloud SB calculates xkm−x∗m−bkA=a+α72 and sends the encrypted result(26)SQU˜=xkm−x∗m−bkA=a+α72,=xkm−x∗m−bkA=a2,xkm−x∗m−bkA=a2⋅α7α72.back to cloudSA. Finally, xkm−x∗m−bkA=a2 is calculated by cloud SA as follows:(27)xkm−x∗m−bkA=a2=,SQU˜xkm−x∗m−bkA=a2⋅α7−1α72−1.The above four subroutines are repeated iteratively until the convergence criterion is satisfied. It is worth noting that the data requester and the workers are not required to stay online during the execution of PFTD.


## 5.5. Privacy Analysis

Theorem 1.
Our PFTD approach is secure under semi-honest and non-colluding cloud serverSA and SB in that both of them learn nothing about the sensitive information of the participant workers.Proof. Given the semantic security of the Paillier cryptosystem [42, 76], the privacy guarantee of our PFTD approach can be proved by analyzing its concrete workflow, based on the modular sequential composition theorem [89, 90].Firstly, cloudSA receives the answers encrypted by the semantically secure Paillier cryptosystem, for which SA does not have the private key for decryption. Then, cloud SA operates over the ciphertexts of the answers and interact with cloud SB when necessary. Throughout the interaction with SB, what the cloud SA learns are the following ciphertexts: ∑m=1Mx˜∗m>0.5˜, ∑m=m+1M+Nx˜∗m>0.5˜, ΨA=a, ckA=a, bkA=a, C, wk, x∗m, bkA=a, σk2. According to the semantic security of the Paillier cryptosystem [42, 76], cloud SA is unable to infer the workers’ private information.Throughout the interaction withSA, what the cloud SB learns are the following perturbed values: x˜∗mm=1M+N, 0.5˜mm=1M+N, b˜kA=a, Ψ˜A=a, c˜kA=a,p˜knum,p˜kden, α6⋅σk2, and xkm−x∗m−bkA=a+α7. Since there is no collusion between SA and SB, cloud SB is unable to infer the private information of each worker.In summary, the weights, answers, bias, and variance of each participant worker will not be disclosed to other entities in our PFTD approach.


## 6. Experiments

### 6.1. Implementation

In this section, we conduct extensive evaluations of a proof-of-concept implementation of our PFTD approach. Since the problem of private and fair truth discovery is still nearly unexplored, we adopt the representative privacy-preserving truth discovery approach PPTD [91] as our baseline. Unless otherwise specified, the results are averaged over five runs.All the experiments are implemented on a server with Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40 GHz× 24 and 96 GB of RAM, running Ubuntu 16.04 LTS. Implementation of the PFTD and the PPTD was done in Python language. For cryptographic primitives, we use the Paillier library (https://pypi.org/project/phe/) for additively homomorphic encryption, and the input size of the key generation algorithm is set to 1024 bits. The rounding parameter R to round the fractional data is set to 107.


### 6.2. Performance on Synthetic Dataset

In this section, we evaluate the performance of the proposed PFTD approach on a synthetic dataset in terms of accuracy, convergence, and fairness guarantee. In this synthetic dataset, the ground truthx^m for each task Tm is generated from a uniform distribution Uni0.3,0.7. The answer of each worker pk for task Tm is then generated as xkm=x^m+bkA=a+N0,σk2, where the bias bkA=a follows N0,0.4 and the standard deviation σk follows N0.01,5. Finally, the answer is transformed to a binary value (xkm=1 if xkm>0.5, and vice versa).


#### 6.2.1. Accuracy

In this part, we compare the accuracy of the proposed PFTD approach with PPTD, which is defined as the percentage of matched values between the estimated truth and the ground truth. Here, the number of tasks is fixed as 30, and we vary the number of the participant workers from 10 to 100. As illustrated in Figure3, the accuracy of our PFTD approach is the same as FairTD regardless of the number of workers, which verifies the correctness of its design and execution. Additionally, the accuracy of PFTD has a slight decrease compared with the fairness-free approach PPTD. This meets our expectations since there exists a trade-off between accuracy and fairness guarantee on the synthetic dataset [22].Figure 3
Accuracy with respect to the number of workers for the synthetic dataset.


#### 6.2.2. Convergence

In this part, we investigate the convergence performance of our proposed PFTD approach. In this set of experiments, the number of workers is set to 100, and the disparity thresholdθ is set to 0.05. In addition, we randomly initialize the truth with five different values. We inspect the evolution of the convergence value defined as x∗t−x∗t−12, where x∗tt≥1 is the vector of the estimated truth at the t-th iteration. As shown in Figure 4, the convergence value of PFTD decreases immediately within several iterations and then reaches a relatively stable stage, which validates its ability to converge quickly.Figure 4
Convergence performance for the synthetic dataset.


#### 6.2.3. Fairness Guarantee

Figure5 depicts the comparison results of the proposed PFTD approach and PPTD in terms of fairness guarantee. The data points for the PPTD are obtained on five different synthetic datasets. As can be seen, the disparity of the inferred truth of PFTD is lower than PFTD for a specific accuracy, which verifies its capability to alleviate the discrimination on the protected attributes. Besides, it can be observed that there exists a trade-off between the disparity of the estimated truth and the accuracy in PFTD on the synthetic dataset. We must note that this is consistent with the results in [22] and further verifies that the ciphertexts computation in PFTD has almost no impact on the fair truth estimation.Figure 5
Fairness guarantee with respect to accuracy for the synthetic dataset.


### 6.3. Performance on Real Dataset

In this section, we evaluate the performance of the proposed PFTD approach on the crowd judgment dataset (https://farid.berkeley.edu/downloads/publications/scienceadvances17/). This dataset contains the answers collected from the workers on Amazon Mechanical Turk (AMT). Each worker received a description of a defendant, including sex, age, and previous criminal history and predicted whether this defendant would recidivate within two years.


#### 6.3.1. Accuracy

In this part, we compare the accuracy of the proposed PFTD approach with PPTD. In this set of experiments, the number of tasks is set to 867, and the disparity thresholdθ is set to 0.05. The results are provided in Figure 6. As can be seen, the PFTD approach exhibits higher accuracy than PPTD in this setting. This is somewhat counterintuitive since there exists a trade-off between accuracy and fairness on the synthetic dataset (Section 6.2). However, we note that this phenomenon is consistent with the results in [22], and the authors explained that bias removal brings a positive impact on the accuracy within a specific range of disparity threshold.Figure 6
Accuracy with respect to the number of workers for the real dataset.


#### 6.3.2. Convergence

In this part, we investigate the convergence of our proposed PFTD approach. In this set of experiments, the number of tasks is set to 867, while the number of workers is set to 30. The convergence valuex∗t−x∗t−12 is also utilized to evaluate the convergence here. As shown in Figure 7, the convergence value drops to a relatively low value quickly within several iterations.Figure 7
Convergence performance for the real dataset.


#### 6.3.3. Fairness Guarantee

Figure8 reveals the fairness guarantee of the PFTD approach and PPTD with respect to accuracy. The datapoints for the PPTD are obtained on five random subsets of the original real dataset. We can see that the truth disparity of PFTD is lower than PPTD for a given accuracy, which again verifies its capability of discrimination alleviating. Additionally, the fairness guarantee of PFTD also improves on the real dataset as the accuracy increases, which is consistent with the results in [22].Figure 8
Fairness guarantee with respect to accuracy for the real dataset.


### 6.4. Resources Required on the Worker Side

#### 6.4.1. Computation Overhead

In this part, we evaluate the computation performance on the worker side. As shown in Figure9, the running time of PFTD is much less than that of PPTD. This makes sense because the workers in PPTD not only undertake the encryption and decryption operation, but also participate in the computations in the encrypted domain. In contrast, the workers in PFTD are only required to encrypt their answers in the initialization phase.Figure 9
Computation overhead on the worker side.


#### 6.4.2. Communication Overhead

Table2 provides a comparison of the communication overhead between the proposed PFTD approach and PPTD. As can be seen, the communication overhead of PFTD is consistently less than PPTD with the increase in the number of tasks. The reason is that in PFTD the workers do not need to be involved in the iteration phase and only need to upload the encrypted answers in the initialization phase.Table 2
Communication overhead (KB) on the worker side.
Number of tasks3691215PPTD4.589.1713.7518.3422.92PFTD3.436.8610.3013.7317.16Number of tasks1821242730PPTD27.5032.0936.6741.2645.84PFTD20.5924.0227.4630.8934.32


### 6.5. Resources Required on the Server Side

#### 6.5.1. Computation Overhead

Figure10 reports the computation overhead at the server side. We can see that the computation cost imposed on cloud SA is less than the cloud server in PPTD. This is because the decryption and encryption of the threshold Paillier cryptosystem performed by the server in PPTD are more involved than the Paillier cryptosystem. Notably, cloud SB undertakes a bit more computation due to the engaged encryption and decryption operations. However, this should not be an acute concern or significantly restrict the applicability of our PFTD approach. In practice, cloud SB is typically implemented by a cloud server with abundant computation resources. In addition, compared to PPTD, our PFTD approach provides fairness guarantee and does not require any worker to stay online during the iteration phase.Figure 10
Computation overhead on the server side.


#### 6.5.2. Communication Overhead

Table3 reveals the communication overhead at the server side. As can be seen, PFTD exhibits relatively larger communication overhead compared to PPTD. This meets our expectations since the truth inference under the constraint of fairness is much more involved. However, we believe this would not greatly constrain the applicability of PFTD. In most cases, the current cloud infrastructure allows network communication with large bandwidth (from 1,000 Mbps to 14,000 Mbps), and therefore the data can be transmitted effectively. Besides, the communication channel between SA and SB is much more reliable than that between the workers and SA, since both SA and SB usually have reliable network infrastructures.Table 3
Communication overhead (KB) on the server side.
Number of workers3691215PPTD0.961.832.703.564.31PFTDSA⟶SB5.4011.2716.2619.3825.30SB⟶SA2.865.958.6410.5413.62Number of data workers1821242730PPTD5.166.016.847.698.53PFTDSA⟶SB27.5534.9140.8440.8449.52SB⟶SA15.0518.8421.9822.3626.79


## 6.1. Implementation

In this section, we conduct extensive evaluations of a proof-of-concept implementation of our PFTD approach. Since the problem of private and fair truth discovery is still nearly unexplored, we adopt the representative privacy-preserving truth discovery approach PPTD [91] as our baseline. Unless otherwise specified, the results are averaged over five runs.All the experiments are implemented on a server with Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40 GHz× 24 and 96 GB of RAM, running Ubuntu 16.04 LTS. Implementation of the PFTD and the PPTD was done in Python language. For cryptographic primitives, we use the Paillier library (https://pypi.org/project/phe/) for additively homomorphic encryption, and the input size of the key generation algorithm is set to 1024 bits. The rounding parameter R to round the fractional data is set to 107.


## 6.2. Performance on Synthetic Dataset

In this section, we evaluate the performance of the proposed PFTD approach on a synthetic dataset in terms of accuracy, convergence, and fairness guarantee. In this synthetic dataset, the ground truthx^m for each task Tm is generated from a uniform distribution Uni0.3,0.7. The answer of each worker pk for task Tm is then generated as xkm=x^m+bkA=a+N0,σk2, where the bias bkA=a follows N0,0.4 and the standard deviation σk follows N0.01,5. Finally, the answer is transformed to a binary value (xkm=1 if xkm>0.5, and vice versa).


### 6.2.1. Accuracy

In this part, we compare the accuracy of the proposed PFTD approach with PPTD, which is defined as the percentage of matched values between the estimated truth and the ground truth. Here, the number of tasks is fixed as 30, and we vary the number of the participant workers from 10 to 100. As illustrated in Figure3, the accuracy of our PFTD approach is the same as FairTD regardless of the number of workers, which verifies the correctness of its design and execution. Additionally, the accuracy of PFTD has a slight decrease compared with the fairness-free approach PPTD. This meets our expectations since there exists a trade-off between accuracy and fairness guarantee on the synthetic dataset [22].Figure 3
Accuracy with respect to the number of workers for the synthetic dataset.


### 6.2.2. Convergence

In this part, we investigate the convergence performance of our proposed PFTD approach. In this set of experiments, the number of workers is set to 100, and the disparity thresholdθ is set to 0.05. In addition, we randomly initialize the truth with five different values. We inspect the evolution of the convergence value defined as x∗t−x∗t−12, where x∗tt≥1 is the vector of the estimated truth at the t-th iteration. As shown in Figure 4, the convergence value of PFTD decreases immediately within several iterations and then reaches a relatively stable stage, which validates its ability to converge quickly.Figure 4
Convergence performance for the synthetic dataset.


### 6.2.3. Fairness Guarantee

Figure5 depicts the comparison results of the proposed PFTD approach and PPTD in terms of fairness guarantee. The data points for the PPTD are obtained on five different synthetic datasets. As can be seen, the disparity of the inferred truth of PFTD is lower than PFTD for a specific accuracy, which verifies its capability to alleviate the discrimination on the protected attributes. Besides, it can be observed that there exists a trade-off between the disparity of the estimated truth and the accuracy in PFTD on the synthetic dataset. We must note that this is consistent with the results in [22] and further verifies that the ciphertexts computation in PFTD has almost no impact on the fair truth estimation.Figure 5
Fairness guarantee with respect to accuracy for the synthetic dataset.


## 6.2.1. Accuracy

In this part, we compare the accuracy of the proposed PFTD approach with PPTD, which is defined as the percentage of matched values between the estimated truth and the ground truth. Here, the number of tasks is fixed as 30, and we vary the number of the participant workers from 10 to 100. As illustrated in Figure3, the accuracy of our PFTD approach is the same as FairTD regardless of the number of workers, which verifies the correctness of its design and execution. Additionally, the accuracy of PFTD has a slight decrease compared with the fairness-free approach PPTD. This meets our expectations since there exists a trade-off between accuracy and fairness guarantee on the synthetic dataset [22].Figure 3
Accuracy with respect to the number of workers for the synthetic dataset.


## 6.2.2. Convergence

In this part, we investigate the convergence performance of our proposed PFTD approach. In this set of experiments, the number of workers is set to 100, and the disparity thresholdθ is set to 0.05. In addition, we randomly initialize the truth with five different values. We inspect the evolution of the convergence value defined as x∗t−x∗t−12, where x∗tt≥1 is the vector of the estimated truth at the t-th iteration. As shown in Figure 4, the convergence value of PFTD decreases immediately within several iterations and then reaches a relatively stable stage, which validates its ability to converge quickly.Figure 4
Convergence performance for the synthetic dataset.


## 6.2.3. Fairness Guarantee

Figure5 depicts the comparison results of the proposed PFTD approach and PPTD in terms of fairness guarantee. The data points for the PPTD are obtained on five different synthetic datasets. As can be seen, the disparity of the inferred truth of PFTD is lower than PFTD for a specific accuracy, which verifies its capability to alleviate the discrimination on the protected attributes. Besides, it can be observed that there exists a trade-off between the disparity of the estimated truth and the accuracy in PFTD on the synthetic dataset. We must note that this is consistent with the results in [22] and further verifies that the ciphertexts computation in PFTD has almost no impact on the fair truth estimation.Figure 5
Fairness guarantee with respect to accuracy for the synthetic dataset.


## 6.3. Performance on Real Dataset

In this section, we evaluate the performance of the proposed PFTD approach on the crowd judgment dataset (https://farid.berkeley.edu/downloads/publications/scienceadvances17/). This dataset contains the answers collected from the workers on Amazon Mechanical Turk (AMT). Each worker received a description of a defendant, including sex, age, and previous criminal history and predicted whether this defendant would recidivate within two years.


### 6.3.1. Accuracy

In this part, we compare the accuracy of the proposed PFTD approach with PPTD. In this set of experiments, the number of tasks is set to 867, and the disparity thresholdθ is set to 0.05. The results are provided in Figure 6. As can be seen, the PFTD approach exhibits higher accuracy than PPTD in this setting. This is somewhat counterintuitive since there exists a trade-off between accuracy and fairness on the synthetic dataset (Section 6.2). However, we note that this phenomenon is consistent with the results in [22], and the authors explained that bias removal brings a positive impact on the accuracy within a specific range of disparity threshold.Figure 6
Accuracy with respect to the number of workers for the real dataset.


### 6.3.2. Convergence

In this part, we investigate the convergence of our proposed PFTD approach. In this set of experiments, the number of tasks is set to 867, while the number of workers is set to 30. The convergence valuex∗t−x∗t−12 is also utilized to evaluate the convergence here. As shown in Figure 7, the convergence value drops to a relatively low value quickly within several iterations.Figure 7
Convergence performance for the real dataset.


### 6.3.3. Fairness Guarantee

Figure8 reveals the fairness guarantee of the PFTD approach and PPTD with respect to accuracy. The datapoints for the PPTD are obtained on five random subsets of the original real dataset. We can see that the truth disparity of PFTD is lower than PPTD for a given accuracy, which again verifies its capability of discrimination alleviating. Additionally, the fairness guarantee of PFTD also improves on the real dataset as the accuracy increases, which is consistent with the results in [22].Figure 8
Fairness guarantee with respect to accuracy for the real dataset.


## 6.3.1. Accuracy

In this part, we compare the accuracy of the proposed PFTD approach with PPTD. In this set of experiments, the number of tasks is set to 867, and the disparity thresholdθ is set to 0.05. The results are provided in Figure 6. As can be seen, the PFTD approach exhibits higher accuracy than PPTD in this setting. This is somewhat counterintuitive since there exists a trade-off between accuracy and fairness on the synthetic dataset (Section 6.2). However, we note that this phenomenon is consistent with the results in [22], and the authors explained that bias removal brings a positive impact on the accuracy within a specific range of disparity threshold.Figure 6
Accuracy with respect to the number of workers for the real dataset.


## 6.3.2. Convergence

In this part, we investigate the convergence of our proposed PFTD approach. In this set of experiments, the number of tasks is set to 867, while the number of workers is set to 30. The convergence valuex∗t−x∗t−12 is also utilized to evaluate the convergence here. As shown in Figure 7, the convergence value drops to a relatively low value quickly within several iterations.Figure 7
Convergence performance for the real dataset.


## 6.3.3. Fairness Guarantee

Figure8 reveals the fairness guarantee of the PFTD approach and PPTD with respect to accuracy. The datapoints for the PPTD are obtained on five random subsets of the original real dataset. We can see that the truth disparity of PFTD is lower than PPTD for a given accuracy, which again verifies its capability of discrimination alleviating. Additionally, the fairness guarantee of PFTD also improves on the real dataset as the accuracy increases, which is consistent with the results in [22].Figure 8
Fairness guarantee with respect to accuracy for the real dataset.


## 6.4. Resources Required on the Worker Side

### 6.4.1. Computation Overhead

In this part, we evaluate the computation performance on the worker side. As shown in Figure9, the running time of PFTD is much less than that of PPTD. This makes sense because the workers in PPTD not only undertake the encryption and decryption operation, but also participate in the computations in the encrypted domain. In contrast, the workers in PFTD are only required to encrypt their answers in the initialization phase.Figure 9
Computation overhead on the worker side.


### 6.4.2. Communication Overhead

Table2 provides a comparison of the communication overhead between the proposed PFTD approach and PPTD. As can be seen, the communication overhead of PFTD is consistently less than PPTD with the increase in the number of tasks. The reason is that in PFTD the workers do not need to be involved in the iteration phase and only need to upload the encrypted answers in the initialization phase.Table 2
Communication overhead (KB) on the worker side.
Number of tasks3691215PPTD4.589.1713.7518.3422.92PFTD3.436.8610.3013.7317.16Number of tasks1821242730PPTD27.5032.0936.6741.2645.84PFTD20.5924.0227.4630.8934.32


## 6.4.1. Computation Overhead

In this part, we evaluate the computation performance on the worker side. As shown in Figure9, the running time of PFTD is much less than that of PPTD. This makes sense because the workers in PPTD not only undertake the encryption and decryption operation, but also participate in the computations in the encrypted domain. In contrast, the workers in PFTD are only required to encrypt their answers in the initialization phase.Figure 9
Computation overhead on the worker side.


## 6.4.2. Communication Overhead

Table2 provides a comparison of the communication overhead between the proposed PFTD approach and PPTD. As can be seen, the communication overhead of PFTD is consistently less than PPTD with the increase in the number of tasks. The reason is that in PFTD the workers do not need to be involved in the iteration phase and only need to upload the encrypted answers in the initialization phase.Table 2
Communication overhead (KB) on the worker side.
Number of tasks3691215PPTD4.589.1713.7518.3422.92PFTD3.436.8610.3013.7317.16Number of tasks1821242730PPTD27.5032.0936.6741.2645.84PFTD20.5924.0227.4630.8934.32


## 6.5. Resources Required on the Server Side

### 6.5.1. Computation Overhead

Figure10 reports the computation overhead at the server side. We can see that the computation cost imposed on cloud SA is less than the cloud server in PPTD. This is because the decryption and encryption of the threshold Paillier cryptosystem performed by the server in PPTD are more involved than the Paillier cryptosystem. Notably, cloud SB undertakes a bit more computation due to the engaged encryption and decryption operations. However, this should not be an acute concern or significantly restrict the applicability of our PFTD approach. In practice, cloud SB is typically implemented by a cloud server with abundant computation resources. In addition, compared to PPTD, our PFTD approach provides fairness guarantee and does not require any worker to stay online during the iteration phase.Figure 10
Computation overhead on the server side.


### 6.5.2. Communication Overhead

Table3 reveals the communication overhead at the server side. As can be seen, PFTD exhibits relatively larger communication overhead compared to PPTD. This meets our expectations since the truth inference under the constraint of fairness is much more involved. However, we believe this would not greatly constrain the applicability of PFTD. In most cases, the current cloud infrastructure allows network communication with large bandwidth (from 1,000 Mbps to 14,000 Mbps), and therefore the data can be transmitted effectively. Besides, the communication channel between SA and SB is much more reliable than that between the workers and SA, since both SA and SB usually have reliable network infrastructures.Table 3
Communication overhead (KB) on the server side.
Number of workers3691215PPTD0.961.832.703.564.31PFTDSA⟶SB5.4011.2716.2619.3825.30SB⟶SA2.865.958.6410.5413.62Number of data workers1821242730PPTD5.166.016.847.698.53PFTDSA⟶SB27.5534.9140.8440.8449.52SB⟶SA15.0518.8421.9822.3626.79


## 6.5.1. Computation Overhead

Figure10 reports the computation overhead at the server side. We can see that the computation cost imposed on cloud SA is less than the cloud server in PPTD. This is because the decryption and encryption of the threshold Paillier cryptosystem performed by the server in PPTD are more involved than the Paillier cryptosystem. Notably, cloud SB undertakes a bit more computation due to the engaged encryption and decryption operations. However, this should not be an acute concern or significantly restrict the applicability of our PFTD approach. In practice, cloud SB is typically implemented by a cloud server with abundant computation resources. In addition, compared to PPTD, our PFTD approach provides fairness guarantee and does not require any worker to stay online during the iteration phase.Figure 10
Computation overhead on the server side.


## 6.5.2. Communication Overhead

Table3 reveals the communication overhead at the server side. As can be seen, PFTD exhibits relatively larger communication overhead compared to PPTD. This meets our expectations since the truth inference under the constraint of fairness is much more involved. However, we believe this would not greatly constrain the applicability of PFTD. In most cases, the current cloud infrastructure allows network communication with large bandwidth (from 1,000 Mbps to 14,000 Mbps), and therefore the data can be transmitted effectively. Besides, the communication channel between SA and SB is much more reliable than that between the workers and SA, since both SA and SB usually have reliable network infrastructures.Table 3
Communication overhead (KB) on the server side.
Number of workers3691215PPTD0.961.832.703.564.31PFTDSA⟶SB5.4011.2716.2619.3825.30SB⟶SA2.865.958.6410.5413.62Number of data workers1821242730PPTD5.166.016.847.698.53PFTDSA⟶SB27.5534.9140.8440.8449.52SB⟶SA15.0518.8421.9822.3626.79


## 7. Conclusion and Future Works

In this paper, to address the privacy and unfairness concerns in crowdsourcing, we propose a private and fair truth discovery approach named PFTD. The key idea of PFTD is to leverage the additively homomorphic cryptosystem and random perturbation to estimate the fair truth without compromising the privacy of the participant workers. Our approach imposes little overhead on the worker side and removes the online requirement of the participant workers. Privacy analysis shows that the sensitive information of the participant worker is well protected throughout the process of fair truth discovery. Extensive experiments conducted on both the synthetic and real dataset verify the effectiveness of the proposed PFTD approach.At the end of this paper, we wish to discuss the limitations of PFTD and point out several future directions. First, it is interesting to design a tailored approach for the scenario where new workers dynamically join the private and fair truth discovery process. Besides, PFTD inevitably introduces additional computation and communications overhead due to the encryption and decryption operations engaged to ensure privacy. Therefore, how to offer privacy and fairness guarantees with lower computation and communication overhead remains an open question. Taken together, our study is the first step towards private and fair truth discovery, and we believe future works in this promising area would push forward the protection of human rights.



---

*Source: 9281729-2022-03-30.xml*