# Selecting Workers Wisely for Crowdsourcing When Copiers and Domain Experts Co-exist
## 2 Related Work
### Truth Discovery in Crowdsourcing  
The most basic truth discovery method is majority voting. This approach regards all workers as equally trustworthy. However, in reality, different worker may have different accuracies [46]. To differentiate worker credibility, early methods, such as D&S [7], use a confusion matrix to model the credibility of workers. After that, more advanced methods, such as TruthFinder [47], LTM [48], and PrecRec [49] have been proposed. They measure worker credibility by additionally applying worker answering models or incorporating more considerations, such as task difficulty. However, these methods do not take worker domain expertise into consideration, assuming that workers have the same credibility for all tasks. Based on this assumption, they measure the accuracy of each work as a unified value or matrix. To relax this assumption, there are some current efforts in the truth discovery step in crowdsourcing [50]. They try to utilize the fine-grained credibility of sources. FaiFcrowd [51] uses a probabilistic graphical method to divide tasks into topic-level clusters and estimate every source's topical credibility accordingly. However, this method needs to determine the number of topics in advance, and the semantics of topic clusters is lacking. IniCrowd [52] is a similar method. It uses similarity metrics and topic models to obtain the similarity and topic distribution of each task. Tasks with high text similarity are assigned to the same domain. Nevertheless, this method may simply divide tasks with a similar text description syntax together. In fact, these tasks are in different domains. [PERSON] et al. [53] proposed a method that considers domains and multi-truth. They inferred the domain expertise of a data source based on its data richness in different domains. However, this method is inapplicable to the crowdsourcing scenario, where we cannot expect all workers to provide answers to a large number of tasks. In crowdsourcing, a large number of truth discovery algorithms for many aspects have been proposed. [PERSON] et al. [54] propose a privacy-preserving truth discovery framework called PPTD. This framework uses the threshold homomorphic cryptosystem to guarantee the confidentiality of workers' values and weights. [PERSON] et al. [55] proposed a non-interactive privacy-preserving truth discovery framework, which protects workers' data while obtaining truth. [PERSON] et al. [56] proposed an unsupervised learning method to quantify the workers' credibility and long-term reputations. This method uses an outlier detection technique to filter out anomalous data items. [PERSON] et al. [57] proposed a protocol called BUR, which can employ nearly the least number of workers while ensuring that the overall accuracy of each task meets the requirements (given threshold). [PERSON] et al. [58] proposed a framework for multi-requester mobile crowdsourcing, called CENTURION, which consists of a truth discovery step and an incentive step. However, none of these studies have considered the factors of the domain and copying. In addition, copying is ubiquitous in crowdsourcing. The credibility of copiers is not in line with their real level. Some truth discovery works have taken source correlation into consideration. [PERSON] et al. (2016) proposed a method to consider the relationship between sources in truth discovery. Source correlations were inferred based on the intuition that \"if two sources provide the same false values, it is very likely that one copies from the other\". However, this model does not precisely demonstrate how a potential correlation can impact the estimation of sources' trustworthiness. MBM ([PERSON] et al., 2017) takes into account both the copier and multi-truth problems. \(IMC^{2}\)([PERSON] et al., 2017) takes into comprehensive consideration the copying and incentive mechanism in crowdsourcing, but it still calculates the probability of copiers among all workers, which is very heavy in the actual crowdsourcing application.