# Selecting Workers Wisely for Crowdsourcing When Copiers and Domain Experts Co-exist
## 4 Task Assignment
### Copier Detection and Removal  
Existing task assignment methods mostly assume that all workers complete their tasks independently. However, in reality, copying is ubiquitous. We classified workers into two groups, i.e., independent workers and copiers. Given a task, it may have multiple distinct false answers, but only one single true answer. It is common to observe two workers sharing the same true answer, but it is a rare event to see two workers providing the same false answer. Therefore, if two workers share many false answers, they are highly likely to be dependent, i.e., \(w_{i}\sim w_{i^{\prime}}\). We applied the Bayesian method to compute the dependent probability of two workers, \(w_{i}\) and \(w_{i^{\prime}}\). The purpose of the algorithm is to avoid assigning tasks to copiers in the task assignment step. Therefore, we not only need to detect whether there is a copying relationship between a pair of workers, but we also need to identify the direction of dependence. Intuitively, copiers normally tend to copy information from highly reliable workers. Therefore, given \(w_{i}\sim w_{i^{\prime}}\), we regard the worker with lower reliability as the copier. We denote \(w_{i}\) a copy from \(w_{i^{\prime}}\) as \(w_{i}\to w_{i^{\prime}}\). In this paper, we focused on the direct copying relationship between any pair of workers, and leave the complex global copying detection as future work. In order to calculate the dependence probability, we first partitioned the tasks completed by \(w_{i}\) and \(w_{i^{\prime}}\) in H into three sets. \(T^{t}\) is the set of tasks on which \(w_{i}\) and \(w_{i^{\prime}}\) provide the same true values; \(T^{f}\) is the set of tasks on which they provide the same false values; \(T^{d}\) is the set of tasks on which they provide different values. We first considered the situation where the two workers \(w_{i}\) and \(w_{i^{\prime}}\) are independent. Since there is only one true value, the probability that \(w_{i}\) and \(w_{i^{\prime}}\) provide the same true value for task \(t_{j}\), denoted by \(P_{t}^{j}\) for convenience, is:  
\[P_{t}^{j}=P\big{(}t_{j}\in T^{t}|w_{i}\perp w_{i^{\prime}}\big{)}=q_{i}^{j} \cdot q_{i^{\prime}}^{j} \tag{4}\]  
Based on the assumption of a uniform false value distribution in Section 2, we think any independent worker has the same probability of providing each false value for task \(t_{j}\). Thus, the probability that worker \(w_{i}\) provides a false value for task \(t_{j}\) is \(\frac{1-q_{i}^{j}}{|t_{j}|-1}\). The probability that \(w_{i}\) and \(w_{i^{\prime}}\) provide the same false value for task \(t_{j}\), denoted by \(P_{f}^{j}\), is:  
\[P_{f}^{j}=P(t_{j}\in T^{f}|w_{i}\perp w_{i^{\prime}})=\frac{(1-q_{i}^{j}) \cdot(1-q_{i^{\prime}}^{j})}{|a|^{j}} \tag{5}\]  
Then, the probability that \(w_{i}\) and \(w_{i^{\prime}}\) provide different values on task \(t_{j}\), denoted by \(P_{d}^{j}\), is:  
\[P_{d}^{j}=P(t_{j}\in T^{d}|w_{i}\perp w_{i^{\prime}})=1-P_{t}^{j}-P_{f}^{j} \tag{6}\]  
Thus, the conditional probability of observing \(\phi\) is:  
\[P(\phi|w_{i}\perp w_{i^{\prime}})=\prod_{t_{j}\in T^{t}}P_{t}^{i}\cdot\prod_{ t_{j}\in T^{f}}P_{f}^{j}\cdot\prod_{t_{j}\in T^{d}}P_{d}^{j} \tag{7}\]  
We next considered the situation where \(w_{i}\to w_{i^{\prime}}\) (similar for \(w_{i^{\prime}}\to w_{i}\)). There are two cases where \(w_{i}\) and \(w_{i^{\prime}}\) provide the same value for the task \(t_{j}\). First, with probability \(r\), one copies the value from the other, and so, the value is true with probability \(q_{i^{\prime}}^{j}\) and false with probability \(1-q_{i^{\prime}}^{j}\). Second, with probability \(1-r\), the two workers provide the value independently, and so, the probability of being true or false is the same as that in the situation where \(w_{i}\) and \(w_{i^{\prime}}\) are independent.