Modeling Uncertainty and Inaccuracy on Data from Crowdsourcing Platforms: MONITOR Crowdsourcing is characterized by the externalization of tasks to a crowd of workers. In some platforms the tasks are easy, open access and remunerated by micropayment. The crowd is very diversified due to the simplicity of the tasks, but the payment can attract malicious workers. It is essential to identify these malicious workers in order not to consider their answers. In addition, not all workers have the same qualification for a task, so it might be interesting to give more weight to those with more qualifications. In this paper we propose a new method for characterizing the profile of contributors and aggregating answers using the theory of belief functions to estimate uncertain and imprecise answers. In order to evaluate the contributor profile we consider both his qualification for the task and his behaviour during its achievement thanks to his reflection.

Evaluating the crowd quality for subjective questions based on a Spark computing environment Abstract   Crowdsourcing is a popular method of data collection or problem solving, which has been widely used in business, scientific research and other fields. Personnel quality control is a key issue in the field of crowdsourcing, and the method of estimating personnel quality is receiving more and more attention. This paper proposes a new algorithm for the quality estimation of crowdsourcing personnel, which is based on subjective questions innovatively. Specifically, confidence intervals are used to indicate personnel quality. To improve its scalability and efficiency in big data environment, algorithm is implemented on Spark, a widelyadopted distributed computing platform. Finally, extensive experiments are conducted on realworld data sets, and results demonstrate that algorithm can effectively and quickly estimate the quality of crowdsourcing personnel in the big data environment.