Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novel worker desirability index (WDI) jointly considering the effect of each worker's reputation, workload and motivation to work on collective productivity. Through evaluating workers' WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing. Crowdsourcing systems refer to systems designed to obtain needed services, ideas, or content by soliciting contributions from a large group of people and especially from the online community 1 . Over the years, crowdsourcing technologies have spawned the "sharing economy", an economic paradigm in which individuals can borrow or rent assets owned by someone else. Examples include Uber (https://www.uber.com), the world's largest "taxi company" which crowdsources rides for commuters, and Airbnb (https://www.airbnb.com), the world's largest "hotel chain" which crowdsources accommodations for travellers. A recent study by PricewaterhouseCoopers (PwC) estimates that the size of the sharing economy will reach $335 billion by 2025 2 . Nowadays, many organisations flexibly outsource work to a global pool of workers on a temporary basis. Indeed, crowdsourcing has been used successfully for a range of tasks, including collaborative sensing [3] [4] [5] [6] [7] [8] [9] , citizen science 10, 11 , and human-powered online security 12 . In crowdsourcing systems, crowdsourcers propose tasks for workers to complete. Such systems often consist of a large number of interacting crowdsourcers and workers. There are typically multiple workers (i.e. resources) available for a crowdsourcer. For example, in Amazon's Mechanical Turk (mTurk) (https://www.mturk.com), workers outnumber crowdsourcers almost 20 to 1 13 . As such, the number of potential ways for the division of work among workers is very large. The cost of delegating a task to a worker in crowdsourcing does not solely depend on the number of crowdsourcers engaging this worker. As a human being, a worker is not only limited in productivity (i.e. how much workload he can process per unit time), but also has different competence which can affect the quality of the results produced 14 . These personal characteristics may vary over time and can only
They proposed a surprise-minimization-value maximization (SMVM) approach to address the NP-hard problem of allocation task to worker and maximize social welfare in crowdsourcing system REF .
Algorithmic Management for Improving Collective Productivity in Crowdsourcing
Scientific Reports
Scientific Reports
Medicine Computer Science