# Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine Interaction
## 2 Background and Scope  
The point of departure for building the taxonomy presented in this article was the existing work found on the intersectional space between human-computer interaction (HCI) and AI from a crowdsourcing perspective. Although the coining of the term 'crowdsourcing' took place in the mid-2000s, some may argue that its origin is rooted in the seminal work of the physicist and astronomer [PERSON], who used news media as a crowdsourcing strategy for obtaining accurate observations on the Leonid meteor shower that was witnessed across the United States in 1833 [20]. What is interesting to note is that the sequential steps and general techniques used by [PERSON] about nineteen decades ago constitute the basis for most of the current crowdsourcing applications. Aligned with this goal, a variety of taxonomies and conceptual frameworks have been developed to better characterize the way as information technology (IT)-enabled crowdsourcing operates. Among the known classifications of crowdsourcing activities, [PERSON] and co-authors [21] were some of the first to frame this phenomenon from a taxonomic point of view by incorporating the nature of the crowd, the payment mechanisms or lack thereof, and the type of task into an integrated framework. In line with this, [PERSON] [22] proposed a taxonomy that comprises the different forms of intrinsic and extrinsic motivation that can lead to a successful crowdsourcing experience (e.g., social status, altruistic behavior, and personal achievement). This taxonomic proposal also addresses a set of aspects that are specific to the nature of the crowdsourcing task being undertaken by encompassing the expertise and complexity that are directly or indirectly involved in such initiatives. On the basis of insights from the history of group support systems, one would notice similar points to [PERSON]'s [8] task circumplex taxons taking into consideration the different task types that can be executed by individuals in a group structure, which may include decision-making, idea generation and information gathering to name just a few examples. To an extent, this research strand led to the proliferation of several taxonomies incorporating task-related elements (e.g., [23; 24; 25; 26; 27; 28; 29; 30]). Consistent with the task properties discussed in most of these studies, a cursory look at the literature reveals certain commonalities related to crowd attributes (e.g., reputation), requester features (e.g., incentivization), and platform facilities such as aggregation and payment mechanisms [29]. Other research works have focused specifically on internal forms of crowdsourcing [31] or even on the use of crowdsourcing as a taxonomy development strategy by itself [32]. On a more generic level, [PERSON] and colleagues [10] made a clear distinction between the IT-enabled crowdsourcing requirements in business and non-business contexts by basing their proposal on the four collective intelligence \"genes\" proposed by [PERSON] et al. [23]. However, these taxonomies fail to fully account for the hybrid nature of crowd-AI interaction and thus are unable to capture the variety of interactions and relations that occur when using a hybrid intelligence system. During the last few years, the advances in the development of AI technologies have been silently leveraging the capacity of a large pool of crowd workers worldwide who provide data on a daily basis and thus contribute to the improvement of several models on a scale that had never been seen before. In fact, this intertwinement of algorithms with crowdsourcing workflows brought important advantages in a multiplicity of set tings. Prior work has employed these principles and proved to be effective in detecting accessibility problems on public surfaces (e.g., sidewalks) through the use of street-level imagery [33]. In the same vein, [PERSON] and associates [34] proposed a system for identifying urban infrastructure damages, such as fallen street signs, when AI-based solutions fail to recognize them.