# Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine Interaction
## 5 Validation and Assessment of the Proposed Taxonomy
Based on the findings from the chosen sample, [PERSON] and co-authors [124] provide one of the few examples of studies calling attention to possible unethical actions associated with the disclosure of sensitive information from images and videos. In a similar way, only 20% of primary studies (\(n=5\)) fully describe machine and human (crowd) agency, governance practices, or control (oversight) (T4), although extensive research has been conducted about the potential risks and unintentional harms associated with the lack of an effective governance strategy able to regulate algorithmic actions [125]. In this regard, trust building [126; 127] appears among the most critical factors affecting technology acceptance when considering human-AI interaction at a massive scale. One enduring taxonomic unit that has been largely addressed since the very beginning of the field of CSCW is concerned with the contextual and situational information (T5) that is then used to support awareness about the environment in which the interaction takes place. This includes what goes on in the environment, who is available, who leaves, and how individuals \"remain sensitive to the conduct of others so that an event or action, which may have some passing significance, can be displayed to each other without it necessarily gaining interactional or sequential import\" [128]. If the entire sample is considered, 48% of studies (\(n=12\)) mentioned some kind of contextual or situational issues. For instance, [PERSON] et al. [39] proposed a crowd-machine hybrid system where the conversation context is used to provide response candidates using recorded facets and previous chat conversation logs. In particular, the task-specific contextual data is captured with the help of the crowd (by using chat logs) to improve the quality of responses based on current and past conversations. Moreover, [PERSON] and associates [129] used self-adapting mechanisms based on reinforcement learning (RL) and contextual features extracted to increase crowdsourcing participation over time, while [PERSON] and co-workers [40] considered the lack of context as a determining factor for failure in smart environments. Turning to the role of infrastructural support (T6) in interactive human-AI practices at a crowd level, the review disclosed a total of 20 studies (80%) where infrastructure or the characteristics of a crowd-computing platform are reported. In CSCW, the concept of 'infrastructure' and its ecological nature [130] has developed over the years to characterize socio-technical assemblages \"that underpins and enables action, engagement, and awareness\" [131]. On the basis of their research review, [PERSON] and colleagues [29] gave a detailed description of the features that are commonly found in crowdsourcing platforms. In line with this, [PERSON] and co-authors [102] stressed that a crowdsourcing system must provide functions and components able to support workflows involving actions such as task assignment, pre-selecting crowd workers, stating rewards, and selecting contributions. From payment mechanisms to result aggregation, a crowd-computing platform must combine crowd-, requester-, task- and platform-related information and facilities (i.e., infrastructural elements) that act in unison to carry out tasks in accordance with the different requirements. From an infrastructural perspective, [PERSON] and associates [39] described the conversational worker interface used for chatting and real-time response modeling along with the automatic response voting and generating algorithms deployed to operate in a continuous manner as the conversation continues. Using a crowd-AI hybrid intelligence lens, the results showed a total of 14 studies addressing algorithmic reasoning, inference, explainability, and interpretability (T7). For instance, human-AI decision-making processes are complex by nature, and AI-infused systems require a certain level of explainability [132] and interpretability [133] to provide insights about the algorithmic actions taken during the AI-enabled experience.