huashen218/convxai-cia-dataset huggingface dataset:
Title: Apps Against the Spread: Privacy Implications and User Acceptance of COVID-19-Related Smartphone Apps on Three Continents
Authors: family: Utz
given: Christine; family: Becker
given: Steffen; family: Schnitzler
given: Theodor; family: Farke
given: Florian M.; family: Herbert
given: Franziska; family: Schaewitz
given: Leonie; family: Degeling
given: Martin; family: Dürmuth
given: Markus
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: COVID-19, privacy, digital contact tracing

Abstract: The COVID-19 pandemic has fueled the development of smartphone applications to assist disease management. Many “corona apps” require widespread adoption to be effective, which has sparked public debates about the privacy, security, and societal implications of government-backed health applications. We conducted a representative online study in Germany (n = 1003), the US (n = 1003), and China (n = 1019) to investigate user acceptance of corona apps, using a vignette design based on the contextual integrity framework. We explored apps for contact tracing, symptom checks, quarantine enforcement, health certificates, and mere information. Our results provide insights into data processing practices that foster adoption and reveal significant differences between countries, with user acceptance being highest in China and lowest in the US. Chinese participants prefer the collection of personalized data, while German and US participants favor anonymity. Across countries, contact tracing is viewed more positively than quarantine enforcement, and technical malfunctions negatively impact user acceptance.
Title: A Digital Safety Dilemma: Analysis of Computer-Mediated Computer Security Interventions for Intimate Partner Violence During COVID-19
Authors: family: Tseng
given: Emily; family: Freed
given: Diana; family: Engel
given: Kristen; family: Ristenpart
given: Thomas; family: Dell
given: Nicola
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: intimate partner violence, computer security and privacy, gender-based violence

Abstract: The shutdown measures necessary to stop the spread of COVID-19 have amplified the role of technology in intimate partner violence (IPV). Survivors may be forced to endure lockdowns with their abusers, intensifying the dangers of technology-enabled abuse (e.g. stalking, harassment, monitoring, surveillance). They may also be forced to rely on potentially compromised devices to reach support networks: a dangerous dilemma for digital safety. This qualitative study examines how technologists with computer security expertise provided remote assistance to IPV survivors during the pandemic. Findings from 24 consults with survivors and five focus groups with technologist consultants show how remote delivery of technology support services raised three fundamental challenges: (1) ensuring safety for survivors and consultants; (2) assessing device security over a remote connection; and (3) navigating new burdens for consultants, including emotional labor. We highlight implications for HCI researchers creating systems that enable access to remote expert services for vulnerable people.
Title: Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative
Authors: family: Levy
given: Ariel; family: Agrawal
given: Monica; family: Satyanarayan
given: Arvind; family: Sontag
given: David
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: human-AI teams, text tagging, agency, mental model, ontology, clinical annotation

Abstract: Automated decision support can accelerate tedious tasks as users can focus their attention where it is needed most. However, a key concern is whether users overly trust or cede agency to automation. In this paper, we investigate the effects of introducing automation to annotating clinical texts — a multi-step, error-prone task of identifying clinical concepts (e.g., procedures) in medical notes, and mapping them to labels in a large ontology. We consider two forms of decision aid: recommending which labels to map concepts to, and pre-populating annotation suggestions. Through laboratory studies, we find that 18 clinicians generally build intuition of when to rely on automation and when to exercise their own judgement. However, when presented with fully pre-populated suggestions, these expert users exhibit less agency: accepting improper mentions, and taking less initiative in creating additional annotations. Our findings inform how systems and algorithms should be designed to mitigate the observed issues.
Title: EventAnchor: Reducing Human Interactions in Event Annotation of Racket Sports Videos
Authors: family: Deng
given: Dazhen; family: Wu
given: Jiang; family: Wang
given: Jiachen; family: Wu
given: Yihong; family: Xie
given: Xiao; family: Zhou
given: Zheng; family: Zhang
given: Hui; family: Zhang
given: Xiaolong (Luke); family: Wu
given: Yingcai
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: visualization, computer vision, Racket sports, data annotation

Abstract: The popularity of racket sports (e.g., tennis and table tennis) leads to high demands for data analysis, such as notational analysis, on player performance. While sports videos offer many benefits for such analysis, retrieving accurate information from sports videos could be challenging. In this paper, we propose EventAnchor, a data analysis framework to facilitate interactive annotation of racket sports video with the support of computer vision algorithms. Our approach uses machine learning models in computer vision to help users acquire essential events from videos (e.g., serve, the ball bouncing on the court) and offers users a set of interactive tools for data annotation. An evaluation study on a table tennis annotation system built on this framework shows significant improvement of user performances in simple annotation tasks on objects of interest and complex annotation tasks requiring domain knowledge.
Title: Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs
Authors: family: Suresh
given: Harini; family: Gomez
given: Steven R.; family: Nam
given: Kevin K.; family: Satyanarayan
given: Arvind
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: goals, framework, knowledge, expertise, needs, machine learning, interpretability, explainability

Abstract: To ensure accountability and mitigate harm, it is critical that diverse stakeholders can interrogate black-box automated systems and find information that is understandable, relevant, and useful to them. In this paper, we eschew prior expertise- and role-based categorizations of interpretability stakeholders in favor of a more granular framework that decouples stakeholders’ knowledge from their interpretability needs. We characterize stakeholders by their formal, instrumental, and personal knowledge and how it manifests in the contexts of machine learning, the data domain, and the general milieu. We additionally distill a hierarchical typology of stakeholder needs that distinguishes higher-level domain goals from lower-level interpretability tasks. In assessing the descriptive, evaluative, and generative powers of our framework, we find our more nuanced treatment of stakeholders reveals gaps and opportunities in the interpretability literature, adds precision to the design and comparison of user studies, and facilitates a more reflexive approach to conducting this research.
Title: Data-Centric Explanations: Explaining Training Data of Machine Learning Systems to Promote Transparency
Authors: family: Anik
given: Ariful Islam; family: Bunt
given: Andrea
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Fairness, Transparency, Machine Learning Systems, Trust, Training Data, Explanations, User Expertise

Abstract: Training datasets fundamentally impact the performance of machine learning (ML) systems. Any biases introduced during training (implicit or explicit) are often reflected in the system's behaviors leading to questions about fairness and loss of trust in the system. Yet, information on training data is rarely communicated to stakeholders. In this work, we explore the concept of data-centric explanations for ML systems that describe the training data to end-users. Through a formative study, we investigate the potential utility of such an approach, including the information about training data that participants find most compelling. In a second study, we investigate reactions to our explanations across four different system scenarios. Our results suggest that data-centric explanations have the potential to impact how users judge the trustworthiness of a system and to assist users in assessing fairness. We discuss the implications of our findings for designing explanations to support users’ perceptions of ML systems.
Title: Method for Exploring Generative Adversarial Networks (GANs) via Automatically Generated Image Galleries
Authors: family: Zhang
given: Enhao; family: Banovic
given: Nikola
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: qualitative model validation., Interactive model exploration

Abstract: Generative Adversarial Networks (GANs) can automatically generate quality images from learned model parameters. However, it remains challenging to explore and objectively assess the quality of all possible images generated using a GAN. Currently, model creators evaluate their GANs via tedious visual examination of generated images sampled from narrow prior probability distributions on model parameters. Here, we introduce an interactive method to explore and sample quality images from GANs. Our first two user studies showed that participants can use the tool to explore a GAN and select quality images. Our third user study showed that images sampled from a posterior probability distribution using a Markov Chain Monte Carlo (MCMC) method on parameters of images collected in our first study resulted in on average higher quality and more diverse images than existing baselines. Our work enables principled qualitative GAN exploration and evaluation.
Title: Player-AI Interaction: What Neural Network Games Reveal About AI as Play
Authors: family: Zhu
given: Jichen; family: Villareale
given: Jennifer; family: Javvaji
given: Nithesh; family: Risi
given: Sebastian; family: Löwe
given: Mathias; family: Weigelt
given: Rush; family: Harteveld
given: Casper
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Neural Networks, Human-AI Interaction, Game Design, User Experience

Abstract: The advent of artificial intelligence (AI) and machine learning (ML) bring human-AI interaction to the forefront of HCI research. This paper argues that games are an ideal domain for studying and experimenting with how humans interact with AI. Through a systematic survey of neural network games (n = 38), we identified the dominant interaction metaphors and AI interaction patterns in these games. In addition, we applied existing human-AI interaction guidelines to further shed light on player-AI interaction in the context of AI-infused systems. Our core finding is that AI as play can expand current notions of human-AI interaction, which are predominantly productivity-based. In particular, our work suggests that game and UX designers should consider flow to structure the learning curve of human-AI interaction, incorporate discovery-based learning to play around with the AI and observe the consequences, and offer users an invitation to play to explore new forms of human-AI interaction.
Title: Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Authors: family: Lu
given: Zhuoran; family: Yin
given: Ming
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: human-AI interaction, appropriate reliance, Machine learning

Abstract: This paper addresses an under-explored problem of AI-assisted decision-making: when objective performance information of the machine learning model underlying a decision aid is absent or scarce, how do people decide their reliance on the model? Through three randomized experiments, we explore the heuristics people may use to adjust their reliance on machine learning models when performance feedback is limited. We find that the level of agreement between people and a model on decision-making tasks that people have high confidence in significantly affects reliance on the model if people receive no information about the model’s performance, but this impact will change after aggregate-level model performance information becomes available. Furthermore, the influence of high confidence human-model agreement on people’s reliance on a model is moderated by people’s confidence in cases where they disagree with the model. We discuss potential risks of these heuristics, and provide design implications on promoting appropriate reliance on AI.
Title: AutoDS: Towards Human-Centered Automation of Data Science
Authors: family: Wang
given: Dakuo; family: Andres
given: Josh; family: Weisz
given: Justin D.; family: Oduor
given: Erick; family: Dugan
given: Casey
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: human-in-the-loop, AutoML, XAI, automated machine learning, human-AI collaboration, AutoDS, automated data science, Data science, model building, collaborative AI, AI

Abstract: Data science (DS) projects often follow a lifecycle that consists of laborious tasks for data scientists and domain experts (e.g., data exploration, model training, etc.). Only till recently, machine learning(ML) researchers have developed promising automation techniques to aid data workers in these tasks. This paper introduces AutoDS, an automated machine learning (AutoML) system that aims to leverage the latest ML automation techniques to support data science projects. Data workers only need to upload their dataset, then the system can automatically suggest ML configurations, preprocess data, select algorithm, and train the model. These suggestions are presented to the user via a web-based graphical user interface and a notebook-based programming user interface. Our goal is to offer a systematic investigation of user interaction and perceptions of using an AutoDS system in solving a data science task. We studied AutoDS with 30 professional data scientists, where one group used AutoDS, and the other did not, to complete a data science project. As expected, AutoDS improves productivity; Yet surprisingly, we find that the models produced by the AutoDS group have higher quality and less errors, but lower human confidence scores. We reflect on the findings by presenting design implications for incorporating automation techniques into human work in the data science lifecycle.
Title: Evaluating the Interpretability of Generative Models by Interactive Reconstruction
Authors: family: Ross
given: Andrew; family: Chen
given: Nina; family: Hang
given: Elisa Zhao; family: Glassman
given: Elena L.; family: Doshi-Velez
given: Finale
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: representation learning, evaluation methods, interpretability

Abstract: For machine learning models to be most useful in numerous sociotechnical systems, many have argued that they must be human-interpretable. However, despite increasing interest in interpretability, there remains no firm consensus on how to measure it. This is especially true in representation learning, where interpretability research has focused on “disentanglement” measures only applicable to synthetic datasets and not grounded in human factors. We introduce a task to quantify the human-interpretability of generative model representations, where users interactively modify representations to reconstruct target instances. On synthetic datasets, we find performance on this task much more reliably differentiates entangled and disentangled models than baseline approaches. On a real dataset, we find it differentiates between representation learning methods widely believed but never shown to produce more or less interpretable models. In both cases, we ran small-scale think-aloud studies and large-scale experiments on Amazon Mechanical Turk to confirm that our qualitative and quantitative results agreed.
Title: Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Authors: family: Bansal
given: Gagan; family: Wu
given: Tongshuang; family: Zhou
given: Joyce; family: Fok
given: Raymond; family: Nushi
given: Besmira; family: Kamar
given: Ece; family: Ribeiro
given: Marco Tulio; family: Weld
given: Daniel
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Augmented intelligence, Explainable AI, Human-AI teams

Abstract: Many researchers motivate explainable AI with studies showing that human-AI team performance on decision-making tasks improves when the AI explains its recommendations. However, prior studies observed improvements from explanations only when the AI, alone, outperformed both the human and the best team. Can explanations help lead to complementary performance, where team accuracy is higher than either the human or the AI working solo? We conduct mixed-method user studies on three datasets, where an AI with accuracy comparable to humans helps participants solve a task (explaining itself in some conditions). While we observed complementary improvements from AI augmentation, they were not increased by explanations. Rather, explanations increased the chance that humans will accept the AI’s recommendation, regardless of its correctness. Our result poses new challenges for human-centered AI: Can we develop explanatory approaches that encourage appropriate trust in AI, and therefore help generate (or improve) complementary performance?
Title: Expanding Explainability: Towards Social Transparency in AI systems
Authors: family: Ehsan
given: Upol; family: Liao
given: Q. Vera; family: Muller
given: Michael; family: Riedl
given: Mark O.; family: Weisz
given: Justin D.
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: socio-organizational context, explanations, sociotechnical, human-AI interaction, Artificial Intelligence, social transparency, Explainable AI

Abstract: As AI-powered systems increasingly mediate consequential decision-making, their explainability is critical for end-users to take informed and accountable actions. Explanations in human-human interactions are socially-situated. AI systems are often socio-organizationally embedded. However, Explainable AI (XAI) approaches have been predominantly algorithm-centered. We take a developmental step towards socially-situated XAI by introducing and exploring Social Transparency (ST), a sociotechnically informed perspective that incorporates the socio-organizational context into explaining AI-mediated decision-making. To explore ST conceptually, we conducted interviews with 29 AI users and practitioners grounded in a speculative design scenario. We suggested constitutive design elements of ST and developed a conceptual framework to unpack ST’s effect and implications at the technical, decision-making, and organizational level. The framework showcases how ST can potentially calibrate trust in AI, improve decision-making, facilitate organizational collective actions, and cultivate holistic explainability. Our work contributes to the discourse of Human-Centered XAI by expanding the design space of XAI.
Title: Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Authors: family: Xin
given: Doris; family: Wu
given: Eva Yiwei; family: Lee
given: Doris Jung-Lin; family: Salehi
given: Niloufar; family: Parameswaran
given: Aditya
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Abstract: Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.
Title: Circumspect Users: Older Adults as Critical Adopters and Resistors of Technology
Authors: family: Barros Pena
given: Belén; family: Clarke
given: Rachel E; family: Holmquist
given: Lars Erik; family: Vines
given: John
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Older adults, Life story interview, Life course, Everyday resistance

Abstract: While HCI research has often addressed the needs of older adults, they are often framed as being sceptical of digital technologies. We argue that while many older adults are circumspect users of digital technology, they bring rich and critical perspectives on the role of technology in society that are grounded in lived experiences across their life courses. We report on 20 technology life story interviews conducted with retirees over the age of 60. Our analysis shows how experiences of technology across their life courses significantly undermined participants’ sense of competency, independence, resilience, agency and control. Dissonances between what our participants valued and the perceived values of technology have led them to become critical adopters of technology, and resist its intrusion into certain aspects of their lives. We discuss how the critical perspectives of older adults and the value dissonances they experience are valuable for designing future digital technologies.
Title: Training and Embedding Cybersecurity Guardians in Older Communities.
Authors: family: Nicholson
given: James; family: Morrison
given: Ben; family: Dixon
given: Matt; family: Holt
given: Jack; family: Coventry
given: Lynne; family: McGlasson
given: Jill
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: community, Older adults, civic engagement, information sharing, cybersecurity

Abstract: Older adults can struggle to access relevant community expertise when faced with new situations. One such situation is the number of cyberattacks they may face when interacting online. This paper reports on an initiative which recruited, trained, and supported older adults to become community cybersecurity educators (CyberGuardians), tasked with promoting cybersecurity best practice within their communities to prevent older adults falling victim to opportunistic cyberattacks. This initiative utilised an embedded peer-to-peer information dissemination strategy, rather than expert-to-citizen, facilitating the inclusion of individuals who would ordinarily be unlikely to seek cybersecurity information and thus may be vulnerable to cyberattacks. We report on ways the CyberGuardians used informal methods to create more aware communities, served as role models for behaviour change and indirectly improved their personal wellbeing. We discuss considerations for supporting CyberGuardians, including implications for sustainability and for replicating this model in other digital contexts, e.g., recognising misinformation or improving mental health.