Miracle-dz/UniFAR-FTUs huggingface dataset:
Tea, Earl Grey, Hot: Designing Speech Interactions from the Imagined Ideal of Star Trek

Speech is now common in daily interactions with our devices, thanks to voice user interfaces (VUIs) like Alexa. Despite their seeming ubiquity, designs often do not match users’ expectations. Science fiction, which is known to influence design of new technologies, has included VUIs for decades. Star Trek: The Next Generation is a prime example of how people envisioned ideal VUIs. Understanding how current VUIs live up to Star Trek’s utopian technologies reveals mismatches between current designs and user expectations, as informed by popular fiction. Combining conversational analysis and VUI user analysis, we study voice interactions with the Enterprise’s computer and compare them to current interactions. Independent of futuristic computing power, we find key design-based differences: Star Trek interactions are brief and functional, not conversational, they are highly multimodal and context-driven, and there is often no spoken computer response. From this, we suggest paths to better align VUIs with user expectations.
Elevate: A Walkable Pin-Array for Large Shape-Changing Terrains

Current head-mounted displays enable users to explore virtual worlds by simply walking through them (i.e., real-walking VR). This led researchers to create haptic displays that can also simulate different types of elevation shapes. However, existing shape-changing floors are limited by their tabletop scale or the coarse resolution of the terrains they can display due to the limited number of actuators and low vertical resolution. To tackle this challenge, we introduce Elevate, a dynamic and walkable pin-array floor on which users can experience not only large variations in shapes but also the details of the underlying terrain. Our system achieves this by packing 1200 pins arranged on a 1.80 × 0.60m platform, in which each pin can be actuated to one of ten height levels (resolution: 15mm/level). To demonstrate its applicability, we present our haptic floor combined with four walkable applications and a user study that reported increased realism and enjoyment.
Hummer: Text Entry by Gaze and Hum

Text entry by gaze is a useful means of hands-free interaction that is applicable in settings where dictation suffers from poor voice recognition or where spoken words and sentences jeopardize privacy or confidentiality. However, text entry by gaze still shows inferior performance and it quickly exhausts its users. We introduce text entry by gaze and hum as a novel hands-free text entry. We review related literature to converge to word-level text entry by analysis of gaze paths that are temporally constrained by humming. We develop and evaluate two design choices: “HumHum” and “Hummer. ” The first method requires short hums to indicate the start and end of a word. The second method interprets one continuous humming as an indication of the start and end of a word. In an experiment with 12 participants, Hummer achieved a commendable text entry rate of 20.45 words per minute, and outperformed HumHum and the gaze-only method EyeSwipe in both quantitative and qualitative measures.
“It's The Same Conflict Every Time, On Repeat.”: How Digital Technology Use Can Contribute Towards Conflict in Parents’ Relationships

Family technology use can create or amplify conflict in parents’ relationships – we found four key factors that contribute to this issue. We conducted a probe and interview study with eight sets of parents, to explore how and why technology use might cause conflict in their relationships. This paper presents data from two particular sets of parents to illustrate our findings. In doing so, it complements existing work that primarily focuses on parent-child relationships, and contributes to a more complete understanding of how family technology use can affect family dynamics. We also suggest directions for further work to address this issue of conflict between parents, associated with their family's use of technology.
Immersive Inclusivity at CHI: Design and Creation of Inclusive User Interactions Through Immersive Media

Immersive media is becoming increasingly common in day-to-day scenarios: from extended reality systems to multimodal interfaces. Such ubiquity opens an opportunity for building more inclusive environments for users with disabilities (permanent, temporary, or situational) by either introducing immersive and multimodal elements into existing applications, or designing and creating immersive applications with inclusivity in mind. Thus the aim of this workshop is to create a discussion platform on intersections between the fields of immersive media, accessibility, and human-computer interaction, outline the key current and future problems of immersive inclusive design, and define a set of methodologies for design and evaluation of immersive systems from inclusivity perspective.
Crowdsourcing More Effective Initializations for Single-Target Trackers Through Automatic Re-querying

In single-target video object tracking, an initial bounding box is drawn around a target object and propagated through a video. When this bounding box is provided by a careful human expert, it is expected to yield strong overall tracking performance that can be mimicked at scale by novice crowd workers with the help of advanced quality control methods. However, we show through an investigation of 900 crowdsourced initializations that such quality control strategies are inadequate for this task in two major ways: first, the high level of redundancy in these methods (e.g., averaging multiple responses to reduce error) is unnecessary, as 23% of crowdsourced initializations perform just as well as the gold-standard initialization. Second, even nearly perfect initializations can lead to degraded long-term performance due to the complexity of object tracking. Considering these findings, we evaluate novel approaches for automatically selecting bounding boxes to re-query, and introduce Smart Replacement, an efficient method that decides whether to use the crowdsourced replacement initialization.
Little Road Driving HUD: Heads-Up Display Complexity Influences Drivers’ Perceptions of Automated Vehicles

Modern vehicles are using AI and increasingly sophisticated sensor suites to improve Advanced Driving Assistance Systems (ADAS) and support automated driving capabilities. Heads-Up-Displays (HUDs) provide an opportunity to visually inform drivers about vehicle perception and interpretation of the driving environment. One approach to HUD design may be to reveal to drivers the vehicle’s full contextual understanding, though it is not clear if the benefits of additional information outweigh the drawbacks of added complexity, or if this balance holds across drivers. We designed and tested an Augmented Reality (AR) HUD in an online study (N = 298), focusing on the influence of HUD visualizations on drivers’ situation awareness and perceptions. Participants viewed two driving scenes with one of three HUD conditions. Results were nuanced: situation awareness declined with increasing driving context complexity, and contrary to expectation, also declined with the presence of a HUD compared to no HUD. Significant differences were found by varying HUD complexity, which led us to explore different characterizations of complexity, including counts of scene items, item categories, and illuminated pixels. Our analysis finds that driving style interacts with driving context and HUD complexity, warranting further study.
Charting the Unknown: Challenges in the Clinical Assessment of Patients’ Technology Use Related to Eating Disorders

A growing body of research in HCI focuses on understanding how social media and other social technologies impact a given user’s mental health, including eating disorders. In this paper, we review the results of an interview study with 10 clinicians spanning various specialties who treat people with eating disorders, in order to understand the clinical contexts of eating disorders and social media use. We found various tensions related to clinician comfort and education into the (mis)use of technologies and balancing the positive and negative aspects of social media use within active disease states as well as in recovery. Understanding these tensions as well as the variation in the current process of diagnosing patients is a critical component in connecting HCI research focused on eating disorders to clinical practice and ultimately assessing how digital self-harm could be addressed clinically in the future.
Towards Explainable and Trustworthy Autonomous Physical Systems

The safe deployment of autonomous physical systems in real-world scenarios requires them to be explainable and trustworthy, especially in critical domains. In contrast with ‘black-box’ systems, explainable and trustworthy autonomous physical systems will lend themselves to easy assessments by system designers and regulators. This promises to pave ways for easy improvements that can lead to enhanced performance, and as well, increased public trust. In this one-day virtual workshop, we aim to gather a globally distributed group of researchers and practitioners to discuss the opportunities and social challenges in the design, implementation, and deployment of explainable and trustworthy autonomous physical systems, especially in a post-pandemic era. Interactions will be fostered through panel discussions and a series of spotlight talks. To ensure lasting impact of the workshop, we will conduct a pre-workshop survey which will examine the public perception of the trustworthiness of autonomous physical systems. Further, we will publish a summary report providing details about the survey as well as the identified challenges resulting from the workshop’s panel discussions.
Assessing MyData Scenarios: Ethics, Concerns, and the Promise

Public controversies around the unethical use of personal data are increasing, spotlighting data ethics as an increasingly important field of study. MyData is a related emerging vision that emphasizes individuals’ control of their personal data. In this paper, we investigate people’s perceptions of various data management scenarios by measuring the perceived ethicality and level of felt concern concerning the scenarios. We deployed a set of 96 unique scenarios to an online crowdsourcing platform for assessment and invited a representative sample of the participants to a second-stage questionnaire about the MyData vision and its potential in the field of healthcare. Our results provide a timely investigation into how topical data-related practices affect the perceived ethicality and the felt concern. The questionnaire analysis reveals great potential in the MyData vision. Through the combined quantitative and qualitative results, we contribute to the field of data ethics.
EarRumble: Discreet Hands- and Eyes-Free Input by Voluntary Tensor Tympani Muscle Contraction

We explore how discreet input can be provided using the tensor tympani - a small muscle in the middle ear that some people can voluntarily contract to induce a dull rumbling sound. We investigate the prevalence and ability to control the muscle through an online questionnaire (N=192) in which 43.2% of respondents reported the ability to “ear rumble”. Data collected from participants (N=16) shows how in-ear barometry can be used to detect voluntary tensor tympani contraction in the sealed ear canal. This data was used to train a classifier based on three simple ear rumble “gestures” which achieved 95% accuracy. Finally, we evaluate the use of ear rumbling for interaction, grounded in three manual, dual-task application scenarios (N=8). This highlights the applicability of EarRumble as a low-effort and discreet eyes- and hands-free interaction technique that users found “magical” and “almost telepathic”.
Toward the Introduction of Google Assistant in Therapy for Children with Neurodevelopmental Disorders: An Exploratory Study

Despite the growing interest in Intelligent Personal Assistants in many domains, limited studies have explored this technology’s usage among persons with neurodevelopmental disorders (NDD) and its potential for improving their cognitive skills. This paper presents an exploratory study that investigated the use of Google Home featured with Google Assistant in a therapeutic setting. We provided three therapists with two Google Home devices for 21 days, and they were welcome to use it as they liked at their ordinary one-to-one therapy sessions. During the study, we gathered different data: history logs from Google Assistant and semi-structured observations and comments by the specialists through questionnaires, forms, and a group interview. Our findings give a first glimpse of the usage patterns of Google Assistant within the therapy for children with NDD. Furthermore, our results point out the benefits and challenges for both therapists and children while interacting with conversational technology.
Mobile Recognition and Tracking of Objects in the Environment through Augmented Reality and 3D Audio Cues for People with Visual Impairments

People with visual impairments face challenges in scene and object recognition, especially in unknown environments. We combined the mobile scene detection framework Apple ARKit with MobileNet-v2 and 3D spatial audio to provide an auditory scene description to people with visual impairments. The combination of ARKit and MobileNet allows keeping recognized objects in the scene even if the user turns away from the object. An object can thus serve as an auditory landmark. With a search function, the system can even guide the user to a particular item. The system also provides spatial audio warnings for nearby objects and walls to avoid collisions. We evaluated the implemented app in a preliminary user study. The results show that users can find items without visual feedback using the proposed application. The study also reveals that the range of local object detection through MobileNet-v2 was insufficient, which we aim to overcome using more accurate object detection frameworks in future work (YOLOv5x).
Mobile, Hands-free, Silent Speech Texting Using SilentSpeller

Voice control provides hands-free access to computing, but there are many situations where audible speech is not appropriate. Most unvoiced speech text entry systems can not be used while on-the-go due to movement artifacts. SilentSpeller enables mobile silent texting using a dental retainer with capacitive touch sensors to track tongue movement. Users type by spelling words without voicing. In offline isolated word testing on a 1164-word dictionary, SilentSpeller achieves an average 97% character accuracy. 97% offline accuracy is also achieved on phrases recorded while walking or seated. Live text entry achieves up to 53 words per minute and 90% accuracy, which is competitive with expert text entry on mini-QWERTY keyboards without encumbering the hands.
Oh, Snap! A Fabrication Pipeline to Magnetically Connect Conventional and 3D-Printed Electronics

3D printing has revolutionized rapid prototyping by speeding up the creation of custom-shaped objects. With the rise of multi-material 3D printers, these custom-shaped objects can now be made interactive in a single pass through passive conductive structures. However, connecting conventional electronics to these conductive structures often still requires time-consuming manual assembly involving many wires, soldering or gluing. To alleviate these shortcomings, we propose : a fabrication pipeline and interfacing concept to magnetically connect a 3D-printed object equipped with passive sensing structures to conventional sensing electronics. To this end, utilizes ferromagnetic and conductive 3D-printed structures, printable in a single pass on standard printers. We further present a proof-of-concept capacitive sensing board that enables easy and robust magnetic assembly to quickly create interactive 3D-printed objects. We evaluate by assessing the robustness and quality of the connection and demonstrate its broad applicability by a series of example applications.
Don’t You Know That You’re Toxic: Normalization of Toxicity in Online Gaming

Video game toxicity, endemic to online play, represents a pervasive and complex problem. Antisocial behaviours in online play directly harm player wellbeing, enjoyment, and retention—but research has also revealed that some players normalize toxicity as an inextricable and acceptable element of the competitive video game experience. In this work, we explore perceptions of toxicity and how they are predicted by player traits, demonstrating that participants reporting a higher tendency towards Conduct Reconstrual, Distorting Consequences, Dehumanization, and Toxic Online Disinhibition perceive online game interactions as less toxic. Through a thematic analysis on willingness to report, we also demonstrate that players abstain from reporting toxic content because they view it as acceptable, typical of games, as banter, or as not their concern. We propose that these traits and themes represent contributing factors to the cyclical normalization of toxicity. These findings further highlight the multifaceted nature of toxicity in online video games.
Communication Skills Training Intervention Based on Automated Recognition of Nonverbal Signals

There have been promising studies that show a potential of providing social signal feedback to improve communication skills. However, these studies have primarily focused on unimodal methods of feedback. In addition to this, studies do not assess whether skills are maintained after a given time. With a sample size of 22 this paper investigates whether multimodal social signal feedback is an effective method of improving communication in the context of media interviews. A pre-post experimental evaluation of media skills training intervention is presented which compares standard feedback with augmented feedback based on automated recognition of multimodal social signals. Results revealed significantly different training effects between the two conditions. However, the initial experiment study failed to show significant differences in human judgement of performance. A 6-month follow-up study revealed human judgement ratings were higher for the experiment group. This study suggests that augmented selective multimodal social signal feedback is an effective method for communication skills training.
Coping with Digital Wellbeing in a Multi-Device World

While Digital Self-Control Tools (DSCTs) mainly target smartphones, more effort should be put into evaluating multi-device ecosystems to enhance digital wellbeing as users typically use multiple devices at a time. In this paper, we first review more than 300 DSCTs by demonstrating that the majority of them implements a single-device conceptualization that poorly adapts to multi-device settings. Then, we report on the results from an interview and a sketching exercise (N=20) exploring how users make sense of their multi-device digital wellbeing. Findings show that digital wellbeing issues extend beyond smartphones, with the most problematic behaviors deriving from the simultaneous usage of different devices to perform uncorrelated tasks. While this suggests the need of DSCTs that can adapt to different and multiple devices, our work also highlights the importance of learning how to properly behave with technology, e.g., through educational courses, which may be more effective than any lock-out mechanism.
“Put it on the Top, I’ll Read it Later”: Investigating Users’ Desired Display Order for Smartphone Notifications

Smartphone users do not deal with notifications strictly in the order they are displayed, but sometimes read them from the middle, suggesting a mismatch between current systems’ display order and users’ needs. We therefore used mixed methods to investigate 34 smartphone users’ desired notification display order and related it with users’ self-reported order of attendance. Classifying using these two orders as dimensions, we obtained seven types of notifications, which helped us not only highlight the distinct attributes but understand the implied roles of these seven types of notifications, as well as the implied meaning of display orders. This is especially manifested in our identification of three main mismatches between the two orders. Qualitative findings reveal several meanings that participants attached to particular positions when arranging notifications. We offer design implications for notification systems, including calling for two-dimensional notification layout to support the multi-purpose roles of smartphone notifications we identified.
Toward Understanding the Cultural Influences on Social Media Use of Middle Class Mothers in India

This paper presents a preliminary investigation into the cultural influences on social media use of middle class mothers in India. We focus on the interplay of cultural beliefs, norms, and social and familial structures that shape their perceptions which in turn influence information sharing and seeking, and social capital building. We conducted semi-structured interviews with 23 middle class mothers in India and found that cultural practices, traditions, and the presence of trusted offline strong-tie networks influenced their social media engagement.
Living with Uncertainty and Stigma: Self-Experimentation and Support-Seeking around Polycystic Ovary Syndrome

Polycystic Ovary Syndrome (PCOS) is a condition that causes hormonal imbalance and infertility in women and people with female reproductive organs. PCOS causes different symptoms for different people, with no singular or universal cure. Being a stigmatized and enigmatic condition, it is challenging to discover, diagnose, and manage PCOS. This work aims to inform the design of inclusive health technologies through an understanding of people’s lived experiences and challenges with PCOS. We conducted semi-structured interviews with 10 women diagnosed with PCOS and analyzed a PCOS-specific subreddit forum. We report people’s support-seeking, sense-making, and self-experimentation practices, and find uncertainty and stigma to be key in shaping their unique experiences of the condition. We further identify potential avenues for designing technology to support their diverse needs, such as personalized and contextual tracking, accelerated self-discovery, and co-management, contributing to a growing body of HCI literature on stigmatized topics in women’s health and well-being.
Drone in Love: Emotional Perception of Facial Expressions on Flying Robots

Drones are rapidly populating human spaces, yet little is known about how these flying robots are perceived and understood by humans. Recent works suggested that their acceptance is predicated upon their sociability. This paper explores the use of facial expressions to represent emotions on social drones. We leveraged design practices from ground robotics and created a set of rendered robotic faces that convey basic emotions. We evaluated individuals’ response to these emotional facial expressions on drones in two empirical studies (N = 98, N = 98). Our results demonstrate that individuals accurately recognize five drone emotional expressions, as well as make sense of intensities within emotion categories. We describe how participants were emotionally affected by the drone, showed empathy towards it, and created narratives to interpret its emotions. As a consequence, we formulate design recommendations for social drones and discuss methodological insights on the use of static versus dynamic stimuli in affective robotics studies.